🩻 [MIDL 2026 Oral] CWCD Category-Specific LoRA Adapters

Shantam Srivastava, Mahesh Bhosale, David Doermann, Mingchen Gao

University at Buffalo, The State University of New York

CWCD was accepted as a full paper at Medical Imaging with Deep Learning (MIDL) 2026 and selected for an oral presentation (oral selection rate below 8%).

📖 Model Description

This repository contains the eight category-specific rank-1 LoRA adapters released with CWCD: Category-Wise Contrastive Decoding for Structured Medical Report Generation. CWCD generates a structured findings report through independent forward passes under eight anatomical categories.

At inference time, each adapter is used for both the normal X-ray and its corresponding category-masked X-ray. CWCD contrasts the resulting base and masked log-probabilities, applies Plausibility-Based Vocabulary Subselection, and greedily selects the next token. The eight category-wise findings are combined into the final structured findings report.

The complete inference implementation and usage instructions are available in the CWCD GitHub repository.

🤗 Available Adapters

Anatomical category	Adapter directory	LoRA rank	Target modules
Lungs and Airways	adapters/lungs	1	q_proj, v_proj
Musculoskeletal and Chest Wall	adapters/wall	1	q_proj, v_proj
Cardiovascular	adapters/cardiovascular	1	q_proj, v_proj
Pleura	adapters/pleura	1	q_proj, v_proj
Hila and Mediastinum	adapters/hila	1	q_proj, v_proj
Tubes, Catheters, and Support Devices	adapters/tubes	1	q_proj, v_proj
Abdominal	adapters/abdominal	1	q_proj, v_proj
Other	adapters/other	1	q_proj, v_proj

Each directory contains:

adapter_config.json
adapter_model.bin

🧠 CWCD Inference

For each anatomical category, the GitHub inference pipeline:

1. loads the corresponding category-specific adapter;
2. obtains the base distribution from the normal X-ray;
3. obtains the masked distribution after blacking out that category's bounding boxes;
4. contrasts the two distributions in log-probability space;
5. applies Plausibility-Based Vocabulary Subselection;
6. generates the category-wise findings and combines all eight sections.

The contrastive score is:

score(token) = (1 + alpha) * log P_base(token) - alpha * log P_masked(token)

The released defaults are alpha=1.0 and beta=0.5.

🚀 Using the Adapters

The adapters are downloaded automatically by inference.py through huggingface_hub.snapshot_download(). Only the adapter configuration and weight files are retrieved.

git clone https://github.com/shantamsrivastava/CWCD.git
cd CWCD

python -m pip install -r requirements.txt
python -m pip install git+https://github.com/microsoft/LLaVA-Rad.git

python inference.py \
  --image chest_xray.png \
  --boxes boxes.example.json \
  --alpha 1.0 \
  --beta 0.5 \
  --output report.json

See the GitHub README for environment setup, input format, standard category-wise inference, prompts, and all CLI options.

📦 Repository Structure

adapters/
├── abdominal/
├── cardiovascular/
├── hila/
├── lungs/
├── other/
├── pleura/
├── tubes/
└── wall/

⚠️ Intended Use and Limitations

These adapters are intended strictly for research and educational use. CWCD has not been approved or validated for clinical or diagnostic deployment and must not be used to make medical decisions or inform patient care.

Generated findings can contain omissions, incorrect statements, or hallucinations. Performance depends on the base model, input image quality, category-specific bounding boxes, and the domain represented by the evaluation data. Users are responsible for protecting patient privacy and following all applicable data-use agreements.

🤝 Acknowledgements

CWCD builds on LLaVA-Rad, LLaVA, Vicuna, and PEFT.

📑 Citation

@inproceedings{srivastava2026cwcd,
  title={CWCD: Category-Wise Contrastive Decoding for Structured Medical Report Generation},
  author={Srivastava, Shantam and Bhosale, Mahesh and Doermann, David and Gao, Mingchen},
  booktitle={Medical Imaging with Deep Learning},
  series={Proceedings of Machine Learning Research},
  volume={315},
  pages={868--893},
  year={2026},
  publisher={PMLR}
}