Full Article
Skip to main content
Research
Publications
Code & data
People
Microsoft Research blog
Artificial intelligence
Audio & acoustics
Computer vision
Graphics & multimedia
Human-computer interaction
Human language technologies
Search & information retrieval
Data platforms and analytics
Hardware & devices
Programming languages & software engineering
Quantum computing
Security, privacy & cryptography
Systems & networking
Algorithms
Mathematics
Ecology & environment
Economics
Medical, health & genomics
Social sciences
Technology for emerging marketsAcademic programs
Events & academic conferences
Microsoft Research ForumBehind the Tech podcast
Microsoft Research blog
Microsoft Research Forum
Microsoft Research podcast
About Microsoft Research
Careers & internships
People
Emeritus program
News & awards
Microsoft Research newsletter
Africa
AI for Science
AI Frontiers
Asia-Pacific
Cambridge
Health Futures
India
Montreal
New England
New York City
Redmond
Applied Sciences
Mixed Reality & AI - Cambridge
Mixed Reality & AI - Zurich
Register: Research Forum
Microsoft Security
Azure
Dynamics 365
Microsoft 365
Microsoft Teams
Windows 365
Microsoft AI
Azure Space
Mixed reality
Microsoft HoloLens
Microsoft Viva
Quantum computing
Sustainability
Education
Automotive
Financial services
Government
Healthcare
Manufacturing
Retail
Find a partner
Become a partner
Partner Network
Microsoft Marketplace
Software companies
Blog
Microsoft Advertising
Developer Center
Documentation
Events
Licensing
Microsoft Learn
Microsoft Research
View Sitemap
Return to Blog HomeMicrosoft Research BlogResearch Note: CARE-X is a research model and not a Microsoft product offering or medical device. It has not been cleared or approved by any regulatory authority and is not intended for clinical diagnosis, screening, or patient care. The results described below are retrospective research findings and do not establish the safety, effectiveness, or suitability of CARE-X for any clinical use. References to potential workflows describe areas for future research, not currently available capabilities or recommended uses. At a glanceThe challenge: Chest X-ray interpretation spans diverse tasks that require both expressive report generation and calibrated diagnostic predictions.CARE-X is a unified chest X-ray VLM for diverse clinical interpretation tasks. It combines generation and structured prediction to provide both free-text reasoning and deterministic outputs.CARE-X uses reinforcement learning (DAPO) to reward clinical correctness in a multi-task setting.In a separate research experiment from CARE-X, we paired Qwen3-VL-4B-Instruct with deterministic measurement tools to evaluate whether direct computation could improve performance on measurement-dependent conditions compared with visual approximation alone. Validated on real-world Indian clinical data from Narayana Health, including rare ICU pathologies and CT-confirmed enlargement conditions.What radiologists need: Task diversity, flexibility, and clinical fidelityA clinically useful radiology AI system must support a wide range of tasks, adapt to different workflows, and produce outputs that are medically accurate.Radiologists and other clinicians use chest X-rays for many different purposes. A clinically useful AI system must be able to support that range of tasks. It may be asked to generate detailed findings and concise impressions for a report, answer questions about the presence, absence, or location of a finding, identify medical devices and assess their placement, or pinpoint exactly where an abnormality appears in an image.These tasks also require different kinds of outputs, from narrative reports to calibrated diagnostic scores. And above all, they require clinical accuracy. A report could ostensibly be perfectly written yet clinically wrong if it misses a finding, reverses a negation, or misidentifies a location. Certain findings could be trivial in one context and vital to identify in another.CARE-X was developed as a research model to explore how a unified approach can address these diverse demands. The system combines generative and discriminative capabilities, clinically aligned optimization, and tool-based reasoning to support a broader range of radiology workflows while maintaining clinical fidelity.PODCAST SERIESAI Testing and Evaluation: Learnings from Science and IndustryDiscover how Microsoft is learning from other domains to advance evaluation and testing as a pillar of AI governance.
Listen now
Opens in a new tabGaps in current radiology vision-language modelsDespite the impressive task breadth of recent models, critical gaps remain between what radiologists need and what current systems deliver:No calibrated confidence for diagnostic decisions. Generative VLMs predict diagnoses as free text, but they typically do not provide calibrated confidence scores. In clinical settings, confidence matters. Clinicians cannot tune sensitivity–specificity trade-offs across clinical contexts—an important requirement for real-world deployment. Discriminative models provide these properties but lack the flexibility of open-ended generation.Cross-entropy loss does not optimize clinical fidelity. Standard training methods treat all token-level errors similarly, regardless of their clinical consequences. A coordinate mistake may be penalized no more than a harmless wording change. A “yes” can be flipped to a “no” even though the clinical meaning is completely different. Missing a life-threatening finding may carry the same training penalty as omitting a minor observation. As a result, models are not explicitly optimized for what matters most in patient care.No capability for measurement-dependent findings. Some radiological findings require more than visual recognition. Radiological signs such as cardiomegaly, mediastinal widening etc. depend on precise measurements. For example, a model may correctly recognize whether a chest radiograph was acquired using an AP or PA view. But determining cardiomegaly requires measuring the cardiac and thoracic widths and determining the cardiothoracic ratio. Those quantities should be measured and computed rather than visually approximated while considering variables such as type of view, exposure, rotation of the patient etc. Together, these gaps call for more than a fluent generative model. The system must combine broad task coverage, structured predictions, clinically aligned optimization, and quantitative tools where direct measurement is required.CARE-X: One model, flexible outputsCARE-X brings these diverse interpretation capabilities into one model, using generative or dual inference according to the needs of each task:
Task type What CARE-X does Inference mode
Report generation: Findings Produces the detailed findings section GenerativeReport generation: Impression Produces the concise diagnostic impression GenerativePresence and negation assessment Determines whether a pathology is present or absent and handles negation Dual: generative + auxiliary headDisease location assessment Identifies where an abnormality appears GenerativeFine-grained multilabel disease classification Categorizes abnormalities across multiple labels GenerativeMultilabel tubes and lines classification Identifies visible medical devices GenerativeAbnormal placement detection of tubes and lines Determines whether a device is positioned incorrectly Dual: generative + auxiliary headAbnormality phrase grounding Localizes a described pathological finding Dual: generative + auxiliary headAnatomical grounding Localizes 29 anatomical regions Dual: generative + auxiliary head
Table 1: CARE-X task coverage and inference modesDual inference means that a single forward pass produces both an autoregressive response and a structured auxiliary-head prediction with a confidence score. This provides free-text flexibility alongside threshold-adjustable outputs for tasks where operating-point control matters.The CARE-X architecture and training approachCARE-X is built on a SigLIP2-so400M vision encoder and a Phi-4-mini-instruct (3.8B) language model connected through a lightweight adapter. To support both free-text generation and structured clinical predictions, the model augments the shared language backbone with task-specific auxiliary heads for classification and visual grounding. These heads provide calibrated diagnostic predictions and spatial localization signals while sharing representations with the generative language model. Rather than being trained independently, they are co-trained with the language-modeling objective, allowing structured supervision to enrich shared representations and improve generative performance on the same tasks.Training. CARE-X uses a three-stage supervised fine-tuning pipeline (vision pre-training, adapter/head training, and LoRA adaptation) followed by DAPO-based reinforcement learning. DAPO optimizes task-specific rewards for clinical reporting, diagnostic accuracy, and spatial grounding quality.Figure 1. The CARE-X model. (Left) Supervised fine-tuning with task-specific heads — classification, grounding, and language modeling — sharing the same Phi-4-mini-instruct backbone. The classification head outputs calibrated P(Yes)/P(No) scores; the grounding head outputs bounding box coordinate with confidence; the language modeling head generates free-text responses. (Right) DAPO with task-specific rewards for multi-task reinforcement alignment across report generation, grounding, and VQA. Auxiliary supervision: Structured prediction strengthens generationA central finding of this work is that co-training discriminative auxiliary heads with a generative VLM enriches shared representations, leading to stronger generative performance on the same tasks while also providing calibrated structured predictions.Grounding improvementsThe auxiliary grounding head consistently improves localization over generative decoding. On anatomical grounding (Chest ImaGenome), mAP and mIoU increase by +28.2 pp and +6.2 pp, while the largest gains occur on phrase grounding (PadChest), with +24.6 pp mAP and +14.1 pp mIoU. The composite spatial loss enhances geometric precision in shared representations.DAPO bridges the gap to dedicated detection headsDAPO-trained generative output approaches or exceeds the SFT auxiliary detection head. On Anatomy grounding, CARE-X generative (0.868 mAP) surpasses the SFT detection head (0.865). This is practically significant—it demonstrates that reward-aligned learning can bring autoregressive spatial decoding to parity with structured prediction, offering clinicians a single generative inference mode without requiring auxiliary heads at test time.Calibrated classification with tunable operating pointsBeyond representation enrichment, the classification head offers a distinct deployment advantage: calibrated probability scores with tunable thresholds allow clinicians to shift between high-sensitivity screening and high-specificity confirmation from a single forward pass—a capability purely generative architectures cannot provide.
Model Inference Setting Sensitivity ↑ PPV ↑ F1 ↑
CARE-X Generative 0.932 0.895 0.913CARE-X (Th=0.5) Auxiliary Head 0.943 0.885 0.913CARE-X (Th=0.6) Auxiliary Head 0.855 0.927 0.890CheXOne Generative 0.878 0.854 0.866MedGemma Generative 0.798 0.886 0.839
Table 2: Abnormality classification performance on Chest ImaGenome. Adjustable thresholds enable operating-point selection.Strong report generation across four benchmarksWithin the paper’s comparison set, CARE-X achieves the strongest performance on most reported metrics across MIMIC-CXR, IU-Xray, CheXpert-Plus, and ReXGradient. CRIMSON, a held-out metric that evaluates abnormal findings and weights errors by clinical severity, suggests these gains reflect clinically meaningful improvements rather than reward-specific optimization.Figure 2. CRIMSON scores (↑) for CARE-X against baseline report-generation models across four chest X-ray datasets — ReXGradient, MIMIC-CXR, IU-Xray, and CheXpert-Plus. CARE-X (highlighted) achieves the highest CRIMSON score on every dataset.CARE-X reaches 94% accuracy on ReXVQACARE-X ranks first on the ReXrank RexVQA leaderboard (opens in new tab) as of August 2026. On the ReXVQA benchmark (41,007 question–answer pairs across five clinically relevant categories), CARE-X reaches 94% overall accuracy , six percentage points above the next-best publicly reported model.Figure 3: ReXVQA accuracy across five findings-quality dimensions — negation, presence, location, differential diagnosis, geometric information, and overall. CARE-X consistently outperforms CheXOne-R1 and MedGemma on every axis, with the largest margins in differential diagnosis, location assessment and negation.Tool-augmented measurement: Interleaving perception and computationSome radiological findings depend on quantitative measurements rather than visual patterns. In a separate research experiment from CARE-X, we built an inference-time pipeline that combines Qwen3-VL-4B-Instruct with deterministic measurement tools, allowing the model to alternate between image understanding and precise computation. Qwen3-VL-4B-Instruct retains visual access to the radiograph throughout inference, invoking tools to identify anatomical landmarks, compute measurements, and evaluate diagnostic thresholds as needed. This creates a multi-turn reasoning loop that interleaves perception and measurement, enabling the model to combine visual context with exact quantitative evidence before reaching a diagnosis.Figure 4. Tool-augmented quantitative reasoning pipeline. The orchestrator mediates a multi-turn loop: the VLM reasons over the image (perception), emits structured tool calls, receives deterministic results, and synthesizes the final diagnosis.Despite requiring no task-specific training, this approach substantially outperforms perception-only inference across all evaluated measurement-based conditions. The results suggest that for threshold-dependent diagnoses, direct computation of clinically defined measurements is more reliable than visual approximation alone.More broadly, this measurement-augmented approach could augment clinical workflows by expanding the set of quantitative assessments routinely derived from chest radiographs. For example, aortic dilation is not typically quantified on CXR and is often detected only incidentally on CT scans obtained for other indications. As delayed detection can contribute to adverse cardiovascular outcomes, reliable CXR-based screening could enable earlier identification and follow-up of aortic dilation.
Condition Perception F1 Tool F1 Δ F1
Cardiomegaly 74.56 96.00 +21.4Mediastinal Widening 72.63 97.47 +24.8Aortic Knob Enlargement 60.31 99.76 +39.5Ascending Aorta Enlargement 39.33 100.00 +60.7Descending Aorta Enlargement† 28.57 100.00 +71.4Average +43.6
Table 3: Perception-only versus tool-augmented measurement. The average F1 improvement is 43.6 percentage points across five conditions.Validation on Indian clinical data: Rare ICU conditions and CT-confirmed enlargementResearch ethics and data use : The Narayana Health evaluations used de-identified, retrospective clinical data under applicable institutional ethics review and data-use approvals. Narayana Health approved publication of the study results described here. Study 1: Inpatient and ICU conditionsTo assess real-world generalizability in a research setting, we evaluated CARE-X on 1,047 de-identified chest radiographs from Narayana Health, annotated for five rare, high-acuity conditions with prevalence ranging from 2.6% to 5.2%—reflecting realistic clinical distributions where missed diagnoses carry severe consequences.
Fracture Mediastinal Shift Pneumoperitoneum Pneumothorax Tubes & Lines Abnormal Placement
Model Sens / Spec Sens / Spec Sens / Spec Sens / Spec Sens / SpecCheXOne 0.41 / 0.90 0.80 / 0.78 0.67 / 0.98 0.85 / 0.72 0.03 / 0.97MedGemma 0.05 / 1.00 1.00 / 0.53 0.00 / 1.00 0.52 / 0.73 0.18 / 0.87CARE-X 0.62 / 0.64 0.83 / 0.86 0.89 / 0.94 0.83 / 0.75 0.66 / 0.77
Table 4: ICU pathology classification on Indian hospital data. CARE-X achieves the most balanced performance.CARE-X achieves the highest sensitivity in three out of five conditions while maintaining reasonable specificity, demonstrating generalization to low-prevalence clinical settings.Study 2: CT-confirmed enlargement conditionsIn a retrospective study to measure pure recall efficacy, we evaluated measurement-dependent conditions such as mediastinal widening findings including aortic enlargement, hilar mass, and pulmonary artery enlargement on a outpatient cohort of 122 positive cases with CT-confirmed ground truth, avoiding the subjectivity of radiologist consensus on borderline enlargement findings on CXR. In the overlay setting, the VLM receives the original radiograph alongside a second image with condition-relevant anatomical segmentation masks — offering spatial guidance without direct access to measurement tools.The tool-augmented variant reached 94.26% recall, a +10.65 percentage-point gain over the best perception-only baseline. Where CT or echocardiography access is limited, reliable triage from a widely available modality like chest X-ray can cut both unnecessary referrals and missed diagnoses.Figure 5: Recall on the CT-confirmed enlargement cohort across perception-only, overlay-assisted, and tool-augmented inference. (Study 2)In a related study (accepted at EACTS conference 2026), for mild aortic dilation, the measurement-driven reasoning approach detected 40 of 43 CT-confirmed cases (93% sensitivity), compared to just 5 of 43 (12%) identified on the initial radiology reads, where aortic enlargement is usually not the primary indication for the chest X-ray. This corresponds to 35 additional mild cases that were surfaced but missed during the initial CXR interpretation. These results suggest that explicit quantitative measurements may help identify borderline enlargement that is difficult to assess through visual inspection alone. What this does and doesn’t show These numbers are all recall, i.e., how many true positives we catch. This was the focus of the initial study because, in triage, a missed diagnosis is typically the costlier failure mode, and CT-confirmed ground truth gave us a clean way to measure it without relying on radiologist consensus for the difficult cases. Recall, however, captures only one dimension of diagnostic performance. A model that flags everything achieves perfect recall and is useless in practice. An extended study is underway that includes CT-confirmed negative cohorts as well. Preliminary results are promising, and further studies are planned to explicitly evaluate the viability of quantitative aortic measurements on chest X-ray as a screening tool for aortic dilation. CARE-X: Toward clinically useful radiology AICARE-X demonstrates that discriminative and generative objectives can be effectively combined within a unified radiology AI model. By jointly training classification, grounding, and language capabilities, the model supports both flexible report generation and calibrated, threshold-adjustable predictions. The separate measurement study further highlights a practical division of labor between learned reasoning and deterministic computation: the VLM provides visual understanding and identifies relevant evidence, while measurement-dependent diagnoses are computed through transparent, tool-based calculations. Retrospective evaluation on clinically challenging Narayana Health cohorts provides encouraging evidence of the potential of this approach for real-world radiology applications. The clinical relevance of this research is underscored by the selection of the AI-based aortic dilatation screening application as a finalist for showcase at the IHF Innovation Hub, World Hospital Congress 2026 , recognizing its potential to support earlier detection and clinical decision-making in cardiovascular care. Looking ahead, CARE-X can be extended beyond its current capabilities through structured report generation, richer differential diagnosis support, and tighter integration of tools within the model itself. The framework could also benefit from incorporating broader clinical context, including laboratory results and patient history, enabling more comprehensive clinical reasoning. CARE-X is a research model, not a Microsoft product offering or medical device. It has not been cleared or approved by any regulatory authority and is not intended or validated for clinical diagnosis, screening, patient care, or clinical decision-making. The results described are retrospective research findings and do not establish safety, effectiveness, or suitability for clinical use. Paper co-authors:Mercy Ranjit , Anirban Porya (opens in new tab) , Niharika Vadlamudi (opens in new tab) , Nikhilesh E (opens in new tab) , Sathvik Joel (opens in new tab) , Prasanth V V (opens in new tab) , Tanuja Ganu , Abhyuday Swamy (opens in new tab) , Pranay Umredkar (opens in new tab) , Pradeep Narayan (opens in new tab) , Vivek Rajagopal (opens in new tab)Collaborators: Medha AI (opens in new tab) , Narayana Health (opens in new tab)Opens in a new tabRelated publicationsCARE-X: Towards Clinically Useful Radiology VLMs with Auxiliary Supervision, Reward-Aligned Learning, and Tool-Augmented Measurement Meet the authorsMercy RanjitPrincipal Research ML EngineerLearn moreNikhilesh EResearch InternMicrosoftLearn moreDr. Abhyuday Kumara SwamySenior Data ScientistNarayana HealthLearn moreTanuja GanuDirector of Research EngineeringLearn moreResearch Areas
Computer vision
Medical, health and genomicsFollow us:
Follow on X
Like on Facebook
Follow on LinkedIn
Subscribe on Youtube
Follow on Instagram
Subscribe to our RSS feedShare this page:
Share on X
Share on Facebook
Share on LinkedIn
Share on Reddit
Surface Pro
Surface Laptop
Surface Laptop Ultra
Surface RTX Spark Dev Box
Copilot for organizations
Copilot for personal use
Explore Microsoft products
Windows 11 apps
Account profile
Download Center
Microsoft Store support
Returns
Order tracking
Certified Refurbished
Microsoft Store Promise
Flexible Payments
Microsoft in education
Devices for education
Microsoft Teams for Education
Microsoft 365 Education
How to buy for your school
Educator training and development
Deals for students and parents
AI for education
Microsoft AI
Microsoft Security
Dynamics 365
Microsoft 365
Microsoft Power Platform
Microsoft Teams
Microsoft 365 Copilot
Small Business
Azure
Microsoft Developer
Microsoft Learn
Support for AI marketplace apps
Microsoft Tech Community
Microsoft Marketplace
Software companies
Visual Studio
Careers
About Microsoft
Company news
Privacy at Microsoft
Investors
Diversity and inclusion
Accessibility
Sustainability
Your Privacy ChoicesConsumer Health Privacy
Sitemap
Contact Microsoft
Privacy
Manage cookies
Terms of use
Trademarks
Safety & eco
Recycling
About our ads