NVIDIA has released NV-Reason-CT, a vision–language foundation model built to natively process full 3D CT volumes. The model produces structured diagnostic reports, generates radiologist-style chain-of-thought reasoning, and supports multi-turn conversational follow-up for chest and abdominal CT studies. NV-Reason-CT is offered as an open research and development foundation model, not an autonomous diagnostic device or cleared clinical product.
Why 3D CT requires a different approach
A single abdominal CT can contain 300–600 axial slices that encode anatomical context across three spatial dimensions. Treating a CT as independent 2D slices discards through-plane relationships that are essential to judge the shape, extent and density of findings. This volumetric complexity poses challenges in three areas:
- Perception: 2D token grids lose the spatial continuity between slices that defines volumetric structures such as masses, effusions and infiltrates.
- Reasoning: many models output labels without explaining why; radiologists reason via systematic anatomic review, differential diagnoses and calibrated confidence, which must be reproducible to audit and teach AI outputs.
- Conversational depth: radiologists commonly ask follow-up questions, reconsider differentials and correlate findings across regions in multiple turns; most existing models lack such multi-turn capabilities.
How NV-Reason-CT provides full 3D reasoning
NV-Reason-CT pairs a dedicated full 3D vision transformer encoder with a language model trained to generate chain-of-thought reasoning that mirrors radiologists’ systematic CT reviews. Unlike slice-based approaches, NV-Reason-CT ingests the CT volume as a true 3D input, preserving through-plane anatomical continuity and enabling holistic reasoning about structures.
Key capabilities:
- Structured report generation: NVIDIA curated a CT ontology covering 30 chest and 29 abdominal abnormalities (for example lung nodules, pneumothorax, hepatic lesions, renal cysts) to guide report generation and evaluation in formats that map to clinical documentation.
- Radiologist-emulating chain-of-thought: the model can output step-by-step internal reasoning — reviewing regions systematically, surfacing findings, weighing differential diagnoses and stating uncertainty.
- Multistep conversational follow-up: users can ask follow-up questions about findings, request clarification on differentials, or probe the model’s reasoning across multiple turns.
- Full 3D ViT encoder: a purpose-built 3D vision encoder extracts volumetric features that 2D approaches cannot recover. The model also passes 3D vision token coordinates to the LLM, with 3D MRoPE to account for spatial inter-token relationships through the LLM layers.
Architecture and technical details
The architecture couples a Qwen3.5-4B language model with a Primus/Colipri-based 3D ViT encoder; all weights are retrained end-to-end on large CT cohorts with structured reports, reasoning traces and multistep VQA. CT volumes are resampled to 192³ voxels at 2 mm isotropic resolution and tokenized using non-overlapping 8×8×8 patches, producing 24×24×24 = 13,824 vision tokens. Rather than merging or downsampling, all vision tokens and their 3D grid coordinates are passed to the LLM, which incorporates 3D MRoPE to model spatial relationships.
The language model is trained to reason like a radiologist: systematic region-by-region review, noting normal alongside abnormal findings, expressing calibrated uncertainty, and producing structured conclusions rather than classifier-style outputs.
Training methodology
NV-Reason-CT follows a two-stage training pipeline similar to NV-Reason-CXR: supervised fine-tuning followed by reinforcement learning.
Stage 1 — Supervised fine-tuning: the model was trained on a mixture of structured reports, expert radiologist reasoning annotations and VQA examples. Radiologists contributed chain-of-thought dictations describing what they examine in each region, which findings they consider significant, the differential diagnoses they weigh, and how they arrive at their assessments. The resulting curriculum includes roughly 550,000 structured QA examples across chest and abdominal regions, covering section-level anatomy QA, localized finding QA, severity-level QA and binary abnormality identification. The training set includes CT-RATE, NIH CT datasets and CancerVerse, supplemented with high-quality synthetic reasoning data distilled from LLMs and grounded by expert annotations.
Stage 2 — Reinforcement learning for reasoning quality: the model was refined with Group Relative Policy Optimization (GRPO). A reward function based on accuracy of identified abnormalities and diagnoses guides training; importantly the GRPO reward is anatomy-aware and provides region-level signals rather than a single global reward, improving calibration across chest–abdomen findings.
This two-stage design aims to teach reasoning patterns first and then reinforce clinical correctness so the model can generalize across diverse CT presentations without exhaustive hand-annotated reasoning chains for every example.
Benchmarking and clinical feedback
NVIDIA reports state-of-the-art performance on public 3D CT benchmarks. On CT-RATE (18 labels, fixed uniform threshold) NV-Reason-CT achieved Macro-F1 = 0.614 and Macro-AUROC = 0.871, outperforming published baselines such as VoxelFM (Macro-F1 0.581, AUROC 0.870), Pillar-0 (0.544 / 0.861), CT-CLIP (0.398 / 0.733), Merlin (0.358 / 0.662), ClinFusion-8B (reported Macro-F1 0.442) and MedGemma 1.5 (0.303). NVIDIA notes this is the first open model to be competitive in both CT classification and report generation concurrently.
NIH radiologists provided favorable clinical reviews for the structured reports and the plausibility of the chain-of-thought traces. Baris Turkbey, M.D., F.S.A.R. (National Institutes of Health) is quoted saying that the ability to review the model’s internal reasoning — not just its conclusions — enables trust and actionability.
Clinical and research implications
NVIDIA highlights two practical benefits noted by reviewers:
- Time savings in structured reporting: generating detailed structured reports covering 60+ abnormalities is time-consuming; NV-Reason-CT can produce such output in seconds, allowing radiologists to scan, validate and amend results more rapidly.
- Explainability for audit: by outputting chain-of-thought traces, the model’s conclusions become auditable — radiologists can identify where model reasoning aligns or diverges from their own.
The model checkpoints and post-training recipes are intended for researchers and developers to fine-tune on institution-specific CT datasets, study chain-of-thought in medical imaging, or integrate NV-Reason-CT into multimodal pipelines. Complementary NVIDIA models include NV-Generate-CTMR (synthetic 3D CT/MR generation) and NV-Segment-CTMR (organ and lesion segmentation).
Several medical AI and radiology workflow companies are mentioned as potential integrators of the foundation model, including Aidoc, HOPPR, Rad AI, Mosaic Clinical Technologies and Raidium.
Availability and regulatory status
NVIDIA emphasizes that NV-Reason-CT is an open research and development foundation model and not a cleared clinical product or autonomous diagnostic system. Checkpoints and code are made available on Hugging Face and GitHub, including inference scripts and training configurations for researchers.
Example usage
The announcement includes an example Python inference snippet showing how to load the model (nvidia/NV-Reason-CT) with AutoModelForImageTextToText and AutoProcessor, prepare a chat-style prompt with an input 3D image (e.g., chest_ct.nii.gz), and generate chain-of-thought reasoning and a structured report. An excerpted model output demonstrates internal thinking tags (<think>) and a stepwise review of a contrast-enhanced chest CT, describing findings such as a right lower lobe mass-associated partial collapse, mediastinal and hilar adenopathy, bilateral ground-glass opacities and indeterminate osseous lesions.
Conclusion
NV-Reason-CT extends chain-of-thought reasoning into volumetric CT interpretation by combining a native 3D ViT encoder with a reasoning-trained language model and an anatomy-aware reinforcement learning stage. NVIDIA positions the model as a research and development foundation for the medical AI community to build on, offering improved volumetric perception, explainability and multi-turn clinical interaction while clarifying that it is not a regulatory-cleared clinical device.
