An AI that identifies a finding in a medical image but cannot show what it saw leaves clinicians with a difficult choice: trust the answer or do the analysis again. A study published in Nature Medicine on 29 October describes EVLF-FM, a vision-language foundation model designed to answer a diagnostic question, indicate a relevant image region and explain its answer. Its most consequential advance is not a single accuracy score but the attempt to connect language to location across different types of medical imaging.[1]
The development and evaluation involved more than 1.3 million samples from 18 datasets spanning ten imaging modalities and six specialties. The investigators also assembled external testing using 13,443 independent samples from 17 additional datasets across ten modalities. The sources range from CT and MRI to histopathology, dermatoscopy, ultrasound and retinal images. That breadth is unusual, but it also creates an interpretive trap: performance averaged across very different tasks can conceal a weak result in one clinically important setting.[1]
What the benchmark numbers measure
On internal disease-diagnosis validation the team reports a highest average accuracy of 0.895 against its selected comparison models. On visual grounding—where a system must identify the relevant region—its average accuracy at an intersection-over-union threshold of 0.5 was 0.842 across nine modalities. These are different tasks, with different numerators and different reference standards. Neither number can be interpreted as the probability that a real patient receives a correct overall diagnosis.[1]
EVLF-FM’s distinct evaluation measures
| Reported result | What it assesses |
|---|---|
| 0.895 mean accuracy | Internal image-diagnosis validationSource 1 |
| 0.842 [email protected] | Internal visual grounding across nine modalitiesSource 1 |
| 4.25 / 5 | Mean physician grading of selected explanationsSource 1 |
| 13,443 samples | External test-set sizeSource 1 |
For explanation quality, 12 physicians, including four junior doctors and eight specialists, graded selected outputs against the S.C.O.R.E. framework: safety, consensus and context, objectivity, reproducibility and explainability. The overall mean was 4.25 out of five, with a reported spread of 0.94. That is a structured human assessment of written reasoning; it is not a trial proving that explanations faithfully expose the internal causes of each prediction or improve treatment decisions.[1]
The explanation is built, not discovered
EVLF-FM combines global disease-related visual representations with higher-resolution pixel information. Supervised adaptation is followed by reinforcement learning using task-specific feedback, including scores for finding the correct image region. The team also uses sentence-level comparisons between generated explanations and reference reasoning, penalizing unsupported claims and contradictions. This gives the system incentives to produce visually grounded, clinically recognizable accounts rather than unrestricted plausible prose.[1]
The model’s explanation pipeline is deliberately constructed: synthetic rationales are filtered and refined before training, and alignment with expert reference reasoning is rewarded. The resulting explanations can be inspected against the image, but a plausible rationale is not automatically a faithful account of the model’s internal decision process.[1]
The researchers also tested robustness to prompt wording and reported small average score changes across ten prompt variants. They acknowledge that visual question answering generally lagged disease classification, that not all datasets meet demanding clinical performance levels, and that prospective testing in diverse hospitals remains necessary. Those limitations are not side notes: they define the difference between a research foundation model and a regulated workflow that patients and clinicians could depend on.[1]
The contribution is a single model that links a prediction to an image location and an inspectable explanation. Its external dataset tests extend the evidence beyond internal validation; prospective studies would establish whether those outputs improve clinicians’ decisions in practice.[1]
Sources & further reading
- A clinical multimodal vision language foundation model with fine-grain explainability Nature Medicine · 29 Oct 2026 · Accessed 30 Oct 2026