Microsoft’s diagnostic AI is being tested on a more demanding task than naming a disease after all the evidence has been supplied. Its MAI-DxO system chooses what to ask, which investigations to request and when to revise a diagnosis. In an October 22 research update, Microsoft reports substantial accuracy gains on both complex medical cases and more typical emergency-department presentations.[1]
The surrounding Sequential Diagnosis Benchmark, or SDBench, begins each case with limited information. Participants request further details as they work. The evaluation scores the final diagnosis, the evidence obtained and estimated investigation costs, making the path to an answer part of the test.[1]
A coordinated diagnostic process
MAI-DxO coordinates several AI roles that propose diagnoses, challenge one another and weigh the value of further tests. The process revisits its assessment as information arrives. This matters because diagnosis involves a sequence of decisions: an expensive or invasive investigation may answer a question, but the system first has to decide whether that information is worth obtaining.[1]
Reported diagnostic accuracy in SDBench
| Case set and condition | MAI-DxO | Physician comparison |
|---|---|---|
| 304 complex NEJM cases; physicians with references | 83% | 36% |
| NEJM cases; physicians without references | 83% | 22% |
| MIMIC-IV-ED emergency scenarios | 80% | 47% |
The physician results differ substantially depending on access to reference material. On the complex NEJM cases, 83% versus 36% is a 47-percentage-point gap; comparing the same system with the 22% no-reference result produces a different headline. Keeping those conditions visible makes the comparison more informative than describing one universal advantage over doctors.[1]
Information has a cost
Microsoft also reports a configuration that puts more weight on cost. It reached 81% accuracy on the NEJM cases at less than half the estimated cost of comparative systems. That is a separate configuration from the highest-scoring 83% result. The cost estimates describe the benchmark’s investigations rather than an observed reduction in hospital bills.[1]
The methodological contribution is the ability to examine a work-up. A model that arrives at the right answer only after requesting every available test has behaved differently from one that selects a small number of informative investigations. A sequential benchmark makes that difference measurable and creates a way to study when further information changes the conclusion.
Microsoft has made a selected set of NEJM cases and the benchmark’s supporting components available for research, including tools that manage information requests, judge diagnoses and estimate costs. That gives other teams a way to test their own systems against the same sequence-based task.[1][2]
The next step is to see how this approach assists clinicians when symptoms evolve, investigations have real delays and decisions reflect a patient’s preferences as well as the diagnostic label. The current evidence concerns interactive case evaluation. Its useful advance is to put information gathering and resource use alongside accuracy, closer to the decisions that clinical work actually requires.
Sources & further reading
- Building Medical Superintelligence Microsoft AI · 22 Oct 2026 · Accessed 25 Oct 2026
- SDBench — Sequential Diagnosis Benchmark Microsoft Research · Accessed 25 Oct 2026