Microsoft’s diagnostic AI reached 83.2% accuracy on a set of complex medical cases by deciding which questions and investigations to request along the way. Generalist physicians with access to reference resources achieved 36.1% in the same benchmark, according to a study published in Science on 22 October.[1][2]
The AI Diagnostic Orchestrator, or MAI-DxO, coordinates several simulated diagnostic roles that revise possible explanations as evidence arrives. Its contribution is the sequence of decisions behind the answer: what information to seek, which test to request and when to stop investigating.[1][2]
A benchmark that releases evidence in stages
The difficult-case evaluation used 304 New England Journal of Medicine clinicopathological conference cases. Participants began with a short clinical presentation and could request further information. Microsoft reports 22% accuracy for physicians without external references and 36% with them; the journal abstract gives the latter comparison more precisely as 36.1%.[1][2]
On a separate set derived from MIMIC-IV-ED emergency-department admissions, the orchestrator achieved 80.3%, compared with 46.8% for physicians. These are distinct case sets and should not be pooled into a single overall accuracy score.[2]
Accuracy in the sequential-diagnosis study
| Case set | AI orchestrator | Generalist physicians |
|---|---|---|
| NEJM clinicopathological cases | 83.2% | 36.1% with reference resources |
| MIMIC-IV-ED-derived cases | 80.3% | 46.8% |
A static medical quiz supplies the relevant evidence in advance. Sequential diagnosis also tests whether the participant can obtain it efficiently. The benchmark records requested investigations and estimates their cost, allowing the researchers to examine the route to a diagnosis as well as whether the final answer is correct.[1][2]
Accuracy and investigation cost can be traded off
Microsoft reports a more cost-constrained configuration that retained about 81% accuracy on the difficult cases at less than half the estimated investigation cost of comparison approaches. That is a benchmark estimate for requested tests. It does not include every cost of clinical care, such as staffing, waiting, incidental findings or follow-up.[1]
The simulated panel also should not be confused with independent human specialists. Several roles can encourage a model to examine alternatives, but they may share underlying capabilities and failure modes. Its effectiveness is established by the resulting benchmark performance, rather than by the appearance of a multidisciplinary conversation.[1]
Prospective work moves the question into care
Microsoft plans a prospective evaluation with Beth Israel Deaconess Medical Center in Boston, including direct patient interaction and comparison with physicians’ independent assessments. That is a concrete step toward examining how the system handles the uncertainty and incomplete information of an actual encounter. It is a planned evaluation, not a reported clinical outcome.[1]
The distinction matters because NEJM cases are selected diagnostic puzzles, while real patients can clarify a history, need urgent treatment or change condition during an investigation. Emergency-department-derived records broaden the benchmark, but they still cannot reproduce all of those interactions.[1][2]
The study’s advance is a way to evaluate diagnosis as an unfolding process. By recording questions, tests and estimated costs, it makes the reasoning path available for examination instead of rewarding only a correct answer at the end.[1][2]
Sources & further reading
- Building Medical Superintelligence Microsoft AI · 22 Oct 2026 · Accessed 27 Oct 2026
- Sequential diagnosis with language models Science / PubMed · 22 Oct 2026 · Accessed 27 Oct 2026