A chatbot can get the name of a disease right and still recommend a bad plan. The tests might be unnecessary. The referral might be excessive. A well-intentioned instruction to investigate everything could create expense, anxiety and delays without improving a patient's outcome. Researchers publishing in NEJM AI in October studied that under-measured side of algorithmic medical advice: the cost of care that a system proposes.[1][2]
The study used 200 deidentified clinical notes from outpatient and emergency settings. Twenty-four language-model systems, including specialised medical products, generated suggested assessments and plans from the case presentations. Investigators extracted their proposed tests and referrals, then applied US Medicare prices. Compared with the plans documented by treating physicians, the model-generated recommendations were notably more expensive.[1][2]
A number that deserves its denominator
The reported mean proposed testing and referral cost was about $314 per visit for the AI plans, versus $97 for the physician comparator—roughly 3.2 times as much. The estimated difference was approximately $216 per visit. These are averages across a curated set of notes and priced recommendations, not actual patient bills paid by insurers or families. Medication suggestions were examined separately but not fully included in the same price total.[1][2]
The models often recommended some investigation when the physician documented none. The research account reports physicians ordered no tests in roughly 66% of visits, whereas every evaluated general-purpose model proposed testing on those cases under the examined setting. That pattern suggests an algorithmic tendency to prefer action over restraint. But one cannot infer that every physician decision was correct or every AI-recommended test was wasteful. Either party could have missed a useful investigation.[1][2]
Why it is easy for an AI to over-order
A general-purpose assistant is usually rewarded for providing a complete-seeming answer. If a case note mentions a symptom, the system can list a medically plausible range of diagnostic possibilities and tests. A clinician faced with a real person has additional context: physical examination, continuity of care, prior tests, the patient's preferences, local protocols and awareness that every investigation has an opportunity cost.[1][2]
In a written-note benchmark the model sees a narrower world. It may respond to uncertainty by recommending everything that could conceivably clarify the picture. That is a form of risk avoidance, but it can shift costs and potentially expose patients to unnecessary procedures or follow-on investigations. A test is not harmless merely because it is technically available.[1][2]
The investigators also tried prompts that encouraged attention to cost and appropriate parsimony. These reduced, but did not eliminate, the spending difference in the reported analyses. That makes wording part of the intervention rather than a neutral wrapper around a model's capabilities. For clinical use, however, spending less is not automatically better; the correct objective is appropriate, safe care at justified cost.[1][2]
The study did not run a clinic
No patient was randomised here to an AI-generated treatment plan versus a physician's plan with subsequent outcomes measured. The cost figures come from recommendations priced under a defined schedule. The case notes are not a prospective, representative sample of current care, and medication costs and downstream complications add further uncertainty.[1][2]
A meaningful next trial would compare clinical outcomes as well as cost: missed serious illnesses, unnecessary investigations, referral timing, patient understanding and clinician workload. It would also give the algorithm access to clinically relevant history while maintaining data security and human oversight.[1][2]
The study is valuable because it asks an uncomfortable question absent from many medical-AI leaderboards. Getting the diagnosis right is only the beginning of a decision. A responsible health system must also ask what the model tells people to do next—and who pays when it recommends doing too much.[1][2]
Sources & further reading
- Comparative Cost of Care Recommended by Large Language Models Versus Treating Physicians NEJM AI · 15 Oct 2026 · Accessed 24 Oct 2026
- AI doctors’ orders come with a sky-high bill Nature Research Highlight · 15 Oct 2026 · Accessed 24 Oct 2026
- AI plans cost 3.2 times physicians’ care across 200 clinic notes longevitydocs News · 21 Oct 2026 · Accessed 24 Oct 2026