Radiographers made fewer fracture errors after checking AI

Thirteen radiographers at Singapore General Hospital made fewer fracture-detection errors after consulting AI during a three-month prospective study. Across 2,496 analysed examinations, accuracy rose from 97.64% to 98.04%.[1]

Missed fractures fell from 38 to 32 and false alarms from 21 to 17. Ten corrections followed correct AI advice; another occurred despite incorrect AI output. One wrong AI suggestion introduced a new error.[1]

Rayvolve 4.1 performed worse alone at the study’s operating threshold. The study lacked an unaided second-read control and measured neither reading time nor downstream patient outcomes.[1]

A second opinion can be useful even when it performs worse overall. It may notice a different set of mistakes, while the person using it accepts useful advice and rejects misleading suggestions. The resulting decision is a property of that working relationship. A ranking of the software against a human reader cannot fully describe it.

The denominator outside the accuracy score

Before the paired analysis, 265 of the 2,995 acquired examinations had absent or incomplete AI output and were excluded.[1]

That creates a different problem from an incorrect suggestion. Someone must recognise that a result has not arrived and continue the established pathway. A hospital needs to know both how decisions change when output is available and how often the service fails to deliver it. High accuracy among completed readings cannot answer the availability question.

What the wider evidence can support

An umbrella review published the same day in npj Digital Medicine examined 50 reviews covering 387 unique primary studies. Pooled sensitivity was 89.7% and specificity 90.4%, but all 25 meta-analyses received low or critically low methodological ratings. Most assessments of evidence certainty were low or very low.[2]

Comparative evidence was too limited to establish a consistent advantage for AI, clinicians or assisted clinicians. That inconclusive comparison does not establish equivalence. None of the included reviews reported adverse events or patient-safety outcomes.[2]

The two papers address different questions. One follows a particular team’s decisions before and after seeing software output. The other assesses what a much larger body of research can reliably support. A positive local finding can coexist with uncertainty about widespread adoption.

Access already has a practical, conditional route in England. NICE permits four fracture-detection products, including Rayvolve, during an evidence-generation period, subject to regulatory and evidence-collection requirements. Its guidance preserves definitive radiology review and calls for local rules when clinical and AI opinions disagree. It also asks services to monitor whether implementation increases fracture-clinic referrals or requests for further imaging.[3]

Those later events connect a reporting decision to the experience of care. A corrected interpretation may change treatment, follow-up or the need to return to hospital; those links need measurement. The Singapore result is encouraging precisely because it counts changes in actual working decisions. Establishing what those changes mean for patients is the next part of the case.

Report an error or suggest a correction