AI Models Are Already Outscoring Doctors on Diagnostic Tests
OpenAI's o1 model scored 78% on real-world diagnoses versus 30% for physicians, and Mayo Clinic AI caught pancreatic cancer up to three years early — recent studies show AI pulling sharply ahead in clinical accuracy.
A run of recent studies suggests AI systems aren't just catching up to doctors on diagnosis — they've pulled ahead, and by a wide margin.
In testing highlighted by Harvard Medical School researchers, OpenAI's o1 model correctly diagnosed real-world cases 78% of the time, compared with 30% for physicians working the same cases. On a set of clinical vignettes, the gap was even larger: o1 scored 89% against a 34% average for human doctors. In a tougher, real-time setting — emergency room triage — o1 hit 67.1% accuracy, ahead of two attending physicians who scored between 50.3% and 55.3%.
"We tested the AI model against virtually every benchmark, and it eclipsed both prior models and our physician baselines," said Harvard Medical School professor Arjun K. Manrai, whose team ran the comparisons.
The pattern isn't limited to one model or one kind of test. A Swedish study found AI-enhanced mammogram screening improved breast cancer detection rates over standard radiologist review. Separately, an AI system at the Mayo Clinic identified pancreatic cancer up to three years before it would have otherwise been caught by clinicians — a lead time that can be the difference between a treatable case and a terminal one.
None of this means AI is ready to replace physicians outright. There's a real regulatory headwind building: several US states have moved to restrict AI from playing any role in mental health care, out of caution about accountability and harm. AI-optimists argue that blanket restrictions risk closing off a technology that, in test after test, is now demonstrably catching what doctors miss — and that regulation should target genuine risk, not the mere presence of AI.
Source: Reason.com via Yahoo News