Microsoft AI Diagnostics Outpace Generalist Doctors in Complex Case Study

For patients facing difficult medical diagnoses, the arrival of advanced artificial intelligence may soon offer a second set of eyes. A new study published in Science this week reveals that a diagnostic system developed by Microsoft correctly identified complex medical conditions more than twice as often as generalist physicians. While this finding does not yet change daily routines for most people, it signals a significant shift in how the industry views the role of machine intelligence in supporting clinical decision making.

The system, known as MAI-DxO or DxO, functions by coordinating multiple AI agents that mimic the behavior of a medical panel. These agents engage in a process that includes questioning the patient, ordering necessary tests, and debating potential diagnoses among themselves. In the study, which utilized complex cases sourced from the New England Journal of Medicine, the AI system achieved an accuracy rate of 83 percent. By comparison, primary care doctors and general internists, who worked individually with available reference materials, reached the correct diagnosis only 36 percent of the time.

Mustafa Suleyman, the CEO of Microsoft AI, described the results as a meaningful step toward what he terms medical superintelligence. He emphasized that the technology represents part of a broader infrastructure designed to drive breakthroughs in healthcare. Suleyman noted that the company is collaborating with the Mayo Clinic to train a specialized foundation model for medical use, with plans to release it by next year. He expressed confidence that this model will allow for substantial progress in the field.

However, the study includes important caveats regarding the scope of the comparison. The complex cases tested were typically handled by specialists or teams of physicians in real-world settings, yet these specialists were not part of the comparison group in the study. This distinction suggests that while the AI outperformed generalists, it has not yet been pitted against the specific experts who usually manage such difficult scenarios.

The research also indicated that much of the improvement in accuracy stems from the underlying power of the AI models rather than just the orchestration system itself. Microsoft tested its system with models from several providers, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The most favorable results were achieved using OpenAI’s GPT-5 and GPT-5.5 models. When using GPT-5 on its own with a basic prompt, the system scored 78.9 percent at an estimated cost of about $8,400 per case. The DxO system reduced that cost by approximately $3,000 while improving accuracy by only four points.

Performance varied depending on the specific model used and the type of case. With the newer GPT-5.5 model, the addition of the DxO orchestration increased accuracy on complex cases from 72 percent to nearly 87 percent at a slightly lower cost. However, for emergency department cases, the difference between using the model alone and using it with the DxO assistance was minimal, with both approaches scoring around 85 percent at similar costs. The data suggests that the orchestration system provides the greatest benefit when paired with less capable models or when dealing with highly complex situations.

To move beyond simulations, Microsoft is entering a new partnership with Beth Israel Deaconess Medical Center in Boston. This initiative will evaluate the clinical reasoning capabilities of the system across various hospital settings. Patients will continue to receive standard care, including triage and specialist consultations. Those who consent will also interact with the AI system. Microsoft and its partners will then compare the AI’s recommendations against the decisions made by human doctors.

The company positions this work as part of a larger manifesto to build medical superintelligence. This concept is defined as the capacity to surpass the strongest human-led health systems in medical reasoning and coordination across the entire arc of care. Microsoft aims to move beyond simple benchmarking tasks and integrate directly with clinicians and healthcare systems. Suleyman believes that progress in this area will accelerate rapidly once real-world usage begins to identify and patch gaps in the models, a process he compares to the rapid development seen in coding and chat applications over the last several years.

As the industry moves forward, Microsoft joins other major tech firms in claiming that AI can match or outperform human doctors. The company already sells tools for dictation and radiology to clinicians and views this new diagnostic system as a natural expansion of those efforts. Suleyman stated that the goal is to become the leader in medical clinical decision support products in the coming years, leveraging the technology to provide foresight that is valuable to practicing clinicians.