Advanced artificial intelligence (AI) models capable of performing complex reasoning can more accurately identify a patient’s primary diagnosis, but they also pose significant risks to the accuracy and consistency of their results. A new study shows that these systems generate more false positive diagnoses than conventional models. Having doctors verify the results could help reduce errors and make AI safer to use in medical practice.
Researchers from the University of Hong Kong, Tsinghua University in China, Harvard Medical School and other medical and academic institutions have evaluated the performance of advanced artificial intelligence (AI) models in making medical diagnoses.
The research results, published on Thursday in the scientific journal Nature Communications, highlight an important problem for integrating these technologies into medicine: improving the ability to identify diseases may be accompanied by an increase in the number of erroneous diagnoses.
The study compared large-scale linguistic models (LLMs), which process and generate natural language information, with a more advanced category of systems, called logical reasoning models (LRMs), designed to solve complex problems through multiple successive stages of analysis.
In medicine, these models can analyze information from patient records to identify the main diagnosis and propose differential diagnoses, or other conditions that could explain symptoms and investigation results.
Advanced models identified the main diagnosis more accurately, but generated more errors
The researchers used an assessment system called AIMS-LDR, built on data from real electronic medical records, which included complex clinical information from 12 medical specialties.
Six pairs of AI models were compared, each pair including a model with advanced reasoning capabilities and a conventional linguistic model.
The analysis followed the accuracy of the main diagnosis, the relevance of the proposed alternative diagnoses, and the risk of generating false positive diagnoses.
Models with advanced reasoning capabilities demonstrated higher accuracy in identifying the main diagnosis. They also proposed more clinically relevant differential diagnoses.
However, the superior performances were accompanied by an increase in the number of false positive diagnoses, namely conditions indicated by AI systems without being supported by the patient’s clinical data.
According to the results, advanced models generated over 34% more false positive diagnoses than conventional linguistic models.
The researchers also found an approximately 40% reduction in the reproducibility of the results, meaning that the advanced models provided less consistent answers when evaluated repeatedly under comparable conditions.
Differences were observed even in experiments that used different wording of instructions, changes in model configurations, and alternative assessment methods.
These results suggest that a greater capacity to analyze and identify possible diseases is not automatically accompanied by a reduction in all types of diagnostic errors.
Verification of diagnoses by doctors improved outcomes
Another component of the research evaluated the possibility of reducing errors through collaboration between doctors and AI systems.
The authors analyzed a strategy in which the AI model is used to identify a larger number of possible diagnoses, and the physician subsequently verifies their accuracy and clinical relevance.
The approach aims to capitalize on the ability of advanced systems to identify conditions that could be missed, while retaining medical evaluation to confirm the diagnosis.
In a validation experiment involving 15 doctors, this strategy improved diagnostic accuracy by 20% and reduced the number of false positive diagnoses by 44%, according to the authors.
The results indicate that medical screening can help reduce errors generated by advanced models, while maintaining their benefits in identifying possible conditions.
What the results mean for the use of AI in medicine
The research highlights the importance of evaluating AI systems through multiple criteria, beyond the ability to identify the main diagnosis.
A model that correctly recognizes multiple conditions may simultaneously generate a greater number of unjustified diagnoses, with possible consequences on the investigation process and medical decisions.
Also, providing different results on repeated assessments may affect the reliability of these systems when used to support clinical decisions.
The authors believe that integrating advanced AI models into the diagnostic process should include mechanisms for verifying results and medical supervision.
The study has important limitations, however. The assessments were conducted under controlled experimental conditions, based on retrospective clinical data, and the experiment on collaboration between doctors and artificial intelligence included only 15 doctors.
The research did not evaluate the impact of using these systems on patient outcomes in current medical practice.
The authors emphasize the need for studies that track the use of these models over time, in real clinical conditions, to determine whether the observed benefits can be confirmed and whether these systems can improve the safety and quality of medical diagnosis.
The results support the development of systems in which AI contributes to identifying possible diagnoses, while the physician retains responsibility for evaluating and confirming them.
Comments