Pancreatitis, party mishaps and panic — the clinical cases where AI fails
Canadian researchers put the AI model through its paces on 150 clinical cases.
Imaging misinterpretation was meant to fall quickly to AI’s diagnostic supremacy, but a study of 150 clinical challenges suggests that imaging, numbers and party hijinks are actually the areas where AI struggles.
Researchers from Western University in Ontario, Canada, put free AI software ChatGPT up against the multiple-choice challenges from website Medscape, finding that the AI correctly answered 49%.
In 74% of cases, ChatGPT successfully ruled out at least some of the incorrect answers as well.
In one example, it correctly diagnosed ankylosing spondylitis in an 58-year-old with elevated CRP and ESR, negative human leucocyte antigen B27 and bilaterally tender sternoclavicular joints that improved with NSAIDs.
