Artificial intelligence (AI) has become a cornerstone in modern healthcare, offering rapid and precise analysis of medical data, from X-rays to patient histories, often detecting issues long before they’re visible to human eyes. However, a groundbreaking study published on December 20, 2024, in the BMJ suggests that AI systems, much like humans, may experience a decline in cognitive performance as they age. This revelation casts doubt on the long-held belief that AI will seamlessly replace human doctors in critical medical roles.
The study, which evaluated several leading AI models, including OpenAI’s ChatGPT, Anthropic’s Sonnet, and Alphabet’s Gemini, utilized the Montreal Cognitive Assessment (MoCA) — a tool traditionally employed to detect cognitive impairments such as Alzheimer’s or dementia in humans. The MoCA assesses various mental faculties, including memory, attention, language, spatial reasoning, and executive function, through tasks like drawing clock faces, recalling word lists, and performing serial subtraction.
While the AI models demonstrated proficiency in areas like language and attention, they struggled significantly with visual-spatial tasks and executive functions, which require higher-order thinking and problem-solving. For instance, the latest iteration of ChatGPT (version 4) achieved a near-passing score of 26 out of 30, while older models like Gemini 1.0 scored a mere 16, highlighting a stark decline in performance over time.
The researchers emphasized that their findings are observational and not directly comparable to human cognitive decline, given the fundamental differences between AI and human brains. However, they warned that these shortcomings could represent a “critical vulnerability” in AI systems, particularly in clinical settings where visual abstraction and executive decision-making are paramount.
This study also introduces an intriguing possibility: as AI systems age and exhibit signs of cognitive deterioration, human neurologists might find themselves diagnosing and treating not just patients, but the very machines designed to assist them.
The implications of this research are far-reaching. While AI continues to revolutionize healthcare, its limitations underscore the need for human oversight, especially in complex medical diagnostics. As the authors aptly noted, “The cognitive decline observed in older AI models challenges the notion that these systems can fully replace human expertise, particularly in fields requiring nuanced judgment and adaptability.”
For now, the dream of AI entirely supplanting human doctors remains just that — a dream. Instead, the future may lie in a collaborative approach, where AI complements human skills rather than replaces them, ensuring both efficiency and reliability in patient care.
