What Will AI Do To The Doctors? (Redux)
Is your brain exercising less with AI assistance? If so, does this diminished cognitive engagement affect our skillsets? Find out in this blog by Kiran Raj Pandey.
A few days ago, we discussed how reliance on an AI tool to localise adenoma appeared to diminish doctors’ endoscopy skills when they were not using the AI tool. Another study appears to corroborate this finding of rapid declining of human skillsets when relying on an AI tool.
In a study from MIT, published as a pre-print in arxiv, researchers set out to study brain activity of study participants who completed an essay with and without the assistance of ChatGPT. To do this, they enrolled 54 participants, and divided them into 3 groups. The first group was asked to write an essay on a topic without the assistance of an AI tool, another group was asked to complete the same task with the help of a search engine, and a third was asked to complete the task with the help of an LLM like ChatGPT.
Researchers then set out to measure brain matter activity in all of these study participants using ElectroEncephalographs (EEG). EEGs detect the electrical activity in the brain and can be a surrogate indicator of cognitive load in the brain. Researchers found that study participants who used the assistance of an LLM to complete the task had markedly low neural activity and participants who did not use the assistance of an external tool had the highest neural activity as measured by an EEG. Participants who used a search engine had moderate neural activity.
In another leg of the study, the researchers switched the LLM-assisted group to brain-only group and the brain-only group to the LLM assisted group. Researchers found that the Brain to LLM group had higher memory recall activity while the LLM to Brian group showed reduced alpha and beta activity in the EEG, demonstrating cognitive under engagement.
Furthermore, while the Brain only group showed greater ownership of their work and had a greater ability to recall and quote their work, the LLM group demonstrated less ownership of their work and had a poorer ability to quote their work.
In addition, over a 4 month period after the study, the LLM-only group consistently underperformed - as measured by neural, linguistic, behavioural tasks.
These findings from the MIT study are consistent with what the Lancet study we discussed last week suggested–that doctors tend to rapidly lose their skills in identifying an adenoma in the colon when they begin to rely on an AI tool to help them identify adenomas.
These findings should raise our hair. What are we gaining with AI, and what are we giving up? Comment your thoughts below.
Kiran Raj Pandey is a physician and a health services & systems researcher. Learn more about him and his work at kiranrajpandey.com.
 | A guest post by
|