I worry doctors and patients are turning off their brains in favour of AI

Last month, a South Australian coroner recommended AI be used to review medical imaging after radiologists failed to detect signs of gastric adenocarcinoma while looking for recurrence of a patient’s bladder cancer.

The coroner likened AI’s role in radiology to “a spell check”.

But GP Dr Rachel Glasson fears relying too heavily on AI could come at another cost: our willingness — and ability — to think for ourselves.

In a somewhat chilling development, two CEOs of major AI companies have warned that AI is likely to wipe us all out within 10 years if appropriate guardrails are not put in place — and soon.

My son is currently studying computer science at university, so I asked him about this, and to my surprise, he said I had nothing to worry about because “it’ll never happen”.

I remember all the Y2K panic that turned out to be a major non-event, but I’m not convinced that the AI panic is going to have the same outcome.

The biggest fear I have is that AI is going to make humans stop using their brains and that our ability to think independently will atrophy as a result.

Leonardo da Vinci knew this in 1500, writing: “Iron rusts from disuse; stagnant water loses its purity and, in cold weather, becomes frozen; even so does inaction sap the vigours of the mind.”

In medicine, I’ve already started noticing how AI is having this effect.

First, the increasing uptake of AI scribe tools to make consultation notes.

With the help of Lyrebird or Heidi, the staccato trickling of your average time-poor, hunt-and-peck typist can become an eloquently flowing stream of prose. I’ve seen examples of this working very well.

I’ve also seen examples where the relevant clinical details end up drowning in a bloated torrent of AI verbosity. If I need to scroll down more than once to read the previous visit’s notes, I’m probably not going to read it at all.

The process of making my own concise clinical notes helps me organise my thoughts and check that nothing essential has been missed. If AI were to just dump everything on the page for me, I’m not sure the effect would be the same.

Saving time at the expense of accuracy, memory and learning? No thanks.

Second, it is not just doctors who are turning off their brains in favour of AI.

Increasingly, I’m seeing patients who present saying, “I wouldn’t have come, but I asked ChatGPT, and it said I should see a doctor right away.”

One was a parent with a child that had been pecked on the finger by a bush turkey about 90 minutes earlier.

The finger had sustained a tiny abrasion, the kind of thing that needs Savlon and a bandaid, not an emergency visit to the doctor.

But ChatGPT — seemingly as obsessed with sepsis as Google is with cancer — had advised a high risk of infection, so common sense went out the window and a visit to urgent care was the result.

Finally, and perhaps most ominously, I have recently noticed AI being used by radiologists.

This was inevitable, really. It makes sense that AI would be better at pattern recognition than humans, but there are significant issues when the use of AI is divorced from clinical assessment.

Working in urgent care, I order a lot of X-rays of acutely injured body parts. And sometimes, when I’m not sure what I’m seeing, I pop next door and ask the radiologist on duty what they think.

About a month ago, I saw a radiologist use an AI tool for the first time — a box that, when dragged over the image, points out possible abnormal findings — and was impressed, until more recently, when I realised what kind of problem this tool can also create.

The reason urgent care doctors are usually (and often much to the patient’s disappointment) not willing to order anything other than immediately necessary tests is because the results — particularly any abnormal ones — will almost certainly become someone else’s problem to check and manage.

You can imagine how much fun this is: you have never seen the patient and yet must figure out from the notes (sometimes an AI soliloquy, sometimes semi-gibberish) if the patient was advised of the finding and treated appropriately.

I always check the report first and the notes second, so if the report has no findings, I can skip the notes altogether.

Last week, three reports came in with significant findings, so I called all the patients involved — only to discover these were AI findings that had nothing to do with site of injury.

Quizzing a parent about exactly where their kid’s hand hurts reminded me of trying to get a patient to take their own pulse over telehealth during COVID-19: a painful and frustrating waste of time.

Recently, AusDoc reported on a coroner’s recommendation that radiologists use AI to avoid missing unexpected but significant findings.

I reckon, for each such instance, there would be at least 10,000 of the opposite.

Are we ready to surf the tidal wave of incidentalomas this will create, or are we more likely to drown in the resulting flood?

Strap on your floaties because I think we’re about to find out.

Dr Rachel Glasson is a GP in Sydney, NSW.

You're missing out on this article.