RICHMOND — Psychiatrist Dr Hemlata Ranga has begun refusing new patients at her Melbourne Clinic practice in Richmond if they do not consent to the use of an AI scribe for transcription of their sessions. Her practice registration form states that patients who do not wish AI to be used must arrange for their referring doctor to refer them to a different service provider. The form also states that patients consent to the use of AI transcription software such as Heidi Health AI and Microsoft to assist with note taking, preparation of clinical letters, reports, and other clinical documents.

A spokesperson for the Melbourne Clinic said the facility does not direct its accredited psychiatrists on their use of AI. The spokesperson said psychiatrists at the clinic will disclose the use of AI to all patients and request consent prior to its use, and that if consent is not granted, AI will not be used.

Two in five general practitioners use AI-driven note taking tools, according to the Royal Australian College of General Practitioners, which said uptake of AI scribes has doubled in 12 months and coincides with increased demand for mental health services. The college also said consumer groups have raised concerns that consent conversations over the use of AI tools are lacking or that there is a power imbalance between patients and doctors.

Heidi AI has been used in 115 million sessions in the past 18 months. Heidi AI co-founder Dr Tom Kelly said the tool processes data in the country the patient is in and does not use it to train the AI or sell it to others. He said the company uses third-party testing and auditing to keep data secure and aims to ensure high standards in transcripts, but doctors still need to check their notes.

Tom Sulston, head of policy at Digital Rights Watch, said AI transcription tools were often trained on male, white, heterosexual, English-speaking users, making mistakes more common for other demographics. He said the tools are exempt from Therapeutic Goods Administration regulations because they do not diagnose patients.

"AI is not implemented to improve healthcare outcomes or patient experience, but to reduce administration costs for a clinic," Sulston said. "In this scenario, patients are not the customer, they're the product."

Sulston said regulators and legislators need to step up and provide Australians with a legally enshrined right to refuse AI systems without facing repercussions to their health.