CINCINNATI — Associate professor Nelly Elsayed at the University of Cincinnati published a peer-reviewed study on July 14, 2026, identifying critical socio-technical risks associated with AI-powered clinical speech-to-text systems used in healthcare documentation. The research, appearing in the International Journal of Medical Informatics, highlights concerns about transparency, privacy, and reliability that could affect patient safety and clinician trust.

Elsayed’s paper, titled “Socio-technical risks of clinical speech-to-text systems: Transparency, privacy, and reliability challenges in AI-driven documentation,” synthesized findings from existing research, ethical guidelines, and government regulations on AI adoption in healthcare settings. The study examined how artificial intelligence tools are integrated into clinical workflows and where potential failures may arise when human oversight is insufficient.

Among the key risks identified was inconsistent disclosure and consent practices around the use of AI systems in patient encounters. The research also pointed to decreased performance of speech-to-text algorithms when processing accented or disordered speech, which may disproportionately affect certain patient populations. Additionally, the study noted that extraneous noises common in clinical environments—such as machines beeping or medical staff conversing— reduce transcription accuracy.

Elsayed emphasized that AI speech-to-text systems are often trained in idealized acoustic conditions that do not reflect real-world clinical settings. Because of this, the study found that large-language models cannot be reliably deployed without scenario-specific training that accounts for diverse accents, speech impairments, and ambient hospital sounds. Without such adaptations, the systems risk generating inaccurate or incomplete medical records.

A central concern raised in the study is the lack of human review over AI-generated clinical notes, which can lead to unchecked errors entering patient records. Elsayed argued that accountability remains unclear when mistakes occur: it is uncertain whether responsibility lies with the software developer or the treating clinician. To address these gaps, she called for stronger governance frameworks and clearer operational protocols.

“We need to have a human in the loop to check whether the text is exactly what has been spoken,” Elsayed said. “And that test needs to be done for the entire text, not just for the first couple statements.” She added that developers must provide clinicians with explicit guidance on system use. “The organization developing the system needs to give guidelines for the doctor, what they can use, what they cannot use and what to look out for.”

Clinical documentation underpins diagnosis, treatment, billing, and legal records in healthcare. Inaccurate or incomplete AI-generated notes could compromise patient care, increase clinician workload through error correction, or create liability issues without clear accountability structures. As healthcare systems increasingly adopt AI tools to reduce administrative burdens, Elsayed’s study underscores the necessity of aligning technological deployment with robust human oversight, inclusive design, and transparent consent processes.