Editor’s notes
In recent weeks, growing MTS has meant engaging with medical creators across the web. The great AI divide is prominent — does our understanding stem as far as what each tool promises to deliver? Like most things in medicine, we must make our own inquiry. With AI scribes knocking on our doorstep, this is where we start. Expect an MTS series exploring AI in clinical medicine
Why the AI Infiltration in Medicine?
Clinical documentation and administrative automation is currently the fastest-growing healthcare technology segment. With AI scribing tools, otherwise known as ambient AI, quietly recording and transcribing in the background of medical consultations. With a mere handful of U.S companies alone operating in this sector generated 600 million in 2025, with an anticipated 2.4x annual revenue return.
We could not easily abbreviate the layers of challenges creating astronomical demand in healthcare. A changing landscape has resulted in high clinical demand, paired with insufficient resource allocation, with those in care burning out faster than ever. In fact, the WHO has stated that countries at all levels of socioeconomic development are facing healthcare workforce challenges, including education, employment, deployment, and retention. Understandably, there are multifactorial degrees of variation by country; however, there are estimates of a projected shortfall of 11 million health workers by 2030. With the impact anticipated to disproportionately affect low- and lower-middle-income countries, of which, importantly, medical workforce demand from high-income countries is a factor.
A growing body of studies exploring the use of AI scribes by clinicians demonstrate reduced clinician time spent on electronic health records (EHRs), reduced documentation time, and a reduction in self-reported burnout. This naturally opens up avenues to consider the downstream impact of such tools, in both financial, productivity and retention of the health workforce.
As clinicians, it is important to take a step back and observe the impact from a holistic point of view, as other studies would demonstrate the counteracting risks of each proposed benefit AI offers currently. The implications of AI tools warrant consideration before we endorse or implement them in our own clinical practice, as the case for caution gathers traction.
Scepticism in Practice
A class action lawsuit, Washington et al v. Sutter Health, has been started in the U.S The core allegations include illegal use of patient identifiable information by third parties. This includes a claim of improper consent, with no consent sought or obtained by the clinicians prior to use of the system to facilitate the consultation. The added layer is the breach of the current U.S legal framework regarding confidential patient data privacy, and the due process for this to be sourced from medical providers. If we make a pale comparison to existing EHR systems, the frameworks around secure informed consent are much clearer and rigorous. Ranging from consenting for EHR onboarding to accessing records and consenting to share specific data with informed consent. Clinicians routinely obtain consent to use patient information in clinical research, even when the information used is non-identifiable. Therefore, when using a novel tool that records personal and sensitive conversations, it is important to consider liability and how your utilisation can equate to granting permission-by-proxy.
In the event that a clinician has armed themselves sufficiently, with the relevant information spanning technical, regulatory and the risks vs benefits profile, as well as consenting every patient pre-consultation, how much time is saved? Further considerations are also required, for example, the rapid rate of AI enhancement whereby ‘up to date’ can be out of date in a short period. This raises questions on the rate of updating our information for patients, and how we can track the emerging implications of use, without a longitudinal view.
It is also important to account for the human error factor, given that AI scribes produce a litany of errors; the clinician has added responsibility of being the validator for AI-written content, in addition to their ordinary responsibility of attesting to the validity and truth of written clinical documentation, important for both capturing an accurate and honest record of an encounter, for both the patient and doctor. Legally binding documents used to retrospectively account for adverse events, decisions, auditing, learning, etc. How can we account for the human error in our validation? Can one say that AI scribe error increases human error levels? With a proposed mechanism of consistent AI error levels vetted by time-poor human clinicians, with reduced vigilance levels for a tool designed to assist them. What could be the implications of a clinician acting on a previous encounter note, with a false clinical finding produced by AI and missed by a colleague? Could the potentially huge implications be worth any time saved?
An Overview of Risks in AI Scribing
Omissions: discussion content is missing from the report.
Hallucinations: fabricated information with no basis on the discussion that occurred.
Factual inaccuracies: misreporting details such as the date of onset.
Systematic performance disparities: AI scribing tools produce higher error rates for patients whose speech patterns differ from the model's training data, typically Standard American English.
Patient trust: personal perceptions of being recorded and on AI itself can impact trust and open communication.
Lack of clinician knowledge for proper consenting: factors such as data privacy and risk vs benefit required for proper consenting.
Human error: AI scribing content verified by doctors has been found to contain errors on a wide basis.
What should you do?
Conduct your own literature review - does the ambient scribe meet the lawful basis for processing patient data? What does your regulator or local guideline say about the use of ambient scribes? How can you establish quality control? Is there any risk posed to your patients?
Follow the necessary steps for adequate consent, with transparency of the known and unknown.
Be aware of the rates of errors and the disparity levels between AI Scribe companies, scrutinise the quality of research, and make an informed decision on use.
Tailor AI tool use, non-clinical administrative burden - low risk and high reward.
Extra vigilance is required when using tools such as AI scribing- vet your documentation in a timely manner, when you are able to recall the details of the consultation.
Be aware of error duplication- do not let your documentation be the source of one.
