One-year review of ambient scribe technology at The Permanente Medical Group finds evidence it could reduce documentation burden
A year into the rollout of a generative artificial intelligence (AI) scribe for notetaking, doctors with The Permanente Medical Group (TPMG) in Northern California have used the technology more than 2.5 million times to assist with visits. Surveys of physician users and their patients found most were pleased with the technology; an analysis also found AI scribe use saved nearly 16,000 hours in documentation time.
The AI scribe was offered to TPMG doctors both to reduce the burden of notetaking during medical visits, and to free them from focusing on the computer keyboard while with patients. Metrics and observations from the first year of use were reported in an article in NEJM Catalyst.
“We have an opportunity and obligation to take advantage of innovative AI that improves patient care and augments our physicians’ capabilities, while supporting their wellness,” said analysis co-author Kristine Lee, MD, TPMG associate executive director of virtual medicine and technology.
Patients are informed that the AI scribe is in use, and doctors have the opportunity to review and edit the tool’s suggested visit summary to ensure accuracy.
Despite the involvement of AI, the app does not make decisions or recommendations to the doctor about patient care. The AI technology involved is generative AI applied to conversations to produce transcripts and summaries in a medical setting, explained co-author Vincent Liu, MD, MSc, a research scientist with the Kaiser Permanente Division of Research (DOR) and TPMG chief data officer. “We have now shown that this technology alleviates workloads for doctors,” Liu said. “Both doctors and patients highly value face-to-face contact during a visit, and the AI scribe supports that.”
Between October 2023 and December 2024, the tool was used by 7,260 TPMG physicians to assist in 2,576,627 patient encounters. Some were particularly heavy users — 3,447 physicians used AI scribes in at least 100 patient encounters each.
Scribe users were of all ages, experience levels, and medical specialties. They were more likely to be in mental health, emergency medicine, or primary care. Allergy and cardiology were also among the top specialties. Physicians least likely to use the scribe were in infectious disease, obstetrics/gynecology, and urology.
“We found the highest adoption rates in departments that typically suffer from the highest levels of documentation and burnout,” said lead author Aaron Tierney, PhD, a DOR staff scientist.
The authors tracked whether AI scribe use was associated with changes in physician time spent on documenting patient visits. They found AI scribe users had reductions in time spent in the EHR outside of the 7 a.m. to 7 p.m. workday and time spent in notetaking per appointment. “The users who benefitted the most were also the highest volume users of the technology,” Tierney said. “Over time, their time savings substantially surpassed the time savings among their peers who used the technology infrequently or not at all.”
Doctors, patients surveyed
Adult and family medicine physicians were surveyed about their use of the scribe, and 102 responses were received. Two-thirds said they use the technology 5 or more days per week, while 8% had never used it. Most (63%) said they used in every one of their in-person visits, and another 29% using it in most visits. They were less likely to use the scribe in telemedicine visits.
Survey respondents overwhelmingly (88%) said AI scribes had a positive impact on their visit interactions. “The feedback from clinicians cited savings in time spent on work and reduced cognitive load, and that the AI scribe helped them recall details of conversations with patients,” Tierney said.
Surveys were also given to patients after a visit that involved a scribe, with 118 responses. More than a third said their doctor spent more time than usual speaking directly to them, while 56% reported no effect. About half (47%) thought their doctor spent less time looking at the computer screen. Two-thirds said they were comfortable with the technology being used, 26% were neutral, and 8% said they had some level of discomfort with it.
The authors also reported differences in adoption of AI scribe after the organization changed technology vendors between the period of an initial 10-week pilot in 2023 and the rollout in 2024. The pace of adoption continued to increase during the rollout with no decrease with the new vendor’s product.
The technology is expected to have enhancements and customization over time that could increase the number of users, the authors said.
The same authors previously published a report in NEJM Catalyst, in February 2024, about lessons learned during the 10-week pilot period, with similar findings as those reported for the complete rollout.
In that article, the authors described efforts to evaluate the accuracy of the AI scribe summaries. They analyzed a sample of 35 of the tool’s AI-generated transcripts, scoring them on average 48 out of 50 on measures including lack of bias, consistency, succinctness, and accuracy. The assessment found few incidents of hallucinations, or false information introduced by the AI.
Additional co-authors were Gregg Gayre, MD, Chief Technology Officer TPMG, Brian Hoberman, MD, MBA, executive vice president and chief information officer of The Permanente Federation, Britt Mattern, MBA, managing director of Kaiser Permanente HealthConnect, Manuel Ballesca, MD, of TPMG, and Patricia Kipnis, PhD, of the Division of Research.
###
About the Kaiser Permanente Division of Research
The Kaiser Permanente Division of Research conducts, publishes, and disseminates epidemiologic and health services research to improve the health and medical care of Kaiser Permanente members and society at large. KPDOR seeks to understand the determinants of illness and well-being and to improve the quality and cost-effectiveness of health care. Currently, DOR’s 720-plus staff, including 73 research and staff scientists, are working on nearly 630 epidemiological and health services research projects. For more information, visit divisionofresearch.kp.org or follow us @KPDOR.
Comments (0)