Kaiser Permanente’s Desktop Medicine team gets AI assist in handling millions of patient messages, resolving a third before they get to doctors’ inboxes
An artificial intelligence (AI) powered system that categorizes patient messages assisted regional staff in resolving about a third of the messages so they never reached the inboxes of busy doctors. The Desktop Medicine project carried out by The Permanente Medical Group (TPMG) was described in JAMA Network Open.
The study looked at nearly 5 million messages sent over 5 months in 2023 and found three-quarters were labeled by the AI system; of the total, 1.5 million messages were resolved by regional staff.
The system, leveraging AI methods, was designed to combat the challenge of handling the expanding inboxes of physicians who must evaluate and respond to messages along with their other duties of caring for patients, documenting care, and more, said senior author Kristine Lee, MD, TPMG associate executive director of virtual medicine and technology.
“Physicians in a modern health care system are managing in-person patient care alongside virtual care such as video visits and secure messaging,” Lee said. “While doctors welcome expanded ways to connect with their patients, they also need assistance with the changing workload. Our program found that AI tools may offer tools that could help when paired with robust workflows.”
The Desktop Medicine program is one part to the organization’s efforts to address the tremendous increase in physician electronic workload, said Gregg Gayre, MD, chief technology officer for TPMG. “We are exploring the safe and careful application of augmented intelligence — a term we prefer over artificial intelligence — to problems such as an unprecedented volume of electronic messages being sent to our medical teams,” Gayre said. “As we explore, we also measure and report our findings in evaluations such as this one. This transparency is crucial for a large medical group like ours, at the leading edge of using AI to improve health care.”
The program uses natural language processing and machine learning to label messages by topic and forward them to be managed by a regional team of medical assistants, teleservice representatives, pharmacists, and doctors. The AI tools include custom-built language processing models designed by TPMG programmers for this purpose.
A set of about 20,000 messages was used to train the natural language processing system, with support from triage nurses and physicians to label them. The labels include topics such as medication requests or respiratory symptoms. Labels were also used to identify the more urgent messages that should be sent along to doctors on an expedited basis.
Once trained, the system was integrated with Kaiser Permanente Northern California’s electronic health record for adult and family medicine in May 2022, expanding to pediatric messages later in the year. The researchers chose 4.7 million patient messages sent between April and August 2023 to analyze for labeling consistency and management.
This study lays the foundation for more applications of AI to enhance the way that we care for our patients through all channels.
—Jennifer Manickam, MD
Of the messages studied, 77.6% were labeled with one of about 30 topics. The most common labels were medications (32.8%), skin conditions (9.3%), messages with attachments (8.3%), and emergent content for expedited review (7.6%).
Of the 4.8 million messages evaluated by regional desktop medicine staff, 31.9% were resolved — never reaching physician inboxes — by regional medical assistants or teleservice representatives (21.4% of total), physicians (7.2%), or pharmacists (3.3%).
“The system was effective in identifying about 1.5 million messages that would have otherwise gone on to physician inboxes,” said study lead author Vincent Liu, MD, MS, a research scientist with the Kaiser Permanente Division of Research and TPMG chief data officer. These may be simple queries such as pharmacy hours or medication refills, he said.
Other advantages of labeling messages, the study authors said, include continual refinement of the system and the capability to identify concerning symptoms in patient messages so they can be addressed. The system also provides a health system with the ability to identify emerging health trends — such as infectious diseases — contained in patient requests.
Future improvements could include the use of newer large language models such as GPT-4 and addressing messages with multiple topics and those with no category labels.
“We certainly have not solved the problem of high volumes of inbox messages for our physicians, but it is gratifying to see how we could use technology to resolve these messages in real-time,” said co-author Jennifer Manickam, MD, TPMG chair of adult and family medicine technology leads. “This study lays the foundation for more applications of AI to enhance the way that we care for our patients through all channels.”
The study was funded by The Permanente Medical Group.
Additional co-authors were Pamela Kaercher, PhD, Eric Smallberg, MAIA, Kanishka Bhutani, MSc, and Michelle Mancha, MS, of The Permanente Medical Group.
# # #
About the Kaiser Permanente Division of Research
The Kaiser Permanente Division of Research conducts, publishes and disseminates epidemiologic and health services research to improve the health and medical care of Kaiser Permanente members and society at large. It seeks to understand the determinants of illness and well-being, and to improve the quality and cost-effectiveness of health care. Currently, DOR’s 600-plus staff is working on more than 450 epidemiological and health services research projects. For more information, visit divisionofresearch.kaiserpermanente.org or follow us @KPDOR.
This Post Has 0 Comments