1
|
Ma SP, Liang AS, Shah SJ, Smith M, Jeong Y, Devon-Sand A, Crowell T, Delahaie C, Hsia C, Lin S, Shanafelt T, Pfeffer MA, Sharp C, Garcia P. Ambient artificial intelligence scribes: utilization and impact on documentation time. J Am Med Inform Assoc 2024:ocae304. [PMID: 39688515 DOI: 10.1093/jamia/ocae304] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2024] [Revised: 10/29/2024] [Accepted: 11/27/2024] [Indexed: 12/18/2024] Open
Abstract
OBJECTIVES To quantify utilization and impact on documentation time of a large language model-powered ambient artificial intelligence (AI) scribe. MATERIALS AND METHODS This prospective quality improvement study was conducted at a large academic medical center with 45 physicians from 8 ambulatory disciplines over 3 months. Utilization and documentation times were derived from electronic health record (EHR) use measures. RESULTS The ambient AI scribe was utilized in 9629 of 17 428 encounters (55.25%) with significant interuser heterogeneity. Compared to baseline, median time per note reduced significantly by 0.57 minutes. Median daily documentation, afterhours, and total EHR time also decreased significantly by 6.89, 5.17, and 19.95 minutes/day, respectively. DISCUSSION An early pilot of an ambient AI scribe demonstrated robust utilization and reduced time spent on documentation and in the EHR. There was notable individual-level heterogeneity. CONCLUSION Large language model-powered ambient AI scribes may reduce documentation burden. Further studies are needed to identify which users benefit most from current technology and how future iterations can support a broader audience.
Collapse
Affiliation(s)
- Stephen P Ma
- Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - April S Liang
- Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Shreya J Shah
- Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, United States
- Stanford Healthcare AI Applied Research Team, Division of Primary Care and Population Health, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Margaret Smith
- Stanford Healthcare AI Applied Research Team, Division of Primary Care and Population Health, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Yejin Jeong
- Stanford Healthcare AI Applied Research Team, Division of Primary Care and Population Health, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Anna Devon-Sand
- Stanford Healthcare AI Applied Research Team, Division of Primary Care and Population Health, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Trevor Crowell
- Stanford Healthcare AI Applied Research Team, Division of Primary Care and Population Health, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Clarissa Delahaie
- Technology and Digital Solutions, Stanford Medicine, Stanford, CA 94305, United States
| | - Caroline Hsia
- Technology and Digital Solutions, Stanford Medicine, Stanford, CA 94305, United States
| | - Steven Lin
- Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, United States
- Stanford Healthcare AI Applied Research Team, Division of Primary Care and Population Health, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Tait Shanafelt
- Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, United States
- WellMD Center, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Michael A Pfeffer
- Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, United States
- Technology and Digital Solutions, Stanford Medicine, Stanford, CA 94305, United States
| | - Christopher Sharp
- Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Patricia Garcia
- Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, United States
| |
Collapse
|
2
|
Shah SJ, Devon-Sand A, Ma SP, Jeong Y, Crowell T, Smith M, Liang AS, Delahaie C, Hsia C, Shanafelt T, Pfeffer MA, Sharp C, Lin S, Garcia P. Ambient artificial intelligence scribes: physician burnout and perspectives on usability and documentation burden. J Am Med Inform Assoc 2024:ocae295. [PMID: 39657021 DOI: 10.1093/jamia/ocae295] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2024] [Revised: 10/28/2024] [Accepted: 11/18/2024] [Indexed: 12/17/2024] Open
Abstract
OBJECTIVE This study evaluates the pilot implementation of ambient AI scribe technology to assess physician perspectives on usability and the impact on physician burden and burnout. MATERIALS AND METHODS This prospective quality improvement study was conducted at Stanford Health Care with 48 physicians over a 3-month period. Outcome measures included burden, burnout, usability, and perceived time savings. RESULTS Paired survey analysis (n = 38) revealed large statistically significant reductions in task load (-24.42, p <.001) and burnout (-1.94, p <.001), and moderate statistically significant improvements in usability scores (+10.9, p <.001). Post-survey responses (n = 46) indicated favorable utility with improved perceptions of efficiency, documentation quality, and ease of use. DISCUSSION In one of the first pilot implementations of ambient AI scribe technology, improvements in physician task load, burnout, and usability were demonstrated. CONCLUSION Ambient AI scribes like DAX Copilot may enhance clinical workflows. Further research is needed to optimize widespread implementation and evaluate long-term impacts.
Collapse
Affiliation(s)
- Shreya J Shah
- Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, United States
- Stanford Healthcare AI Applied Research Team, Division of Primary Care and Population Health, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Anna Devon-Sand
- Stanford Healthcare AI Applied Research Team, Division of Primary Care and Population Health, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Stephen P Ma
- Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Yejin Jeong
- Stanford Healthcare AI Applied Research Team, Division of Primary Care and Population Health, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Trevor Crowell
- Stanford Healthcare AI Applied Research Team, Division of Primary Care and Population Health, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Margaret Smith
- Stanford Healthcare AI Applied Research Team, Division of Primary Care and Population Health, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - April S Liang
- Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Clarissa Delahaie
- Technology and Digital Solutions, Stanford Medicine, Stanford, CA 94305, United States
| | - Caroline Hsia
- Technology and Digital Solutions, Stanford Medicine, Stanford, CA 94305, United States
| | - Tait Shanafelt
- Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, United States
- WellMD Center, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Michael A Pfeffer
- Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, United States
- Technology and Digital Solutions, Stanford Medicine, Stanford, CA 94305, United States
| | - Christopher Sharp
- Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Steven Lin
- Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, United States
- Stanford Healthcare AI Applied Research Team, Division of Primary Care and Population Health, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Patricia Garcia
- Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, United States
| |
Collapse
|
3
|
Levi BH, Ekpa N, Lin A, Smith CW, Volpe RL. The Experience of Medical Scribing: No Disparities Identified. ADVANCES IN MEDICAL EDUCATION AND PRACTICE 2024; 15:153-160. [PMID: 38476633 PMCID: PMC10929157 DOI: 10.2147/amep.s439826] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 09/12/2023] [Accepted: 12/20/2023] [Indexed: 03/14/2024]
Abstract
Introduction The chronic failure to significantly increase the number of underrepresented minorities (URM) in medicine requires that we look for new mechanisms for channelling URM students through pre-medical education and into medical school. One potential mechanism is medical scribing, which involves a person helping a physician engage in real-time documentation in the electronic medical record. Methods As a precursor to evaluating this mechanism, this survey pilot study explored individuals' experiences working as a medical scribe to look for any differences related to URM status. Of 248 scribes, 159 (64% response rate) completed an online survey. The survey was comprised of 11 items: demographics (4 items), role and length of time spent as a scribe (2 items), and experience working as a scribe (5 items). Results The vast majority (>80%) of participants reported that working as a medical scribe gave them useful insight into being a clinician, provided valuable mentoring, and reinforced their commitment to pursue a career in medicine. The experiences reported by scribes who identified as URM did not differ from those reported by their majority counterparts. Discussion It remains to be seen whether medical scribing can serve as an effective pipeline for URM individuals to matriculate into medical school. But the present findings suggest that the experience of working as a medical scribe is a positive one for URM.
Collapse
Affiliation(s)
- Benjamin H Levi
- Department of Humanities, Penn State College of Medicine, Hershey, PA, USA
- Department of Pediatrics, Penn State College of Medicine, Hershey, PA, USA
| | - Ndifreke Ekpa
- University of Houston, HCA Houston Healthcare Kingwood, Houston, TX, USA
| | - Andrea Lin
- Penn State College of Medicine, Hershey, PA, USA
| | | | - Rebecca L Volpe
- Department of Humanities, Penn State College of Medicine, Hershey, PA, USA
| |
Collapse
|
4
|
Kang C, Sarkar IN. Interventions to Reduce Electronic Health Record-Related Burnout: A Systematic Review. Appl Clin Inform 2024; 15:10-25. [PMID: 37923381 PMCID: PMC10764123 DOI: 10.1055/a-2203-3787] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2023] [Accepted: 11/02/2023] [Indexed: 11/07/2023] Open
Abstract
BACKGROUND Electronic health records are a significant contributing factor in clinician burnout, which negatively impacts patient care. OBJECTIVES To identify and appraise published solutions that aim to reduce EHR-related burnout in clinicians. METHODS A literature search strategy was developed following the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses. Six databases were searched for articles published between January 1950 and March 2023. The inclusion criteria were peer-reviewed, full-text, English language articles that described interventions targeting EHR-related burnout in any type of clinician, with reported outcomes related to burnout, wellness, EHR satisfaction, or documentation workload. Studies describing interventions without an explicit focus on reducing burnout or enhancing EHR-related satisfaction were excluded. RESULTS We identified 44 articles describing interventions to reduce EHR-related burnout. These interventions included the use of scribes, EHR training, and EHR modifications. These interventions were generally well received by the clinicians and patients, with subjective improvements in documentation time and EHR satisfaction, although objective data were limited. CONCLUSION The findings of this review underscore the potential benefits of interventions to reduce EHR-related burnout as well as the need for further research with more robust study designs involving randomized trials, control groups, longer study durations, and validated, objective outcome measurements.
Collapse
Affiliation(s)
- Chaerim Kang
- Center for Biomedical Informatics, Brown University, Providence, Rhode Island, United States
| | - Indra Neil Sarkar
- Center for Biomedical Informatics, Brown University, Providence, Rhode Island, United States
- Rhode Island Quality Institute, Providence, Rhode Island, United States
| |
Collapse
|
5
|
Getting by With Less: How to do More With Less Staff After COVID-19? Am J Gastroenterol 2022; 117:1547-1549. [PMID: 36194043 DOI: 10.14309/ajg.0000000000001853] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/09/2022] [Accepted: 05/31/2022] [Indexed: 12/11/2022]
|
6
|
Monteith C, Ní Bhuinneáin M, Geary MP. Mitigating the stress of transition: An exploration of the effects and effectiveness of a preparatory course for junior obstetric trainees transitioning to senior training roles. Eur J Obstet Gynecol Reprod Biol 2022; 276:154-159. [PMID: 35914418 DOI: 10.1016/j.ejogrb.2022.07.019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2022] [Revised: 07/20/2022] [Accepted: 07/21/2022] [Indexed: 11/16/2022]
Abstract
BACKGROUND Attrition amongst obstetrics trainees is high worldwide and attributed to sources of stress and burnout. The role of formal education and simulation as a means to prepare trainees for stressful periods such as transition into senior roles is underexplored. OBJECTIVE This study set out to explore whether the creation of a dedicated educational intervention might positively influence burnout and self-estimated preparedness for practice among obstetric trainees transitioning into more senior roles. STUDY DESIGN A six-week preparatory training programme for year 2 trainees was created specifically for this study. The intervention used the flipped classroom design incorporating online learning that prepared participants for six simulation-based workshops. Participants were randomised by training cluster into an intervention group (n = 4) who participated in the educational intervention and a control group (n = 7) who received standard online and workplace training. The effects on trainee well-being was assessed using the Maslach burnout inventory (MBI) and a self-report questionnaire estimating preparedness for practice. Technical and non-technical skills were assessed using standardised OSAT and NOTSS assessment tools. The primary outcomes were MBI and preparedness for practice scores. Secondary outcomes included OSAT and NOTSS scores. Group comparisons were made using by t-test or Pearson Chi2 analysis where appropriate. RESULTS The study indicated a positive, non-significant trend in pre-post burnout scores in the intervention group. The following improving trends were noted in all subscales: emotional exhaustion 21.5 ± 2.6 (pre-intervention 23 ± 6.2); depersonalisation 9.8 ± 4.0 (pre-intervention 12.3 ± 2.8); personal accomplishment 35.5 ± 6.51 (pre-intervention 33 ± 5.5). The educational intervention engendered an increase in self estimated preparedness for practice amongst the intervention group (p = 0.006). From a training perspective, increased preparedness was noted for the following practical skills: forceps delivery (p = 0.0001), rotational forceps delivery (p = 0.02), delivery of twins vaginally (p = 0.0007) and performing a pudendal block (p = 0.001). CONCLUSION This is one of the first studies to investigate whether the provision of a targeted training module can improve burnout scores and preparedness for practice amongst obstetrics trainees at an important time of transition. The positive but largely non-significant findings of this study should be examined in larger longitudinal and adequately powered studies.
Collapse
Affiliation(s)
- Cathy Monteith
- Department of Obstetrics and Gynaecology, Rotunda Hospital, Dublin, Ireland; Department of Obstetrics and Gynaecology, National Maternity Hospital, Dublin Ireland.
| | - Méabh Ní Bhuinneáin
- Department of Obstetrics and Gynaecology, Rotunda Hospital, Dublin, Ireland; Department of Obstetrics and Gynaecology, National Maternity Hospital, Dublin Ireland.
| | - Michael P Geary
- Department of Obstetrics and Gynaecology, Rotunda Hospital, Dublin, Ireland; Department of Obstetrics and Gynaecology, National Maternity Hospital, Dublin Ireland.
| |
Collapse
|
7
|
Nguyen OT, Turner K, Apathy NC, Magoc T, Hanna K, Merlo LJ, Harle CA, Thompson LA, Berner ES, Feldman SS. Primary care physicians' electronic health record proficiency and efficiency behaviors and time interacting with electronic health records: a quantile regression analysis. J Am Med Inform Assoc 2021; 29:461-471. [PMID: 34897493 PMCID: PMC8800512 DOI: 10.1093/jamia/ocab272] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2021] [Revised: 11/05/2021] [Accepted: 11/23/2021] [Indexed: 12/15/2022] Open
Abstract
OBJECTIVE This study aimed to understand the association between primary care physician (PCP) proficiency with the electronic health record (EHR) system and time spent interacting with the EHR. MATERIALS AND METHODS We examined the use of EHR proficiency tools among PCPs at one large academic health system using EHR-derived measures of clinician EHR proficiency and efficiency. Our main predictors were the use of EHR proficiency tools and our outcomes focused on 4 measures assessing time spent in the EHR: (1) total time spent interacting with the EHR, (2) time spent outside scheduled clinical hours, (3) time spent documenting, and (4) time spent on inbox management. We conducted multivariable quantile regression models with fixed effects for physician-level factors and time in order to identify factors that were independently associated with time spent in the EHR. RESULTS Across 441 primary care physicians, we found mixed associations between certain EHR proficiency behaviors and time spent in the EHR. Across EHR activities studied, QuickActions, SmartPhrases, and documentation length were positively associated with increased time spent in the EHR. Models also showed a greater amount of help from team members in note writing was associated with less time spent in the EHR and documenting. DISCUSSION Examining the prevalence of EHR proficiency behaviors may suggest targeted areas for initial and ongoing EHR training. Although documentation behaviors are key areas for training, team-based models for documentation and inbox management require further study. CONCLUSIONS A nuanced association exists between physician EHR proficiency and time spent in the EHR.
Collapse
Affiliation(s)
- Oliver T Nguyen
- Corresponding Author: Oliver T. Nguyen, MSHI, Department of Community Health and Family Medicine, University of Florida, College of Medicine, PO Box 100211, Gainesville, FL 32610, USA;
| | - Kea Turner
- Department of Health Outcomes and Behavior, Moffitt Cancer Center, Tampa, Florida, USA,Department of Oncological Sciences, University of South Florida, Tampa, Florida, USA
| | - Nate C Apathy
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Tanja Magoc
- Clinical and Translational Science Institute, University of Florida, Gainesville, Florida, USA
| | - Karim Hanna
- Department of Family Medicine, Morsani College of Medicine, University of South Florida, Tampa, Florida, USA
| | - Lisa J Merlo
- Department of Psychiatry, University of Florida, Gainesville, Florida, USA
| | - Christopher A Harle
- Clinical and Translational Science Institute, University of Florida, Gainesville, Florida, USA,Department of Health Outcomes and Biomedical Informatics, University of Florida, Gainesville, Florida, USA
| | - Lindsay A Thompson
- Department of Health Outcomes and Biomedical Informatics, University of Florida, Gainesville, Florida, USA,Department of Pediatrics, University of Florida, Gainesville, Florida, USA
| | - Eta S Berner
- Department of Health Services Administration, University of Alabama at Birmingham, Birmingham, Alabama, USA
| | - Sue S Feldman
- Department of Health Services Administration, University of Alabama at Birmingham, Birmingham, Alabama, USA
| |
Collapse
|
8
|
Waller R, Ekpa N, Kass L. The Scribe Effect: the Impact of a Pre-matriculation Experience on Subsequent Medical School Education. MEDICAL SCIENCE EDUCATOR 2021; 31:1983-1989. [PMID: 34567836 PMCID: PMC8455115 DOI: 10.1007/s40670-021-01407-7] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 09/14/2021] [Indexed: 06/13/2023]
Abstract
Medical scribes have been utilized since the 1970s but have been in ever-increasing demand over the past 25 years. The reasons for this growth have been well documented, with positive impacts on provider well-being, patient satisfaction, clinical efficiency, and revenue generation. Many aspiring healthcare providers become medical scribes during or immediately after college, believing it will provide them with helpful experience and increase their chances of gaining entrance into medical education. However, little data exists to justify those beliefs. Through written surveys and semi-structured interviews, we found that scribes feel that their experience shaped their futures in medicine in two broad themes, specifically confirming their commitment to medicine (with subthemes of specialty choice, establishing mentorship, and exposure to difficult topics) and the essential skills of a physician (with subthemes of communication, professionalism, history and physical, terminology and jargon, and clinical reasoning). Understanding the impact of a scribe experience may provide medical school admissions personnel a more thorough sense of the scribe's strengths and likelihood of success in training, and should generate testable hypotheses for further studies into the learning processes of medical scribes.
Collapse
Affiliation(s)
- Robert Waller
- Department of Emergency Medicine, Penn State Milton S. Hershey Medical Center, 500 University Dr, PO Box 850, Hershey, PA USA
| | | | - Lawrence Kass
- Department of Emergency Medicine, Penn State Milton S. Hershey Medical Center, 500 University Dr, PO Box 850, Hershey, PA USA
| |
Collapse
|