How does medical scribes' work inform development of speech-based clinical documentation technologies? A systematic review

Evaluating the Impact of Artificial Intelligence (AI) on Clinical Documentation Efficiency and Accuracy Across Clinical Settings: A Scoping Review

Artificial intelligence (AI) technologies (natural language processing (NLP), speech recognition (SR), and machine learning (ML)) can transform clinical documentation in healthcare. This scoping review evaluates the impact of AI on the accuracy and efficiency of clinical documentation across various clinical settings (hospital wards, emergency departments, and outpatient clinics). We found 176 articles by applying a specific search string on Ovid. To ensure a more comprehensive search process, we also performed manual searches on PubMed and BMJ, examining any relevant references we encountered. In this way, we were able to add 46 more articles, resulting in 222 articles in total. After removing duplicates, 208 articles were screened. This led to the inclusion of 36 studies. We were mostly interested in articles discussing the impact of AI technologies, such as NLP, ML, and SR, and their accuracy and efficiency in clinical documentation. To ensure that our research reflected recent work, we focused our efforts on studies published in 2019 and beyond. This criterion was pilot-tested beforehand and necessary adjustments were made. After comparing screened articles independently, we ensured inter-rater reliability (Cohen's kappa=1.0), and data extraction was completed on these 36 articles. We conducted this study according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. This scoping review shows improvements in clinical documentation using AI technologies, with an emphasis on accuracy and efficiency. There was a reduction in clinician workload, with the streamlining of the documentation processes. Subsequently, doctors also had more time for patient care. However, these articles also raised various challenges surrounding the use of AI in clinical settings. These challenges included the management of errors, legal liability, and integration of AI with electronic health records (EHRs). There were also some ethical concerns regarding the use of AI with patient data. AI shows massive potential for improving the day-to-day work life of doctors across various clinical settings. However, more research is needed to address the many challenges associated with its use. Studies demonstrate improved accuracy and efficiency in clinical documentation with the use of AI. With better regulatory frameworks, implementation, and research, AI can significantly reduce the burden placed on doctors by documentation.

The Utility and Implications of Ambient Scribes in Primary Care

Ambient scribe technology, utilizing large language models, represents an opportunity for addressing several current pain points in the delivery of primary care. We explore the evolution of ambient scribes and their current use in primary care. We discuss the suitability of primary care for ambient scribe integration, considering the varied nature of patient presentations and the emphasis on comprehensive care. We also propose the stages of maturation in the use of ambient scribes in primary care and their impact on care delivery. Finally, we call for focused research on safety, bias, patient impact, and privacy in ambient scribe technology, emphasizing the need for early training and education of health care providers in artificial intelligence and digital health tools.

Aci-bench: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generation

Recent immense breakthroughs in generative models such as in GPT4 have precipitated re-imagined ubiquitous usage of these models in all applications. One area that can benefit by improvements in artificial intelligence (AI) is healthcare. The note generation task from doctor-patient encounters, and its associated electronic medical record documentation, is one of the most arduous time-consuming tasks for physicians. It is also a natural prime potential beneficiary to advances in generative models. However with such advances, benchmarking is more critical than ever. Whether studying model weaknesses or developing new evaluation metrics, shared open datasets are an imperative part of understanding the current state-of-the-art. Unfortunately as clinic encounter conversations are not routinely recorded and are difficult to ethically share due to patient confidentiality, there are no sufficiently large clinic dialogue-note datasets to benchmark this task. Here we present the Ambient Clinical Intelligence Benchmark (ACI-BENCH) corpus, the largest dataset to date tackling the problem of AI-assisted note generation from visit dialogue. We also present the benchmark performances of several common state-of-the-art approaches.

"Mm-hm," "Uh-uh": are non-lexical conversational sounds deal breakers for the ambient clinical documentation technology?

OBJECTIVES

Ambient clinical documentation technology uses automatic speech recognition (ASR) and natural language processing (NLP) to turn patient-clinician conversations into clinical documentation. It is a promising approach to reducing clinician burden and improving documentation quality. However, the performance of current-generation ASR remains inadequately validated. In this study, we investigated the impact of non-lexical conversational sounds (NLCS) on ASR performance. NLCS, such as Mm-hm and Uh-uh, are commonly used to convey important information in clinical conversations, for example, Mm-hm as a "yes" response from the patient to the clinician question "are you allergic to antibiotics?"

MATERIALS AND METHODS

In this study, we evaluated 2 contemporary ASR engines, Google Speech-to-Text Clinical Conversation ("Google ASR"), and Amazon Transcribe Medical ("Amazon ASR"), both of which have their language models specifically tailored to clinical conversations. The empirical data used were from 36 primary care encounters. We conducted a series of quantitative and qualitative analyses to examine the word error rate (WER) and the potential impact of misrecognized NLCS on the quality of clinical documentation.

RESULTS

Out of a total of 135 647 spoken words contained in the evaluation data, 3284 (2.4%) were NLCS. Among these NLCS, 76 (0.06% of total words, 2.3% of all NLCS) were used to convey clinically relevant information. The overall WER, of all spoken words, was 11.8% for Google ASR and 12.8% for Amazon ASR. However, both ASR engines demonstrated poor performance in recognizing NLCS: the WERs across frequently used NLCS were 40.8% (Google) and 57.2% (Amazon), respectively; and among the NLCS that conveyed clinically relevant information, 94.7% and 98.7%, respectively.

DISCUSSION AND CONCLUSION

Current ASR solutions are not capable of properly recognizing NLCS, particularly those that convey clinically relevant information. Although the volume of NLCS in our evaluation data was very small (2.4% of the total corpus; and for NLCS that conveyed clinically relevant information: 0.06%), incorrect recognition of them could result in inaccuracies in clinical documentation and introduce new patient safety risks.

Medical Scribes Have a Variable Impact on Documentation Workflows

Physicians can reduce their documentation time by working with a scribe. However, what scribes document and how their actions affect existing documentation workflows is unclear. This study leverages electronic health record (EHR) audit logs to observe how scribes affected the documentation workflows of seven physicians and their staff across 13,000 outpatient ophthalmology visits. In addition to editing progress notes, scribes routinely edited exam findings and diagnoses. Scribes with clinical training also edited items such as vital signs that a scribe without clinical training did not. Every physician edited patient records later in the day when working with a scribe and those who deferred their editing the most had some of the largest reductions in EHR time. These results suggest that what scribes document, how physicians work with scribes, and scribe impact on documentation time are all highly variable, highlighting the need for evidence-based best practices.

Reduced Cognitive Burden and Increased Focus: A Mixed-methods Study Exploring How Implementing Scribes Impacted Physicians

BACKGROUND

Understanding how medical scribes impact care delivery can inform decision-makers who must balance the cost of hiring scribes with their contribution to alleviating clinician burden.

OBJECTIVE

The objective of this study was to understand how scribes impacted provider efficiency and satisfaction.

DESIGN

This was mixed-methods study.

PARTICIPANTS

Internal and family medicine clinicians were included.

MEASURES

We administered structured surveys and conducted unstructured interviews with clinicians who adopted scribes. We collected average days to close charts and quantity of after-hours clinical work in the 6 months before and after implementation using electronic health record data. We conducted a difference in difference (DID) analysis using a multilevel Poisson regression.

RESULTS

Three themes emerged from the interviews: (1) charting time is less after training; (2) clinicians wanted to continue working with scribes; and (3) scribes did not reduce the overall inbox burden. In the 6-month survey, 76% of clinicians endorsed that working with a scribe improved work satisfaction versus 50% at 1 month. After implementation, days to chart closure decreased [DID=0.38 fewer days; 95% confidence interval (CI): -0.61, -0.15] the average minutes worked after hours on clinic days decreased (DID=-11.5 min/d; 95% CI: -13.1, -9.9) as did minutes worked on nonclinical days (DID=-24.9 min/d; 95% CI: -28.1, -21.7).

CONCLUSIONS

Working with scribes was associated with reduced time to close charts and reduced time using the electronic health record, markers of efficiency. Increased satisfaction accrued once scribes had experience.

[Artificial Intelligence: Challenges and Applications in Intensive Care Medicine]

The high workload in intensive care medicine arises from the exponential growth of medical knowledge, the flood of data generated by the permanent and intensive monitoring of intensive care patients, and the documentation burden. Artificial intelligence (AI) is predicted to have a great impact on ICU work in the near future as it will be applicable in many areas of critical care medicine. These applications include documentation through speech recognition, predictions for decision support, algorithms for parameter optimisation and the development of personalised intensive care medicine. AI-based decision support systems can augment human therapy decisions. Primarily through machine learning, a sub-discipline of AI, self-adaptive algorithms can learn to recognise patterns and make predictions. For actual use in clinical settings, the explainability of such systems is a prerequisite. Intensive care staff spends a large amount of their working hours on documentation, which has increased up to 50% of work time with the introduction of PDMS. Speech recognition has the potential to reduce this documentation burden. It is not yet precise enough to be usable in the clinic. The application of AI in medicine, with the help of large data sets, promises to identify diagnoses more quickly, develop individualised, precise treatments, support therapeutic decisions, use resources with maximum effectiveness and thus optimise the patient experience in the near future.

Comparing Scribed and Non-scribed Outpatient Progress Notes

Working with scribes can reduce provider documentation time, but few studies have examined how scribes affect clinical notes. In this retrospective cross-sectional study, we examine over 50,000 outpatient progress notes written with and without scribe assistance by 70 providers across 27 specialties in 2017-2018. We find scribed notes were consistently longer than those written without scribe assistance, with most additional text coming from note templates. Scribed notes were also more likely to contain certain templated lists, such as the patient's medications or past medical history. However, there was significant variation in how working with scribes affected a provider's mix of typed, templated, and copied note text, suggesting providers adapt their documentation workflows to varying degrees when working with scribes. These results suggest working with scribes may contribute to note bloat, but that providers' individual documentation workflows, including their note templates, may have a large impact on scribed note contents.

An Ethical Case for Medical Scribes

This article addresses ethical concerns with the use of electronic health records (EHRs) by physicians in clinical practice. It presents arguments for two claims. First, requiring physicians to maintain patient EHRs for medically unnecessary tasks is likely contributing to increased burnout, decreased quality of care, and potential risks to patient safety. Second, medical institutions have ethical reasons to employ medical scribes to maintain patient EHRs. Finally, this article reviews central objections to employing medical scribes and provides responses to each.

Conceptualizing the digitalization of healthcare work: A metaphor-based Critical Interpretive Synthesis

The digitalization of healthcare work has gained center stage in academic debates spanning disciplines as diverse as medicine, sociology and STS. The different analytical interests and methodological traditions of these three strains of scholarship have, however, resulted in quite diverging approaches to this issue. Points of interest have ranged from the (disattended) promise of increased efficiency of healthcare work, to dynamics of task delegation, (re-)professionalization and (re-)distribution of invisible work, to the disruption of informal organization. Instead of studying these dynamics in practice, in this paper we foreground the potentiality for theory-making inherent in the systematic cross-contamination of different theoretical and disciplinary perspectives. We perform a Critical Interpretive Synthesis (CIS) centering the ways the digitalization of healthcare work has been investigated in recent STS, sociological and medical literature. To open up assumptions and insights intrinsic to each body of literature for scholars and practitioners in other fields, we propose here a metaphor-based variation on CIS approaches. We probe, in turn, what slime molds can teach us about STS's focus on interconnections and materiality, how we can better understand sociological analyses of invisible work exploring them through theatrical performances, and which lessons river engineering offers concerning medical scholarship's discussion of efficiency and proper healthcare work. Thinking through these metaphors, we conceptualize the digitalization of healthcare work as a phenomenon spanning, at once, the directionality of technological innovation trajectories and the open-endedness of situated changes in work practices. Based on our analysis, we propose focusing on technological scripts, and various forms of invisible work and informal organization as entry points into the study of the tension between directionality and open-endedness in the context of the digitalization of healthcare work.

An interview study with medical scribes on how their work may alleviate clinician burnout through delegated health IT tasks

OBJECTIVES

To understand how medical scribes' work may contribute to alleviating clinician burnout attributable directly or indirectly to the use of health IT.

MATERIALS AND METHODS

Qualitative analysis of semistructured interviews with 32 participants who had scribing experience in a variety of clinical settings.

RESULTS

We identified 7 categories of clinical tasks that clinicians commonly choose to offload to medical scribes, many of which involve delegated use of health IT. These range from notes-taking and computerized data entry to foraging, assembling, and tracking information scattered across multiple clinical information systems. Some common characteristics shared among these tasks include: (1) time-consuming to perform; (2) difficult to remember or keep track of; (3) disruptive to clinical workflow, clinicians' cognitive processes, or patient-provider interactions; (4) perceived to be low-skill "clerical" work; and (5) deemed as adding no value to direct patient care.

DISCUSSION

The fact that clinicians opt to "outsource" certain clinical tasks to medical scribes is a strong indication that performing these tasks is not perceived to be the best use of their time. Given that a vast majority of healthcare practices in the US do not have the luxury of affording medical scribes, the burden would inevitably fall onto clinicians' shoulders, which could be a major source for clinician burnout.

CONCLUSIONS

Medical scribes help to offload a substantial amount of burden from clinicians-particularly with tasks that involve onerous interactions with health IT. Developing a better understanding of medical scribes' work provides useful insights into the sources of clinician burnout and potential solutions to it.

Interaction Time with Electronic Health Records: A Systematic Review

BACKGROUND

The amount of time that health care clinicians (physicians and nurses) spend interacting with the electronic health record is not well understood.

OBJECTIVE

This study aimed to evaluate the time that health care providers spend interacting with electronic health records (EHR).

METHODS

Data are retrieved from Ovid MEDLINE(R) and Epub Ahead of Print, In-Process and Other Non-Indexed Citations and Daily, (Ovid) Embase, CINAHL, and SCOPUS.

STUDY ELIGIBILITY CRITERIA

Peer-reviewed studies that describe the use of EHR and include measurement of time either in hours, minutes, or in the percentage of a clinician's workday. Papers were written in English and published between 1990 and 2021.

PARTICIPANTS

All physicians and nurses involved in inpatient and outpatient settings.

STUDY APPRAISAL AND SYNTHESIS METHODS

A narrative synthesis of the results, providing summaries of interaction time with EHR. The studies were rated according to Quality Assessment Tool for Studies with Diverse Designs.

RESULTS

Out of 5,133 de-duplicated references identified through database searching, 18 met inclusion criteria. Most were time-motion studies (50%) that followed by logged-based analysis (44%). Most were conducted in the United States (94%) and examined a clinician workflow in the inpatient settings (83%). The average time was nearly 37% of time of their workday by physicians in both inpatient and outpatient settings and 22% of the workday by nurses in inpatient settings. The studies showed methodological heterogeneity.

CONCLUSION

This systematic review evaluates the time that health care providers spend interacting with EHR. Interaction time with EHR varies depending on clinicians' roles and clinical settings, computer systems, and users' experience. The average time spent by physicians on EHR exceeded one-third of their workday. The finding is a possible indicator that the EHR has room for usability, functionality improvement, and workflow optimization.

Electronic health record note review in an outpatient specialty clinic: who is looking?

Note entry and review in electronic health records (EHRs) are time-consuming. While some clinics have adopted team-based models of note entry, how these models have impacted note review is unknown in outpatient specialty clinics such as ophthalmology. We hypothesized that ophthalmologists and ancillary staff review very few notes. Using audit log data from 9775 follow-up office visits in an academic ophthalmology clinic, we found ophthalmologists reviewed a median of 1 note per visit (2.6 ± 5.3% of available notes), while ancillary staff reviewed a median of 2 notes per visit (4.1 ± 6.2% of available notes). While prior ophthalmic office visit notes were the most frequently reviewed note type, ophthalmologists and staff reviewed no such notes in 51% and 31% of visits, respectively. These results highlight the collaborative nature of note review and raise concerns about how cumbersome EHR designs affect efficient note review and the utility of prior notes in ophthalmic clinical care.

Contribution of Free-Text Comments to the Burden of Documentation: Assessment and Analysis of Vital Sign Comments in Flowsheets

Background

Documentation burden is a common problem with modern electronic health record (EHR) systems. To reduce this burden, various recording methods (eg, voice recorders or motion sensors) have been proposed. However, these solutions are in an early prototype phase and are unlikely to transition into practice in the near future. A more pragmatic alternative is to directly modify the implementation of the existing functionalities of an EHR system.

Objective

This study aims to assess the nature of free-text comments entered into EHR flowsheets that supplement quantitative vital sign values and examine opportunities to simplify functionality and reduce documentation burden.

Methods

We evaluated 209,055 vital sign comments in flowsheets that were generated in the Epic EHR system at the Vanderbilt University Medical Center in 2018. We applied topic modeling, as well as the natural language processing Clinical Language Annotation, Modeling, and Processing software system, to extract generally discussed topics and detailed medical terms (expressed as probability distribution) to investigate the stories communicated in these comments.

Results

Our analysis showed that 63.33% (6053/9557) of the users who entered vital signs made at least one free-text comment in vital sign flowsheet entries. The user roles that were most likely to compose comments were registered nurse, technician, and licensed nurse. The most frequently identified topics were the notification of a result to health care providers (0.347), the context of a measurement (0.307), and an inability to obtain a vital sign (0.224). There were 4187 unique medical terms that were extracted from 46,029 (0.220) comments, including many symptom-related terms such as “pain,” “upset,” “dizziness,” “coughing,” “anxiety,” “distress,” and “fever” and drug-related terms such as “tylenol,” “anesthesia,” “cannula,” “oxygen,” “motrin,” “rituxan,” and “labetalol.”

Conclusions

Considering that flowsheet comments are generally not displayed or automatically pulled into any clinical notes, our findings suggest that the flowsheet comment functionality can be simplified (eg, via structured response fields instead of a text input dialog) to reduce health care provider effort. Moreover, rich and clinically important medical terms such as medications and symptoms should be explicitly recorded in clinical notes for better visibility.

Envisioning an artificial intelligence documentation assistant for future primary care consultations: A co-design study with general practitioners

OBJECTIVE

The study sought to understand the potential roles of a future artificial intelligence (AI) documentation assistant in primary care consultations and to identify implications for doctors, patients, healthcare system, and technology design from the perspective of general practitioners.

MATERIALS AND METHODS

Co-design workshops with general practitioners were conducted. The workshops focused on (1) understanding the current consultation context and identifying existing problems, (2) ideating future solutions to these problems, and (3) discussing future roles for AI in primary care. The workshop activities included affinity diagramming, brainwriting, and video prototyping methods. The workshops were audio-recorded and transcribed verbatim. Inductive thematic analysis of the transcripts of conversations was performed.

RESULTS

Two researchers facilitated 3 co-design workshops with 16 general practitioners. Three main themes emerged: professional autonomy, human-AI collaboration, and new models of care. Major implications identified within these themes included (1) concerns with medico-legal aspects arising from constant recording and accessibility of full consultation records, (2) future consultations taking place out of the exam rooms in a distributed system involving empowered patients, (3) human conversation and empathy remaining the core tasks of doctors in any future AI-enabled consultations, and (4) questioning the current focus of AI initiatives on improved efficiency as opposed to patient care.

CONCLUSIONS

AI documentation assistants will likely to be integral to the future primary care consultations. However, these technologies will still need to be supervised by a human until strong evidence for reliable autonomous performance is available. Therefore, different human-AI collaboration models will need to be designed and evaluated to ensure patient safety, quality of care, doctor safety, and doctor autonomy.