Professional and interprofessional differences in electronic health records use and recognition of safety issues in critically ill patients

Acute vital signs changes are underrepresented by a conventional electronic health record when compared with automatically acquired data in a single-center tertiary pediatric cardiac intensive care unit

OBJECTIVE

We sought to evaluate the fidelity with which the patient's clinical state is represented by the electronic health record (EHR) flow sheet vital signs data compared to a commercially available automated data aggregation platform in a pediatric cardiac intensive care unit (CICU).

METHODS

This is a retrospective observational study of heart rate (HR), systolic blood pressure (SBP), respiratory rate (RR), and pulse oximetry (SpO2) data archived in a conventional EHR and an automated data platform for 857 pediatric patients admitted postoperatively to a tertiary pediatric CICU. Automated data captured for 72 h after admission were analyzed for significant HR, SBP, RR, and SpO2 deviations from baseline (events). Missed events were identified when the EHR failed to reflect the events reflected in the automated platform.

RESULTS

Analysis of 132 054 622 data entries, including 264 966 (0.2%) EHR entries and 131 789 656 (99.8%) automated entries, identified 15 839 HR events, 5851 SBP events, 9648 RR events, and 2768 SpO2 events lasting 3-60 min; these events were missing in the EHR 48%, 58%, 50%, and 54% of the time, respectively. Subanalysis identified 329 physiologically implausible events (eg, likely operator or device error), of which 104 (32%) were nonetheless documented in the EHR.

CONCLUSION

In this single-center retrospective study of CICU patients, EHR vital sign documentation was incomplete compared to an automated data aggregation platform. Significant events were underrepresented by the conventional EHR, regardless of event duration. Enrichment of the EHR with automated data aggregation capabilities may improve representation of patient condition.

Utilizing eye tracking to assess electronic health record use by pharmacists in the intensive care unit

DISCLAIMER

In an effort to expedite the publication of articles related to the COVID-19 pandemic, AJHP is posting these manuscripts online as soon as possible after acceptance. Accepted manuscripts have been peer-reviewed and copyedited, but are posted online before technical formatting and author proofing. These manuscripts are not the final version of record and will be replaced with the final article (formatted per AJHP style and proofed by the authors) at a later time.

PURPOSE

A study was conducted using high-fidelity electronic health record (EHR)-based simulations with incorporated eye tracking to understand the workflow of critical care pharmacists within the EHR, with specific attention to the data elements most frequently viewed.

METHODS

Eight critical care pharmacists were given 25 minutes to review 3 simulated intensive care unit (ICU) charts deployed in the simulation instance of the EHR. Using monitor-based eye trackers, time spent reviewing screens, clinical information accessed, and screens used to access specific information were reviewed and quantified to look for trends.

RESULTS

Overall, pharmacists viewed 25.5 total and 15.1 unique EHR screens per case. The majority of time was spent looking at screens focused on medications, followed by screens displaying notes, laboratory values, and vital signs. With regard to medication data, the vast majority of screen visitations were to view information on opioids/sedatives and antibiotics. With regard to laboratory values, the majority of views were focused on basic chemistry and hematology data. While there was significant variance between pharmacists, individual navigation patterns remained constant across cases.

CONCLUSION

The study results suggest that in addition to medication information, laboratory data and clinical notes are key focuses of ICU pharmacist review of patient records and that navigation to multiple screens is required in order to view these data with the EHR. New pharmacy-specific EHR interfaces should consolidate these elements within a primary interface.

A qualitative study of clinician perspectives and desired characteristics of a clinical decision support system for early sepsis recognition (Preprint)

Background

Sepsis is a major burden for health care systems in the United States, with over 750,000 cases annually and a total cost of approximately US $20 billion. The hallmark of sepsis treatment is early and appropriate initiation of antibiotic therapy. Although sepsis clinical decision support (CDS) systems can provide clinicians with early predictions of suspected sepsis or imminent clinical decline, such systems have not reliably demonstrated improvements in clinical outcomes or care processes. Growing evidence suggests that the challenges of integrating sepsis CDS systems into clinical workflows, gaining the trust of clinicians, and making sepsis CDS systems clinically relevant at the bedside are all obstacles to successful deployment. However, there are significant knowledge gaps regarding the achievement of these implementation and deployment goals.

Objective

We aimed to identify perceptions of predictive information in sepsis CDS systems based on clinicians’ past experiences, explore clinicians’ perceptions of a hypothetical sepsis CDS system, and identify the characteristics of a CDS system that would be helpful in promoting timely recognition and management of suspected sepsis in a multidisciplinary, team-based clinical setting.

Methods

We conducted semistructured interviews with practicing bedside nurses, advanced practice providers, and physicians at a large academic medical center between September 2020 and March 2021. We used modified human factor methods (contextual interview and cognitive walkthrough performed over video calls because of the COVID-19 pandemic) and conducted a thematic analysis using an abductive approach for coding to identify important patterns and concepts in the interview transcripts.

Results

We interviewed 6 bedside nurses and 9 clinicians responsible for ordering antibiotics (advanced practice providers or physicians) who had a median of 4 (IQR 4-6.5) years of experience working in an inpatient setting. We then synthesized critical content from the thematic analysis of the data into four domains: clinician perceptions of prediction models and alerts; previous experiences of clinician encounters with predictive information and risk scores; desired characteristics of a CDS system build, including predictions, supporting information, and delivery methods for a potential alert; and the clinical relevance and potential utility of a CDS system. These 4 domains were strongly linked to clinicians’ perceptions of the likelihood of adoption and the impact on clinical workflows when diagnosing and managing patients with suspected sepsis. Ultimately, clinicians desired a trusted and actionable CDS system to improve sepsis care.

Conclusions

Building a trusted and actionable sepsis CDS alert is paramount to achieving acceptability and use among clinicians. These findings can inform the development, implementation, and deployment strategies for CDS systems that support the early detection and treatment of sepsis. This study also highlights several key opportunities when eliciting clinician input before the development and deployment of prediction models.

Facilitating interprofessional affective learning in health professional students through digital client documentation: a comparison of simulation modes

Digital Interprofessional Learning Client Documentation (D-IPL Client Docs) is an initiative designed to develop student interprofessional communication skills through electronic record writing and a virtual simulation (VS) or live virtual simulation (LVS) case conference. The aims of the study were to (a) identify whether D-IPL Client Docs supports student learning in the affective domain and (b) compare the learning outcomes for students participating in the VS versus the LVS case conference. Data were drawn from 83 Bachelor of Social Work students who had participated with other health professional students in the D-IPL Client Docs activities. The reflective journals submitted by this cohort of social work students were analyzed qualitatively and quantitatively using the Griffith University Affective Learning Scale. Qualitative analyses revealed that the activities enabled students in both groups to learn about themselves, their roles, and the roles of others, and the benefits of interprofessional collaboration in optimizing client outcomes. Quantitatively, the VS mode appeared to be more effective in supporting students to develop higher order affective learning; however, the effect size was small. Future studies should involve a larger sample size and include students from various professions to ascertain the transferability of findings.

Not another box to check! Using the UTAUT to explore nurses' psychological adaptation to electronic health record usability

BACKGROUND

As the integration of electronic health records (EHRs) continues to expand throughout healthcare organizations worldwide, nurses are faced with the challenge to acclimate to a dynamic technology-based environment of patient care. As technology advances, EHR usability may influence nurses' clinical performance and require nurses to adapt to a wide range of situations associated with electronic documentation. While perceived benefits of EHRs include improvements in healthcare delivery and patient outcomes, there are also unintended consequences associated with EHR usability.

AIMS

The aim of this article is to explore dimensions of EHR usability that may influence nurses' psychological adaptation to the use of EHRs in daily practice.

MATERIALS AND METHODS

The unified theory of acceptance and use of technology (UTAUT) consists of four constructs theorized to have a direct influence on end-user behavior and acceptance of technology: performance expectancy, effort expectancy, social influence, and facilitating conditions. The UTAUT provides the framework to explore EHR usability as found in literature and describe the positive and negative psychological responses of nurses related to the use of EHRs in daily practice.

RESULTS

Integration of EHRs into daily nursing practice can result in positive experiences or have a negative impact on nurses ability to perform their role as patient caregivers.

DISCUSSION

While integration of EHRs into healthcare systems can facilitate improvements in patient outcomes, the delivery of patient care and nurses' daily practices can simultaneously be complicated by EHR usability issues.

CONCLUSION

Although positive and negative experiences of nurses in relationship to EHR usability can be found in literature, there is limited evidence on nurses' psychological adaptation to use of EHRs. Further research on EHR usability is needed based on nursing perspectives and feedback to determine the psychological impact EHRs exert on nurses, discover effective methods for resolving issues, and promote positive trends associated with EHR usability.

Pilot Testing of Simulation in the Evaluation of a Novel, Rapidly Deployable Electronic Health Record for use in Disaster Intensive Care

OBJECTIVES

The SARS-CoV-2 pandemic has highlighted the need for rapid creation and management of ICU field hospitals with effective remote monitoring which is dependent on the rapid deployment and integration of an Electronic Health Record (EHR). We describe the use of simulation to evaluate a rapidly scalable hub-and-spoke model for EHR deployment and monitoring using asynchronous training.

METHODS

We adapted existing commercial EHR products to serve as the point of entry from a simulated hospital and a separate system for tele-ICU support and monitoring of the interfaced data. To train our users we created a modular video-based curriculum to facilitate asynchronous training. Effectiveness of the curriculum was assessed through completion of common ICU documentation tasks in a high-fidelity simulation. Additional endpoints include assessment of EHR navigation, user satisfaction (Net Promoter), system usability (System Usability Scale-SUS), and cognitive load (NASA-TLX).

RESULTS

A total of 5 participants achieved a 100% task completion on all domains except ventilator data (91%). Systems demonstrated high degrees of satisfaction (Net Promoter = 65.2), acceptable usability (SUS = 66.5), and acceptable cognitive load (NASA-TLX = 41.5); with higher levels of cognitive load correlating with the number of screens employed.

CONCLUSIONS

Clinical usability of a comprehensive and rapidly deployable EHR was acceptable in an intensive care simulation which was preceded by < 1 hour of video education about the EHR. This model should be considered in plans for integrated clinical response with remote and accessory facilities.

Using A New Model of Electronic Health Record Training To Reduce Physician Burnout: A Plan For Action

UNSTRUCTURED

Physician burnout has been growing in the US at an alarming rate, and healthcare organizations are beginning to invest significant resources in combating this phenomenon. While the causes for burnout are multifactorial, a key issue that affects physicians is that they spend a significant proportion of their time interacting with their electronic health record (EHR), primarily because of the need to sift through increasing amounts of patient data coupled with a significant documentation burden. This has led to physicians spending increasing amounts time with the EHR after hours trying to catch up on paperwork ("pajama time"), which is a factor linked to burnout. In this paper we propose an innovative model of EHR training utilizing high-fidelity EHR simulation designed to facilitate efficient optimization of EHR use by clinicians, and emphasize the importance of both lifelong learning and physician well-being.

Sex Differences in Electronic Health Record Navigation Strategies: Secondary Data Analysis

Background

Use of electronic health records (EHRs) has increased dramatically over the past decade. Their widespread adoption has been plagued with numerous complaints about usability, with subsequent impacts on patient safety and provider well-being. Data in other fields suggest biological sex impacts basic patterns of navigation in electronic media.

Objective

This study aimed to determine whether biological sex impacted physicians’ navigational strategies while using EHRs.

Methods

This is a secondary analysis of a prior study where physicians were given verbal and written signout, and then, while being monitored with an eye tracker, were asked to review a simulated record in our institution’s EHR system, which contained 14 patient safety items. Afterward, the number of safety items recognized was recorded.

Results

A total of 93 physicians (female: n=46, male: n=47) participated in the study. Two gaze patterns were identified: one characterized more so by saccadic (“scanning”) eye movements and the other characterized more so by longer fixations (“staring”). Female physicians were more likely to use the scanning pattern; they had a shorter mean fixation duration (P=.005), traveled more distance per minute of screen time (P=.03), had more saccades per minute of screen time (P=.02), and had longer periods of saccadic movement (P=.03). The average proportion of time spent staring compared to scanning (the Gaze Index [GI]) across all participants was approximately 3:1. Females were more likely than males to have a GI value <3.0 (P=.003). At the extremes, males were more likely to have a GI value >5, while females were more likely to have a GI value <1. Differences in navigational strategy had no impact on task performance.

Conclusions

Females and males demonstrate fundamentally different navigational strategies while navigating the EHR. This has potentially significant impacts for usability testing in EHR training and design. Further studies are needed to determine if the detected differences in gaze patterns produce meaningful differences in cognitive load while using EHRs.

Nursing workload associated with the frequency of multidisciplinary rounds: a cross-sectional study

OBJECTIVE

To assess the frequency of multidisciplinary rounds during ICU days, to evaluate the participation of diverse healthcare professionals, to identify the reasons why rounds were not performed on specific days, and whether bed occupancy rate and nurse workload were associated with the conduction of multidisciplinary rounds.

METHODS

We performed a cross-sectional study to assess the frequency of multidisciplinary rounds in four intensive care units in a cancer center. We also collected data on rates of professional participation, reasons for not performing rounds when they did not occur, and daily bed occupancy rates and assessed nurse workload by measuring the Nursing Activity Score.

RESULTS

Rounds were conducted on 595 (65.8%) of 889 surveyed intensive care unit days. Nurses, physicians, respiratory therapists, pharmacists, and infection control practitioners participated most often. Rounds did not occur due to admission of new patients at the scheduled time (136; 44.7%) and involvement of nurses in activities unrelated to patients' care (97; 31.9%). In multivariate analysis, higher Nursing Activity Scores were associated with greater odds of conducting multidisciplinary rounds (OR = 1.06; 95%CI 1.04 - 1.10; p < 0.01), whereas bed occupancy rates were not (OR = 0.99; 95%CI 0.97 - 1.00; p = 0.18).

CONCLUSION

Multidisciplinary rounds were conducted on less than two-thirds of surveyed intensive care unit days. Many rounds were cancelled due to activities unrelated to patient care. Unexpectedly, increased workload was associated with higher odds of conducting rounds. Workload is a possible trigger to discuss daily goals to improve patient outcomes and to enhance the effectiveness of multidisciplinary teams.

The Impact of Electronic Health Record-Based Simulation During Intern Boot Camp: Interventional Study

BACKGROUND

Accurate data retrieval is an essential part of patient care in the intensive care unit (ICU). The electronic health record (EHR) is the primary method for data storage and data review. We previously reported that residents participating in EHR-based simulations have varied and nonstandard approaches to finding data in the ICU, with subsequent errors in recognizing patient safety issues. We hypothesized that a novel EHR simulation-based training exercise would decrease EHR use variability among intervention interns, irrespective of prior EHR experience.

OBJECTIVE

This study aims to understand the impact of a novel, short, high-fidelity, simulation-based EHR learning activity on the intern data gathering workflow and satisfaction.

METHODS

A total of 72 internal medicine interns across the 2018 and 2019 academic years underwent a dedicated EHR training session as part of a week-long boot camp early in their training. We collected data on previous EHR and ICU experience for all subjects. Training consisted of 1 hour of guided review of a high-fidelity, simulated ICU patient chart focusing on best navigation practices for data retrieval. Specifically, the activity focused on using high- and low-yield data visualization screens determined by expert consensus. The intervention group interns then had 20 minutes to review a new simulated patient chart before the group review. EHR screen navigation was captured using screen recording software and compared with data from existing ICU residents performing the same task on the same medical charts (N=62). Learners were surveyed immediately and 6 months after the activity to assess satisfaction and preferred EHR screen use.

RESULTS

Participants found the activity useful and enjoyable immediately and after 6 months. Intervention interns used more individual screens than reference residents (18 vs 20; P=.008), but the total number of screens used was the same (35 vs 38; P=.30). Significantly more intervention interns used the 10 most common screens (73% vs 45%; P=.001). Intervention interns used high-yield screens more often and low-yield screens less often than the reference residents, which are persistent on self-report 6 months later.

CONCLUSIONS

A short, high-fidelity, simulation-based learning activity focused on provider-specific data gathering was found to be enjoyable and to modify navigation patterns persistently. This suggests that workflow-specific simulation-based EHR training throughout training is of educational benefit to residents.

Interprofessional Evidence-Based Strategies to Enhance Provider and Patient Interactions During Electronic Health Record Use

The purpose of this study was to develop and disseminate evidence-based interprofessional strategies to enhance provider-patient interactions, including ethical issues, that arise during electronic documentation. An interprofessional simulation scenario was implemented with students, and strategies developed were then used to train hospital staff. Nurses reported being significantly more likely to use the interprofessional strategies after completing the program. Interprofessional simulation and training is an effective method to address challenges that arise during electronic health record use.

The national health information technology human factors and ergonomics agenda

Health information technology (IT) implementation has encompassed much of the United States health care system over the past decade, and user frustration with health IT has steadily increased. Human factors and ergonomics (HFE) methods and approaches can improve the design, implementation, and use of health IT for clinicians and consumers. To better understand the effect of federal HFE in health IT research funding, the authors conducted a review of several key, specific initiatives. The review focused on the goals and accomplishments of these initiatives. Findings to date show that HFE is improving the usefulness of health IT, but additional research and new methods are needed. Corresponding research funding and policy priorities are identified. New HFE work and innovative approaches are needed to capitalize on HFE knowledge, principles, and methods to improve the design, implementation, and use of health IT at a broader scale.

Data Omission by Physician Trainees on ICU Rounds

OBJECTIVES

Incomplete patient data, either due to difficulty gathering and synthesizing or inappropriate data filtering, can lead clinicians to misdiagnosis and medical error. How completely ICU interprofessional rounding teams appraise the patient data set that informs clinical decision-making is unknown. This study measures how frequently physician trainees omit data from prerounding notes ("artifacts") and verbal presentations during daily rounds.

DESIGN

Observational study.

SETTING

Tertiary academic medical ICU with an established electronic health record and where physician trainees are the primary presenters during daily rounds.

SUBJECTS

Presenters (medical student or resident physician), interprofessional rounding team.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We quantified the amount and types of patient data omitted from photocopies of physician trainees' artifacts and audio recordings of oral ICU rounds presentations when compared with source electronic health record data. An audit of 157 patient presentations including 6,055 data elements across nine domains revealed 100% of presentations contained omissions. Overall, 22.9% of data were missing from artifacts and 42.4% from presentations. The interprofessional team supplemented only 4.1% of additional available data. Frequency of trainee data omission varied by data type and sociotechnical factors. The strongest predictor of trainee verbal omissions was a preceding failure to include the data on the artifact. Passive data gathering via electronic health record macros resulted in extremely complete artifacts but paradoxically predicted greater likelihood of verbal omission when compared with manual notation. Interns verbally omitted the most data, whereas medical students omitted the least.

CONCLUSIONS

In an academic rounding model reliant on trainees to preview and select data for presentation during ICU rounds, verbal appraisal of patient data was highly incomplete. Additional trainee oversight and education, improved electronic health record tools, and novel academic rounding paradigms are needed to address this potential source of medical error.

Use of a Novel, Electronic Health Record-Centered, Interprofessional ICU Rounding Simulation to Understand Latent Safety Issues

OBJECTIVES

The electronic health record is a primary source of information for all professional groups participating in ICU rounds. We previously demonstrated that, individually, all professional groups involved in rounds have significant blind spots in recognition of patient safety issues in the electronic health record. However, it is unclear how team dynamics impacts identification and verbalization of viewed data. Therefore, we created an ICU rounding simulation to assess how the interprofessional team recognized and reported data and its impact on decision-making.

DESIGN

Each member of the ICU team reviewed a simulated ICU chart in the electronic health record which contained embedded patient safety issues. The team conducted simulated rounds according to the ICU's existing rounding script and was assessed for recognition of safety issues.

SETTING

Academic medical center.

SUBJECTS

ICU residents, nurses, and pharmacists.

INTERVENTION

None.

MEASUREMENTS AND MAIN RESULTS

Twenty-eight teams recognized 68.6% of safety issues with only 50% teams having the primary diagnosis in their differential. Individually, interns, nurses, and pharmacists recognized 30.4%, 15.6%, and 19.6% of safety items, respectively. However, there was a negative correlation between the intern's performance and the nurse's or the pharmacist's performance within a given team. The wide variance in recognition of data resulted in wide variance in orders. Overall, there were 21.8 orders requested and 21.6 orders placed per case resulting in 3.6 order entry inconsistencies/case. Between the two cases, there were 145 distinct orders place with 43% being unique to a specific team and only 2% placed by all teams.

CONCLUSIONS

Although significant blind spots exist in the interprofessional team's ability to recognize safety issues in the electronic health record, the inclusion of other professional groups does serve as a partial safety net to improve recognition. Electronic health record-based, ICU rounding simulations can serve as a test-bed for innovations in ICU rounding structure and data collection.

An interprofessional team simulation exercise about a complex geriatric patient

Interprofessional collaboration is an essential skill to optimize the care of older adults with complex problems. We successfully developed and evaluated an interprofessional teamwork simulation exercise for medical, nursing, pharmacy, and social work students. Pharmacy students participated via video conferencing. Before the simulation, students watched a teamwork video and reviewed the patient case. Following an icebreaker exercise, interdisciplinary faculty facilitated a discussion highlighting effective teamwork strategies. Students then collaborated to develop a discharge plan, followed by a simulated family meeting with a theater student. Interdisciplinary faculty again provided structured debriefing highlighting principles of effective teamwork. Students self-rated interprofessional practice core competencies were evaluated using a retrospective pre/post survey and analyzed using paired t-tests. We qualitatively examined the use of distance technology and assessed learner's satisfaction with the project. All core competency categories for all disciplines demonstrated significant improvements in mean scores. Students' qualitative comments demonstrated positive impact on learning interprofessional core competencies.

Impact of a Web-Based Electronic Health Record on Behavioral Health Service Delivery for Children and Adolescents: Randomized Controlled Trial

Background

Electronic health records (EHRs) have been widely proposed as a mechanism for improving health care quality. However, rigorous research on the impact of EHR systems on behavioral health service delivery is scant, especially for children and adolescents.

Objective

The current study evaluated the usability of an EHR developed to support the implementation of the Wraparound care coordination model for children and youth with complex behavioral health needs, and impact of the EHR on service processes, fidelity, and proximal outcomes.

Methods

Thirty-four Wraparound facilitators working in two programs in two states were randomized to either use the new EHR (19/34, 56%) or to continue to implement Wraparound services as usual (SAU) using paper-based documentation (15/34, 44%). Key functions of the EHR included standard fields such as youth and family information, diagnoses, assessment data, and progress notes. In addition, there was the maintenance of a coordinated plan of care, progress measurement on strategies and services, communication among team members, and reporting on services, expenditures, and outcomes. All children and youth referred to services for eight months (N=211) were eligible for the study. After excluding those who were ineligible (69/211, 33%) and who declined to participate (59/211, 28%), a total of 83/211 (39%) children and youth were enrolled in the study with 49/211 (23%) in the EHR condition and 34/211 (16%) in the SAU condition. Facilitators serving these youth and families and their supervisors completed measures of EHR usability and appropriateness, supervision processes and activities, work satisfaction, and use of and attitudes toward standardized assessments. Data from facilitators were collected by web survey and, where necessary, by phone interviews. Parents and caregivers completed measures via phone interviews. Related to fidelity and quality of behavioral health care, including Wraparound team climate, working alliance with providers, fidelity to the Wraparound model, and satisfaction with services.

Results

EHR-assigned facilitators from both sites demonstrated the robust use of the system. Facilitators in the EHR group reported spending significantly more time reviewing client progress (P=.03) in supervision, and less time overall sending reminders to youth/families (P=.04). A trend toward less time on administrative tasks (P=.098) in supervision was also found. Facilitators in both groups reported significantly increased use of measurement-based care strategies overall, which may reflect cross-group contamination (given that randomization of staff to the EHR occurred within agencies and supervisors supervised both types of staff). Although not significant at P<.05, there was a trend (P=.10) toward caregivers in the EHR group reporting poorer shared agreement on tasks on the measure of working alliance with providers. No other significant between-group differences were found.

Conclusions

Results support the proposal that use of EHR systems can promote the use of client progress data and promote efficiency; however, there was little evidence of any impact (positive or negative) on overall service quality, fidelity, or client satisfaction. The field of children’s behavioral health services would benefit from additional research on EHR systems using designs that include larger sample sizes and longer follow-up periods.

Trial Registration

ClinicalTrials.gov NCT02421874; https://clinicaltrials.gov/ct2/show/NCT02421874 (Archived by WebCite at http://www.webcitation.org/6yyGPJ3NA)

Use of Simulation Based on an Electronic Health Records Environment to Evaluate the Structure and Accuracy of Notes Generated by Medical Scribes: Proof-of-Concept Study

Background

The increasing adoption of electronic health records (EHRs) has been associated with a number of unintended negative consequences with provider efficiency and job satisfaction. To address this, there has been a dramatic increase in the use of medical scribes to perform many of the required EHR functions. Despite this rapid growth, little has been published on the training or assessment tools to appraise the safety and efficacy of scribe-related EHR activities. Given the number of reports documenting that other professional groups suffer from a number of performance errors in EHR interface and data gathering, scribes likely face similar challenges. This highlights the need for new assessment tools for medical scribes.

Objective

The objective of this study was to develop a virtual video-based simulation to demonstrate and quantify the variability and accuracy of scribes’ transcribed notes in the EHR.

Methods

From a pool of 8 scribes in one department, a total of 5 female scribes, intent on pursuing careers in health care, with at least 6 months of experience were recruited for our simulation study. We created three simulated patient-provider scenarios. Each scenario contained a corresponding medical record in our simulation instance of our EHR. For each scenario, we video-recorded a standardized patient-provider encounter. Five scribes with at least 6 months of experience both with our EHR and in the specialty of the simulated cases were recruited. Each scribe watched the simulated encounter and transcribed notes into a simulated EHR environment. Transcribed notes were evaluated for interscribe variability and compared with a gold standard for accuracy.

Results

All scribes completed all simulated cases. There was significant interscribe variability in note structure and content. Overall, only 26% of all data elements were unique to the scribe writing them. The term data element was used to define the individual pieces of data that scribes perceived from the simulation. Note length was determined by counting the number of words varied by 31%, 37%, and 57% between longest and shortest note between the three cases, and word economy ranged between 23% and 71%. Overall, there was a wide inter- and intrascribe variation in accuracy for each section of the notes with ranges from 50% to 76%, resulting in an overall positive predictive value for each note between 38% and 81%.

Conclusions

We created a high-fidelity, video-based EHR simulation, capable of assessing multiple performance indicators in medical scribes. In this cohort, we demonstrate significant variability both in terms of structure and accuracy in clinical documentation. This form of simulation can provide a valuable tool for future development of scribe curriculum and assessment of competency.

Health Information Technology (HIT) Adaptation: Refocusing on the Journey to Successful HIT Implementation

In past years, policies and regulations required hospitals to implement advanced capabilities of certified electronic health records (EHRs) in order to receive financial incentives. This has led to accelerated implementation of health information technologies (HIT) in health care settings. However, measures commonly used to evaluate the success of HIT implementation, such as HIT adoption, technology acceptance, and clinical quality, fail to account for complex sociotechnical variability across contexts and the different trajectories within organizations because of different implementation plans and timelines. We propose a new focus, HIT adaptation, to illuminate factors that facilitate or hinder the connection between use of the EHR and improved quality of care as well as to explore the trajectory of changes in the HIT implementation journey as it is impacted by frequent system upgrades and optimizations. Future research should develop instruments to evaluate the progress of HIT adaptation in both its longitudinal design and its focus on adaptation progress rather than on one cross-sectional outcome, allowing for more generalizability and knowledge transfer.

Use of EHR-based simulation to diagnose aetiology of information gathering issues in struggling learners: a proof of concept study

Learners who struggle with clinical decision making are often the most challenging to identify and remediate. While for some learners, struggles can be directly traced to a poor knowledge base, for many others, it is more difficult to understand the reason for their struggles. One of the main component of effective decision making is access to accurate and complete clinical information. The electronic health record (EHR) is the main source of clinical information and, with its widespread adoption, has come increased realisation that a large fraction of users have difficulty in effectively gathering and subsequently processing information out of the EHR. We previously documented that high-fidelity EHR-based simulation improves EHR usability and, when combined with eye and screen tracking, generates important measures of usability. We hypothesised that the same simulation exercise could help distinguish whether learners had difficulty in knowledge, information gathering or information processing. We report the results of the first three struggling learners who participated in this exercise. In each case, the simulation was able to ‘diagnose’ the aetiology for the learners’ struggle and assist in formulating an appropriate solution. We suggest that high-fidelity EHR-based simulation can be a powerful tool in the standard approach to understanding struggling learners.