A usability and safety analysis of electronic health records: a multi-center study

HCI-modelling for improving the clinical usability of digital health technologies

INTRODUCTION

Digital Health Technologies (DHTs) have been shown to have variable usability as measured by efficiency, effectiveness and user satisfaction despite large-scale government projects to regulate and standardise user interface (UI) design. We hypothesised that Human-Computer Interaction (HCI) modelling could improve the methodology for DHT design and regulation, and support the creation of future evidence-based UI standards and guidelines for DHTs.

METHODOLOGY

Using a Design Science Research (DSR) framework, we developed novel UI components that adhered to existing standards and guidelines (combining the NHS Common User Interface (CUI) standard and the NHS Design System). We firstly evaluated the Patient Banner UI component for compliance with the two guidelines and then used HCI-modelling to evaluate the "Add New Patient" workflow to measure time to task completion and cognitive load.

RESULTS

Combining the two guidelines to produce new UI elements is technically feasible for the Patient Banner and the Patient Name Input components. There are some inconsistencies between the NHS Design System and the NHS CUI when implementing the Patient Banner. HCI-modelling successfully quantified challenges adhering to the NHS CUI and the NHS Design system for the "Add New Patient" workflow.

DISCUSSION

We successfully developed new design artefacts combing two major design guidelines for DHTs. By quantifying usability issues using HCI-modelling, we have demonstrated the feasibility of a methodology that combines HCI-modelling into a human-centred design (HCD) process could enable the development of standardised UI elements for DHTs that is more scientifically robust than HCD alone.

CONCLUSION

Combining HCI-modelling and Human-Centred Design could improve scientific progress towards developing safer and more user-friendly DHTs.

Utilization and efficacy of DotPhrases in the electronic medical record for improving physician documentation

Clinical documentation of patient visits has changed over the last 20 years, with the increasing use of electronic records causing a seismic shift in how notes are taken. Electronic note-taking aims at reducing the time taken to document a visit, and the introduction of dot phrases, or DotPhrases, in electronic medical records is a step toward reducing the time required to update patients' charts, which might allow doctors to spend more time with their patients. DotPhrases, abbreviated phrases used in the electronic medical record, help in the simplification of note-taking and the standardization of notes. They also allow for a more comprehensive note from physicians and ensure that no information is undocumented. On the contrary, however, excessive usage of DotPhrases can lead to an excessively long and cumbersome note. This can overwhelm physicians and lead to them missing crucial information that is buried somewhere in the notes. Although there is ample research studying the benefits of DotPhrases, adequate research must also be carried out to understand their shortcomings and disadvantages. This article aims to shed some light on use of DotPhrases and to outline their advantages and disadvantages affecting patient management and care.

Adapted large language models can outperform medical experts in clinical text summarization

Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown promise in natural language processing (NLP) tasks, their effectiveness on a diverse range of clinical summarization tasks remains unproven. Here we applied adaptation methods to eight LLMs, spanning four distinct clinical summarization tasks: radiology reports, patient questions, progress notes and doctor-patient dialogue. Quantitative assessments with syntactic, semantic and conceptual NLP metrics reveal trade-offs between models and adaptation methods. A clinical reader study with 10 physicians evaluated summary completeness, correctness and conciseness; in most cases, summaries from our best-adapted LLMs were deemed either equivalent (45%) or superior (36%) compared with summaries from medical experts. The ensuing safety analysis highlights challenges faced by both LLMs and medical experts, as we connect errors to potential medical harm and categorize types of fabricated information. Our research provides evidence of LLMs outperforming medical experts in clinical text summarization across multiple tasks. This suggests that integrating LLMs into clinical workflows could alleviate documentation burden, allowing clinicians to focus more on patient care.

[Inability of hospital computerised physician order entry systems to secure the use of concentrated potassium intravenous solutions]

OBJECTIVES

To determine whether hospital computerised physician order entry (CPOE) systems contribute to securing intravenous potassium chloride (KCl) prescriptions with reference to the recommendations issued by French healthcare agencies.

METHODS

We sent a questionnaire to the members of the Association pour le Digital et l'Information en Pharmacie.

RESULTS

More than three quarters of the 84 responses received involving 23 CPOE systems indicate that it is possible to: prescribe an ampoule of concentrated potassium chloride 10% 10mL intravenously without any diluents (80%); prescribe 4g of KCl in a bag of 500mL of NaCl 0,9% (98%); prescribe a solution that contains 6 grams of KCl per liter (94%); prescribe the administration of an injectable ampoule orally by means of a free text comment (83%). Nearly half of the responses indicate that it is possible to prescribe: concentrated KCl ampoules as administration solvent (50%); an injectable vial to be administered by oral route (52%).

CONCLUSION

At least 23 hospital CPOE systems are unable to secure the prescriptions of injectable KCl. This finding lifts the veil on an unthought, namely the role of CPOE systems in securing high-risk medications. In order to solve this problem, it should be mandatory that health information technology vendors pay particular attention to these drugs. With regard to injectable KCl, the utilisation of a dilution vehicle, maximum concentration and maximum infusion flow rate are the first four constraints to be satisfied.

Collecting patient-reported outcome measures in the electronic health record: Lessons from the NIH pragmatic trials Collaboratory

The NIH Pragmatic Trials Collaboratory supports the design and conduct of 27 embedded pragmatic clinical trials, and many of the studies collect patient reported outcome measures as primary or secondary outcomes. Study teams have encountered challenges in the collection of these measures, including challenges related to competing health care system priorities, clinician's buy-in for adoption of patient-reported outcome measures, low adoption and reach of technology in low resource settings, and lack of consensus and standardization of patient-reported outcome measure selection and administration in the electronic health record. In this article, we share case examples and lessons learned, and suggest that, when using patient-reported outcome measures for embedded pragmatic clinical trials, investigators must make important decisions about whether to use data collected from the participating health system's electronic health record, integrate externally collected patient-reported outcome data into the electronic health record, or collect these data in separate systems for their studies.

Design of an interface to communicate artificial intelligence-based prognosis for patients with advanced solid tumors: a user-centered approach

OBJECTIVES

To design an interface to support communication of machine learning (ML)-based prognosis for patients with advanced solid tumors, incorporating oncologists' needs and feedback throughout design.

MATERIALS AND METHODS

Using an interdisciplinary user-centered design approach, we performed 5 rounds of iterative design to refine an interface, involving expert review based on usability heuristics, input from a color-blind adult, and 13 individual semi-structured interviews with oncologists. Individual interviews included patient vignettes and a series of interfaces populated with representative patient data and predicted survival for each treatment decision point when a new line of therapy (LoT) was being considered. Ongoing feedback informed design decisions, and directed qualitative content analysis of interview transcripts was used to evaluate usability and identify enhancement requirements.

RESULTS

Design processes resulted in an interface with 7 sections, each addressing user-focused questions, supporting oncologists to "tell a story" as they discuss prognosis during a clinical encounter. The iteratively enhanced interface both triggered and reflected design decisions relevant when attempting to communicate ML-based prognosis, and exposed misassumptions. Clinicians requested enhancements that emphasized interpretability over explainability. Qualitative findings confirmed that previously identified issues were resolved and clarified necessary enhancements (eg, use months not days) and concerns about usability and trust (eg, address LoT received elsewhere). Appropriate use should be in the context of a conversation with an oncologist.

CONCLUSION

User-centered design, ongoing clinical input, and a visualization to communicate ML-related outcomes are important elements for designing any decision support tool enabled by artificial intelligence, particularly when communicating prognosis risk.

Clinical Text Summarization: Adapting Large Language Models Can Outperform Human Experts

Sifting through vast textual data and summarizing key information from electronic health records (EHR) imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown immense promise in natural language processing (NLP) tasks, their efficacy on a diverse range of clinical summarization tasks has not yet been rigorously demonstrated. In this work, we apply domain adaptation methods to eight LLMs, spanning six datasets and four distinct clinical summarization tasks: radiology reports, patient questions, progress notes, and doctor-patient dialogue. Our thorough quantitative assessment reveals trade-offs between models and adaptation methods in addition to instances where recent advances in LLMs may not improve results. Further, in a clinical reader study with ten physicians, we show that summaries from our best-adapted LLMs are preferable to human summaries in terms of completeness and correctness. Our ensuing qualitative analysis highlights challenges faced by both LLMs and human experts. Lastly, we correlate traditional quantitative NLP metrics with reader study scores to enhance our understanding of how these metrics align with physician preferences. Our research marks the first evidence of LLMs outperforming human experts in clinical text summarization across multiple tasks. This implies that integrating LLMs into clinical workflows could alleviate documentation burden, empowering clinicians to focus more on personalized patient care and the inherently human aspects of medicine.

Assessing Electronic Health Record (EHR) Use during a Major EHR Transition: An Innovative Mixed Methods Approach

BACKGROUND

Electronic health record (EHR) transitions are inherently disruptive to healthcare workers who must rapidly learn a new EHR and adapt to altered clinical workflows. Healthcare workers' perceptions of EHR usability and their EHR use patterns following transitions are poorly understood. The Department of Veterans Affairs (VA) is currently replacing its homegrown EHR with a commercial Cerner EHR, presenting a unique opportunity to examine EHR use trends and usability perceptions.

OBJECTIVE

To assess EHR usability and uptake up to 1-year post-transition at the first VA EHR transition site using a novel longitudinal, mixed methods approach.

DESIGN

A concurrent mixed methods strategy using EHR use metrics and qualitative interview data.

PARTICIPANTS

141 clinicians with data from select EHR use metrics in Cerner Lights On Network®. Interviews with 25 healthcare workers in various clinical and administrative roles.

APPROACH

We assessed changes in total EHR time, documentation time, and order time per patient post-transition. Interview transcripts (n = 90) were coded and analyzed for content specific to EHR usability.

KEY RESULTS

Total EHR time, documentation time, and order time all decreased precipitously within the first four months after go-live and demonstrated gradual improvements over 12 months. Interview participants expressed ongoing concerns with the EHR's usability and functionality up to a year after go-live such as tasks taking longer than the old system and inefficiencies related to inadequate training and inherent features of the new system. These sentiments did not seem to reflect the observed improvements in EHR use metrics.

CONCLUSIONS

The integration of quantitative and qualitative data yielded a complex picture of EHR usability. Participants described persistent challenges with EHR usability 1 year after go-live contrasting with observed improvements in EHR use metrics. Combining findings across methods can provide a clearer, contextualized understanding of EHR adoption and use patterns during EHR transitions.

Foundational domains and competencies for baccalaureate health informatics education

BACKGROUND

Foundational domains are the building blocks of educational programs. The lack of foundational domains in undergraduate health informatics (HI) education can adversely affect the development of rigorous curricula and may impede the attainment of CAHIIM accreditation of academic programs.

OBJECTIVE

This White Paper presents foundational domains developed by AMIA's Academic Forum Baccalaureate Education Committee (BEC) which include corresponding competencies (knowledge, skills, and attitudes) that are intended for curriculum development and CAHIIM accreditation quality assessment for undergraduate education in applied health informatics.

METHODS

The AMIA BEC used the previously published master's foundational domains as a guide to creating a set of competencies for health informatics at the undergraduate level to assess graduates from undergraduate health informatics programs for competence at graduation. A consensus method was used to adapt the domains for undergraduate level course work and harmonize the foundational domains with the currently adapted domains for HI master's education.

RESULTS

Ten foundational domains were developed to support the development and evaluation of baccalaureate health informatics education.

DISCUSSION

This article will inform future work towards building CAHIIM accreditation standards to ensure that higher education institutions meet acceptable levels of quality for undergraduate health informatics education.

Inpatient EHR User Experience and Hospital EHR Safety Performance

IMPORTANCE

Despite the broad adoption and optimization of electronic health record (EHR) systems across the continuum of care, serious usability and safety problems persist.

OBJECTIVE

To assess whether EHR safety performance is associated with EHR frontline user experience in a national sample of hospitals.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study included all US adult hospitals that used the National Quality Forum Leapfrog Health IT Safety Measure and also used the ARCH Collaborative EHR User experience survey from January 1, 2017, to January 1, 2019. Data analysis was performed from September 2020 to November 2022.

MAIN OUTCOMES AND MEASURES

The primary outcomes were hospital performance on the Leapfrog Health IT Safety measure (overall and 10 subcomponents) and the ARCH collaborative frontline user experience scores (overall and 8 subcomponents). Ordinary least squares models with survey responses clustered by hospital were used to assess associations between the overall measures and their subcomponents.

RESULTS

There were 112 hospitals and 5689 frontline user surveys included in the study. Hospitals scored a mean of 0.673 (range, 0.297-0.973) on the Leapfrog Health IT safety measure; the mean ARCH EHR user experience score was 3.377 (range, 1 [best] to 5 [worst]). The adjusted β coefficient between the overall safety score and overall user experience score was 0.011 (95% CI, 0.006-0.016). The ARCH overall score was also significantly associated with 10 subcategory scores of the Leapfrog Health IT safety score, and the overall Leapfrog score was associated with the 8 subcategory scores of the ARCH user experience score.

CONCLUSIONS AND RELEVANCE

This cross-sectional study found a positive association between frontline user-rated EHR usability and EHR safety performance. This finding suggests that improving EHR usability, which is a current well-known pain point for EHR users, could have direct benefits in terms of improved EHR safety.

Designing an Effective Organizational Culture to Guard Against the Cyber Risks of Emerging Technologies

The Internet of Medical Things (IoMT) and interoperable technologies have transformed how patient data affect medical care; such technological innovations revolutionize how healthcare organizations (HCOs) improve cost, quality, and access. New cyber risks, however, accompany developing cyber ecosystems. Although immediate data exchange is beneficial, risk arises from the IoMT's increased susceptibility to human influence. The success of quality care relies on protecting health information technology (HIT) against newly developing cyber vulnerabilities. Therefore, managers must be just as invested in their HCO's cybersecurity protocols as cybercriminals are in bypassing those protocols. This essay proposes a healthcare cyber resiliency model that leverages human and technical factors through a cycle of feedback and process improvement. It intends to equip healthcare administrators with the foundational philosophy necessary to secure their emerging technologies.

Antibiotic clinical decision support for pneumonia in the ED: A randomized trial

BACKGROUND

Electronic health record-based clinical decision support (CDS) is a promising antibiotic stewardship strategy. Few studies have evaluated the effectiveness of antibiotic CDS in the pediatric emergency department (ED).

OBJECTIVE

To compare the effectiveness of antibiotic CDS vs. usual care for promoting guideline-concordant antibiotic prescribing for pneumonia in the pediatric ED.

DESIGN

Pragmatic randomized clinical trial.

SETTING AND PARTICIPANTS

Encounters for children (6 months-18 years) with pneumonia presenting to two tertiary care children s hospital EDs in the United States.

INTERVENTION

CDS or usual care was randomly assigned during 4-week periods within each site. The CDS intervention provided antibiotic recommendations tailored to each encounter and in accordance with national guidelines.

MAIN OUTCOME AND MEASURES

The primary outcome was exclusive guideline-concordant antibiotic prescribing within the first 24 h of care. Safety outcomes included time to first antibiotic order, encounter length of stay, delayed intensive care, and 3- and 7-day revisits.

RESULTS

1027 encounters were included, encompassing 478 randomized to usual care and 549 to CDS. Exclusive guideline-concordant prescribing did not differ at 24 h (CDS, 51.7% vs. usual care, 53.3%; odds ratio [OR] 0.94 [95% confidence interval [CI]: 0.73, 1.20]). In pre-specified stratified analyses, CDS was associated with guideline-concordant prescribing among encounters discharged from the ED (74.9% vs. 66.0%; OR 1.53 [95% CI: 1.01, 2.33]), but not among hospitalized encounters. Mean time to first antibiotic was shorter in the CDS group (3.0 vs 3.4 h; p = .024). There were no differences in safety outcomes.

CONCLUSIONS

Effectiveness of ED-based antibiotic CDS was greatest among those discharged from the ED. Longitudinal interventions designed to target both ED and inpatient clinicians and to address common implementation challenges may enhance the effectiveness of CDS as a stewardship tool.

Using Time Series Clustering to Segment and Infer Emergency Department Nursing Shifts from Electronic Health Record Log Files

Few computational approaches exist for abstracting electronic health record (EHR) log files into clinically meaningful phenomena like clinician shifts. Because shifts are a fundamental unit of work recognized in clinical settings, shifts may serve as a primary unit of analysis in the study of documentation burden. We conducted a proof- of-concept study to investigate the feasibility of a novel approach using time series clustering to segment and infer clinician shifts from EHR log files. From 33,535,585 events captured between April-June 2021, we computationally identified 43,911 potential shifts among 2,285 (74.2%) emergency department nurses. On average, computationally-identified shifts were 10.6±3.1 hours long. Based on data distributions, we classified these shifts based on type: day, evening, night; and length: 12-hour, 8-hour, other. We validated our method through manual chart review of computationally-identified 12-hour shifts achieving 92.0% accuracy. Preliminary results suggest unsupervised clustering methods may be a reasonable approach for rapidly identifying clinician shifts.

Understanding the perceived role of electronic health records and workflow fragmentation on clinician documentation burden in emergency departments

OBJECTIVE

Understand the perceived role of electronic health records (EHR) and workflow fragmentation on clinician documentation burden in the emergency department (ED).

METHODS

From February to June 2022, we conducted semistructured interviews among a national sample of US prescribing providers and registered nurses who actively practice in the adult ED setting and use Epic Systems' EHR. We recruited participants through professional listservs, social media, and email invitations sent to healthcare professionals. We analyzed interview transcripts using inductive thematic analysis and interviewed participants until we achieved thematic saturation. We finalized themes through a consensus-building process.

RESULTS

We conducted interviews with 12 prescribing providers and 12 registered nurses. Six themes were identified related to EHR factors perceived to contribute to documentation burden including lack of advanced EHR capabilities, absence of EHR optimization for clinicians, poor user interface design, hindered communication, increased manual work, and added workflow blockages, and five themes associated with cognitive load. Two themes emerged in the relationship between workflow fragmentation and EHR documentation burden: underlying sources and adverse consequences.

DISCUSSION

Obtaining further stakeholder input and consensus is essential to determine whether these perceived burdensome EHR factors could be extended to broader contexts and addressed through optimizing existing EHR systems alone or through a broad overhaul of the EHR's architecture and primary purpose.

CONCLUSION

While most clinicians perceived that the EHR added value to patient care and care quality, our findings underscore the importance of designing EHRs that are in harmony with ED clinical workflows to alleviate the clinician documentation burden.

Medical and nursing clinician perspectives on the usability of the hospital electronic medical record: A qualitative analysis

BACKGROUND

Electronic medical records (EMRs) have been widely implemented in Australian hospitals. Their usability and design to support clinicians to effectively deliver and document care is essential, as is their impact on clinical workflow, safety and quality, communication, and collaboration across health systems. Perceptions of, and data about, usability of EMRs implemented in Australian hospitals are key to successful adoption.

OBJECTIVE

To explore perspectives of medical and nursing clinicians on EMR usability utilising free-text data collected in a survey.

METHOD

Qualitative analysis of one free-text optional question included in a web-based survey. Respondents included medical and nursing/midwifery professionals in Australian hospitals (85 doctors and 27 nurses), who commented on the usability of the main EMR used.

RESULTS

Themes identified related to the status of EMR implementation, system design, human factors, safety and risk, system response time, and stability, alerts, and supporting the collaboration between healthcare sectors. Positive factors included ability to view information from any location; ease of medication documentation; and capacity to access diagnostic test results. Usability concerns included lack of intuitiveness; complexity; difficulties communicating with primary and other care sectors; and time taken to perform clinical tasks.

CONCLUSION

If the benefits of EMRs are to be realised, there are good reasons to address the usability challenges identified by clinicians. Easy solutions that could improve the usability experience of hospital-based clinicians include resolving sign-on issues, use of templates, and more intelligent alerts and warnings to avoid errors.

IMPLICATIONS

These essential improvements to the usability of the EMR, which are the foundation of the digital health system, will enable hospital clinicians to deliver safer and more effective health care.

Extended Reality Applications for Space Health

INTRODUCTION: Spaceflight has detrimental effects on human health, imposing significant and unique risks to crewmembers due to physiological adaptations, exposure to physical and psychological stressors, and limited capabilities to provide medical care. Previous research has proposed and evaluated several strategies to support and mitigate the risks related to astronauts' health and medical exploration capabilities. Among these, extended reality (XR) technologies, including augmented reality (AR), virtual reality (VR), and mixed reality (MR) have increasingly been adopted for training, real-time clinical, and operational support in both terrestrial and aerospace settings, and only a few studies have reported research results on the applications of XR technologies for improving space health. This study aims to systematically review the scientific literature that has explored the application of XR technologies in the space health field. We also discuss the methodological and design characteristics of the existing studies in this realm, informing future research and development efforts on applying XR technologies to improve space health and enhance crew safety and performance.Ebnali M, Paladugu P, Miccile C, Park SH, Burian B, Yule S, Dias RD. Extended reality applications for space health. Aerosp Med Hum Perform. 2023; 94(3):122-130.

Human factors in healthcare IT: Management considerations and trends

A range of human factors issues are recognized as critical to the success of projects involving Health Information Technology (HIT). Problems related to the usability of HIT have come to the fore, with continued reports of systems that are non-intuitive and difficult to use and that may even pose safety risks. In this article, we consider a number of approaches from usability engineering and human factors that can be applied to improve the chances of system success and adoption. A range of methods focused around human factors can be employed throughout the system development cycle of HIT. The purpose of this article is to discuss human factors approaches that can be used to improve the likelihood of successful system adoption and also provide input into the selection and procurement process of HIT. The article concludes with recommendations regarding how understanding of human factors can be integrated into healthcare organizational decision making.

Unintended consequences of pharmacy information systems: A case study

BACKGROUND

Pharmacy information systems (PhIS) can cause medication errors that pharmacists may overlook due to their increased workload and lack of understanding of maintaining information quality. This study seeks to identify factors influencing unintended consequences of PhIS and how they affect the information quality, which can pose a risk to patient safety.

MATERIALS AND METHODS

This qualitative, explanatory case study evaluated PhIS in ambulatory pharmacies in a hospital and a clinic. Data were collected through observations, interviews, and document analysis. We applied the socio-technical interactive analysis (ISTA) framework to investigate the socio-technical interactions of pharmacy information systems that lead to unintended consequences. We then adopted the human-organization-process-technology-fit (HOPT-fit) framework to identify their contributing and dominant factors, misfits, and mitigation measures.

RESULTS

We identified 28 unintended consequences of PhIS, their key contributing factors, and their interrelations with the systems. The primary causes of unintended consequences include system rigidity and complexity, unclear knowledge, understanding, skills, and purpose of using the system, use of hybrid paper and electronic documentation, unclear and confusing transitions, additions and duplication of tasks and roles in the workflow, and time pressure, causing cognitive overload and workarounds. Recommended mitigating mechanisms include human factor principles in system design, data quality improvement for PhIS in terms of effective use of workspace, training, PhIS master data management, and communication by standardizing workarounds.

CONCLUSION

Threats to information quality emerge in PhIS because of its poor design, a failure to coordinate its functions and clinical tasks, and pharmacists' lack of understanding of the system use. Therefore, safe system design, fostering awareness in maintaining the information quality of PhIS and cultivating its safe use in organizations is essential to ensure patient safety. The proposed evaluation approach facilitates the evaluator to identify complex socio-technical interactions and unintended consequences factors, impact, and mitigation mechanisms.

A Systematic Review of Quantitative Methods for Evaluating Electronic Medication Administration Record and Bar-Coded Medication Administration Usability

BACKGROUND

Although electronic medication administration records (eMARs) and bar-coded medication administration (BCMA) have improved medication safety, poor usability of these technologies can increase patient safety risks.

OBJECTIVES

The objective of our systematic review was to identify the impact of eMAR and BCMA design on usability, operationalized as efficiency, effectiveness, and satisfaction.

METHODS

We retrieved peer-reviewed journal articles on BCMA and eMAR quantitative usability measures from PsycInfo and MEDLINE (1946-August 20, 2019), and EMBASE (1976-October 23, 2019). Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we screened articles, extracted and categorized data into the usability categories of effectiveness, efficiency, and satisfaction, and evaluated article quality.

RESULTS

We identified 1,922 articles and extracted data from 41 articles. Twenty-four articles (58.5%) investigated BCMA only, 10 (24.4%) eMAR only, and seven (17.1%) both BCMA and eMAR. Twenty-four articles (58.5%) measured effectiveness, 8 (19.5%) efficiency, and 17 (41.5%) satisfaction. Study designs included randomized controlled trial (n = 1; 2.4%), interrupted time series (n = 1; 2.4%), pretest/posttest (n = 21; 51.2%), posttest only (n = 14; 34.1%), and pretest/posttest and posttest only for different dependent variables (n = 4; 9.8%). Data collection occurred through observations (n = 19, 46.3%), surveys (n = 17, 41.5%), patient safety event reports (n = 9, 22.0%), surveillance (n = 6, 14.6%), and audits (n = 3, 7.3%).

CONCLUSION

Of the 100 measures across the 41 articles, implementing BCMA and/or eMAR broadly resulted in an increase in measures of effectiveness (n = 23, 52.3%) and satisfaction (n = 28, 62.2%) compared to measures of efficiency (n = 3, 27.3%). Future research should focus on eMAR efficiency measures, utilize rigorous study designs, and generate specific design requirements.

Improving the User Experience with Discount Site-Specific User Testing

OBJECTIVES

Poor electronic health record (EHR) usability is associated with patient safety concerns, user dissatisfaction, and provider burnout. EHR certification requires vendors to perform user testing. However, there are no such requirements for site-specific implementations. Health care organizations customize EHR implementations, potentially introducing usability problems. Site-specific usability evaluations may help to identify these concerns, and "discount" usability methods afford health systems a means of doing so even without dedicated usability specialists. This report characterizes a site-specific discount user testing program launched at an academic medical center. We describe lessons learned and highlight three of the EHR features in detail to demonstrate the impact of testing on implementation decisions and on users.

METHODS

Thirteen new EHR features which had already undergone heuristic evaluation and iterative design were evaluated over the course of three user test events. Each event included five to six users. Participants used think aloud technique. Measures of user efficiency, effectiveness, and satisfaction were collected. Usability concerns were characterized by the type of usability heuristic violated and by correctability.

RESULTS

Usability concerns occurred at a rate of 2.5 per feature tested. Seventy percent of the usability concerns were deemed correctable prior to implementation. The first highlighted feature was moved to production despite low single ease question (SEQ) scores which may have predicted its subsequent withdrawal from production based on post implementation feedback. Another feature was rebuilt based on usability findings, and a new version was retested and moved to production. A third feature highlights an easily correctable usability concern identified in user testing. Quantitative usability metrics generally reinforced qualitative findings.

CONCLUSION

Simplified user testing with a limited number of participants identifies correctable usability concerns, even after heuristic evaluation. Our discount usability approach to site-specific usability has a role in implementations and may improve the usability of the EHR for the end user.

Patient Safety Issues From Information Overload in Electronic Medical Records

BACKGROUND AND OBJECTIVE

Electronic health records (EHRs) have become ubiquitous in medicine and continue to grow in informational content. Little has been documented regarding patient safety from the resultant information overload. The objective of this literature review is to better understand how information overload in EHR affects patient safety.

METHODS

A literature search was performed using the Transparent Reporting of Systematic Reviews and Meta-Analyses standards for literature review. PubMed and Web of Science were searched and articles selected that were relevant to EHR information overload based on keywords.

RESULTS

The literature search yielded 28 articles meeting the criteria for the study. Information overload was found to increase physician cognitive load and error rates in clinical simulations. Overabundance of clinically irrelevant information, poor data display, and excessive alerting were consistently identified as issues that may lead to information overload.

CONCLUSIONS

Information overload in EHRs may result in higher error rates and negatively impact patient safety. Further studies are necessary to define the role of EHR in adverse patient safety events and to determine methods to mitigate these errors. Changes focused on the usability of EHR should be considered with the end user (physician) in mind. Federal agencies have a role to play in encouraging faster adoption of improved EHR interfaces.

Efficiency and Safety of Electronic Health Records in Switzerland-A Comparative Analysis of 2 Commercial Systems in Hospitals

OBJECTIVES

Differences in efficiency and safety between 2 electronic health record (systems A and B) in Swiss hospitals were investigated.

METHODS

In a scenario-based usability test under experimental conditions, a total of 100 physicians at 4 hospitals were asked to complete typical routine tasks, like medication or imaging orders. Differences in number of mouse clicks and time-on-task as indicators of efficiency and error type, error count, and rate as indicators of patient safety between hospital sites were analyzed. Time-on-task and clicks were correlated with error count.

RESULTS

There were differences in efficiency and safety between hospitals. Overall, physicians working with system B required less clicks (A: 511, B: 442, P = 0.001) and time (A: 2055 seconds, B: 1713 seconds, P = 0.055) and made fewer errors (A: 40%, B: 27%, P < 0.001). No participant completed all tasks correctly. The most frequent error in medication and radiology ordering was a wrong dose and a wrong level, respectively. Time errors were particularly prevalent in laboratory orders. Higher error counts coincided with longer time-on-task (r = 0.50, P < 0.001) and more clicks (r = 0.47, P < 0.001).

CONCLUSIONS

The variations in clicks, time, and errors are likely due to naive functionality and design of the systems and differences in their implementation. The high error rates coincide with inefficiency and jeopardize patient safety and produce economic costs and burden on physicians. The results raise usability concerns with potential for severe patient harm. A deeper understanding of differences as well as regulative guidelines and policy making are needed.

Development and pilot evaluation of an electronic health record usability and safety self-assessment tool

Objective

Poor electronic health record (EHR) usability contributes to clinician burnout and poses patent safety risks. Site-specific customization and configuration of EHRs require individual EHR system usability and safety testing which is resource intensive. We developed and pilot-tested a self-administered EHR usability and safety assessment tool, focused on computerized provider order entry (CPOE), which can be used by any facility to identify specific issues. In addition, the tool provides recommendations for improvement.

Materials and Methods

An assessment tool consisting of 104 questions was developed and pilot-tested at 2 hospitals, one using a Cerner EHR and the other using Epic. Five physicians at each site participated in and completed the assessment. Participant response accuracy compared to actual EHR interactions, consistency across participants, and usability issues identified through the tool were measured at each site.

Results

Across sites, participants answered an average of 46 questions in 23 min with 89.9% of responses either correct or partially correct. The tool identified 8 usability and safety issues at one site and 7 at the other site across medication, laboratory, and radiology CPOE functions.

Discussion

The tool shows promise as a method to rapidly evaluate EHR usability and safety and provide guidance on specific areas for improvement. Important improvements to the evaluation tool were identified including the need to clarify certain questions and provide definitions for usability terminology.

Conclusion

A self-administered usability and safety assessment tool can serve to identify specific usability and safety issues in the EHR and provide guidance for improvements.

Pediatric Dose Calculation Issues and the Need for Human Factors–Informed Preventative Technology Optimizations

Background: Dose calculation errors are one of the most common types of medication errors impacting children and they can result in significant harm. Technology-based solutions, such as computerized provider order entry, can effectively reduce dose calculation issues; however, these technologies are not always optimized, resulting in potential benefits not being fully realized. Methods: We analyzed pediatric dose-related patient safety event reports submitted to the Pennsylvania Patient Safety Reporting System using a task-analytic approach that focused on information being used in the dose calculation, calculation errors during ordering, and errors during dose preparation or administration. From these reports, we identified whether the patient was impacted by the error, the type of medication involved, and whether a technology optimization could have mitigated the issue. Results: Of the 356 reports reviewed, 326 (91.6%) met the criteria for a dose calculation issue. The 326 reports meeting criteria had the following dose calculation issue types: wrong information used in the calculation (49 of 326, 15.0%), incorrect calculation during ordering (97 of 326, 29.8%), and calculated dose was not properly used or incorrect calculation during preparation/administration (180 of 326, 55.2%). Most of these dose calculation issues impacted the patient (219 of 326, 67.2%). Analysis of these issues by patient age group and drug class also revealed interesting patterns. Technology optimizations potentially could have addressed 81.6% of the dose calculation issues identified. Conclusion: While many healthcare facilities have adopted health information technology and other devices to support the medication process, these technologies are not always optimized to address dose calculation issues. Human factors–informed recommendations, a safety checklist, and test cases for optimizing technology are provided in the context of these findings.

Impact of a Vendor-Developed Opioid Clinical Decision Support Intervention on Adherence to Prescribing Guidelines, Opioid Prescribing, and Rates of Opioid-Related Encounters

BACKGROUND

 Provider prescribing practices contribute to an excess of opioid-related deaths in the United States. Clinical guidelines exist to assist providers with improving prescribing practices and promoting patient safety. Clinical decision support systems (CDSS) may promote adherence to these guidelines and improve prescribing practices. The aim of this project was to improve opioid guideline adherence, prescribing practices, and rates of opioid-related encounters through the implementation of an opioid CDSS.

METHODS

 A vendor-developed, provider-targeted CDSS package was implemented in a multi-location academic health center. An interrupted time-series analysis was performed, evaluating 30 weeks pre- and post-implementation time periods. Outcomes were derived from vendor-supplied key performance indicators and directly from the electronic health record (EHR) database. Opioid-prescribing outcomes included count of opioid prescriptions, morphine milligram equivalents per prescription, counts of opioids with concurrent benzodiazepines, and counts of short-acting opioids in opioid-naïve patients. Encounter outcomes included rates of encounters for opioid abuse and dependence and rates of encounters for opioid poisoning and overdose. Guideline adherence outcomes included rates of provision of naloxone and documentation of opioid treatment agreements.

RESULTS

 The opioid CDSS generated an average of 1,637 alerts per week. Rates of provision of naloxone and opioid treatment agreements improved after CDSS implementation. Vendor-supplied prescribing outcomes were consistent with prescribing outcomes derived directly from the EHR, but all prescribing and encounter outcomes were unchanged.

CONCLUSION

 A vendor-developed, provider-targeted opioid CDSS did not improve opioid-prescribing practices or rates of opioid-related encounters. The CDSS improved some measures of provider adherence to opioid-prescribing guidelines. Further work is needed to determine the optimal configuration of opioid CDSS so that opioid-prescribing patterns are appropriately modified and encounter outcomes are improved.

Early Changes in Billing and Notes After Evaluation and Management Guideline Change

BACKGROUND

The American Medical Association updated guidance in 2021 for frequently used billing codes for outpatient evaluation and management (E/M) visits. The intent was to account for provider time outside of face-to-face encounters and to reduce onerous documentation requirements.

OBJECTIVE

To analyze E/M visit use, documentation length, and time spent in the electronic health record (EHR) before and after the guideline change.

DESIGN

Observational, retrospective, pre-post study.

SETTING

U.S.-based ambulatory practices using the Epic Systems EHR.

PARTICIPANTS

303 547 advanced practice providers and physicians across 389 organizations.

MEASUREMENTS

Data from September 2020 through April 2021 containing weekly provider-level E/M code and EHR use metadata were extracted from the Epic Signal database. We descriptively analyzed overall and specialty-specific changes in E/M visit use, note length, and time spent in the EHR before and after the new guidelines using provider-level paired t tests.

RESULTS

Following the new guidelines, level 3 visits decreased by 2.41 percentage points (95% CI, -2.48 to -2.34 percentage points) to 38.5% of all E/M visits, a 5.9% relative decrease from fall 2020. Level 4 visits increased by 0.89 percentage points (CI, 0.82 to 0.96 percentage points) to 40.9% of E/M visits, a 2.2% relative increase. Level 5 visits (the highest acuity level) increased by 1.85 percentage points (CI, 1.81 to 1.89 percentage points) to 10.1% of E/M visits, a 22.6% relative increase. These changes varied by specialty. We found no meaningful changes in measures of note length or time spent in the EHR.

LIMITATION

The Epic ambulatory client base may underrepresent smaller and independent practices.

CONCLUSION

Immediate changes in E/M coding contrast with null findings for changes in both note length and EHR time. Provider organizations are positioned to respond more rapidly to billing process changes than to changes in care delivery and associated EHR use behaviors. Fully realizing the intended benefits of this guideline change will require more time, facilitation, and scaling of best practices that more directly address EHR documentation practices and associated burden.

PRIMARY FUNDING SOURCE

None.

Collaborative Research and Development of a Novel, Patient-Centered Digital Platform (MyEyeSite) for Rare Inherited Retinal Disease Data: Acceptability and Feasibility Study

BACKGROUND

Inherited retinal diseases (IRDs) are a leading cause of blindness in children and working age adults in the United Kingdom and other countries, with an appreciable socioeconomic impact. However, by definition, IRD data are individually rare, and as a result, this patient group has been underserved by research. Researchers need larger amounts of these rare data to make progress in this field, for example, through the development of gene therapies. The challenge has been how to find and make these data available to researchers in the most productive way. MyEyeSite is a research collaboration aiming to design and develop a digital platform (the MyEyeSite platform) for people with rare IRDs that will enable patients, doctors, and researchers to aggregate and share specialist eye health data. A crucial component of this platform is the MyEyeSite patient application, which will provide the means for patients with IRD to interact with the system and, in particular, to collate, manage, and share their personal specialist IRD data both for research and their own health care.

OBJECTIVE

This study aims to test the acceptability and feasibility of the MyEyeSite platform in the target IRD population through a collaborative patient-centered study.

METHODS

Qualitative data were generated through focus groups and workshops, and quantitative data were obtained through a survey of patients with IRD. Participants were recruited through clinics at Moorfields Eye Hospital National Health Service (NHS) Foundation Trust and the National Institute for Health Research (NIHR) Moorfields Biomedical Research Centre through their patient and public involvement databases.

RESULTS

Our IRD focus group sample (n=50) highlighted the following themes: frustration with the current system regarding data sharing within the United Kingdom's NHS; positive expectations of the potential benefits of the MyEyeSite patient application, resulting from increased access to this specialized data; and concerns regarding data security, including potentially unethical use of the data outside the NHS. Of the surveyed 80 participants, 68 (85%) were motivated to have a more active role in their eye care and share their data for research purposes using a secure technology, such as a web application or mobile app.

CONCLUSIONS

This study demonstrates that patients with IRD are highly motivated to be actively involved in managing their own data for research and their own eye care. It demonstrates the feasibility of involving patients with IRD in the detailed design of the MyEyeSite platform exemplar, with input from the patient with IRD workshops playing a key role in determining both the functionality and accessibility of the designs and prototypes. The development of a user-centered technological solution to the problem of rare health data has the potential to benefit not only the patient with IRD community but also others with rare diseases.

FACTORS FOR SUCCESSFULLY PASSING CERTIFICATION EXAMS: A SYSTEMATIC REVIEW

This study explored possible success factors for health information management certification exams. Based on the American Health Information Management Association (AHIMA) website, in 2018 and 2019, only 70 percent of first-time test takers passed the Registered Health Information Administrator (RHIA) exam; 26 percent passed the Certified Health Data Analyst (CHDA) exam in 2018; and only 10 percent passed the Certified Health Data Analyst exam in 2019. A quantitative systematic review and meta-analysis offered insight into factors related to passing certification exams. Sources included existing, relevant peer-reviewed and published literature since 1990 within 87 educational and health/medicine databases and 62 other articles and journal databases available at the University of South Dakota library. Outcomes from the systematic review include illumination of factors for passing health information management, healthcare, and education certification exams. Ultimately, this new information will help improve pass rates on certification exams.

Scenario-Based Evaluation of Team Health Information Technology to Support Pediatric Trauma Care Transitions

BACKGROUND

Clinicians need health information technology (IT) that better supports their work. Currently, most health IT is designed to support individuals; however, more and more often, clinicians work in cross-functional teams. Trauma is one of the leading preventable causes of children's death. Trauma care by its very nature is team based but due to the emergent nature of trauma, critical clinical information is often missed in the transition of these patients from one service or unit to another. Teamwork transition technology can help support these transitions and minimize information loss while enhancing information gathering and storage. In this study, we created a large screen technology to support shared situational awareness across multiple clinical roles and departments.

OBJECTIVES

This study aimed to examine if the Teamwork Transition Technology (T3) supports teams and team cognition.

METHODS

We used a scenario-based mock-up methodology with 36 clinicians and staff from the different units and departments who are involved in pediatric trauma to examine T3.

RESULTS

Results of the evaluation show that most participants agreed that the technology helps achieve the goals set out in the design phase. Respondents thought that T3 organizes and presents information in a different way that was helpful to them.

CONCLUSION

In this study, we examined a health IT (T3) that was designed to support teams and team cognition. The results of our evaluation show that participants agreed that T3 does support them in their work and increases their situation awareness.

Emergency Physician Perceptions of Electronic Health Record Usability and Safety

OBJECTIVES

Despite requirements for electronic health record (EHR) vendor usability testing, usability challenges persist, contributing to patient safety concerns. We sought to identify emergency physicians' perceived EHR usability and safety strengths and shortcomings across major EHR vendor products.

METHODS

Fifty-five emergency physicians from 4 different hospitals were interviewed. The interviews were qualitatively analyzed, and physician comments were aligned with a usability taxonomy to identify emerging themes by vendor and hospital.

RESULTS

Of the 194 comments about usability, the 3 most commonly discussed usability topics were Workflow Support (33.5% of comments), Visual Display (20.1%), and Data Entry (14.4%). Electronic health record usability strengths were centered on Visual Display, and the most common shortcoming was the lack of Workflow Support. Fourteen cross-hospital/cross-vendor themes, 6 vendor-specific themes, and 4 hospital-specific themes were identified.

CONCLUSIONS

Usability shortcomings that spanned across hospitals and vendors may suggest a need for more applied research and improved design to resolve these issues. Shortcomings that are localized to a specific product or hospital may be due to customization and may be addressable by learning from other organizations.

Patient Safety Threats in Information Management Using Health Information Technology in Ambulatory Cancer Care: An Exploratory, Prospective Study

BACKGROUND

Cancer care is complex, involving highly toxic drugs, critically ill patients, and various different care providers. Because it is important for clinicians to have the latest and complete information about the patient available, this study focused on patient safety issues in information management developing from health information technology (HIT) use in oncology ambulatory infusion centers.

OBJECTIVE

The aim was to exploratively and prospectively assess patient safety risks from an expert perspective: instead of retrospectively analyzing safety events, we assessed the information management hazards inherent to the daily work processes; instead of asking healthcare workers at the front line, we used them as information sources to construct our patient safety expert view on the hazards.

METHODS

The work processes of clinicians in three ambulatory infusion centers were assessed and evaluated based on interviews and observations with a nurse and a physician of each unit. The 125 identified patient safety issues were described and sorted into thematic groups.

RESULTS

A broad range of patient safety issues was identified, such as data fragmentation, or information islands, meaning that patient data are stored across different cases or software and that different professional groups do not use the same set of information.

CONCLUSIONS

The current design and implementation of HIT systems do not support adequate information management: clinicians needed to play very close attention and improvise to avoid errors in using HIT and treat cancer patients safely. It is important to take the clinical front-end practice into account when evaluating or planning further HIT improvements.

Emerging digital technologies in cancer treatment, prevention, and control

The very first issue of the journal of Translational Behavioral Medicine (TBM) was dedicated, in part, to the theme of Health Information Technology as a platform for evidence implementation. The topic was timely: legislation in the USA was passed with the intent of stimulating the adoption of electronic health records; mobile smartphones, tablets, and other devices were gaining traction in the consumer market, while members within the Society of Behavioral Medicine were gaining scientific understanding on how to use these tools to effect healthy behavior change. For the anniversary issue of TBM, we evaluated the progress and problems associated with deploying digital health technologies to support cancer treatment, prevention, and control over the last decade. We conducted a narrative review of published literature to identify the role that emerging digital technologies may take in achieving national and international objectives in the decade to come. We tracked our evaluation of the literature across three phases in the cancer control continuum: (a) prevention, (b) early detection/screening, and (c) treatment/survivorship. From our targeted review and analyses, we noted that significant progress had been made in the adoption of digital health technologies in the cancer space over the past decade but that significant work remains to be done to integrate these technologies effectively into the cancer control systems needed to improve outcomes equitably across populations. The challenge for the next 10 years is inherently translational.

Workflow integration analysis of a human factors-based clinical decision support in the emergency department

Numerous challenges with the implementation, acceptance, and use of health IT are related to poor usability and a lack of integration of the technologies into clinical workflow, and have, therefore, limited the potential of these technologies to improve patient safety. We propose a definition and conceptual model of health IT workflow integration. Using interviews of 12 emergency department (ED) physicians, we identify 134 excerpts of barriers and facilitators to workflow integration of a human factors (HF)-based clinical decision support (CDS) implemented in the ED. Using data on these 134 barriers and facilitators, we distinguish 25 components of workflow integration of the CDS, which are described according to four dimensions of workflow integration: time, flow, scope of patient journey, and level. The proposed definition and conceptual model of workflow integration can be used to inform health IT design; this is the purpose of the proposed checklist that can help to ensure consideration of workflow integration during the development of health IT.

Association Between Proficiency and Efficiency in Electronic Health Records Among Pediatricians at a Major Academic Health System

Objective: The purpose of this study was to evaluate the variations in electronic health record (EHR) activity among General and Specialty pediatricians by investigating the time spent and documentation length, normalized for workload.

Materials and Methods: We conducted a cross-sectional study of pediatric physicians using Epic EHR at a major Southeastern academic healthcare system. We collected user-level EHR activity data of 104 pediatric physicians over 91 days from April 1 to June 30, 2020.

Results: Of the 104 pediatrics physicians, 56 (54%) were General pediatricians and 48 (46%) were Specialists pediatricians. General pediatricians spent an average of 17.6 min [interquartile range (IQR): 12.9–37] using the EHR per appointment, while Specialists spent 35.7 min (IQR: 28–48.4) per appointment.

Significant negative associations were found between proficiency scores and the amount of time spent in the system for Generalists (p < 0.001). On the contrary, significant positive associations were found between proficiency scores and the amount of time spent in the system for Specialists (p < 0.01).

Conclusions: We report an association between EHR proficiency and efficiency levels among pediatricians within the same healthcare system, receiving the same EHR training, and using the same EHR system. The profound differences in EHR activity suggest that higher priority should be given to redesigning EHR training methods to accommodate the learning needs of physicians.

A Time and Motion Analysis of Nursing Workload and Electronic Health Record Use in the Emergency Department

INTRODUCTION

The use of an electronic health record may create unanticipated consequences for emergency care delivery. We sought to describe emergency department nursing task distribution and the use of the electronic health record.

METHODS

This was a prospective observational study of nurses in the emergency department using a time-and-motion methodology. Three trained research assistants conducted 1:1 observations between March and September 2019. Nurse tasks were classified into 6 established categories: electronic health record, direct/indirect patient care, communication, personal time, and other. Nurses' perceived workload was assessed using the National Aeronautics and Space Administration (NASA) Task Load Index.

RESULTS

Twenty-three observations were conducted over 46 hours. Overall, nurses spent 27% of their time on electronic health record tasks, 25% on direct patient care, 17% on personal time, 15% on indirect patient care, and 6% on communication. During morning (7 am-12 pm) and afternoon shifts (12 pm-3 pm), the use of the health record was the most commonly performed task, whereas indirect patient care was the task most performed during evening shifts (3 pm-12 pm). Using the National Aeronautics and Space Administration (NASA) Task Load Index, nurses reported an increase in mental demand and effort during afternoon shifts compared with morning shifts.

DISCUSSION

We observed that emergency nurses spent more time using the electronic health record as compared to other tasks. Increased usability of the electronic health record, particularly during high occupancy periods, may be a target for improvement.

Classification of Electronic Health Record-Related Patient Safety Incidents: Development and Validation Study

Background

It is assumed that the implementation of health information technology introduces new vulnerabilities within a complex sociotechnical health care system, but no international consensus exists on a standardized format for enhancing the collection, analysis, and interpretation of technology-induced errors.

Objective

This study aims to develop a classification for patient safety incident reporting associated with the use of mature electronic health records (EHRs). It also aims to validate the classification by using a data set of incidents during a 6-month period immediately after the implementation of a new EHR system.

Methods

The starting point of the classification development was the Finnish Technology-Induced Error Risk Assessment Scale tool, based on research on commonly recognized error types. A multiprofessional research team used iterative tests on consensus building to develop a classification system. The final classification, with preliminary descriptions of classes, was validated by applying it to analyze EHR-related error incidents (n=428) during the implementation phase of a new EHR system and also to evaluate this classification’s characteristics and applicability for reporting incidents. Interrater agreement was applied.

Results

The number of EHR-related patient safety incidents during the implementation period (n=501) was five-fold when compared with the preimplementation period (n=82). The literature identified new error types that were added to the emerging classification. Error types were adapted iteratively after several test rounds to develop a classification for reporting patient safety incidents in the clinical use of a high-maturity EHR system. Of the 427 classified patient safety incidents, interface problems accounted for 96 (22.5%) incident reports, usability problems for 73 (17.1%), documentation problems for 60 (14.1%), and clinical workflow problems for 33 (7.7%). Altogether, 20.8% (89/427) of reports were related to medication section problems, and downtime problems were rare (n=8). During the classification work, 14.8% (74/501) of reports of the original sample were rejected because of insufficient information, even though the reports were deemed to be related to EHRs. The interrater agreement during the blinded review was 97.7%.

Conclusions

This study presents a new classification for EHR-related patient safety incidents applicable to mature EHRs. The number of EHR-related patient safety incidents during the implementation period may reflect patient safety challenges during the implementation of a new type of high-maturity EHR system. The results indicate that the types of errors previously identified in the literature change with the EHR development cycle.

Moving toward a sociotechnical systems approach to continuous health information technology design: the path forward for improving electronic health record usability and reducing clinician burnout

Based on our analysis of descriptions provided by four EHR vendors on their EHR usability efforts, we provide three recommendations aimed at improving the usability of health information technology and reducing clinician burnout. First, EHR vendors need to dedicate increased attention to the design of the entire sociotechnical (work) system, including the EHR technology and its usability as well as the interactions of the technology with other system elements. Second, EHR vendors need to deepen and broaden their understanding of the work of clinicians and care teams by using diverse and mixed method. Third, in collaboration with health care organizations, EHR vendors should engage in cycles of continuous design and learning in order to improve the usability of health IT.

Physician experiences of screen-level features in a prominent electronic health record: Design recommendations from a qualitative study

The goal of this qualitative study was to assess physicians' perceptions around features of key screens within a prominent commercial EHR, and to solicit end-user recommendations for improved retrieval of high-priority clinical information. We conducted a qualitative, descriptive study of 25 physicians in a medical ICU setting. at a tertiary academic medical center. An in-depth, semi-structured interview guide was developed to elicit physician perceptions on information retrieval as well as favorable and unfavorable features of specific EHR screens. Transcripts were independently coded in a qualitative software management tool by at least two trained coders using a common code book. We successfully obtained vendor permission to map physicians perception's on full Epic© screenshots. Among the 25 physician participants (13 female; 5 attending physicians, 9 fellows, 11 residents), the majority of participants reported experiencing challenges finding clinical information in the EHR. We present the most favorable and unfavorable screen-level features for four central EHR screens: Flowsheet, Notes/Chart Review, Results Review, and Vital Signs. We also compiled participants' recommendations for a comprehensive EHR dashboard screen to better support clinical workflow and information retrieval in the medical ICU through User-Centered Design. ICU physicians demonstrated a mix of positive and negative attitudes toward specific screen-level features in a major vendor-based EHR system. Physician perceptions of information overload emerged as a theme across multiple EHR screens. Our findings underscore the importance of qualitative research and end-user feedback in EHR software design and interface optimization at both the vendor and institutional level.

Characterizing physician EHR use with vendor derived data: a feasibility study and cross-sectional analysis

OBJECTIVE

To derive 7 proposed core electronic health record (EHR) use metrics across 2 healthcare systems with different EHR vendor product installations and examine factors associated with EHR time.

MATERIALS AND METHODS

A cross-sectional analysis of ambulatory physicians EHR use across the Yale-New Haven and MedStar Health systems was performed for August 2019 using 7 proposed core EHR use metrics normalized to 8 hours of patient scheduled time.

RESULTS

Five out of 7 proposed metrics could be measured in a population of nonteaching, exclusively ambulatory physicians. Among 573 physicians (Yale-New Haven N = 290, MedStar N = 283) in the analysis, median EHR-Time8 was 5.23 hours. Gender, additional clinical hours scheduled, and certain medical specialties were associated with EHR-Time8 after adjusting for age and health system on multivariable analysis. For every 8 hours of scheduled patient time, the model predicted these differences in EHR time (P < .001, unless otherwise indicated): female physicians +0.58 hours; each additional clinical hour scheduled per month -0.01 hours; practicing cardiology -1.30 hours; medical subspecialties -0.89 hours (except gastroenterology, P = .002); neurology/psychiatry -2.60 hours; obstetrics/gynecology -1.88 hours; pediatrics -1.05 hours (P = .001); sports/physical medicine and rehabilitation -3.25 hours; and surgical specialties -3.65 hours.

CONCLUSIONS

For every 8 hours of scheduled patient time, ambulatory physicians spend more than 5 hours on the EHR. Physician gender, specialty, and number of clinical hours practicing are associated with differences in EHR time. While audit logs remain a powerful tool for understanding physician EHR use, additional transparency, granularity, and standardization of vendor-derived EHR use data definitions are still necessary to standardize EHR use measurement.

Electronic Health Record Usability: Associations With Nurse and Patient Outcomes in Hospitals

BACKGROUND

Electronic health record (EHR) usability issues represent an emerging threat to the wellbeing of nurses and patients; however, few large studies have examined these relationships.

OBJECTIVE

To examine associations between EHR usability and nurse job (burnout, job dissatisfaction, and intention to leave) and surgical patient (inpatient mortality and 30-day readmission) outcomes.

METHODS

A cross-sectional analysis of linked American Hospital Association, state patient discharge, and nurse survey data was conducted. The sample included 343 hospitals, 1,281,848 surgical patients, and 12,004 nurses. Logistic regression models were used to assess relationships between EHR usability and outcomes, before and after accounting for EHR adoption level (comprehensive vs. basic or less) and other confounders.

RESULTS

In fully adjusted models, nurses who worked in hospitals with poorer EHR usability had significantly higher odds of burnout [odds ratio (OR), 1.41; 95% confidence interval (CI), 1.21-1.64], job dissatisfaction (OR, 1.61; 95% CI, 1.37-1.90) and intention to leave (OR, 1.31; 95% CI, 1.09-1.58) compared with nurses working in hospitals with better usability. Surgical patients treated in hospitals with poorer EHR usability had significantly higher odds of inpatient mortality (OR, 1.21; 95% CI, 1.09-1.35) and 30-day readmission (OR, 1.06; 95% CI, 1.01-1.12) compared with patients in hospitals with better usability. Comprehensive EHR adoption was associated with higher odds of nurse burnout (OR, 1.14; 95% CI, 1.01-1.28).

CONCLUSION

Employing EHR systems with suboptimal usability was associated with higher odds of adverse nurse job outcomes and surgical patient mortality and readmission. EHR usability may be more important to nurse job and patient outcomes than comprehensive EHR adoption.

Investigation of usability problems of electronic medical record systems in the emergency department

BACKGROUND

Despite the benefits of using electronic medical record (EMR) systems, existing studies show that many healthcare providers are uncertain regarding their usability. The usability issues of these systems decrease their efficiency, discourage clinicians, and cause dissatisfaction among patients, which may result in safety risks and harm.

OBJECTIVE

The aim of this study was to collect and analyze EMR system usability problems from actual users. Practical user interface guidelines were presented based on the medical practices of these users.

METHODS

Employing an online questionnaire with a seven-point Likert scale, usability issues of EMR systems were collected from 200 emergency department healthcare providers (103 physicians (medical doctors) and 97 nurses) from South Korea.

RESULTS

The most common usability problem among the physicians and nurses was generating in-patient selection. This pertained to the difficulty in finding the required information on-screen because of poor visibility and a lack of distinctiveness.

CONCLUSIONS

The major problems of EMR systems and their causes were identified. It is recommended that intensive visual enhancement of EMR system interfaces should be implemented to support user tasks. By providing a better understanding of the current usability problems among medical practitioners, the results of this study can be useful for developing EMR systems with increased effectiveness and efficiency.

Integrating Option Grid Patient Decision Aids in the Epic Electronic Health Record: Case Study at 5 Health Systems

Background

Some researchers argue that the successful implementation of patient decision aids (PDAs) into clinical workflows depends on their integration into electronic health records (EHRs). Anecdotally, we know that EHR integration is a complex and time-consuming task; yet, the process has not been examined in detail. As part of an implementation project, we examined the work involved in integrating an encounter PDA for symptomatic uterine fibroids into Epic EHR systems.

Objective

This study aims to identify the steps and time required to integrate a PDA into the Epic EHR system and examine facilitators and barriers to the integration effort.

Methods

We conducted a case study at 5 academic medical centers in the United States. A clinical champion at each institution liaised with their Epic EHR team to initiate the integration of the uterine fibroid Option Grid PDAs into clinician-facing menus. We scheduled regular meetings with the Epic software analysts and an expert Epic technologist to discuss how best to integrate the tools into Epic for use by clinicians with patients. The meetings were then recorded and transcribed. Two researchers independently coded the transcripts and field notes before categorizing the codes and conducting a thematic analysis to identify the facilitators and barriers to EHR integration. The steps were reviewed and edited by an Epic technologist to ensure their accuracy.

Results

Integrating the uterine fibroid Option Grid PDA into clinician-facing menus required an 18-month timeline and a 6-step process, as follows: task priority negotiation with Epic software teams, security risk assessment, technical review, Epic configuration; troubleshooting, and launch. The key facilitators of the process were the clinical champions who advocated for integration at the institutional level and the presence of an experienced technologist who guided Epic software analysts during the build. Another facilitator was the use of an emerging industry standard app platform (Health Level 7 Substitutable Medical Applications and Reusable Technologies on Fast Healthcare Interoperability Resources) as a means of integrating the Option Grid into existing systems. This standard platform enabled clinicians to access the tools by using single sign-on credentials and prevented protected health information from leaving the EHR. Key barriers were the lack of control over the Option Grid product developed by EBSCO (Elton B Stephens Company) Health; the periodic Epic upgrades that can result in a pause on new software configurations; and the unforeseen software problems with Option Grid (ie, inability to print the PDA), which delayed the launch of the PDA.

Conclusions

The integration of PDAs into the Epic EHR system requires a 6-step process and an 18-month timeline. The process required support and prioritization from a clinical champion, guidance from an experienced technologist, and a willing EHR software developer team.

Defining Data Migration Across Multidisciplinary Ambulatory Clinics Using Participatory Design

OBJECTIVE

This study aimed to develop an institutional approach for defining data migration based on participatory design principles.

METHODS

We outline a collaborative approach to define data migration as part of an electronic health record (EHR) transition at an urban hospital with 20 ambulatory clinics, based on participatory design. We developed an institution-specific list of data for migration based on physician end-user feedback. In this paper, we review the project planning phases, multidisciplinary governance, and methods used.

RESULTS

Detailed data migration feedback was obtained from 90% of participants. Depending on the specialty, requests for historical laboratory values ranged from 2 to as many as 145 unique laboratory types. Lookback periods requested by physicians varied and were ultimately assigned to provide the most clinical data. This clinical information was then combined to synthesize an overall proposed data migration request on behalf of the institution.

CONCLUSION

Institutions undergoing an EHR transition should actively involve physician end-users and key stakeholders. Physician feedback is vital for developing a clinically relevant EHR environment but is often difficult to obtain. Challenges include physician time constraints and overall knowledge about health information technology. This study demonstrates how a participatory design can serve to improve the clinical end-user's understanding of the technical aspects of an EHR implementation, as well as enhance the outcomes of such projects.