A usability and safety analysis of electronic health records: a multi-center study

2021 ISHNE/HRS/EHRA/APHRS collaborative statement on mHealth in Arrhythmia Management: Digital Medical Tools for Heart Rhythm Professionals: From the International Society for Holter and Noninvasive Electrocardiology/Heart Rhythm Society/European Heart Rhythm Association/Asia Pacific Heart Rhythm Society

This collaborative statement from the International Society for Holter and Noninvasive Electrocardiology/Heart Rhythm Society/European Heart Rhythm Association/Asia Pacific Heart Rhythm Society describes the current status of mobile health ("mHealth") technologies in arrhythmia management. The range of digital medical tools and heart rhythm disorders that they may be applied to and clinical decisions that may be enabled are discussed. The facilitation of comorbidity and lifestyle management (increasingly recognized to play a role in heart rhythm disorders) and patient self-management are novel aspects of mHealth. The promises of predictive analytics but also operational challenges in embedding mHealth into routine clinical care are explored.

Defining Data Migration Across Multidisciplinary Ambulatory Clinics Using Participatory Design

OBJECTIVE

This study aimed to develop an institutional approach for defining data migration based on participatory design principles.

METHODS

We outline a collaborative approach to define data migration as part of an electronic health record (EHR) transition at an urban hospital with 20 ambulatory clinics, based on participatory design. We developed an institution-specific list of data for migration based on physician end-user feedback. In this paper, we review the project planning phases, multidisciplinary governance, and methods used.

RESULTS

Detailed data migration feedback was obtained from 90% of participants. Depending on the specialty, requests for historical laboratory values ranged from 2 to as many as 145 unique laboratory types. Lookback periods requested by physicians varied and were ultimately assigned to provide the most clinical data. This clinical information was then combined to synthesize an overall proposed data migration request on behalf of the institution.

CONCLUSION

Institutions undergoing an EHR transition should actively involve physician end-users and key stakeholders. Physician feedback is vital for developing a clinically relevant EHR environment but is often difficult to obtain. Challenges include physician time constraints and overall knowledge about health information technology. This study demonstrates how a participatory design can serve to improve the clinical end-user's understanding of the technical aspects of an EHR implementation, as well as enhance the outcomes of such projects.

Usability of electronic health record systems in UK EDs

BACKGROUND

The large volume of patients, rapid staff turnover and high work pressure mean that the usability of all systems within the ED is important. The transition to electronic health records (EHRs) has brought many benefits to emergency care but imposes a significant burden on staff to enter data. Poor usability has a direct consequence and opportunity cost in staff time and resources that could otherwise be employed in patient care. This research measures the usability of EHR systems in UK EDs using a validated assessment tool.

METHODS

This was a survey completed by members and fellows of the Royal College of Emergency Medicine conducted during summer 2019. The primary outcome was the System Usability Scale Score, which ranges from 0 (worst) to 100 (best). Scores were compared with an internationally recognised measure of acceptable usability of 68. Results were analysed by EHR system, country, healthcare organisation and physician grade. Only EHR systems with at least 20 responses were analysed.

RESULTS

There were 1663 responses from a total population of 8794 (19%) representing 192 healthcare organisations (mainly UK NHS), and 25 EHR systems. Fifteen EHR systems had at least 20 responses and were included in the analysis. No EHR system achieved a median usability score that met the industry standard of acceptable usability.The median usability score was 53 (IQR 35-68). Individual EHR systems' scores ranged from 35 (IQR 26-53) to 65 (IQR 44-80).

CONCLUSION

In this survey, no UK ED EHR system met the internationally validated standard of acceptable usability for information technology.

2021 ISHNE/ HRS/ EHRA/ APHRS collaborative statement on mHealth in Arrhythmia Management: Digital Medical Tools for Heart Rhythm Professionals: From the International Society for Holter and Noninvasive Electrocardiology/Heart Rhythm Society/European Heart Rhythm Association/Asia Pacific Heart Rhythm Society

This collaborative statement from the International Society for Holter and Noninvasive Electrocardiology/ Heart Rhythm Society/ European Heart Rhythm Association/ Asia Pacific Heart Rhythm Society describes the current status of mobile health ("mHealth") technologies in arrhythmia management. The range of digital medical tools and heart rhythm disorders that they may be applied to and clinical decisions that may be enabled are discussed. The facilitation of comorbidity and lifestyle management (increasingly recognized to play a role in heart rhythm disorders) and patient self-management are novel aspects of mHealth. The promises of predictive analytics but also operational challenges in embedding mHealth into routine clinical care are explored.

A patient and family reporting system for perceived ambulatory note mistakes: experience at 3 U.S. healthcare centers

OBJECTIVE

The study sought to test a patient and family online reporting system for perceived ambulatory visit note inaccuracies.

MATERIALS AND METHODS

We implemented a patient and family electronic reporting system at 3 U.S. healthcare centers: a northeast urban academic adult medical center (AD), a northeast urban academic pediatric medical center (PED), and a southeast nonprofit hospital network (NET). Patients and families reported potential documentation inaccuracies after reading primary care and subspecialty visit notes. Results were characterized using descriptive statistics and coded for clinical relevance.

RESULTS

We received 1440 patient and family reports (780 AD, 402 PED, and 258 NET), and 27% of the reports identified a potential inaccuracy (25% AD, 35% PED, 28% NET). Among these, patients and families indicated that the potential inaccuracy was important or very important in 58% of reports (55% AD, 55% PED, 71% NET). The most common types of potential inaccuracies included description of symptoms (21%), past medical problems (21%), medications (18%), and important information that was missing (15%). Most patient- and family-reported inaccuracies resulted in a change to care or to the medical record (55% AD, 67% PED, data not available at NET).

DISCUSSION

About one-quarter of patients and families using an online reporting system identified potential documentation inaccuracies in visit notes and more than half were considered important by patients and clinicians, underscoring the potential role of patients and families as ambulatory safety partners.

CONCLUSIONS

Partnering with patients and families to obtain reports on inaccuracies in visit notes may contribute to safer care. Mechanisms to encourage greater use of patient and family reporting systems are needed.

Effect of an Electronic Health Record "Nudge" on Opioid Prescribing and Electronic Health Record Keystrokes in Ambulatory Care

BACKGROUND

Multiple policy initiatives encourage more cautious prescribing of opioids in light of their risks. Electronic health record (EHR) redesign can influence prescriber choices, but some redesigns add to workload.

OBJECTIVE

To estimate the effect of an EHR prescribing redesign on both opioid prescribing choices and keystrokes.

DESIGN

Quality improvement quasi-experiment, analyzed as interrupted time series.

PARTICIPANTS

Adult patients of an academic multispecialty practice and a federally qualified health center (FQHC) who received new prescriptions for short-acting opioids, and their providers.

INTERVENTION

In the redesign, new prescriptions of short-acting opioids defaulted to the CDC-recommended minimum for opioid-naïve patients, with no alerts or hard stops, such that 9 keystrokes were required for a guideline-concordant prescription and 24 for a non-concordant prescription.

MAIN MEASURES

Proportion of guideline-concordant prescriptions, defined as new prescriptions with a 3-day supply or less, calculated per 2-week period. Number of mouse clicks and keystrokes needed to place prescriptions.

KEY RESULTS

Across the 2 sites, 22,113 patients received a new short-acting opioid prescription from 821 providers. Before the intervention, both settings showed secular trends toward smaller-quantity prescriptions. At the academic practice, the intervention was associated with an immediate increase in guideline-concordant prescriptions from an average of 12% to 31% of all prescriptions. At the FQHC, about 44% of prescriptions were concordant at the time of the intervention, which was not associated with an additional significant increase. However, total keystrokes needed to place the concordant prescriptions decreased 62.7% from 3552 in the 6 months before the intervention to 1323 in the 6 months afterwards.

CONCLUSIONS

Autocompleting prescription forms with guideline-recommended values was associated with a large increase in guideline concordance in an organization where baseline concordance was low, but not in an organization where it was already high. The redesign markedly reduced the number of keystrokes needed to place orders, with important implications for EHR-related stress.

TRIAL REGISTRATION

www.ClinicalTrials.gov protocol 1710018646.

2021 ISHNE/HRS/EHRA/APHRS Collaborative Statement on mHealth in Arrhythmia Management: Digital Medical Tools for Heart Rhythm Professionals

This collaborative statement from the International Society for Holter and Noninvasive Electrocardiology/Heart Rhythm Society/European Heart Rhythm Association/Asia Pacific Heart Rhythm Society describes the current status of mobile health ("mHealth") technologies in arrhythmia management. The range of digital medical tools and heart rhythm disorders that they may be applied to and clinical decisions that may be enabled are discussed. The facilitation of comorbidity and lifestyle management (increasingly recognized to play a role in heart rhythm disorders) and patient self-management are novel aspects of mHealth. The promises of predictive analytics but also operational challenges in embedding mHealth into routine clinical care are explored.

2021 ISHNE/HRS/EHRA/APHRS Expert Collaborative Statement on mHealth in Arrhythmia Management: Digital Medical Tools for Heart Rhythm Professionals: From the International Society for Holter and Noninvasive Electrocardiology/Heart Rhythm Society/European Heart Rhythm Association/Asia-Pacific Heart Rhythm Society

This collaborative statement from the International Society for Holter and Noninvasive Electrocardiology/Heart Rhythm Society/European Heart Rhythm Association/Asia-Pacific Heart Rhythm Society describes the current status of mobile health technologies in arrhythmia management. The range of digital medical tools and heart rhythm disorders that they may be applied to and clinical decisions that may be enabled are discussed. The facilitation of comorbidity and lifestyle management (increasingly recognized to play a role in heart rhythm disorders) and patient self-management are novel aspects of mobile health. The promises of predictive analytics but also operational challenges in embedding mobile health into routine clinical care are explored.

2021 ISHNE / HRS / EHRA / APHRS Collaborative Statement on mHealth in Arrhythmia Management: Digital Medical Tools for Heart Rhythm Professionals: From the International Society for Holter and Noninvasive Electrocardiology / Heart Rhythm Society / European Heart Rhythm Association / Asia Pacific Heart Rhythm Society

This collaborative statement from the International Society for Holter and Noninvasive Electrocardiology / Heart Rhythm Society / European Heart Rhythm Association / Asia Pacific Heart Rhythm Society describes the current status of mobile health ("mHealth") technologies in arrhythmia management. The range of digital medical tools and heart rhythm disorders that they may be applied to and clinical decisions that may be enabled are discussed. The facilitation of comorbidity and lifestyle management (increasingly recognized to play a role in heart rhythm disorders) and patient self-management are novel aspects of mHealth. The promises of predictive analytics but also operational challenges in embedding mHealth into routine clinical care are explored.

Impact of electronic health records on predefined safety outcomes in patients admitted to hospital: a scoping review

OBJECTIVES

Review available evidence for impact of electronic health records (EHRs) on predefined patient safety outcomes in interventional studies to identify gaps in current knowledge and design interventions for future research.

DESIGN

Scoping review to map existing evidence and identify gaps for future research.

DATA SOURCES

PubMed, the Cochrane Library, EMBASE, Trial registers.

STUDY SELECTION

Eligibility criteria: We conducted a scoping review of bibliographic databases and the grey literature of randomised and non-randomised trials describing interventions targeting a list of fourteen predefined areas of safety. The search was limited to manuscripts published between January 2008 and December 2018 of studies in adult inpatient settings and complemented by a targeted search for studies using a sample of EHR vendors. Studies were categorised according to methodology, intervention characteristics and safety outcome.Results from identified studies were grouped around common themes of safety measures.

RESULTS

The search yielded 583 articles of which 24 articles were included. The identified studies were largely from US academic medical centres, heterogeneous in study conduct, definitions, treatment protocols and study outcome reporting. Of the 24 included studies effective safety themes included medication reconciliation, decision support for prescribing medications, communication between teams, infection prevention and measures of EHR-specific harm. Heterogeneity of the interventions and study characteristics precluded a systematic meta-analysis. Most studies reported process measures and not patient-level safety outcomes: We found no or limited evidence in 13 of 14 predefined safety areas, with good evidence limited to medication safety.

CONCLUSIONS

Published evidence for EHR impact on safety outcomes from interventional studies is limited and does not permit firm conclusions regarding the full safety impact of EHRs or support recommendations about ideal design features. The review highlights the need for greater transparency in quality assurance of existing EHRs and further research into suitable metrics and study designs.

Frontiers in human factors: embedding specialists in multi-disciplinary efforts to improve healthcare

Despite the application of a huge range of human factors (HF) principles in a growing range of care contexts, there is much more that could be done to realize this expertise for patient benefit, staff well-being and organizational performance. Healthcare has struggled to embrace system safety approaches, misapplied or misinterpreted others, and has stuck to a range of outdated and potentially counter-productive myths even has safety science has developed. One consequence of these persistent misunderstandings is that few opportunities exist in clinical settings for qualified HF professionals. Instead, HF has been applied by clinicians and others, to highly variable degrees-sometimes great success, but frequently in limited and sometimes counter-productive ways. Meanwhile, HF professionals have struggled to make a meaningful impact on frontline care and have had little career structure or support. However, in the last few years, embedded clinical HF practitioners have begun to have considerable success that are now being supported and amplified by professional networks. The recent coronavirus disease of 2019 (COVID-19) experiences confirm this. Closer collaboration between healthcare and HF professionals will result in significant and ultimately beneficial changes to both professions and clinical care.

Development and Optimization of Clinical Informatics Infrastructure to Support Bioinformatics at an Oncology Center

Translational bioinformatics for therapeutic discovery requires the infrastructure of clinical informatics. In this chapter, we describe the clinical informatics components needed for successful implementation of translational research at a cancer center. This chapter is meant to be an introduction to those clinical informatics concepts that are needed for translational research. For a detailed account of clinical informatics, the authors will guide the reader to comprehensive resources. We provide examples of workflows from Moffitt Cancer Center led by Drs. Perkins and Markowitz. This perspective represents an interesting collaboration as Dr. Perkins is the Chief Medical Information Officer and Dr. Markowitz is a translational researcher in Melanoma with an active informatics component to his laboratory to study the mechanisms of resistance to checkpoint blockade and an active member of the clinical informatics team.

The complex case of EHRs: examining the factors impacting the EHR user experience

Physicians can spend more time completing administrative tasks in their electronic health record (EHR) than engaging in direct face time with patients. Increasing rates of burnout associated with EHR use necessitate improvements in how EHRs are developed and used. Although EHR design often bears the brunt of the blame for frustrations expressed by physicians, the EHR user experience is influenced by a variety of factors, including decisions made by entities other than the developers and end users, such as regulators, policymakers, and administrators. Identifying these key influences can help create a deeper understanding of the challenges in developing a better EHR user experience. There are multiple opportunities for regulators, policymakers, EHR developers, payers, health system leadership, and users each to make changes to collectively improve the use and efficacy of EHRs.

Perceived Electronic Health Record Usability as a Predictor of Task Load and Burnout Among US Physicians: Mediation Analysis

BACKGROUND

Electronic health record (EHR) usability and physician task load both contribute to physician professional burnout. The association between perceived EHR usability and workload has not previously been studied at a national level. Better understanding these interactions could give further information as to the drivers of extraneous task load.

OBJECTIVE

This study aimed to determine the relationship between physician-perceived EHR usability and workload by specialty and evaluate for associations with professional burnout.

METHODS

A secondary analysis of a cross-sectional survey of US physicians from all specialties was conducted from October 2017 to March 2018. Among the 1250 physicians invited to respond to the subsurvey analyzed here, 848 (67.8%) completed it. EHR usability was assessed with the System Usability Scale (SUS; range: 0-100). Provider task load (PTL) was assessed using the mental demand, physical demand, temporal demand, and effort required subscales of the National Aeronautics and Space Administration Task Load Index (range: 0-400). Burnout was measured using the Maslach Burnout Inventory.

RESULTS

The mean scores were 46.1 (SD 22.1) for SUS and 262.5 (SD 71.7) for PTL. On multivariable analysis adjusting for age, gender, relationship status, medical specialty, practice setting, hours worked per week, and number of nights on call per week, physician-rated EHR usability was associated with PTL, with each 1-point increase in SUS score (indicating more favorable) associated with a 0.57-point decrease in PTL score (P<.001). On mediation analysis, higher SUS score was associated with lower PTL score, which was associated with lower odds of burnout.

CONCLUSIONS

A strong association was observed between EHR usability and workload among US physicians, with more favorable usability associated with less workload. Both outcomes were associated with the odds of burnout, with task load acting as a mediator between EHR usability and burnout. Improving EHR usability while decreasing task load has the potential to allow practicing physicians more working memory for medical decision making and patient communication.

Limiting the number of open charts does not impact wrong patient order entry in the emergency department

OBJECTIVE

We sought to examine the impact of limiting the number of open active charts on wrong patient order entry events among 13 emergency departments (EDs) in a large integrated health system.

METHODS

A retrospective chart review of all orders placed between September 2017 and September 2019 was conducted. The rate of retract and reorder events was analyzed with no overlap in both the period pre- and post-intervention period. Secondary analysis of error rate by clinician type, clinician patient load, and time of day was performed.

RESULTS

The order retraction rate was not improved pre- and post-intervention. Retraction rates varied by clinician type with residents retracting more often than physicians (odds ratio [OR] = 1.443 [1.349, 1.545]). Advanced practice providers also showed a slightly higher rate than physicians (OR = 1.114 [1.071, 1.160]). Pharmacists showed very low rates compared to physicians (OR = 0.191 [0.048, 0.764]). Time of day and staffing ratios appear to be a factor with wrong patient order entry rates slightly lower during the night (1900-0700) than the day (OR 0.958 [0.923, 0.995]), and increasing slightly with every additional patient per provider (OR 1.019 [1.005, 1.032]). The Academic Medical Center had more retractions that the other EDs. OR for the various ED types compared to the Academic Medical Center included Community (OR 0.908 [0.859, 0.959]), Teaching Hospitals (OR 0.850 [0.802, 0.900]), and Freestanding (OR 0.932 [0.864, 1.006]).

CONCLUSIONS

Limiting the number of open active charts from 4 to 2 did not significantly reduce the incidence of wrong patient order entry. Further investigation into other factors contributing to order entry errors is warranted.

A micro-analytic approach to understanding electronic health record navigation paths

Clinician task performance is significantly impacted by the navigational efficiency of the system interface. Here we propose and evaluate a navigational complexity framework useful for examining differences in electronic health record (EHR) interface systems and their impact on task performance. The methodological approach includes 1) expert-based methods-specifically, representational analysis (focused on interface elements), keystroke level modeling (KLM), and cognitive walkthrough; and 2) quantitative analysis of interactive behaviors based on video-captured observations. Medication administration record (MAR) tasks completed by nurses during preoperative (PreOp) patient assessment were studied across three Mayo Clinic regional campuses and three different EHR systems. By analyzing the steps executed within the interfaces involved to complete the MAR tasks, we characterized complexities in EHR navigation. These complexities were reflected in time spent on task, click counts, and screen transitions, and were found to potentially influence nurses' performance. Two of the EHR systems, employing a single screen format, required less time to complete (mean 101.5, range 106-97 s), respectively, compared to one system employing multiple screens (176 s, 73% increase). These complexities surfaced through trade-offs in cognitive processes that could potentially influence nurses' performance. Factors such as perceptual-motor activity, visual search, and memory load impacted navigational complexity. An implication of this work is that small tractable changes in interface design can substantially improve EHR navigation, overall usability, and workflow.

The Effect of Electronic Health Record Usability Redesign on Annual Screening Rates in an Ambulatory Setting

OBJECTIVES

Improving the usability of electronic health records (EHR) continues to be a focus of clinicians, vendors, researchers, and regulatory bodies. To understand the impact of usability redesign of an existing, site-configurable feature, we evaluated the user interface (UI) used to screen for depression, alcohol and drug misuse, fall risk, and the existence of advance directive information in ambulatory settings.

METHODS

As part of a quality improvement project, based on heuristic analysis, the existing UI was redesigned. Using an iterative, user-centered design process, several usability defects were corrected. Summative usability testing was performed as part of the product development and implementation cycle. Clinical quality measures reflecting rolling 12-month rates of screening were examined over 8 months prior to the implementation of the redesigned UI and 9 months after implementation.

RESULTS

Summative usability testing demonstrated improvements in task time, error rates, and System Usability Scale scores. Interrupted time series analysis demonstrated significant improvements in all screening rates after implementation of the redesigned UI compared with the original implementation.

CONCLUSION

User-centered redesign of an existing site-specific UI may lead to significant improvements in measures of usability and quality of patient care.

Human-Computer Interaction, Ethics, and Biomedical Informatics

Objectives : To provide an overview of recent work at the intersection of Biomedical Informatics, Human-Computer Interaction, and Ethics.

Methods : Search terms for Human-Computer Interaction, Biomedical Informatics, and Ethics were used to identify relevant papers published between 2017 and 2019.Relevant papers were identified through multiple methods, including database searches, manual reviews of citations, recent publications, and special collections, as well as through peer recommendations. Identified articles were reviewed and organized into broad themes.

Results : We identified relevant papers at the intersection of Biomedical Informatics, Human-Computer Interactions, and Ethics in over a dozen journals. The content of these papers was organized into three broad themes: ethical issues associated with systems in use, systems design, and responsible conduct of research.

Conclusions : The results of this overview demonstrate an active interest in exploring the ethical implications of Human-Computer Interaction concerns in Biomedical Informatics. Papers emphasizing ethical concerns associated with patient-facing tools, mobile devices, social media, privacy, inclusivity, and e-consent reflect the growing prominence of these topics in biomedical informatics research. New questions in these areas will likely continue to arise with the growth of precision medicine and citizen science.

The influence of electronic health record use on collaboration among medical specialties

Background

One of the main objectives of Electronic Health Records (EHRs) is to enhance collaboration among healthcare professionals. However, our knowledge of how EHRs actually affect collaborative practices is limited. This study examines how an EHR facilitates and constrains collaboration in five outpatient clinics.

Methods

We conducted an embedded case study at five outpatient clinics of a Dutch hospital that had implemented an organization-wide EHR. Data were collected through interviews with representatives of medical specialties, administration, nursing, and management. Documents were analyzed to contextualize these data. We examined the following collaborative affordances of EHRs: (1) portability, (2) co-located access, (3) shared overviews, (4) mutual awareness, (5) messaging, and (6) orchestrating.

Results

Our findings demonstrate how an EHR will both facilitate and constrain collaboration among specialties and disciplines. Affordances that were inscribed in the system for collaboration purposes were not fully actualized in the hospital because:

(a) The EHR helps health professionals coordinate patient care on an informed basis at any time and in any place but only allows asynchronous patient record use.

(b) The comprehensive patient file affords joint clinical decision-making based on shared data, but specialty- and discipline-specific user-interfaces constrain mutual understanding of that data. Moreover, not all relevant information can be easily shared across specialties and outside the hospital.

(c) The reduced necessity for face-to-face communication saves time but is experienced as hindering collective responsibility for a smooth workflow.

(d) The EHR affords registration at the source and registration of activities through orders, but the heightened administrative burden for physicians and the strict authorization rules on inputting data constrain the flexible, multidisciplinary collaboration.

(e) While the EHR affords a complete overview, information overload occurs due to the parallel generation of individually owned notes and the high frequency of asynchronous communication through messages of varying clinical priority.

Conclusions

For the optimal actualization of EHRs’ collaborative affordances in hospitals, coordinated use of these affordances by health professionals is a prerequisite. Such coordinated use requires organizational, technical, and behavioral adaptations. Suggestions for hospital-wide policies to enhance trust in both the EHR and in its coordinated use for effective collaboration are offered.

The national health information technology human factors and ergonomics agenda

Health information technology (IT) implementation has encompassed much of the United States health care system over the past decade, and user frustration with health IT has steadily increased. Human factors and ergonomics (HFE) methods and approaches can improve the design, implementation, and use of health IT for clinicians and consumers. To better understand the effect of federal HFE in health IT research funding, the authors conducted a review of several key, specific initiatives. The review focused on the goals and accomplishments of these initiatives. Findings to date show that HFE is improving the usefulness of health IT, but additional research and new methods are needed. Corresponding research funding and policy priorities are identified. New HFE work and innovative approaches are needed to capitalize on HFE knowledge, principles, and methods to improve the design, implementation, and use of health IT at a broader scale.

Studying Workflow and Workarounds in Electronic Health Record-Supported Work to Improve Health System Performance

Clinical workflow is the enactment of a series of steps to perform a clinical activity. The transition from paper to electronic health records (EHRs) over the past decade has been characterized by profound challenges supporting clinical workflow, impeding frontline clinicians' ability to deliver safe, efficient, and effective care. In response, there has been substantial effort to study clinical workflow as well as workarounds-exceptions to routine workflow-in order to identify opportunities for improvement. This article describes predominant methods of studying workflow and workarounds and provides examples of the applications of these methods along with the resulting insights. Challenges to studying workflow and workarounds are described, and recommendations for how to approach such studies are given. Although there is not yet a set of standard approaches, this article helps advance workflow research that ultimately serves to inform how to coevolve the design of EHR systems and organizational decisions about processes, roles, and responsibilities in order to support clinical workflow that more consistently delivers on the potential benefits of a digitized health care system.

Assessment of Health Information Technology-Related Outpatient Diagnostic Delays in the US Veterans Affairs Health Care System: A Qualitative Study of Aggregated Root Cause Analysis Data

IMPORTANCE

Diagnostic delay in the outpatient setting is an emerging safety priority that health information technology (HIT) should help address. However, diagnostic delays have persisted, and new safety concerns associated with the use of HIT have emerged.

OBJECTIVE

To analyze HIT-related outpatient diagnostic delays within a large, integrated health care system.

DESIGN, SETTING, AND PARTICIPANTS

This cohort study involved qualitative content analysis of safety concerns identified in aggregated root cause analysis (RCA) data related to HIT and outpatient diagnostic delays. The setting was the US Department of Veterans Affairs using all RCAs submitted to the Veterans Affairs (VA) National Center for Patient Safety from January 1, 2013, to July 31, 2018.

MAIN OUTCOMES AND MEASURES

Common themes associated with the role of HIT-related safety concerns were identified and categorized according to the Health IT Safety framework for measuring, monitoring, and improving HIT safety. This framework includes 3 related domains (ie, safe HIT, safe use of HIT, and using HIT to improve safety) situated within an 8-dimensional sociotechnical model accounting for interacting technical and nontechnical variables associated with safety. Hence, themes identified enhanced understanding of the sociotechnical context and domain of HIT safety involved.

RESULTS

Of 214 RCAs categorized by the terms delay and outpatient submitted during the study period, 88 were identified as involving diagnostic delays and HIT, from which 172 unique HIT-related safety concerns were extracted (mean [SD], 1.97 [1.53] per RCA). Most safety concerns (82.6% [142 of 172]) involved problems with safe use of HIT, predominantly sociotechnical factors associated with people, workflow and communication, and a poorly designed human-computer interface. Fewer safety concerns involved problems with safe HIT (14.5% [25 of 172]) or using HIT to improve safety (0.3% [5 of 172]). The following 5 key high-risk areas for diagnostic delays emerged: managing electronic health record inbox notifications and communication, clinicians gathering key diagnostic information, technical problems, data entry problems, and failure of a system to track test results.

CONCLUSIONS AND RELEVANCE

This qualitative study of a national RCA data set suggests that interventions to reduce outpatient diagnostic delays could aim to improve test result management, interoperability, data visualization, and order entry, as well as to decrease information overload.

Varying rates of patient identity verification when using computerized provider order entry

OBJECTIVE

We sought to determine rates of computerized provider order entry (CPOE) patient identity verification and when and where in the ordering process verification occurred.

MATERIALS AND METHODS

Fifty-five physicians from 4 healthcare systems completed simulated patient scenarios using their respective CPOE system (Epic or Cerner). Eye movements were recorded and analyzed.

RESULTS

Across all participants patient id was verified significantly more often than not (62.4% vs 37.6%). Vendor A had significantly higher verification rates than not; vendor B had no difference. Participants using vendor A verified information significantly more often before signing the order than after (88.4% vs 11.6%); there was no difference in vendor B. The banner bar was the most frequent verification location.

DISCUSSION

Factors such as CPOE design, physician training, and the use of a simulated methodology may be impacting verification rates.

CONCLUSIONS

Verification rates vary by CPOE product, and this can have patient safety consequences.

Assessing the cognitive and work load of an inpatient safety dashboard in the context of opioid management

For health information technology to realize its potential to improve flow, care, and patient safety, applications should be intuitive to use and burden neutral for frontline clinicians. We assessed the impact of a patient safety dashboard on clinician cognitive and work load within a simulated information-seeking task for safe inpatient opioid medication management. Compared to use of an electronic health record for the same task, the dashboard was associated with significantly reduced time on task, mouse clicks, and mouse movement (each p < 0.001), with no significant increases in cognitive load nor task inaccuracy. Cognitive burden was higher for users with less experience, possibly partly attributable to usability issues identified during this study. Findings underscore the importance of assessing the usability, cognitive, and work load analysis during the design and implementation of health information technology applications.

The Association Between Perceived Electronic Health Record Usability and Professional Burnout Among US Physicians

OBJECTIVE

To describe and benchmark physician-perceived electronic health record (EHR) usability as defined by a standardized metric of technology usability and evaluate the association with professional burnout among physicians.

PARTICIPANTS AND METHODS

This cross-sectional survey of US physicians from all specialty disciplines was conducted between October 12, 2017, and March 15, 2018, using the American Medical Association Physician Masterfile. Among the 30,456 invited physicians, 5197 (17.1%) completed surveys. A random 25% (n=1250) of respondents in the primary survey received a subsurvey evaluating EHR usability, and 870 (69.6%) completed it. EHR usability was assessed using the System Usability Scale (SUS; range 0-100). SUS scores were normalized to percentile rankings across more than 1300 previous studies from other industries. Burnout was measured using the Maslach Burnout Inventory.

RESULTS

Mean ± SD SUS score was 45.9±21.9. A score of 45.9 is in the bottom 9% of scores across previous studies and categorized in the "not acceptable" range or with a grade of F. On multivariate analysis adjusting for age, sex, medical specialty, practice setting, hours worked, and number of nights on call weekly, physician-rated EHR usability was independently associated with the odds of burnout with each 1 point more favorable SUS score associated with a 3% lower odds of burnout (odds ratio, 0.97; 95% CI, 0.97-0.98; P<.001).

CONCLUSION

The usability of current EHR systems received a grade of F by physician users when evaluated using a standardized metric of technology usability. A strong dose-response relationship between EHR usability and the odds of burnout was observed.

Health information technology:Fallacies and Sober realities - Redux A homage to Bentzi Karsh and Robert Wears

Since the publication of "Health Information Technology: Fallacies and Sober Realities" in 2010, health information technology (HIT) has become nearly ubiquitous in US healthcare facilities. Yet, HIT has yet to achieve its putative benefits of higher quality, safer, and lower cost care. There has been variable but largely marginal progress at addressing the 12 HIT fallacies delineated in the original paper. Here, we revisit several of the original fallacies and add five new ones. These fallacies must be understood and addressed by all stakeholders for HIT to be a positive force in achieving the high value healthcare system the nation deserves. Foundational cognitive and human factors engineering research and development continue to be essential to HIT development, deployment, and use.

Overcoming challenges to data quality in the ASPREE clinical trial

Background

Large-scale studies risk generating inaccurate and missing data due to the complexity of data collection. Technology has the potential to improve data quality by providing operational support to data collectors. However, this potential is under-explored in community-based trials. The Aspirin in reducing events in the elderly (ASPREE) trial developed a data suite that was specifically designed to support data collectors: the ASPREE Web Accessible Relational Database (AWARD). This paper describes AWARD and the impact of system design on data quality.

Methods

AWARD’s operational requirements, conceptual design, key challenges and design solutions for data quality are presented. Impact of design features is assessed through comparison of baseline data collected prior to implementation of key functionality (n = 1000) with data collected post implementation (n = 18,114). Overall data quality is assessed according to data category.

Results

At baseline, implementation of user-driven functionality reduced staff error (from 0.3% to 0.01%), out-of-range data entry (from 0.14% to 0.04%) and protocol deviations (from 0.4% to 0.08%). In the longitudinal data set, which contained more than 39 million data values collected within AWARD, 96.6% of data values were entered within specified query range or found to be accurate upon querying. The remaining data were missing (3.4%). Participant non-attendance at scheduled study activity was the most common cause of missing data. Costs associated with cleaning data in ASPREE were lower than expected compared with reports from other trials.

Conclusions

Clinical trials undertake complex operational activity in order to collect data, but technology rarely provides sufficient support. We find the AWARD suite provides proof of principle that designing technology to support data collectors can mitigate known causes of poor data quality and produce higher-quality data. Health information technology (IT) products that support the conduct of scheduled activity in addition to traditional data entry will enhance community-based clinical trials. A standardised framework for reporting data quality would aid comparisons across clinical trials.

Trial registration

International Standard Randomized Controlled Trial Number Register, ISRCTN83772183. Registered on 3 March 2005.

How well is the electronic health record supporting the clinical tasks of hospital physicians? A survey of physicians at three Norwegian hospitals

Background

The electronic health record is expected to improve the quality and efficiency of health care. Many novel functionalities have been introduced in order to improve medical decision making and communication between health care personnel. There is however limited evidence on whether these new functionalities are useful. The aim of our study was to investigate how well the electronic health record system supports physicians in performing basic clinical tasks.

Methods

Physicians of three prominent Norwegian hospitals participated in the survey. They were asked, in an online questionnaire, how well the hospital’s electronic health record system DIPS supported 49 clinical tasks as well as how satisfied they were with the system in general, including the technical performance. Two hundred and eight of 402 physicians (52%) submitted a completely answered questionnaire.

Results

Seventy-two percent of the physicians had their work interrupted or delayed because the electronic health record hangs or crashes at least once a week, while 22% had experienced this problem daily. Fifty-three percent of the physicians indicated that the electronic health record is cumbersome to use and adds to their workload. The majority of physicians were satisfied with managing tests, e.g., requesting laboratory tests, reading test results and managing radiological investigations and electrocardiograms. Physicians were less satisfied with managing referrals. There was high satisfaction with some of the decision support functionalities available for prescribing drugs. This includes drug interaction alerts and drug allergy warnings, which are displayed automatically. However, physicians were less satisfied with other aspects of prescribing drugs, including getting an overview of the ongoing drug therapy.

Conclusions

In the survey physicians asked for improvements of certain electronic health record functionalities like medication, clinical workflow support including planning and better overviews. In addition, there is apparently a need to focus on system stability, number of logins, reliability and better instructions on available electronic health record features. Considerable development is needed in current electronic health record systems to improve usefulness and satisfaction.

Six habits of highly successful health information technology: powerful strategies for design and implementation

Healthcare information technologies are now a routine component of patient-clinician interactions. Originally designed for operational functions including billing and regulatory compliance, these systems have had unintended consequences including increased exam room documentation, divided attention during the visit, and use of scribes to alleviate documentation burdens. In an age in which technology is ubiquitous in everyday life, we must re-envision healthcare technology to support both clinical operations and, above all, the patient-clinician relationship. We present 6 habits for designing user-centered health technologies: (1) put patient care first, (2) assemble a team with the right skills, (3) relentlessly ask WHY, (4) keep it simple, (5) be Darwinian, and (6) don't lose the forest for the trees. These habits should open dialogues between developers, implementers, end users, and stakeholders, as well as outline a path for better, more usable technology that puts patients and their clinicians back at the center of care.

Assessment of Deep Natural Language Processing in Ascertaining Oncologic Outcomes From Radiology Reports

Importance

A rapid learning health care system for oncology will require scalable methods for extracting clinical end points from electronic health records (EHRs). Outside of clinical trials, end points such as cancer progression and response are not routinely encoded into structured data.

Objective

To determine whether deep natural language processing can extract relevant cancer outcomes from radiologic reports, a ubiquitous but unstructured EHR data source.

Design, Setting, and Participants

A retrospective cohort study evaluated 1112 patients who underwent tumor genotyping for a diagnosis of lung cancer and participated in the Dana-Farber Cancer Institute PROFILE study from June 26, 2013, to July 2, 2018.

Exposures

Patients were divided into curation and reserve sets. Human abstractors applied a structured framework to radiologic reports for the curation set to ascertain the presence of cancer and changes in cancer status over time (ie, worsening/progressing vs improving/responding). Deep learning models were then trained to capture these outcomes from report text and subsequently evaluated in a 10% held-out test subset of curation patients. Cox proportional hazards regression models compared human and machine curations of disease-free survival, progression-free survival, and time to improvement/response in the curation set, and measured associations between report classification and overall survival in the curation and reserve sets.

Main Outcomes and Measures

The primary outcome was area under the receiver operating characteristic curve (AUC) for deep learning models; secondary outcomes were time to improvement/response, disease-free survival, progression-free survival, and overall survival.

Results

A total of 2406 patients were included (mean [SD] age, 66.5 [10.8] years; 1428 female [59.7%]; 2170 [90.2%] white). Radiologic reports (n = 14 230) were manually reviewed for 1112 patients in the curation set. In the test subset (n = 109), deep learning models identified the presence of cancer, improvement/response, and worsening/progression with accurate discrimination (AUC >0.90). Machine and human curation yielded similar measurements of disease-free survival (hazard ratio [HR] for machine vs human curation, 1.18; 95% CI, 0.71-1.95); progression-free survival (HR, 1.11; 95% CI, 0.71-1.71); and time to improvement/response (HR, 1.03; 95% CI, 0.65-1.64). Among 15 000 additional reports for 1294 reserve set patients, algorithm-detected cancer worsening/progression was associated with decreased overall survival (HR for mortality, 4.04; 95% CI, 2.78-5.85), and improvement/response was associated with increased overall survival (HR, 0.41; 95% CI, 0.22-0.77).

Conclusions and Relevance

Deep natural language processing appears to speed curation of relevant cancer outcomes and facilitate rapid learning from EHR data.

Human Factors in Health(care) Informatics: Toward Continuous Sociotechnical System Design

Developing structures and processes for continuous sociotechnical system design is key to sustaining human factors (HF) knowledge in the context of rapid health care changes and technological innovations. Two research studies illustrate how to embed HF in organizational learning processes and structures. We need to develop innovative HF methods for continuous sociotechnical system design.

A Visual Analytics Dashboard to Summarize Serial Anesthesia Records in Pediatric Radiation Treatment

BACKGROUND

Young children who undergo radiation therapy may require general anesthesia to remain still during weeks of radiation sessions. On a typical day at our hospital, an anesthesia team will care for 10 patients in the radiation therapy suite, and each patient will have multiple prior anesthetic records. Daily review of prior anesthesia records is important to maintain anesthetic consistency and to identify potential improvement, yet our electronic health record (EHR) made such review time-consuming and cumbersome.

OBJECTIVES

This article aims to design a visual analytics interface that simultaneously displays data from multiple anesthesia encounters to support clinical consistency in medications and airway management.

METHODS

Documentation from the EHR is available in the clinical data warehouse following daily backups. A visual analytics interface was built to aggregate important components of multiple anesthesia encounters in pediatric radiation oncology on a single screen. The application was embedded in the EHR's anesthesia module and updated daily.

RESULTS

Each anesthesia encounter was represented by a vertical line with the date at the bottom of the screen. Each vertical line was divided into sections corresponding to the medications, type of airway device, type of radiation oncology procedure, days between treatments, and recovery score and time. Information about the medications, airways, and procedures was shown with icon legends. This layout enabled users to quickly see the key components of multiple anesthetics and make inferences between, for example, the medications used and the recovery score.

CONCLUSION

The dashboard provides a high-level summary of all radiation therapy anesthesia records for children receiving recurrent treatments. In this clinical scenario, it is desirable to replicate an optimal anesthetic approach for daily or near-daily treatments or adjust the anesthetic based on observed patterns.

Human Factors and Usability for Health Information Technology: Old and New Challenges

OBJECTIVES

Despite national mandates, incentives, and other programs, the design of health information technology (IT) remains problematic and usability problems continue to be reported. This paper reviews recent literature on human factors and usability of health IT, with a specific focus on research aimed at applying human factors methods and principles to improve the actual design of health IT, its use, and associated patient and clinician outcomes.

METHODS

We reviewed recent literature on human factors and usability problems of health IT and research on human-centered design of health IT for clinicians and patients.

RESULTS

Studies continue to show usability problems of health IT experienced by multiple groups of health care professionals (e.g., physicians and nurses) as well as patients. Recent research shows that usability is influenced by both designers (e.g., IT vendors) and implementers in health care organizations, and that the application of human-centered design practices needs to be further improved and extended. We welcome emerging research on the design of health IT for teams as team-based care is increasingly implemented throughout health care.

CONCLUSIONS

Progress in the application of human factors methods and principles to the design of health IT is occurring, with important information provided on their actual impact on care processes and patient outcomes. Future research should examine the work of health IT designers and implementers, which would help to develop strategies for further embedding human factors engineering in IT design processes.

Identifying Electronic Health Record Usability And Safety Challenges In Pediatric Settings

Pediatric populations are uniquely vulnerable to the usability and safety challenges of electronic health records (EHRs), particularly those related to medication, yet little is known about the specific issues contributing to hazards. To understand specific usability issues and medication errors in the care of children, we analyzed 9,000 patient safety reports, made in the period 2012-17, from three different health care institutions that were likely related to EHR use. Of the 9,000 reports, 3,243 (36 percent) had a usability issue that contributed to the medication event, and 609 (18.8 percent) of the 3,243 might have resulted in patient harm. The general pattern of usability challenges and medication errors were the same across the three sites. The most common usability challenges were associated with system feedback and the visual display. The most common medication error was improper dosing.

Reinterpreting the nursing record for an electronic context

BACKGROUND

This article seeks to facilitate the re-imagining of nursing records purposefully within an electronic context. It questions existing approaches to nursing documentation, critically examines existing nursing record systems and identifies new requirements.

METHODS

A comprehensive literature review was conducted to identify themes, that might meaningfully contribute to a new approach to nursing record systems development, around four key interrelated areas - standards, decision making, abstraction and summarization, and documenting. Studies were analyzed using narrative synthesis to provide a critical analysis of the current 'state of the art', and recommendations for the future.

RESULTS

Included studies collectively described aspects of current best practice, both in terms of nursing record systems themselves, and how nurses and other health professionals contribute to and engage with those systems. A number of cross-cutting themes identified more novel approaches taken by nurses to systems development: going back to basics in determining purpose; firming up informatics foundations; nuancing or tailoring to suit different requirements; and engagement, involvement and participation.

CONCLUSION

There is a paucity of research that specifically focuses on the nature of the electronic nursing record and its impact on patient care processes and outcomes. In addition to further research in these areas, there is a need: to reinterpret nurses as knowledge workers rather than as 'data collectors'; to agree on the application in practice of appropriate standards and terminologies; and to work together with system developers to change the ways in which data are captured and care is documented.

What is the impact of introducing inpatient electronic prescribing on prescribing errors? A naturalistic stepped wedge study in an English teaching hospital

Most studies evaluating the impact of electronic prescribing on prescribing safety have used comparatively weak study designs such as uncontrolled before-and-after studies. This study aimed to apply a more robust naturalistic stepped wedge study design to compare the prevalence and types of prescribing errors for electronic prescribing and paper prescribing. Data were collected weekly during a phased electronic prescribing implementation across 20 wards in a large English hospital. We identified 511 (7.8%) erroneous orders in 6523 paper medication orders, and 312 (6.0%) in 5237 electronic prescribing orders. Logistic regression suggested no statistically significant effect of electronic prescribing use or of study week; patient and ward had significant effects. Errors involving incorrect doses and illegible or incomplete orders were less common with electronic prescribing; those involving duplication, omission, incorrect drug and incorrect formulation were more common. Actions are needed to mitigate these error types; future studies should give more consideration to the effects of patient and ward.

Comparison of a Prototype for Indications-Based Prescribing With 2 Commercial Prescribing Systems

IMPORTANCE

The indication (reason for use) for a medication is rarely included on prescriptions despite repeated recommendations to do so. One barrier has been the way existing electronic prescribing systems have been designed.

OBJECTIVE

To evaluate, in comparison with the prescribing modules of 2 leading electronic health record prescribing systems, the efficiency, error rate, and satisfaction with a new computerized provider order entry prototype for the outpatient setting that allows clinicians to initiate prescribing using the indication.

DESIGN, SETTING, AND PARTICIPANTS

This quality improvement study used usability tests requiring internal medicine physicians, residents, and physician assistants to enter prescriptions electronically, including indication, for 8 clinical scenarios. The tool order assignments were randomized and prescribers were asked to use the prototype for 4 of the scenarios and their usual system for the other 4. Time on task, number of clicks, and order details were captured. User satisfaction was measured using posttask ratings and a validated system usability scale. The study participants practiced in 2 health systems' outpatient practices. Usability tests were conducted between April and October of 2017.

MAIN OUTCOMES AND MEASURES

Usability (efficiency, error rate, and satisfaction) of indications-based computerized provider order entry prototype vs the electronic prescribing interface of 2 electronic health record vendors.

RESULTS

Thirty-two participants (17 attending physicians, 13 residents, and 2 physician assistants) used the prototype to complete 256 usability test scenarios. The mean (SD) time on task was 1.78 (1.17) minutes. For the 20 participants who used vendor 1's system, it took a mean (SD) of 3.37 (1.90) minutes to complete a prescription, and for the 12 participants using vendor 2's system, it took a mean (SD) of 2.93 (1.52) minutes. Across all scenarios, when comparing number of clicks, for those participants using the prototype and vendor 1, there was a statistically significant difference from the mean (SD) number of clicks needed (18.39 [12.62] vs 46.50 [27.29]; difference, 28.11; 95% CI, 21.47-34.75; P < .001). For those using the prototype and vendor 2, there was also a statistically significant difference in number of clicks (20.10 [11.52] vs 38.25 [19.77]; difference, 18.14; 95% CI, 11.59-24.70; P < .001). A blinded review of the order details revealed medication errors (eg, drug-allergy interactions) in 38 of 128 prescribing sessions using a vendor system vs 7 of 128 with the prototype.

CONCLUSIONS AND RELEVANCE

Reengineering prescribing to start with the drug indication allowed indications to be captured in an easy and useful way, which may be associated with saved time and effort, reduced medication errors, and increased clinician satisfaction.

Electronic medical record implementation for a healthcare system caring for homeless people

Objective

Electronic medical record (EMR) implementation at centers caring for homeless people is constrained by limited resources and the increased disease burden of the patient population. Few informatics articles address this issue. This report describes Boston Health Care for the Homeless Program’s migration to new EMR software without loss of unique care elements and processes.

Materials and methods

Workflows for clinical and operational functions were analyzed and modeled, focusing particularly on resource constraints and comorbidities. Workflows were optimized, standardized, and validated before go-live by user groups who provided design input. Software tools were configured to support optimized workflows. Customization was minimal. Training used the optimized configuration in a live training environment allowing users to learn and use the software before go-live.

Results

Implementation was rapidly accomplished over 6 months. Productivity was reduced at most minimally over the initial 3 months. During the first full year, quality indicator levels were maintained. Keys to success were completing before go-live workflow analysis, workflow mapping, building of documentation templates, creation of screen shot guides, role-based phased training, and standardization of processes. Change management strategies were valuable. The early availability of a configured training environment was essential. With this methodology, the software tools were chosen and workflows optimized that addressed the challenges unique to caring for homeless people.

Conclusions

Successful implementation of an EMR to care for homeless people was achieved through detailed workflow analysis, optimizing and standardizing workflows, configuring software, and initiating training all well before go-live. This approach was particularly suitable for a homeless population.