The Development and Piloting of the Ambulatory Electronic Health Record Evaluation Tool: Lessons Learned

Electronic Health Record Usability, Satisfaction, and Burnout for Family Physicians

Importance

Electronic health record (EHR) work has been associated with decreased physician well-being. Understanding the association between EHR usability and physician satisfaction and burnout, and whether team and technology strategies moderate this association, is critical to informing efforts to address EHR-associated physician burnout.

Objectives

To measure family physician satisfaction with their EHR and EHR usability across functions and evaluate the association of EHR usability with satisfaction and burnout, as well as the moderating association of 4 team and technology EHR efficiency strategies.

Design, Setting, and Participants

This study uses data from a cross-sectional survey conducted from December 12, 2021, to October 17, 2022, of all family physicians seeking American Board of Family Medicine recertification in 2022.

Exposure

Physicians perceived EHR usability across 6 domains, as well as adoption of 4 EHR efficiency strategies: scribes, support from other staff, templated text, and voice recognition or transcription.

Main Outcomes and Measures

Physician EHR satisfaction and frequency of experiencing burnout measured with a single survey item ("I feel burned out from my work"), with answers ranging from "never" to "every day."

Results

Of the 2067 physicians (1246 [60.3%] younger than 50 years; 1051 men [50.9%]; and 1729 [86.0%] practicing in an urban area) who responded to the survey, 562 (27.2%) were very satisfied and 775 (37.5%) were somewhat satisfied, while 346 (16.7%) were somewhat dissatisfied and 198 (9.6%) were very dissatisfied with their EHR. Readability of information had the highest usability, with 543 physicians (26.3%) rating it as excellent, while usefulness of alerts had the lowest usability, with 262 physicians (12.7%) rating it as excellent. In multivariable models, good or excellent usability for entering data (β = 0.09 [95% CI, 0.05-0.14]; P < .001), alignment with workflow processes (β = 0.11 [95% CI, 0.06-0.16]; P < .001), ease of finding information (β = 0.14 [95% CI, 0.09-0.19]; P < .001), and usefulness of alerts (β = 0.11 [95% CI, 0.06-0.16]; P < .001) were associated with physicians being very satisfied with their EHR. In addition, being very satisfied with the EHR was associated with reduced frequency of burnout (β = -0.64 [95% CI, -1.06 to -0.22]; P < .001). In moderation analysis, only physicians with highly usable EHRs saw improvements in satisfaction from adopting efficiency strategies.

Conclusions and Relevance

In this survey study of physician EHR usability and satisfaction, approximately one-fourth of family physicians reported being very satisfied with their EHR, while another one-fourth reported being somewhat or very dissatisfied, a concerning finding amplified by the inverse association between EHR satisfaction and burnout. Electronic health record-based alerts had the lowest reported usability, suggesting EHR vendors should focus their efforts on improving alerts. Electronic health record efficiency strategies were broadly adopted, but only physicians with highly usable EHRs realized gains in EHR satisfaction from using these strategies, suggesting that EHR burden-reduction interventions are likely to have heterogenous associations across physicians with different EHRs.

Interprofessional Evaluation of a Medication Clinical Decision Support System Prior to Implementation

BACKGROUND

 Computerized physician order entry (CPOE) and clinical decision support systems (CDSS) are widespread due to increasing digitalization of hospitals. They can be associated with reduced medication errors and improved patient safety, but also with well-known risks (e.g., overalerting, nonadoption).

OBJECTIVES

 Therefore, we aimed to evaluate a commonly used CDSS containing Medication-Safety-Validators (e.g., drug-drug interactions), which can be locally activated or deactivated, to identify limitations and thereby potentially optimize the use of the CDSS in clinical routine.

METHODS

 Within the implementation process of Meona (commercial CPOE/CDSS) at a German University hospital, we conducted an interprofessional evaluation of the CDSS and its included Medication-Safety-Validators following a defined algorithm: (1) general evaluation, (2) systematic technical and content-related validation, (3) decision of activation or deactivation, and possibly (4) choosing the activation mode (interruptive or passive). We completed the in-depth evaluation for exemplarily chosen Medication-Safety-Validators. Moreover, we performed a survey among 12 German University hospitals using Meona to compare their configurations.

RESULTS

 Based on the evaluation, we deactivated 3 of 10 Medication-Safety-Validators due to technical or content-related limitations. For the seven activated Medication-Safety-Validators, we chose the interruptive option ["PUSH-(&PULL)-modus"] four times (4/7), and a new, on-demand option ["only-PULL-modus"] three times (3/7). The site-specific configuration (activation or deactivation) differed across all participating hospitals in the survey and led to varying medication safety alerts for identical patient cases.

CONCLUSION

 An interprofessional evaluation of CPOE and CDSS prior to implementation in clinical routine is crucial to detect limitations. This can contribute to a sustainable utilization and thereby possibly increase medication safety.

Electronic Medication Reconciliation Tools Aimed at Healthcare Professionals to Support Medication Reconciliation: a Systematic Review

The development of health information technology available and accessible to professionals is increasing in the last few years. However, a low number of electronic health tools included some kind of information about medication reconciliation. To identify all the electronic medication reconciliation tools aimed at healthcare professionals and summarize their main features, availability, and clinical impact on patient safety. A systematic review of studies that included a description of an electronic medication reconciliation tool (web-based or mobile app) aimed at healthcare professionals was conducted. The review protocol was registered with PROSPERO: registration number CRD42022366662, and followed PRISMA guidelines. The literature search was performed using four healthcare databases: PubMed, EMBASE, Cochrane Library, and Scopus with no language or publication date restrictions. We identified a total of 1227 articles, of which only 12 met the inclusion criteria.Through these articles,12 electronic tools were detected. Viewing and comparing different medication lists and grouping medications into multiple categories were some of the more recurring features of the tools. With respect to the clinical impact on patient safety, a reduction in adverse drug events or medication discrepancies was detected in up to four tools, but no significant differences in emergency room visits or hospital readmissions were found. 12 e-MedRec tools aimed at health professionals have been developed to date but none was designed as a mobile app. The main features that healthcare professionals requested to be included in e-MedRec tools were interoperability, "user-friendly" information, and integration with the ordering process.

How Safe are Outpatient Electronic Health Records? An Evaluation of Medication-Related Decision Support using the Ambulatory Electronic Health Record Evaluation Tool

BACKGROUND

The purpose of the Ambulatory Electronic Health Record (EHR) Evaluation Tool is to provide outpatient clinics with an assessment that they can use to measure the ability of the EHR system to detect and prevent common prescriber errors. The tool consists of a medication safety test and a medication reconciliation module.

OBJECTIVES

The goal of this study was to perform a broad evaluation of outpatient medication-related decision support using the Ambulatory EHR Evaluation Tool.

METHODS

We performed a cross-sectional study with 10 outpatient clinics using the Ambulatory EHR Evaluation Tool. For the medication safety test, clinics were provided test patients and associated medication test orders to enter in their EHR, where they recorded any advice or information they received. Once finished, clinics received an overall percentage score of unsafe orders detected and individual order category scores. For the medication reconciliation module, clinics were asked to electronically reconcile two medication lists, where modifications were made by adding and removing medications and changing the dosage of select medications.

RESULTS

For the medication safety test, the mean overall score was 57%, with the highest score being 70%, and the lowest score being 40%. Clinics performed well in the drug allergy (100%), drug dose daily (85%), and inappropriate medication combinations (74%) order categories. Order categories with the lowest performance were drug laboratory (10%) and drug monitoring (3%). Most clinics (90%) scored a 0% in at least one order category. For the medication reconciliation module, only one clinic (10%) could reconcile medication lists electronically; however, there was no clinical decision support available that checked for drug interactions.

CONCLUSION

We evaluated a sample of ambulatory practices around their medication-related decision support and found that advanced capabilities within these systems have yet to be widely implemented. The tool was practical to use and identified substantial opportunities for improvement in outpatient medication safety.

Medication Reconciliation during Transitions of Care Across Institutions: A Quantitative Analysis of Challenges and Opportunities

OBJECTIVE

Medication discrepancies between clinical systems may pose a patient safety hazard. In this paper, we identify challenges and quantify medication discrepancies across transitions of care.

METHODS

We used structured clinical data and free-text hospital discharge summaries to compare active medications' lists at four time points: preadmission (outpatient), at-admission (inpatient), at-discharge (inpatient), and postdischarge (outpatient). Medication lists were normalized to RxNorm. RxNorm identifiers were further processed using the RxNav API to identify the ingredient. The specific drugs and ingredients from inpatient and outpatient medication lists were compared.

RESULTS

Using RxNorm drugs, the median percentage intersection when comparing active medication lists within the same electronic health record system ranged between 94.1 and 100% indicating substantial overlap. Similarly, when using RxNorm ingredients the median percentage intersection was 94.1 to 100%. In contrast, the median percentage intersection when comparing active medication lists across EHR systems was significantly lower (RxNorm drugs: 6.1-7.1%; RxNorm ingredients: 29.4-35.0%) indicating that the active medication lists were significantly less similar (p < 0.05).Medication lists in the same EHR system are more similar to each other (fewer discrepancies) than medication lists in different EHR systems when comparing specific RxNorm drug and the more general RxNorm ingredients at transitions of care. Transitions of care that require interoperability between two EHR systems are associated with more discrepancies than transitions where medication changes are expected (e.g., at-admission vs. at-discharge). Challenges included lack of access to structured, standardized medication data across systems, and difficulty distinguishing medications from orderable supplies such as lancets and diabetic test strips.

CONCLUSION

Despite the challenges to medication normalization, there are opportunities to identify and assist with medication reconciliation across transitions of care between institutions.

System Configuration Evaluation for a Province-Wide Clinical Information System Using the eSafety Checklist

BACKGROUND

According to Digital Health Canada 2013 eSafety Guidelines, an estimated one-third of patient safety incidents following implementation of clinical information systems (CISs) are technology-related. An eSafety checklist was previously developed to improve CIS safety by providing a comprehensive listing of system-agnostic, evidence-based configuration recommendations.

OBJECTIVES

We sought to use the checklist to support safe initial configuration of a provincial system-wide CIS (Alberta, Canada), referred to as Connect Care.

METHODS

The checklist was applied to 13 Connect Care modules in three successive phases. First, the checklist was adapted to an abbreviated high-priority version. Second, demonstrations of each module were recorded. Finally, independent evaluation of each recording was conducted by two eSafety evaluators using the abbreviated eSafety checklist.

RESULTS

All modules achieved greater than 72% compliance, with an average of 84%. Overall, 273 opportunities for improvement were identified, with four major areas or themes emerging: (1) inconsistent date and time, (2) unclear patient identification, (3) ineffective alert system, and (4) insufficient decision support. These opportunities were forwarded to the appropriate build teams for review and implementation.

CONCLUSION

This work is the first to utilize the eSafety checklist in a real-world CIS, which will become one of the largest in Canada. The checklist has shown clinical applicability in identifying gaps in CIS configuration and should be considered for use in future and pre-existing CISs.

Orenstein E, Umontuen U, McNeill D, Kandaswamy S. Reducing Therapeutic Duplication in Inpatient Medication Orders

BACKGROUND

Therapeutic duplication, the presence of multiple agents prescribed for the same indication without clarification for when each should be used, can contribute to serious medical errors. Joint Commission standards require that orders contain clarifying information about when each order should be given. In our system, as needed (PRN) acetaminophen and ibuprofen orders are major contributors to therapeutic duplication.

OBJECTIVE

The objective of this study is to design and evaluate effectiveness of clinical decision support (CDS) to reduce therapeutic duplication with acetaminophen and ibuprofen orders.

METHODS

This study was done in a pediatric health system with three freestanding hospitals. We iteratively designed and implemented two CDS strategies aimed at reducing the therapeutic duplication with these agents: (1) interruptive alert prompting clinicians for clarifying PRN comments at order entry and (2) addition of discrete "first-line" and "second-line" PRN reasons to orders. Therapeutic duplications were measured by manual review of orders for 30-day periods before and after each intervention and 6 months later.

RESULTS

Therapeutic duplications decreased from 1,485 in the 30 days prior to the first alert implementation to 818 in the 30 days after but rose back to 1,208 in the 30 days prior to the second intervention. After discrete reasons were added to the order, therapeutic duplication decreased to 336 in the immediate 30 days and 6 months later remained at 277. Alerts firing rates decreased from 76.0 per 1,000 PRN acetaminophen or ibuprofen orders to 42.9 after the second intervention.

CONCLUSION

Interruptive alerts may reduce therapeutic duplication but are associated with high rates of user frustration and alert fatigue. Leveraging discrete PRN reasons for "first line" and "second line" produced a greater reduction in therapeutic duplication as well as fewer interruptive alerts and less manual entry for providers.