The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review

Recommendations to promote fairness and inclusion in biomedical AI research and clinical use

OBJECTIVE

Understanding and quantifying biases when designing and implementing actionable approaches to increase fairness and inclusion is critical for artificial intelligence (AI) in biomedical applications.

METHODS

In this Special Communication, we discuss how bias is introduced at different stages of the development and use of AI applications in biomedical sciences and health care. We describe various AI applications and their implications for fairness and inclusion in sections on 1) Bias in Data Source Landscapes, 2) Algorithmic Fairness, 3) Uncertainty in AI Predictions, 4) Explainable AI for Fairness and Equity, and 5) Sociological/Ethnographic Issues in Data and Results Representation.

RESULTS

We provide recommendations to address biases when developing and using AI in clinical applications.

CONCLUSION

These recommendations can be applied to informatics research and practice to foster more equitable and inclusive health care systems and research discoveries.

Event Analysis for Automated Estimation of Absent and Persistent Medication Alerts: Novel Methodology

Background

Event analysis is a promising approach to estimate the acceptance of medication alerts issued by computerized physician order entry (CPOE) systems with an integrated clinical decision support system (CDSS), particularly when alerts cannot be interactively confirmed in the CPOE-CDSS due to its system architecture. Medication documentation is then reviewed for documented evidence of alert acceptance, which can be a time-consuming process, especially when performed manually.

Objective

We present a new automated event analysis approach, which was applied to a large data set generated in a CPOE-CDSS with passive, noninterruptive alerts.

Methods

Medication and alert data generated over 3.5 months within the CPOE-CDSS at Heidelberg University Hospital were divided into 24-hour time intervals in which the alert display was correlated with associated prescription changes. Alerts were considered "persistent" if they were displayed in every consecutive 24-hour time interval due to a respective active prescription until patient discharge and were considered "absent" if they were no longer displayed during continuous prescriptions in the subsequent interval.

Results

Overall, 1670 patient cases with 11,428 alerts were analyzed. Alerts were displayed for a median of 3 (IQR 1-7) consecutive 24-hour time intervals, with the shortest alerts displayed for drug-allergy interactions and the longest alerts displayed for potentially inappropriate medication for the elderly (PIM). Among the total 11,428 alerts, 56.1% (n=6413) became absent, most commonly among alerts for drug-drug interactions (1915/2366, 80.9%) and least commonly among PIM alerts (199/499, 39.9%).

Conclusions

This new approach to estimate alert acceptance based on event analysis can be flexibly adapted to the automated evaluation of passive, noninterruptive alerts. This enables large data sets of longitudinal patient cases to be processed, allows for the derivation of the ratios of persistent and absent alerts, and facilitates the comparison and prospective monitoring of these alerts.

Is the European Crohn's and Colitis organisation (ECCO) e-guide an acceptable and feasible tool for increasing gastroenterologists' guideline adherence? A mixed methods evaluation

BACKGROUND AND AIMS

Management of inflammatory bowel disease is constantly evolving, increasing the importance for gastroenterologists to keep up to date with guidelines. Traditional implementation strategies have had only small positive impacts on clinical practice. eHealth strategies such as the European Crohn's and Colitis Organisation e-guide may be beneficial for clinician decision making in keeping with guidelines. The aim of this study was to evaluate the feasibility and acceptability of the e-guide.

METHODS

A mixed methods approach was used to evaluate feasibility and acceptability. Cognitive (think-aloud) interviews were conducted with Australian gastroenterologists while using the e-guide. Two clinical scenarios were developed to allow evaluation of various aspects of the e-guide. Content analysis was applied to the qualitative interview data and descriptive analysis to the quantitative and observational data.

RESULTS

Seventeen participants completed the study. Data saturation were reached. The ECCO e-guide was largely feasible and acceptable, as demonstrated by most clinical questions answered correctly, 87% reaching the answer within 3 min, and most feeling it was useful, would be beneficial to their practice and would use it again. Issues raised included difficulties with website navigation, layout of the e-guide and difficulties with access (network firewalls, paid subscription required).

CONCLUSIONS

The ECCO e-guide is largely acceptable and feasible for gastroenterologists to use. Aspects of the e-guide could be modified to improve user experience. This study highlights the importance of engaging end-users in the development and evaluation of clinician educational tools.

Evaluation of an Intensive Care Unit Sepsis Alert in Critically Ill Medical Patients

BACKGROUND

Sepsis alerts commonly used for intensive care unit (ICU) patients can lead to alert fatigue because these patients generally meet 1 or more of the criteria for systemic inflammatory response syndrome. To identify ICU patients at greatest risk for sepsis-related consequences, an ICU-specific sepsis alert was implemented.

OBJECTIVE

To evaluate an ICU sepsis alert based on modified criteria for systemic inflammatory response syndrome among critically ill medical patients.

METHODS

This retrospective evaluation was conducted at a comprehensive tertiary referral center. Patients included were at least 18 years old, were admitted to the critical care medicine service, and had at least 1 sepsis alert between January 1 and February 29, 2020. The sepsis alert identified patients meeting at least 2 modified systemic inflammatory response syndrome criteria (white blood cell count, <4000/μL or >12 000/μL; body temperature, <36 °C or >38.3 °C; heart rate, >110/min; and respiratory rate, >21/min), with at least 1 of the 2 criteria being white blood cell count or body temperature.

RESULTS

For 128 alerts evaluated, the positive predictive value was 72%. Of 713 patients who were admitted to the critical care medicine service and did not have a sepsis alert, 7 received a sepsis diagnosis. The ICU sepsis alert had a negative predictive value of 99%, sensitivity of 92.9%, and specificity of 95.1%.

CONCLUSIONS

An ICU sepsis alert using modified systemic inflammatory response syndrome criteria was effective for identifying sepsis in critically ill medical patients.

Interprofessional Evaluation of a Medication Clinical Decision Support System Prior to Implementation

BACKGROUND

 Computerized physician order entry (CPOE) and clinical decision support systems (CDSS) are widespread due to increasing digitalization of hospitals. They can be associated with reduced medication errors and improved patient safety, but also with well-known risks (e.g., overalerting, nonadoption).

OBJECTIVES

 Therefore, we aimed to evaluate a commonly used CDSS containing Medication-Safety-Validators (e.g., drug-drug interactions), which can be locally activated or deactivated, to identify limitations and thereby potentially optimize the use of the CDSS in clinical routine.

METHODS

 Within the implementation process of Meona (commercial CPOE/CDSS) at a German University hospital, we conducted an interprofessional evaluation of the CDSS and its included Medication-Safety-Validators following a defined algorithm: (1) general evaluation, (2) systematic technical and content-related validation, (3) decision of activation or deactivation, and possibly (4) choosing the activation mode (interruptive or passive). We completed the in-depth evaluation for exemplarily chosen Medication-Safety-Validators. Moreover, we performed a survey among 12 German University hospitals using Meona to compare their configurations.

RESULTS

 Based on the evaluation, we deactivated 3 of 10 Medication-Safety-Validators due to technical or content-related limitations. For the seven activated Medication-Safety-Validators, we chose the interruptive option ["PUSH-(&PULL)-modus"] four times (4/7), and a new, on-demand option ["only-PULL-modus"] three times (3/7). The site-specific configuration (activation or deactivation) differed across all participating hospitals in the survey and led to varying medication safety alerts for identical patient cases.

CONCLUSION

 An interprofessional evaluation of CPOE and CDSS prior to implementation in clinical routine is crucial to detect limitations. This can contribute to a sustainable utilization and thereby possibly increase medication safety.

Development and implementation of medication-related clinical rules for obstetrics, gynaecology, and paediatric outpatients

OBJECTIVES

Prescription errors can cause serious adverse drug events. Clinical decision support systems prevent prescription errors; however, real-time clinical rules in obstetrics, gynaecology, and paediatric outpatients remain unexplored. We evaluated the effects of localised, real-time clinical rules on alert rates and acceptance rates compared with manual prescription review.

METHODS

We developed real-time clinical rules that incorporate information systems to obtain characteristic information and laboratory values. We conducted a retrospective cohort study to compare the alert and recommendation acceptance rates of all prescription error types before and after clinical rule implementation in obstetrics, gynaecology, and paediatrics. Clinical rules, prescription error types, and alerts were determined by a prescribing review committee comprising physicians, pharmacists, nurses, and administrators. The difference in alert and acceptance rates between the groups was analysed using relative risk.

RESULTS

The number of alerts increased after clinical rules implementation; the number of on-duty pharmacists for review decreased from 10 to 2. Compared with those with manual review, the alert rates for paediatrics and obstetrics and gynaecology increased with the clinical rules by 3.97- and 11.26-fold, respectively, and the alert rates for drug-drug interactions (DDIs) and combined medication errors in obstetrics and gynaecology increased with the clinical rules by 26.10- and 26.54-fold, respectively. In paediatrics, the alert rate for all prescription error types was higher with the clinical rules review than with the manual review; the alert rates for DDI, dosage, and combination medication errors were significantly different between the clinical rules and the manual review. However, there was no difference in the recommendation acceptance rate between the manual review and the clinical rules.

CONCLUSIONS

Clinical rules can identify prescription errors that manual review cannot detect and ensure real-time review efficiency in high-volume outpatient prescription settings. The high acceptance rate and modification of prescriptions may be relevant to highly customised and localised clinical rules.

The Learning Electronic Health Record

Electronic medical records (EMRs) constitute the electronic version of all medical information included in a patient's paper chart. The electronic health record (EHR) technology has witnessed massive expansion in developed countries and to a lesser extent in underresourced countries during the last 2 decades. We will review factors leading to this expansion, how the emergence of EHRs is affecting several health-care stakeholders; some of the growing pains associated with EHRs with a particular emphasis on the delivery of care to the critically ill; and ongoing developments on the path to improve the quality of research, health-care delivery, and stakeholder satisfaction.

Inpatient EHR User Experience and Hospital EHR Safety Performance

IMPORTANCE

Despite the broad adoption and optimization of electronic health record (EHR) systems across the continuum of care, serious usability and safety problems persist.

OBJECTIVE

To assess whether EHR safety performance is associated with EHR frontline user experience in a national sample of hospitals.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study included all US adult hospitals that used the National Quality Forum Leapfrog Health IT Safety Measure and also used the ARCH Collaborative EHR User experience survey from January 1, 2017, to January 1, 2019. Data analysis was performed from September 2020 to November 2022.

MAIN OUTCOMES AND MEASURES

The primary outcomes were hospital performance on the Leapfrog Health IT Safety measure (overall and 10 subcomponents) and the ARCH collaborative frontline user experience scores (overall and 8 subcomponents). Ordinary least squares models with survey responses clustered by hospital were used to assess associations between the overall measures and their subcomponents.

RESULTS

There were 112 hospitals and 5689 frontline user surveys included in the study. Hospitals scored a mean of 0.673 (range, 0.297-0.973) on the Leapfrog Health IT safety measure; the mean ARCH EHR user experience score was 3.377 (range, 1 [best] to 5 [worst]). The adjusted β coefficient between the overall safety score and overall user experience score was 0.011 (95% CI, 0.006-0.016). The ARCH overall score was also significantly associated with 10 subcategory scores of the Leapfrog Health IT safety score, and the overall Leapfrog score was associated with the 8 subcategory scores of the ARCH user experience score.

CONCLUSIONS AND RELEVANCE

This cross-sectional study found a positive association between frontline user-rated EHR usability and EHR safety performance. This finding suggests that improving EHR usability, which is a current well-known pain point for EHR users, could have direct benefits in terms of improved EHR safety.

An Evaluation of Drug-Drug Interaction Alerts Produced by Clinical Decision Support Systems in a Tertiary Hospital

Introduction Drug-drug interactions (DDIs) have the potential to harm patients. Hence, DDI alerts are meant to prevent harm; as a result, their usefulness is reduced when most alerts displayed to providers are ignored. This study aims to explore the rates and reasons for overriding alerts of DDI. Methods This is a retrospective study of DDI alert overrides that occurred between January 2020 and December 2020 within the inpatient medical records at a tertiary hospital, Medina City, Kingdom of Saudi Arabia. Results A total of 7,098 DDI alerts were generated from inpatient settings, of which 6,551(92.2%) were overridden by the physicians at the point of prescribing. "Will Monitor as Recommended" (33%) was the most common reason for the override, followed by 'Will Adjust the Dose as Recommended (27.1%)," "The Patient Has Already Tolerated the Combination" (25.7%), and "No Overridden Reason Selected" (13.0%). Discussion The DDI alert overriding is still high and is comparable to other studies. However, this study reveals that physicians are ready to deal with the consequences of around 58% of DDI alerts. Additionally, 13% of physicians were not willing to report the reason for overriding. This indicates an urgent need to review and restructure the DDI alert system. Conclusion The DDI alert override rates are high, and this is undesirable. It is recommended to revise the DDI alert system. Future studies should dig deep for real reasons for overriding and seek inputs from all stakeholders, including developing actionable metrics to track and monitor DDI alerting system.

Digitalisation of the drug prescribing process in Swiss hospitals - results of a survey

BACKGROUND

The state of digitalisation in the healthcare sector in Switzerland is lagging, even as the national electronic health record (EHR) is being gradually implemented. Little is known about the implementation of electronic prescribing systems, their auxiliary features or drug datasets in Swiss hospitals.The aim of this study was to understand which electronic systems are implemented to support doctors in Swiss hospitals during the medication prescribing process.

METHODS

The survey was sent in spring 2021 to the chief pharmacists of the main Swiss hospitals. The survey focused on the introduction of the EHR, the clinical information system (CIS) and its prescribing module, as well as drug information data and clinical decision support systems (CDSS).

RESULTS

The response rate was 98% (58/59 hospitals). Almost half of the hospitals (47%) were connected to the national EHR, almost all hospitals (86%) used a CIS and a vast majority of the hospitals (84%) had implemented electronic prescribing systems in their CIS. 10 years ago, around 63% of hospitals used a CIS and 40% were equipped with an electronic prescribing system. Today, CDSS of any kind were implemented in 50% of the hospitals, predominantly for drug-drug interactions. Drug master data were maintained in most hospitals (76%) via an automated interface, but mostly supplemented manually. Clinical drug information data were maintained in 74% of hospitals. In 67% of hospitals, datasets were imported via an automated interface.

CONCLUSIONS

The digitalisation of the medical prescribing process in Swiss hospitals has progressed over the last decade. Drug prescriptions via electronic prescribing systems were introduced in most hospitals. However, this survey suggests that the current use of CDSS is far from exhausted, and that clinical drug information data could be maintained more efficiently. Optimising electronic support for healthcare professionals during the prescribing process still has considerable potential.

Qualitative Evaluation of an Artificial Intelligence–Based Clinical Decision Support System to Guide Rhythm Management of Atrial Fibrillation: Survey Study

Background

Despite the numerous studies evaluating various rhythm control strategies for atrial fibrillation (AF), determination of the optimal strategy in a single patient is often based on trial and error, with no one-size-fits-all approach based on international guidelines/recommendations. The decision, therefore, remains personal and lends itself well to help from a clinical decision support system, specifically one guided by artificial intelligence (AI). QRhythm utilizes a 2-stage machine learning (ML) model to identify the optimal rhythm management strategy in a given patient based on a set of clinical factors, in which the model first uses supervised learning to predict the actions of an expert clinician and identifies the best strategy through reinforcement learning to obtain the best clinical outcome—a composite of symptomatic recurrence, hospitalization, and stroke.

Objective

We qualitatively evaluated a novel, AI-based, clinical decision support system (CDSS) for AF rhythm management, called QRhythm, which uses both supervised and reinforcement learning to recommend either a rate control or one of 3 types of rhythm control strategies—external cardioversion, antiarrhythmic medication, or ablation—based on individual patient characteristics.

Methods

Thirty-three clinicians, including cardiology attendings and fellows and internal medicine attendings and residents, performed an assessment of QRhythm, followed by a survey to assess relative comfort with automated CDSS in rhythm management and to examine areas for future development.

Results

The 33 providers were surveyed with training levels ranging from resident to fellow to attending. Of the characteristics of the app surveyed, safety was most important to providers, with an average importance rating of 4.7 out of 5 (SD 0.72). This priority was followed by clinical integrity (a desire for the advice provided to make clinical sense; importance rating 4.5, SD 0.9), backward interpretability (transparency in the population used to create the algorithm; importance rating 4.3, SD 0.65), transparency of the algorithm (reasoning underlying the decisions made; importance rating 4.3, SD 0.88), and provider autonomy (the ability to challenge the decisions made by the model; importance rating 3.85, SD 0.83). Providers who used the app ranked the integrity of recommendations as their highest concern with ongoing clinical use of the model, followed by efficacy of the application and patient data security. Trust in the app varied; 1 (17%) provider responded that they somewhat disagreed with the statement, “I trust the recommendations provided by the QRhythm app,” 2 (33%) providers responded with neutrality to the statement, and 3 (50%) somewhat agreed with the statement.

Conclusions

Safety of ML applications was the highest priority of the providers surveyed, and trust of such models remains varied. Widespread clinical acceptance of ML in health care is dependent on how much providers trust the algorithms. Building this trust involves ensuring transparency and interpretability of the model.

Application of Prescription Reviews for Traditional Chinese Medicine to Improve Medical Disputes and Patient Satisfaction

Objective

Medical disputes and patient satisfaction are related to inappropriate prescribing practices. We aim to investigate the clinical application of prescription reviews for traditional Chinese medicine (TCM).

Method

TCM prescriptions performed prescription reviews in 372 patients from the year 2019 to 2020 were set as the observation group and those from the year 2017 to 2018 without prescription reviews as the control group (n = 341). According to the Criteria for Assessing Prescription Quality in Chinese Hospitals (CAPQCH) items, “Irrational” and “Rational” TCM prescriptions were determined mainly based on the following category: nonstandard prescriptions, inappropriate prescriptions, and hypernormal prescriptions. The incidence of medical disputes and the degree of patient satisfaction were compared between the two groups.

Result

No difference was found in age and gender between the control group and the observation group. The number of irrational TCM prescriptions from the year 2017 to 2020 was 6, 8, 2, and 3, respectively, with the percentage of 3.725%, 4.480%, 1.201%, and 1.446%. The irrational rate in the observation group (1.344%) was significantly lower than that in the control group (4.106%). Specifically, a higher rate of nonstandard prescriptions was revealed in the control group as compared with the observation group. Moreover, a reduced incidence of medical disputes was revealed in the observation group relative to the control group accompanying with the increased degree of patient satisfaction.

Conclusion

Prescription reviews have high application value in the management of Chinese pharmacies, which can improve the rationality of prescriptions, increase patient satisfaction, and reduce medical disputes.

Impact of Computerized Provider Order Entry on Chemotherapy Medication Errors: A Systematic Review

Context: Chemotherapy errors are considered the second most common cause of fatal medication errors (ME). Currently, computerized provider order entry (CPOE) is increasingly used to prevent or decrease ME and improve the safety of the medication process. Objectives: This study was conducted to systematically review the impacts of CPOE on the incidence of chemotherapy ME, the severity of errors, and adverse drug events (ADEs) in cancer care units. Data Sources: The literature search was conducted, using 5 databases of PubMed, EMBASE, Scopus, Web of Science, and ScienceDirect between 2000 and 2020. Search terms included keywords and MESH terms related to CPOE, ME, chemotherapy, and cancer care unit. Study Selection: Articles were included in this research if they investigated the CPOE system, reported ME, and were carried out in the oncology department. Non-English papers, duplications, review studies, and conference papers were excluded. Data Extraction: The selected papers were read repeatedly and related papers were extracted. All eligible articles were qualitatively evaluated with a tool provided by Downs. The extracted information included the author’s name, year of publication, study location, type of study, study objectives, and main findings. Results: A total of 829 studies were retrieved. Fourteen articles met the inclusion criteria. Ten studies (71%) reported the impact of CPOE on chemotherapy ME in comparison with the paper-based ordering method. In 4 studies (29%), researchers developed a CPOE for the oncology department, and the system was, then, assessed concerning user experience, safety challenges as well as the effects of CPOE on ME. Nine articles (64%) reported the impact of the CPOE system on ME only in the prescribing phase, and 5 studies (36%) examined ME in all phases of the chemotherapy process. Five studies (36%) reported the impact of the CPOE system on ADEs and the severity of errors. Conclusions: Implementing CPOE is associated with a significant reduction in ME in all phases of the chemotherapy process. However, the CPOE does not prevent all MEs and may cause new errors. The rigorous analysis of the chemotherapy process and considering the designing principles could help develop the CPOE systems and minimize ME.

Long-term impact of a geriatric prescribing context

BACKGROUND

The medication-related death of a hospitalized older adult elucidated the inappropriateness of medication default doses in our electronic health record (EHR) for older adults. In response, we created and implemented the Geriatric Prescribing Context (GPC), an EHR-based set of age-specific dose and frequency defaults for patients 75 years and older, in July 2017. Inpatient medication orders aligned with GPC defaults and showed significant dose decreases at one year for nine of ten most commonly used medications. This follow-up investigation examined GPC alignment of dose and frequency over the 42-month time period after its implementation.

METHODS

Order data for the ten most commonly used medications at OHSU Hospital were collected retrospectively from July 2016 through December 2020. We used Statistical Process Control charts to assess the proportion of medication orders aligning with the GPC's recommendations. Signals of special cause were evaluated to identify time periods when shifts in process averages likely occurred and suspected shifts were assessed using binomial proportion tests. We used RStudio (RStudio, Inc., version 1.2.5001) and Microsoft Excel (2016) to perform statistical analyses and control charts, respectively.

RESULTS

The preimplementation phase of all medications displayed no special causes. After significant initial improvement in 2017, control charts revealed three different patterns of performance. Eight medications maintained the initial improvement with one medication displaying a second significant improvement at a later date. Two medications showed a subsequent decline in performance not statistically different from baseline. Overall, eight of the ten medications were prescribed at more age-friendly doses and frequencies compared to baseline after 42 months.

CONCLUSIONS

The GPC is an effective method to support safer prescribing for hospitalized older patients, but long-term impacts may be medication-specific. Further investigation is needed to ensure appropriate prescribing across drug classes and understand the GPC's impact on patient outcomes like adverse drug events.

Effect of Computerized Physician Order Entry and Clinical Decision Support System on Adverse Drug Events Prevention in the Emergency Department: A Systematic Review

Objective: An adverse drug event (ADE) is an injury resulting from a medical intervention related to a drug. The emergency department (ED) is a ward vulnerable to more ADEs because of overcrowding. Information technologies such as computerized physician order entry (CPOE) and clinical decision support system (CDSS) may decrease the occurrence of ADEs. This study aims to review research that reported the evaluation of the effectiveness of CPOE and CDSS on lowering the occurrence of ADEs in the ED. Data Sources: PubMed, EMBASE, and Web of Science databases were used to find studies published from 2003 to 2018. The search was conducted in November 2018. Study Selection and Data Extraction: The search resulted in 1700 retrieved articles. After applying inclusion and exclusion criteria, 11 articles were included. Data on the date, country, type of system, medication process stages, study design, participants, sample size, and outcomes were extracted. Data Synthesis: Results showed that CPOE and CDSS may prevent ADEs in the ED through significantly decreasing the rate of errors, ADEs, excessive dose, and inappropriate prescribing (in 54.5% of articles); furthermore, CPOE and CDSS may significantly increase the rate of appropriate prescribing and dosing in compliance with established guidelines (45.5% of articles). Conclusion: This study revealed that the use of CPOE and CDSS can lower the occurrence of ADEs in the ED; however, further randomized controlled trials are needed to address the effect of a CDSS, with basic or advanced features, on the occurrence of ADEs in the ED.

Renal medication-related clinical decision support (CDS) alerts and overrides in the inpatient setting following implementation of a commercial electronic health record: implications for designing more effective alerts

Objective

To assess the appropriateness of medication-related clinical decision support (CDS) alerts associated with renal insufficiency and the potential/actual harm from overriding the alerts.

Materials and Methods

Override rate frequency was recorded for all inpatients who had a renal CDS alert trigger between 05/2017 and 04/2018. Two random samples of 300 for each of 2 types of medication-related CDS alerts associated with renal insufficiency—“dose change” and “avoid medication”—were evaluated by 2 independent reviewers using predetermined criteria for appropriateness of alert trigger, appropriateness of override, and patient harm.

Results

We identified 37 100 “dose change” and 5095 “avoid medication” alerts in the population evaluated, and 100% of each were overridden. Dose change triggers were classified as 12.5% appropriate and overrides of these alerts classified as 90.5% appropriate. Avoid medication triggers were classified as 29.6% appropriate and overrides 76.5% appropriate. We identified 5 adverse drug events, and, of these, 4 of the 5 were due to inappropriately overridden alerts.

Conclusion

Alerts were nearly always presented inappropriately and were all overridden during the 1-year period studied. Alert fatigue resulting from receiving too many poor-quality alerts may result in failure to recognize errors that could lead to patient harm. Although medication-related CDS alerts associated with renal insufficiency had previously been found to be the most clinically beneficial alerts in a legacy system, in this system they were ineffective. These findings underscore the need for improvements in alert design, implementation, and monitoring of alert performance to make alerts more patient-specific and clinically appropriate.

How effective are electronic medication systems in reducing medication error rates and associated harm among hospital inpatients? A systematic review and meta-analysis

OBJECTIVE

To conduct a systematic review and meta-analysis to assess: 1) changes in medication error rates and associated patient harm following electronic medication system (EMS) implementation; and 2) evidence of system-related medication errors facilitated by the use of an EMS.

MATERIALS AND METHODS

We searched Medline, Scopus, Embase, and CINAHL for studies published between January 2005 and March 2019, comparing medication errors rates with or without assessments of related harm (actual or potential) before and after EMS implementation. EMS was defined as a computer-based system enabling the prescribing, supply, and/or administration of medicines. Study quality was assessed.

RESULTS

There was substantial heterogeneity in outcomes of the 18 included studies. Only 2 were strong quality. Meta-analysis of 5 studies reporting change in actual harm post-EMS showed no reduced risk (RR: 1.22, 95% CI: 0.18-8.38, P = .8) and meta-analysis of 3 studies reporting change in administration errors found a significant reduction in error rates (RR: 0.77, 95% CI: 0.72-0.83, P = .004). Of 10 studies of prescribing error rates, 9 reported a reduction but variable denominators precluded meta-analysis. Twelve studies provided specific examples of system-related medication errors; 5 quantified their occurrence.

DISCUSSION AND CONCLUSION

Despite the wide-scale adoption of EMS in hospitals around the world, the quality of evidence about their effectiveness in medication error and associated harm reduction is variable. Some confidence can be placed in the ability of systems to reduce prescribing error rates. However, much is still unknown about mechanisms which may be most effective in improving medication safety and design features which facilitate new error risks.

Identification of Design Criteria to Improve Patient Care in Electronic Health Record Downtime

OBJECTIVE

Design criteria specifications (needs, obstacles, and context-of-use considerations) for continuing safe and efficient patient care activities during downtime were identified by using phenomenological analysis.

METHODS

Interview transcripts from medical personnel who had experience with downtime incidents were examined using a phenomenological approach. This process allowed for the identification of design criteria for performing downtime patient care activities.

RESULTS

A substantial variation in criteria was found from participants in different roles. The differences suggest opportunities to address downtime that may require attention to individual roles.

CONCLUSIONS

Workload distribution and communication are significant issues in patient care during downtime. There may not be an equal work distribution, leading to an increased workload for some personnel during downtime. Phenomenological analysis was completed after participants were interviewed, indicating it is a viable post hoc approach. Some downtime criteria were identified as potential guidelines for the development of better downtime contingency plans.

Clinical decision support systems-based interventions to improve medication outcomes: A systematic literature review on features and effects

Background: Clinical decision support systems (CDSSs) interventions were used to improve the life quality and safety in patients and also to improve practitioner performance, especially in the field of medication. Therefore, the aim of the paper was to summarize the available evidence on the impact, outcomes and significant factors on the implementation of CDSS in the field of medicine. Methods: This study is a systematic literature review. PubMed, Cochrane Library, Web of Science, Scopus, EMBASE, and ProQuest were investigated by 15 February 2017. The inclusion requirements were met by 98 papers, from which 13 had described important factors in the implementation of CDSS, and 86 were medicated-related. We categorized the system in terms of its correlation with medication in which a system was implemented, and our intended results were examined. In this study, the process outcomes (such as; prescription, drug-drug interaction, drug adherence, etc.), patient outcomes, and significant factors affecting the implementation of CDSS were reviewed. Results: We found evidence that the use of medication-related CDSS improves clinical outcomes. Also, significant results were obtained regarding the reduction of prescription errors, and the improvement in quality and safety of medication prescribed. Conclusion: The results of this study show that, although computer systems such as CDSS may cause errors, in most cases, it has helped to improve prescribing, reduce side effects and drug interactions, and improve patient safety. Although these systems have improved the performance of practitioners and processes, there has not been much research on the impact of these systems on patient outcomes.

Comparison of Medication Alerts from Two Commercial Applications in the USA

Introduction

Medication organizations across the USA have adopted electronic health records, and one of the most anticipated benefits of these was improved medication safety, but alert fatigue has been a major issue.

Objective

We compared the appropriateness of medication-related clinical decision support alerts triggered by two commercial applications: EPIC and Seegnal’s platform.

Methods

This was a retrospective comparison of two commercial applications. We provided Seegnal with deidentified inpatient, outpatient, and inpatient genetic electronic medical record (EMR)-extracted datasets for 657, 2731, and 413 patients, respectively. Seegnal then provided the alerts that would have triggered, which we compared with those triggered by EPIC in clinical care. A random sample of the alerts triggered were reviewed for appropriateness, and the positive predictive value (PPV) and negative predictive value (NPV) were calculated. We also reviewed all the inpatient and outpatient charts for patients within our cohort who were receiving ten or more concomitant medications with alerts we found to be appropriate to assess whether any adverse events had occurred and whether Seegnal’s platform could have prevented them.

Results

Results from EPIC and the Seegnal platform were compared based on alert load, PPV, NPV, and potential adverse events. Overall, compared with EPIC, the Seegnal platform triggered fewer alerts in the inpatient (1697 vs. 27,540), outpatient (2341 vs. 35,134), and inpatient genetic (1493 vs. 20,975) cohorts. The Seegnal platform had higher specificity in the inpatient (99 vs. 0.3%; p < 0.0001), outpatient (99 vs. 0.3%; p < 0.0001), and inpatient genetic (97.9 vs. 1.2%; p < 0.0001) groups and higher sensitivity in the inpatient (100 vs. 68.8%; p < 0.0001) and outpatient (88.6 vs.78.3%; p < 0.0001) groups but not in the inpatient genetic cohort (81 vs. 78.5%; p = 0.11). We identified 16 adverse events that occurred in the inpatient setting, 11 (69%) of which potentially could have been prevented with the Seegnal platform.

Conclusions

Overall, the Seegnal platform triggered 94% fewer alerts than EPIC in the inpatient setting and 93% fewer in the outpatient setting, with much higher sensitivity and specificity. This application could substantially reduce alert fatigue and improve medication safety at the same time.

Supplementary Information

The online version of this article (10.1007/s40264-021-01048-0) contains supplementary material, which is available to authorized users.

A novel method to retrieve alerts from a homegrown Computerized Physician Order Entry (CPOE) system of an academic medical center: Comprehensive alert characteristic analysis

Background

The collection and analysis of alert logs are necessary for hospital administrators to understand the types and distribution of alert categories within the organization and reduce alert fatigue. However, this is not readily available in most homegrown Computerized Physician Order Entry (CPOE) systems.

Objective

To present a novel method that can collect alert information from a homegrown CPOE system (at an academic medical center in Taiwan) and conduct a comprehensive analysis of the number of alerts triggered and alert characteristics.

Methods

An alert log collector was developed using the Golang programming language and was implemented to collect all triggered interruptive alerts from a homegrown CPOE system of a 726-bed academic medical center from November 2017 to June 2018. Two physicians categorized the alerts from the log collector as either clinical or non-clinical (administrative).

Results

Overall, 1,625,341 interruptive alerts were collected and classified into 1,474 different categories based on message content. The sum of the top 20, 50, and 100 categories of most frequently triggered alerts accounted for approximately 80, 90 and 97 percent of the total triggered alerts, respectively. Among alerts from the 100 most frequently triggered categories, 1,266,818 (80.2%) were administrative and 312,593 (19.8%) were clinical alerts.

Conclusion

We have successfully developed an alert log collector that can serve as an extended function to retrieve alerts from a homegrown CPOE system. The insight generated from the present study could also potentially bring value to hospital system designers and hospital administrators when redesigning their CPOE system.

Nurses' perception of teamwork and its relationship with the occurrence and reporting of adverse events: A questionnaire survey in teaching hospitals

AIMS

To identify the levels of teamwork and its relationship with the occurrence and reporting of adverse events among Iranian nurses.

BACKGROUND

Strengthening teamwork is emphasized worldwide for enhancing quality care and patient safety.

METHODS

This study applied a cross-sectional survey design. A total of 327 Iranian nurses from eight teaching hospitals participated in a self-administered survey using simple random sampling. The Teamwork Perceptions Questionnaire was used to measure the teamwork. The frequency of occurrence and reporting of adverse events were measured with two questions. Data were analysed using descriptive analyses, independent t tests and logistic regression analysis.

RESULTS

The mean teamwork score was 3.81 out of 5. Among the nurses, 48.0% had experienced adverse events in the past 6 months and 79.8% reported having an appropriate performance in adverse events reporting. Teamwork was significantly associated with lower occurrences of adverse events and better adverse events reporting. Specifically, nurses with higher situation monitoring (odds ratio (OR) = 0.47), mutual support (OR = 3.18) and team leadership (OR = 2.09) scores were more likely to report adverse events. Nurses with higher situation monitoring scores were less likely to experience the occurrence of adverse events (OR = 0.38).

CONCLUSIONS

Nurses' perception of teamwork was moderate to high. Teamwork was associated with the occurrence and reporting of adverse events. Further study is needed to identify the effects of teamwork training on the learning outcomes, including teamwork, occurrence and the reporting of adverse events among nurses.

IMPLICATIOS FOR NURSING MANAGEMENT

Nursing managers should consider multiple educational strategies including structured teamwork training to improve staff nurses' teamwork competency. Administrative initiatives and quality improvement projects are needed to increase nurses' performance in the reporting of adverse events through an accreditation process.

Rationale and design of the Novel Uses of adaptive Designs to Guide provider Engagement in Electronic Health Records (NUDGE-EHR) pragmatic adaptive randomized trial: a trial protocol

BACKGROUND

The prescribing of high-risk medications to older adults remains extremely common and results in potentially avoidable health consequences. Efforts to reduce prescribing have had limited success, in part because they have been sub-optimally timed, poorly designed, or not provided actionable information. Electronic health record (EHR)-based tools are commonly used but have had limited application in facilitating deprescribing in older adults. The objective is to determine whether designing EHR tools using behavioral science principles reduces inappropriate prescribing and clinical outcomes in older adults.

METHODS

The Novel Uses of Designs to Guide provider Engagement in Electronic Health Records (NUDGE-EHR) project uses a two-stage, 16-arm adaptive randomized pragmatic trial with a "pick-the-winner" design to identify the most effective of many potential EHR tools among primary care providers and their patients ≥ 65 years chronically using benzodiazepines, sedative hypnotic ("Z-drugs"), or anticholinergics in a large integrated delivery system. In stage 1, we randomized providers and their patients to usual care (n = 81 providers) or one of 15 EHR tools (n = 8 providers per arm) designed using behavioral principles including salience, choice architecture, or defaulting. After 6 months of follow-up, we will rank order the arms based upon their impact on the trial's primary outcome (for both stages): reduction in inappropriate prescribing (via discontinuation or tapering). In stage 2, we will randomize (a) stage 1 usual care providers in a 1:1 ratio to one of the up to 5 most promising stage 1 interventions or continue usual care and (b) stage 1 providers in the unselected arms in a 1:1 ratio to one of the 5 most promising interventions or usual care. Secondary and tertiary outcomes include quantities of medication prescribed and utilized and clinically significant adverse outcomes.

DISCUSSION

Stage 1 launched in October 2020. We plan to complete stage 2 follow-up in December 2021. These results will advance understanding about how behavioral science can optimize EHR decision support to improve prescribing and health outcomes. Adaptive trials have rarely been used in implementation science, so these findings also provide insight into how trials in this field could be more efficiently conducted.

TRIAL REGISTRATION

Clinicaltrials.gov ( NCT04284553 , registered: February 26, 2020).

Integrating the Practical Robust Implementation and Sustainability Model With Best Practices in Clinical Decision Support Design: Implementation Science Approach

Background

Clinical decision support (CDS) design best practices are intended to provide a narrative representation of factors that influence the success of CDS tools. However, they provide incomplete direction on evidence-based implementation principles.

Objective

This study aims to describe an integrated approach toward applying an existing implementation science (IS) framework with CDS design best practices to improve the effectiveness, sustainability, and reproducibility of CDS implementations.

Methods

We selected the Practical Robust Implementation and Sustainability Model (PRISM) IS framework. We identified areas where PRISM and CDS design best practices complemented each other and defined methods to address each. Lessons learned from applying these methods were then used to further refine the integrated approach.

Results

Our integrated approach to applying PRISM with CDS design best practices consists of 5 key phases that iteratively interact and inform each other: multilevel stakeholder engagement, designing the CDS, design and usability testing, thoughtful deployment, and performance evaluation and maintenance. The approach is led by a dedicated implementation team that includes clinical informatics and analyst builder expertise.

Conclusions

Integrating PRISM with CDS design best practices extends user-centered design and accounts for the multilevel, interacting, and dynamic factors that influence CDS implementation in health care. Integrating PRISM with CDS design best practices synthesizes the many known contextual factors that can influence the success of CDS tools, thereby enhancing the reproducibility and sustainability of CDS implementations. Others can adapt this approach to their situation to maximize and sustain CDS implementation success.

Adoption rates of electronic health records in Turkish Hospitals and the relation with hospital sizes

BACKGROUND

Nation-wide adoption of electronic health records (EHRs) in hospitals has become a Turkish policy priority in recognition of their benefits in maintaining the overall quality of clinical care. The electronic medical record maturity model (EMRAM) is a widely used survey tool developed by the Healthcare Information and Management Systems Society (HIMSS) to measure the rate of adoption of EHR functions in a hospital or a secondary care setting. Turkey completed many standardizations and infrastructural improvement initiatives in the health information technology (IT) domain during the first phase of the Health Transformation Program between 2003 and 2017. Like the United States of America (USA), the Turkish Ministry of Health (MoH) applied a bottom-up approach to adopting EHRs in state hospitals. This study aims to measure adoption rates and levels of EHR use in state hospitals in Turkey and investigate any relationship between adoption and use and hospital size.

METHODS

EMRAM surveys were completed by 600 (68.9%) state hospitals in Turkey between 2014 and 2017. The availability and prevalence of medical information systems and EHR functions and their use were measured. The association between hospital size and the availability/prevalence of EHR functions was also calculated.

RESULTS

We found that 63.1% of all hospitals in Turkey have at least basic EHR functions, and 36% have comprehensive EHR functions, which compares favourably to the results of Korean hospitals in 2017, but unfavorably to the results of US hospitals in 2015 and 2017. Our findings suggest that smaller hospitals are better at adopting certain EHR functions than larger hospitals.

CONCLUSION

Measuring the overall adoption rates of EHR functions is an emerging approach and a beneficial tool for the strategic management of countries. This study is the first one covering all state hospitals in a country using EMRAM. The bottom-up approach to adopting EHR in state hospitals that was successful in the USA has also been found to be successful in Turkey. The results are used by the Turkish MoH to disseminate the nation-wide benefits of EHR functions.

The effect of medication related clinical decision support at the time of physician order entry

Background In advanced clinical decision support systems, patient characteristics and laboratory values are included in the algorithms that generate alerts. These alerts have a higher specificity than basic medication surveillance alerts. The alerts of advanced clinical decision support systems can be shown directly to the prescriber during order entry, without the risk of generating an overload of irrelevant alerts. We implemented five advanced algorithms that are shown directly to the prescriber. These algorithms are for gastrointestinal prophylaxis, folic or folinic acid prescribed with orally or subcutaneously administered methotrexate, vitamin D prescribed with bisphosphonates, hyponatremia and measuring plasma levels for vancomycin and gentamicin. Objective We evaluated the effect of the implementation of the algorithms. Setting We performed prospective intervention studies with a historical group for comparison in both inpatients and outpatients at a teaching hospital in the Netherlands. Methods We compared the time period after implementation of the algorithm with the time period before implementation, using data from the hospital information system Epic. Difference in guideline adherence were analyzed using Chi square tests. Main outcome measure The outcome measures were the number of alerts, the acceptance rate of the advice in the alert, and for the algorithm measuring plasma levels for vancomycin and gentamicin the time to the correct dose. Results For all algorithms, the implementation resulted in a significant increase in guideline adherence, varying from 11 to 36%. The acceptance rate varied from 14% for hyponatremia to 90% for methotrexate. For gastrointestinal prophylaxis the acceptance rate was 4.4% for basic drug-drug interaction alerts when no gastrointestinal prophylaxis was prescribed and increased to 44.7% after implementation of the advanced algorithm. This algorithm substantially decreased the number of alerts from 812 before implementation to 217 after implementation. After implementation of the algorithm for measuring plasma levels for vancomycin and gentamicin, the proportion of patients receiving the correct dose after 48 h increased from 73 to 84% (p = 0.03). Conclusion Implementation of advanced algorithms that take patient characteristics into account and are shown directly to the physician during order entry, result in an increased guideline adherence.

From sensitization to adoption? A qualitative study of the implementation of a digitally supported intervention for clinical decision making in polypharmacy

OBJECTIVE

Formative evaluation of the implementation process for a digitally supported intervention in polypharmacy in Germany. Qualitative research was conducted within a cluster randomized controlled trial (C-RCT). It focused on understanding how the intervention influences behavior-related outcomes in the prescription and medication review process.

METHODS/SETTING

Twenty-seven general practitioners (GPs) were included in the study in the two groups of the C-RCT, the intervention, and the wait list control group. Behavior-related outcomes were investigated using three-step data analysis (content analytic approach, documentary method, and design of a model of implementation pathways).

RESULTS

Content analysis showed that physicians were more intensely aware of polypharmacy-related risks, described positive learning effects of the digital technology on their prescribing behavior, and perceived a change in communication with patients and pharmacists. Conversely, they felt uncertain about their own responsibility when prescribing. Three main dimensions were discovered which influenced adoption behavior: (1) the physicians' interpretation of the relevance of pharmaceutical knowledge provided by the intervention in changing decision-making situations in polypharmacy; (2) their medical code of ethics for clinical decision making in the context of progressing digitalization; and (3) their concepts of evidence-based medicine on the basis of professional experiences with polypharmacy in primary care settings. In our sample, both simple and complex pathways from sensitization to adoption were observed. The resulting model on adoption behavior includes a paradigmatic description of different pathways and a visualization of different observed levels and applied methodological approaches. We assumed that the GP habitus can weaken or strengthen interventional effects towards intervention uptake. This formative evaluation strategy is beneficial for the identification of behavior-related implementation barriers and facilitators.

CONCLUSION

Our analyses of the adoption behavior of a digitally supported intervention in polypharmacy revealed both simple and complex pathways from awareness to adoption, which may impact the implementation of the intervention and therefore, its effectiveness. Future consideration of adoption behavior in the planning and evaluation of digitally supported interventions may enhance uptake and support the interpretation of effects.

TRIAL REGISTRATION

NCT03430336 , 12 February 2018.

Understanding the scope of downtime threats: A scoping review of downtime-focused literature and news media

Electronic health record downtimes are any period where the computer systems are unavailable, either for planned or unexpected events. During an unexpected downtime, healthcare workers are rapidly forced to use rarely-practiced, paper-based methods for healthcare delivery. In some instances, patient safety is compromised or data exposed to parties seeking profit. This review provides a foundational perspective of the current state of downtime readiness as organizations prepare to handle downtime events. A search of technical news media related to healthcare informatics and a scoping review of the research literature were conducted. Findings ranged from theoretical exploration of downtime to empirical direct comparison of downtime versus normal operation. Overall, 166 US hospitals experienced a total of 701 days of downtime in 43 events between 2012 and 2018. Almost half (48.8%) of the published downtime events involved some form of cyber-attacks. Downtime contingency planning is still predominantly considered through a top-down organizational focus. We propose that a bottom-up approach, involving the front-line clinical staff responsible for executing the downtime procedure, will be beneficial. Significant new research support for the development of contingency plans will be needed.

Bibliometric Analysis of Medication Errors and Adverse Drug Events Studies

OBJECTIVES

Medication errors and adverse drug events are a key concern of the health-care industry. The objectives of this study were to map the intellectual structure of the studies of medication errors and adverse drug events and to investigate the developing path of this literature and interrelationships among the main topics.

METHODS

The Web of Science database was searched for documentation of medication errors and adverse drug events from 1961 to 2013. The most cited articles and references were profiled and analyzed using HistCite software to draw a historiograph and Ucinet software to draw a sociogram.

RESULTS

The database search revealed 3343 medication errors and 3342 adverse drug event documents. The most cited articles on medication errors focused on 3 key themes from 1961 to 2013, namely, medication errors in adult inpatients, computerized physician order entry in medication error studies, and medication errors in pediatric inpatients. The developing path for the most cited articles about adverse drug events from 1987 to 2013 was as follows: detection, analysis, effect, and prevention from adult inpatient to pediatric inpatient settings and from hospitalized care to ambulatory care. In addition, social network analysis based on the most cited references revealed a close relationship between medication errors and adverse drug events.

CONCLUSIONS

The mapping results provide a valuable tool for researchers to access the literature in this field and can be used to help identify the direction of medication errors and adverse drug events research.

Effects of CPOE-based medication ordering on outcomes: an overview of systematic reviews

BACKGROUND

Computerised provider order entry (CPOE) systems are widely used in clinical settings for the electronic ordering of medications, laboratory tests and radiological therapies. However, evidence regarding effects of CPOE-based medication ordering on clinical and safety outcomes is mixed. We conducted an overview of systematic reviews (SRs) to characterise the cumulative effects of CPOE use for medication ordering in clinical settings.

METHODS

MEDLINE, EMBASE, CINAHL and the Cochrane Library were searched to identify published SRs from inception to 12 February 2018. SRs investigating the effects of the use of CPOE for medication ordering were included. Two reviewers independently extracted data and assessed the methodological quality of included SRs.

RESULTS

Seven SRs covering 118 primary studies were included for review. Pooled studies from the SRs in inpatient settings showed that CPOE use resulted in statistically significant decreases in medication errors and adverse drug events (ADEs); however, there was considerable variation in the magnitude of their relative risk reduction (54%-92% for errors, 35%-53% for ADEs). There was no significant relative risk reduction on hospital mortality or length of stay. Bibliographic analysis showed limited overlap (24%) among studies included across all SRs.

CONCLUSION

SRs on CPOEs included predominantly non-randomised controlled trials and observational studies with varying foci. SRs predominantly focused on inpatient settings and often lacked comparison groups; SRs used inconsistent definitions of outcomes, lacked descriptions regarding the effects on patient harm and did not differentiate among the levels of available decision support. With five of the seven SRs having low to moderate quality, findings from the SRs must be interpreted with caution. We discuss potential directions for future primary studies and SRs of CPOE.

Testing the face validity and inter-rater agreement of a simple approach to drug-drug interaction evidence assessment

Low concordance between drug-drug interaction (DDI) knowledge bases is a well-documented concern. One potential cause of inconsistency is variability between drug experts in approach to assessing evidence about potential DDIs. In this study, we examined the face validity and inter-rater reliability of a novel DDI evidence evaluation instrument designed to be simple and easy to use.

METHODS

A convenience sample of participants with professional experience evaluating DDI evidence was recruited. Participants independently evaluated pre-selected evidence items for 5 drug pairs using the new instrument. For each drug pair, participants labeled each evidence item as sufficient or insufficient to establish the existence of a DDI based on the evidence categories provided by the instrument. Participants also decided if the overall body of evidence supported a DDI involving the drug pair. Agreement was computed both at the evidence item and drug pair levels. A cut-off of ≥ 70% was chosen as the agreement threshold for percent agreement, while a coefficient > 0.6 was used as the cut-off for chance-corrected agreement. Open ended comments were collected and coded to identify themes related to the participants' experience using the novel approach.

RESULTS

The face validity of the new instrument was established by two rounds of evaluation involving a total of 6 experts. Fifteen experts agreed to participate in the reliability assessment, and 14 completed the study. Participant agreement on the sufficiency of 22 of the 34 evidence items (65%) did not exceed the a priori agreement threshold. Similarly, agreement on the sufficiency of evidence for 3 of the 5 drug pairs (60%) was poor. Chance-corrected agreement at the drug pair level further confirmed the poor interrater reliability of the instrument (Gwet's AC1 = 0.24, Conger's Kappa = 0.24). Participant comments suggested several possible reasons for the disagreements including unaddressed subjectivity in assessing an evidence item's type and study design, an infeasible separation of evidence evaluation from the consideration of clinical relevance, and potential issues related to the evaluation of DDI case reports.

CONCLUSIONS

Even though the key findings were negative, the study's results shed light on how experts approach DDI evidence assessment, including the importance situating evidence assessment within the context of consideration of clinical relevance. Analysis of participant comments within the context of the negative findings identified several promising future research directions including: novel computer-based support for evidence assessment; formal evaluation of a more comprehensive evidence assessment approach that requires consideration of specific, explicitly stated, clinical consequences; and more formal investigation of DDI case report assessment instruments.

Unexpected Increase in Benzodiazepine Prescriptions Related to the Introduction of an Electronic Prescribing Tool: Evidence from Multicenter Hospital Data

Electronic Prescribing tools (e-prescribing) have shown several benefits in terms of prescribing process adequacy and health care quality in hospital settings. We hypothesize however, that an undesired effect of digitalisation, due to the easier and faster prescribing process allowing patients to skip face-to-face conversations with patients and nurses, is that it could facilitate the prescription of medications at high risk of overuse or abuse, such as benzodiazepines (BZDs). We conducted a panel data study to investigate, the impact of the introduction of an e-prescribing system on new BZD prescriptions in hospitalised patients in a network of five teaching hospitals. During the observation period 1 July 2014-30 April 2019, 43,320 admissions were analysed. A fixed-effects model was adopted to estimate the effect of e-prescribing on new BZD prescriptions. E-prescribing implementation was associated with a significant increase of new BZD prescriptions: absolute +1.5%, and relative +43% (p < 0.001). The effect was similar in males and females (respectively, absolute +2.3%, relative +65% (p < 0.001); absolute +1.8%, relative +58% (p = 0.01)) and in patients ≥70 years old (absolute +1.6%, relative +59% (p < 0.001)). After controlling for time-varying explanatory variables, the implementation of the e-prescribing tool showed similar significant effects. E-prescribing implementation was associated with a significant increase of new in-hospital BZD prescriptions. For classes of drugs at risk of overuse or abuse, e-prescribing should be used cautiously, to minimize the risk of over-prescriptions. Further research in other settings and countries is needed to analyse causal interactions between e-prescribing and BZD prescriptions in the hospital setting, and to promote the ultimate goal of high-value care.

Effect of Electronic Prescribing Strategies on Medication Error and Harm in Hospital: a Systematic Review and Meta-analysis

BACKGROUND

Computerized physician order entry and clinical decision support systems are electronic prescribing strategies that are increasingly used to improve patient safety. Previous reviews show limited effect on patient outcomes. Our objective was to assess the impact of electronic prescribing strategies on medication errors and patient harm in hospitalized patients.

METHODS

MEDLINE, EMBASE, CENTRAL, and CINAHL were searched from January 2007 to January 2018. We included prospective studies that compared hospital-based electronic prescribing strategies with control, and reported on medication error or patient harm. Data were abstracted by two reviewers and pooled using random effects model. Study quality was assessed using the Effective Practice and Organisation of Care and evidence quality was assessed using Grading of Recommendations Assessment, Development, and Evaluation.

RESULTS

Thirty-eight studies were included; comprised of 11 randomized control trials and 27 non-randomized interventional studies. Electronic prescribing strategies reduced medication errors (RR 0.24 (95% CI 0.13, 0.46), I² 98%, n = 11) and dosing errors (RR 0.17 (95% CI 0.08, 0.38), I² 96%, n = 9), with both risk ratios significantly affected by advancing year of publication. There was a significant effect of electronic prescribing strategies on adverse drug events (ADEs) (RR 0.52 (95% CI 0.40, 0.68), I² 0%, n = 2), but not on preventable ADEs (RR 0.55 (95% CI 0.30, 1.01), I² 78%, n = 3), hypoglycemia (RR 1.03 (95% CI 0.62-1.70), I² 28%, n = 7), length of stay (MD - 0.18 (95% - 1.42, 1.05), I² 94%, n = 7), or mortality (RR 0.97 (95% CI 0.79, 1.19), I² 74%, n = 9). The quality of evidence was rated very low.

DISCUSSION

Electronic prescribing strategies decrease medication errors and adverse drug events, but had no effect on other patient outcomes. Conservative interpretations of these findings are supported by significant heterogeneity and the preponderance of low-quality studies.

Development and Evaluation of a Clinical Decision Support System to Improve Medication Safety

BACKGROUND

Clinical decision support systems (CDSSs) are a good strategy for preventing medication errors and reducing the incidence and severity of adverse drug events (ADEs). However, these systems are not very effective and are subject to multiple limitations that prevent their implementation in clinical practice.

OBJECTIVES

The objective of this study was to evaluate the effectiveness of an advanced CDSS, HIGEA, which generates alerts based on predefined clinical rules to identify patients at risk of an ADE.

METHODS

A multidisciplinary team defined the system and the clinical rules focusing on medication errors commonly encountered in clinical practice. Four intervention programs were defined: (1) dose adjustment in renal impairment; (2) adjustment of anticoagulation/antiplatelet therapy; (3) detection of biochemical/hematologic toxicities; and (4) therapeutic drug monitoring. We performed a 6-month observational prospective study to analyze the effectiveness of these clinical rules by calculating the positive predictive value (PPV).

RESULTS

The team defined 211 clinical rules. During the study period, HIGEA generated 1,086 alerts (8.9 alerts per working day), which were reviewed by pharmacists. Fifty-one percent (554/1,086) of alerts generated an intervention to prevent a possible ADE; of these, 66% (368/554) required a documented modification to therapy owing to a real prescription error intercepted. The intervention program that induced the highest number of modifications to therapy was the dose adjustment in renal impairment program (PPV = 0.51), followed by the adjustment of anticoagulation/antiplatelet therapy program (PPV = 0.24). The percentage of accepted interventions was similar in surgical units (68%), medical units (67%), and critical care units (63%).

CONCLUSION

Our study offers evidence that HIGEA is highly effective in preventing potential ADEs at the prescription stage.

A Quality Improvement Initiative to Decrease Platelet Ordering Errors and a Proposed Model for Evaluating Clinical Decision Support Effectiveness

BACKGROUND AND OBJECTIVES

Clinical decision support (CDS) and computerized provider order entry have been shown to improve health care quality and safety, but may also generate previously unanticipated errors. We identified multiple CDS tools for platelet transfusion orders. In this study, we sought to evaluate and improve the effectiveness of those CDS tools while creating and testing a framework for future evaluation of other CDS tools.

METHODS

Using a query of an enterprise data warehouse at a tertiary care pediatric hospital, we conducted a retrospective analysis to assess baseline use and performance of existing CDS for platelet transfusion orders. Our outcome measure was the percentage of platelet undertransfusion ordering errors. Errors were defined as platelet transfusion volumes ordered which were less than the amount recommended by the order set used. We then redesigned our CDS and measured the impact of our intervention prospectively using statistical process control methodology.

RESULTS

We identified that 62% of all platelet transfusion orders were placed with one of two order sets (Inpatient Service 1 and Inpatient Service 2). The Inpatient Service 1 order set had a significantly higher occurrence of ordering errors (3.10% compared with 1.20%). After our interventions, platelet transfusion order error occurrence on Inpatient Service 1 decreased from 3.10 to 0.33%.

CONCLUSION

We successfully reduced platelet transfusion ordering errors by redesigning our CDS tools. We suggest that the use of collections of clinical data may help identify patterns in erroneous ordering, which could otherwise go undetected. We have created a framework which can be used to evaluate the effectiveness of other similar CDS tools.

Electronic health record (EHR) training program identifies a new tool to quantify the EHR time burden and improves providers' perceived control over their workload in the EHR

OBJECTIVE

To understand if providers who had additional electronic health record (EHR) training improved their satisfaction, decreased personal EHR-use time, and decreased turnaround time on tasks.

MATERIALS AND METHODS

This pre-post study with no controls evaluated the impact of a supplemental EHR training program on a group of academic and community practice clinicians that previously had go-live group EHR training and 20 months experience using this EHR on self-reported data, calculated EHR time, and vendor-reported metrics.

RESULTS

Providers self-reported significant improvements in their knowledge of efficiency tools in the EHR after training and doubled (significant) their preference list entries (mean pre = 38.1 [65.88], post = 63.5 [90.47], P < .01). Of the 7 EHR satisfaction variables, only 1 self-reported variable significantly improved after training: Control over my workload in the EHR (mean pre = 2.7 [0.96], post = 3.0 [1.04], P < .01). There was no significant decrease in their calculated EHR usage outside of clinic (mean pre = 0.39 [0.77] to post = 0.37 [0.48], P = .73). No significant difference was seen in turnaround time for patient calls (mean pre = 2.3 [2.06] days, post = 1.9 [1.76] days, P = .08) and results (mean before = 4.0 [2.79] days, after = 3.2 [2.33] days, P = .03).

DISCUSSION

Multiple sources of data provide a holistic view of the provider experience in the EHR. This study suggests that individualized EHR training can improve the knowledge of EHR tools and satisfaction with their perceived control of EHR workload, however this did not translate into less Clinician Logged-In Outside Clinic (CLOC) time, a calculated metric, nor quicker turnaround on in box tasks. CLOC time emerged as a potential less-costly surrogate metric for provider satisfaction in EHR work than surveying clinicians. Further study is required to understand the cost-benefit of various interventions to decrease CLOC time.

CONCLUSIONS

This supplemental EHR training session, 20 months post go-live, where most participants elected to receive 2 or fewer sessions did significantly improve provider satisfaction with perceived control over their workload in the EHR, but it was not effective in decreasing EHR-use time outside of clinic. CLOC time, a calculated metric, could be a practical trackable surrogate for provider satisfaction (inverse correlation) with after-hours time spent in the EHR. Further study into interventions that decrease CLOC time and improve turnaround time to respond to inbox tasks are suggested next steps.

Systematic review of computerized prescriber order entry and clinical decision support

PURPOSE

Results of a systematic review of published data on the effect of computerized prescriber order entry (CPOE) with clinical decision support on medication error (ME) and adverse drug event (ADE) rates are presented.

METHODS

Literature searches of MEDLINE, Embase, and other databases were conducted to identify English- and Spanish-language articles on selected CPOE outcomes published from 1995 through 2016; in addition, 5 specific journals were searched for pertinent articles published during the period 2010-16. Publications on controlled prospective studies and before-and-after studies that assessed MEs and/or ADEs as main outcomes were selected for inclusion in the review.

RESULTS

Nineteen studies met the inclusion criteria. Data on MEs and ADEs could not be pooled, mainly due to heterogeneity in outcome definitions and study methodologies. The reviewed evidence indicated that CPOE implementation led to an overall reduction in errors at the prescription stage of the medication-use process (relative risk reduction, 0.29 [95% confidence interval, 0.10-0.85]; I ² = 99%) and reductions in most types of prescription errors, but CPOE also resulted in the emergence of other types of errors.

CONCLUSION

CPOE reduces the overall ME rate in the prescription process, as well as specific types of errors, such as wrong dose or strength, wrong drug, frequency, administration route, and drug-drug interaction errors. The implementation of CPOE can lead to new errors, such as wrong drug selection from drop-down menus.

Clinical relevance of alerts from a decision support system, PHARAO, for drug safety assessment in the older adults

Background

PHARAO is a decision support system developed to evaluate the risk for a set of either common or serious side-effects resulting from a combination of pharmacodynamic effects from a patient’s medications. The objective of this study was to investigate the validity of the risk scores for the common side-effects generated by PHARAO in older patients.

Methods

Side-effects included were sedation, constipation, orthostatic symptoms, anticholinergic and serotonergic effects. The alerts generated by PHARAO were tested in 745 persons ≥65 years old. Dispensed prescriptions retrieved from the Swedish prescribed drug register were used to generate the pharmacological risk scores of patients’ medications. Symptoms possibly related to side-effects were extracted from medical records data.

Results

The PHARAO system generated 776 alerts, most often for the risk of anticholinergic symptoms. The total specificity estimates of the PHARAO system were 0.95, 0.89 and 0.78 for high, intermediate and low risk alerts, respectively. The corresponding sensitivity estimates were between 0.12 and 0.37. The negative predictive value was 0.90 and the positive predictive value ranged between 0.20–0.25.

Conclusions

The PHARAO system had a high specificity and negative predictive value to detect symptoms possibly associated with the of patients’ medications, while the sensitivity and positive predictive value were low. The PHARAO system has the potential to minimise the risk of over-alerts in combination with a drug-drug interaction alert system, but should be used in connection with a medical evaluation of the patient.

Electronic supplementary material

The online version of this article (10.1186/s12877-019-1179-y) contains supplementary material, which is available to authorized users.

Continuing Patient Care during Electronic Health Record Downtime

INTRODUCTION

Electronic health record (EHR) downtime is any period during which the EHR system is fully or partially unavailable. These periods are operationally disruptive and pose risks to patients. EHR downtime has not sufficiently been studied in the literature, and most hospitals are not adequately prepared.

OBJECTIVE

The objective of this study was to assess the operational implications of downtime with a focus on the clinical laboratory, and to derive recommendations for improved downtime contingency planning.

METHODS

A hybrid qualitative-quantitative study based on historic performance data and semistructured interviews was performed at two mid-Atlantic hospitals. In the quantitative analysis, paper records from downtime events were analyzed and compared with normal operations. To enrich this quantitative analysis, interviews were conducted with 17 hospital employees, who had experienced several downtime events, including a hospital-wide EHR shutdown.

RESULTS

During downtime, laboratory testing results were delayed by an average of 62% compared with normal operation. However, the archival data were incomplete due to inconsistencies in the downtime paper records. The qualitative interview data confirmed that delays in laboratory result reporting are significant, and further uncovered that the delays are often due to improper procedural execution, and incomplete or incorrect documentation. Interviewees provided a variety of perspectives on the operational implications of downtime, and how to best address them. Based on these insights, recommendations for improved downtime contingency planning were derived, which provide a foundation to enhance Safety Assurance Factors for EHR Resilience guides.

CONCLUSION

This study documents the extent to which downtime events are disruptive to hospital operations. It further highlights the challenge of quantitatively assessing the implication of downtimes events, due to a lack of otherwise EHR-recorded data. Organizations that seek to improve and evaluate their downtime contingency plans need to find more effective methods to collect data during these times.

The usability aspects of medication-related decision support systems in the inpatient setting: A systematic review

Effort has been made to study the effect of medication-related clinical decision support systems in the inpatient setting; however, there is not much known about the usability of these systems. The goal of this study is to systematically review studies that focused on the usability aspects such as effectiveness, efficiency, and satisfaction of these systems. We systematically searched relevant articles in Scopus, Embase, and PubMed from 1 January 2000 to 1 January 2016, and found 22 articles. Based on Van Welie’s usability model, we categorized usability aspects in terms of usage indicators and means. Our results showed that evidence was mainly found for effectiveness and efficiency. They showed positive results in the usage indicators errors and safety and performance speed. The means warnings and adaptability also had mostly positive results. To date, the effects satisfaction of clinical decision support system remains understudied. Aspects such as memorability, learnability, and consistency require more attention.

Implications of electronic health record downtime: an analysis of patient safety event reports

Objective

We sought to understand the types of clinical processes, such as image and medication ordering, that are disrupted during electronic health record (EHR) downtime periods by analyzing the narratives of patient safety event report data.

Materials and Methods

From a database of 80 381 event reports, 76 reports were identified as explicitly describing a safety event associated with an EHR downtime period. These reports were analyzed and categorized based on a developed code book to identify the clinical processes that were impacted by downtime. We also examined whether downtime procedures were in place and followed.

Results

The reports were coded into categories related to their reported clinical process: Laboratory, Medication, Imaging, Registration, Patient Handoff, Documentation, History Viewing, Delay of Procedure, and General. A majority of reports (48.7%, n = 37) were associated with lab orders and results, followed by medication ordering and administration (14.5%, n = 11). Incidents commonly involved patient identification and communication of clinical information. A majority of reports (46%, n = 35) indicated that downtime procedures either were not followed or were not in place. Only 27.6% of incidents (n = 21) indicated that downtime procedures were successfully executed.

Discussion

Patient safety report data offer a lens into EHR downtime-related safety hazards. Important areas of risk during EHR downtime periods were patient identification and communication of clinical information; these should be a focus of downtime procedure planning to reduce safety hazards.

Conclusion

EHR downtime events pose patient safety hazards, and we highlight critical areas for downtime procedure improvement.

Evaluation of clinical decision support systems for diabetes care: An overview of current evidence

BACKGROUND

Systematic reviews (SRs) have shown that clinical decision support systems (CDSSs) have the potential to improve diabetes care. However, methods of measuring and presenting outcomes are varied, and conclusions have been inconsistent. In addition, the reporting and methodological quality in this field is unknown, which could affect the integrity and accuracy of research. Therefore, it is difficult to confirm whether CDSSs are effective in improving diabetes care.

OBJECTIVE

To comprehensively evaluate the effects of CDSS on diabetes care and to examine the methodological and reporting qualities.

METHODS

We searched PubMed, EMBASE, and Cochrane Library from their inception to February 2017. Systematic reviews investigating the effects of CDSS on diabetes care were included. Outcomes were determined in advance and assessed separately for process of care and patient outcomes. Methodological and reporting qualities were assessed by AMSTAR and PRISMA, respectively.

RESULTS

Seventeen SRs, consisting of 222 unique randomized controlled trials and 102 nonrandomized controlled trials, were included. Evidence that CDDS significantly impacted patient outcomes was found in 32 of 102 unique studies of the 15 SRs that examined this effect (31%). A significant impact of CDSS on process of care was found in 117 out of 143 unique studies of the 11 SRs that examined this effect (82%). Ratings for overall scores of AMSTAR resulted in a mean score of 6.5 with a range of scores from 3.5 to 10.0. Reporting quality related to methodological domains was particularly incomplete.

CONCLUSIONS

Clinical decision support systems improved the quality of diabetes care by inconsistently improving process of care or patient outcomes. There is evidence that CDSS for providing alerts, reminders, or feedback to participants were most likely to impact diabetes care. Poor reporting of methodological domains, together with qualitative or narrative methods to combine findings, may limit the confidence in research evidence.

Reduced Verification of Medication Alerts Increases Prescribing Errors

OBJECTIVE

Clinicians using clinical decision support (CDS) to prescribe medications have an obligation to ensure that prescriptions are safe. One option is to verify the safety of prescriptions if there is uncertainty, for example, by using drug references. Supervisory control experiments in aviation and process control have associated errors, with reduced verification arising from overreliance on decision support. However, it is unknown whether this relationship extends to clinical decision-making. Therefore, we examine whether there is a relationship between verification behaviors and prescribing errors, with and without CDS medication alerts, and whether task complexity mediates this.

METHODS

A total of 120 students in the final 2 years of a medical degree prescribed medicines for patient scenarios using a simulated electronic prescribing system. CDS (correct, incorrect, and no CDS) and task complexity (low and high) were varied. Outcomes were omission (missed prescribing errors) and commission errors (accepted false-positive alerts). Verification measures were access of drug references and view time percentage of task time.

RESULTS

Failure to access references for medicines with prescribing errors increased omission errors with no CDS (high-complexity: χ ²(1) = 12.716; p < 0.001) and incorrect CDS (Fisher's exact; low-complexity: p = 0.002; high-complexity: p = 0.001). Failure to access references for false-positive alerts increased commission errors (low-complexity: χ ²(1) = 16.673, p < 0.001; high-complexity: χ ²(1) = 18.690, p < 0.001). Fewer participants accessed relevant references with incorrect CDS compared with no CDS (McNemar; low-complexity: p < 0.001; high-complexity: p < 0.001). Lower view time percentages increased omission (F(3, 361.914) = 4.498; p = 0.035) and commission errors (F(1, 346.223) = 2.712; p = 0.045). View time percentages were lower in CDS-assisted conditions compared with unassisted conditions (F(2, 335.743) = 10.443; p < 0.001).

DISCUSSION

The presence of CDS reduced verification of prescription safety. When CDS was incorrect, reduced verification was associated with increased prescribing errors.

CONCLUSION

CDS can be incorrect, and verification provides one mechanism to detect errors. System designers need to facilitate verification without increasing workload or eliminating the benefits of correct CDS.

International Group for Reducing Inappropriate Medication Use & Polypharmacy (IGRIMUP): Position Statement and 10 Recommendations for Action

Globally, the number of drug prescriptions is increasing causing more adverse drug events, which is now a significant cause of mortality, morbidity, and disability that has reached epidemic proportions. The risk of adverse drug events is correlated to very old age, multiple co-morbidities, dementia, frailty, and limited life expectancy, with the major contributor being polypharmacy. Each characteristic alters the risk-benefit balance of medications, typically reducing anticipated benefits and amplifying risk. Current clinical guidelines are based on evidence proven in younger/healthier adult populations using a single disease model and their application to older adults with multimorbidity, in whom testing has not been conducted, yields a different risk-benefit prospect and makes inappropriate medication use and polypharmacy inevitable. Applying inappropriate clinical practice guidelines to older adults is antithetical to good healthcare, is likely to increase health inequity, and is associated with substantial negative clinical, economic, and social implications for health systems. The casualties are on the scale of a war or epidemic, yet are usually invisible in measures of healthcare quality and formal recommendations. Radical and rapid action is required to achieve a better quality of life for older populations and to remain true to the principles of medical professionalism and evidence-based medicine that place patients' interests and autonomy at the fore. This first International Group for Reducing Inappropriate Medication Use & Polypharmacy position statement briefly details the causes, consequences, and extent of inappropriate medication use and polypharmacy. This article outlines current strategies to reduce inappropriate medication use, provides evidence for their effect, and then proposes recommendations for moving forward with 10 recommendations for action and 12 recommendations for research. We conclude that an urgent integrated effort to reduce inappropriate medication use and polypharmacy should be a leading global target of the highest priority. The cornerstone of this position statement from the International Group for Reducing Inappropriate Medication Use & Polypharmacy is the understanding that without evidence of definite relevant benefit, when it comes to prescribing, for many older patients 'less is more'. This approach differs from most other current recommendations and guidance in medical care, as the focus is on what, when, and how to stop, rather than on when to start medications/interventions. Disrupting the framework that indiscriminately applies standard guidelines to older adults requires a new approach that better serves patients with multimorbidity. This transition requires a shift in medical education, research, and diagnostic frameworks, and re-examination of the measures used as quality indicators. In achieving this objective, we promote a return to some of the original concepts of evidence-based medicine: which considers scientific data (where it exists), clinical judgment, patient/family preference, and context. A shift is needed: from the current model that focuses on single conditions to one that simultaneously considers multiple conditions and patient priorities. This approach reframes the clinician's role as a professional providing care, rather than a disease technician.

Qualitative study exploring the phenomenon of multiple electronic prescribing systems within single hospital organisations

Background

A previous census of electronic prescribing (EP) systems in England showed that more than half of hospitals with EP reported more than one EP system within the same hospital. Our objectives were to describe the rationale for having multiple EP systems within a single hospital, and to explore perceptions of stakeholders about the advantages and disadvantages of multiple systems including any impact on patient safety.

Methods

Hospitals were selected from previous census respondents. A decision matrix was developed to achieve a maximum variation sample, and snowball sampling used to recruit stakeholders of different professional backgrounds. We then used an a priori framework to guide and analyse semi-structured interviews.

Results

Ten participants, comprising pharmacists and doctors and a nurse, were interviewed from four hospitals. The findings suggest that use of multiple EP systems was not strategically planned. Three co-existing models of EP systems adoption in hospitals were identified: organisation-led, clinician-led and clinical network-led, which may have contributed to multiple systems use. Although there were some perceived benefits of multiple EP systems, particularly in niche specialities, many disadvantages were described. These included issues related to access, staff training, workflow, work duplication, and system interfacing. Fragmentation of documentation of the patient’s journey was a major safety concern.

Discussion

The complexity of EP systems’ adoption and deficiencies in IT strategic planning may have contributed to multiple EP systems use in the NHS. In the near to mid-term, multiple EP systems may remain in place in many English hospitals, which may create challenges to quality and patient safety.

Electronic supplementary material

The online version of this article (10.1186/s12913-018-3750-1) contains supplementary material, which is available to authorized users.