Quality of Decision Support in Computerized Provider Order Entry: Systematic Literature Review

Assessing the applicability and appropriateness of ChatGPT in answering clinical pharmacy questions

OBJECTIVES

Clinical pharmacists rely on different scientific references to ensure appropriate, safe, and cost-effective drug use. Tools based on artificial intelligence (AI) such as ChatGPT (Generative Pre-trained Transformer) could offer valuable support. The objective of this study was to assess ChatGPT's capacity to correctly respond to clinical pharmacy questions asked by healthcare professionals in our university hospital.

MATERIAL AND METHODS

ChatGPT's capacity to respond correctly to the last 100 consecutive questions recorded in our clinical pharmacy database was assessed. Questions were copied from our FileMaker Pro database and pasted into ChatGPT March 14 version online platform. The generated answers were then copied verbatim into an Excel file. Two blinded clinical pharmacists reviewed all the questions and the answers given by the software. In case of disagreements, a third blinded pharmacist intervened to decide.

RESULTS

Documentation-related issues (n=36) and drug administration mode (n=30) were preponderantly recorded. Among 69 applicable questions, the rate of correct answers varied from 30 to 57.1% depending on questions type with a global rate of 44.9%. Regarding inappropriate answers (n=38), 20 were incorrect, 18 gave no answers and 8 were incomplete with 8 answers belonging to 2 different categories. No better answers than the pharmacists were observed.

CONCLUSIONS

ChatGPT demonstrated a mitigated performance in answering clinical pharmacy questions. It should not replace human expertise as a high rate of inappropriate answers was highlighted. Future studies should focus on the optimization of ChatGPT for specific clinical pharmacy questions and explore the potential benefits and limitations of integrating this technology into clinical practice.

Effect of knowledgebase transition of a clinical decision support system on medication order and alert patterns in an emergency department

A knowledgebase (KB) transition of a clinical decision support (CDS) system occurred at the study site. The transition was made from one commercial database to another, provided by a different vendor. The change was applied to all medications in the institute. The aim of this study was to analyze the effect of KB transition on medication-related orders and alert patterns in an emergency department (ED). Data of patients, medication-related orders and alerts, and physicians in the ED from January 2018 to December 2020 were analyzed in this study. A set of definitions was set to define orders, alerts, and alert overrides. Changes in order and alert patterns before and after the conversion, which took place in May 2019, were assessed. Overall, 101,450 patients visited the ED, and 1325 physicians made 829,474 prescription orders to patients during visit and at discharge. Alert rates (alert count divided by order count) for periods A and B were 12.6% and 14.1%, and override rates (alert override count divided by alert count) were 60.8% and 67.4%, respectively. Of the 296 drugs that were used more than 100 times during each period, 64.5% of the drugs had an increase in alert rate after the transition. Changes in alert rates were tested using chi-squared test and Fisher's exact test. We found that the CDS system knowledgebase transition was associated with a significant change in alert patterns at the medication level in the ED. Careful consideration is advised when such a transition is performed.

Tackling alert fatigue with a semi-automated clinical decision support system: quantitative evaluation and end-user survey

STUDY AIMS

Clinical decision support systems (CDSS) embedded in hospital electronic health records efficiently reduce medication errors, but there is a risk of low physician adherence due to alert fatigue. At the Cantonal Hospital Aarau, a CDSS is being developed that allows the highly accurate detection and correction of medication errors. The semi-automated CDSS sends its alerts either directly to the physician or to a clinical pharmacist for review first. Our aim was to evaluate the performance of the recently implemented CDSS in terms of acceptance rate and alert burden, as well as physicians' satisfaction with the CDSS.

METHODS

All alerts generated by the clinical decision support systems between January and December 2021 were included in a retrospective quantitative evaluation. A team of clinical pharmacists performed a follow-up to determine whether the recommendation made by the CDSS was implemented by the physician. The acceptance rate was calculated including all alerts for which it was possible to determine an outcome. A web-based survey was conducted amongst physicians to assess their attitude towards the CDSS. The survey questions included overall satisfaction, helpfulness of individual algorithms, and perceived alert burden.

RESULTS

In 2021, a total of 10,556 alerts were generated, of which 619 triggered a direct notification to the physician and 2,231 notifications were send to the physician after evaluation by a clinical pharmacist. The acceptance rates were 89.8% and 68.4%, respectively, which translates as an overall acceptance rate of 72.4%. On average, clinical pharmacists received 17.2 alerts per day, while all of the hospital physicians together received 7.8 notifications per day. In the survey, 94.5% of physicians reported being satisfied or very satisfied with the CDSS. Algorithms addressing potential medication errors concerning anticoagulants received the highest usefulness ratings.

CONCLUSION

The development of this semi-automated clinical decision support system with context-based algorithms resulted in alerts with a high acceptance rate. Involving clinical pharmacists proved a promising approach to limit the alert burden of physicians and thus tackle alert fatigue. The CDSS is well accepted by our physicians.

Improvement in Pharmacy Student Responses to Medication-Related Problems with and without Clinical Decision Support Alerts

OBJECTIVE

To assess pharmacy student responses to medication problems with and without clinical decision support (CDS) alerts during simulated order verification.

METHODS

Three classes of students completed an order verification simulation. The simulation randomized students to a different series of 10 orders with varying CDS alert frequency. Two of the orders contained medication-related problems. The appropriateness of the students' interventions and responses to the CDS alerts were evaluated. In the following semester for 2 classes, 2 similar simulations were completed. All 3 simulations contained 1 problem with and 1 without an alert.

RESULTS

During the first simulation, 384 students reviewed an order with a problem and an alert. Students exposed to prior inappropriate alerts within the simulation had less appropriate responses (66% vs 75%). Of 321 students who viewed a second order with a problem, those reviewing an order lacking an alert recommended an appropriate change less often (45% vs 87%). Among 351 students completing the second simulation, those who participated in the first simulation appropriately responded to the alert for a problem more often than those who only received a didactic debrief (95% vs 87%). Among those completing all 3 simulations, appropriate responses increased between simulations for problems with (n = 238, 72-95-93%) and without alerts (n = 49, 53-71-90%).

CONCLUSIONS

Some pharmacy students displayed baseline alert fatigue and overreliance on CDS alerts for medication problem detection during order verification simulations. Exposure to the simulations improved CDS alert response appropriateness and detection of problems.

Development and retrospective evaluation of a clinical decision support system for the efficient detection of drug-related problems by clinical pharmacists

BACKGROUND

Clinical decision support systems (CDSS) can help identify drug-related problems (DRPs). However, the alert specificity remains variable. Defining more relevant alerts for detecting DRPs would improve CDSS.

AIM

Develop electronic queries that assist pharmacists in conducting medication reviews and an assessment of the performance of this model to detect DRPs.

METHOD

Electronic queries were set up in CDSS using "triggers" from electronic health records: drug prescriptions, laboratory values, medical problems, vital signs, demographics. They were based on a previous study where 315 patients admitted in internal medicine benefited from a multidisciplinary medication review (gold-standard) to highlight potential DRPs. Electronic queries were retrospectively tested to assess performance in detecting DRPs revealed with gold-standard. For each electronic query, sensitivity, specificity, positive and negative predictive value were computed.

RESULTS

Of 909 DRPs, 700 (77.8%) were used to create 366 electronic queries. Electronic queries correctly detected 77.1% of DRPs, median sensitivity and specificity reached 100.0% (IQRs, 100.0%-100.0%) and 99.7% (IQRs, 97.0%-100.0%); median positive predictive value and negative predictive value reached 50.0% (IQRs, 12.5%-100.0%) and 100.0% (IQRs, 100.0%-100.0%). Performances varied according to "triggers" (p < 0.001, best performance in terms of predictive positive value when exclusively involving drug prescriptions).

CONCLUSION

Electronic queries based on electronic heath records had high sensitivity and negative predictive value and acceptable specificity and positive predictive value and may contribute to facilitate medication review. Implementing some of these electronic queries (the most effective and clinically relevant) in current practice will allow a better assessment of their impact on the efficiency of the clinical pharmacist.

Added value of patient- and drug-related factors to stratify drug-drug interaction alerts for risk of QT prolongation: Development and validation of a risk prediction model

AIMS

Many clinical decision support systems trigger warning alerts for drug-drug interactions potentially leading to QT prolongation and torsades de pointes (QT-DDIs). Unfortunately, there is overalerting and underalerting because stratification is only based on a fixed QT-DDI severity level. We aimed to improve QT-DDI alerting by developing and validating a risk prediction model considering patient- and drug-related factors.

METHODS

We fitted 31 predictor candidates to a stepwise linear regression for 1000 bootstrap samples and selected the predictors present in 95% of the 1000 models. A final linear regression model with those variables was fitted on the original development sample (350 QT-DDIs). This model was validated on an external dataset (143 QT-DDIs). Both true QTc and predicted QTc were stratified into three risk levels (low, moderate and high). Stratification of QT-DDIs could be appropriate (predicted risk = true risk), acceptable (one risk level difference) or inappropriate (two risk levels difference).

RESULTS

The final model included 11 predictors with the three most important being use of antiarrhythmics, age and baseline QTc. Comparing current practice to the prediction model, appropriate stratification increased significantly from 37% to 54% appropriate QT-DDIs (increase of 17.5% on average [95% CI +5.4% to +29.6%], p_adj  = 0.006) and inappropriate stratification decreased significantly from 13% to 1% inappropriate QT-DDIs (decrease of 11.2% on average [95% CI -17.7% to -4.7%], p_adj  ≤ 0.001).

CONCLUSION

The prediction model including patient- and drug-related factors outperformed QT alerting based on QT-DDI severity alone and therefore is a promising strategy to improve DDI alerting.

Electronic monitoring of potential adverse drug events related to lopinavir/ritonavir and hydroxychloroquine during the first wave of COVID-19

During Switzerland's first wave of COVID-19, clinical pharmacy activities during medical rounds in Geneva University Hospitals were replaced by targeted remote interventions. We describe using the electronic PharmaCheck system to screen high-risk situations of adverse drug events (ADEs), particularly targeting prescriptions of lopinavir/ritonavir (LPVr) and hydroxychloroquine (HCQ) in the presence of contraindications or prescriptions outside institutional guidelines. Of 416 patients receiving LPVr and/or HCQ, 182 alerts were triggered for 164 (39.4%) patients. The main associated risk factors of ADEs were drug-drug interactions, QTc interval prolongation, electrolyte disorder and inadequate LPVr dosage. Therapeutic optimisation recommended by a pharmacist or proposals for additional monitoring were accepted in 80% (n=36) of cases. Combined with pharmacist contextualisation to the clinical context, PharmaCheck made it possible to successfully adapt clinical pharmacist activities by switching from a global to a targeted analysis mode in an emergency context.

Workload and workflow implications associated with the use of electronic clinical decision support tools used by health professionals in general practice: a scoping review

BACKGROUND

Electronic clinical decision support tools (eCDS) are increasingly available to assist General Practitioners (GP) with the diagnosis and management of a range of health conditions. It is unclear whether the use of eCDS tools has an impact on GP workload. This scoping review aimed to identify the available evidence on the use of eCDS tools by health professionals in general practice in relation to their impact on workload and workflow.

METHODS

A scoping review was carried out using the Arksey and O'Malley methodological framework. The search strategy was developed iteratively, with three main aspects: general practice/primary care contexts, risk assessment/decision support tools, and workload-related factors. Three databases were searched in 2019, and updated in 2021, covering articles published since 2009: Medline (Ovid), HMIC (Ovid) and Web of Science (TR). Double screening was completed by two reviewers, and data extracted from included articles were analysed.

RESULTS

The search resulted in 5,594 references, leading to 95 full articles, referring to 87 studies, after screening. Of these, 36 studies were based in the USA, 21 in the UK and 11 in Australia. A further 18 originated from Canada or Europe, with the remaining studies conducted in New Zealand, South Africa and Malaysia. Studies examined the use of eCDS tools and reported some findings related to their impact on workload, including on consultation duration. Most studies were qualitative and exploratory in nature, reporting health professionals' subjective perceptions of consultation duration as opposed to objectively-measured time spent using tools or consultation durations. Other workload-related findings included impacts on cognitive workload, "workflow" and dialogue with patients, and clinicians' experience of "alert fatigue".

CONCLUSIONS

The published literature on the impact of eCDS tools in general practice showed that limited efforts have focused on investigating the impact of such tools on workload and workflow. To gain an understanding of this area, further research, including quantitative measurement of consultation durations, would be useful to inform the future design and implementation of eCDS tools.

Development and assessment of PharmaCheck: an electronic screening tool for the prevention of twenty major adverse drug events

BACKGROUND

Adverse drug events (ADEs) can be prevented by deploying clinical decision support systems (CDSS) that directly assist physicians, via computerized order entry systems, and clinical pharmacists performing medication reviews as part of medical rounds. However, physicians using CDSS are known to be exposed to the alert-fatigue phenomenon. Our study aimed to assess the performance of PharmaCheck-a CDSS to help clinical pharmacists detect high-risk situations with the potential to lead to ADEs-and its impact on clinical pharmacists' activities.

METHODS

Twenty clinical rules, divided into four risk classes, were set for the daily screening of high-risk situations in the electronic health records of patients admitted to our General Internal Medicine Department. Alerts to clinical pharmacists encouraged them to telephone prescribers and suggest any necessary treatment adjustments. PharmaCheck's performance was assessed using the intervention's positive predictive value (PPV), which characterizes the proportion of interventions for each alert triggered. PharmaCheck's impact was assessed by considering clinical pharmacists as a filter for ruling out futile alerts and by comparing the final clinical PPV with a pharmacist (the proportion of interventions that led to a change in the medical regimen) to the final clinical PPV without a pharmacist.

RESULTS

Over 132 days, 447 alerts were triggered for 383 patients, leading to 90 interventions (overall intervention PPV = 20.1%). By risk class, intervention PPVs made up 26.9% (n = 65/242) of abnormal laboratory value alerts, 3.1% (4/127) of alerts for contraindicated medications or medications to be used with caution, 28.2% (20/71) of drug-drug interaction alerts, and 14.3% (1/7) of inadequate mode of administration alerts. Clinical PPVs reached 71.0% (64/90) when pharmacists filtered alerts and 14% (64/242) if they were not doing it.

CONCLUSION

PharmaCheck enabled clinical pharmacists to improve their traditional processes and broaden their coverage by focusing on 20 high-risk situations. Alert management by pharmacists seemed to be a more effective way of preventing risky situations and alert-fatigue than a model addressing alerts to physicians exclusively. Some fine-tuning could enhance PharmaCheck's performance by considering the information quality of triggers, the variability of clinical settings, and the fact that some prescription processes are already highly secured.

Pharmacogenomic Clinical Decision Support: A Review, How-to Guide, and Future Vision

Clinical decision support (CDS) is an essential part of any pharmacogenomics (PGx) implementation. Increasingly, institutions have implemented CDS tools in the clinical setting to bring PGx data into patient care, and several have published their experiences with these implementations. However, barriers remain that limit the ability of some programs to create CDS tools to fit their PGx needs. Therefore, the purpose of this review is to summarize the types, functions, and limitations of PGx CDS currently in practice. Then, we provide an approachable step‐by‐step how‐to guide with a case example to help implementers bring PGx to the front lines of care regardless of their setting. Particular focus is paid to the five “rights” of CDS as a core around designing PGx CDS tools. Finally, we conclude with a discussion of opportunities and areas of growth for PGx CDS.

Methods for the detection of adverse drug reactions in hospitalized children: a systematic review

Introduction: Adverse drug reactions (ADR) are a problem for healthcare systems worldwide. Pediatric patients constitute a vulnerable group with regard to ADRs. However, although pediatric patients are at increased risk for these reactions, there is little progress on ADR detection methods in this group.Areas covered: In this systematic search, performed according to PRISMA statements, we selected studies, published in PubMed/Medline databases; Scopus; LILACS; Web of Science; Embase and Cochrane Library until April, 2020, on ADRs in hospitalized pediatric patients.Expert opinion: The increase of pediatric drug safety data is essential to the improvement of childcare. Health services must continuously stimulate educational programs focused on ADR detection tools to minimize the barriers and raise awareness among professionals. Therefore, it is necessary to consider that each method has advantages and disadvantages and must be analyzed in detail to be implemented according to the peculiarities of each practice scenario. Triggers tools (active method) correlated with electronic medical notes seems a good strategy for ADR identification, whether pediatric parameters are well checked and adapted with each age group. In any event, combined methods will add data to identification and clearer ADR assessment.

Evaluation of an optimized context-aware clinical decision support system for drug-drug interaction screening

OBJECTIVE

Evaluation of the effect of six optimization strategies in a clinical decision support system (CDSS) for drug-drug interaction (DDI) screening on alert burden and alert acceptance and description of clinical pharmacist intervention acceptance.

METHODS

Optimizations in the new CDSS were the customization of the knowledge base (with addition of 67 extra DDIs and changes in severity classification), a new alert design, required override reasons for the most serious alerts, the creation of DDI-specific screening intervals, patient-specific alerting, and a real-time follow-up system of all alerts by clinical pharmacists with interventions by telephone was introduced. The alert acceptance was evaluated both at the prescription level (i.e. prescription acceptance, was the DDI prescribed?) and at the administration level (i.e. administration acceptance, did the DDI actually take place?). Finally, the new follow-up system was evaluated by assessing the acceptance of clinical pharmacist's interventions.

RESULTS

In the pre-intervention period, 1087 alerts (92.0 % level 1 alerts) were triggered, accounting for 19 different DDIs. In the post-intervention period, 2630 alerts (38.4 % level 1 alerts) were triggered, representing 86 different DDIs. The relative risk forprescription acceptance in the post-intervention period compared to the pre-intervention period was 4.02 (95 % confidence interval (CI) 3.17-5.10; 25.5 % versus 6.3 %). The relative risk for administration acceptance was 1.16 (95 % CI 1.08-1.25; 54.4 % versus 46.7 %). Finally, 86.9 % of the clinical pharmacist interventions were accepted.

CONCLUSION

Six concurrently implemented CDSS optimization strategies resulted in a high alert acceptance and clinical pharmacist intervention acceptance. Administration acceptance was remarkably higher than prescription acceptance.

Electronic Health Record Reminders for Chlamydia Screening in an American Indian Population

OBJECTIVES

Indian Health Service (IHS) screening rates for Chlamydia trachomatis are lower than national rates of chlamydia screening in the Southwest. We describe and evaluate the effect of a public health intervention consisting of electronic health record (EHR) reminders to alert health care providers to screen for chlamydia at an IHS facility. We also conducted an awareness presentation among health care providers on chlamydia screening.

METHODS

We conducted our intervention from November 1, 2013, through October 31, 2015, at an IHS facility in the Southwest. We implemented algorithms that queried database values to assess chlamydia screening performance in 6 clinical departments. We presented data on the screening performance of clinical departments and health care providers (de-identified) in the awareness presentations. We re-queried database values 1 and 2 years after implementation of the EHR reminder intervention to evaluate before-and-after screening rates, comparing data among all patients and among female patients only.

RESULTS

We found small, sustained relative increases in chlamydia screening rates during the 2012-2015 evaluation period: 20.8% pre-intervention to 24.9% and 24.2% one and two years postintervention, respectively, across all patients; 32.3% preintervention to 36.6% and 35.6% one and two years postintervention, respectively, among female patients. Increases in clinical department-specific screening rates varied and were most prominent in internal medicine (35.8% preintervention to peak 65.8% postintervention). The 1 clinic (obstetrics-gynecology) that did not receive an awareness presentation showed a consistent downward trend in screening rates, although absolute rates were consistently higher in that clinic than in other clinics.

CONCLUSIONS

Awareness presentations that offer feedback to health care providers on screening performance, heighten provider awareness of the importance of chlamydia screening, and promote development of novel provider-initiated screening protocols may help to increase screening rates when combined with EHR reminders.

Machine Learning Approach to Reduce Alert Fatigue Using a Disease Medication-Related Clinical Decision Support System: Model Development and Validation

Background

Computerized physician order entry (CPOE) systems are incorporated into clinical decision support systems (CDSSs) to reduce medication errors and improve patient safety. Automatic alerts generated from CDSSs can directly assist physicians in making useful clinical decisions and can help shape prescribing behavior. Multiple studies reported that approximately 90%-96% of alerts are overridden by physicians, which raises questions about the effectiveness of CDSSs. There is intense interest in developing sophisticated methods to combat alert fatigue, but there is no consensus on the optimal approaches so far.

Objective

Our objective was to develop machine learning prediction models to predict physicians’ responses in order to reduce alert fatigue from disease medication–related CDSSs.

Methods

We collected data from a disease medication–related CDSS from a university teaching hospital in Taiwan. We considered prescriptions that triggered alerts in the CDSS between August 2018 and May 2019. Machine learning models, such as artificial neural network (ANN), random forest (RF), naïve Bayes (NB), gradient boosting (GB), and support vector machine (SVM), were used to develop prediction models. The data were randomly split into training (80%) and testing (20%) datasets.

Results

A total of 6453 prescriptions were used in our model. The ANN machine learning prediction model demonstrated excellent discrimination (area under the receiver operating characteristic curve [AUROC] 0.94; accuracy 0.85), whereas the RF, NB, GB, and SVM models had AUROCs of 0.93, 0.91, 0.91, and 0.80, respectively. The sensitivity and specificity of the ANN model were 0.87 and 0.83, respectively.

Conclusions

In this study, ANN showed substantially better performance in predicting individual physician responses to an alert from a disease medication–related CDSS, as compared to the other models. To our knowledge, this is the first study to use machine learning models to predict physician responses to alerts; furthermore, it can help to develop sophisticated CDSSs in real-world clinical settings.

Impact of Clinical Decision Support on Antibiotic Prescribing for Acute Respiratory Infections: a Cluster Randomized Implementation Trial

BACKGROUND

Clinical decision support (CDS) is a promising tool for reducing antibiotic prescribing for acute respiratory infections (ARIs).

OBJECTIVE

To assess the impact of previously effective CDS on antibiotic-prescribing rates for ARIs when adapted and implemented in diverse primary care settings.

DESIGN

Cluster randomized clinical trial (RCT) implementing a CDS tool designed to guide evidence-based evaluation and treatment of streptococcal pharyngitis and pneumonia.

SETTING

Two large academic health system primary care networks with a mix of providers.

PARTICIPANTS

All primary care practices within each health system were invited. All providers within participating clinic were considered a participant. Practices were randomized selection to a control or intervention group.

INTERVENTIONS

Intervention practice providers had access to an integrated clinical prediction rule (iCPR) system designed to determine the risk of bacterial infection from reason for visit of sore throat, cough, or upper respiratory infection and guide evidence-based evaluation and treatment.

MAIN OUTCOME(S)

Change in overall antibiotic prescription rates.

MEASURE(S)

Frequency, rates, and type of antibiotics prescribed in intervention and controls groups.

RESULTS

33 primary care practices participated with 541 providers and 100,573 patient visits. Intervention providers completed the tool in 6.9% of eligible visits. Antibiotics were prescribed in 35% and 36% of intervention and control visits, respectively, showing no statistically significant difference. There were also no differences in rates of orders for rapid streptococcal tests (RR, 0.94; P = 0.11) or chest X-rays (RR, 1.01; P = 0.999) between groups.

CONCLUSIONS

The iCPR tool was not effective in reducing antibiotic prescription rates for upper respiratory infections in diverse primary care settings. This has implications for the generalizability of CDS tools as they are adapted to heterogeneous clinical contexts.

TRIAL REGISTRATION

Clinicaltrials.gov (NCT02534987). Registered August 26, 2015 at https://clinicaltrials.gov.

Reducing Alert Fatigue by Sharing Low-Level Alerts With Patients and Enhancing Collaborative Decision Making Using Blockchain Technology: Scoping Review and Proposed Framework (MedAlert)

Background

Clinical decision support (CDS) is a tool that helps clinicians in decision making by generating clinical alerts to supplement their previous knowledge and experience. However, CDS generates a high volume of irrelevant alerts, resulting in alert fatigue among clinicians. Alert fatigue is the mental state of alerts consuming too much time and mental energy, which often results in relevant alerts being overridden unjustifiably, along with clinically irrelevant ones. Consequently, clinicians become less responsive to important alerts, which opens the door to medication errors.

Objective

This study aims to explore how a blockchain-based solution can reduce alert fatigue through collaborative alert sharing in the health sector, thus improving overall health care quality for both patients and clinicians.

Methods

We have designed a 4-step approach to answer this research question. First, we identified five potential challenges based on the published literature through a scoping review. Second, a framework is designed to reduce alert fatigue by addressing the identified challenges with different digital components. Third, an evaluation is made by comparing MedAlert with other proposed solutions. Finally, the limitations and future work are also discussed.

Results

Of the 341 academic papers collected, 8 were selected and analyzed. MedAlert securely distributes low-level (nonlife-threatening) clinical alerts to patients, enabling a collaborative clinical decision. Among the solutions in our framework, Hyperledger (private permissioned blockchain) and BankID (federated digital identity management) have been selected to overcome challenges such as data integrity, user identity, and privacy issues.

Conclusions

MedAlert can reduce alert fatigue by attracting the attention of patients and clinicians, instead of solely reducing the total number of alerts. MedAlert offers other advantages, such as ensuring a higher degree of patient privacy and faster transaction times compared with other frameworks. This framework may not be suitable for elderly patients who are not technology savvy or in-patients. Future work in validating this framework based on real health care scenarios is needed to provide the performance evaluations of MedAlert and thus gain support for the better development of this idea.

Applying population health approaches to improve safe anticoagulant use in the outpatient setting: the DOAC Dashboard multi-cohort implementation evaluation study protocol

BACKGROUND

Use of direct oral anticoagulants (DOAC) is rapidly growing for treatment of atrial fibrillation and venous thromboembolism. However, incorrect dosing of these medications is common and puts patients at risk of adverse drug events. One way to improve safe prescribing is the use of population health tools, including interactive dashboards built into the electronic health record (EHR). As such tools become more common, exploring ways to understand which aspects are effective in specific settings and how to effectively adapt and implement in existing anticoagulation clinics across different health systems is vital.

METHODS

This three-phase project will evaluate a current nation-wide implementation effort of the DOAC Dashboard in the Veterans Health Administration (VHA) using both quantitative and qualitative methods. Informed by this evaluation, the DOAC Dashboard will be implemented in four new health systems using an implementation strategy derived from the VHA experience and interviews with providers in those new health systems. Quantitative evaluation of the VHA and non-VHA implementation will follow the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework. Qualitative interviews with stakeholders will be analyzed using the Consolidated Framework for Implementation Research and Technology Acceptance Models to identify key determinants of implementation success.

DISCUSSION

This study will (1) evaluate the implementation of an EHR-based population health tool for medication management within a large, nation-wide, highly integrated health system; (2) guide the adoption in a set of four different health systems; and (3) evaluation that multi-center implementation effort. These findings will help to inform future EHR-based implementation efforts in a wide variety of health care settings.

Clinical Decision Support and Implications for the Clinician Burnout Crisis

OBJECTIVES

This survey aimed to review aspects of clinical decision support (CDS) that contribute to burnout and identify key themes for improving the acceptability of CDS to clinicians, with the goal of decreasing said burnout.

METHODS

We performed a survey of relevant articles from 2018-2019 addressing CDS and aspects of clinician burnout from PubMed and Web of Science™. Themes were manually extracted from publications that met inclusion criteria.

RESULTS

Eighty-nine articles met inclusion criteria, including 12 review articles. Review articles were either prescriptive, describing how CDS should work, or analytic, describing how current CDS tools are deployed. The non-review articles largely demonstrated poor relevance and acceptability of current tools, and few studies showed benefits in terms of efficiency or patient outcomes from implemented CDS. Encouragingly, multiple studies highlighted steps that succeeded in improving both acceptability and relevance of CDS.

CONCLUSIONS

CDS can contribute to clinician frustration and burnout. Using the techniques of improving relevance, soliciting feedback, customization, measurement of outcomes and metrics, and iteration, the effects of CDS on burnout can be ameliorated.

Evaluating the Impact of the Grading and Assessment of Predictive Tools Framework on Clinicians and Health Care Professionals' Decisions in Selecting Clinical Predictive Tools: Randomized Controlled Trial

BACKGROUND

While selecting predictive tools for implementation in clinical practice or for recommendation in clinical guidelines, clinicians and health care professionals are challenged with an overwhelming number of tools. Many of these tools have never been implemented or evaluated for comparative effectiveness. To overcome this challenge, the authors developed and validated an evidence-based framework for grading and assessment of predictive tools (the GRASP framework). This framework was based on the critical appraisal of the published evidence on such tools.

OBJECTIVE

The aim of the study was to examine the impact of using the GRASP framework on clinicians' and health care professionals' decisions in selecting clinical predictive tools.

METHODS

A controlled experiment was conducted through a web-based survey. Participants were randomized to either review the derivation publications, such as studies describing the development of the predictive tools, on common traumatic brain injury predictive tools (control group) or to review an evidence-based summary, where each tool had been graded and assessed using the GRASP framework (intervention group). Participants in both groups were asked to select the best tool based on the greatest validation or implementation. A wide group of international clinicians and health care professionals were invited to participate in the survey. Task completion time, rate of correct decisions, rate of objective versus subjective decisions, and level of decisional conflict were measured.

RESULTS

We received a total of 194 valid responses. In comparison with not using GRASP, using the framework significantly increased correct decisions by 64%, from 53.7% to 88.1% (88.1/53.7=1.64; t₁₉₃=8.53; P<.001); increased objective decision making by 32%, from 62% (3.11/5) to 82% (4.10/5; t₁₈₉=9.24; P<.001); decreased subjective decision making based on guessing by 20%, from 49% (2.48/5) to 39% (1.98/5; t₁₈₈=-5.47; P<.001); and decreased prior knowledge or experience by 8%, from 71% (3.55/5) to 65% (3.27/5; t₁₈₇=-2.99; P=.003). Using GRASP significantly decreased decisional conflict and increased the confidence and satisfaction of participants with their decisions by 11%, from 71% (3.55/5) to 79% (3.96/5; t₁₈₈=4.27; P<.001), and by 13%, from 70% (3.54/5) to 79% (3.99/5; t₁₈₈=4.89; P<.001), respectively. Using GRASP decreased the task completion time, on the 90th percentile, by 52%, from 12.4 to 6.4 min (t₁₉₃=-0.87; P=.38). The average System Usability Scale of the GRASP framework was very good: 72.5% and 88% (108/122) of the participants found the GRASP useful.

CONCLUSIONS

Using GRASP has positively supported and significantly improved evidence-based decision making. It has increased the accuracy and efficiency of selecting predictive tools. GRASP is not meant to be prescriptive; it represents a high-level approach and an effective, evidence-based, and comprehensive yet simple and feasible method to evaluate, compare, and select clinical predictive tools.

Is the Climb Worth the View? The Savings/Alert Ratio for Reducing Vitamin D Testing

BACKGROUND

Despite guideline recommendations, vitamin D testing has increased substantially. Clinical decision support (CDS) presents an opportunity to reduce inappropriate laboratory testing.

OBJECTIVES AND METHODS

To reduce inappropriate testing of vitamin D at the Vanderbilt University Medical Center, a CDS assigned providers to receive or not receive an electronic alert each time a 25-hydroxyvitamin D assay was ordered for an adult patient unless the order was associated with a diagnosis in the patient's chart for which vitamin D testing is recommended. The CDS ran for 80 days, collecting data on number of tests, provider information, and basic patient demographics.

RESULTS

During the 80 days, providers placed 12,368 orders for 25-hydroxyvitamin D. The intervention group ordered a vitamin D assay and received the alert for potentially inappropriate testing 2,181 times and completed the 25-hydroxyvitamin D order in 89.9% of encounters, while the control group ordered a vitamin D assay (without receiving an alert) 2,032 times and completed the order in 98.1% of encounters, for an absolute reduction of testing of 8% (p < 0.001).

CONCLUSION

This CDS reduced vitamin D ordering by utilizing a soft-stop approach. At a charge of $179.00 per test and a cost to the laboratory of $4.20 per test, each display of the alert led to an average reduction of $14.70 in charges and of $0.34 in spending by the laboratory (the savings/alert ratio). By describing the effectiveness of an electronic alert in terms of the savings/alert ratio, the impact of this intervention can be better appreciated and compared with other interventions.

Acute kidney injury risk-based screening in pediatric inpatients: a pragmatic randomized trial

BACKGROUND

Pediatric acute kidney injury (AKI) is common and associated with increased morbidity, mortality, and length of stay. We performed a pragmatic randomized trial testing the hypothesis that AKI risk alerts increase AKI screening.

METHODS

All intensive care and ward admissions of children aged 28 days through 21 years without chronic kidney disease from 12/6/2016 to 11/1/2017 were included. The intervention alert displayed if calculated AKI risk was > 50% and no serum creatinine (SCr) was ordered within 24 h. The primary outcome was SCr testing within 48 h of AKI risk > 50%.

RESULTS

Among intensive care admissions, 973/1909 (51%) were randomized to the intervention. Among those at risk, more SCr tests were ordered for the intervention group than for controls (418/606, 69% vs. 361/597, 60%, p = 0.002). AKI incidence and severity were the same in intervention and control groups. Among ward admissions, 5492/10997 (50%) were randomized to the intervention, and there were no differences between groups in SCr testing, AKI incidence, or severity of AKI.

CONCLUSIONS

Alerts based on real-time prediction of AKI risk increased screening rates in intensive care but not pediatric ward settings. Pragmatic clinical trials provide the opportunity to assess clinical decision support and potentially eliminate ineffective alerts.

Unlocking the Power of Artificial Intelligence and Big Data in Medicine

Data-driven science and its corollaries in machine learning and the wider field of artificial intelligence have the potential to drive important changes in medicine. However, medicine is not a science like any other: It is deeply and tightly bound with a large and wide network of legal, ethical, regulatory, economical, and societal dependencies. As a consequence, the scientific and technological progresses in handling information and its further processing and cross-linking for decision support and predictive systems must be accompanied by parallel changes in the global environment, with numerous stakeholders, including citizen and society. What can be seen at the first glance as a barrier and a mechanism slowing down the progression of data science must, however, be considered an important asset. Only global adoption can transform the potential of big data and artificial intelligence into an effective breakthroughs in handling health and medicine. This requires science and society, scientists and citizens, to progress together.

Clinical Pharmacist-Led Impact on Inappropriate Albumin Use and Costs in the Critically Ill

Background:Optimal albumin use in the intensive care unit (ICU) remains challenging with inappropriate use approaching 50%. No published reports have described clinical pharmacist impact aimed at mitigating inappropriate albumin use in the ICU. Objective: To evaluate the clinical and economic impact of a clinical pharmacist-led intervention strategy targeting inappropriate albumin in the ICU. Methods: A retrospective cohort study evaluated all adult (≥18 years) ICU patients administered albumin at an academic medical center over a 2-year period. Institutional guidelines were developed with clinical pharmacists targeting inappropriate albumin use. The primary end point was to compare inappropriate use of albumin administered before and after pharmacist intervention implementation. Secondary analyses compared the overall albumin use between study periods. In-hospital mortality, length of stay, and albumin-related costs between study periods were also compared. Results: A total of 4419 patients were identified, with 2448 (55.4%) critically ill patients included. The pharmacist-led strategy resulted in a 50.9% reduction of inappropriate albumin use (P < 0.001). The rate of inappropriate albumin use was 44.3 ± 10.5 and 5.5 ± 2.9 g per patient-day in the preimplementation and postimplementation periods, respectively (P < 0.001). Costs associated with overall and inappropriate albumin use in the ICU decreased by 34.8% and 87.1%, respectively. Total annual cost-savings was $355 393 in the ICUs. No differences in clinical outcomes were found. Conclusion and Relevance: Clinical pharmacist-led interventions reduced overall and inappropriate albumin use and costs without negatively affecting clinical outcomes.

Training Aspects of Laboratory-Based Decision Support

Pathology has a large role to play in the proper development, implementation, and optimization of clinical decision support (CDS). CDS training must be supported by an educational foundation in clinical and pathology informatics. Educational opportunities are currently limited, but expanding, in the pathology residency space with Pathology Informatics Essentials for Residents. The use of an educational version of electronic clinical systems is an important educational tool to support the needed outcomes-driven and exercise-based informatics and CDS training. With the multidisciplinary nature of informatics, it is advantageous to include laboratory professionals in the training exercises as appropriate.

Decision Support Tools within the Electronic Health Record

Laboratory tests are an integral part of the electronic health record (EHR). Providing clinical decision support (CDS) for the ordering, collection, reporting, viewing, and interpretation of laboratory testing is a fundamental function of the EHR. The implementation of a sustainable, effective laboratory CDS program requires a commitment to standardization and harmonization of the laboratory dictionaries that are the foundation of laboratory-based CDS. In this review, the authors provide an overview of the tools available within the EHR to improve decision making throughout the entire laboratory testing process, from test order to clinical action.

Using Machine Learning-Based Multianalyte Delta Checks to Detect Wrong Blood in Tube Errors

OBJECTIVES

An unfortunate reality of laboratory medicine is that blood specimens collected from one patient occasionally get mislabeled with identifiers from a different patient, resulting in so-called "wrong blood in tube" (WBIT) errors and potential patient harm. Here, we sought to develop a machine learning-based, multianalyte delta check algorithm to detect WBIT errors and mitigate patient harm.

METHODS

We simulated WBIT errors within sets of routine inpatient chemistry test results to develop, train, and evaluate five machine learning-based WBIT detection algorithms.

RESULTS

The best-performing WBIT detection algorithm we developed was based on a support vector machine and incorporated changes in test results between consecutive collections across 11 analytes. This algorithm achieved an area under the curve of 0.97 and considerably outperformed traditional single-analyte delta checks.

CONCLUSIONS

Machine learning-based multianalyte delta checks may offer a practical strategy to identify WBIT errors prior to test reporting and improve patient safety.