CDS in a Learning Health Care System: Identifying Physicians' Reasons for Rejection of Best-Practice Recommendations in Pneumonia through Computerized Clinical Decision Support

Effects of a Pharmacist-Designed Clinical Decision Support System on Antimicrobial Stewardship

BACKGROUND

 Clinical decision support systems (CDSSs) are computer applications, which can be applied to give guidance to practitioners in antimicrobial stewardship (AS) activities; however, further information is needed for their optimal use.

OBJECTIVES

 Our objective was to analyze the implementation of a CDSS program in a second-level hospital, describing alerts, recommendations, and the effects on consumption and clinical outcomes.

METHODS

 In October 2020, a pharmacist-driven CDSS designed for AS was implemented in a second-level hospital. The program provides a list of alerts related to antimicrobial treatment and microbiology, which were automatized for revision by the AS professionals. To analyze the implementation of the CDSS, a pre-post-intervention, retrospective study was designed. AS-triggered alerts and recommendations (total number and rate of acceptance) were compiled. The effect of the CDSS was measured using antimicrobial consumption, duration of antimicrobial treatments, in-hospital mortality, and length of stay (LOS) for patients admitted for infectious causes.

RESULTS

 The AS team revised a total of 7,543 alerts and 772 patients had at least one recommendation, with an acceptance rate of 79.3%. Antimicrobial consumption decreased from 691.1 to 656.8 defined daily doses (DDD)/1,000 beds-month (p = 0.04) and the duration of antimicrobial treatment from 3.6 to 3.3 days (p < 0.01). In-hospital mortality decreased from 6.6 to 6.2% (p = 0.46) and mean LOS from 7.2 to 6.2 days (p < 0.01).

CONCLUSION

 The implementation of a CDSS resulted in a significant reduction of antimicrobial DDD, duration of antimicrobial treatments, and hospital LOS. There was no significant difference in mortality.

Learning health systems on the front lines to strengthen care against future pandemics and climate change: a rapid review

BACKGROUND

An essential component of future-proofing health systems against future pandemics and climate change is strengthening the front lines of care: principally, emergency departments and primary care settings. To achieve this, these settings can adopt learning health system (LHS) principles, integrating data, evidence, and experience to continuously improve care delivery. This rapid review aimed to understand the ways in which LHS principles have been applied to primary care and emergency departments, the extent to which LHS approaches have been adopted in these key settings, and the factors that affect their adoption.

METHODS

Three academic databases (Embase, Scopus, and PubMed) were searched for full text articles reporting on LHSs in primary care and/or emergency departments published in the last five years. Articles were included if they had a primary focus on LHSs in primary care settings (general practice, allied health, multidisciplinary primary care, and community-based care) and/or emergency care settings. Data from included articles were catalogued and synthesised according to the modified Institute of Medicine's five-component framework for LHSs (science and informatics, patient-clinician partnerships, incentives, continuous learning culture, and structure and governance).

RESULTS

Thirty-seven articles were included, 32 of which reported LHSs in primary care settings and seven of which reported LHSs in emergency departments. Science and informatics was the most commonly reported LHS component, followed closely by continuous learning culture and structure and governance. Most articles (n = 30) reported on LHSs that had been adopted, and many of the included articles (n = 17) were descriptive reports of LHS approaches.

CONCLUSIONS

Developing LHSs at the front lines of care is essential for future-proofing against current and new threats to health system sustainability, such as pandemic- and climate change-induced events. Limited research has examined the application of LHS concepts to emergency care settings. Implementation science should be utilised to better understand the factors influencing adoption of LHS approaches on the front lines of care, so that all five LHS components can be progressed in these settings.

Factors influencing clinician and patient interaction with machine learning-based risk prediction models: a systematic review

Machine learning (ML)-based risk prediction models hold the potential to support the health-care setting in several ways; however, use of such models is scarce. We aimed to review health-care professional (HCP) and patient perceptions of ML risk prediction models in published literature, to inform future risk prediction model development. Following database and citation searches, we identified 41 articles suitable for inclusion. Article quality varied with qualitative studies performing strongest. Overall, perceptions of ML risk prediction models were positive. HCPs and patients considered that models have the potential to add benefit in the health-care setting. However, reservations remain; for example, concerns regarding data quality for model development and fears of unintended consequences following ML model use. We identified that public views regarding these models might be more negative than HCPs and that concerns (eg, extra demands on workload) were not always borne out in practice. Conclusions are tempered by the low number of patient and public studies, the absence of participant ethnic diversity, and variation in article quality. We identified gaps in knowledge (particularly views from under-represented groups) and optimum methods for model explanation and alerts, which require future research.

Evaluation of an Antimicrobial Stewardship Decision Support for Pediatric Infections

OBJECTIVES

Clinical decision support (CDS) has promise for the implementation of antimicrobial stewardship programs (ASPs) in the emergency department (ED). We sought to assess the usability of a newly developed automated CDS to improve guideline-adherent antibiotic prescribing for pediatric community-acquired pneumonia (CAP) and urinary tract infection (UTI).

METHODS

We conducted comparative usability testing between an automated, prototype CDS-enhanced discharge order set and standard order set, for pediatric CAP and UTI antibiotic prescribing. After an extensive user-centered design process, the prototype CDS was integrated into the electronic health record, used passive activation, and embedded locally adapted prescribing guidelines. Participants were randomized to interact with three simulated ED scenarios of children with CAP or UTI, across both systems. Measures included task completion, decision-making and usability errors, clinical actions (order set use and correct antibiotic selection), as well as objective measures of system usability, utility, and workload using the National Aeronautics and Space Administration Task Load Index (NASA-TLX). The prototype CDS was iteratively refined to optimize usability and workflow.

RESULTS

Usability testing in 21 ED clinical providers demonstrated that, compared to the standard order sets, providers preferred the prototype CDS, with improvements in domains such as explanations of suggested antibiotic choices (p < 0.001) and provision of additional resources on antibiotic prescription (p < 0.001). Simulated use of the CDS also led to overall improved guideline-adherent prescribing, with a 31% improvement for CAP. A trend was present toward absolute workload reduction. Using the NASA-TLX, workload scores for the current system were median 26, interquartile ranges (IQR): 11 to 41 versus median 25, and IQR: 10.5 to 39.5 for the CDS system (p = 0.117).

CONCLUSION

Our CDS-enhanced discharge order set for ED antibiotic prescribing was strongly preferred by users, improved the accuracy of antibiotic prescribing, and trended toward reduced provider workload. The CDS was optimized for impact on guideline-adherent antibiotic prescribing from the ED and end-user acceptability to support future evaluative trials of ED ASPs.

Real-Time User Feedback to Support Clinical Decision Support System Improvement

OBJECTIVES

To improve clinical decision support (CDS) by allowing users to provide real-time feedback when they interact with CDS tools and by creating processes for responding to and acting on this feedback.

METHODS

Two organizations implemented similar real-time feedback tools and processes in their electronic health record and gathered data over a 30-month period. At both sites, users could provide feedback by using Likert feedback links embedded in all end-user facing alerts, with results stored outside the electronic health record, and provide feedback as a comment when they overrode an alert. Both systems are monitored daily by clinical informatics teams.

RESULTS

The two sites received 2,639 Likert feedback comments and 623,270 override comments over a 30-month period. Through four case studies, we describe our use of end-user feedback to rapidly respond to build errors, as well as identifying inaccurate knowledge management, user-interface issues, and unique workflows.

CONCLUSION

Feedback on CDS tools can be solicited in multiple ways, and it contains valuable and actionable suggestions to improve CDS alerts. Additionally, end users appreciate knowing their feedback is being received and may also make other suggestions to improve the electronic health record. Incorporation of end-user feedback into CDS monitoring, evaluation, and remediation is a way to improve CDS.

A Pragmatic, Stepped-Wedge, Cluster-controlled Clinical Trial of Real-Time Pneumonia Clinical Decision Support

Rationale: Care of emergency department (ED) patients with pneumonia can be challenging. Clinical decision support may decrease unnecessary variation and improve care. Objectives: To report patient outcomes and processes of care after deployment of electronic pneumonia clinical decision support (ePNa): a comprehensive, open loop, real-time clinical decision support embedded within the electronic health record. Methods: We conducted a pragmatic, stepped-wedge, cluster-controlled trial with deployment at 2-month intervals in 16 community hospitals. ePNa extracts real-time and historical data to guide diagnosis, risk stratification, microbiological studies, site of care, and antibiotic therapy. We included all adult ED patients with pneumonia over the course of 3 years identified by International Classification of Diseases, 10th Revision discharge coding confirmed by chest imaging. Measurements and Main Results: The median age of the 6,848 patients was 67 years (interquartile range, 50-79), and 48% were female; 64.8% were hospital admitted. Unadjusted mortality was 8.6% before and 4.8% after deployment. A mixed effects logistic regression model adjusting for severity of illness with hospital cluster as the random effect showed an adjusted odds ratio of 0.62 (0.49-0.79; P < 0.001) for 30-day all-cause mortality after deployment. Lower mortality was consistent across hospital clusters. ePNa-concordant antibiotic prescribing increased from 83.5% to 90.2% (P < 0.001). The mean time from ED admission to first antibiotic was 159.4 (156.9-161.9) minutes at baseline and 150.9 (144.1-157.8) minutes after deployment (P < 0.001). Outpatient disposition from the ED increased from 29.2% to 46.9%, whereas 7-day secondary hospital admission was unchanged (5.2% vs. 6.1%). ePNa was used by ED clinicians in 67% of eligible patients. Conclusions: ePNa deployment was associated with improved processes of care and lower mortality. Clinical trial registered with www.clinicaltrials.gov (NCT03358342).

The development and implementation of a guideline-based clinical decision support system to improve empirical antibiotic prescribing

BACKGROUND

To describe and evaluate a clinical decision support system (CDSS) for empirical antibiotic therapy using a systematic framework.

METHODS

A reporting framework for behavior change intervention implementation was used, which includes several domains: development, evaluation and implementation. Within the development domain a description is given of the engagement of stakeholders, a rationale for how the CDSS may influence antibiotic prescribing and a detailed outline of how the system was developed. Within the evaluation domain a technical validation is performed and the interaction between potential users and the CDSS is analyzed. Within the domain of implementation a description is given on how the CDSS was tested in the real world and the strategies that were used for implementation and adoption of the CDSS.

RESULTS

Development: a CDSS was developed, with the involvement of stakeholders, to assist empirical antibiotic prescribing by physicians.

EVALUATION

Technical problems were determined during the validation process and corrected in a new CDSS version. A usability study was performed to assess problems in the system-user interaction.

IMPLEMENTATION

In 114 patients the antibiotic advice that was generated by the CDSS was followed. For 54 patients the recommendations were not adhered to.

CONCLUSIONS

This study describes the development and validation of a CDSS for empirical antibiotic therapy and shows the usefulness of the systematic framework for reporting CDSS interventions. In addition it shows that CDSS recommendations are not always adhered to which is associated with incorrect use of the system.

How to use new antibiotics in the therapy of ventilator-associated pneumonia

PURPOSE OF REVIEW

Ventilator-associated pneumonia (VAP) is a common nosocomial infection in critically ill patients requiring endotracheal intubation and mechanical ventilation. Recently, the emergence of multidrug-resistant Gram-negative bacteria, including carbapenem-resistant Enterobacterales, multidrug-resistant Pseudomonas aeruginosa and Acinetobacter species, has complicated the selection of appropriate antimicrobials and contributed to treatment failure. Although novel antimicrobials are crucial to treating VAP caused by these multidrug-resistant organisms, knowledge of how to optimize their efficacy while minimizing the development of resistance should be a requirement for their use.

RECENT FINDINGS

Several studies have assessed the efficacy of novel antimicrobials against multidrug-resistant organisms, but high-quality studies focusing on optimal dosing, infusion time and duration of therapy in patients with VAP are still lacking. Antimicrobial and diagnostic stewardship should be combined to optimize the use of these novel agents.

SUMMARY

Improvements in diagnostic tests, stewardship practices and a better understanding of dosing, infusion time, duration of treatment and the effects of combining various antimicrobials should help optimize the use of novel antimicrobials for VAP and maximize clinical outcomes while minimizing the development of resistance.

Deploying an Electronic Clinical Decision Support Tool for Diagnosis and Treatment of Pneumonia Into Rural and Critical Access Hospitals: Utilization, Effect on Processes of Care, and Clinician Satisfaction

PURPOSE

Electronic clinical decision support (CDS) for treatment of community-acquired pneumonia (ePNa) is associated with improved guideline adherence and decreased mortality. How rural providers respond to CDS developed for urban hospitals could shed light on extending CDS to resource-limited settings.

METHODS

ePNa was deployed into 10 rural and critical access hospital emergency departments (EDs) in Utah and Idaho in 2018. We reviewed pneumonia cases identified through ICD-10 codes after local deployment to measure ePNa utilization and guideline adherence. ED providers were surveyed to assess quantitative and qualitative aspects of satisfaction.

FINDINGS

ePNa was used in 109/301 patients with pneumonia (36%, range 0%-67% across hospitals) and was associated with appropriate antibiotic selection (93% vs 65%, P < .001). Fifty percent of survey recipients responded, 87% were physicians, 87% were men, and the median ED experience was 10 years. Mean satisfaction with ePNa was 3.3 (range 1.7-4.8) on a 5-point Likert scale. Providers with a favorable opinion of ePNa were more likely to favor implementation of additional CDS (P = .005). Satisfaction was not associated with provider type, age, years of experience or experience with ePNa. Ninety percent of respondents provided qualitative feedback. The most common theme in high and low utilization hospitals was concern about usability. Compared to high utilization hospitals, low utilization hospitals more frequently identified concerns about adaptation for local needs.

CONCLUSIONS

ePNa deployment to rural and critical access EDs was moderately successful and associated with improved antibiotic use. Concerns about usability and adapting ePNa for local use predominated the qualitative feedback.

A Randomized Controlled Trial of an Electronic Clinical Decision Support Tool for Inpatient Antimicrobial Stewardship

BACKGROUND

The weighted incidence syndromic combination antibiogram (WISCA) is an antimicrobial stewardship tool that utilizes electronic medical record data to provide real-time clinical decision support regarding empiric antibiotic prescription in the hospital setting. The aim of this study was to determine the impact of WISCA utilization for empiric antibiotic prescription on hospital length of stay (LOS).

METHODS

We performed a crossover randomized controlled trial of the WISCA tool at 4 hospitals. Study participants included adult inpatients receiving empiric antibiotics for urinary tract infection (UTI), abdominal-biliary infection (ABI), pneumonia, or nonpurulent cellulitis. Antimicrobial stewardship (ASP) physicians utilized WISCA and clinical guidelines to provide empiric antibiotic recommendations. The primary outcome was LOS. Secondary outcomes included 30-day mortality, 30-day readmission, Clostridioides difficile infection, acquisition of multidrug-resistant gram-negative organism (MDRO), and antibiotics costs.

RESULTS

In total, 6849 participants enrolled in the study. There were no overall differences in outcomes among the intervention versus control groups. Participants with cellulitis in the intervention group had significantly shorter mean LOS compared to participants with cellulitis in the control group (coefficient estimate = 0.53 [-0.97, -0.09], P = .0186). For patients with community acquired pneumonia (CAP), the intervention group had significantly lower odds of 30-day mortality compared to the control group (adjusted odds ratio [aOR] .58, 95% confidence interval [CI], .396, .854, P = .02).

CONCLUSIONS

Use of WISCA was not associated with improved outcomes for UTI and ABI. Guidelines-based interventions were associated with decreased LOS for cellulitis and decreased mortality for CAP.

Development and Internal Validation of a Prediction Model to Risk Stratify Children with Suspected Community-Acquired Pneumonia

BACKGROUND

Although community-acquired pneumonia (CAP) is one of the most common infections in children, no tools exist to risk stratify children with suspected CAP. We developed and validated a prediction model to risk stratify and inform hospitalization decisions in children with suspected CAP.

METHODS

We performed a prospective cohort study of children age 3 months to 18 years with suspected CAP in a pediatric emergency department (ED). Primary outcome was disease severity, defined as mild (discharge home or hospitalization for <24 hours with no oxygen or intravenous (IV) fluids), moderate (hospitalization <24 hours with oxygen or IV fluids, or hospitalization >24 hours), or severe (intensive care unit (ICU) stay for >24 hours, septic shock, vasoactive agents, positive-pressure ventilation, chest drainage, extracorporeal membrane oxygenation, or death). Ordinal logistic regression and bootstrapped backwards selection were used to derive and internally validate our model.

RESULTS

Of 1128 children, 371 (32.9%) developed moderate disease and 48 (4.3%) severe disease. Severity models demonstrated excellent discrimination (optimism-corrected c-indices of 0.81) and outstanding calibration. Severity predictors in the final model included respiratory rate, systolic blood pressure, oxygenation, retractions, capillary refill, atelectasis or pneumonia on chest radiograph, and pleural effusion.

CONCLUSIONS

We derived and internally validated a score that accurately predicts disease severity in children with suspected CAP. Once externally validated, this score has potential to facilitate management decisions by providing individualized risk estimates that can be used in conjunction with clinical judgment to improve the care of children with suspected CAP.

Implementation of Real-Time Electronic Clinical Decision Support for Emergency Department Patients with Pneumonia Across a Healthcare System

A real-time electronic CDS for pneumonia (ePNa) identifies possible pneumonia patients, measures severity and antimicrobial resistance risk, and then recommends disposition, antibiotics, and microbiology studies. Use is voluntary, and clinicians may modify treatment recommendations. ePNa was associated with lower mortality in emergency department (ED) patients versus usual care (Annals EM 66:511). We adapted ePNa for the Cerner EHR, and implemented it across Intermountain Healthcare EDs (Utah, USA) throughout 2018. We introduced ePNa through didactic, interactive presentations to ED clinicians; follow-up visits identified barriers and facilitators to use. Email reminded clinicians and answered questions. Hospital admitting clinicians encouraged ePNa use to smooth care transitions. Audit-and-feedback measured utilization, showing variations from best practice when ePNa and associated electronic order sets were not used. Use was initially low, but gradually increased especially at larger hospitals. A user-friendly interface, frequent reminders, audit-and- feedback, a user survey, a nurse educator, and local physician champions are additive towards implementation success.

Towards a Maturity Model for Clinical Decision Support Operations

Clinical decision support (CDS) systems delivered through the electronic health record are an important element of quality and safety initiatives within a health care system. However, managing a large CDS knowledge base can be an overwhelming task for informatics teams. Additionally, it can be difficult for these informatics teams to communicate their goals with external operational stakeholders and define concrete steps for improvement. We aimed to develop a maturity model that describes a roadmap toward organizational functions and processes that help health care systems use CDS more effectively to drive better outcomes. We developed a maturity model for CDS operations through discussions with health care leaders at 80 organizations, iterative model development by four clinical informaticists, and subsequent review with 19 health care organizations. We ceased iterations when feedback from three organizations did not result in any changes to the model. The proposed CDS maturity model includes three main "pillars": "Content Creation," "Analytics and Reporting," and "Governance and Management." Each pillar contains five levels-advancing along each pillar provides CDS teams a deeper understanding of the processes CDS systems are intended to improve. A "roof" represents the CDS functions that become attainable after advancing along each of the pillars. Organizations are not required to advance in order and can develop in one pillar separately from another. However, we hypothesize that optimal deployment of preceding levels and advancing in tandem along the pillars increase the value of organizational investment in higher levels of CDS maturity. In addition to describing the maturity model and its development, we also provide three case studies of health care organizations using the model for self-assessment and determine next steps in CDS development.

Clinical Decision Support Systems and Their Role in Antibiotic Stewardship: a Systematic Review

PURPOSE OF REVIEW

The purpose of this article is to perform a systematic review over the past 5 years on the role and effectiveness of clinical decision support systems (CDSSs) on antibiotic stewardship.

RECENT FINDINGS

CDDS interventions found a significant impact on multiple outcomes relevant to antibiotic stewardship. There are various types of CDSS implementations, both active and passive (provider initiated). Passive interventions were associated with more significant outcomes; however, both interventions appeared effective. In the reviewed literature, CDSSs were consistently associated with decreasing antibiotic consumption and narrowing the spectrum of antibiotic usage. Generally, guideline adherence was improved with CDSS, although this was not universal. The effect on other outcomes, such as mortality, Clostridiodes difficile infections, length of stay, and cost, inconsistently showed a significant difference. Overall, CDDS implementation has effectively decreased antibiotic consumption and improved guideline adherence across the various types of CDSS. Other positive outcomes were noted in certain settings, but were not universal. When creating a new intervention, it is important to identify the optimal structure and deployment of a CDSS for a specific setting.

Physicians Voluntarily Using an EHR-Based CDS Tool Improved Patients' Guideline-Related Statin Prescription Rates: A Retrospective Cohort Study

BACKGROUND

In 2013, the American College of Cardiology (ACC) and the American Heart Association (AHA) released a revised guideline on statin therapy initiation. The guideline included a 10-year risk calculation based on regression modeling, which made hand calculation infeasible. Compliance to the guideline has been suboptimal, as many patients were recommended but not prescribed statin therapy. Clinical decision support (CDS) tools may improve statin guideline compliance. Few statin guideline CDS tools evaluated clinical outcome.

OBJECTIVES

We determined if use of a CDS tool, the statin macro, was associated with increased 2013 ACC/AHA statin guideline compliance at the level of statin prescription versus no statin prescription. We did not determine if each patient's statin prescription met ACC/AHA 2013 therapy intensity recommendations (high vs. moderate vs. low).

METHODS

The authors developed a clinician-initiated, EHR-embedded statin macro command ("statin macro") that displayed the 2013 ACC/AHA statin guideline recommendation in the electronic health record documentation. We included patients who had a primary care visit during the study period (January 1-June 30, 2016), were eligible for statin therapy based on the ACC/AHA guideline prior to the study period, and were not prescribed statin therapy prior to the study period. We tested the association of macro usage and statin therapy prescription during the study period using relative risk and mixed effect logistic regression.

RESULTS

Subjects included 11,877 patients seen in primary care, who were retrospectively recommended statin therapy at study initiation based on the ACC/AHA guideline, but who had not received statin therapy. During the study period, 125 clinicians used the statin macro command for 389 of the 11,877 patients (3.2%). Of the 389 patients for whom that statin macro was used, 108 patients (28%) had a statin prescribed during the study period. Of the 11,488 for whom the statin macro was not used, 1,360 (13%) patients received a clinician-prescribed statin (relative risk 2.3, p < 0.001). Controlling for patient covariates and clinicians, statin macro usage was significantly associated with statin therapy prescription (odds ratio 2.86, p < 0.001).

CONCLUSION

Although the statin macro had low uptake, its use was associated with a greater rate of statin prescriptions (dosage not determined) for patients whom 2013 ACC/AHA guidelines required statin therapy.