Interruptive Versus Noninterruptive Clinical Decision Support: Usability Study

Barriers, facilitators, and recommendations to increase the use of a clinical decision support tool for managing chronic pain in primary care

BACKGROUND AND OBJECTIVE

Primary care providers (PCPs) use poorly organized patient information in electronic health records (EHR) within a limited time when treating patients with chronic pain. Clinical decision support (CDS) tools assist PCPs by synthesizing patient information and prompting guideline-concordant treatment decisions. A CDS tool- Chronic Pain OneSheet was developed through a user-centered design process to support PCP's decision-making for patients with chronic noncancer pain. OneSheet aggregates relevant patient information in one place in the EHR. OneSheet also guides PCPs in completing guideline-recommended opioid risk management tasks, tracking patient treatments, and documenting pain-related symptoms. Our objective was to identify barriers, facilitators, and recommendations to increase OneSheet use for chronic noncancer pain management in primary care.

METHODS

We conducted 19 qualitative interviews with PCPs from two academic health systems who had access to OneSheet in their EHR. Interview transcripts were coded to identify common themes using a modified thematic approach.

RESULTS

PCPs identified several barriers to using OneSheet, including limited time to address patient needs associated with multiple chronic conditions, resistance to changing established workflows, and complex OneSheet display. PCPs reported several facilitators to using OneSheet, such as OneSheet's ability to serve as a hub for chronic pain data, easy access to features that facilitate completing mandatory tasks and improved planning for certain patient visits. PCPs recommended prioritizing access to commonly used features, adding display customization capabilities, and expanding access to patients and other team members to increase OneSheet use.

CONCLUSION

Our findings highlight the importance of acknowledging the PCP workflow and task load when designing CDS tools. Future CDS tools should balance the extent of information provided with assisting PCPs to fulfill mandatory tasks. Expanding CDS tools to multiple care team members and patients can also lead to higher use by facilitating data entry, leading to more streamlined care delivery.

Development and validation of a clinical decision support system to prevent anticoagulant duplications

BACKGROUND AND OBJECTIVE

Unintended duplicate prescriptions of anticoagulants increase the risk of serious adverse events. Clinical Decision Support Systems (CDSSs) can help prevent such medication errors; however, sophisticated algorithms are needed to avoid alert fatigue. This article describes the steps taken in our hospital to develop a CDSS to prevent anticoagulant duplication (AD).

METHODS

The project was composed of three phases. In phase I, the status quo was established. In phase II, a clinical pharmacist developed an algorithm to detect ADs using daily data exports. In phase III, the algorithm was integrated into the hospital's electronic health record system. Alerts were reviewed by clinical pharmacists before being sent to the prescribing physician. We conducted a retrospective analysis of all three phases to assess the impact of the interventions on the occurrence and duration of ADs. Phase III was analyzed in more detail regarding the acceptance rate, sensitivity, and specificity of the alerts.

RESULTS

We identified 91 ADs in 1581 patients receiving two or more anticoagulants during phase I, 70 ADs in 1692 patients in phase II, and 57 ADs in 1575 patients in phase III. Mean durations of ADs were 1.8, 1.4, and 1.1 calendar days during phases I, II, and III, respectively. In comparison to the baseline in phase I, the relative risk reduction of AD in patients treated with at least two different anticoagulants during phase III was 42% (RR: 0.58, CI: 0.42-0.81). A total of 429 alerts were generated during phase III, many of which were self-limiting, and 186 alerts were sent to the respective prescribing physician. The acceptance rate was high at 97%. We calculated a sensitivity of 87.4% and a specificity of 87.9%.

CONCLUSION

The stepwise development of a CDSS for the detection of AD markedly reduced the frequency and duration of medication errors in our hospital, thereby improving patient safety.

Randomized Controlled Comparative Effectiveness Trial of Risk Model-Guided Clinical Decision Support for Suicide Screening

Suicide prevention requires risk identification, appropriate intervention, and follow-up. Traditional risk identification relies on patient self-reporting, support network reporting, or face-to-face screening with validated instruments or history and physical exam. In the last decade, statistical risk models have been studied and more recently deployed to augment clinical judgment. Models have generally been found to be low precision or problematic at scale due to low incidence. Few have been tested in clinical practice, and none have been tested in clinical trials to our knowledge.

Methods

We report the results of a pragmatic randomized controlled trial (RCT) in three outpatient adult Neurology clinic settings. This two-arm trial compared the effectiveness of Interruptive and Non-Interruptive Clinical Decision Support (CDS) to prompt further screening of suicidal ideation for those predicted to be high risk using a real-time, validated statistical risk model of suicide attempt risk, with the decision to screen as the primary end point. Secondary outcomes included rates of suicidal ideation and attempts in both arms. Manual chart review of every trial encounter was used to determine if suicide risk assessment was subsequently documented.

Results

From August 16, 2022, through February 16, 2023, our study randomized 596 patient encounters across 561 patients for providers to receive either Interruptive or Non-Interruptive CDS in a 1:1 ratio. Adjusting for provider cluster effects, Interruptive CDS led to significantly higher numbers of decisions to screen (42%=121/289 encounters) compared to Non-Interruptive CDS (4%=12/307) (odds ratio=17.7, p-value <0.001). Secondarily, no documented episodes of suicidal ideation or attempts occurred in either arm. While the proportion of documented assessments among those noting the decision to screen was higher for providers in the Non-Interruptive arm (92%=11/12) than in the Interruptive arm (52%=63/121), the interruptive CDS was associated with more frequent documentation of suicide risk assessment (63/289 encounters compared to 11/307, p-value<0.001).

Conclusions

In this pragmatic RCT of real-time predictive CDS to guide suicide risk assessment, Interruptive CDS led to higher numbers of decisions to screen and documented suicide risk assessments. Well-powered large-scale trials randomizing this type of CDS compared to standard of care are indicated to measure effectiveness in reducing suicidal self-harm. ClinicalTrials.gov Identifier: NCT05312437.

Impact of a Clinical Decision Support Alert on Informed Consent Documentation in the Neonatal Intensive Care Unit

Background

Informed consent is necessary to preserve patient autonomy and shared decision-making, yet compliant consent documentation is suboptimal in the intensive care unit (ICU). We aimed to increase compliance with bundled consent documentation, which provides consent for a predefined set of common procedures in the neonatal ICU from 0% to 50% over 1 year.

Methods

We used the Plan-Do-Study-Act model for quality improvement. Interventions included education and performance awareness, delineation of the preferred consenting process, consent form revision, overlay tool creation, and clinical decision support (CDS) alert use within the electronic health record. Monthly audits categorized consent forms as missing, present but noncompliant, or compliant. We analyzed consent compliance on a run chart using standard run chart interpretation rules and obtained feedback on the CDS as a countermeasure.

Results

We conducted 564 audits over 37 months. Overall, median consent compliance increased from 0% to 86.6%. Upon initiating the CDS alert, we observed the highest monthly compliance of 93.3%, followed by a decrease to 33.3% with an inadvertent discontinuation of the CDS. Compliance subsequently increased to 73.3% after the restoration of the alert. We created a consultant opt-out selection to address negative feedback associated with CDS. There were no missing consent forms within the last 7 months of monitoring.

Conclusions

A multi-faceted approach led to sustained improvement in bundled consent documentation compliance in our neonatal intensive care unit, with the direct contribution of the CDS observed. A CDS intervention directed at the informed consenting process may similarly benefit other ICUs.

Tackling alert fatigue with a semi-automated clinical decision support system: quantitative evaluation and end-user survey

STUDY AIMS

Clinical decision support systems (CDSS) embedded in hospital electronic health records efficiently reduce medication errors, but there is a risk of low physician adherence due to alert fatigue. At the Cantonal Hospital Aarau, a CDSS is being developed that allows the highly accurate detection and correction of medication errors. The semi-automated CDSS sends its alerts either directly to the physician or to a clinical pharmacist for review first. Our aim was to evaluate the performance of the recently implemented CDSS in terms of acceptance rate and alert burden, as well as physicians' satisfaction with the CDSS.

METHODS

All alerts generated by the clinical decision support systems between January and December 2021 were included in a retrospective quantitative evaluation. A team of clinical pharmacists performed a follow-up to determine whether the recommendation made by the CDSS was implemented by the physician. The acceptance rate was calculated including all alerts for which it was possible to determine an outcome. A web-based survey was conducted amongst physicians to assess their attitude towards the CDSS. The survey questions included overall satisfaction, helpfulness of individual algorithms, and perceived alert burden.

RESULTS

In 2021, a total of 10,556 alerts were generated, of which 619 triggered a direct notification to the physician and 2,231 notifications were send to the physician after evaluation by a clinical pharmacist. The acceptance rates were 89.8% and 68.4%, respectively, which translates as an overall acceptance rate of 72.4%. On average, clinical pharmacists received 17.2 alerts per day, while all of the hospital physicians together received 7.8 notifications per day. In the survey, 94.5% of physicians reported being satisfied or very satisfied with the CDSS. Algorithms addressing potential medication errors concerning anticoagulants received the highest usefulness ratings.

CONCLUSION

The development of this semi-automated clinical decision support system with context-based algorithms resulted in alerts with a high acceptance rate. Involving clinical pharmacists proved a promising approach to limit the alert burden of physicians and thus tackle alert fatigue. The CDSS is well accepted by our physicians.

Meaningful time-related aspects of alerts in Clinical Decision Support Systems. A unified framework

Alerts are a common functionality of clinical decision support systems (CDSSs). Although they have proven to be useful in clinical practice, the alert burden can lead to alert fatigue and significantly reduce their usability and acceptance. Based on a literature review, we propose a unified framework consisting of a set of meaningful timestamps that allows the use of state-of-the-art measures for alert burden, such as alert dwell time, alert think time, and response time. In addition, it can be used to investigate other measures that could be relevant as regards dealing with this problem. Furthermore, we provide a case study concerning three different types of alerts to which the framework was successfully applied. We consider that our framework can easily be adapted to other CDSSs and that it could be useful for dealing with alert burden measurement thus contributing to its appropriate management.

The effectiveness of a noninterruptive alert to increase prescription of take-home naloxone in emergency departments

OBJECTIVE

Opioid-related overdose (OD) deaths continue to increase. Take-home naloxone (THN), after treatment for an OD in an emergency department (ED), is a recommended but under-utilized practice. To promote THN prescription, we developed a noninterruptive decision support intervention that combined a detailed OD documentation template with a reminder to use the template that is automatically inserted into a provider's note by decision rules. We studied the impact of the combined intervention on THN prescribing in a longitudinal observational study.

METHODS

ED encounters involving an OD were reviewed before and after implementation of the reminder embedded in the physicians' note to use an advanced OD documentation template for changes in: (1) use of the template and (2) prescription of THN. Chi square tests and interrupted time series analyses were used to assess the impact. Usability and satisfaction were measured using the System Usability Scale (SUS) and the Net Promoter Score.

RESULTS

In 736 OD cases defined by International Classification of Disease version 10 diagnosis codes (247 prereminder and 489 postreminder), the documentation template was used in 0.0% and 21.3%, respectively (P < .0001). The sensitivity and specificity of the reminder for OD cases were 95.9% and 99.8%, respectively. Use of the documentation template led to twice the rate of prescribing of THN (25.7% vs 50.0%, P < .001). Of 19 providers responding to the survey, 74% of SUS responses were in the good-to-excellent range and 53% of providers were Net Promoters.

CONCLUSIONS

A noninterruptive decision support intervention was associated with higher THN prescribing in a pre-post study across a multiinstitution health system.

Novel Note Templates to Enhance Signal and Reduce Noise in Medical Documentation: a Prospective Improvement Study

BACKGROUND

The introduction of electronic workflows has allowed for the flow of raw un-contextualized clinical data into medical documentation. As a result, many electronic notes have become replete of "noise" and deplete of clinically significant "signals". There is an urgent need to develop and implement innovative approaches in electronic clinical documentation that improve note quality and reduce unnecessary bloating.

OBJECTIVE

To describe the development and impact of a novel set of templates designed to change the flow of information in medical documentation.

METHODS

This is a multi-hospital nonrandomized prospective improvement study conducted on the Inpatient General Internal Medicine Service across three hospital campuses at the New York University (NYU) Langone Health System. A group of physician leaders representing each campus met biweekly for six months. The output of these meetings included 1) a conceptualization of the note bloat problem as a dysfunction in information flow 2) a set of guiding principles for organizational documentation improvement 3) the design and build of novel electronic templates that reduced the flow of extraneous information into provider notes by providing link outs to best practice data visualizations and 4) a documentation improvement curriculum for inpatient medicine providers. Prior to go-live, pragmatic usability testing was performed with the new progress note template, and the overall user experience measured using the System Usability Scale (SUS). Primary outcomes measures after go-live include template utilization rate and note length in characters.

RESULTS

In usability testing amongst 22 medicine providers, the new progress note template averaged a usability score of 90.6/100 on the System Usability Scale. 77% of providers strongly agreed that the new template was easy to use. 68% strongly agreed that they would like to use the template frequently. In the three months after template implementation, General Internal Medicine providers wrote 65% of all inpatient notes with the new templates. During this period of time the organization saw a 46%, 47%, and 32% reduction in note length for general medicine progress notes, consults, and H&Ps, respectively, when compared to a baseline measurement period prior to interventions.

CONCLUSIONS

A bundled intervention that included deployment of novel templates for inpatient general medicine providers significantly reduced average note length on the clinical service. Templates designed to reduce the flow of extraneous information into provider notes performed well during usability testing, and these templates were rapidly adopted across all hospital campuses. Further research is needed to assess the impact of novel templates on note quality, provider efficiency and patient outcomes.

CLINICALTRIAL

User-Centered Design to Reduce Inappropriate Blood Transfusion Orders

BACKGROUND

To improve blood transfusion practices, we applied user-centered design (UCD) to evaluate potential changes to blood transfusion orders.

OBJECTIVES

The aim of the study is to build effective transfusion orders with different designs to improve guideline adherence.

METHODS

We developed three different versions of transfusion orders that varied how information was presented to clinicians ordering blood transfusions. We engaged 14 clinicians (residents, advanced practice providers [APPs], and attending physicians) from different specialties. We used the think aloud technique and rapid qualitative analysis to generate themes to incorporate into our modified orders.

RESULTS

Most end-users who participated in the semi-structured interviews preferred the interruptive alert design plus behavioral nudges (n = 8/14, 57%). The predominant rationale was that the in-line alert was not visually effective in capturing the end-user's attention, while the interruptive alert forced a brief stop in the workflow to consider the guidelines. All users supported the general improvements, though for different reasons, and as a result, the general improvements remained in the designs for the forthcoming trial.

CONCLUSION

The user experience uncovered through the think aloud approach produced a clear and rich understanding of potentially confounding factors in the initial design of different intervention versions. Input from end-users guided the creation of all three designs so each was addressing human factors with parity, which ensured that the results of our study reflected differences in interruptive properties of the alerts and not differences in design.

Usability and Utility of Human Immunodeficiency Virus Pre-exposure Prophylaxis Clinical Decision Support to Increase Knowledge and Pre-exposure Prophylaxis Initiations among Pediatric Providers

OBJECTIVES

 An effective clinical decision support system (CDSS) may address the current provider training barrier to offering preexposure prophylaxis (PrEP) to youth at risk for human immunodeficiency virus (HIV) infection. This study evaluated change in provider knowledge and the likelihood to initiate PrEP after exposure to a PrEP CDSS. A secondary objective explored perceived provider utility of the CDSS and suggestions for improving CDSS effectiveness.

METHODS

 This was a prospective study using survey responses from a convenience sample of pediatric providers who launched the interruptive PrEP CDSS when ordering an HIV test. McNemar's test evaluated change in provider PrEP knowledge and likelihood to initiate PrEP. Qualitative responses on CDSS utility and suggested improvements were analyzed using framework analysis and were connected to quantitative analysis elements using the merge approach.

RESULTS

 Of the 73 invited providers, 43 had available outcome data and were included in the analysis. Prior to using the CDSS, 86% of participants had never been prescribed PrEP. Compared to before CDSS exposure, there were significant increases in the proportion of providers who were knowledgeable about PrEP (p = 0.0001), likely to prescribe PrEP (p < 0.0001) and likely to refer their patient for PrEP (p < 0.0001). Suggestions for improving the CDSS included alternative "triggers" for the CDSS earlier in visit workflows, having a noninterruptive CDSS, additional provider educational materials, access to patient-facing PrEP materials, and additional CDSS support for adolescent confidentiality and navigating financial implications of PrEP.

CONCLUSION

 Our findings suggest that an interruptive PrEP CDSS attached to HIV test orders can be an effective tool to increase knowledge and likelihood to initiate PrEP among pediatric providers. Continual improvement of the PrEP CDSS based on provider feedback is required to optimize usability, effectiveness, and adoption. A highly usable PrEP CDSS may be a powerful tool to close the gap in youth PrEP access and uptake.

An Accessible Clinical Decision Support System to Curtail Anesthetic Greenhouse Gases in a Large Health Network: Implementation Study

BACKGROUND

Inhaled anesthetics in the operating room are potent greenhouse gases and are a key contributor to carbon emissions from health care facilities. Real-time clinical decision support (CDS) systems lower anesthetic gas waste by prompting anesthesia professionals to reduce fresh gas flow (FGF) when a set threshold is exceeded. However, previous CDS systems have relied on proprietary or highly customized anesthesia information management systems, significantly reducing other institutions' accessibility to the technology and thus limiting overall environmental benefit.

OBJECTIVE

In 2018, a CDS system that lowers anesthetic gas waste using methods that can be easily adopted by other institutions was developed at the University of California San Francisco (UCSF). This study aims to facilitate wider uptake of our CDS system and further reduce gas waste by describing the implementation of the FGF CDS toolkit at UCSF and the subsequent implementation at other medical campuses within the University of California Health network.

METHODS

We developed a noninterruptive active CDS system to alert anesthesia professionals when FGF rates exceeded 0.7 L per minute for common volatile anesthetics. The implementation process at UCSF was documented and assembled into an informational toolkit to aid in the integration of the CDS system at other health care institutions. Before implementation, presentation-based education initiatives were used to disseminate information regarding the safety of low FGF use and its relationship to environmental sustainability. Our FGF CDS toolkit consisted of 4 main components for implementation: sustainability-focused education of anesthesia professionals, hardware integration of the CDS technology, software build of the CDS system, and data reporting of measured outcomes.

RESULTS

The FGF CDS system was successfully deployed at 5 University of California Health network campuses. Four of the institutions are independent from the institution that created the CDS system. The CDS system was deployed at each facility using the FGF CDS toolkit, which describes the main components of the technology and implementation. Each campus made modifications to the CDS tool to best suit their institution, emphasizing the versatility and adoptability of the technology and implementation framework.

CONCLUSIONS

It has previously been shown that the FGF CDS system reduces anesthetic gas waste, leading to environmental and fiscal benefits. Here, we demonstrate that the CDS system can be transferred to other medical facilities using our toolkit for implementation, making the technology and associated benefits globally accessible to advance mitigation of health care-related emissions.

Interruptive Electronic Alerts for Choosing Wisely Recommendations: A Cluster Randomized Controlled Trial

OBJECTIVE

To assess the efficacy of interruptive electronic alerts in improving adherence to the American Board of Internal Medicine's Choosing Wisely recommendations to reduce unnecessary laboratory testing.

MATERIALS AND METHODS

We administered 5 cluster randomized controlled trials simultaneously, using electronic medical record alerts regarding prostate-specific antigen (PSA) testing, acute sinusitis treatment, vitamin D testing, carotid artery ultrasound screening, and human papillomavirus testing. For each alert, we assigned 5 outpatient clinics to an interruptive alert and 5 were observed as a control. Primary and secondary outcomes were the number of postalert orders per 100 patients at each clinic and number of triggered alerts divided by orders, respectively. Post hoc analysis evaluated whether physicians experiencing interruptive alerts reduced their alert-triggering behaviors.

RESULTS

Median postalert orders per 100 patients did not differ significantly between treatment and control groups; absolute median differences ranging from 0.04 to 0.40 for PSA testing. Median alerts per 100 orders did not differ significantly between treatment and control groups; absolute median differences ranged from 0.004 to 0.03. In post hoc analysis, providers receiving alerts regarding PSA testing in men were significantly less likely to trigger additional PSA alerts than those in the control sites (Incidence Rate Ratio 0.12, 95% CI [0.03-0.52]).

DISCUSSION

Interruptive point-of-care alerts did not yield detectable changes in the overall rate of undesired orders or the order-to-alert ratio between active and silent sites. Complementary behavioral or educational interventions are likely needed to improve efforts to curb medical overuse.

CONCLUSION

Implementation of interruptive alerts at the time of ordering was not associated with improved adherence to 5 Choosing Wisely guidelines.

TRIAL REGISTRATION

NCT02709772.

Combined Expert and User-Driven Usability Assessment of Trauma Decision Support Systems Improves User-Centered Design

BACKGROUND

Trauma clinical decision support systems improve adherence with evidence-based practice but suffer from poor usability and the lack of a user-centered design. The objective of this study was to compare the effectiveness of user and expert-driven usability testing methods to detect usability issues in a rib fracture clinical decision support system and identify guiding principles for trauma clinical decision support systems.

METHODS

A user-driven and expert-driven usability investigation was conducted using a clinical decision support system developed for patients with rib fractures. The user-driven usability evaluation was as follows: 10 clinicians were selected for simulation-based usability testing using snowball sampling, and each clinician completed 3 simulations using a video-conferencing platform. End-users participated in a novel team-based approach that simulated realistic clinical workflows. The expert-driven heuristic evaluation was as follows: 2 usability experts conducted a heuristic evaluation of the clinical decision support system using 10 common usability heuristics. Usability issues were identified, cataloged, and ranked for severity using a 4-level ordinal scale. Thematic analysis was utilized to categorize the identified usability issues.

RESULTS

Seventy-nine usability issues were identified; 63% were identified by experts and 48% by end-users. Notably, 58% of severe usability issues were identified by experts alone. Only 11% of issues were identified by both methods. Five themes were identified that could guide the design of clinical decision support systems-transparency, functionality and integration into workflow, automated and noninterruptive, flexibility, and layout and appearance. Themes were preferentially identified by different methods.

CONCLUSION

We found that a dual-method usability evaluation involving usability experts and end-users drastically improved detection of usability issues over single-method alone. We identified 5 themes to guide trauma clinical decision support system design. Performing usability testing via a remote video-conferencing platform facilitated multi-site involvement despite a global pandemic.

Barriers to and Facilitators for Acceptance of Comprehensive Clinical Decision Support System-Driven Care Maps for Patients With Thoracic Trauma: Interview Study Among Health Care Providers and Nurses

BACKGROUND

Comprehensive clinical decision support (CDS) care maps can improve the delivery of care and clinical outcomes. However, they are frequently plagued by usability problems and poor user acceptance.

OBJECTIVE

This study aims to characterize factors influencing successful design and use of comprehensive CDS care maps and identify themes associated with end-user acceptance of a thoracic trauma CDS care map earlier in the process than has traditionally been done. This was a planned adaptive redesign stage of a User Acceptance and System Adaptation Design development and implementation strategy for a CDS care map. This stage was based on a previously developed prototype CDS care map guided by the Unified Theory of Acceptance and Use of Technology.

METHODS

A total of 22 multidisciplinary end users (physicians, advanced practice providers, and nurses) were identified and recruited using snowball sampling. Qualitative interviews were conducted, audio-recorded, and transcribed verbatim. Generation of prespecified codes and the interview guide was informed by the Unified Theory of Acceptance and Use of Technology constructs and investigative team experience. Interviews were blinded and double-coded. Thematic analysis of interview scripts was conducted and yielded descriptive themes about factors influencing the construction and potential use of an acceptable CDS care map.

RESULTS

A total of eight dominant themes were identified: alert fatigue (theme 1), automation (theme 2), redundancy (theme 3), minimalistic design (theme 4), evidence based (theme 5), prevent errors (theme 6), comprehensive across the spectrum of disease (theme 7), and malleability (theme 8). Themes 1 to 4 addressed factors directly affecting end users, and themes 5 to 8 addressed factors affecting patient outcomes. More experienced providers prioritized a system that is easy to use. Nurses prioritized a system that incorporated evidence into decision support. Clinicians across specialties, roles, and ages agreed that the amount of extra work generated should be minimal and that the system should help them administer optimal care efficiently.

CONCLUSIONS

End user feedback reinforces attention toward factors that improve the acceptance and use of a CDS care map for patients with thoracic trauma. Common themes focused on system complexity, the ability of the system to fit different populations and settings, and optimal care provision. Identifying these factors early in the development and implementation process may facilitate user-centered design and improve adoption.

Electronic Medication Management Systems: Analysis of Enhancements to Reduce Errors and Improve Workflow

BACKGROUND

Electronic medication management (eMM) has been shown to reduce medication errors; however, new safety risks have also been introduced that are associated with system use. No research has specifically examined the changes made to eMM systems to mitigate these risks.

OBJECTIVES

To (1) identify system-related medication errors or workflow blocks that were the target of eMM system updates, including the types of medications involved, and (2) describe and classify the system enhancements made to target these risks.

METHODS

In this retrospective qualitative study, documents detailing updates made from November 2014 to December 2019 to an eMM system were reviewed. Medication-related updates were classified according to "rationale for changes" and "changes made to the system."

RESULTS

One hundred and seventeen updates, totaling 147 individual changes, were made to the eMM system over the 4-year period. The most frequent reasons for changes being made to the eMM were to prevent medication errors (24% of reasons), optimize workflow (22%), and support "work as done" on paper (16%). The most frequent changes made to the eMM were options added to lists (14% of all changes), extra information made available on the screen (8%), and the wording or phrasing of text modified (8%). Approximately a third of the updates (37%) related to high-risk medications. The reasons for system changes appeared to vary over time, as eMM functionality and use expanded.

CONCLUSION

To our knowledge, this is the first study to systematically review and categorize system updates made to overcome new safety risks associated with eMM use. Optimization of eMM is an ongoing process, which changes over time as users become more familiar with the system and use is expanded to more sites. Continuous monitoring of the system is necessary to detect areas for improvement and capitalize on the benefits an electronic system can provide.

A Fully Collaborative, Noteless Electronic Medical Record Designed to Minimize Information Chaos: Software Design and Feasibility Study

Background

Clinicians spend large amounts of their workday using electronic medical records (EMRs). Poorly designed documentation systems contribute to the proliferation of out-of-date information, increased time spent on medical records, clinician burnout, and medical errors. Beyond software interfaces, examining the underlying paradigms and organizational structures for clinical information may provide insights into ways to improve documentation systems. In particular, our attachment to the note as the major organizational unit for storing unstructured medical data may be a cause of many of the problems with modern clinical documentation. Notes, as currently understood, systematically incentivize information duplication and information scattering, both within a single clinician’s notes over time and across multiple clinicians’ notes. Therefore, it is worthwhile to explore alternative paradigms for unstructured data organization.

Objective

The aim of this study is to demonstrate the feasibility of building an EMR that does not use notes as the core organizational unit for unstructured data and which is designed specifically to disincentivize information duplication and information scattering.

Methods

We used specific design principles to minimize the incentive for users to duplicate and scatter information. By default, the majority of a patient’s medical history remains the same over time, so users should not have to redocument that information. Clinicians on different teams or services mostly share the same medical information, so all data should be collaboratively shared across teams and services (while still allowing for disagreement and nuance). In all cases where a clinician must state that information has remained the same, they should be able to attest to the information without redocumenting it. We designed and built a web-based EMR based on these design principles.

Results

We built a medical documentation system that does not use notes and instead treats the chart as a single, dynamically updating, and fully collaborative workspace. All information is organized by clinical topic or problem. Version history functionality is used to enable granular tracking of changes over time. Our system is highly customizable to individual workflows and enables each individual user to decide which data should be structured and which should be unstructured, enabling individuals to leverage the advantages of structured templating and clinical decision support as desired without requiring programming knowledge. The system is designed to facilitate real-time, fully collaborative documentation and communication among multiple clinicians.

Conclusions

We demonstrated the feasibility of building a non–note-based, fully collaborative EMR system. Our attachment to the note as the only possible atomic unit of unstructured medical data should be reevaluated, and alternative models should be considered.

A Behavioral Economics-Electronic Health Record Module to Promote Appropriate Diabetes Management in Older Adults: Protocol for a Pragmatic Cluster Randomized Controlled Trial

Background

The integration of behavioral economics (BE) principles and electronic health records (EHRs) using clinical decision support (CDS) tools is a novel approach to improving health outcomes. Meanwhile, the American Geriatrics Society has created the Choosing Wisely (CW) initiative to promote less aggressive glycemic targets and reduction in pharmacologic therapy in older adults with type 2 diabetes mellitus. To date, few studies have shown the effectiveness of combined BE and EHR approaches for managing chronic conditions, and none have addressed guideline-driven deprescribing specifically in type 2 diabetes. We previously conducted a pilot study aimed at promoting appropriate CW guideline adherence using BE nudges and EHRs embedded within CDS tools at 5 clinics within the New York University Langone Health (NYULH) system. The BE-EHR module intervention was tested for usability, adoption, and early effectiveness. Preliminary results suggested a modest improvement of 5.1% in CW compliance.

Objective

This paper presents the protocol for a study that will investigate the effectiveness of a BE-EHR module intervention that leverages BE nudges with EHR technology and CDS tools to reduce overtreatment of type 2 diabetes in adults aged 76 years and older, per the CW guideline.

Methods

A pragmatic, investigator-blind, cluster randomized controlled trial was designed to evaluate the BE-EHR module. A total of 66 NYULH clinics will be randomized 1:1 to receive for 18 months either (1) a 6-component BE-EHR module intervention + standard care within the NYULH EHR, or (2) standard care only. The intervention will be administered to clinicians during any patient encounter (eg, in person, telemedicine, medication refill, etc). The primary outcome will be patient-level CW compliance. Secondary outcomes will measure the frequency of intervention component firings within the NYULH EHR, and provider utilization and interaction with the BE-EHR module components.

Results

Study recruitment commenced on December 7, 2020, with the activation of all 6 BE-EHR components in the NYULH EHR.

Conclusions

This study will test the effectiveness of a previously developed, iteratively refined, user-tested, and pilot-tested BE-EHR module aimed at providing appropriate diabetes care to elderly adults, compared to usual care via a cluster randomized controlled trial. This innovative research will be the first pragmatic randomized controlled trial to use BE principles embedded within the EHR and delivered using CDS tools to specifically promote CW guideline adherence in type 2 diabetes. The study will also collect valuable information on clinician workflow and interaction with the BE-EHR module, guiding future research in optimizing the timely delivery of BE nudges within CDS tools. This work will address the effectiveness of BE-inspired interventions in diabetes and chronic disease management.

Trial Registration

ClinicalTrials.gov NCT04181307; https://clinicaltrials.gov/ct2/show/NCT04181307

International Registered Report Identifier (IRRID)

DERR1-10.2196/28723

Development and implementation of a clinical decision support-based initiative to drive intravenous fluid prescribing

OBJECTIVE

Studies suggest superior outcomes with use of intravenous (IV) balanced fluids compared to normal saline (NS). However, significant fluid prescribing variability persists, highlighting the knowledge-to-practice gap. We sought to identify contributors to prescribing variation and utilize a clinical decision support system (CDSS) to increase institutional balanced fluid prescribing.

MATERIALS AND METHODS

This single-center informatics-enabled quality improvement initiative for patients hospitalized or treated in the emergency department included stepwise interventions of 1) identification of design factors within the computerized provider order entry (CPOE) of our electronic health record (EHR) that contribute to preferential NS ordering, 2) clinician education, 3) fluid stocking modifications, 4) re-design and implementation of a CDSS-integrated IV fluid ordering panel, and 5) comparison of fluid prescribing before and after the intervention. EHR-derived prescribing data was analyzed via single interrupted time series.

RESULTS

Pre-intervention (3/2019-9/2019), balanced fluids comprised 33% of isotonic fluid orders, with gradual uptake (1.4%/month) of balanced fluid prescribing. Clinician education (10/2019-2/2020) yielded a modest (4.4%/month, 95% CI 1.6-7.2, p = 0.01) proportional increase in balanced fluid prescribing, while CPOE redesign (3/2020) yielded an immediate (20.7%, 95% CI 17.7-23.6, p < 0.0001) and sustained increase (72% of fluid orders in 12/2020). The intervention proved most effective among those with lower baseline balanced fluids utilization, including emergency medicine (57% increase, 95% CI 0.7-1.8, p < 0.0001) and internal medicine/subspecialties (18% increase, 95% CI 14.4-21.3, p < 0.0001) clinicians and substantially reduced institutional prescribing variation.

CONCLUSION

Integration of CDSS into an EHR yielded a robust and sustained increase in balanced fluid prescribing. This impact far exceeded that of clinician education highlighting the importance of CDSS.

An evaluation of clinical decision support tools for Patient Health Questionnaire-9 administration

Introduction

Many health care institutions are working to improve depression screening and management with the use of the Patient Health Questionnaire 9 (PHQ-9). Clinical decision support (CDS) within the EHR is one strategy, but little is known about effective approaches to design or implement such CDS. The purpose of this study is to compare implementation outcomes of two versions of a CDS tool to improve PHQ-9 administration for patients with depression.

Methods

This was a retrospective, observational study comparing two versions of a CDS. Version 1 interrupted clinician workflow, and version 2 did not interrupt workflow. Outcomes of interest included reach, adoption, and effectiveness. PHQ-9 administration was determined by chart review. Chi-square tests were used to evaluate associations between PHQ-9 administration with versions 1 and 2.

Results

Version 1 resulted in PHQ-9 administration 77 times (15.3% of 504 unique encounters) compared with 49 times (9.8% of 502 unique encounters) with version 2 (P = .011).

Discussion

An interruptive CDS tool may be more effective at increasing PHQ-9 administration, but a noninterruptive CDS tool may be preferred to minimize alert fatigue despite a decrease in effectiveness.

eSIMPLER: A Dynamic, Electronic Health Record-Integrated Checklist for Clinical Decision Support During PICU Daily Rounds

OBJECTIVES

Design, implement, and evaluate a rounding checklist with deeply embedded, dynamic electronic health record integration.

DESIGN

Before-after quality-improvement study.

SETTING

Quaternary PICU in an academic, free-standing children's hospital.

PATIENTS

All patients in the PICU during daily morning rounds.

INTERVENTIONS

Implementation of an updated dynamic checklist (eSIMPLER) providing clinical decision support prompts with display of relevant data automatically pulled from the electronic health record.

MEASUREMENTS AND MAIN RESULTS

The prior daily rounding checklist, eSIMPLE, was implemented for 49,709 patient-days (7,779 patients) between October 30, 2011, and October 7, 2018. eSIMPLER was implemented for 5,306 patient-days (971 patients) over 6 months. Checklist completion rates were similar (eSIMPLE: 95% [95% CI, 88-98%] vs eSIMPLER: 98% [95% CI, 92-100%] of patient-days; p = 0.40). eSIMPLER required less time per patient (28 ± 1 vs 47 ± 24 s; p < 0.001). Users reported improved satisfaction with eSIMPLER (p = 0.009). Several checklist-driven process measures-discordance between electronic health record orders for stress ulcer prophylaxis and user-recorded indication for stress ulcer prophylaxis, rate of venous thromboembolism prophylaxis prescribing, and recognition of reduced renal function-improved during the eSIMPLER phase.

CONCLUSIONS

eSIMPLER, a dynamic, electronic health record-informed checklist, required less time to complete and improved certain care processes compared with a prior, static checklist with limited electronic health record data. By focusing on the "Five Rights" of clinical decision support, we created a well-accepted clinical decision support tool that was integrated efficiently into daily rounds. Generalizability of eSIMPLER's effectiveness and its impact on patient outcomes need to be examined.

The appropriateness of clinical decision support systems alerts in supporting clinical workflows: A systematic review

A CDSS generates a high number of inappropriate alerts that interrupt the clinical workflow. As a result, clinicians silence, disable, or ignore alerts, thereby undermining patient safety. Therefore, the effectiveness and appropriateness of CDSS alerts need to be evaluated. A systematic review was carried out to identify the factors that affect CDSS alert appropriateness in supporting clinical workflow. Seven electronic databases (PubMed, Scopus, ACM, Science Direct, IEEE, Ovid Medline, and Ebscohost) were searched for English language articles published between 1997 and 2018. Seventy six papers met the inclusion criteria, of which 26, 24, 15, and 11 papers are retrospective cohort, qualitative, quantitative, and mixed-method studies, respectively. The review highlights various factors influencing the appropriateness and efficiencies of CDSS alerts. These factors are categorized into technology, human, organization, and process aspects using a combination of approaches, including socio-technical framework, five rights of CDSS, and Lean. Most CDSS alerts were not properly designed based on human factor methods and principles, explaining high alert overrides in clinical practices. The identified factors and recommendations from the review may offer valuable insights into how CDSS alerts can be designed appropriately to support clinical workflow.

Associations of physicians' prescribing experience, work hours, and workload with prescription errors

OBJECTIVE

We aimed to assess associations of physician's work overload, successive work shifts, and work experience with physicians' risk to err.

MATERIALS AND METHODS

This large-scale study included physicians who prescribed at least 100 systemic medications at Sheba Medical Center during 2012-2017 in all acute care departments, excluding intensive care units. Presumed medication errors were flagged by a high-accuracy computerized decision support system that uses machine-learning algorithms to detect potential medication prescription errors. Physicians' successive work shifts (first or only shift, second, and third shifts), workload (assessed by the number of prescriptions during a shift) and work-experience, as well as a novel measurement of physicians' prescribing experience with a specific drug, were assessed per prescription. The risk to err was determined for various work conditions.

RESULTS

1 652 896 medical orders were prescribed by 1066 physicians; The system flagged 3738 (0.23%) prescriptions as erroneous. Physicians were 8.2 times more likely to err during high than normal-low workload shifts (5.19% vs 0.63%, P < .0001). Physicians on their third or second successive shift (compared to a first or single shift) were more likely to err (2.1%, 1.8%, and 0.88%, respectively, P < .001). Lack of experience in prescribing a specific medication was associated with higher error rate (0.37% for the first 5 prescriptions vs 0.13% after over 40, P < .001).

DISCUSSION

Longer hours and less experience in prescribing a specific medication increase risk of erroneous prescribing.

CONCLUSION

Restricting successive shifts, reducing workload, increasing training and supervision, and implementing smart clinical decision support systems may help reduce prescription errors.

Use of deep learning to develop continuous-risk models for adverse event prediction from electronic health records

Early prediction of patient outcomes is important for targeting preventive care. This protocol describes a practical workflow for developing deep-learning risk models that can predict various clinical and operational outcomes from structured electronic health record (EHR) data. The protocol comprises five main stages: formal problem definition, data pre-processing, architecture selection, calibration and uncertainty, and generalizability evaluation. We have applied the workflow to four endpoints (acute kidney injury, mortality, length of stay and 30-day hospital readmission). The workflow can enable continuous (e.g., triggered every 6 h) and static (e.g., triggered at 24 h after admission) predictions. We also provide an open-source codebase that illustrates some key principles in EHR modeling. This protocol can be used by interdisciplinary teams with programming and clinical expertise to build deep-learning prediction models with alternate data sources and prediction tasks.

Applying Clinical Decision Support Design Best Practices With the Practical Robust Implementation and Sustainability Model Versus Reliance on Commercially Available Clinical Decision Support Tools: Randomized Controlled Trial

BACKGROUND

Limited consideration of clinical decision support (CDS) design best practices, such as a user-centered design, is often cited as a key barrier to CDS adoption and effectiveness. The application of CDS best practices is resource intensive; thus, institutions often rely on commercially available CDS tools that are created to meet the generalized needs of many institutions and are not user centered. Beyond resource availability, insufficient guidance on how to address key aspects of implementation, such as contextual factors, may also limit the application of CDS best practices. An implementation science (IS) framework could provide needed guidance and increase the reproducibility of CDS implementations.

OBJECTIVE

This study aims to compare the effectiveness of an enhanced CDS tool informed by CDS best practices and an IS framework with a generic, commercially available CDS tool.

METHODS

We conducted an explanatory sequential mixed methods study. An IS-enhanced and commercial CDS alert were compared in a cluster randomized trial across 28 primary care clinics. Both alerts aimed to improve beta-blocker prescribing for heart failure. The enhanced alert was informed by CDS best practices and the Practical, Robust, Implementation, and Sustainability Model (PRISM) IS framework, whereas the commercial alert followed vendor-supplied specifications. Following PRISM, the enhanced alert was informed by iterative, multilevel stakeholder input and the dynamic interactions of the internal and external environment. Outcomes aligned with PRISM's evaluation measures, including patient reach, clinician adoption, and changes in prescribing behavior. Clinicians exposed to each alert were interviewed to identify design features that might influence adoption. The interviews were analyzed using a thematic approach.

RESULTS

Between March 15 and August 23, 2019, the enhanced alert fired for 61 patients (106 alerts, 87 clinicians) and the commercial alert fired for 26 patients (59 alerts, 31 clinicians). The adoption and effectiveness of the enhanced alert were significantly higher than those of the commercial alert (62% vs 29% alerts adopted, P<.001; 14% vs 0% changed prescribing, P=.006). Of the 21 clinicians interviewed, most stated that they preferred the enhanced alert.

CONCLUSIONS

The results of this study suggest that applying CDS best practices with an IS framework to create CDS tools improves implementation success compared with a commercially available tool.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04028557; http://clinicaltrials.gov/ct2/show/NCT04028557.

Evaluation of a clinical decision support system for rare diseases: a qualitative study

BACKGROUND

Rare Diseases (RDs) are difficult to diagnose. Clinical Decision Support Systems (CDSS) could support the diagnosis for RDs. The Medical Informatics in Research and Medicine (MIRACUM) consortium developed a CDSS for RDs based on distributed clinical data from eight German university hospitals. To support the diagnosis for difficult patient cases, the CDSS uses data from the different hospitals to perform a patient similarity analysis to obtain an indication of a diagnosis. To optimize our CDSS, we conducted a qualitative study to investigate usability and functionality of our designed CDSS.

METHODS

We performed a Thinking Aloud Test (TA-Test) with RDs experts working in Rare Diseases Centers (RDCs) at MIRACUM locations which are specialized in diagnosis and treatment of RDs. An instruction sheet with tasks was prepared that the participants should perform with the CDSS during the study. The TA-Test was recorded on audio and video, whereas the resulting transcripts were analysed with a qualitative content analysis, as a ruled-guided fixed procedure to analyse text-based data. Furthermore, a questionnaire was handed out at the end of the study including the System Usability Scale (SUS).

RESULTS

A total of eight experts from eight MIRACUM locations with an established RDC were included in the study. Results indicate that more detailed information about patients, such as descriptive attributes or findings, can help the system perform better. The system was rated positively in terms of functionality, such as functions that enable the user to obtain an overview of similar patients or medical history of a patient. However, there is a lack of transparency in the results of the CDSS patient similarity analysis. The study participants often stated that the system should present the user with an overview of exact symptoms, diagnosis, and other characteristics that define two patients as similar. In the usability section, the CDSS received a score of 73.21 points, which is ranked as good usability.

CONCLUSIONS

This qualitative study investigated the usability and functionality of a CDSS of RDs. Despite positive feedback about functionality of system, the CDSS still requires some revisions and improvement in transparency of the patient similarity analysis.

Unpacking Prevalence and Dichotomy in Quick Sequential Organ Failure Assessment and Systemic Inflammatory Response Syndrome Parameters: Observational Data-Driven Approach Backed by Sepsis Pathophysiology

BACKGROUND

Considering morbidity, mortality, and annual treatment costs, the dramatic rise in the incidence of sepsis and septic shock among intensive care unit (ICU) admissions in US hospitals is an increasing concern. Recent changes in the sepsis definition (sepsis-3), based on the quick Sequential Organ Failure Assessment (qSOFA), have motivated the international medical informatics research community to investigate score recalculation and information retrieval, and to study the intersection between sepsis-3 and the previous definition (sepsis-2) based on systemic inflammatory response syndrome (SIRS) parameters.

OBJECTIVE

The objective of this study was three-fold. First, we aimed to unpack the most prevalent criterion for sepsis (for both sepsis-3 and sepsis-2 predictors). Second, we intended to determine the most prevalent sepsis scenario in the ICU among 4 possible scenarios for qSOFA and 11 possible scenarios for SIRS. Third, we investigated the multicollinearity or dichotomy among qSOFA and SIRS predictors.

METHODS

This observational study was conducted according to the most recent update of Medical Information Mart for Intensive Care (MIMIC-III, Version 1.4), the critical care database developed by MIT. The qSOFA (sepsis-3) and SIRS (sepsis-2) parameters were analyzed for patients admitted to critical care units from 2001 to 2012 in Beth Israel Deaconess Medical Center (Boston, MA, USA) to determine the prevalence and underlying relation between these parameters among patients undergoing sepsis screening. We adopted a multiblind Delphi method to seek a rationale for decisions in several stages of the research design regarding handling missing data and outlier values, statistical imputations and biases, and generalizability of the study.

RESULTS

Altered mental status in the Glasgow Coma Scale (59.28%, 38,854/65,545 observations) was the most prevalent sepsis-3 (qSOFA) criterion and the white blood cell count (53.12%, 17,163/32,311 observations) was the most prevalent sepsis-2 (SIRS) criterion confronted in the ICU. In addition, the two-factored sepsis criterion of high respiratory rate (≥22 breaths/minute) and altered mental status (28.19%, among four possible qSOFA scenarios besides no sepsis) was the most prevalent sepsis-3 (qSOFA) scenario, and the three-factored sepsis criterion of tachypnea, high heart rate, and high white blood cell count (12.32%, among 11 possible scenarios besides no sepsis) was the most prevalent sepsis-2 (SIRS) scenario in the ICU. Moreover, the absolute Pearson correlation coefficients were not significant, thereby nullifying the likelihood of any linear correlation among the critical parameters and assuring the lack of multicollinearity between the parameters. Although this further bolsters evidence for their dichotomy, the absence of multicollinearity cannot guarantee that two random variables are statistically independent.

CONCLUSIONS

Quantifying the prevalence of the qSOFA criteria of sepsis-3 in comparison with the SIRS criteria of sepsis-2, and understanding the underlying dichotomy among these parameters provides significant inferences for sepsis treatment initiatives in the ICU and informing hospital resource allocation. These data-driven results further offer design implications for multiparameter intelligent sepsis prediction in the ICU.

Machine Learning Approach to Reduce Alert Fatigue Using a Disease Medication-Related Clinical Decision Support System: Model Development and Validation

Background

Computerized physician order entry (CPOE) systems are incorporated into clinical decision support systems (CDSSs) to reduce medication errors and improve patient safety. Automatic alerts generated from CDSSs can directly assist physicians in making useful clinical decisions and can help shape prescribing behavior. Multiple studies reported that approximately 90%-96% of alerts are overridden by physicians, which raises questions about the effectiveness of CDSSs. There is intense interest in developing sophisticated methods to combat alert fatigue, but there is no consensus on the optimal approaches so far.

Objective

Our objective was to develop machine learning prediction models to predict physicians’ responses in order to reduce alert fatigue from disease medication–related CDSSs.

Methods

We collected data from a disease medication–related CDSS from a university teaching hospital in Taiwan. We considered prescriptions that triggered alerts in the CDSS between August 2018 and May 2019. Machine learning models, such as artificial neural network (ANN), random forest (RF), naïve Bayes (NB), gradient boosting (GB), and support vector machine (SVM), were used to develop prediction models. The data were randomly split into training (80%) and testing (20%) datasets.

Results

A total of 6453 prescriptions were used in our model. The ANN machine learning prediction model demonstrated excellent discrimination (area under the receiver operating characteristic curve [AUROC] 0.94; accuracy 0.85), whereas the RF, NB, GB, and SVM models had AUROCs of 0.93, 0.91, 0.91, and 0.80, respectively. The sensitivity and specificity of the ANN model were 0.87 and 0.83, respectively.

Conclusions

In this study, ANN showed substantially better performance in predicting individual physician responses to an alert from a disease medication–related CDSS, as compared to the other models. To our knowledge, this is the first study to use machine learning models to predict physician responses to alerts; furthermore, it can help to develop sophisticated CDSSs in real-world clinical settings.

Clinical Decision Support and Implications for the Clinician Burnout Crisis

OBJECTIVES

This survey aimed to review aspects of clinical decision support (CDS) that contribute to burnout and identify key themes for improving the acceptability of CDS to clinicians, with the goal of decreasing said burnout.

METHODS

We performed a survey of relevant articles from 2018-2019 addressing CDS and aspects of clinician burnout from PubMed and Web of Science™. Themes were manually extracted from publications that met inclusion criteria.

RESULTS

Eighty-nine articles met inclusion criteria, including 12 review articles. Review articles were either prescriptive, describing how CDS should work, or analytic, describing how current CDS tools are deployed. The non-review articles largely demonstrated poor relevance and acceptability of current tools, and few studies showed benefits in terms of efficiency or patient outcomes from implemented CDS. Encouragingly, multiple studies highlighted steps that succeeded in improving both acceptability and relevance of CDS.

CONCLUSIONS

CDS can contribute to clinician frustration and burnout. Using the techniques of improving relevance, soliciting feedback, customization, measurement of outcomes and metrics, and iteration, the effects of CDS on burnout can be ameliorated.

Impact of a "Chart Closure" Hard Stop Alert on Prescribing for Elevated Blood Pressures Among Patients With Diabetes: Quasi-Experimental Study

BACKGROUND

University of California at Los Angeles Health implemented a Best Practice Advisory (BPA) alert for the initiation of an angiotensin-converting enzyme inhibitor (ACEI) or angiotensin-receptor blocker (ARB) for individuals with diabetes. The BPA alert was configured with a "chart closure" hard stop, which demanded a response before closing the chart.

OBJECTIVE

The aim of the study was to evaluate whether the implementation of the BPA was associated with changes in ACEI and ARB prescribing during primary care encounters for patients with diabetes.

METHODS

We defined ACEI and ARB prescribing opportunities as primary care encounters in which the patient had a diabetes diagnosis, elevated blood pressure in recent encounters, no active ACEI or ARB prescription, and no contraindications. We used a multivariate logistic regression model to compare the change in the probability of an ACEI or ARB prescription during opportunity encounters before and after BPA implementation in primary care sites that did (n=30) and did not (n=31) implement the BPA. In an additional subgroup analysis, we compared ACEI and ARB prescribing in BPA implementation sites that had also implemented a pharmacist-led medication management program.

RESULTS

We identified a total of 2438 opportunity encounters across 61 primary care sites. The predicted probability of an ACEI or ARB prescription increased significantly from 11.46% to 22.17% during opportunity encounters in BPA implementation sites after BPA implementation. However, in the subgroup analysis, we only observed a significant improvement in ACEI and ARB prescribing in BPA implementation sites that had also implemented the pharmacist-led program. Overall, the change in the predicted probability of an ACEI or ARB prescription from before to after BPA implementation was significantly greater in BPA implementation sites compared with nonimplementation sites (difference-in-differences of 11.82; P<.001).

CONCLUSIONS

A BPA with a "chart closure" hard stop is a promising tool for the treatment of patients with comorbid diabetes and hypertension with an ACEI or ARB, especially when implemented within the context of team-based care, wherein clinical pharmacists support the work of primary care providers.