Comparing the Effectiveness of Alerts and Dynamically Annotated Visualizations (DAVs) in Improving Clinical Decision Making

A Bayesian Spatiotemporal Nowcasting Model for Public Health Decision-Making and Surveillance

As coronavirus disease 2019 (COVID-19) spread through the United States in 2020, states began to set up alert systems to inform policy decisions and serve as risk communication tools for the general public. Many of these systems included indicators based on an assessment of trends in numbers of reported cases. However, when cases are indexed by date of disease onset, reporting delays complicate the interpretation of trends. Despite a foundation of statistical literature with which to address this problem, these methods have not been widely applied in practice. In this paper, we develop a Bayesian spatiotemporal nowcasting model for assessing trends in county-level COVID-19 cases in Ohio. We compare the performance of our model with the approach used in Ohio and the approach included in decision support materials from the Centers for Disease Control and Prevention. We demonstrate gains in performance while still retaining interpretability using our model. In addition, we are able to fully account for uncertainty in both the time series of cases and the reporting process. While we cannot eliminate all of the uncertainty in public health surveillance and subsequent decision-making, we must use approaches that embrace these challenges and deliver more accurate and honest assessments to policy-makers.

Optimized Use of the Electronic Health Record and Other Clinical Resources to Enhance the Management of Hypogonadal Men

Electronic health records (EHRs) have enabled electronic documentation of a tremendous amount of clinical data. EHRs have the potential to improve communication between patients and their providers, facilitate quality improvement and outcomes research, and reduce medical errors. Conversely, EHRs have also increased clinician burnout, information clutter, and depersonalization of the interactions between patients and their providers. Increasing clinician input into EHR design, providing access to technical help, streamlining of workflow, and the use of custom templates that have fewer requirements for evaluation and management coding can reduce this burnout and increase the utility of this advancing technology.

Reducing Alert Fatigue by Sharing Low-Level Alerts With Patients and Enhancing Collaborative Decision Making Using Blockchain Technology: Scoping Review and Proposed Framework (MedAlert)

Background

Clinical decision support (CDS) is a tool that helps clinicians in decision making by generating clinical alerts to supplement their previous knowledge and experience. However, CDS generates a high volume of irrelevant alerts, resulting in alert fatigue among clinicians. Alert fatigue is the mental state of alerts consuming too much time and mental energy, which often results in relevant alerts being overridden unjustifiably, along with clinically irrelevant ones. Consequently, clinicians become less responsive to important alerts, which opens the door to medication errors.

Objective

This study aims to explore how a blockchain-based solution can reduce alert fatigue through collaborative alert sharing in the health sector, thus improving overall health care quality for both patients and clinicians.

Methods

We have designed a 4-step approach to answer this research question. First, we identified five potential challenges based on the published literature through a scoping review. Second, a framework is designed to reduce alert fatigue by addressing the identified challenges with different digital components. Third, an evaluation is made by comparing MedAlert with other proposed solutions. Finally, the limitations and future work are also discussed.

Results

Of the 341 academic papers collected, 8 were selected and analyzed. MedAlert securely distributes low-level (nonlife-threatening) clinical alerts to patients, enabling a collaborative clinical decision. Among the solutions in our framework, Hyperledger (private permissioned blockchain) and BankID (federated digital identity management) have been selected to overcome challenges such as data integrity, user identity, and privacy issues.

Conclusions

MedAlert can reduce alert fatigue by attracting the attention of patients and clinicians, instead of solely reducing the total number of alerts. MedAlert offers other advantages, such as ensuring a higher degree of patient privacy and faster transaction times compared with other frameworks. This framework may not be suitable for elderly patients who are not technology savvy or in-patients. Future work in validating this framework based on real health care scenarios is needed to provide the performance evaluations of MedAlert and thus gain support for the better development of this idea.

Comparative Effectiveness of Best Practice Alerts with Active and Passive Presentations: A Retrospective Study

We assess the relationship of active or passive presentation of Best Practice Advisories (BPAs) for hospital clinicians with compliance rates of recommended actions. We identify the design characteristics of alerts that can be used to assess the effectiveness of design choices with superior usability. Alerts in Electronic Health Records (EHRs) are frequently overridden by healthcare providers. Identifying characteristics of effective alerts can increase the frequency that actions recommended in evidence-based care guidelines are done, reduce user frustration, and improve interface usability along with the willingness to use alerts. We conducted a retrospective analysis of data for 11 BPAs between June 2014 and May 2015. The outcome measure was the percent correspondence with recommended actions. A repeated measures regression model was used for the correlation of the BPA presentation type with the outcome measure. The BPA presentation type was significant such that the odds are 7.7 times greater that a recommended action would be taken by a provider with an active BPA presentation type after adjusting for whether an action was required. Active presentation alerts achieve higher compliance rates. CDS alerts that actively interrupted the provider’s workflow were associated with a higher compliance rate with recommended actions.

Designing an electronic blood-borne virus risk alert to improve uptake of testing

The primary aim of the current study was to test the effect of the presentation design of a test alert system on healthcare workers’ (HCWs’) decision-making regarding blood-borne virus (BBV) testing. The secondary aim was to determine HCWs’ acceptance of the system. An online survey used a within-subjects research design with four design factors as independent variables. The dependent variable was clinical decision. Ten realistic descriptions of hypothetical patients were presented to participants who were asked to decide whether to request BBV testing. The effect of a pre-set course of action to request BBV testing was significant when additional information (cost-effectiveness, date of last BBV test or risk assessment) was not presented, with a 16% increase from 30 to 46% accept decisions. When risk assessment information was presented without a pre-set course of action, the effects of cost-effectiveness (27% increase) and last test date (23% decrease) were significant. The main reason for declining to test was insufficient risk. HCWs’ acceptance of the test alert system was high and resistance was low. We make recommendations from the results for the design of a subsequent real-world trial of the test alert system.

Assessment of Health Information Technology-Related Outpatient Diagnostic Delays in the US Veterans Affairs Health Care System: A Qualitative Study of Aggregated Root Cause Analysis Data

IMPORTANCE

Diagnostic delay in the outpatient setting is an emerging safety priority that health information technology (HIT) should help address. However, diagnostic delays have persisted, and new safety concerns associated with the use of HIT have emerged.

OBJECTIVE

To analyze HIT-related outpatient diagnostic delays within a large, integrated health care system.

DESIGN, SETTING, AND PARTICIPANTS

This cohort study involved qualitative content analysis of safety concerns identified in aggregated root cause analysis (RCA) data related to HIT and outpatient diagnostic delays. The setting was the US Department of Veterans Affairs using all RCAs submitted to the Veterans Affairs (VA) National Center for Patient Safety from January 1, 2013, to July 31, 2018.

MAIN OUTCOMES AND MEASURES

Common themes associated with the role of HIT-related safety concerns were identified and categorized according to the Health IT Safety framework for measuring, monitoring, and improving HIT safety. This framework includes 3 related domains (ie, safe HIT, safe use of HIT, and using HIT to improve safety) situated within an 8-dimensional sociotechnical model accounting for interacting technical and nontechnical variables associated with safety. Hence, themes identified enhanced understanding of the sociotechnical context and domain of HIT safety involved.

RESULTS

Of 214 RCAs categorized by the terms delay and outpatient submitted during the study period, 88 were identified as involving diagnostic delays and HIT, from which 172 unique HIT-related safety concerns were extracted (mean [SD], 1.97 [1.53] per RCA). Most safety concerns (82.6% [142 of 172]) involved problems with safe use of HIT, predominantly sociotechnical factors associated with people, workflow and communication, and a poorly designed human-computer interface. Fewer safety concerns involved problems with safe HIT (14.5% [25 of 172]) or using HIT to improve safety (0.3% [5 of 172]). The following 5 key high-risk areas for diagnostic delays emerged: managing electronic health record inbox notifications and communication, clinicians gathering key diagnostic information, technical problems, data entry problems, and failure of a system to track test results.

CONCLUSIONS AND RELEVANCE

This qualitative study of a national RCA data set suggests that interventions to reduce outpatient diagnostic delays could aim to improve test result management, interoperability, data visualization, and order entry, as well as to decrease information overload.

Reducing Interruptive Alert Burden Using Quality Improvement Methodology

BACKGROUND

Increased adoption of electronic health records (EHR) with integrated clinical decision support (CDS) systems has reduced some sources of error but has led to unintended consequences including alert fatigue. The "pop-up" or interruptive alert is often employed as it requires providers to acknowledge receipt of an alert by taking an action despite the potential negative effects of workflow interruption. We noted a persistent upward trend of interruptive alerts at our institution and increasing requests for new interruptive alerts.

OBJECTIVES

Using Institute for Healthcare Improvement (IHI) quality improvement (QI) methodology, the primary objective was to reduce the total volume of interruptive alerts received by providers.

METHODS

We created an interactive dashboard for baseline alert data and to monitor frequency and outcomes of alerts as well as to prioritize interventions. A key driver diagram was developed with a specific aim to decrease the number of interruptive alerts from a baseline of 7,250 to 4,700 per week (35%) over 6 months. Interventions focused on the following key drivers: appropriate alert display within workflow, clear alert content, alert governance and standardization, user feedback regarding overrides, and respect for user knowledge.

RESULTS

A total of 25 unique alerts accounted for 90% of the total interruptive alert volume. By focusing on these 25 alerts, we reduced interruptive alerts from 7,250 to 4,400 per week.

CONCLUSION

Systematic and structured improvements to interruptive alerts can lead to overall reduced interruptive alert burden. Using QI methods to prioritize our interventions allowed us to maximize our impact. Further evaluation should be done on the effects of reduced interruptive alerts on patient care outcomes, usability heuristics on cognitive burden, and direct feedback mechanisms on alert utility.

PARTICIPATORY BULLSEYE TOOLKIT INTERVIEW: IDENTIFYING PHYSICIANS' RELATIVE PRIORITIZATION OF DECISION FACTORS WHEN ORDERING RADIOLOGIC IMAGING IN A HOSPITAL SETTING

Critical Decision Method (CDM), a popular cognitive task analysis (CTA) method, is an in-depth retrospective interview that uses a historical non-routine incident to identify experts' decision-making factors in complex socio-technical settings with high consequences for failure. However, it is challenging to use CDM to make comparisons, including those between experts and trainees. We describe an alternative CTA method used to study physicians' decision making for ordering diagnostic imaging. After being primed with 11 simulated patient scenarios, nine attending and 11 resident physicians were asked to map out and present their decision-making process with a bullseye participatory design toolkit. Interviews were analyzed qualitatively, revealing four common decision factors: diagnostic efficacy, patient safety, organizational constraints, and patient comfort. The bullseye maps were used to quantitatively measure priority differences between these decision factors. Attending and resident physicians both prioritized diagnostic efficacy over the other factors (2.38 vs. 3.71, p <.01, and 2.59 vs. 3.52, p<.01, respectively), but attending physicians' decisions had a higher proportion of non-diagnostic items (65% vs. 50%, p = .008). Our results demonstrate the usefulness of this method in eliciting decision factors for a complex, face-valid task and for identifying differences due to levels of expertise and training.

Principles of Automation for Patient Safety in Intensive Care: Learning From Aviation

BACKGROUND

The transition away from written documentation and analog methods has opened up the possibility of leveraging data science and analytic techniques to improve health care. In the implementation of data science techniques and methodologies, high-acuity patients in the ICU can particularly benefit. The Principles of Automation for Patient Safety in Intensive Care (PASPIC) framework draws on Billings's principles of human-centered aviation (HCA) automation and helps in identifying the advantages, pitfalls, and unintended consequences of automation in health care.

THE FRAMEWORK AND ITS KEY CHARACTERISTICS

Billings's HCA principles are based on the premise that human operators must remain "in command," so that they are continuously informed and actively involved in all aspects of system operations. In addition, automated systems need to be predictable, simple to train, to learn, and to operate, and must be able to monitor the human operators, and every intelligent system element must know the intent of other intelligent system elements. In applying Billings's HCA principles to the ICU setting, PAPSIC has three key characteristics: (1) integration and better interoperability, (2) multidimensional analysis, and (3) enhanced situation awareness.

RECOMMENDATIONS

PAPSIC suggests that health care professionals reduce overreliance on automation and implement "cooperative automation" and that vendors reduce mode errors and embrace interoperability.

CONCLUSION

Much can be learned from the aviation industry in automating the ICU. Because it combines "smart" technology with the necessary controls to withstand unintended consequences, PAPSIC could help ensure more informed decision making in the ICU and better patient care.

Introducing pharmacogenetic testing with clinical decision support into primary care: a feasibility study

BACKGROUND

Inappropriate prescribing increases patient illness and death owing to adverse drug events. The inclusion of genetic information into primary care medication practices is one solution. Our aim was to assess the ability to obtain and genotype saliva samples and to determine the levels of use of a decision support tool that creates medication options adjusted for patient characteristics, drug-drug interactions and pharmacogenetics.

METHODS

We conducted a cohort study in 6 primary care settings (5 family practices and 1 pharmacy), enrolling 191 adults with at least 1 of 10 common diseases. Saliva samples were obtained in the physician's office or pharmacy and sent to our laboratory, where DNA was extracted and genotyped and reports were generated. The reports were sent directly to the family physician/pharmacist and linked to an evidence-based prescribing decision support system. The primary outcome was ability to obtain and genotype samples. The secondary outcomes were yield and purity of DNA samples, ability to link results to decision support software and use of the decision support software.

RESULTS

Genotyping resulted in linking of 189 patients (99%) with pharmacogenetic reports to the decision support program. A total of 96.8% of samples had at least 1 actionable genotype for medications included in the decision support system. The medication support system was used by the physicians and pharmacists 236 times over 3 months.

INTERPRETATION

Physicians and pharmacists can collect saliva samples of sufficient quantity and quality for DNA extraction, purification and genotyping. A clinical decision support system with integrated data from pharmacogenetic tests may enable personalized prescribing within primary care. Trial registration: ClinicalTrials.gov, NCT02383290.