Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial

Introduction

Several methods have been developed to electronically monitor patients for severe sepsis, but few provide predictive capabilities to enable early intervention; furthermore, no severe sepsis prediction systems have been previously validated in a randomised study. We tested the use of a machine learning-based severe sepsis prediction system for reductions in average length of stay and in-hospital mortality rate.

Methods

We conducted a randomised controlled clinical trial at two medical-surgical intensive care units at the University of California, San Francisco Medical Center, evaluating the primary outcome of average length of stay, and secondary outcome of in-hospital mortality rate from December 2016 to February 2017. Adult patients (18+) admitted to participating units were eligible for this factorial, open-label study. Enrolled patients were assigned to a trial arm by a random allocation sequence. In the control group, only the current severe sepsis detector was used; in the experimental group, the machine learning algorithm (MLA) was also used. On receiving an alert, the care team evaluated the patient and initiated the severe sepsis bundle, if appropriate. Although participants were randomly assigned to a trial arm, group assignments were automatically revealed for any patients who received MLA alerts.

Results

Outcomes from 75 patients in the control and 67 patients in the experimental group were analysed. Average length of stay decreased from 13.0 days in the control to 10.3 days in the experimental group (p=0.042). In-hospital mortality decreased by 12.4 percentage points when using the MLA (p=0.018), a relative reduction of 58.0%. No adverse events were reported during this trial.

Conclusion

The MLA was associated with improved patient outcomes. This is the first randomised controlled trial of a sepsis surveillance system to demonstrate statistically significant differences in length of stay and in-hospital mortality.

Trial registration

NCT03015454.

A review of human factors principles for the design and implementation of medication safety alerts in clinical information systems

The objective of this review is to describe the implementation of human factors principles for the design of alerts in clinical information systems. First, we conduct a review of alarm systems to identify human factors principles that are employed in the design and implementation of alerts. Second, we review the medical informatics literature to provide examples of the implementation of human factors principles in current clinical information systems using alerts to provide medication decision support. Last, we suggest actionable recommendations for delivering effective clinical decision support using alerts. A review of studies from the medical informatics literature suggests that many basic human factors principles are not followed, possibly contributing to the lack of acceptance of alerts in clinical information systems. We evaluate the limitations of current alerting philosophies and provide recommendations for improving acceptance of alerts by incorporating human factors principles in their design.

Recommendations to improve the usability of drug-drug interaction clinical decision support alerts

OBJECTIVE

To establish preferred strategies for presenting drug-drug interaction (DDI) clinical decision support alerts.

MATERIALS AND METHODS

A DDI Clinical Decision Support Conference Series included a workgroup consisting of 24 clinical, usability, and informatics experts representing academia, health information technology (IT) vendors, healthcare organizations, and the Office of the National Coordinator for Health IT. Workgroup members met via web-based meetings 12 times from January 2013 to February 2014, and two in-person meetings to reach consensus on recommendations to improve decision support for DDIs. We addressed three key questions: (1) what, how, where, and when do we display DDI decision support? (2) should presentation of DDI decision support vary by clinicians? and (3) how should effectiveness of DDI decision support be measured?

RESULTS

Our recommendations include the consistent use of terminology, visual cues, minimal text, formatting, content, and reporting standards to facilitate usability. All clinicians involved in the medication use process should be able to view DDI alerts and actions by other clinicians. Override rates are common but may not be a good measure of effectiveness.

DISCUSSION

Seven core elements should be included with DDI decision support. DDI information should be presented to all clinicians. Finally, in their current form, override rates have limited capability to evaluate alert effectiveness.

CONCLUSION

DDI clinical decision support alerts need major improvements. We provide recommendations for healthcare organizations and IT vendors to improve the clinician interface of DDI alerts, with the aim of reducing alert fatigue and improving patient safety.

A systematic review of the types and causes of prescribing errors generated from using computerized provider order entry systems in primary and secondary care

Objective

To understand the different types and causes of prescribing errors associated with computerized provider order entry (CPOE) systems, and recommend improvements in these systems.

Materials and Methods

We conducted a systematic review of the literature published between January 2004 and June 2015 using three large databases: the Cumulative Index to Nursing and Allied Health Literature, Embase, and Medline. Studies that reported qualitative data about the types and causes of these errors were included. A narrative synthesis of all eligible studies was undertaken.

Results

A total of 1185 publications were identified, of which 34 were included in the review. We identified 8 key themes associated with CPOE-related prescribing errors: computer screen display, drop-down menus and auto-population, wording, default settings, nonintuitive or inflexible ordering, repeat prescriptions and automated processes, users' work processes, and clinical decision support systems. Displaying an incomplete list of a patient's medications on the computer screen often contributed to prescribing errors. Lack of system flexibility resulted in users employing error-prone workarounds, such as the addition of contradictory free-text comments. Users' misinterpretations of how text was presented in CPOE systems were also linked with the occurrence of prescribing errors.

Discussion and Conclusions

Human factors design is important to reduce error rates. Drop-down menus should be designed with safeguards to decrease the likelihood of selection errors. Development of more sophisticated clinical decision support, which can perform checks on free-text, may also prevent errors. Further research is needed to ensure that systems minimize error likelihood and meet users' workflow expectations.

The MOnitoring Resynchronization dEvices and CARdiac patiEnts (MORE-CARE) randomized controlled trial: phase 1 results on dynamics of early intervention with remote monitoring

BACKGROUND

Remote monitoring (RM) in patients with advanced heart failure and cardiac resynchronization therapy defibrillators (CRT-D) may reduce delays in clinical decisions by transmitting automatic alerts. However, this strategy has never been tested specifically in this patient population, with alerts for lung fluid overload, and in a European setting.

OBJECTIVE

The main objective of Phase 1 (presented here) is to evaluate if RM strategy is able to reduce time from device-detected events to clinical decisions.

METHODS

In this multicenter randomized controlled trial, patients with moderate to severe heart failure implanted with CRT-D devices were randomized to a Remote group (with remote follow-up and wireless automatic alerts) or to a Control group (with standard follow-up without alerts). The primary endpoint of Phase 1 was the delay between an alert event and clinical decisions related to the event in the first 154 enrolled patients followed for 1 year.

RESULTS

The median delay from device-detected events to clinical decisions was considerably shorter in the Remote group compared to the Control group: 2 (25(th)-75(th) percentile, 1-4) days vs 29 (25(th)-75(th) percentile, 3-51) days respectively, P=.004. In-hospital visits were reduced in the Remote group (2.0 visits/patient/year vs 3.2 visits/patient/year in the Control group, 37.5% relative reduction, P<.001). Automatic alerts were successfully transmitted in 93% of events occurring outside the hospital in the Remote group. The annual rate of all-cause hospitalizations per patient did not differ between the two groups (P=.65).

CONCLUSIONS

RM in CRT-D patients with advanced heart failure allows physicians to promptly react to clinically relevant automatic alerts and significantly reduces the burden of in-hospital visits.

TRIAL REGISTRATION

Clinicaltrials.gov NCT00885677; http://clinicaltrials.gov/show/NCT00885677 (Archived by WebCite at http://www.webcitation.org/6IkcCJ7NF).

More Than a Text Message: Dismantling Digital Triggers to Curate Behavior Change in Patient-Centered Health Interventions

Digital triggers such as text messages, emails, and push alerts are designed to focus an individual on a desired goal by prompting an internal or external reaction at the appropriate time. Triggers therefore have an essential role in engaging individuals with digital interventions delivered outside of traditional health care settings, where other events in daily lives and fluctuating motivation to engage in effortful behavior exist. There is an emerging body of literature examining the use of digital triggers for short-term action and longer-term behavior change. However, little attention has been given to understanding the components of digital triggers. Using tailoring as an overarching framework, we separated digital triggers into 5 primary components: (1) who (sender), (2) how (stimulus type, delivery medium, heterogeneity), (3) when (delivered), (4) how much (frequency, intensity), and (5) what (trigger's target, trigger's structure, trigger's narrative). We highlighted key considerations when tailoring each component and the pitfalls of ignoring common mistakes, such as alert fatigue and habituation. As evidenced throughout the paper, there is a broad literature base from which to draw when tailoring triggers to curate behavior change in health interventions. More research is needed, however, to examine differences in efficacy based on component tailoring, to best use triggers to facilitate behavior change over time, and to keep individuals engaged in physical and mental health behavior change efforts. Dismantling digital triggers into their component parts and reassembling them according to the gestalt of one's change goals is the first step in this development work.

Using a mobile health application to support self-management in chronic obstructive pulmonary disease: a six-month cohort study

BACKGROUND

Self-management strategies have the potential to support patients with chronic obstructive pulmonary disease (COPD). Telehealth interventions may have a role in delivering this support along with the opportunity to monitor symptoms and physiological variables. This paper reports findings from a six-month, clinical, cohort study of COPD patients' use of a mobile telehealth based (mHealth) application and how individually determined alerts in oxygen saturation levels, pulse rate and symptoms scores related to patient self-initiated treatment for exacerbations.

METHODS

The development of the mHealth intervention involved a patient focus group and multidisciplinary team of researchers, engineers and clinicians. Individual data thresholds to set alerts were determined, and the relationship to exacerbations, defined by the initiation of stand-by medications, was measured. The sample comprised 18 patients (age range of 50-85 years) with varied levels of computer skills.

RESULTS

Patients identified no difficulties in using the mHealth application and used all functions available. 40% of exacerbations had an alert signal during the three days prior to a patient starting medication. Patients were able to use the mHealth application to support self- management, including monitoring of clinical data. Within three months, 95% of symptom reporting sessions were completed in less than 100 s.

CONCLUSIONS

Home based, unassisted, daily use of the mHealth platform is feasible and acceptable to people with COPD for reporting daily symptoms and medicine use, and to measure physiological variables such as pulse rate and oxygen saturation. These findings provide evidence for integrating telehealth interventions with clinical care pathways to support self-management in COPD.

Criteria for assessing high-priority drug-drug interactions for clinical decision support in electronic health records

BACKGROUND

High override rates for drug-drug interaction (DDI) alerts in electronic health records (EHRs) result in the potentially dangerous consequence of providers ignoring clinically significant alerts. Lack of uniformity of criteria for determining the severity or validity of these interactions often results in discrepancies in how these are evaluated. The purpose of this study was to identify a set of criteria for assessing DDIs that should be used for the generation of clinical decision support (CDS) alerts in EHRs.

METHODS

We conducted a 20-year systematic literature review of MEDLINE and EMBASE to identify characteristics of high-priority DDIs. These criteria were validated by an expert panel consisting of medication knowledge base vendors, EHR vendors, in-house knowledge base developers from academic medical centers, and both federal and private agencies involved in the regulation of medication use.

RESULTS

Forty-four articles met the inclusion criteria for assessing characteristics of high-priority DDIs. The panel considered five criteria to be most important when assessing an interaction- Severity, Probability, Clinical Implications of the interaction, Patient characteristics, and the Evidence supporting the interaction. In addition, the panel identified barriers and considerations for being able to utilize these criteria in medication knowledge bases used by EHRs.

CONCLUSIONS

A multi-dimensional approach is needed to understanding the importance of an interaction for inclusion in medication knowledge bases for the purpose of CDS alerting. The criteria identified in this study can serve as a first step towards a uniform approach in assessing which interactions are critical and warrant interruption of a provider's workflow.

Efficacy of a clinical decision-support system in an HIV practice: a randomized trial

BACKGROUND

Data to support improved patient outcomes from clinical decision-support systems (CDSSs) are lacking in HIV care.

OBJECTIVE

To test the efficacy of a CDSS in improving HIV outcomes in an outpatient clinic.

DESIGN

Randomized, controlled trial. (ClinicalTrials.gov registration number: NCT00678600)

SETTING

Massachusetts General Hospital HIV Clinic.

PARTICIPANTS

HIV care providers and their patients.

INTERVENTION

Computer alerts were generated for virologic failure (HIV RNA level >400 copies/mL after a previous HIV RNA level ≤400 copies/mL), evidence of suboptimal follow-up, and 11 abnormal laboratory test results. Providers received interactive computer alerts, facilitating appointment rescheduling and repeated laboratory testing, for half of their patients and static alerts for the other half.

MEASUREMENTS

The primary end point was change in CD4 cell count. Other end points included time to clinical event, 6-month suboptimal follow-up, and severe laboratory toxicity.

RESULTS

Thirty-three HIV care providers followed 1011 patients with HIV. In the intervention group, the mean increase in CD4 cell count was greater (0.0053 vs. 0.0032 × 109 cells/L per month; difference, 0.0021 × 109 cells/L per month [95% CI, 0.0001 to 0.004]; P = 0.040) and the rate of 6-month suboptimal follow-up was lower (20.6 vs. 30.1 events per 100 patient-years; P = 0.022) than those in the control group. Median time to next scheduled appointment was shorter in the intervention group than in the control group after a suboptimal follow-up alert (1.71 vs. 3.48 months; P < 0.001) and after a toxicity alert (2.79 vs. >6 months; P = 0.072). More than 90% of providers supported adopting the CDSS as part of standard care.

LIMITATION

This was a 1-year informatics study conducted at a single hospital subspecialty clinic.

CONCLUSION

A CDSS using interactive provider alerts improved CD4 cell counts and clinic follow-up for patients with HIV. Wider implementation of such systems can provide important clinical benefits.

PRIMARY FUNDING SOURCE

National Institute of Allergy and Infectious Diseases.

The effect of medication cost transparency alerts on prescriber behavior

OBJECTIVE

The purpose of this study was to determine if medication cost transparency alerts provided at time of prescribing led ambulatory prescribers to reduce their use of low-value medications.

MATERIALS AND METHODS

Provider-level alerts were deployed to ambulatory practices of a single health system from February 2018 through April 2018. Practice sites included 58 primary care and 152 specialty care clinics totaling 1896 attending physicians, residents, and advanced practice nurses throughout western Washington. Prescribers in the randomly assigned intervention arm received a computerized alert whenever they ordered a medication among 4 high-cost medication classes. For each class, a lower cost, equally effective, and safe alternative was available. The primary outcome was the change in prescribing volume for each of the 4 selected medication classes during the 12-week intervention period relative to a prior 24-week baseline.

RESULTS

A total of 15 456 prescriptions for high-cost medications were written during the baseline period including 7223 in the intervention arm and 8233 in the control arm. During the intervention period, a decrease in daily prescribing volume was noted for all high-cost medications including 33% for clobetasol propionate (p < .0001), 59% for doxycycline hyclate (p < .0001), 43% for fluoxetine tablets (p < .0001), and a non-significant 3% decrease for high-cost triptans (p = .65). Prescribing volume for the high-cost medications overall decreased by 32% (p < .0001).

CONCLUSION

Medication cost transparency alerts in an ambulatory setting lead to more cost-conscious prescribing. Future work is needed to predict which alerts will be most effective.

The vulnerabilities of computerized physician order entry systems: a qualitative study

OBJECTIVE

To test the vulnerabilities of a wide range of computerized physician order entry (CPOE) systems to different types of medication errors, and develop a more comprehensive qualitative understanding of how their design could be improved.

MATERIALS AND METHODS

The authors reviewed a random sample of 63,040 medication error reports from the US Pharmacopeia (USP) MEDMARX reporting system where CPOE systems were considered a "contributing factor" to errors and flagged test scenarios that could be tested in current CPOE systems. Testers entered these orders in 13 commercial and homegrown CPOE systems across 16 different sites in the United States and Canada, using both usual practice and where-needed workarounds. Overarching themes relevant to interface design and usability/workflow issues were identified.

RESULTS

CPOE systems often failed to detect and prevent important medication errors. Generation of electronic alert warnings varied widely between systems, and depended on a number of factors, including how the order information was entered. Alerts were often confusing, with unrelated warnings appearing on the same screen as those more relevant to the current erroneous entry. Dangerous drug-drug interaction warnings were displayed only after the order was placed rather than at the time of ordering. Testers illustrated various workarounds that allowed them to enter these erroneous orders.

DISCUSSION AND CONCLUSION

The authors found high variability in ordering approaches between different CPOE systems, with major deficiencies identified in some systems. It is important that developers reflect on these findings and build in safeguards to ensure safer prescribing for patients.

Clinical decisions support malfunctions in a commercial electronic health record

OBJECTIVES

Determine if clinical decision support (CDS) malfunctions occur in a commercial electronic health record (EHR) system, characterize their pathways and describe methods of detection.

METHODS

We retrospectively examined the firing rate for 226 alert type CDS rules for detection of anomalies using both expert visualization and statistical process control (SPC) methods over a five year period. Candidate anomalies were investigated and validated.

RESULTS

Twenty-one candidate CDS anomalies were identified from 8,300 alert-months. Of these candidate anomalies, four were confirmed as CDS malfunctions, eight as false-positives, and nine could not be classified. The four CDS malfunctions were a result of errors in knowledge management: 1) inadvertent addition and removal of a medication code to the electronic formulary list; 2) a seasonal alert which was not activated; 3) a change in the base data structures; and 4) direct editing of an alert related to its medications. 154 CDS rules (68%) were amenable to SPC methods and the test characteristics were calculated as a sensitivity of 95%, positive predictive value of 29% and F-measure 0.44.

DISCUSSION

CDS malfunctions were found to occur in our EHR. All of the pathways for these malfunctions can be described as knowledge management errors. Expert visualization is a robust method of detection, but is resource intensive. SPC-based methods, when applicable, perform reasonably well retrospectively.

CONCLUSION

CDS anomalies were found to occur in a commercial EHR and visual detection along with SPC analysis represents promising methods of malfunction detection.

Structured override reasons for drug-drug interaction alerts in electronic health records

OBJECTIVE

The study sought to determine availability and use of structured override reasons for drug-drug interaction (DDI) alerts in electronic health records.

MATERIALS AND METHODS

We collected data on DDI alerts and override reasons from 10 clinical sites across the United States using a variety of electronic health records. We used a multistage iterative card sort method to categorize the override reasons from all sites and identified best practices.

RESULTS

Our methodology established 177 unique override reasons across the 10 sites. The number of coded override reasons at each site ranged from 3 to 100. Many sites offered override reasons not relevant to DDIs. Twelve categories of override reasons were identified. Three categories accounted for 78% of all overrides: "will monitor or take precautions," "not clinically significant," and "benefit outweighs risk."

DISCUSSION

We found wide variability in override reasons between sites and many opportunities to improve alerts. Some override reasons were irrelevant to DDIs. Many override reasons attested to a future action (eg, decreasing a dose or ordering monitoring tests), which requires an additional step after the alert is overridden, unless the alert is made actionable. Some override reasons deferred to another party, although override reasons often are not visible to other users. Many override reasons stated that the alert was inaccurate, suggesting that specificity of alerts could be improved.

CONCLUSIONS

Organizations should improve the options available to providers who choose to override DDI alerts. DDI alerting systems should be actionable and alerts should be tailored to the patient and drug pairs.

Asynchronous automated electronic laboratory result notifications: a systematic review

Objective

To systematically review the literature pertaining to asynchronous automated electronic notifications of laboratory results to clinicians.

Methods

PubMed, Web of Science, and the Cochrane Collaboration were queried for studies pertaining to automated electronic notifications of laboratory results. A title review was performed on the primary results, with a further abstract review and full review to produce the final set of included articles.

Results

The full review included 34 articles, representing 19 institutions. Of these, 19 reported implementation and design of systems, 11 reported quasi-experimental studies, 3 reported a randomized controlled trial, and 1 was a meta-analysis. Twenty-seven articles included alerts of critical results, while 5 focused on urgent notifications and 2 on elective notifications. There was considerable variability in clinical setting, system implementation, and results presented.

Conclusion

Several asynchronous automated electronic notification systems for laboratory results have been evaluated, most from >10 years ago. Further research on the effect of notifications on clinicians as well as the use of modern electronic health records and new methods of notification is warranted to determine their effects on workflow and clinical outcomes.

Practical considerations in the use of real-time continuous glucose monitoring alerts

BACKGROUND

The safety and efficacy of real-time (RT) continuous glucose monitoring (CGM) systems in the management of type 1 diabetes are increasingly apparent. Clinical trials have demonstrated the utility of these systems in lowering hemoglobin A1c, minimizing hypoglycemia, and reducing glycemic variability. These RT systems allow patients to conveniently monitor their glucose levels by displaying concentration and trending information. Several of these RT systems provide preset alerts that sound when absolute glucose thresholds are reached. Additionally, some systems allow for predictive algorithm-based alerts that incorporate rates of change. However, clinical trials have identified significant noncompliance in the use of these devices, most notably in the pediatric and adolescent populations. A retrospective review of CGM reports shows that many patients set high and low alert thresholds at levels that result in frequent alerts, potentially resulting in patient nuisance, dismissal of consequential alerts, and eventual product abandonment. Therefore, setting the alert thresholds at appropriate high and low settings can determine the balance between either a perceived benefit by the patient and their long-term use of CGM systems or annoyance to the patient and discontinuation.

CONCLUSION

Care should be taken to set CGM alerts at levels that result in a manageable number of notifications per day. In some cases, providers should consider not using alerts at all or consider using broad targets when initiating CGM to maximize alert specificity. Real-time CGM is safe and generally well tolerated; however, individualization of alert settings is necessary maximize the system's benefits and patient adherence.

Evaluating a digital sepsis alert in a London multisite hospital network: a natural experiment using electronic health record data

Objective

The study sought to determine the impact of a digital sepsis alert on patient outcomes in a UK multisite hospital network.

Materials and Methods

A natural experiment utilizing the phased introduction (without randomization) of a digital sepsis alert into a multisite hospital network. Sepsis alerts were either visible to clinicians (patients in the intervention group) or running silently and not visible (the control group). Inverse probability of treatment-weighted multivariable logistic regression was used to estimate the effect of the intervention on individual patient outcomes.

Outcomes

In-hospital 30-day mortality (all inpatients), prolonged hospital stay (≥7 days) and timely antibiotics (≤60 minutes of the alert) for patients who alerted in the emergency department.

Results

The introduction of the alert was associated with lower odds of death (odds ratio, 0.76; 95% confidence interval [CI], 0.70-0.84; n = 21 183), lower odds of prolonged hospital stay ≥7 days (OR, 0.93; 95% CI, 0.88-0.99; n = 9988), and in patients who required antibiotics, an increased odds of receiving timely antibiotics (OR, 1.71; 95% CI, 1.57-1.87; n = 4622).

Discussion

Current evidence that digital sepsis alerts are effective is mixed. In this large UK study, a digital sepsis alert has been shown to be associated with improved outcomes, including timely antibiotics. It is not known whether the presence of alerting is responsible for improved outcomes or whether the alert acted as a useful driver for quality improvement initiatives.

Conclusions

These findings strongly suggest that the introduction of a network-wide digital sepsis alert is associated with improvements in patient outcomes, demonstrating that digital based interventions can be successfully introduced and readily evaluated.

Mixed methods study of medication-related decision support alerts experienced during electronic prescribing for inpatients at an English hospital

Objectives

Electronic prescribing and medication administration systems are being introduced in many hospitals worldwide, with varying degrees of clinical decision support including pop-up alerts. Previous research suggests that prescribers override a high proportion of alerts, but little research has been carried out in the UK. Our objective was to explore rates of alert overriding in different prescribing situations and prescribers’ perceptions around the use of decision support alerts in a UK hospital.

Methods

We conducted a mixed methods study on three cardiology wards, directly observing medical and non-medical prescribers’ alert override rates during both ward round and non-ward round prescribing; observations were followed by semi-structured interviews with prescribers, which were then transcribed and analysed thematically.

Results

Overall, 69% of 199 observed alerts were overridden. Alerts experienced during ward rounds were significantly more likely to be overridden than those outside of ward rounds (80% of 56 vs 51% of 63; p=0.001, Χ² test). While respondents acknowledged that alerts could be useful, several also described negative unintended consequences. Many were of the view that usefulness of alerts was limited if the alert was reminding them to do something they would do anyway, or suggesting something they did not feel was relevant. Findings suggest that targeting, timing and additional features of alerts are critical factors in determining whether they are acted on or overridden.

Conclusion

The majority of alerts were overridden. Alerts may be less likely to be overridden if they are built into the prescribing workflow.

Computerized pharmacy surveillance and alert system for drug-related problems

WHAT IS KNOWN AND OBJECTIVE

Because of the impact of drug-related problems (DRPs) on morbidity and mortality, there is a need for computerized strategies to increase drug safety. The detection and identification of the causes of potential DRPs can be facilitated by the incorporation of a pharmacy warning system (PWS) in the computerized prescriber order entry (CPOE) and its application in the routine validation of inpatient drug therapy. A limited number of studies have evaluated a clinical decision support system to monitor drug treatment. Most of these applications have utilized a small range of drugs with alerts and/or types of alert. The objective of this study was to describe the implementation of a PWS integrated in the electronic medical record (EMR).

METHODS

The PWS was developed in 2003-2004. Pharmacological information to generate drug alerts was entered on demographic data, drug dosage, laboratory tests related to the prescribed drug and drug combinations (interactions, duplications and necessary combinations). The PWS was applied in the prescription reviews conducted in patients admitted to the hospital in 2012.

RESULTS AND DISCUSSION

Information on 83% of the drugs included in the pharmacopeia was introduced into the PWS, allowing detection of 2808 potential DRPs, representing 79·1% of all potential DRPs detected during the study period. Twenty per cent of PWS DRPs were clinically relevant, requiring pharmacist intervention.

WHAT IS NEW AND CONCLUSION

The PWS detected most potential DRPs, thus increasing inpatient safety. The detection ability of the PWS was higher than that reported for other tools described in the literature.

The frequency of inappropriate nonformulary medication alert overrides in the inpatient setting

BACKGROUND

Experts suggest that formulary alerts at the time of medication order entry are the most effective form of clinical decision support to automate formulary management.

OBJECTIVE

Our objectives were to quantify the frequency of inappropriate nonformulary medication (NFM) alert overrides in the inpatient setting and provide insight on how the design of formulary alerts could be improved.

METHODS

Alert overrides of the top 11 (n = 206) most-utilized and highest-costing NFMs, from January 1 to December 31, 2012, were randomly selected for appropriateness evaluation. Using an empirically developed appropriateness algorithm, appropriateness of NFM alert overrides was assessed by 2 pharmacists via chart review. Appropriateness agreement of overrides was assessed with a Cohen's kappa. We also assessed which types of NFMs were most likely to be inappropriately overridden, the override reasons that were disproportionately provided in the inappropriate overrides, and the specific reasons the overrides were considered inappropriate.

RESULTS

Approximately 17.2% (n = 35.4/206) of NFM alerts were inappropriately overridden. Non-oral NFM alerts were more likely to be inappropriately overridden compared to orals. Alerts overridden with "blank" reasons were more likely to be inappropriate. The failure to first try a formulary alternative was the most common reason for alerts being overridden inappropriately.

CONCLUSION

Approximately 1 in 5 NFM alert overrides are overridden inappropriately. Future research should evaluate the impact of mandating a valid override reason and adding a list of formulary alternatives to each NFM alert; we speculate these NFM alert features may decrease the frequency of inappropriate overrides.

Intrusion-aware alert validation algorithm for cooperative distributed intrusion detection schemes of wireless sensor networks

Existing anomaly and intrusion detection schemes of wireless sensor networks have mainly focused on the detection of intrusions. Once the intrusion is detected, an alerts or claims will be generated. However, any unidentified malicious nodes in the network could send faulty anomaly and intrusion claims about the legitimate nodes to the other nodes. Verifying the validity of such claims is a critical and challenging issue that is not considered in the existing cooperative-based distributed anomaly and intrusion detection schemes of wireless sensor networks. In this paper, we propose a validation algorithm that addresses this problem. This algorithm utilizes the concept of intrusion-aware reliability that helps to provide adequate reliability at a modest communication cost. In this paper, we also provide a security resiliency analysis of the proposed intrusion-aware alert validation algorithm.

Smart Electronic Eyedrop Bottle for Unobtrusive Monitoring of Glaucoma Medication Adherence

Glaucoma, the leading cause of irreversible blindness, affects >70 million people worldwide. Lowering intraocular pressure via topical administration of eye drops is the most common first-line therapy for glaucoma. This treatment paradigm has notoriously high non-adherence rates: ranging from 30% to 80%. The advent of smart phone enabled technologies creates promise for improving eyedrop adherence. However, previous eyedrop electronic monitoring solutions had awkward medication bottle adjuncts and crude software for monitoring the administration of a drop that adversely affected their ability to foster sustainable improvements in adherence. The current work begins to address this unmet need for wireless technology by creating a “smart drop” bottle. This medication bottle is instrumented with sensing electronics that enable detection of each eyedrop administered while maintaining the shape and size of the bottle. This is achieved by a thin electronic force sensor wrapped around the bottle and underneath the label, interfaced with a thin electronic circuit underneath the bottle that allows for detection and wireless transmission to a smart-phone application. We demonstrate 100% success rate of wireless communication over 75 feet with <1% false positive and false negative rates of single drop deliveries, thus providing a viable solution for eyedrop monitoring for glaucoma patients.

Improving the specificity of drug-drug interaction alerts: Can it be done?

PURPOSE

Inaccurate and nonspecific medication alerts contribute to high override rates, alert fatigue, and ultimately patient harm. Drug-drug interaction (DDI) alerts often fail to account for factors that could reduce risk; further, drugs that trigger alerts are often inconsistently grouped into value sets. Toward improving the specificity of DDI alerts, the objectives of this study were to (1) highlight the inconsistency of drug value sets for triggering DDI alerts and (2) demonstrate a method of classifying factors that can be used to modify the risk of harm from a DDI.

METHODS

This was a proof-of-concept study focused on 15 well-known DDIs. Using 3 drug interaction references, we extracted 2 drug value sets and any available order- and patient-related factors for each DDI. Fleiss' kappa was used to measure the consistency of value sets among references. Risk-modifying factors were classified as order parameters (eg, route and dose) or patient characteristics (eg, comorbidities and laboratory results).

RESULTS

Seventeen value sets (56%) had nonsignificant agreement. Agreement among the remaining 13 value sets was on average moderate. Thirty-three factors that could reduce risk in 14 of 15 DDIs (93%) were identified. Most risk-modifying factors (67%) were classified as order parameters.

CONCLUSION

This study demonstrates the importance of increasing the consistency of drug value sets that trigger DDI alerts and how alert specificity and usefulness can be improved with risk-modifying factors obtained from drug references. It may be difficult to operationalize certain factors to reduce unnecessary alerts; however, factors can be used to support decisions by providing contextual information.

Cognitive analysis of decision support for antibiotic ordering in a neonatal intensive care unit

BACKGROUND

Clinical decision support systems (CDSS) are a method used to support prescribing accuracy when deployed within a computerized provider order entry system (CPOE). Divergence from using CDSS is exemplified by high alert override rates. Excessive cognitive load imposed by the CDSS may help to explain such high rates.

OBJECTIVES

The aim of this study was to describe the cognitive impact of a CPOE-integrated CDSS by categorizing system use problems according to the type of mental processing required to resolve them.

METHODS

A qualitative, descriptive design was used employing two methods; a cognitive walkthrough and a think-aloud protocol. Data analysis was guided by Norman's Theory of Action and a theory of cognitive distances which is an extension to Norman's theory.

RESULTS

The most frequently occurring source of excess cognitive effort was poor information timing. Information presented by the CDSS was often presented after clinicians required the information for decision making. Additional sources of effort included use of language that was not clear to the user, vague icons, and lack of cues to guide users through tasks.

CONCLUSIONS

Lack of coordination between clinician's task-related thought processes and those presented by a CDSS results in excessive cognitive work required to use the system. This can lead to alert overrides and user errors. Close attention to user's cognitive processes as they carry out clinical tasks prior to CDSS development may provide key information for system design that supports clinical tasks and reduces cognitive effort.