Understanding User Acceptance of Clinical Decision Support Systems to Promote Increased Cancer Screening Rates in a Primary Care Practice

OBJECTIVE

Clinical decision support systems (CDDSs) in the electronic medical record (EMR) have been implemented in primary care settings to identify patients due for cancer screening tests, while functioning as a real time reminder system. There is little known about primary care providers (PCPs) perspective or user acceptance of CDSS. The purpose of this study was to investigate primary care provider perceptions of utilizing CDSS alerts in the EMR to promote increased screening rates for breast cancer, cervical cancer, and colorectal cancer.

METHODS

An electronic survey was administered to PCPs in a Midwest Health Institution community internal medicine practice from September 25, 2019 through November 27, 2019.

RESULTS

Among 37 participants (9 NP/Pas and 28 MD/DOs), the NP/PA group was more likely to agree that alerts were helpful (50%; P-value = .0335) and the number of alerts (89%; P = .0227) in the EMR was appropriate. The NP/PA group also was more likely to find alerts straightforward to use (78%, P = .0239). Both groups agreed about feeling comfortable using the health maintenance alerts (MD/DO = 79%; NP/PA = 100%).

CONCLUSION

CDSSs can promote and facilitate ordering of cancer screening tests. The use of technology can promptly identify patients due for a test and act as a reminder to the PCP. PCPs identify these alerts to be a beneficial tool in the EMR when they do not interrupt workflow and provide value to patient care. More work is needed to identify factors that could optimize alerts to be even more helpful, particularly to MD/DO groups.

Designing Interactive Alerts to Improve Recognition of Critical Events in Medical Emergencies

Vital sign values during medical emergencies can help clinicians recognize and treat patients with life-threatening injuries. Identifying abnormal vital signs, however, is frequently delayed and the values may not be documented at all. In this mixed-methods study, we designed and evaluated a two-phased visual alert approach for a digital checklist in trauma resuscitation that informs users about undocumented vital signs. Using an interrupted time series analysis, we compared documentation in the periods before (two years) and after (four months) the introduction of the alerts. We found that introducing alerts led to an increase in documentation throughout the post-intervention period, with clinicians documenting vital signs earlier. Interviews with users and video review of cases showed that alerts were ineffective when clinicians engaged less with the checklist or set the checklist down to perform another activity. From these findings, we discuss approaches to designing alerts for dynamic team-based settings.

Evaluating a digital sepsis alert in a London multisite hospital network: a natural experiment using electronic health record data

Objective

The study sought to determine the impact of a digital sepsis alert on patient outcomes in a UK multisite hospital network.

Materials and Methods

A natural experiment utilizing the phased introduction (without randomization) of a digital sepsis alert into a multisite hospital network. Sepsis alerts were either visible to clinicians (patients in the intervention group) or running silently and not visible (the control group). Inverse probability of treatment-weighted multivariable logistic regression was used to estimate the effect of the intervention on individual patient outcomes.

Outcomes

In-hospital 30-day mortality (all inpatients), prolonged hospital stay (≥7 days) and timely antibiotics (≤60 minutes of the alert) for patients who alerted in the emergency department.

Results

The introduction of the alert was associated with lower odds of death (odds ratio, 0.76; 95% confidence interval [CI], 0.70-0.84; n = 21 183), lower odds of prolonged hospital stay ≥7 days (OR, 0.93; 95% CI, 0.88-0.99; n = 9988), and in patients who required antibiotics, an increased odds of receiving timely antibiotics (OR, 1.71; 95% CI, 1.57-1.87; n = 4622).

Discussion

Current evidence that digital sepsis alerts are effective is mixed. In this large UK study, a digital sepsis alert has been shown to be associated with improved outcomes, including timely antibiotics. It is not known whether the presence of alerting is responsible for improved outcomes or whether the alert acted as a useful driver for quality improvement initiatives.

Conclusions

These findings strongly suggest that the introduction of a network-wide digital sepsis alert is associated with improvements in patient outcomes, demonstrating that digital based interventions can be successfully introduced and readily evaluated.

The effect of medication cost transparency alerts on prescriber behavior

OBJECTIVE

The purpose of this study was to determine if medication cost transparency alerts provided at time of prescribing led ambulatory prescribers to reduce their use of low-value medications.

MATERIALS AND METHODS

Provider-level alerts were deployed to ambulatory practices of a single health system from February 2018 through April 2018. Practice sites included 58 primary care and 152 specialty care clinics totaling 1896 attending physicians, residents, and advanced practice nurses throughout western Washington. Prescribers in the randomly assigned intervention arm received a computerized alert whenever they ordered a medication among 4 high-cost medication classes. For each class, a lower cost, equally effective, and safe alternative was available. The primary outcome was the change in prescribing volume for each of the 4 selected medication classes during the 12-week intervention period relative to a prior 24-week baseline.

RESULTS

A total of 15 456 prescriptions for high-cost medications were written during the baseline period including 7223 in the intervention arm and 8233 in the control arm. During the intervention period, a decrease in daily prescribing volume was noted for all high-cost medications including 33% for clobetasol propionate (p < .0001), 59% for doxycycline hyclate (p < .0001), 43% for fluoxetine tablets (p < .0001), and a non-significant 3% decrease for high-cost triptans (p = .65). Prescribing volume for the high-cost medications overall decreased by 32% (p < .0001).

CONCLUSION

Medication cost transparency alerts in an ambulatory setting lead to more cost-conscious prescribing. Future work is needed to predict which alerts will be most effective.

Smart Electronic Eyedrop Bottle for Unobtrusive Monitoring of Glaucoma Medication Adherence

Glaucoma, the leading cause of irreversible blindness, affects >70 million people worldwide. Lowering intraocular pressure via topical administration of eye drops is the most common first-line therapy for glaucoma. This treatment paradigm has notoriously high non-adherence rates: ranging from 30% to 80%. The advent of smart phone enabled technologies creates promise for improving eyedrop adherence. However, previous eyedrop electronic monitoring solutions had awkward medication bottle adjuncts and crude software for monitoring the administration of a drop that adversely affected their ability to foster sustainable improvements in adherence. The current work begins to address this unmet need for wireless technology by creating a “smart drop” bottle. This medication bottle is instrumented with sensing electronics that enable detection of each eyedrop administered while maintaining the shape and size of the bottle. This is achieved by a thin electronic force sensor wrapped around the bottle and underneath the label, interfaced with a thin electronic circuit underneath the bottle that allows for detection and wireless transmission to a smart-phone application. We demonstrate 100% success rate of wireless communication over 75 feet with <1% false positive and false negative rates of single drop deliveries, thus providing a viable solution for eyedrop monitoring for glaucoma patients.

Using journal alerts to support your continuing professional development

There are many ways to keep up to date with research that affects your role and personal development. You can regularly use PubMed or Scholar to find recent papers using keyword searches, you can rely on others to do the work for you with literature reviews, share the job with Journal clubs or using Journal alerts you can have the papers and research you want delivered to your inbox.

The Incidence of Paediatric Acute Kidney Injury Identified Using an AKI E-Alert Algorithm in Six English Hospitals

Objective: Acute kidney injury (AKI) is a significant cause of morbidity and mortality among hospitalised patients. The objectives in this study were (i) to investigate the incidence of AKI using the National Health Services (NHS) AKI e-alert algorithm as a means of identifying AKI; and (ii) in a randomly selected sub-group of children with AKI identified using the algorithm, to evaluate the recognition and management of AKI. Patients and Methods: Retrospective cross-sectional study with initial electronic retrieval of creatinine measurements at six hospitals in England over a six-month period. Results were evaluated using the NHS AKI e-alert algorithm with recognition and management of AKI stages 1, 2 and 3 reviewed in a sub-set of randomly selected patient case notes. Patients aged 29 to 17 years were included. AKI stage 1 was defined as a rise of 1.5 - ≤2x baseline creatinine level; AKI stage 2 a rise of ≤ 2.0 and < 3.0; AKI stage 3 a rise of ≥ 3.0. Urine output was not considered for AKI staging. Results: 57,278 creatinine measurements were analysed. 5,325 (10.8%) AKI alerts were noted in 1,112 patients with AKI 1 (62%), AKI 2 (16%) and AKI 3 (22%). There were 222 (20%) <1y, 432 (39%) 1 ≤ 6y, 192 (17%) 6 ≤ 11y, 207 (19%) 11 ≤ 16y, and 59 (5%) 16-17y. Case notes of 123 of 1,112 [11.1%] children with AKI alerts were reviewed. Confirmed AKI was recognised with a documented management plan following its identification in n = 32 [26%] patients only. Conclusions: In this first multicentre study of the incidence of AKI in children admitted to selected hospitals across England, the incidence of AKI was 10.8% with most patients under the age of 6 years and with AKI stage 1. Recognition and management of AKI was seen in just over 25% children. These data highlight the need to improve recognition of AKI in hospitalised children in the UK.

Real-World Hypoglycemia Avoidance With a Predictive Low Glucose Alert Does Not Depend on Frequent Screen Views

BACKGROUND

Frequent real-time continuous glucose monitoring (rtCGM) data viewing has been associated with reduced mean glucose and frequent scanning of an intermittently scanned continuous glucose monitoring (isCGM) system has been associated with reduced hypoglycemia for patients with diabetes. However, requiring patients to frequently interact with their glucose monitoring devices to detect actual or impending hypoglycemia is burdensome. We hypothesized that a predictive low glucose alert, which forecasts glucose ≤55 mg/dL within 20 minutes and is included in a new rtCGM system, could mitigate hypoglycemia without requiring frequent device interaction.

METHODS

We analyzed estimated glucose values (EGVs) from an anonymized convenience sample of 15,000 patients who used Dexcom G6 (Dexcom, Inc, San Diego, CA, USA) and its mobile app for at least 30 days with or without the "Urgent Low Soon" alert (ULS) enabled. Screen view frequency was determined as the frequency with which the trend screen was accessed on the app. Multiple screen views within any 5-minute interval were counted as one. Hypoglycemia exposure for patients in the top and bottom quartiles of screen view frequency (>8.25 and <3.30 per day, respectively) was calculated as the percentage of EGVs below various thresholds.

RESULTS

Over 93% of users enabled the ULS alert; its use was associated with significantly reduced hypoglycemia <55 and <70 mg/dL, independent of screen view frequency.

CONCLUSION

Use of the G6 ULS alert may disencumber rtCGM users by promoting significant reductions in hypoglycemia without requiring frequent device interactions.

Structured override reasons for drug-drug interaction alerts in electronic health records

Objective

The study sought to determine availability and use of structured override reasons for drug-drug interaction (DDI) alerts in electronic health records.

Materials and Methods

We collected data on DDI alerts and override reasons from 10 clinical sites across the United States using a variety of electronic health records. We used a multistage iterative card sort method to categorize the override reasons from all sites and identified best practices.

Results

Our methodology established 177 unique override reasons across the 10 sites. The number of coded override reasons at each site ranged from 3 to 100. Many sites offered override reasons not relevant to DDIs. Twelve categories of override reasons were identified. Three categories accounted for 78% of all overrides: “will monitor or take precautions,” “not clinically significant,” and “benefit outweighs risk.”

Discussion

We found wide variability in override reasons between sites and many opportunities to improve alerts. Some override reasons were irrelevant to DDIs. Many override reasons attested to a future action (eg, decreasing a dose or ordering monitoring tests), which requires an additional step after the alert is overridden, unless the alert is made actionable. Some override reasons deferred to another party, although override reasons often are not visible to other users. Many override reasons stated that the alert was inaccurate, suggesting that specificity of alerts could be improved.

Conclusions

Organizations should improve the options available to providers who choose to override DDI alerts. DDI alerting systems should be actionable and alerts should be tailored to the patient and drug pairs.

Do user preferences align with human factors assessment scores of drug-drug interaction alerts?

This study aimed to assess drug-drug interaction alert interfaces and to examine the relationship between compliance with human factors principles and user-preferences of alerts. Three reviewers independently evaluated drug-drug interaction alert interfaces in seven electronic systems using the Instrument-for-Evaluating-Human-Factors-Principles-in-Medication-Related-Decision-Support-Alerts (I-MeDeSA). Fifty-three doctors and pharmacists completed a survey to rate the alert interfaces from best to worst and reported on liked and disliked features. Human factors compliance and user-preferences of alerts were compared. Statistical analysis revealed no significant association between I-MeDeSA scores and user-preferences. However, the strengths and weaknesses of drug-drug interaction alerts from users' perspectives were in-line with the human factors constructs evaluated by the I-MeDeSA. I-MeDeSA in its current form, is unable to identify alerts that are preferred by the users. The design principles assessed by I-MeDeSA appear to be sound, but its arbitrary allocation of points to each human factors construct may not reflect the relative importance that the end-users place on different aspects of alert design.

Timely Interventions for Children with ADHD through Web-Based Monitoring Algorithms

The aim of this study was to evaluate an automated trigger algorithm designed to detect potentially adverse events in children with Attention-Deficit/Hyperactivity Disorder (ADHD), who were monitored remotely between visits. We embedded a trigger algorithm derived from parent-reported ADHD rating scales within an electronic patient monitoring system. We categorized clinicians’ alert resolution outcomes and compared Vanderbilt ADHD rating scale scores between patients who did or did not have triggered alerts. A total of 146 out of 1738 parent reports (8%) triggered alerts for 98 patients. One hundred and eleven alerts (76%) required immediate clinician review. Nurses successfully contacted parents for 68 (61%) of actionable alerts; 46% (31/68) led to a change in care plan prior to the next scheduled appointment. Compared to patients without alerts, patients with alerts demonstrated worsened ADHD severity (β = 5.8, 95% CI: 3.5–8.1 [p < 0.001] within 90 days prior to an alert. The trigger algorithm facilitated timely changes in the care plan in between face-to-face visits.

The Triad of Risk-Related Behaviors (TriRB): A Three-Dimensional Model of Cyber Risk Taking

OBJECTIVE

We identify three risk-related behaviors in coping with cyber threats-the exposure to risk a person chooses, use of security features, and responses to security indications. The combinations of behaviors that users choose determine how well they cope with threats and the severity of adverse events they experience.

BACKGROUND

End users' coping with risks is a major factor in cybersecurity. This behavior results from a combination of risk-related behaviors rather than from a single risk-taking tendency.

METHOD

In two experiments, participants played a Tetris-like game, attempting to maximize their gains, while exogenous occasional attacks could diminish earnings. An alerting system provided indications about possible attacks, and participants could take protective actions to limit the losses from attacks.

RESULTS

Variables such as the costs of protective actions, reliability of the alerting system, and attack severity affected the three behaviors differently. Also, users dynamically adjusted each of the three risk-related behaviors after gaining experience with the system.

CONCLUSION

The results demonstrate that users' risk taking is the complex combination of three behaviors rather than the expression of a general risk-taking tendency. The use of security features, exposure to risk, and responses to security indications reflect long-term strategy, short-term tactical decisions, and immediate maneuvering in coping with risks in dynamic environments.

APPLICATION

The results have implications for the analysis of cybersecurity-related decisions and actions as well as for the evaluation and design of systems and targeted interventions in other domains.

Mixed methods study of medication-related decision support alerts experienced during electronic prescribing for inpatients at an English hospital

Objectives

Electronic prescribing and medication administration systems are being introduced in many hospitals worldwide, with varying degrees of clinical decision support including pop-up alerts. Previous research suggests that prescribers override a high proportion of alerts, but little research has been carried out in the UK. Our objective was to explore rates of alert overriding in different prescribing situations and prescribers’ perceptions around the use of decision support alerts in a UK hospital.

Methods

We conducted a mixed methods study on three cardiology wards, directly observing medical and non-medical prescribers’ alert override rates during both ward round and non-ward round prescribing; observations were followed by semi-structured interviews with prescribers, which were then transcribed and analysed thematically.

Results

Overall, 69% of 199 observed alerts were overridden. Alerts experienced during ward rounds were significantly more likely to be overridden than those outside of ward rounds (80% of 56 vs 51% of 63; p=0.001, Χ² test). While respondents acknowledged that alerts could be useful, several also described negative unintended consequences. Many were of the view that usefulness of alerts was limited if the alert was reminding them to do something they would do anyway, or suggesting something they did not feel was relevant. Findings suggest that targeting, timing and additional features of alerts are critical factors in determining whether they are acted on or overridden.

Conclusion

The majority of alerts were overridden. Alerts may be less likely to be overridden if they are built into the prescribing workflow.

Asynchronous automated electronic laboratory result notifications: a systematic review

Objective

To systematically review the literature pertaining to asynchronous automated electronic notifications of laboratory results to clinicians.

Methods

PubMed, Web of Science, and the Cochrane Collaboration were queried for studies pertaining to automated electronic notifications of laboratory results. A title review was performed on the primary results, with a further abstract review and full review to produce the final set of included articles.

Results

The full review included 34 articles, representing 19 institutions. Of these, 19 reported implementation and design of systems, 11 reported quasi-experimental studies, 3 reported a randomized controlled trial, and 1 was a meta-analysis. Twenty-seven articles included alerts of critical results, while 5 focused on urgent notifications and 2 on elective notifications. There was considerable variability in clinical setting, system implementation, and results presented.

Conclusion

Several asynchronous automated electronic notification systems for laboratory results have been evaluated, most from >10 years ago. Further research on the effect of notifications on clinicians as well as the use of modern electronic health records and new methods of notification is warranted to determine their effects on workflow and clinical outcomes.

A systematic review of the types and causes of prescribing errors generated from using computerized provider order entry systems in primary and secondary care

Objective

To understand the different types and causes of prescribing errors associated with computerized provider order entry (CPOE) systems, and recommend improvements in these systems.

Materials and Methods

We conducted a systematic review of the literature published between January 2004 and June 2015 using three large databases: the Cumulative Index to Nursing and Allied Health Literature, Embase, and Medline. Studies that reported qualitative data about the types and causes of these errors were included. A narrative synthesis of all eligible studies was undertaken.

Results

A total of 1185 publications were identified, of which 34 were included in the review. We identified 8 key themes associated with CPOE-related prescribing errors: computer screen display, drop-down menus and auto-population, wording, default settings, nonintuitive or inflexible ordering, repeat prescriptions and automated processes, users' work processes, and clinical decision support systems. Displaying an incomplete list of a patient's medications on the computer screen often contributed to prescribing errors. Lack of system flexibility resulted in users employing error-prone workarounds, such as the addition of contradictory free-text comments. Users' misinterpretations of how text was presented in CPOE systems were also linked with the occurrence of prescribing errors.

Discussion and Conclusions

Human factors design is important to reduce error rates. Drop-down menus should be designed with safeguards to decrease the likelihood of selection errors. Development of more sophisticated clinical decision support, which can perform checks on free-text, may also prevent errors. Further research is needed to ensure that systems minimize error likelihood and meet users' workflow expectations.

Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial

Introduction

Several methods have been developed to electronically monitor patients for severe sepsis, but few provide predictive capabilities to enable early intervention; furthermore, no severe sepsis prediction systems have been previously validated in a randomised study. We tested the use of a machine learning-based severe sepsis prediction system for reductions in average length of stay and in-hospital mortality rate.

Methods

We conducted a randomised controlled clinical trial at two medical-surgical intensive care units at the University of California, San Francisco Medical Center, evaluating the primary outcome of average length of stay, and secondary outcome of in-hospital mortality rate from December 2016 to February 2017. Adult patients (18+) admitted to participating units were eligible for this factorial, open-label study. Enrolled patients were assigned to a trial arm by a random allocation sequence. In the control group, only the current severe sepsis detector was used; in the experimental group, the machine learning algorithm (MLA) was also used. On receiving an alert, the care team evaluated the patient and initiated the severe sepsis bundle, if appropriate. Although participants were randomly assigned to a trial arm, group assignments were automatically revealed for any patients who received MLA alerts.

Results

Outcomes from 75 patients in the control and 67 patients in the experimental group were analysed. Average length of stay decreased from 13.0 days in the control to 10.3 days in the experimental group (p=0.042). In-hospital mortality decreased by 12.4 percentage points when using the MLA (p=0.018), a relative reduction of 58.0%. No adverse events were reported during this trial.

Conclusion

The MLA was associated with improved patient outcomes. This is the first randomised controlled trial of a sepsis surveillance system to demonstrate statistically significant differences in length of stay and in-hospital mortality.

Trial registration

NCT03015454.

Clinical decisions support malfunctions in a commercial electronic health record

OBJECTIVES

Determine if clinical decision support (CDS) malfunctions occur in a commercial electronic health record (EHR) system, characterize their pathways and describe methods of detection.

METHODS

We retrospectively examined the firing rate for 226 alert type CDS rules for detection of anomalies using both expert visualization and statistical process control (SPC) methods over a five year period. Candidate anomalies were investigated and validated.

RESULTS

Twenty-one candidate CDS anomalies were identified from 8,300 alert-months. Of these candidate anomalies, four were confirmed as CDS malfunctions, eight as false-positives, and nine could not be classified. The four CDS malfunctions were a result of errors in knowledge management: 1) inadvertent addition and removal of a medication code to the electronic formulary list; 2) a seasonal alert which was not activated; 3) a change in the base data structures; and 4) direct editing of an alert related to its medications. 154 CDS rules (68%) were amenable to SPC methods and the test characteristics were calculated as a sensitivity of 95%, positive predictive value of 29% and F-measure 0.44.

DISCUSSION

CDS malfunctions were found to occur in our EHR. All of the pathways for these malfunctions can be described as knowledge management errors. Expert visualization is a robust method of detection, but is resource intensive. SPC-based methods, when applicable, perform reasonably well retrospectively.

CONCLUSION

CDS anomalies were found to occur in a commercial EHR and visual detection along with SPC analysis represents promising methods of malfunction detection.

More Than a Text Message: Dismantling Digital Triggers to Curate Behavior Change in Patient-Centered Health Interventions

Digital triggers such as text messages, emails, and push alerts are designed to focus an individual on a desired goal by prompting an internal or external reaction at the appropriate time. Triggers therefore have an essential role in engaging individuals with digital interventions delivered outside of traditional health care settings, where other events in daily lives and fluctuating motivation to engage in effortful behavior exist. There is an emerging body of literature examining the use of digital triggers for short-term action and longer-term behavior change. However, little attention has been given to understanding the components of digital triggers. Using tailoring as an overarching framework, we separated digital triggers into 5 primary components: (1) who (sender), (2) how (stimulus type, delivery medium, heterogeneity), (3) when (delivered), (4) how much (frequency, intensity), and (5) what (trigger's target, trigger's structure, trigger's narrative). We highlighted key considerations when tailoring each component and the pitfalls of ignoring common mistakes, such as alert fatigue and habituation. As evidenced throughout the paper, there is a broad literature base from which to draw when tailoring triggers to curate behavior change in health interventions. More research is needed, however, to examine differences in efficacy based on component tailoring, to best use triggers to facilitate behavior change over time, and to keep individuals engaged in physical and mental health behavior change efforts. Dismantling digital triggers into their component parts and reassembling them according to the gestalt of one's change goals is the first step in this development work.

Computerized pharmacy surveillance and alert system for drug-related problems

WHAT IS KNOWN AND OBJECTIVE

Because of the impact of drug-related problems (DRPs) on morbidity and mortality, there is a need for computerized strategies to increase drug safety. The detection and identification of the causes of potential DRPs can be facilitated by the incorporation of a pharmacy warning system (PWS) in the computerized prescriber order entry (CPOE) and its application in the routine validation of inpatient drug therapy. A limited number of studies have evaluated a clinical decision support system to monitor drug treatment. Most of these applications have utilized a small range of drugs with alerts and/or types of alert. The objective of this study was to describe the implementation of a PWS integrated in the electronic medical record (EMR).

METHODS

The PWS was developed in 2003-2004. Pharmacological information to generate drug alerts was entered on demographic data, drug dosage, laboratory tests related to the prescribed drug and drug combinations (interactions, duplications and necessary combinations). The PWS was applied in the prescription reviews conducted in patients admitted to the hospital in 2012.

RESULTS AND DISCUSSION

Information on 83% of the drugs included in the pharmacopeia was introduced into the PWS, allowing detection of 2808 potential DRPs, representing 79·1% of all potential DRPs detected during the study period. Twenty per cent of PWS DRPs were clinically relevant, requiring pharmacist intervention.

WHAT IS NEW AND CONCLUSION

The PWS detected most potential DRPs, thus increasing inpatient safety. The detection ability of the PWS was higher than that reported for other tools described in the literature.

Feedback and Training to Improve Use of an Electronic Prescribing System: A Randomised Controlled Trial

Excessive presentation of alerts in electronic prescribing systems (ePS) results in 'alert fatigue' which reduces alert effectiveness and frustrates users. Previous research at our study site showed high rates of duplication alerts, some of which were the result of doctors not using available short-cut functions in the ePS. This study aimed to improve uptake of short-cut functions and so reduce alert fatigue by trialing two interventions: feedback and training. Fifty doctors were randomised to one of three groups: Control, Feedback or Training. The Feedback group received an individualised feedback report via email and the Training group received brief face-to-face refresher training. Participants partook in informal interviews to discuss the training and the ePS in use. The proportion of orders which triggered a duplication alert was our primary outcome measure. Neither intervention had a significant impact on duplication alert rate (Feedback: 80.8% vs. 77.8% of orders, Training: 77.5% vs. 76.5% of orders; all p&gt;0.05). We identified a number of factors related to the intervention, ePS and prescribing environment that contributed to this result. Rather than focusing on changing prescribing behaviour, we suggest a more effective and appropriate approach is to redesign the ePS so that fewer and more meaningful alerts are presented.

The frequency of inappropriate nonformulary medication alert overrides in the inpatient setting

BACKGROUND

Experts suggest that formulary alerts at the time of medication order entry are the most effective form of clinical decision support to automate formulary management.

OBJECTIVE

Our objectives were to quantify the frequency of inappropriate nonformulary medication (NFM) alert overrides in the inpatient setting and provide insight on how the design of formulary alerts could be improved.

METHODS

Alert overrides of the top 11 (n = 206) most-utilized and highest-costing NFMs, from January 1 to December 31, 2012, were randomly selected for appropriateness evaluation. Using an empirically developed appropriateness algorithm, appropriateness of NFM alert overrides was assessed by 2 pharmacists via chart review. Appropriateness agreement of overrides was assessed with a Cohen's kappa. We also assessed which types of NFMs were most likely to be inappropriately overridden, the override reasons that were disproportionately provided in the inappropriate overrides, and the specific reasons the overrides were considered inappropriate.

RESULTS

Approximately 17.2% (n = 35.4/206) of NFM alerts were inappropriately overridden. Non-oral NFM alerts were more likely to be inappropriately overridden compared to orals. Alerts overridden with "blank" reasons were more likely to be inappropriate. The failure to first try a formulary alternative was the most common reason for alerts being overridden inappropriately.

CONCLUSION

Approximately 1 in 5 NFM alert overrides are overridden inappropriately. Future research should evaluate the impact of mandating a valid override reason and adding a list of formulary alternatives to each NFM alert; we speculate these NFM alert features may decrease the frequency of inappropriate overrides.

The vulnerabilities of computerized physician order entry systems: a qualitative study

OBJECTIVE

To test the vulnerabilities of a wide range of computerized physician order entry (CPOE) systems to different types of medication errors, and develop a more comprehensive qualitative understanding of how their design could be improved.

MATERIALS AND METHODS

The authors reviewed a random sample of 63,040 medication error reports from the US Pharmacopeia (USP) MEDMARX reporting system where CPOE systems were considered a "contributing factor" to errors and flagged test scenarios that could be tested in current CPOE systems. Testers entered these orders in 13 commercial and homegrown CPOE systems across 16 different sites in the United States and Canada, using both usual practice and where-needed workarounds. Overarching themes relevant to interface design and usability/workflow issues were identified.

RESULTS

CPOE systems often failed to detect and prevent important medication errors. Generation of electronic alert warnings varied widely between systems, and depended on a number of factors, including how the order information was entered. Alerts were often confusing, with unrelated warnings appearing on the same screen as those more relevant to the current erroneous entry. Dangerous drug-drug interaction warnings were displayed only after the order was placed rather than at the time of ordering. Testers illustrated various workarounds that allowed them to enter these erroneous orders.

DISCUSSION AND CONCLUSION

The authors found high variability in ordering approaches between different CPOE systems, with major deficiencies identified in some systems. It is important that developers reflect on these findings and build in safeguards to ensure safer prescribing for patients.

Using a mobile health application to support self-management in chronic obstructive pulmonary disease: a six-month cohort study

BACKGROUND

Self-management strategies have the potential to support patients with chronic obstructive pulmonary disease (COPD). Telehealth interventions may have a role in delivering this support along with the opportunity to monitor symptoms and physiological variables. This paper reports findings from a six-month, clinical, cohort study of COPD patients' use of a mobile telehealth based (mHealth) application and how individually determined alerts in oxygen saturation levels, pulse rate and symptoms scores related to patient self-initiated treatment for exacerbations.

METHODS

The development of the mHealth intervention involved a patient focus group and multidisciplinary team of researchers, engineers and clinicians. Individual data thresholds to set alerts were determined, and the relationship to exacerbations, defined by the initiation of stand-by medications, was measured. The sample comprised 18 patients (age range of 50-85 years) with varied levels of computer skills.

RESULTS

Patients identified no difficulties in using the mHealth application and used all functions available. 40% of exacerbations had an alert signal during the three days prior to a patient starting medication. Patients were able to use the mHealth application to support self- management, including monitoring of clinical data. Within three months, 95% of symptom reporting sessions were completed in less than 100 s.

CONCLUSIONS

Home based, unassisted, daily use of the mHealth platform is feasible and acceptable to people with COPD for reporting daily symptoms and medicine use, and to measure physiological variables such as pulse rate and oxygen saturation. These findings provide evidence for integrating telehealth interventions with clinical care pathways to support self-management in COPD.