Clinical Relevance of Automated Drug Alerts From the Perspective of Medical Providers

The authors used a real-time survey instrument and subsequent focus group among primary care clinicians at a large healthcare system to assess usefulness of automated drug alerts. Of 108 alerts encountered, 0.9% (n = 1) represented critical alerts, and 16% (n = 17) were significant drug interaction alerts. Sixty-one percent (n = 66) involved duplication of a medication or medication class. The rest (n = 24) involved topical medications, inhalers, or vaccines. Of the 84 potentially relevant alerts, providers classified 11% (9/84), or about 1 in 9, as useful. Drug interaction alerts were more often deemed useful than drug duplication alerts (44.4% versus 1.5%, P < .001). Focus group participants generally echoed these results when ranking the relevance of 15 selected alerts, although there was wide variance in ratings for individual alerts. Hence, a "smarter" system that utilizes a set of mandatory alerts while allowing providers to tailor use of other automated warnings may improve clinical relevance of drug alert systems.

A Novel Design for Drug-Drug Interaction Alerts Improves Prescribing Efficiency

BACKGROUND

Drug-drug interactions (DDIs) are common in clinical care and pose serious risks for patients. Electronic health records display DDI alerts that can influence prescribers, but the interface design of DDI alerts has largely been unstudied. In this study, the objective was to apply human factors engineering principles to alert design. It was hypothesized that redesigned DDI alerts would significantly improve prescribers' efficiency and reduce prescribing errors.

METHODS

In a counterbalanced, crossover study with prescribers, two DDI alert designs were evaluated. Department of Veterans Affairs (VA) prescribers were video recorded as they completed fictitious patient scenarios, which included DDI alerts of varying severity. Efficiency was measured from time-stamped recordings. Prescribing errors were evaluated against predefined criteria. Efficiency and prescribing errors were analyzed with the Wilcoxon signed-rank test. Other usability data were collected on the adequacy of alert content, prescribers' use of the DDI monograph, and alert navigation.

RESULTS

Twenty prescribers completed patient scenarios for both designs. Prescribers resolved redesigned alerts in about half the time (redesign: 52 seconds versus original design: 97 seconds; p<.001). Prescribing errors were not significantly different between the two designs. Usability results indicate that DDI alerts might be enhanced by facilitating easier access to laboratory data and dosing information and by allowing prescribers to cancel either interacting medication directly from the alert. Results also suggest that neither design provided adequate information for decision making via the primary interface.

CONCLUSION

Applying human factors principles to DDI alerts improved overall efficiency. Aspects of DDI alert design that could be further enhanced prior to implementation were also identified.

Reducing prescribing errors through creatinine clearance alert redesign

BACKGROUND

Literature has shown that computerized creatinine clearance alerts reduce errors during prescribing, and applying human factors principles may further reduce errors. Our objective was to apply human factors principles to creatinine clearance alert design and assess whether the redesigned alerts increase usability and reduce prescribing errors compared with the original alerts.

METHODS

Twenty Veterans Affairs (VA) outpatient providers (14 physicians, 2 nurse practitioners, and 4 clinical pharmacists) completed 2 usability sessions in a counterbalanced study to evaluate original and redesigned alerts. Each session consisted of fictional patient scenarios with 3 medications that warranted prescribing changes because of renal impairment, each associated with creatinine clearance alerts. Quantitative and qualitative data were collected to assess alert usability and the occurrence of prescribing errors.

RESULTS

There were 43% fewer prescribing errors with the redesigned alerts compared with the original alerts (P = .001). Compared with the original alerts, redesigned alerts significantly reduced prescribing errors for allopurinol and ibuprofen (85% vs 40% and 65% vs 25%, P = .012 and P = .008, respectively), but not for spironolactone (85% vs 65%). Nine providers (45%) voiced confusion about why the alert was appearing when they encountered the original alert design. When laboratory links were presented on the redesigned alert, laboratory information was accessed 3.5 times more frequently.

CONCLUSIONS

Although prescribing errors were high with both alert designs, the redesigned alerts significantly improved prescribing outcomes. This investigation provides some of the first evidence on how alerts may be designed to support safer prescribing for patients with renal impairment.

Design and Evaluation of an Electronic Override Mechanism for Medication Alerts to Facilitate Communication Between Prescribers and Pharmacists

Background: Computerized medication alerts can often be bypassed by entering an override rationale, but prescribers’ override reasons are frequently ambiguous to pharmacists who review orders. Objective: To develop and evaluate a new override mechanism for adverse reaction and drug-drug interaction alerts. We hypothesized that the new mechanism would improve usability for prescribers and increase the clinical appropriateness of override reasons. Methods: A counterbalanced, crossover study was conducted with 20 prescribers in a simulated prescribing environment. We modified the override mechanism timing, navigation, and text entry. Instead of free-text entry, the new mechanism presented prescribers with a predefined set of override reasons. We assessed usability (learnability, perceived efficiency, and usability errors) and used a priori criteria to evaluate the clinical appropriateness of override reasons entered. Results: Prescribers rated the new mechanism as more efficient (Wilcoxon signed-rank test, P = 0.032). When first using the new design, 5 prescribers had difficulty finding the new mechanism, and 3 interpreted the navigation to mean that the alert could not be overridden. The number of appropriate override reasons significantly increased with the new mechanism compared with the original mechanism (median change of 3.0; interquartile range = 3.0; P < 0.0001). Conclusions: When prescribers were given a menu-based choice for override reasons, clinical appropriateness of these reasons significantly improved. Further enhancements are necessary, but this study is an important first step toward a more standardized menu of override choices. Findings may be used to improve communication through e-prescribing systems between prescribers and pharmacists.

Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-based simulation

OBJECTIVE

To apply human factors engineering principles to improve alert interface design. We hypothesized that incorporating human factors principles into alerts would improve usability, reduce workload for prescribers, and reduce prescribing errors.

MATERIALS AND METHODS

We performed a scenario-based simulation study using a counterbalanced, crossover design with 20 Veterans Affairs prescribers to compare original versus redesigned alerts. We redesigned drug-allergy, drug-drug interaction, and drug-disease alerts based upon human factors principles. We assessed usability (learnability of redesign, efficiency, satisfaction, and usability errors), perceived workload, and prescribing errors.

RESULTS

Although prescribers received no training on the design changes, prescribers were able to resolve redesigned alerts more efficiently (median (IQR): 56 (47) s) compared to the original alerts (85 (71) s; p=0.015). In addition, prescribers rated redesigned alerts significantly higher than original alerts across several dimensions of satisfaction. Redesigned alerts led to a modest but significant reduction in workload (p=0.042) and significantly reduced the number of prescribing errors per prescriber (median (range): 2 (1-5) compared to original alerts: 4 (1-7); p=0.024).

DISCUSSION

Aspects of the redesigned alerts that likely contributed to better prescribing include design modifications that reduced usability-related errors, providing clinical data closer to the point of decision, and displaying alert text in a tabular format. Displaying alert text in a tabular format may help prescribers extract information quickly and thereby increase responsiveness to alerts.

CONCLUSIONS

This simulation study provides evidence that applying human factors design principles to medication alerts can improve usability and prescribing outcomes.

Potential safety gaps in order entry and automated drug alerts: a nationwide survey of VA physician self-reported practices with computerized order entry

OBJECTIVE

Understanding provider perceptions of and experiences with order entry and order checks (drug alerts) in an electronic prescribing system may help improve medication safety technology.

DESIGN

Cross-sectional, national survey of Veterans Administration physicians practicing in various specialties.

MEASUREMENT

Thirty-five question instrument was divided into 4 content domains. Response options included dichotomous, numeric, multiple choices, and Likert-like scales. Statistical methods included logistic regression.

RESULTS

The adjusted response rate was 1543 of 3588 (43%). Almost all providers (90%) felt that the VA electronic prescribing system, including its order checks, improved prescribing safety to some degree. Most respondents (72%) reported that they always or almost always document outside medications in a clinic note, although only 44% always or almost always entered outside medications in the non-VA medication data field. Most physicians (88%) who encountered serious allergic or adverse drug reactions reported either notifying a pharmacist or entering the information in the allergies/adverse reactions field. Generalists and physicians with higher numbers of prescriptions were more likely to enter relevant data into the electronic medical record (or notify a pharmacist, in the case of adverse reactions). In addition, 48% of providers described critical drug-drug interaction alerts as very useful; medical specialists found these less useful, whereas surgical specialists found these more useful when compared with generalists.

LIMITATIONS

Survey was conducted within a single healthcare system.

CONCLUSION

Computerized provider order entry and related order checks are perceived to improve prescribing safety; however, provider entry of some relevant information into the appropriate electronic fields may not be optimal.