Evaluating the positive predictive values of antidote signals to detect potential adverse drug reactions (ADRs) in the medical intensive care unit (ICU)

Analysis of medication-related triggers to determine adverse drug events

PURPOSE

Voluntary event reporting systems continue to be the most common method used to identify adverse events in most US hospitals; however, this method fails to capture more than 90% of adverse drug events (ADEs). The purpose of this study is to examine which medication-related triggers have the highest positive predictive values (PPV) for detecting ADEs at a large academic medical centre.

METHODS

A 1-year, single-centre, retrospective quality improvement study was conducted to assess the PPV of four medication-related triggers: flumazenil, naloxone, glucose <70 mg/dL or dextrose 50%. Retrospective chart review was conducted on a random sample of eligible patients to establish if an ADE occurred and determine its preventability. Assessed triggers were also compared against the hospital's voluntary event reporting system to determine whether the events were previously reported.

RESULTS

A total of 161 triggers were reviewed. PPV values for detection of ADEs were 0.55, 0.58, 0.76 and 0.68 for flumazenil, naloxone, glucose <70 mg/dL and dextrose 50%, respectively. PPV values for detection of preventable ADEs were 0.09, 0.16, 0.32 and 0.34 for flumazenil, naloxone, glucose <70 mg/dL and dextrose 50%, respectively. Of the 107 ADEs identified, three events were reported through the hospital's voluntary event reporting system (2.8%).

CONCLUSIONS

Trigger tools successfully detected both preventable and non-preventable ADEs. Events detected using trigger tools are unlikely to be reported through voluntary event reporting systems; therefore, trigger tools can serve as a useful adjunct for adverse event detection.

Artificial Intelligence-Enabled Software Prototype to Inform Opioid Pharmacovigilance From Electronic Health Records: Development and Usability Study

Background

The use of patient health and treatment information captured in structured and unstructured formats in computerized electronic health record (EHR) repositories could potentially augment the detection of safety signals for drug products regulated by the US Food and Drug Administration (FDA). Natural language processing and other artificial intelligence (AI) techniques provide novel methodologies that could be leveraged to extract clinically useful information from EHR resources.

Objective

Our aim is to develop a novel AI-enabled software prototype to identify adverse drug event (ADE) safety signals from free-text discharge summaries in EHRs to enhance opioid drug safety and research activities at the FDA.

Methods

We developed a prototype for web-based software that leverages keyword and trigger-phrase searching with rule-based algorithms and deep learning to extract candidate ADEs for specific opioid drugs from discharge summaries in the Medical Information Mart for Intensive Care III (MIMIC III) database. The prototype uses MedSpacy components to identify relevant sections of discharge summaries and a pretrained natural language processing (NLP) model, Spark NLP for Healthcare, for named entity recognition. Fifteen FDA staff members provided feedback on the prototype's features and functionalities.

Results

Using the prototype, we were able to identify known, labeled, opioid-related adverse drug reactions from text in EHRs. The AI-enabled model achieved accuracy, recall, precision, and F1-scores of 0.66, 0.69, 0.64, and 0.67, respectively. FDA participants assessed the prototype as highly desirable in user satisfaction, visualizations, and in the potential to support drug safety signal detection for opioid drugs from EHR data while saving time and manual effort. Actionable design recommendations included (1) enlarging the tabs and visualizations; (2) enabling more flexibility and customizations to fit end users' individual needs; (3) providing additional instructional resources; (4) adding multiple graph export functionality; and (5) adding project summaries.

Conclusions

The novel prototype uses innovative AI-based techniques to automate searching for, extracting, and analyzing clinically useful information captured in unstructured text in EHRs. It increases efficiency in harnessing real-world data for opioid drug safety and increases the usability of the data to support regulatory review while decreasing the manual research burden.

Outcomes and Adverse Effects With Peramivir for the Treatment of Influenza H1N1 in Critically Ill Pediatric Patients

OBJECTIVES

Influenza is an environmental pathogen and infection presents as a range from asymptomatic to fulminant illness. Though treatment is supportive, antiviral agents have a role in the management of infection. Pediatric use of peramivir is largely based on reports and extrapolations of pharmacokinetic data. We seek to describe efficacy and safety of peramivir in critically ill pediatric patients.

METHODS

This is a retrospective, institutional review board-approved chart review of all patients under 21 years of age, admitted to the PICU, and treated with peramivir for influenza H1N1 infection between January 1, 2016, and March 31, 2016, at a single-center, 12-bed PICU. The primary outcome was time to sustained resolution of fever; secondary outcomes included dose, duration, and adverse effects of peramivir therapy.

RESULTS

Seven patients were included with median age of 3.7 years. Median time to sustained resolution of fever was 49.3 hours, median duration of mechanical ventilation was 14.2 days, median ICU LOS was 18.7 days, and hospital LOS was 24.7 days. No patients suffered mortality. Three patients experienced leukopenia, one of which experienced a concurrent neutropenia. Three patients experienced hyperglycemia, 2 experienced hypertension, 1 experienced increased aspartate aminotransferase and increased alanine aminotransferase, and 1 experienced diarrhea. All adverse events assessed were classified as possible using published adverse event causality assessments.

CONCLUSIONS

Peramivir has been shown to be an effective therapy for the treatment of influenza H1N1 in critically ill pediatric patients. In our experience with 7 pediatric patients, peramivir was well tolerated at typical durations of therapy; however, increased vigilance is warranted during prolonged courses or in patients with reasons for altered pharmacokinetics and pharmacodynamics.

Clinical Practice Guideline: Safe Medication Use in the ICU

OBJECTIVE

To provide ICU clinicians with evidence-based guidance on safe medication use practices for the critically ill.

DATA SOURCES

PubMed, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, CINAHL, Scopus, and ISI Web of Science for relevant material to December 2015.

STUDY SELECTION

Based on three key components: 1) environment and patients, 2) the medication use process, and 3) the patient safety surveillance system. The committee collectively developed Population, Intervention, Comparator, Outcome questions and quality of evidence statements pertaining to medication errors and adverse drug events addressing the key components. A total of 34 Population, Intervention, Comparator, Outcome questions, five quality of evidence statements, and one commentary on disclosure was developed.

DATA EXTRACTION

Subcommittee members were assigned selected Population, Intervention, Comparator, Outcome questions or quality of evidence statements. Subcommittee members completed their Grading of Recommendations Assessment, Development, and Evaluation of the question with his/her quality of evidence assessment and proposed strength of recommendation, then the draft was reviewed by the relevant subcommittee. The subcommittee collectively reviewed the evidence profiles for each question they developed. After the draft was discussed and approved by the entire committee, then the document was circulated among all members for voting on the quality of evidence and strength of recommendation.

DATA SYNTHESIS

The committee followed the principles of the Grading of Recommendations Assessment, Development, and Evaluation system to determine quality of evidence and strength of recommendations.

CONCLUSIONS

This guideline evaluates the ICU environment as a risk for medication-related events and the environmental changes that are possible to improve safe medication use. Prevention strategies for medication-related events are reviewed by medication use process node (prescribing, distribution, administration, monitoring). Detailed considerations to an active surveillance system that includes reporting, identification, and evaluation are discussed. Also, highlighted is the need for future research for safe medication practices that is specific to critically ill patients.

Detection of adverse drug events using an electronic trigger tool

PURPOSE

Implementation and refinement of an integrated electronic "trigger tool" for detecting adverse drug events (ADEs) is described.

METHODS

A three-month prospective study was conducted at a large medical center to test and improve the positive predictive value (PPV) of an electronic health record-based tool for detecting ADEs associated with use of four "trigger drugs": the reversal agents flumazenil, naloxone, phytonadione, and protamine. On administration of a trigger drug to an adult patient, an electronic message was transmitted to two pharmacists, who reviewed cases in near real time (typically, on the same day) to detect actual or potential ADEs. In phase 1 of the study, any use of a trigger drug resulted in an alert message; in subsequent phases, the alerting criteria were narrowed on the basis of clinical criteria and laboratory data with the goal of refining the trigger tool's PPV.

RESULTS

A total of 87 drug administrations were reviewed during the three-month study period, with 27 ADEs detected. PPV values in phases 1, 2, and 3 were 0.33, 0.21, and 0.36, respectively. The relatively low overall PPV of the trigger tool was largely attributable to false-positive trigger messages associated with phytonadione use (such messages were reduced from 35 in phase 1 to 7 in phase 3).

CONCLUSION

Evaluation and refinement of an electronic trigger tool based on detecting the use of the reversal agents flumazenil, naloxone, phytonadione, and protamine found an overall PPV of 0.31 during a three-month study period.

Adverse drug reactions in therapeutic hypothermia after cardiac arrest

BACKGROUND

Therapeutic hypothermia (TH) improves survival and neurologic function in comatose survivors of cardiac arrest. Many medications used to support TH have altered pharmacokinetics and pharmacodynamics during this treatment. It is unknown if or at what frequency the medications used during TH cause adverse drug reactions (ADRs).

METHODS

A retrospective chart review was conducted for patients admitted to an intensive care unit (ICU) after cardiac arrest and treated with TH from January 2009 to June 2012 at two urban, university-affiliated, tertiary-care medical centres. Medications commonly used during TH were screened for association with significant ADRs (grade 3 or greater per Common Terminology Criteria for Adverse Events) using three published ADR detection instruments.

RESULTS

A total of 229 patients were included, the majority being males with median age of 62 presenting with an out-of-hospital cardiac arrest in pulseless electrical activity or asystole. The most common comorbidities were hypertension, coronary artery disease, and diabetes mellitus. There were 670 possible ADRs and 69 probable ADRs identified. Of the 670 possible ADRs, propofol, fentanyl, and acetaminophen were the most common drugs associated with ADRs. Whereas fentanyl, insulin, and propofol were the most common drugs associated with a probable ADR. Patients were managed with TH for a median of 22 hours, with 38% of patients surviving to hospital discharge.

CONCLUSIONS

Patients undergoing TH after cardiac arrest frequently experience possible adverse reactions associated with medications and the corresponding laboratory abnormalities are significant. There is a need for judicious use and close monitoring of drugs in the setting of TH until recommendations for dose adjustments are available to help prevent ADRs.

Clinical decision support for drug related events: Moving towards better prevention

Clinical decision support (CDS) systems with automated alerts integrated into electronic medical records demonstrate efficacy for detecting medication errors (ME) and adverse drug events (ADEs). Critically ill patients are at increased risk for ME, ADEs and serious negative outcomes related to these events. Capitalizing on CDS to detect ME and prevent adverse drug related events has the potential to improve patient outcomes. The key to an effective medication safety surveillance system incorporating CDS is advancing the signals for alerts by using trajectory analyses to predict clinical events, instead of waiting for these events to occur. Additionally, incorporating cutting-edge biomarkers into alert knowledge in an effort to identify the need to adjust medication therapy portending harm will advance the current state of CDS. CDS can be taken a step further to identify drug related physiological events, which are less commonly included in surveillance systems. Predictive models for adverse events that combine patient factors with laboratory values and biomarkers are being established and these models can be the foundation for individualized CDS alerts to prevent impending ADEs.

Use of Text Searching for Trigger Words in Medical Records to Identify Adverse Drug Reactions within an Intensive Care Unit Discharge Summary

PURPOSE

To evaluate the performance of using trigger words (e.g. clues to an adverse drug reaction) in unstructured, narrative text to detect adverse drug reactions (ADRs) and compare the use of these trigger words to a targeted chart review for ADR detection within the intensive care unit (ICU) discharge summary note.

MATERIALS

A retrospective medical record review was conducted. Evaluation of ADRs occurred in two phases - targeted chart review of the ICU discharge summary notes in Phase 1 and targeted chart review using specific words and phrases as triggers for ADRs in Phase 2.

RESULTS

Four hundred ADRs were documented in 223 patients for Phase 1. For Phase 2, there were 219 ADRs identified in 120 patients. 138 real or accurate ADRs were identified from Phase 1 and 47 duplicate events. 34 ADRs from Phase 2 were not identified in Phase 1. Fifteen of the ADRs were inaccurately presumed in Phase 2. Fifty-eight of 127 text triggers identified at least one ADR. Low and moderate frequency trigger words were more likely to have PPVs > 5%.

CONCLUSIONS

Targeted chart review using specific words and phrases as triggers for ADRs is a reasonable approach to identify ADRs and may save time compared to other methods after further refinement leads to a more accurately performing trigger word list.

Evaluation of an automated surveillance system using trigger alerts to prevent adverse drug events in the intensive care unit and general ward

INTRODUCTION

Adverse events in the intensive care unit (ICU) may be associated with several possible causes, so determining a drug-related causal assessment is more challenging than in general ward patients. Therefore, the hypothesis was that automated trigger alerts may perform differently in various patient care settings. The purpose of this study was to compare the frequency and type of clinically significant automated trigger alerts in critically ill and general ward patients as well as evaluate the performance of alerts for drug-related hazardous conditions (DRHCs).

METHODS

A retrospective cohort study was conducted in adult ICU and general ward patients at three institutions (academic, community, and rural hospital) in a health system. Automated trigger alerts generated during two nonconsecutive months were obtained from a centralized database. Pharmacist responses to alerts and prescriber response to recommendations were evaluated for all alerts. A clinical significant event was defined as an actionable intervention requiring drug therapy changes that the pharmacist determined to be appropriate for patient safety and where the physician accepted the pharmacist's recommendation. The positive predictive value (PPV) was calculated for each trigger alert considered a DRHC (i.e., abnormal laboratory values and suspected drug causes).

RESULTS

A total of 751 alerts were generated in 623 patients during the study period. Pharmacists intervened on 39.8 and 44.8 % alerts generated in the ICU and general ward, respectively. Overall, the physician acceptance rate of approximately 90 % was comparable irrespective of patient care setting. Therefore, the number of clinically significant alerts was 88.9 and 83.4 % for the ICU and non-ICU, respectively. The types of drug therapy changes were similar between settings. The PPV of alerts identifying a DRHC was 0.66 in the ICU and 0.76 in general ward patients.

CONCLUSIONS

The number and type of clinically significant alerts were similar irrespective of patient population, suggesting that the alerts may be equally as beneficial in the ICU population, despite the challenges in drug-related event adjudication. An opportunity exists to improve the performance of alerts in both settings, so quality improvement programs for measuring alert performance and making refinements is needed.

A multicenter study of the point prevalence of drug-induced hypotension in the ICU

OBJECTIVE

To determine the point prevalence of drug-induced hypotension episodes in critically ill patients, to assess the episodes resulting from error, and to describe how episodes are treated.

DESIGN

Multicenter observational, 24-hour snapshot study.

SETTING

Forty-seven ICUs in 27 institutions located in the United States, Canada, and Singapore.

PATIENTS

A total of 688 ICU patients were evaluated.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Patients were included in the study if they had an episode of hypotension in the 24 hours prior to the clinical pharmacists' evaluation. The definition for a hypotensive episode is either a systolic blood pressure less than 90 mm Hg or a decrease in systolic blood pressure of 30 mm Hg over a 2-hour period. Each episode of unintentional hypotension was assessed for suspected drug-related causes. When a drug-related cause was suspected, an objective assessment tool, the modified Kramer, was used to determine causality. A score of at least "possible" was considered drug induced, referred to as a "drug-related hazardous condition." A drug-related hazardous condition is the temporal gap (intermediate stage) between the identification of an adverse drug reaction and the subsequent onset of drug-induced injury, known as an "adverse drug event." Drug-induced episodes were evaluated for medication errors and treatment. One hundred fifty-eight patients experienced 204 hypotensive episodes that were considered unintentional and drug related. Common drugs implicated included propofol, fentanyl, metoprolol, lorazepam, hydralazine, and furosemide. A total of 54 episodes (26.5%) resulted from medication errors. Common error types were improper dose/quantity (46%) and prescribing (25%). A total of 56.9% episodes were treated.

CONCLUSIONS

Many hypotensive episodes in the ICU are drug related and require treatment. A substantial portion of these episodes result from errors and are therefore preventable. This presents opportunities to improve prescribing including optimizing drug dosing to avoid possible patient harm from drug-induced hypotension.

Improving adverse drug event detection in critically ill patients through screening intensive care unit transfer summaries

PURPOSE

This study aimed to determine the frequency and type of adverse drug events (ADEs) identified in intensive care unit (ICU) transfer summaries and in the hospital discharge summaries to demonstrate the effectiveness of ICU transfer summary surveillance in the identification of ADEs.

METHODS

A retrospective electronic medical record review was conducted for medical ICU patients admitted between January 2009 and April 2009 to a large, academic medical center. The Harvard Practice Scale and the modified Leonard Assessment Scale were used to evaluate the presence of an ADE from the ICU transfer and hospital discharge summaries.

RESULTS

Two hundred and fifty-four patients were identified for inclusion with a median medical ICU length of stay of 4.5 days and hospital length of stay of 13 days. The ICU transfer summary review revealed 173 ADEs among 124 unique patients with a rate of 33.9 ADEs per 1000 hospital patient days. Sixty-nine ADEs among 63 unique patients were identified through the hospital discharge summary with a rate of 13.5 ADEs per 1000 hospital patient days. Only 23.1% of ADEs discussed in the ICU transfer summary were also discussed in the hospital discharge summary.

CONCLUSIONS

The use of ICU transfer summaries is an effective tool to increase ADE detection. The use of an ICU transfer summary should be considered as an adjunct method to an existing ADE surveillance system for heightened pharmacovigilance.

Analysis of risk factors for adverse drug events in critically ill patients*

OBJECTIVES

An evaluation of risk factors for adverse drug events in critically ill patients has not been previously studied. The purpose of this original study was to determine risk factors for adverse drug events in critically ill adult patients.

DESIGN

This retrospective case-control study includes patients who were admitted to the intensive care unit during a 7.5-yr period.

SETTING

Academic medical center with 647 beds that contains approximately 120 intensive care unit beds.

PATIENTS

Patients in the case group experienced an adverse drug event as documented in the hospital's database. The control group comprised the next two patients admitted to the same intensive care unit by the same admitting service.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Twenty-nine suspected risk factors identified from the literature were evaluated, including patient characteristics, drug characteristics, and laboratory values using a multiple logistic regression. A sample of 1101 cases and controls (54% male), with a mean age of 59.4 ± 17.5 yrs, were identified. In 367 cases, there was a total of 499 documented adverse drug events. Patients with kidney injury, thrombocytopenia, and those admitted emergently were 16-times, 3-times, and 2-times more likely to have an adverse drug event, respectively. Patients who were administered intravenous medications had a 3% higher risk of having an adverse drug event for each drug dispensed. Overall, the case group received more drugs per intensive care unit day and more drugs per intensive care unit stay.

CONCLUSIONS

Several patient and drug-related characteristics contribute to the risk of adverse drug events in critically ill patients. Diligent monitoring of factors that can influence the pharmacokinetic properties for existing drug therapies is necessary. Drug regimens should be evaluated daily for minimization. Based on previous studies, pharmacists as part of the interdisciplinary team could help to manage these risks.

Use of an abnormal laboratory value-drug combination alert to detect drug-induced thrombocytopenia in critically Ill patients

PURPOSE

The aim of this study was to assess the performance of a commercially available clinical decision support system (CDSS) drug-laboratory result alert in detecting drug-induced thrombocytopenia in critically ill patients.

MATERIALS AND METHODS

Adult patients admitted to the medical and cardiac intensive care unit during an 8-week period and identified by 1 of 3 signals in the CDSS, TheraDoc, were eligible. Alerts were generated when the patient had a low platelet count and was ordered a potentially causal drug. Patients were evaluated in real time for the occurrence of an adverse drug reaction using 3 causality instruments. Positive predictive values were calculated for the alert.

RESULTS

Sixty-four patients with a mean age of 54 years met the inclusion criteria, generating 350 alerts. Positive predictive values were 0.36, 0.83, and 0.40 for signals 1, 2, and 3, respectively. Overall, there were 137 adverse drug reactions identified in the 350 alerts, with heparin, vancomycin, and famotidine as the 3 most common potential causes.

CONCLUSIONS

A commercial CDSS drug-laboratory alert is effective at identifying drug-induced thrombocytopenia in the intensive care unit and may improve patient safety. Compared with previous studies, the combination alert performs better than alerts based exclusively on laboratory values and should be considered to reduce alert fatigue.

Transitioning knowledge gained from simulation to pharmacy practice

Using simulation to teach pharmacy practice skills may result in knowledge that is transferable to patient care. Key areas in which simulation is being used in pharmacy education include therapeutics, communication, physical assessment, patient safety, and populations to which students may have infrequent exposure. Enhancing interprofessional healthcare team dynamics and the skills of practicing healthcare professionals are other practical applications for simulation education. Educators should continue to be creative in the incorporation of simulation into pharmacy education and conduct more studies on the impact of simulation education on patient care to demonstrate the efficacy of this teaching modality.

Computerized detection of adverse drug reactions in the medical intensive care unit

OBJECTIVE

Clinical event monitors are a type of active medication monitoring system that can use signals to alert clinicians to possible adverse drug reactions. The primary goal was to evaluate the positive predictive values of select signals used to automate the detection of ADRs in the medical intensive care unit.

METHOD

This is a prospective, case series of adult patients in the medical intensive care unit during a six-week period who had one of five signals presents: an elevated blood urea nitrogen, vancomycin, or quinidine concentration, or a low sodium or glucose concentration. Alerts were assessed using 3 objective published adverse drug reaction determination instruments. An event was considered an adverse drug reaction when 2 out of 3 instruments had agreement of possible, probable or definite. Positive predictive values were calculated as the proportion of alerts that occurred, divided by the number of times that alerts occurred and adverse drug reactions were confirmed.

RESULTS

145 patients were eligible for evaluation. For the 48 patients (50% male) having an alert, the mean±SD age was 62±19 years. A total of 253 alerts were generated. Positive predictive values were 1.0, 0.55, 0.38 and 0.33 for vancomycin, glucose, sodium, and blood urea nitrogen, respectively. A quinidine alert was not generated during the evaluation.

CONCLUSIONS

Computerized clinical event monitoring systems should be considered when developing methods to detect adverse drug reactions as part of intensive care unit patient safety surveillance systems, since they can automate the detection of these events using signals that have good performance characteristics by processing commonly available laboratory and medication information.

Developing a patient safety surveillance system to identify adverse events in the intensive care unit

Aggregation of adverse drug event data has evolved in the last decade. Several approaches are available to augment the standard voluntary incident reporting system. Most of these methods are applicable to nonmedication adverse events as well. To identify appropriately system trends as well as process failures, intensive care units should participate in various collection methods. Several different methods are available for robust adverse drug event data collection, such as target chart review, nontargeted chart review, and direct observation. As the various methods usually capture different types of events, employing more than one technique will improve the assessment of intensive care unit care. Some of these surveillance methods offer real-time or near real-time identification of adverse drug events and potentially afford the practitioner time for intervention. Continued development of adverse drug event detection will allow for further quality improvement efforts and preventive strategies to be utilized.