Application and optimisation of the Comparison on Extreme Laboratory Tests (CERT) algorithm for detection of adverse drug reactions: Transferability across national boundaries

Use of Electronic Health Record Data for Drug Safety Signal Identification: A Scoping Review

INTRODUCTION

Pharmacovigilance programs protect patient health and safety by identifying adverse event signals through postmarketing surveillance of claims data and spontaneous reports. Electronic health records (EHRs) provide new opportunities to address limitations of traditional approaches and promote discovery-oriented pharmacovigilance.

METHODS

To evaluate the current state of EHR-based medication safety signal identification, we conducted a scoping literature review of studies aimed at identifying safety signals from routinely collected patient-level EHR data. We extracted information on study design, EHR data elements utilized, analytic methods employed, drugs and outcomes evaluated, and key statistical and data analysis choices.

RESULTS

We identified 81 eligible studies. Disproportionality methods were the predominant analytic approach, followed by data mining and regression. Variability in study design makes direct comparisons difficult. Studies varied widely in terms of data, confounding adjustment, and statistical considerations.

CONCLUSION

Despite broad interest in utilizing EHRs for safety signal identification, current efforts fail to leverage the full breadth and depth of available data or to rigorously control for confounding. The development of best practices and application of common data models would promote the expansion of EHR-based pharmacovigilance.

Methods for drug safety signal detection using routinely collected observational electronic health care data: A systematic review

PURPOSE

Signal detection is a crucial step in the discovery of post-marketing adverse drug reactions. There is a growing interest in using routinely collected data to complement established spontaneous report analyses. This work aims to systematically review the methods for drug safety signal detection using routinely collected healthcare data and their performance, both in general and for specific types of drugs and outcomes.

METHODS

We conducted a systematic review following the PRISMA guidelines, and registered a protocol in PROSPERO. MEDLINE, EMBASE, PubMed, Web of Science, Scopus, and the Cochrane Library were searched until July 13, 2021.

RESULTS

The review included 101 articles, among which there were 39 methodological works, 25 performance assessment papers, and 24 observational studies. Methods included adaptations from those used with spontaneous reports, traditional epidemiological designs, methods specific to signal detection with real-world data. More recently, implementations of machine learning have been studied in the literature. Twenty-five studies evaluated method performances, 16 of them using the area under the curve (AUC) for a range of positive and negative controls as their main measure. Despite the likelihood that performance measurement could vary by drug-event pair, only 10 studies reported performance stratified by drugs and outcomes, in a heterogeneous manner. The replicability of the performance assessment results was limited due to lack of transparency in reporting and the lack of a gold standard reference set.

CONCLUSIONS

A variety of methods have been described in the literature for signal detection with routinely collected data. No method showed superior performance in all papers and across all drugs and outcomes, performance assessment and reporting were heterogeneous. However, there is limited evidence that self-controlled designs, high dimensional propensity scores, and machine learning can achieve higher performances than other methods.

Signal Detection of Potentially Drug-Induced Liver Injury in Children Using Electronic Health Records

Background: This study proposes a quantitative 2-stage procedure to detect potential drug-induced liver injury (DILI) signals in pediatric inpatients using an data warehouse of electronic health records (EHRs). Methods: Eight years of medical data from a constructed database were used. A two-stage procedure was adopted: (i) stage 1: the drugs suspected of inducing DILI were selected and (ii) stage 2: the associations between the drugs and DILI were identified in a retrospective cohort study. Results: 1,196 drugs were filtered initially and 12 drugs were further potentially identified as suspect drugs inducing DILI. Eleven drugs (fluconazole, omeprazole, sulfamethoxazole, vancomycin, granulocyte colony-stimulating factor (G-CSF), acetaminophen, nifedipine, fusidine, oseltamivir, nystatin and meropenem) were showed to be associated with DILI. Of these, two drugs, nystatin [odds ratio[OR]=1.39, 95%CI:1.10-1.75] and G-CSF (OR = 1.91, 95%CI:1.55-2.35), were found to be new potential signals in adults and children. Three drugs [nifedipine [OR = 1.77, 95%CI:1.26-2.46], fusidine [OR = 1.43, 95%CI:1.08-1.86], and oseltamivi r [OR = 1.64, 95%CI:1.23-2.18]] were demonstrated to be new signals in pediatrics. The other drug-DILI associations had been confirmed in previous studies. Conclusions: A quantitative algorithm to detect potential signals of DILI has been described. Our work promotes the application of EHR data in pharmacovigilance and provides candidate drugs for further causality assessment studies.

Machine learning model combining features from algorithms with different analytical methodologies to detect laboratory-event-related adverse drug reaction signals

Background

The importance of identifying and evaluating adverse drug reactions (ADRs) has been widely recognized. Many studies have developed algorithms for ADR signal detection using electronic health record (EHR) data. In this study, we propose a machine learning (ML) model that enables accurate ADR signal detection by integrating features from existing algorithms based on inpatient EHR laboratory results.

Materials and methods

To construct an ADR reference dataset, we extracted known drug–laboratory event pairs represented by a laboratory test from the EU-SPC and SIDER databases. All possible drug–laboratory event pairs, except known ones, are considered unknown. To detect a known drug–laboratory event pair, three existing algorithms—CERT, CLEAR, and PACE—were applied to 21-year inpatient EHR data. We also constructed ML models (based on random forest, L1 regularized logistic regression, support vector machine, and a neural network) that use the intermediate products of the CERT, CLEAR, and PACE algorithms as inputs and determine whether a drug–laboratory event pair is associated. For performance comparison, we evaluated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), F1-measure, and area under receiver operating characteristic (AUROC).

Results

All measures of ML models outperformed those of existing algorithms with sensitivity of 0.593–0.793, specificity of 0.619–0.796, NPV of 0.645–0.727, PPV of 0.680–0.777, F1-measure of 0.629–0.709, and AUROC of 0.737–0.816. Features related to change or distribution of shape were considered important for detecting ADR signals.

Conclusions

Improved performance of ML models indicated that applying our model to EHR data is feasible and promising for detecting more accurate and comprehensive ADR signals.