Supervised signal detection for adverse drug reactions in medication dispensing data

Predicting Drugs Suspected of Causing Adverse Drug Reactions Using Graph Features and Attention Mechanisms

BACKGROUND

Adverse drug reactions (ADRs) refer to an unintended harmful reaction that occurs after the administration of a medication for therapeutic purposes, which is unrelated to the intended pharmacological action of the drug. In the United States, ADRs account for 6% of all hospital admissions annually. The cost of ADR-related illnesses in 2016 was estimated at USD 528.4 billion. Increasing the awareness of ADRs is an effective measure to prevent them. Assessing suspected drugs in adverse events helps to enhance the awareness of ADRs.

METHODS

In this study, a suspect drug assisted judgment model (SDAJM) is designed to identify suspected drugs in adverse events. This framework utilizes the graph isomorphism network (GIN) and an attention mechanism to extract features based on patients' demographic information, drug information, and ADR information.

RESULTS

By comparing it with other models, the results of various tests show that this model performs well in predicting the suspected drugs in adverse reaction events. ADR signal detection was conducted on a group of cardiovascular system drugs, and case analyses were performed on two classic drugs, Mexiletine and Captopril, as well as on two classic antithyroid drugs. The results indicate that the model can accomplish the task of predicting drug ADRs. Validation using benchmark datasets from ten drug discovery domains shows that the model is applicable to classification tasks on the Tox21 and SIDER datasets.

CONCLUSIONS

This study applies deep learning methods to construct the SDAJM model for three purposes: (1) identifying drugs suspected to cause adverse drug events (ADEs), (2) predicting the ADRs of drugs, and (3) other drug discovery tasks. The results indicate that this method can offer new directions for research in the field of ADRs.

A real-world pharmacovigilance study on cardiovascular adverse events of tisagenlecleucel using machine learning approach

Chimeric antigen receptor T-cell (CAR-T) therapies are a paradigm-shifting therapeutic in patients with hematological malignancies. However, some concerns remain that they may cause serious cardiovascular adverse events (AEs), for which data are scarce. In this study, gradient boosting machine algorithm-based model was fitted to identify safety signals of serious cardiovascular AEs reported for tisagenlecleucel in the World Health Organization Vigibase up until February 2024. Input dataset, comprised of positive and negative controls of tisagenlecleucel based on its labeling information and literature search, was used to train the model. Then, we implemented the model to calculate the predicted probability of serious cardiovascular AEs defined by preferred terms included in the important medical event list from European Medicine Agency. There were 467 distinct AEs from 3,280 safety cases reports for tisagenlecleucel, of which 363 (77.7%) were classified as positive controls, 66 (14.2%) as negative controls, and 37 (7.9%) as unknown AEs. The prediction model had area under the receiver operating characteristic curve of 0.76 in the test dataset application. Of the unknown AEs, six cardiovascular AEs were predicted as the safety signals: bradycardia (predicted probability 0.99), pleural effusion (0.98), pulseless electrical activity (0.89), cardiotoxicity (0.83), cardio-respiratory arrest (0.69), and acute myocardial infarction (0.58). Our findings underscore vigilant monitoring of acute cardiotoxicities with tisagenlecleucel therapy.

Early Detection of Adverse Drug Reaction Signals by Association Rule Mining Using Large-Scale Administrative Claims Data

INTRODUCTION

Adverse drug reactions (ADRs) are a leading cause of mortality worldwide and should be detected promptly to reduce health risks to patients. A data-mining approach using large-scale medical records might be a useful method for the early detection of ADRs. Many studies have analyzed medical records to detect ADRs; however, most of them have focused on a narrow range of ADRs, limiting their usefulness.

OBJECTIVE

This study aimed to identify methods for the early detection of a wide range of ADR signals.

METHODS

First, to evaluate the performance in signal detection of ADRs by data-mining, we attempted to create a gold standard based on clinical evidence. Second, association rule mining (ARM) was applied to patient symptoms and medications registered in claims data, followed by evaluating ADR signal detection performance.

RESULTS

We created a new gold standard consisting of 92 positive and 88 negative controls. In the assessment of ARM using claims data, the areas under the receiver-operating characteristic curve and the precision-recall curve were 0.80 and 0.83, respectively. If the detection criteria were defined as lift > 1, conviction > 1, and p-value < 0.05, ARM could identify 156 signals, of which 90 were true positive controls (sensitivity: 0.98, specificity: 0.25). Evaluation of the capability of ARM with short periods of data revealed that ARM could detect a greater number of positive controls than the conventional analysis method.

CONCLUSIONS

ARM of claims data may be effective in the early detection of a wide range of ADR signals.

Methods for drug safety signal detection using routinely collected observational electronic health care data: A systematic review

PURPOSE

Signal detection is a crucial step in the discovery of post-marketing adverse drug reactions. There is a growing interest in using routinely collected data to complement established spontaneous report analyses. This work aims to systematically review the methods for drug safety signal detection using routinely collected healthcare data and their performance, both in general and for specific types of drugs and outcomes.

METHODS

We conducted a systematic review following the PRISMA guidelines, and registered a protocol in PROSPERO. MEDLINE, EMBASE, PubMed, Web of Science, Scopus, and the Cochrane Library were searched until July 13, 2021.

RESULTS

The review included 101 articles, among which there were 39 methodological works, 25 performance assessment papers, and 24 observational studies. Methods included adaptations from those used with spontaneous reports, traditional epidemiological designs, methods specific to signal detection with real-world data. More recently, implementations of machine learning have been studied in the literature. Twenty-five studies evaluated method performances, 16 of them using the area under the curve (AUC) for a range of positive and negative controls as their main measure. Despite the likelihood that performance measurement could vary by drug-event pair, only 10 studies reported performance stratified by drugs and outcomes, in a heterogeneous manner. The replicability of the performance assessment results was limited due to lack of transparency in reporting and the lack of a gold standard reference set.

CONCLUSIONS

A variety of methods have been described in the literature for signal detection with routinely collected data. No method showed superior performance in all papers and across all drugs and outcomes, performance assessment and reporting were heterogeneous. However, there is limited evidence that self-controlled designs, high dimensional propensity scores, and machine learning can achieve higher performances than other methods.

Detecting early safety signals of infliximab using machine learning algorithms in the Korea adverse event reporting system

There has been a growing attention on using machine learning (ML) in pharmacovigilance. This study aimed to investigate the utility of supervised ML algorithms on timely detection of safety signals in the Korea Adverse Event Reporting System (KAERS), using infliximab as a case drug, between 2009 and 2018. Input data set for ML training was constructed based on the drug label information and spontaneous reports in the KAERS. Gold standard dataset containing known AEs was randomly divided into the training and test sets. Two supervised ML algorithms (gradient boosting machine [GBM], random forest [RF]) were fitted with hyperparameters tuned on the training set by using a fivefold validation. Then, we stratified the KAERS data by calendar year to create 10 cumulative yearly datasets, in which ML algorithms were applied to detect five pre-specified AEs of infliximab identified during post-marketing surveillance. Four AEs were detected by both GBM and RF in the first year they appeared in the KAERS and earlier than they were updated in the drug label of infliximab. We further applied our models to data retrieved from the US Food and Drug Administration Adverse Event Reporting System repository and found that they outperformed existing disproportionality methods. Both GBM and RF demonstrated reliable performance in detecting early safety signals and showed promise for applying such approaches to pharmacovigilance.

Analyzing adverse drug reaction using statistical and machine learning methods: A systematic review

BACKGROUND

Adverse drug reactions (ADRs) are unintended negative drug-induced responses. Determining the association between drugs and ADRs is crucial, and several methods have been proposed to demonstrate this association. This systematic review aimed to examine the analytical tools by considering original articles that utilized statistical and machine learning methods for detecting ADRs.

METHODS

A systematic literature review was conducted based on articles published between 2015 and 2020. The keywords used were statistical, machine learning, and deep learning methods for detecting ADR signals. The study was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement (PRISMA) guidelines.

RESULTS

We reviewed 72 articles, of which 51 and 21 addressed statistical and machine learning methods, respectively. Electronic medical record (EMR) data were exclusively analyzed using the regression method. For FDA Adverse Event Reporting System (FAERS) data, components of the disproportionality method were preferable. DrugBank was the most used database for machine learning. Other methods accounted for the highest and supervised methods accounted for the second highest.

CONCLUSIONS

Using the 72 main articles, this review provides guidelines on which databases are frequently utilized and which analysis methods can be connected. For statistical analysis, >90% of the cases were analyzed by disproportionate or regression analysis with each spontaneous reporting system (SRS) data or electronic medical record (EMR) data; for machine learning research, however, there was a strong tendency to analyze various data combinations. Only half of the DrugBank database was occupied, and the k-nearest neighbor method accounted for the greatest proportion.

The potential for leveraging machine learning to filter medication alerts

OBJECTIVE

To evaluate the potential for machine learning to predict medication alerts that might be ignored by a user, and intelligently filter out those alerts from the user's view.

MATERIALS AND METHODS

We identified features (eg, patient and provider characteristics) proposed to modulate user responses to medication alerts through the literature; these features were then refined through expert review. Models were developed using rule-based and machine learning techniques (logistic regression, random forest, support vector machine, neural network, and LightGBM). We collected log data on alerts shown to users throughout 2019 at University of Utah Health. We sought to maximize precision while maintaining a false-negative rate <0.01, a threshold predefined through discussion with physicians and pharmacists. We developed models while maintaining a sensitivity of 0.99. Two null hypotheses were developed: H1-there is no difference in precision among prediction models; and H2-the removal of any feature category does not change precision.

RESULTS

A total of 3,481,634 medication alerts with 751 features were evaluated. With sensitivity fixed at 0.99, LightGBM achieved the highest precision of 0.192 and less than 0.01 for the pre-defined maximal false-negative rate by subject-matter experts (H1) (P < 0.001). This model could reduce alert volume by 54.1%. We removed different combinations of features (H2) and found that not all features significantly contributed to precision. Removing medication order features (eg, dosage) most significantly decreased precision (-0.147, P = 0.001).

CONCLUSIONS

Machine learning potentially enables the intelligent filtering of medication alerts.

Key use cases for artificial intelligence to reduce the frequency of adverse drug events: a scoping review

Adverse drug events (ADEs) represent one of the most prevalent types of health-care-related harm, and there is substantial room for improvement in the way that they are currently predicted and detected. We conducted a scoping review to identify key use cases in which artificial intelligence (AI) could be leveraged to reduce the frequency of ADEs. We focused on modern machine learning techniques and natural language processing. 78 articles were included in the scoping review. Studies were heterogeneous and applied various AI techniques covering a wide range of medications and ADEs. We identified several key use cases in which AI could contribute to reducing the frequency and consequences of ADEs, through prediction to prevent ADEs and early detection to mitigate the effects. Most studies (73 [94%] of 78) assessed technical algorithm performance, and few studies evaluated the use of AI in clinical settings. Most articles (58 [74%] of 78) were published within the past 5 years, highlighting an emerging area of study. Availability of new types of data, such as genetic information, and access to unstructured clinical notes might further advance the field.

Proton pump inhibitors may contribute to progression or development of chronic obstructive pulmonary disease-A sequence symmetry analysis approach

WHAT IS KNOWN AND OBJECTIVE

Proton pump inhibitors (PPIs), used to treat and prevent gastro-oesophageal conditions, are well-tolerated but have been associated with risk including pneumonia. The extent to which initiation of PPIs can contribute to other respiratory conditions such as chronic obstructive pulmonary disease (COPD) is largely unknown.

METHODS

A sequence symmetry analysis (SSA) approach was applied to the Australian Department of Human Services, Pharmaceutical Benefits Scheme 10% extract. Participants were aged 45 years and older and were dispensed PPIs (ATC Codes A02BC01, A02BC02, A02BC03, A02BC04 and A02BC05) and long-acting bronchodilators (LABDs) for COPD (ATC Codes R03BB04 (PBS Item Code 10509D and 08626B), R03BB05, R03BB06, R03BB07 and R03AC18 (PBS Item Code 05137J and 05134F)) between 2013 and 2019. The analysis included patients initiated on an LABD within 12 months before or after their first prescription of a PPI. The crude sequence ratio (cSR) was calculated as the number of patients prescribed their first LABD after starting a PPI divided by the number of patients prescribed their first LABD before starting a PPI. Calculation of the adjusted sequence ratio (aSR) accounted for prescribing trends over time in initiation of each of the medicines. A signal was identified where the aSR lower 95% confidence interval (CI) was greater than one.

RESULTS AND DISCUSSION

Initiation of omeprazole was associated with a 29% increased risk of initiating a LABD (ASR = 1.29 95% CI 1.22-1.36). Initiation of esomeprazole, rabeprazole, pantoprazole or lansoprazole was associated with 25%, 15%, 8% and 8% increased risk, respectively.

WHAT IS NEW AND CONCLUSION

There is an established association between gastro-oesophageal reflux disease and COPD which has been confirmed by implementation of a sequence symmetry-based approach which demonstrated that PPI initiation is potentially associated with progression or exacerbation of COPD. The impact PPI use has directly on this association requires further investigation.

Evaluating the use of prescription sequence symmetry analysis as a pharmacovigilance tool: A scoping review

BACKGROUND

The (prescription) sequence symmetry analysis (PSSA) design has been used to identify potential prescribing cascade signals by assessing the prescribing sequence of an index drug relative to a marker drug presumed to treat an adverse drug event provoked by the index drug.

OBJECTIVES

This review aimed to explore the use of the PSSA design as a pharmacovigilance tool with a particular focus on the breadth of identified signals and advances in PSSA methodology.

METHODS

We searched Embase, PubMed/Medline, Google Scholar, Web of Science and grey literature to identify studies that used the PSSA methodology. Two reviewers independently extracted relevant data for each included article. Study characteristics including signals identified, exposure time window, stratified analyses, and use of controls were extracted.

RESULTS

We identified 53 studies which reported original results obtained using PSSA methodology or quantified the validity of components of the PSSA design. Of those, nine studies provided validation metrics showing reasonable sensitivity and high specificity of PSSA to identify prescribing cascade signals. We identified 340 unique index drug - marker drug signals published in the PSSA literature, representing 281 unique index - marker pharmacological class dyads (i.e., unique fourth-level Anatomical Therapeutic Chemical [ATC] classification dyads). Commonly observed signals were identified for index drugs acting upon the nervous system (34%), cardiovascular system (21%), and blood and blood-forming organs (15%), and many marker drugs were related to the nervous system (25%), alimentary tract and metabolism (23%), cardiovascular system (17%), and genitourinary system and sex hormones (14%). Negative controls and positive controls were utilized in 21% and 13% of studies, respectively.

CONCLUSIONS

The PSSA methodology has been used in 53 studies worldwide to detect and evaluate over 300 unique prescribing cascades signals. Researchers should consider sensitivity analyses incorporating negative and/or positive controls and additional time windows to evaluate time-varying biases when designing PSSA studies.

Artificial Intelligence in Pharmacoepidemiology: A Systematic Review. Part 2-Comparison of the Performance of Artificial Intelligence and Traditional Pharmacoepidemiological Techniques

Aim: To summarize the evidence on the performance of artificial intelligence vs. traditional pharmacoepidemiological techniques.

Methods: Ovid MEDLINE (01/1950 to 05/2019) was searched to identify observational studies, meta-analyses, and clinical trials using artificial intelligence techniques having a drug as the exposure or the outcome of the study. Only studies with an available full text in the English language were evaluated.

Results: In all, 72 original articles and five reviews were identified via Ovid MEDLINE of which 19 (26.4%) compared the performance of artificial intelligence techniques with traditional pharmacoepidemiological methods. In total, 44 comparisons have been performed in articles that aimed at 1) predicting the needed dosage given the patient’s characteristics (31.8%), 2) predicting the clinical response following a pharmacological treatment (29.5%), 3) predicting the occurrence/severity of adverse drug reactions (20.5%), 4) predicting the propensity score (9.1%), 5) identifying subpopulation more at risk of drug inefficacy (4.5%), 6) predicting drug consumption (2.3%), and 7) predicting drug-induced lengths of stay in hospital (2.3%). In 22 out of 44 (50.0%) comparisons, artificial intelligence performed better than traditional pharmacoepidemiological techniques. Random forest (seven out of 11 comparisons; 63.6%) and artificial neural network (six out of 10 comparisons; 60.0%) were the techniques that in most of the comparisons outperformed traditional pharmacoepidemiological methods.

Conclusion: Only a small fraction of articles compared the performance of artificial intelligence techniques with traditional pharmacoepidemiological methods and not all artificial intelligence techniques have been compared in a Pharmacoepidemiological setting. However, in 50% of comparisons, artificial intelligence performed better than pharmacoepidemiological techniques.

Machine Learning for Detection of Safety Signals From Spontaneous Reporting System Data: Example of Nivolumab and Docetaxel

Introduction: Various methods have been implemented to detect adverse drug reaction (ADR) signals. However, the applicability of machine learning methods has not yet been fully evaluated.

Objective: To evaluate the feasibility of machine learning algorithms in detecting ADR signals of nivolumab and docetaxel, new and old anticancer agents.

Methods: We conducted a safety surveillance study of nivolumab and docetaxel using the Korea national spontaneous reporting database from 2009 to 2018. We constructed a novel input dataset for each study drug comprised of known ADRs that were listed in the drug labels and unknown ADRs. Given the known ADRs, we trained machine learning algorithms and evaluated predictive performance in generating safety signals of machine learning algorithms (gradient boosting machine [GBM] and random forest [RF]) compared with traditional disproportionality analysis methods (reporting odds ratio [ROR] and information component [IC]) by using the area under the curve (AUC). Each method then was implemented to detect new safety signals from the unknown ADR datasets.

Results: Of all methods implemented, GBM achieved the best average predictive performance (AUC: 0.97 and 0.93 for nivolumab and docetaxel). The AUC achieved by each method was 0.95 and 0.92 (RF), 0.55 and 0.51 (ROR), and 0.49 and 0.48 (IC) for respective drug. GBM detected additional 24 and nine signals for nivolumab and 82 and 76 for docetaxel compared to ROR and IC, respectively, from the unknown ADR datasets.

Conclusion: Machine learning algorithm based on GBM performed better and detected more new ADR signals than traditional disproportionality analysis methods.

Considering additive effects of polypharmacy : Analysis of adverse events in geriatric patients in long-term care facilities

Background

Potential additive effects of polypharmacy are rarely considered in adverse events of geriatric patients living in long-term care facilities. Our aim, therefore, was to identify adverse events in this setting and to assess plausible concomitant drug causes.

Methods

A cross-sectional observational study was performed in three facilities as follows: (i) adverse event identification: we structurally identified adverse events using nurses’ interviews and chart review. (ii) Analysis of the concomitantly administered drugs per patient was performed in two ways: (ii.a) a review of summary of product characteristics for listed adverse drug reactions to identify possible causing drugs and (ii.b) a causality assessment according to Naranjo algorithm.

Results

(i) We found 424 adverse events with a median of 4 per patient (range 1–14) in 103 of the 104 enrolled patients (99%). (ii.a) We identified a median of 3 drugs (range 0–11) with actually occurring adverse events listed as an adverse drug reaction in the summary of product characteristics. (ii.b) Causality was classified in 198 (46.9%) of adverse events as “doubtful,” in 218 (51.2%) as “possible,” in 7 (1.7%) as “probable,” and in 1 (0.2%) adverse event as a “definitive” cause of the administered drugs. In 340 (80.2%) of all identified adverse events several drugs simultaneously reached the highest respective Naranjo score.

Conclusion

Patients in long-term facilities frequently suffer from many adverse events. Concomitantly administered drugs have to be frequently considered as plausible causes for adverse events. These additive effects of drugs should be more focused in patient care and research.

Artificial Intelligence in Pharmacoepidemiology: A Systematic Review. Part 1-Overview of Knowledge Discovery Techniques in Artificial Intelligence

Aim

To perform a systematic review on the application of artificial intelligence (AI) based knowledge discovery techniques in pharmacoepidemiology.

Study Eligibility Criteria

Clinical trials, meta-analyses, narrative/systematic review, and observational studies using (or mentioning articles using) artificial intelligence techniques were eligible. Articles without a full text available in the English language were excluded.

Data Sources

Articles recorded from 1950/01/01 to 2019/05/06 in Ovid MEDLINE were screened.

Participants

Studies including humans (real or simulated) exposed to a drug.

Results

In total, 72 original articles and 5 reviews were identified via Ovid MEDLINE. Twenty different knowledge discovery methods were identified, mainly from the area of machine learning (66/72; 91.7%). Classification/regression (44/72; 61.1%), classification/regression + model optimization (13/72; 18.0%), and classification/regression + features selection (12/72; 16.7%) were the three most frequent tasks in reviewed literature that machine learning methods has been applied to solve. The top three used techniques were artificial neural networks, random forest, and support vector machines models.

Conclusions

The use of knowledge discovery techniques of artificial intelligence techniques has increased exponentially over the years covering numerous sub-topics of pharmacoepidemiology.

Systematic Review Registration

Systematic review registration number in PROSPERO: CRD42019136552.

Drug-Target Interaction Prediction via Dual Laplacian Graph Regularized Matrix Completion

Drug-target interactions play an important role for biomedical drug discovery and development. However, it is expensive and time-consuming to accomplish this task by experimental determination. Therefore, developing computational techniques for drug-target interaction prediction is urgent and has practical significance. In this work, we propose an effective computational model of dual Laplacian graph regularized matrix completion, referred to as DLGRMC briefly, to infer the unknown drug-target interactions. Specifically, DLGRMC transforms the task of drug-target interaction prediction into a matrix completion problem, in which the potential interactions between drugs and targets can be obtained based on the prediction scores after the matrix completion procedure. In DLGRMC, the drug pairwise chemical structure similarities and the target pairwise genomic sequence similarities are fully exploited to serve the matrix completion by using a dual Laplacian graph regularization term; i.e., drugs with similar chemical structure are more likely to have interactions with similar targets and targets with similar genomic sequence similarity are more likely to have interactions with similar drugs. In addition, during the matrix completion process, an indicator matrix with binary values which indicates the indices of the observed drug-target interactions is deployed to preserve the experimental confirmed interactions. Furthermore, we develop an alternative iterative strategy to solve the constrained matrix completion problem based on Augmented Lagrange Multiplier algorithm. We evaluate DLGRMC on five benchmark datasets and the results show that DLGRMC outperforms several state-of-the-art approaches in terms of 10-fold cross validation based AUPR values and PR curves. In addition, case studies also demonstrate that DLGRMC can successfully predict most of the experimental validated drug-target interactions.

Assessment of Medication Safety Using Only Dispensing Data

PURPOSE OF REVIEW

The purpose of this review is to provide an overview of the published studies that have been used to generate evidence on the safety of medicine use when only medication dispensing data are available.

RECENT FINDINGS

Medication dispensing databases are increasingly available for research on large populations, particularly in countries that provide universal coverage for medicines. These data are often used for drug utilisation studies to identify inappropriate medicine use at the population level that may be associated with known safety issues. Lack of coded diagnoses, to identify outcomes, and lack of data on confounders can limit use of these data in practice for medication safety assessment. To overcome these issues, studies have exploited the fact that symptoms of adverse effects of medications can be treated with other medications, for example antidepressants to treat depression or oxybutynin to treat urinary incontinence. The challenge of unmeasured confounding has been addressed by implementing self-controlled study designs that use within-person comparisons and provide inherent control for confounding. Prescription sequence symmetry analysis (SSA) is a within-person study design that has been demonstrated as a useful tool for safety signal generation in dispensing data.

SUMMARY

Using medicine initiation as a proxy for the development of adverse events can help to generate evidence of the safety of medicines when only medication dispensing data are available. Careful consideration, however, should be given to the sensitivity and specificity of the proxy medicine for the adverse event and potential for time-varying confounding due to trends in medicine utilisation. Data-mining approaches using dispensing data have the potential to improve safety assessments; however, the challenge of unmeasured confounding with these methods remains to be investigated.