1
|
Taylor LG, Kürzinger ML, Hermans R, Enshaeifar S, Dwan B, Chhikara P, Li X, Thummisetti S, Colas S, Duverne M, Juhaeri J. Considerations for practical use of tree-based scan statistics for signal detection using electronic healthcare data: a case study with insulin glargine. Expert Opin Drug Saf 2024:1-11. [PMID: 39162331 DOI: 10.1080/14740338.2024.2393274] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2024] [Accepted: 07/29/2024] [Indexed: 08/21/2024]
Abstract
BACKGROUND Hypothesis-free signal detection (HFSD) methods such as tree-based scan statistics (TBSS) applied to longitudinal electronic healthcare data (EHD) are increasingly used in safety monitoring. However, challenges may arise in interpreting HFSD results alongside results from disproportionality analysis of spontaneous reporting. RESEARCH DESIGN AND METHODS Using the anti-diabetes drug insulin glargine (Lantus®) we apply two different tree-based scan designs using TreeScan™ software on retrospective EHD and compare the results to one another as well as to results from a disproportionality analysis using SRD. RESULTS The self-controlled tree temporal scan method produced the larger number of alerts relative to propensity-score matched approach; however, far fewer alerts were observed when analyses were limited to EHD in inpatient/emergency room settings only. Very few reference adverse events were observed using TBSS methods on EHD relative to disproportionality methods in SRD. CONCLUSION Differences in detected alerts between TBSS methods and between TBSS and disproportionality analysis of SRD are likely attributable to differences in data, comparator, and study design. Our results suggest that HFDS methods like TBSS applied to EHD may complement more traditional approaches such as disproportionality analysis of SRD to provide a more complete picture of product safety in the post-approval setting.
Collapse
Affiliation(s)
| | | | | | | | | | | | - Xinyu Li
- Sanofi, Epidemiology and Benefit-Risk, PSPV, Bridgewater, MA, USA
| | | | - Sandrine Colas
- Sanofi Epidemiology and Benefit-Risk, PSPV, Toronto, Canada
| | | | - Juhaeri Juhaeri
- Sanofi, Epidemiology and Benefit-Risk, PSPV, Bridgewater, MA, USA
| |
Collapse
|
2
|
Jung J, Kim JH, Bae JH, Woo SS, Lee H, Shin JY. A real-world pharmacovigilance study on cardiovascular adverse events of tisagenlecleucel using machine learning approach. Sci Rep 2024; 14:13641. [PMID: 38871843 DOI: 10.1038/s41598-024-64466-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2024] [Accepted: 06/10/2024] [Indexed: 06/15/2024] Open
Abstract
Chimeric antigen receptor T-cell (CAR-T) therapies are a paradigm-shifting therapeutic in patients with hematological malignancies. However, some concerns remain that they may cause serious cardiovascular adverse events (AEs), for which data are scarce. In this study, gradient boosting machine algorithm-based model was fitted to identify safety signals of serious cardiovascular AEs reported for tisagenlecleucel in the World Health Organization Vigibase up until February 2024. Input dataset, comprised of positive and negative controls of tisagenlecleucel based on its labeling information and literature search, was used to train the model. Then, we implemented the model to calculate the predicted probability of serious cardiovascular AEs defined by preferred terms included in the important medical event list from European Medicine Agency. There were 467 distinct AEs from 3,280 safety cases reports for tisagenlecleucel, of which 363 (77.7%) were classified as positive controls, 66 (14.2%) as negative controls, and 37 (7.9%) as unknown AEs. The prediction model had area under the receiver operating characteristic curve of 0.76 in the test dataset application. Of the unknown AEs, six cardiovascular AEs were predicted as the safety signals: bradycardia (predicted probability 0.99), pleural effusion (0.98), pulseless electrical activity (0.89), cardiotoxicity (0.83), cardio-respiratory arrest (0.69), and acute myocardial infarction (0.58). Our findings underscore vigilant monitoring of acute cardiotoxicities with tisagenlecleucel therapy.
Collapse
Affiliation(s)
- Juhong Jung
- Department of Biohealth Regulatory Science, Sungkyunkwan University, Suwon, Republic of Korea
| | - Ju Hwan Kim
- Department of Biohealth Regulatory Science, Sungkyunkwan University, Suwon, Republic of Korea
- School of Pharmacy, Sungkyunkwan University, 2066, Seobu-ro, Jangan-gu, Suwon, Gyeonggi-do, 16419, Republic of Korea
| | - Ji-Hwan Bae
- School of Pharmacy, Sungkyunkwan University, 2066, Seobu-ro, Jangan-gu, Suwon, Gyeonggi-do, 16419, Republic of Korea
| | - Simon S Woo
- Department of Artificial Intelligence, College of Computing and Informatics, Sungkyunkwan University, Suwon, Republic of Korea
| | - Hyesung Lee
- Department of Biohealth Regulatory Science, Sungkyunkwan University, Suwon, Republic of Korea.
- School of Pharmacy, Sungkyunkwan University, 2066, Seobu-ro, Jangan-gu, Suwon, Gyeonggi-do, 16419, Republic of Korea.
| | - Ju-Young Shin
- Department of Biohealth Regulatory Science, Sungkyunkwan University, Suwon, Republic of Korea.
- School of Pharmacy, Sungkyunkwan University, 2066, Seobu-ro, Jangan-gu, Suwon, Gyeonggi-do, 16419, Republic of Korea.
- Samsung Advanced Institute for Health Sciences & Technology, Sungkyunkwan University, Seoul, Republic of Korea.
| |
Collapse
|
3
|
Coste A, Wong AY, Warren-Gash C, Matthewman J, Bate A, Douglas IJ. Implementation of a Taxonomy-Based Framework for the Selection of Appropriate Drugs and Outcomes for Real-World Data Signal Detection Studies. Drug Saf 2024; 47:183-192. [PMID: 38093083 PMCID: PMC10821990 DOI: 10.1007/s40264-023-01382-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/14/2023] [Indexed: 01/28/2024]
Abstract
INTRODUCTION For signal detection studies investigating either drug safety or method evaluation, the choice of drug-outcome pairs needs to be tailored to the planned study design and vice versa. While this is well understood in hypothesis-testing epidemiology, it should be as important in signal detection, but this has not widely been considered. There is a need for a taxonomy framework to provide guidance and a systematic reproducible approach to the selection of appropriate drugs and outcomes for signal detection studies either investigating drug safety or assessing method performance using real-world data. OBJECTIVE The aim was to design a general framework for the selection of appropriate drugs and outcomes for signal detection studies given a study design of interest. As a motivating example, we illustrate how the framework is applied to build a reference set for a study aiming to assess the performance of the self-controlled case series with active comparators. METHODS We reviewed criteria presented in two published studies which aimed to provide practical advice for choosing the appropriate signal evaluation methodology, and assessed their relevance for signal detection. Further characteristics specific to signal detection were added. The final framework is based on: the application of study design requirements, the database(s) of interest, and the clinical importance of the drug(s) and outcome(s) under consideration. This structure was applied by selecting drug-outcome pairs as a reference set (i.e. list of drug-outcome pairs classified as positive or negative controls) for which the method is expected to work well for a signal detection study aiming to assess the performance of self-controlled case series. Eight criteria were used, related to the application of self-controlled case series assumptions, choice of active comparators, coverage in the database of interest and clinical importance of the outcomes. RESULTS After application of the framework, two classes of antibiotics (seven drugs) were selected for the study, and 28 outcomes from all organ classes were chosen from the drug labels, out of the 273 investigated. In total, this corresponds to 104 positive controls (drug-outcome pairs) and 58 negative controls. CONCLUSIONS We proposed and applied a framework for the selection of drugs and outcomes for both drug safety signal detection and method assessment used in signal detection to optimise their performance given a study design. This framework will eliminate part of the bias relating to drugs and outcomes not being suited to the method or database. The main difficulty lies in the choice of the criteria and their application to ensure systematic selection, especially as some information remains unknown in signal detection, and clinical judgement was needed on occasions. The same framework could be adapted for other methods.
Collapse
Affiliation(s)
- Astrid Coste
- Department of Non-Communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine, Keppel St, WC1E 7HT, London, UK.
| | - Angel Ys Wong
- Department of Non-Communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine, Keppel St, WC1E 7HT, London, UK
| | - Charlotte Warren-Gash
- Department of Non-Communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine, Keppel St, WC1E 7HT, London, UK
| | - Julian Matthewman
- Department of Non-Communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine, Keppel St, WC1E 7HT, London, UK
| | - Andrew Bate
- Department of Non-Communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine, Keppel St, WC1E 7HT, London, UK
- GlaxoSmithKline, Brentford, UK
| | - Ian J Douglas
- Department of Non-Communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine, Keppel St, WC1E 7HT, London, UK
| |
Collapse
|
4
|
Detecting early safety signals of infliximab using machine learning algorithms in the Korea adverse event reporting system. Sci Rep 2022; 12:14869. [PMID: 36050484 PMCID: PMC9436954 DOI: 10.1038/s41598-022-18522-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2021] [Accepted: 08/16/2022] [Indexed: 11/26/2022] Open
Abstract
There has been a growing attention on using machine learning (ML) in pharmacovigilance. This study aimed to investigate the utility of supervised ML algorithms on timely detection of safety signals in the Korea Adverse Event Reporting System (KAERS), using infliximab as a case drug, between 2009 and 2018. Input data set for ML training was constructed based on the drug label information and spontaneous reports in the KAERS. Gold standard dataset containing known AEs was randomly divided into the training and test sets. Two supervised ML algorithms (gradient boosting machine [GBM], random forest [RF]) were fitted with hyperparameters tuned on the training set by using a fivefold validation. Then, we stratified the KAERS data by calendar year to create 10 cumulative yearly datasets, in which ML algorithms were applied to detect five pre-specified AEs of infliximab identified during post-marketing surveillance. Four AEs were detected by both GBM and RF in the first year they appeared in the KAERS and earlier than they were updated in the drug label of infliximab. We further applied our models to data retrieved from the US Food and Drug Administration Adverse Event Reporting System repository and found that they outperformed existing disproportionality methods. Both GBM and RF demonstrated reliable performance in detecting early safety signals and showed promise for applying such approaches to pharmacovigilance.
Collapse
|
5
|
Lee S, Lee JH, Kim GJ, Kim JY, Shin H, Ko I, Choe S, Kim JH. Development of a Data-Driven Reference Standard for Adverse Drug Reaction (RS-ADR) Signal Assessment (Preprint). J Med Internet Res 2021; 24:e35464. [PMID: 36201386 PMCID: PMC9585444 DOI: 10.2196/35464] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2021] [Revised: 04/29/2022] [Accepted: 07/14/2022] [Indexed: 11/13/2022] Open
Abstract
Background Pharmacovigilance using real-world data (RWD), such as multicenter electronic health records (EHRs), yields massively parallel adverse drug reaction (ADR) signals. However, proper validation of computationally detected ADR signals is not possible due to the lack of a reference standard for positive and negative associations. Objective This study aimed to develop a reference standard for ADR (RS-ADR) to streamline the systematic detection, assessment, and understanding of almost all drug-ADR associations suggested by RWD analyses. Methods We integrated well-known reference sets for drug-ADR pairs, including Side Effect Resource, Observational Medical Outcomes Partnership, and EU-ADR. We created a pharmacovigilance dictionary using controlled vocabularies and systematically annotated EHR data. Drug-ADR associations computed from MetaLAB and MetaNurse analyses of multicenter EHRs and extracted from the Food and Drug Administration Adverse Event Reporting System were integrated as “empirically determined” positive and negative reference sets by means of cross-validation between institutions. Results The RS-ADR consisted of 1344 drugs, 4485 ADRs, and 6,027,840 drug-ADR pairs with positive and negative consensus votes as pharmacovigilance reference sets. After the curation of the initial version of RS-ADR, novel ADR signals such as “famotidine–hepatic function abnormal” were detected and reasonably validated by RS-ADR. Although the validation of the entire reference standard is challenging, especially with this initial version, the reference standard will improve as more RWD participate in the consensus voting with advanced pharmacovigilance dictionaries and analytic algorithms. One can check if a drug-ADR pair has been reported by our web-based search interface for RS-ADRs. Conclusions RS-ADRs enriched with the pharmacovigilance dictionary, ADR knowledge, and real-world evidence from EHRs may streamline the systematic detection, evaluation, and causality assessment of computationally detected ADR signals.
Collapse
Affiliation(s)
- Suehyun Lee
- Department of Biomedical Informatics, College of Medicine, Konyang University, Daejeon, Republic of Korea
| | - Jeong Hoon Lee
- Seoul National University Biomedical Informatics (SNUBI), Division of Biomedical Informatics, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Grace Juyun Kim
- Seoul National University Biomedical Informatics (SNUBI), Division of Biomedical Informatics, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Jong-Yeup Kim
- Healthcare Data Science Center, Konyang University Hospital, Daejeon, Republic of Korea
| | - Hyunah Shin
- Healthcare Data Science Center, Konyang University Hospital, Daejeon, Republic of Korea
| | - Inseok Ko
- Healthcare Data Science Center, Konyang University Hospital, Daejeon, Republic of Korea
| | - Seon Choe
- Seoul National University Biomedical Informatics (SNUBI), Division of Biomedical Informatics, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Ju Han Kim
- Seoul National University Biomedical Informatics (SNUBI), Division of Biomedical Informatics, Seoul National University College of Medicine, Seoul, Republic of Korea
| |
Collapse
|
6
|
Khouri C, Nguyen T, Revol B, Lepelley M, Pariente A, Roustit M, Cracowski JL. Leveraging the Variability of Pharmacovigilance Disproportionality Analyses to Improve Signal Detection Performances. Front Pharmacol 2021; 12:668765. [PMID: 34122089 PMCID: PMC8193489 DOI: 10.3389/fphar.2021.668765] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2021] [Accepted: 05/14/2021] [Indexed: 11/13/2022] Open
Abstract
Background: A plethora of methods and models of disproportionality analyses for safety surveillance have been developed to date without consensus nor a gold standard, leading to methodological heterogeneity and substantial variability in results. We hypothesized that this variability is inversely correlated to the robustness of a signal of disproportionate reporting (SDR) and could be used to improve signal detection performances. Methods: We used a validated reference set containing 399 true and false drug-event pairs and performed, with a frequentist and a Bayesian disproportionality method, seven types of analyses (model) for which the results were very unlikely to be related to actual differences in absolute risks of ADR. We calculated sensitivity, specificity and plotted ROC curves for each model. We then evaluated the predictive capacities of all models and assessed the impact of combining such models with the number of positive SDR for a given drug-event pair through binomial regression models. Results: We found considerable variability in disproportionality analysis results, both positive and negative SDR could be generated for 60% of all drug-event pairs depending on the model used whatever their truthfulness. Furthermore, using the number of positive SDR for a given drug-event pair largely improved the signal detection performances of all models. Conclusion: We therefore advocate for the pre-registration of protocols and the presentation of a set of secondary and sensitivity analyses instead of a unique result to avoid selective outcome reporting and because variability in the results may reflect the likelihood of a signal being a true adverse drug reaction.
Collapse
Affiliation(s)
- Charles Khouri
- Pharmacovigilance Unit, Grenoble Alpes University Hospital, Grenoble, France.,Clinical Pharmacology Department INSERM CIC 1406, Grenoble Alpes University Hospital, Grenoble, France.,Hypoxia and PhysioPathology, UMR 1300, INSERM, University Grenoble Alpes, Grenoble, France
| | - Thuy Nguyen
- Pharmacovigilance Unit, Grenoble Alpes University Hospital, Grenoble, France
| | - Bruno Revol
- Pharmacovigilance Unit, Grenoble Alpes University Hospital, Grenoble, France.,Clinical Pharmacology Department INSERM CIC 1406, Grenoble Alpes University Hospital, Grenoble, France.,Hypoxia and PhysioPathology, UMR 1300, INSERM, University Grenoble Alpes, Grenoble, France
| | - Marion Lepelley
- Pharmacovigilance Unit, Grenoble Alpes University Hospital, Grenoble, France.,Clinical Pharmacology Department INSERM CIC 1406, Grenoble Alpes University Hospital, Grenoble, France
| | - Antoine Pariente
- INSERM U1219, Bordeaux Population Health, Team Pharmacoepidemiology, University of Bordeaux, Bordeaux, France.,Service de Pharmacologie Médicale, Pôle de Santé Publique, CHU de Bordeaux, Bordeaux, France
| | - Matthieu Roustit
- Clinical Pharmacology Department INSERM CIC 1406, Grenoble Alpes University Hospital, Grenoble, France.,Hypoxia and PhysioPathology, UMR 1300, INSERM, University Grenoble Alpes, Grenoble, France
| | - Jean-Luc Cracowski
- Pharmacovigilance Unit, Grenoble Alpes University Hospital, Grenoble, France.,Hypoxia and PhysioPathology, UMR 1300, INSERM, University Grenoble Alpes, Grenoble, France
| |
Collapse
|
7
|
Malec SA, Wei P, Bernstam EV, Boyce RD, Cohen T. Using computable knowledge mined from the literature to elucidate confounders for EHR-based pharmacovigilance. J Biomed Inform 2021; 117:103719. [PMID: 33716168 PMCID: PMC8559730 DOI: 10.1016/j.jbi.2021.103719] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2020] [Revised: 12/31/2020] [Accepted: 01/04/2021] [Indexed: 10/21/2022]
Abstract
INTRODUCTION Drug safety research asks causal questions but relies on observational data. Confounding bias threatens the reliability of studies using such data. The successful control of confounding requires knowledge of variables called confounders affecting both the exposure and outcome of interest. However, causal knowledge of dynamic biological systems is complex and challenging. Fortunately, computable knowledge mined from the literature may hold clues about confounders. In this paper, we tested the hypothesis that incorporating literature-derived confounders can improve causal inference from observational data. METHODS We introduce two methods (semantic vector-based and string-based confounder search) that query literature-derived information for confounder candidates to control, using SemMedDB, a database of computable knowledge mined from the biomedical literature. These methods search SemMedDB for confounders by applying semantic constraint search for indications treated by the drug (exposure) and that are also known to cause the adverse event (outcome). We then include the literature-derived confounder candidates in statistical and causal models derived from free-text clinical notes. For evaluation, we use a reference dataset widely used in drug safety containing labeled pairwise relationships between drugs and adverse events and attempt to rediscover these relationships from a corpus of 2.2 M NLP-processed free-text clinical notes. We employ standard adjustment and causal inference procedures to predict and estimate causal effects by informing the models with varying numbers of literature-derived confounders and instantiating the exposure, outcome, and confounder variables in the models with dichotomous EHR-derived data. Finally, we compare the results from applying these procedures with naive measures of association (χ2 and reporting odds ratio) and with each other. RESULTS AND CONCLUSIONS We found semantic vector-based search to be superior to string-based search at reducing confounding bias. However, the effect of including more rather than fewer literature-derived confounders was inconclusive. We recommend using targeted learning estimation methods that can address treatment-confounder feedback, where confounders also behave as intermediate variables, and engaging subject-matter experts to adjudicate the handling of problematic covariates.
Collapse
Affiliation(s)
- Scott A Malec
- University of Pittsburgh School of Medicine, Department of Biomedical Informatics, Pittsburgh, PA, United States.
| | - Peng Wei
- The University of Texas MD Anderson Cancer Center, Department of Biostatistics, Houston, TX, United States
| | - Elmer V Bernstam
- University of Texas Health Science Center at Houston, School of Biomedical Informatics, Houston, TX, United States
| | - Richard D Boyce
- University of Pittsburgh School of Medicine, Department of Biomedical Informatics, Pittsburgh, PA, United States
| | - Trevor Cohen
- University of Washington, Department of Biomedical Informatics and Medical Education, Seattle, WA, United States
| |
Collapse
|
8
|
Bae JH, Baek YH, Lee JE, Song I, Lee JH, Shin JY. Machine Learning for Detection of Safety Signals From Spontaneous Reporting System Data: Example of Nivolumab and Docetaxel. Front Pharmacol 2021; 11:602365. [PMID: 33628176 PMCID: PMC7898680 DOI: 10.3389/fphar.2020.602365] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2020] [Accepted: 11/02/2020] [Indexed: 12/14/2022] Open
Abstract
Introduction: Various methods have been implemented to detect adverse drug reaction (ADR) signals. However, the applicability of machine learning methods has not yet been fully evaluated. Objective: To evaluate the feasibility of machine learning algorithms in detecting ADR signals of nivolumab and docetaxel, new and old anticancer agents. Methods: We conducted a safety surveillance study of nivolumab and docetaxel using the Korea national spontaneous reporting database from 2009 to 2018. We constructed a novel input dataset for each study drug comprised of known ADRs that were listed in the drug labels and unknown ADRs. Given the known ADRs, we trained machine learning algorithms and evaluated predictive performance in generating safety signals of machine learning algorithms (gradient boosting machine [GBM] and random forest [RF]) compared with traditional disproportionality analysis methods (reporting odds ratio [ROR] and information component [IC]) by using the area under the curve (AUC). Each method then was implemented to detect new safety signals from the unknown ADR datasets. Results: Of all methods implemented, GBM achieved the best average predictive performance (AUC: 0.97 and 0.93 for nivolumab and docetaxel). The AUC achieved by each method was 0.95 and 0.92 (RF), 0.55 and 0.51 (ROR), and 0.49 and 0.48 (IC) for respective drug. GBM detected additional 24 and nine signals for nivolumab and 82 and 76 for docetaxel compared to ROR and IC, respectively, from the unknown ADR datasets. Conclusion: Machine learning algorithm based on GBM performed better and detected more new ADR signals than traditional disproportionality analysis methods.
Collapse
Affiliation(s)
- Ji-Hwan Bae
- School of Pharmacy, Sungkyunkwan University, Suwon-si, South Korea
| | - Yeon-Hee Baek
- School of Pharmacy, Sungkyunkwan University, Suwon-si, South Korea
| | - Jeong-Eun Lee
- School of Pharmacy, Sungkyunkwan University, Suwon-si, South Korea
| | - Inmyung Song
- Department of Health Administration, College of Nursing and Health, Kongju National University, Gongju-si, South Korea
| | - Jee-Hyong Lee
- Department of Artificial Intelligence, Sungkyunkwan University, Suwon-si, South Korea
| | - Ju-Young Shin
- School of Pharmacy, Sungkyunkwan University, Suwon-si, South Korea.,Samsung Advanced Institute for Health Sciences and Technology, Sungkyunkwan University, Jongno-gu, South Korea
| |
Collapse
|
9
|
Thurin NH, Lassalle R, Schuemie M, Pénichon M, Gagne JJ, Rassen JA, Benichou J, Weill A, Blin P, Moore N, Droz-Perroteau C. Empirical assessment of case-based methods for drug safety alert identification in the French National Healthcare System database (SNDS): Methodology of the ALCAPONE project. Pharmacoepidemiol Drug Saf 2020; 29:993-1000. [PMID: 32133717 DOI: 10.1002/pds.4983] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2019] [Revised: 01/02/2020] [Accepted: 02/12/2020] [Indexed: 01/22/2023]
Abstract
OBJECTIVES To introduce the methodology of the ALCAPONE project. BACKGROUND The French National Healthcare System Database (SNDS), covering 99% of the French population, provides a potentially valuable opportunity for drug safety alert generation. ALCAPONE aimed to assess empirically in the SNDS case-based designs for alert generation related to four health outcomes of interest. METHODS ALCAPONE used a reference set adapted from observational medical outcomes partnership (OMOP) and Exploring and Understanding Adverse Drug Reactions (EU-ADR) project, with four outcomes-acute liver injury (ALI), myocardial infarction (MI), acute kidney injury (AKI), and upper gastrointestinal bleeding (UGIB)-and positive and negative drug controls. ALCAPONE consisted of four main phases: (1) data preparation to fit the OMOP Common Data Model and select the drug controls; (2) detection of the selected controls via three case-based designs: case-population, case-control, and self-controlled case series, including design variants (varying risk window, adjustment strategy, etc.); (3) comparison of design variant performance (area under the ROC curve, mean square error, etc.); and (4) selection of the optimal design variants and their calibration for each outcome. RESULTS Over 2009-2014, 5225 cases of ALI, 354 109 MI, 12 633 AKI, and 156 057 UGIB were identified using specific definitions. The number of detectable drugs ranged from 61 for MI to 25 for ALI. Design variants generated more than 50 000 points estimates. Results by outcome will be published in forthcoming papers. CONCLUSIONS ALCAPONE has shown the interest of the empirical assessment of pharmacoepidemiological approaches for drug safety alert generation and may encourage other researchers to do the same in other databases.
Collapse
Affiliation(s)
- Nicolas H Thurin
- Bordeaux PharmacoEpi, INSERM CIC1401, Université de Bordeaux, Bordeaux, France.,INSERM U1219, Université de Bordeaux, Bordeaux, France
| | - Régis Lassalle
- Bordeaux PharmacoEpi, INSERM CIC1401, Université de Bordeaux, Bordeaux, France
| | - Martijn Schuemie
- Epidemiology Analytics, Janssen Research and Development, Titusville, New Jersey, USA.,Observational Health Data Sciences and Informatics (OHDSI), New York, New York, USA
| | - Marine Pénichon
- Bordeaux PharmacoEpi, INSERM CIC1401, Université de Bordeaux, Bordeaux, France
| | - Joshua J Gagne
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts, USA
| | | | - Jacques Benichou
- Department of Biostatistics and Clinical Research, Rouen University Hospital, Rouen, France.,INSERM U1181, Paris, France
| | - Alain Weill
- Caisse Nationale de l'Assurance Maladie, Paris, France
| | - Patrick Blin
- Bordeaux PharmacoEpi, INSERM CIC1401, Université de Bordeaux, Bordeaux, France
| | - Nicholas Moore
- Bordeaux PharmacoEpi, INSERM CIC1401, Université de Bordeaux, Bordeaux, France.,INSERM U1219, Université de Bordeaux, Bordeaux, France.,CHU de Bordeaux, Bordeaux, France
| | | |
Collapse
|
10
|
Hauben M, Reynolds R, Caubel P. Deconstructing the Pharmacovigilance Hype Cycle. Clin Ther 2019; 40:1981-1990.e3. [PMID: 30545608 DOI: 10.1016/j.clinthera.2018.10.021] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2018] [Revised: 10/11/2018] [Accepted: 10/24/2018] [Indexed: 12/31/2022]
Abstract
Data science is making increasing contributions to pharmacovigilance. Although the technical innovation of these works are indisputable, efficient progress in real-world pharmacovigilance signal detection may be hampered by corresponding technology life cycle effects, with a resulting tendency to conclude that, with large enough datasets and intricate algorithms, "the numbers speak for themselves," discounting the importance of clinical and scientific judgment. A practical consequence is overzealous declarations regarding the safety or lack of safety of drugs. We describe these concerns through a critical discussion of key results and conclusions from case studies selected to illustrate these points.
Collapse
|
11
|
Gavrielov-Yusim N, Kürzinger ML, Nishikawa C, Pan C, Pouget J, Epstein LB, Golant Y, Tcherny-Lessenot S, Lin S, Hamelin B, Juhaeri J. Comparison of text processing methods in social media-based signal detection. Pharmacoepidemiol Drug Saf 2019; 28:1309-1317. [PMID: 31392844 DOI: 10.1002/pds.4857] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2018] [Revised: 06/12/2019] [Accepted: 06/14/2019] [Indexed: 11/08/2022]
Abstract
PURPOSE Adverse event (AE) identification in social media (SM) can be performed using various types of natural language processing (NLP) and machine learning (ML). These methods can be categorized by complexity and precision level. Co-occurrence-based ML methods are rather basic, as they identify simultaneous appearance of drugs and clinical events in a single post. In contrast, statistical learning methods involve more complex NLP and identify drugs, events, and associations between them. We aimed to compare the ability of co-occurrence and NLP to identify AEs and signals of disproportionate reporting (SDR) in patient-generated SM. We also examined the performance of lift in SM-based signal detection (SD). METHODS Our examination was performed in a corpus of SM posts crawled from open online patient forums and communities, using the spontaneously reported VigiBase data as reference data set. RESULTS We found that co-occurrence and NLP produce AEs, which are 57% and 93% consistent with VigiBase AEs, respectively. Among the SDRs identified both in SM and in VigiBase, up to 55.3% were identified earlier in co-occurrence, and up to 32.1% were identified earlier in NLP-processed SM. Using lift in SM SD provided performance similar to frequentist methods, both in co-occurrence and in NLP-processed AEs. CONCLUSION Our results indicate that using SM as a data source complementary to traditional pharmacovigilance sources should be considered further. Various levels of SM processing may be considered, depending on the preferred policies and tolerance for false-positive to false-negative balance in routine pharmacovigilance processes.
Collapse
Affiliation(s)
| | | | - Chihiro Nishikawa
- Epidemiology and Benefit Risk Evaluation, Sanofi, Chilly-Mazarin, France
| | - Chunshen Pan
- Epidemiology and Benefit Risk Evaluation, Sanofi, Bridgewater, NJ, USA
| | - Julie Pouget
- Information Technology and Solutions, R&D CMO - SC Real World Evidence, Sanofi, Lyon, France
| | | | | | | | - Stephen Lin
- Global Pharmacovigilance, Sanofi, Bridgewater, NJ, USA
| | | | - Juhaeri Juhaeri
- Epidemiology and Benefit Risk Evaluation, Sanofi, Bridgewater, NJ, USA
| |
Collapse
|
12
|
Kürzinger ML, Schück S, Texier N, Abdellaoui R, Faviez C, Pouget J, Zhang L, Tcherny-Lessenot S, Lin S, Juhaeri J. Web-Based Signal Detection Using Medical Forums Data in France: Comparative Analysis. J Med Internet Res 2018; 20:e10466. [PMID: 30459145 PMCID: PMC6280030 DOI: 10.2196/10466] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2018] [Revised: 06/29/2018] [Accepted: 06/29/2018] [Indexed: 01/20/2023] Open
Abstract
BACKGROUND While traditional signal detection methods in pharmacovigilance are based on spontaneous reports, the use of social media is emerging. The potential strength of Web-based data relies on their volume and real-time availability, allowing early detection of signals of disproportionate reporting (SDRs). OBJECTIVE This study aimed (1) to assess the consistency of SDRs detected from patients' medical forums in France compared with those detected from the traditional reporting systems and (2) to assess the ability of SDRs in identifying earlier than the traditional reporting systems. METHODS Messages posted on patients' forums between 2005 and 2015 were used. We retained 8 disproportionality definitions. Comparison of SDRs from the forums with SDRs detected in VigiBase was done by describing the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, receiver operating characteristics curve, and the area under the curve (AUC). The time difference in months between the detection dates of SDRs from the forums and VigiBase was provided. RESULTS The comparison analysis showed that the sensitivity ranged from 29% to 50.6%, the specificity from 86.1% to 95.5%, the PPV from 51.2% to 75.4%, the NPV from 68.5% to 91.6%, and the accuracy from 68% to 87.7%. The AUC reached 0.85 when using the metric empirical Bayes geometric mean. Up to 38% (12/32) of the SDRs were detected earlier in the forums than that in VigiBase. CONCLUSIONS The specificity, PPV, and NPV were high. The overall performance was good, showing that data from medical forums may be a valuable source for signal detection. In total, up to 38% (12/32) of the SDRs could have been detected earlier, thus, ensuring the increased safety of patients. Further enhancements are needed to investigate the reliability and validation of patients' medical forums worldwide, the extension of this analysis to all possible drugs or at least to a wider selection of drugs, as well as to further assess performance against established signals.
Collapse
Affiliation(s)
| | | | | | | | | | - Julie Pouget
- Information Technology and Solutions, Sanofi, Lyon, France
| | - Ling Zhang
- Global Pharmacovigilance, Sanofi, Bridgewater, NJ, United States
| | | | - Stephen Lin
- Global Pharmacovigilance, Sanofi, Bridgewater, NJ, United States
| | - Juhaeri Juhaeri
- Epidemiology and Benefit Risk Evaluation, Sanofi, Bridgewater, NJ, United States
| |
Collapse
|
13
|
Trinh NTH, Solé E, Benkebil M. Benefits of combining change-point analysis with disproportionality analysis in pharmacovigilance signal detection. Pharmacoepidemiol Drug Saf 2018; 28:370-376. [PMID: 29992679 DOI: 10.1002/pds.4613] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2017] [Revised: 01/08/2018] [Accepted: 06/04/2018] [Indexed: 11/08/2022]
Abstract
BACKGROUND Change-point analysis (CPA) is a powerful method to analyse pharmacovigilance data but it has never been used on the disproportionality metric. OBJECTIVES To optimize signal detection investigating the interest of time-series analysis in pharmacovigilance and the benefits of combining CPA with the proportional reporting ratio (PRR). METHODS We investigated the couple benfluorex and aortic valve incompetence (AVI) using the French National Pharmacovigilance and EudraVigilance databases: CPA was applied on monthly counts of reports and the lower bound of monthly computed PRR (PRR-). We stated a CPA hypothesis that the substance-event combination is more likely to be a signal when the 2 following criteria are fulfilled: PRR- is greater than 1 with at least 5 cases, and CPA method detects at least 2 successive change points of PRR- which made consecutively increasing segments. We tested this hypothesis by 95 test cases identified from a drug safety reference set and 2 validated signals from EudraVigilance database: CPA was applied on PRR-. RESULTS For benfluorex and AVI, change points detected by CPA on PRR- were more meaningful compared with monthly counts of reports: More change points detected and detected earlier. In the reference set, 14 positive controls satisfied CPA hypothesis, 6 positive controls only met first requirements, 3 negative controls only met first requirement, and 2 validated signals satisfied CPA hypothesis. CONCLUSIONS The combination of CPA and PRR represents a significant advantage in detecting earlier signals and reducing false-positive signals. This approach should be confirmed in further studies.
Collapse
Affiliation(s)
- Nhung T H Trinh
- Inserm UMR 1153, Obstetrical, Perinatal and Pediatric Epidemiology Research Team, Research Center for Epidemiology and Biostatistics Sorbonne Paris Cité (CRESS), Paris Descartes University, Paris, France.,Adverse Events and incidents Department-Surveillance Division, Agence nationale de sécurité du médicament et des produits de santé (ANSM), Saint Denis, France
| | - Elodie Solé
- Adverse Events and incidents Department-Surveillance Division, Agence nationale de sécurité du médicament et des produits de santé (ANSM), Saint Denis, France
| | - Mehdi Benkebil
- Adverse Events and incidents Department-Surveillance Division, Agence nationale de sécurité du médicament et des produits de santé (ANSM), Saint Denis, France
| |
Collapse
|
14
|
An MCEM Framework for Drug Safety Signal Detection and Combination from Heterogeneous Real World Evidence. Sci Rep 2018; 8:1806. [PMID: 29379048 PMCID: PMC5789130 DOI: 10.1038/s41598-018-19979-7] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2017] [Accepted: 01/11/2018] [Indexed: 11/08/2022] Open
Abstract
Delayed drug safety insights can impact patients, pharmaceutical companies, and the whole society. Post-market drug safety surveillance plays a critical role in providing drug safety insights, where real world evidence such as spontaneous reporting systems (SRS) and a series of disproportional analysis serve as a cornerstone of proactive and predictive drug safety surveillance. However, they still face several challenges including concomitant drugs confounders, rare adverse drug reaction (ADR) detection, data bias, and the under-reporting issue. In this paper, we are developing a new framework that detects improved drug safety signals from multiple data sources via Monte Carlo Expectation-Maximization (MCEM) and signal combination. In MCEM procedure, we propose a new sampling approach to generate more accurate SRS signals for each ADR through iteratively down-weighting their associations with irrelevant drugs in case reports. While in signal combination step, we adopt Bayesian hierarchical model and propose a new summary statistic such that SRS signals can be combined with signals derived from other observational health data allowing for related signals to borrow statistical support with adjustment of data reliability. They combined effectively alleviate the concomitant confounders, data bias, rare ADR and under-reporting issues. Experimental results demonstrated the effectiveness and usefulness of the proposed framework.
Collapse
|
15
|
Harpaz R, DuMouchel W, Schuemie M, Bodenreider O, Friedman C, Horvitz E, Ripple A, Sorbello A, White RW, Winnenburg R, Shah NH. Toward multimodal signal detection of adverse drug reactions. J Biomed Inform 2017; 76:41-49. [PMID: 29081385 PMCID: PMC8502488 DOI: 10.1016/j.jbi.2017.10.013] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2017] [Revised: 10/14/2017] [Accepted: 10/24/2017] [Indexed: 11/27/2022]
Abstract
OBJECTIVE Improving mechanisms to detect adverse drug reactions (ADRs) is key to strengthening post-marketing drug safety surveillance. Signal detection is presently unimodal, relying on a single information source. Multimodal signal detection is based on jointly analyzing multiple information sources. Building on, and expanding the work done in prior studies, the aim of the article is to further research on multimodal signal detection, explore its potential benefits, and propose methods for its construction and evaluation. MATERIAL AND METHODS Four data sources are investigated; FDA's adverse event reporting system, insurance claims, the MEDLINE citation database, and the logs of major Web search engines. Published methods are used to generate and combine signals from each data source. Two distinct reference benchmarks corresponding to well-established and recently labeled ADRs respectively are used to evaluate the performance of multimodal signal detection in terms of area under the ROC curve (AUC) and lead-time-to-detection, with the latter relative to labeling revision dates. RESULTS Limited to our reference benchmarks, multimodal signal detection provides AUC improvements ranging from 0.04 to 0.09 based on a widely used evaluation benchmark, and a comparative added lead-time of 7-22 months relative to labeling revision dates from a time-indexed benchmark. CONCLUSIONS The results support the notion that utilizing and jointly analyzing multiple data sources may lead to improved signal detection. Given certain data and benchmark limitations, the early stage of development, and the complexity of ADRs, it is currently not possible to make definitive statements about the ultimate utility of the concept. Continued development of multimodal signal detection requires a deeper understanding the data sources used, additional benchmarks, and further research on methods to generate and synthesize signals.
Collapse
Affiliation(s)
- Rave Harpaz
- Oracle Health Sciences, Bedford, MA, United States.
| | | | | | | | | | | | - Anna Ripple
- National Library of Medicine, NIH, Bethesda, MD, United States
| | | | | | | | - Nigam H Shah
- Stanford University, Stanford, CA, United States
| |
Collapse
|
16
|
Capó-Lugo CE, Kho AN, O'Dwyer LC, Rosenman MB. Data Sharing and Data Registries in Physical Medicine and Rehabilitation. PM R 2017; 9:S59-S74. [PMID: 28527505 DOI: 10.1016/j.pmrj.2017.04.003] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2017] [Revised: 04/04/2017] [Accepted: 04/06/2017] [Indexed: 11/26/2022]
Abstract
The field of physical medicine & rehabilitation (PM&R), along with all the disciplines it encompasses, has evolved rapidly in the past 50 years. The number of controlled trials, systematic reviews, and meta-analyses in PM&R increased 5-fold from 1998 to 2013. In recent years, professional, private, and governmental institutions have identified the need to track function and functional status across providers and settings of care and on a larger scale. Because function and functional status are key aspects of PM&R, access to and sharing of reliable data will have an important impact on clinical practice. We reviewed the current landscape of PM&R databases and data repositories, the clinical applicability and practice implications of data sharing, and challenges and future directions. We included articles that (1) addressed any aspect of function, disability, or participation; (2) focused on recovery or maintenance of any function; and (3) used data repositories or research databases. We identified 398 articles that cited 244 data sources. The data sources included 66 data repositories and 179 research databases. We categorized the data sources based on their purposes and uses, geographic distribution, and other characteristics. This study collates the range of databases, data repositories, and data-sharing mechanisms that have been used in PM&R internationally. In recent years, these data sources have provided significant information for the field, especially at the population-health level. Implications and future directions for data sources also are discussed.
Collapse
Affiliation(s)
- Carmen E Capó-Lugo
- Center for Education in Health Sciences, Institute for Public Health and Medicine, Feinberg School of Medicine, Northwestern University, 633 N. St. Clair St, 20th Floor, Chicago, IL 60611(∗).
| | - Abel N Kho
- Center for Health Information Partnerships, Institute for Public Health and Medicine and Department of Medicine, Feinberg School of Medicine, Northwestern University, Chicago, IL(†)
| | - Linda C O'Dwyer
- Galter Health Sciences Library, Feinberg School of Medicine, Northwestern University, Chicago, IL(‡)
| | - Marc B Rosenman
- Center for Health Information Partnerships, Institute for Public Health and Medicine and Department of Pediatrics, Feinberg School of Medicine, Northwestern University, Chicago, IL(§)
| |
Collapse
|
17
|
Voss EA, Boyce RD, Ryan PB, van der Lei J, Rijnbeek PR, Schuemie MJ. Accuracy of an automated knowledge base for identifying drug adverse reactions. J Biomed Inform 2016; 66:72-81. [PMID: 27993747 DOI: 10.1016/j.jbi.2016.12.005] [Citation(s) in RCA: 68] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2016] [Revised: 12/08/2016] [Accepted: 12/10/2016] [Indexed: 11/15/2022]
Abstract
INTRODUCTION Drug safety researchers seek to know the degree of certainty with which a particular drug is associated with an adverse drug reaction. There are different sources of information used in pharmacovigilance to identify, evaluate, and disseminate medical product safety evidence including spontaneous reports, published peer-reviewed literature, and product labels. Automated data processing and classification using these evidence sources can greatly reduce the manual curation currently required to develop reference sets of positive and negative controls (i.e. drugs that cause adverse drug events and those that do not) to be used in drug safety research. METHODS In this paper we explore a method for automatically aggregating disparate sources of information together into a single repository, developing a predictive model to classify drug-adverse event relationships, and applying those predictions to a real world problem of identifying negative controls for statistical method calibration. RESULTS Our results showed high predictive accuracy for the models combining all available evidence, with an area under the receiver-operator curve of ⩾0.92 when tested on three manually generated lists of drugs and conditions that are known to either have or not have an association with an adverse drug event. CONCLUSIONS Results from a pilot implementation of the method suggests that it is feasible to develop a scalable alternative to the time-and-resource-intensive, manual curation exercise previously applied to develop reference sets of positive and negative controls to be used in drug safety research.
Collapse
Affiliation(s)
- E A Voss
- Epidemiology Analytics, Janssen Research & Development, LLC, Raritan, NJ, United States; Erasmus University Medical Center, Rotterdam, Netherlands; Observational Health Data Sciences and Informatics (OHDSI), New York, NY, United States.
| | - R D Boyce
- University of Pittsburgh, Pittsburgh, PA, United States; Observational Health Data Sciences and Informatics (OHDSI), New York, NY, United States
| | - P B Ryan
- Epidemiology Analytics, Janssen Research & Development, LLC, Raritan, NJ, United States; Columbia University, New York, NY, United States; Observational Health Data Sciences and Informatics (OHDSI), New York, NY, United States
| | - J van der Lei
- Erasmus University Medical Center, Rotterdam, Netherlands; Observational Health Data Sciences and Informatics (OHDSI), New York, NY, United States
| | - P R Rijnbeek
- Erasmus University Medical Center, Rotterdam, Netherlands; Observational Health Data Sciences and Informatics (OHDSI), New York, NY, United States
| | - M J Schuemie
- Epidemiology Analytics, Janssen Research & Development, LLC, Raritan, NJ, United States; Observational Health Data Sciences and Informatics (OHDSI), New York, NY, United States
| |
Collapse
|
18
|
Hauben M, Hung EY. Revisiting the reported signal of acute pancreatitis with rasburicase: an object lesson in pharmacovigilance. Ther Adv Drug Saf 2016; 7:94-101. [PMID: 27298720 DOI: 10.1177/2042098616647955] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/25/2023] Open
Abstract
INTRODUCTION There is an interest in methodologies to expeditiously detect credible signals of drug-induced pancreatitis. An example is the reported signal of pancreatitis with rasburicase emerging from a study [the 'index publication' (IP)] combining quantitative signal detection findings from a spontaneous reporting system (SRS) and electronic health records (EHRs). The signal was reportedly supported by a clinical review with a case series manuscript in progress. The reported signal is noteworthy, being initially classified as a false-positive finding for the chosen reference standard, but reclassified as a 'clinically supported' signal. OBJECTIVE This paper has dual objectives: to revisit the signal of rasburicase and acute pancreatitis and extend the original analysis via reexamination of its findings, in light of more contemporary data; and to motivate discussions on key issues in signal detection and evaluation, including recent findings from a major international pharmacovigilance research initiative. METHODOLOGY We used the same methodology as the IP, including the same disproportionality analysis software/dataset for calculating observed to expected reporting frequencies (O/Es), Medical Dictionary for Regulatory Activities Preferred Term, and O/E metric/threshold combination defining a signal of disproportionate reporting. Baseline analysis results prompted supplementary analyses using alternative analytical choices. We performed a comprehensive literature search to identify additional published case reports of rasburicase and pancreatitis. RESULTS We could not replicate positive findings (e.g. a signal or statistic of disproportionate reporting) from the SRS data using the same algorithm, software, dataset and vendor specified in the IP. The reporting association was statistically highlighted in default and supplemental analysis when more sensitive forms of disproportionality analysis were used. Two of three reports in the FAERS database were assessed as likely duplicate reports. We did not identify any additional reports in the FAERS corresponding to the three cases identified in the IP using EHRs. We did not identify additional published reports of pancreatitis associated with rasburicase. DISCUSSION Our exercise stimulated interesting discussions of key points in signal detection and evaluation, including causality assessment, signal detection algorithm performance, pharmacovigilance terminology, duplicate reporting, mechanisms for communicating signals, the structure of the FAERs database, and recent results from a major international pharmacovigilance research initiative.
Collapse
Affiliation(s)
- Manfred Hauben
- New York University School of Medicine, and Pfizer Inc., Safety Sciences Research, 235 East 42nd Street, Mail Stop 219-9-W, New York, NY 10017, USA
| | - Eric Y Hung
- Pfizer Inc., Safety Sciences Research, New York, NY, USA
| |
Collapse
|
19
|
Slattery J. Measuring Signal Detection Performance: Can We Trust Negative Controls and Do We Need Them? Drug Saf 2016; 39:371-3. [PMID: 26895343 DOI: 10.1007/s40264-016-0407-z] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
|