Performance of Stratified and Subgrouped Disproportionality Analyses in Spontaneous Databases

Caveats of Covariate Adjustment in Disproportionality Analysis for Best Practices

Spontaneous reporting systems (SRS) provide valuable data for detecting unidentified adverse events not observed in clinical trials and for conducting safety assessments that accurately reflect real-world clinical practice. With the increasing number of publications using the SRS for disproportionality analysis (DA), there is an increasing demand for a comprehensive understanding of the research limitations associated with the SRS. However, there is a lack of understanding of the caveats associated with adjusting covariates in DA of the SRS. Herein, we summarized the use of covariate adjustment and its caveats in DA. The Council for International Organizations of Medical Sciences VIII suggests considering adjustments such as stratification when they can enhance the sensitivity and/or specificity of statistical analysis. However, several database-specific and statistical caveats have been identified when adjusting for covariates derived from the SRS. Disproportionality analysis may be affected not only by reporting bias at the time of enrollment but also by sparse-data bias due to variations in the number of enrollment reports. Statistical evidence is needed to determine in which cases and to what extent sensitivity and/or specificity are affected. Nevertheless, it is important for researchers to acknowledge that certain limitations discussed in this context may be inherent and cannot be rectified. Based on this understanding, they can then make an informed decision on whether to perform a covariate adjustment.

The evolving role of disproportionality analysis in pharmacovigilance

INTRODUCTION

From 2009 to 2015, the IMI PROTECT conducted rigorous studies addressing questions about optimal implementation and significance of disproportionality analyses, leading to the development of Good Signal Detection Practices. The ensuing period witnessed the independent exploration of research paths proposed by IMI PROTECT, accumulating valuable experience and insights that have yet to be seamlessly integrated.

AREAS COVERED

This state-of-the-art review integrates IMI PROTECT recommendations with recent acquisitions and evolving challenges. It deals with defining the object of study, disproportionality methods, subgrouping, masking, drug-drug interaction, duplication, expectedness, the debated use of disproportionality results as risk measures, integration with other types of data.

EXPERT OPINION

Despite the ongoing skepticism regarding the usefulness of disproportionality analyses and individual case safety reports, their ability to timely detect safety signals regarding rare and unpredictable adverse reactions remains unparalleled. Moreover, recent exploration into their potential for characterizing safety signals revealed valuable insights concerning potential risk factors and the patient's perspective. To fully realize their potential beyond hypothesis generation and achieve a comprehensive evidence synthesis with other kinds of data and studies, each with their unique limitations and contributions, we need to investigate methods for more transparently communicating disproportionality results and mapping and addressing pharmacovigilance biases.

In-silico approaches to assessing multiple high-level drug-drug and drug-disease adverse drug effects

INTRODUCTION

Pharmacovigilance plays a pivotal role in monitoring adverse events (AEs) related to chemical substances in human/animal populations. With increasing spontaneous-reporting systems, researchers turned to in-silico approaches to efficiently analyze drug safety profiles. Here, we review in-silico methods employed for assessing multiple drug-drug/drug-disease AEs covered by comparative analyses and visualization strategies.

AREAS COVERED

Disproportionality, involving multi-stage statistical methodologies and data processing, identifies safety signals among drug-AE pairs. By stratifying data based on disease indications/demographics, researchers address confounders and assess drug safety. Comparative analyses, including clustering techniques and visualization techniques, assess drug similarities, patterns, and trends, calculate correlations, and identify distinct toxicities. Furthermore, we conducted a thorough Scopus search on 'pharmacovigilance,' yielding 5,836 publications spanning 2003 to 2023.

EXPERT OPINION

Pharmacovigilance relies on diverse data sources, presenting challenges in the integration of in-silico approaches and requiring compliance with regulations and AI adoption. Systematic use of statistical analyses enables identifications of potential risks with drugs. Frequentist and Bayesian methods are used in disproportionalities, each with its strengths and weaknesses. Integration of pharmacogenomics with pharmacovigilance enables personalized medicine, with AI further enhancing patient engagement. This multidisciplinary approach holds promise, improving drug efficacy and safety, and should be a core mission of One-Health studies.

The REporting of A Disproportionality Analysis for DrUg Safety Signal Detection Using Individual Case Safety Reports in PharmacoVigilance (READUS-PV): Explanation and Elaboration

In pharmacovigilance, disproportionality analyses based on individual case safety reports are widely used to detect safety signals. Unfortunately, publishing disproportionality analyses lacks specific guidelines, often leading to incomplete and ambiguous reporting, and carries the risk of incorrect conclusions when data are not placed in the correct context. The REporting of A Disproportionality analysis for drUg Safety signal detection using individual case safety reports in PharmacoVigilance (READUS-PV) statement was developed to address this issue by promoting transparent and comprehensive reporting of disproportionality studies. While the statement paper explains in greater detail the procedure followed to develop these guidelines, with this explanation paper we present the 14 items retained for READUS-PV guidelines, together with an in-depth explanation of their rationale and bullet points to illustrate their practical implementation. Our primary objective is to foster the adoption of the READUS-PV guidelines among authors, editors, peer reviewers, and readers of disproportionality analyses. Enhancing transparency, completeness, and accuracy of reporting, as well as proper interpretation of their results, READUS-PV guidelines will ultimately facilitate evidence-based decision making in pharmacovigilance.

Cardiovascular Safety Evaluation of Febuxostat and Allopurinol: Findings from the FDA Adverse Event Reporting System

BACKGROUND

Febuxostat and allopurinol are the most commonly used uric acid-lowering medications, and their safety is of great concern, especially the cardiovascular adverse reactions associated with febuxostat. We propose to study the cardiovascular toxicity of febuxostat and allopurinol using the FDA Adverse Event Reporting System (FAERS) database.

METHODS

A total of 64 quarters of FAERS data were downloaded from 2004 to 2019. Febuxostat- and allopurinol-related cardiovascular adverse events were extracted after data cleaning. Signal detection was conducted by reporting odds ratio (ROR) and proportional reporting ratio (PRR).

RESULTS

There were 2939 and 25,219 reports of febuxostat- and allopurinol-related cardiovascular adverse events (CVAEs), respectively. The most frequent CVAEs with febuxostat and allopurinol were edema peripheral (14.38%) and peripheral swelling (8.76%), respectively. In elderly gout patients, febuxostat is associated with an increased risk of heart failure, ischemic heart disease, hypertension, and cardiomyopathy. Febuxostat in combination with acetic acid derivatives nonsteroidal anti-inflammatory drug (NSAIDS) also increases the risk of cardiovascular adverse events.

CONCLUSIONS

Compared with allopurinol, febuxostat may increase cardiovascular toxicity in patients with gout.

Stacking Ensemble of Disproportionality Indicators for Adverse Vaccine Reactions Detection-An Empirical Study on Predicting Adverse Reactions of COVID-19 Vaccines

Vaccine safety is a critical issue for public health, which has recently become more crucial than ever since COVID-19 started to spread worldwide in 2020. Many COVID-19 vaccines have been developed and used without following the traditional three clinical trial stages. Instead, most COVID-19 vaccines were approved through emergency use approval (EUA) within one year, significantly raising the risk of rare and severe adverse events. Reporting systems like the Vaccine Adverse Event Reporting System (VAERS) have been established worldwide to detect unknown and severe adverse reactions as early as possible. Although experts and researchers have been working hard to find ways to detect adverse vaccine event (AVE) signals from VAERS data, most of the contemporary methods are statistical methods based on measuring the disproportionality between vaccine-induced events and non-vaccine-induced events. This paper proposes a novel ensemble AVE detection method, which adopts a stacking ensemble of various disproportionality indicators, fusing dual-scale contingency values measured in single and cumulative yearly duration, and embraces the concept of feature concatenation. Experiments conducted on US VAERS data to predict AVE caused by COVID-19 vaccines show that our proposed method is effective. We observed that: (1) Stacking ensemble of various disproportionality indicators is superior to any single disproportionality indicator and voting ensemble method; (2) Fusing dual-scale contingency values and feature concatenation brings synergy to our proposed stacking ensemble AVE detection. Compared to the best disproportionality metric in this study, our top-performing ensemble version exhibited a 34% improvement in accuracy, 71% in precision, 29% in recall, and 77% in F-measure, with a slight decrease (8%) in specificity.

Performance of subgrouped proportional reporting ratios in the US Food and Drug Administration (FDA) adverse event reporting system

BACKGROUND

Many signal detection algorithms give the same weight to information from all products and patients, which may result in signals being masked or false positives being flagged as potential signals. Subgrouped analysis can be used to help correct for this.

RESEARCH DESIGN AND METHODS

The publicly available US Food and Drug Administration Adverse Event Reporting System quarterly data extract files from 1 January 2015 through 30 September 2017 were utilized. A proportional reporting ratio (PRR) analysis subgrouped by either age, sex, ADE report type, seriousness of ADE, or reporter was compared to the crude PRR analysis using sensitivity, specificity, precision, and c-statistic.

RESULTS

Subgrouping by age (n = 78, 34.5% increase), sex (n = 67, 15.5% increase), and reporter (n = 64, 10.3% increase) identified more signals than the crude analysis. Subgrouping by either age or sex increased both the sensitivity and precision. Subgrouping by report type or seriousness resulted in fewer signals (n = 50, -13.8% for both). Subgrouped analyses had higher c-statistic values, with age having the highest (0.468).

CONCLUSIONS

Subgrouping by either age or sex produced more signals with higher sensitivity and precision than the crude PRR analysis. Subgrouping by these variables can unmask potentially important associations.

Identifying Safety Subgroups at Risk: Assessing the Agreement Between Statistical Alerting and Patient Subgroup Risk

INTRODUCTION

Identifying individual characteristics or underlying conditions linked to adverse drug reactions (ADRs) can help optimise the benefit-risk ratio for individuals. A systematic evaluation of statistical methods to identify subgroups potentially at risk using spontaneous ADR report datasets is lacking.

OBJECTIVES

In this study, we aimed to assess concordance between subgroup disproportionality scores and European Medicines Agency Pharmacovigilance Risk Assessment Committee (PRAC) discussions of potential subgroup risk.

METHODS

The subgroup disproportionality method described by Sandberg et al., and variants, were applied to statistically screen for subgroups at potential increased risk of ADRs, using data from the US FDA Adverse Event Reporting System (FAERS) cumulative from 2004 to quarter 2 2021. The reference set used to assess concordance was manually extracted from PRAC minutes from 2015 to 2019. Mentions of subgroups presenting potential differentiated risk and overlapping with the Sandberg method were included.

RESULTS

Twenty-seven PRAC subgroup examples representing 1719 subgroup drug-event combinations (DECs) in FAERS were included. Using the Sandberg methodology, 2 of the 27 could be detected (one for age and one for sex). No subgroup examples for pregnancy and underlying condition were detected. With a methodological variant, 14 of 27 examples could be detected.

CONCLUSIONS

We observed low concordance between subgroup disproportionality scores and PRAC discussions of potential subgroup risk. Subgroup analyses performed better for age and sex, while for covariates not well-captured in FAERS, such as underlying condition and pregnancy, additional data sources should be considered.

Safety profile of monoclonal antibodies targeting the calcitonin gene-related peptide system in pregnancy: Updated analysis in VigiBase®

BACKGROUND

Safety data on the use of migraine preventive monoclonal antibodies targeting the calcitonin gene-related peptide (CGRP) system in pregnancy are limited.

METHODS

Updated pharmacovigilance assessment of the safety reports related to pregnancy associated with erenumab, galcanezumab, fremanezumab and eptinezumab, retrieved from VigiBase® as of 31 December 2021. As primary outcome, the whole group of monoclonal antibodies targeting the CGRP system was considered and sex and age subgroup disproportionality analyses using the reporting odds ratio (ROR) were conducted.

RESULTS

286 safety reports were found: 116 (40.6%) on erenumab, 125 (43.7%) on galcanezumab, 39 (13.6%) on fremanezumab, 6 (2.1%) on eptinezumab. One hundred and forty-nine (52.1%) safety reports reported only drug exposure in relation to pregnancy while 137 (47.9%) also included ≥1 pregnancy outcomes: maternal outcomes (n = 64), spontaneous abortion (n = 63), foetal growth restriction (n = 1), prematurity (n = 8), neonatal outcomes (n = 13), and poor breastfeeding (n = 1). No specific patterns of maternal, foetal and neonatal toxicity were observed. Spontaneous abortion was not disproportionally more frequently reported with erenumab, galcanezumab, fremanezumab and eptinezumab compared with the entire database (ROR 1.1, 95% confidence interval, CI, 0.8-1.5), the entire database since 2018 (ROR 1.3, 95% CI 1.0-1.8), and triptans (ROR 1.2, 95% CI 0.8-1.9).

CONCLUSIONS

This updated safety analysis on erenumab, galcanezumab, fremanezumab and eptinezumab in pregnancy showed no signals of foeto-maternal toxicity according to VigiBase® safety reports.

Immune Checkpoint Inhibitors and Pregnancy: Analysis of the VigiBase® Spontaneous Reporting System

In pregnancy, immune checkpoint pathways are involved in the maintenance of fetomaternal immune tolerance. Preclinical studies have shown that immune checkpoint inhibitors (ICIs) increase the risk of fetal death. Despite the fact that using ICIs in pregnant women and women of childbearing potential is not recommended, some case reports of ICI exposure in pregnancy have been published showing favorable fetal outcomes. This study aimed to gain further insight into ICI safety in pregnancy by querying VigiBase^®, the World Health Organization's spontaneous reporting system. We performed raw and subgroup disproportionality analyses using the reporting odds ratio and comparing ICIs with the entire database, other antineoplastic agents, and other antineoplastic agents gathered in VigiBase^® since 2011. Across 103 safety reports referring to ICI exposure during the peri-pregnancy period, 56 reported pregnancy-related outcomes, of which 46 were without concomitant drugs as potential confounding factors. No signals of disproportionate reporting were found for spontaneous abortion, fetal growth restriction, and prematurity. In light of the expanding indications of ICIs, continuous surveillance by clinicians and pharmacovigilance experts is warranted, along with pharmacoepidemiological studies on other sources of real-world evidence, such as birth records, to precisely assess ICI exposure during the peri-pregnancy period and further characterize relevant outcomes.

Advancing drug safety science by integrating molecular knowledge with post-marketing adverse event reports

Promising drug development efforts may frequently fail due to unintended adverse reactions. Several methods have been developed to analyze such data, aiming to improve pharmacovigilance and drug safety. In this work, we provide a brief review of key directions to quantitatively analyzing adverse events and explore the potential of augmenting these methods using additional molecular data descriptors. We argue that molecular expansion of adverse event data may provide a path to improving the insights gained through more traditional pharmacovigilance approaches. Examples include the ability to assess statistical relevance with respect to underlying biomolecular mechanisms, the ability to generate plausible causative hypotheses and/or confirmation where possible, the ability to computationally study potential clinical trial designs and/or results, as well as the further provision of advanced features incorporated in innovative methods, such as machine learning. In summary, molecular data expansion provides an elegant way to extend mechanistic modeling, systems pharmacology, and patient‐centered approaches for the assessment of drug safety. We anticipate that such advances in real‐world data informatics and outcome analytics will help to better inform public health, via the improved ability to prospectively understand and predict various types of drug‐induced molecular perturbations and adverse events.

The Impact of Mandatory Reporting of Non-Serious Safety Reports to EudraVigilance on the Detection of Adverse Reactions

INTRODUCTION AND OBJECTIVE

European Union legislation has mandated the submission of European Economic Area non-serious reports to the EudraVigilance database since November 2017. As spontaneous reports of suspected adverse reactions to medicines represent a key source of safety signals, the European Medicines Agency has undertaken this work to assess the effects of this requirement on the characteristics of the reports submitted to EudraVigilance and on the detection of adverse drug reactions through routine analyses of the database.

METHODS

Changes in the numbers of serious and non-serious reports transmitted to EudraVigilance were examined over the period during which the legislation was implemented. The numbers and nature of potential safety signals emerging from established statistical algorithms used at the European Medicines Agency applied either to only the serious reports or to all reports in EudraVigilance were compared.

RESULTS

Up to November 2017, less than 25% of European Economic Area reports in EudraVigilance were classified as non-serious, since than this figure was slightly above 60%. This change accompanied an increase in the total number of reports received. Addition of non-serious reports to the signal detection process resulted in a small overall increase in signals of disproportionate reporting with some new signals of disproportionate reporting appearing and some existing signals of disproportionate reporting disappearing; the sensitivity of the signal detection system was slightly increased and the proportion of signals of disproportionate reporting that corresponded to known adverse drug reactions (a measure of efficiency) was unchanged.

CONCLUSIONS

The change in legislation has led to a small increase in sensitivity, without affecting the efficiency of the routine statistical measures used. The number of non-serious reports as a proportion of reports in EudraVigilance is likely to increase over time and further monitoring of the impact on signal detection is required. Further work is also required on the qualitative impact of non-serious reports on the nature of signals detected and on their evaluation.

Safety Profile of Ibrutinib: An Analysis of the WHO Pharmacovigilance Database

As ibrutinib has become a standard of care in B-cell malignancies in monotherapy or in combination with other agents, definition of its safety profile appears essential. The aim of this study was to further characterize the safety profile of ibrutinib through the identification of potential safety signals in a large-scale pharmacovigilance database. All serious individual case safety reports (ICSRs) in patients aged ≥18 years involving ibrutinib suspected in the occurrence of serious adverse drug reactions or drug interacting from November 13th, 2013 to December 31st, 2020 were extracted from VigiBase, the World Health Organization global safety database. Disproportionality reporting was assessed using the information component (IC) and the proportional reporting ratio (PRR), with all other anticancer drugs used as the reference group. To mitigate the confounding of age, two subgroups were considered: patients aged<75 years and ≥75 years. A signal of disproportionate reporting (SDR) was defined if both IC and PRR were significant. A total of 16,196 ICSRs were included. The median age of patients was 72.9 years, 42.6% of ICSRs concerned patients aged ≥75 years, and 64.2% male patients. More than half (56.2%) of ICSRs resulted in hospitalization or prolonged hospitalization. Among 713 SDRs, 36 potential safety signals emerged in ibrutinib-treated patients, mainly ischemic heart diseases, pericarditis, uveitis, retinal disorders and fractures. All potential safety signals having arisen in this analysis may support patient care and monitoring of ongoing clinical trials. However, owing to the mandatory limitations of this study, our results need further confirmation using population-based studies.

Reporting of Acute Inflammatory Neuropathies with COVID-19 Vaccines: Subgroup Disproportionality Analyses in VigiBase

Since marketing authorization, cases of neuralgic amyotrophy (NA), facial paralysis/Bell’s palsy (FP/BP), and Guillain-Barré syndrome (GBS) were reported with COVID-19 vaccines of different technologies. This study aimed to assess whether NA, FP/BP, and GBS were more frequently reported in VigiBase with COVID-19 vaccines (of any technologies) than with other viral vaccines, over the full database and across potential risk groups by sex and age. The reporting odds ratio (ROR) with 95% confidence interval (95% CI) was used as the measure of disproportionality and subgroup disproportionality analyses were performed by sex and age. Out of 808,906 safety reports with COVID-19 vaccines, 57 (0.01%) reported NA, 3320 (0.4%) FP/BP, and 632 (0.1%) GBS. There were not signals of disproportionate reporting for NA and GBS with COVID-19 vaccines against other viral vaccines. FP/BP was disproportionately more frequently reported with COVID-19 vaccines than with other viral vaccines over the full database (ROR 1.12, 95%CI 1.07–1.17), in males (ROR 1.65, 95%CI 1.54–1.78) and in age subgroups 65–74 years (ROR 1.21, 95%CI 1.05–1.39) and ≥75 years (ROR 1.84, 95%CI 1.52–2.22). Albeit not proving causation, these findings might support clinicians in decision-making for patients potentially at risk for developing an acute inflammatory neuropathy with COVID-19 vaccines.

Risk Factor Considerations in Statistical Signal Detection: Using Subgroup Disproportionality to Uncover Risk Groups for Adverse Drug Reactions in VigiBase

Introduction

In the treatment of the individual patient, a vision is to achieve the best possible balance between benefit and harm. Such tailored therapy relies upon the identification and characterisation of risk factors for adverse drug reactions. Information relevant to risk factor considerations can be captured in adverse event reports and could be utilised in statistical signal detection.

Objective

The aim of this study was to explore whether statistical screening of a broad range of risk factors within a global database of adverse event reports could uncover signals of risk groups for adverse drug reactions.

Methods

Subgroup disproportionality analysis was applied to 15.4 million reports entered in VigiBase, the World Health Organization (WHO) global database of individual case safety reports, up to August 2017. Disproportionality analyses for drug–adverse event pairs were performed (1) in the full database and (2) across a range of subgroups defined by the following covariates: patient age, sex, body mass index, pregnancy, underlying condition, reporting country, and geographical region. Drug–adverse event pairs disproportionately over-reported in such subgroups, but not in the full database, and with a substantial difference between the two observed-to-expected ratios, were highlighted as statistical signals. These were further prioritised, through filtering and sorting, for clinical assessment, whereafter clinically relevant signals were communicated to the pharmacovigilance community and the public.

Results

Assessments were performed for 354 prioritised statistical signals, resulting in seven communicated signals describing previously unrecognised potential risk groups related to age (elderly), sex (male and female), body mass index (underweight and obese), and geographical region (Asia), all except one for already established adverse drug reactions. Important aspects considered in the assessments included an evaluation of the disproportionate over-reporting in the subgroup by reviewing alternative explanations and reporting patterns for similar drugs/adverse events/subgroups, and a search for plausible mechanisms to support the risk hypothesis.

Conclusions

This study reveals that it is possible to uncover signals of risk groups for adverse drug reactions through incorporation of broad risk factor screening into statistical signal detection in a global database of adverse event reports. Our findings suggest the potential to use such statistical methodologies for risk characterisation in subpopulations of concern.

Electronic supplementary material

The online version of this article (10.1007/s40264-020-00957-w) contains supplementary material, which is available to authorized users.

A meta-epidemiological study found lack of transparency and poor reporting of disproportionality analyses for signal detection in pharmacovigilance databases

OBJECTIVE

To review and appraise methods and reporting characteristics of pharmacovigilance disproportionality analyses.

STUDY DESIGN AND SETTING

We randomly selected 100 disproportionality analyses indexed in Medline found during a systematic literature search. We then extracted and synthetized methodological and reporting characteristics using 7 key items: 1) title transparency; 2) protocol pre-registration; 3) date of data extraction and analysis; 4) outcome, population, exposure and comparator definitions; 5) adjustment and stratification of results; 6) method and threshold for signal detection; 7) secondary and sensitivity analyses.

RESULTS

We found that methods used to generate disproportionality signals were extremely heterogeneous; there were nearly as many unique analyses as studies. The authors used various populations, methods, signal detection thresholds, adjustment or stratification variables, generally without justification for their choice or pre-specification in protocols. Moreover, 78% of studies failed to report methods for case, adverse drug reactions or comparator selection and 32 studies did not define the threshold for signal generation.

CONCLUSION

Our survey raises major concerns regarding all aspects of disproportionality analyses that could lead to misleading results and generate unjustified alarms. We advocate for a strong and transparent rationale for variable selection, choice of population and comparators pre-specified in a protocol and assessed by sensitivity analyses.

Disproportionality Analysis for Pharmacovigilance Signal Detection in Small Databases or Subsets: Recommendations for Limiting False-Positive Associations

Introduction

Uncovering safety signals through the collection and assessment of individual case reports remains a core pharmacovigilance activity. Despite the widespread use of disproportionality analysis in signal detection, recommendations are lacking on the minimum size of databases or subsets of databases required to yield robust results.

Objective

This study aims to investigate the relationship between database size and robustness of disproportionality analysis, with regards to limiting spurious associations.

Methods

Three types of subsets were created from the global database VigiBase: random subsets (500 replicates each of 11 fixed subset sizes between 250 and 100,000 reports), country-specific subsets (all 131 countries available in the original VigiBase extract) and subsets based on the Anatomical Therapeutic Chemical classification. For each subset, a spuriousness rate was computed as the ratio between the number of drug–event combinations highlighted by disproportionality analysis in a permuted version of the subset and the corresponding number in the original subset. In the permuted data, all true reporting associations between drugs and adverse events were broken. Subsets with fewer than five original associations were excluded. Additionally, the set of disproportionately over-reported drug–event combinations in three specific countries at three different time points were clinically assessed for labelledness. These time points corresponded to database sizes of less than 10,000, 5000 and 1000 reports, respectively. All disproportionality analysis was based on the Information Component (IC), implemented as IC₀₂₅ > 0.

Results

Spuriousness rates were below 0.15 for all 110 included countries regardless of subset size, with only seven countries (6%) exceeding the empirical threshold of 0.10 observed for large subsets. All 21 excluded countries had < 500 reports. For random subsets containing 3000–5000 or more reports, the higher end of observed spuriousness rates was close to 0.10. In the clinical assessment, the proportion of labelled or otherwise known drug–event combinations was very high (87–100%) across all countries and time points studied.

Conclusions

To mitigate the risk of highlighting spurious associations with disproportionality analysis, a minimum size of 500 reports is recommended for national databases. For databases or subsets that are not country-specific, our recommendation is 5000 reports. This study does not consider sensitivity, which is expected to be poor in smaller databases.

Leveraging the Variability of Pharmacovigilance Disproportionality Analyses to Improve Signal Detection Performances

Background: A plethora of methods and models of disproportionality analyses for safety surveillance have been developed to date without consensus nor a gold standard, leading to methodological heterogeneity and substantial variability in results. We hypothesized that this variability is inversely correlated to the robustness of a signal of disproportionate reporting (SDR) and could be used to improve signal detection performances.

Methods: We used a validated reference set containing 399 true and false drug-event pairs and performed, with a frequentist and a Bayesian disproportionality method, seven types of analyses (model) for which the results were very unlikely to be related to actual differences in absolute risks of ADR. We calculated sensitivity, specificity and plotted ROC curves for each model. We then evaluated the predictive capacities of all models and assessed the impact of combining such models with the number of positive SDR for a given drug-event pair through binomial regression models.

Results: We found considerable variability in disproportionality analysis results, both positive and negative SDR could be generated for 60% of all drug-event pairs depending on the model used whatever their truthfulness. Furthermore, using the number of positive SDR for a given drug-event pair largely improved the signal detection performances of all models.

Conclusion: We therefore advocate for the pre-registration of protocols and the presentation of a set of secondary and sensitivity analyses instead of a unique result to avoid selective outcome reporting and because variability in the results may reflect the likelihood of a signal being a true adverse drug reaction.

Adverse Event Reporting with Immune Checkpoint Inhibitors in Older Patients: Age Subgroup Disproportionality Analysis in VigiBase

Simple Summary

Due to the changes that occur with aging in the immune system, older patients represent a subpopulation of concern for immune checkpoint inhibitor toxicity. This pharmacovigilance study aimed to assess whether older patient age (65 years and older) was a risk factor for increased reporting of adverse drug reactions with immune checkpoint inhibitors as compared to other antineoplastic drugs in VigiBase, the World Health Organization global database of spontaneous reporting. Disproportionality analysis by age subgroups (<18 years, 18–64 years, 65–74 years, 75–84 years and ≥85 years) did not highlight older patient age as risk factor for increased reporting of any specific toxicity with immune checkpoint inhibitors as compared to other antineoplastic drugs. A signal of disproportionate reporting emerged for eye disorders with immune checkpoint inhibitors in patients aged 18–64 years, which deserves further investigation aimed at elucidating risk factors and defining management strategies.

Abstract

Older patients represent a subpopulation of concern for immune checkpoint inhibitor (ICI) toxicity because of changes in the aging immune system and the potentially relevant clinical implications for their quality of life. Current evidence on ICI safety in older patients is conflicting. This study aimed to assess whether older patient age was a risk factor for increased reporting with ICIs as compared to other antineoplastic drugs in VigiBase, the World Health Organization database of suspected adverse drug reactions. Disproportionality analyses computing the reporting odds ratios (RORs) were performed by age subgroups (<18 years, 18–64 years, 65–74 years, 75–84 years and ≥85 years). There were not signals of disproportionate reporting with ICIs specifically detected in older patient age subgroups (≥65 years), which were not present in the disproportionality analysis over the entire dataset. A signal of disproportionate reporting with ICIs emerged for eye disorders only in the age subgroup 18–64 years (ROR 1.13, 95% confidence interval 1.05–1.23). These findings showed that adverse event reporting with ICIs in older patients was comparable to that in the overall patient cohort and prompt for the further investigation of eye disorders with ICIs to elucidating risk factors and defining management strategies.

A New Drug-Drug Interaction Between Hydroxychloroquine and Metformin? A Signal Detection Study

Introduction

Hydroxychloroquine was recently promoted in patients infected with COVID-19 infection. A recent experimental study has suggested an increased toxicity of hydroxychloroquine in association with metformin in mice.

Objective

The present study was undertaken to investigate the reality of this putative drug–drug interaction between hydroxychloroquine and metformin using pharmacovigilance data.

Methods

Using VigiBase^®, the WHO pharmacovigilance database, we performed a disproportionality analysis (case/non-case study). Cases were reports of fatal outcomes with the drugs of interest and non-cases were all other reports for these drugs registered between 1 January 2000 and 31 December 2019. Data with hydroxychloroquine (or metformin) alone were compared with the association hydroxychloroquine + metformin. Results are reported as ROR (reporting odds ratio) with their 95% confidence interval.

Results

Of the 10,771 Individual Case Safety Reports (ICSR) involving hydroxychloroquine, 52 were recorded as ‘fatal outcomes’. In comparison with hydroxychloroquine alone, hydroxychloroquine + metformin was associated with an ROR value of 57.7 (23.9–139.3). In comparison with metformin alone, hydroxychloroquine + metformin was associated with an ROR value of 6.0 (2.6–13.8).

Conclusion

Our study identified a signal for the association hydroxychloroquine + metformin that appears to be more at risk of fatal outcomes (particularly by completed suicides) than one of the two drugs when given alone.

Clustering-Based Hybrid Approach for Identifying Quantitative Multidimensional Associations Between Patient Attributes, Drugs and Adverse Drug Reactions

The activity of post-marketing surveillance results in a collection of large amount of data. The analysis of data is very useful for raising early warnings on possible adverse reactions of drugs. Association rule mining techniques have been heavily explored by the research community for identifying binary association between drugs and their adverse effects. But these techniques perform poorly and miss out several interesting associations when it comes to analysis of multidimensional data which may include multiple patient attributes, drugs and adverse drug reactions. In the present work, a clustering-based hybrid approach has been presented for finding quantitative multidimensional association from the large amount of data. Firstly, it employs clustering technique for segmentation of data into semantically coherent clusters. Furthermore, disproportionality method called proportional reporting ratio is applied on clustered data for generating statistically strong associations. The performance of the proposed methodology has been examined on the data taken from the U.S. Food and Drug Administration Adverse Event Reporting System database corresponding to Aspirin and nine other drugs which are prescribed along with Aspirin. The experimental results show that the proposed approach discovered a number of association rules which are very comprehensive and informative regarding relationship of patient traits and drugs with adverse drug reactions. On comparing experimental results with LPMiner, it is observed that the quantitative association rules discovered by LPMiner are just 8.3% of what have been discovered by the proposed methodology.

An evaluation of statistical approaches to postmarketing surveillance

Safety of medical products presents a serious concern worldwide. Surveillance systems of postmarket medical products have been established for continual monitoring of adverse events (AEs) in many countries, and the proliferation of electronic health record systems further facilitates continual monitoring for AEs. We review existing statistical methods for signal detection that are mostly in use in postmarketing safety surveillance of spontaneously reported AEs and we study their performance characteristics by simulation. We compare those with the likelihood ratio test (LRT) method (appropriately modified for use in pharmacovigilance) and use three different methods to generate data (AE based, drug based, and a modification of the method of Ahmed et al). Performance metrics include type I error, power, sensitivity, and false discovery rate, among others. The results show superior performance of the LRT method in almost all simulation experiments. An application to the FDA Adverse Event Reporting System database is illustrated using rhabdomyolysis-related preferred terms reported to FDA during the third-quarter of 2014 to the first-quarter of 2017 for statin drugs. We present a critical discussion and recommendations for use of these methods.

Pooling Different Safety Data Sources: Impact of Combining Solicited and Spontaneous Reports on Signal Detection In Pharmacovigilance

INTRODUCTION

The volume of adverse events (AEs) collected, analysed, and reported has been increasing at a rapid rate for over the past 10 years, largely due to the growth of solicited programmes. The proportion of various forms of solicited case data has evolved over time, with the main relative volume increase coming from Patient Support Programmes. In this study, we sought to examine the impact of the pooling of AE report data from solicited sources with data from spontaneous sources to safety signal detection using disproportionality analysis methods.

METHODS

Two conditions were explored in which disproportionality scores from hypothetical drugs were evaluated in a simulated safety database. The first condition held occurrence of events constant and varied solicited case volume, while the second condition varied both proportion of occurrence of events and solicited case volume.

RESULTS

In the first setting, where all AE terms have the same probability to occur with any drug, increasing volumes of solicited cases while keeping occurrence of events constant leads to reduced variability in disproportionality scores, consequently reducing or eliminating identified signals of disproportionate reporting. In the second setting, varying both case volume and reporting rates can mask true safety signals and falsely identify signals where there are none.

CONCLUSIONS

This analysis of simulated data suggests that pooling AE data from solicited sources with spontaneous case data may impact the results of disproportionality analyses, masking true safety signals and identifying false positives. Therefore, increased volumes of safety data do not necessarily correlate with improved safety signal detection.

Safety of Biologics Approved for the Treatment of Rheumatoid Arthritis and Other Autoimmune Diseases: A Disproportionality Analysis from the FDA Adverse Event Reporting System (FAERS)

INTRODUCTION

The molecular and pharmacological complexity of biologic disease-modifying antirheumatic drugs used for the management of rheumatoid arthritis (RA) favors the occurrence of adverse drug reactions (ADRs), which should be constantly monitored in post-marketing safety studies.

OBJECTIVE

The aim of this study was to identify signals of disproportionate reporting (SDR) of clinical relevance related to the use of biologic drugs approved for RA and other autoimmune diseases.

METHODS

All suspected ADRs registered in the FDA Adverse Event Reporting System between January 2003 and June 2016 were collected. The reporting odds ratio was used as a measure of disproportionality to identify possible SDRs related to biologics. Those involving important medical events and designated medical events (DME) were prioritized.

RESULTS

In total, 2602 SDRs were prioritized. The most commonly reported were 'Infections and infestations' (32.2%) and 'Neoplasms benign, malignant, and unspecified' (20.4%), and were mainly related to use of infliximab (25.3%, p < 0.001, and 28.8%, p = 0.002, respectively). Sixty-three signals involving DMEs were identified, most of which were related to rituximab (n = 27), and were mainly due to 'blood disorders'. Amongst the DMEs detected for more than one biologic, 'intestinal perforation' and 'pulmonary fibrosis' were related to most of them.

CONCLUSIONS

The results of this study highlight possible safety issues associated with biologics, whose relationship should be more thoroughly investigated. Our results contribute to future research on the identification of clinically relevant risks associated with these drugs, and may help contribute to their rational and safe use.

Paediatric safety signals identified in VigiBase: Methods and results from Uppsala Monitoring Centre

Purpose

The purpose of this study is to uncover previously unrecognised risks of medicines in paediatric pharmacovigilance reports and thereby advance a safer use of medicines in paediatrics.

Methods

Individual case safety reports (ICSRs) with ages less than 18 years were retrieved from VigiBase, the World Health Organization (WHO) global database of ICSRs, in September 2014. The reports were grouped according to the following age spans: 0 to 27 days; 28 days to 23 months; 2 to 11 years; and 12 to 17 years. vigiRank, a data‐driven predictive model for emerging safety signals, was used to prioritise the list of drug events by age groups. The list was manually assessed, and potential signals were identified to undergo in‐depth assessment to determine whether a signal should be communicated.

Results

A total of 472 drug‐event pairs by paediatric age groups were the subject of an initial manual assessment. Twenty‐seven drug events from the two older age groups were classified as potential signals. An in‐depth assessment resulted in eight signals, of which one concerned harm in connection with off‐label use of dextromethorphan and another with accidental overdose of olanzapine by young children, and the remaining signals referred to potentially new causal associations for atomoxetine (two signals), temozolamide, deferasirox, levetiracetam, and desloratadine that could be relevant also for adults.

Conclusions

Clinically relevant signals were uncovered in VigiBase by using vigiRank applied to paediatric age groups. Further refinement of the methodology is needed to identify signals in reports with ages under 2 years and to capture signals specific to the paediatric population as a risk group.

Drug-induced Stevens-Johnson syndrome and toxic epidermal necrolysis in vietnamese spontaneous adverse drug reaction database: A subgroup approach to disproportionality analysis

WHAT IS KNOWN AND OBJECTIVE

Despite the numerous studies investigating drug-induced Stevens-Johnson syndrome/toxic epidermal necrolysis (SJS/TEN), the understanding and quantitative data in developing countries remain limited. The study aimed to describe and quantify the drug-related risk of SJS/TEN in a resource-limited context using the Vietnamese spontaneous reporting database (VSRD) of adverse drug reactions.

METHODS

Spontaneous reports relating to medium- and late-onset severe cutaneous adverse reactions (MLOSCAR) and SJS/TEN recorded in the VSRD from 2010 to 2015 were retrospectively analysed. The demographic characteristics and drug information were described and compared between SJS/TEN and other MLOSCAR reports. The drug-induced SJS/TEN signals were estimated using subgrouped disproportionality analysis with calculation of the reporting odds ratio (ROR) and the respective 95% confidence interval (CI).

RESULTS

The VSRD received 2,849 MLOSCAR reports, 136 of which focus on SJS/TEN over a 6-year period. About 60% of SJS/TEN patients were male, and the majority of them were adults (mean age 42.5 ± 22.9). Up to 91.8% of drugs induced SJS/TEN within 1-28 days, and 45% SJS/TEN cases were evaluated as life-threatening. Positive signals were generated with carbamazepine (n = 25, ROR [95% CI] = 11.99 [7.07-19.92]), allopurinol (n = 15, ROR [95% CI] = 4.2 [2.20-7.59]), traditional/herbal medicines (n = 7, ROR [95% CI] = 2.76 [1.12-5.86]), colchicine (n = 4, ROR [95% CI] = 6.22 [1.69-18.72]), valproic acid (n = 3, ROR [95% CI] = 8.71 [1.89-30.19]) and meloxicam (n = 3, ROR [95% CI] = 7.09 [1.55-24.29]), which are well known for SJS/TEN. Cefixime (n = 5, ROR [95% CI] = 3.34 [1.13-8.00]) and paracetamol (n = 22, ROR [95% CI] = 5.23 [3.10-8.49]) also generated positive signals despite their popularity in Vietnam.

WHAT IS NEW AND CONCLUSION

This first Vietnamese population-based study has highlighted original characteristics and signals of drug-induced SJS/TEN, which are relatively consistent with other worldwide data and typical for a developing country.

Pharmacovigilance of sodium-glucose co-transporter-2 inhibitors: What a clinician should know on disproportionality analysis of spontaneous reporting systems

Sodium-glucose co-transporter-2 inhibitors (SGLT2-Is) have consistently demonstrated a clinically significant reduction of cardiovascular mortality. However, their safety in clinical practice is still incompletely characterized, and post-marketing monitoring is required considering the expected increase in clinical use. Different analyses of international spontaneous reporting systems, known as disproportionality analyses (DAs), have highlighted the occurrence of ketoacidosis, amputations, acute renal failure and skin toxicity. In this viewpoint, we critically appraise these pharmacovigilance data on SGLT2-Is, with the aim of supporting clinicians in proper interpretation of these studies, and discussing their risk-benefit profile. To this aim, we offer a broad perspective on basic technical aspects subtending DAs of spontaneous reporting databases (describing peculiarities of the Food and Drug Administration Adverse Event Reporting System), their common and evolving uses, key pitfalls in presenting study results (in terms of "risk" or "association") and relevant strategies to account for major confounders. This will also facilitate reviewers and editors in proper evaluation of DAs, and prompt pharmacovigilance experts in converging towards a set of minimum requirements in standardization of design, performance and reporting of DAs. A consensus on quality assessment of DAs will finally establish their transferability to clinical practice. It is anticipated that DAs cannot be used per se as a standalone approach to assess a drug-related risk and cannot replace clinical judgment in the individual patient.

Good Signal Detection Practices: Evidence from IMI PROTECT

Over a period of 5 years, the Innovative Medicines Initiative PROTECT (Pharmacoepidemiological Research on Outcomes of Therapeutics by a European ConsorTium) project has addressed key research questions relevant to the science of safety signal detection. The results of studies conducted into quantitative signal detection in spontaneous reporting, clinical trial and electronic health records databases are summarised and 39 recommendations have been formulated, many based on comparative analyses across a range of databases (e.g. regulatory, pharmaceutical company). The recommendations point to pragmatic steps that those working in the pharmacovigilance community can take to improve signal detection practices, whether in a national or international agency or in a pharmaceutical company setting. PROTECT has also pointed to areas of potentially fruitful future research and some areas where further effort is likely to yield less.