Leveraging the Variability of Pharmacovigilance Disproportionality Analyses to Improve Signal Detection Performances

The evolving role of disproportionality analysis in pharmacovigilance

INTRODUCTION

From 2009 to 2015, the IMI PROTECT conducted rigorous studies addressing questions about optimal implementation and significance of disproportionality analyses, leading to the development of Good Signal Detection Practices. The ensuing period witnessed the independent exploration of research paths proposed by IMI PROTECT, accumulating valuable experience and insights that have yet to be seamlessly integrated.

AREAS COVERED

This state-of-the-art review integrates IMI PROTECT recommendations with recent acquisitions and evolving challenges. It deals with defining the object of study, disproportionality methods, subgrouping, masking, drug-drug interaction, duplication, expectedness, the debated use of disproportionality results as risk measures, integration with other types of data.

EXPERT OPINION

Despite the ongoing skepticism regarding the usefulness of disproportionality analyses and individual case safety reports, their ability to timely detect safety signals regarding rare and unpredictable adverse reactions remains unparalleled. Moreover, recent exploration into their potential for characterizing safety signals revealed valuable insights concerning potential risk factors and the patient's perspective. To fully realize their potential beyond hypothesis generation and achieve a comprehensive evidence synthesis with other kinds of data and studies, each with their unique limitations and contributions, we need to investigate methods for more transparently communicating disproportionality results and mapping and addressing pharmacovigilance biases.

Integrase inhibitor drugs during pregnancy and congenital anomalies: A case/non-case study from the global pharmacovigilance database VigiBase®

In 2018, a significant neural tube defects (NTD) signal was reported after pre-conceptional exposure to dolutegravir, but was not confirmed in further analysis. Since 2019, dolutegravir-based regimen, an integrase inhibitor (INI), is recommended by WHO as the most-effective first-line therapy in all patients living with HIV. To explore the potential INI-related teratogenic effect, we searched disproportionate signals between exposure to INI-class drugs and congenital anomalies, compared to non-INI drugs, using the international pharmacovigilance database, VigiBase®. We selected all the reports registered in VigiBase® between 01/01/2007 and 30/03/2021 on any antiretroviral drug-related fetal or neonatal adverse drug reactions, declared either in children (<2 years) exposed in utero or in pregnant women (12-50 years). A case/non-case study was conducted to detected signals between congenital anomalies and prenatal exposure to any INI-class drug, compared to non-INI drugs, by estimating adjusted reporting odds ratios (aROR) with 95% confidence intervals (95%CI). We identified 2521 unique reports, among which 664 (26.3%) were related to INI-class use. Overall, 520 congenital anomalies were cited from 327 unique reports, of whom 31.0% were INI-related. Compared to non-INI drugs, no significant disproportionate reporting signal between prenatal exposure to INI-class drugs and congenital anomalies was found (aROR 1.13; 95% CI:0.85-1.51). However, specific significant signals were reported for raltegravir/elvitegravir/dolutegravir drug exposure and urinary malformations (aROR 2.43; 95%CI:1.08-5.43), digestive malformations (aROR 3.09; 95%CI:1.22-7.84), and NTDs (aROR 3.02; 95%CI:1.09-8.37). Although specific congenital anomalies signals associated with raltegravir/elvitegravir/dolutegravir exposure were notified, causal relationship needs to be further investigated in prospective studies.

Comparators in Pharmacovigilance: A Quasi-Quantification Bias Analysis

BACKGROUND AND OBJECTIVE

It is unclear which comparator is the most appropriate for bias reduction in disproportionality analyses based on spontaneous reports. We conducted a quasi-quantitative bias analysis using two well-studied drug-event combinations to assess how different comparators influence the directionality of bias in pharmacovigilance.

METHODS

We used the US Food and Drug Administration Adverse Event Reporting System focusing on two drug-event combinations with a propensity for stimulated reporting: rivaroxaban and hepatotoxicity, and canagliflozin and acute kidney injury. We assessed the directionality of three disproportionality analysis estimates (reporting odds ratio, proportional reporting ratio, information component) using one unrestricted comparator (full data) and two restricted comparators (active comparator, active comparator with class exclusion). Analyses were conducted within two calendar time periods, defined based on external events (approval of direct oral anticoagulants, Food and Drug Administration safety warning on acute kidney injury with sodium-glucose cotransporter 2 inhibitors) hypothesized to alter reporting rates.

RESULTS

There were no false-positive signals for rivaroxaban and hepatotoxicity irrespective of the comparator. Restricting to the initial post-approval period led to false-positive signals, with restricted comparators performing worse. There were false-positive signals for canagliflozin and acute kidney injury, with restricted comparators performing better. Restricting to the period before the Food and Drug Administration warning weakened the false-positive signal for canagliflozin and acute kidney injury across comparators.

CONCLUSIONS

We could not identify a consistent and predictable pattern to the directionality of disproportionality analysis estimates with specific comparators. Calendar time-based restrictions anchored on relevant external events had a considerable impact.

The REporting of A Disproportionality Analysis for DrUg Safety Signal Detection Using Individual Case Safety Reports in PharmacoVigilance (READUS-PV): Explanation and Elaboration

In pharmacovigilance, disproportionality analyses based on individual case safety reports are widely used to detect safety signals. Unfortunately, publishing disproportionality analyses lacks specific guidelines, often leading to incomplete and ambiguous reporting, and carries the risk of incorrect conclusions when data are not placed in the correct context. The REporting of A Disproportionality analysis for drUg Safety signal detection using individual case safety reports in PharmacoVigilance (READUS-PV) statement was developed to address this issue by promoting transparent and comprehensive reporting of disproportionality studies. While the statement paper explains in greater detail the procedure followed to develop these guidelines, with this explanation paper we present the 14 items retained for READUS-PV guidelines, together with an in-depth explanation of their rationale and bullet points to illustrate their practical implementation. Our primary objective is to foster the adoption of the READUS-PV guidelines among authors, editors, peer reviewers, and readers of disproportionality analyses. Enhancing transparency, completeness, and accuracy of reporting, as well as proper interpretation of their results, READUS-PV guidelines will ultimately facilitate evidence-based decision making in pharmacovigilance.

Adverse drug reaction signal detection methods in spontaneous reporting system: A systematic review

BACKGROUND

A series of signal detection methods have been developed to detect adverse drug reaction (ADR) signals in spontaneous reporting system. However, different signal detection methods yield quite different signal detection results, and we do not know which method has the best detection performance. How to choose the most suitable signal detection method is an urgent problem to be solved. In this study, we systematically reviewed the characteristics and application scopes of current signal detection methods, with the goal of providing references for the optimization selection of signal detection methods in spontaneous reporting system.

METHODS

We searched six databases from inception to January 2023. The search strategy targeted literatures regarding signal detection methods in spontaneous reporting system. We used thematic analysis approach to summarize the advantages, disadvantages, and application scope of each signal detection method.

RESULTS

A total of 93 literatures were included, including 27 reviews and 66 methodological studies. Moreover, 31 signal detection methods were identified in these literatures. Each signal detection method has its inherent advantages and disadvantages, resulting in different application scopes of these methods.

CONCLUSION

Our systematic review finds that there are variabilities in the advantages, disadvantages, and application scopes of different signal detection methods. This finding indicates that the most suitable signal detection method varies across different drug safety scenarios. Moreover, when selecting signal detection method in a particular drug safety scenario, the following factors need to be considered: purpose of research, database size, drug characteristics, adverse event characteristics, and characteristics of the relations between drugs and adverse events.

Drug-induced tumoral disease: A global pharmacovigilance database analysis

INTRODUCTION

Cancer remains a worldwide threat, having caused almost 10 million deaths in 2020. The American Cancer Society has identified both known and probable carcinogens, including commonly used drugs. The aim of this study is to describe the drugs most frequently reported in the occurrence of cancer.

METHODS

Among all individual case safety reports (ICSRs) in the global pharmacovigilance database VigiBase, we searched for the 50 most reported drugs with an adverse drug reaction term belonging to the query "Malignant or unspecified tumors" until June 30, 2023. Then, we extracted the disproportionality measurement data, information component (IC), and reporting odds ratio (ROR) in order to assess a disproportionality signal.

RESULTS

Among all ICSRs in VigiBase, 871,925 contained an ADR belonging to the SMQ "Malignant or unspecified tumors". Ranitidine was the drug with the most reported ADRs related to cancer (n=106,484), followed by lenalidomide (n=13,466), and etanercept (n=8014). The drugs with the highest IC were ranitidine (IC=5.2, 95% confidence interval [95% CI]=5.2-5.2), pioglitazone (1353 ICSRs, IC=4.2, 95% CI=4.2-4.2), and regorafenib (1272 ICSRs, IC=2.8, 95% CI=2.8-2.8).

DISCUSSION

Our results show that the main pharmacological mechanisms are associated with ranitidine (link with levels of N-nitrosodimethylamine in ranitidine-based drugs), gene-activating drugs (pioglitazone: link with agonist effects on PPAR-γ gene activation), various pharmacological families with immunosuppressive effects (protein kinase inhibitors, immunomodulators, azathioprine, etc.), certain types of protein kinase inhibitors whose oncogenic mechanisms remain unclear (regorafenib, sorafenib, imatinib, ibrutinib, etc.), and hormone antagonists (tamoxifen, letrozole). Special monitoring of patients exposed to these drugs may be required. Further studies are needed to assess the risk with certain drugs in this ranking.

Drug clustering to anticipate new aspects of drug safety profile: Application to gabapentinoids and other voltage-gated calcium channel ligand drugs

AIMS

Gabapentin and pregabalin bind to α2-δ subunit of voltage-gated calcium channels (Cav ). Other drugs targeting Cav include cardiovascular calcium channel blockers (CCBs) and anticonvulsants (levetiracetam, ethosuximide and zonisamide). In addition to pharmacodynamics, the safety profile of gabapentinoids seems to overlap with the one of cardiovascular CCBs (oedema) and Cav -blocking anticonvulsants (suicide and ataxia). The objective of this study was to cluster the safety profile of different Cav -ligand drugs by focusing on whether gabapentinoids present a distinct adverse drug reaction (ADR) signature from cardiovascular CCBs and anticonvulsants.

METHODS

We extracted all ADRs with at least one significant disproportionate reporting (reporting odds ratio) related to gabapentinoids, CCBs or anticonvulsants in VigiBase. After principal component analysis preprocessing, a hierarchical ascendent classification was performed to cluster gabapentinoids and other Cav -ligand drugs that share a similar ADR signature. The robustness of the results was determined through four sensitivity analyses, varying on the dataset or the clustering method.

RESULTS

A total of 16 drugs and 65 ADRs were included. Gabapentinoids were in Cluster #1, which included eight other drugs (isradipine, nicardipine, lacidipine, lercanidipine, ethosuximide, levetiracetam, zonisamide and nimodipine). Cluster #2 contained two drugs (diltiazem and verapamil) and Cluster #3 contained four drugs (amlodipine, felodipine, nifedipine and nitrendipine). The clustering results were consistent in all sensitivity analyses.

CONCLUSIONS

The safety profile of gabapentinoids overlaps with those of some dihydropyridine CCBs and Cav -blocking anticonvulsants. These results could be used to anticipate some unidentified ADRs of gabapentinoids from information accumulated with older drugs and sharing a common molecular target and ADR signature.

Development and Application of a Data-Driven Signal Detection Method for Surveillance of Adverse Event Variability Across Manufacturing Lots of Biologics

INTRODUCTION

Postmarketing drug safety surveillance research has focused on the product-patient interaction as the primary source of variability in clinical outcomes. However, the inherent complexity of pharmaceutical manufacturing and distribution, especially of biologic drugs, also underscores the importance of risks related to variability in manufacturing and supply chain conditions that could potentially impact clinical outcomes. We propose a data-driven signal detection method called HMMScan to monitor for manufacturing lot-dependent changes in adverse event (AE) rates, and herein apply it to a biologic drug.

METHODS

The HMMScan method chooses the best-fitting candidate from a family of probabilistic Hidden Markov Models to detect temporal correlations in per lot AE rates that could signal clinically relevant variability in manufacturing and supply chain conditions. Additionally, HMMScan indicates the particular lots most likely to be related to risky states of the manufacturing or supply chain condition. The HMMScan method was validated on extensive simulated data and applied to three actual lot sequences of a major biologic drug by combining lot metadata from the manufacturer with AE reports from the US FDA Adverse Event Reporting System (FAERS).

RESULTS

Extensive method validation on simulated data indicated that HMMScan is able to correctly detect the presence or absence of variable manufacturing and supply chain conditions for contiguous sequences of 100 lots or more when changes in these conditions have a meaningful impact on AE rates. Applying the HMMScan method to FAERS data, two of the three actual lot sequences examined exhibited evidence of potential manufacturing or supply chain-related variability.

CONCLUSIONS

HMMScan could be utilized by both manufacturers and regulators to automate lot variability monitoring and inform targeted root-cause analysis. Broad application of HMMScan would rely on a well-developed data input pipeline. The proposed method is implemented in an open-source GitHub repository.

Mapping Strategies to Assess and Increase the Validity of Published Disproportionality Signals: A Meta-Research Study

BACKGROUND AND AIM

Disproportionality analysis is traditionally used in spontaneous reporting systems to generate working hypotheses about potential adverse drug reactions: the so-called disproportionality signals. We aim to map the methods used by researchers to assess and increase the validity of their published disproportionality signals.

METHODS

From a systematic literature search of published disproportionality analyses up until 1 January 2020, we randomly selected and analyzed 100 studies. We considered five domains: (1) rationale for the study, (2) design of disproportionality analyses, (3) case-by-case assessment, (4) use of complementary data sources, and (5) contextualization of the results within existing evidence.

RESULTS

Among the articles, multiple strategies were adopted to assess and enhance the results validity. The rationale, in 95 articles, was explicitly referred to the accrued evidence, mostly observational data (n = 46) and regulatory documents (n = 45). A statistical adjustment was performed in 34 studies, and specific strategies to correct for biases were implemented in 33 studies. A case-by-case assessment was complementarily performed in 35 studies, most often by investigating temporal plausibility (n = 26). Complementary data sources were used in 25 articles. In 78 articles, results were contextualized using accrued evidence from the literature and regulatory documents, the most important sources being observational (n = 45), other disproportionalities (n = 37), and case reports (n = 36).

CONCLUSIONS

This meta-research study highlighted the heterogeneity in methods and strategies used by researchers to assess the validity of disproportionality signals. Mapping these strategies is a first step towards testing their utility in different scenarios and developing guidelines for designing future disproportionality analysis.

Sensitivity and specificity in signal detection with the reporting odds ratio and the information component

PURPOSE

As measures of association between an adverse drug reaction (ADR) and exposure to a drug the reporting odds ratio (ROR) and the information component (IC) can be used. We sought to test the reliability of signal detection with these.

METHODS

We simulated ADR counts as binomially distributed random numbers for different expected ADR frequencies and theoretical reporting odds ratios (RORs). We then calculated the empirical IC and the empirical ROR and their confidence intervals. The rate of signals that was detected despite a theoretical ROR of 1 represented the false positive rate, and represented the sensitivity if the ROR was >1.

RESULTS

For expected case counts below 1 the false positive rate oscillates from 0.01 to 0.1 even though 0.025 were intended. Even beyond expected case counts of 5 oscillations can cover a range of 0.018 to 0.035. The first n oscillations with the largest amplitude are eliminated if a minimum case count of n is required. To detect an ROR of 2 with a sensitivity of 0.8, a minimum of 12 expected ADRs are required. In contrast, 2 expected ADRs suffice to detect an ROR of 4.

CONCLUSION

Summaries of measures for disproportionality should include the expected number of cases in the group of interest if a signal was detected. If no signal was detected the sensitivity for the detection of a representative ROR or the minimum ROR that could be detected with probability 0.8 should be reported.

Association between Anti-Osteoporotic Drugs and Risk of Acute Kidney Injury: A Cross-Sectional Study Using Disproportional Analysis and a Pharmacovigilance Database

The number of fractures related to osteoporosis is expected to increase. Therefore, clarifying the risk of acute kidney injury (AKI) associated with each type of anti-osteoporotic drug may avoid discontinuation of osteoporosis pharmacotherapy due to onset of AKI. This cross-sectional study using disproportional analysis and a pharmacovigilance database assessed the risk of AKI with various anti-osteoporotic drugs by analyzing data entered into the US Food and Drug Administration's Adverse Event Reporting System (FAERS) from April 2014 to March 2021 and the Medical Data Vision (MDV) database in Japan in November 2021. All anti-osteoporotic drugs were investigated, including bisphosphonates, selective estrogen receptor modulators, denosumab, romosozumab, abaloparatide, and teriparatide. In the analysis of FAERS data, disproportionality for decreasing AKI was observed for oral ibandronate [reporting odds ratios (ROR) 0.22, 95% confidence interval (CI) 0.09-0.45, P < 0.01], bazedoxifene (ROR 0.26, 95% CI 0.05-0.77, P = 0.01), and intravenous ibandronate (ROR 0.39, 95% CI 0.14-0.86, P = 0.01). In the analysis of the MDV data, the incidence of AKI was lower in patients taking intravenous ibandronate [odds ratio (OR) 0.22, 95% CI 0.06-0.89, P = 0.03], and the incidence of AKI was higher in patients taking oral alendronate (OR 2.40, 95% CI 2.08-2.77, P < 0.01). Risk of AKI may differ even among oral anti-osteoporotic drugs, and the evidence of this association should be assessed further in future drug safety studies. This article is protected by copyright. All rights reserved.

Development of a Network-Based Signal Detection Tool: The COVID-19 Adversome in the FDA Adverse Event Reporting System

Introduction: The analysis of pharmacovigilance databases is crucial for the safety profiling of new and repurposed drugs, especially in the COVID-19 era. Traditional pharmacovigilance analyses-based on disproportionality approaches-cannot usually account for the complexity of spontaneous reports often with multiple concomitant drugs and events. We propose a network-based approach on co-reported events to help assessing disproportionalities and to effectively and timely identify disease-, comorbidity- and drug-related syndromes, especially in a rapidly changing low-resources environment such as that of COVID-19. Materials and Methods: Reports on medications administered for COVID-19 were extracted from the FDA Adverse Event Reporting System quarterly data (January-September 2020) and queried for disproportionalities (Reporting Odds Ratio corrected for multiple comparisons). A network (the Adversome) was estimated considering events as nodes and conditional co-reporting as links. Communities of significantly co-reported events were identified. All data and scripts employed are available in a public repository. Results: Among the 7,082 COVID-19 reports extracted, the seven most frequently suspected drugs (remdesivir, hydroxychloroquine, azithromycin, tocilizumab, lopinavir/ritonavir, sarilumab, and ethanol) have shown disproportionalities with 54 events. Of interest, myasthenia gravis with hydroxychloroquine, and cerebrovascular vein thrombosis with azithromycin. Automatic clustering identified 13 communities, including a methanol-related neurotoxicity associated with alcohol-based hand-sanitizers and a long QT/hepatotoxicity cluster associated with azithromycin, hydroxychloroquine and lopinavir-ritonavir interactions. Conclusion: Findings from the Adversome detect plausible new signals and iatrogenic syndromes. Our network approach complements traditional pharmacovigilance analyses, and may represent a more effective signal detection technique to guide clinical recommendations by regulators and specific follow-up confirmatory studies.

A meta-epidemiological study found lack of transparency and poor reporting of disproportionality analyses for signal detection in pharmacovigilance databases

OBJECTIVE

To review and appraise methods and reporting characteristics of pharmacovigilance disproportionality analyses.

STUDY DESIGN AND SETTING

We randomly selected 100 disproportionality analyses indexed in Medline found during a systematic literature search. We then extracted and synthetized methodological and reporting characteristics using 7 key items: 1) title transparency; 2) protocol pre-registration; 3) date of data extraction and analysis; 4) outcome, population, exposure and comparator definitions; 5) adjustment and stratification of results; 6) method and threshold for signal detection; 7) secondary and sensitivity analyses.

RESULTS

We found that methods used to generate disproportionality signals were extremely heterogeneous; there were nearly as many unique analyses as studies. The authors used various populations, methods, signal detection thresholds, adjustment or stratification variables, generally without justification for their choice or pre-specification in protocols. Moreover, 78% of studies failed to report methods for case, adverse drug reactions or comparator selection and 32 studies did not define the threshold for signal generation.

CONCLUSION

Our survey raises major concerns regarding all aspects of disproportionality analyses that could lead to misleading results and generate unjustified alarms. We advocate for a strong and transparent rationale for variable selection, choice of population and comparators pre-specified in a protocol and assessed by sensitivity analyses.