Optimization of a quantitative signal detection algorithm for spontaneous reports of adverse events post immunization

Adverse drug reaction signal detection methods in spontaneous reporting system: A systematic review

BACKGROUND

A series of signal detection methods have been developed to detect adverse drug reaction (ADR) signals in spontaneous reporting system. However, different signal detection methods yield quite different signal detection results, and we do not know which method has the best detection performance. How to choose the most suitable signal detection method is an urgent problem to be solved. In this study, we systematically reviewed the characteristics and application scopes of current signal detection methods, with the goal of providing references for the optimization selection of signal detection methods in spontaneous reporting system.

METHODS

We searched six databases from inception to January 2023. The search strategy targeted literatures regarding signal detection methods in spontaneous reporting system. We used thematic analysis approach to summarize the advantages, disadvantages, and application scope of each signal detection method.

RESULTS

A total of 93 literatures were included, including 27 reviews and 66 methodological studies. Moreover, 31 signal detection methods were identified in these literatures. Each signal detection method has its inherent advantages and disadvantages, resulting in different application scopes of these methods.

CONCLUSION

Our systematic review finds that there are variabilities in the advantages, disadvantages, and application scopes of different signal detection methods. This finding indicates that the most suitable signal detection method varies across different drug safety scenarios. Moreover, when selecting signal detection method in a particular drug safety scenario, the following factors need to be considered: purpose of research, database size, drug characteristics, adverse event characteristics, and characteristics of the relations between drugs and adverse events.

Application of biclustering algorithm in adverse drug reaction monitoring system of China

PURPOSE

Signal evaluation is considered to be a tedious process owing to the large number of disproportional signals detected. This study aimed to apply a biclustering algorithm in the spontaneous reporting system of China and to obtain the optimal parameters. The biclustering algorithm is expected to improve the efficiency of signal evaluation by identifying similar signal groups.

METHODS

Information component (IC) was the method used for disproportionality analysis. By using IC thresholds of various strengths (0.05-4.00), the original quantitative data matrix was transformed into 80 different binary data matrices, where each cell contained either a 1 or 0. The biclustering results were obtained using a total of 720 Bimax algorithm parameters (minimal number of columns and rows was 3, 4, or 5). Next, the optimal parameters were determined through the comprehensive evaluation of the rank sum ration. Finally, we examined the biclustering results under the optimal parameters and evaluated the effect of biclustering analysis on adverse drug reaction (ADR) data in China.

RESULTS

The optimal strength of the IC threshold was 0.80, and the minimum number of rows and columns was 3. After taxonomic evaluation, we also found that 1836 biclusters (42.8%) contained similar drugs or similar ADRs, which accounted for 72.3% of signals unevaluated.

CONCLUSIONS

Applying biclustering analysis in spontaneous reporting system could provide support in confirming unrecognized ADRs, identifying rare ADRs, and screening drug-ADR pairs, which need more attention. Biclustering algorithm could improve the efficiency of signal detection and evaluation in China.

Adjuvanted (AS03) A/H1N1 2009 Pandemic Influenza Vaccines and Solid Organ Transplant Rejection: Systematic Signal Evaluation and Lessons Learnt

Introduction

We investigated a signal of solid organ transplant (SOT) rejection after immunisation with (AS03) A/H1N1 2009 pandemic influenza vaccines.

Methods

Potential immunological mechanisms were reviewed and quantitative analyses were conducted. The feasibility of pharmacoepidemiological studies was explored.

Results

Overall results, including data from a pharmacoepidemiological study, support the safety of adjuvanted (AS03) pandemic influenza vaccination in SOT recipients. The regulatory commitment to evaluate the signal through a stepwise investigation was closed in 2014.

Conclusion

Lessons learned highlight the importance of investigating plausible biological mechanisms between vaccines and potentially associated adverse outcomes, and the importance of selecting appropriate study settings and designs for safety signal investigations.

Electronic supplementary material

The online version of this article (doi:10.1007/s40264-017-0532-3) contains supplementary material, which is available to authorized users.

Good Signal Detection Practices: Evidence from IMI PROTECT

Over a period of 5 years, the Innovative Medicines Initiative PROTECT (Pharmacoepidemiological Research on Outcomes of Therapeutics by a European ConsorTium) project has addressed key research questions relevant to the science of safety signal detection. The results of studies conducted into quantitative signal detection in spontaneous reporting, clinical trial and electronic health records databases are summarised and 39 recommendations have been formulated, many based on comparative analyses across a range of databases (e.g. regulatory, pharmaceutical company). The recommendations point to pragmatic steps that those working in the pharmacovigilance community can take to improve signal detection practices, whether in a national or international agency or in a pharmaceutical company setting. PROTECT has also pointed to areas of potentially fruitful future research and some areas where further effort is likely to yield less.

The upper bound to the Relative Reporting Ratio-a measure of the impact of the violation of hidden assumptions underlying some disproportionality methods used in signal detection

Purpose

For disproportionality measures based on the Relative Reporting Ratio (RRR) such as the Information Component (IC) and the Empirical Bayesian Geometrical Mean (EBGM), each product and event is assumed to represent a negligible fraction of the spontaneous report database (SRD). Here, we provide the tools for allowing signal detection experts to assess the consequence of the violation of this assumption on their specific SRD.

Methods

For each product–event pair (P–E), a worst-case scenario associated all the reported events-of-interest with the product of interest. The values of the RRR under this scenario were measured for different sets of stratification factors using the GlaxoSmithKline vaccines SRD. These values represent the RRR upper bound that RRR cannot exceed whatever the true strength of association.

Results

Depending on the choice of stratification factors, the RRR could not exceed an upper bound of 2 for up to 2.4% of the P–Es. For Engerix™, 23.4% of all reports in the SDR, the RRR could not exceed an upper bound of 2 for up to 13.8% of pairs. For the P–E Rotarix™-Intussusception, the choice of stratification factors impacted the upper bound to RRR: from 52.5 for an unstratified RRR to 2.0 for a fully stratified RRR.

Conclusions

The quantification of the upper bound can indicate whether measures such as EBGM, IC, or RRR can be used for SRD for which products or events represent a non-negligible fraction of the entire SRD. In addition, at the level of the product or P–E, it can also highlight detrimental impact of overstratification. © 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd.

Signal detection on spontaneous reports of adverse events following immunisation: a comparison of the performance of a disproportionality-based algorithm and a time-to-onset-based algorithm

PURPOSE

Disproportionality methods measure how unexpected the observed number of adverse events is. Time-to-onset (TTO) methods measure how unexpected the TTO distribution of a vaccine-event pair is compared with what is expected from other vaccines and events. Our purpose is to compare the performance associated with each method.

METHODS

For the disproportionality algorithms, we defined 336 combinations of stratification factors (sex, age, region and year) and threshold values of the multi-item gamma Poisson shrinker (MGPS). For the TTO algorithms, we defined 18 combinations of significance level and time windows. We used spontaneous reports of adverse events recorded for eight vaccines. The vaccine product labels were used as proxies for true safety signals. Algorithms were ranked according to their positive predictive value (PPV) for each vaccine separately; amedian rank was attributed to each algorithm across vaccines.

RESULTS

The algorithm with the highest median rank was based on TTO with a significance level of 0.01 and a time window of 60 days after immunisation. It had an overall PPV 2.5 times higher than for the highest-ranked MGPS algorithm, 16(th) rank overall, which was fully stratified and had a threshold value of 0.8. A TTO algorithm with roughly the same sensitivity as the highest-ranked MGPS had better specificity but longer time-to-detection.

CONCLUSIONS

Within the scope of this study, the majority of the TTO algorithms presented a higher PPV than for any MGPS algorithm. Considering the complementarity of TTO and disproportionality methods, a signal detection strategy combining them merits further investigation. © 2013 GlaxoSmithKline. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd.