Performance of subgrouped proportional reporting ratios in the US Food and Drug Administration (FDA) adverse event reporting system

The evolving role of disproportionality analysis in pharmacovigilance

INTRODUCTION

From 2009 to 2015, the IMI PROTECT conducted rigorous studies addressing questions about optimal implementation and significance of disproportionality analyses, leading to the development of Good Signal Detection Practices. The ensuing period witnessed the independent exploration of research paths proposed by IMI PROTECT, accumulating valuable experience and insights that have yet to be seamlessly integrated.

AREAS COVERED

This state-of-the-art review integrates IMI PROTECT recommendations with recent acquisitions and evolving challenges. It deals with defining the object of study, disproportionality methods, subgrouping, masking, drug-drug interaction, duplication, expectedness, the debated use of disproportionality results as risk measures, integration with other types of data.

EXPERT OPINION

Despite the ongoing skepticism regarding the usefulness of disproportionality analyses and individual case safety reports, their ability to timely detect safety signals regarding rare and unpredictable adverse reactions remains unparalleled. Moreover, recent exploration into their potential for characterizing safety signals revealed valuable insights concerning potential risk factors and the patient's perspective. To fully realize their potential beyond hypothesis generation and achieve a comprehensive evidence synthesis with other kinds of data and studies, each with their unique limitations and contributions, we need to investigate methods for more transparently communicating disproportionality results and mapping and addressing pharmacovigilance biases.

Safety assessment of Yasmin: Real-world adverse event analysis using the FAERS database

BACKGROUND

This study aimed to elucidate the scope and nature of adverse events (AEs) associated with Yasmin.

METHODS

Among the 17,035,572 AE reports collected from the Food and Drug Administration Adverse Event Reporting System (FAERS) database between January 2004 and September 2023, 25,949 reports involved Yasmin. The demographic details, clinical outcomes, and sources of reports were extracted, and four algorithms were used to evaluate adverse drug reactions.

RESULTS

The majority of the AE reports involved females aged 18-45 years. Hospitalization was the most frequently reported serious outcome (46.84 %), with death occurring in 292 patients (1.82 %). The highest number of reports originated from the United States. Adverse reactions spanned across 24 system organ categories (SOCs), and hepatobiliary, vascular, and psychiatric disorders were the most frequently reported AEs. A total of 229 Preferred Terms (PTs) were identified for adverse reactions, with high signal strength observed for conditions such as post-cholecystectomy syndrome. In addition, fear of disease, which has not been previously identified as an AE related to Yasmin, was also identified as a high signal strength side effect.

CONCLUSION

The findings of the present study underscore the importance of monitoring and identifying potential AEs in patients receiving Yasmin, including those not currently listed in the medication instructions.

Evaluation of four machine learning models for signal detection

Background

Logistic regression-based signal detection algorithms have benefits over disproportionality analysis due to their ability to handle potential confounders and masking factors. Feature exploration and developing alternative machine learning algorithms can further strengthen signal detection.

Objectives

Our objective was to compare the signal detection performance of logistic regression, gradient-boosted trees, random forest and support vector machine models utilizing Food and Drug Administration adverse event reporting system data.

Design

Cross-sectional study.

Methods

The quarterly data extract files from 1 October 2017 through 31 December 2020 were downloaded. Due to an imbalanced outcome, two training sets were used: one stratified on the outcome variable and another using Synthetic Minority Oversampling Technique (SMOTE). A crude model and a model with tuned hyperparameters were developed for each algorithm. Model performance was compared against a reference set using accuracy, precision, F1 score, recall, the receiver operating characteristic area under the curve (ROCAUC), and the precision-recall curve area under the curve (PRCAUC).

Results

Models trained on the balanced training set had higher accuracy, F1 score and recall compared to models trained on the SMOTE training set. When using the balanced training set, logistic regression, gradient-boosted trees, random forest and support vector machine models obtained similar performance evaluation metrics. The gradient-boosted trees hyperparameter tuned model had the highest ROCAUC (0.646) and the random forest crude model had the highest PRCAUC (0.839) when using the balanced training set.

Conclusion

All models trained on the balanced training set performed similarly. Logistic regression models had higher accuracy, precision and recall. Logistic regression, random forest and gradient-boosted trees hyperparameter tuned models had a PRCAUC ⩾ 0.8. All models had an ROCAUC ⩾ 0.5. Including both disproportionality analysis results and additional case report information in models resulted in higher performance evaluation metrics than disproportionality analysis alone.