Time to onset in statistical signal detection revisited: A follow-up study in long-term onset adverse drug reactions

Frequency and timing of adverse reactions to COVID-19 vaccines; A multi-country cohort event monitoring study

INTRODUCTION

During the COVID-19 pandemic, EMA set-up a large-scale cohort event monitoring (CEM) system to estimate incidence rates of patient-reported adverse drug reactions (ADRs) of different COVID-19 vaccines across the participating countries. This study aims to give an up to date and in-depth analysis of the frequency of patient-reported ADRs after the 1st, 2nd, and booster vaccination, to identify potential predictors in developing ADRs and to describe time-to-onset (TTO) and time-to-recovery (TTR) of ADRs.

METHODS

A CEM study was rolled out in a period ranging from February 2021 to February 2023 across multiple European countries; The Netherlands, Belgium, France, the United Kingdom, Italy, Portugal, Romania, Slovakia and Spain. Analysis consisted of a descriptive analyses of frequencies of COVID-19 vaccine-related ADRs for 1st, 2nd and booster vaccination, analysis of potential predictors in developing ADRs with a generalized linear mixed-effects model, analysis of TTO and TTR of ADRs and a sensitivity analysis for loss to follow-up (L2FU).

RESULTS

A total of 29,837 participants completed at least the baseline and the first follow-up questionnaire for 1st and 2nd vaccination and 7,250 participants for the booster. The percentage of participants who reported at least one ADR is 74.32% (95%CI 73.82-74.81). Solicited ADRs, including injection site reactions, are very common across vaccination moments. Potential predictors for these reactions are the brand of vaccine used, the patient's age, sex and prior SARS-CoV-2 infection. The percentage of serious ADRs in the study is low for 1st and 2nd vaccination (0.24%, 95%CI 0.19--0.31) and booster (0.26%, 95%CI 0.15, 0.41). The TTO was 14 h (median) for dose 1 and slightly longer for dose 2 and booster dose. TTR is generally also within a few days. The effect of L2FU on estimations of frequency is limited.

CONCLUSION

Despite some limitations due to study design and study-roll out, CEM studies can allow prompt and almost real-time observations of the safety of medications directly from a patient-centered perspective, which can play a crucial role for regulatory bodies during an emergency setting such as the COVID-19 pandemic.

Detection algorithms and attentive points of safety signal using spontaneous reporting systems as a clinical data source

Continuous evaluation of drug safety is needed following approval to determine adverse events (AEs) in patient populations with diverse backgrounds. Spontaneous reporting systems are an important source of information for the detection of AEs not identified in clinical trials and for safety assessments that reflect the real-world use of drugs in specific populations and clinical settings. The use of spontaneous reporting systems is expected to detect drug-related AEs early after the launch of a new drug. Spontaneous reporting systems do not contain data on the total number of patients that use a drug; therefore, signal detection by disproportionality analysis, focusing on differences in the ratio of AE reports, is frequently used. In recent years, new analyses have been devised, including signal detection methods focused on the difference in the time to onset of an AE, methods that consider the patient background and those that identify drug-drug interactions. However, unlike commonly used statistics, the results of these analyses are open to misinterpretation if the method and the characteristics of the spontaneous reporting system cannot be evaluated properly. Therefore, this review describes signal detection using data mining, considering traditional methods and the latest knowledge, and their limitations.

Time to onset in statistical signal detection revisited: A follow-up study in long-term onset adverse drug reactions

Purpose

In a previous study, we developed a signal detection method using the time to onset (TTO) of adverse drug reactions (ADRs). The aim of the current study was to investigate this method in a subset of ADRs with a longer TTO and to compare its performance with disproportionality analysis.

Methods

Using The Netherlands's spontaneous reporting database, TTO distributions for drug—ADR associations with a median TTO of 7 days or more were compared with other drugs with the same ADR using the two‐sample Anderson–Darling (AD) test. Presence in the Summary of Product Characteristics (SPC) was used as the gold standard for identification of a true ADR. Twelve combinations with different values for the number of reports and median TTO were tested. Performance in terms of sensitivity and positive predictive value (PPV) was compared with disproportionality analysis. A sensitivity analysis was performed to compare the results with those from the previous study.

Results

A total of 38 017 case reports, containing 32 478 unique drug—ADR associations. Sensitivity was lower for the TTO method (range 0.08‐0.34) compared with disproportionality analysis (range 0.60‐0.87), whereas PPV was similar for both methods (range 0.93‐1.0). The results from the sensitivity analysis were similar to the original analysis.

Conclusions

Because of its low sensitivity, the developed TTO method cannot replace disproportionality analysis as a signal detection tool. It may be useful in combination with other methods.