Vaccine Safety Surveillance Using Routinely Collected Healthcare Data-An Empirical Evaluation of Epidemiological Designs

A comparison of four self-controlled study designs in an analysis of COVID-19 vaccines and myocarditis using five European databases

INTRODUCTION

The aim of this study was to assess the possible extent of bias due to violation of a core assumption (event-dependent exposures) when using self-controlled designs to analyse the association between COVID-19 vaccines and myocarditis.

METHODS

We used data from five European databases (Spain: BIFAP, FISABIO VID, and SIDIAP; Italy: ARS-Tuscany; England: CPRD Aurum) converted to the ConcePTION Common Data Model. Individuals who experienced both myocarditis and were vaccinated against COVID-19 between 1 September 2020 and the end of data availability in each country were included. We compared a self-controlled risk interval study (SCRI) using a pre-vaccination control window, an SCRI using a post-vaccination control window, a standard SCCS and an extension of the SCCS designed to handle violations of the assumption of event-dependent exposures.

RESULTS

We included 1,757 cases of myocarditis. For analyses of the first dose of the Pfizer vaccine, to which all databases contributed information, we found results consistent with a null effect in both of the SCRI and extended SCCS, but some indication of a harmful effect in a standard SCCS. For the second dose, we found evidence of a harmful association for all study designs, with relatively similar effect sizes (SCRI pre = 1.99, 1.40 - 2.82; SCRI post 2.13, 95 %CI - 1.43, 3.18; standard SCCS 1.79, 95 %CI 1.31 - 2.44, extended SCCS 1.52, 95 %CI = 1.08 - 2.15). Adjustment for calendar time did not change these conclusions. Findings using all designs were also consistent with a harmful effect following a second dose of the Moderna vaccine.

CONCLUSIONS

In the context of the known association between COVID-19 vaccines and myocarditis, we have demonstrated that two forms of SCRI and two forms of SCCS led to largely comparable results, possibly because of limited violation of the assumption of event-dependent exposures.

Health-Analytics Data to Evidence Suite (HADES): Open-Source Software for Observational Research

The Health-Analytics Data to Evidence Suite (HADES) is an open-source software collection developed by Observational Health Data Sciences and Informatics (OHDSI). It executes directly against healthcare data such as electronic health records and administrative claims, that have been converted to the Observational Medical Outcomes Partnership (OMOP) Common Data Model. Using advanced analytics, HADES performs characterization, population-level causal effect estimation, and patient-level prediction, potentially across a federated data network, allowing patient-level data to remain locally while only aggregated statistics are shared. Designed to run across a wide array of technical environments, including different operating systems and database platforms, HADES uses continuous integration with a large set of unit tests to maintain reliability. HADES implements OHDSI best practices, and is used in almost all published OHDSI studies, including some that have directly informed regulatory decisions.

Changes to Public Health Surveillance Methods Due to the COVID-19 Pandemic: Scoping Review

BACKGROUND

Public health surveillance plays a vital role in informing public health decision-making. The onset of the COVID-19 pandemic in early 2020 caused a widespread shift in public health priorities. Global efforts focused on COVID-19 monitoring and contact tracing. Existing public health programs were interrupted due to physical distancing measures and reallocation of resources. The onset of the COVID-19 pandemic intersected with advancements in technologies that have the potential to support public health surveillance efforts.

OBJECTIVE

This scoping review aims to explore emergent public health surveillance methods during the early COVID-19 pandemic to characterize the impact of the pandemic on surveillance methods.

METHODS

A scoping search was conducted in multiple databases and by scanning key government and public health organization websites from March 2020 to January 2022. Published papers and gray literature that described the application of new or revised approaches to public health surveillance were included. Papers that discussed the implications of novel public health surveillance approaches from ethical, legal, security, and equity perspectives were also included. The surveillance subject, method, location, and setting were extracted from each paper to identify trends in surveillance practices. Two public health epidemiologists were invited to provide their perspectives as peer reviewers.

RESULTS

Of the 14,238 unique papers, a total of 241 papers describing novel surveillance methods and changes to surveillance methods are included. Eighty papers were review papers and 161 were single studies. Overall, the literature heavily featured papers detailing surveillance of COVID-19 transmission (n=187). Surveillance of other infectious diseases was also described, including other pathogens (n=12). Other public health topics included vaccines (n=9), mental health (n=11), substance use (n=4), healthy nutrition (n=1), maternal and child health (n=3), antimicrobial resistance (n=2), and misinformation (n=6). The literature was dominated by applications of digital surveillance, for example, by using big data through mobility tracking and infodemiology (n=163). Wastewater surveillance was also heavily represented (n=48). Other papers described adaptations to programs or methods that existed prior to the COVID-19 pandemic (n=9). The scoping search also found 109 papers that discuss the ethical, legal, security, and equity implications of emerging surveillance methods. The peer reviewer public health epidemiologists noted that additional changes likely exist, beyond what has been reported and available for evidence syntheses.

CONCLUSIONS

The COVID-19 pandemic accelerated advancements in surveillance and the adoption of new technologies, especially for digital and wastewater surveillance methods. Given the investments in these systems, further applications for public health surveillance are likely. The literature for surveillance methods was dominated by surveillance of infectious diseases, particularly COVID-19. A substantial amount of literature on the ethical, legal, security, and equity implications of these emerging surveillance methods also points to a need for cautious consideration of potential harm.

Serially Combining Epidemiological Designs Does Not Improve Overall Signal Detection in Vaccine Safety Surveillance

INTRODUCTION

Vaccine safety surveillance commonly includes a serial testing approach with a sensitive method for 'signal generation' and specific method for 'signal validation.' The extent to which serial testing in real-world studies improves or hinders overall performance in terms of sensitivity and specificity remains unknown.

METHODS

We assessed the overall performance of serial testing using three administrative claims and one electronic health record database. We compared type I and II errors before and after empirical calibration for historical comparator, self-controlled case series (SCCS), and the serial combination of those designs against six vaccine exposure groups with 93 negative control and 279 imputed positive control outcomes.

RESULTS

The historical comparator design mostly had fewer type II errors than SCCS. SCCS had fewer type I errors than the historical comparator. Before empirical calibration, the serial combination increased specificity and decreased sensitivity. Type II errors mostly exceeded 50%. After empirical calibration, type I errors returned to nominal; sensitivity was lowest when the methods were combined.

CONCLUSION

While serial combination produced fewer false-positive signals compared with the most specific method, it generated more false-negative signals compared with the most sensitive method. Using a historical comparator design followed by an SCCS analysis yielded decreased sensitivity in evaluating safety signals relative to a one-stage SCCS approach. While the current use of serial testing in vaccine surveillance may provide a practical paradigm for signal identification and triage, single epidemiological designs should be explored as valuable approaches to detecting signals.

Herpes zoster vaccine safety in the Aotearoa New Zealand population: a self-controlled case series study

In Aotearoa New Zealand, zoster vaccine live is used for the prevention of zoster and associated complications in adults. This study assessed the risk of pre-specified serious adverse events following zoster vaccine live immunisation among adults in routine clinical practice. We conducted a self-controlled case series study using routinely collected national data. We compared the incidence of serious adverse events during the at-risk period with the control period. Rate ratios were estimated using Conditional Poisson regression models. Falsification outcomes analyses were used to evaluate biases in our study population. From April 2018 to July 2021, 278,375 received the vaccine. The rate ratio of serious adverse events following immunisation was 0·43 (95% confidence interval [CI]: 0·37-0·50). There was no significant increase in the risk of cerebrovascular accidents, acute myocardial infarction, acute pericarditis, acute myocarditis, and Ramsay-Hunt Syndrome. The herpes zoster vaccine is safe in adults in Aotearoa New Zealand.

Comparative risk of thrombosis with thrombocytopenia syndrome or thromboembolic events associated with different covid-19 vaccines: international network cohort study from five European countries and the US

OBJECTIVE

To quantify the comparative risk of thrombosis with thrombocytopenia syndrome or thromboembolic events associated with use of adenovirus based covid-19 vaccines versus mRNA based covid-19 vaccines.

DESIGN

International network cohort study.

SETTING

Routinely collected health data from contributing datasets in France, Germany, the Netherlands, Spain, the UK, and the US.

PARTICIPANTS

Adults (age ≥18 years) registered at any contributing database and who received at least one dose of a covid-19 vaccine (ChAdOx1-S (Oxford-AstraZeneca), BNT162b2 (Pfizer-BioNTech), mRNA-1273 (Moderna), or Ad26.COV2.S (Janssen/Johnson & Johnson)), from December 2020 to mid-2021.

MAIN OUTCOME MEASURES

Thrombosis with thrombocytopenia syndrome or venous or arterial thromboembolic events within the 28 days after covid-19 vaccination. Incidence rate ratios were estimated after propensity scores matching and were calibrated using negative control outcomes. Estimates specific to the database were pooled by use of random effects meta-analyses.

RESULTS

Overall, 1 332 719 of 3 829 822 first dose ChAdOx1-S recipients were matched to 2 124 339 of 2 149 679 BNT162b2 recipients from Germany and the UK. Additionally, 762 517 of 772 678 people receiving Ad26.COV2.S were matched to 2 851 976 of 7 606 693 receiving BNT162b2 in Germany, Spain, and the US. All 628 164 Ad26.COV2.S recipients from the US were matched to 2 230 157 of 3 923 371 mRNA-1273 recipients. A total of 862 thrombocytopenia events were observed in the matched first dose ChAdOx1-S recipients from Germany and the UK, and 520 events after a first dose of BNT162b2. Comparing ChAdOx1-S with a first dose of BNT162b2 revealed an increased risk of thrombocytopenia (pooled calibrated incidence rate ratio 1.33 (95% confidence interval 1.18 to 1.50) and calibrated incidence rate difference of 1.18 (0.57 to 1.8) per 1000 person years). Additionally, a pooled calibrated incidence rate ratio of 2.26 (0.93 to 5.52) for venous thrombosis with thrombocytopenia syndrome was seen with Ad26.COV2.S compared with BNT162b2.

CONCLUSIONS

In this multinational study, a pooled 30% increased risk of thrombocytopenia after a first dose of the ChAdOx1-S vaccine was observed, as was a trend towards an increased risk of venous thrombosis with thrombocytopenia syndrome after Ad26.COV2.S compared with BNT162b2. Although rare, the observed risks after adenovirus based vaccines should be considered when planning further immunisation campaigns and future vaccine development.