Temporal Confounding in the Test-Negative Design

Hypothesis testing and sample size considerations for the test-negative design

The test-negative design (TND) is an observational study design to evaluate vaccine effectiveness (VE) that enrolls individuals receiving diagnostic testing for a target disease as part of routine care. VE is estimated as one minus the adjusted odds ratio of testing positive versus negative comparing vaccinated and unvaccinated patients. Although the TND is related to case-control studies, it is distinct in that the ratio of test-positive cases to test-negative controls is not typically pre-specified. For both types of studies, sparse cells are common when vaccines are highly effective. We consider the implications of these features on power for the TND. We use simulation studies to explore three hypothesis-testing procedures and associated sample size calculations for case-control and TND studies. These tests, all based on a simple logistic regression model, are a standard Wald test, a continuity-corrected Wald test, and a score test. The Wald test performs poorly in both case-control and TND when VE is high because the number of vaccinated test-positive cases can be low or zero. Continuity corrections help to stabilize the variance but induce bias. We observe superior performance with the score test as the variance is pooled under the null hypothesis of no group differences. We recommend using a score-based approach to design and analyze both case-control and TND. We propose a modification to the TND score sample size to account for additional variability in the ratio of controls over cases. This work enhances our understanding of the data generating mechanism in a test-negative design (TND) and how it is distinct from that of a case-control study due to its passive recruitment of controls.

Hypothesis testing and sample size considerations for the test-negative design

The test-negative design (TND) is an observational study design to evaluate vaccine effectiveness (VE) that enrolls individuals receiving diagnostic testing for a target disease as part of routine care. VE is estimated as one minus the adjusted odds ratio of testing positive versus negative comparing vaccinated and unvaccinated patients. Although the TND is related to case-control studies, it is distinct in that the ratio of test-positive cases to test-negative controls is not typically pre-specified. For both types of studies, sparse cells are common when vaccines are highly effective. We consider the implications of these features on power for the TND. We use simulation studies to explore three hypothesis-testing procedures and associated sample size calculations for case-control and TND studies. These tests, all based on a simple logistic regression model, are a standard Wald test, a continuity-corrected Wald test, and a score test. The Wald test performs poorly in both case-control and TND when VE is high because the number of vaccinated test-positive cases can be low or zero. Continuity corrections help to stabilize the variance but induce bias. We observe superior performance with the score test as the variance is pooled under the null hypothesis of no group differences. We recommend using a score-based approach to design and analyze both case-control and TND. We propose a modification to the TND score sample size to account for additional variability in the ratio of controls over cases. This work expands our understanding of the data mechanisms of the TND.

Comparative effectiveness of different primary vaccination courses on mRNA-based booster vaccines against SARs-COV-2 infections: a time-varying cohort analysis using trial emulation in the Virus Watch community cohort

BACKGROUND

The Omicron B.1.1.529 variant increased severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infections in doubly vaccinated individuals, particularly in the Oxford-AstraZeneca COVID-19 vaccine (ChAdOx1) recipients. To tackle infections, the UK's booster vaccination programmes used messenger ribonucleic acid (mRNA) vaccines irrespective of an individual's primary course vaccine type, and prioritized the clinically vulnerable. These mRNA vaccines included the Pfizer-BioNTech COVID-19 vaccine (BNT162b2) the Moderna COVID-19 vaccine (mRNA-1273). There is limited understanding of the effectiveness of different primary vaccination courses on mRNA booster vaccines against SARs-COV-2 infections and how time-varying confounders affect these evaluations.

METHODS

Trial emulation was applied to a prospective community observational cohort in England and Wales to reduce time-varying confounding-by-indication driven by prioritizing vaccination based upon age, vulnerability and exposure. Trial emulation was conducted by meta-analysing eight adult cohort results whose booster vaccinations were staggered between 16 September 2021 and 05 January 2022 and followed until 23 January 2022. Time from booster vaccination until SARS-CoV-2 infection, loss of follow-up or end of study was modelled using Cox proportional hazard models and adjusted for age, sex, minority ethnic status, clinically vulnerability and deprivation.

RESULTS

A total of 19 159 participants were analysed, with 11 709 ChAdOx1 primary courses and 7450 BNT162b2 primary courses. Median age, clinical vulnerability status and infection rates fluctuate through time. In mRNA-boosted adults, 7.4% (n = 863) of boosted adults with a ChAdOx1 primary course experienced a SARS-CoV-2 infection compared with 7.7% (n = 571) of those who had BNT162b2 as a primary course. The pooled adjusted hazard ratio (aHR) was 1.01 with a 95% confidence interval (CI) of: 0.90 to 1.13.

CONCLUSION

After an mRNA booster dose, we found no difference in protection comparing those with a primary course of BNT162b2 with those with a ChAdOx1 primary course. This contrasts with pre-booster findings where previous research shows greater effectiveness of BNT162b2 than ChAdOx1 in preventing infection.

Current Challenges With the Use of Test-Negative Designs for Modeling COVID-19 Vaccination and Outcomes

The widespread testing for severe acute respiratory syndrome coronavirus 2 infection has facilitated the use of test-negative designs (TNDs) for modeling coronavirus disease 2019 (COVID-19) vaccination and outcomes. Despite the comprehensive literature on TND, the use of TND in COVID-19 studies is relatively new and calls for robust design and analysis to adapt to a rapidly changing and dynamically evolving pandemic and to account for changes in testing and reporting practices. In this commentary, we aim to draw the attention of researchers to COVID-specific challenges in using TND as we are analyzing data amassed over more than two years of the pandemic. We first review when and why TND works and general challenges in TND studies presented in the literature. We then discuss COVID-specific challenges which have not received adequate acknowledgment but may add to the risk of invalid conclusions in TND studies of COVID-19.

Typhoid conjugate vaccine effectiveness in Malawi: evaluation of a test-negative design using randomised, controlled clinical trial data

BACKGROUND

Typhoid conjugate vaccines are being introduced in low-income and middle-income countries to prevent typhoid illness in children. Vaccine effectiveness studies assess vaccine performance after introduction. The test-negative design is a commonly used method to estimate vaccine effectiveness that has not been applied to typhoid vaccines because of concerns over blood culture insensitivity. The overall aim of the study was to evaluate the appropriateness of using a test-negative design to assess typhoid Vi polysaccharide-tetanus toxoid conjugate vaccine (Vi-TT) effectiveness using a gold standard randomised controlled trial database.

METHODS

Using blood culture data from a randomised controlled trial of Vi-TT in Malawi, we simulated a test-negative design to derive vaccine effectiveness estimates using three different approaches and compared these to randomised trial efficacy results. In the randomised trial, 27 882 children aged 9 months to 12 years were randomly assigned (1:1) to receive a single dose of Vi-TT or meningococcal capsular group A conjugate vaccine between Feb 21 and Sept 27, 2018, and were followed up for blood culture-confirmed typhoid fever until Sept 30, 2021.

FINDINGS

For all three test-negative design approaches, vaccine effectiveness estimates (test-negative design A, 80·3% [95% CI 66·2 to 88·5] vs test-negative design B, 80·5% [66·5 to 88·6] vs test-negative design C, 80·4% [66·9 to 88·4]) were almost identical to the randomised trial results (80·4% [95% CI 66·4 to 88·5]). Receipt of Vi-TT did not affect the risk of non-typhoid fever (vaccine efficacy against non-typhoid fever -0·4% [95% CI -4·9 to 3·9] vs -1% [-5·6 to 3·3] vs -2·5% [-6·4 to 1·3] for test-negative design A, test-negative design B, and test-negative design C, respectively).

INTERPRETATION

This study validates the test-negative design core assumption for typhoid vaccine effectiveness estimation and shows the accuracy and precision of the estimates compared with the randomised controlled trial. These results show that the test-negative design is suitable for assessing typhoid conjugate vaccine effectiveness in post-introduction studies using blood culture surveillance.

FUNDING

Bill & Melinda Gates Foundation.

The variants of SARS-CoV-2 and the challenges of vaccines

Since the outbreak of coronavirus disease 2019 (COVID-19), countries all over the world have suffered severe losses. It affects not only human life and health but also the economy. In response to COVID-19, countries have made tremendous efforts to vaccine development. The newly discovered variants of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) have brought major challenges to the effectiveness and research of vaccines. This article reviews the existing literature and summarizes the main variants of the SARS-CoV-2 and its impact on vaccines, and provides new ideas for the later development of vaccines. An excellent job in developing and applying vaccines will be an important measure for epidemic prevention and control.

SARS-CoV-2 Variants and Vaccines

Viral variants of concern may emerge with dangerous resistance to the immunity generated by the current vaccines to prevent coronavirus disease 2019 (Covid-19). Moreover, if some variants of concern have increased transmissibility or virulence, the importance of efficient public health measures and vaccination programs will increase. The global response must be both timely and science based.