Measuring Vaccine Efficacy Against Infection and Disease in Clinical Trials: Sources and Magnitude of Bias in Coronavirus Disease 2019 (COVID-19) Vaccine Efficacy Estimates

Implementation and adherence to regular asymptomatic testing in a COVID-19 vaccine trial

BACKGROUND

For pathogens which cause infections that present asymptomatically, evaluating vaccine efficacy (VE) against asymptomatic infection is important for understanding a vaccine's potential epidemiological impact. Regular testing for subclinical infections is a potentially valuable strategy but its success hinges on participant adherence and minimising false positives. This paper describes the implementation and adherence to weekly testing in a COVID-19 vaccine trial.

METHODS

COV002 was a phase 2/3 trial assessing the ChAdOx1 nCoV-19 vaccine against SARS-CoV-2. Asymptomatic infections were detected using weekly self-administered swabs for RT-PCR testing. We analysed adherence using mixed-effects regression models and estimated the probability of true and false positive asymptomatic infections using estimates of adherence and testing characteristics.

FINDINGS

356,551 tests were self-administered by 10,811 participants during the 13-month follow-up. Median adherence was 75.0% (IQR 42·6-90·9), which translated to a 74·5% (IQR 50·9-78·8) probability of detecting a positive asymptomatic infection during the swabbing period, and between 21 and 96 false positives during VE evaluation. The odds of returning a swab declined by 8% per week and further after testing positive and unblinding. Adherence was higher in older age groups, females and non-healthcare workers.

INTERPRETATION

The COV002 trial demonstrated the feasibility of running a long-term regular asymptomatic testing strategy. This information could be valuable for designing future phase III vaccine trials in which infection is an outcome.

FUNDING

UK Research and Innovation, National Institutes for Health Research (NIHR), Coalition for Epidemic Preparedness Innovations, NIHR Oxford Biomedical Research Centre, Thames Valley and South Midland's NIHR Clinical Research Network, AstraZeneca.

How has research on the effectiveness and safety of COVID-19 vaccination been evaluated: a scope review with emphasis on CoronaVac

Introduction

The control of the COVID-19 epidemic has been focused on the development of vaccines against SARS-CoV-2. All developed vaccines have reported safety and efficacy results in preventing infection and its consequences, although the quality of evidence varies depending on the vaccine considered. Different methodological designs have been used for their evaluation, which can influence our understanding of the effects of these interventions. CoronaVac is an inactivated vaccine, and it has been assessed in various studies, including clinical trials and observational studies. Given these differences, our objective was to explore the published information to answer the question: how has the efficacy/effectiveness and safety of CoronaVac been evaluated in different studies? This is to identify potential gaps and challenges to be addressed in understanding its effect.

Methods

A scoping review was carried out following the methodology proposed by the Joanna Briggs Institute, which included studies carried out in humans as of 2020, corresponding to systematic reviews, clinical trials, analytical or descriptive observational studies, in which the effectiveness and/or safety of vaccines for COVID19 were evaluated or described. There were no age restrictions for the study participants.

Results

The efficacy/effectiveness and safety of this vaccine was assessed through 113 studies. Nineteen corresponded to experimental studies, 7 of Phase II, 5 of Phase IV, and 4 were clinical trials with random assignment. Although some clinical trials with random assignment have been carried out, these have limitations in terms of feasibility, follow-up times, and with this, the possibility of evaluating safety outcomes that occur with low frequencies. Not all studies have used homogeneous methods of analysis. Both the prevention of infection, and the prevention of outcomes such as hospitalization or death, have been valued through similar outcomes, but some through multivariate analysis of dependencies, and others through analysis that try to infer causally through different control methods of confounding.

Conclusion

Published information on the evaluation of the efficacy/effectiveness and safety of the CoronaVac is abundant. However, there are differences in terms of vaccine application schedules, population definition, outcomes evaluated, follow-up times, and safety assessment, as well as non-standardization in the reporting of results, which may hinder the generalizability of the findings. It is important to generate meetings and consensus strategies for the methods and reporting of this type of studies, which will allow to reduce the heterogeneity in their presentation and a better understanding of the effect of these vaccines.

A multi-country analysis of COVID-19 hospitalizations by vaccination status

BACKGROUND

Individuals vaccinated against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), when infected, can still develop disease that requires hospitalization. It remains unclear whether these patients differ from hospitalized unvaccinated patients with regard to presentation, coexisting comorbidities, and outcomes.

METHODS

Here, we use data from an international consortium to study this question and assess whether differences between these groups are context specific. Data from 83,163 hospitalized COVID-19 patients (34,843 vaccinated, 48,320 unvaccinated) from 38 countries were analyzed.

FINDINGS

While typical symptoms were more often reported in unvaccinated patients, comorbidities, including some associated with worse prognosis in previous studies, were more common in vaccinated patients. Considerable between-country variation in both in-hospital fatality risk and vaccinated-versus-unvaccinated difference in this outcome was observed.

CONCLUSIONS

These findings will inform allocation of healthcare resources in future surges as well as design of longer-term international studies to characterize changes in clinical profile of hospitalized COVID-19 patients related to vaccination history.

FUNDING

This work was made possible by the UK Foreign, Commonwealth and Development Office and Wellcome (215091/Z/18/Z, 222410/Z/21/Z, 225288/Z/22/Z, and 220757/Z/20/Z); the Bill & Melinda Gates Foundation (OPP1209135); and the philanthropic support of the donors to the University of Oxford's COVID-19 Research Response Fund (0009109). Additional funders are listed in the "acknowledgments" section.

Individual variation in vaccine immune response can produce bimodal distributions of protection

The ability for vaccines to protect against infectious diseases varies among individuals, but computational models employed to inform policy typically do not account for this variation. Here we examine this issue: we implement a model of vaccine efficacy developed in the context of SARS-CoV-2 in order to evaluate the general implications of modelling correlates of protection on the individual level. Due to high levels of variation in immune response, the distributions of individual-level protection emerging from this model tend to be highly dispersed, and are often bimodal. We describe the specification of the model, provide an intuitive parameterisation, and comment on its general robustness. We show that the model can be viewed as an intermediate between the typical approaches that consider the mode of vaccine action to be either "all-or-nothing" or "leaky". Our view based on this analysis is that individual variation in correlates of protection is an important consideration that may be crucial to designing and implementing models for estimating population-level impacts of vaccination programs.

Observed negative vaccine effectiveness could be the canary in the coal mine for biases in observational COVID-19 studies

Since the emergence of the SARS-CoV-2 Omicron variant, multiple observational studies have reported negative vaccine effectiveness (VE) against infection, symptomatic infection, and even severity (hospitalization), potentially leading to an interpretation that vaccines were facilitating infection and disease. However, current observations of negative VE likely stem from the presence of various biases (e.g., exposure differences, testing differences). Although negative VE is more likely to arise when true biological efficacy is generally low and biases are large, positive VE measurements can also be subject to the same mechanisms of bias. In this perspective, we first outline the different mechanisms of bias that could lead to false-negative VE measurements and then discuss their ability to potentially influence other protection measurements. We conclude by discussing the use of suspected false-negative VE measurements as a signal to interrogate the estimates (quantitative bias analysis) and to discuss potential biases when communicating real-world immunity research.

Expert review of global real-world data on COVID-19 vaccine booster effectiveness and safety during the omicron-dominant phase of the pandemic

INTRODUCTION

COVID-19 vaccines have been highly effective in reducing morbidity and mortality during the pandemic. However, the emergence of the Omicron variant and subvariants as the globally dominant strains have raised doubts about the effectiveness of currently available vaccines and prompted debate about potential future vaccination strategies.

AREAS COVERED

Using the publicly available IVAC VIEW-hub platform, we reviewed 52 studies on vaccine effectiveness (VE) after booster vaccinations. VE were reported for SARS-CoV-2 symptomatic infection, severe disease and death and stratified by vaccine schedule and age. In addition, a non-systematic literature review of safety was performed to identify single or multi-country studies investigating adverse event rates for at least two of the currently available COVID-19 vaccines.

EXPERT OPINION

Booster shots of the current COVID-19 vaccines provide consistently high protection against Omicron-related severe disease and death. Additionally, this protection appears to be conserved for at least 3 months, with a small but significant waning after that. The positive risk-benefit ratio of these vaccines is well established, giving us confidence to administer additional doses as required. Future vaccination strategies will likely include a combination of schedules based on risk profile, as overly frequent boosting may be neither beneficial nor sustainable for the general population.