Incidence and Estimated Vaccine Effectiveness Against Symptomatic SARS-CoV-2 Infection Among Persons Tested in US Retail Locations, May 1 to August 7, 2021

Test negative design for vaccine effectiveness estimation in the context of the COVID-19 pandemic: A systematic methodology review

BACKGROUND

During the height of the global COVID-19 pandemic, the test-negative design (TND) was extensively used in many countries to evaluate COVID-19 vaccine effectiveness (VE). Typically, the TND involves the recruitment of care-seeking individuals who meet a common clinical case definition. All participants are then tested for an infection of interest.

OBJECTIVES

To review and describe the variation in TND methodology, and disclosure of potential biases, as applied to the evaluation of COVID-19 VE during the early vaccination phase of the pandemic.

METHODS

We conducted a systematic review by searching four biomedical databases using defined keywords to identify peer-reviewed articles published between January 1, 2020, and January 25, 2022. We included only original articles that employed a TND to estimate VE of COVID-19 vaccines in which cases and controls were evaluated based on SARS-CoV-2 laboratory test results.

RESULTS

We identified 96 studies, 35 of which met the defined criteria. Most studies were from North America (16 studies) and targeted the general population (28 studies). Outcome case definitions were based primarily on COVID-19-like symptoms; however, several papers did not consider or specify symptoms. Cases and controls had the same inclusion criteria in only half of the studies. Most studies relied upon administrative or hospital databases assembled for a different (non-evaluation) clinical purpose. Potential unmeasured confounding (20 studies), misclassification of current SARS-CoV-2 infection (16 studies) and selection bias (10 studies) were disclosed as limitations by some studies.

CONCLUSION

We observed potentially meaningful deviations from the validated design in the application of the TND during the COVID-19 pandemic.

Risk factors affecting COVID-19 vaccine effectiveness identified from 290 cross-country observational studies until February 2022: a meta-analysis and meta-regression

BACKGROUND

Observational studies made it possible to assess the impact of risk factors on the long-term effectiveness of mRNA and adenoviral vector (AdV) vaccines against COVID-19.

METHODS

A computerized literature search was undertaken using the MEDLINE, EMBASE, and MedRxiv databases to identify eligible studies, with no language restrictions, published up to 28 February 2022. Eligible were observational studies assessing vaccine effectiveness (VE) by disease severity with reference groups of unvaccinated participants or participants immunized with one, two, or three vaccine doses. Our study was carried out in compliance with the PRISMA and MOOSE guidelines. The risk of study bias was identified using the Newcastle-Ottawa Quality Assessment Scale. The GRADE guidelines were applied to assess the strength of evidence for the primary outcome. The synthesis was conducted using a meta-analysis and meta-regression.

RESULTS

Out of a total of 14,155 publications, 290 studies were included. Early VE of full vaccination against COVID-19 of any symptomatology and severity decreased from 96% (95% CI, 95-96%) for mRNA and from 86% (95% CI, 83-89%) for AdV vaccines to 67% for both vaccine types in the last 2 months of 2021. A similar 1-year decline from 98 to 86% was found for severe COVID-19 after full immunization with mRNA, but not with AdV vaccines providing persistent 82-87% effectiveness. Variant-reduced VE was only associated with Omicron regardless of disease severity, vaccine type, or vaccination completeness. The level of protection was reduced in participants aged >65 years, with a comorbidity or those in long-term care or residential homes independently of the number of doses received. The booster effect of the third mRNA dose was unclear because incompletely restored effectiveness, regardless of disease severity, declined within a short-term interval of 4 months.

CONCLUSIONS

Full vaccination provided an early high, yet waning level of protection against COVID-19 of any severity with a strong impact on the high-risk population. Moreover, the potential risk of new antigenically distinct variants should not be underestimated, and any future immunization strategy should include variant-updated vaccines.

COVID-19 cross-sectional study in Maricá, Brazil: The impact of vaccination coverage on viral incidence

Population surveillance in COVID-19 Pandemic is crucial to follow up the pace of disease and its related immunological status. Here we present a cross-sectional study done in Maricá, a seaside town close to the city of Rio de Janeiro, Brazil. Three rounds of study sampling, enrolling a total of 1134 subjects, were performed during May to August 2021. Here we show that the number of individuals carrying detectable IgG antibodies and the neutralizing antibody (NAb) levels were greater in vaccinated groups compared to unvaccinated ones, highlighting the importance of vaccination to attain noticeable levels of populational immunity against SARS-CoV-2. Moreover, we found a decreased incidence of COVID-19 throughout the study, clearly correlated with the level of vaccinated individuals as well as the proportion of individuals with detectable levels of IgG anti-SARS-CoV-2 and NAb. The observed drop occurred even during the introduction of the Delta variant in Maricá, what suggests that the vaccination slowed down the widespread transmission of this variant. Overall, our data clearly support the use of vaccines to drop the incidence associated to SARS-CoV-2.

COVID-19 vaccination effectiveness rates by week and sources of bias: a retrospective cohort study

OBJECTIVE

To examine COVID-19 vaccine effectiveness over six 7-day intervals after the first dose and assess underlying bias in observational data.

DESIGN AND SETTING

Retrospective cohort study using Columbia University Irving Medical Center data linked to state and city immunisation registries.

OUTCOMES AND MEASURES

We used large-scale propensity score matching with up to 54 987 covariates, fitted Cox proportional hazards models and constructed Kaplan-Meier plots for two main outcomes (COVID-19 infection and COVID-19-associated hospitalisation). We conducted manual chart review of cases in week 1 in both groups along with a set of secondary analyses for other index date, outcome and population choices.

RESULTS

The study included 179 666 patients. We observed increasing effectiveness after the first dose of mRNA vaccines with week 6 effectiveness approximating 84% (95% CI 72% to 91%) for COVID-19 infection and 86% (95% CI 69% to 95%) for COVID-19-associated hospitalisation. When analysing unexpectedly high effectiveness in week 1, chart review revealed that vaccinated patients are less likely to seek care after vaccination and are more likely to be diagnosed with COVID-19 during the encounters for other conditions. Secondary analyses highlighted potential outcome misclassification for International Classification of Diseases, Tenth Revision, Clinical Modification diagnosis, the influence of excluding patients with prior COVID-19 infection and anchoring in the unexposed group. Long-term vaccine effectiveness in fully vaccinated patients matched the results of the randomised trials.

CONCLUSIONS

For vaccine effectiveness studies, observational data need to be scrutinised to ensure compared groups exhibit similar health-seeking behaviour and are equally likely to be captured in the data. While we found that studies may be capable of accurately estimating long-term effectiveness despite bias in early weeks, the early week results should be reported in every study so that we may gain a better understanding of the biases. Given the difference in temporal trends of vaccine exposure and patients' baseline characteristics, indirect comparison of vaccines may produce biased results.

Association of COVID-19 Vaccination With Symptomatic SARS-CoV-2 Infection by Time Since Vaccination and Delta Variant Predominance

IMPORTANCE

Monitoring COVID-19 vaccine performance over time since vaccination and against emerging variants informs control measures and vaccine policies.

OBJECTIVE

To estimate the associations between symptomatic SARS-CoV-2 infection and receipt of BNT162b2, mRNA-1273, and Ad26.COV2.S by day since vaccination before and during Delta variant predominance (pre-Delta period: March 13-May 29, 2021; Delta period: July 18-October 17, 2021).

DESIGN, SETTING, AND PARTICIPANTS

Test-negative, case-control design with data from 6884 US COVID-19 testing sites in the pharmacy-based Increasing Community Access to Testing platform. This study included 1 634 271 laboratory-based SARS-CoV-2 nucleic acid amplification tests (NAATs) from adults 20 years and older and 180 112 NAATs from adolescents 12 to 19 years old with COVID-19-like illness from March 13 to October 17, 2021.

EXPOSURES

COVID-19 vaccination (1 Ad26.COV2.S dose or 2 mRNA doses) 14 or more days prior.

MAIN OUTCOMES AND MEASURES

Association between symptomatic infection and prior vaccination measured using the odds ratio (OR) from spline-based multivariable logistic regression.

RESULTS

The analysis included 390 762 test-positive cases (21.5%) and 1 423 621 test-negative controls (78.5%) (59.9% were 20-44 years old; 9.9% were 12-19 years old; 58.9% were female; 71.8% were White). Among adults 20 years and older, the BNT162b2 mean OR for days 14 to 60 after a second dose (initial OR) was lower during the pre-Delta period (0.10 [95% CI, 0.09-0.11]) than during the Delta period (0.16 [95% CI, 0.16-0.17]) and increased with time since vaccination (per-month change in OR, pre-Delta: 0.04 [95% CI, 0.02-0.05]; Delta: 0.03 [95% CI, 0.02-0.03]). The initial mRNA-1273 OR was 0.05 (95% CI, 0.04-0.05) during the pre-Delta period, 0.10 (95% CI, 0.10-0.11) during the Delta period, and increased with time (per-month change in OR, pre-Delta: 0.02 [95% CI, 0.005-0.03]; Delta: 0.03 [95% CI, 0.03-0.04]). The Ad26.COV2.S initial OR was 0.42 (95% CI, 0.37-0.47) during the pre-Delta period and 0.62 (95% CI, 0.58-0.65) during the Delta period and did not significantly increase with time since vaccination. Among adolescents, the BNT162b2 initial OR during the Delta period was 0.06 (95% CI, 0.05-0.06) among 12- to 15-year-olds, increasing by 0.02 (95% CI, 0.01-0.03) per month, and 0.10 (95% CI, 0.09-0.11) among 16- to 19-year-olds, increasing by 0.04 (95% CI, 0.03-0.06) per month.

CONCLUSIONS AND RELEVANCE

Among adults, the OR for the association between symptomatic SARS-CoV-2 infection and COVID-19 vaccination (as an estimate of vaccine effectiveness) was higher during Delta variant predominance, suggesting lower protection. For mRNA vaccination, the steady increase in OR by month since vaccination was consistent with attenuation of estimated effectiveness over time; attenuation related to time was greater than that related to variant.

Association Between 3 Doses of mRNA COVID-19 Vaccine and Symptomatic Infection Caused by the SARS-CoV-2 Omicron and Delta Variants

IMPORTANCE

Assessing COVID-19 vaccine performance against the rapidly spreading SARS-CoV-2 Omicron variant is critical to inform public health guidance.

OBJECTIVE

To estimate the association between receipt of 3 doses of Pfizer-BioNTech BNT162b2 or Moderna mRNA-1273 vaccine and symptomatic SARS-CoV-2 infection, stratified by variant (Omicron and Delta).

DESIGN, SETTING, AND PARTICIPANTS

A test-negative case-control analysis among adults 18 years or older with COVID-like illness tested December 10, 2021, through January 1, 2022, by a national pharmacy-based testing program (4666 COVID-19 testing sites across 49 US states).

EXPOSURES

Three doses of mRNA COVID-19 vaccine (third dose ≥14 days before test and ≥6 months after second dose) vs unvaccinated and vs 2 doses 6 months or more before test (ie, eligible for a booster dose).

MAIN OUTCOMES AND MEASURES

Association between symptomatic SARS-CoV-2 infection (stratified by Omicron or Delta variants defined using S-gene target failure) and vaccination (3 doses vs unvaccinated and 3 doses vs 2 doses). Associations were measured with multivariable multinomial regression. Among cases, a secondary outcome was median cycle threshold values (inversely proportional to the amount of target nucleic acid present) for 3 viral genes, stratified by variant and vaccination status.

RESULTS

Overall, 23 391 cases (13 098 Omicron; 10 293 Delta) and 46 764 controls were included (mean age, 40.3 [SD, 15.6] years; 42 050 [60.1%] women). Prior receipt of 3 mRNA vaccine doses was reported for 18.6% (n = 2441) of Omicron cases, 6.6% (n = 679) of Delta cases, and 39.7% (n = 18 587) of controls; prior receipt of 2 mRNA vaccine doses was reported for 55.3% (n = 7245), 44.4% (n = 4570), and 41.6% (n = 19 456), respectively; and being unvaccinated was reported for 26.0% (n = 3412), 49.0% (n = 5044), and 18.6% (n = 8721), respectively. The adjusted odds ratio for 3 doses vs unvaccinated was 0.33 (95% CI, 0.31-0.35) for Omicron and 0.065 (95% CI, 0.059-0.071) for Delta; for 3 vaccine doses vs 2 doses the adjusted odds ratio was 0.34 (95% CI, 0.32-0.36) for Omicron and 0.16 (95% CI, 0.14-0.17) for Delta. Median cycle threshold values were significantly higher in cases with 3 doses vs 2 doses for both Omicron and Delta (Omicron N gene: 19.35 vs 18.52; Omicron ORF1ab gene: 19.25 vs 18.40; Delta N gene: 19.07 vs 17.52; Delta ORF1ab gene: 18.70 vs 17.28; Delta S gene: 23.62 vs 20.24).

CONCLUSIONS AND RELEVANCE

Among individuals seeking testing for COVID-like illness in the US in December 2021, receipt of 3 doses of mRNA COVID-19 vaccine (compared with unvaccinated and with receipt of 2 doses) was less likely among cases with symptomatic SARS-CoV-2 infection compared with test-negative controls. These findings suggest that receipt of 3 doses of mRNA vaccine, relative to being unvaccinated and to receipt of 2 doses, was associated with protection against both the Omicron and Delta variants, although the higher odds ratios for Omicron suggest less protection for Omicron than for Delta.