Pertussis vaccine effectiveness in a frequency matched population-based case-control Canadian Immunization Research Network study in Ontario, Canada 2009–2015

Validating pertussis data measures using electronic medical record data in Ontario, Canada 1986-2016

Background

Pertussis is a reportable disease in many countries, but ascertainment bias has limited data accuracy. This study aims to validate pertussis data measures using a reference standard that incorporates different suspected case severities, allowing for the impact of case severity on accuracy and detection to be explored.

Methods

We evaluated 25 pertussis detection algorithms in a primary care electronic medical record database between January 1, 1986 and December 30, 2016. We estimated sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). We used sensitivity analyses to explore areas of uncertainty and evaluated reasons for lack of detection.

Results

The algorithm including all data measures achieved the highest sensitivity at 20.6%. Sensitivity increased to 100% after reclassifying symptom-only cases as non-cases, but the PPV remained low. Age at first episode was significantly associated with detection in half of the tested scenarios, and false negatives often had some history of immunization.

Conclusions

Sensitivity improved by reclassifying symptom-only cases but remained low unless multiple data sources were used. Results demonstrate a trade-off between PPV and sensitivity. EMRs can enhance detection through patient history and clinical note data. It is essential to improve case identification of older individuals with vaccination history to reduce ascertainment bias.

Pertussis immunisation strategies to optimise infant pertussis control: A narrative systematic review

OBJECTIVE

Countries routinely offering acellular pertussis vaccine, where long-term protection is not sustained, have the challenge of selecting an optimal schedule to minimise disease among young infants. We conducted a narrative systematic review and synthesis of information to evaluate different pertussis immunisation strategies at controlling pertussis disease, hospitalisation, deaths, and vaccine effectiveness among young infants.

METHODS

We conducted a review of the literature on studies about the primary, booster, and/or maternal vaccination series and synthesised findings narratively. Countries offering the first three doses of vaccine within six-months of life and a booster on or before the second year or life were defined as accelerated primary and booster schedules, respectively. Countries offering primary and booster doses later were defined as extended primary and booster schedules. All search results were screened, and articles reviewed and reconciled, by two authors. The Risk of Bias in Non-randomised Studies of Intervention tool was used to evaluate the risk of bias.

FINDINGS

A total of 98 studies were included in the analyses and the following recurring themes were described: timing of vaccination, vaccine coverage, waning immunity/vaccine effectiveness, direct and indirect effectiveness, switching from an accelerated to extended schedule, impact of changes in testing. The risk of bias was generally low to moderate for most studies.

CONCLUSION

Comparing schedules is challenging and there was insufficient evidence to that one schedule was superior to another. Countries must select a schedule that maintains high vaccine coverage and reduced the risk of delaying the delivery vaccines to protect infants.

Developing a reference standard for pertussis by applying a stratified sampling strategy to electronic medical record data

PURPOSE

Pertussis surveillance remains essential in Canada, but ascertainment bias limits the accuracy of surveillance data. Introducing other sources to improve detection has highlighted the importance of validation. However, challenges arise due to low prevalence, and oversampling suspected cases can introduce partial verification bias. The aim of this study was to build a reference standard for pertussis validation studies that provides adequate analytic precision and minimizes bias.

METHODS

We used a stratified strategy to sample the reference standard from a primary care electronic medical record cohort. We incorporated abstractor notes into definite, possible, ruled-out, and no mention of pertussis classifications which were based on surveillance case definitions.

RESULTS

We abstracted eight hundred records from the cohort of 404,922. There were 208 (26%) definite and 261 (32.6%) possible prevalent pertussis cases. Classifications demonstrated a wide variety of case severities. Abstraction reliability was moderate to substantial based on Cohen's kappa and raw percent agreement.

CONCLUSIONS

When conducting validation studies for pertussis and other low prevalence diseases, this stratified sampling strategy can be used to develop a reference standard using limited resources. This approach mitigates verification and spectrum bias while providing sufficient precision and incorporating a range of case severities.

Durability of protection after 5 doses of acellular pertussis vaccine among 5-9 year old children in King County, Washington

PURPOSE

Waning of immunity after vaccination with the acellular Pertussis (aP) vaccine has been proposed as one of the main reasons for pertussis resurgence in the US. In this study, we estimated time-varying vaccine effectiveness after 5 doses of aP vaccine.

METHODS

We conducted a retrospective cohort study among children 5-9 years old (born between 2008 and 2012) living in King County, Washington, USA, who participated in the Washington State Immunization Information System. We estimated time-varying vaccine effectiveness after 5 doses of aP using smoothed scaled Schoenfeld residuals obtained from fitting Cox proportional hazards models to the data as well as piecewise constant Poisson regression.

RESULTS

There were 55 pertussis cases in this cohort, of whom 22 (40%) were fully-vaccinated and 33 (60%) were under-vaccinated. Vaccine effectiveness (VE) remained high for up to 42 months after the fifth dose (VE(t) = 89%; 95% CI: 64%, 97%) as estimated using survival analysis methods and up to 4 years (VE(t) = 93%; 95% CI: 67%, 98%) as estimated using Poisson regression.

CONCLUSION

We did not find evidence for waning of vaccine effectiveness for up to four years after 5 doses of aP among 5 -9 years old children in King County, WA.

Using observational epidemiology to evaluate COVID-19 vaccines: integrating traditional methods with new data sources and tools

Although clinical trials are necessary for vaccine approval, observational epidemiology will be required to evaluate the long-term effectiveness, safety, and population impacts of newly approved COVID-19 vaccines under real-world field conditions. In this commentary, I argue that a hybrid approach that combines new data sources and tools, including COVID-19 vaccine registries, with traditional epidemiological methods will be needed to evaluate COVID-19 vaccines using observational epidemiology. Wherever possible, primary data collection, active surveillance, and linkage with existing population-based cohorts should be leveraged to supplement secondary data sources and passive surveillance systems. Evidence-informed public health decision making around provincial COVID-19 immunization programs will need to account for potential biases, incomplete or conflicting information, and heterogeneity across subpopulations.

Waning immunity after receipt of Pertussis, Diphtheria, Tetanus and Polio-related vaccines: a systematic review and meta-analysis

DTP-containing (pertussis, diphtheria and tetanus) combined with polio vaccines are recommended by WHO as part of routine immunization programs. The decline of immunity after vaccination has been considered as a possible reason for the re-emergence of vaccine-preventable diseases worldwide. In this study, we evaluated the potential duration of protective immunity of pertussis, diphtheria, tetanus and polio through a systematic review and meta-analysis. We examined data on immunological and clinical outcomes. We observed evidence of waning post-vaccination immunity for pertussis and diphtheria, while tetanus and polio vaccines provided sustained protection. Further research on the risk factors of waning immunity after vaccination and the optimal timing of booster doses for pertussis and diphtheria are needed.

Exploring the reasons for low pertussis vaccine effectiveness in Ontario, Canada, 2006-2008: a Canadian Immunization Research Network study

OBJECTIVES

Although pertussis vaccines have been widely used for many decades, a burden of illness persists. Resurgences in Ontario, Canada, have not been substantial in the past decade, but an outbreak of pertussis occurred in Toronto between 1 October 2005 and 31 March 2006. Previous Ontario studies found high vaccine effectiveness (VE) in the initial years post-immunization. In order to explore the impact of outbreaks and external factors on VE, we investigated pertussis VE during the period 2006-2008.

METHODS

We assessed pertussis VE using a frequency-matched case-control study for the period 1 March 2006 to 31 December 2008. We used logistic regression to estimate VE by age, time since last vaccination, and vaccination status according to the Ontario recommended schedule. We compared analyses including and excluding cases from Toronto, and to two recent Ontario pertussis VE studies.

RESULTS

We included 1797 confirmed cases and 7188 matched controls. Most cases were under 4 years of age during the study period. Pertussis VE was 3.8% (95% CI: - 21.0, 24.0) in the period 15-364 days following the last pertussis vaccine dose, and increased with increasing time since vaccination. Pertussis VE in the first 15-364 days excluding Toronto increased to 57.1% (95% CI: 26.0, 75.1), but the trend of increasing VE with time since vaccination persisted. Although VE was higher in older (6-11 years) than younger (0-5 years) children, it was lower at 12-13 years than after 14 years.

CONCLUSION

VE was lower in comparison with other studies conducted in Ontario, particularly in younger children. Various factors occurring during the study period may have influenced the results, including clinical testing of asymptomatic contacts, laboratory testing and methods and reporting practice, and a sensitive case definition. Further studies are needed to optimize methods for measuring VE to inform pertussis vaccine policy.

A Call for Caution in Use of Pertussis Vaccine Effectiveness Studies to Estimate Waning Immunity: A Canadian Immunization Research Network Study

BACKGROUND

Vaccine effectiveness (VE) studies provide essential evidence on waning vaccine-derived immunity, a major threat to pertussis control. We evaluated how study design affects estimates by comparing 2 case-control studies conducted in Ontario, Canada.

METHODS

We compared results from a test-negative design (TND) with a frequency-matched design (FMD) case-control study using pertussis cases from 2005-2015. In the first study, we identified test-negative controls from the public health laboratory that diagnosed cases and, in the second, randomly selected controls from patients attending the same physicians that reported cases, frequency matched on age and year. We compared characteristics of cases and controls using standardized differences.

RESULTS

In both designs, VE estimates for the early years postimmunization were consistent with clinical trials (TND, 84%; FMD, 89% at 1-3 years postvaccination) but diverged as time since last vaccination increased (TND, 41%; FMD, 74% by 8 years postvaccination). Overall, we observed lower VE and faster waning in the TND than the FMD. In the TND but not FMD, controls differed from cases in important confounders, being younger, having more comorbidities, and higher healthcare use. Differences between the controls of each design were greater than differences between cases. TND controls were more likely to be unvaccinated or incompletely vaccinated than FMD controls (P < .001).

CONCLUSIONS

The FMD adjusted better for healthcare-seeking behavior than the TND. Duration of protection from pertussis vaccines is unclear because estimates vary by study design. Caution should be exercised by experts, researchers, and decision makers when evaluating evidence on optimal timing of boosters.

Pertussis vaccine effectiveness and duration of protection - A systematic review and meta-analysis

A comprehensive review of observational pertussis vaccine effectiveness (VE) studies is needed to update gaps from previous reviews. We conducted a systematic review of VE and duration of protection studies for the whole-cell (wP) and acellular (aP) pertussis vaccines and conducted a formal meta-analysis using random effects models. Evidence continues to suggest that receipt of any pertussis vaccine confers protection in the short-term against disease although this protection wanes rapidly for aP vaccine. We detected significant heterogeneity in pooled estimates due, in part, to factors such as bias and confounding which may be mitigated by study design. Our review of possible sources of heterogeneity may help interpretation of other VE studies and aid design decisions in future pertussis VE research.

A systematic review of quantitative bias analysis applied to epidemiological research

BACKGROUND

Quantitative bias analysis (QBA) measures study errors in terms of direction, magnitude and uncertainty. This systematic review aimed to describe how QBA has been applied in epidemiological research in 2006-19.

METHODS

We searched PubMed for English peer-reviewed studies applying QBA to real-data applications. We also included studies citing selected sources or which were identified in a previous QBA review in pharmacoepidemiology. For each study, we extracted the rationale, methodology, bias-adjusted results and interpretation and assessed factors associated with reproducibility.

RESULTS

Of the 238 studies, the majority were embedded within papers whose main inferences were drawn from conventional approaches as secondary (sensitivity) analyses to quantity-specific biases (52%) or to assess the extent of bias required to shift the point estimate to the null (25%); 10% were standalone papers. The most common approach was probabilistic (57%). Misclassification was modelled in 57%, uncontrolled confounder(s) in 40% and selection bias in 17%. Most did not consider multiple biases or correlations between errors. When specified, bias parameters came from the literature (48%) more often than internal validation studies (29%). The majority (60%) of analyses resulted in >10% change from the conventional point estimate; however, most investigators (63%) did not alter their original interpretation. Degree of reproducibility related to inclusion of code, formulas, sensitivity analyses and supplementary materials, as well as the QBA rationale.

CONCLUSIONS

QBA applications were rare though increased over time. Future investigators should reference good practices and include details to promote transparency and to serve as a reference for other researchers.

A multisite study of pertussis vaccine effectiveness by time since last vaccine dose from three Canadian provinces: A Canadian Immunization Research Network study

BACKGROUND

Pertussis remains poorly controlled relative to other diseases targeted by childhood vaccination programs. We combined estimates from four population-based studies of pertussis vaccine effectiveness (VE) in three Canadian provinces using a meta-analytic approach to improve precision and explore regional variation in VE and durability of protection.

METHODS

Studies were conducted in Alberta, Manitoba, and Ontario over periods ranging from 1996 to 2015. Adjusted log odds ratios (OR; VE = 100*[1-OR]) of the effect of vaccination on pertussis risk were estimated by time since last vaccination in each study and pooled using DerSimonian and Laird random-effects models. We used the I² statistic to estimate between-study heterogeneity and assessed methodological and clinical heterogeneity through subgroup analyses of study design and age.

RESULTS

Data on 3,270 pertussis cases and 23,863 controls were available. Pertussis VE declined from 86% (95% CI 79%-90%, I² = 81.5%) at < 1 year since last vaccination to 51% (11%-74%, I² = 80.9%) by ≥ 8 years. Effect estimates were the most heterogeneous in the least and most elapsed time periods since last vaccine dose. This was attributable mostly to variation between provinces in the distribution of age groups and number of vaccine doses received within time periods, as well as study design and small numbers in the most elapsed time period.

INTERPRETATION

Consistent trends of decreasing pertussis VE with increasing time since last vaccination across three Canadian provinces indicate the need for immunization schedules and vaccine development to optimize protection for all individuals, especially for adolescents and young adults at greatest risk of infection.

Effectiveness of four doses of pertussis vaccine during infancy diminished in elementary school age: A test-negative case-control study in Japan

OBJECTIVE

The Japanese national immunization program recommends that children receive 4 doses of acellular pertussis vaccine between 3 months and 2 years of age. Nevertheless, the number of pertussis cases is increasing in elementary school children aged 6-12 years. Therefore, a test-negative case-control study was conducted to assess the effectiveness of the pertussis vaccine program.

METHODS

Subjects included children aged ≥3 months who visited a collaborating hospital due to pertussis-specific cough between October 2017 and November 2019. All subjects underwent diagnostic tests for pertussis, and those diagnosed as positive were regarded as cases. Subjects diagnosed as pertussis-negative were classified as controls. Vaccination history was collected using a questionnaire administered to parents with reference to immunization records. Logistic regression models were employed to calculate the odds ratio (OR) and 95% confidence interval for laboratory-confirmed pertussis.

RESULTS

Of 187 recruited subjects (120 cases and 67 controls), questionnaire responses were obtained for 145 subjects (95 cases and 50 controls). Compared with unvaccinated subjects, the vaccine effectiveness (VE) of 4 doses was 70% among all subjects and reached to 90% with marginal significance among subjects under 6 years of age. However, among school-aged subjects, the VE was not suggestive of protection against pertussis (VE: 8%). For vaccinees given 4 doses, the OR for developing pertussis increased significantly with longer duration since the fourth dose (compared with <4.5 years, OR of 6.0-8.2 years = 5.74; OR of ≥8.3 years = 3.88; P for trend by duration < 0.01).

CONCLUSION

Effectiveness of administering 4 doses of pertussis vaccine during infancy decreases with time passed since the fourth dose. This regimen does not protect school-aged children against pertussis.