Assessing misclassification of vaccination status: Implications for studies of the safety of the childhood immunization schedule

National trends in patterns of under-vaccination in early childhood: National Immunization Survey-Child, United States, 2011-2021

BACKGROUND

The study's objective was to examine national trends in patterns of under-vaccination in the United States.

RESEARCH DESIGN AND METHODS

The National Immunization Survey-Child (NIS-Child) is an annual cross-sectional survey that collects provider-verified vaccination records from a large national probability sample of children. Records from the 2011-2021 NIS-Child were used to assess receipt of the combined 7-vaccine series by age 24 months. Based on prior work, patterns indicative of hesitancy included zero vaccines, not starting ≥1 series, and consistent vaccine-limiting. Patterns indicative of practical issues included starting all series but missing doses. Up-to-date (UTD) was defined as receiving all doses in the combined 7-vaccine series.

RESULTS

The study population comprised 127,257 children. Over the observation period, patterns indicative of hesitancy significantly decreased (p-trend < 0.0001), patterns indicative of practical issues significantly decreased (p-trend < 0.0001), and UTD significantly increased (p-trend < 0.0001). In 2021, the weighted percentage in each category was as follows: probable hesitancy 6.3% (95% confidence interval [CI] 5.4%, 7.2%), probable practical issues 26.0% (95% CI 24.4%, 27.6%), and UTD 67.7% (95% CI 66.0%, 69.4%).

CONCLUSION

Over an 11-year period, vaccination coverage in the United States for the combined 7-vaccine series has improved, with patterns suggestive of practical issues or hesitancy declining.

A comparison of electronic health records and the Oregon state immunization registry for human papilloma virus vaccine delivery (2005-2022)

INTRODUCTION

Immunization Information Systems (IIS) play an important information-sharing role at the point of care, and provide vital vaccination data for research studies and policy-makers. Previous validation studies comparing the accuracy of state registry data to health records have had mixed results.

METHODS

We conducted a retrospective review of EHR vaccination data for 9-17 year-old patients from 10 Oregon primary care clinics who had at least one ambulatory care visit in the past 3 years from the date of validation data collection. Data on 100 age eligible youth were captured per clinic. We compared HPV and Tdap vaccinations captured in the EHR to the Oregon ALERT IIS. All clinics were located in rural areas with both family medicine (n = 7) and pediatric (n = 3) primary care clinics.

RESULTS

Overall agreement for HPV vaccination between EHR and ALERT IIS was 89.4 % (k = 0.83; p < 0.05). For Tdap vaccination overall agreement was 80.8 % (k = 0.60; p < 0.05). Pediatric clinics showed a higher overall vaccine agreement for both HPV at 93.3 % (k = 0.89; p < 0.05) and Tdap at 95.3 % (k = 0.90; p < 0.05). Among clinics that used bidirectional data exchange (only family medicine clinics), HPV agreement was higher at 91 % (k = 0.85) versus 88 % (k = 0.81; p < 0.05) and was lower for Tdap 75 % with bidirectional data exchange (k = 0.50) versus 86 % without bidirectional data exchange (k = 0.70; p < 0.05). When the EHR and ALERT IIS disagreed, ALERT ISS usually had additional vaccines.

CONCLUSIONS

ALERT IIS data provides more accurate data than EHRs can provide when measuring vaccine delivery among adolescents in rural Oregon.

Temporal Trends in Undervaccination: A Population-Based Cohort Study

INTRODUCTION

Monitoring the trends in undervaccination, including that because of parental vaccine refusal or delay, can inform public health responses directed at improving vaccine confidence and vaccination coverage.

METHODS

A retrospective cohort study was conducted in the Vaccine Safety Datalink. The cohort included all children born in 2004-2017 with ≥3 well-child visits between ages 2 and 23 months. Using electronic health record-based vaccination data, the average days undervaccinated was calculated for each child. Undervaccination patterns were assessed through age 23 months. Temporal trends were inspected for inflection points and were analyzed using linear regression. Nested within the cohort study, a survey was conducted to compare parent reports of vaccine refusal or delay with observed vaccination patterns. Data were analyzed in 2020.

RESULTS

The study cohort consisted of 808,170 children. The percentage of children with average days undervaccinated=0 (fully vaccinated, no delays) rose from a nadir of 47.1% for the birth year 2008 to 68.4% for the birth year 2017 (p_trend<0.001). The percentage with no vaccines rose from 0.35% for the birth year 2004 to 1.28% for the birth year 2017 (p_trend<0.001). Consistent vaccine limiting was observed in 2.04% for the birth year 2017. Omission of measles, mumps, and rubella vaccine peaked at 4.76% in the birth year 2007 and declined thereafter (p_trend<0.001). On the parent survey (response rate 60.2%), a high proportion of parents of the most undervaccinated children reported refusing or delaying vaccines.

CONCLUSIONS

In a 14-year cohort study, vaccination timeliness has improved. However, the small but increasing number of children who received no vaccines by age 23 months warrants additional attention.

Timeliness of Early Childhood Vaccinations and Undervaccination Patterns in Montana

INTRODUCTION

Early childhood vaccination rates are lower in rural areas than those in urban areas of the U.S. This study's objective is to quantify vaccine timeliness and the prevalence of undervaccination patterns in Montana and to measure the associations between timeliness and series completion by age 24 months.

METHODS

Using records from January 2015 to November 2019 in Montana's centralized immunization information system, days undervaccinated were calculated for the combined 7-vaccine series. Undervaccination patterns indicative of certain barriers to vaccination, including parental vaccine hesitancy, were identified. Using multivariable log-linked binomial regression, the association between timing of vaccine delay and not completing the combined 7-vaccine series by age 24 months was assessed. Analyses were conducted in March 2020-August 2020.

RESULTS

Among 31,422 children, 38.0% received all vaccine doses on time; 24.3% received all doses, but some were received late; and 37.7% had not completed the combined 7-vaccine series. Approximately 18.7% had an undervaccination pattern suggestive of parental vaccine hesitancy, and 19.7% started all series but were missing doses needed for multidose series completion. Although falling behind on vaccinations at any age was associated with failing to complete the combined 7-vaccine series, being late at age 12-15 months had the strongest association (adjusted prevalence ratio=3.73, 95% CI=3.56, 3.91) compared with being on time at age 12-15 months.

CONCLUSIONS

Fewer than 2 in 5 Montana children were fully vaccinated on time for the combined 7-vaccine series. To increase vaccination rates, initiatives to increase vaccine confidence and remind parents to complete vaccine series are needed.

Addressing logistical barriers to childhood vaccination using an automated reminder system and online resource intervention: A randomized controlled trial

BACKGROUND

As the rates of vaccination decline in children with logistical barriers to vaccination, new strategies to increase vaccination are needed. The goal of this study was to develop and evaluate the effectiveness of the Vaccines For Babies (VFB) intervention, an automated reminder system with online resources to address logistical barriers to vaccination in caregivers of children enrolled in an integrated healthcare system. Effectiveness was evaluated in a randomized controlled trial.

METHODS

Qualitative interviews were conducted with parents of children less than two years old to identify logistical barriers to vaccination that were used to develop the VFB intervention. VFB included automated reminders to schedule the 6- and 12-month vaccine visit linking caregivers to resources to address logistic barriers, sent to the preferred mode of outreach (text, email, and/or phone). Parents of children between 3 and 10 months of age with indicators of logistical barriers to vaccination were randomized to receive VFB or usual well child care (UC). The primary outcome was percentage of days undervaccinated at 2 years of life. A difference in differences analysis was conducted.

RESULTS

Qualitative interviews with 6 parents of children less than 2 years of age identified transportation, scheduling challenges, and knowledge of vaccine timing as logistical barriers to vaccination. We enrolled 250 participants in the trial, 45% were loss to follow-up. There were no significant differences in vaccination uptake between those enrolled in UC or the VFB intervention (0.51%, p = 0.86). In Medicaid enrolled participants, there was a modest decrease in percentage of days undervaccinated in the VFB intervention compared to UC (6.3%, p = 0.07).

CONCLUSION

Automated Reminders and with links to heath system resources was not shown to increase infant vaccination uptake demonstrating additional resources are needed to address the needs of caregivers experiencing logistical barriers to vaccination.

Hepatitis B birth dose vaccination patterns in the military health System, 2014-2018

BACKGROUND

Since 2005, the universal hepatitis B (HepB) birth dose has been recommended for all medically stable infants weighing ≥2,000 g at birth. The timing of the birth dose provides a critical safeguard and prevents infection among infants born to HBsAg-positive mothers not identified prenatally. We assess infant HepB vaccination in the U.S. Department of Defense's Military Health System (MHS) to identify trends in vaccination coverage and sociodemographic factors associated with non-receipt of the birth dose, receiving the first HepB vaccine >3 days of life, and not receiving any HepB vaccine in the first 18 months of life utilizing parental refusal codes. To our knowledge, this is one of the first studies assessing trends in parental refusal of the HepB birth dose utilizing administrative claims parental refusal codes.

METHODS

We conducted a retrospective cohort analysis of MHS live births from January 1, 2014 through December 31, 2018 utilizing administrative claims data. Data were included from 44 hospitals in 24 unique states, territories, or countries. We analyzed diagnosis codes for vaccine refusal and vaccination and current procedural terminology (CPT) codes to identify vaccination patterns. Generalized linear mixed effects models with a logit link were used to assess factors associated with vaccination patterns.

RESULTS

HepB birth dose vaccination coverage increased from 79.6% in 2014 to 88.1% in 2018 (p < .0001). Refusal rates also increased from 3.7% in 2014 to 4.5% in 2018 (p < .0001). The percentage of patients with missing diagnosis codes for vaccine refusal or vaccination decreased from 16.7% in 2014 to 7.4% in 2018. Factors associated with non-receipt of the birth dose included earlier year of birth, white maternal race, higher maternal age, higher birth order, and longer infant length of stay in hospital.

CONCLUSION

Vaccination coverage for HepB birth dose is high in the MHS and increased over time; concurrently, refusal rates also increased over time. Utilizing administrative claims data has the benefit of differentiating reasons for non-receipt of the birth dose over time.

Web-Based Tailored Messaging to Increase Vaccination: A Randomized Clinical Trial

BACKGROUND

To increase vaccine acceptance, we created a Web-based the "Vaccines and Your Baby" intervention (VAYB) that provided new parents with vaccine information messages tailored to vaccine beliefs and values. We evaluated the effectiveness of the VAYB by comparing timely uptake of infant vaccines to an untailored version of the intervention (UT) or usual care intervention (UC) only.

METHODS

Between April 2016 and June 2019, we conducted a randomized clinical trial. Pregnant women and new parents were randomly assigned to the VAYB, UT, or UC arms. In the VAYB and UT arms, participants were exposed to interventions at 4 time points from pregnancy until their child was 15 months of age. The primary outcome was up-to-date status for recommended vaccines from birth to 200 days of age. A modified intent-to-treat analysis was conducted. Data were analyzed with logistic regression to generate odds ratios (ORs) and 95% confidence intervals (CIs).

RESULTS

We enrolled 824 participants (276 VAYB, 274 UT, 274 UC), 143 (17.4%) of whom were lost to follow-up. The up-to-date rates in the VAYB, UT, and UC arms were 91.44%, 92.86%, and 92.31%, respectively. Infants in the VAYB arm were not more likely to be up to date than infants in the UC arm (OR = 0.89; 95% CI, 0.45-1.76) or in the UT arm (OR = 0.82; 95% CI, 0.42-1.63). The odds of being up to date did not differ between UT and UC arms (OR = 1.08; 95% CI, 0.54-2.18).

CONCLUSIONS

Delivering Web-based vaccine messages tailored to parents' vaccine attitudes and values did not positively impact the timely uptake of infant vaccines.

Order of Live and Inactivated Vaccines and Risk of Non-vaccine-targeted Infections in US Children 11-23 Months of Age

BACKGROUND

Some findings from observational studies have suggested that recent receipt of live vaccines may be associated with decreased non-vaccine-targeted infection risk and mortality. Our objective was to estimate risk of non-vaccine-targeted infections based on most recent vaccine type (live vaccines only, inactivated vaccines only or both concurrently) received in US children 11-23 months of age.

METHODS

We conducted a retrospective cohort study within the Vaccine Safety Datalink. We examined electronic health record and immunization data from children born in 2003-2013 who received 3 diphtheria-tetanus-acellular pertussis vaccines before their first birthday. We modeled vaccine type as a time-varying exposure and estimated risk of non-vaccine-targeted infections identified in emergency department and inpatient settings, adjusting for multiple confounders.

RESULTS

Among 428,608 children, 48.9% were female, 4.9% had ≥1 immunization visit with live vaccines only and 10.3% had a non-vaccine-targeted infection. In males, lower risk of non-vaccine-targeted infections was observed following last receipt of live vaccines only or live and inactivated vaccines concurrently as compared with last receipt of inactivated vaccines only [live vaccines-only adjusted hazard ratio (aHR) = 0.83, 95% confidence interval (CI): 0.72-0.94; live and inactivated vaccines concurrently aHR: 0.91, 95% CI: 0.88-0.94]. Among females, last receipt of live and inactivated vaccines concurrently was significantly associated with non-vaccine-targeted infection risk (aHR = 0.94, 95% CI: 0.91-0.97 vs. last receipt of inactivated vaccines only).

CONCLUSIONS

We observed modest associations between live vaccine receipt and non-vaccine-targeted infections. In this observational study, multiple factors, including healthcare-seeking behavior, may have influenced results.

Impact of vaccine delays at the 2, 4, 6 and 12 month visits on incomplete vaccination status by 24 months of age in Quebec, Canada

BACKGROUND

Timeliness in the administration of recommended vaccines is often evaluated using vaccine delays and provides more information regarding the susceptibility of children to vaccine-preventable diseases compared with vaccine coverage at a given age. The importance of on-time administration of vaccines scheduled at the first visit is well documented, but data are scarce about the impact of vaccine delays at other visits on vaccination status by 24 months of age. Using vaccine delays for the first three doses of DTaP-containing vaccines and for the first dose of measles-containing vaccines as markers of timeliness at the 2, 4, 6 and 12 month visits, we estimated the proportion of incomplete vaccination status by 24 months of age attributable to a vaccine delay at each of these visits.

METHODS

We used the data from six cross-sectional coverage surveys conducted in the Province of Quebec from 2006 to 2016 which included 7183 children randomly selected from the universal health insurance database. A vaccine dose was considered delayed if received 30 days or more after the recommended age. The impact of new vaccine delays at each visit on incomplete vaccination status by 24 months of age was estimated with the attributable risk in the population.

RESULTS

The proportion of children with vaccine delay was 5.4% at 2 months, 13.3% at 4 months, 23.1% at 6 months and 23.6% at 12 months. Overall, 72.5% of all 2-year-old children with an incomplete status by 24 months were attributable with a vaccine delay, of which 16.1% were attributable with a first vaccine delay at 2 months, 10.6% at 4 months, 14.0% at 6 months and 31.8% at 12 months.

CONCLUSIONS

While great emphasis has been put on vaccine delays at the first vaccination visit, the prevalence of vaccine delays was greater with later visits and most children with an incomplete vaccination status by 24 months had a vaccine delay occurring during these later visits. Interventions to improve timeliness should address vaccine delays at each visit and not only focus on the first visit.

Assessing Potential Confounding and Misclassification Bias When Studying the Safety of the Childhood Immunization Schedule

OBJECTIVES

Some parents are concerned the childhood immunization schedule could increase risk for allergic disorders, including asthma. To inform future safety studies of this speculated association, a parent survey was conducted to examine the risk of misclassification of vaccination status in electronic health record data, and to assess the potential for confounding if asthma risk factors varied by vaccination status.

METHODS

A survey was conducted among parents of children 19 to 35 months old at 6 medical organizations within the Vaccine Safety Datalink. Parents of children in 4 vaccination groups were surveyed: 1) no vaccines by 12 months of age and a diagnosis of parental vaccine refusal; 2) consistent vaccine limiting (≤2 vaccines per visit); 3) not consistently vaccine limiting but otherwise undervaccinated with a vaccine refusal diagnosis; and 4) fully vaccinated with no delays and no vaccine refusal. Parents were surveyed about their child's vaccination status and whether asthma risk factors existed.

RESULTS

Among a survey sample of 2043 parents, 1209 responded (59.2%). For receiving no vaccines, the observed agreement between parent report and electronic health record data was 94.0% (κ = 0.79); for receiving all vaccines with no delays, the observed agreement was 87.3% (κ = 0.73). Although most asthma risk factors (allergic rhinitis, eczema, food allergies, family asthma history) reported by parents did not differ significantly between children in the vaccination groups studied, several factors (aeroallergen sensitivity, breastfeeding) differed significantly between groups.

CONCLUSIONS

Measurement and control of disease risk factors should be carefully considered in observational studies of the safety of the immunization schedule.

The impact of time since vaccination and study design on validity in parental recall of childhood vaccination status in the All Our Families cohort

INTRODUCTION

Parental reporting of childhood vaccination status is often used for policy and program evaluation and research purposes. Many factors can bias parental reporting of childhood vaccination status, however, to our knowledge, no analysis has assessed whether time since vaccination impacts reporting accuracy. Therefore, using the Calgary electronic vaccine registry (PHANTIM) as the gold standard, we aimed to test the accuracy of parental reporting of childhood vaccination status at three different time-points since vaccination.

METHODS

The All Our Families (AOF) cohort study asked parents to report their child's 2, 4, 6, 12 and 18 month vaccines (vaccination time-point) on questionnaires given when the child was 1, 2 and 3 years of age (survey time-point). We linked the AOF parental reporting of vaccination status to the PHANTIM registry and calculated the percent agreement and difference in coverage estimates between PHANTIM and AOF at each vaccination and survey time-point combination. Furthermore, we measured the sensitivity and specificity, and negative (NPV) and positive predictive values (PPV) of parental vaccine recall across time.

RESULTS

AOF parent reports of coverage rates were consistently higher than the PHANTIM estimates. While we saw significant differences in percent agreement for certain vaccination time-points, we saw no consistent directional difference by survey time-point, suggesting that parental accuracy did not change with time. We found a uniformly high sensitivity across all vaccination and survey time-points, and no consistent patterns in the specificity, PPV and NPV results.

CONCLUSION

Time since vaccination may not be the most important consideration when designing and implementing a vaccination survey. Other factors that may contribute to the bias associated with parental reporting of vaccination status include the complexity of the vaccine schedule, schedule changes over time, and the wording and structure of the questionnaires.

Association Between Estimated Cumulative Vaccine Antigen Exposure Through the First 23 Months of Life and Non-Vaccine-Targeted Infections From 24 Through 47 Months of Age

IMPORTANCE

Some parents are concerned that multiple vaccines in early childhood could weaken their child's immune system. Biological data suggest that increased vaccine antigen exposure could increase the risk for infections not targeted by vaccines.

OBJECTIVE

To examine estimated cumulative vaccine antigen exposure through the first 23 months of life in children with and without non-vaccine-targeted infections from 24 through 47 months of age.

DESIGN, SETTING, AND PARTICIPANTS

A nested case-control study was conducted in 6 US health care organizations participating in the Vaccine Safety Datalink. Cases were identified by International Classification of Diseases codes for infectious diseases in the emergency department and inpatient medical settings and then validated by medical record review. Cases of non-vaccine-targeted infection were matched to controls by age, sex, health care organization site, and chronic disease status. Participants were children ages 24 through 47 months, born between January 1, 2003, and September 31, 2013, followed up until December 31, 2015.

EXPOSURES

Cumulative vaccine antigen exposure, estimated by summing the number of antigens in each vaccine dose received from birth through age 23 months.

MAIN OUTCOMES AND MEASURES

Non-vaccine-targeted infections, including upper and lower respiratory infections and gastrointestinal infections, from 24 through 47 months of age, and the association between these infections and estimated cumulative vaccine exposure from birth through 23 months. Conditional logistic regression was used to estimate matched odds ratios representing the odds of non-vaccine-targeted infections for every 30-unit increase in estimated cumulative number of antigens received.

RESULTS

Among the 944 patients (193 cases and 751 controls), the mean (SD) age was 32.5 (6.3) months, 422 (45%) were female, and 61 (7%) had a complex chronic condition. Through the first 23 months, the estimated mean (SD) cumulative vaccine antigen exposure was 240.6 (48.3) for cases and 242.9 (51.1) for controls. The between-group difference for estimated cumulative antigen exposure was -2.3 (95% CI, -10.1 to 5.4; P = .55). Among children with vs without non-vaccine-targeted infections from 24 through 47 months of age, the matched odds ratio for estimated cumulative antigen exposure through age 23 months was not significant (matched odds ratio, 0.94; 95% CI, 0.84 to 1.07).

CONCLUSIONS AND RELEVANCE

Among children from 24 through 47 months of age with emergency department and inpatient visits for infectious diseases not targeted by vaccines, compared with children without such visits, there was no significant difference in estimated cumulative vaccine antigen exposure through the first 23 months of life.

Bias from outcome misclassification in immunization schedule safety research

PURPOSE

The Institute of Medicine recommended conducting observational studies of childhood immunization schedule safety. Such studies could be biased by outcome misclassification, leading to incorrect inferences. Using simulations, we evaluated (1) outcome positive predictive values (PPVs) as indicators of bias of an exposure-outcome association, and (2) quantitative bias analyses (QBA) for bias correction.

METHODS

Simulations were conducted based on proposed or ongoing Vaccine Safety Datalink studies. We simulated 4 studies of 2 exposure groups (children with no vaccines or on alternative schedules) and 2 baseline outcome levels (100 and 1000/100 000 person-years), with 3 relative risk (RR) levels (RR = 0.50, 1.00, and 2.00), across 1000 replications using probabilistic modeling. We quantified bias from non-differential and differential outcome misclassification, based on levels previously measured in database research (sensitivity > 95%; specificity > 99%). We calculated median outcome PPVs, median observed RRs, Type 1 error, and bias-corrected RRs following QBA.

RESULTS

We observed PPVs from 34% to 98%. With non-differential misclassification and true RR = 2.00, median bias was toward the null, with severe bias (median observed RR = 1.33) with PPV = 34% and modest bias (median observed RR = 1.83) with PPV = 83%. With differential misclassification, PPVs did not reflect median bias, and there was Type 1 error of 100% with PPV = 90%. QBA was generally effective in correcting misclassification bias.

CONCLUSIONS

In immunization schedule studies, outcome misclassification may be non-differential or differential to exposure. Overall outcome PPVs do not reflect the distribution of false positives by exposure and are poor indicators of bias in individual studies. Our results support QBA for immunization schedule safety research.

Patterns of childhood immunization and all-cause mortality

BACKGROUND

Evidence supports the safety of the recommended childhood immunization schedule as a whole. However, additional research is warranted as parents' refusing or delaying vaccinations has increased in recent years. All-cause mortality has been identified as a priority outcome to study in the context of the recommended immunization schedule.

METHODS

We included children born January 1, 2004 through December 31, 2009, enrolled in the Vaccine Safety Datalink (VSD) from birth through 18 months of age. We examined vaccination patterns during the first 18 months of life among 8 vaccines, and identified deaths occurring between 19 and 48 months of age. We excluded children with complex chronic conditions, contraindications to vaccination, and deaths due to injuries, congenital anomalies, or diseases with onset prior to 19 months of age. We calculated mortality rates among children with different patterns of immunization, and incidence rate ratios (IRR) using the Cox proportional hazards model for children vaccinated according to the schedule versus undervaccinated children, adjusting for outpatient healthcare utilization, influenza vaccination, sex, and VSD site.

RESULTS

Among 312,388 children in the study, 199,661 (64%) were vaccinated according to the schedule, and 112,727 (36%) were delayed or not vaccinated for at least one vaccine dose. Of 18 deaths eligible for analysis, 11 occurred in children following the schedule (2.28 per 100,000 person-years), and seven occurred in undervaccinated children (2.57 per 100,000 person-years). Mortality rates among children following the schedule were not significantly different from those of undervaccinated children when excluding deaths with unknown causes (IRR = 1.29, 95% CI = 0.33-4.99), as well as when including deaths with unknown causes (IRR = 0.84, 95% CI = 0.32-2.99).

CONCLUSION

Although there were few deaths, our results do not indicate a difference in risk of all-cause mortality among fully vaccinated versus undervaccinated children. Our findings support the safety of the currently recommended immunization schedule with regard to all-cause mortality.