1
|
Liu S, Durantini MR, Calabrese C, Sanchez F, Albarracin D. A systematic review and meta-analysis of strategies to promote vaccination uptake. Nat Hum Behav 2024; 8:1689-1705. [PMID: 39090405 DOI: 10.1038/s41562-024-01940-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Accepted: 07/01/2024] [Indexed: 08/04/2024]
Abstract
Although immunization can dramatically curb the mortality and morbidity associated with vaccine-preventable diseases, vaccination uptake remains suboptimal in many areas of the world. Here, in this meta-analysis, we analysed the results from 88 eligible randomized controlled trials testing interventions to increase vaccination uptake with 1,628,768 participants from 17 countries with variable development levels (for example, Human Development Index ranging from 0.485 to 0.955). We estimated the efficacy of seven intervention strategies including increasing access to vaccination, sending vaccination reminders, providing incentives, supplying information, correcting misinformation, promoting both active and passive motivation and teaching behavioural skills. We showed that the odds of vaccination were 1.5 (95% confidence interval, 1.27 to 1.77) times higher for intervention than control conditions. Among the intervention strategies, using incentives and increasing access were most promising in improving vaccination uptake, with the access strategy being particularly effective in countries with lower incomes and less access to healthcare.
Collapse
Affiliation(s)
- Sicong Liu
- School of Physical Education and Sports Science, South China Normal University, Guangzhou, China.
- Annenberg Public Policy Center, University of Pennsylvania, Philadelphia, PA, USA.
| | - Marta R Durantini
- Annenberg Public Policy Center, University of Pennsylvania, Philadelphia, PA, USA
| | - Christopher Calabrese
- Annenberg Public Policy Center, University of Pennsylvania, Philadelphia, PA, USA
- College of Behavioral, Social and Health Sciences, Clemson University, Clemson, SC, USA
| | - Flor Sanchez
- Department of Psychology, Universidad Autónoma de Madrid, Madrid, Spain
| | - Dolores Albarracin
- Annenberg Public Policy Center, University of Pennsylvania, Philadelphia, PA, USA.
- Department of Psychology, University of Pennsylvania, Philadelphia, PA, USA.
- Annenberg School for Communication, University of Pennsylvania, Philadelphia, PA, USA.
| |
Collapse
|
2
|
Malik AA, Ahmed N, Shafiq M, Elharake JA, James E, Nyhan K, Paintsil E, Melchinger HC, Team YBI, Malik FA, Omer SB. Behavioral interventions for vaccination uptake: A systematic review and meta-analysis. Health Policy 2023; 137:104894. [PMID: 37714082 PMCID: PMC10885629 DOI: 10.1016/j.healthpol.2023.104894] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Revised: 06/22/2023] [Accepted: 08/15/2023] [Indexed: 09/17/2023]
Abstract
BACKGROUND Human behavior and more specifically behavioral insight-based approaches to vaccine uptake have often been overlooked. While there have been a few narrative reviews indexed in Medline on behavioral interventions to increase vaccine uptake, to our knowledge, none have been systematic reviews and meta-analyses covering not just high but also low-and-middle income countries. METHODS We included 613 studies from the Medline database in our systematic review and meta-analysis categorizing different behavioral interventions in 9 domains: education campaigns, on-site vaccination, incentives, free vaccination, institutional recommendation, provider recommendation, reminder and recall, message framing, and vaccine champion. Additionally, considering that there is variability in the acceptance of vaccines among different populations, we assessed studies from both high-income countries (HICs) and low- to middle-income countries (LMICs), separately. FINDINGS Our results showed that behavioral interventions can considerably improve vaccine uptake in most settings. All domains that we examined improved vaccine uptake with the highest effect size associated with provider recommendation (OR: 3.4 (95%CI: 2.5-4.6); Domain: motivation) and on-site vaccination (OR: 2.9 (95%CI: 2.3-3.7); Domain: practical issues). While the number of studies conducted in LMICs was smaller, the quality of studies was similar with those conducted in HICs. Nevertheless, there were variations in the observed effect sizes. INTERPRETATION Our findings indicate that "provider recommendation" and "on-site vaccination" along with other behavioral interventions can be employed to increase vaccination rates globally.
Collapse
Affiliation(s)
- Amyn A Malik
- Yale Institute for Global Health, New Haven, CT 06510, USA; Analysis Group, Inc, Boston, MA 02199, USA
| | - Noureen Ahmed
- UT Southwestern Peter O'Donnell Jr. School of Public Health, Dallas, TX 75390, USA
| | - Mehr Shafiq
- Yale Institute for Global Health, New Haven, CT 06510, USA; Columbia University School of Public Health, New York, NY 10032, USA
| | - Jad A Elharake
- Yale Institute for Global Health, New Haven, CT 06510, USA; UT Southwestern Peter O'Donnell Jr. School of Public Health, Dallas, TX 75390, USA; The Ohio State University College of Medicine, Columbus, OH 43210, USA
| | - Erin James
- Yale Institute for Global Health, New Haven, CT 06510, USA
| | - Kate Nyhan
- Yale University, New Haven, CT 06510, USA
| | - Elliott Paintsil
- Yale Institute for Global Health, New Haven, CT 06510, USA; Columbia University Institute of Human Nutrition, New York, NY 10032, USA
| | | | | | - Fauzia A Malik
- UT Southwestern Peter O'Donnell Jr. School of Public Health, Dallas, TX 75390, USA
| | - Saad B Omer
- UT Southwestern Peter O'Donnell Jr. School of Public Health, Dallas, TX 75390, USA.
| |
Collapse
|
3
|
Tsamandouras I, Spyromitrou-Xioufi P, Matalliotakis M, Matalliotaki C, Ladomenou F. Influenza and Pertussis Vaccine Uptake during Pregnancy: Determinants Found through a Multi-Center Questionnaire Study of Pregnant Women and Healthcare Professionals. Behav Med 2023; 49:1-6. [PMID: 34791991 DOI: 10.1080/08964289.2021.1987853] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/27/2023]
Abstract
The objective of this article is to assess the adherence of pregnant women to the national recommendations for influenza and pertussis vaccination and the reasons behind their non-adherence. This was a retrospective observational study conducted in a well-defined puerperant population of adequate healthcare standards from December 2018 to December 2019. The study was carried out with 1006 puerperants and 66 health care practitioners. Data were collected, including demographic-obstetric features of pregnant women, whether they received antenatal vaccination, the reasons for having been vaccinated or not as well as health professional's opinion regarding antenatal immunization. The uptake of influenza and pertussis vaccine during pregnancy was suboptimal with lack of recommendation of the vaccine by the healthcare providers being the main barrier. Factors positively associated with antenatal vaccination against influenza were higher level of maternal education and advanced maternal age while antenatal vaccination against pertussis was positively associated with higher level of maternal education. This large-scale retrospective study reveals the inadequacy of antenatal vaccination rates against pertussis and influenza in Crete, Greece. Results suggest that obstetricians' confidence in vaccination is of outmost importance for implementing immunization in pregnancy and any doubts on vaccine effectiveness and safety should be resolved. Routine antenatal vaccination counseling and pregnancy immunization campaigns are essential to improve vaccine uptake during pregnancy.
Collapse
Affiliation(s)
| | | | - Michail Matalliotakis
- Department of Obstetrics & Gynaecology, Venizeleion General Hospital, Heraklion, Greece
| | - Charoula Matalliotaki
- Department of Obstetrics & Gynaecology, Venizeleion General Hospital, Heraklion, Greece
| | - Fani Ladomenou
- Department of Paediatrics, Venizeleion General Hospital, Heraklion, Greece
| |
Collapse
|
4
|
Regan AK, Fiddian-Green A. Protecting pregnant people & infants against influenza: A landscape review of influenza vaccine hesitancy during pregnancy and strategies for vaccine promotion. Hum Vaccin Immunother 2022; 18:2156229. [PMID: 36535646 PMCID: PMC10019833 DOI: 10.1080/21645515.2022.2156229] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2022] [Revised: 11/23/2022] [Accepted: 12/05/2022] [Indexed: 12/24/2022] Open
Abstract
Before COVID-19, influenza vaccines were the most widely recommended vaccine during pregnancy worldwide. In response to immunization during pregnancy, maternal antibodies offer protection against potentially life-threatening disease in both pregnant people and their infants up to six months of age. Despite this, influenza vaccine hesitancy is common, with few countries reporting immunization rates in pregnant people above 50%. In this review, we highlight individual, institutional, and social factors associated with influenza vaccine hesitancy during pregnancy. In addition, we present an overview of the evidence evaluating interventions to address influenza vaccine hesitancy during pregnancy. While some studies have indicated promising results, no single intervention has consistently effectively increased influenza vaccine uptake during pregnancy. Using a social-ecological model of health framework, future strategies addressing multiple levels of vaccine hesitancy will be needed to realize the potential health benefits of prenatal immunization programs.
Collapse
Affiliation(s)
- Annette K. Regan
- School of Nursing and Health Professions, University of San Francisco, San Francisco, CA, USA
- Fielding School of Public Health, University of California Los Angeles, Los Angeles, CA, USA
| | - Alice Fiddian-Green
- School of Nursing and Health Professions, University of San Francisco, San Francisco, CA, USA
| |
Collapse
|
5
|
Omer SB, O'Leary ST, Bednarczyk RA, Ellingson MK, Spina CI, Dudley MZ, Chamberlain AT, Limaye RJ, Brewer SE, Frew PM, Malik FA, Orenstein W, Halsey N, Ault K, Salmon DA. Multi-tiered intervention to increase maternal immunization coverage: A randomized, controlled trial. Vaccine 2022; 40:4955-4963. [PMID: 35817646 DOI: 10.1016/j.vaccine.2022.06.055] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2021] [Revised: 05/13/2022] [Accepted: 06/20/2022] [Indexed: 10/17/2022]
Abstract
OBJECTIVE To evaluate the impact of a multi-component intervention package of maternal immunization uptake in obstetric care clinics. METHODS In a multi-level, cluster- and individually-randomized controlled trial we implemented an evidence-based intervention that targeted practice-, provider- and patient-level barriers to vaccine uptake. Obstetric practices were randomized to receive the practice and provider-level interventions or continue their normal standard of care. We enrolled pregnant women at practices in Georgia and Colorado and randomized women into patient-level intervention and control groups, resulting in four study arms. The primary outcomes were receipt of the influenza and tetanus, diphtheria and acellular pertussis (Tdap) vaccines during pregnancy. A sample size of 550 women per arm (2200 total) was planned and enrolled to compare the intervention between the four study arms. RESULTS Between June 2017 and July 2018, 4907 women were screened and 2200 women were randomized, 550 to each of the four study arms. We were unable to follow-up with 108 women, for a final sample size of 2092. Sample characteristics and sample size were similar among study arms. There was no significant increase in Tdap or influenza vaccine uptake overall. Among women who had no intention of or were unsure about receiving the influenza vaccine during pregnancy, those who received just the patient-level intervention were 61% more likely to receive the influenza vaccine than those in the control arm (Relative risk: 1.61; 95% Confidence Interval: 1.18-2.21). There was no significant difference in vaccine uptake for either influenza or tetanus, diphtheria and acellular pertussis between the four arms of the study. CONCLUSIONS This trial highlights the need for more targeted interventions to improve vaccine uptake. Future work should focus on clinics with low baseline vaccine uptake and the patient-level intervention should be expanded and targeted towards women with low vaccine confidence.
Collapse
Affiliation(s)
- Saad B Omer
- Yale Institute for Global Health, Yale University, New Haven, CT, United States; Department of Health Policy and Management, Yale School of Public Health, Yale University, New Haven, CT, United States; Department of Epidemiology of Microbial Diseases, Yale School of Public Health, Yale University, New Haven, CT, United States; Yale School of Nursing, Yale University, New Haven, CT, United States
| | - Sean T O'Leary
- Adult and Child Consortium for Health Outcomes Research and Delivery Science, University of Colorado Anschutz Medical Campus and Children's Hospital Colorado, Aurora, CO, United States; Department of Pediatrics, University of Colorado Anschutz Medical Campus, Aurora, CO, United States
| | - Robert A Bednarczyk
- Hubert Department of Global Health, Rollins School of Public Health, Emory University, Atlanta, GA, United States; Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA, United States; Emory Vaccine Center, Emory University, Atlanta, GA, United States
| | - Mallory K Ellingson
- Department of Epidemiology of Microbial Diseases, Yale School of Public Health, Yale University, New Haven, CT, United States.
| | - Christine I Spina
- Adult and Child Consortium for Health Outcomes Research and Delivery Science, University of Colorado Anschutz Medical Campus and Children's Hospital Colorado, Aurora, CO, United States
| | - Matthew Z Dudley
- Department of International Health, Johns Hopkins University Bloomberg School of Public Health, Baltimore, MD, United States; Institute for Vaccine Safety, Johns Hopkins University Bloomberg School of Public Health, Baltimore, MD, United States
| | - Allison T Chamberlain
- Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA, United States
| | - Rupali J Limaye
- Department of International Health, Johns Hopkins University Bloomberg School of Public Health, Baltimore, MD, United States; Department of Health, Behavior and Society, Johns Hopkins University Bloomberg School of Public Health, Baltimore, MD, United States; Department of Epidemiology, Johns Hopkins University Bloomberg School of Public Health, Baltimore, MD, United States; International Vaccine Access Center, Johns Hopkins University Bloomberg School of Public Health, Baltimore, MD, United States
| | - Sarah E Brewer
- Adult and Child Consortium for Health Outcomes Research and Delivery Science, University of Colorado Anschutz Medical Campus and Children's Hospital Colorado, Aurora, CO, United States; Department of Family Medicine, University of Colorado Anschutz Medical Campus, United States
| | - Paula M Frew
- Hubert Department of Global Health, Rollins School of Public Health, Emory University, Atlanta, GA, United States; Department of Medicine, School of Medicine, Emory University, Atlanta, GA, United States
| | - Fauzia A Malik
- Department of Health Policy and Management, Yale School of Public Health, Yale University, New Haven, CT, United States; Dean's Office, Yale School of Public Health, New Haven, CT, United States
| | - Walter Orenstein
- Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA, United States; Emory Vaccine Center, Emory University, Atlanta, GA, United States; Department of Medicine, School of Medicine, Emory University, Atlanta, GA, United States
| | - Neal Halsey
- Department of International Health, Johns Hopkins University Bloomberg School of Public Health, Baltimore, MD, United States; Institute for Vaccine Safety, Johns Hopkins University Bloomberg School of Public Health, Baltimore, MD, United States
| | - Kevin Ault
- Department of Obstetrics and Gynecology, University of Kansas School of Medicine, Kansas City, KS, United States
| | - Daniel A Salmon
- Department of International Health, Johns Hopkins University Bloomberg School of Public Health, Baltimore, MD, United States; Institute for Vaccine Safety, Johns Hopkins University Bloomberg School of Public Health, Baltimore, MD, United States; Department of Health, Behavior and Society, Johns Hopkins University Bloomberg School of Public Health, Baltimore, MD, United States
| |
Collapse
|
6
|
Callahan AG, Coleman-Cowger VH, Schulkin J, Power ML. Racial disparities in influenza immunization during pregnancy in the United States: A narrative review of the evidence for disparities and potential interventions. Vaccine 2021; 39:4938-4948. [PMID: 34312009 DOI: 10.1016/j.vaccine.2021.07.028] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2021] [Revised: 07/09/2021] [Accepted: 07/12/2021] [Indexed: 10/20/2022]
Abstract
OBJECTIVE The objectives of this review were to summarize existing data on racial disparities in maternal immunization for influenza in the U.S. and to review the literature on interventions to improve the uptake of the influenza vaccine among Black pregnant women. DATA SOURCES U.S. survey data on maternal influenza immunization by racial and ethnic group were summarized in narrative form. To review intervention studies, PubMed, CINAHL, EMBASE, and the Cochrane Library databases were searched for English language articles published 2017 to 2021, in addition to studies identified by a previous systematic review. STUDY ELIGIBILITY CRITERIA Peer-reviewed studies conducted in the U.S. and reporting interventions designed to increase the uptake of the influenza vaccine in pregnancy with study populations including at least 20% of participants identifying as Black were included. STUDY APPRAISAL AND SYNTHESIS METHODS Studies were grouped and reviewed in a narrative manner according to whether they were conducted in predominantly Black populations or in more racially diverse populations, and whether they tested multicomponent or single-component interventions. RESULTS A decade of survey data show that Black women in the U.S. consistently have the lowest rate of influenza immunization in pregnancy. Black women report a lower rate of being recommended or offered the vaccine, and provider recommendation is associated with greater vaccine uptake. Intervention studies to increase influenza immunization among Black pregnant women have reported mixed results. Successful interventions include multicomponent practice-based interventions, group prenatal care, and culturally competent patient educational messages. CONCLUSIONS Racial disparities in maternal uptake of the influenza vaccine are long-standing, but not intractable. More research is needed to test interventions to address this disparity, with a focus on increasing provider recommendation and offer of the vaccine, addressing patients' concerns about vaccine safety and efficacy, improving providers' cultural competence, and building trust between providers and patients.
Collapse
Affiliation(s)
- Alice G Callahan
- Health Professions Division, Lane Community College, 4000 East 30th Ave., Eugene, OR 97405, USA.
| | | | - Jay Schulkin
- Department of Obstetrics and Gynecology, University of Washington, 1959 NE Pacific Street, Box 356460, Seattle, WA 98195, USA.
| | - Michael L Power
- Center for Species Survival, Smithsonian National Zoological Park & Conservation Biology Institute, P.O. Box 37012, MRC 5503, Washington, DC 20013-7012, USA.
| |
Collapse
|
7
|
A systematic review of interventions to improve uptake of pertussis vaccination in pregnancy. PLoS One 2019; 14:e0214538. [PMID: 30921421 PMCID: PMC6438510 DOI: 10.1371/journal.pone.0214538] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2018] [Accepted: 03/14/2019] [Indexed: 12/22/2022] Open
Abstract
BACKGROUND Maternal pertussis vaccination has been introduced in several countries to prevent pertussis morbidity and mortality in infants too young to be vaccinated. Our review aimed to systematically collect and summarize the available evidence on the effectiveness of interventions used to improve pertussis vaccination uptake in pregnant women. METHODS We conducted a systematic search of MEDLINE/PubMed, PMC and CINAHL. Before and after studies and those with a concurrent control group were considered for inclusion. Standardized effect sizes were described as the ratio of the odds to be vaccinated in the intervention group compared with the standard care group and absolute benefit increase (ABI) were calculated. RESULTS Six studies were included in the review, of which three were randomized controlled trials (RCTs). Strategies to improve uptake were focused on healthcare providers, pregnant women, or enhancing vaccine access. Healthcare provider interventions included provider reminder, education, feedback and standing orders. Interventions directed at pregnant women focused solely on education. Observational studies showed: (1) the provision of maternal pertussis vaccination by midwives at the place of antenatal care has improved uptake of pertussis vaccine during pregnancy from 20% to 90%; (2) introduction of an automated reminder within the electronic medical record was associated with an improvement in the pertussis immunization rate from 48% to 97%; (3) an increase in prenatal pertussis vaccine uptake from 36% to 61% after strategies to increase provider awareness of recommendations were introduced. In contrast to these findings, interventions in all three RCTs (2 involved education of pregnant women, 1 had multi-component interventions) did not demonstrate improved vaccination uptake. CONCLUSIONS Based on the existing research, we recommend incorporating midwife delivered maternal immunization programs at antenatal clinics, use of a provider reminder system to target unvaccinated pregnant women and include maternal pertussis immunization as part of standard antenatal care.
Collapse
|
8
|
Ellingson MK, Dudley MZ, Limaye RJ, Salmon DA, O'Leary ST, Omer SB. Enhancing uptake of influenza maternal vaccine. Expert Rev Vaccines 2019; 18:191-204. [PMID: 30587042 DOI: 10.1080/14760584.2019.1562907] [Citation(s) in RCA: 55] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2023]
Abstract
INTRODUCTION Influenza vaccination during pregnancy can offer many benefits to both mother and infant. Despite recommendations from the Advisory Committee on Immunization Practices, vaccine coverage rates among pregnant women during pregnancy are below 40% in the United States. There is a need for a greater understanding of what interventions can improve vaccine uptake among pregnant women. AREAS COVERED This review synthesizes the existing evidence on the effectiveness of interventions to improve maternal influenza vaccine uptake. These interventions are examined within the framework of the three psychological propositions: thoughts and feelings, social processes and changing behavior directly. EXPERT COMMENTARY A number of promising and effective interventions were identified in this review. Nudge-based interventions that build on favorable intentions to vaccinate such as provider prompts and standing orders have demonstrated significant success in improving influenza vaccine uptake. However, substantial gaps in the literature still exist. Provider recommendations are the most important predictor of vaccine receipt among pregnant women, yet few studies evaluated intervening to improve the dialogue between patient and provider. With the potential for even more vaccines to be added to the maternal immunization schedule, it is vitally important to understand how to improve uptake.
Collapse
Affiliation(s)
- Mallory K Ellingson
- a Hubert Department of Global Health, Rollins School of Public Health , Emory University , Atlanta , GA , USA
| | - Matthew Z Dudley
- b Department of International Health , Johns Hopkins University Bloomberg School of Public Health , Baltimore , MD , USA.,c Institute for Vaccine Safety , Johns Hopkins University Bloomberg School of Public Health , Baltimore , MD , USA
| | - Rupali J Limaye
- b Department of International Health , Johns Hopkins University Bloomberg School of Public Health , Baltimore , MD , USA.,c Institute for Vaccine Safety , Johns Hopkins University Bloomberg School of Public Health , Baltimore , MD , USA.,d Department of Epidemiology , Johns Hopkins University Bloomberg School of Public Health , Baltimore , MD , USA.,e Department of Health, Behavior and Society , Johns Hopkins University Bloomberg School of Public Health , Baltimore , MD , USA
| | - Daniel A Salmon
- b Department of International Health , Johns Hopkins University Bloomberg School of Public Health , Baltimore , MD , USA.,c Institute for Vaccine Safety , Johns Hopkins University Bloomberg School of Public Health , Baltimore , MD , USA.,e Department of Health, Behavior and Society , Johns Hopkins University Bloomberg School of Public Health , Baltimore , MD , USA
| | - Sean T O'Leary
- f Adult and Child Consortium for Health Outcomes Research and Delivery Science , University of Colorado Anschutz Medical Campus and Children's Hospital , Aurora , CO , USA.,g Department of Pediatrics , University of Colorado Anschutz Medical Campus , Aurora , CO , USA
| | - Saad B Omer
- a Hubert Department of Global Health, Rollins School of Public Health , Emory University , Atlanta , GA , USA.,h Department of Epidemiology, Rollins School of Public Health , Emory University , Atlanta , GA , USA.,i Department of Pediatrics, School of Medicine , Emory University , Atlanta , GA , USA.,j Emory Vaccine Center , Emory University , Atlanta , GA , USA
| |
Collapse
|
9
|
Kaufman J, Ryan R, Walsh L, Horey D, Leask J, Robinson P, Hill S. Face-to-face interventions for informing or educating parents about early childhood vaccination. Cochrane Database Syst Rev 2018; 5:CD010038. [PMID: 29736980 PMCID: PMC6494431 DOI: 10.1002/14651858.cd010038.pub3] [Citation(s) in RCA: 88] [Impact Index Per Article: 14.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/12/2023]
Abstract
BACKGROUND Early childhood vaccination is an essential global public health practice that saves two to three million lives each year, but many children do not receive all the recommended vaccines. To achieve and maintain appropriate coverage rates, vaccination programmes rely on people having sufficient awareness and acceptance of vaccines.Face-to-face information or educational interventions are widely used to help parents understand why vaccines are important; explain where, how and when to access services; and address hesitancy and concerns about vaccine safety or efficacy. Such interventions are interactive, and can be adapted to target particular populations or identified barriers.This is an update of a review originally published in 2013. OBJECTIVES To assess the effects of face-to-face interventions for informing or educating parents about early childhood vaccination on vaccination status and parental knowledge, attitudes and intention to vaccinate. SEARCH METHODS We searched the CENTRAL, MEDLINE, Embase, five other databases, and two trial registries (July and August 2017). We screened reference lists of relevant articles, and contacted authors of included studies and experts in the field. We had no language or date restrictions. SELECTION CRITERIA We included randomised controlled trials (RCTs) and cluster-RCTs evaluating the effects of face-to-face interventions delivered to parents or expectant parents to inform or educate them about early childhood vaccination, compared with control or with another face-to-face intervention. The World Health Organization recommends that children receive all early childhood vaccines, with the exception of human papillomavirus vaccine (HPV), which is delivered to adolescents. DATA COLLECTION AND ANALYSIS We used standard methodological procedures expected by Cochrane. Two authors independently reviewed all search results, extracted data and assessed the risk of bias of included studies. MAIN RESULTS In this update, we found four new studies, for a total of ten studies. We included seven RCTs and three cluster-RCTs involving a total of 4527 participants, although we were unable to pool the data from one cluster-RCT. Three of the ten studies were conducted in low- or middle- income countries.All included studies compared face-to-face interventions with control. Most studies evaluated the effectiveness of a single intervention session delivered to individual parents. The interventions were an even mix of short (ten minutes or less) and longer sessions (15 minutes to several hours).Overall, elements of the study designs put them at moderate to high risk of bias. All studies but one were at low risk of bias for sequence generation (i.e. used a random number sequence). For allocation concealment (i.e. the person randomising participants was unaware of the study group to which participant would be allocated), three were at high risk and one was judged at unclear risk of bias. Due to the educational nature of the intervention, blinding of participants and personnel was not possible in any studies. The risk of bias due to blinding of outcome assessors was judged as low for four studies. Most studies were at unclear risk of bias for incomplete outcome data and selective reporting. Other potential sources of bias included failure to account for clustering in a cluster-RCT and significant unexplained baseline differences between groups. One cluster-RCT was at high risk for selective recruitment of participants.We judged the certainty of the evidence to be low for the outcomes of children's vaccination status, parents' attitudes or beliefs, intention to vaccinate, adverse effects (e.g. anxiety), and immunisation cost, and moderate for parents' knowledge or understanding. All studies had limitations in design. We downgraded the certainty of the evidence where we judged that studies had problems with randomisation or allocation concealment, or when outcomes were self-reported by participants who knew whether they'd received the intervention or not. We also downgraded the certainty for inconsistency (vaccination status), imprecision (intention to vaccinate and adverse effects), and indirectness (attitudes or beliefs, and cost).Low-certainty evidence from seven studies (3004 participants) suggested that face-to-face interventions to inform or educate parents may improve vaccination status (risk ratio (RR) 1.20, 95% confidence interval (CI) 1.04 to 1.37). Moderate-certainty evidence from four studies (657 participants) found that face-to-face interventions probably slightly improved parent knowledge (standardised mean difference (SMD) 0.19, 95% CI 0.00 to 0.38), and low-certainty evidence from two studies (179 participants) suggested they may slightly improve intention to vaccinate (SMD 0.55, 95% CI 0.24 to 0.85). Low-certainty evidence found the interventions may lead to little or no change in parent attitudes or beliefs about vaccination (SMD 0.03, 95% CI -0.20 to 0.27; three studies, 292 participants), or in parents' anxiety (mean difference (MD) -1.93, 95% CI -7.27 to 3.41; one study, 90 participants). Only one study (365 participants) measured the intervention cost of a case management strategy, reporting that the estimated additional cost per fully immunised child for the intervention was approximately eight times higher than usual care (low-certainty evidence). No included studies reported outcomes associated with parents' experience of the intervention (e.g. satisfaction). AUTHORS' CONCLUSIONS There is low- to moderate-certainty evidence suggesting that face-to-face information or education may improve or slightly improve children's vaccination status, parents' knowledge, and parents' intention to vaccinate.Face-to-face interventions may be more effective in populations where lack of awareness or understanding of vaccination is identified as a barrier (e.g. where people are unaware of new or optional vaccines). The effect of the intervention in a population where concerns about vaccines or vaccine hesitancy is the primary barrier is less clear. Reliable and validated scales for measuring more complex outcomes, such as attitudes or beliefs, are necessary in order to improve comparisons of the effects across studies.
Collapse
Affiliation(s)
- Jessica Kaufman
- La Trobe UniversityCentre for Health Communication and Participation, School of Psychology and Public HealthBundooraVICAustralia3086
| | - Rebecca Ryan
- La Trobe UniversityCentre for Health Communication and Participation, School of Psychology and Public HealthBundooraVICAustralia3086
| | - Louisa Walsh
- La Trobe UniversityCentre for Health Communication and Participation, School of Psychology and Public HealthBundooraVICAustralia3086
| | - Dell Horey
- La Trobe UniversityCollege of Science, Health and EngineeringBundooraVICAustralia3086
| | - Julie Leask
- The University of SydneySydney Nursing SchoolSydneyNSWAustralia2050
| | - Priscilla Robinson
- La Trobe UniversityDepartment of Public Health, School of Psychology and Public HealthBundooraVICAustralia3086
| | - Sophie Hill
- La Trobe UniversityCentre for Health Communication and Participation, School of Psychology and Public HealthBundooraVICAustralia3086
| | | |
Collapse
|
10
|
"When you are injected, the baby is protected:" Assessing the acceptability of a maternal Tdap vaccine based on mothers' knowledge, attitudes, and beliefs of pertussis and vaccinations in Lusaka, Zambia. Vaccine 2018; 36:3048-3053. [PMID: 29653846 PMCID: PMC5952816 DOI: 10.1016/j.vaccine.2018.03.081] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2018] [Revised: 03/28/2018] [Accepted: 03/29/2018] [Indexed: 11/22/2022]
Abstract
INTRODUCTION Severe and fatal pertussis infections are concentrated among infants who are too young to be protected through routine vaccinations. Maternal Tdap (mTdap), which is now the standard of care in the US and UK, is considered to be the most effective way to address this gap in preventative care. Little is known about how pregnant women in low-resource settings might view mTdap. To inform strategies for mTdap implementation in these contexts, public health researchers sought to understand knowledge, attitudes, and beliefs toward pertussis and maternal vaccines and assess the barriers to vaccine acceptance. METHODS We conducted focus group discussions (FGDs) among mothers who participated in a longitudinal birth cohort study at the Chawama primary health center in Lusaka, Zambia. Since SAMIPS was not a clinical trial, but instead an observational cohort study, registration on clinicaltrials.gov was not required. Trained interviewers conducted the FGDs in January 2016 using a semi-structured interview guide, exploring participants' knowledge, attitudes and beliefs toward pertussis and vaccines. We analyzed the transcripts using Nvivo v.11 software. RESULTS Fifty mothers participated across 7 FGDs. Mothers had limited knowledge of pertussis and vaccines, yet expressed generally positive views of vaccinating themselves and their children. Participants conveyed very little vaccine hesitancy around maternal vaccinations, and discussed how they could protect their children's health. Mothers also highlighted barriers and facilitators to vaccine uptake, which included partner involvement, feelings of maternal authority over healthcare decision-making, and community rumors about Western medicine. CONCLUSION Mothers viewed vaccinations as an important method to keep their children healthy, despite cultural myths and misconceptions about pertussis and vaccines. FGD results suggest that vaccine acceptability is high in Zambia, which is a critical factor to vaccine uptake. A strategy addressing myths and misconceptions should be prioritized if and when mTdap is introduced across low-resource settings.
Collapse
|
11
|
Frew PM, Randall LA, Malik F, Limaye RJ, Wilson A, O'Leary ST, Salmon D, Donnelly M, Ault K, Dudley MZ, Fenimore VL, Omer SB. Clinician perspectives on strategies to improve patient maternal immunization acceptability in obstetrics and gynecology practice settings. Hum Vaccin Immunother 2018; 14:1548-1557. [PMID: 29313458 DOI: 10.1080/21645515.2018.1425116] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022] Open
Abstract
Pregnancy is an ideal time to communicate with women about vaccines for themselves and their infants, yet maternal immunization rates remain suboptimal. This study aimed to identify clinic, provider, and staff-related attributes and facilitators to be utilized for a comprehensive vaccine intervention in ob-gyn clinical settings. We conducted in-depth interviews with 24 providers, both healthcare providers (e.g., physicians, nurse practitioners, midwives) and practice managers, from urban and suburban ob-gyn practices in Georgia and Colorado about their immunization attitudes, practices, and patient experiences. Qualitative analyses included Pearson correlation tests to evaluate patterns and relationships within the data to determine themes. Six major themes emerged: 1) strong provider "buy in" for maternal immunization; 2) the supporting role of clinical/interpersonal cues for vaccine promotion; 3) varying provider-patient communication approaches and its influence on maternal and pediatric uptake; 4) an urgent need for a designated office immunization champion; 5) reimbursement and practice implementation challenges; and 6) region differences in attitudes and values toward maternal immunization. Although providers expressed strong support for maternal immunization practices and offered environmental cues for vaccine promotion, practices often lacked a designated, structured role for an immunization champion equipped to manage delicate conversations with patients. The findings reflect needs for immunization champion identification, training, and support, along with best practices guidelines to improve coordination of vaccine promotion and delivery efforts in ob-gyn provider offices. Additionally, provider training on communication approaches to enhance acceptance and uptake of maternal vaccines is warranted.
Collapse
Affiliation(s)
- Paula M Frew
- a Emory University School of Medicine , Department of Medicine, Division of Infectious Diseases , Atlanta , GA , USA.,b Emory University Rollins School of Public Health , Hubert Department of Global Health , Atlanta , GA , USA
| | - Laura A Randall
- a Emory University School of Medicine , Department of Medicine, Division of Infectious Diseases , Atlanta , GA , USA
| | - Fauzia Malik
- b Emory University Rollins School of Public Health , Hubert Department of Global Health , Atlanta , GA , USA
| | - Rupali J Limaye
- c Johns Hopkins Bloomberg School of Public Health , Department of International Health, Division of Global Disease Epidemiology and Control , Baltimore , MD , USA
| | - Andrew Wilson
- b Emory University Rollins School of Public Health , Hubert Department of Global Health , Atlanta , GA , USA
| | - Sean T O'Leary
- d University of Colorado Denver , Department of Pediatrics, Division of Infectious Diseases , Denver , CO , USA
| | - Daniel Salmon
- c Johns Hopkins Bloomberg School of Public Health , Department of International Health, Division of Global Disease Epidemiology and Control , Baltimore , MD , USA
| | - Meghan Donnelly
- e University of Colorado School of Medicine , Department of Obstetrics and Gynecology, Division of Maternal Fetal Medicine , Denver , CO , USA
| | - Kevin Ault
- f University of Kansas Medical Center , Department of Obstetrics and Gynecology , Kansas City , KS , USA
| | - Matthew Z Dudley
- c Johns Hopkins Bloomberg School of Public Health , Department of International Health, Division of Global Disease Epidemiology and Control , Baltimore , MD , USA
| | - Vincent L Fenimore
- a Emory University School of Medicine , Department of Medicine, Division of Infectious Diseases , Atlanta , GA , USA
| | - Saad B Omer
- b Emory University Rollins School of Public Health , Hubert Department of Global Health , Atlanta , GA , USA.,g Emory University Rollins School of Public Health , Department of Epidemiology , Atlanta , GA , USA.,h Emory University School of Medicine , Department of Medicine, Division of Pediatrics , Atlanta , GA , USA
| |
Collapse
|
12
|
Danchin MH, Costa-Pinto J, Attwell K, Willaby H, Wiley K, Hoq M, Leask J, Perrett KP, O'Keefe J, Giles ML, Marshall H. Vaccine decision-making begins in pregnancy: Correlation between vaccine concerns, intentions and maternal vaccination with subsequent childhood vaccine uptake. Vaccine 2017; 36:6473-6479. [PMID: 28811050 DOI: 10.1016/j.vaccine.2017.08.003] [Citation(s) in RCA: 106] [Impact Index Per Article: 15.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2017] [Revised: 06/09/2017] [Accepted: 08/02/2017] [Indexed: 11/15/2022]
Abstract
INTRODUCTION Maternal and childhood vaccine decision-making begins prenatally. Amongst pregnant Australian women we aimed to ascertain vaccine information received, maternal immunisation uptake and attitudes and concerns regarding childhood vaccination. We also aimed to determine any correlation between a) intentions and concerns regarding childhood vaccination, (b) concerns about pregnancy vaccination, (c) socioeconomic status (SES) and (d) uptake of influenza and pertussis vaccines during pregnancy and routine vaccines during childhood. METHODS Women attending public antenatal clinics were recruited in three Australian states. Surveys were completed on iPads. Follow-up phone surveys were done three to six months post delivery, and infant vaccination status obtained via the Australian Childhood Immunisation Register (ACIR). RESULTS Between October 2015 and March 2016, 975 (82%) of 1184 mothers consented and 406 (42%) agreed to a follow up survey, post delivery. First-time mothers (445; 49%) had significantly more vaccine concerns in pregnancy and only 73% had made a decision about childhood vaccination compared to 89% of mothers with existing children (p-value<0.001). 66% of mothers reported receiving enough information during pregnancy on childhood vaccination. In the post delivery survey, 46% and 82% of mothers reported receiving pregnancy influenza and pertussis vaccines respectively. The mother's degree of vaccine hesitancy and two attitudinal factors were correlated with vaccine uptake post delivery. There was no association between reported maternal vaccine uptake or SES and childhood vaccine uptake. CONCLUSION First time mothers are more vaccine hesitant and undecided about childhood vaccination, and only two thirds of all mothers believed they received enough information during pregnancy. New interventions to improve both education and communication on childhood and maternal vaccines, delivered by midwives and obstetricians in the Australian public hospital system, may reduce vaccine hesitancy for all mothers in pregnancy and post delivery, particularly first-time mothers.
Collapse
Affiliation(s)
- M H Danchin
- Vaccine and Immunisation Research Group, Murdoch Childrens Research Institute, Australia; Department of General Medicine, The Royal Children's Hospital, Australia; School of Population and Global Health, The University of Melbourne, Australia.
| | - J Costa-Pinto
- Department of General Medicine, The Royal Children's Hospital, Australia
| | - K Attwell
- Wesfarmers Centre of Vaccines and Infectious Diseases, Telethon Kids Institute, Western Australia, Australia; Sir Walter Murdoch School of Public Policy and International Affairs, Murdoch University, Australia
| | - H Willaby
- Sydney School of Public Health, Sydney Medical School, University of Sydney, Australia
| | - K Wiley
- National Centre for Immunisation Research and Surveillance, Australia
| | - M Hoq
- Clinical Epidemiology and Biostatistics Unit, Murdoch Childrens Research Institute, Australia
| | - J Leask
- Sydney School of Public Health, Sydney Medical School, University of Sydney, Australia; National Centre for Immunisation Research and Surveillance, Australia
| | - K P Perrett
- Vaccine and Immunisation Research Group, Murdoch Childrens Research Institute, Australia; Department of General Medicine, The Royal Children's Hospital, Australia; School of Population and Global Health, The University of Melbourne, Australia
| | - Jacinta O'Keefe
- Vaccine and Immunisation Research Group, Murdoch Childrens Research Institute, Australia
| | - M L Giles
- The Alfred Hospital, Royal Women's Hospital and Monash Health and Monash University, Australia
| | - H Marshall
- Women's and Children's Hospital and Robinson Research Institute, The University of Adelaide, South Australia, Australia
| |
Collapse
|
13
|
Abstract
Maternal immunization has undergone a paradigm shift in recent years, as women and healthcare providers accept and recognize the benefits of this strategy not only for the pregnant woman but also for the developing fetus and young infant. This article reviews the evidence for active immunization during pregnancy, with an emphasis on perinatal and infant outcomes. Current recommendations for immunization during pregnancy are presented, with particular focus on the routinely recommended vaccines during pregnancy: influenza and Tdap (tetanus, diphtheria, and pertussis). We discuss future research directions, maternal vaccines in development, and considerations for optimizing and advancing this underutilized strategy.
Collapse
|
14
|
Kriss JL, Frew PM, Cortes M, Malik FA, Chamberlain AT, Seib K, Flowers L, Ault KA, Howards PP, Orenstein WA, Omer SB. Evaluation of two vaccine education interventions to improve pertussis vaccination among pregnant African American women: A randomized controlled trial. Vaccine 2017; 35:1551-1558. [PMID: 28216190 DOI: 10.1016/j.vaccine.2017.01.037] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2016] [Revised: 01/13/2017] [Accepted: 01/17/2017] [Indexed: 11/27/2022]
Abstract
BACKGROUND Vaccination coverage with tetanus toxoid, reduced diphtheria toxoid, and acellular pertussis (Tdap) vaccine in pregnancy or immediately postpartum has been low. Limited data exist on rigorously evaluated interventions to increase maternal vaccination, including Tdap. Tailored messaging based on the Elaboration Likelihood Model (ELM) framework has been successful in improving uptake of some public health interventions. We evaluated the effect of two ELM-based vaccine educational interventions on Tdap vaccination among pregnant African American women, a group of women who tend to have lower vaccine uptake compared with other groups. METHODS We conducted a prospective randomized controlled trial to pilot test two interventions - an affective messaging video and a cognitive messaging iBook - among pregnant African American women recruited during routine prenatal care visits. We measured Tdap vaccination during the perinatal period (during pregnancy and immediately postpartum), reasons for non-vaccination, and intention to receive Tdap in the next pregnancy. RESULTS Among the enrolled women (n=106), 90% completed follow-up. Tdap vaccination in the perinatal period was 18% in the control group; 50% in the iBook group (Risk Ratio [vs. control group]: 2.83; 95% CI, 1.26-6.37), and 29% in the video group (RR: 1.65; 95% CI, 0.66-4.09). From baseline to follow-up, women's reported intention to receive Tdap during the next pregnancy improved in all three groups. Among unvaccinated women, the most common reason reported for non-vaccination was lack of a recommendation for Tdap by the woman's physician. CONCLUSIONS Education interventions that provide targeted information for pregnant women in an interactive manner may be useful to improve Tdap vaccination during the perinatal period. However, larger studies including multiple racial and ethnic groups are needed to evaluate robustness of our findings. TRIAL REGISTRATION clinicaltrials.gov Identifier: NCT01740310.
Collapse
Affiliation(s)
- Jennifer L Kriss
- Emory University, Department of Epidemiology and Laney Graduate School, Atlanta, GA, United States.
| | - Paula M Frew
- Emory University, Rollins School of Public Health, Department of Behavioral Sciences and Health Education, and Division of Infectious Diseases, Department of Medicine, Atlanta, GA, United States; Emory University, Rollins School of Public Health, Hubert Department of Global Health, Atlanta, GA, United States
| | - Marielysse Cortes
- Emory University, Rollins School of Public Health, Hubert Department of Global Health, Atlanta, GA, United States
| | - Fauzia A Malik
- Emory University, Rollins School of Public Health, Hubert Department of Global Health, Atlanta, GA, United States
| | - Allison T Chamberlain
- Emory University, Rollins School of Public Health, Department of Epidemiology, Atlanta, GA, United States
| | - Katherine Seib
- Emory University, Rollins School of Public Health, Hubert Department of Global Health, Atlanta, GA, United States
| | - Lisa Flowers
- Emory University, School of Medicine, Department of Gynecology and Obstetrics, Atlanta, GA, United States
| | - Kevin A Ault
- University of Kansas Medical Center, Department of Obstetrics and Gynecology, Kansas City, KS, United States
| | - Penelope P Howards
- Emory University, Rollins School of Public Health, Department of Epidemiology, Atlanta, GA, United States
| | - Walter A Orenstein
- Emory University, Emory Vaccine Center and School of Medicine, Atlanta, GA, United States
| | - Saad B Omer
- Emory University, Rollins School of Public Health, Hubert Department of Global Health, Atlanta, GA, United States; Emory University, Rollins School of Public Health, Department of Epidemiology, Atlanta, GA, United States; Emory University, Department of Pediatrics, Atlanta, GA, United States
| |
Collapse
|
15
|
Frew PM, Chung Y, Fisher AK, Schamel J, Basket MM. Parental experiences with vaccine information statements: Implications for timing, delivery, and parent-provider immunization communication. Vaccine 2016; 34:5840-5844. [PMID: 27789148 DOI: 10.1016/j.vaccine.2016.10.026] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2016] [Revised: 10/04/2016] [Accepted: 10/06/2016] [Indexed: 10/20/2022]
Abstract
OBJECTIVE We examined Vaccine Information Statements (VIS) dissemination practices and parental use and perceptions. METHODS We conducted a national online panel survey of 2603 US parents of children aged <7. Primary outcomes included reported VIS receipt, delivery timing, reading experiences, and perceived utility. RESULTS Most parents received a VIS (77.2%; [95% CI: 74.5-79.7%]), 59.7% [56.6-62.7%] before vaccination but 14.5% [12.5-16.8%] reported receiving it after their child's immunization; 15.1% [13.0-17.6%] were unsure of receipt status or timing; another 10.7% [9.0-12.6%] reported non-receipt of a VIS. Less than half who received a VIS before vaccination completed it before vaccination (46.2% [42.4, 50.0%]), but most who read at least some found the information useful (95.7% [93.8-97.0%]). Parents who delayed or refused at least one recommended non-influenza vaccine reported fewer opportunities to ask providers VIS questions. CONCLUSIONS Most parents report receiving VIS before vaccination as per federal guidelines. Continued effort is needed to enhance VIS distribution practice and parent-provider VIS content communication.
Collapse
Affiliation(s)
- Paula M Frew
- Emory University School of Medicine, Department of Medicine, Division of Infectious Diseases, United States; Emory University, Rollins School of Public Health, Department of Behavioral Sciences and Health Education, United States; Emory University, Rollins School of Public Health, Hubert Department of Global Health, United States; U.S. Centers for Disease Control and Prevention, National Center for Immunization and Respiratory Diseases, United States.
| | - Yunmi Chung
- Oak Ridge Institute for Science and Education (ORISE), United States
| | - Allison Kennedy Fisher
- U.S. Centers for Disease Control and Prevention, National Center for Immunization and Respiratory Diseases, United States
| | - Jay Schamel
- Oak Ridge Institute for Science and Education (ORISE), United States
| | - Michelle M Basket
- U.S. Centers for Disease Control and Prevention, National Center for Immunization and Respiratory Diseases, United States
| |
Collapse
|