The Food and Drug Administration's Post-Licensure Rapid Immunization Safety Monitoring program: strengthening the federal vaccine safety enterprise

Supporting Pharmacovigilance Signal Validation and Prioritization with Analyses of Routinely Collected Health Data: Lessons Learned from an EHDEN Network Study

INTRODUCTION

Individual case reports are the main asset in pharmacovigilance signal management. Signal validation is the first stage after signal detection and aims to determine if there is sufficient evidence to justify further assessment. Throughout signal management, a prioritization of signals is continually made. Routinely collected health data can provide relevant contextual information but are primarily used at a later stage in pharmacoepidemiological studies to assess communicated signals.

OBJECTIVE

The aim of this study was to examine the feasibility and utility of analysing routine health data from a multinational distributed network to support signal validation and prioritization and to reflect on key user requirements for these analyses to become an integral part of this process.

METHODS

Statistical signal detection was performed in VigiBase, the WHO global database of individual case safety reports, targeting generic manufacturer drugs and 16 prespecified adverse events. During a 5-day study-a-thon, signal validation and prioritization were performed using information from VigiBase, regulatory documents and the scientific literature alongside descriptive analyses of routine health data from 10 partners of the European Health Data and Evidence Network (EHDEN). Databases included in the study were from the UK, Spain, Norway, the Netherlands and Serbia, capturing records from primary care and/or hospitals.

RESULTS

Ninety-five statistical signals were subjected to signal validation, of which eight were considered for descriptive analyses in the routine health data. Design, execution and interpretation of results from these analyses took up to a few hours for each signal (of which 15-60 minutes were for execution) and informed decisions for five out of eight signals. The impact of insights from the routine health data varied and included possible alternative explanations, potential public health and clinical impact and feasibility of follow-up pharmacoepidemiological studies. Three signals were selected for signal assessment, two of these decisions were supported by insights from the routine health data. Standardization of analytical code, availability of adverse event phenotypes including bridges between different source vocabularies, and governance around the access and use of routine health data were identified as important aspects for future development.

CONCLUSIONS

Analyses of routine health data from a distributed network to support signal validation and prioritization are feasible in the given time limits and can inform decision making. The cost-benefit of integrating these analyses at this stage of signal management requires further research.

Vaccine Preparedness for the Next Influenza Pandemic: A Regulatory Perspective

The response to SARS-CoV-2 demonstrated the tremendous potential of investments in vaccine research and development to impact a global pandemic, resulting in the rapid development and deployment of lifesaving vaccines. However, this unprecedented speed was insufficient to either effectively combat initial waves of the pandemic or adapt in real time to new variants. This review focuses on opportunities from a public health oriented regulatory perspective for enhancing research, development, evaluation, production, and monitoring of safety and effectiveness to facilitate more rapid availability of pandemic influenza vaccines. We briefly review regulatory pathways and processes relevant to pandemic influenza, including how they can be strengthened and globally coordinated. We then focus on what we believe are critical opportunities to provide better approaches, tools, and methods to accelerate and improve vaccine development and evaluation and thus greatly enhance pandemic preparedness. In particular, for the improved vaccines needed to respond to a future influenza pandemic better and more rapidly, moving as much of the development and evaluation process as possible into the pre-pandemic period is critical, including through approval and use of analogous seasonal influenza vaccines with defined immune correlates of protection.

Risk of Myocarditis and Pericarditis among Young Adults following mRNA COVID-19 Vaccinations

There have been reports of cases of myocarditis and pericarditis as rare complications following mRNA COVID-19 vaccinations among young adults. While most reported cases are mild, this potential vaccine safety signal should be closely monitored. Using data from the CDC and the Vaccine Adverse Event Reporting System (VAERS), we calculated the combined reporting rate of myocarditis and pericarditis stratified by age group, sex, vaccine dose, and manufacturer, and compared these rates to the crude background incidence rates. Compared to the general population prior to the administration of the first COVID-19 vaccines in December 2020, we identified a higher-than-expected reporting rate of myocarditis and pericarditis following mRNA vaccination; the risk was higher after a second vaccine dose, higher in males than in females, and decreased with age. The highest risk was seen in males 12–17 years of age with approximately 6 cases per 100,000 second doses. Our findings suggest an increased risk of myocarditis and pericarditis in young males following a second dose of an mRNA COVID-19 vaccine. Since these findings are based on safety signals derived from passive surveillance data, confirmatory epidemiological studies should be undertaken.

The Use of Real-World Data to Assess the Impact of Safety-Related Regulatory Interventions

The regulation of medicines seeks to ensure the efficacy, safety, and quality of prescription and non-prescription medicines. Given that the conditions under which a medicine's benefits outweigh its risks are complex, it is essential that communications about the safe and effective use of medicines be clear and actionable. Assessing the impact of interventions to improve the safe and effective use of medicines is a developing area, and one in which real-world data are playing an increasingly important role. Although real-world data are commonly used to assess the impact of regulatory interventions, there are several areas where their use could be improved. Specific areas for improvement include assessing regulatory interventions across a wider range of medicines, rather than concentrating on a relatively few therapeutic areas; assessing more clinically relevant outcomes rather than relying on measures such as changes in the number of prescriptions, which may not always correlate with the desired impact; assessing the potential unintended or negative consequences of regulatory interventions; applying methods to address potential confounders; assessing long-term, rather than just short-term, impacts of an intervention; increasing the use of comparator groups, when feasible; and evaluating the impact of regulatory interventions from multiple dimensions, rather than from a single dimension. Expanded use of real-world data could inform some of these efforts, although data sources beyond administrative claims data will likely be necessary to achieve all these goals.

Vaccine Safety Datalink infrastructure enhancements for evaluating the safety of maternal vaccination

BACKGROUND

Identifying pregnancy episodes and accurately estimating their beginning and end dates are imperative for observational maternal vaccine safety studies using electronic health record (EHR) data.

METHODS

We modified the Vaccine Safety Datalink (VSD) Pregnancy Episode Algorithm (PEA) to include both the International Classification of Disease, ninth revision (ICD-9 system) and ICD-10 diagnosis codes, incorporated additional gestational age data, and validated this enhanced algorithm with manual medical record review. We also developed the new Dynamic Pregnancy Algorithm (DPA) to identify pregnancy episodes in real time.

RESULTS

Around 75% of the pregnancy episodes identified by the enhanced VSD PEA were live births, 12% were spontaneous abortions (SABs), 10% were induced abortions (IABs), and 0.4% were stillbirths (SBs). Gestational age was identified for 99% of live births, 89% of SBs, 69% of SABs, and 42% of IABs. Agreement between the PEA-assigned and abstractor-identified pregnancy outcome and outcome date was 100% for live births, but was lower for pregnancy losses. When gestational age was available in the medical record, the agreement was higher for live births (97%), but lower for pregnancy losses (75%). The DPA demonstrated strong concordance with the PEA and identified pregnancy episodes ⩾6 months prior to the outcome date for 89% of live births.

CONCLUSION

The enhanced VSD PEA is a useful tool for identifying pregnancy episodes in EHR databases. The DPA improves the timeliness of pregnancy identification and can be used for near real-time maternal vaccine safety studies.

PLAIN LANGUAGE SUMMARY

Improving identification of pregnancies in the Vaccine Safety Datalink electronic medical record databases to allow for better and faster monitoring of vaccination safety during pregnancy Introduction: It is important to monitor of the safety of vaccines after they have been approved and licensed by the Food and Drug Administration, especially among women vaccinated during pregnancy. The Vaccine Safety Datalink (VSD) monitors vaccine safety through observational studies within large databases of electronic medical records. Since 2012, VSD researchers have used an algorithm called the Pregnancy Episode Algorithm (PEA) to identify the medical records of women who have been pregnant. Researchers then use these medical records to study whether receiving a particular vaccine is linked to any negative outcomes for the woman or her child.Methods: The goal of this study was to update and enhance the PEA to include the full set of medical record diagnostic codes [both from the older International Classification of Disease, ninth revision (ICD-9 system) and the newer ICD-10 system] and to incorporate additional sources of data about gestational age. To ensure the validity of the PEA following these enhancements, we manually reviewed medical records and compared the results with the algorithm. We also developed a new algorithm, the Dynamic Pregnancy Algorithm (DPA), to identify women earlier in pregnancy, allowing us to conduct more timely vaccine safety assessments.Results: The new version of the PEA identified 2,485,410 pregnancies in the VSD database. The enhanced algorithm more precisely estimated the beginning of pregnancies, especially those that did not result in live births, due to the new sources of gestational age data.Conclusion: Our new algorithm, the DPA, was successful at identifying pregnancies earlier in gestation than the PEA. The enhanced PEA and the new DPA will allow us to better evaluate the safety of current and future vaccinations administered during or around the time of pregnancy.

Active Vaccine Safety Surveillance: Global Trends and Challenges in China

Importance. The great success in vaccine-preventable diseases has been accompanied by vaccine safety concerns. This has caused vaccine hesitancy to be the top 10 in threats to global health. The comprehensive understanding of adverse events following immunization should be entirely based on clinical trials and postapproval surveillance. It has increasingly been recognized worldwide that the active surveillance of vaccine safety should be an essential part of immunization programs due to its complementary advantages to passive surveillance and clinical trials.Highlights. In the present study, the framework of vaccine safety surveillance was summarized to illustrate the importance of active surveillance and address vaccine hesitancy or safety concerns. Then, the global progress of active surveillance systems was reviewed, mainly focusing on population-based or hospital-based active surveillance. With these successful paradigms, the practical and reliable ways to create robust and similar systems in China were discussed and presented from the perspective of available databases, methodology challenges, policy supports, and ethical considerations.Conclusion. In the inevitable trend of the global vaccine safety ecosystem, the establishment of an active surveillance system for vaccine safety in China is urgent and feasible. This process can be accelerated with the consensus and cooperation of regulatory departments, research institutions, and data owners.

Challenges in Phase 4 post-licensure safety studies using real world data in the United States: Hepatitis B vaccine example

Post-licensure vaccine safety studies are essential to identify adverse events that may not have been detected in pre-licensure clinical trials and to address questions that arose during the pre-licensure phase. These studies are increasingly conducted using real-world data collected as part of routine health care delivery. However, design of post-licensure vaccine safety studies involves many pragmatic and scientific decisions, which must be made while balancing diverse stakeholder opinions. Challenges include selecting exposure and comparison groups, deciding on the most appropriate outcome, determining sample size and length of follow-up time, and other analytic considerations. As an example of this process and to inform other post-licensure vaccine safety studies in real-world settings, we discuss our experience with design of an FDA-required Phase 4 post-licensure safety study of a hepatitis B vaccine in a large integrated health care organization in the United States.

Safety of vaccines used for routine immunization in the United States: An updated systematic review and meta-analysis

BACKGROUND

Understanding the safety of vaccines is critical to inform decisions about vaccination. Our objective was to conduct a systematic review of the safety of vaccines recommended for children, adults, and pregnant women in the United States.

METHODS

We searched the literature in November 2020 to update a 2014 Agency for Healthcare Research and Quality review by integrating newly available data. Studies of vaccines that used a comparator and reported the presence or absence of key adverse events were eligible. Adhering to Evidence-based Practice Center methodology, we assessed the strength of evidence (SoE) for all evidence statements. The systematic review is registered in PROSPERO (CRD42020180089).

RESULTS

Of 56,603 reviewed citations, 338 studies reported in 518 publications met inclusion criteria. For children, SoE was high for no increased risk of autism following measles, mumps, and rubella (MMR) vaccine. SoE was high for increased risk of febrile seizures with MMR. There was no evidence of increased risk of  intussusception with rotavirus vaccine at the latest follow-up (moderate SoE), nor of diabetes (high SoE). There was no evidence of increased risk or insufficient evidence for key adverse events for newer vaccines such as 9-valent human papillomavirus and meningococcal B vaccines. For adults, there was no evidence of increased risk (varied SoE) or insufficient evidence for key adverse events for the new adjuvanted inactivated influenza vaccine and recombinant adjuvanted zoster vaccine. We found no evidence of increased risk (varied SoE) for key adverse events among pregnant women following tetanus, diphtheria, and acellular pertussis vaccine, including stillbirth (moderate SoE).

CONCLUSIONS

Across a large body of research we found few associations of vaccines and serious key adverse events; however, rare events are challenging to study. Any adverse events should be weighed against the protective benefits that vaccines provide.

Suitability of databases in the Asia-Pacific for collaborative monitoring of vaccine safety

INTRODUCTION

Information regarding availability of electronic healthcare databases in the Asia-Pacific region is critical for planning vaccine safety assessments particularly, as COVID-19 vaccines are introduced. This study aimed to identify data sources in the region, potentially suitable for vaccine safety surveillance. This manuscript is endorsed by the International Society for Pharmacoepidemiology (ISPE).

METHODS

Nineteen countries targeted for database reporting were identified using published country lists and review articles. Surveillance capacity was assessed using two surveys: a 9-item introductory survey and a 51-item full survey. Survey questions related to database characteristics, covariate and health outcome variables, vaccine exposure characteristics, access and governance, and dataset linkage capability. Other questions collated research/regulatory applications of the data and local publications detailing database use for research.

RESULTS

Eleven databases containing vaccine-specific information were identified across 8 countries. Databases were largely national in coverage (8/11, 73%), encompassed all ages (9/11, 82%) with population size from 1.4 to 52 million persons. Vaccine exposure information varied particularly for standardized vaccine codes (5/11, 46%), brand (7/11, 64%) and manufacturer (5/11, 46%). Outcome data were integrated with vaccine data in 6 (55%) databases and available via linkage in 5 (46%) databases. Data approval processes varied, impacting on timeliness of data access.

CONCLUSIONS

Variation in vaccine data availability, complexities in data access including, governance and data release approval procedures, together with requirement for data linkage for outcome information, all contribute to the challenges in building a distributed network for vaccine safety assessment in the Asia-Pacific and globally. Common data models (CDMs) may help expedite vaccine safety research across the region.

Real-World Evidence for Assessing Pharmaceutical Treatments in the Context of COVID-19

The emergence and global spread of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has resulted in an urgent need for evidence on medical interventions and outcomes of the resulting disease, coronavirus disease 2019 (COVID-19). Although many randomized controlled trials (RCTs) evaluating treatments and vaccines for COVID-19 are already in progress, the number of clinical questions of interest greatly outpaces the available resources to conduct RCTs. Therefore, there is growing interest in whether nonrandomized real-world evidence (RWE) can be used to supplement RCT evidence and aid in clinical decision making, but concerns about nonrandomized RWE have been highlighted by a proliferation of RWE studies on medications and COVID-19 outcomes with widely varying conclusions. The objective of this paper is to review some clinical questions of interest, potential data types, challenges, and merits of RWE in COVID-19, resulting in recommendations for nonrandomized RWE designs and analyses based on established RWE principles.

Leveraging the Capabilities of the FDA's Sentinel System To Improve Kidney Care

The Sentinel System is a national electronic postmarketing resource established by the US Food and Drug Administration to support assessment of the safety and effectiveness of marketed medical products. It has built a large, multi-institutional, distributed data network that contains comprehensive electronic health data, covering about 700 million person-years of longitudinal observation time nationwide. With its sophisticated infrastructure and a large selection of flexible analytic tools, the Sentinel System permits rapid and secure analyses, while preserving patient privacy and health-system autonomy. The Sentinel System also offers enhanced capabilities, including accessing full-text medical records, supporting randomized clinical trials embedded in healthcare delivery systems, and facilitating effective collection of patient-reported data using mobile devices, among many other research programs. The nephrology research community can use the infrastructure, tools, and data that this national resource offers for evidence generation. This review summarizes the Sentinel System and its ability to rapidly generate high-quality, real-world evidence; discusses the program's experience in, and potential for, addressing gaps in kidney care; and outlines avenues for conducting research, leveraging this national resource in collaboration with Sentinel investigators.

The role of epidemiology in informing United States childhood immunization policy and practice

One of the ten greatest public health achievements is childhood vaccination because of its impact on controlling and eliminating vaccine-preventable diseases (VPDs). Evidence-based immunization policies and practices are responsible for this success and are supported by epidemiology that has generated scientific evidence for informing policy and practice. The purpose of this report is to highlight the role of epidemiology in the development of immunization policy and successful intervention in public health practice that has resulted in a measurable public health impact: the control and elimination of VPDs in the United States. Examples in which epidemiology informed immunization policy were collected from a literature review and consultation with experts who have been working in this field for the past 30 years. Epidemiologic examples (e.g., thimerosal-containing vaccines and the alleged association between the measles, mumps, and rubella (MMR) vaccine and autism) are presented to describe challenges that epidemiologists have addressed. Finally, we describe ongoing challenges to the nation's ability to sustain high vaccination coverage, particularly with concerns about vaccine safety and effectiveness, increasing use of religious and philosophical belief exemptions to vaccination, and vaccine hesitancy. Learning from past and current experiences may help epidemiologists anticipate and address current and future challenges to respond to emerging infectious diseases, such as COVID-19, with new vaccines and enhance the public health impact of immunization programs for years to come.

Active Surveillance of Adverse Events Following Human Papillomavirus Vaccination: Feasibility Pilot Study Based on the Regional Health Care Information Platform in the City of Ningbo, China

BACKGROUND

Comprehensive safety data for vaccines from post-licensure surveillance, especially active surveillance, could guide administrations and individuals to make reasonable decisions on vaccination. Therefore, we designed a pilot study to assess the capability of a regional health care information platform to actively monitor the safety of a newly licensed vaccine.

OBJECTIVE

This study aimed to conduct active surveillance of human papillomavirus (HPV) vaccine safety based on this information platform.

METHODS

In 2017, one of China's most mature information platforms with superior data linkage was selected. A structured questionnaire and open-ended interview guidelines were developed to investigate the feasibility of active surveillance following HPV vaccination using the regional health care information platform in Ningbo. The questionnaire was sent to participants via email, and a face-to-face interview was conducted to confirm details or resolve discrepancies.

RESULTS

Five databases that could be considered essential to active surveillance of vaccine safety were integrated into the platform starting in 2015. Except for residents' health records, which had a coverage rate of 87%, the data sources covered more than 95% of the records that were documented in Ningbo. All the data could be inherently linked using the national identity card. There were 19,328 women who received the HPV vaccine, and 37,988 doses were administered in 2017 and 2018. Women aged 30-40 years accounted for the largest proportion. Quadrivalent vaccination accounted for 73.1% of total vaccination, a much higher proportion than that of bivalent vaccination. Of the first doses, 60 (60/19,328, 0.31%) occurred outside Ningbo. There were no missing data for vaccination-relevant variables, such as identity card, vaccine name, vaccination doses, vaccination date, and manufacturer. ICD-10 coding could be used to identify 9,180 cases using a predefined list of the outcomes of interest, and 1.88% of these cases were missing the identity card. During the 90 days following HPV vaccination, 4 incident cases were found through the linked vaccination history and electronic medical records. The combined incident rate of rheumatoid arthritis, optic neuritis, and Henoch-Schonlein purpura was 8.84/100,000 doses of bivalent HPV, and the incidence rate of rheumatoid arthritis was 3.75/100,000 doses of quadrivalent HPV.

CONCLUSIONS

This study presents an available approach to initiate an active surveillance system for adverse events following HPV vaccination, based on a regional health care information platform in China. An extended observation period or the inclusion of additional functional sites is warranted to conduct future hypothesis-generating and hypothesis-confirming studies for vaccine safety concerns.

Big Data in the Assessment of Pediatric Medication Safety

Big data (BD) in pediatric medication safety research provides many opportunities to improve the safety and health of children. The number of pediatric medication and device trials has increased in part because of the past 20 years of US legislation requiring and incentivizing study of the effects of medical products in children (Food and Drug Administration Modernization Act of 1997, Pediatric Rule in 1998, Best Pharmaceuticals for Children Act of 2002, and Pediatric Research Equity Act of 2003). There are some limitations of traditional approaches to studying medication safety in children. Randomized clinical trials within the regulatory context may not enroll patients who are representative of the general pediatric population, provide the power to detect rare safety signals, or provide long-term safety data. BD sources may have these capabilities. In recent years, medical records have become digitized, and cell phones and personal devices have proliferated. In this process, the field of biomedical science has progressively used BD from those records coupled with other data sources, both digital and traditional. Additionally, large distributed databases that include pediatric-specific outcome variables are available. A workshop entitled "Advancing the Development of Pediatric Therapeutics: Application of 'Big Data' to Pediatric Safety Studies" held September 18 to 19, 2017, in Silver Spring, Maryland, formed the basis of many of the ideas outlined in this article, which are intended to identify key examples, critical issues, and future directions in this early phase of an anticipated dramatic change in the availability and use of BD.

The science of vaccine safety: Summary of meeting at Wellcome Trust

Vaccines are everywhere hugely successful but are also under attack. The reason for the latter is the perception by some people that vaccines are unsafe. However that may be, vaccine safety, life any other scientific subject, must be constantly studied. It was from this point of view that a meeting was organized at the Wellcome Trust in London in May 2019 to assess some aspects of vaccine safety as subjects for scientific study. The objective of the meeting was to assess what is known beyond reasonable doubt and conversely what areas need additional studies. Although the meeting could not cover all aspects of vaccine safety science, many of the most important issues were addressed by a group of about 30 experts to determine what is already known and what additional studies are merited to assess the safety of the vaccines currently in use. The meeting began with reviews of the current situation in different parts of the world, followed by reviews of specific controversial areas, including the incidence of certain conditions after vaccination and the safety of certain vaccine components. Lastly, information about the human papillomavirus vaccine was considered because its safety has been particularly challenged by vaccine opponents. The following is a summary of the meeting findings. In addition to this summary, the meeting organizers will explore opportunities to perform studies that would enlarge knowledge of vaccine safety.

Vaccinating pregnant women against influenza needs to be a priority for all countries: An expert commentary

BACKGROUND

In 2012, the World Health Organization recommended influenza vaccination for all pregnant women worldwide and the prioritisation of pregnant women in national influenza vaccination programmes. Nevertheless, vaccination rates in pregnant women often remain much lower than national targets.

OBJECTIVES

To assess the benefits and risks associated with influenza infection and vaccination during pregnancy, and to consider obstacles that work against influenza vaccine uptake during pregnancy.

RESULTS

There is strong evidence that maternal and foetal outcomes can be compromised if women develop influenza infections during pregnancy. Influenza vaccines have been administered to millions of pregnant women and have demonstrated benefits in terms of disease prevention in mothers and their infants. There is a consensus amongst several recommending authorities that influenza vaccines may be safely administered during all stages of pregnancy. Healthcare professionals are recognised as the most important influencers of vaccine uptake, being well placed to recommend vaccination and directly address safety concerns.

CONCLUSIONS

Despite data supporting the value of influenza vaccination during pregnancy, vaccine uptake remains low globally. Low uptake appears to be largely due to ineffective communication with pregnant women about the risks and benefits of influenza vaccination. A graphical abstract is available online.

ADVANCE system testing: Benefit-risk analysis of a marketed vaccine using multi-criteria decision analysis and individual-level state transition modelling

BACKGROUND

The Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public-private collaboration aiming to develop and test a system for rapid benefit-risk (B/R) monitoring of vaccines using electronic health record (eHR) databases in Europe. Proof-of-concept studies were designed to assess the proposed processes and system for generating the required evidence to perform B/R assessment and near-real time monitoring of vaccines. We aimed to test B/R methodologies for vaccines, using the comparison of the B/R profiles of whole-cell (wP) and acellular pertussis (aP) vaccine formulations in children as an example.

METHODS

We used multi-criteria decision analysis (MCDA) to structure the B/R assessment combined with individual-level state transition modelling to build the B/R effects table. In the state transition model, we simulated the number of events in two hypothetical cohorts of 1 million children followed from first pertussis dose till pre-school-entry booster (or six years of age, whichever occurred first), with one cohort receiving wP, and the other aP. The benefits were reductions in pertussis incidence and complications. The risks were increased incidences of febrile convulsions, fever, hypotonic-hyporesponsive episodes, injection-site reactions and persistent crying. Most model parameters were informed by estimates (coverage, background incidences, relative risks) from eHR databases from Denmark (SSI), Spain (BIFAP and SIDIAP), Italy (Pedianet) and the UK (RCGP-RSC and THIN). Preferences were elicited from clinical and epidemiological experts.

RESULTS

Using state transition modelling to build the B/R effects table facilitated the comparison of different vaccine effects (e.g. immediate vaccine risks vs long-term vaccine benefits). Estimates from eHR databases could be used to inform the simulation model. The model results could be easily combined with preference weights to obtain B/R scores.

CONCLUSION

Existing B/R methodology, modelling and estimates from eHR databases can be successfully used for B/R assessment of vaccines.

Leveraging the entire cohort in drug safety monitoring: part 1 methods for sequential surveillance that use regression adjustment or weighting to control confounding in a multisite, rare event, distributed data setting

OBJECTIVE

Study designs involving self-controlled or exposure-matched samples are commonly used to monitor postmarket vaccine and drug safety, and they use a subset of the available larger cohort. This article overviews group sequential methods designed for observational data safety monitoring that use the whole exposed and unexposed cohorts by implementing regression adjustment or weighting to control confounding.

METHODS

We summarize what is known about the performance of "whole cohort" methods in multisite health plan data networks such as the Sentinel System of the Food and Drug Administration, where outcomes are rare, individual-level patient data cannot be pooled across sites, site heterogeneity is large, and data are dynamically updated over time.

RESULTS

Group sequential estimation and testing methods that use regression or weighting can flexibly handle electronic health care data's unpredictability, including an uncertain rate of new product uptake, variable composition of the population over time, and data changes due to dynamic administrative updates. Regression and weighting methods generally have higher power, faster signal detection, and fewer practical challenges compared with some design-based confounder adjustment methods.

CONCLUSION

Group sequential regression adjustment and weighting approaches are feasible and underused in practice. They leverage more information than designs that involved sampling and increase power to detect rare adverse effects without increasing bias.

Applying sequential surveillance methods that use regression adjustment or weighting to control confounding in a multisite, rare-event, distributed setting: Part 2 in-depth example of a reanalysis of the measles-mumps-rubella-varicella combination vaccine and seizure risk

OBJECTIVE

In-depth example of two new group sequential methods for postmarket safety monitoring of new medical products.

STUDY DESIGN AND SETTING

Existing trial-based group sequential approaches have been extended to adjust for confounders, accommodate rare events, and address privacy-related constraints on data sharing. Most adaptations have involved design-based confounder strategies, for example, self-controlled or exposure matching, while analysis-based approaches like regression and weighting have received less attention. We describe the methodology of two new group sequential approaches that use analysis-based confounder adjustment (GS GEE) and weighting (GS IPTW). Using data from the Food and Drug Administration's Sentinel network, we apply both methods in the context of a known positive association: the measles-mumps-rubella-varicella vaccine and seizure risk in infants.

RESULTS

Estimates from both new approaches were similar and comparable to prior studies using design-based methods to address confounding. The time to detection of a safety signal was considerably shorter for GS IPTW, which estimates a risk difference, compared to GS GEE, which provides relative estimates of excess risk.

CONCLUSION

Future group sequential safety surveillance efforts should consider analysis-based confounder adjustment techniques that evaluate safety signals on the risk difference scale to achieve greater statistical power and more timely results.

The MMR Vaccine and Autism

Autism is a developmental disability that can cause significant social, communication, and behavioral challenges. A report published in 1998, but subsequently retracted by the journal, suggested that measles, mumps, and rubella (MMR) vaccine causes autism. However, autism is a neurodevelopmental condition that has a strong genetic component with genesis before one year of age, when MMR vaccine is typically administered. Several epidemiologic studies have not found an association between MMR vaccination and autism, including a study that found that MMR vaccine was not associated with an increased risk of autism even among high-risk children whose older siblings had autism. Despite strong evidence of its safety, some parents are still hesitant to accept MMR vaccination of their children. Decreasing acceptance of MMR vaccination has led to outbreaks or resurgence of measles. Health-care providers have a vital role in maintaining confidence in vaccination and preventing suffering, disability, and death from measles and other vaccine-preventable diseases.

Vaccination and risk of lone atrial fibrillation in the active component United States military

PURPOSE

To evaluate the hypothesis that receipt of anthrax vaccine adsorbed (AVA) increases the risk of atrial fibrillation in the absence of identifiable underlying risk factors or structural heart disease (lone atrial fibrillation).

METHODS

We conducted a retrospective population-based cohort study among U.S. military personnel who were on active duty during the period from January 1, 1998 through December 31, 2006. International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes were used to identify individuals diagnosed with atrial fibrillation in the Defense Medical Surveillance System, and electronic records were screened to include only individuals without evidence of predisposing medical conditions. We used multivariable Poisson regression to estimate the risk of lone atrial fibrillation after exposure to AVA. We also evaluated possible associations with influenza and smallpox vaccines.

RESULTS

Our study population consisted of 2,957,091individuals followed for 11,329,746 person-years of service. Of these, 2,435 met our case definition for lone atrial fibrillation, contributing approximately 8,383 person-years of service. 1,062,176 (36%) individuals received at least one dose of AVA; the median person time observed post-exposure was 3.6 years. We found no elevated risk of diagnosed lone atrial fibrillation associated with AVA (adjusted risk ratio = 0.99; 95% confidence interval = 0.90, 1.09; p = 0.84). No elevated risk was observed for lone atrial fibrillation associated with influenza or smallpox vaccines given during military service.

CONCLUSIONS

We did not find an increased risk of lone atrial fibrillation after AVA, influenza or smallpox vaccine. These findings may be helpful in planning future vaccine safety research.

Bias due to differential and non-differential disease- and exposure misclassification in studies of vaccine effectiveness

BACKGROUND

Studies of vaccine effectiveness (VE) rely on accurate identification of vaccination and cases of vaccine-preventable disease. In practice, diagnostic tests, clinical case definitions and vaccination records often present inaccuracies, leading to biased VE estimates. Previous studies investigated the impact of non-differential disease misclassification on VE estimation.

METHODS

We explored, through simulation, the impact of non-differential and differential disease- and exposure misclassification when estimating VE using cohort, case-control, test-negative case-control and case-cohort designs. The impact of misclassification on the estimated VE is demonstrated for VE studies on childhood seasonal influenza and pertussis vaccination. We additionally developed a web-application graphically presenting bias for user-selected parameters.

RESULTS

Depending on the scenario, the misclassification parameters had differing impacts. Decreased exposure specificity had greatest impact for influenza VE estimation when vaccination coverage was low. Decreased exposure sensitivity had greatest impact for pertussis VE estimation for which high vaccination coverage is typically achieved. The impact of the exposure misclassification parameters was found to be more noticeable than that of the disease misclassification parameters. When misclassification is limited, all study designs perform equally. In case of substantial (differential) disease misclassification, the test-negative design performs worse.

CONCLUSIONS

Misclassification can lead to significant bias in VE estimates and its impact strongly depends on the scenario. We developed a web-application for assessing the potential (joint) impact of possibly differential disease- and exposure misclassification that can be modified by users to their own study scenario. Our results and the simulation tool may be used to guide better design, conduct and interpretation of future VE studies.

Assessment of Quadrivalent Human Papillomavirus Vaccine Safety Using the Self-Controlled Tree-Temporal Scan Statistic Signal-Detection Method in the Sentinel System

The self-controlled tree-temporal scan statistic-a new signal-detection method-can evaluate whether any of a wide variety of health outcomes are temporally associated with receipt of a specific vaccine, while adjusting for multiple testing. Neither health outcomes nor postvaccination potential periods of increased risk need be prespecified. Using US medical claims data in the Food and Drug Administration's Sentinel system, we employed the method to evaluate adverse events occurring after receipt of quadrivalent human papillomavirus vaccine (4vHPV). Incident outcomes recorded in emergency department or inpatient settings within 56 days after first doses of 4vHPV received by 9- through 26.9-year-olds in 2006-2014 were identified using International Classification of Diseases, Ninth Revision, diagnosis codes and analyzed by pairing the new method with a standard hierarchical classification of diagnoses. On scanning diagnoses of 1.9 million 4vHPV recipients, 2 statistically significant categories of adverse events were found: cellulitis on days 2-3 after vaccination and "other complications of surgical and medical procedures" on days 1-3 after vaccination. Cellulitis is a known adverse event. Clinically informed investigation of electronic claims records of the patients with "other complications" did not suggest any previously unknown vaccine safety problem. Considering that thousands of potential short-term adverse events and hundreds of potential risk intervals were evaluated, these findings add significantly to the growing safety record of 4vHPV.

Analyzing self-controlled case series data when case confirmation rates are estimated from an internal validation sample

Vaccine safety studies are often electronic health record (EHR)-based observational studies. These studies often face significant methodological challenges, including confounding and misclassification of adverse event. Vaccine safety researchers use self-controlled case series (SCCS) study design to handle confounding effect and employ medical chart review to ascertain cases that are identified using EHR data. However, for common adverse events, limited resources often make it impossible to adjudicate all adverse events observed in electronic data. In this paper, we considered four approaches for analyzing SCCS data with confirmation rates estimated from an internal validation sample: (1) observed cases, (2) confirmed cases only, (3) known confirmation rate, and (4) multiple imputation (MI). We conducted a simulation study to evaluate these four approaches using type I error rates, percent bias, and empirical power. Our simulation results suggest that when misclassification of adverse events is present, approaches such as observed cases, confirmed case only, and known confirmation rate may inflate the type I error, yield biased point estimates, and affect statistical power. The multiple imputation approach considers the uncertainty of estimated confirmation rates from an internal validation sample, yields a proper type I error rate, largely unbiased point estimate, proper variance estimate, and statistical power.

Bias from outcome misclassification in immunization schedule safety research

PURPOSE

The Institute of Medicine recommended conducting observational studies of childhood immunization schedule safety. Such studies could be biased by outcome misclassification, leading to incorrect inferences. Using simulations, we evaluated (1) outcome positive predictive values (PPVs) as indicators of bias of an exposure-outcome association, and (2) quantitative bias analyses (QBA) for bias correction.

METHODS

Simulations were conducted based on proposed or ongoing Vaccine Safety Datalink studies. We simulated 4 studies of 2 exposure groups (children with no vaccines or on alternative schedules) and 2 baseline outcome levels (100 and 1000/100 000 person-years), with 3 relative risk (RR) levels (RR = 0.50, 1.00, and 2.00), across 1000 replications using probabilistic modeling. We quantified bias from non-differential and differential outcome misclassification, based on levels previously measured in database research (sensitivity > 95%; specificity > 99%). We calculated median outcome PPVs, median observed RRs, Type 1 error, and bias-corrected RRs following QBA.

RESULTS

We observed PPVs from 34% to 98%. With non-differential misclassification and true RR = 2.00, median bias was toward the null, with severe bias (median observed RR = 1.33) with PPV = 34% and modest bias (median observed RR = 1.83) with PPV = 83%. With differential misclassification, PPVs did not reflect median bias, and there was Type 1 error of 100% with PPV = 90%. QBA was generally effective in correcting misclassification bias.

CONCLUSIONS

In immunization schedule studies, outcome misclassification may be non-differential or differential to exposure. Overall outcome PPVs do not reflect the distribution of false positives by exposure and are poor indicators of bias in individual studies. Our results support QBA for immunization schedule safety research.

Development and application of two semi-automated tools for targeted medical product surveillance in a distributed data network

PURPOSE OF REVIEW

An important component of the Food and Drug Administration's Sentinel Initiative is the active post-market risk identification and analysis (ARIA) system, which utilizes semi-automated, parameterized computer programs to implement propensity-score adjusted and self-controlled risk interval designs to conduct targeted surveillance of medical products in the Sentinel Distributed Database. In this manuscript, we review literature relevant to the development of these programs and describe their application within the Sentinel Initiative.

RECENT FINDINGS

These quality-checked and publicly available tools have been successfully used to conduct rapid, replicable, and targeted safety analyses of several medical products. In addition to speed and reproducibility, use of semi-automated tools allows investigators to focus on decisions regarding key methodological parameters. We also identified challenges associated with the use of these methods in distributed and prospective datasets like the Sentinel Distributed Database, namely uncertainty regarding the optimal approach to estimating propensity scores in dynamic data among data partners of heterogeneous size.

SUMMARY

Future research should focus on the methodological challenges raised by these applications as well as developing new modular programs for targeted surveillance of medical products.

A Synthesis of Current Surveillance Planning Methods for the Sequential Monitoring of Drug and Vaccine Adverse Effects Using Electronic Health Care Data

INTRODUCTION

The large-scale assembly of electronic health care data combined with the use of sequential monitoring has made proactive postmarket drug- and vaccine-safety surveillance possible. Although sequential designs have been used extensively in randomized trials, less attention has been given to methods for applying them in observational electronic health care database settings.

EXISTING METHODS

We review current sequential-surveillance planning methods from randomized trials, and the Vaccine Safety Datalink (VSD) and Mini-Sentinel Pilot projects-two national observational electronic health care database safety monitoring programs.

FUTURE SURVEILLANCE PLANNING

Based on this examination, we suggest three steps for future surveillance planning in health care databases: (1) prespecify the sequential design and analysis plan, using available feasibility data to reduce assumptions and minimize later changes to initial plans; (2) assess existing drug or vaccine uptake, to determine if there is adequate information to proceed with surveillance, before conducting more resource-intensive planning; and (3) statistically evaluate and clearly communicate the sequential design with all those designing and interpreting the safety-surveillance results prior to implementation. Plans should also be flexible enough to accommodate dynamic and often unpredictable changes to the database information made by the health plans for administrative purposes.

CONCLUSIONS

This paper is intended to encourage dialogue about establishing a more systematic, scalable, and transparent sequential design-planning process for medical-product safety-surveillance systems utilizing observational electronic health care databases. Creating such a framework could yield improvements over existing practices, such as designs with increased power to assess serious adverse events.

Diagnostic accuracy of administrative data algorithms in the diagnosis of osteoarthritis: a systematic review

Background

Administrative health care data are frequently used to study disease burden and treatment outcomes in many conditions including osteoarthritis (OA). OA is a chronic condition with significant disease burden affecting over 27 million adults in the US. There are few studies examining the performance of administrative data algorithms to diagnose OA. The purpose of this study is to perform a systematic review of administrative data algorithms for OA diagnosis; and, to evaluate the diagnostic characteristics of algorithms based on restrictiveness and reference standards.

Methods

Two reviewers independently screened English-language articles published in Medline, Embase, PubMed, and Cochrane databases that used administrative data to identify OA cases. Each algorithm was classified as restrictive or less restrictive based on number and type of administrative codes required to satisfy the case definition. We recorded sensitivity and specificity of algorithms and calculated positive likelihood ratio (LR+) and positive predictive value (PPV) based on assumed OA prevalence of 0.1, 0.25, and 0.50.

Results

The search identified 7 studies that used 13 algorithms. Of these 13 algorithms, 5 were classified as restrictive and 8 as less restrictive. Restrictive algorithms had lower median sensitivity and higher median specificity compared to less restrictive algorithms when reference standards were self-report and American college of Rheumatology (ACR) criteria. The algorithms compared to reference standard of physician diagnosis had higher sensitivity and specificity than those compared to self-reported diagnosis or ACR criteria.

Conclusions

Restrictive algorithms are more specific for OA diagnosis and can be used to identify cases when false positives have higher costs e.g. interventional studies. Less restrictive algorithms are more sensitive and suited for studies that attempt to identify all cases e.g. screening programs.

Electronic supplementary material

The online version of this article (doi:10.1186/s12911-016-0319-y) contains supplementary material, which is available to authorized users.

Development and Application of an Alert System to Detect Cases of Food Poisoning in Japan

Recent public health concerns regarding commercial food products have increased the need to develop an automated method to detect food product-related health events. We developed and verified a method for the early detection of potentially harmful events caused by commercial food products. We collected data from daily internet-based questionnaires examining the presence or absence of symptoms and information about food purchased by the respondents. Using these data, we developed a method to detect possible health concerns regarding commercialized food products. To achieve this, we combined the signal detection method used in the reporting system of adverse effects of pharmaceutical products and the Early Aberration Reporting System (EARS) used by the United States Centers for Disease Control. Whiteleg shrimp (Litopenaeus vannamei), which had odds ratio and Odds(−) of 8.99 and 4.13, respectively, was identified as a possible causative food product for diarrhea and vomiting. In conclusion, this study demonstrated that food distributors can implement post-marketing monitoring of the safety of food products purchased via the internet.

Enhancing Vaccine Safety Capacity Globally: A Lifecycle Perspective

Major vaccine safety controversies have arisen in several countries beginning in the last decades of 20th century. Such periodic vaccine safety controversies are unlikely to go away in the near future as more national immunization programs mature with near elimination of target vaccine-preventable diseases that result in relative greater prominence of adverse events following immunizations, both true reactions and temporally coincidental events. There are several ways in which vaccine safety capacity can be improved to potentially mitigate the impact of future vaccine safety controversies. This paper aims to take a "lifecycle" approach, examining some potential pre- and post-licensure opportunities to improve vaccine safety, in both developed (specifically U.S. and Europe) and low- and middle-income countries.

Enhancing vaccine safety capacity globally: A lifecycle perspective

Major vaccine safety controversies have arisen in several countries beginning in the last decades of 20th century. Such periodic vaccine safety controversies are unlikely to go away in the near future as more national immunization programs mature with near elimination of target vaccine-preventable diseases that result in relative greater prominence of adverse events following immunizations, both true reactions and temporally coincidental events. There are several ways in which vaccine safety capacity can be improved to potentially mitigate the impact of future vaccine safety controversies. This paper aims to take a "lifecycle" approach, examining some potential pre- and post-licensure opportunities to improve vaccine safety, in both developed (specifically U.S. and Europe) and low- and middle-income countries.

Prospective influenza vaccine safety surveillance using fresh data in the Sentinel System

Purpose

To develop the infrastructure to conduct timely active surveillance for safety of influenza vaccines and other medical countermeasures in the Sentinel System (formerly the Mini‐Sentinel Pilot), a Food and Drug Administration‐sponsored national surveillance system that typically relies on data that are mature, settled, and updated quarterly.

Methods

Three Data Partners provided their earliest available (“fresh”) cumulative claims data on influenza vaccination and health outcomes 3–4 times on a staggered basis during the 2013–2014 influenza season, collectively producing 10 data updates. We monitored anaphylaxis in the entire population using a cohort design and seizures in children ≤4 years of age using both a self‐controlled risk interval design (primary) and a cohort design (secondary). After each data update, we conducted sequential analysis for inactivated (IIV) and live (LAIV) influenza vaccines using the Maximized Sequential Probability Ratio Test, adjusting for data‐lag.

Results

Most of the 10 sequential analyses were conducted within 6 weeks of the last care‐date in the cumulative dataset. A total of 6 682 336 doses of IIV and 782 125 doses of LAIV were captured. The primary analyses did not identify any statistical signals following IIV or LAIV. In secondary analysis, the risk of seizures was higher following concomitant IIV and PCV13 than historically after IIV in 6‐ to 23‐month‐olds (relative risk = 2.7), which requires further investigation.

Conclusions

The Sentinel System can implement a sequential analysis system that uses fresh data for medical product safety surveillance. Active surveillance using sequential analysis of fresh data holds promise for detecting clinically significant health risks early. Limitations of employing fresh data for surveillance include cost and the need for careful scrutiny of signals. © 2015 The Authors. Pharmacoepidemiology and Drug Safety Published by John Wiley & Sons Ltd.

Evaluation of the risk of venous thromboembolism after quadrivalent human papillomavirus vaccination among US females

After the Food and Drug Administration (FDA) licensed quadrivalent human papillomavirus vaccine (HPV4) in 2006, reports suggesting a possible association with venous thromboembolism (VTE) emerged from the Vaccine Adverse Event Reporting System and the Vaccine Safety Datalink. Our objective was to determine whether HPV4 increased VTE risk. The subjects were 9-26-year-old female members of five data partners in the FDA's Mini-Sentinel pilot project receiving HPV4 during 2006-2013. The outcome was radiologically confirmed first-ever VTE among potential cases identified by diagnosis codes in administrative data during Days 1-77 after HPV4 vaccination. With a self-controlled risk interval design, we compared counts of first-ever VTE in risk intervals (Days 1-28 and Days 1-7 post-vaccination) and control intervals (Days 36-56 for Dose 1 and Days 36-63 for Doses 2 and 3). Combined hormonal contraceptive use was treated as a potential confounder. The main analyses were: (1) unadjusted for time-varying VTE risk from contraceptive use, (2) unadjusted but restricted to cases without such time-varying risk, and (3) adjusted by incorporating the modeled risk of VTE by week of contraceptive use in the analysis. Of 279 potential VTE cases identified following 1,423,399 HPV4 doses administered, 225 had obtainable charts, and 53 were confirmed first-ever VTE. All 30 with onsets in risk or control intervals had known risk factors for VTE. VTE risk was not elevated in the first 7 or 28 days following any dose of HPV in any analysis (e.g. relative risk estimate (95% CI) from both unrestricted analyses, for all-doses, 28-day risk interval: 0.7 (0.3-1.4)). Temporal scan statistics found no clustering of VTE onsets after any dose. Thus, we found no evidence of an increased risk of VTE associated with HPV4 among 9-26-year-old females. A particular strength of this evaluation was its control for both time-invariant and contraceptive-related time-varying potential confounding.

Overall conceptual framework for studying the genetics of autoimmune diseases following vaccination: a regulatory perspective

The US Vaccine Adverse Event Reporting System contains case reports of autoimmune diseases (ADs) occurring following vaccinations. ADs are rare and occur in unvaccinated people, making the potential association between vaccines and ADs challenging to evaluate. Developing mechanistic pathways that link genes, immune mediators, vaccine components and ADs would be helpful for hypothesis generation, enhancing theories of biologic plausibility and grouping rare autoimmune adverse events to increase the ability to detect and evaluate safety signals. Here, we propose a conceptual framework for investigating the genetics of ADs as safety signals following vaccination, potentially contributing to the identification of relevant biomarkers. We also discuss a study design that incorporates genetic information into postmarket clinical evaluation of autoimmune adverse events following vaccination.

Febrile Seizures After 2010-2011 Trivalent Inactivated Influenza Vaccine

OBJECTIVES

In the Post-Licensure Rapid Immunization Safety Monitoring Program, we examined risk of febrile seizures (FS) after trivalent inactivated influenza vaccine (TIV) and 13-valent pneumococcal conjugate vaccine (PCV13) during the 2010-2011 influenza season, adjusted for concomitant diphtheria tetanus acellular pertussis-containing vaccines (DTaP). Assuming children would receive both vaccines, we examined whether same-day TIV and PCV13 vaccination was associated with greater FS risk when compared with separate-day vaccination.

METHODS

We used a self-controlled risk interval design, comparing the FS rate in a risk interval (0-1 days) versus control interval (14-20 days). Vaccinations were identified in claims and immunization registry data. FS were confirmed with medical records.

RESULTS

No statistically significant TIV-FS associations were found in unadjusted or adjusted models (incidence rate ratio [IRR] adjusted for age, seasonality, and concomitant PCV13 and DTaP: 1.36, 95% confidence interval [CI] 0.78 to 2.39). Adjusted for age and seasonality, PCV13 was significantly associated with FS (IRR 1.74, 95% CI 1.06 to 2.86), but not when further adjusting for concomitant TIV and DTaP (IRR 1.61, 95% CI 0.91 to 2.82). Same-day TIV and PCV13 vaccination was not associated with excess risk of FS when compared with separate-day vaccination (1.08 fewer FS per 100 000 with same day administration, 95% CI -5.68 to 6.09).

CONCLUSIONS

No statistically significant increased risk of FS was found for 2010-2011 TIV or PCV13, when adjusting for concomitant vaccines. Same-day TIV and PCV13 vaccination was not associated with more FS compared with separate-day vaccination.

Safety monitoring in the Vaccine Adverse Event Reporting System (VAERS)

The Centers for Disease Control and Prevention (CDC) and the U.S. Food and Drug Administration (FDA) conduct post-licensure vaccine safety monitoring using the Vaccine Adverse Event Reporting System (VAERS), a spontaneous (or passive) reporting system. This means that after a vaccine is approved, CDC and FDA continue to monitor safety while it is distributed in the marketplace for use by collecting and analyzing spontaneous reports of adverse events that occur in persons following vaccination. Various methods and statistical techniques are used to analyze VAERS data, which CDC and FDA use to guide further safety evaluations and inform decisions around vaccine recommendations and regulatory action. VAERS data must be interpreted with caution due to the inherent limitations of passive surveillance. VAERS is primarily a safety signal detection and hypothesis generating system. Generally, VAERS data cannot be used to determine if a vaccine caused an adverse event. VAERS data interpreted alone or out of context can lead to erroneous conclusions about cause and effect as well as the risk of adverse events occurring following vaccination. CDC makes VAERS data available to the public and readily accessible online. We describe fundamental vaccine safety concepts, provide an overview of VAERS for healthcare professionals who provide vaccinations and might want to report or better understand a vaccine adverse event, and explain how CDC and FDA analyze VAERS data. We also describe strengths and limitations, and address common misconceptions about VAERS. Information in this review will be helpful for healthcare professionals counseling patients, parents, and others on vaccine safety and benefit-risk balance of vaccination.

Advanced Clinical Decision Support for Vaccine Adverse Event Detection and Reporting

BACKGROUND

Reporting of adverse events (AEs) following vaccination can help identify rare or unexpected complications of immunizations and aid in characterizing potential vaccine safety signals. We developed an open-source, generalizable clinical decision support system called Electronic Support for Public Health-Vaccine Adverse Event Reporting System (ESP-VAERS) to assist clinicians with AE detection and reporting.

METHODS

ESP-VAERS monitors patients' electronic health records for new diagnoses, changes in laboratory values, and new allergies following vaccinations. When suggestive events are found, ESP-VAERS sends the patient's clinician a secure electronic message with an invitation to affirm or refute the message, add comments, and submit an automated, prepopulated electronic report to VAERS. High-probability AEs are reported automatically if the clinician does not respond. We implemented ESP-VAERS in December 2012 throughout the MetroHealth System, an integrated healthcare system in Ohio. We queried the VAERS database to determine MetroHealth's baseline reporting rates from January 2009 to March 2012 and then assessed changes in reporting rates with ESP-VAERS.

RESULTS

In the 8 months following implementation, 91 622 vaccinations were given. ESP-VAERS sent 1385 messages to responsible clinicians describing potential AEs. Clinicians opened 1304 (94.2%) messages, responded to 209 (15.1%), and confirmed 16 for transmission to VAERS. An additional 16 high-probability AEs were sent automatically. Reported events included seizure, pleural effusion, and lymphocytopenia. The odds of a VAERS report submission during the implementation period were 30.2 (95% confidence interval, 9.52-95.5) times greater than the odds during the comparable preimplementation period.

CONCLUSIONS

An open-source, electronic health record-based clinical decision support system can increase AE detection and reporting rates in VAERS.

The impact of epidemics of vaccine-preventable disease on vaccine uptake: lessons from the 2011-2012 US pertussis epidemic

Conventional wisdom suggests that if there is a vaccine that is effective in preventing a disease, vaccine uptake will increase when the disease risk is high. Recent evidence, however, suggests that this may not always be the case. In a study we conducted in Washington State, we found no population-level increase in pertussis vaccination of infants during a pertussis epidemic. In this paper, we aim to review what is known about the history of vaccine uptake during epidemics of vaccine-preventable disease, the challenges facing public health campaigns responding to these epidemics, and how the effect of a vaccine-preventable disease epidemic on vaccine uptake can be studied.

A vaccine study design selection framework for the postlicensure rapid immunization safety monitoring program

The Postlicensure Rapid Immunization Safety Monitoring Program, the vaccination safety monitoring component of the US Food and Drug Administration's Mini-Sentinel project, is currently the largest cohort in the US general population for vaccine safety surveillance. We developed a study design selection framework to provide a roadmap and description of methods that may be utilized to evaluate potential associations between vaccines and health outcomes of interest in the Postlicensure Rapid Immunization Safety Monitoring Program and other systems using administrative data. The strengths and weaknesses of designs for vaccine safety monitoring, including the cohort design, the case-centered design, the risk interval design, the case-control design, the self-controlled risk interval design, the self-controlled case series method, and the case-crossover design, are described and summarized in tabular form. A structured decision table is provided to aid in planning of future vaccine safety monitoring activities, and the data components comprising the structured decision table are delineated. The study design selection framework provides a starting point for planning vaccine safety evaluations using claims-based data sources.

The complementary roles of Phase 3 trials and post-licensure surveillance in the evaluation of new vaccines

Vaccines have led to significant reductions in morbidity and saved countless lives from many infectious diseases and are one of the most important public health successes of the modern era. Both vaccines' effectiveness and safety are keys for the success of immunisation programmes. The role of post-licensure surveillance has become increasingly recognised by regulatory authorities in the overall vaccine development process. Safety, purity, and effectiveness of vaccines are carefully assessed before licensure, but some safety and effectiveness aspects need continuing monitoring after licensure; Post-marketing activities are a necessary complement to pre-licensure activities for monitoring vaccine quality and to inform public health programmes. In the recent past, the availability of large databases together with data-mining and cross-linkage techniques have significantly improved the potentialities of post-licensure surveillance. The scope of this review is to present challenges and opportunities offered by vaccine post-licensure surveillance. While pre-licensure activities form the foundation for the development of effective and safe vaccines, post-licensure monitoring and assessment, are necessary to assure that vaccines are effective and safe when translated in real world settings. Strong partnerships and collaboration at an international level between different stakeholders is necessary for finding and optimally allocating resources and establishing robust post-licensure processes.

Responding to vaccine safety signals during pandemic influenza: a modeling study

Background

Managing emerging vaccine safety signals during an influenza pandemic is challenging. Federal regulators must balance vaccine risks against benefits while maintaining public confidence in the public health system.

Methods

We developed a multi-criteria decision analysis model to explore regulatory decision-making in the context of emerging vaccine safety signals during a pandemic. We simulated vaccine safety surveillance system capabilities and used an age-structured compartmental model to develop potential pandemic scenarios. We used an expert-derived multi-attribute utility function to evaluate potential regulatory responses by combining four outcome measures into a single measure of interest: 1) expected vaccination benefit from averted influenza; 2) expected vaccination risk from vaccine-associated febrile seizures; 3) expected vaccination risk from vaccine-associated Guillain-Barre Syndrome; and 4) expected change in vaccine-seeking behavior in future influenza seasons.

Results

Over multiple scenarios, risk communication, with or without suspension of vaccination of high-risk persons, were the consistently preferred regulatory responses over no action or general suspension when safety signals were detected during a pandemic influenza. On average, the expert panel valued near-term vaccine-related outcomes relative to long-term projected outcomes by 3∶1. However, when decision-makers had minimal ability to influence near-term outcomes, the response was selected primarily by projected impacts on future vaccine-seeking behavior.

Conclusions

The selected regulatory response depends on how quickly a vaccine safety signal is identified relative to the peak of the pandemic and the initiation of vaccination. Our analysis suggested two areas for future investment: efforts to improve the size and timeliness of the surveillance system and behavioral research to understand changes in vaccine-seeking behavior.