The effectiveness of U.S. public health surveillance systems for situational awareness during the 2009 H1N1 pandemic: a retrospective analysis

Metrics and indicators used to assess health system resilience in response to shocks to health systems in high income countries-A systematic review

Health system resilience has never been more important than with the COVID-19 pandemic. There is need to identify feasible measures of resilience, potential strategies to build resilience and weaknesses of health systems experiencing shocks. The purpose of this systematic review is to examine how the resilience of health systems has been measured across various health system shocks. Following PRISMA guidelines, with double screening at each stage, the review identified 3175 studies of which 68 studies were finally included for analysis. Almost half (46%) were focused on COVID-19, followed by the economic crises, disasters and previous pandemics. Over 80% of studies included quantitative metrics. The most common WHO health system functions studied were resources and service delivery. In relation to the shock cycle, most studies reported metrics related to the management stage (79%) with the fewest addressing recovery and learning (22%). Common metrics related to staff headcount, staff wellbeing, bed number and type, impact on utilisation and quality, public and private health spending, access and coverage, and information systems. Limited progress has been made with developing standardised qualitative metrics particularly around governance. Quantitative metrics need to be analysed in relation to change and the impact of the shock. The review notes problems with measuring preparedness and the fact that few studies have really assessed the legacy or enduring impact of shocks.

Missing science: A scoping study of COVID-19 epidemiological data in the United States

Systematic approaches to epidemiologic data collection are critical for informing pandemic responses, providing information for the targeting and timing of mitigations, for judging the efficacy and efficiency of alternative response strategies, and for conducting real-world impact assessments. Here, we report on a scoping study to assess the completeness of epidemiological data available for COVID-19 pandemic management in the United States, enumerating authoritative US government estimates of parameters of infectious transmission, infection severity, and disease burden and characterizing the extent and scope of US public health affiliated epidemiological investigations published through November 2021. While we found authoritative estimates for most expected transmission and disease severity parameters, some were lacking, and others had significant uncertainties. Moreover, most transmission parameters were not validated domestically or re-assessed over the course of the pandemic. Publicly available disease surveillance measures did grow appreciably in scope and resolution over time; however, their resolution with regards to specific populations and exposure settings remained limited. We identified 283 published epidemiological reports authored by investigators affiliated with U.S. governmental public health entities. Most reported on descriptive studies. Published analytic studies did not appear to fully respond to knowledge gaps or to provide systematic evidence to support, evaluate or tailor community mitigation strategies. The existence of epidemiological data gaps 18 months after the declaration of the COVID-19 pandemic underscores the need for more timely standardization of data collection practices and for anticipatory research priorities and protocols for emerging infectious disease epidemics.

Pandemic Awareness Scale (PAS): Evidence of Validity and Reliability in a Turkish Sample During the COVID-19 Pandemic

Public awareness is crucial in the prevention and management of the pandemic outbreaks. Accordingly, this study aimed to develop a scale that measures individuals’ awareness on pandemic outbreaks. The study investigated psychometric properties of the Pandemic Awareness Scale (PAS) in a Turkish sample (n = 1303) during the COVID-19 pandemic. The exploratory-factor-analysis (EFA) results (n = 903) indicated a high internal consistency reliability (α = .89) and suggested one-factor structure. The factor structure was confirmed in a different sample (n = 400) by a confirmatory-factor-analysis (CFA). The CFA results indicated that the one-factor model fits the data well (x²/df = 3.79, GFI = .96, IFI = .97, TLI = .94, CFI = .97, RMSEA = .084). The results suggested that the PAS is a valid and reliable tool to measure Turkish individuals’ pandemic awareness level.

Socioeconomic bias in influenza surveillance

Individuals in low socioeconomic brackets are considered at-risk for developing influenza-related complications and often exhibit higher than average influenza-related hospitalization rates. This disparity has been attributed to various factors, including restricted access to preventative and therapeutic health care, limited sick leave, and household structure. Adequate influenza surveillance in these at-risk populations is a critical precursor to accurate risk assessments and effective intervention. However, the United States of America's primary national influenza surveillance system (ILINet) monitors outpatient healthcare providers, which may be largely inaccessible to lower socioeconomic populations. Recent initiatives to incorporate Internet-source and hospital electronic medical records data into surveillance systems seek to improve the timeliness, coverage, and accuracy of outbreak detection and situational awareness. Here, we use a flexible statistical framework for integrating multiple surveillance data sources to evaluate the adequacy of traditional (ILINet) and next generation (BioSense 2.0 and Google Flu Trends) data for situational awareness of influenza across poverty levels. We find that ZIP Codes in the highest poverty quartile are a critical vulnerability for ILINet that the integration of next generation data fails to ameliorate.

Public Health Emergency Preparedness System Evaluation Criteria and Performance Metrics: A Review of Contributions of the CDC-Funded Preparedness and Emergency Response Research Centers

OBJECTIVES

The US Centers for Disease Control and Prevention (CDC)-funded Preparedness and Emergency Response Research Centers (PERRCs) conducted research from 2008 to 2015 aimed to improve the complex public health emergency preparedness and response (PHEPR) system. This paper summarizes PERRC studies that addressed the development and assessment of criteria for evaluating PHEPR and metrics for measuring their efficiency and effectiveness.

METHODS

We reviewed 171 PERRC publications indexed in PubMed between 2009 and 2016. These publications derived from 34 PERRC research projects. We identified publications that addressed the development or assessment of criteria and metrics pertaining to PHEPR systems and describe the evaluation methods used and tools developed, the system domains evaluated, and the metrics developed or assessed.

RESULTS

We identified 29 publications from 12 of the 34 PERRC projects that addressed PHEPR system evaluation criteria and metrics. We grouped each study into 1 of 3 system domains, based on the metrics developed or assessed: (1) organizational characteristics (n = 9), (2) emergency response performance (n = 12), and (3) workforce capacity or capability (n = 8). These studies addressed PHEPR system activities including responses to the 2009 H1N1 pandemic and the 2011 tsunami, as well as emergency exercise performance, situational awareness, and workforce willingness to respond. Both PHEPR system process and outcome metrics were developed or assessed by PERRC studies.

CONCLUSIONS

PERRC researchers developed and evaluated a range of PHEPR system evaluation criteria and metrics that should be considered by system partners interested in assessing the efficiency and effectiveness of their activities. Nonetheless, the monitoring and measurement problem in PHEPR is far from solved. Lack of standard measures that are readily obtained or computed at local levels remains a challenge for the public health preparedness field. (Disaster Med Public Health Preparedness. 2019;13:626-638).

Public Health Surveillance for Zika Virus: Data Interpretation and Report Validity

Zika virus provides an example for which public health surveillance is based primarily on health care provider notifications to health departments of potential cases. This case-based surveillance is commonly used to understand the spread of disease in a population. However, case-based surveillance is often biased-whether testing is done and which tests are used and the accuracy of the results depend on a variety of factors including test availability, patient demand, perceptions of transmission, and patient and provider awareness, leading to surveillance artifacts that can provide misleading information on the spread of a disease in a population and have significant public health practice implications. To better understand this challenge, we first summarize the process that health departments use to generate surveillance reports, then describe factors influencing testing and reporting patterns at the patient, provider, and contextual level. We then describe public health activities, including active surveillance, that influence both patient and provider behavior as well as surveillance reports, and conclude with a discussion about the interpretation of surveillance data and approaches that could improve the validity of surveillance reports.

Assessing the severity of influenza: a role for longitudinal telephone surveys?

During the 2009 influenza pandemic, a rapid assessment of disease severity was a challenge as a significant proportion of cases did not seek medical care; care-seeking behaviour changed and the proportion asymptomatic was unknown. A random-digit-dialling telephone survey was undertaken during the 2011/12 winter season in England and Wales to address the feasibility of answering these questions. A proportional quota sampling strategy was employed based on gender, age group, geographical location, employment status and level of education. Households were recruited pre-season and re-contacted immediately following peak seasonal influenza activity. The pre-peak survey was undertaken in October 2011 with 1061 individuals recruited and the post-peak telephone survey in March 2012. Eight hundred and thirty-four of the 1061 (78.6%) participants were successfully re-contacted. Their demographic characteristics compared well to national census data. In total, 8.4% of participants self-reported an influenza-like illness (ILI) in the previous 2 weeks, with 3.2% conforming to the World Health Organization (WHO) ILI case definition. In total, 29.6% of the cases reported consulting their general practitioner. 54.1% of the 1061 participants agreed to be re-contacted about providing biological samples. A population-based cohort was successfully recruited and followed up. Longitudinal survey methodology provides a practical tool to assess disease severity during future pandemics.

A review of documents prepared by international organizations about influenza pandemics, including the 2009 pandemic: a bibliometric analysis

Background

World Health Organization (WHO), the World Bank, UN System Influenza Coordination (UNSIC) and other international organizations released a series of documents to fight against the influenza pandemic. Those documents have great significance on guiding influenza pandemic preparedness and responses and providing a multilevel, multi-directional influenza pandemic prevention and control network for their member countries. This study focuses on the above-mentioned influenza pandemic preparedness guidelines with the aim of exploring the roles of the society, defining the relationship of different interventions and evaluating the planning on influenza pandemic preparedness.

Methods

Documents about pandemic influenza preparedness were retrieved from the official websites of the following three international organizations, World Health Organization (WHO), the World Bank, UN System Influenza Coordination (UNSIC) with the key words ‘pandemic’, ‘influenza’ and the Boolean combinations of these words as the retrieval strategy. Guidelines, research study and meeting reports were included in the study. The categories of the ministries/departments involved and their roles/responsibilities in pandemic influenza preparedness were summarized. Word frequency of selected vocabularies about pandemic influenza preventive measures were collected from the documents and the correlations between the word frequency of these measures were analyzed. Ochiai coefficient was employed to show the correlation between the word vocabularies.

Results

A total of 38 records on the topic of pandemic influenza preparedness were included. The responsibilities of the whole-of-society mentioned in the international organizations’ documents varied across the 2009 influenza pandemic period. Meanwhile, it had been emphasized that a comprehensive influenza prevention and control plan in every sector should be developed and evaluated. Because various measures were emphasized in the guidelines after 2009 pandemic influenza, the correlations between the word frequencies of the various influenza preventive measures became stronger after the pandemic influenza.

Conclusions

Responsibilities of ministries of education, ministries of energy, ministries of agriculture and animal health, ministries of communication and the business sector in the pandemic influenza preparedness were described more comprehensively in the international organizations’ documents in 2017. Better understanding the variations of the guidelines delivered by international organizations would be useful for the member countries to strengthen their influenza control network.

Electronic supplementary material

The online version of this article (10.1186/s12879-018-3286-3) contains supplementary material, which is available to authorized users.

Evaluating oseltamivir prescriptions in Centers for Medicare and Medicaid Services medical claims records as an indicator of seasonal influenza in the United States

Background

Over 34 million residents of the United States aged 65 years and older are also Medicare prescription drug beneficiaries. Medical claims records for this age group potentially provide a wealth of data for better understanding influenza epidemiology.

Objective

The purpose of this study was to evaluate data on oseltamivir dispensing extracted from medical claims records as an indicator of influenza activity in the United States for the 2010‐11 through 2014‐15 influenza seasons.

Methods

We used Centers for Medicare and Medicaid Services (CMS) medical claims data to evaluate the weekly number of therapeutic oseltamivir prescriptions dispensed following a rapid influenza diagnostic test among beneficiaries 65 years old and older as an indicator of influenza timing and intensity. We compared the temporal changes in this indicator to changes in the proportion of influenza‐like illnesses among outpatient visits in the US Outpatient Influenza‐like Illness Surveillance Network (ILINet) by administrative regions defined by the US Department of Health and Human Services. Using the moving epidemic method, we determined intensity thresholds and categorized the severity of seasons for both CMS and ILINet data.

Results

Centers for Medicare and Medicaid Services oseltamivir data and ILINet data were strongly correlated by administrative region (median Spearman's ρ = 0.78; interquartile range = 0.73‐0.80). CMS oseltamivir data and ILINet data substantially agreed (Cohen's weighted κ = 0.62) as to the seasonal severity across administrative regions.

Conclusions

Our results support the use of oseltamivir dispensing in medical claims data as an indicator of US influenza activity.

ORBiT: Oak Ridge biosurveillance toolkit for public health dynamics

Background

The digitization of health-related information through electronic health records (EHR) and electronic healthcare reimbursement claims and the continued growth of self-reported health information through social media provides both tremendous opportunities and challenges in developing effective biosurveillance tools. With novel emerging infectious diseases being reported across different parts of the world, there is a need to build systems that can track, monitor and report such events in a timely manner. Further, it is also important to identify susceptible geographic regions and populations where emerging diseases may have a significant impact.

Methods

In this paper, we present an overview of Oak Ridge Biosurveillance Toolkit (ORBiT), which we have developed specifically to address data analytic challenges in the realm of public health surveillance. In particular, ORBiT provides an extensible environment to pull together diverse, large-scale datasets and analyze them to identify spatial and temporal patterns for various biosurveillance-related tasks.

Results

We demonstrate the utility of ORBiT in automatically extracting a small number of spatial and temporal patterns during the 2009-2010 pandemic H1N1 flu season using claims data. These patterns provide quantitative insights into the dynamics of how the pandemic flu spread across different parts of the country. We discovered that the claims data exhibits multi-scale patterns from which we could identify a small number of states in the United States (US) that act as "bridge regions" contributing to one or more specific influenza spread patterns. Similar to previous studies, the patterns show that the south-eastern regions of the US were widely affected by the H1N1 flu pandemic. Several of these south-eastern states act as bridge regions, which connect the north-east and central US in terms of flu occurrences.

Conclusions

These quantitative insights show how the claims data combined with novel analytical techniques can provide important information to decision makers when an epidemic spreads throughout the country. Taken together ORBiT provides a scalable and extensible platform for public health surveillance.

Biosurveillance capability requirements for the global health security agenda: lessons from the 2009 H1N1 pandemic

The biosurveillance capabilities needed to rapidly detect and characterize emerging biological threats are an essential part of the Global Health Security Agenda (GHSA). The analyses of the global public health system's functioning during the 2009 H1N1 pandemic suggest that while capacities such as those identified in the GHSA are essential building blocks, the global biosurveillance system must possess 3 critical capabilities: (1) the ability to detect outbreaks and determine whether they are of significant global concern, (2) the ability to describe the epidemiologic characteristics of the pathogen responsible, and (3) the ability to track the pathogen's spread through national populations and around the world and to measure the impact of control strategies. The GHSA capacities-laboratory and diagnostic capacity, reporting networks, and so on-were essential in 2009 and surely will be in future events. But the 2009 H1N1 experience reminds us that it is not just detection but epidemiologic characterization that is necessary. Similarly, real-time biosurveillance systems are important, but as the 2009 H1N1 experience shows, they may contain inaccurate information about epidemiologic risks. Rather, the ability of scientists in Mexico, the United States, and other countries to make sense of the emerging laboratory and epidemiologic information that was critical-an example of global social capital-enabled an effective global response. Thus, to ensure that it is meeting its goals, the GHSA must track capabilities as well as capacities.

Performance assessment of communicable disease surveillance in disasters: a systematic review

BACKGROUND

This study aimed to identify the indices and frameworks that have been used to assess the performance of communicable disease surveillance (CDS) in response to disasters and other emergencies, including infectious disease outbreaks.

METHOD

In this systematic review, PubMed, Google Scholar, Scopus, ScienceDirect, ProQuest databases and grey literature were searched until the end of 2013. All retrieved titles were examined in accordance with inclusion criteria. Abstracts of the relevant titles were reviewed and eligible abstracts were included in a list for data abstraction. Finally, the study variables were extracted.

RESULTS

Sixteen articles and one book were found relevant to our study objectives. In these articles, 31 criteria and 35 indicators were used or suggested for the assessment/evaluation of the performance of surveillance systems in disasters. The Centers for Disease Control (CDC) updated guidelines for the evaluation of public health surveillance systems were the most widely used.

CONCLUSION

Despite the importance of performance assessment in improving CDS in response to disasters, there is a lack of clear and accepted frameworks. There is also no agreement on the use of existing criteria and indices. The only relevant framework is the CDC guideline, which is a common framework for assessing public health surveillance systems as a whole. There is an urgent need to develop appropriate frameworks, criteria, and indices for specifically assessing the performance of CDS in response to disasters and other emergencies, including infectious diseases outbreaks. Key words: Disasters, Emergencies, Communicable Diseases, Surveillance System, Performance Assessment.

The epidemiology and surveillance workforce among local health departments in California: mutual aid and surge capacity for routine and emergency infectious disease situations

OBJECTIVE

Public health surveillance and epidemiologic investigations are critical public health functions for identifying threats to the health of a community. We conducted a survey of local health departments (LHDs) in California to describe the workforce that supports public health surveillance and epidemiologic functions during routine and emergency infectious disease situations.

METHODS

The target population consisted of the 61 LHDs in California. The online survey instrument was designed to collect information about the workforce involved in key epidemiologic functions. We also examined how the public health workforce increases its epidemiologic capacity during infectious disease emergencies.

RESULTS

Of 61 LHDs in California, 31 (51%) completed the survey. A wide range of job classifications contribute to epidemiologic functions routinely, and LHDs rely on both internal and external sources of epidemiologic surge capacity during infectious disease emergencies. This study found that while 17 (55%) LHDs reported having a mutual aid agreement with at least one other organization for emergency response, only nine (29%) LHDs have a mutual aid agreement specifically for epidemiology and surveillance functions.

CONCLUSIONS

LHDs rely on a diverse workforce to conduct epidemiology and public health surveillance functions, emphasizing the need to identify and describe the types of staff positions that could benefit from public health surveillance and epidemiology training. While some organizations collaborate with external partners to support these functions during an emergency, many LHDs do not rely on mutual aid agreements for epidemiology and surveillance activities.

Characterizing Influenza surveillance systems performance: application of a Bayesian hierarchical statistical model to Hong Kong surveillance data

BACKGROUND

Infectious disease surveillance is a process the product of which reflects both actual disease trends and public awareness of the disease. Decisions made by patients, health care providers, and public health professionals about seeking and providing health care and about reporting cases to health authorities are all influenced by the information environment, which changes constantly. Biases are therefore imbedded in surveillance systems; these biases need to be characterized to provide better situational awareness for decision-making purposes. Our goal is to develop a statistical framework to characterize influenza surveillance systems, particularly their correlation with the information environment.

METHODS

We identified Hong Kong influenza surveillance data systems covering healthcare providers, laboratories, daycare centers and residential care homes for the elderly. A Bayesian hierarchical statistical model was developed to examine the statistical relationships between the influenza surveillance data and the information environment represented by alerts from HealthMap and web queries from Google. Different models were fitted for non-pandemic and pandemic periods and model goodness-of-fit was assessed using common model selection procedures.

RESULTS

Some surveillance systems - especially ad hoc systems developed in response to the pandemic flu outbreak - are more correlated with the information environment than others. General practitioner (percentage of influenza-like-illness related patient visits among all patient visits) and laboratory (percentage of specimen tested positive) seem to proportionally reflect the actual disease trends and are less representative of the information environment. Surveillance systems using influenza-specific code for reporting tend to reflect biases of both healthcare seekers and providers.

CONCLUSIONS

This study shows certain influenza surveillance systems are less correlated with the information environment than others, and therefore, might represent more reliable indicators of disease activity in future outbreaks. Although the patterns identified in this study might change in future outbreaks, the potential susceptibility of surveillance data is likely to persist in the future, and should be considered when interpreting surveillance data.

Measuring and assessing public health emergency preparedness

The National Health Security Strategy calls for a systematic quality improvement efforts to improving health security. In addition, since 2001, billions of dollars have been spent on public health emergency preparedness at the federal, state, and local levels. Policy makers must now ask has this investment been effective, and in what way to measure this investment. Whether the goal is to ensure accountability to policy makers or to facilitate quality improvement, valid and reliable measures of preparedness are needed. Measuring and assessing the state of the nation's preparedness, however, are challenging. This article explores the current measurement climate and potential for improvement.

Effect of race/ethnicity and socioeconomic status on pandemic H1N1-related outcomes in Massachusetts

OBJECTIVES

We linked hospital discharge and American Community Survey and US Census data to investigate 2009 H1N1 influenza (H1N1)-related outcomes by racial/ethnic groups and socioeconomic status (SES).

METHODS

We examined the population discharged from any acute care hospital in Massachusetts and calculated rates of intensive care unit (ICU) stay by racial/ethnic and SES groups between April 26 and September 30, 2009. We used logistic regression models to identify predictors of ICU stay.

RESULTS

Of 4874 H1N1-related hospitalizations, 526 (11%) were admitted to the ICU. Those in less affluent SES groups had lower risk of ICU stay than the most affluent SES group. Compared with Whites, Hispanics had significantly lower risk of 2009 H1N1-related ICU stay (odds ratio = 0.52; 95% confidence interval = 0.32, 0.86). Only 13% of Whites admitted to the ICU were in the lowest SES group, compared with 63% of Hispanics and 43% of Blacks.

CONCLUSIONS

To our knowledge, this is the first statewide description of 2009 H1N1 influenza-related ICU stays according to racial/ethnic group and SES in the United States. Future work should investigate evidence related to social determinants of health among racial/ethnic groups to reduce disparities in relation to pandemic influenza.

A Bayesian Approach to Characterize Hong Kong Influenza Surveillance Systems

Objective

Introduction

Methods

Results

Conclusions