Forecasting peaks of seasonal influenza epidemics

Integrating information from historical data into mechanistic models for influenza forecasting

Seasonal influenza causes significant annual morbidity and mortality worldwide. In France, it is estimated that, on average, 2 million individuals consult their GP for influenza-like-illness (ILI) every year. Traditionally, mathematical models used for epidemic forecasting can either include parameters capturing the infection process (mechanistic or compartmental models) or rely on time series analysis approaches that do not make mechanistic assumptions (statistical or phenomenological models). While the latter make extensive use of past epidemic data, mechanistic models are usually independently initialized in each season. As a result, forecasts from such models can contain trajectories that are vastly different from past epidemics. We developed a mechanistic model that takes into account epidemic data from training seasons when producing forecasts. The parameters of the model are estimated via a first particle filter running on the observed data. A second particle filter is then used to produce forecasts compatible with epidemic trajectories from the training set. The model was calibrated and tested on 35 years' worth of surveillance data from the French Sentinelles Network, representing the weekly number of patients consulting for ILI over the period 1985-2019. Our results show that the new method improves upon standard mechanistic approaches. In particular, when retrospectively tested on the available data, our model provides increased accuracy for short-term forecasts (from one to four weeks into the future) and peak timing and intensity. Our new approach for epidemic forecasting allows the integration of key strengths of the statistical approach into the mechanistic modelling framework and represents an attempt to provide accurate forecasts by making full use of the rich surveillance dataset collected in France since 1985.

Combinational Recommendation of Vaccinations, Mask-Wearing, and Home-Quarantine to Control Influenza in Megacities: An Agent-Based Modeling Study With Large-Scale Trajectory Data

The outbreak of COVID-19 stimulated a new round of discussion on how to deal with respiratory infectious diseases. Influenza viruses have led to several pandemics worldwide. The spatiotemporal characteristics of influenza transmission in modern cities, especially megacities, are not well-known, which increases the difficulty of influenza prevention and control for populous urban areas. For a long time, influenza prevention and control measures have focused on vaccination of the elderly and children, and school closure. Since the outbreak of COVID-19, the public's awareness of measures such as vaccinations, mask-wearing, and home-quarantine has generally increased in some regions of the world. To control the influenza epidemic and reduce the proportion of infected people with high mortality, the combination of these three measures needs quantitative evaluation based on the spatiotemporal transmission characteristics of influenza in megacities. Given that the agent-based model with both demographic attributes and fine-grained mobility is a key planning tool in deploying intervention strategies, this study proposes a spatially explicit agent-based influenza model for assessing and recommending the combinations of influenza control measures. This study considers Shenzhen city, China as the research area. First, a spatially explicit agent-based influenza transmission model was developed by integrating large-scale individual trajectory data and human response behavior. Then, the model was evaluated across multiple intra-urban spatial scales based on confirmed influenza cases. Finally, the model was used to evaluate the combined effects of the three interventions (V: vaccinations, M: mask-wearing, and Q: home-quarantining) under different compliance rates, and their optimal combinations for given control objectives were recommended. This study reveals that adults were a high-risk population with a low reporting rate, and children formed the lowest infected proportion and had the highest reporting rate in Shenzhen. In addition, this study systematically recommended different combinations of vaccinations, mask-wearing, and home-quarantine with different compliance rates for different control objectives to deal with the influenza epidemic. For example, the "V45%-M60%-Q20%" strategy can maintain the infection percentage below 5%, while the "V20%-M60%-Q20%" strategy can maintain the infection percentage below 15%. The model and policy recommendations from this study provide a tool and intervention reference for influenza epidemic management in the post-COVID-19 era.

Predicting seasonal influenza using supermarket retail records

Increased availability of epidemiological data, novel digital data streams, and the rise of powerful machine learning approaches have generated a surge of research activity on real-time epidemic forecast systems. In this paper, we propose the use of a novel data source, namely retail market data to improve seasonal influenza forecasting. Specifically, we consider supermarket retail data as a proxy signal for influenza, through the identification of sentinel baskets, i.e., products bought together by a population of selected customers. We develop a nowcasting and forecasting framework that provides estimates for influenza incidence in Italy up to 4 weeks ahead. We make use of the Support Vector Regression (SVR) model to produce the predictions of seasonal flu incidence. Our predictions outperform both a baseline autoregressive model and a second baseline based on product purchases. The results show quantitatively the value of incorporating retail market data in forecasting models, acting as a proxy that can be used for the real-time analysis of epidemics.

Using digital surveillance tools for near real-time mapping of the risk of infectious disease spread

Data from digital disease surveillance tools such as ProMED and HealthMap can complement the field surveillance during ongoing outbreaks. Our aim was to investigate the use of data collected through ProMED and HealthMap in real-time outbreak analysis. We developed a flexible statistical model to quantify spatial heterogeneity in the risk of spread of an outbreak and to forecast short term incidence trends. The model was applied retrospectively to data collected by ProMED and HealthMap during the 2013-2016 West African Ebola epidemic and for comparison, to WHO data. Using ProMED and HealthMap data, the model was able to robustly quantify the risk of disease spread 1-4 weeks in advance and for countries at risk of case importations, quantify where this risk comes from. Our study highlights that ProMED and HealthMap data could be used in real-time to quantify the spatial heterogeneity in risk of spread of an outbreak.

Highlighting the compound risk of COVID-19 and environmental pollutants using geospatial technology

The new COVID-19 coronavirus disease has emerged as a global threat and not just to human health but also the global economy. Due to the pandemic, most countries affected have therefore imposed periods of full or partial lockdowns to restrict community transmission. This has had the welcome but unexpected side effect that existing levels of atmospheric pollutants, particularly in cities, have temporarily declined. As found by several authors, air quality can inherently exacerbate the risks linked to respiratory diseases, including COVID-19. In this study, we explore patterns of air pollution for ten of the most affected countries in the world, in the context of the 2020 development of the COVID-19 pandemic. We find that the concentrations of some of the principal atmospheric pollutants were temporarily reduced during the extensive lockdowns in the spring. Secondly, we show that the seasonality of the atmospheric pollutants is not significantly affected by these temporary changes, indicating that observed variations in COVID-19 conditions are likely to be linked to air quality. On this background, we confirm that air pollution may be a good predictor for the local and national severity of COVID-19 infections.

Forecasting and planning during a pandemic: COVID-19 growth rates, supply chain disruptions, and governmental decisions

Policymakers during COVID-19 operate in uncharted territory and must make tough decisions. Operational Research - the ubiquitous 'science of better' - plays a vital role in supporting this decision-making process. To that end, using data from the USA, India, UK, Germany, and Singapore up to mid-April 2020, we provide predictive analytics tools for forecasting and planning during a pandemic. We forecast COVID-19 growth rates with statistical, epidemiological, machine- and deep-learning models, and a new hybrid forecasting method based on nearest neighbors and clustering. We further model and forecast the excess demand for products and services during the pandemic using auxiliary data (google trends) and simulating governmental decisions (lockdown). Our empirical results can immediately help policymakers and planners make better decisions during the ongoing and future pandemics.

Forecasting Weekly Influenza Outpatient Visits Using a Two-Dimensional Hierarchical Decision Tree Scheme

Influenza is a serious public health issue, as it can cause acute suffering and even death, social disruption, and economic loss. Effective forecasting of influenza outpatient visits is beneficial to anticipate and prevent medical resource shortages. This study uses regional data on influenza outpatient visits to propose a two-dimensional hierarchical decision tree scheme for forecasting influenza outpatient visits. The Taiwan weekly influenza outpatient visit data were collected from the national infectious disease statistics system and used for an empirical example. The 788 data points start in the first week of 2005 and end in the second week of 2020. The empirical results revealed that the proposed forecasting scheme outperformed five competing models and was able to forecast one to four weeks of anticipated influenza outpatient visits. The scheme may be an effective and promising alternative for forecasting one to four steps (weeks) ahead of nationwide influenza outpatient visits in Taiwan. Our results also suggest that, for forecasting nationwide influenza outpatient visits in Taiwan, one- and two-time lag information and regional information from the Taipei, North, and South regions are significant.

The design and evaluation of a Bayesian system for detecting and characterizing outbreaks of influenza

The prediction and characterization of outbreaks of infectious diseases such as influenza remains an open and important problem. This paper describes a framework for detecting and characterizing outbreaks of influenza and the results of testing it on data from ten outbreaks collected from two locations over five years. We model outbreaks with compartment models and explicitly model non-influenza influenza-like illnesses.

Flucast: A Real-Time Tool to Predict Severity of an Influenza Season

Background

Influenza causes serious illness requiring annual health system surge capacity, yet annual seasonal variation makes it difficult to forecast and plan for the severity of an upcoming season. Research shows that hospital and health system stakeholders indicate a preference for forecasting tools that are easy to use and understand to assist with surge capacity planning for influenza.

Objective

This study aimed to develop a simple risk prediction tool, Flucast, to predict the severity of an emerging influenza season.

Methods

Study data were obtained from the National Notifiable Diseases Surveillance System and Australian Influenza Surveillance Reports from the Department of Health, Australia. We tested Flucast using retrospective seasonal data for 11 Australian influenza seasons. We compared five different models using parameters known early in the season that may be associated with the severity of the season. To calibrate the tool, the resulting estimates of seasonal severity were validated against independent reports of influenza-attributable morbidity and mortality. The model with the highest predictive accuracy against retrospective seasonal activity was chosen as a best-fit model to develop the Flucast tool. The tool was prospectively tested against the 2018 and the emerging 2019 influenza season.

Results

The Flucast tool predicted the severity of all retrospectively studied years correctly for influenza seasonal activity in Australia. With the use of real-time data, the tool provided a reasonable early prediction of a low to moderate season for the 2018 and severe seasonal activity for the upcoming 2019 season. The tool meets stakeholder preferences for simplicity and ease of use to assist with surge capacity planning.

Conclusions

The Flucast tool may be useful to inform future health system influenza preparedness planning, surge capacity, and intervention programs in real time, and can be adapted for different settings and geographic locations.

Assessing the performance of real-time epidemic forecasts: A case study of Ebola in the Western Area region of Sierra Leone, 2014-15

Real-time forecasts based on mathematical models can inform critical decision-making during infectious disease outbreaks. Yet, epidemic forecasts are rarely evaluated during or after the event, and there is little guidance on the best metrics for assessment. Here, we propose an evaluation approach that disentangles different components of forecasting ability using metrics that separately assess the calibration, sharpness and bias of forecasts. This makes it possible to assess not just how close a forecast was to reality but also how well uncertainty has been quantified. We used this approach to analyse the performance of weekly forecasts we generated in real time for Western Area, Sierra Leone, during the 2013-16 Ebola epidemic in West Africa. We investigated a range of forecast model variants based on the model fits generated at the time with a semi-mechanistic model, and found that good probabilistic calibration was achievable at short time horizons of one or two weeks ahead but model predictions were increasingly unreliable at longer forecasting horizons. This suggests that forecasts may have been of good enough quality to inform decision making based on predictions a few weeks ahead of time but not longer, reflecting the high level of uncertainty in the processes driving the trajectory of the epidemic. Comparing forecasts based on the semi-mechanistic model to simpler null models showed that the best semi-mechanistic model variant performed better than the null models with respect to probabilistic calibration, and that this would have been identified from the earliest stages of the outbreak. As forecasts become a routine part of the toolkit in public health, standards for evaluation of performance will be important for assessing quality and improving credibility of mathematical models, and for elucidating difficulties and trade-offs when aiming to make the most useful and reliable forecasts.

Even a good influenza forecasting model can benefit from internet-based nowcasts, but those benefits are limited

The ability to produce timely and accurate flu forecasts in the United States can significantly impact public health. Augmenting forecasts with internet data has shown promise for improving forecast accuracy and timeliness in controlled settings, but results in practice are less convincing, as models augmented with internet data have not consistently outperformed models without internet data. In this paper, we perform a controlled experiment, taking into account data backfill, to improve clarity on the benefits and limitations of augmenting an already good flu forecasting model with internet-based nowcasts. Our results show that a good flu forecasting model can benefit from the augmentation of internet-based nowcasts in practice for all considered public health-relevant forecasting targets. The degree of forecast improvement due to nowcasting, however, is uneven across forecasting targets, with short-term forecasting targets seeing the largest improvements and seasonal targets such as the peak timing and intensity seeing relatively marginal improvements. The uneven forecasting improvements across targets hold even when "perfect" nowcasts are used. These findings suggest that further improvements to flu forecasting, particularly seasonal targets, will need to derive from other, non-nowcasting approaches.

Optimal multi-source forecasting of seasonal influenza

Forecasting the emergence and spread of influenza viruses is an important public health challenge. Timely and accurate estimates of influenza prevalence, particularly of severe cases requiring hospitalization, can improve control measures to reduce transmission and mortality. Here, we extend a previously published machine learning method for influenza forecasting to integrate multiple diverse data sources, including traditional surveillance data, electronic health records, internet search traffic, and social media activity. Our hierarchical framework uses multi-linear regression to combine forecasts from multiple data sources and greedy optimization with forward selection to sequentially choose the most predictive combinations of data sources. We show that the systematic integration of complementary data sources can substantially improve forecast accuracy over single data sources. When forecasting the Center for Disease Control and Prevention (CDC) influenza-like-illness reports (ILINet) from week 48 through week 20, the optimal combination of predictors includes public health surveillance data and commercially available electronic medical records, but neither search engine nor social media data.

Influenza forecast optimization when using different surveillance data types and geographic scale

Background

Advance warning of influenza incidence levels from skillful forecasts could help public health officials and healthcare providers implement more timely preparedness and intervention measures to combat outbreaks. Compared to influenza predictions generated at regional and national levels, those generated at finer scales could offer greater value in determining locally appropriate measures; however, to date, the various influenza surveillance data that are collected by state and county departments of health have not been well utilized in influenza prediction.

Objectives

To assess whether an influenza forecast model system can be optimized to generate accurate forecasts using novel surveillance data streams.

Methods

Here, we generate retrospective influenza forecasts with a dynamic, compartmental model‐inference system using surveillance data for influenza‐like illness (ILI), laboratory‐confirmed cases, and pneumonia and influenza mortality at state and county levels. We evaluate how specification of 3 system inputs—scaling, observational error variance (OEV), and filter divergence (lambda)—affects forecast accuracy.

Results

In retrospective forecasts, and across data types, there were no clear optimal combinations for the 3 system inputs; however, scaling was most critical to forecast accuracy, whereas OEV and lambda were not.

Conclusions

Forecasts using new data streams should be tested to determine an appropriate scaling value using historical data and analyzed for forecast accuracy.

Lessons from a decade of individual-based models for infectious disease transmission: a systematic review (2006-2015)

Background

Individual-based models (IBMs) are useful to simulate events subject to stochasticity and/or heterogeneity, and have become well established to model the potential (re)emergence of pathogens (e.g., pandemic influenza, bioterrorism). Individual heterogeneity at the host and pathogen level is increasingly documented to influence transmission of endemic diseases and it is well understood that the final stages of elimination strategies for vaccine-preventable childhood diseases (e.g., polio, measles) are subject to stochasticity. Even so it appears IBMs for both these phenomena are not well established. We review a decade of IBM publications aiming to obtain insights in their advantages, pitfalls and rationale for use and to make recommendations facilitating knowledge transfer within and across disciplines.

Methods

We systematically identified publications in Web of Science and PubMed from 2006-2015 based on title/abstract/keywords screening (and full-text if necessary) to retrieve topics, modeling purposes and general specifications. We extracted detailed modeling features from papers on established vaccine-preventable childhood diseases based on full-text screening.

Results

We identified 698 papers, which applied an IBM for infectious disease transmission, and listed these in a reference database, describing their general characteristics. The diversity of disease-topics and overall publication frequency have increased over time (38 to 115 annual publications from 2006 to 2015). The inclusion of intervention strategies (8 to 52) and economic consequences (1 to 20) are increasing, to the detriment of purely theoretical explorations. Unfortunately, terminology used to describe IBMs is inconsistent and ambiguous. We retrieved 24 studies on a vaccine-preventable childhood disease (covering 7 different diseases), with publication frequency increasing from the first such study published in 2008. IBMs have been useful to explore heterogeneous between- and within-host interactions, but combined applications are still sparse. The amount of missing information on model characteristics and study design is remarkable.

Conclusions

IBMs are suited to combine heterogeneous within- and between-host interactions, which offers many opportunities, especially to analyze targeted interventions for endemic infections. We advocate the exchange of (open-source) platforms and stress the need for consistent “branding”. Using (existing) conventions and reporting protocols would stimulate cross-fertilization between research groups and fields, and ultimately policy making in decades to come.

Electronic supplementary material

The online version of this article (doi:10.1186/s12879-017-2699-8) contains supplementary material, which is available to authorized users.

Translation of Real-Time Infectious Disease Modeling into Routine Public Health Practice

Infectious disease dynamic modeling can support outbreak emergency responses. We conducted a workshop to canvas the needs of stakeholders in Australia for practical, real-time modeling tools for infectious disease emergencies. The workshop was attended by 29 participants who represented government, defense, general practice, and academia stakeholders. We found that modeling is underused in Australia and its potential is poorly understood by practitioners involved in epidemic responses. The development of better modeling tools is desired. Ideal modeling tools for operational use would be easy to use, clearly indicate underlying parameterization and assumptions, and assist with policy and decision making.

Forecasting influenza in Hong Kong with Google search queries and statistical model fusion

BACKGROUND

The objective of this study is to investigate predictive utility of online social media and web search queries, particularly, Google search data, to forecast new cases of influenza-like-illness (ILI) in general outpatient clinics (GOPC) in Hong Kong. To mitigate the impact of sensitivity to self-excitement (i.e., fickle media interest) and other artifacts of online social media data, in our approach we fuse multiple offline and online data sources.

METHODS

Four individual models: generalized linear model (GLM), least absolute shrinkage and selection operator (LASSO), autoregressive integrated moving average (ARIMA), and deep learning (DL) with Feedforward Neural Networks (FNN) are employed to forecast ILI-GOPC both one week and two weeks in advance. The covariates include Google search queries, meteorological data, and previously recorded offline ILI. To our knowledge, this is the first study that introduces deep learning methodology into surveillance of infectious diseases and investigates its predictive utility. Furthermore, to exploit the strength from each individual forecasting models, we use statistical model fusion, using Bayesian model averaging (BMA), which allows a systematic integration of multiple forecast scenarios. For each model, an adaptive approach is used to capture the recent relationship between ILI and covariates.

RESULTS

DL with FNN appears to deliver the most competitive predictive performance among the four considered individual models. Combing all four models in a comprehensive BMA framework allows to further improve such predictive evaluation metrics as root mean squared error (RMSE) and mean absolute predictive error (MAPE). Nevertheless, DL with FNN remains the preferred method for predicting locations of influenza peaks.

CONCLUSIONS

The proposed approach can be viewed a feasible alternative to forecast ILI in Hong Kong or other countries where ILI has no constant seasonal trend and influenza data resources are limited. The proposed methodology is easily tractable and computationally efficient.

Developing a synthetic national population to investigate the impact of different cardiovascular disease risk management strategies: A derivation and validation study

BACKGROUND

Many national cardiovascular disease (CVD) risk factor management guidelines now recommend that drug treatment decisions should be informed primarily by patients' multi-variable predicted risk of CVD, rather than on the basis of single risk factor thresholds. To investigate the potential impact of treatment guidelines based on CVD risk thresholds at a national level requires individual level data representing the multi-variable CVD risk factor profiles for a country's total adult population. As these data are seldom, if ever, available, we aimed to create a synthetic population, representing the joint CVD risk factor distributions of the adult New Zealand population.

METHODS AND RESULTS

A synthetic population of 2,451,278 individuals, representing the actual age, gender, ethnicity and social deprivation composition of people aged 30-84 years who completed the 2013 New Zealand census was generated using Monte Carlo sampling. Each 'synthetic' person was then probabilistically assigned values of the remaining cardiovascular disease (CVD) risk factors required for predicting their CVD risk, based on data from the national census national hospitalisation and drug dispensing databases and a large regional cohort study, using Monte Carlo sampling and multiple imputation. Where possible, the synthetic population CVD risk distributions for each non-demographic risk factor were validated against independent New Zealand data sources.

CONCLUSIONS

We were able to develop a synthetic national population with realistic multi-variable CVD risk characteristics. The construction of this population is the first step in the development of a micro-simulation model intended to investigate the likely impact of a range of national CVD risk management strategies that will inform CVD risk management guideline updates in New Zealand and elsewhere.

Forecasting seasonal influenza with a state-space SIR model

Seasonal influenza is a serious public health and societal problem due to its consequences resulting from absenteeism, hospitalizations, and deaths. The overall burden of influenza is captured by the Centers for Disease Control and Prevention's influenza-like illness network, which provides invaluable information about the current incidence. This information is used to provide decision support regarding prevention and response efforts. Despite the relatively rich surveillance data and the recurrent nature of seasonal influenza, forecasting the timing and intensity of seasonal influenza in the U.S. remains challenging because the form of the disease transmission process is uncertain, the disease dynamics are only partially observed, and the public health observations are noisy. Fitting a probabilistic state-space model motivated by a deterministic mathematical model [a susceptible-infectious-recovered (SIR) model] is a promising approach for forecasting seasonal influenza while simultaneously accounting for multiple sources of uncertainty. A significant finding of this work is the importance of thoughtfully specifying the prior, as results critically depend on its specification. Our conditionally specified prior allows us to exploit known relationships between latent SIR initial conditions and parameters and functions of surveillance data. We demonstrate advantages of our approach relative to alternatives via a forecasting comparison using several forecast accuracy metrics.

Twitter Influenza Surveillance: Quantifying Seasonal Misdiagnosis Patterns and their Impact on Surveillance Estimates

BACKGROUND

Influenza (flu) surveillance using Twitter data can potentially save lives and increase efficiency by providing governments and healthcare organizations with greater situational awareness. However, research is needed to determine the impact of Twitter users' misdiagnoses on surveillance estimates.

OBJECTIVE

This study establishes the importance of Twitter users' misdiagnoses by showing that Twitter flu surveillance in the United States failed during the 2011-2012 flu season, estimates the extent of misdiagnoses, and tests several methods for reducing the adverse effects of misdiagnoses.

METHODS

Metrics representing flu prevalence, seasonal misdiagnosis patterns, diagnosis uncertainty, flu symptoms, and noise were produced using Twitter data in conjunction with OpenSextant for geo-inferencing, and a maximum entropy classifier for identifying tweets related to illness. These metrics were tested for correlations with World Health Organization (WHO) positive specimen counts of flu from 2011 to 2014.

RESULTS

Twitter flu surveillance erroneously indicated a typical flu season during 2011-2012, even though the flu season peaked three months late, and erroneously indicated plateaus of flu tweets before the 2012-2013 and 2013-2014 flu seasons. Enhancements based on estimates of misdiagnoses removed the erroneous plateaus and increased the Pearson correlation coefficients by .04 and .23, but failed to correct the 2011-2012 flu season estimate. A rough estimate indicates that approximately 40% of flu tweets reflected misdiagnoses.

CONCLUSIONS

Further research into factors affecting Twitter users' misdiagnoses, in conjunction with data from additional atypical flu seasons, is needed to enable Twitter flu surveillance systems to produce reliable estimates during atypical flu seasons.

Forecasting Influenza Outbreaks in Boroughs and Neighborhoods of New York City

The ideal spatial scale, or granularity, at which infectious disease incidence should be monitored and forecast has been little explored. By identifying the optimal granularity for a given disease and host population, and matching surveillance and prediction efforts to this scale, response to emergent and recurrent outbreaks can be improved. Here we explore how granularity and representation of spatial structure affect influenza forecast accuracy within New York City. We develop network models at the borough and neighborhood levels, and use them in conjunction with surveillance data and a data assimilation method to forecast influenza activity. These forecasts are compared to an alternate system that predicts influenza for each borough or neighborhood in isolation. At the borough scale, influenza epidemics are highly synchronous despite substantial differences in intensity, and inclusion of network connectivity among boroughs generally improves forecast accuracy. At the neighborhood scale, we observe much greater spatial heterogeneity among influenza outbreaks including substantial differences in local outbreak timing and structure; however, inclusion of the network model structure generally degrades forecast accuracy. One notable exception is that local outbreak onset, particularly when signal is modest, is better predicted with the network model. These findings suggest that observation and forecast at sub-municipal scales within New York City provides richer, more discriminant information on influenza incidence, particularly at the neighborhood scale where greater heterogeneity exists, and that the spatial spread of influenza among localities can be forecast.

Influenza, or the flu, causes significant morbidity and mortality during both seasonal and pandemic outbreaks. Recently developed influenza forecast systems have the potential to aid public health planning for and mitigation of the burden of this disease. However, current forecasts are often generated at spatial scales (e.g. national level) that are coarser than the scales at which public health measures and interventions are implemented (e.g. community level). Here we build and test influenza forecast systems at county and community levels, which either include spatial connectivity among locations or are run in isolation. We test these four flu forecast systems (i.e. 2 models × 2 spatial scales) using data collected from 2008 to 2013, including the 2009 pandemic, for the five boroughs (corresponding to county level) and 42 neighborhoods (corresponding to community level) in New York City. We compare the performance of the four forecast systems in predicting the onset, duration, and intensity of flu outbreaks and found that the performance varied by spatial scale (borough vs. neighborhood), season (non-pandemic vs. pandemic) and metric (onset, duration, and intensity). In general, the inclusion of spatial network connectivity in the forecast model improves forecast accuracy at the borough scale but degrades accuracy at the neighborhood scale.

Superensemble forecasts of dengue outbreaks

In recent years, a number of systems capable of predicting future infectious disease incidence have been developed. As more of these systems are operationalized, it is important that the forecasts generated by these different approaches be formally reconciled so that individual forecast error and bias are reduced. Here we present a first example of such multi-system, or superensemble, forecast. We develop three distinct systems for predicting dengue, which are applied retrospectively to forecast outbreak characteristics in San Juan, Puerto Rico. We then use Bayesian averaging methods to combine the predictions from these systems and create superensemble forecasts. We demonstrate that on average, the superensemble approach produces more accurate forecasts than those made from any of the individual forecasting systems.

Accurate estimation of influenza epidemics using Google search data via ARGO

Accurate real-time tracking of influenza outbreaks helps public health officials make timely and meaningful decisions that could save lives. We propose an influenza tracking model, ARGO (AutoRegression with GOogle search data), that uses publicly available online search data. In addition to having a rigorous statistical foundation, ARGO outperforms all previously available Google-search-based tracking models, including the latest version of Google Flu Trends, even though it uses only low-quality search data as input from publicly available Google Trends and Google Correlate websites. ARGO not only incorporates the seasonality in influenza epidemics but also captures changes in people's online search behavior over time. ARGO is also flexible, self-correcting, robust, and scalable, making it a potentially powerful tool that can be used for real-time tracking of other social events at multiple temporal and spatial resolutions.

EpiCaster: An Integrated Web Application For Situation Assessment and Forecasting of Global Epidemics

Public health decision makers need access to high resolution situation assessment tools for understanding the extent of various epidemics in different regions of the world. In addition, they need insights into the future course of epidemics by way of forecasts. Such forecasts are essential for planning the allocation of limited resources and for implementing several policy-level and behavioral intervention strategies. The need for such forecasting systems became evident in the wake of the recent Ebola outbreak in West Africa. We have developed EpiCaster, an integrated Web application for situation assessment and forecasting of various epidemics, such as Flu and Ebola, that are prevalent in different regions of the world. Using EpiCaster, users can assess the magnitude and severity of different epidemics at highly resolved spatio-temporal levels. EpiCaster provides time-varying heat maps and graphical plots to view trends in the disease dynamics. EpiCaster also allows users to visualize data gathered through surveillance mechanisms, such as Google Flu Trends (GFT) and the World Health Organization (WHO). The forecasts provided by EpiCaster are generated using different epidemiological models, and the users can select the models through the interface to filter the corresponding forecasts. EpiCaster also allows the users to study epidemic propagation in the presence of a number of intervention strategies specific to certain diseases. Here we describe the modeling techniques, methodologies and computational infrastructure that EpiCaster relies on to support large-scale predictive analytics for situation assessment and forecasting of global epidemics.

Flexible Modeling of Epidemics with an Empirical Bayes Framework

Seasonal influenza epidemics cause consistent, considerable, widespread loss annually in terms of economic burden, morbidity, and mortality. With access to accurate and reliable forecasts of a current or upcoming influenza epidemic’s behavior, policy makers can design and implement more effective countermeasures. This past year, the Centers for Disease Control and Prevention hosted the “Predict the Influenza Season Challenge”, with the task of predicting key epidemiological measures for the 2013–2014 U.S. influenza season with the help of digital surveillance data. We developed a framework for in-season forecasts of epidemics using a semiparametric Empirical Bayes framework, and applied it to predict the weekly percentage of outpatient doctors visits for influenza-like illness, and the season onset, duration, peak time, and peak height, with and without using Google Flu Trends data. Previous work on epidemic modeling has focused on developing mechanistic models of disease behavior and applying time series tools to explain historical data. However, tailoring these models to certain types of surveillance data can be challenging, and overly complex models with many parameters can compromise forecasting ability. Our approach instead produces possibilities for the epidemic curve of the season of interest using modified versions of data from previous seasons, allowing for reasonable variations in the timing, pace, and intensity of the seasonal epidemics, as well as noise in observations. Since the framework does not make strict domain-specific assumptions, it can easily be applied to some other diseases with seasonal epidemics. This method produces a complete posterior distribution over epidemic curves, rather than, for example, solely point predictions of forecasting targets. We report prospective influenza-like-illness forecasts made for the 2013–2014 U.S. influenza season, and compare the framework’s cross-validated prediction error on historical data to that of a variety of simpler baseline predictors.

Influenza epidemics occur annually, and incur significant losses in terms of lost productivity, sickness, and death. Policy makers employ countermeasures, such as vaccination campaigns, to combat the occurrence and spread of infectious diseases, but epidemics exhibit a wide range of behavior, which makes designing and planning these efforts difficult. Accurate and reliable numerical forecasts of how an epidemic will behave, as well as advance notice of key events, could enable policy makers to further specialize countermeasures for a particular season. While a large amount of work already exists on modeling epidemics in past seasons, work on forecasting is relatively sparse. Specially tailored models for historical data may be overly strict and fail to produce behavior similar to the current season. We designed a framework for predicting epidemics without making strong assumptions about how the disease propagates by relying on slightly modified versions of past epidemics to form possibilities for the current season. We report forecasts generated for the 2013–2014 Centers for Disease Control and Prevention (CDC) “Predict the Influenza Season Challenge”, and assess its accuracy retrospectively.

Infectious disease modeling methods as tools for informing response to novel influenza viruses of unknown pandemic potential

The rising importance of infectious disease modeling makes this an appropriate time for a guide for public health practitioners tasked with preparing for, and responding to, an influenza pandemic. We list several questions that public health practitioners commonly ask about pandemic influenza and match these with analytical methods, giving details on when during a pandemic the methods can be used, how long it might take to implement them, and what data are required. Although software to perform these tasks is available, care needs to be taken to understand: (1) the type of data needed, (2) the implementation of the methods, and (3) the interpretation of results in terms of model uncertainty and sensitivity. Public health leaders can use this article to evaluate the modeling literature, determine which methods can provide appropriate evidence for decision-making, and to help them request modeling work from in-house teams or academic groups.

Forecasting the 2013-2014 influenza season using Wikipedia

Infectious diseases are one of the leading causes of morbidity and mortality around the world; thus, forecasting their impact is crucial for planning an effective response strategy. According to the Centers for Disease Control and Prevention (CDC), seasonal influenza affects 5% to 20% of the U.S. population and causes major economic impacts resulting from hospitalization and absenteeism. Understanding influenza dynamics and forecasting its impact is fundamental for developing prevention and mitigation strategies. We combine modern data assimilation methods with Wikipedia access logs and CDC influenza-like illness (ILI) reports to create a weekly forecast for seasonal influenza. The methods are applied to the 2013-2014 influenza season but are sufficiently general to forecast any disease outbreak, given incidence or case count data. We adjust the initialization and parametrization of a disease model and show that this allows us to determine systematic model bias. In addition, we provide a way to determine where the model diverges from observation and evaluate forecast accuracy. Wikipedia article access logs are shown to be highly correlated with historical ILI records and allow for accurate prediction of ILI data several weeks before it becomes available. The results show that prior to the peak of the flu season, our forecasting method produced 50% and 95% credible intervals for the 2013-2014 ILI observations that contained the actual observations for most weeks in the forecast. However, since our model does not account for re-infection or multiple strains of influenza, the tail of the epidemic is not predicted well after the peak of flu season has passed.

Improving Google Flu Trends estimates for the United States through transformation

Google Flu Trends (GFT) uses Internet search queries in an effort to provide early warning of increases in influenza-like illness (ILI). In the United States, GFT estimates the percentage of physician visits related to ILI (%ILINet) reported by the Centers for Disease Control and Prevention (CDC). However, during the 2012-13 influenza season, GFT overestimated %ILINet by an appreciable amount and estimated the peak in incidence three weeks late. Using data from 2010-14, we investigated the relationship between GFT estimates (%GFT) and %ILINet. Based on the relationship between the relative change in %GFT and the relative change in %ILINet, we transformed %GFT estimates to better correspond with %ILINet values. In 2010-13, our transformed %GFT estimates were within ± 10% of %ILINet values for 17 of the 29 weeks that %ILINet was above the seasonal baseline value determined by the CDC; in contrast, the original %GFT estimates were within ± 10% of %ILINet values for only two of these 29 weeks. Relative to the %ILINet peak in 2012-13, the peak in our transformed %GFT estimates was 2% lower and one week later, whereas the peak in the original %GFT estimates was 74% higher and three weeks later. The same transformation improved %GFT estimates using the recalibrated 2013 GFT model in early 2013-14. Our transformed %GFT estimates can be calculated approximately one week before %ILINet values are reported by the CDC and the transformation equation was stable over the time period investigated (2010-13). We anticipate our results will facilitate future use of GFT.

Twitter improves influenza forecasting

Accurate disease forecasts are imperative when preparing for influenza epidemic outbreaks; nevertheless, these forecasts are often limited by the time required to collect new, accurate data. In this paper, we show that data from the microblogging community Twitter significantly improves influenza forecasting. Most prior influenza forecast models are tested against historical influenza-like illness (ILI) data from the U.S. Centers for Disease Control and Prevention (CDC). These data are released with a one-week lag and are often initially inaccurate until the CDC revises them weeks later. Since previous studies utilize the final, revised data in evaluation, their evaluations do not properly determine the effectiveness of forecasting. Our experiments using ILI data available at the time of the forecast show that models incorporating data derived from Twitter can reduce forecasting error by 17-30% over a baseline that only uses historical data. For a given level of accuracy, using Twitter data produces forecasts that are two to four weeks ahead of baseline models. Additionally, we find that models using Twitter data are, on average, better predictors of influenza prevalence than are models using data from Google Flu Trends, the leading web data source.

What are health-related users tweeting? A qualitative content analysis of health-related users and their messages on twitter

Background

Twitter is home to many health professionals who send messages about a variety of health-related topics. Amid concerns about physicians posting inappropriate content online, more in-depth knowledge about these messages is needed to understand health professionals’ behavior on Twitter.

Objective

Our goal was to characterize the content of Twitter messages, specifically focusing on health professionals and their tweets relating to health.

Methods

We performed an in-depth content analysis of 700 tweets. Qualitative content analysis was conducted on tweets by health users on Twitter. The primary objective was to describe the general type of content (ie, health-related versus non-health related) on Twitter authored by health professionals and further to describe health-related tweets on the basis of the type of statement made. Specific attention was given to whether a tweet was personal (as opposed to professional) or made a claim that users would expect to be supported by some level of medical evidence (ie, a “testable” claim). A secondary objective was to compare content types among different users, including patients, physicians, nurses, health care organizations, and others.

Results

Health-related users are posting a wide range of content on Twitter. Among health-related tweets, 53.2% (184/346) contained a testable claim. Of health-related tweets by providers, 17.6% (61/346) were personal in nature; 61% (59/96) made testable statements. While organizations and businesses use Twitter to promote their services and products, patient advocates are using this tool to share their personal experiences with health.

Conclusions

Twitter users in health-related fields tweet about both testable claims and personal experiences. Future work should assess the relationship between testable tweets and the actual level of evidence supporting them, including how Twitter users—especially patients—interpret the content of tweets posted by health providers.

Influenza forecasting in human populations: a scoping review

Forecasts of influenza activity in human populations could help guide key preparedness tasks. We conducted a scoping review to characterize these methodological approaches and identify research gaps. Adapting the PRISMA methodology for systematic reviews, we searched PubMed, CINAHL, Project Euclid, and Cochrane Database of Systematic Reviews for publications in English since January 1, 2000 using the terms “influenza AND (forecast* OR predict*)”, excluding studies that did not validate forecasts against independent data or incorporate influenza-related surveillance data from the season or pandemic for which the forecasts were applied. We included 35 publications describing population-based (N = 27), medical facility-based (N = 4), and regional or global pandemic spread (N = 4) forecasts. They included areas of North America (N = 15), Europe (N = 14), and/or Asia-Pacific region (N = 4), or had global scope (N = 3). Forecasting models were statistical (N = 18) or epidemiological (N = 17). Five studies used data assimilation methods to update forecasts with new surveillance data. Models used virological (N = 14), syndromic (N = 13), meteorological (N = 6), internet search query (N = 4), and/or other surveillance data as inputs. Forecasting outcomes and validation metrics varied widely. Two studies compared distinct modeling approaches using common data, 2 assessed model calibration, and 1 systematically incorporated expert input. Of the 17 studies using epidemiological models, 8 included sensitivity analysis. This review suggests need for use of good practices in influenza forecasting (e.g., sensitivity analysis); direct comparisons of diverse approaches; assessment of model calibration; integration of subjective expert input; operational research in pilot, real-world applications; and improved mutual understanding among modelers and public health officials.

An operational epidemiological model for calibrating agent-based simulations of pandemic influenza outbreaks

Uncertainty of pandemic influenza viruses continue to cause major preparedness challenges for public health policymakers. Decisions to mitigate influenza outbreaks often involve tradeoff between the social costs of interventions (e.g., school closure) and the cost of uncontrolled spread of the virus. To achieve a balance, policymakers must assess the impact of mitigation strategies once an outbreak begins and the virus characteristics are known. Agent-based (AB) simulation is a useful tool for building highly granular disease spread models incorporating the epidemiological features of the virus as well as the demographic and social behavioral attributes of tens of millions of affected people. Such disease spread models provide excellent basis on which various mitigation strategies can be tested, before they are adopted and implemented by the policymakers. However, to serve as a testbed for the mitigation strategies, the AB simulation models must be operational. A critical requirement for operational AB models is that they are amenable for quick and simple calibration. The calibration process works as follows: the AB model accepts information available from the field and uses those to update its parameters such that some of its outputs in turn replicate the field data. In this paper, we present our epidemiological model based calibration methodology that has a low computational complexity and is easy to interpret. Our model accepts a field estimate of the basic reproduction number, and then uses it to update (calibrate) the infection probabilities in a way that its effect combined with the effects of the given virus epidemiology, demographics, and social behavior results in an infection pattern yielding a similar value of the basic reproduction number. We evaluate the accuracy of the calibration methodology by applying it for an AB simulation model mimicking a regional outbreak in the US. The calibrated model is shown to yield infection patterns closely replicating the input estimates of the basic reproduction number. The calibration method is also tested to replicate an initial infection incidence trend for a H1N1 outbreak like that of 2009.

A systematic review of studies on forecasting the dynamics of influenza outbreaks

Forecasting the dynamics of influenza outbreaks could be useful for decision-making regarding the allocation of public health resources. Reliable forecasts could also aid in the selection and implementation of interventions to reduce morbidity and mortality due to influenza illness. This paper reviews methods for influenza forecasting proposed during previous influenza outbreaks and those evaluated in hindsight. We discuss the various approaches, in addition to the variability in measures of accuracy and precision of predicted measures. PubMed and Google Scholar searches for articles on influenza forecasting retrieved sixteen studies that matched the study criteria. We focused on studies that aimed at forecasting influenza outbreaks at the local, regional, national, or global level. The selected studies spanned a wide range of regions including USA, Sweden, Hong Kong, Japan, Singapore, United Kingdom, Canada, France, and Cuba. The methods were also applied to forecast a single measure or multiple measures. Typical measures predicted included peak timing, peak height, daily/weekly case counts, and outbreak magnitude. Due to differences in measures used to assess accuracy, a single estimate of predictive error for each of the measures was difficult to obtain. However, collectively, the results suggest that these diverse approaches to influenza forecasting are capable of capturing specific outbreak measures with some degree of accuracy given reliable data and correct disease assumptions. Nonetheless, several of these approaches need to be evaluated and their performance quantified in real-time predictions.

National and local influenza surveillance through Twitter: an analysis of the 2012-2013 influenza epidemic

Social media have been proposed as a data source for influenza surveillance because they have the potential to offer real-time access to millions of short, geographically localized messages containing information regarding personal well-being. However, accuracy of social media surveillance systems declines with media attention because media attention increases "chatter" - messages that are about influenza but that do not pertain to an actual infection - masking signs of true influenza prevalence. This paper summarizes our recently developed influenza infection detection algorithm that automatically distinguishes relevant tweets from other chatter, and we describe our current influenza surveillance system which was actively deployed during the full 2012-2013 influenza season. Our objective was to analyze the performance of this system during the most recent 2012-2013 influenza season and to analyze the performance at multiple levels of geographic granularity, unlike past studies that focused on national or regional surveillance. Our system's influenza prevalence estimates were strongly correlated with surveillance data from the Centers for Disease Control and Prevention for the United States (r = 0.93, p < 0.001) as well as surveillance data from the Department of Health and Mental Hygiene of New York City (r = 0.88, p < 0.001). Our system detected the weekly change in direction (increasing or decreasing) of influenza prevalence with 85% accuracy, a nearly twofold increase over a simpler model, demonstrating the utility of explicitly distinguishing infection tweets from other chatter.