Counteracting structural errors in ensemble forecast of influenza outbreaks

Application progress of ensemble forecast technology in influenza forecast based on infectious disease model

To comprehensively understand the application progress of ensemble forecast technology in influenza forecast based on infectious disease model, so as to provide scientific references for further research. In this study, two keywords of "influenza" and "ensemble forecast" are selected to search and select the relevant literatures, which are then outlined and summarized. It is found that: In recent years, some studies about ensemble forecast technology for influenza have been reported in the literature, and some well-performed influenza ensemble forecast systems have already been operationally implemented and provide references for scientific prevention and control. In general, ensemble forecast can well represent various uncertainties in forecasting influenza cases based on infectious disease models, and can achieve more accurate forecasts and more valuable information than single deterministic forecast. However, there are still some shortcomings in the current studies, it is suggested that scientists engaged in influenza forecast based on infectious disease models strengthen cooperation with scholars in the field of numerical weather forecast, which is expected to further improve the skills and application level of ensemble forecast for influenza.

Pharmacoinformatics and Breed-Based De Novo Hybridization Studies to Develop New Neuraminidase Inhibitors as Potential Anti-Influenza Agents

Influenza represents a profoundly transmissible viral ailment primarily afflicting the respiratory system. Neuraminidase inhibitors constitute a class of antiviral therapeutics employed in the management of influenza. These inhibitors impede the liberation of the viral neuraminidase protein, thereby impeding viral dissemination from the infected cell to host cells. As such, neuraminidase has emerged as a pivotal target for mitigating influenza and its associated complications. Here, we apply a de novo hybridization approach based on a breed-centric methodology to elucidate novel neuraminidase inhibitors. The breed technique amalgamates established ligand frameworks with the shared target, neuraminidase, resulting in innovative inhibitor constructs. Molecular docking analysis revealed that the seven synthesized breed molecules (designated Breeds 1-7) formed more robust complexes with the neuraminidase receptor than conventional clinical neuraminidase inhibitors such as zanamivir, oseltamivir, and peramivir. Pharmacokinetic evaluations of the seven breed molecules (Breeds 1-7) demonstrated favorable bioavailability and optimal permeability, all falling within the specified parameters for human application. Molecular dynamics simulations spanning 100 nanoseconds corroborated the stability of these breed molecules within the active site of neuraminidase, shedding light on their structural dynamics. Binding energy assessments, which were conducted through MM-PBSA analysis, substantiated the enduring complexes formed by the seven types of molecules and the neuraminidase receptor. Last, the investigation employed a reaction-based enumeration technique to ascertain the synthetic pathways for the synthesis of the seven breed molecules.

Development of Accurate Long-lead COVID-19 Forecast

Coronavirus disease 2019 (COVID-19) will likely remain a major public health burden; accurate forecast of COVID-19 epidemic outcomes several months into the future is needed to support more proactive planning. Here, we propose strategies to address three major forecast challenges, i.e., error growth, the emergence of new variants, and infection seasonality. Using these strategies in combination we generate retrospective predictions of COVID-19 cases and deaths 6 months in the future for 10 representative US states. Tallied over >25,000 retrospective predictions through September 2022, the forecast approach using all three strategies consistently outperformed a baseline forecast approach without these strategies across different variant waves and locations, for all forecast targets. Overall, probabilistic forecast accuracy improved by 64% and 38% and point prediction accuracy by 133% and 87% for cases and deaths, respectively. Real-time 6-month lead predictions made in early October 2022 suggested large attack rates in most states but a lower burden of deaths than previous waves during October 2022 -March 2023; these predictions are in general accurate compared to reported data. The superior skill of the forecast methods developed here demonstrate means for generating more accurate long-lead forecast of COVID-19 and possibly other infectious diseases.

Dengue transmission dynamics prediction by combining metapopulation networks and Kalman filter algorithm

Predicting the specific magnitude and the temporal peak of the epidemic of individual local outbreaks is critical for infectious disease control. Previous studies have indicated that significant differences in spatial transmission and epidemic magnitude of dengue were influenced by multiple factors, such as mosquito population density, climatic conditions, and population movement patterns. However, there is a lack of studies that combine the above factors to explain their complex nonlinear relationships in dengue transmission and generate accurate predictions. Therefore, to study the complex spatial diffusion of dengue, this research combined the above factors and developed a network model for spatiotemporal transmission prediction of dengue fever using metapopulation networks based on human mobility. For improving the prediction accuracy of the epidemic model, the ensemble adjusted Kalman filter (EAKF), a data assimilation algorithm, was used to iteratively assimilate the observed case data and adjust the model and parameters. Our study demonstrated that the metapopulation network-EAKF system provided accurate predictions for city-level dengue transmission trajectories in retrospective forecasts of 12 cities in Guangdong province, China. Specifically, the system accurately predicts local dengue outbreak magnitude and the temporal peak of the epidemic up to 10 wk in advance. In addition, the system predicted the peak time, peak intensity, and total number of dengue cases more accurately than isolated city-specific forecasts. The general metapopulation assimilation framework presented in our study provides a methodological foundation for establishing an accurate system with finer temporal and spatial resolution for retrospectively forecasting the magnitude and temporal peak of dengue fever outbreaks. These forecasts based on the proposed method can be interoperated to better support intervention decisions and inform the public of potential risks of disease transmission.

Development of Accurate Long-lead COVID-19 Forecast

Coronavirus disease 2019 (COVID-19) will likely remain a major public health burden; accurate forecast of COVID-19 epidemic outcomes several months into the future is needed to support more proactive planning. Here, we propose strategies to address three major forecast challenges, i.e., error growth, the emergence of new variants, and infection seasonality. Using these strategies in combination we generate retrospective predictions of COVID-19 cases and deaths 6 months in the future for 10 representative US states. Tallied over >25,000 retrospective predictions through September 2022, the forecast approach using all three strategies consistently outperformed a baseline forecast approach without these strategies across different variant waves and locations, for all forecast targets. Overall, probabilistic forecast accuracy improved by 64% and 38% and point prediction accuracy by 133% and 87% for cases and deaths, respectively. Real-time 6-month lead predictions made in early October 2022 suggested large attack rates in most states but a lower burden of deaths than previous waves during October 2022 - March 2023; these predictions are in general accurate compared to reported data. The superior skill of the forecast methods developed here demonstrate means for generating more accurate long-lead forecast of COVID-19 and possibly other infectious diseases.

AUTHOR SUMMARY

Infectious disease forecast aims to reliably predict the most likely future outcomes during an epidemic. To date, reliable COVID-19 forecast remains elusive and is needed to support more proactive planning. Here, we pinpoint the major challenges facing COVID-19 forecast and propose three strategies. Comprehensive testing shows the forecast approach using all three strategies consistently outperforms a baseline approach without these strategies across different variant waves and locations in the United States for all forecast targets, improving the probabilistic forecast accuracy by ∼50% and point prediction accuracy by ∼100%. The superior skills of the forecast methods developed here demonstrate means for generating more accurate long-lead COVID-19 forecasts. The methods may be also applicable to other infectious diseases.

ONE SENTENCE SUMMARY

To support more proactive planning, we develop COVID-19 forecast methods that substantially improve accuracy with lead time up to 6 months.

Evaluating the impact of stay-at-home and quarantine measures on COVID-19 spread

BACKGROUND

During the early stage of the COVID-19 pandemic, many countries implemented non-pharmaceutical interventions (NPIs) to control the transmission of SARS-CoV-2, the causative pathogen of COVID-19. Among those NPIs, stay-at-home and quarantine measures were widely adopted and enforced. Understanding the effectiveness of stay-at-home and quarantine measures can inform decision-making and control planning during the ongoing COVID-19 pandemic and for future disease outbreaks.

METHODS

In this study, we use mathematical models to evaluate the impact of stay-at-home and quarantine measures on COVID-19 spread in four cities that experienced large-scale outbreaks in the spring of 2020: Wuhan, New York, Milan, and London. We develop a susceptible-exposed-infected-removed (SEIR)-type model with components of self-isolation and quarantine and couple this disease transmission model with a data assimilation method. By calibrating the model to case data, we estimate key epidemiological parameters before lockdown in each city. We further examine the impact of stay-at-home and quarantine rates on COVID-19 spread after lockdown using counterfactual model simulations.

RESULTS

Results indicate that self-isolation of susceptible population is necessary to contain the outbreak. At a given rate, self-isolation of susceptible population induced by stay-at-home orders is more effective than quarantine of SARS-CoV-2 contacts in reducing effective reproductive numbers [Formula: see text]. Variation in self-isolation and quarantine rates can also considerably affect the duration of outbreaks, attack rates and peak timing. We generate counterfactual simulations to estimate effectiveness of stay-at-home and quarantine measures. Without these two measures, the cumulative confirmed cases could be much higher than reported numbers within 40 days after lockdown in Wuhan, New York, Milan, and London.

CONCLUSIONS

Our findings underscore the essential role of stay-at-home orders and quarantine of SARS-CoV-2 contacts during the early phase of the pandemic.

An ensemble forecast system for tracking dynamics of dengue outbreaks and its validation in China

As a common vector-borne disease, dengue fever remains challenging to predict due to large variations in epidemic size across seasons driven by a number of factors including population susceptibility, mosquito density, meteorological conditions, geographical factors, and human mobility. An ensemble forecast system for dengue fever is first proposed that addresses the difficulty of predicting outbreaks with drastically different scales. The ensemble forecast system based on a susceptible-infected-recovered (SIR) type of compartmental model coupled with a data assimilation method called the ensemble adjusted Kalman filter (EAKF) is constructed to generate real-time forecasts of dengue fever spread dynamics. The model was informed by meteorological and mosquito density information to depict the transmission of dengue virus among human and mosquito populations, and generate predictions. To account for the dramatic variations of outbreak size in different seasons, the effective population size parameter that is sequentially updated to adjust the predicted outbreak scale is introduced into the model. Before optimizing the transmission model, we update the effective population size using the most recent observations and historical records so that the predicted outbreak size is dynamically adjusted. In the retrospective forecast of dengue outbreaks in Guangzhou, China during the 2011-2017 seasons, the proposed forecast model generates accurate projections of peak timing, peak intensity, and total incidence, outperforming a generalized additive model approach. The ensemble forecast system can be operated in real-time and inform control planning to reduce the burden of dengue fever.

The Rise in Norovirus-Related Acute Gastroenteritis During the Fight Against the COVID-19 Pandemic in Southern China

Background: There has been a significant decline in the morbidity of almost all infectious diseases during the COVID-19 pandemic. However, while the incidence of norovirus-related acute gastroenteritis declined in Guangzhou, China during the initial period of the pandemic, incidence increased significantly once the new school year began in September 2020.

Methods: Norovirus-related acute gastroenteritis clusters and outbreaks were assessed in Guangzhou from 2015 to 2020. Medians and interquartile ranges were compared between groups using the Mann–Whitney U-test, and attack rates were calculated.

Results: While 78,579 cases of infectious diarrhea were reported from 2015 to 2019, with an average of 15,716 cases per year, only 12,065 cases of infectious diarrhea were reported in 2020. The numbers of sporadic cases and outbreaks reported from January to August 2020 were lower than the average numbers reported during the same time period each year from 2015 to 2019 but began to increase in September 2020. The number of cases in each reported cluster ranged from 10 to 70 in 2020, with a total of 1,280 cases and an average attack rate of 5.85%. The median number of reported cases, the cumulative number of cases, and the attack rate were higher than the average number reported each year from 2015 to 2019. The intervention time in 2020 was also higher than the average intervention time reported during 2015–2019. The main norovirus genotypes circulating in Guangzhou during 2015–2020 included genogroup 2 type 2 (GII.2) (n = 79, 26.69%), GII.17 (n = 36, 12.16%), GII.3 (n = 27, 9.12%), GII.6 (n = 8, 2.7%), GII.4 Sydney_2012 (n = 7, 2.36%), and GII.4 (n = 6, 2.03%).

Conclusions: Our findings illustrate the importance of maintaining epidemiological surveillance for viral gastroenteritis during the COVID-19 pandemic. Local disease prevention and control institutions need to devote sufficient human resources to control norovirus clusters.

Quantifying the Impact of COVID-19 Nonpharmaceutical Interventions on Influenza Transmission in the United States

BACKGROUND

Nonpharmaceutical interventions (NPIs) have been implemented to suppress transmission of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Evidence indicates that NPIs against coronavirus disease 2019 (COVID-19) may also have effects on transmission of seasonal influenza.

METHODS

In this study, we use an absolute humidity-driven susceptible-infectious-recovered-susceptible (SIRS) model to quantify the reduction of influenza incidence and transmission in the United States and US Department of Health and Human Services regions after implementation of NPIs in 2020. We investigate long-term effect of NPIs on influenza incidence by projecting influenza transmission at the national scale over the next 5 years, using the SIRS model.

RESULTS

We estimate that incidence of influenza A/H1 and B, which circulated in early 2020, was reduced by more than 60% in the United States during the first 10 weeks following implementation of NPIs. The reduction of influenza transmission exhibits clear geographical variation. After the control measures are relaxed, potential accumulation of susceptibility to influenza infection may lead to a large outbreak, the scale of which may be affected by length of the intervention period and duration of immunity to influenza.

CONCLUSIONS

Healthcare systems need to prepare for potential influenza patient surges and advocate vaccination and continued precautions.

Identifying asymptomatic spreaders of antimicrobial-resistant pathogens in hospital settings

Antimicrobial-resistant organisms (AMROs) can colonize people without symptoms for long periods of time, during which these agents can spread unnoticed to other patients in healthcare systems. The accurate identification of asymptomatic spreaders of AMRO in hospital settings is essential for supporting the design of interventions against healthcare-associated infections (HAIs). However, this task remains challenging because of limited observations of colonization and the complicated transmission dynamics occurring within hospitals and the broader community. Here, we study the transmission of methicillin-resistant Staphylococcus aureus (MRSA), a prevalent AMRO, in 66 Swedish hospitals and healthcare facilities with inpatients using a data-driven, agent-based model informed by deidentified real-world hospitalization records. Combining the transmission model, patient-to-patient contact networks, and sparse observations of colonization, we develop and validate an individual-level inference approach that estimates the colonization probability of individual hospitalized patients. For both model-simulated and historical outbreaks, the proposed method supports the more accurate identification of asymptomatic MRSA carriers than other traditional approaches. In addition, in silica control experiments indicate that interventions targeted to inpatients with a high-colonization probability outperform heuristic strategies informed by hospitalization history and contact tracing.

Trade-offs between individual and ensemble forecasts of an emerging infectious disease

Probabilistic forecasts play an indispensable role in answering questions about the spread of newly emerged pathogens. However, uncertainties about the epidemiology of emerging pathogens can make it difficult to choose among alternative model structures and assumptions. To assess the potential for uncertainties about emerging pathogens to affect forecasts of their spread, we evaluated the performance 16 forecasting models in the context of the 2015-2016 Zika epidemic in Colombia. Each model featured a different combination of assumptions about human mobility, spatiotemporal variation in transmission potential, and the number of virus introductions. We found that which model assumptions had the most ensemble weight changed through time. We additionally identified a trade-off whereby some individual models outperformed ensemble models early in the epidemic, but on average the ensembles outperformed all individual models. Our results suggest that multiple models spanning uncertainty across alternative assumptions are necessary to obtain robust forecasts for emerging infectious diseases.

Multiscale influenza forecasting

Influenza forecasting in the United States (US) is complex and challenging due to spatial and temporal variability, nested geographic scales of interest, and heterogeneous surveillance participation. Here we present Dante, a multiscale influenza forecasting model that learns rather than prescribes spatial, temporal, and surveillance data structure and generates coherent forecasts across state, regional, and national scales. We retrospectively compare Dante's short-term and seasonal forecasts for previous flu seasons to the Dynamic Bayesian Model (DBM), a leading competitor. Dante outperformed DBM for nearly all spatial units, flu seasons, geographic scales, and forecasting targets. Dante's sharper and more accurate forecasts also suggest greater public health utility. Dante placed 1st in the Centers for Disease Control and Prevention's prospective 2018/19 FluSight challenge in both the national and regional competition and the state competition. The methodology underpinning Dante can be used in other seasonal disease forecasting contexts having nested geographic scales of interest.

The effect of influenza virus infection on pregnancy outcomes: A systematic review and meta-analysis of cohort studies

BACKGROUND

Adverse pregnancy outcomes are risk factors for neonatal mortality and morbidity. While some studies have demonstrated notable associations between influenza and adverse pregnancy outcomes, the findings have contrasted with other studies. This meta-analysis was conducted to assess the effect of influenza infection on pregnancy outcomes.

METHODS

We searched PubMed, Embase, Cochrane Library and Web of Science from inception to 4 November 2020. Relative risks (RRs) with 95% confidence intervals (CIs) were pooled using random-effects or fixed-effects models.

RESULTS

A total of 17 studies involving 2,351,204 participants were included. Influenza infection increased the risk of stillbirth (RR = 3.62, 95% CI: 1.60-8.20), with no significant effect on preterm birth (RR = 1.17, 95%CI: 0.95-1.45), fetal death (RR = 0.93, 95%CI: 0.73-1.18), small for gestational age (SGA) (RR = 1.10, 95%CI: 0.98-1.24) and low birth weight (LBW) (RR = 1.88, 95%CI: 0.46-7.66). In a subgroup analysis of LBW, the association was evident in studies conducted during the 2009 H1N1 pandemic (RR = 2.28, 95%CI: 1.81-2.87), with no evidence of an association in pre-pandemic or post-pandemic studies.

CONCLUSIONS

Influenza virus infection was associated with an increased risk of stillbirth, but its effect on preterm birth, fetal death, SGA and LBW is still uncertain.

Spatial transmission network construction of influenza-like illness using dynamic Bayesian network and vector-autoregressive moving average model

BACKGROUND

Although vaccination is one of the main countermeasures against influenza epidemic, it is highly essential to make informed prevention decisions to guarantee that limited vaccination resources are allocated to the places where they are most needed. Hence, one of the fundamental steps for decision making in influenza prevention is to characterize its spatio-temporal trend, especially on the key problem about how influenza transmits among adjacent places and how much impact the influenza of one place could have on its neighbors. To solve this problem while avoiding too much additional time-consuming work on data collection, this study proposed a new concept of spatio-temporal route as well as its estimation methods to construct the influenza transmission network.

METHODS

The influenza-like illness (ILI) data of Sichuan province in 21 cities was collected from 2010 to 2016. A joint pattern based on the dynamic Bayesian network (DBN) model and the vector autoregressive moving average (VARMA) model was utilized to estimate the spatio-temporal routes, which were applied to the two stages of learning process respectively, namely structure learning and parameter learning. In structure learning, the first-order conditional dependencies approximation algorithm was used to generate the DBN, which could visualize the spatio-temporal routes of influenza among adjacent cities and infer which cities have impacts on others in influenza transmission. In parameter learning, the VARMA model was adopted to estimate the strength of these impacts. Finally, all the estimated spatio-temporal routes were put together to form the final influenza transmission network.

RESULTS

The results showed that the period of influenza transmission cycle was longer in Western Sichuan and Chengdu Plain than that in Northeastern Sichuan, and there would be potential spatio-temporal routes of influenza from bordering provinces or municipalities into Sichuan province. Furthermore, this study also pointed out several estimated spatio-temporal routes with relatively high strength of associations, which could serve as clues of hot spot areas detection for influenza surveillance.

CONCLUSIONS

This study proposed a new framework for exploring the potentially stable spatio-temporal routes between different places and measuring specific the sizes of transmission effects. It could help making timely and reliable prediction of the spatio-temporal trend of infectious diseases, and further determining the possible key areas of the next epidemic by considering their neighbors' incidence and the transmission relationships.

Optimizing respiratory virus surveillance networks using uncertainty propagation

Infectious disease prevention, control and forecasting rely on sentinel observations; however, many locations lack the capacity for routine surveillance. Here we show that, by using data from multiple sites collectively, accurate estimation and forecasting of respiratory diseases for locations without surveillance is feasible. We develop a framework to optimize surveillance sites that suppresses uncertainty propagation in a networked disease transmission model. Using influenza outbreaks from 35 US states, the optimized system generates better near-term predictions than alternate systems designed using population and human mobility. We also find that monitoring regional population centers serves as a reasonable proxy for the optimized network and could direct surveillance for diseases with limited records. The  proxy method is validated using model simulations for 3,108 US counties and historical data for two other respiratory pathogens - human metapneumovirus and seasonal coronavirus - from 35 US states and can be used to guide systemic allocation of surveillance efforts.

Differential effects of intervention timing on COVID-19 spread in the United States

Assessing the effects of early nonpharmaceutical interventions on coronavirus disease 2019 (COVID-19) spread is crucial for understanding and planning future control measures to combat the pandemic. We use observations of reported infections and deaths, human mobility data, and a metapopulation transmission model to quantify changes in disease transmission rates in U.S. counties from 15 March to 3 May 2020. We find that marked, asynchronous reductions of the basic reproductive number occurred throughout the United States in association with social distancing and other control measures. Counterfactual simulations indicate that, had these same measures been implemented 1 to 2 weeks earlier, substantial cases and deaths could have been averted and that delayed responses to future increased incidence will facilitate a stronger rebound of infections and death. Our findings underscore the importance of early intervention and aggressive control in combatting the COVID-19 pandemic.

Predicting dengue outbreaks at neighbourhood level using human mobility in urban areas

Dengue is a vector-borne disease transmitted by the Aedes genus mosquito. It causes financial burdens on public health systems and considerable morbidity and mortality. Tropical regions in the Americas and Asia are the areas most affected by the virus. Fortaleza is a city with approximately 2.6 million inhabitants in northeastern Brazil that, during the recent decades, has been suffering from endemic dengue transmission, interspersed with larger epidemics. The objective of this paper is to study the impact of human mobility in urban areas on the spread of the dengue virus, and to test whether human mobility data can be used to improve predictions of dengue virus transmission at the neighbourhood level. We present two distinct forecasting systems for dengue transmission in Fortaleza: the first using artificial neural network methods and the second developed using a mechanistic model of disease transmission. We then present enhanced versions of the two forecasting systems that incorporate bus transportation data cataloguing movement among 119 neighbourhoods in Fortaleza. Each forecasting system was used to perform retrospective forecasts for historical dengue outbreaks from 2007 to 2015. Results show that both artificial neural networks and mechanistic models can accurately forecast dengue cases, and that the inclusion of human mobility data substantially improves the performance of both forecasting systems. While the mechanistic models perform better in capturing seasons with large-scale outbreaks, the neural networks more accurately forecast outbreak peak timing, peak intensity and annual dengue time series. These results have two practical implications: they support the creation of public policies from the use of the models created here to combat the disease and help to understand the impact of urban mobility on the epidemic in large cities.

Aggregating forecasts of multiple respiratory pathogens supports more accurate forecasting of influenza-like illness

Influenza-like illness (ILI) is a commonly measured syndromic signal representative of a range of acute respiratory infections. Reliable forecasts of ILI can support better preparation for patient surges in healthcare systems. Although ILI is an amalgamation of multiple pathogens with variable seasonal phasing and attack rates, most existing process-based forecasting systems treat ILI as a single infectious agent. Here, using ILI records and virologic surveillance data, we show that ILI signal can be disaggregated into distinct viral components. We generate separate predictions for six contributing pathogens (influenza A/H1, A/H3, B, respiratory syncytial virus, and human parainfluenza virus types 1-2 and 3), and develop a method to forecast ILI by aggregating these predictions. The relative contribution of each pathogen to the total ILI signal is estimated using a Markov Chain Monte Carlo (MCMC) method upon forecast aggregation. We find highly variable overall contributions from influenza type A viruses across seasons, but relatively stable contributions for the other pathogens. Using historical data from 1997 to 2014 at US national and regional levels, the proposed forecasting system generates improved predictions of both seasonal and near-term targets relative to a baseline method that simulates ILI as a single pathogen. The hierarchical forecasting system can generate predictions for each viral component, as well as infer and predict their contributions to ILI, which may additionally help physicians determine the etiological causes of ILI in clinical settings.

Influencer identification in dynamical complex systems

The integrity and functionality of many real-world complex systems hinge on a small set of pivotal nodes, or influencers. In different contexts, these influencers are defined as either structurally important nodes that maintain the connectivity of networks, or dynamically crucial units that can disproportionately impact certain dynamical processes. In practice, identification of the optimal set of influencers in a given system has profound implications in a variety of disciplines. In this review, we survey recent advances in the study of influencer identification developed from different perspectives, and present state-of-the-art solutions designed for different objectives. In particular, we first discuss the problem of finding the minimal number of nodes whose removal would breakdown the network (i.e. the optimal percolation or network dismantle problem), and then survey methods to locate the essential nodes that are capable of shaping global dynamics with either continuous (e.g. independent cascading models) or discontinuous phase transitions (e.g. threshold models). We conclude the review with a summary and an outlook.

Forecasting national and regional influenza-like illness for the USA

Health planners use forecasts of key metrics associated with influenza-like illness (ILI); near-term weekly incidence, week of season onset, week of peak, and intensity of peak. Here, we describe our participation in a weekly prospective ILI forecasting challenge for the United States for the 2016-17 season and subsequent evaluation of our performance. We implemented a metapopulation model framework with 32 model variants. Variants differed from each other in their assumptions about: the force-of-infection (FOI); use of uninformative priors; the use of discounted historical data for not-yet-observed time points; and the treatment of regions as either independent or coupled. Individual model variants were chosen subjectively as the basis for our weekly forecasts; however, a subset of coupled models were only available part way through the season. Most frequently, during the 2016-17 season, we chose; FOI variants with both school vacations and humidity terms; uninformative priors; the inclusion of discounted historical data for not-yet-observed time points; and coupled regions (when available). Our near-term weekly forecasts substantially over-estimated incidence early in the season when coupled models were not available. However, our forecast accuracy improved in absolute terms and relative to other teams once coupled solutions were available. In retrospective analysis, we found that the 2016-17 season was not typical: on average, coupled models performed better when fit without historically augmented data. Also, we tested a simple ensemble model for the 2016-17 season and found that it underperformed our subjective choice for all forecast targets. In this study, we were able to improve accuracy during a prospective forecasting exercise by coupling dynamics between regions. Although reduction of forecast subjectivity should be a long-term goal, some degree of human intervention is likely to improve forecast accuracy in the medium-term in parallel with the systematic consideration of more sophisticated ensemble approaches.

Systematic biases in disease forecasting - The role of behavior change

In a simple susceptible-infected-recovered (SIR) model, the initial speed at which infected cases increase is indicative of the long-term trajectory of the outbreak. Yet during real-world outbreaks, individuals may modify their behavior and take preventative steps to reduce infection risk. As a consequence, the relationship between the initial rate of spread and the final case count may become tenuous. Here, we evaluate this hypothesis by comparing the dynamics arising from a simple SIR epidemic model with those from a modified SIR model in which individuals reduce contacts as a function of the current or cumulative number of cases. Dynamics with behavior change exhibit significantly reduced final case counts even though the initial speed of disease spread is nearly identical for both of the models. We show that this difference in final size projections depends critically in the behavior change of individuals. These results also provide a rationale for integrating behavior change into iterative forecast models. Hence, we propose to use a Kalman filter to update models with and without behavior change as part of iterative forecasts. When the ground truth outbreak includes behavior change, sequential predictions using a simple SIR model perform poorly despite repeated observations while predictions using the modified SIR model are able to correct for initial forecast errors. These findings highlight the value of incorporating behavior change into baseline epidemic and dynamic forecast models.

A collaborative multiyear, multimodel assessment of seasonal influenza forecasting in the United States

Influenza infects an estimated 9-35 million individuals each year in the United States and is a contributing cause for between 12,000 and 56,000 deaths annually. Seasonal outbreaks of influenza are common in temperate regions of the world, with highest incidence typically occurring in colder and drier months of the year. Real-time forecasts of influenza transmission can inform public health response to outbreaks. We present the results of a multiinstitution collaborative effort to standardize the collection and evaluation of forecasting models for influenza in the United States for the 2010/2011 through 2016/2017 influenza seasons. For these seven seasons, we assembled weekly real-time forecasts of seven targets of public health interest from 22 different models. We compared forecast accuracy of each model relative to a historical baseline seasonal average. Across all regions of the United States, over half of the models showed consistently better performance than the historical baseline when forecasting incidence of influenza-like illness 1 wk, 2 wk, and 3 wk ahead of available data and when forecasting the timing and magnitude of the seasonal peak. In some regions, delays in data reporting were strongly and negatively associated with forecast accuracy. More timely reporting and an improved overall accessibility to novel and traditional data sources are needed to improve forecasting accuracy and its integration with real-time public health decision making.

Assessing the performance of real-time epidemic forecasts: A case study of Ebola in the Western Area region of Sierra Leone, 2014-15

Real-time forecasts based on mathematical models can inform critical decision-making during infectious disease outbreaks. Yet, epidemic forecasts are rarely evaluated during or after the event, and there is little guidance on the best metrics for assessment. Here, we propose an evaluation approach that disentangles different components of forecasting ability using metrics that separately assess the calibration, sharpness and bias of forecasts. This makes it possible to assess not just how close a forecast was to reality but also how well uncertainty has been quantified. We used this approach to analyse the performance of weekly forecasts we generated in real time for Western Area, Sierra Leone, during the 2013-16 Ebola epidemic in West Africa. We investigated a range of forecast model variants based on the model fits generated at the time with a semi-mechanistic model, and found that good probabilistic calibration was achievable at short time horizons of one or two weeks ahead but model predictions were increasingly unreliable at longer forecasting horizons. This suggests that forecasts may have been of good enough quality to inform decision making based on predictions a few weeks ahead of time but not longer, reflecting the high level of uncertainty in the processes driving the trajectory of the epidemic. Comparing forecasts based on the semi-mechanistic model to simpler null models showed that the best semi-mechanistic model variant performed better than the null models with respect to probabilistic calibration, and that this would have been identified from the earliest stages of the outbreak. As forecasts become a routine part of the toolkit in public health, standards for evaluation of performance will be important for assessing quality and improving credibility of mathematical models, and for elucidating difficulties and trade-offs when aiming to make the most useful and reliable forecasts.

Predictability in process-based ensemble forecast of influenza

Process-based models have been used to simulate and forecast a number of nonlinear dynamical systems, including influenza and other infectious diseases. In this work, we evaluate the effects of model initial condition error and stochastic fluctuation on forecast accuracy in a compartmental model of influenza transmission. These two types of errors are found to have qualitatively similar growth patterns during model integration, indicating that dynamic error growth, regardless of source, is a dominant component of forecast inaccuracy. We therefore examine the nonlinear growth of model initial error and compute the fastest growing directions using singular vector analysis. Using this information, we generate perturbations in an ensemble forecast system of influenza to obtain more optimal ensemble spread. In retrospective forecasts of historical outbreaks for 95 US cities from 2003 to 2014, this approach improves short-term forecast of incidence over the next one to four weeks.

Inference and control of the nosocomial transmission of methicillin-resistant Staphylococcus aureus

Methicillin-resistant Staphylococcus aureus (MRSA) is a continued threat to human health in both community and healthcare settings. In hospitals, control efforts would benefit from accurate estimation of asymptomatic colonization and infection importation rates from the community. However, developing such estimates remains challenging due to limited observation of colonization and complicated transmission dynamics within hospitals and the community. Here, we develop an inference framework that can estimate these key quantities by combining statistical filtering techniques, an agent-based model, and real-world patient-to-patient contact networks, and use this framework to infer nosocomial transmission and infection importation over an outbreak spanning 6 years in 66 Swedish hospitals. In particular, we identify a small number of patients with disproportionately high risk of colonization. In retrospective control experiments, interventions targeted to these individuals yield a substantial improvement over heuristic strategies informed by number of contacts, length of stay and contact tracing.

Evaluation of mechanistic and statistical methods in forecasting influenza-like illness

A variety of mechanistic and statistical methods to forecast seasonal influenza have been proposed and are in use; however, the effects of various data issues and design choices (statistical versus mechanistic methods, for example) on the accuracy of these approaches have not been thoroughly assessed. Here, we compare the accuracy of three forecasting approaches-a mechanistic method, a weighted average of two statistical methods and a super-ensemble of eight statistical and mechanistic models-in predicting seven outbreak characteristics of seasonal influenza during the 2016-2017 season at the national and 10 regional levels in the USA. For each of these approaches, we report the effects of real time under- and over-reporting in surveillance systems, use of non-surveillance proxies of influenza activity and manual override of model predictions on forecast quality. Our results suggest that a meta-ensemble of statistical and mechanistic methods has better overall accuracy than the individual methods. Supplementing surveillance data with proxy estimates generally improves the quality of forecasts and transient reporting errors degrade the performance of all three approaches considerably. The improvement in quality from ad hoc and post-forecast changes suggests that domain experts continue to possess information that is not being sufficiently captured by current forecasting approaches.

Forecasting the spatial transmission of influenza in the United States

Recurrent outbreaks of seasonal and pandemic influenza create a need for forecasts of the geographic spread of this pathogen. Although it is well established that the spatial progression of infection is largely attributable to human mobility, difficulty obtaining real-time information on human movement has limited its incorporation into existing infectious disease forecasting techniques. In this study, we develop and validate an ensemble forecast system for predicting the spatiotemporal spread of influenza that uses readily accessible human mobility data and a metapopulation model. In retrospective state-level forecasts for 35 US states, the system accurately predicts local influenza outbreak onset,-i.e., spatial spread, defined as the week that local incidence increases above a baseline threshold-up to 6 wk in advance of this event. In addition, the metapopulation prediction system forecasts influenza outbreak onset, peak timing, and peak intensity more accurately than isolated location-specific forecasts. The proposed framework could be applied to emergent respiratory viruses and, with appropriate modifications, other infectious diseases.

Counteracting structural errors in ensemble forecast of influenza outbreaks

For influenza forecasts generated using dynamical models, forecast inaccuracy is partly attributable to the nonlinear growth of error. As a consequence, quantification of the nonlinear error structure in current forecast models is needed so that this growth can be corrected and forecast skill improved. Here, we inspect the error growth of a compartmental influenza model and find that a robust error structure arises naturally from the nonlinear model dynamics. By counteracting these structural errors, diagnosed using error breeding, we develop a new forecast approach that combines dynamical error correction and statistical filtering techniques. In retrospective forecasts of historical influenza outbreaks for 95 US cities from 2003 to 2014, overall forecast accuracy for outbreak peak timing, peak intensity and attack rate, are substantially improved for predicted lead times up to 10 weeks. This error growth correction method can be generalized to improve the forecast accuracy of other infectious disease dynamical models.

Inaccuracy of influenza forecasts based on dynamical models is partly due to nonlinear error growth. Here the authors address the error structure of a compartmental influenza model, and develop a new improved forecast approach combining dynamical error correction and statistical filtering techniques.