Superensemble forecast of respiratory syncytial virus outbreaks at national, regional, and state levels in the United States

Challenges of COVID-19 Case Forecasting in the US, 2020-2021

During the COVID-19 pandemic, forecasting COVID-19 trends to support planning and response was a priority for scientists and decision makers alike. In the United States, COVID-19 forecasting was coordinated by a large group of universities, companies, and government entities led by the Centers for Disease Control and Prevention and the US COVID-19 Forecast Hub (https://covid19forecasthub.org). We evaluated approximately 9.7 million forecasts of weekly state-level COVID-19 cases for predictions 1-4 weeks into the future submitted by 24 teams from August 2020 to December 2021. We assessed coverage of central prediction intervals and weighted interval scores (WIS), adjusting for missing forecasts relative to a baseline forecast, and used a Gaussian generalized estimating equation (GEE) model to evaluate differences in skill across epidemic phases that were defined by the effective reproduction number. Overall, we found high variation in skill across individual models, with ensemble-based forecasts outperforming other approaches. Forecast skill relative to the baseline was generally higher for larger jurisdictions (e.g., states compared to counties). Over time, forecasts generally performed worst in periods of rapid changes in reported cases (either in increasing or decreasing epidemic phases) with 95% prediction interval coverage dropping below 50% during the growth phases of the winter 2020, Delta, and Omicron waves. Ideally, case forecasts could serve as a leading indicator of changes in transmission dynamics. However, while most COVID-19 case forecasts outperformed a naïve baseline model, even the most accurate case forecasts were unreliable in key phases. Further research could improve forecasts of leading indicators, like COVID-19 cases, by leveraging additional real-time data, addressing performance across phases, improving the characterization of forecast confidence, and ensuring that forecasts were coherent across spatial scales. In the meantime, it is critical for forecast users to appreciate current limitations and use a broad set of indicators to inform pandemic-related decision making.

Determining the timing of respiratory syncytial virus (RSV) epidemics: a systematic review, 2016 to 2021; method categorisation and identification of influencing factors

BackgroundThere is currently no standardised approach to estimate respiratory syncytial virus (RSV) epidemics' timing (or seasonality), a critical information for their effective prevention and control.AimWe aimed to provide an overview of methods to define RSV seasonality and identify factors supporting method choice or interpretation/comparison of seasonal estimates.MethodsWe systematically searched PubMed and Embase (2016-2021) for studies using quantitative approaches to determine the start and end of RSV epidemics. Studies' features (data-collection purpose, location, regional/(sub)national scope), methods, and assessment characteristics (case definitions, sampled population's age, in/outpatient status, setting, diagnostics) were extracted. Methods were categorised by their need of a denominator (i.e. numbers of specimens tested) and their retrospective vs real-time application. Factors worth considering when choosing methods and assessing seasonal estimates were sought by analysing studies.ResultsWe included 32 articles presenting 49 seasonality estimates (18 thereof through the 10% positivity threshold method). Methods were classified into eight categories, two requiring a denominator (1 retrospective; 1 real-time) and six not (3 retrospective; 3 real-time). A wide range of assessment characteristics was observed. Several studies showed that seasonality estimates varied when methods differed, or data with dissimilar assessment characteristics were employed. Five factors (comprising study purpose, application time, assessment characteristics, healthcare system and policies, and context) were identified that could support method choice and result interpretation.ConclusionMethods and assessment characteristics used to define RSV seasonality are heterogeneous. Our categorisation of methods and proposed framework of factors may assist in choosing RSV seasonality methods and interpretating results.

Multi-faceted analysis and prediction for the outbreak of pediatric respiratory syncytial virus

OBJECTIVES

Respiratory syncytial virus (RSV) is a significant cause of pediatric hospitalizations. This article aims to utilize multisource data and leverage the tensor methods to uncover distinct RSV geographic clusters and develop an accurate RSV prediction model for future seasons.

MATERIALS AND METHODS

This study utilizes 5-year RSV data from sources, including medical claims, CDC surveillance data, and Google search trends. We conduct spatiotemporal tensor analysis and prediction for pediatric RSV in the United States by designing (i) a nonnegative tensor factorization model for pediatric RSV diseases and location clustering; (ii) and a recurrent neural network tensor regression model for county-level trend prediction using the disease and location features.

RESULTS

We identify a clustering hierarchy of pediatric diseases: Three common geographic clusters of RSV outbreaks were identified from independent sources, showing an annual RSV trend shifting across different US regions, from the South and Southeast regions to the Central and Northeast regions and then to the West and Northwest regions, while precipitation and temperature were found as correlative factors with the coefficient of determination R2≈0.5, respectively. Our regression model accurately predicted the 2022-2023 RSV season at the county level, achieving R2≈0.3 mean absolute error MAE < 0.4 and a Pearson correlation greater than 0.75, which significantly outperforms the baselines with P-values <.05.

CONCLUSION

Our proposed framework provides a thorough analysis of RSV disease in the United States, which enables healthcare providers to better prepare for potential outbreaks, anticipate increased demand for services and supplies, and save more lives with timely interventions.

Comparing trained and untrained probabilistic ensemble forecasts of COVID-19 cases and deaths in the United States

The U.S. COVID-19 Forecast Hub aggregates forecasts of the short-term burden of COVID-19 in the United States from many contributing teams. We study methods for building an ensemble that combines forecasts from these teams. These experiments have informed the ensemble methods used by the Hub. To be most useful to policy makers, ensemble forecasts must have stable performance in the presence of two key characteristics of the component forecasts: (1) occasional misalignment with the reported data, and (2) instability in the relative performance of component forecasters over time. Our results indicate that in the presence of these challenges, an untrained and robust approach to ensembling using an equally weighted median of all component forecasts is a good choice to support public health decision makers. In settings where some contributing forecasters have a stable record of good performance, trained ensembles that give those forecasters higher weight can also be helpful.

Use of mathematical modelling to assess respiratory syncytial virus epidemiology and interventions: a literature review

Respiratory syncytial virus (RSV) is a leading cause of acute lower respiratory tract infection worldwide, resulting in approximately sixty thousand annual hospitalizations of< 5-year-olds in the United States alone and three million annual hospitalizations globally. The development of over 40 vaccines and immunoprophylactic interventions targeting RSV has the potential to significantly reduce the disease burden from RSV infection in the near future. In the context of RSV, a highly contagious pathogen, dynamic transmission models (DTMs) are valuable tools in the evaluation and comparison of the effectiveness of different interventions. This review, the first of its kind for RSV DTMs, provides a valuable foundation for future modelling efforts and highlights important gaps in our understanding of RSV epidemics. Specifically, we have searched the literature using Web of Science, Scopus, Embase, and PubMed to identify all published manuscripts reporting the development of DTMs focused on the population transmission of RSV. We reviewed the resulting studies and summarized the structure, parameterization, and results of the models developed therein. We anticipate that future RSV DTMs, combined with cost-effectiveness evaluations, will play a significant role in shaping decision making in the development and implementation of intervention programs.

Simulation of the COVID-19 patient flow and investigation of the future patient arrival using a time-series prediction model: a real-case study

COVID-19 looks to be the worst pandemic disease in the last decades due to its number of infected people, deaths, and the staggering demand for healthcare services, especially hospitals. The first and most important step is to identify the patient flow through a certain process. For the second step, there is a crucial need for predicting the future patient arrivals for planning especially at the administrative level of a hospital. This study aims to first simulate the patient flow process and then predict the future entry of patients in a hospital as the case study. Also, according to the system status, this study suggests some policies based on different probable scenarios and assesses the outcome of each decision to improve the policies. The simulation model is conducted by Arena.15 software. The seasonal auto-regressive integrated moving average (SARIMA) model is used for patient’s arrival prediction within 30 days. Different scenarios are evaluated through a data envelopment analysis (DEA) method. The simulation model runs for predicted patient’s arrival for the least efficient scenario and the outputs compare the base run scenario. Results show that the system collapses after 14 days according to the predictions and simulation and the bottleneck of the ICU and CCU departments becomes problematic. Hospitals can use simulation and also prediction tools to avoid the crisis to plan for the future in the pandemic.

The roles of machine learning methods in limiting the spread of deadly diseases: A systematic review

Machine learning (ML) methods can be leveraged to prevent the spread of deadly infectious disease outbreak (e.g., COVID-19). This can be done by applying machine learning methods in predicting and detecting the deadly infectious disease. Most reviews did not discuss about the machine learning algorithms, datasets and performance measurements used for various applications in predicting and detecting the deadly infectious disease. In contrast, this paper outlines the literature review based on two major ways (e.g., prediction, detection) to limit the spread of deadly disease outbreaks. Hence, this study aims to investigate the state of the art, challenges and future works of leveraging ML methods to detect and predict deadly disease outbreaks according to two categories mentioned earlier. Specifically, this study provides a review on various approaches (e.g., individual and ensemble models), types of datasets, parameters or variables and performance measures used in the previous works. The literature review included all articles from journals and conference proceedings published from 2010 through 2020 in Scopus indexed databases using the search terms Predicting Disease Outbreaks and/or Detecting Disease using Machine Learning. The findings from this review focus on commonly used machine learning approaches, challenges and future works to limit the spread of deadly disease outbreaks through preventions and detections.

COVID-19 Outbreak Prediction with Machine Learning

Several outbreak prediction models for COVID-19 are being used by officials around the world to make informed decisions and enforce relevant control measures. Among the standard models for COVID-19 global pandemic prediction, simple epidemiological and statistical models have received more attention by authorities, and these models are popular in the media. Due to a high level of uncertainty and lack of essential data, standard models have shown low accuracy for long-term prediction. Although the literature includes several attempts to address this issue, the essential generalization and robustness abilities of existing models need to be improved. This paper presents a comparative analysis of machine learning and soft computing models to predict the COVID-19 outbreak as an alternative to susceptible–infected–recovered (SIR) and susceptible-exposed-infectious-removed (SEIR) models. Among a wide range of machine learning models investigated, two models showed promising results (i.e., multi-layered perceptron, MLP; and adaptive network-based fuzzy inference system, ANFIS). Based on the results reported here, and due to the highly complex nature of the COVID-19 outbreak and variation in its behavior across nations, this study suggests machine learning as an effective tool to model the outbreak. This paper provides an initial benchmarking to demonstrate the potential of machine learning for future research. This paper further suggests that a genuine novelty in outbreak prediction can be realized by integrating machine learning and SEIR models.

COVID-19 Pandemic Prediction for Hungary; A Hybrid Machine Learning Approach

Several epidemiological models are being used around the world to project the number of infected individuals and the mortality rates of the COVID-19 outbreak. Advancing accurate prediction models is of utmost importance to take proper actions. Due to the lack of essential data and uncertainty, the epidemiological models have been challenged regarding the delivery of higher accuracy for long-term prediction. As an alternative to the susceptible-infected-resistant (SIR)-based models, this study proposes a hybrid machine learning approach to predict the COVID-19, and we exemplify its potential using data from Hungary. The hybrid machine learning methods of adaptive network-based fuzzy inference system (ANFIS) and multi-layered perceptron-imperialist competitive algorithm (MLP-ICA) are proposed to predict time series of infected individuals and mortality rate. The models predict that by late May, the outbreak and the total morality will drop substantially. The validation is performed for 9 days with promising results, which confirms the model accuracy. It is expected that the model maintains its accuracy as long as no significant interruption occurs. This paper provides an initial benchmarking to demonstrate the potential of machine learning for future research.

Projections of epidemic transmission and estimation of vaccination impact during an ongoing Ebola virus disease outbreak in Northeastern Democratic Republic of Congo, as of Feb. 25, 2019

BACKGROUND

As of February 25, 2019, 875 cases of Ebola virus disease (EVD) were reported in North Kivu and Ituri Provinces, Democratic Republic of Congo. Since the beginning of October 2018, the outbreak has largely shifted into regions in which active armed conflict has occurred, and in which EVD cases and their contacts have been difficult for health workers to reach. We used available data on the current outbreak, with case-count time series from prior outbreaks, to project the short-term and long-term course of the outbreak.

METHODS

For short- and long-term projections, we modeled Ebola virus transmission using a stochastic branching process that assumes gradually quenching transmission rates estimated from past EVD outbreaks, with outbreak trajectories conditioned on agreement with the course of the current outbreak, and with multiple levels of vaccination coverage. We used two regression models to estimate similar projection periods. Short- and long-term projections were estimated using negative binomial autoregression and Theil-Sen regression, respectively. We also used Gott's rule to estimate a baseline minimum-information projection. We then constructed an ensemble of forecasts to be compared and recorded for future evaluation against final outcomes. From August 20, 2018 to February 25, 2019, short-term model projections were validated against known case counts.

RESULTS

During validation of short-term projections, from one week to four weeks, we found models consistently scored higher on shorter-term forecasts. Based on case counts as of February 25, the stochastic model projected a median case count of 933 cases by February 18 (95% prediction interval: 872-1054) and 955 cases by March 4 (95% prediction interval: 874-1105), while the auto-regression model projects median case counts of 889 (95% prediction interval: 876-933) and 898 (95% prediction interval: 877-983) cases for those dates, respectively. Projected median final counts range from 953 to 1,749. Although the outbreak is already larger than all past Ebola outbreaks other than the 2013-2016 outbreak of over 26,000 cases, our models do not project that it is likely to grow to that scale. The stochastic model estimates that vaccination coverage in this outbreak is lower than reported in its trial setting in Sierra Leone.

CONCLUSIONS

Our projections are concentrated in a range up to about 300 cases beyond those already reported. While a catastrophic outbreak is not projected, it is not ruled out, and prevention and vigilance are warranted. Prospective validation of our models in real time allowed us to generate more accurate short-term forecasts, and this process may prove useful for future real-time short-term forecasting. We estimate that transmission rates are higher than would be seen under target levels of 62% coverage due to contact tracing and vaccination, and this model estimate may offer a surrogate indicator for the outbreak response challenges.