1
|
Chowell G, Tariq A, Dahal S, Bleichrodt A, Luo R, Hyman JM. SpatialWavePredict: a tutorial-based primer and toolbox for forecasting growth trajectories using the ensemble spatial wave sub-epidemic modeling framework. BMC Med Res Methodol 2024; 24:131. [PMID: 38849766 PMCID: PMC11157887 DOI: 10.1186/s12874-024-02241-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2023] [Accepted: 05/06/2024] [Indexed: 06/09/2024] Open
Abstract
BACKGROUND Dynamical mathematical models defined by a system of differential equations are typically not easily accessible to non-experts. However, forecasts based on these types of models can help gain insights into the mechanisms driving the process and may outcompete simpler phenomenological growth models. Here we introduce a friendly toolbox, SpatialWavePredict, to characterize and forecast the spatial wave sub-epidemic model, which captures diverse wave dynamics by aggregating multiple asynchronous growth processes and has outperformed simpler phenomenological growth models in short-term forecasts of various infectious diseases outbreaks including SARS, Ebola, and the early waves of the COVID-19 pandemic in the US. RESULTS This tutorial-based primer introduces and illustrates a user-friendly MATLAB toolbox for fitting and forecasting time-series trajectories using an ensemble spatial wave sub-epidemic model based on ordinary differential equations. Scientists, policymakers, and students can use the toolbox to conduct real-time short-term forecasts. The five-parameter epidemic wave model in the toolbox aggregates linked overlapping sub-epidemics and captures a rich spectrum of epidemic wave dynamics, including oscillatory wave behavior and plateaus. An ensemble strategy aims to improve forecasting performance by combining the resulting top-ranked models. The toolbox provides a tutorial for forecasting time-series trajectories, including the full uncertainty distribution derived through parametric bootstrapping, which is needed to construct prediction intervals and evaluate their accuracy. Functions are available to assess forecasting performance, estimation methods, error structures in the data, and forecasting horizons. The toolbox also includes functions to quantify forecasting performance using metrics that evaluate point and distributional forecasts, including the weighted interval score. CONCLUSIONS We have developed the first comprehensive toolbox to characterize and forecast time-series data using an ensemble spatial wave sub-epidemic wave model. As an epidemic situation or contagion occurs, the tools presented in this tutorial can facilitate policymakers to guide the implementation of containment strategies and assess the impact of control interventions. We demonstrate the functionality of the toolbox with examples, including a tutorial video, and is illustrated using daily data on the COVID-19 pandemic in the USA.
Collapse
Affiliation(s)
- Gerardo Chowell
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA.
- Department of Applied Mathematics, Kyung Hee University, Yongin, 17104, Korea.
| | - Amna Tariq
- Department of Pediatrics, School of Medicine, Stanford University, Palo Alto, CA, USA
| | - Sushma Dahal
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| | - Amanda Bleichrodt
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| | - Ruiyan Luo
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| | - James M Hyman
- Department of Mathematics, Tulane University, New Orleans, LA, USA
| |
Collapse
|
2
|
Chowell G, Dahal S, Bleichrodt A, Tariq A, Hyman JM, Luo R. SubEpiPredict: A tutorial-based primer and toolbox for fitting and forecasting growth trajectories using the ensemble n-sub-epidemic modeling framework. Infect Dis Model 2024; 9:411-436. [PMID: 38385022 PMCID: PMC10879680 DOI: 10.1016/j.idm.2024.02.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2023] [Revised: 02/01/2024] [Accepted: 02/01/2024] [Indexed: 02/23/2024] Open
Abstract
An ensemble n-sub-epidemic modeling framework that integrates sub-epidemics to capture complex temporal dynamics has demonstrated powerful forecasting capability in previous works. This modeling framework can characterize complex epidemic patterns, including plateaus, epidemic resurgences, and epidemic waves characterized by multiple peaks of different sizes. In this tutorial paper, we introduce and illustrate SubEpiPredict, a user-friendly MATLAB toolbox for fitting and forecasting time series data using an ensemble n-sub-epidemic modeling framework. The toolbox can be used for model fitting, forecasting, and evaluation of model performance of the calibration and forecasting periods using metrics such as the weighted interval score (WIS). We also provide a detailed description of these methods including the concept of the n-sub-epidemic model, constructing ensemble forecasts from the top-ranking models, etc. For the illustration of the toolbox, we utilize publicly available daily COVID-19 death data at the national level for the United States. The MATLAB toolbox introduced in this paper can be very useful for a wider group of audiences, including policymakers, and can be easily utilized by those without extensive coding and modeling backgrounds.
Collapse
Affiliation(s)
- Gerardo Chowell
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
- Department of International Epidemiology and Population Studies, Fogarty International Center, National Institutes of Health, Bethesda, MD, USA
| | - Sushma Dahal
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| | - Amanda Bleichrodt
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| | - Amna Tariq
- Department of Pediatrics, School of Medicine, Stanford University, Palo Alto, CA, USA
| | - James M. Hyman
- Department of Mathematics, Center for Computational Science, Tulane University, New Orleans, LA, USA
| | - Ruiyan Luo
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| |
Collapse
|
3
|
Manley H, Bayley T, Danelian G, Burton L, Finnie T, Charlett A, Watkins NA, Birrell P, De Angelis D, Keeling M, Funk S, Medley G, Pellis L, Baguelin M, Ackland GJ, Hutchinson J, Riley S, Panovska-Griffiths J. Combining models to generate consensus medium-term projections of hospital admissions, occupancy and deaths relating to COVID-19 in England. ROYAL SOCIETY OPEN SCIENCE 2024; 11:231832. [PMID: 39076350 PMCID: PMC11285879 DOI: 10.1098/rsos.231832] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Revised: 02/29/2024] [Accepted: 03/09/2024] [Indexed: 07/31/2024]
Abstract
Mathematical modelling has played an important role in offering informed advice during the COVID-19 pandemic. In England, a cross government and academia collaboration generated medium-term projections (MTPs) of possible epidemic trajectories over the future 4-6 weeks from a collection of epidemiological models. In this article, we outline this collaborative modelling approach and evaluate the accuracy of the combined and individual model projections against the data over the period November 2021-December 2022 when various Omicron subvariants were spreading across England. Using a number of statistical methods, we quantify the predictive performance of the model projections for both the combined and individual MTPs, by evaluating the point and probabilistic accuracy. Our results illustrate that the combined MTPs, produced from an ensemble of heterogeneous epidemiological models, were a closer fit to the data than the individual models during the periods of epidemic growth or decline, with the 90% confidence intervals widest around the epidemic peaks. We also show that the combined MTPs increase the robustness and reduce the biases associated with a single model projection. Learning from our experience of ensemble modelling during the COVID-19 epidemic, our findings highlight the importance of developing cross-institutional multi-model infectious disease hubs for future outbreak control.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | - Paul Birrell
- UK Health Security Agency, London, UK
- MRC Biostatistics Unit, University of Cambridge, , UK
| | - Daniela De Angelis
- UK Health Security Agency, London, UK
- MRC Biostatistics Unit, University of Cambridge, , UK
| | - Matt Keeling
- Department of Mathematics, University of Warwick, Coventry, UK
| | - Sebastian Funk
- London School of Hygiene and Tropical Medicine, London, UK
| | - Graham Medley
- London School of Hygiene and Tropical Medicine, London, UK
| | | | | | | | | | | | - Jasmina Panovska-Griffiths
- UK Health Security Agency, London, UK
- Queen’s College, University of Oxford, Oxford, UK
- The Big Data Institute and the Pandemic Sciences Institute, University of Oxford, Oxford, UK
| |
Collapse
|
4
|
Chowell G, Bleichrodt A, Dahal S, Tariq A, Roosa K, Hyman JM, Luo R. GrowthPredict: A toolbox and tutorial-based primer for fitting and forecasting growth trajectories using phenomenological growth models. Sci Rep 2024; 14:1630. [PMID: 38238407 PMCID: PMC10796326 DOI: 10.1038/s41598-024-51852-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2023] [Accepted: 01/10/2024] [Indexed: 01/22/2024] Open
Abstract
Simple dynamic modeling tools can help generate real-time short-term forecasts with quantified uncertainty of the trajectory of diverse growth processes unfolding in nature and society, including disease outbreaks. An easy-to-use and flexible toolbox for this purpose is lacking. This tutorial-based primer introduces and illustrates GrowthPredict, a user-friendly MATLAB toolbox for fitting and forecasting time-series trajectories using phenomenological dynamic growth models based on ordinary differential equations. This toolbox is accessible to a broad audience, including students training in mathematical biology, applied statistics, and infectious disease modeling, as well as researchers and policymakers who need to conduct short-term forecasts in real-time. The models included in the toolbox capture exponential and sub-exponential growth patterns that typically follow a rising pattern followed by a decline phase, a common feature of contagion processes. Models include the 1-parameter exponential growth model and the 2-parameter generalized-growth model, which have proven useful in characterizing and forecasting the ascending phase of epidemic outbreaks. It also includes the 2-parameter Gompertz model, the 3-parameter generalized logistic-growth model, and the 3-parameter Richards model, which have demonstrated competitive performance in forecasting single peak outbreaks. We provide detailed guidance on forecasting time-series trajectories and available software ( https://github.com/gchowell/forecasting_growthmodels ), including the full uncertainty distribution derived through parametric bootstrapping, which is needed to construct prediction intervals and evaluate their accuracy. Functions are available to assess forecasting performance across different models, estimation methods, error structures in the data, and forecasting horizons. The toolbox also includes functions to quantify forecasting performance using metrics that evaluate point and distributional forecasts, including the weighted interval score. This tutorial and toolbox can be broadly applied to characterizing and forecasting time-series data using simple phenomenological growth models. As a contagion process takes off, the tools presented in this tutorial can help create forecasts to guide policy regarding implementing control strategies and assess the impact of interventions. The toolbox functionality is demonstrated through various examples, including a tutorial video, and the examples use publicly available data on the monkeypox (mpox) epidemic in the USA.
Collapse
Affiliation(s)
- Gerardo Chowell
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA.
| | - Amanda Bleichrodt
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| | - Sushma Dahal
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| | - Amna Tariq
- School of Medicine, Stanford University, Stanford, CA, USA
| | - Kimberlyn Roosa
- National Institute for Mathematical and Biological Synthesis (NIMBioS), University of Tennessee, Knoxville, TN, USA
| | - James M Hyman
- Department of Mathematics, Center for Computational Science, Tulane University, New Orleans, LA, USA
| | - Ruiyan Luo
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| |
Collapse
|
5
|
Chowell G, Bleichrodt A, Dahal S, Tariq A, Roosa K, Hyman JM, Luo R. A MATLAB toolbox to fit and forecast growth trajectories using phenomenological growth models: Application to epidemic outbreaks. RESEARCH SQUARE 2023:rs.3.rs-2724940. [PMID: 37034746 PMCID: PMC10081381 DOI: 10.21203/rs.3.rs-2724940/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/26/2023]
Abstract
Background Simple dynamic modeling tools can be useful for generating real-time short-term forecasts with quantified uncertainty of the trajectory of diverse growth processes unfolding in nature and society, including disease outbreaks. An easy-to-use and flexible toolbox for this purpose is lacking. Results In this tutorial-based primer, we introduce and illustrate a user-friendly MATLAB toolbox for fitting and forecasting time-series trajectories using phenomenological dynamic growth models based on ordinary differential equations. This toolbox is accessible to various audiences, including students training in time-series forecasting, dynamic growth modeling, parameter estimation, parameter uncertainty and identifiability, model comparison, performance metrics, and forecast evaluation, as well as researchers and policymakers who need to conduct short-term forecasts in real-time. The models included in the toolbox capture exponential and sub-exponential growth patterns that typically follow a rising pattern followed by a decline phase, a common feature of contagion processes. Models include the 2-parameter generalized-growth model, which has proved useful to characterize and forecast the ascending phase of epidemic outbreaks, and the Gompertz model as well as the 3-parameter generalized logistic-growth model and the Richards model, which have demonstrated competitive performance in forecasting single peak outbreaks.The toolbox provides a tutorial for forecasting time-series trajectories that include the full uncertainty distribution, derived through parametric bootstrapping, which is needed to construct prediction intervals and evaluate their accuracy. Functions are available to assess forecasting performance across different models, estimation methods, error structures in the data, and forecasting horizons. The toolbox also includes functions to quantify forecasting performance using metrics that evaluate point and distributional forecasts, including the weighted interval score. Conclusions We have developed the first comprehensive toolbox to characterize and forecast time-series data using simple phenomenological growth models. As a contagion process takes off, the tools presented in this tutorial can facilitate policymaking to guide the implementation of control strategies and assess the impact of interventions. The toolbox functionality is demonstrated through various examples, including a tutorial video, and is illustrated using weekly data on the monkeypox epidemic in the USA.
Collapse
Affiliation(s)
- Gerardo Chowell
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| | - Amanda Bleichrodt
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| | - Sushma Dahal
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| | - Amna Tariq
- Stanford University, School of Medicine, CA, USA
| | - Kimberlyn Roosa
- National Institute for Mathematical and Biological Synthesis (NIMBioS), University of Tennessee, Knoxville, TN, USA
| | - James M Hyman
- Department of Mathematics, Center for Computational Science, Tulane University, New Orleans, LA, USA
| | - Ruiyan Luo
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| |
Collapse
|
6
|
Chowell G, Bleichrodt A, Dahal S, Tariq A, Roosa K, Hyman JM, Luo R. A MATLAB toolbox to fit and forecast growth trajectories using phenomenological growth models: Application to epidemic outbreaks. RESEARCH SQUARE 2023:rs.3.rs-2724940. [PMID: 37034746 PMCID: PMC10081381 DOI: 10.21203/rs.3.rs-2724940/v2] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
BACKGROUND Simple dynamic modeling tools can be useful for generating real-time short-term forecasts with quantified uncertainty of the trajectory of diverse growth processes unfolding in nature and society, including disease outbreaks. An easy-to-use and flexible toolbox for this purpose is lacking. RESULTS In this tutorial-based primer, we introduce and illustrate a user-friendly MATLAB toolbox for fitting and forecasting time-series trajectories using phenomenological dynamic growth models based on ordinary differential equations. This toolbox is accessible to various audiences, including students training in time-series forecasting, dynamic growth modeling, parameter estimation, parameter uncertainty and identifiability, model comparison, performance metrics, and forecast evaluation, as well as researchers and policymakers who need to conduct short-term forecasts in real-time. The models included in the toolbox capture exponential and sub-exponential growth patterns that typically follow a rising pattern followed by a decline phase, a common feature of contagion processes. Models include the 2-parameter generalized-growth model, which has proved useful to characterize and forecast the ascending phase of epidemic outbreaks, and the Gompertz model as well as the 3-parameter generalized logistic-growth model and the Richards model, which have demonstrated competitive performance in forecasting single peak outbreaks.The toolbox provides a tutorial for forecasting time-series trajectories that include the full uncertainty distribution, derived through parametric bootstrapping, which is needed to construct prediction intervals and evaluate their accuracy. Functions are available to assess forecasting performance across different models, estimation methods, error structures in the data, and forecasting horizons. The toolbox also includes functions to quantify forecasting performance using metrics that evaluate point and distributional forecasts, including the weighted interval score. CONCLUSIONS We have developed the first comprehensive toolbox to characterize and forecast time-series data using simple phenomenological growth models. As a contagion process takes off, the tools presented in this tutorial can facilitate policymaking to guide the implementation of control strategies and assess the impact of interventions. The toolbox functionality is demonstrated through various examples, including a tutorial video, and is illustrated using weekly data on the monkeypox epidemic in the USA.
Collapse
Affiliation(s)
- Gerardo Chowell
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| | - Amanda Bleichrodt
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| | - Sushma Dahal
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| | - Amna Tariq
- Stanford University, School of Medicine, CA, USA
| | - Kimberlyn Roosa
- National Institute for Mathematical and Biological Synthesis (NIMBioS), University of Tennessee, Knoxville, TN, USA
| | - James M Hyman
- Department of Mathematics, Center for Computational Science, Tulane University, New Orleans, LA, USA
| | - Ruiyan Luo
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| |
Collapse
|
7
|
Quiroga BF, Vásquez C, Vicuña MI. Nonlinear time-series forecasts for decision support: short-term demand for ICU beds in Santiago, Chile, during the 2021 COVID-19 pandemic. INTERNATIONAL TRANSACTIONS IN OPERATIONAL RESEARCH : A JOURNAL OF THE INTERNATIONAL FEDERATION OF OPERATIONAL RESEARCH SOCIETIES 2022; 30:ITOR13222. [PMID: 36712286 PMCID: PMC9874731 DOI: 10.1111/itor.13222] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/13/2021] [Revised: 04/22/2022] [Accepted: 09/22/2022] [Indexed: 06/18/2023]
Abstract
In Chile, due to the explosive increase of new Coronavirus disease 2019 (COVID-19) cases during the first part of 2021, the ability of health services to accommodate new incoming cases was jeopardized. It has become necessary to be able to manage intensive care unit (ICU) capacity, and for this purpose, monitoring both the evolution of new cases and the demand for ICU beds has become urgent. This paper presents short-term forecast models for the number of new cases and the number of COVID-19 patients admitted to ICUs in the Metropolitan Region in Chile.
Collapse
Affiliation(s)
- Bernardo F. Quiroga
- School of ManagementPontificia Universidad Católica de ChileAv. Vicuña Mackenna 4860MaculSantiagoRM 7820436Chile
| | - Cristián Vásquez
- School of ManagementPontificia Universidad Católica de ChileAv. Vicuña Mackenna 4860MaculSantiagoRM 7820436Chile
| | - M. Ignacia Vicuña
- School of ManagementPontificia Universidad Católica de ChileAv. Vicuña Mackenna 4860MaculSantiagoRM 7820436Chile
| |
Collapse
|
8
|
An ensemble n-sub-epidemic modeling framework for short-term forecasting epidemic trajectories: Application to the COVID-19 pandemic in the USA. PLoS Comput Biol 2022; 18:e1010602. [PMID: 36201534 PMCID: PMC9578588 DOI: 10.1371/journal.pcbi.1010602] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2022] [Revised: 10/18/2022] [Accepted: 09/26/2022] [Indexed: 11/19/2022] Open
Abstract
We analyze an ensemble of n-sub-epidemic modeling for forecasting the trajectory of epidemics and pandemics. These ensemble modeling approaches, and models that integrate sub-epidemics to capture complex temporal dynamics, have demonstrated powerful forecasting capability. This modeling framework can characterize complex epidemic patterns, including plateaus, epidemic resurgences, and epidemic waves characterized by multiple peaks of different sizes. We systematically assess their calibration and short-term forecasting performance in short-term forecasts for the COVID-19 pandemic in the USA from late April 2020 to late February 2022. We compare their performance with two commonly used statistical ARIMA models. The best fit sub-epidemic model and three ensemble models constructed using the top-ranking sub-epidemic models consistently outperformed the ARIMA models in terms of the weighted interval score (WIS) and the coverage of the 95% prediction interval across the 10-, 20-, and 30-day short-term forecasts. In our 30-day forecasts, the average WIS ranged from 377.6 to 421.3 for the sub-epidemic models, whereas it ranged from 439.29 to 767.05 for the ARIMA models. Across 98 short-term forecasts, the ensemble model incorporating the top four ranking sub-epidemic models (Ensemble(4)) outperformed the (log) ARIMA model 66.3% of the time, and the ARIMA model, 69.4% of the time in 30-day ahead forecasts in terms of the WIS. Ensemble(4) consistently yielded the best performance in terms of the metrics that account for the uncertainty of the predictions. This framework can be readily applied to investigate the spread of epidemics and pandemics beyond COVID-19, as well as other dynamic growth processes found in nature and society that would benefit from short-term predictions.
Collapse
|
9
|
Chowell G, Dahal S, Tariq A, Roosa K, Hyman JM, Luo R. An ensemble n-sub-epidemic modeling framework for short-term forecasting epidemic trajectories: Application to the COVID-19 pandemic in the USA.. [PMID: 35794886 PMCID: PMC9258290 DOI: 10.1101/2022.06.19.22276608] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Abstract
We analyze an ensemble of n-sub-epidemic modeling for forecasting the trajectory of epidemics and pandemics. These ensemble modeling approaches, and models that integrate sub-epidemics to capture complex temporal dynamics, have demonstrated powerful forecasting capability. This modeling framework can characterize complex epidemic patterns, including plateaus, epidemic resurgences, and epidemic waves characterized by multiple peaks of different sizes. We systematically assess their calibration and short-term forecasting performance in short-term forecasts for the COVID-19 pandemic in the USA from late April 2020 to late February 2022. We compare their performance with two commonly used statistical ARIMA models. The best fit sub-epidemic model and three ensemble models constructed using the top-ranking sub-epidemic models consistently outperformed the ARIMA models in terms of the weighted interval score (WIS) and the coverage of the 95% prediction interval across the 10-, 20-, and 30-day short-term forecasts. In the 30-day forecasts, the average WIS ranged from 377.6 to 421.3 for the sub-epidemic models, whereas it ranged from 439.29 to 767.05 for the ARIMA models. Across 98 short-term forecasts, the ensemble model incorporating the top four ranking sub-epidemic models (Ensemble(4)) outperformed the (log) ARIMA model 66.3% of the time, and the ARIMA model 69.4% of the time in 30-day ahead forecasts in terms of the WIS. Ensemble(4) consistently yielded the best performance in terms of the metrics that account for the uncertainty of the predictions. This framework could be readily applied to investigate the spread of epidemics and pandemics beyond COVID-19, as well as other dynamic growth processes found in nature and society that would benefit from short-term predictions. The COVID-19 pandemic has highlighted the urgent need to develop reliable tools to forecast the trajectory of epidemics and pandemics in near real-time. We describe and apply an ensemble n-sub-epidemic modeling framework for forecasting the trajectory of epidemics and pandemics. We systematically assess its calibration and short-term forecasting performance in weekly 10–30 days ahead forecasts for the COVID-19 pandemic in the USA from late April 2020 to late February 2022 and compare its performance with two different statistical ARIMA models. This framework demonstrated reliable forecasting performance and substantially outcompeted the ARIMA models. The forecasting performance was consistently best for the ensemble sub-epidemic models incorporating a higher number of top-ranking sub-epidemic models. The ensemble model incorporating the top four ranking sub-epidemic models consistently yielded the best performance, particularly in terms of the coverage rate of the 95% prediction interval and the weighted interval score. This framework can be applied to forecast other growth processes found in nature and society including the spread of information through social media.
Collapse
|
10
|
Vossler H, Akilimali P, Pan Y, KhudaBukhsh WR, Kenah E, Rempała GA. Analysis of individual-level data from 2018-2020 Ebola outbreak in Democratic Republic of the Congo. Sci Rep 2022; 12:5534. [PMID: 35365724 PMCID: PMC8972744 DOI: 10.1038/s41598-022-09564-4] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Accepted: 03/17/2022] [Indexed: 11/08/2022] Open
Abstract
The 2018-2020 Ebola virus disease epidemic in Democratic Republic of the Congo (DRC) resulted in 3481 cases (probable and confirmed) and 2299 deaths. In this paper, we use a novel statistical method to analyze the individual-level incidence and hospitalization data on DRC Ebola victims. Our analysis suggests that an increase in the rate of quarantine and isolation that has shortened the infectiousness period by approximately one day during the epidemic's third and final wave was likely responsible for the eventual containment of the outbreak. The analysis further reveals that the total effective population size or the average number of individuals at risk for the disease exposure in three epidemic waves over the period of 24 months was around 16,000-a much smaller number than previously estimated and likely an evidence of at least partial protection of the population at risk through ring vaccination and contact tracing as well as adherence to strict quarantine and isolation policies.
Collapse
Affiliation(s)
- Harley Vossler
- College of Public Health, The Ohio State University, Columbus, OH, USA
| | - Pierre Akilimali
- College of Public Health, University of Kinshasa, Kinshasa, Congo
| | - Yuhan Pan
- College of Public Health, The Ohio State University, Columbus, OH, USA
| | | | - Eben Kenah
- College of Public Health, The Ohio State University, Columbus, OH, USA
| | | |
Collapse
|
11
|
Miyama T, Jung SM, Hayashi K, Anzai A, Kinoshita R, Kobayashi T, Linton NM, Suzuki A, Yang Y, Yuan B, Kayano T, Akhmetzhanov AR, Nishiura H. Phenomenological and mechanistic models for predicting early transmission data of COVID-19. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2022; 19:2043-2055. [PMID: 35135241 DOI: 10.3934/mbe.2022096] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Forecasting future epidemics helps inform policy decisions regarding interventions. During the early coronavirus disease 2019 epidemic period in January-February 2020, limited information was available, and it was too challenging to build detailed mechanistic models reflecting population behavior. This study compared the performance of phenomenological and mechanistic models for forecasting epidemics. For the former, we employed the Richards model and the approximate solution of the susceptible-infected-recovered (SIR) model. For the latter, we examined the exponential growth (with lockdown) model and SIR model with lockdown. The phenomenological models yielded higher root mean square error (RMSE) values than the mechanistic models. When using the numbers from reported data for February 1 and 5, the Richards model had the highest RMSE, whereas when using the February 9 data, the SIR approximation model was the highest. The exponential model with a lockdown effect had the lowest RMSE, except when using the February 9 data. Once interventions or other factors that influence transmission patterns are identified, they should be additionally taken into account to improve forecasting.
Collapse
Affiliation(s)
- Takeshi Miyama
- Division of Public Health, Osaka Institute of Public Health, Osaka, Japan
- School of Public Health, Kyoto University, Kyoto, Japan
| | - Sung-Mok Jung
- School of Public Health, Kyoto University, Kyoto, Japan
- Graduate School of Medicine, Hokkaido University, Sapporo, Japan
| | | | - Asami Anzai
- School of Public Health, Kyoto University, Kyoto, Japan
| | - Ryo Kinoshita
- School of Public Health, Kyoto University, Kyoto, Japan
| | | | - Natalie M Linton
- School of Public Health, Kyoto University, Kyoto, Japan
- Graduate School of Medicine, Hokkaido University, Sapporo, Japan
| | - Ayako Suzuki
- School of Public Health, Kyoto University, Kyoto, Japan
| | - Yichi Yang
- Graduate School of Medicine, Hokkaido University, Sapporo, Japan
| | - Baoyin Yuan
- Graduate School of Medicine, Hokkaido University, Sapporo, Japan
| | - Taishi Kayano
- School of Public Health, Kyoto University, Kyoto, Japan
| | | | | |
Collapse
|
12
|
Multivariate phenomenological models for real-time short-term forecasts of hospital capacity for COVID-19 in Belgium from March to June 2020. Epidemiol Infect 2021. [PMCID: PMC8755551 DOI: 10.1017/s0950268821002491] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
Phenomenological models are popular for describing the epidemic curve. We present how they can be used at different phases in the epidemic, by modelling the daily number of new hospitalisations (or cases). As real-time prediction of the hospital capacity is important, a joint model of the new hospitalisations, number of patients in hospital and in intensive care unit (ICU) is proposed. This model allows estimation of the length of stay in hospital and ICU, even if no (or limited) individual level information on length of stay is available. Estimation is done in a Bayesian framework. In this framework, real-time alarms, defined as the probability of exceeding hospital capacity, can be easily derived. The methods are illustrated using data from the COVID-19 pandemic in March–June 2020 in Belgium, but are widely applicable.
Collapse
|
13
|
Vicuña MI, Vásquez C, Quiroga BF. Forecasting the 2020 COVID-19 Epidemic: A Multivariate Quasi-Poisson Regression to Model the Evolution of New Cases in Chile. Front Public Health 2021; 9:610479. [PMID: 33968875 PMCID: PMC8102770 DOI: 10.3389/fpubh.2021.610479] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2020] [Accepted: 03/25/2021] [Indexed: 02/01/2023] Open
Abstract
Objectives: To understand and forecast the evolution of COVID-19 (Coronavirus disease 2019) in Chile, and analyze alternative simulated scenarios to better predict alternative paths, in order to implement policy solutions to stop the spread and minimize damage. Methods: We have specified a novel multi-parameter generalized logistic growth model, which does not only look at the trend of the data, but also includes explanatory covariates, using a quasi-Poisson regression specification to account for overdispersion of the count data. We fitted our model to data from the onset of the disease (February 28) until September 15. Estimating the parameters from our model, we predicted the growth of the epidemic for the evolution of the disease until the end of October 2020. We also evaluated via simulations different fictional scenarios for the outcome of alternative policies (those analyses are included in the Supplementary Material). Results and Conclusions: The evolution of the disease has not followed an exponential growth, but rather, stabilized and moved downward after July 2020, starting to increase again after the implementation of the Step-by-Step policy. The lockdown policy implemented in the majority of the country has proven effective in stopping the spread, and the lockdown-relaxation policies, however gradual, appear to have caused an upward break in the trend.
Collapse
Affiliation(s)
- María Ignacia Vicuña
- Escuela de Administración, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Cristián Vásquez
- Escuela de Administración, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Bernardo F Quiroga
- Escuela de Administración, Pontificia Universidad Católica de Chile, Santiago, Chile
| |
Collapse
|
14
|
Chowell G, Luo R. Ensemble bootstrap methodology for forecasting dynamic growth processes using differential equations: application to epidemic outbreaks. BMC Med Res Methodol 2021; 21:34. [PMID: 33583405 PMCID: PMC7882252 DOI: 10.1186/s12874-021-01226-9] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2020] [Accepted: 01/18/2021] [Indexed: 11/30/2022] Open
Abstract
BACKGROUND Ensemble modeling aims to boost the forecasting performance by systematically integrating the predictive accuracy across individual models. Here we introduce a simple-yet-powerful ensemble methodology for forecasting the trajectory of dynamic growth processes that are defined by a system of non-linear differential equations with applications to infectious disease spread. METHODS We propose and assess the performance of two ensemble modeling schemes with different parametric bootstrapping procedures for trajectory forecasting and uncertainty quantification. Specifically, we conduct sequential probabilistic forecasts to evaluate their forecasting performance using simple dynamical growth models with good track records including the Richards model, the generalized-logistic growth model, and the Gompertz model. We first test and verify the functionality of the method using simulated data from phenomenological models and a mechanistic transmission model. Next, the performance of the method is demonstrated using a diversity of epidemic datasets including scenario outbreak data of the Ebola Forecasting Challenge and real-world epidemic data outbreaks of including influenza, plague, Zika, and COVID-19. RESULTS We found that the ensemble method that randomly selects a model from the set of individual models for each time point of the trajectory of the epidemic frequently outcompeted the individual models as well as an alternative ensemble method based on the weighted combination of the individual models and yields broader and more realistic uncertainty bounds for the trajectory envelope, achieving not only better coverage rate of the 95% prediction interval but also improved mean interval scores across a diversity of epidemic datasets. CONCLUSION Our new methodology for ensemble forecasting outcompete component models and an alternative ensemble model that differ in how the variance is evaluated for the generation of the prediction intervals of the forecasts.
Collapse
Affiliation(s)
- Gerardo Chowell
- Department of Population Heath Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA.
- Division of International Epidemiology and Population Studies, Fogarty International Center, National Institutes of Health, Bethesda, MD, USA.
| | - Ruiyan Luo
- Department of Population Heath Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| |
Collapse
|
15
|
Lee Y, Roosa K, Chowell G. Systematic comparison of epidemic growth patterns using two different estimation approaches. Infect Dis Model 2020; 6:5-14. [PMID: 33294744 PMCID: PMC7691176 DOI: 10.1016/j.idm.2020.10.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2019] [Revised: 09/15/2020] [Accepted: 10/11/2020] [Indexed: 11/25/2022] Open
Abstract
Background Different estimation approaches are frequently used to calibrate mathematical models to epidemiological data, particularly for analyzing infectious disease outbreaks. Here, we use two common methods to estimate parameters that characterize growth patterns using the generalized growth model (GGM) calibrated to real outbreak datasets. Materials and methods Data from 31 outbreaks are used to fit the GGM to the ascending phase of each outbreak and estimate the parameters using both least squares (LSQ) and maximum likelihood estimation (MLE) methods. We utilize parametric bootstrapping to construct confidence intervals for parameter estimates. We compare the results including RMSE, Anscombe residual, and 95% prediction interval coverage. We also evaluate the correlation between the estimates from both methods. Results Comparing LSQ and MLE estimates, most outbreaks have similar parameter estimates, RMSE, Anscombe, and 95% prediction interval coverage. Parameter estimates do not differ across methods when the model yields a good fit to the early growth phase. However, for two outbreaks, there are systematic deviations in model fit to the data that explain differences in parameter estimates (e.g., residuals represent random error rather than systematic deviation). Conclusion Our findings indicate that utilizing LSQ and MLE methods produce similar results in the context of characterizing epidemic growth patterns with the GGM, provided that the model yields a good fit to the data.
Collapse
Affiliation(s)
- Yiseul Lee
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, 30303, USA
| | - Kimberlyn Roosa
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, 30303, USA.,UT One Health Initiative, University of Tennessee, Knoxville, TN, 37996, USA
| | - Gerardo Chowell
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, 30303, USA.,Division of International Epidemiology and Population Studies, Fogarty International Center, National Institute of Health, Bethesda, MD, 20892, USA
| |
Collapse
|