|
1
|
Bracher J, Wolffram D, Deuschel J, Görgen K, Ketterer JL, Ullrich A, Abbott S, Barbarossa MV, Bertsimas D, Bhatia S, Bodych M, Bosse NI, Burgard JP, Castro L, Fairchild G, Fiedler J, Fuhrmann J, Funk S, Gambin A, Gogolewski K, Heyder S, Hotz T, Kheifetz Y, Kirsten H, Krueger T, Krymova E, Leithäuser N, Li ML, Meinke JH, Miasojedow B, Michaud IJ, Mohring J, Nouvellet P, Nowosielski JM, Ozanski T, Radwan M, Rakowski F, Scholz M, Soni S, Srivastava A, Gneiting T, Schienle M. National and subnational short-term forecasting of COVID-19 in Germany and Poland during early 2021. COMMUNICATIONS MEDICINE 2022; 2:136. [PMID: 36352249 PMCID: PMC9622804 DOI: 10.1038/s43856-022-00191-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2021] [Accepted: 09/22/2022] [Indexed: 11/07/2022] Open
Abstract
BACKGROUND During the COVID-19 pandemic there has been a strong interest in forecasts of the short-term development of epidemiological indicators to inform decision makers. In this study we evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland for the period from January through April 2021. METHODS We evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland. These were issued by 15 different forecasting models, run by independent research teams. Moreover, we study the performance of combined ensemble forecasts. Evaluation of probabilistic forecasts is based on proper scoring rules, along with interval coverage proportions to assess calibration. The presented work is part of a pre-registered evaluation study. RESULTS We find that many, though not all, models outperform a simple baseline model up to four weeks ahead for the considered targets. Ensemble methods show very good relative performance. The addressed time period is characterized by rather stable non-pharmaceutical interventions in both countries, making short-term predictions more straightforward than in previous periods. However, major trend changes in reported cases, like the rebound in cases due to the rise of the B.1.1.7 (Alpha) variant in March 2021, prove challenging to predict. CONCLUSIONS Multi-model approaches can help to improve the performance of epidemiological forecasts. However, while death numbers can be predicted with some success based on current case and hospitalization data, predictability of case numbers remains low beyond quite short time horizons. Additional data sources including sequencing and mobility data, which were not extensively used in the present study, may help to improve performance.
Collapse
Affiliation(s)
- Johannes Bracher
- Chair of Statistical Methods and Econometrics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany.
- Computational Statistics Group, Heidelberg Institute for Theoretical Studies (HITS), Heidelberg, Germany.
| | - Daniel Wolffram
- Chair of Statistical Methods and Econometrics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
- Computational Statistics Group, Heidelberg Institute for Theoretical Studies (HITS), Heidelberg, Germany
- HIDSS4Health - Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany
| | - Jannik Deuschel
- Chair of Statistical Methods and Econometrics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
| | - Konstantin Görgen
- Chair of Statistical Methods and Econometrics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
| | - Jakob L Ketterer
- Chair of Statistical Methods and Econometrics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
| | | | - Sam Abbott
- London School of Hygiene and Tropical Medicine, London, UK
| | | | - Dimitris Bertsimas
- Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Sangeeta Bhatia
- MRC Centre for Global Infectious Disease Analysis, Abdul Latif Jameel Institute for Disease and Emergency Analytics (J-IDEA), Imperial College London, London, UK
| | - Marcin Bodych
- Wroclaw University of Science and Technology, Wroclaw, Poland
| | - Nikos I Bosse
- London School of Hygiene and Tropical Medicine, London, UK
| | - Jan Pablo Burgard
- Economic and Social Statistics Department, University of Trier, Trier, Germany
| | - Lauren Castro
- Information Systems and Modeling, Los Alamos National Laboratory, Los Alamos, NM, USA
| | - Geoffrey Fairchild
- Information Systems and Modeling, Los Alamos National Laboratory, Los Alamos, NM, USA
| | - Jochen Fiedler
- Fraunhofer Institute for Industrial Mathematics (ITWM), Kaiserslautern, Germany
| | - Jan Fuhrmann
- Institute for Applied Mathematics, University of Heidelberg, Heidelberg, Germany
| | - Sebastian Funk
- London School of Hygiene and Tropical Medicine, London, UK
| | - Anna Gambin
- Faculty of Mathematics, Informatics, and Mechanics, University of Warsaw, Warsaw, Poland
| | - Krzysztof Gogolewski
- Faculty of Mathematics, Informatics, and Mechanics, University of Warsaw, Warsaw, Poland
| | - Stefan Heyder
- Institute of Mathematics, Technische Universität Ilmenau, Ilmenau, Germany
| | - Thomas Hotz
- Institute of Mathematics, Technische Universität Ilmenau, Ilmenau, Germany
| | - Yuri Kheifetz
- Institute for Medical Informatics, Statistics and Epidemiology, University of Leipzig, Leipzig, Germany
| | - Holger Kirsten
- Institute for Medical Informatics, Statistics and Epidemiology, University of Leipzig, Leipzig, Germany
| | - Tyll Krueger
- Wroclaw University of Science and Technology, Wroclaw, Poland
| | - Ekaterina Krymova
- Swiss Data Science Center, ETH Zürich and EPF Lausanne, Zürich, Switzerland
| | - Neele Leithäuser
- Fraunhofer Institute for Industrial Mathematics (ITWM), Kaiserslautern, Germany
| | - Michael L Li
- Operations Research Center, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Jan H Meinke
- Jülich Supercomputing Centre, Forschungszentrum Jülich, Jülich, Germany
| | - Błażej Miasojedow
- Faculty of Mathematics, Informatics, and Mechanics, University of Warsaw, Warsaw, Poland
| | - Isaac J Michaud
- Statistical Sciences Group, Los Alamos National Laboratory, Los Alamos, NM, USA
| | - Jan Mohring
- Fraunhofer Institute for Industrial Mathematics (ITWM), Kaiserslautern, Germany
| | | | - Jedrzej M Nowosielski
- Interdisciplinary Centre for Mathematical and Computational Modelling, University of Warsaw, Warsaw, Poland
| | - Tomasz Ozanski
- Wroclaw University of Science and Technology, Wroclaw, Poland
| | - Maciej Radwan
- Interdisciplinary Centre for Mathematical and Computational Modelling, University of Warsaw, Warsaw, Poland
| | - Franciszek Rakowski
- Interdisciplinary Centre for Mathematical and Computational Modelling, University of Warsaw, Warsaw, Poland
| | - Markus Scholz
- Institute for Medical Informatics, Statistics and Epidemiology, University of Leipzig, Leipzig, Germany
| | - Saksham Soni
- Operations Research Center, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Ajitesh Srivastava
- Ming Hsieh Department of Computer and Electrical Engineering, University of Southern California, Los Angeles, CA, USA
| | - Tilmann Gneiting
- Computational Statistics Group, Heidelberg Institute for Theoretical Studies (HITS), Heidelberg, Germany
- Institute for Stochastics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
| | - Melanie Schienle
- Chair of Statistical Methods and Econometrics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany.
- Computational Statistics Group, Heidelberg Institute for Theoretical Studies (HITS), Heidelberg, Germany.
| |
Collapse
|