Adaptively stacking ensembles for influenza forecasting

Challenges of COVID-19 Case Forecasting in the US, 2020-2021

During the COVID-19 pandemic, forecasting COVID-19 trends to support planning and response was a priority for scientists and decision makers alike. In the United States, COVID-19 forecasting was coordinated by a large group of universities, companies, and government entities led by the Centers for Disease Control and Prevention and the US COVID-19 Forecast Hub (https://covid19forecasthub.org). We evaluated approximately 9.7 million forecasts of weekly state-level COVID-19 cases for predictions 1-4 weeks into the future submitted by 24 teams from August 2020 to December 2021. We assessed coverage of central prediction intervals and weighted interval scores (WIS), adjusting for missing forecasts relative to a baseline forecast, and used a Gaussian generalized estimating equation (GEE) model to evaluate differences in skill across epidemic phases that were defined by the effective reproduction number. Overall, we found high variation in skill across individual models, with ensemble-based forecasts outperforming other approaches. Forecast skill relative to the baseline was generally higher for larger jurisdictions (e.g., states compared to counties). Over time, forecasts generally performed worst in periods of rapid changes in reported cases (either in increasing or decreasing epidemic phases) with 95% prediction interval coverage dropping below 50% during the growth phases of the winter 2020, Delta, and Omicron waves. Ideally, case forecasts could serve as a leading indicator of changes in transmission dynamics. However, while most COVID-19 case forecasts outperformed a naïve baseline model, even the most accurate case forecasts were unreliable in key phases. Further research could improve forecasts of leading indicators, like COVID-19 cases, by leveraging additional real-time data, addressing performance across phases, improving the characterization of forecast confidence, and ensuring that forecasts were coherent across spatial scales. In the meantime, it is critical for forecast users to appreciate current limitations and use a broad set of indicators to inform pandemic-related decision making.

Application progress of ensemble forecast technology in influenza forecast based on infectious disease model

To comprehensively understand the application progress of ensemble forecast technology in influenza forecast based on infectious disease model, so as to provide scientific references for further research. In this study, two keywords of "influenza" and "ensemble forecast" are selected to search and select the relevant literatures, which are then outlined and summarized. It is found that: In recent years, some studies about ensemble forecast technology for influenza have been reported in the literature, and some well-performed influenza ensemble forecast systems have already been operationally implemented and provide references for scientific prevention and control. In general, ensemble forecast can well represent various uncertainties in forecasting influenza cases based on infectious disease models, and can achieve more accurate forecasts and more valuable information than single deterministic forecast. However, there are still some shortcomings in the current studies, it is suggested that scientists engaged in influenza forecast based on infectious disease models strengthen cooperation with scholars in the field of numerical weather forecast, which is expected to further improve the skills and application level of ensemble forecast for influenza.

Repeatability and timing of tropical influenza epidemics

Much of the world experiences influenza in yearly recurring seasons, particularly in temperate areas. These patterns can be considered repeatable if they occur predictably and consistently at the same time of year. In tropical areas, including southeast Asia, timing of influenza epidemics is less consistent, leading to a lack of consensus regarding whether influenza is repeatable. This study aimed to assess repeatability of influenza in Vietnam, with repeatability defined as seasonality that occurs at a consistent time of year with low variation. We developed a mathematical model incorporating parameters to represent periods of increased transmission and then fitted the model to data collected from sentinel hospitals throughout Vietnam as well as four temperate locations. We fitted the model for individual (sub)types of influenza as well as all combined influenza throughout northern, central, and southern Vietnam. Repeatability was evaluated through the variance of the timings of peak transmission. Model fits from Vietnam show high variance (sd = 64-179 days) in peak transmission timing, with peaks occurring at irregular intervals and throughout different times of year. Fits from temperate locations showed regular, annual epidemics in winter months, with low variance in peak timings (sd = 32-57 days). This suggests that influenza patterns are not repeatable or seasonal in Vietnam. Influenza prevention in Vietnam therefore cannot rely on anticipation of regularly occurring outbreaks.

Supporting COVID-19 policy-making with a predictive epidemiological multi-model warning system

BACKGROUND

In response to the SARS-CoV-2 pandemic, the Austrian governmental crisis unit commissioned a forecast consortium with regularly projections of case numbers and demand for hospital beds. The goal was to assess how likely Austrian ICUs would become overburdened with COVID-19 patients in the upcoming weeks.

METHODS

We consolidated the output of three epidemiological models (ranging from agent-based micro simulation to parsimonious compartmental models) and published weekly short-term forecasts for the number of confirmed cases as well as estimates and upper bounds for the required hospital beds.

RESULTS

We report on three key contributions by which our forecasting and reporting system has helped shaping Austria's policy to navigate the crisis, namely (i) when and where case numbers and bed occupancy are expected to peak during multiple waves, (ii) whether to ease or strengthen non-pharmaceutical intervention in response to changing incidences, and (iii) how to provide hospital managers guidance to plan health-care capacities.

CONCLUSIONS

Complex mathematical epidemiological models play an important role in guiding governmental responses during pandemic crises, in particular when they are used as a monitoring system to detect epidemiological change points.

Chimeric forecasting: combining probabilistic predictions from computational models and human judgment

Forecasts of the trajectory of an infectious agent can help guide public health decision making. A traditional approach to forecasting fits a computational model to structured data and generates a predictive distribution. However, human judgment has access to the same data as computational models plus experience, intuition, and subjective data. We propose a chimeric ensemble-a combination of computational and human judgment forecasts-as a novel approach to predicting the trajectory of an infectious agent. Each month from January, 2021 to June, 2021 we asked two generalist crowds, using the same criteria as the COVID-19 Forecast Hub, to submit a predictive distribution over incident cases and deaths at the US national level either two or three weeks into the future and combined these human judgment forecasts with forecasts from computational models submitted to the COVID-19 Forecasthub into a chimeric ensemble. We find a chimeric ensemble compared to an ensemble including only computational models improves predictions of incident cases and shows similar performance for predictions of incident deaths. A chimeric ensemble is a flexible, supportive public health tool and shows promising results for predictions of the spread of an infectious agent.

Aggregating human judgment probabilistic predictions of the safety, efficacy, and timing of a COVID-19 vaccine

Safe, efficacious vaccines were developed to reduce the transmission of SARS-CoV-2 during the COVID-19 pandemic. But in the middle of 2020, vaccine effectiveness, safety, and the timeline for when a vaccine would be approved and distributed to the public was uncertain. To support public health decision making, we solicited trained forecasters and experts in vaccinology and infectious disease to provide monthly probabilistic predictions from July to September of 2020 of the efficacy, safety, timing, and delivery of a COVID-19 vaccine. We found, that despite sparse historical data, a linear pool—a combination of human judgment probabilistic predictions—can quantify the uncertainty in clinical significance and timing of a potential vaccine. The linear pool underestimated how fast a therapy would show a survival benefit and the high efficacy of approved COVID-19 vaccines. However, the linear pool did make an accurate prediction for when a vaccine would be approved by the FDA. Compared to individual forecasters, the linear pool was consistently above the median of the most accurate forecasts. A linear pool is a fast and versatile method to build probabilistic predictions of a developing vaccine that is robust to poor individual predictions. Though experts and trained forecasters did underestimate the speed of development and the high efficacy of a SARS-CoV-2 vaccine, linear pool predictions can improve situational awareness for public health officials and for the public make clearer the risks, rewards, and timing of a vaccine.