Epidemic tracking and forecasting: Lessons learned from a tumultuous year

Oscillating spatiotemporal patterns of COVID-19 in the United States

COVID-19 case rates in the US wax and wane in wave-like patterns over time, but the spatial patterns of these temporal epidemic waves have not been well characterized. By analyzing state- and county-level COVID-19 case rate data for spatiotemporal decomposition modes and oscillatory patterns, we demonstrate that the transmission dynamics of COVID-19 feature recurrent spatiotemporal patterns. In addition to the well-recognized national-level annual mid-winter surges, we demonstrate a prominent but previously unrecognized six-month north-south oscillation in the eastern US (Eastern US COVID-19 Oscillator-EUCO) that gives rise to regional sub-epidemics and travelling epidemic waves. We also demonstrate a second less prominent pattern that oscillates east-west in the northern US (Northern US COVID-19 Oscillator-NUCO). The drivers of these newly recognized oscillatory epidemic patterns remain to be elucidated.

A modeling pipeline to relate municipal wastewater surveillance and regional public health data

As COVID-19 becomes endemic, public health departments benefit from improved passive indicators, which are independent of voluntary testing data, to estimate the prevalence of COVID-19 in local communities. Quantification of SARS-CoV-2 RNA from wastewater has the potential to be a powerful passive indicator. However, connecting measured SARS-CoV-2 RNA to community prevalence is challenging due to the high noise typical of environmental samples. We have developed a generalized pipeline using in- and out-of-sample model selection to test the ability of different correction models to reduce the variance in wastewater measurements and applied it to data collected from treatment plants in the Chicago area. We built and compared a set of multi-linear regression models, which incorporate pepper mild mottle virus (PMMoV) as a population biomarker, Bovine coronavirus (BCoV) as a recovery control, and wastewater system flow rate into a corrected estimate for SARS-CoV-2 RNA concentration. For our data, models with BCoV performed better than those with PMMoV, but the pipeline should be used to reevaluate any new data set as the sources of variance may change across locations, lab methods, and disease states. Using our best-fit model, we investigated the utility of RNA measurements in wastewater as a leading indicator of COVID-19 trends. We did this in a rolling manner for corrected wastewater data and for other prevalence indicators and statistically compared the temporal relationship between new increases in the wastewater data and those in other prevalence indicators. We found that wastewater trends often lead other COVID-19 indicators in predicting new surges.

Bias-Adjusted Predictions of County-Level Vaccination Coverage from the COVID-19 Trends and Impact Survey

BACKGROUND

The potential for selection bias in nonrepresentative, large-scale, low-cost survey data can limit their utility for population health measurement and public health decision making. We developed an approach to bias adjust county-level COVID-19 vaccination coverage predictions from the large-scale US COVID-19 Trends and Impact Survey.

DESIGN

We developed a multistep regression framework to adjust for selection bias in predicted county-level vaccination coverage plateaus. Our approach included poststratification to the American Community Survey, adjusting for differences in observed covariates, and secondary normalization to an unbiased reference indicator. As a case study, we prospectively applied this framework to predict county-level long-run vaccination coverage among children ages 5 to 11 y. We evaluated our approach against an interim observed measure of 3-mo coverage for children ages 5 to 11 y and used long-term coverage estimates to monitor equity in the pace of vaccination scale up.

RESULTS

Our predictions suggested a low ceiling on long-term national vaccination coverage (46%), detected substantial geographic heterogeneity (ranging from 11% to 91% across counties in the United States), and highlighted widespread disparities in the pace of scale up in the 3 mo following Emergency Use Authorization of COVID-19 vaccination for 5- to 11-y-olds.

LIMITATIONS

We relied on historical relationships between vaccination hesitancy and observed coverage, which may not capture rapid changes in the COVID-19 policy and epidemiologic landscape.

CONCLUSIONS

Our analysis demonstrates an approach to leverage differing strengths of multiple sources of information to produce estimates on the time scale and geographic scale necessary for proactive decision making.

IMPLICATIONS

Designing integrated health measurement systems that combine sources with different advantages across the spectrum of timeliness, spatial resolution, and representativeness can maximize the benefits of data collection relative to costs.

HIGHLIGHTS

The COVID-19 pandemic catalyzed massive survey data collection efforts that prioritized timeliness and sample size over population representativeness.The potential for selection bias in these large-scale, low-cost, nonrepresentative data has led to questions about their utility for population health measurement.We developed a multistep regression framework to bias adjust county-level vaccination coverage predictions from the largest public health survey conducted in the United States to date: the US COVID-19 Trends and Impact Survey.Our study demonstrates the value of leveraging differing strengths of multiple data sources to generate estimates on the time scale and geographic scale necessary for proactive public health decision making.

"Back to the future" projections for COVID-19 surges

We argue that information from countries who had earlier COVID-19 surges can be used to inform another country's current model, then generating what we call back-to-the-future (BTF) projections. We show that these projections can be used to accurately predict future COVID-19 surges prior to an inflection point of the daily infection curve. We show, across 12 different countries from all populated continents around the world, that our method can often predict future surges in scenarios where the traditional approaches would always predict no future surges. However, as expected, BTF projections cannot accurately predict a surge due to the emergence of a new variant. To generate BTF projections, we make use of a matching scheme for asynchronous time series combined with a response coaching SIR model.

A semi-empirical risk panel to monitor epidemics: multi-faceted tool to assist healthcare and public health professionals

Introduction

Bronchiolitis, mostly caused by Respiratory Syncytial Virus (RSV), and influenza among other respiratory infections, lead to seasonal saturation at healthcare centers in temperate areas. There is no gold standard to characterize the stages of epidemics, nor the risk of respiratory infections growing. We aimed to define a set of indicators to assess the risk level of respiratory viral epidemics, based on both incidence and their short-term dynamics, and considering epidemical thresholds.

Methods

We used publicly available data on daily cases of influenza for the whole population and bronchiolitis in children <2 years from the Information System for Infection Surveillance in Catalonia (SIVIC). We included a Moving Epidemic Method (MEM) variation to define epidemic threshold and levels. We pre-processed the data with two different nowcasting approaches and performed a 7-day moving average. Weekly incidences (cases per 105 population) were computed and the 5-day growth rate was defined to create the effective potential growth (EPG) indicator. We performed a correlation analysis to define the forecasting ability of this index.

Results

Our adaptation of the MEM method allowed us to define epidemic weekly incidence levels and epidemic thresholds for bronchiolitis and influenza. EPG was able to anticipate daily 7-day cumulative incidence by 4-5 (bronchiolitis) or 6-7 (influenza) days.

Discussion

We developed a semi-empirical risk panel incorporating the EPG index, which effectively anticipates surpassing epidemic thresholds for bronchiolitis and influenza. This panel could serve as a robust surveillance tool, applicable to respiratory infectious diseases characterized by seasonal epidemics, easy to handle for individuals lacking a mathematical background.

Estimating national and state-level suicide deaths using a novel online symptom search data source

BACKGROUND

Suicide mortality data are a critical source of information for understanding suicide-related trends in the United States. However, official suicide mortality data experience significant delays. The Google Symptom Search Dataset (SSD), a novel population-level data source derived from online search behavior, has not been evaluated for its utility in predicting suicide mortality trends.

METHODS

We identified five mental health related variables (suicidal ideation, self-harm, depression, major depressive disorder, and pain) from the SSD. Daily search trends for these symptoms were utilized to estimate national and state suicide counts in 2020, the most recent year for which data was available, via a linear regression model. We compared the performance of this model to a baseline autoregressive integrated moving average (ARIMA) model and a model including all 422 symptoms (All Symptoms) in the SSD.

RESULTS

Our Mental Health Model estimated the national number of suicide deaths with an error of -3.86 %, compared to an error of 7.17 % and 28.49 % for the ARIMA baseline and All Symptoms models. At the state level, 70 % (N = 35) of states had a prediction error of <10 % with the Mental Health Model, with accuracy generally favoring larger population states with higher number of suicide deaths.

CONCLUSION

The Google SSD is a new real-time data source that can be used to make accurate predictions of suicide mortality monthly trends at the national level. Additional research is needed to optimize state level predictions for states with low suicide counts.

Wastewater-based epidemiology predicts COVID-19-induced weekly new hospital admissions in over 150 USA counties

Although the coronavirus disease (COVID-19) emergency status is easing, the COVID-19 pandemic continues to affect healthcare systems globally. It is crucial to have a reliable and population-wide prediction tool for estimating COVID-19-induced hospital admissions. We evaluated the feasibility of using wastewater-based epidemiology (WBE) to predict COVID-19-induced weekly new hospitalizations in 159 counties across 45 states in the United States of America (USA), covering a population of nearly 100 million. Using county-level weekly wastewater surveillance data (over 20 months), WBE-based models were established through the random forest algorithm. WBE-based models accurately predicted the county-level weekly new admissions, allowing a preparation window of 1-4 weeks. In real applications, periodically updated WBE-based models showed good accuracy and transferability, with mean absolute error within 4-6 patients/100k population for upcoming weekly new hospitalization numbers. Our study demonstrated the potential of using WBE as an effective method to provide early warnings for healthcare systems.

Assessing the accuracy of California county level COVID-19 hospitalization forecasts to inform public policy decision making

BACKGROUND

The COVID-19 pandemic has highlighted the role of infectious disease forecasting in informing public policy. However, significant barriers remain for effectively linking infectious disease forecasts to public health decision making, including a lack of model validation. Forecasting model performance and accuracy should be evaluated retrospectively to understand under which conditions models were reliable and could be improved in the future.

METHODS

Using archived forecasts from the California Department of Public Health's California COVID Assessment Tool ( https://calcat.covid19.ca.gov/cacovidmodels/ ), we compared how well different forecasting models predicted COVID-19 hospitalization census across California counties and regions during periods of Alpha, Delta, and Omicron variant predominance.

RESULTS

Based on mean absolute error estimates, forecasting models had variable performance across counties and through time. When accounting for model availability across counties and dates, some individual models performed consistently better than the ensemble model, but model rankings still differed across counties. Local transmission trends, variant prevalence, and county population size were informative predictors for determining which model performed best for a given county based on a random forest classification analysis. Overall, the ensemble model performed worse in less populous counties, in part because of fewer model contributors in these locations.

CONCLUSIONS

Ensemble model predictions could be improved by incorporating geographic heterogeneity in model coverage and performance. Consistency in model reporting and improved model validation can strengthen the role of infectious disease forecasting in real-time public health decision making.

Enhancing long-term forecasting: Learning from COVID-19 models

While much effort has gone into building predictive models of the COVID-19 pandemic, some have argued that early exponential growth combined with the stochastic nature of epidemics make the long-term prediction of contagion trajectories impossible. We conduct two complementary studies to assess model features supporting better long-term predictions. First, we leverage the diverse models contributing to the CDC repository of COVID-19 USA death projections to identify factors associated with prediction accuracy across different projection horizons. We find that better long-term predictions correlate with: (1) capturing the physics of transmission (instead of using black-box models); (2) projecting human behavioral reactions to an evolving pandemic; and (3) resetting state variables to account for randomness not captured in the model before starting projection. Second, we introduce a very simple model, SEIRb, that incorporates these features, and few other nuances, offers informative predictions for as far as 20-weeks ahead, with accuracy comparable with the best models in the CDC set. Key to the long-term predictive power of multi-wave COVID-19 trajectories is capturing behavioral responses endogenously: balancing feedbacks where the perceived risk of death continuously changes transmission rates through the adoption and relaxation of various Non-Pharmaceutical Interventions (NPIs).

Long-term projections of COVID-19 trajectory have been used to inform various policies and decisions such as planning intensive care capacity, selecting clinical trial locations, and deciding on economic policy packages. However, these types of long-term forecasts are challenging as epidemics are complex: they include reinforcing contagion mechanisms that create exponential growth, are moderated by randomness in environmental and social determinants of transmission, and are subject to endogenous human responses to evolving risk perceptions. In this study we take a step towards systematically examining the modeling choices that regulate COVID-19 forecasting accuracy in two complementary studies. First, we leverage the diverse models contributing to the CDC repository of COVID-19 USA death projections to identify factors associated with prediction accuracy across different projection horizons. Second, we design a very simple forecasting model that only incorporates the key features identified in the first study, and show that the long-term prediction accuracy of this model is comparable with the best models in the CDC set. We conclude that forecasting models responding to future epidemics would benefit from starting small: first incorporating key mechanistic features, important behavioral feedbacks, and simple state-resetting approaches and then expanding to capture other features. Our study shows that the key to the long-term predictive power of epidemic models is an endogenous representation of human behavior in interaction with the evolving epidemic.

COVIDNearTerm: A simple method to forecast COVID-19 hospitalizations

Introduction

COVID-19 has caused tremendous death and suffering since it first emerged in 2019. Soon after its emergence, models were developed to help predict the course of various disease metrics, and these models have been relied upon to help guide public health policy.

Methods

Here we present a method called COVIDNearTerm to "forecast" hospitalizations in the short term, two to four weeks from the time of prediction. COVIDNearTerm is based on an autoregressive model and utilizes a parametric bootstrap approach to make predictions. It is easy to use as it requires only previous hospitalization data, and there is an open-source R package that implements the algorithm. We evaluated COVIDNearTerm on San Francisco Bay Area hospitalizations and compared it to models from the California COVID Assessment Tool (CalCAT).

Results

We found that COVIDNearTerm predictions were more accurate than the CalCAT ensemble predictions for all comparisons and any CalCAT component for a majority of comparisons. For instance, at the county level our 14-day hospitalization median absolute percentage errors ranged from 16 to 36%. For those same comparisons, the CalCAT ensemble errors were between 30 and 59%.

Conclusion

COVIDNearTerm is a simple and useful tool for predicting near-term COVID-19 hospitalizations.