A Bayesian ensemble approach for epidemiological projections

An ensemble n-sub-epidemic modeling framework for short-term forecasting epidemic trajectories: Application to the COVID-19 pandemic in the USA

We analyze an ensemble of n-sub-epidemic modeling for forecasting the trajectory of epidemics and pandemics. These ensemble modeling approaches, and models that integrate sub-epidemics to capture complex temporal dynamics, have demonstrated powerful forecasting capability. This modeling framework can characterize complex epidemic patterns, including plateaus, epidemic resurgences, and epidemic waves characterized by multiple peaks of different sizes. We systematically assess their calibration and short-term forecasting performance in short-term forecasts for the COVID-19 pandemic in the USA from late April 2020 to late February 2022. We compare their performance with two commonly used statistical ARIMA models. The best fit sub-epidemic model and three ensemble models constructed using the top-ranking sub-epidemic models consistently outperformed the ARIMA models in terms of the weighted interval score (WIS) and the coverage of the 95% prediction interval across the 10-, 20-, and 30-day short-term forecasts. In our 30-day forecasts, the average WIS ranged from 377.6 to 421.3 for the sub-epidemic models, whereas it ranged from 439.29 to 767.05 for the ARIMA models. Across 98 short-term forecasts, the ensemble model incorporating the top four ranking sub-epidemic models (Ensemble(4)) outperformed the (log) ARIMA model 66.3% of the time, and the ARIMA model, 69.4% of the time in 30-day ahead forecasts in terms of the WIS. Ensemble(4) consistently yielded the best performance in terms of the metrics that account for the uncertainty of the predictions. This framework can be readily applied to investigate the spread of epidemics and pandemics beyond COVID-19, as well as other dynamic growth processes found in nature and society that would benefit from short-term predictions.

Vote-processing rules for combining control recommendations from multiple models

Mathematical modelling is used during disease outbreaks to compare control interventions. Using multiple models, the best method to combine model recommendations is unclear. Existing methods weight model projections, then rank control interventions using the combined projections, presuming model outputs are directly comparable. However, the way each model represents the epidemiological system will vary. We apply electoral vote-processing rules to combine model-generated rankings of interventions. Combining rankings of interventions, instead of combining model projections, avoids assuming that projections are comparable as all comparisons of projections are made within each model. We investigate four rules: First-past-the-post, Alternative Vote (AV), Coombs Method and Borda Count. We investigate rule sensitivity by including models that favour only one action or including those that rank interventions randomly. We investigate two case studies: the 2014 Ebola outbreak in West Africa (37 compartmental models) and a hypothetical foot-and-mouth disease outbreak in UK (four individual-based models). The Coombs Method was least susceptible to adding models that favoured a single action, Borda Count and AV were most susceptible to adding models that ranked interventions randomly. Each rule chose the same intervention as when ranking interventions by mean projections, suggesting that combining rankings provides similar recommendations with fewer assumptions about model comparability. This article is part of the theme issue 'Technical challenges of modelling real-life epidemics and examples of overcoming these'.

Statistical methods used to combine the effective reproduction number, R(t), and other related measures of COVID-19 in the UK

In the recent COVID-19 pandemic, a wide range of epidemiological modelling approaches were used to predict the effective reproduction number, R(t), and other COVID-19-related measures such as the daily rate of exponential growth, r(t). These candidate models use different modelling approaches or differing assumptions about spatial or age-mixing, and some capture genuine uncertainty in scientific understanding of disease dynamics. Combining estimates using appropriate statistical methodology from multiple candidate models is important to better understand the variation of these outcome measures to help inform decision-making. In this paper, we combine estimates for specific UK nations/regions using random-effects meta-analyses techniques, utilising the restricted maximum-likelihood (REML) method to estimate the heterogeneity variance parameter, and two approaches to calculate the confidence interval for the combined estimate: the standard Wald-type and the Knapp and Hartung (KNHA) method. As estimates in this setting are derived using model predictions, each with varying degrees of uncertainty, equal-weighting is favoured over the standard inverse-variance weighting to avoid potential up-weighting of models providing estimates with lower levels of uncertainty that are not fully accounting for inherent uncertainties. Both equally-weighted models using REML alone and REML+KNHA approaches were found to provide similar variation for R(t) and r(t), with both approaches providing wider, and therefore more conservative, confidence intervals around the combined estimate compared to the standard inverse-variance weighting approach. Utilising these meta-analysis techniques has allowed for statistically robust combined estimates to be calculated for key COVID-19 outcome measures. This in turn allows timely and informed decision-making based on all available information.

An ensemble n-sub-epidemic modeling framework for short-term forecasting epidemic trajectories: Application to the COVID-19 pandemic in the USA.. [PMID: 35794886 PMCID: PMC9258290 DOI: 10.1101/2022.06.19.22276608] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

We analyze an ensemble of n-sub-epidemic modeling for forecasting the trajectory of epidemics and pandemics. These ensemble modeling approaches, and models that integrate sub-epidemics to capture complex temporal dynamics, have demonstrated powerful forecasting capability. This modeling framework can characterize complex epidemic patterns, including plateaus, epidemic resurgences, and epidemic waves characterized by multiple peaks of different sizes. We systematically assess their calibration and short-term forecasting performance in short-term forecasts for the COVID-19 pandemic in the USA from late April 2020 to late February 2022. We compare their performance with two commonly used statistical ARIMA models. The best fit sub-epidemic model and three ensemble models constructed using the top-ranking sub-epidemic models consistently outperformed the ARIMA models in terms of the weighted interval score (WIS) and the coverage of the 95% prediction interval across the 10-, 20-, and 30-day short-term forecasts. In the 30-day forecasts, the average WIS ranged from 377.6 to 421.3 for the sub-epidemic models, whereas it ranged from 439.29 to 767.05 for the ARIMA models. Across 98 short-term forecasts, the ensemble model incorporating the top four ranking sub-epidemic models (Ensemble(4)) outperformed the (log) ARIMA model 66.3% of the time, and the ARIMA model 69.4% of the time in 30-day ahead forecasts in terms of the WIS. Ensemble(4) consistently yielded the best performance in terms of the metrics that account for the uncertainty of the predictions. This framework could be readily applied to investigate the spread of epidemics and pandemics beyond COVID-19, as well as other dynamic growth processes found in nature and society that would benefit from short-term predictions.

The COVID-19 pandemic has highlighted the urgent need to develop reliable tools to forecast the trajectory of epidemics and pandemics in near real-time. We describe and apply an ensemble n-sub-epidemic modeling framework for forecasting the trajectory of epidemics and pandemics. We systematically assess its calibration and short-term forecasting performance in weekly 10–30 days ahead forecasts for the COVID-19 pandemic in the USA from late April 2020 to late February 2022 and compare its performance with two different statistical ARIMA models. This framework demonstrated reliable forecasting performance and substantially outcompeted the ARIMA models. The forecasting performance was consistently best for the ensemble sub-epidemic models incorporating a higher number of top-ranking sub-epidemic models. The ensemble model incorporating the top four ranking sub-epidemic models consistently yielded the best performance, particularly in terms of the coverage rate of the 95% prediction interval and the weighted interval score. This framework can be applied to forecast other growth processes found in nature and society including the spread of information through social media.

Improving astronaut cancer risk assessment from space radiation with an ensemble model framework

A new approach to NASA space radiation risk modeling has successfully extended the current NASA probabilistic cancer risk model to an ensemble framework able to consider sub-model parameter uncertainty as well as model-form uncertainty associated with differing theoretical or empirical formalisms. Ensemble methodologies are already widely used in weather prediction, modeling of infectious disease outbreaks, and certain terrestrial radiation protection applications to better understand how uncertainty may influence risk decision-making. Applying ensemble methodologies to space radiation risk projections offers the potential to efficiently incorporate emerging research results, allow for the incorporation of future models, improve uncertainty quantification, and reduce the impact of subjective bias. Moreover, risk forecasting across an ensemble of multiple predictive models can provide stakeholders additional information on risk acceptance if current health/medical standards cannot be met for future space exploration missions, such as human missions to Mars. In this work, ensemble risk projections implementing multiple sub-models of radiation quality, dose and dose-rate effectiveness factors, excess risk, and latency are presented. Initial consensus methods for ensemble model weights and correlations to account for individual model bias are discussed. In these analyses, the ensemble forecast compares well to results from NASA's current operational cancer risk projection model used to assess permissible mission durations for astronauts. However, a large range of projected risk values are obtained at the upper 95th confidence level where models must extrapolate beyond available biological data sets. Closer agreement is seen at the median ± one sigma due to the inherent similarities in available models. Identification of potential new models, epidemiological data, and methods for statistical correlation between predictive ensemble members are discussed. Alternate ways of communicating risk and acceptable uncertainty with respect to NASA's current permissible exposure limits are explored.

Trade-offs between individual and ensemble forecasts of an emerging infectious disease

Probabilistic forecasts play an indispensable role in answering questions about the spread of newly emerged pathogens. However, uncertainties about the epidemiology of emerging pathogens can make it difficult to choose among alternative model structures and assumptions. To assess the potential for uncertainties about emerging pathogens to affect forecasts of their spread, we evaluated the performance 16 forecasting models in the context of the 2015-2016 Zika epidemic in Colombia. Each model featured a different combination of assumptions about human mobility, spatiotemporal variation in transmission potential, and the number of virus introductions. We found that which model assumptions had the most ensemble weight changed through time. We additionally identified a trade-off whereby some individual models outperformed ensemble models early in the epidemic, but on average the ensembles outperformed all individual models. Our results suggest that multiple models spanning uncertainty across alternative assumptions are necessary to obtain robust forecasts for emerging infectious diseases.

The Use and Misuse of Mathematical Modeling for Infectious Disease Policymaking: Lessons for the COVID-19 Pandemic

Mathematical modeling has played a prominent and necessary role in the current coronavirus disease 2019 (COVID-19) pandemic, with an increasing number of models being developed to track and project the spread of the disease, as well as major decisions being made based on the results of these studies. A proliferation of models, often diverging widely in their projections, has been accompanied by criticism of the validity of modeled analyses and uncertainty as to when and to what extent results can be trusted. Drawing on examples from COVID-19 and other infectious diseases of global importance, we review key limitations of mathematical modeling as a tool for interpreting empirical data and informing individual and public decision making. We present several approaches that have been used to strengthen the validity of inferences drawn from these analyses, approaches that will enable better decision making in the current COVID-19 crisis and beyond.

Ensemble bootstrap methodology for forecasting dynamic growth processes using differential equations: application to epidemic outbreaks

BACKGROUND

Ensemble modeling aims to boost the forecasting performance by systematically integrating the predictive accuracy across individual models. Here we introduce a simple-yet-powerful ensemble methodology for forecasting the trajectory of dynamic growth processes that are defined by a system of non-linear differential equations with applications to infectious disease spread.

METHODS

We propose and assess the performance of two ensemble modeling schemes with different parametric bootstrapping procedures for trajectory forecasting and uncertainty quantification. Specifically, we conduct sequential probabilistic forecasts to evaluate their forecasting performance using simple dynamical growth models with good track records including the Richards model, the generalized-logistic growth model, and the Gompertz model. We first test and verify the functionality of the method using simulated data from phenomenological models and a mechanistic transmission model. Next, the performance of the method is demonstrated using a diversity of epidemic datasets including scenario outbreak data of the Ebola Forecasting Challenge and real-world epidemic data outbreaks of including influenza, plague, Zika, and COVID-19.

RESULTS

We found that the ensemble method that randomly selects a model from the set of individual models for each time point of the trajectory of the epidemic frequently outcompeted the individual models as well as an alternative ensemble method based on the weighted combination of the individual models and yields broader and more realistic uncertainty bounds for the trajectory envelope, achieving not only better coverage rate of the 95% prediction interval but also improved mean interval scores across a diversity of epidemic datasets.

CONCLUSION

Our new methodology for ensemble forecasting outcompete component models and an alternative ensemble model that differ in how the variance is evaluated for the generation of the prediction intervals of the forecasts.

The predicted impact of tuberculosis preventive therapy: the importance of disease progression assumptions

BACKGROUND

Following infection with Mycobacterium tuberculosis (M.tb), individuals may rapidly develop tuberculosis (TB) disease or enter a "latent" infection state with a low risk of progression to disease. Mathematical models use a variety of structures and parameterisations to represent this process. The effect of these different assumptions on the predicted impact of TB interventions has not been assessed.

METHODS

We explored how the assumptions made about progression from infection to disease affect the predicted impact of TB preventive therapy. We compared the predictions using three commonly used model structures, and parameters derived from two different data sources.

RESULTS

The predicted impact of preventive therapy depended on both the model structure and parameterisation. At a baseline annual TB incidence of 500/100,000, there was a greater than 2.5-fold difference in the predicted reduction in incidence due to preventive therapy (ranging from 6 to 16%), and the number needed to treat to avert one TB case varied between 67 and 157. The relative importance of structure and parameters depended on baseline TB incidence and assumptions about the efficacy of preventive therapy, with the choice of structure becoming more important at higher incidence.

CONCLUSIONS

The assumptions use to represent progression to disease in models are likely to influence the predicted impact of preventive therapy and other TB interventions. Modelling estimates of TB preventive therapy should consider routinely incorporating structural uncertainty, particularly in higher burden settings. Not doing so may lead to inaccurate and over confident conclusions, and sub-optimal evidence for decision making.

FMD-VS: A virtual sensor to index FMD virus scattering

Foot-and-mouth disease (FMD) models—analytical models for tracking and analyzing FMD outbreaks—are known as dominant tools for examining the spread of the disease under various conditions and assessing the effectiveness of countermeasures. There has been some remarkable progress in modeling research since the UK epidemic in 2001. Several modeling methods have been introduced, developed, and are still growing. However, in 2010 when a FMD outbreak occurred in the Miyazaki prefecture, a crucial problem reported: Once a regional FMD outbreak occurs, municipal officials in the region must make various day-to-day decisions throughout this period of vulnerability. The deliverables of FMD modeling research in its current state appear insufficient to support the daily judgments required in such cases. FMD model can be an efficient support tool for prevention decisions. It requires being conversant with modeling and its preconditions. Therefore, most municipal officials with no knowledge or experience found full use of the model difficult. Given this limitation, the authors consider methods and systems to support users of FMD models who must make real-time epidemic-related judgments in the infected areas. We propose a virtual sensor, designated “FMD-VS,” to index FMD virus scattering in conditions where there is once a notion of FMD; and (2) shows how we apply the developed FMD-VS technique during an outbreak. In (1), we show our approach to constructing FMD-VS based on the existing FMD model and offer an analysis and evaluation method to assess its performance. We again present the results produced when the technique applied to 2010 infection data from the Miyazaki Prefecture. For (2), we outline the concept of a method that supports the prevention judgment of municipal officials and show how to use FMD-VS.

Epidemiologically and Socio-economically Optimal Policies via Bayesian Optimization

Mass public quarantining, colloquially known as a lock-down, is a non-pharmaceutical intervention to check spread of disease. This paper presents ESOP (Epidemiologically and Socio-economically Optimal Policies), a novel application of active machine learning techniques using Bayesian optimization, that interacts with an epidemiological model to arrive at lock-down schedules that optimally balance public health benefits and socio-economic downsides of reduced economic activity during lock-down periods. The utility of ESOP is demonstrated using case studies with VIPER (Virus-Individual-Policy-EnviRonment), a stochastic agent-based simulator that this paper also proposes. However, ESOP is flexible enough to interact with arbitrary epidemiological simulators in a black-box manner, and produce schedules that involve multiple phases of lock-downs.

Systematic selection between age and household structure for models aimed at emerging epidemic predictions

Numerous epidemic models have been developed to capture aspects of human contact patterns, making model selection challenging when they fit (often-scarce) early epidemic data equally well but differ in predictions. Here we consider the invasion of a novel directly transmissible infection and perform an extensive, systematic and transparent comparison of models with explicit age and/or household structure, to determine the accuracy loss in predictions in the absence of interventions when ignoring either or both social components. We conclude that, with heterogeneous and assortative contact patterns relevant to respiratory infections, the model’s age stratification is crucial for accurate predictions. Conversely, the household structure is only needed if transmission is highly concentrated in households, as suggested by an empirical but robust rule of thumb based on household secondary attack rate. This work serves as a template to guide the simplicity/accuracy trade-off in designing models aimed at initial, rapid assessment of potential epidemic severity.

Models of emerging epidemics can be exceedingly helpful in planning the response, but early on model selection is a difficult task. Here, the authors explore the joint contribution of age stratification and household structure on epidemic spread, and provides a rule of thumb to guide model choice.

Real-time forecasting of epidemic trajectories using computational dynamic ensembles

Forecasting the trajectory of social dynamic processes, such as the spread of infectious diseases, poses significant challenges that call for methods that account for data and model uncertainty. Here we introduce an ensemble model for sequential forecasting that weights a set of plausible models and use a frequentist computational bootstrap approach to evaluate its uncertainty. We demonstrate the feasibility of our approach using simple dynamic differential-equation models and the trajectory of outbreak scenarios of the Ebola Forecasting Challenge. Specifically, we generate sequential short-term forecasts of epidemic outbreaks by combining phenomenological models that incorporate flexible epidemic growth scaling, namely the Generalized-Growth Model (GGM) and the Generalized Logistic Model (GLM). We rely on the root-mean-square error (RMSE) to quantify the quality of the models' fits during the calibration periods for weighting their contribution to the ensemble model while forecasting performance was evaluated using the RMSE of the forecasts. For a given forecasting horizon (1-4 weeks), we report the performance for each model as the percentage of the number of times each model outperforms the other models. The overall mean RMSE performance of the GLM and the GGM-GLM ensemble models outcompeted that of participant models of the Ebola Forecasting Challenge. We also found that the ensemble model provided more accurate forecasts with higher frequency than the GGM and GLM models, but its performance varied across forecasting horizons. For instance, across all of the Ebola Challenge Scenarios, the ensemble model outperformed the other models at horizons of 2 and 3 weeks while the GLM outperformed other models at horizons of 1 and 4 weeks.

Uncertainty in malaria simulations in the highlands of Kenya: Relative contributions of model parameter setting, driving climate and initial condition errors

In this study, experiments are conducted to gauge the relative importance of model, initial condition, and driving climate uncertainty for simulations of malaria transmission at a highland plantation in Kericho, Kenya. A genetic algorithm calibrates each of these three factors within their assessed prior uncertainty in turn to see which allows the best fit to a timeseries of confirmed cases. It is shown that for high altitude locations close to the threshold for transmission, the spatial representativeness uncertainty for climate, in particular temperature, dominates the uncertainty due to model parameter settings. Initial condition uncertainty plays little role after the first two years, and is thus important in the early warning system context, but negligible for decadal and climate change investigations. Thus, while reducing uncertainty in the model parameters would improve the quality of the simulations, the uncertainty in the temperature driving data is critical. It is emphasized that this result is a function of the mean climate of the location itself, and it is shown that model uncertainty would be relatively more important at warmer, lower altitude locations.

A comparison of two mathematical models of the impact of mass drug administration on the transmission and control of schistosomiasis

•

This paper compares two mathematical models describing the transmission dynamics of schistosome infection and the impact of mass drug administration.

•

The models differ structurally in a number of ways, including the dynamics of the intermediate snail host and the treatment of adult worms within the human host.

•

The models are validated against data taken from a mass-drug administration trial in Mozambique.

•

The differences between the model predictions and the data are discussed in the context of the structural differences between the models.

The predictions of two mathematical models describing the transmission dynamics of schistosome infection and the impact of mass drug administration are compared. The models differ in their description of the dynamics of the parasites within the host population and in their representation of the stages of the parasite lifecycle outside of the host. Key parameters are estimated from data collected in northern Mozambique from 2011 to 2015. This type of data set is valuable for model validation as treatment prior to the study was minimal. Predictions from both models are compared with each other and with epidemiological observations. Both models have difficulty matching both the intensity and prevalence of disease in the datasets and are only partially successful at predicting the impact of treatment. The models also differ from each other in their predictions, both quantitatively and qualitatively, of the long-term impact of 10 years’ school-based mass drug administration. We trace the dynamical differences back to basic assumptions about worm aggregation, force of infection and the dynamics of the parasite in the snail population in the two models and suggest data which could discriminate between them. We also discuss limitations with the datasets used and ways in which data collection could be improved.

Predicting lymphatic filariasis transmission and elimination dynamics using a multi-model ensemble framework

•

No single mathematical model captures all features of parasite transmission dynamics.

•

Multi-model ensemble modelling can overcome biases of single models.

•

A multi-model ensemble of three lymphatic filariasis models is proposed and evaluated.

•

The multi-model ensemble outperformed the single models in predicting infection.

•

The ensemble approach may improve use of models to inform disease control policy.

Mathematical models of parasite transmission provide powerful tools for assessing the impacts of interventions. Owing to complexity and uncertainty, no single model may capture all features of transmission and elimination dynamics. Multi-model ensemble modelling offers a framework to help overcome biases of single models. We report on the development of a first multi-model ensemble of three lymphatic filariasis (LF) models (EPIFIL, LYMFASIM, and TRANSFIL), and evaluate its predictive performance in comparison with that of the constituents using calibration and validation data from three case study sites, one each from the three major LF endemic regions: Africa, Southeast Asia and Papua New Guinea (PNG). We assessed the performance of the respective models for predicting the outcomes of annual MDA strategies for various baseline scenarios thought to exemplify the current endemic conditions in the three regions. The results show that the constructed multi-model ensemble outperformed the single models when evaluated across all sites. Single models that best fitted calibration data tended to do less well in simulating the out-of-sample, or validation, intervention data. Scenario modelling results demonstrate that the multi-model ensemble is able to compensate for variance between single models in order to produce more plausible predictions of intervention impacts. Our results highlight the value of an ensemble approach to modelling parasite control dynamics. However, its optimal use will require further methodological improvements as well as consideration of the organizational mechanisms required to ensure that modelling results and data are shared effectively between all stakeholders.

Estimation of anthracnose dynamics by nonlinear filtering

In this work, we apply the nonlinear filtering theory to the estimation of the partially observed dynamics of anthracnose which is a phytopathology. The signal here is the inhibition rate and the observations are the fruit volume and the rotted volume. We propose stochastic models based on deterministic models studied previously in the literature, in order to represent the noise introduced by uncontrolled variations on parameters and errors on the measurements. Under the assumption of Brownian noises, we prove the well-posedness of the models in either they take into account the space variable or not. The filtering problem is solved for the nonspatial model giving Zakai and Kushner–Stratonovich equations satisfied respectively by the unnormalized and the normalized conditional distribution of the signal with respect to the observations. A prevision problem and a discrete filtering problem are also studied for the realistic cases of discrete and possibly incomplete observations. We illustrate the filter behavior through figures displaying the average estimation relative error and a 95% confidence region obtained after a hundred of numerical simulations with initial conditions taken randomly with respect to uniform law.

An ensemble approach to predicting the impact of vaccination on rotavirus disease in Niger

Recently developed vaccines provide a new way of controlling rotavirus in sub-Saharan Africa. Models for the transmission dynamics of rotavirus are critical both for estimating current burden from imperfect surveillance and for assessing potential effects of vaccine intervention strategies. We examine rotavirus infection in the Maradi area in southern Niger using hospital surveillance data provided by Epicentre collected over two years. Additionally, a cluster survey of households in the region allows us to estimate the proportion of children with diarrhea who consulted at a health structure. Model fit and future projections are necessarily particular to a given model; thus, where there are competing models for the underlying epidemiology an ensemble approach can account for that uncertainty. We compare our results across several variants of Susceptible-Infectious-Recovered (SIR) compartmental models to quantify the impact of modeling assumptions on our estimates. Model-specific parameters are estimated by Bayesian inference using Markov chain Monte Carlo. We then use Bayesian model averaging to generate ensemble estimates of the current dynamics, including estimates of R0, the burden of infection in the region, as well as the impact of vaccination on both the short-term dynamics and the long-term reduction of rotavirus incidence under varying levels of coverage. The ensemble of models predicts that the current burden of severe rotavirus disease is 2.6–3.7% of the population each year and that a 2-dose vaccine schedule achieving 70% coverage could reduce burden by 39–42%.

Essential information: Uncertainty and optimal control of Ebola outbreaks

Early resolution of uncertainty during an epidemic outbreak can lead to rapid and efficient decision making, provided that the uncertainty affects prioritization of actions. The wide range in caseload projections for the 2014 Ebola outbreak caused great concern and debate about the utility of models. By coding and running 37 published Ebola models with five candidate interventions, we found that, despite this large variation in caseload projection, the ranking of management options was relatively consistent. Reducing funeral transmission and reducing community transmission were generally ranked as the two best options. Value of information (VoI) analyses show that caseloads could be reduced by 11% by resolving all model-specific uncertainties, with information about model structure accounting for 82% of this reduction and uncertainty about caseload only accounting for 12%. Our study shows that the uncertainty that is of most interest epidemiologically may not be the same as the uncertainty that is most relevant for management. If the goal is to improve management outcomes, then the focus of study should be to identify and resolve those uncertainties that most hinder the choice of an optimal intervention. Our study further shows that simplifying multiple alternative models into a smaller number of relevant groups (here, with shared structure) could streamline the decision-making process and may allow for a better integration of epidemiological modeling and decision making for policy.

Essential information: Uncertainty and optimal control of Ebola outbreaks

Early resolution of uncertainty during an epidemic outbreak can lead to rapid and efficient decision making, provided that the uncertainty affects prioritization of actions. The wide range in caseload projections for the 2014 Ebola outbreak caused great concern and debate about the utility of models. By coding and running 37 published Ebola models with five candidate interventions, we found that, despite this large variation in caseload projection, the ranking of management options was relatively consistent. Reducing funeral transmission and reducing community transmission were generally ranked as the two best options. Value of information (VoI) analyses show that caseloads could be reduced by 11% by resolving all model-specific uncertainties, with information about model structure accounting for 82% of this reduction and uncertainty about caseload only accounting for 12%. Our study shows that the uncertainty that is of most interest epidemiologically may not be the same as the uncertainty that is most relevant for management. If the goal is to improve management outcomes, then the focus of study should be to identify and resolve those uncertainties that most hinder the choice of an optimal intervention. Our study further shows that simplifying multiple alternative models into a smaller number of relevant groups (here, with shared structure) could streamline the decision-making process and may allow for a better integration of epidemiological modeling and decision making for policy.

Ensemble modelling and structured decision-making to support Emergency Disease Management

Epidemiological models in animal health are commonly used as decision-support tools to understand the impact of various control actions on infection spread in susceptible populations. Different models contain different assumptions and parameterizations, and policy decisions might be improved by considering outputs from multiple models. However, a transparent decision-support framework to integrate outputs from multiple models is nascent in epidemiology. Ensemble modelling and structured decision-making integrate the outputs of multiple models, compare policy actions and support policy decision-making. We briefly review the epidemiological application of ensemble modelling and structured decision-making and illustrate the potential of these methods using foot and mouth disease (FMD) models. In case study one, we apply structured decision-making to compare five possible control actions across three FMD models and show which control actions and outbreak costs are robustly supported and which are impacted by model uncertainty. In case study two, we develop a methodology for weighting the outputs of different models and show how different weighting schemes may impact the choice of control action. Using these case studies, we broadly illustrate the potential of ensemble modelling and structured decision-making in epidemiology to provide better information for decision-making and outline necessary development of these methods for their further application.

Spatiotemporal Variation in Distance Dependent Animal Movement Contacts: One Size Doesn't Fit All

The structure of contacts that mediate transmission has a pronounced effect on the outbreak dynamics of infectious disease and simulation models are powerful tools to inform policy decisions. Most simulation models of livestock disease spread rely to some degree on predictions of animal movement between holdings. Typically, movements are more common between nearby farms than between those located far away from each other. Here, we assessed spatiotemporal variation in such distance dependence of animal movement contacts from an epidemiological perspective. We evaluated and compared nine statistical models, applied to Swedish movement data from 2008. The models differed in at what level (if at all), they accounted for regional and/or seasonal heterogeneities in the distance dependence of the contacts. Using a kernel approach to describe how probability of contacts between farms changes with distance, we developed a hierarchical Bayesian framework and estimated parameters by using Markov Chain Monte Carlo techniques. We evaluated models by three different approaches of model selection. First, we used Deviance Information Criterion to evaluate their performance relative to each other. Secondly, we estimated the log predictive posterior distribution, this was also used to evaluate their relative performance. Thirdly, we performed posterior predictive checks by simulating movements with each of the parameterized models and evaluated their ability to recapture relevant summary statistics. Independent of selection criteria, we found that accounting for regional heterogeneity improved model accuracy. We also found that accounting for seasonal heterogeneity was beneficial, in terms of model accuracy, according to two of three methods used for model selection. Our results have important implications for livestock disease spread models where movement is an important risk factor for between farm transmission. We argue that modelers should refrain from using methods to simulate animal movements that assume the same pattern across all regions and seasons without explicitly testing for spatiotemporal variation.

Trends in the Mechanistic and Dynamic Modeling of Infectious Diseases

The dynamics of infectious disease epidemics are driven by interactions between individuals with differing disease status (e.g., susceptible, infected, immune). Mechanistic models that capture the dynamics of such “dependent happenings” are a fundamental tool of infectious disease epidemiology. Recent methodological advances combined with access to new data sources and computational power have resulted in an explosion in the use of dynamic models in the analysis of emerging and established infectious diseases. Increasing use of models to inform practical public health decision making has challenged the field to develop new methods to exploit available data and appropriately characterize the uncertainty in the results. Here, we discuss recent advances and areas of active research in the mechanistic and dynamic modeling of infectious disease. We highlight how a growing emphasis on data and inference, novel forecasting methods, and increasing access to “big data” are changing the field of infectious disease dynamics. We showcase the application of these methods in phylodynamic research, which combines mechanistic models with rich sources of molecular data to tie genetic data to population-level disease dynamics. As dynamics and mechanistic modeling methods mature and are increasingly tied to principled statistical approaches, the historic separation between the infectious disease dynamics and “traditional” epidemiologic methods is beginning to erode; this presents new opportunities for cross pollination between fields and novel applications.

Validation of Models Used to Inform Colorectal Cancer Screening Guidelines: Accuracy and Implications

BACKGROUND

Microsimulation models synthesize evidence about disease processes and interventions, providing a method for predicting long-term benefits and harms of prevention, screening, and treatment strategies. Because models often require assumptions about unobservable processes, assessing a model's predictive accuracy is important.

METHODS

We validated 3 colorectal cancer (CRC) microsimulation models against outcomes from the United Kingdom Flexible Sigmoidoscopy Screening (UKFSS) Trial, a randomized controlled trial that examined the effectiveness of one-time flexible sigmoidoscopy screening to reduce CRC mortality. The models incorporate different assumptions about the time from adenoma initiation to development of preclinical and symptomatic CRC. Analyses compare model predictions to study estimates across a range of outcomes to provide insight into the accuracy of model assumptions.

RESULTS

All 3 models accurately predicted the relative reduction in CRC mortality 10 years after screening (predicted hazard ratios, with 95% percentile intervals: 0.56 [0.44, 0.71], 0.63 [0.51, 0.75], 0.68 [0.53, 0.83]; estimated with 95% confidence interval: 0.56 [0.45, 0.69]). Two models with longer average preclinical duration accurately predicted the relative reduction in 10-year CRC incidence. Two models with longer mean sojourn time accurately predicted the number of screen-detected cancers. All 3 models predicted too many proximal adenomas among patients referred to colonoscopy.

CONCLUSION

Model accuracy can only be established through external validation. Analyses such as these are therefore essential for any decision model. Results supported the assumptions that the average time from adenoma initiation to development of preclinical cancer is long (up to 25 years), and mean sojourn time is close to 4 years, suggesting the window for early detection and intervention by screening is relatively long. Variation in dwell time remains uncertain and could have important clinical and policy implications.

Forecasting influenza outbreak dynamics in Melbourne from Internet search query surveillance data

BACKGROUND

Accurate forecasting of seasonal influenza epidemics is of great concern to healthcare providers in temperate climates, as these epidemics vary substantially in their size, timing and duration from year to year, making it a challenge to deliver timely and proportionate responses. Previous studies have shown that Bayesian estimation techniques can accurately predict when an influenza epidemic will peak many weeks in advance, using existing surveillance data, but these methods must be tailored both to the target population and to the surveillance system.

OBJECTIVES

Our aim was to evaluate whether forecasts of similar accuracy could be obtained for metropolitan Melbourne (Australia).

METHODS

We used the bootstrap particle filter and a mechanistic infection model to generate epidemic forecasts for metropolitan Melbourne (Australia) from weekly Internet search query surveillance data reported by Google Flu Trends for 2006-14.

RESULTS AND CONCLUSIONS

Optimal observation models were selected from hundreds of candidates using a novel approach that treats forecasts akin to receiver operating characteristic (ROC) curves. We show that the timing of the epidemic peak can be accurately predicted 4-6 weeks in advance, but that the magnitude of the epidemic peak and the overall burden are much harder to predict. We then discuss how the infection and observation models and the filtering process may be refined to improve forecast robustness, thereby improving the utility of these methods for healthcare decision support.

Decision-making for foot-and-mouth disease control: Objectives matter

Formal decision-analytic methods can be used to frame disease control problems, the first step of which is to define a clear and specific objective. We demonstrate the imperative of framing clearly-defined management objectives in finding optimal control actions for control of disease outbreaks. We illustrate an analysis that can be applied rapidly at the start of an outbreak when there are multiple stakeholders involved with potentially multiple objectives, and when there are also multiple disease models upon which to compare control actions. The output of our analysis frames subsequent discourse between policy-makers, modellers and other stakeholders, by highlighting areas of discord among different management objectives and also among different models used in the analysis. We illustrate this approach in the context of a hypothetical foot-and-mouth disease (FMD) outbreak in Cumbria, UK using outputs from five rigorously-studied simulation models of FMD spread. We present both relative rankings and relative performance of controls within each model and across a range of objectives. Results illustrate how control actions change across both the base metric used to measure management success and across the statistic used to rank control actions according to said metric. This work represents a first step towards reconciling the extensive modelling work on disease control problems with frameworks for structured decision making.

Progress and Challenges in Infectious Disease Cartography

Quantitatively mapping the spatial distributions of infectious diseases is key to both investigating their epidemiology and identifying populations at risk of infection. Important advances in data quality and methodologies have allowed for better investigation of disease risk and its association with environmental factors. However, incorporating dynamic human behavioural processes in disease mapping remains challenging. For example, connectivity among human populations, a key driver of pathogen dispersal, has increased sharply over the past century, along with the availability of data derived from mobile phones and other dynamic data sources. Future work must be targeted towards the rapid updating and dissemination of appropriately designed disease maps to guide the public health community in reducing the global burden of infectious disease.

Defining the relationship between infection prevalence and clinical incidence of Plasmodium falciparum malaria

In many countries health system data remain too weak to accurately enumerate Plasmodium falciparum malaria cases. In response, cartographic approaches have been developed that link maps of infection prevalence with mathematical relationships to predict the incidence rate of clinical malaria. Microsimulation (or ‘agent-based') models represent a powerful new paradigm for defining such relationships; however, differences in model structure and calibration data mean that no consensus yet exists on the optimal form for use in disease-burden estimation. Here we develop a Bayesian statistical procedure combining functional regression-based model emulation with Markov Chain Monte Carlo sampling to calibrate three selected microsimulation models against a purpose-built data set of age-structured prevalence and incidence counts. This allows the generation of ensemble forecasts of the prevalence–incidence relationship stratified by age, transmission seasonality, treatment level and exposure history, from which we predict accelerating returns on investments in large-scale intervention campaigns as transmission and prevalence are progressively reduced.

Mathematical models are used to predict malaria burden to inform disease control efforts. Here, Cameron et al. use Bayesian statistics to calibrate previous models against a data set of age-structured prevalence and incidence, generating stratified forecasts of the prevalence–incidence relationship.