Challenges in estimation, uncertainty quantification and elicitation for pandemic modelling

Precognition of Known And Unknown Biothreats: A Risk-Based Approach

Data mining and artificial intelligence algorithms can estimate the probability of future occurrences with defined precision. Yet, the prediction of infectious disease outbreaks remains a complex and difficult task. This is demonstrated by the limited accuracy and sensitivity of current models in predicting the emergence of previously unknown pathogens such as Zika, Chikungunya, and SARS-CoV-2, and the resurgence of Mpox, along with their impacts on global health, trade, and security. Comprehensive analysis of infectious disease risk profiles, vulnerabilities, and mitigation capacities, along with their spatiotemporal dynamics at the international level, is essential for preventing their transnational propagation. However, annual indexes about the impact of infectious diseases provide a low level of granularity to allow stakeholders to craft better mitigation strategies. A quantitative risk assessment by analytical platforms requires billions of near real-time data points from heterogeneous sources, integrating and analyzing univariable or multivariable data with different levels of complexity and latency that, in most cases, overwhelm human cognitive capabilities. Autonomous biosurveillance can open the possibility for near real-time, risk- and evidence-based policymaking and operational decision support.

Characterising information gains and losses when collecting multiple epidemic model outputs

BACKGROUND

Collaborative comparisons and combinations of epidemic models are used as policy-relevant evidence during epidemic outbreaks. In the process of collecting multiple model projections, such collaborations may gain or lose relevant information. Typically, modellers contribute a probabilistic summary at each time-step. We compared this to directly collecting simulated trajectories. We aimed to explore information on key epidemic quantities; ensemble uncertainty; and performance against data, investigating potential to continuously gain information from a single cross-sectional collection of model results.

METHODS

We compared projections from the European COVID-19 Scenario Modelling Hub. Five teams modelled incidence in Belgium, the Netherlands, and Spain. We compared July 2022 projections by incidence, peaks, and cumulative totals. We created a probabilistic ensemble drawn from all trajectories, and compared to ensembles from a median across each model's quantiles, or a linear opinion pool. We measured the predictive accuracy of individual trajectories against observations, using this in a weighted ensemble. We repeated this sequentially against increasing weeks of observed data. We evaluated these ensembles to reflect performance with varying observed data.

RESULTS

By collecting modelled trajectories, we showed policy-relevant epidemic characteristics. Trajectories contained a right-skewed distribution well represented by an ensemble of trajectories or a linear opinion pool, but not models' quantile intervals. Ensembles weighted by performance typically retained the range of plausible incidence over time, and in some cases narrowed this by excluding some epidemic shapes.

CONCLUSIONS

We observed several information gains from collecting modelled trajectories rather than quantile distributions, including potential for continuously updated information from a single model collection. The value of information gains and losses may vary with each collaborative effort's aims, depending on the needs of projection users. Understanding the differing information potential of methods to collect model projections can support the accuracy, sustainability, and communication of collaborative infectious disease modelling efforts.

Precision epidemiology at the nexus of mathematics and nanotechnology: Unraveling the dance of viral dynamics

The intersection of mathematical modeling, nanotechnology, and epidemiology marks a paradigm shift in our battle against infectious diseases, aligning with the focus of the journal on the regulation, expression, function, and evolution of genes in diverse biological contexts. This exploration navigates the intricate dance of viral transmission dynamics, highlighting mathematical models as dual tools of insight and precision instruments, a theme relevant to the diverse sections of Gene. In the context of virology, ethical considerations loom large, necessitating robust frameworks to protect individual rights, an aspect essential in infectious disease research. Global collaboration emerges as a critical pillar in our response to emerging infectious diseases, fortified by the predictive prowess of mathematical models enriched by nanotechnology. The synergy of interdisciplinary collaboration, training the next generation to bridge mathematical rigor, biology, and epidemiology, promises accelerated discoveries and robust models that account for real-world complexities, fostering innovation and exploration in the field. In this intricate review, mathematical modeling in viral transmission dynamics and epidemiology serves as a guiding beacon, illuminating the path toward precision interventions, global preparedness, and the collective endeavor to safeguard human health, resonating with the aim of advancing knowledge in gene regulation and expression.

Human judgement forecasting of COVID-19 in the UK

Background

In the past, two studies found ensembles of human judgement forecasts of COVID-19 to show predictive performance comparable to ensembles of computational models, at least when predicting case incidences. We present a follow-up to a study conducted in Germany and Poland and investigate a novel joint approach to combine human judgement and epidemiological modelling.

Methods

From May 24th to August 16th 2021, we elicited weekly one to four week ahead forecasts of cases and deaths from COVID-19 in the UK from a crowd of human forecasters. A median ensemble of all forecasts was submitted to the European Forecast Hub. Participants could use two distinct interfaces: in one, forecasters submitted a predictive distribution directly, in the other forecasters instead submitted a forecast of the effective reproduction number R t. This was then used to forecast cases and deaths using simulation methods from the EpiNow2 R package. Forecasts were scored using the weighted interval score on the original forecasts, as well as after applying the natural logarithm to both forecasts and observations.

Results

The ensemble of human forecasters overall performed comparably to the official European Forecast Hub ensemble on both cases and deaths, although results were sensitive to changes in details of the evaluation. R t forecasts performed comparably to direct forecasts on cases, but worse on deaths. Self-identified "experts" tended to be better calibrated than "non-experts" for cases, but not for deaths.

Conclusions

Human judgement forecasts and computational models can produce forecasts of similar quality for infectious disease such as COVID-19. The results of forecast evaluations can change depending on what metrics are chosen and judgement on what does or doesn't constitute a "good" forecast is dependent on the forecast consumer. Combinations of human and computational forecasts hold potential but present real-world challenges that need to be solved.

Modeling Supply and Demand Dynamics of Vaccines against Epidemic-Prone Pathogens: Case Study of Ebola Virus Disease

Health emergencies caused by epidemic-prone pathogens (EPPs) have increased exponentially in recent decades. Although vaccines have proven beneficial, they are unavailable for many pathogens. Furthermore, achieving timely and equitable access to vaccines against EPPs is not trivial. It requires decision-makers to capture numerous interrelated factors across temporal and spatial scales, with significant uncertainties, variability, delays, and feedback loops that give rise to dynamic and unexpected behavior. Therefore, despite progress in filling R&D gaps, the path to licensure and the long-term viability of vaccines against EPPs continues to be unclear. This paper presents a quantitative system dynamics modeling framework to evaluate the long-term sustainability of vaccine supply under different vaccination strategies. Data from both literature and 50 expert interviews are used to model the supply and demand of a prototypical Ebolavirus Zaire (EBOV) vaccine. Specifically, the case study evaluates dynamics associated with proactive vaccination ahead of an outbreak of similar magnitude as the 2018-2020 epidemic in North Kivu, Democratic Republic of the Congo. The scenarios presented demonstrate how uncertainties (e.g., duration of vaccine-induced protection) and design criteria (e.g., priority geographies and groups, target coverage, frequency of boosters) lead to important tradeoffs across policy aims, public health outcomes, and feasibility (e.g., technical, operational, financial). With sufficient context and data, the framework provides a foundation to apply the model to a broad range of additional geographies and priority pathogens. Furthermore, the ability to identify leverage points for long-term preparedness offers directions for further research.

Reflections On Epidemiological Modeling To Inform Policy During The COVID-19 Pandemic In Western Europe, 2020-23

We reflect on epidemiological modeling conducted throughout the COVID-19 pandemic in Western Europe, specifically in Belgium, France, Italy, the Netherlands, Portugal, Switzerland, and the United Kingdom. Western Europe was initially one of the worst-hit regions during the COVID-19 pandemic. Western European countries deployed a range of policy responses to the pandemic, which were often informed by mathematical, computational, and statistical models. Models differed in terms of temporal scope, pandemic stage, interventions modeled, and analytical form. This diversity was modulated by differences in data availability and quality, government interventions, societal responses, and technical capacity. Many of these models were decisive to policy making at key junctures, such as during the introduction of vaccination and the emergence of the Alpha, Delta, and Omicron variants. However, models also faced intense criticism from the press, other scientists, and politicians around their accuracy and appropriateness for decision making. Hence, evaluating the success of models in terms of accuracy and influence is an essential task. Modeling needs to be supported by infrastructure for systems to collect and share data, model development, and collaboration between groups, as well as two-way engagement between modelers and both policy makers and the public.

Inference on a Multi-Patch Epidemic Model with Partial Mobility, Residency, and Demography: Case of the 2020 COVID-19 Outbreak in Hermosillo, Mexico

Most studies modeling population mobility and the spread of infectious diseases, particularly those using meta-population multi-patch models, tend to focus on the theoretical properties and numerical simulation of such models. As such, there is relatively scant literature focused on numerical fit, inference, and uncertainty quantification of epidemic models with population mobility. In this research, we use three estimation techniques to solve an inverse problem and quantify its uncertainty for a human-mobility-based multi-patch epidemic model using mobile phone sensing data and confirmed COVID-19-positive cases in Hermosillo, Mexico. First, we utilize a Brownian bridge model using mobile phone GPS data to estimate the residence and mobility parameters of the epidemic model. In the second step, we estimate the optimal model epidemiological parameters by deterministically inverting the model using a Darwinian-inspired evolutionary algorithm (EA)-that is, a genetic algorithm (GA). The third part of the analysis involves performing inference and uncertainty quantification in the epidemic model using two Bayesian Monte Carlo sampling methods: t-walk and Hamiltonian Monte Carlo (HMC). The results demonstrate that the estimated model parameters and incidence adequately fit the observed daily COVID-19 incidence in Hermosillo. Moreover, the estimated parameters from the HMC method yield large credible intervals, improving their coverage for the observed and predicted daily incidences. Furthermore, we observe that the use of a multi-patch model with mobility yields improved predictions when compared to a single-patch model.

Tracking the national and regional COVID-19 epidemic status in the UK using weighted principal component analysis

One of the difficulties in monitoring an ongoing pandemic is deciding on the metric that best describes its status when multiple intercorrelated measurements are available. Having a single measure, such as the effective reproduction number [Formula: see text], has been a simple and useful metric for tracking the epidemic and for imposing policy interventions to curb the increase when [Formula: see text]. While [Formula: see text] is easy to interpret in a fully susceptible population, it is more difficult to interpret for a population with heterogeneous prior immunity, e.g. from vaccination and prior infection. We propose an additional metric for tracking the UK epidemic that can capture the different spatial scales. These are the principal scores from a weighted principal component analysis. In this paper, we have used the methodology across the four UK nations and across the first two epidemic waves (January 2020-March 2021) to show that first principal score across nations and epidemic waves is a representative indicator of the state of the pandemic and is correlated with the trend in R. Hospitalizations are shown to be consistently representative; however, the precise dominant indicator, i.e. the principal loading(s) of the analysis, can vary geographically and across epidemic waves. This article is part of the theme issue 'Technical challenges of modelling real-life epidemics and examples of overcoming these'.

The challenges of data in future pandemics

The use of data has been essential throughout the unfolding COVID-19 pandemic. We have needed it to populate our models, inform our understanding, and shape our responses to the disease. However, data has not always been easy to find and access, it has varied in quality and coverage, been difficult to reuse or repurpose. This paper reviews these and other challenges and recommends steps to develop a data ecosystem better able to deal with future pandemics by better supporting preparedness, prevention, detection and response.

Modelling: Understanding pandemics and how to control them

New disease challenges, societal demands and better or novel types of data, drive innovations in the structure, formulation and analysis of epidemic models. Innovations in modelling can lead to new insights into epidemic processes and better use of available data, yielding improved disease control and stimulating collection of better data and new data types. Here we identify key challenges for the structure, formulation, analysis and use of mathematical models of pathogen transmission relevant to current and future pandemics.

Challenges for modelling interventions for future pandemics

Mathematical modelling and statistical inference provide a framework to evaluate different non-pharmaceutical and pharmaceutical interventions for the control of epidemics that has been widely used during the COVID-19 pandemic. In this paper, lessons learned from this and previous epidemics are used to highlight the challenges for future pandemic control. We consider the availability and use of data, as well as the need for correct parameterisation and calibration for different model frameworks. We discuss challenges that arise in describing and distinguishing between different interventions, within different modelling structures, and allowing both within and between host dynamics. We also highlight challenges in modelling the health economic and political aspects of interventions. Given the diversity of these challenges, a broad variety of interdisciplinary expertise is needed to address them, combining mathematical knowledge with biological and social insights, and including health economics and communication skills. Addressing these challenges for the future requires strong cross-disciplinary collaboration together with close communication between scientists and policy makers.