Real-time decision-making during emergency disease outbreaks

When do we need multiple infectious disease models? Agreement between projection rank and magnitude in a multi-model setting

Mathematical models are useful for public health planning and response to infectious disease threats. However, different models can provide differing results, which can hamper decision making if not synthesized appropriately. To address this challenge, multi-model hubs convene independent modeling groups to generate ensembles, known to provide more accurate predictions of future outcomes. Yet, these hubs are resource intensive, and how many models are sufficient in a hub is not known. Here, we compare the benefit of predictions from multiple models in different contexts: (1) decision settings that depend on predictions of quantitative outcomes (e.g., hospital capacity planning), where assessments of the benefits of multi-model ensembles have largely focused; and (2) decisions settings that require the ranking of alternative epidemic scenarios (e.g., comparing outcomes under multiple possible interventions and biological uncertainties). We develop a mathematical framework to mimic a multi-model prediction setting, and use this framework to quantify how frequently predictions from different models agree. We further explore multi-model agreement using real-world, empirical data from 14 rounds of U.S. COVID-19 Scenario Modeling Hub projections. Our results suggest that the value of multiple models could be different in different decision contexts, and if only a few models are available, focusing on the rank of alternative epidemic scenarios could be more robust than focusing on quantitative outcomes. Although additional exploration of the sufficient number of models for different contexts is still needed, our results indicate that it may be possible to identify decision contexts where it is robust to rely on fewer models, a finding that can inform the use of modeling resources during future public health crises.

Modeling target-density-based cull strategies to contain foot-and-mouth disease outbreaks

Total ring depopulation is sometimes used as a management strategy for emerging infectious diseases in livestock, which raises ethical concerns regarding the potential slaughter of large numbers of healthy animals. We evaluated a farm-density-based ring culling strategy to control foot-and-mouth disease (FMD) in the United Kingdom (UK), which may allow for some farms within rings around infected premises (IPs) to escape depopulation. We simulated this reduced farm density, or "target density", strategy using a spatially-explicit, stochastic, state-transition algorithm. We modeled FMD spread in four counties in the UK that have different farm demographics, using 740,000 simulations in a full-factorial analysis of epidemic impact measures (i.e., culled animals, culled farms, and epidemic length) and cull strategy parameters (i.e., target farm density, daily farm cull capacity, and cull radius). All of the cull strategy parameters listed above were drivers of epidemic impact. Our simulated target density strategy was usually more effective at combatting FMD compared with traditional total ring depopulation when considering mean culled animals and culled farms and was especially effective when daily farm cull capacity was low. The differences in epidemic impact measures among the counties are likely driven by farm demography, especially differences in cattle and farm density. To prevent over-culling and the associated economic, organizational, ethical, and psychological impacts, the target density strategy may be worth considering in decision-making processes for future control of FMD and other diseases.

A systematic review of mechanistic models used to study avian influenza virus transmission and control

The global spread of avian influenza A viruses in domestic birds is causing increasing socioeconomic devastation. Various mechanistic models have been developed to better understand avian influenza transmission and evaluate the effectiveness of control measures in mitigating the socioeconomic losses caused by these viruses. However, the results of models of avian influenza transmission and control have not yet been subject to a comprehensive review. Such a review could help inform policy makers and guide future modeling work. To help fill this gap, we conducted a systematic review of the mechanistic models that have been applied to field outbreaks. Our three objectives were to: (1) describe the type of models and their epidemiological context, (2) list estimates of commonly used parameters of low pathogenicity and highly pathogenic avian influenza transmission, and (3) review the characteristics of avian influenza transmission and the efficacy of control strategies according to the mechanistic models. We reviewed a total of 46 articles. Of these, 26 articles estimated parameters by fitting the model to data, one evaluated the effectiveness of control strategies, and 19 did both. Values of the between-individual reproduction number ranged widely: from 2.18 to 86 for highly pathogenic avian influenza viruses, and from 4.7 to 45.9 for low pathogenicity avian influenza viruses, depending on epidemiological settings, virus subtypes and host species. Other parameters, such as the durations of the latent and infectious periods, were often taken from the literature, limiting the models' potential insights. Concerning control strategies, many models evaluated culling (n = 15), while vaccination received less attention (n = 6). According to the articles reviewed, optimal control strategies varied between virus subtypes and local conditions, and depended on the overall objective of the intervention. For instance, vaccination was optimal when the objective was to limit the overall number of culled flocks. In contrast, pre-emptive culling was preferred for reducing the size and duration of an epidemic. Early implementation consistently improved the overall efficacy of interventions, highlighting the need for effective surveillance and epidemic preparedness.

Approaches for disease prioritization and decision-making in animal health, 2000-2021: a structured scoping review

This scoping review identifies and describes the methods used to prioritize diseases for resource allocation across disease control, surveillance, and research and the methods used generally in decision-making on animal health policy. Three electronic databases (Medline/PubMed, Embase, and CAB Abstracts) were searched for articles from 2000 to 2021. Searches identified 6, 395 articles after de-duplication, with an additional 64 articles added manually. A total of 6, 460 articles were imported to online document review management software (sysrev.com) for screening. Based on inclusion and exclusion criteria, 532 articles passed the first screening, and after a second round of screening, 336 articles were recommended for full review. A total of 40 articles were removed after data extraction. Another 11 articles were added, having been obtained from cross-citations of already identified articles, providing a total of 307 articles to be considered in the scoping review. The results show that the main methods used for disease prioritization were based on economic analysis, multi-criteria evaluation, risk assessment, simple ranking, spatial risk mapping, and simulation modeling. Disease prioritization was performed to aid in decision-making related to various categories: (1) disease control, prevention, or eradication strategies, (2) general organizational strategy, (3) identification of high-risk areas or populations, (4) assessment of risk of disease introduction or occurrence, (5) disease surveillance, and (6) research priority setting. Of the articles included in data extraction, 50.5% had a national focus, 12.3% were local, 11.9% were regional, 6.5% were sub-national, and 3.9% were global. In 15.2% of the articles, the geographic focus was not specified. The scoping review revealed the lack of comprehensive, integrated, and mutually compatible approaches to disease prioritization and decision support tools for animal health. We recommend that future studies should focus on creating comprehensive and harmonized frameworks describing methods for disease prioritization and decision-making tools in animal health.

Optimal Epidemic Control under Uncertainty: Tradeoffs between Information Collection and Other Actions

BACKGROUND

Recent epidemics and measures taken to control them-through vaccination or other actions-have highlighted the role and importance of uncertainty in public health. There is generally a tradeoff between information collection and other uses of resources. Whether this tradeoff is solved explicitly or implicitly, the concept of value of information is central to inform policy makers in an uncertain environment.

METHOD

We use a deterministic SIR (susceptible, infectious, recovered) disease emergence and transmission model with vaccination that can be administered as 1 or 2 doses. The disease parameters and vaccine characteristics are uncertain. We study the tradeoffs between information acquisition and 2 other measures: bringing vaccination forward and acquiring more vaccine doses. To do this, we quantify the expected value of perfect information (EVPI) under different constraints faced by public health authorities (i.e., the time of the vaccination campaign implementation and the number of vaccine doses available).

RESULTS

We discuss the appropriateness of different responses under uncertainty. We show that, in some cases, vaccinating later or with less vaccine doses but more information about the epidemic, and the efficacy of control strategies may bring better results than vaccinating earlier or with more doses and less information, respectively.

CONCLUSION

In the present methodological article, we show in an abstract setting how clearly defining and treating the tradeoff between information acquisition and the relaxation of constraints can improve public health decision making.

HIGHLIGHTS

Uncertainties can seriously hinder epidemic control, but resolving them is costly. Thus, there are tradeoffs between information collection and alternative uses of resources.We use a generic SIR model with vaccination and a value-of-information framework to explore these tradeoffs.We show in which cases vaccinating later with more information about the epidemic and the efficacy of control measures may be better-or not-than vaccinating earlier with less information.We show in which cases vaccinating with fewer vaccine doses and more information about the epidemic and the efficacy of control measures may be better-or not-than vaccinating with more doses and less information.

Stochastic optimization for vaccine and testing kit allocation for the COVID-19 pandemic

We present a formal mathematical modeling framework for a multi-agent sequential decision problem during an epidemic. The problem is formulated as a collaboration between a vaccination agent and learning agent to allocate stockpiles of vaccines and tests to a set of zones under various types of uncertainty. The model is able to capture passive information processes and maintain beliefs over the uncertain state of the world. We designed a parameterized direct lookahead approximation which is robust and scalable under different scenarios, resource scarcity, and beliefs about the environment. We design a test allocation policy designed to capture the value of information and demonstrate that it outperforms other learning policies when there is an extreme shortage of resources (information is scarce). We simulate the model with two scenarios including a resource allocation problem to each state in the United States and another for the nursing homes in Nevada. The US example demonstrates the scalability of the model and the nursing home example demonstrates the robustness under extreme resource shortages.

Vote-processing rules for combining control recommendations from multiple models

Mathematical modelling is used during disease outbreaks to compare control interventions. Using multiple models, the best method to combine model recommendations is unclear. Existing methods weight model projections, then rank control interventions using the combined projections, presuming model outputs are directly comparable. However, the way each model represents the epidemiological system will vary. We apply electoral vote-processing rules to combine model-generated rankings of interventions. Combining rankings of interventions, instead of combining model projections, avoids assuming that projections are comparable as all comparisons of projections are made within each model. We investigate four rules: First-past-the-post, Alternative Vote (AV), Coombs Method and Borda Count. We investigate rule sensitivity by including models that favour only one action or including those that rank interventions randomly. We investigate two case studies: the 2014 Ebola outbreak in West Africa (37 compartmental models) and a hypothetical foot-and-mouth disease outbreak in UK (four individual-based models). The Coombs Method was least susceptible to adding models that favoured a single action, Borda Count and AV were most susceptible to adding models that ranked interventions randomly. Each rule chose the same intervention as when ranking interventions by mean projections, suggesting that combining rankings provides similar recommendations with fewer assumptions about model comparability. This article is part of the theme issue 'Technical challenges of modelling real-life epidemics and examples of overcoming these'.

Optimal health and economic impact of non-pharmaceutical intervention measures prior and post vaccination in England: a mathematical modelling study

Background. Even with good progress on vaccination, SARS-CoV-2 infections in the UK may continue to impose a high burden of disease and therefore pose substantial challenges for health policy decision makers. Stringent government-mandated physical distancing measures (lockdown) have been demonstrated to be epidemiologically effective, but can have both positive and negative economic consequences. The duration and frequency of any intervention policy could, in theory, be optimized to maximize economic benefits while achieving substantial reductions in disease. Methods. Here, we use a pre-existing SARS-CoV-2 transmission model to assess the health and economic implications of different strengths of control through time in order to identify optimal approaches to non-pharmaceutical intervention stringency in the UK, considering the role of vaccination in reducing the need for future physical distancing measures. The model is calibrated to the COVID-19 epidemic in England and we carry out retrospective analysis of the optimal timing of precautionary breaks in 2020 and the optimal relaxation policy from the January 2021 lockdown, considering the willingness to pay (WTP) for health improvement. Results. We find that the precise timing and intensity of interventions is highly dependent upon the objective of control. As intervention measures are relaxed, we predict a resurgence in cases, but the optimal intervention policy can be established dependent upon the WTP per quality adjusted life year loss avoided. Our results show that establishing an optimal level of control can result in a reduction in net monetary loss of billions of pounds, dependent upon the precise WTP value. Conclusion. It is vital, as the UK emerges from lockdown, but continues to face an on-going pandemic, to accurately establish the overall health and economic costs when making policy decisions. We demonstrate how some of these can be quantified, employing mechanistic infectious disease transmission models to establish optimal levels of control for the ongoing COVID-19 pandemic.

Modelling livestock infectious disease control policy under differing social perspectives on vaccination behaviour

The spread of infection amongst livestock depends not only on the traits of the pathogen and the livestock themselves, but also on the veterinary health behaviours of farmers and how this impacts their implementation of disease control measures. Controls that are costly may make it beneficial for individuals to rely on the protection offered by others, though that may be sub-optimal for the population. Failing to account for socio-behavioural properties may produce a substantial layer of bias in infectious disease models. We investigated the role of heterogeneity in vaccine response across a population of farmers on epidemic outbreaks amongst livestock, caused by pathogens with differential speed of spread over spatial landscapes of farms for two counties in England (Cumbria and Devon). Under different compositions of three vaccine behaviour groups (precautionary, reactionary, non-vaccination), we evaluated from population- and individual-level perspectives the optimum threshold distance to premises with notified infection that would trigger responsive vaccination by the reactionary vaccination group. We demonstrate a divergence between population and individual perspectives in the optimal scale of reactive voluntary vaccination response. In general, minimising the population-level perspective cost requires a broader reactive uptake of the intervention, whilst optimising the outcome for the average individual increased the likelihood of larger scale disease outbreaks. When the relative cost of vaccination was low and the majority of premises had undergone precautionary vaccination, then adopting a perspective that optimised the outcome for an individual gave a broader spatial extent of reactive response compared to a perspective wanting to optimise outcomes for everyone in the population. Under our assumed epidemiological context, the findings identify livestock disease intervention receptiveness and cost combinations where one would expect strong disagreement between the intervention stringency that is best from the perspective of a stakeholder responsible for supporting the livestock industry compared to a sole livestock owner. Were such discord anticipated and achieving a consensus view across perspectives desired, the findings may also inform those managing veterinary health policy the requisite reduction in intervention cost and/or the required extent of nurturing beneficial community attitudes towards interventions.

The COVID-19 pandemic has shown how crucial human behaviour is in controlling the spread of an infectious disease. The same is true of livestock, where farmer behaviour is critical to reduce the spread of an infection to enhance animal welfare and reduce economic losses. An ongoing concern for livestock owners is therefore ensuring they have adequate disease management procedures. However, what an individual farmer considers an appropriate way to control an infection in their own livestock may not be the best way to prevent an infection for every farmer’s livestock in the population. We describe a mathematical model combining epidemiological and behavioural elements to study the tension between individual and population-level control of livestock diseases. Applied to representative livestock systems in two counties in England (Cumbria and Devon), and splitting farmers into three types of vaccine behaviour groups (precautionary, reactionary, non-vaccination), we show what individual farmers see as an effective way to reduce infection is not the same as would benefit every farmer. The preferred response to protect every farmer’s livestock is to encourage wider uptake of reactive vaccination, whereas optimising the spatial extent of reactive vaccination for the average individual increases the chance of larger disease outbreaks.

When Is Mass Prophylaxis Cost-Effective for Epidemic Control? A Comparison of Decision Approaches

BACKGROUND

For certain communicable disease outbreaks, mass prophylaxis of uninfected individuals can curtail new infections. When an outbreak emerges, decision makers could benefit from methods to quickly determine whether mass prophylaxis is cost-effective. We consider 2 approaches: a simple decision model and machine learning meta-models. The motivating example is plague in Madagascar.

METHODS

We use a susceptible-exposed-infectious-removed (SEIR) epidemic model to derive a decision rule based on the fraction of the population infected, effective reproduction ratio, infection fatality rate, quality-adjusted life-year loss associated with death, prophylaxis effectiveness and cost, time horizon, and willingness-to-pay threshold. We also develop machine learning meta-models of a detailed model of plague in Madagascar using logistic regression, random forest, and neural network models. In numerical experiments, we compare results using the decision rule and the meta-models to results obtained using the simulation model. We vary the initial fraction of the population infected, the effective reproduction ratio, the intervention start date and duration, and the cost of prophylaxis.

LIMITATIONS

We assume homogeneous mixing and no negative side effects due to antibiotic prophylaxis.

RESULTS

The simple decision rule matched the SEIR model outcome in 85.4% of scenarios. Using data for a 2017 plague outbreak in Madagascar, the decision rule correctly indicated that mass prophylaxis was not cost-effective. The meta-models were significantly more accurate, with an accuracy of 92.8% for logistic regression, 95.8% for the neural network model, and 96.9% for the random forest model.

CONCLUSIONS

A simple decision rule using minimal information about an outbreak can accurately evaluate the cost-effectiveness of mass prophylaxis for outbreak mitigation. Meta-models of a complex disease simulation can achieve higher accuracy but with greater computational and data requirements and less interpretability.

HIGHLIGHTS

We use a susceptible-exposed-infectious-removed model and net monetary benefit to derive a simple decision rule to evaluate the cost-effectiveness of mass prophylaxis.We use the example of plague in Madagascar to compare the performance of the analytically derived decision rule to that of machine learning meta-models trained on a stochastic dynamic transmission model.We assess the accuracy of each approach for different combinations of disease dynamics and intervention scenarios.The machine learning meta-models are more accurate predictors of mass prophylaxis cost-effectiveness. However, the simple decision rule is also accurate and may be a preferred substitute in low-resource settings.

Evaluating COVID-19 decision-making in a humanitarian setting: The case study of Somalia

The global COVID-19 pandemic is unprecedented in its scope and impact. While a great deal of research has been directed towards the response in high-income countries, relatively little is known about the way in which decision-makers in low-income and crisis-affected countries have contended with the epidemic. Through use of an a priori decision framework, we aimed to evaluate the process of policy and operational decision-making in relation to the COVID-19 response in Somalia, a chronically fragile country, focusing particularly on the use of information and the role of transparency. We undertook a desk review, observed a number of key decision-making fora and conducted a series of key informant and focus group discussions with a range of decision-makers including state authority, civil society, humanitarian and development actors. We found that nearly all actors struggled to make sense of the scale of the epidemic and form an appropriate response. Decisions made during the early months had a large impact on the course of the epidemic response. Decision-makers relied heavily on international norms and were constrained by a number of factors within the political environment including resource limitations, political contestation and low population adherence to response measures. Important aspects of the response suffered from a transparency deficit and would have benefitted from more inclusive decision-making. Development of decision support tools appropriate for crisis-affected settings that explicitly deal with individual and environmental decision factors could lead to more effective and timely epidemic response.

Bayesian nonparametric inference for heterogeneously mixing infectious disease models

Mathematical models of infectious disease transmission continue to play a vital role in understanding, mitigating, and preventing outbreaks. The vast majority of epidemic models in the literature are parametric, meaning that they contain inherent assumptions about how transmission occurs in a population. However, such assumptions can be lacking in appropriate biological or epidemiological justification and in consequence lead to erroneous scientific conclusions and misleading predictions. We propose a flexible Bayesian nonparametric framework that avoids the need to make strict model assumptions about the infection process and enables a far more data-driven modeling approach for inferring the mechanisms governing transmission. We use our methods to enhance our understanding of the transmission mechanisms of the 2001 UK foot and mouth disease outbreak.

Infectious disease transmission models require assumptions about how the pathogen spreads between individuals. These assumptions may be somewhat arbitrary, particularly when it comes to describing how transmission varies between individuals of different types or in different locations, and may in turn lead to incorrect conclusions or policy decisions. We develop a general Bayesian nonparametric framework for transmission modeling that removes the need to make such specific assumptions with regard to the infection process. We use multioutput Gaussian process prior distributions to model different infection rates in populations containing multiple types of individuals. Further challenges arise because the transmission process itself is unobserved, and large outbreaks can be computationally demanding to analyze. We address these issues by data augmentation and a suitable efficient approximation method. Simulation studies using synthetic data demonstrate that our framework gives accurate results. We analyze an outbreak of foot and mouth disease in the United Kingdom, quantifying the spatial transmission mechanism between farms with different combinations of livestock.

Spatially targeted digital chest radiography to reduce tuberculosis in high-burden settings: a study of adaptive decision making

Background:

Spatially-targeted approaches to screen for tuberculosis (TB) could accelerate TB control in high-burden populations. We aimed to estimate gains in case-finding yield under an adaptive decision-making approach for spatially-targeted, mobile digital chest radiography (dCXR)-based screening in communities with varying levels of TB prevalence.

Methods:

We used a Monte-Carlo simulation model to simulate a spatially-targeted screening intervention in 24 communities with TB prevalence estimates derived from a large community-randomized trial. We implemented a Thompson sampling algorithm to allocate screening units based on Bayesian probabilities of local TB prevalence that are continuously updated during weekly screening rounds. Four mobile units for dCXR-based screening and subsequent Xpert Ultra-based testing were allocated among the communities during a 52-week period. We estimated the yield of bacteriologically-confirmed TB per 1000 screenings comparing scenarios of spatially-targeted and untargeted resource allocation.

Results:

We estimated that under the untargeted scenario, an expected 666 (95% uncertainty interval 522–825) TB cases would be detected over one year, equivalent to 8.9 (7.5–10.3) per 1000 individuals screened. Allocating the screening units to the communities with the highest (prior-year) cases notification rates resulted in an expected 760 (617–926) TB cases detected, 10.1 (8.6–11.8) per 1000 screened. Adaptive, spatially-targeted screening resulted in an expected 1241 (995–1502) TB cases detected, 16.5 (14.5–18.7) per 1000 screened. Numbers of dCXR-based screenings needed to detect one additional TB case declined during the first 12–14 weeks as a result of Bayesian learning.

Conclusion:

We introduce a spatially-targeted screening strategy that could reduce the number of screenings necessary to detect additional TB in high-burden settings and thus improve the efficiency of screening interventions. Empirical trials are needed to determine whether this approach could be successfully implemented.

A model exploration of carrier and movement transmission as potential explanatory causes for the persistence of foot-and-mouth disease in endemic regions

Foot-and-mouth disease (FMD) is a virulent and economically important disease of livestock, still endemic in many areas of Asia and sub-Saharan Africa. Transmission from persistently infected livestock, also known as carriers, has been proposed as a mechanism to support the persistence of FMD in endemic regions. However, whether carrier livestock can infect susceptible animals is controversial; recovered virus is infectious and there are claims of field transmission, but it remains undemonstrated experimentally. Alternate hypotheses for persistence include the movement of livestock within and between regions, and fomite contamination of the environment. Using a stochastic compartmental ordinary differential equation (ODE) model, we investigate the minimum rates of carrier transmission necessary to contribute to the maintenance of FMD in a region, and compare this to the alternate mechanism of persistence through cattle shipments. We find that carrier transmission can theoretically support persistence even at transmission rates much lower than the highest realistic rates previously proposed, and that the parameters with the most effect on the feasibility of carrier-mediated persistence are the average duration of both the carrier phase and natural immunity. However, shipment-mediated persistence remains a viable alternate mechanism for persistence without carrier transmission.

One Health practices across key agencies in Sweden - Uncovering barriers to cooperation, communication and coordination

AIM

This study examined the barriers and opportunities in Sweden for integrating One Health practices. Sweden's long tradition of working with One Health was used as a case to analyse persistent barriers as well as opportunities.

METHOD

Thirteen semi-structured interviews with experts from the Swedish Veterinary Agency, Food Agency, Public Health Agency, and Environmental Protection Agency were carried out. A thematic content analysis was conducted on the interviews using inductive coding in NVivo.

RESULTS

The study revealed that while collaboration is the general aspiration across the Swedish agencies, barriers persist regarding the understanding of One Health, the integration of the environment sector and awareness of the different terminologies employed within the disciplines. There are legislative challenges and barriers to science to policy translation. Disease outbreak was identified as an opportunity for One Health integration.

CONCLUSIONS

A One Health strategy needs to be developed at agency level to define One Health and clarify the roles and responsibilities. To overcome practical challenges, experts need to be aware of different terminologies and practices when collaborating. Further prospects for One Health integration include employing policy entrepreneurs to push One Health onto the political agenda. Preparations for disease outbreaks need to focus on reducing barriers to effectively integrate One Health. Experiences of One Health projects must be disseminated, and to raise awareness, education programmes must integrate One Health into curricula.

Causes of delayed outbreak responses and their impacts on epidemic spread

Livestock diseases have devastating consequences economically, socially and politically across the globe. In certain systems, pathogens remain viable after host death, which enables residual transmissions from infected carcasses. Rapid culling and carcass disposal are well-established strategies for stamping out an outbreak and limiting its impact; however, wait-times for these procedures, i.e. response delays, are typically farm-specific and time-varying due to logistical constraints. Failing to incorporate variable response delays in epidemiological models may understate outbreak projections and mislead management decisions. We revisited the 2001 foot-and-mouth epidemic in the United Kingdom and sought to understand how misrepresented response delays can influence model predictions. Survival analysis identified farm size and control demand as key factors that impeded timely culling and disposal activities on individual farms. Using these factors in the context of an existing policy to predict local variation in response times significantly affected predictions at the national scale. Models that assumed fixed, timely responses grossly underestimated epidemic severity and its long-term consequences. As a result, this study demonstrates how general inclusion of response dynamics and recognition of partial controllability of interventions can help inform management priorities during epidemics of livestock diseases.

Anticipating future learning affects current control decisions: A comparison between passive and active adaptive management in an epidemiological setting

•

Adaptive epidemic control.

•

Using real-time outbreak information to improve epidemic control.

•

Active Adaptive Management in an epidemiological setting.

•

Analysing the interaction between control and monitoring during an epidemic.

Infectious disease epidemics present a difficult task for policymakers, requiring the implementation of control strategies under significant time constraints and uncertainty. Mathematical models can be used to predict the outcome of control interventions, providing useful information to policymakers in the event of such an epidemic. However, these models suffer in the early stages of an outbreak from a lack of accurate, relevant information regarding the dynamics and spread of the disease and the efficacy of control. As such, recommendations provided by these models are often incorporated in an ad hoc fashion, as and when more reliable information becomes available. In this work, we show that such trial-and-error-type approaches to management, which do not formally take into account the resolution of uncertainty and how control actions affect this, can lead to sub-optimal management outcomes. We compare three approaches to managing a theoretical epidemic: a non-adaptive management (AM) approach that does not use real-time outbreak information to adapt control, a passive AM approach that incorporates real-time information if and when it becomes available, and an active AM approach that explicitly incorporates the future resolution of uncertainty through gathering real-time information into its initial recommendations. The structured framework of active AM encourages the specification of quantifiable objectives, models of system behaviour and possible control and monitoring actions, followed by an iterative learning and control phase that is able to employ complex control optimisations and resolve system uncertainty. The result is a management framework that is able to provide dynamic, long-term projections to help policymakers meet the objectives of management. We investigate in detail the effect of different methods of incorporating up-to-date outbreak information. We find that, even in a highly simplified system, the method of incorporating new data can lead to different results that may influence initial policy decisions, with an active AM approach to management providing better information that can lead to more desirable outcomes from an epidemic.

COVID-19 and the epistemology of epidemiological models at the dawn of AI

The models used to estimate disease transmission, susceptibility and severity determine what epidemiology can (and cannot tell) us about COVID-19. These include: 'model organisms' chosen for their phylogenetic/aetiological similarities; multivariable statistical models to estimate the strength/direction of (potentially causal) relationships between variables (through 'causal inference'), and the (past/future) value of unmeasured variables (through 'classification/prediction'); and a range of modelling techniques to predict beyond the available data (through 'extrapolation'), compare different hypothetical scenarios (through 'simulation'), and estimate key features of dynamic processes (through 'projection'). Each of these models: address different questions using different techniques; involve assumptions that require careful assessment; and are vulnerable to generic and specific biases that can undermine the validity and interpretation of their findings. It is therefore necessary that the models used: can actually address the questions posed; and have been competently applied. In this regard, it is important to stress that extrapolation, simulation and projection cannot offer accurate predictions of future events when the underlying mechanisms (and the contexts involved) are poorly understood and subject to change. Given the importance of understanding such mechanisms/contexts, and the limited opportunity for experimentation during outbreaks of novel diseases, the use of multivariable statistical models to estimate the strength/direction of potentially causal relationships between two variables (and the biases incurred through their misapplication/misinterpretation) warrant particular attention. Such models must be carefully designed to address: 'selection-collider bias', 'unadjusted confounding bias' and 'inferential mediator adjustment bias' - all of which can introduce effects capable of enhancing, masking or reversing the estimated (true) causal relationship between the two variables examined.1 Selection-collider bias occurs when these two variables independently cause a third (the 'collider'), and when this collider determines/reflects the basis for selection in the analysis. It is likely to affect all incompletely representative samples, although its effects will be most pronounced wherever selection is constrained (e.g. analyses focusing on infected/hospitalised individuals). Unadjusted confounding bias disrupts the estimated (true) causal relationship between two variables when: these share one (or more) common cause(s); and when the effects of these causes have not been adjusted for in the analyses (e.g. whenever confounders are unknown/unmeasured). Inferentially similar biases can occur when: one (or more) variable(s) (or 'mediators') fall on the causal path between the two variables examined (i.e. when such mediators are caused by one of the variables and are causes of the other); and when these mediators are adjusted for in the analysis. Such adjustment is commonplace when: mediators are mistaken for confounders; prediction models are mistakenly repurposed for causal inference; or mediator adjustment is used to estimate direct and indirect causal relationships (in a mistaken attempt at 'mediation analysis'). These three biases are central to ongoing and unresolved epistemological tensions within epidemiology. All have substantive implications for our understanding of COVID-19, and the future application of artificial intelligence to 'data-driven' modelling of similar phenomena. Nonetheless, competently applied and carefully interpreted, multivariable statistical models may yet provide sufficient insight into mechanisms and contexts to permit more accurate projections of future disease outbreaks.

Challenges to the Application of Spatially Explicit Stochastic Simulation Models for Foot-and-Mouth Disease Control in Endemic Settings: A Systematic Review

Simulation modeling has become common for estimating the spread of highly contagious animal diseases. Several models have been developed to mimic the spread of foot-and-mouth disease (FMD) in specific regions or countries, conduct risk assessment, analyze outbreaks using historical data or hypothetical scenarios, assist in policy decisions during epidemics, formulate preparedness plans, and evaluate economic impacts. Majority of the available FMD simulation models were designed for and applied in disease-free countries, while there has been limited use of such models in FMD endemic countries. This paper's objective was to report the findings from a study conducted to review the existing published original research literature on spatially explicit stochastic simulation (SESS) models of FMD spread, focusing on assessing these models for their potential use in endemic settings. The goal was to identify the specific components of endemic FMD needed to adapt these SESS models for their potential application in FMD endemic settings. This systematic review followed the PRISMA guidelines, and three databases were searched, which resulted in 1176 citations. Eighty citations finally met the inclusion criteria and were included in the qualitative synthesis, identifying nine unique SESS models. These SESS models were assessed for their potential application in endemic settings. The assessed SESS models can be adapted for use in FMD endemic countries by modifying the underlying code to include multiple cocirculating serotypes, routine prophylactic vaccination (RPV), and livestock population dynamics to more realistically mimic the endemic characteristics of FMD. The application of SESS models in endemic settings will help evaluate strategies for FMD control, which will improve livestock health, provide economic gains for producers, help alleviate poverty and hunger, and will complement efforts to achieve the Sustainable Development Goals.

Predictive accuracy of a hierarchical logistic model of cumulative SARS-CoV-2 case growth until May 2020

BACKGROUND

Infectious disease predictions models, including virtually all epidemiological models describing the spread of the SARS-CoV-2 pandemic, are rarely evaluated empirically. The aim of the present study was to investigate the predictive accuracy of a prognostic model for forecasting the development of the cumulative number of reported SARS-CoV-2 cases in countries and administrative regions worldwide until the end of May 2020.

METHODS

The cumulative number of reported SARS-CoV-2 cases was forecasted in 251 regions with a horizon of two weeks, one month, and two months using a hierarchical logistic model at the end of March 2020. Forecasts were compared to actual observations by using a series of evaluation metrics.

RESULTS

On average, predictive accuracy was very high in nearly all regions at the two weeks forecast, high in most regions at the one month forecast, and notable in the majority of the regions at the two months forecast. Higher accuracy was associated with the availability of more data for estimation and with a more pronounced cumulative case growth from the first case to the date of estimation. In some strongly affected regions, cumulative case counts were considerably underestimated.

CONCLUSIONS

With keeping its limitations in mind, the investigated model may be used for the preparation and distribution of resources during the initial phase of epidemics. Future research should primarily address the model's assumptions and its scope of applicability. In addition, establishing a relationship with known mechanisms and traditional epidemiological models of disease transmission would be desirable.

COVID-19 reopening strategies at the county level in the face of uncertainty: Multiple Models for Outbreak Decision Support

Policymakers make decisions about COVID-19 management in the face of considerable uncertainty. We convened multiple modeling teams to evaluate reopening strategies for a mid-sized county in the United States, in a novel process designed to fully express scientific uncertainty while reducing linguistic uncertainty and cognitive biases. For the scenarios considered, the consensus from 17 distinct models was that a second outbreak will occur within 6 months of reopening, unless schools and non-essential workplaces remain closed. Up to half the population could be infected with full workplace reopening; non-essential business closures reduced median cumulative infections by 82%. Intermediate reopening interventions identified no win-win situations; there was a trade-off between public health outcomes and duration of workplace closures. Aggregate results captured twice the uncertainty of individual models, providing a more complete expression of risk for decision-making purposes.

Challenges and opportunities for using national animal datasets to support foot-and-mouth disease control

National level databases of animal numbers, locations and movements provide the essential foundations for disease preparedness, outbreak investigations and control activities. These activities are particularly important for managing and mitigating the risks of high-impact transboundary animal disease outbreaks such as foot-and-mouth disease (FMD), which can significantly affect international trade access and domestic food security. In countries where livestock production systems are heavily subsidized by the government, producers are often required to provide detailed animal movement and demographic data as a condition of business. In the remaining countries, it can be difficult to maintain these types of databases and impossible to estimate the extent of missing or inaccurate information due to the absence of gold standard datasets for comparison. Consequently, competent authorities are often required to make decisions about disease preparedness and control based on available data, which may result in suboptimal outcomes for their livestock industries. It is important to understand the limitations of poor data quality as well as the range of methods that have been developed to compensate in both disease-free and endemic situations. Using FMD as a case example, this review first discusses the different activities that competent authorities use farm-level animal population data for to support (1) preparedness activities in disease-free countries, (2) response activities during an acute outbreak in a disease-free country, and (3) eradication and control activities in an endemic country. We then discuss (4) data requirements needed to support epidemiological investigations, surveillance, and disease spread modelling both in disease-free and endemic countries.

Real-time estimation of disease activity in emerging outbreaks using internet search information

Understanding the behavior of emerging disease outbreaks in, or ahead of, real-time could help healthcare officials better design interventions to mitigate impacts on affected populations. Most healthcare-based disease surveillance systems, however, have significant inherent reporting delays due to data collection, aggregation, and distribution processes. Recent work has shown that machine learning methods leveraging a combination of traditionally collected epidemiological information and novel Internet-based data sources, such as disease-related Internet search activity, can produce meaningful "nowcasts" of disease incidence ahead of healthcare-based estimates, with most successful case studies focusing on endemic and seasonal diseases such as influenza and dengue. Here, we apply similar computational methods to emerging outbreaks in geographic regions where no historical presence of the disease of interest has been observed. By combining limited available historical epidemiological data available with disease-related Internet search activity, we retrospectively estimate disease activity in five recent outbreaks weeks ahead of traditional surveillance methods. We find that the proposed computational methods frequently provide useful real-time incidence estimates that can help fill temporal data gaps resulting from surveillance reporting delays. However, the proposed methods are limited by issues of sample bias and skew in search query volumes, perhaps as a result of media coverage.

Effects of regional differences and demography in modelling foot-and-mouth disease in cattle at the national scale

Foot-and-mouth disease (FMD) is a fast-spreading viral infection that can produce large and costly outbreaks in livestock populations. Transmission occurs at multiple spatial scales, as can the actions used to control outbreaks. The US cattle industry is spatially expansive, with heterogeneous distributions of animals and infrastructure. We have developed a model that incorporates the effects of scale for both disease transmission and control actions, applied here in simulating FMD outbreaks in US cattle. We simulated infection initiating in each of the 3049 counties in the contiguous US, 100 times per county. When initial infection was located in specific regions, large outbreaks were more likely to occur, driven by infrastructure and other demographic attributes such as premises clustering and number of cattle on premises. Sensitivity analyses suggest these attributes had more impact on outbreak metrics than the ranges of estimated disease parameter values. Additionally, although shipping accounted for a small percentage of overall transmission, areas receiving the most animal shipments tended to have other attributes that increase the probability of large outbreaks. The importance of including spatial and demographic heterogeneity in modelling outbreak trajectories and control actions is illustrated by specific regions consistently producing larger outbreaks than others.

Epidemic growth rates and host movement patterns shape management performance for pathogen spillover at the wildlife-livestock interface

Managing pathogen spillover at the wildlife-livestock interface is a key step towards improving global animal health, food security and wildlife conservation. However, predicting the effectiveness of management actions across host-pathogen systems with different life histories is an on-going challenge since data on intervention effectiveness are expensive to collect and results are system-specific. We developed a simulation model to explore how the efficacies of different management strategies vary according to host movement patterns and epidemic growth rates. The model suggested that fast-growing, fast-moving epidemics like avian influenza were best-managed with actions like biosecurity or containment, which limited and localized overall spillover risk. For fast-growing, slower-moving diseases like foot-and-mouth disease, depopulation or prophylactic vaccination were competitive management options. Many actions performed competitively when epidemics grew slowly and host movements were limited, and how management efficacy related to epidemic growth rate or host movement propensity depended on what objective was used to evaluate management performance. This framework offers one means of classifying and prioritizing responses to novel pathogen spillover threats, and evaluating current management actions for pathogens emerging at the wildlife-livestock interface. This article is part of the theme issue 'Dynamic and integrative approaches to understanding pathogen spillover'.

The Role of Movement Restrictions in Limiting the Economic Impact of Livestock Infections

Movements are essential for the economic success of the livestock industry. These movements however bring the risk of long-range spread of infection, potentially bringing infection to previously disease-free areas where subsequent localised transmission can be devastating. Mechanistic predictive models usually consider controls that minimize the number of livestock affected without considering other costs of an ongoing epidemic. However, it is more appropriate to consider the economic burden, as movement restrictions have major consequences for the economic revenue of farms. Using mechanistic models of foot-and-mouth disease (FMD), bluetongue virus (BTV) and bovine tuberculosis (bTB) in the UK, we contrast the economically optimal control strategies for these diseases. We show that for FMD, the optimal strategy is to ban movements in a small radius around infected farms; the balance between disease control and maintaining 'business as usual' varies between regions. For BTV and bTB, we find that the cost of any movement ban is more than the epidemiological benefits due to the low within-farm prevalence and slow rate of disease spread. This work suggests that movement controls need to be carefully matched to the epidemiological and economic consequences of the disease, and optimal movement bans are often far shorter than existing policy.

A Novel Integrated and Labile eHealth System for Monitoring Dog Rabies Vaccination Campaigns

The elimination of canine rabies through the implementation of high coverage mass dog vaccination campaigns is a complex task, particularly in the resource-limited countries of the rabies endemic world. Here we demonstrated the feasibility of applying targeted rabies vaccination campaigns to deliver more impactful intervention campaigns in resource-limited settings using evidence and lessons learnt from other diseases. With the use of strategic rabies intervention programs, we demonstrate the noteworthy reduction of rabies cases in two very different African settings. The strategic intervention was most significantly aided by the use of a custom-developed vaccination tracking device (the Global Alliance for Rabies Control (GARC) Data Logger) and an integrated rabies surveillance system (the Rabies Epidemiological Bulletin). Our first case study, an island-wide strategic dog vaccination on Tanzania's Unguja island, reduced the incidence of rabies by 71% in the first 16 months of implementation. In the second case study, a similar approach was applied in the metropolitan capital city of Zimbabwe and the incidence of rabies declined by 13% during the first 13 months of implementation. The methodologies and results presented here suggest that, in resource-limited settings, an optimal approach towards the elimination of dog rabies would revolve around strategic interventions, subject to the use of appropriate planning, surveillance, and vaccination tools.

Context matters: using reinforcement learning to develop human-readable, state-dependent outbreak response policies

The number of all possible epidemics of a given infectious disease that could occur on a given landscape is large for systems of real-world complexity. Furthermore, there is no guarantee that the control actions that are optimal, on average, over all possible epidemics are also best for each possible epidemic. Reinforcement learning (RL) and Monte Carlo control have been used to develop machine-readable context-dependent solutions for complex problems with many possible realizations ranging from video-games to the game of Go. RL could be a valuable tool to generate context-dependent policies for outbreak response, though translating the resulting policies into simple rules that can be read and interpreted by human decision-makers remains a challenge. Here we illustrate the application of RL to the development of context-dependent outbreak response policies to minimize outbreaks of foot-and-mouth disease. We show that control based on the resulting context-dependent policies, which adapt interventions to the specific outbreak, result in smaller outbreaks than static policies. We further illustrate two approaches for translating the complex machine-readable policies into simple heuristics that can be evaluated by human decision-makers. This article is part of the theme issue 'Modelling infectious disease outbreaks in humans, animals and plants: epidemic forecasting and control'. This theme issue is linked with the earlier issue 'Modelling infectious disease outbreaks in humans, animals and plants: approaches and important themes'.

Concurrent assessment of epidemiological and operational uncertainties for optimal outbreak control: Ebola as a case study

Determining how best to manage an infectious disease outbreak may be hindered by both epidemiological uncertainty (i.e. about epidemiological processes) and operational uncertainty (i.e. about the effectiveness of candidate interventions). However, these two uncertainties are rarely addressed concurrently in epidemic studies. We present an approach to simultaneously address both sources of uncertainty, to elucidate which source most impedes decision-making. In the case of the 2014 West African Ebola outbreak, epidemiological uncertainty is represented by a large ensemble of published models. Operational uncertainty about three classes of interventions is assessed for a wide range of potential intervention effectiveness. We ranked each intervention by caseload reduction in each model, initially assuming an unlimited budget as a counterfactual. We then assessed the influence of three candidate cost functions relating intervention effectiveness and cost for different budget levels. The improvement in management outcomes to be gained by resolving uncertainty is generally high in this study; appropriate information gain could reduce expected caseload by more than 50%. The ranking of interventions is jointly determined by the underlying epidemiological process, the effectiveness of the interventions and the size of the budget. An epidemiologically effective intervention might not be optimal if its costs outweigh its epidemiological benefit. Under higher-budget conditions, resolution of epidemiological uncertainty is most valuable. When budgets are tight, however, operational and epidemiological uncertainty are equally important. Overall, our study demonstrates that significant reductions in caseload could result from a careful examination of both epidemiological and operational uncertainties within the same modelling structure. This approach can be applied to decision-making for the management of other diseases for which multiple models and multiple interventions are available.

Assessing the performance of real-time epidemic forecasts: A case study of Ebola in the Western Area region of Sierra Leone, 2014-15

Real-time forecasts based on mathematical models can inform critical decision-making during infectious disease outbreaks. Yet, epidemic forecasts are rarely evaluated during or after the event, and there is little guidance on the best metrics for assessment. Here, we propose an evaluation approach that disentangles different components of forecasting ability using metrics that separately assess the calibration, sharpness and bias of forecasts. This makes it possible to assess not just how close a forecast was to reality but also how well uncertainty has been quantified. We used this approach to analyse the performance of weekly forecasts we generated in real time for Western Area, Sierra Leone, during the 2013-16 Ebola epidemic in West Africa. We investigated a range of forecast model variants based on the model fits generated at the time with a semi-mechanistic model, and found that good probabilistic calibration was achievable at short time horizons of one or two weeks ahead but model predictions were increasingly unreliable at longer forecasting horizons. This suggests that forecasts may have been of good enough quality to inform decision making based on predictions a few weeks ahead of time but not longer, reflecting the high level of uncertainty in the processes driving the trajectory of the epidemic. Comparing forecasts based on the semi-mechanistic model to simpler null models showed that the best semi-mechanistic model variant performed better than the null models with respect to probabilistic calibration, and that this would have been identified from the earliest stages of the outbreak. As forecasts become a routine part of the toolkit in public health, standards for evaluation of performance will be important for assessing quality and improving credibility of mathematical models, and for elucidating difficulties and trade-offs when aiming to make the most useful and reliable forecasts.

Dynamics of the 2004 avian influenza H5N1 outbreak in Thailand: The role of duck farming, sequential model fitting and control

The Highly Pathogenic Avian Influenza (HPAI) subtype H5N1 virus persists in many countries and has been circulating in poultry, wild birds. In addition, the virus has emerged in other species and frequent zoonotic spillover events indicate that there remains a significant risk to human health. It is crucial to understand the dynamics of the disease in the poultry industry to develop a more comprehensive knowledge of the risks of transmission and to establish a better distribution of resources when implementing control. In this paper, we develop a set of mathematical models that simulate the spread of HPAI H5N1 in the poultry industry in Thailand, utilising data from the 2004 epidemic. The model that incorporates the intensity of duck farming when assessing transmision risk provides the best fit to the spatiotemporal characteristics of the observed outbreak, implying that intensive duck farming drives transmission of HPAI in Thailand. We also extend our models using a sequential model fitting approach to explore the ability of the models to be used in “real time” during novel disease outbreaks. We conclude that, whilst predictions of epidemic size are estimated poorly in the early stages of disease outbreaks, the model can infer the preferred control policy that should be deployed to minimise the impact of the disease.

Real-time decision-making during emergency disease outbreaks

In the event of a new infectious disease outbreak, mathematical and simulation models are commonly used to inform policy by evaluating which control strategies will minimize the impact of the epidemic. In the early stages of such outbreaks, substantial parameter uncertainty may limit the ability of models to provide accurate predictions, and policymakers do not have the luxury of waiting for data to alleviate this state of uncertainty. For policymakers, however, it is the selection of the optimal control intervention in the face of uncertainty, rather than accuracy of model predictions, that is the measure of success that counts. We simulate the process of real-time decision-making by fitting an epidemic model to observed, spatially-explicit, infection data at weekly intervals throughout two historical outbreaks of foot-and-mouth disease, UK in 2001 and Miyazaki, Japan in 2010, and compare forward simulations of the impact of switching to an alternative control intervention at the time point in question. These are compared to policy recommendations generated in hindsight using data from the entire outbreak, thereby comparing the best we could have done at the time with the best we could have done in retrospect. Our results show that the control policy that would have been chosen using all the data is also identified from an early stage in an outbreak using only the available data, despite high variability in projections of epidemic size. Critically, we find that it is an improved understanding of the locations of infected farms, rather than improved estimates of transmission parameters, that drives improved prediction of the relative performance of control interventions. However, the ability to estimate undetected infectious premises is a function of uncertainty in the transmission parameters. Here, we demonstrate the need for both real-time model fitting and generating projections to evaluate alternative control interventions throughout an outbreak. Our results highlight the use of using models at outbreak onset to inform policy and the importance of state-dependent interventions that adapt in response to additional information throughout an outbreak.