A systematic review to identify areas of enhancements of pandemic simulation models for operational use at provincial and local levels

A competency framework on simulation modelling-supported decision-making for Master of Public Health graduates

BACKGROUND

Simulation models are increasingly important for supporting decision-making in public health. However, due to lack of training, many public health professionals remain unfamiliar with constructing simulation models and using their outputs for decision-making. This study contributes to filling this gap by developing a competency framework on simulation model-supported decision-making targeting Master of Public Health education.

METHODS

The study combined a literature review, a two-stage online Delphi survey and an online consensus workshop. A draft competency framework was developed based on 28 peer-reviewed publications. A two-stage online Delphi survey involving 15 experts was conducted to refine the framework. Finally, an online consensus workshop, including six experts, evaluated the competency framework and discussed its implementation.

RESULTS

The competency framework identified 20 competencies related to stakeholder engagement, problem definition, evidence identification, participatory system mapping, model creation and calibration and the interpretation and dissemination of model results. The expert evaluation recommended differentiating professional profiles and levels of expertise and synergizing with existing course contents to support its implementation.

CONCLUSIONS

The competency framework developed in this study is instrumental to including simulation model-supported decision-making in public health training. Future research is required to differentiate expertise levels and develop implementation strategies.

Computational decision-support tools for urban design to improve resilience against COVID-19 and other infectious diseases: A systematic review

The COVID-19 pandemic highlighted the need for decision-support tools to help cities become more resilient to infectious diseases. Through urban design and planning, non-pharmaceutical interventions can be enabled, impelling behaviour change and facilitating the construction of lower risk buildings and public spaces. Computational tools, including computer simulation, statistical models, and artificial intelligence, have been used to support responses to the current pandemic as well as to the spread of previous infectious diseases. Our multidisciplinary research group systematically reviewed state-of-the-art literature to propose a toolkit that employs computational modelling for various interventions and urban design processes. We selected 109 out of 8,737 studies retrieved from databases and analysed them based on the pathogen type, transmission mode and phase, design intervention and process, as well as modelling methodology (method, goal, motivation, focus, and indication to urban design). We also explored the relationship between infectious disease and urban design, as well as computational modelling support, including specific models and parameters. The proposed toolkit will help designers, planners, and computer modellers to select relevant approaches for evaluating design decisions depending on the target disease, geographic context, design stages, and spatial and temporal scales. The findings herein can be regarded as stand-alone tools, particularly for fighting against COVID-19, or be incorporated into broader frameworks to help cities become more resilient to future disasters.

Dynamic downscaling and daily nowcasting from influenza surveillance data

Real-time trends from surveillance data are important to assess and develop preparedness for influenza outbreaks. The overwhelming testing demand and limited capacity of testing laboratories for viral positivity render daily confirmed case data inaccurate and delay its availability in preparedness. Using Bayesian dynamic downscaling models, we obtained posterior estimates for daily influenza incidences from weekly estimates of the Centers for Disease Control and Prevention and daily reported constitutional and respiratory complaints during emergency department (ED) visits obtained from the state health departments. Our model provides one-day and seven-day lead forecasts along with 95 % $$ \% $$ prediction intervals. Our hybrid Markov Chain Monte Carlo and Kalman filter algorithms facilitate faster computation and enable us to update our estimates as new data become available. Our method is tested and validated using the State of Michigan data over the years 2009-2013. Reported constitutional and respiratory complaints at the EDs showed strong correlations of 0.81 and 0.68 respectively, with influenza rates. In general, our forecast model can be adapted to track an outbreak with only one respiratory virus as a causative agent.

Infectious Disease Modeling: Recommendations for Public Health Decision-Makers

Infectious disease modeling plays an important role in the response to infectious disease outbreaks, perhaps most notably during the coronavirus disease 2019 (COVID-19) pandemic. In our experience working with state and local governments during COVID-19 and previous public health crises, we have observed that, while the scientific literature focuses on models' accuracy and underlying assumptions, an important limitation on the effective application of modeling to public health decision-making is the ability of decision-makers and modelers to work together productively. We therefore propose a set of guiding principles, informed by our experience, for working relationships between decision-makers and modelers. We hypothesize that these guidelines will improve the utility of infectious disease modeling for public health decision-making, irrespective of the particular outbreak in question and of the precise modeling approaches being used.

Timeliness of provisional United States mortality data releases during the COVID-19 pandemic: delays associated with electronic death registration system and weekly mortality

All-cause mortality counts allow public health authorities to identify populations experiencing excess deaths from pandemics, natural disasters, and other emergencies. Delays in the completeness of mortality counts may contribute to misinformation because death counts take weeks to become accurate. We estimate the timeliness of all-cause mortality releases during the COVID-19 pandemic for the dates 3 April to 5 September 2020 by estimating the number of weekly data releases of the NCHS Fluview Mortality Surveillance System until mortality comes within 99% of the counts in the 19 March 19 2021 provisional mortality data release. States' mortality counts take 5 weeks at median (interquartile range 4-7 weeks) to completion. The fastest states were Maine, New Hampshire, Vermont, New York, Utah, Idaho, and Hawaii. States that had not adopted the electronic death registration system (EDRS) were 4.8 weeks slower to achieve complete mortality counts, and each weekly death per 10^8 was associated with a 0.8 week delay. Emergency planning should improve the timeliness of mortality data by improving state vital statistics digital infrastructure.

Systematic review of predictive mathematical models of COVID-19 epidemic

BACKGROUND

Various mathematical models were published to predict the epidemiological consequences of the COVID-19 pandemic. This systematic review has studied the initial epidemiological models.

METHODS

Articles published from January to June 2020 were extracted from databases using search strings and those peer-reviewed with full text in English were included in the study. They were analysed as to whether they made definite predictions in terms of time and numbers, or contained only mathematical assumptions and open-ended predictions. Factors such as early vs. late prediction models, long-term vs. curve-fitting models and comparisons based on modelling techniques were analysed in detail.

RESULTS

Among 56,922 hits in 05 databases, screening yielded 434 abstracts, of which 72 articles were included. Predictive models comprised over 70% (51/72) of the articles, with susceptible, exposed, infectious and recovered (SEIR) being the commonest type (mean duration of prediction being 3 months). Common predictions were regarding cumulative cases (44/72, 61.1%), time to reach total numbers (41/72, 56.9%), peak numbers (22/72, 30.5%), time to peak (24/72, 33.3%), hospital utilisation (7/72, 9.7%) and effect of lockdown and NPIs (50/72, 69.4%). The commonest countries for which models were predicted were China followed by USA, South Korea, Japan and India. Models were published by various professionals including Engineers (12.5%), Mathematicians (9.7%), Epidemiologists (11.1%) and Physicians (9.7%) with a third (32.9%) being the result of collaborative efforts between two or more professions.

CONCLUSION

There was a wide diversity in the type of models, duration of prediction and the variable that they predicted, with SEIR model being the commonest type.

Excess mortality in the United States during the first three months of the COVID-19 pandemic

Deaths are frequently under-estimated during emergencies, times when accurate mortality estimates are crucial for emergency response. This study estimates excess all-cause, pneumonia and influenza mortality during the coronavirus disease 2019 (COVID-19) pandemic using the 11 September 2020 release of weekly mortality data from the United States (U.S.) Mortality Surveillance System (MSS) from 27 September 2015 to 9 May 2020, using semiparametric and conventional time-series models in 13 states with high reported COVID-19 deaths and apparently complete mortality data: California, Colorado, Connecticut, Florida, Illinois, Indiana, Louisiana, Massachusetts, Michigan, New Jersey, New York, Pennsylvania and Washington. We estimated greater excess mortality than official COVID-19 mortality in the U.S. (excess mortality 95% confidence interval (CI) 100 013-127 501 vs. 78 834 COVID-19 deaths) and 9 states: California (excess mortality 95% CI 3338-6344) vs. 2849 COVID-19 deaths); Connecticut (excess mortality 95% CI 3095-3952) vs. 2932 COVID-19 deaths); Illinois (95% CI 4646-6111) vs. 3525 COVID-19 deaths); Louisiana (excess mortality 95% CI 2341-3183 vs. 2267 COVID-19 deaths); Massachusetts (95% CI 5562-7201 vs. 5050 COVID-19 deaths); New Jersey (95% CI 13 170-16 058 vs. 10 465 COVID-19 deaths); New York (95% CI 32 538-39 960 vs. 26 584 COVID-19 deaths); and Pennsylvania (95% CI 5125-6560 vs. 3793 COVID-19 deaths). Conventional model results were consistent with semiparametric results but less precise. Significant excess pneumonia deaths were also found for all locations and we estimated hundreds of excess influenza deaths in New York. We find that official COVID-19 mortality substantially understates actual mortality, excess deaths cannot be explained entirely by official COVID-19 death counts. Mortality reporting lags appeared to worsen during the pandemic, when timeliness in surveillance systems was most crucial for improving pandemic response.

The impact of opening dedicated clinics on disease transmission during an influenza pandemic

Dedicated clinics can be established in an influenza pandemic to isolate people and potentially reduce opportunities for influenza transmission. However, their operation requires resources and their existence may attract the worried-well. In this study, we quantify the impact of opening dedicated influenza clinics during a pandemic based on an agent-based simulation model across a time-varying social network of households, workplaces, schools, community locations, and health facilities in the state of Georgia. We calculate performance measures, including peak prevalence and total attack rate, while accounting for clinic operations, including timing and location. We find that opening clinics can reduce disease spread and hospitalizations even when visited by the worried-well, open for limited weeks, or open in limited locations, and especially when the clinics are in operation during times of highest prevalence. Specifically, peak prevalence, total attack rate, and hospitalization reduced 0.07–0.32%, 0.40–1.51%, 0.02–0.09%, respectively, by operating clinics for the pandemic duration.

Model Comparisons of the Effectiveness and Cost-Effectiveness of Vaccination: A Systematic Review of the Literature

OBJECTIVES

To describe all published articles that have conducted comparisons of model-based effectiveness and cost-effectiveness results in the field of vaccination. Specific objectives were to 1) describe the methodologies used and 2) identify the strengths and limitations of the studies.

METHODS

We systematically searched MEDLINE and Embase databases for studies that compared predictions of effectiveness and cost-effectiveness of vaccination of two or more mathematical models. We categorized studies into two groups on the basis of their data source for comparison (previously published results or new simulation results) and performed a qualitative synthesis of study conclusions.

RESULTS

We identified 115 eligible articles (only 5% generated new simulations from the reviewed models) examining the effectiveness and cost-effectiveness of vaccination against 14 pathogens (69% of studies examined human papillomavirus, influenza, and/or pneumococcal vaccines). The goal of most of studies was to summarize evidence for vaccination policy decisions, and cost-effectiveness was the most frequent outcome examined. Only 33%, 25%, and 3% of studies followed a systematic approach to identify eligible studies, assessed the quality of studies, and performed a quantitative synthesis of results, respectively. A greater proportion of model comparisons using published studies followed a systematic approach to identify eligible studies and to assess their quality, whereas more studies using new simulations performed quantitative synthesis of results and identified drivers of model conclusions. Most comparative modeling studies concluded that vaccination was cost-effective.

CONCLUSIONS

Given the variability in methods used to conduct/report comparative modeling studies, guidelines are required to enhance their quality and transparency and to provide better tools for decision making.

Effectiveness of workplace social distancing measures in reducing influenza transmission: a systematic review

BACKGROUND

Social distancing is one of the community mitigation measures that may be recommended during influenza pandemics. Social distancing can reduce virus transmission by increasing physical distance or reducing frequency of congregation in socially dense community settings, such as schools or workplaces. We conducted a systematic review to assess the evidence that social distancing in non-healthcare workplaces reduces or slows influenza transmission.

METHODS

Electronic searches were conducted using MEDLINE, Embase, Scopus, Cochrane Library, PsycINFO, CINAHL, NIOSHTIC-2, and EconLit to identify studies published in English from January 1, 2000, through May 3, 2017. Data extraction was done by two reviewers independently. A narrative synthesis was performed.

RESULTS

Fifteen studies, representing 12 modeling and three epidemiological, met the eligibility criteria. The epidemiological studies showed that social distancing was associated with a reduction in influenza-like illness and seroconversion to 2009 influenza A (H1N1). However, the overall risk of bias in the epidemiological studies was serious. The modeling studies estimated that workplace social distancing measures alone produced a median reduction of 23% in the cumulative influenza attack rate in the general population. It also delayed and reduced the peak influenza attack rate. The reduction in the cumulative attack rate was more pronounced when workplace social distancing was combined with other nonpharmaceutical or pharmaceutical interventions. However, the effectiveness was estimated to decline with higher basic reproduction number values, delayed triggering of workplace social distancing, or lower compliance.

CONCLUSIONS

Modeling studies support social distancing in non-healthcare workplaces, but there is a paucity of well-designed epidemiological studies.

SYSTEMATIC REVIEW REGISTRATION NUMBER

PROSPERO registration # CRD42017065310.

Effectiveness of workplace social distancing measures in reducing influenza transmission: a systematic review

BACKGROUND

Social distancing is one of the community mitigation measures that may be recommended during influenza pandemics. Social distancing can reduce virus transmission by increasing physical distance or reducing frequency of congregation in socially dense community settings, such as schools or workplaces. We conducted a systematic review to assess the evidence that social distancing in non-healthcare workplaces reduces or slows influenza transmission.

METHODS

Electronic searches were conducted using MEDLINE, Embase, Scopus, Cochrane Library, PsycINFO, CINAHL, NIOSHTIC-2, and EconLit to identify studies published in English from January 1, 2000, through May 3, 2017. Data extraction was done by two reviewers independently. A narrative synthesis was performed.

RESULTS

Fifteen studies, representing 12 modeling and three epidemiological, met the eligibility criteria. The epidemiological studies showed that social distancing was associated with a reduction in influenza-like illness and seroconversion to 2009 influenza A (H1N1). However, the overall risk of bias in the epidemiological studies was serious. The modeling studies estimated that workplace social distancing measures alone produced a median reduction of 23% in the cumulative influenza attack rate in the general population. It also delayed and reduced the peak influenza attack rate. The reduction in the cumulative attack rate was more pronounced when workplace social distancing was combined with other nonpharmaceutical or pharmaceutical interventions. However, the effectiveness was estimated to decline with higher basic reproduction number values, delayed triggering of workplace social distancing, or lower compliance.

CONCLUSIONS

Modeling studies support social distancing in non-healthcare workplaces, but there is a paucity of well-designed epidemiological studies.

SYSTEMATIC REVIEW REGISTRATION NUMBER

PROSPERO registration # CRD42017065310.

Estimating disease burden of a potential A(H7N9) pandemic influenza outbreak in the United States

BACKGROUND

Since spring 2013, periodic emergence of avian influenza A(H7N9) virus in China has heightened the concern for a possible pandemic outbreak among humans, though it is believed that the virus is not yet human-to-human transmittable. Till June 2017, A(H7N9) has resulted in 1533 laboratory-confirmed cases of human infections causing 592 deaths. The aim of this paper is to present disease burden estimates (measured by infection attack rates (IAR) and number of deaths) in the event of a possible pandemic outbreak caused by human-to-human transmission capability acquired by A(H7N9) virus. Even though such a pandemic will likely spread worldwide, our focus in this paper is to estimate the impact on the United States alone.

METHOD

The method first uses a data clustering technique to divide 50 states in the U.S. into a small number of clusters. Thereafter, for a few selected states in each cluster, the method employs an agent-based (AB) model to simulate human A(H7N9) influenza pandemic outbreaks. The model uses demographic and epidemiological data. A few selected non-pharmaceutical intervention (NPI) measures are applied to mitigate the outbreaks. Disease burden for the U.S. is estimated by combining results from the clusters applying a method used in stratified sampling.

RESULTS

Two possible pandemic scenarios with R 0 = 1.5 and 1.8 are examined. Infection attack rates with 95% C.I. (Confidence Interval) for R 0 = 1.5 and 1.8 are estimated to be 18.78% (17.3-20.27) and 25.05% (23.11-26.99), respectively. The corresponding number of deaths (95% C.I.), per 100,000, are 7252.3 (6598.45-7907.33) and 9670.99 (8953.66-10,389.95).

CONCLUSIONS

The results reflect a possible worst-case scenario where the outbreak extends over all states of the U.S. and antivirals and vaccines are not administered. Our disease burden estimations are also likely to be somewhat high due to the fact that only dense urban regions covering approximately 3% of the geographic area and 81% of the population are used for simulating sample outbreaks. Outcomes from these simulations are extrapolated over the remaining 19% of the population spread sparsely over 97% of the area. Furthermore, the full extent of possible NPIs, if deployed, could also have lowered the disease burden estimates.

Modeling and Forecasting Influenza-like Illness (ILI) in Houston, Texas Using Three Surveillance Data Capture Mechanisms

Objective The objective was to forecast and validate prediction estimates of influenza activity in Houston, TX using four years of historical influenza-like illness (ILI) from three surveillance data capture mechanisms. Background Using novel surveillance methods and historical data to estimate future trends of influenza-like illness can lead to early detection of influenza activity increases and decreases. Anticipating surges gives public health professionals more time to prepare and increase prevention efforts. Methods Data was obtained from three surveillance systems, Flu Near You, ILINet, and hospital emergency center (EC) visits, with diverse data capture mechanisms. Autoregressive integrated moving average (ARIMA) models were fitted to data from each source for week 27 of 2012 through week 26 of 2016 and used to forecast influenza-like activity for the subsequent 10 weeks. Estimates were then compared to actual ILI percentages for the same period. Results Forecasted estimates had wide confidence intervals that crossed zero. The forecasted trend direction differed by data source, resulting in lack of consensus about future influenza activity. ILINet forecasted estimates and actual percentages had the least differences. ILINet performed best when forecasting influenza activity in Houston, TX. Conclusion Though the three forecasted estimates did not agree on the trend directions, and thus, were considered imprecise predictors of long-term ILI activity based on existing data, pooling predictions and careful interpretations may be helpful for short term intervention efforts. Further work is needed to improve forecast accuracy considering the promise forecasting holds for seasonal influenza prevention and control, and pandemic preparedness.

Pandemic Risk Assessment Model (PRAM): a mathematical modeling approach to pandemic influenza planning

The Pandemic Risk Assessment Model (PRAM) is a mathematical model developed to analyse two pandemic influenza control measures available to public health: antiviral treatment and immunization. PRAM is parameterized using surveillance data from Alberta, Canada during pandemic H1N1. Age structure and risk level are incorporated in the compartmental, deterministic model through a contact matrix. The model characterizes pandemic influenza scenarios by transmissibility and severity properties. Simulating a worst-case scenario similar to the 1918 pandemic with immediate stockpile release, antiviral demand is 20·3% of the population. With concurrent, effective and timely immunization strategies, antiviral demand would be significantly less. PRAM will be useful in informing policy decisions such as the size of the Alberta antiviral stockpile and can contribute to other pandemic influenza planning activities and scenario analyses.

Network epidemiology and plant trade networks

Models of epidemics in complex networks are improving our predictive understanding of infectious disease outbreaks. Nonetheless, applying network theory to plant pathology is still a challenge. This overview summarizes some key developments in network epidemiology that are likely to facilitate its application in the study and management of plant diseases. Recent surveys have provided much-needed datasets on contact patterns and human mobility in social networks, but plant trade networks are still understudied. Human (and plant) mobility levels across the planet are unprecedented-there is thus much potential in the use of network theory by plant health authorities and researchers. Given the directed and hierarchical nature of plant trade networks, there is a need for plant epidemiologists to further develop models based on undirected and homogeneous networks. More realistic plant health scenarios would also be obtained by developing epidemic models in dynamic, rather than static, networks. For plant diseases spread by the horticultural and ornamental trade, there is the challenge of developing spatio-temporal epidemic simulations integrating network data. The use of network theory in plant epidemiology is a promising avenue and could contribute to anticipating and preventing plant health emergencies such as European ash dieback.

Influenza forecasting in human populations: a scoping review

Forecasts of influenza activity in human populations could help guide key preparedness tasks. We conducted a scoping review to characterize these methodological approaches and identify research gaps. Adapting the PRISMA methodology for systematic reviews, we searched PubMed, CINAHL, Project Euclid, and Cochrane Database of Systematic Reviews for publications in English since January 1, 2000 using the terms “influenza AND (forecast* OR predict*)”, excluding studies that did not validate forecasts against independent data or incorporate influenza-related surveillance data from the season or pandemic for which the forecasts were applied. We included 35 publications describing population-based (N = 27), medical facility-based (N = 4), and regional or global pandemic spread (N = 4) forecasts. They included areas of North America (N = 15), Europe (N = 14), and/or Asia-Pacific region (N = 4), or had global scope (N = 3). Forecasting models were statistical (N = 18) or epidemiological (N = 17). Five studies used data assimilation methods to update forecasts with new surveillance data. Models used virological (N = 14), syndromic (N = 13), meteorological (N = 6), internet search query (N = 4), and/or other surveillance data as inputs. Forecasting outcomes and validation metrics varied widely. Two studies compared distinct modeling approaches using common data, 2 assessed model calibration, and 1 systematically incorporated expert input. Of the 17 studies using epidemiological models, 8 included sensitivity analysis. This review suggests need for use of good practices in influenza forecasting (e.g., sensitivity analysis); direct comparisons of diverse approaches; assessment of model calibration; integration of subjective expert input; operational research in pilot, real-world applications; and improved mutual understanding among modelers and public health officials.

An operational epidemiological model for calibrating agent-based simulations of pandemic influenza outbreaks

Uncertainty of pandemic influenza viruses continue to cause major preparedness challenges for public health policymakers. Decisions to mitigate influenza outbreaks often involve tradeoff between the social costs of interventions (e.g., school closure) and the cost of uncontrolled spread of the virus. To achieve a balance, policymakers must assess the impact of mitigation strategies once an outbreak begins and the virus characteristics are known. Agent-based (AB) simulation is a useful tool for building highly granular disease spread models incorporating the epidemiological features of the virus as well as the demographic and social behavioral attributes of tens of millions of affected people. Such disease spread models provide excellent basis on which various mitigation strategies can be tested, before they are adopted and implemented by the policymakers. However, to serve as a testbed for the mitigation strategies, the AB simulation models must be operational. A critical requirement for operational AB models is that they are amenable for quick and simple calibration. The calibration process works as follows: the AB model accepts information available from the field and uses those to update its parameters such that some of its outputs in turn replicate the field data. In this paper, we present our epidemiological model based calibration methodology that has a low computational complexity and is easy to interpret. Our model accepts a field estimate of the basic reproduction number, and then uses it to update (calibrate) the infection probabilities in a way that its effect combined with the effects of the given virus epidemiology, demographics, and social behavior results in an infection pattern yielding a similar value of the basic reproduction number. We evaluate the accuracy of the calibration methodology by applying it for an AB simulation model mimicking a regional outbreak in the US. The calibrated model is shown to yield infection patterns closely replicating the input estimates of the basic reproduction number. The calibration method is also tested to replicate an initial infection incidence trend for a H1N1 outbreak like that of 2009.

Disease prediction models and operational readiness

The objective of this manuscript is to present a systematic review of biosurveillance models that operate on select agents and can forecast the occurrence of a disease event. We define a disease event to be a biological event with focus on the One Health paradigm. These events are characterized by evidence of infection and or disease condition. We reviewed models that attempted to predict a disease event, not merely its transmission dynamics and we considered models involving pathogens of concern as determined by the US National Select Agent Registry (as of June 2011). We searched commercial and government databases and harvested Google search results for eligible models, using terms and phrases provided by public health analysts relating to biosurveillance, remote sensing, risk assessments, spatial epidemiology, and ecological niche modeling. After removal of duplications and extraneous material, a core collection of 6,524 items was established, and these publications along with their abstracts are presented in a semantic wiki at http://BioCat.pnnl.gov. As a result, we systematically reviewed 44 papers, and the results are presented in this analysis. We identified 44 models, classified as one or more of the following: event prediction (4), spatial (26), ecological niche (28), diagnostic or clinical (6), spread or response (9), and reviews (3). The model parameters (e.g., etiology, climatic, spatial, cultural) and data sources (e.g., remote sensing, non-governmental organizations, expert opinion, epidemiological) were recorded and reviewed. A component of this review is the identification of verification and validation (V&V) methods applied to each model, if any V&V method was reported. All models were classified as either having undergone Some Verification or Validation method, or No Verification or Validation. We close by outlining an initial set of operational readiness level guidelines for disease prediction models based upon established Technology Readiness Level definitions.