Guidelines for multi-model comparisons of the impact of infectious disease interventions

When do we need multiple infectious disease models? Agreement between projection rank and magnitude in a multi-model setting

Mathematical models are useful for public health planning and response to infectious disease threats. However, different models can provide differing results, which can hamper decision making if not synthesized appropriately. To address this challenge, multi-model hubs convene independent modeling groups to generate ensembles, known to provide more accurate predictions of future outcomes. Yet, these hubs are resource intensive, and how many models are sufficient in a hub is not known. Here, we compare the benefit of predictions from multiple models in different contexts: (1) decision settings that depend on predictions of quantitative outcomes (e.g., hospital capacity planning), where assessments of the benefits of multi-model ensembles have largely focused; and (2) decisions settings that require the ranking of alternative epidemic scenarios (e.g., comparing outcomes under multiple possible interventions and biological uncertainties). We develop a mathematical framework to mimic a multi-model prediction setting, and use this framework to quantify how frequently predictions from different models agree. We further explore multi-model agreement using real-world, empirical data from 14 rounds of U.S. COVID-19 Scenario Modeling Hub projections. Our results suggest that the value of multiple models could be different in different decision contexts, and if only a few models are available, focusing on the rank of alternative epidemic scenarios could be more robust than focusing on quantitative outcomes. Although additional exploration of the sufficient number of models for different contexts is still needed, our results indicate that it may be possible to identify decision contexts where it is robust to rely on fewer models, a finding that can inform the use of modeling resources during future public health crises.

The COVID-19 vaccination campaign in Switzerland and its impact on disease spread

We analyse infectious disease case surveillance data to estimate COVID-19 spread and gain an understanding of the impact of introducing vaccines to counter the disease in Switzerland. The data used in this work is extensive and detailed and includes information on weekly number of cases and vaccination rates by age and region. Our approach takes into account waning immunity. The statistical analysis allows us to determine the effects of choosing alternative vaccination strategies. Our results indicate greater uptake of vaccine would have led to fewer cases with a particularly large effect on undervaccinated regions. An alternative distribution scheme not targeting specific age groups also leads to fewer cases overall but could lead to more cases among the elderly (a potentially vulnerable population) during the early stage of prophylaxis rollout.

Informing Public Health Policies with Models for Disease Burden, Impact Evaluation, and Economic Evaluation

Conducting real-world public health experiments is often costly, time-consuming, and ethically challenging, so mathematical models have a long-standing history of being used to inform policy. Applications include estimating disease burden, performing economic evaluation of interventions, and responding to health emergencies such as pandemics. Models played a pivotal role during the COVID-19 pandemic, providing early detection of SARS-CoV-2's pandemic potential and informing subsequent public health measures. While models offer valuable policy insights, they often carry limitations, especially when they depend on assumptions and incomplete data. Striking a balance between accuracy and timely decision-making in rapidly evolving situations such as disease outbreaks is challenging. Modelers need to explore the extent to which their models deviate from representing the real world. The uncertainties inherent in models must be effectively communicated to policy makers and the public. As the field becomes increasingly influential, it needs to develop reporting standards that enable rigorous external scrutiny.

Comparison of a static cohort model and dynamic transmission model for respiratory syncytial virus intervention programs for infants in England and Wales

BACKGROUND

A recent study comparing results of multiple cost-effectiveness analyses (CEAs) in a hypothetical population found that monoclonal antibody (mAb) immunoprophylaxis for respiratory syncytial virus (RSV) in infants averted fewer medically attended cases when estimated using dynamic transmission models (DTMs) versus static cohort models (SCMs). We aimed to investigate whether model calibration or parameterization could be the primary driver of inconsistencies between SCM and DTM predictions.

METHODS

A recently published DTM evaluating the CEA of infant mAb immunoprophylaxis in England and Wales (EW) was selected as the reference model. We adapted our previously published SCM for US infants to EW by utilizing the same data sources used by the DTM. Both models parameterized mAb efficacy from a randomized clinical trial (RCT) that estimated an average efficacy of 74.5% against all medically attended RSV episodes and 62.1% against RSV hospitalizations. To align model assumptions, we modified the SCM to incorporate waning efficacy. Since the estimated indirect effects from the DTM were small (i.e., approximately 100-fold smaller in magnitude than direct effects), we hypothesized that alignment of model parameters should result in alignment of model predictions. Outputs for model comparison comprised averted hospitalizations and averted GP visits, estimated for seasonal (S) and seasonal-with-catchup (SC) immunization strategies.

RESULTS

When we aligned the SCM intervention parameters to DTM intervention parameters, significantly more averted hospitalizations were predicted by the SCM (S: 32.3%; SC: 51.3%) than the DTM (S: 17.8%; SC: 28.6%). The SCM most closely replicated the DTM results when the initial efficacy of the mAb intervention was 62.1%, leading to an average efficacy of 39.3%. Under this parameterization the SCM predicted 17.4% (S) and 27.7% (SC) averted hospitalizations. Results were similar for averted GP visits.

CONCLUSIONS

Parameterization of the RSV mAb intervention efficacy is a plausible primary driver of differences between SCM versus DTM model predictions.

A systematic review of mechanistic models used to study avian influenza virus transmission and control

The global spread of avian influenza A viruses in domestic birds is causing increasing socioeconomic devastation. Various mechanistic models have been developed to better understand avian influenza transmission and evaluate the effectiveness of control measures in mitigating the socioeconomic losses caused by these viruses. However, the results of models of avian influenza transmission and control have not yet been subject to a comprehensive review. Such a review could help inform policy makers and guide future modeling work. To help fill this gap, we conducted a systematic review of the mechanistic models that have been applied to field outbreaks. Our three objectives were to: (1) describe the type of models and their epidemiological context, (2) list estimates of commonly used parameters of low pathogenicity and highly pathogenic avian influenza transmission, and (3) review the characteristics of avian influenza transmission and the efficacy of control strategies according to the mechanistic models. We reviewed a total of 46 articles. Of these, 26 articles estimated parameters by fitting the model to data, one evaluated the effectiveness of control strategies, and 19 did both. Values of the between-individual reproduction number ranged widely: from 2.18 to 86 for highly pathogenic avian influenza viruses, and from 4.7 to 45.9 for low pathogenicity avian influenza viruses, depending on epidemiological settings, virus subtypes and host species. Other parameters, such as the durations of the latent and infectious periods, were often taken from the literature, limiting the models' potential insights. Concerning control strategies, many models evaluated culling (n = 15), while vaccination received less attention (n = 6). According to the articles reviewed, optimal control strategies varied between virus subtypes and local conditions, and depended on the overall objective of the intervention. For instance, vaccination was optimal when the objective was to limit the overall number of culled flocks. In contrast, pre-emptive culling was preferred for reducing the size and duration of an epidemic. Early implementation consistently improved the overall efficacy of interventions, highlighting the need for effective surveillance and epidemic preparedness.

A cyclic behavioral modeling aspect to understand the effects of vaccination and treatment on epidemic transmission dynamics

Evolutionary epidemiological models have played an active part in analyzing various contagious diseases and intervention policies in the biological sciences. The design in this effort is the addition of compartments for treatment and vaccination, so the system is designated as susceptible, vaccinated, infected, treated, and recovered (SVITR) epidemic dynamic. The contact of a susceptible individual with a vaccinated or an infected individual makes the individual either immunized or infected. Inventively, the assumption that infected individuals enter the treatment and recover state at different rates after a time interval is also deliberated through the presence of behavioral aspects. The rate of change from susceptible to vaccinated and infected to treatment is studied in a comprehensive evolutionary game theory with a cyclic epidemic model. We theoretically investigate the cyclic SVITR epidemic model framework for disease-free and endemic equilibrium to show stable conditions. Then, the embedded vaccination and treatment strategies are present using extensive evolutionary game theory aspects among the individuals in society through a ridiculous phase diagram. Extensive numerical simulation suggests that effective vaccination and treatment may implicitly reduce the community risk of infection when reliable and cheap. The results exhibited the dilemma and benefitted situation, in which the interplay between vaccination and treatment evolution and coexistence are investigated by the indicators of social efficiency deficit and socially benefited individuals.

Cost-effectiveness of pharmaceutical strategies to prevent respiratory syncytial virus disease in young children: a decision-support model for use in low-income and middle-income countries

BACKGROUND

Respiratory syncytial virus (RSV) is a leading cause of respiratory disease in young children. A number of mathematical models have been used to assess the cost-effectiveness of RSV prevention strategies, but these have not been designed for ease of use by multidisciplinary teams working in low-income and middle-income countries (LMICs).

METHODS

We describe the UNIVAC decision-support model (a proportionate outcomes static cohort model) and its approach to exploring the potential cost-effectiveness of two RSV prevention strategies: a single-dose maternal vaccine and a single-dose long-lasting monoclonal antibody (mAb) for infants. We identified model input parameters for 133 LMICs using evidence from the literature and selected national datasets. We calculated the potential cost-effectiveness of each RSV prevention strategy (compared to nothing and to each other) over the lifetimes of all children born in the year 2025 and compared our results to a separate model published by PATH. We ran sensitivity and scenario analyses to identify the inputs with the largest influence on the cost-effectiveness results.

RESULTS

Our illustrative results assuming base case input assumptions for maternal vaccination ($3.50 per dose, 69% efficacy, 6 months protection) and infant mAb ($3.50 per dose, 77% efficacy, 5 months protection) showed that both interventions were cost-saving compared to status quo in around one-third of 133 LMICs, and had a cost per DALY averted below 0.5 times the national GDP per capita in the remaining LMICs. UNIVAC generated similar results to a separate model published by PATH. Cost-effectiveness results were most sensitive to changes in the price, efficacy and duration of protection of each strategy, and the rate (and cost) of RSV hospital admissions.

CONCLUSIONS

Forthcoming RSV interventions (maternal vaccines and infant mAbs) are worth serious consideration in LMICs, but there is a good deal of uncertainty around several influential inputs, including intervention price, efficacy, and duration of protection. The UNIVAC decision-support model provides a framework for country teams to build consensus on data inputs, explore scenarios, and strengthen the local ownership and policy-relevance of results.

Cost-Effectiveness of Respiratory Syncytial Virus Preventive Interventions in Children: A Model Comparison Study

OBJECTIVES

Model-based cost-effectiveness analyses on maternal vaccine (MV) and monoclonal antibody (mAb) interventions against respiratory syncytial virus (RSV) use context-specific data and produce varied results. Through model comparison, we aim to characterize RSV cost-effectiveness models and examine drivers for their outputs.

METHODS

We compared 3 static and 2 dynamic models using a common input parameter set for a hypothetical birth cohort of 100 000 infants. Year-round and seasonal programs were evaluated for MV and mAb interventions, using available evidence during the study period (eg, phase III MV and phase IIb mAb efficacy).

RESULTS

Three static models estimated comparable medically attended (MA) cases averted versus no intervention (MV, 1019-1073; mAb, 5075-5487), with the year-round MV directly saving ∼€1 million medical and €0.3 million nonmedical costs, while gaining 4 to 5 discounted quality-adjusted life years (QALYs) annually in <1-year-olds, and mAb resulting in €4 million medical and €1.5 million nonmedical cost savings, and 21 to 25 discounted QALYs gained. In contrast, both dynamic models estimated fewer MA cases averted (MV, 402-752; mAb, 3362-4622); one showed an age shift of RSV cases, whereas the other one reported many non-MA symptomatic cases averted, especially by MV (2014). These differences can be explained by model types, assumptions on non-MA burden, and interventions' effectiveness over time.

CONCLUSIONS

Our static and dynamic models produced overall similar hospitalization and death estimates, but also important differences, especially in non-MA cases averted. Despite the small QALY decrement per non-MA case, their larger number makes them influential for the costs per QALY gained of RSV interventions.

Comparison of model predictions of typhoid conjugate vaccine public health impact and cost-effectiveness

Models are useful to inform policy decisions on typhoid conjugate vaccine (TCV) deployment in endemic settings. However, methodological choices can influence model-predicted outcomes. To provide robust estimates for the potential public health impact of TCVs that account for structural model differences, we compared four dynamic and one static mathematical model of typhoid transmission and vaccine impact. All models were fitted to a common dataset of age-specific typhoid fever cases in Kolkata, India. We evaluated three TCV strategies: no vaccination, routine vaccination at 9 months of age, and routine vaccination at 9 months with a one-time catch-up campaign (ages 9 months to 15 years). The primary outcome was the predicted percent reduction in symptomatic typhoid cases over 10 years after vaccine introduction. For three models with economic analyses (Models A-C), we also compared the incremental cost-effectiveness ratios (ICERs), calculated as the incremental cost (US$) per disability-adjusted life-year (DALY) averted. Routine vaccination was predicted to reduce symptomatic cases by 10-46 % over a 10-year time horizon under an optimistic scenario (95 % initial vaccine efficacy and 19-year mean duration of protection), and by 2-16 % under a pessimistic scenario (82 % initial efficacy and 6-year mean protection). Adding a catch-up campaign predicted a reduction in incidence of 36-90 % and 6-35 % in the optimistic and pessimistic scenarios, respectively. Vaccine impact was predicted to decrease as the relative contribution of chronic carriers to transmission increased. Models A-C all predicted routine vaccination with or without a catch-up campaign to be cost-effective compared to no vaccination, with ICERs varying from $95-789 per DALY averted; two models predicted the ICER of routine vaccination alone to be greater than with the addition of catch-up campaign. Despite differences in model-predicted vaccine impact and cost-effectiveness, routine vaccination plus a catch-up campaign is likely to be impactful and cost-effective in high incidence settings such as Kolkata.

Health economic analyses of latent tuberculosis infection screening and preventive treatment among people living with HIV in lower tuberculosis incidence settings: a systematic review

Introduction: In lower tuberculosis (TB) incidence countries (<100 cases/100,000/year), screening and preventive treatment (PT) for latent TB infection (LTBI) among people living with HIV (PLWH) is often recommended, yet guidelines advising which groups to prioritise for screening can be contradictory and implementation patchy. Evidence of LTBI screening cost-effectiveness may improve uptake and health outcomes at reasonable cost. Methods: Our systematic review assessed cost-effectiveness estimates of LTBI screening/PT strategies among PLWH in lower TB incidence countries to identify model-driving inputs and methodological differences. Databases were searched 1980-2020. Studies including health economic evaluation of LTBI screening of PLWH in lower TB incidence countries (<100 cases/100,000/year) were included. Results: Of 2,644 articles screened, nine studies were included. Cost-effectiveness estimates of LTBI screening/PT for PLWH varied widely, with universal screening/PT found highly cost-effective by some studies, while only targeting to high-risk groups (such as those from mid/high TB incidence countries) deemed cost-effective by others. Cost-effectiveness of strategies screening all PLWH from studies published in the past five years varied from US$2828 to US$144,929/quality-adjusted life-year gained (2018 prices). Study quality varied, with inconsistent reporting of methods and results limiting comparability of studies. Cost-effectiveness varied markedly by screening guideline, with British HIV Association guidelines more cost-effective than NICE guidelines in the UK. Discussion: Cost-effectiveness studies of LTBI screening/PT for PLWH in lower TB incidence settings are scarce, with large variations in methods and assumptions used, target populations and screening/PT strategies evaluated. The limited evidence suggests LTBI screening/PT may be cost-effective for some PLWH groups but further research is required, particularly on strategies targeting screening/PT to PLWH at higher risk. Standardisation of model descriptions and results reporting could facilitate reliable comparisons between studies, particularly to identify those factors driving the wide disparity between cost-effectiveness estimates. Registration: PROSPERO CRD42020166338 (18/03/2020).

Vote-processing rules for combining control recommendations from multiple models

Mathematical modelling is used during disease outbreaks to compare control interventions. Using multiple models, the best method to combine model recommendations is unclear. Existing methods weight model projections, then rank control interventions using the combined projections, presuming model outputs are directly comparable. However, the way each model represents the epidemiological system will vary. We apply electoral vote-processing rules to combine model-generated rankings of interventions. Combining rankings of interventions, instead of combining model projections, avoids assuming that projections are comparable as all comparisons of projections are made within each model. We investigate four rules: First-past-the-post, Alternative Vote (AV), Coombs Method and Borda Count. We investigate rule sensitivity by including models that favour only one action or including those that rank interventions randomly. We investigate two case studies: the 2014 Ebola outbreak in West Africa (37 compartmental models) and a hypothetical foot-and-mouth disease outbreak in UK (four individual-based models). The Coombs Method was least susceptible to adding models that favoured a single action, Borda Count and AV were most susceptible to adding models that ranked interventions randomly. Each rule chose the same intervention as when ranking interventions by mean projections, suggesting that combining rankings provides similar recommendations with fewer assumptions about model comparability. This article is part of the theme issue 'Technical challenges of modelling real-life epidemics and examples of overcoming these'.

Contribution and quality of mathematical modeling evidence in World Health Organization guidelines: A systematic review

Mathematical modeling studies are frequently conducted to guide policy in global health. However, the contribution of mathematical modeling studies to World Health Organization (WHO) guideline recommendations, and the quality of evidence contributed by these studies remains unknown. We conducted a systematic review of the WHO Guidelines Review Committee database to identify guideline recommendations that included evidence from mathematical modeling studies since inception of the Guidelines Review Committee on 1 December, 2007. We included WHO guideline recommendations citing a mathematical modeling study in the primary evidence base. We defined a mathematical model as a framework that predicted epidemiologic, health or economic impact of an intervention or decision in the clinical or public health context. The primary outcome was inclusion of evidence from mathematical modeling studies in a guideline recommendation. We evaluated each unique modeling study across multiple domains of quality. Between 1 December 2007 and 1 April 2019, the WHO Guidelines Review Committee approved 154 guidelines providing 1619 guideline recommendations. Mathematical modeling studies informed 46 WHO guidelines (29.9%) and 101 unique guideline recommendations (6.2%). Modeling evidence addressed topics related to infectious diseases in 38 guidelines (82.6%) and 81 recommendations (80.2%), most commonly for HIV and tuberculosis. Evidence from modeling studies was assessed in the GRADE evidence profile for 12 recommendations (12.9%) and GRADE evidence-to-decision framework for 45 recommendations (44.6%). Modeling-informed recommendations were more likely than other recommendations within the same guidelines to be issued with a "conditional" rather than "strong" strength of recommendation (53.5% versus 37.8%), and the evidence underlying modeling-informed recommendations was more likely to be assessed as very low quality (41.6% versus 24.1%). Upon review of individual modeling studies, we estimated that 33.8% of models performed a calibration, 29.4% of models performed a validation of results, and 20.6% of models reported a change in the study conclusion in the sensitivity analysis. While policy recommendations in WHO guidelines are informed by evidence from modeling studies, the validity of modeling studies included in guidelines development is heterogeneous. Quality assessment is needed to support the evaluation and incorporation of evidence from mathematical modeling studies in guidelines development.

A review of epidemiological models of Clostridioides difficile transmission and control (2009-2021)

Clostridioides difficile is the leading cause of infectious diarrhea and one of the most common healthcare-acquired infections worldwide. We performed a systematic search and a bibliometric analysis of mathematical and computational models for Clostridioides difficile transmission. We identified 33 publications from 2009 to 2021. Models have underscored the importance of asymptomatic colonized patients in maintaining transmission in health-care settings. Infection control, antimicrobial stewardship, active testing, and vaccination have often been evaluated in models. Despite active testing and vaccination being not currently implemented, they are the most commonly evaluated interventions. Some aspects of C. difficile transmission, such community transmission and interventions in health-care settings other than in acute-care hospitals, remained less evaluated through modeling.

Projections of human papillomavirus (HPV) vaccination impact in Ethiopia, India, Nigeria and Pakistan: a comparative modelling study

Introduction

Cervical cancer is the second most common cancer among women in Ethiopia, India, Nigeria and Pakistan. Our study objective was to assess similarities and differences in vaccine-impact projections through comparative modelling analysis by independently estimating the potential health impact of human papillomavirus (HPV) vaccination.

Methods

Using two widely published models (Harvard and Papillomavirus Rapid Interface for Modelling and Economics (PRIME)) to estimate HPV vaccination impact, we simulated a vaccination scenario of 90% annual coverage among 10 cohorts of 9-year-old girls from 2021 to 2030 in Ethiopia, India, Nigeria and Pakistan. We estimated potential health impact in terms of cervical cancer cases, deaths and disability-adjusted life years averted among vaccinated cohorts from the time of vaccination until 2100. We harmonised the two models by standardising input data to comparatively estimate HPV vaccination impact.

Results

Prior to harmonising model assumptions, the range between PRIME and Harvard models for number of cervical cancer cases averted by HPV vaccination was: 262 000 to 2 70 000 in Ethiopia; 1 640 000 to 1 970 000 in India; 330 000 to 3 36 000 in Nigeria and 111 000 to 1 33 000 in Pakistan. When harmonising model assumptions, alignment on HPV type distribution significantly narrowed differences in vaccine-impact estimates.

Conclusion

Despite model differences, the Harvard and PRIME models yielded similar vaccine-impact estimates. The main differences in estimates are due to variation in interpretation around data on cervical cancer attribution to HPV-16/18. As countries make progress towards WHO targets for cervical cancer elimination, continued explorations of underlying differences in model inputs, assumptions and results when examining cervical cancer prevention policy will be critical.

Generating, Presenting, and Interpreting Cost-Effectiveness Results in the Context of Uncertainty: A Tutorial for Deeper Knowledge and Better Practice

This tutorial aims to help make the best available methods for generating and presenting cost-effectiveness results with uncertainty common practice. We believe there is a need for such type of tutorial because some erroneous practices persist (e.g., identifying the cost-effective intervention as the one with the highest probability to be cost-effective), while some of the more advanced methods are hardly used (e.g., the net loss statistic ‘NL’, expected net loss curves and frontier). The tutorial explains with simple examples the pros and cons of using ICER, incremental net benefit and NL to identify the cost-effective intervention, both with and without uncertainty accounted for probabilistically. A flowchart provides practical guidance on when and how to use ICER, incremental net benefit or NL. Different ways to express and present uncertainty in the results are described, including confidence and credible intervals, the probability that a strategy is cost-effective (as usually shown with cost-effectiveness acceptability curves (CEACs)) and the expected value of perfect information (EVPI). The tutorial clarifies and illustrates why EVPI is the only measure accounting fully for decision uncertainty, and why NL curves and the NL frontier may be preferred over CEACs and other plots for presenting cost-effectiveness results in the context of uncertainty. The easy calculations and a worked-out real-life example will help users to thoroughly understand and correctly interpret key cost-effectiveness results. Examples with mathematical calculations, interpretation, plots and R code are provided.

Probability bound analysis: A novel approach for quantifying parameter uncertainty in decision-analytic modeling and cost-effectiveness analysis

Decisions about health interventions are often made using limited evidence. Mathematical models used to inform such decisions often include uncertainty analysis to account for the effect of uncertainty in the current evidence base on decision-relevant quantities. However, current uncertainty quantification methodologies, including probabilistic sensitivity analysis (PSA), require modelers to specify a precise probability distribution to represent the uncertainty of a model parameter. This study introduces a novel approach for representing and propagating parameter uncertainty, probability bounds analysis (PBA), where the uncertainty about the unknown probability distribution of a model parameter is expressed in terms of an interval bounded by lower and upper bounds on the unknown cumulative distribution function (p-box) and without assuming a particular form of the distribution function. We give the formulas of the p-boxes for common situations (given combinations of data on minimum, maximum, median, mean, or standard deviation), describe an approach to propagate p-boxes into a black-box mathematical model, and introduce an approach for decision-making based on the results of PBA. We demonstrate the characteristics and utility of PBA vs PSA using two case studies. In sum, this study provides modelers with practical tools to conduct parameter uncertainty quantification given the constraints of available data and with the fewest assumptions.

An approach for improving the quality of country-level TB modelling

Mathematical modelling is increasingly used to inform budgeting and strategic decision-making by national TB programmes. Despite the importance of these decisions, there is currently no mechanism to review and confirm the appropriateness of modelling analyses. We have developed a benchmarking, reporting, and review (BRR) approach and accompanying tools to allow constructive review of country-level TB modelling applications. This approach has been piloted in five modelling applications and the results of this study have been used to revise and finalise the approach. The BRR approach consists of 1) quantitative benchmarks against which model assumptions and results can be compared, 2) standardised reporting templates and review criteria, and 3) a multi-stage review process providing feedback to modellers during the application, as well as a summary evaluation after completion. During the pilot, use of the tools prompted important changes in the approaches taken to modelling. The pilot also identified issues beyond the scope of a review mechanism, such as a lack of empirical evidence and capacity constraints. This approach provides independent evaluation of the appropriateness of modelling decisions during the course of an application, allowing meaningful changes to be made before results are used to inform decision-making. The use of these tools can improve the quality and transparency of country-level TB modelling applications.

A data-driven metapopulation model for the Belgian COVID-19 epidemic: assessing the impact of lockdown and exit strategies

BACKGROUND

In response to the ongoing COVID-19 pandemic, several countries adopted measures of social distancing to a different degree. For many countries, after successfully curbing the initial wave, lockdown measures were gradually lifted. In Belgium, such relief started on May 4th with phase 1, followed by several subsequent phases over the next few weeks.

METHODS

We analysed the expected impact of relaxing stringent lockdown measures taken according to the phased Belgian exit strategy. We developed a stochastic, data-informed, meta-population model that accounts for mixing and mobility of the age-structured population of Belgium. The model is calibrated to daily hospitalization data and is able to reproduce the outbreak at the national level. We consider different scenarios for relieving the lockdown, quantified in terms of relative reductions in pre-pandemic social mixing and mobility. We validate our assumptions by making comparisons with social contact data collected during and after the lockdown.

RESULTS

Our model is able to successfully describe the initial wave of COVID-19 in Belgium and identifies interactions during leisure/other activities as pivotal in the exit strategy. Indeed, we find a smaller impact of school re-openings as compared to restarting leisure activities and re-openings of work places. We also assess the impact of case isolation of new (suspected) infections, and find that it allows re-establishing relatively more social interactions while still ensuring epidemic control. Scenarios predicting a second wave of hospitalizations were not observed, suggesting that the per-contact probability of infection has changed with respect to the pre-lockdown period.

CONCLUSIONS

Contacts during leisure activities are found to be most influential, followed by professional contacts and school contacts, respectively, for an impending second wave of COVID-19. Regular re-assessment of social contacts in the population is therefore crucial to adjust to evolving behavioral changes that can affect epidemic diffusion.

Assessing fitness-for-purpose and comparing the suitability of COVID-19 multi-country models for local contexts and users

Background: Mathematical models have been used throughout the COVID-19 pandemic to inform policymaking decisions. The COVID-19 Multi-Model Comparison Collaboration (CMCC) was established to provide country governments, particularly low- and middle-income countries (LMICs), and other model users with an overview of the aims, capabilities and limits of the main multi-country COVID-19 models to optimise their usefulness in the COVID-19 response. Methods: Seven models were identified that satisfied the inclusion criteria for the model comparison and had creators that were willing to participate in this analysis. A questionnaire, extraction tables and interview structure were developed to be used for each model, these tools had the aim of capturing the model characteristics deemed of greatest importance based on discussions with the Policy Group. The questionnaires were first completed by the CMCC Technical group using publicly available information, before further clarification and verification was obtained during interviews with the model developers. The fitness-for-purpose flow chart for assessing the appropriateness for use of different COVID-19 models was developed jointly by the CMCC Technical Group and Policy Group. Results: A flow chart of key questions to assess the fitness-for-purpose of commonly used COVID-19 epidemiological models was developed, with focus placed on their use in LMICs. Furthermore, each model was summarised with a description of the main characteristics, as well as the level of engagement and expertise required to use or adapt these models to LMIC settings. Conclusions: This work formalises a process for engagement with models, which is often done on an ad-hoc basis, with recommendations for both policymakers and model developers and should improve modelling use in policy decision making.

The Use and Misuse of Mathematical Modeling for Infectious Disease Policymaking: Lessons for the COVID-19 Pandemic

Mathematical modeling has played a prominent and necessary role in the current coronavirus disease 2019 (COVID-19) pandemic, with an increasing number of models being developed to track and project the spread of the disease, as well as major decisions being made based on the results of these studies. A proliferation of models, often diverging widely in their projections, has been accompanied by criticism of the validity of modeled analyses and uncertainty as to when and to what extent results can be trusted. Drawing on examples from COVID-19 and other infectious diseases of global importance, we review key limitations of mathematical modeling as a tool for interpreting empirical data and informing individual and public decision making. We present several approaches that have been used to strengthen the validity of inferences drawn from these analyses, approaches that will enable better decision making in the current COVID-19 crisis and beyond.

Partnership dynamics in mathematical models and implications for representation of sexually transmitted infections: a review

Mathematical models of sexually transmitted disease (STI) are increasingly relied on to inform policy, practice, and resource allocation. Because STI transmission requires sexual contact between two or more people, a model's ability to represent the dynamics of sexual partnerships can influence the validity of findings. This ability is to a large extent constrained by the model type, as different modeling frameworks vary in their capability to capture patterns of sexual contact at individual, partnership, and network levels. In this paper, we classify models into three groups: compartmental, individual-based, and statistical network models. For each framework, we describe the basic model structure and discuss key aspects of sexual partnership dynamics: how and with whom partnerships are formed, partnership duration and dissolution, and temporal overlap in partnerships (concurrency). We illustrate the potential implications of accurately accounting for partnership dynamics, but these effects depend on characteristics of both the population and pathogen; the combined impact of these partnership and epidemiologic dynamics can be difficult to predict. While each of the reviewed model frameworks may be appropriate to inform certain research or policy questions, modelers and consumers of models should carefully consider the implications of sexual partnership dynamics for the questions under study.

A participatory modelling approach for investigating the spread of COVID-19 in countries of the Eastern Mediterranean Region to support public health decision-making

Early on in the COVID-19 pandemic, the WHO Eastern Mediterranean Regional Office recognised the importance of epidemiological modelling to forecast the progression of the COVID-19 pandemic to support decisions guiding the implementation of response measures. We established a modelling support team to facilitate the application of epidemiological modelling analyses in the Eastern Mediterranean Region (EMR) countries. Here, we present an innovative, stepwise approach to participatory modelling of the COVID-19 pandemic that engaged decision-makers and public health professionals from countries throughout all stages of the modelling process. Our approach consisted of first identifying the relevant policy questions, collecting country-specific data and interpreting model findings from a decision-maker's perspective, as well as communicating model uncertainty. We used a simple modelling methodology that was adaptable to the shortage of epidemiological data, and the limited modelling capacity, in our region. We discuss the benefits of using models to produce rapid decision-making guidance for COVID-19 control in the WHO EMR, as well as challenges that we have experienced regarding conveying uncertainty associated with model results, synthesising and comparing results across multiple modelling approaches, and modelling fragile and conflict-affected states.

When, Who, and How to Sample: Designing Practical Surveillance for 7 Neglected Tropical Diseases as We Approach Elimination

As neglected tropical disease programs look to consolidate the successes of moving towards elimination, we need to understand the dynamics of transmission at low prevalence to inform surveillance strategies for detecting elimination and resurgence. In this special collection, modelling insights are used to highlight drivers of local elimination, evaluate strategies for detecting resurgence, and show the importance of rational spatial sampling schemes for several neglected tropical diseases (specifically schistosomiasis, soil-transmitted helminths, lymphatic filariasis, trachoma, onchocerciasis, visceral leishmaniasis, and gambiense sleeping sickness).

Rational policymaking during a pandemic

Policymaking during a pandemic can be extremely challenging. As COVID-19 is a new disease and its global impacts are unprecedented, decisions are taken in a highly uncertain, complex, and rapidly changing environment. In such a context, in which human lives and the economy are at stake, we argue that using ideas and constructs from modern decision theory, even informally, will make policymaking a more responsible and transparent process.

Development and dissemination of infectious disease dynamic transmission models during the COVID-19 pandemic: what can we learn from other pathogens and how can we move forward?

The current COVID-19 pandemic has resulted in the unprecedented development and integration of infectious disease dynamic transmission models into policy making and public health practice. Models offer a systematic way to investigate transmission dynamics and produce short-term and long-term predictions that explicitly integrate assumptions about biological, behavioural, and epidemiological processes that affect disease transmission, burden, and surveillance. Models have been valuable tools during the COVID-19 pandemic and other infectious disease outbreaks, able to generate possible trajectories of disease burden, evaluate the effectiveness of intervention strategies, and estimate key transmission variables. Particularly given the rapid pace of model development, evaluation, and integration with decision making in emergency situations, it is necessary to understand the benefits and pitfalls of transmission models. We review and highlight key aspects of the history of infectious disease dynamic models, the role of rigorous testing and evaluation, the integration with data, and the successful application of models to guide public health. Rather than being an expansive history of infectious disease models, this Review focuses on how the integration of modelling can continue to be advanced through policy and practice in appropriate and conscientious ways to support the current pandemic response.

Modeling Infectious Diseases in Healthcare Network (MInD-Healthcare) Framework for Describing and Reporting Multidrug-resistant Organism and Healthcare-Associated Infections Agent-based Modeling Methods

Mathematical modeling of healthcare-associated infections and multidrug-resistant organisms improves our understanding of pathogen transmission dynamics and provides a framework for evaluating prevention strategies. One way of improving the communication among modelers is by providing a standardized way of describing and reporting models, thereby instilling confidence in the reproducibility and generalizability of such models. We updated the Overview, Design concepts, and Details protocol developed by Grimm et al [11] for describing agent-based models (ABMs) to better align with elements commonly included in healthcare-related ABMs. The Modeling Infectious Diseases in Healthcare Network (MInD-Healthcare) framework includes the following 9 key elements: (1) Purpose and scope; (2) Entities, state variables, and scales; (3) Initialization; (4) Process overview and scheduling; (5) Input data; (6) Agent interactions and organism transmission; (7) Stochasticity; (8) Submodels; and (9) Model verification, calibration, and validation. Our objective is that this framework will improve the quality of evidence generated utilizing these models.

Responsible modelling: Unit testing for infectious disease epidemiology

Infectious disease epidemiology is increasingly reliant on large-scale computation and inference. Models have guided health policy for epidemics including COVID-19 and Ebola and endemic diseases including malaria and tuberculosis. Yet a coding bug may bias results, yielding incorrect conclusions and actions causing avoidable harm. We are ethically obliged to make our code as free of error as possible. Unit testing is a coding method to avoid such bugs, but it is rarely used in epidemiology. We demonstrate how unit testing can handle the particular quirks of infectious disease models and aim to increase the uptake of this methodology in our field.

COVID-19 reopening strategies at the county level in the face of uncertainty: Multiple Models for Outbreak Decision Support

Policymakers make decisions about COVID-19 management in the face of considerable uncertainty. We convened multiple modeling teams to evaluate reopening strategies for a mid-sized county in the United States, in a novel process designed to fully express scientific uncertainty while reducing linguistic uncertainty and cognitive biases. For the scenarios considered, the consensus from 17 distinct models was that a second outbreak will occur within 6 months of reopening, unless schools and non-essential workplaces remain closed. Up to half the population could be infected with full workplace reopening; non-essential business closures reduced median cumulative infections by 82%. Intermediate reopening interventions identified no win-win situations; there was a trade-off between public health outcomes and duration of workplace closures. Aggregate results captured twice the uncertainty of individual models, providing a more complete expression of risk for decision-making purposes.

The impact of maternal RSV vaccine to protect infants in Gavi-supported countries: Estimates from two models

•

First study examining potential impact of a maternal RSV vaccine across LMICs.

•

Results from independent models to inform Gavi’s 2018 Vaccine Investment Strategy.

•

Experts and stakeholders advised on methods, assumptions, and key model inputs.

•

Substantial potential to reduce infant morbidity and mortality in Gavi countries.

Background

Interventions to protect young infants against respiratory syncytial virus (RSV) are in advanced phases of development and are expected to be available in the foreseeable future. Gavi, the Vaccine Alliance, included maternal vaccines and infant monoclonal antibodies for RSV as part of the 2018 vaccine investment strategy (VIS) and decided to support these products subject to licensure, World Health Organization prequalification, Strategic Advisory Group of Experts recommendation, and meeting the financial assumptions used as the basis of the investment case. Impact estimates reported in this manuscript were used to inform the Gavi VIS.

Methods

We compared two independent vaccine impact models to evaluate a potential maternal RSV vaccine’s impact on infant health in 73 Gavi-supported countries. Key inputs were harmonized across both models. We analyzed various scenarios to evaluate the effect of uncertain model parameters such as vaccine efficacy, duration of infant protection, and infant disease burden. Estimates of averted cases, severe cases, hospitalizations, deaths, and disability-adjusted life years (DALYs) were calculated over the 2023–2035 horizon.

Findings

A maternal RSV vaccine with 60% efficacy offering 5 months of infant protection implemented across 73 low- and middle-income countries could avert 10.1–12.5 million cases, 2.8–4.0 million hospitalizations, 123.7–177.7 thousand deaths, and 8.5–11.9 million DALYs among infants under 6 months of age for the duration of analysis (2023–2035). Maternal RSV vaccination was projected to avert up to 42% of estimated RSV deaths among infants under 6 months in year 2035. Alternative scenario analyses with higher disease burden assumptions showed that a maternal vaccine could avert as many as 325–355 thousand deaths among infants under 6 months.

Interpretation

RSV maternal immunization is projected to substantially reduce mortality and morbidity among young infants if introduced across Gavi-supported countries.

Funding

This work was supported by Bill & Melinda Gates Foundation, Seattle, WA, and Respiratory Syncytial Virus Consortium in Europe. The findings and conclusions contained within are those of the authors and do not necessarily reflect positions or policies of the Bill & Melinda Gates Foundation or of the Respiratory Syncytial Virus Consortium. LW is supported by Research Foundation–Flanders (1234620 N).

Impact of HPV vaccination and cervical screening on cervical cancer elimination: a comparative modelling analysis in 78 low-income and lower-middle-income countries

BACKGROUND

The WHO Director-General has issued a call for action to eliminate cervical cancer as a public health problem. To help inform global efforts, we modelled potential human papillomavirus (HPV) vaccination and cervical screening scenarios in low-income and lower-middle-income countries (LMICs) to examine the feasibility and timing of elimination at different thresholds, and to estimate the number of cervical cancer cases averted on the path to elimination.

METHODS

The WHO Cervical Cancer Elimination Modelling Consortium (CCEMC), which consists of three independent transmission-dynamic models identified by WHO according to predefined criteria, projected reductions in cervical cancer incidence over time in 78 LMICs for three standardised base-case scenarios: girls-only vaccination; girls-only vaccination and once-lifetime screening; and girls-only vaccination and twice-lifetime screening. Girls were vaccinated at age 9 years (with a catch-up to age 14 years), assuming 90% coverage and 100% lifetime protection against HPV types 16, 18, 31, 33, 45, 52, and 58. Cervical screening involved HPV testing once or twice per lifetime at ages 35 years and 45 years, with uptake increasing from 45% (2023) to 90% (2045 onwards). The elimination thresholds examined were an average age-standardised cervical cancer incidence of four or fewer cases per 100 000 women-years and ten or fewer cases per 100 000 women-years, and an 85% or greater reduction in incidence. Sensitivity analyses were done, varying vaccination and screening strategies and assumptions. We summarised results using the median (range) of model predictions.

FINDINGS

Girls-only HPV vaccination was predicted to reduce the median age-standardised cervical cancer incidence in LMICs from 19·8 (range 19·4-19·8) to 2·1 (2·0-2·6) cases per 100 000 women-years over the next century (89·4% [86·2-90·1] reduction), and to avert 61·0 million (60·5-63·0) cases during this period. Adding twice-lifetime screening reduced the incidence to 0·7 (0·6-1·6) cases per 100 000 women-years (96·7% [91·3-96·7] reduction) and averted an extra 12·1 million (9·5-13·7) cases. Girls-only vaccination was predicted to result in elimination in 60% (58-65) of LMICs based on the threshold of four or fewer cases per 100 000 women-years, in 99% (89-100) of LMICs based on the threshold of ten or fewer cases per 100 000 women-years, and in 87% (37-99) of LMICs based on the 85% or greater reduction threshold. When adding twice-lifetime screening, 100% (71-100) of LMICs reached elimination for all three thresholds. In regions in which all countries can achieve cervical cancer elimination with girls-only vaccination, elimination could occur between 2059 and 2102, depending on the threshold and region. Introducing twice-lifetime screening accelerated elimination by 11-31 years. Long-term vaccine protection was required for elimination.

INTERPRETATION

Predictions were consistent across our three models and suggest that high HPV vaccination coverage of girls can lead to cervical cancer elimination in most LMICs by the end of the century. Screening with high uptake will expedite reductions and will be necessary to eliminate cervical cancer in countries with the highest burden.

FUNDING

WHO, UNDP, UN Population Fund, UNICEF-WHO-World Bank Special Program of Research, Development and Research Training in Human Reproduction, Canadian Institute of Health Research, Fonds de recherche du Québec-Santé, Compute Canada, National Health and Medical Research Council Australia Centre for Research Excellence in Cervical Cancer Control.