Active learning to understand infectious disease models and improve policy making

How can machine learning predict cholera: insights from experiments and design science for action research

Cholera is a leading cause of mortality in Nigeria. The two most significant predictors of cholera are a lack of access to clean water and poor sanitary conditions. Other factors such as natural disasters, illiteracy, and internal conflicts that drive people to seek sanctuary in refugee camps may contribute to the spread of cholera in Nigeria. The aim of this research is to develop a cholera outbreak risk prediction (CORP) model using machine learning tools and data science. In this study, we developed a CORP model using design science perspectives and machine learning to detect cholera outbreaks in Nigeria. Nonnegative matrix factorization (NMF) was used for dimensionality reduction, and synthetic minority oversampling technique (SMOTE) was used for data balancing. Outliers were detected using density-based spatial clustering of applications with noise (DBSCAN) were removed improving the overall performance of the model, and the extreme-gradient boost algorithm was used for prediction. The findings revealed that the CORP model outcomes resulted in the best accuracy of 99.62%, Matthews's correlation coefficient of 0.976, and area under the curve of 99.2%, which were improved compared with the previous findings. The developed model can be helpful to healthcare providers in predicting possible cholera outbreaks.

Modelling the impact of combining HIV prevention interventions on HIV dynamics in fishing communities in Uganda

BACKGROUND

In countries with mature generalized HIV epidemics such as Uganda, there are still groups of individuals that are disproportionately affected. Among the key populations in Uganda are fishing communities, which make up about 10% of the population. Compared to the general population, HIV prevalence and incidence among individuals living in these communities is high. This high HIV burden has been attributed to several factors including limited access to prevention and treatment services as well as ongoing high-risk sexual behaviour.

METHODS

We investigated the impact of combined HIV prevention interventions on HIV transmission dynamics in high-risk fishing communities in Uganda using a deterministic compartmental model. The model was calibrated to seroprevalence data from a census performed in 2014. To account for remaining uncertainty in the calibrated model parameters, 50 000 simulated scenarios were modelled to investigate the impact of combined prevention interventions.

RESULTS

The projected HIV incidence decreased from 1.87 per 100 PY without intervention scale-up to 0.25 per 100 PY after 15 years (2014-2029) of intervention scale-up. A potential combination achieving this 87% reduction in incidence over 15 years in Ugandan FCs included condom use in about 60% of sexual acts, 23% of susceptible men circumcised, 87% of people living with HIV aware of their status, 75% of those on ART, and about 3% of susceptible individuals on oral PrEP. Uncertainty analysis revealed relative reductions in incidence ranging from 30.9 to 86.8%. Sensitivity analyses suggested that condom use and early ART were the most important interventions.

CONCLUSION

Reducing HIV incidence, as well as prevalence and AIDS-related mortality, in these high-risk fishing communities in Uganda is attainable over 15 years with a combination prevention package. Our projected intervention coverage levels are well within the national targets set by the Uganda government and enable coming close to reaching the UNAIDS 95-95-95 targets to end AIDS as a public health threat by 2030.

Individual Factors Associated With COVID-19 Infection: A Machine Learning Study

The fast, exponential increase of COVID-19 infections and their catastrophic effects on patients' health have required the development of tools that support health systems in the quick and efficient diagnosis and prognosis of this disease. In this context, the present study aims to identify the potential factors associated with COVID-19 infections, applying machine learning techniques, particularly random forest, chi-squared, xgboost, and rpart for feature selection; ROSE and SMOTE were used as resampling methods due to the existence of class imbalance. Similarly, machine and deep learning algorithms such as support vector machines, C4.5, random forest, rpart, and deep neural networks were explored during the train/test phase to select the best prediction model. The dataset used in this study contains clinical data, anthropometric measurements, and other health parameters related to smoking habits, alcohol consumption, quality of sleep, physical activity, and health status during confinement due to the pandemic associated with COVID-19. The results showed that the XGBoost model got the best features associated with COVID-19 infection, and random forest approximated the best predictive model with a balanced accuracy of 90.41% using SMOTE as a resampling technique. The model with the best performance provides a tool to help prevent contracting SARS-CoV-2 since the variables with the highest risk factor are detected, and some of them are, to a certain extent controllable.

Influence of sexual risk behaviour and STI co-infection dynamics on the evolution of HIV set point viral load in MSM

HIV viral load (VL) is an important predictor of HIV progression and transmission. Anti-retroviral therapy (ART) has been reported to reduce HIV transmission by lowering VL. However, apart from this beneficial effect, increased levels of population mean set-point viral load (SPVL), an estimator for HIV virulence, have been observed in men who have sex with men (MSM) in the decade following the introduction of ART in The Netherlands. Several studies have been devoted to explain these counter-intuitive trends in SPVL. However, to our knowledge, none of these studies has investigated an explanation in which it arises as the result of a sexually transmitted infection (STI) co-factor in detail. In this study, we adapted an event-based, individual-based model to investigate how STI co-infection and sexual risk behaviour affect the evolution of HIV SPVL in MSM before and after the introduction of ART. The results suggest that sexual risk behaviour has an effect on SPVL and indicate that more data are needed to test the effect of STI co-factors on SPVL. Furthermore, the observed trends in SPVL cannot be explained by sexual risk behaviour and STI co-factors only. We recommend to develop mathematical models including also factors related to viral evolution as reported earlier in the literature. However, this requires more complex models, and the collection of more data for parameter estimation than what is currently available.

Choosing a Metamodel of a Simulation Model for Uncertainty Quantification

BACKGROUND

Metamodeling may substantially reduce the computational expense of individual-level state transition simulation models (IL-STM) for calibration, uncertainty quantification, and health policy evaluation. However, because of the lack of guidance and readily available computer code, metamodels are still not widely used in health economics and public health. In this study, we provide guidance on how to choose a metamodel for uncertainty quantification.

METHODS

We built a simulation study to evaluate the prediction accuracy and computational expense of metamodels for uncertainty quantification using life-years gained (LYG) by treatment as the IL-STM outcome. We analyzed how metamodel accuracy changes with the characteristics of the simulation model using a linear model (LM), Gaussian process regression (GP), generalized additive models (GAMs), and artificial neural networks (ANNs). Finally, we tested these metamodels in a case study consisting of a probabilistic analysis of a lung cancer IL-STM.

RESULTS

In a scenario with low uncertainty in model parameters (i.e., small confidence interval), sufficient numbers of simulated life histories, and simulation model runs, commonly used metamodels (LM, ANNs, GAMs, and GP) have similar, good accuracy, with errors smaller than 1% for predicting LYG. With a higher level of uncertainty in model parameters, the prediction accuracy of GP and ANN is superior to LM. In the case study, we found that in the worst case, the best metamodel had an error of about 2.1%.

CONCLUSION

To obtain good prediction accuracy, in an efficient way, we recommend starting with LM, and if the resulting accuracy is insufficient, we recommend trying ANNs and eventually also GP regression.

Workplace influenza vaccination to reduce employee absenteeism: An economic analysis from the employers' perspective

BACKGROUND

Each year, up to 10% of unvaccinated adults contracts seasonal influenza, with half of this proportion developing symptoms. As a result, employers experience significant economic losses in terms of employee absenteeism. Influenza vaccines can be instrumental in reducing this burden. Workplace vaccination is expected to reduce employee absenteeism more than linearly as a result of positive externalities. It remains unclear whether workplace influenza vaccination yields a positive return on investment.

METHODS

We simulated the spread of influenza in the seasons 2011-12 up to 2017-18 in Belgium by means of a compartmental transmission model. We accounted for age-specific social contact patterns and included reduced contact behavior when symptomatically infected. We simulated the impact of employer-funded influenza vaccination at the workplace and performed a cost-benefit analysis to assess the employers' return on workplace vaccination. Furthermore, we look into the cost-benefit of rewarding vaccinated employees by offering an additional day off.

RESULTS

Workplace vaccination reduced the burden of influenza both on the workplace and in the population at large. Compared to the current vaccine coverage - 21% in the population at large - an employee vaccine coverage of 90% could avert an additional 355 000 cases, of which about 150 000 in the employed population and 205 000 in the unemployed population. While seasonal influenza vaccination has been cost-saving on average at about €10 per vaccinated employee, the cost-benefit analysis was prone to between-season variability.

CONCLUSIONS

Vaccinated employees can serve as a barrier to limit the spread of influenza in the population, reducing the attack rate by 78% at an employee coverage of 90%. While workplace vaccination is relatively inexpensive (due to economies of scale) and convenient, the return on investment is volatile. Government subsidies can be pivotal to encourage employers to provide vaccination at the workplace with positive externalities to society as a whole.

Introduction to Metamodeling for Reducing Computational Burden of Advanced Analyses with Health Economic Models: A Structured Overview of Metamodeling Methods in a 6-Step Application Process

Metamodels can be used to reduce the computational burden associated with computationally demanding analyses of simulation models, although applications within health economics are still scarce. Besides a lack of awareness of their potential within health economics, the absence of guidance on the conceivably complex and time-consuming process of developing and validating metamodels may contribute to their limited uptake. To address these issues, this article introduces metamodeling to the wider health economic audience and presents a process for applying metamodeling in this context, including suitable methods and directions for their selection and use. General (i.e., non-health economic specific) metamodeling literature, clinical prediction modeling literature, and a previously published literature review were exploited to consolidate a process and to identify candidate metamodeling methods. Methods were considered applicable to health economics if they are able to account for mixed (i.e., continuous and discrete) input parameters and continuous outcomes. Six steps were identified as relevant for applying metamodeling methods within health economics: 1) the identification of a suitable metamodeling technique, 2) simulation of data sets according to a design of experiments, 3) fitting of the metamodel, 4) assessment of metamodel performance, 5) conducting the required analysis using the metamodel, and 6) verification of the results. Different methods are discussed to support each step, including their characteristics, directions for use, key references, and relevant R and Python packages. To address challenges regarding metamodeling methods selection, a first guide was developed toward using metamodels to reduce the computational burden of analyses of health economic models. This guidance may increase applications of metamodeling in health economics, enabling increased use of state-of-the-art analyses (e.g., value of information analysis) with computationally burdensome simulation models.

The use of mixture density networks in the emulation of complex epidemiological individual-based models

Complex, highly-computational, individual-based models are abundant in epidemiology. For epidemics such as macro-parasitic diseases, detailed modelling of human behaviour and pathogen life-cycle are required in order to produce accurate results. This can often lead to models that are computationally-expensive to analyse and perform model fitting, and often require many simulation runs in order to build up sufficient statistics. Emulation can provide a more computationally-efficient output of the individual-based model, by approximating it using a statistical model. Previous work has used Gaussian processes (GPs) in order to achieve this, but these can not deal with multi-modal, heavy-tailed, or discrete distributions. Here, we introduce the concept of a mixture density network (MDN) in its application in the emulation of epidemiological models. MDNs incorporate both a mixture model and a neural network to provide a flexible tool for emulating a variety of models and outputs. We develop an MDN emulation methodology and demonstrate its use on a number of simple models incorporating both normal, gamma and beta distribution outputs. We then explore its use on the stochastic SIR model to predict the final size distribution and infection dynamics. MDNs have the potential to faithfully reproduce multiple outputs of an individual-based model and allow for rapid analysis from a range of users. As such, an open-access library of the method has been released alongside this manuscript.

Infectious disease modellers have a growing need to expose their models to a variety of stakeholders in interactive, engaging ways that allow them to explore different scenarios. This approach can come with a considerable computational cost that motivates providing a simpler representation of the complex model. We propose the use of mixture density networks as a solution to this problem. MDNs are highly flexible, deep neural network-based models that can emulate a variety of data, including counts and over-dispersion. We explore their use firstly through emulating a negative binomial distribution, which arises in many places in ecology and parasite epidemiology. Then, we explore the approach using a stochastic SIR model. We also provide an accompanying Python library with code for all examples given in the manuscript. We believe that the use of emulation will provide a method to package an infectious disease model such that it can be disseminated to the widest audience possible.

Lessons from a decade of individual-based models for infectious disease transmission: a systematic review (2006-2015)

Background

Individual-based models (IBMs) are useful to simulate events subject to stochasticity and/or heterogeneity, and have become well established to model the potential (re)emergence of pathogens (e.g., pandemic influenza, bioterrorism). Individual heterogeneity at the host and pathogen level is increasingly documented to influence transmission of endemic diseases and it is well understood that the final stages of elimination strategies for vaccine-preventable childhood diseases (e.g., polio, measles) are subject to stochasticity. Even so it appears IBMs for both these phenomena are not well established. We review a decade of IBM publications aiming to obtain insights in their advantages, pitfalls and rationale for use and to make recommendations facilitating knowledge transfer within and across disciplines.

Methods

We systematically identified publications in Web of Science and PubMed from 2006-2015 based on title/abstract/keywords screening (and full-text if necessary) to retrieve topics, modeling purposes and general specifications. We extracted detailed modeling features from papers on established vaccine-preventable childhood diseases based on full-text screening.

Results

We identified 698 papers, which applied an IBM for infectious disease transmission, and listed these in a reference database, describing their general characteristics. The diversity of disease-topics and overall publication frequency have increased over time (38 to 115 annual publications from 2006 to 2015). The inclusion of intervention strategies (8 to 52) and economic consequences (1 to 20) are increasing, to the detriment of purely theoretical explorations. Unfortunately, terminology used to describe IBMs is inconsistent and ambiguous. We retrieved 24 studies on a vaccine-preventable childhood disease (covering 7 different diseases), with publication frequency increasing from the first such study published in 2008. IBMs have been useful to explore heterogeneous between- and within-host interactions, but combined applications are still sparse. The amount of missing information on model characteristics and study design is remarkable.

Conclusions

IBMs are suited to combine heterogeneous within- and between-host interactions, which offers many opportunities, especially to analyze targeted interventions for endemic infections. We advocate the exchange of (open-source) platforms and stress the need for consistent “branding”. Using (existing) conventions and reporting protocols would stimulate cross-fertilization between research groups and fields, and ultimately policy making in decades to come.

Electronic supplementary material

The online version of this article (doi:10.1186/s12879-017-2699-8) contains supplementary material, which is available to authorized users.

Connecting the dots: network data and models in HIV epidemiology

Effective HIV prevention requires knowledge of the structure and dynamics of the social networks across which infections are transmitted. These networks most commonly comprise chains of sexual relationships, but in some populations, sharing of contaminated needles is also an important, or even the main mechanism that connects people in the network. Whereas network data have long been collected during survey interviews, new data sources have become increasingly common in recent years, because of advances in molecular biology and the use of partner notification services in HIV prevention and treatment programmes. We review current and emerging methods for collecting HIV-related network data, as well as modelling frameworks commonly used to infer network parameters and map potential HIV transmission pathways within the network. We discuss the relative strengths and weaknesses of existing methods and models, and we propose a research agenda for advancing network analysis in HIV epidemiology. We make the case for a combination approach that integrates multiple data sources into a coherent statistical framework.

Methods for Health Economic Evaluation of Vaccines and Immunization Decision Frameworks: A Consensus Framework from a European Vaccine Economics Community

BACKGROUND

Incremental cost-effectiveness and cost-utility analyses [health economic evaluations (HEEs)] of vaccines are routinely considered in decision making on immunization in various industrialized countries. While guidelines advocating more standardization of such HEEs (mainly for curative drugs) exist, several immunization-specific aspects (e.g. indirect effects or discounting approach) are still a subject of debate within the scientific community.

OBJECTIVE

The objective of this study was to develop a consensus framework for HEEs of vaccines to support the development of national guidelines in Europe.

METHODS

A systematic literature review was conducted to identify prevailing issues related to HEEs of vaccines. Furthermore, European experts in the field of health economics and immunization decision making were nominated and asked to select relevant aspects for discussion. Based on this, a workshop was held with these experts. Aspects on 'mathematical modelling', 'health economics' and 'decision making' were debated in group-work sessions (GWS) to formulate recommendations and/or--if applicable--to state 'pros' and 'contras'.

RESULTS

A total of 13 different aspects were identified for modelling and HEE: model selection, time horizon of models, natural disease history, measures of vaccine-induced protection, duration of vaccine-induced protection, indirect effects apart from herd protection, target population, model calibration and validation, handling uncertainty, discounting, health-related quality of life, cost components, and perspectives. For decision making, there were four aspects regarding the purpose and the integration of HEEs of vaccines in decision making as well as the variation of parameters within uncertainty analyses and the reporting of results from HEEs. For each aspect, background information and an expert consensus were formulated.

CONCLUSIONS

There was consensus that when HEEs are used to prioritize healthcare funding, this should be done in a consistent way across all interventions, including vaccines. However, proper evaluation of vaccines implies using tools that are not commonly used for therapeutic drugs. Due to the complexity of and uncertainties around vaccination, transparency in the documentation of HEEs and during subsequent decision making is essential.

Optimizing agent-based transmission models for infectious diseases

BACKGROUND

Infectious disease modeling and computational power have evolved such that large-scale agent-based models (ABMs) have become feasible. However, the increasing hardware complexity requires adapted software designs to achieve the full potential of current high-performance workstations.

RESULTS

We have found large performance differences with a discrete-time ABM for close-contact disease transmission due to data locality. Sorting the population according to the social contact clusters reduced simulation time by a factor of two. Data locality and model performance can also be improved by storing person attributes separately instead of using person objects. Next, decreasing the number of operations by sorting people by health status before processing disease transmission has also a large impact on model performance. Depending of the clinical attack rate, target population and computer hardware, the introduction of the sort phase decreased the run time from 26% up to more than 70%. We have investigated the application of parallel programming techniques and found that the speedup is significant but it drops quickly with the number of cores. We observed that the effect of scheduling and workload chunk size is model specific and can make a large difference.

CONCLUSIONS

Investment in performance optimization of ABM simulator code can lead to significant run time reductions. The key steps are straightforward: the data structure for the population and sorting people on health status before effecting disease propagation. We believe these conclusions to be valid for a wide range of infectious disease ABMs. We recommend that future studies evaluate the impact of data management, algorithmic procedures and parallelization on model performance.

Estimating dynamic transmission model parameters for seasonal influenza by fitting to age and season-specific influenza-like illness incidence

Dynamic transmission models are essential to design and evaluate control strategies for airborne infections. Our objective was to develop a dynamic transmission model for seasonal influenza allowing to evaluate the impact of vaccinating specific age groups on the incidence of infection, disease and mortality. Projections based on such models heavily rely on assumed 'input' parameter values. In previous seasonal influenza models, these parameter values were commonly chosen ad hoc, ignoring between-season variability and without formal model validation or sensitivity analyses. We propose to directly estimate the parameters by fitting the model to age-specific influenza-like illness (ILI) incidence data over multiple influenza seasons. We used a weighted least squares (WLS) criterion to assess model fit and applied our method to Belgian ILI data over six influenza seasons. After exploring parameter importance using symbolic regression, we evaluated a set of candidate models of differing complexity according to the number of season-specific parameters. The transmission parameters (average R0, seasonal amplitude and timing of the seasonal peak), waning rates and the scale factor used for WLS optimization, influenced the fit to the observed ILI incidence the most. Our results demonstrate the importance of between-season variability in influenza transmission and our estimates are in line with the classification of influenza seasons according to intensity and vaccine matching.