Personalization of Medical Treatment Decisions: Simplifying Complex Models while Maintaining Patient Health Outcomes

Emulator-Based Bayesian Calibration of the CISNET Colorectal Cancer Models

PURPOSE

To calibrate Cancer Intervention and Surveillance Modeling Network (CISNET)'s SimCRC, MISCAN-Colon, and CRC-SPIN simulation models of the natural history colorectal cancer (CRC) with an emulator-based Bayesian algorithm and internally validate the model-predicted outcomes to calibration targets.

METHODS

We used Latin hypercube sampling to sample up to 50,000 parameter sets for each CISNET-CRC model and generated the corresponding outputs. We trained multilayer perceptron artificial neural networks (ANNs) as emulators using the input and output samples for each CISNET-CRC model. We selected ANN structures with corresponding hyperparameters (i.e., number of hidden layers, nodes, activation functions, epochs, and optimizer) that minimize the predicted mean square error on the validation sample. We implemented the ANN emulators in a probabilistic programming language and calibrated the input parameters with Hamiltonian Monte Carlo-based algorithms to obtain the joint posterior distributions of the CISNET-CRC models' parameters. We internally validated each calibrated emulator by comparing the model-predicted posterior outputs against the calibration targets.

RESULTS

The optimal ANN for SimCRC had 4 hidden layers and 360 hidden nodes, MISCAN-Colon had 4 hidden layers and 114 hidden nodes, and CRC-SPIN had 1 hidden layer and 140 hidden nodes. The total time for training and calibrating the emulators was 7.3, 4.0, and 0.66 h for SimCRC, MISCAN-Colon, and CRC-SPIN, respectively. The mean of the model-predicted outputs fell within the 95% confidence intervals of the calibration targets in 98 of 110 for SimCRC, 65 of 93 for MISCAN, and 31 of 41 targets for CRC-SPIN.

CONCLUSIONS

Using ANN emulators is a practical solution to reduce the computational burden and complexity for Bayesian calibration of individual-level simulation models used for policy analysis, such as the CISNET CRC models. In this work, we present a step-by-step guide to constructing emulators for calibrating 3 realistic CRC individual-level models using a Bayesian approach.

HIGHLIGHTS

We use artificial neural networks (ANNs) to build emulators that surrogate complex individual-based models to reduce the computational burden in the Bayesian calibration process.ANNs showed good performance in emulating the CISNET-CRC microsimulation models, despite having many input parameters and outputs.Using ANN emulators is a practical solution to reduce the computational burden and complexity for Bayesian calibration of individual-level simulation models used for policy analysis.This work aims to support health decision scientists who want to quantify the uncertainty of calibrated parameters of computationally intensive simulation models under a Bayesian framework.

Emulator-based Bayesian calibration of the CISNET colorectal cancer models

Purpose

To calibrate Cancer Intervention and Surveillance Modeling Network (CISNET) 's SimCRC, MISCAN-Colon, and CRC-SPIN simulation models of the natural history colorectal cancer (CRC) with an emulator-based Bayesian algorithm and internally validate the model-predicted outcomes to calibration targets.

Methods

We used Latin hypercube sampling to sample up to 50,000 parameter sets for each CISNET-CRC model and generated the corresponding outputs. We trained multilayer perceptron artificial neural networks (ANN) as emulators using the input and output samples for each CISNET-CRC model. We selected ANN structures with corresponding hyperparameters (i.e., number of hidden layers, nodes, activation functions, epochs, and optimizer) that minimize the predicted mean square error on the validation sample. We implemented the ANN emulators in a probabilistic programming language and calibrated the input parameters with Hamiltonian Monte Carlo-based algorithms to obtain the joint posterior distributions of the CISNET-CRC models' parameters. We internally validated each calibrated emulator by comparing the model-predicted posterior outputs against the calibration targets.

Results

The optimal ANN for SimCRC had four hidden layers and 360 hidden nodes, MISCAN-Colon had 4 hidden layers and 114 hidden nodes, and CRC-SPIN had one hidden layer and 140 hidden nodes. The total time for training and calibrating the emulators was 7.3, 4.0, and 0.66 hours for SimCRC, MISCAN-Colon, and CRC-SPIN, respectively. The mean of the model-predicted outputs fell within the 95% confidence intervals of the calibration targets in 98 of 110 for SimCRC, 65 of 93 for MISCAN, and 31 of 41 targets for CRC-SPIN.

Conclusions

Using ANN emulators is a practical solution to reduce the computational burden and complexity for Bayesian calibration of individual-level simulation models used for policy analysis, like the CISNET CRC models. In this work, we present a step-by-step guide to constructing emulators for calibrating three realistic CRC individual-level models using a Bayesian approach.

The role of INTERCheckWEB digital innovation in supporting polytherapy management

The study aims at defining the factors affecting the clinicians' decision of changing or confirming the treatment options for frail patients in polytherapy, supporting prescribing patterns, thus also figuring out if the inclination of the clinicians towards digital solutions (INTERCheckWEB) and specific guidelines, could play a role in their decision. A literature review was performed, revealing the main individual, organizational and decisional factors, impacting on the clinicians' propensity to change the current patients' therapy: the clinician perceptions of support in case of clinical guidelines use or INTERCheckWEB use were studied. A qualitative approach was implemented, and thirty-five clinicians completed a questionnaire, aimed at evaluating fifteen different clinical cases, defining if they would change the patient's current therapy depending on the level of information received. Three methodological approaches were implemented. (1) Bivariate correlations to test the relationships between variables. (2) Hierarchical sequential linear regression model to define the predictors of the clinician propensity to change therapy. (3) Fuzzy Qualitative Comparative Analysis-fsQCA, to figure out the combination of variables leading to the outcome. Patient's age and autonomy (p value = 0.000), as well as clinician's perception regarding IT ease of use (p value = 0.043) and seniority (p value = 0.009), number of drugs assumed by the patients (p value = 0.000) and number of concomitant diseases (p value = 0.000) are factors influencing a potential change in the current therapy. The fsQCA-crisp confirms that the clinical conditions of the patients are the driving factors that prompt the clinicians towards a therapy change.

Multi-omics data integration methods and their applications in psychiatric disorders

To study mental illness and health, in the past researchers have often broken down their complexity into individual subsystems (e.g., genomics, transcriptomics, proteomics, clinical data) and explored the components independently. Technological advancements and decreasing costs of high throughput sequencing has led to an unprecedented increase in data generation. Furthermore, over the years it has become increasingly clear that these subsystems do not act in isolation but instead interact with each other to drive mental illness and health. Consequently, individual subsystems are now analysed jointly to promote a holistic understanding of the underlying biological complexity of health and disease. Complementing the increasing data availability, current research is geared towards developing novel methods that can efficiently combine the information rich multi-omics data to discover biologically meaningful biomarkers for diagnosis, treatment, and prognosis. However, clinical translation of the research is still challenging. In this review, we summarise conventional and state-of-the-art statistical and machine learning approaches for discovery of biomarker, diagnosis, as well as outcome and treatment response prediction through integrating multi-omics and clinical data. In addition, we describe the role of biological model systems and in silico multi-omics model designs in clinical translation of psychiatric research from bench to bedside. Finally, we discuss the current challenges and explore the application of multi-omics integration in future psychiatric research. The review provides a structured overview and latest updates in the field of multi-omics in psychiatry.

Metamodeling for Policy Simulations with Multivariate Outcomes

PURPOSE

Metamodels are simplified approximations of more complex models that can be used as surrogates for the original models. Challenges in using metamodels for policy analysis arise when there are multiple correlated outputs of interest. We develop a framework for metamodeling with policy simulations to accommodate multivariate outcomes.

METHODS

We combine 2 algorithm adaptation methods-multitarget stacking and regression chain with maximum correlation-with different base learners including linear regression (LR), elastic net (EE) with second-order terms, Gaussian process regression (GPR), random forests (RFs), and neural networks. We optimize integrated models using variable selection and hyperparameter tuning. We compare the accuracy, efficiency, and interpretability of different approaches. As an example application, we develop metamodels to emulate a microsimulation model of testing and treatment strategies for hepatitis C in correctional settings.

RESULTS

Output variables from the simulation model were correlated (average ρ = 0.58). Without multioutput algorithm adaptation methods, in-sample fit (measured by R2) ranged from 0.881 for LR to 0.987 for GPR. The multioutput algorithm adaptation method increased R2 by an average 0.002 across base learners. Variable selection and hyperparameter tuning increased R2 by 0.009. Simpler models such as LR, EE, and RF required minimal training and prediction time. LR and EE had advantages in model interpretability, and we considered methods for improving the interpretability of other models.

CONCLUSIONS

In our example application, the choice of base learner had the largest impact on R2; multioutput algorithm adaptation and variable selection and hyperparameter tuning had a modest impact. Although advantages and disadvantages of specific learning algorithms may vary across different modeling applications, our framework for metamodeling in policy analyses with multivariate outcomes has broad applicability to decision analysis in health and medicine.

Parents' priorities for decision-making of pediatric epilepsy treatments and perceived needs for decision support in multi-ethnic Asian clinical setting: A qualitative analysis

OBJECTIVE

To identify parents' priorities when making a decision on genetic testing and antiseizure drug (ASD) options for pediatric epilepsy and their support needs for informed decision-making in multi-ethnic Asian clinical settings.

METHODS

Qualitative in-depth interviews, using a semi-structured interview guide, were conducted with purposively selected parents of pediatric patients with newly diagnosed epilepsy or known diagnosis of epilepsy (n = 26). Interviews were audio recorded and transcribed verbatim. Thematic analysis was undertaken to generate themes.

RESULTS

Parents' narratives showed difficulty assimilating information, while knowledge deficit and emotional vulnerability led parents' desire to defer a decision for testing and ASDs to mitigate decisional burden. Priorities for decisions were primarily based on intuitive ideas of the treatment's risks and benefits, yet very few could elaborate on tradeoffs between risks and efficacy. Priorities outside the purview of the healthcare team, such as children's emotional wellbeing and family burden of ASD administration, were also considered important. Authority-of-medical-professional heuristic facilitated the ASD decision for parents who preferred shared rather than sole responsibility for a decision. Importantly, parents' support needs for informed decision-making were very much related to the availability of support mechanisms in post-treatment decisions owing to perceived uncertainty of the chosen ASD.

CONCLUSIONS

Findings suggest that multiple priorities influenced ASD decision process. To address support needs of parents for informed decision-making, more consideration should be given to post-treatment decision support through the provision of educational opportunities, building peer support networks, and developing a novel communication channel between healthcare providers and parents.

When Is Mass Prophylaxis Cost-Effective for Epidemic Control? A Comparison of Decision Approaches

BACKGROUND

For certain communicable disease outbreaks, mass prophylaxis of uninfected individuals can curtail new infections. When an outbreak emerges, decision makers could benefit from methods to quickly determine whether mass prophylaxis is cost-effective. We consider 2 approaches: a simple decision model and machine learning meta-models. The motivating example is plague in Madagascar.

METHODS

We use a susceptible-exposed-infectious-removed (SEIR) epidemic model to derive a decision rule based on the fraction of the population infected, effective reproduction ratio, infection fatality rate, quality-adjusted life-year loss associated with death, prophylaxis effectiveness and cost, time horizon, and willingness-to-pay threshold. We also develop machine learning meta-models of a detailed model of plague in Madagascar using logistic regression, random forest, and neural network models. In numerical experiments, we compare results using the decision rule and the meta-models to results obtained using the simulation model. We vary the initial fraction of the population infected, the effective reproduction ratio, the intervention start date and duration, and the cost of prophylaxis.

LIMITATIONS

We assume homogeneous mixing and no negative side effects due to antibiotic prophylaxis.

RESULTS

The simple decision rule matched the SEIR model outcome in 85.4% of scenarios. Using data for a 2017 plague outbreak in Madagascar, the decision rule correctly indicated that mass prophylaxis was not cost-effective. The meta-models were significantly more accurate, with an accuracy of 92.8% for logistic regression, 95.8% for the neural network model, and 96.9% for the random forest model.

CONCLUSIONS

A simple decision rule using minimal information about an outbreak can accurately evaluate the cost-effectiveness of mass prophylaxis for outbreak mitigation. Meta-models of a complex disease simulation can achieve higher accuracy but with greater computational and data requirements and less interpretability.

HIGHLIGHTS

We use a susceptible-exposed-infectious-removed model and net monetary benefit to derive a simple decision rule to evaluate the cost-effectiveness of mass prophylaxis.We use the example of plague in Madagascar to compare the performance of the analytically derived decision rule to that of machine learning meta-models trained on a stochastic dynamic transmission model.We assess the accuracy of each approach for different combinations of disease dynamics and intervention scenarios.The machine learning meta-models are more accurate predictors of mass prophylaxis cost-effectiveness. However, the simple decision rule is also accurate and may be a preferred substitute in low-resource settings.