Exploring Structural Uncertainty and Impact of Health State Utility Values on Lifetime Outcomes in Diabetes Economic Simulation Models: Findings from the Ninth Mount Hood Diabetes Quality-of-Life Challenge

BACKGROUND

Structural uncertainty can affect model-based economic simulation estimates and study conclusions. Unfortunately, unlike parameter uncertainty, relatively little is known about its magnitude of impact on life-years (LYs) and quality-adjusted life-years (QALYs) in modeling of diabetes. We leveraged the Mount Hood Diabetes Challenge Network, a biennial conference attended by international diabetes modeling groups, to assess structural uncertainty in simulating QALYs in type 2 diabetes simulation models.

METHODS

Eleven type 2 diabetes simulation modeling groups participated in the 9th Mount Hood Diabetes Challenge. Modeling groups simulated 5 diabetes-related intervention profiles using predefined baseline characteristics and a standard utility value set for diabetes-related complications. LYs and QALYs were reported. Simulations were repeated using lower and upper limits of the 95% confidence intervals of utility inputs. Changes in LYs and QALYs from tested interventions were compared across models. Additional analyses were conducted postchallenge to investigate drivers of cross-model differences.

RESULTS

Substantial cross-model variability in incremental LYs and QALYs was observed, particularly for HbA1c and body mass index (BMI) intervention profiles. For a 0.5%-point permanent HbA1c reduction, LY gains ranged from 0.050 to 0.750. For a 1-unit permanent BMI reduction, incremental QALYs varied from a small decrease in QALYs (-0.024) to an increase of 0.203. Changes in utility values of health states had a much smaller impact (to the hundredth of a decimal place) on incremental QALYs. Microsimulation models were found to generate a mean of 3.41 more LYs than cohort simulation models (P = 0.049).

CONCLUSIONS

Variations in utility values contribute to a lesser extent than uncertainty captured as structural uncertainty. These findings reinforce the importance of assessing structural uncertainty thoroughly because the choice of model (or models) can influence study results, which can serve as evidence for resource allocation decisions.HighlightsThe findings indicate substantial cross-model variability in QALY predictions for a standardized set of simulation scenarios and is considerably larger than within model variability to alternative health state utility values (e.g., lower and upper limits of the 95% confidence intervals of utility inputs).There is a need to understand and assess structural uncertainty, as the choice of model to inform resource allocation decisions can matter more than the choice of health state utility values.

Evaluating the Ability of Economic Models of Diabetes to Simulate New Cardiovascular Outcomes Trials: A Report on the Ninth Mount Hood Diabetes Challenge

OBJECTIVES

The cardiovascular outcomes challenge examined the predictive accuracy of 10 diabetes models in estimating hard outcomes in 2 recent cardiovascular outcomes trials (CVOTs) and whether recalibration can be used to improve replication.

METHODS

Participating groups were asked to reproduce the results of the Empagliflozin Cardiovascular Outcome Event Trial in Type 2 Diabetes Mellitus Patients (EMPA-REG OUTCOME) and the Canagliflozin Cardiovascular Assessment Study (CANVAS) Program. Calibration was performed and additional analyses assessed model ability to replicate absolute event rates, hazard ratios (HRs), and the generalizability of calibration across CVOTs within a drug class.

RESULTS

Ten groups submitted results. Models underestimated treatment effects (ie, HRs) using uncalibrated models for both trials. Calibration to the placebo arm of EMPA-REG OUTCOME greatly improved the prediction of event rates in the placebo, but less so in the active comparator arm. Calibrating to both arms of EMPA-REG OUTCOME individually enabled replication of the observed outcomes. Using EMPA-REG OUTCOME-calibrated models to predict CANVAS Program outcomes was an improvement over uncalibrated models but failed to capture treatment effects adequately. Applying canagliflozin HRs directly provided the best fit.

CONCLUSIONS

The Ninth Mount Hood Diabetes Challenge demonstrated that commonly used risk equations were generally unable to capture recent CVOT treatment effects but that calibration of the risk equations can improve predictive accuracy. Although calibration serves as a practical approach to improve predictive accuracy for CVOT outcomes, it does not extrapolate generally to other settings, time horizons, and comparators. New methods and/or new risk equations for capturing these CV benefits are needed.

Methods and quality of disease models incorporating more than two sexually transmitted infections: a protocol for a systematic review of the evidence

INTRODUCTION

Disease models can be useful tools for policy makers to inform their decisions. They can help to estimate the costs and benefits of interventions without conducting clinical trials and help to extrapolate the findings of clinical trials to a population level.Sexually transmitted infections (STIs) do not operate in isolation. Risk-taking behaviours and biological interactions can increase the likelihood of an individual being coinfected with more than one STI.Currently, few STI models consider coinfection or the interaction between STIs. We aim to identify and summarise STI models for two or more STIs and describe their modelling approaches.

METHODS AND ANALYSIS

Six databases (Cochrane, Embase, PLOS, ProQuest, Medline and Web of Science) were searched on 27 November 2018 to identify studies that focus on the reporting of the methodology and quality of models for at least two different STIs. The quality of all eligible studies will be accessed using a percentage scale published by Kopec et al . We will summarise all used approaches to model two or more STIs in one model. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework will be used to report all outcomes.

ETHICS AND DISSEMINATION

Ethical approval is not required for this systematic review. The results of this review will be published in a peer-reviewed journal and presented at a suitable conference. The findings from this review will be used to inform the development of a new multi-STI model.

PROSPERO REGISTRATION NUMBER

CRD42017076837.

Healthy, Sick, Dead - An Educational Blueprint to State Transition Disease Modelling

Disease Modelling of chronic diseases such as diabetes or asthma plays an important role in medical decision making. State transition models are the most frequently used method. The objective is to illustrate the elements and the most important underlying procedures for designing a decision analytic Markov model with only three-states.

METHOD

Being "healthy" can be interpreted as a norm state, being "sick" as a temporary state and "dead" as an absorbing state. Transitions with accompanying transition probabilities that allow a cohort of model objects "to flow" between the cumulative exhaustive and mutually exclusive states complete the model structure. Half-cycle correction helps in overcoming the fitting problem of the discrete time valuation of Markov models. A model with the three states healthy, sick and dead is the easiest way to define a reasonable model that covers almost all aspects of a Markov disease model. The absorbing state dead helps in terminating a model. The temporary state sick acts as an event counter and the state healthy serves as a reservoir of modelling objects. The definition of the number and length of cycles completes the definition of a simple state transition model. Additional supplementary material with a functional sample model is provided.

PROSIT Open Source Disease Models for Diabetes Mellitus

There has been legitimate criticism with regard to the quality and the transparency of health economic modelling studies. For that reason, the aim of the PROSIT Disease Modelling Community is to develop transparent open source health economic disease models for diabetes mellitus.

RESULTS

Markov type models were developed in the open source spread sheet software OpenOffice Calc for myocardial infarction, stroke, retinopathy, nephropathy, diabetic foot syndrome, and hypoglycemia. The basic concept is to describe a disease as a cascade of disease states with transitions between them. The transition probability is based on time, gender, age, disease related risks and medical interventions. An internet platform hosts the models and the documentation for public download. Incidence rates of complications were derived from population data and clinical studies. The models have to be adapted according to the specific needs and type of health economic analysis. The software is prepared to allow validation and model testing. The PROSIT Disease Modelling Community with its Markov models for diabetes mellitus suggests a new approach and methodology for developing health economic disease models in a transparent and sustainable manner. Going open source with disease models could overcome the lack in credibility that hampers modelling based health economic studies.

Prediction of 5-Year Survival with Data Mining Algorithms

Survival time prediction at the time of diagnosis is of great importance to make decisions about treatment and long-term follow-up care. However, predicting the outcome of cancer on the basis of clinical information is a challenging task. We now examined the ability of ten different data mining algorithms (Perceptron, Rule Induction, Support Vector Machine, Linear Regression, Naïve Bayes, Decision Tree, k-nearest Neighbor, Logistic Regression, Neural Network, Random Forest) to predict the dichotomous attribute "5-year-survival" based on seven attributes (sex, UICC-stage, etc.) which are available at the time of diagnosis. For this study we made use of the nationwide German research data set on colon cancer provided by the Robert Koch Institute. To assess the results a comparison between data mining algorithms and physicians' opinions was performed. Therefore, physicians guessed the survival time by leveraging the same seven attributes. The average accuracy of the physicians' opinion was 59%, the average accuracy of the machine learning algorithms was 67.7%.