Validation of the Economic and Health Outcomes Model of Type 2 Diabetes Mellitus (ECHO-T2DM)

Adverse drug events in cost-effectiveness models of pharmacological interventions for diabetes, diabetic retinopathy, and diabetic macular edema: a scoping review

OBJECTIVE

The objective of this review was to examine the role of adverse drug events (ADEs) caused by pharmacological interventions in cost-effectiveness models for diabetes mellitus, diabetic retinopathy, and diabetic macular edema.

INTRODUCTION

Guidelines for economic evaluation recognize the importance of including ADEs in the analysis, but in practice, consideration of ADEs in cost-effectiveness models seem to be vague. Inadequate inclusion of these harmful outcomes affects the reliability of the results, and the information provided by economic evaluation could be misleading. Reviewing whether and how ADEs are incorporated in cost-effectiveness models is necessary to understand the current practices of economic evaluation.

INCLUSION CRITERIA

Studies included were published between 2011-2022 in English, representing cost-effectiveness analyses using modeling framework for pharmacological interventions in the treatment of diabetes mellitus, diabetic retinopathy, or diabetic macular edema. Other types of analyses and other types of conditions were excluded.

METHODS

The databases searched included MEDLINE (PubMed), CINAHL (EBSCOhost), Scopus, Web of Science Core Collection, and NHS Economic Evaluation Database. Gray literature was searched via the National Institute for Health and Care Excellence, European Network for Health Technology Assessment, the National Institute for Health and Care Research, and the International Network of Agencies for Health Technology Assessment. The search was conducted on January 1, 2023. Titles and abstracts were screened for inclusion by 2 independent reviewers. Full-text review was conducted by 3 independent reviewers. A data extraction form was used to extract and analyze the data. Results were presented in tabular format with a narrative summary, and discussed in the context of existing literature and guidelines.

RESULTS

A total of 242 reports were extracted and analyzed in this scoping review. For the included analyses, type 2 diabetes was the most common disease (86%) followed by type 1 diabetes (10%), diabetic macular edema (9%), and diabetic retinopathy (0.4%). The majority of the included analyses used a health care payer perspective (88%) and had a time horizon of 30 years or more (75%). The most common model type was a simulation model (57%), followed by a Markov simulation model (18%). Of the included cost-effectiveness analyses, 26% included ADEs in the modeling, and 13% of the analyses excluded them. Most of the analyses (61%) partly considered ADEs; that is, only 1 or 2 ADEs were included. No difference in overall inclusion of ADEs between the different conditions existed, but the models for diabetic retinopathy and diabetic macular edema more often omitted the ADE-related impact on quality of life compared with the models for diabetes mellitus. Most analyses included ADEs in the models as probabilities (55%) or as a submodel (40%), and the most common source for ADE incidences were clinical trials (65%).

CONCLUSIONS

The inclusion of ADEs in cost-effectiveness models is suboptimal. The ADE-related costs were better captured than the ADE-related impact on quality of life, which was most pronounced in the models for diabetic retinopathy and diabetic macular edema. Future research should investigate the potential impact of ADEs on the results, and identify the criteria and policies for practical inclusion of ADEs in economic evaluation.

Examining the Impact of Structural Uncertainty Across 10 Type 2 Diabetes Models: Results From the 2022 Mount Hood Challenge

OBJECTIVES

The Mount Hood Diabetes Challenge Network aimed to examine the impact of model structural uncertainty on the estimated cost-effectiveness of interventions for type 2 diabetes.

METHODS

Ten independent modeling groups completed a blinded simulation exercise to estimate the cost-effectiveness of 3 interventions in 2 type 2 diabetes populations. Modeling groups were provided with a common baseline population, cost and utility values associated with different model health states, and instructions regarding time horizon and discounting. We collated the results to identify variation in predictions of net monetary benefit (NMB) and the drivers of those differences.

RESULTS

Overall, modeling groups agreed which interventions had a positive NMB (ie, were cost-effective), Although estimates of NMB varied substantially-by up to £23 696 for 1 intervention. Variation was mainly driven through differences in risk equations for complications of diabetes and their implementation between models. The number of modeled health states was also a significant predictor of NMB.

CONCLUSIONS

This exercise demonstrates that structural uncertainty between different health economic models affects cost-effectiveness estimates. Although it is reassuring that a decision maker would likely reach similar conclusions on which interventions were cost-effective using most models, the range in numerical estimates generated across different models would nevertheless be important for price-setting negotiations with intervention developers. Minimizing the impact of structural uncertainty on healthcare decision making therefore remains an important priority. Model registries, which record and compare the impact of structural assumptions, offer one potential avenue to improve confidence in the robustness of health economic modeling.

A Blueprint for Multi-use Disease Modeling in Health Economics: Results from Two Expert-Panel Consultations

BACKGROUND

The current use of health economic decision models in HTA is mostly confined to single use cases, which may be inefficient and result in little consistency over different treatment comparisons, and consequently inconsistent health policy decisions, for the same disorder. Multi-use disease models (MUDMs) (other terms: generic models, whole disease models, disease models) may offer a solution. However, much is uncertain about their definition and application. The current research aimed to develop a blueprint for the application of MUDMs.

METHODS

We elicited expert opinion using a two-round modified Delphi process. The panel consisted of experts and stakeholders in health economic modelling from various professional backgrounds. The first questionnaire concerned definition, terminology, potential applications, issues and recommendations for MUDMs and was based on an exploratory scoping review. In the second round, the panel members were asked to reconsider their input, based on feedback regarding first-round results, and to score issues and recommendations for priority. Finally, adding input from external advisors and policy makers in a structured way, an overview of issues and challenges was developed during two team consensus meetings.

RESULTS

In total, 54 respondents contributed to the panel results. The term 'multi-use disease models' was proposed and agreed upon, and a definition was provided. The panel prioritized 10 potential applications (with comparing alternative policies and supporting resource allocation decisions as the top 2), while 20 issues (with model transparency and stakeholders' roles as the top 2) were identified as challenges. Opinions on potential features concerning operationalization of multi-use models were given, with 11 of these subsequently receiving high priority scores (regular updates and revalidation after updates were the top 2).

CONCLUSIONS

MUDMs would improve on current decision support regarding cost-effectiveness information. Given feasibility challenges, this would be most relevant for diseases with multiple treatments, large burden of disease and requiring more complex models. The current overview offers policy makers a starting point to organize the development, use, and maintenance of MUDMs and to support choices concerning which diseases and policy decisions they will be helpful for.

A Systematic Review of Methodologies Used in Models of the Treatment of Diabetes Mellitus

BACKGROUND

Diabetes mellitus is a chronic and complex disease, increasing in prevalence and consequent health expenditure. Cost-effectiveness models with long time horizons are commonly used to perform economic evaluations of diabetes' treatments. As such, prediction accuracy and structural uncertainty are important features in cost-effectiveness models of chronic conditions.

OBJECTIVES

The aim of this systematic review is to identify and review published cost-effectiveness models of diabetes treatments developed between 2011 and 2022 regarding their methodological characteristics. Further, it also appraises the quality of the methods used, and discusses opportunities for further methodological research.

METHODS

A systematic literature review was conducted in MEDLINE and Embase to identify peer-reviewed papers reporting cost-effectiveness models of diabetes treatments, with time horizons of more than 5 years, published in English between 1 January 2011 and 31 of December 2022. Screening, full-text inclusion, data extraction, quality assessment and data synthesis using narrative synthesis were performed. The Philips checklist was used for quality assessment of the included studies. The study was registered in PROSPERO (CRD42021248999).

RESULTS

The literature search identified 30 studies presenting 29 unique cost-effectiveness models of type 1 and/or type 2 diabetes treatments. The review identified 26 type 2 diabetes mellitus (T2DM) models, 3 type 1 DM (T1DM) models and one model for both types of diabetes. Fifteen models were patient-level models, whereas 14 were at cohort level. Parameter uncertainty was assessed thoroughly in most of the models, whereas structural uncertainty was seldom addressed. All the models where validation was conducted performed well. The methodological quality of the models with respect to structure was high, whereas with respect to data modelling it was moderate.

CONCLUSIONS

Models developed in the past 12 years for health economic evaluations of diabetes treatments are of high-quality and make use of advanced methods. However, further developments are needed to improve the statistical modelling component of cost-effectiveness models and to provide better assessment of structural uncertainty.

A Systematic Review of Cost-Effectiveness Studies of Newer Non-Insulin Antidiabetic Drugs: Trends in Decision-Analytical Models for Modelling of Type 2 Diabetes Mellitus

BACKGROUND

We performed a systematic overview of the cost-effectiveness analyses (CEAs) comparing Non-insulin antidiabetic drugs (NIADs) with other NIADs for the treatment of type 2 diabetes mellitus (T2DM), using decision-analytical modelling (DAM), focusing on both the economic results and the underlying methodological choices.

METHODS

Eligible studies were CEAs using DAM to compare NIADs within the glucagon-like peptide-1 (GLP1) receptor agonists, sodium-glucose cotransporter-2 (SGLT2) inhibitors, or dipeptidyl peptidase-4 (DPP4) inhibitor classes with other NIADs within those classes for the treatment of T2DM. The PubMed, Embase and Econlit databases were searched from 1 January 2018 to 15 November 2022. Two reviewers screened the studies for relevance by titles and abstracts and then for eligibility via full-text screening, extracted the data from the full texts and appendices, and then stored the data in a spreadsheet.

RESULTS

The search yielded 890 records and 50 studies were eligible for inclusion. The studies were mainly based on a European setting (60%). Industry sponsorship was found in 82% of studies. The CORE diabetes model was used in 48% of the studies. GLP1 and SGLT2 products were the main comparators in 31 and 16 studies, respectively, while one study had DPP4 and two had no easily discernible main comparator. Direct comparison between SGLT2 and GLP1 occurred in 19 studies. At a class level, SGLT2 dominated GLP1 in six studies and was cost effective against GLP1 once as part of a treatment pathway. GLP1 was cost effective in nine studies and not cost effective against SGLT2 in three studies. At a product level, oral and injectable semaglutide, and empagliflozin, were cost effective against other within-class products. Injectable and oral semaglutide were more frequently found cost effective in these comparisons, with some conflicting results. Most of the modelled cohorts and treatment effects were sourced from randomised controlled trials. The following model assumptions varied depending on the class of the main comparator: choice of and reasoning behind risk equations, the time until the treatment switch, and how often the comparators were discontinued. Diabetes-related complications were emphasised on par with quality-adjusted life-years as model outputs. The main quality issues were regarding the description of alternatives, the perspective of analysis, the measurement of costs and consequences, and patient subgroups.

CONCLUSION

The included CEAs using DAMs have limitations that hinder their ability to inform decision makers on the cost-effective choice: lack of updated reasoning behind the choice of key model assumptions, over-reliance on risk equations based on older treatment practices, and sponsorship bias. The question of which NIAD is cost effective for the treatment of which T2DM patient is a pressing one and the answer remains unclear.

Prediction of complications in health economic models of type 2 diabetes: a review of methods used

AIM

Diabetes health economic (HE) models play important roles in decision making. For most HE models of diabetes 2 diabetes (T2D), the core model concerns the prediction of complications. However, reviews of HE models pay little attention to the incorporation of prediction models. The objective of the current review is to investigate how prediction models have been incorporated into HE models of T2D and to identify challenges and possible solutions.

METHODS

PubMed, Web of Science, Embase, and Cochrane were searched from January 1, 1997, to November 15, 2022, to identify published HE models for T2D. All models that participated in The Mount Hood Diabetes Simulation Modeling Database or previous challenges were manually searched. Data extraction was performed by two independent authors. Characteristics of HE models, their underlying prediction models, and methods of incorporating prediction models were investigated.

RESULTS

The scoping review identified 34 HE models, including a continuous-time object-oriented model (n = 1), discrete-time state transition models (n = 18), and discrete-time discrete event simulation models (n = 15). Published prediction models were often applied to simulate complication risks, such as the UKPDS (n = 20), Framingham (n = 7), BRAVO (n = 2), NDR (n = 2), and RECODe (n = 2). Four methods were identified to combine interdependent prediction models for different complications, including random order evaluation (n = 12), simultaneous evaluation (n = 4), the 'sunflower method' (n = 3), and pre-defined order (n = 1). The remaining studies did not consider interdependency or reported unclearly.

CONCLUSIONS

The methodology of integrating prediction models in HE models requires further attention, especially regarding how prediction models are selected, adjusted, and ordered.

Pharmacoeconomic analysis (CER) of Dulaglutide and Liraglutide in the treatment of patients with type 2 diabetes

AIM

To evaluate the treatment effect Fand pharmacoeconomic value of Dugaglutide in women with type 2 diabetes.

METHODS

Women (n=96) with type 2 diabetes recruited from June 2019 to December 2021 were randomized into two equal groups. The control group was treated with Liraglutide, and the observation group was treated with Dulaglutide, both for 24 weeks. The blood glucose levels, biochemical index, insulin resistance index (HOMA-IR), cost-effect ratio (CER), and drug safety were determined and compared between the two groups.

RESULTS

Blood glucose levels, the biochemical index, and HOMA-IR were lower in both groups after the treatment (P < 0.05), and there was no statistical difference in the blood glucose levels, biochemical index and HOMA-IR between the two groups (P > 0.05). The CER levels did not differ statistically between the two groups (P > 0.05). Both the cost and the incidence of drug side effects during solution injection were lower in the observation group than in the control group after 24 weeks of treatment (P < 0.05).

CONCLUSION

Both Dulaglutide and Liraglutide can reduce blood glucose levels, improve biochemical index, and HOMA-IR levels in women with type 2 diabetes. Dulaglutide is more cost-effective and safe.

CLINICAL TRIAL REGISTRATION

https://www.chictr.org.cn/index.aspx, identifier ChiCTR1900026514.

Does the Structure Matter? An External Validation and Health Economic Results Comparison of Event Simulation Approaches in Severe Obesity

OBJECTIVES

As obesity-associated events impact long-term survival, health economic (HE) modelling is commonly applied, but modelling approaches are diverse. This research aimed to compare the events simulation and the HE outcomes produced by different obesity modelling approaches.

METHODS

An external validation, using the Swedish obesity subjects (SOS) study, of three main structural event modelling approaches was performed: (1) continuous body mass index (BMI) approach; (2) risk equation approach; and (3) categorical BMI-related approach. Outcomes evaluated were mortality, cardiovascular events, and type 2 diabetes (T2D) for both the surgery and the control arms. Concordance between modelling results and the SOS study were investigated by different state-of-the-art measurements, and categorized by the grade of deviation observed (grades 1-4 expressing mild, moderate, severe, and very severe deviations). Furthermore, the costs per quality-adjusted life-year (QALY) gained of surgery versus controls were compared.

RESULTS

Overall and by study arm, the risk equation approach presented the lowest average grade of deviation (overall grade 2.50; control arm 2.25; surgery arm 2.75), followed by the continuous BMI approach (overall 3.25; control 3.50; surgery 3.00) and by the categorial BMI approach (overall 3.63; control 3.50; surgery 3.75). Considering different confidence interval limits, the costs per QALY gained were fairly comparable between all structural approaches (ranging from £2,055 to £6,206 simulating a lifetime horizon).

CONCLUSION

None of the structural approaches provided perfect external event validation, although the risk equation approach showed the lowest overall deviations. The economic outcomes resulting from the three approaches were fairly comparable.

The transferability and validity of a population-level simulation model for the economic evaluation of interventions in diabetes: the MICADO model

AIMS

Valid health economic models are essential to inform the adoption and reimbursement of therapies for diabetes mellitus. Often existing health economic models are applied in other countries and settings than those where they were developed. This practice requires assessing the transferability of a model developed from one setting to another. We evaluate the transferability of the MICADO model, developed for the Dutch 2007 setting, in two different settings using a range of adjustment steps. MICADO predicts micro- and macrovascular events at the population level.

METHODS

MICADO simulation results were compared to observed events in an Italian 2000-2015 cohort (Casale Monferrato Survey [CMS]) and in a Dutch 2008-2019 (Hoorn Diabetes Care Center [DCS]) cohort after adjusting the demographic characteristics. Additional adjustments were performed to: (1) risk factors prevalence at baseline, (2) prevalence of complications, and (3) all-cause mortality risks by age and sex. Model validity was assessed by mean average percentage error (MAPE) of cumulative incidences over 10 years of follow-up, where lower values mean better accuracy.

RESULTS

For mortality, MAPE was lower for CMS compared to DCS (0.38 vs. 0.70 following demographic adjustment) and adjustment step 3 improved it to 0.20 in CMS, whereas step 2 showed best results in DCS (0.65). MAPE for heart failure and stroke in DCS were 0.11 and 0.22, respectively, while for CMS was 0.42 and 0.41.

CONCLUSIONS

The transferability of the MICADO model varied by event and per cohort. Additional adjustments improved prediction of events for MICADO. To ensure a valid model in a new setting it is imperative to assess the impact of adjustments in terms of model accuracy, even when this involves the same country, but a new time period.

Cost-Effectiveness of Once-Weekly Semaglutide 1 mg versus Canagliflozin 300 mg in Patients with Type 2 Diabetes Mellitus in a Canadian Setting

OBJECTIVE

Our objective was to evaluate the long-term cost-effectiveness of once-weekly semaglutide 1 mg versus once-daily canagliflozin 300 mg in patients with type 2 diabetes mellitus (T2DM) uncontrolled with metformin from the healthcare payer and societal perspectives in Canada.

METHODS

Head-to-head data from the SUSTAIN 8 randomised trial (NCT03136484) were extrapolated over 40 years using economic simulation modelling. The cost-effectiveness of once-weekly semaglutide 1 mg versus canagliflozin 300 mg for treating T2DM was estimated using the Swedish Institute for Health Economics-Diabetes Cohort Model (IHE-DCM) and the Economic and Health Outcomes Model of T2DM (ECHO-T2DM). Unit costs and disutility weights capturing treatments and key macro- and microvascular complications were sourced from the literature to best match the Canadian setting. A probabilistic base-case simulation and sensitivity analyses were conducted.

RESULTS

Once-weekly semaglutide 1 mg was associated with reductions in macro- and microvascular complications, yielding incremental cost-effectiveness ratios (ICERs) of (Canadian dollars [CAD]) CAD16,392 and 18,098 per incremental quality-adjusted life-year (QALY) gained versus canagliflozin 300 mg for IHE-DCM and ECHO-T2DM, respectively, from a healthcare payer perspective. Accounting for productivity loss as well, ICERs were CAD14,127 and 13,188 per QALY gained for IHE-DCM and ECHO-T2DM, respectively, from a societal perspective. Sensitivity analyses confirmed that the base-case results were robust to changes in input parameters and assumptions used.

CONCLUSIONS

At a willingness-to-pay threshold of CAD50,000 per QALY gained, once-weekly semaglutide 1 mg was cost-effective over 40 years versus once-daily canagliflozin 300 mg for the treatment of T2DM in patients failing to maintain glycemic control with metformin alone.

Modeling Chronic Kidney Disease in Type 2 Diabetes Mellitus: A Systematic Literature Review of Models, Data Sources, and Derivation Cohorts

INTRODUCTION

As novel therapies for chronic kidney disease (CKD) in type 2 diabetes mellitus (T2DM) become available, their long-term benefits should be evaluated using CKD progression models. Existing models offer different modeling approaches that could be reused, but it may be challenging for modelers to assess commonalities and differences between the many available models. Additionally, the data and underlying population characteristics informing model parameters may not always be evident. Therefore, this study reviewed and summarized existing modeling approaches and data sources for CKD in T2DM, as a reference for future model development.

METHODS

This systematic literature review included computer simulation models of CKD in T2DM populations. Searches were implemented in PubMed (including MEDLINE), Embase, and the Cochrane Library, up to October 2021. Models were classified as cohort state-transition models (cSTM) or individual patient simulation (IPS) models. Information was extracted on modeled kidney disease states, risk equations for CKD, data sources, and baseline characteristics of derivation cohorts in primary data sources.

RESULTS

The review identified 49 models (21 IPS, 28 cSTM). A five-state structure was standard among state-transition models, comprising one kidney disease-free state, three kidney disease states [frequently including albuminuria and end-stage kidney disease (ESKD)], and one death state. Five models captured CKD regression and three included cardiovascular disease (CVD). Risk equations most commonly predicted albuminuria and ESKD incidence, while the most predicted CKD sequelae were mortality and CVD. Most data sources were well-established registries, cohort studies, and clinical trials often initiated decades ago in predominantly White populations in high-income countries. Some recent models were developed from country-specific data, particularly for Asian countries, or from clinical outcomes trials.

CONCLUSION

Modeling CKD in T2DM is an active research area, with a trend towards IPS models developed from non-Western data and single data sources, primarily recent outcomes trials of novel renoprotective treatments.

Performance of the UK Prospective Diabetes Study Outcomes Model 2 in a Contemporary UK Type 2 Diabetes Trial Cohort

OBJECTIVES

The UK Prospective Diabetes Study (UKPDS) Outcomes Model (UKPDS-OM) developed using 30-year (1977-2007) data from the UKPDS is widely used for health outcomes' projections and economic evaluations of therapies for patients with type 2 diabetes (T2D). Nevertheless, its reliability for contemporary UK T2D populations is unclear. We assessed the performance of version 2 of the model (UKPDS-OM2) using data from A Study of Cardiovascular Events in Diabetes (ASCEND), which followed participants with diabetes in the UK between 2005 and 2017.

METHODS

The UKPDS-OM2 was used to predict the occurrence of myocardial infarction (MI), other ischemic heart disease, stroke, cardiovascular (CV) death, and other death among the 14 569 participants with T2D in ASCEND, all without previous CV disease at study entry. Calibration (comparison of predicted and observed year-on-year cumulative incidence over 10 years) and discrimination (c-statistics) of the model were assessed for each endpoint. The percentage error in event rates at year 7 (mean duration of follow up) was used to quantify model bias.

RESULTS

The UKPDS-OM2 substantially overpredicted MI, stroke, CV death, and other death over the 10-year follow-up period (by 149%, 42%, 269%, and 52%, respectively, at year 7). Discrimination of the model for MI and other ischemic heart disease (c-statistics 0.58 and 0.60, respectively) was poorer than that for other outcomes (c-statistics ranging from 0.66 to 0.72).

CONCLUSIONS

The UKPDS-OM2 substantially overpredicted risks of key CV outcomes and death in people with T2D in ASCEND. Appropriate adjustments or a new model may be required for assessments of long-term effects of treatments in contemporary T2D cohorts.

Cardiovascular risk in patients with type 2 diabetes: A systematic review of prediction models

AIMS

To identify all cardiovascular disease risk prediction models developed in patients with type 2 diabetes or in the general population with diabetes as a covariate updating previous studies, describing model performance and analysing both their risk of bias and their applicability METHODS: A systematic search for predictive models of cardiovascular risk was performed in PubMed. The CHARMS and PROBAST guidelines for data extraction and for the assessment of risk of bias and applicability were followed. Google Scholar citations of the selected articles were reviewed to identify studies that conducted external validations.

RESULTS

The titles of 10,556 references were extracted to ultimately identify 19 studies with models developed in a population with diabetes and 46 studies in the general population. Within models developed in a population with diabetes, only six were classified as having a low risk of bias, 17 had a favourable assessment of applicability, 11 reported complete model information, and also 11 were externally validated.

CONCLUSIONS

There exists an overabundance of cardiovascular risk prediction models applicable to patients with diabetes, but many have a high risk of bias due to methodological shortcomings and independent validations are scarce. We recommend following the existing guidelines to facilitate their applicability.

Prediction of mortality and major cardiovascular complications in type 2 diabetes: External validation of UK Prospective Diabetes Study outcomes model version 2 in two European observational cohorts

AIM

To externally validate the UK Prospective Diabetes Study Outcomes Model version 2 (UKPDS-OM2) by comparing the predicted and observed outcomes in two European population-based cohorts of people with type 2 diabetes.

MATERIALS AND METHODS

We used data from the Casale Monferrato Survey (CMS; n = 1931) and a subgroup of the Hoorn Diabetes Care System (DCS) cohort (n = 5188). The following outcomes were analysed: all-cause mortality, myocardial infarction (MI), ischaemic heart disease (IHD), stroke, and congestive heart failure (CHF). Model performance was assessed by comparing predictions with observed cumulative incidences in each cohort during follow-up.

RESULTS

All-cause mortality was overestimated by the UKPDS-OM2 in both the cohorts, with a bias of 0.05 in the CMS and 0.12 in the DCS at 10 years of follow-up. For MI, predictions were consistently higher than observed incidence over the entire follow-up in both cohorts (10 years bias 0.07 for CMS and 0.10 for DCS). The model performed well for stroke and IHD outcomes in both cohorts. CHF incidence was predicted well for the DCS (5 years bias -0.001), but underestimated for the CMS cohort.

CONCLUSIONS

The UKPDS-OM2 consistently overpredicted the risk of mortality and MI in both cohorts during follow-up. Period effects may partially explain the differences. Results indicate that transferability is not satisfactory for all outcomes, and new or adjusted risk equations may be needed before applying the model to the Italian or Dutch settings.

Macrovascular Risk Equations Based on the CANVAS Program

BACKGROUND

Widely used risk equations for cardiovascular outcomes for individuals with type 2 diabetes mellitus (T2DM) have been incapable of predicting cardioprotective effects observed in recent cardiovascular outcomes trials (CVOTs) involving individuals with T2DM at high risk for or with established cardiovascular disease (CVD).

OBJECTIVE

We developed cardiovascular and mortality risk equations using patient-level data from the CANVAS (CANagliflozin cardioVascular Assessment Study) Program to address this shortcoming.

METHODS

Data from 10,142 patients with T2DM at high risk for or with established CVD, randomized to canagliflozin + standard of care (SoC) or SoC alone and followed for a mean duration of 3.6 years in the CANVAS Program were used to derive parametric risk equations for myocardial infarction (MI), stroke, hospitalization for heart failure (HHF), and death. Accumulated knowledge from the widely used UKPDS-OM2 (United Kingdom Prospective Diabetes Study Outcomes Model 2) was leveraged, and any departures in parameterization were limited to those necessary to provide adequate goodness of fit. Candidate explanatory covariates were selected using only the placebo arm to minimize confounding effects. Internal validation was performed separately by study treatment arm.

RESULTS

UKPDS-OM2 predicted CANVAS Program outcomes poorly. Recalibrating UKPDS-OM2 intercepts improved calibration in some cases. Refitting the coefficients but otherwise preserving the UKPDS-OM2 structure improved the fit substantially, which was sufficient for stroke and death. For MI, reselecting UKPDS-OM2 covariates and functional form proved sufficient. For HHF, selection from a broad set of candidate covariates and inclusion of a canagliflozin indicator was required.

CONCLUSION

These risk equations address some of the limitations of widely used risk equations, such as the UKPDS-OM2, for modeling cardioprotective treatments for individuals with T2DM and high cardiovascular risk, including derivation from overly healthy patients treated with agents that lack cardioprotection and have been described as reflecting a different therapeutic era. Future work is needed to examine external validity.

Development and Internal Validation of a Discrete Event Simulation Model of Diabetic Kidney Disease Using CREDENCE Trial Data

INTRODUCTION

The Canagliflozin and Renal Endpoints in Diabetes with Established Nephropathy Clinical Evaluation (CREDENCE) study showed that compared with placebo, canagliflozin 100 mg significantly reduced the risk of major cardiovascular events and adverse renal outcomes in patients with diabetic kidney disease (DKD). We developed a simulation model that can be used to estimate the long-term health and economic consequences of DKD treatment interventions for patients matching the CREDENCE study population.

METHODS

The CREDENCE Economic Model of DKD (CREDEM-DKD) was developed using patient-level data from CREDENCE (which recruited patients with estimated glomerular filtration rate 30 to < 90 mL/min/1.73 m2, urinary albumin to creatinine ratio > 300-5000 mg/g, and taking the maximum tolerated dose of a renin-angiotensin-aldosterone system inhibitor). Risk prediction equations were fit for start of maintenance dialysis, doubling of serum creatinine, hospitalization for heart failure, nonfatal myocardial infarction, nonfatal stroke, and all-cause mortality. A micro-simulation model was constructed using these risk equations combined with user-definable kidney transplant event risks. Internal validation was performed by loading the model to replicate the CREDENCE study and comparing predictions with trial Kaplan-Meier estimate curves. External validation was performed by loading the model to replicate a subgroup of the CANagliflozin cardioVascular Assessment Study (CANVAS) Program with patient characteristics that would have qualified for inclusion in CREDENCE.

RESULTS

Risk prediction equations generally fit well and exhibited good concordance, especially for the placebo arm. In the canagliflozin arm, modest underprediction was observed for myocardial infarction, along with overprediction of dialysis, doubling of serum creatinine, and all-cause mortality. Discrimination was strong (0.85) for the renal outcomes, but weaker for the macrovascular outcomes and all-cause mortality (0.60-0.68). The model performed well in internal and external validation exercises.

CONCLUSION

CREDEM-DKD is an important new tool in the evaluation of treatment interventions in the DKD population.

TRIAL REGISTRATION

ClinicalTrials.gov identifier, NCT02065791.

External Validation of the Core Obesity Model to Assess the Cost-Effectiveness of Weight Management Interventions

BACKGROUND

For economic models to be considered fit for purpose, it is vital that their outputs can be interpreted with confidence by clinicians, budget holders and other stakeholders. Consequently, thorough validation of models should be carried out to enhance confidence in their predictions. Here, we present results of external dependent and independent validations of the Core Obesity Model (COM), which was developed to assess the cost-effectiveness of weight management interventions.

OBJECTIVE

The aim was to assess the external validity of the COM (version 6.1), in line with best practice guidance from the International Society for Pharmacoeconomics and Outcomes Research and the Society for Medical Decision Making.

METHODS

For validation, suitable sources and outcomes were identified, and used to populate the COM with relevant inputs to allow prediction of study outcomes. Study characteristics were entered into the COM to replicate either the studies used to develop the model (dependent validation) or those not included in the model (independent validation). The concordance between predicted and observed outcomes was then assessed using established statistical methods and generation of mean error estimates.

RESULTS

For most outcomes, the predictions of the COM showed good linear correlation with observed outcomes, as evidenced by the high coefficients of determination (R² values). The independent validation revealed a degree of underestimation in predictions of cardiovascular (CV) disease and mortality, and type 2 diabetes.

CONCLUSION

The predictions generated by the risk equations used in the COM showed good concordance both with the studies used to develop the model and with studies not included in the model. In particular, the concordance observed in the external dependent validation suggests that the COM accurately predicts obesity-related event rates observed in the studies used to develop the model. However, the impact of existing CV risk, as well as mortality, is a key area for future refinement of the COM. Our results should increase confidence in the estimates derived from the COM and reduce uncertainty associated with analyses using this model.

Comparing the Cohort and Micro-Simulation Modeling Approaches in Cost-Effectiveness Modeling of Type 2 Diabetes Mellitus: A Case Study of the IHE Diabetes Cohort Model and the Economics and Health Outcomes Model of T2DM

BACKGROUND

Economic modeling is widely used in estimating cost-effectiveness in type 2 diabetes mellitus. Because type 2 diabetes is complex and patients are heterogenous, the cohort modeling approach may generate biased estimates of costeffectiveness. The IHE Diabetes Cohort Model (IHE-DCM) was constructed using the cohort approach as an alternative for stakeholders with limited resources, some of whom have voiced reasonable concerns about a lack of transparency with type 2 diabetes micro-simulation models and long run times.

OBJECTIVES

The objective of this study was to inform decision makers by investigating the direction and magnitude of bias of IHE-DCM cost-effectiveness estimates that can be attributed to the cohort modeling approach.

METHODS

Simulation scenarios inspired by the 9th Mount Hood Diabetes Challenge were simulated with IHE-DCM and with a micro-simulation model, the Economic and Health Outcomes Model of T2DM (ECHO-T2DM), and key metrics (absolute and incremental costs and quality-adjusted life-years, event rates, and cost-effectiveness) were compared for evidence of systematic differences. The models were harmonized to the extent possible to ensure that differences were driven primarily by the unit of observation and not by other model differences.

RESULTS

IHE-DCM run times were faster and IHE-DCM produced uniformly larger estimates of absolute life-years, quality-adjusted life-years, and costs than ECHO-T2DM but smaller between-arm (incremental) differences. Estimated incremental cost-effectiveness ratios and net monetary benefits varied similarly and predictably across the scenarios. On average, IHE-DCM estimates of incremental cost-effectiveness ratios and net monetary benefits were CAN$269 (3%) and CAN$2935 (10%) smaller, respectively, than ECHO-T2DM.

CONCLUSIONS

There was little evidence that estimated cost-effectiveness metrics, the outcomes that matter most to stakeholders, differed systematically.

Performance of the UKPDS Outcomes Model 2 for Predicting Death and Cardiovascular Events in Patients with Type 2 Diabetes Mellitus from a German Population-Based Cohort

BACKGROUND AND OBJECTIVE

Accurate prediction of relevant outcomes is important for targeting therapies and to support health economic evaluations of healthcare interventions in patients with diabetes. The United Kingdom Prospective Diabetes Study (UKPDS) risk equations are some of the most frequently used risk equations. This study aims to analyze the calibration and discrimination of the updated UKPDS risk equations as implemented in the UKPDS Outcomes Model 2 (UKPDS-OM2) for predicting cardiovascular (CV) events and death in patients with type 2 diabetes mellitus (T2DM) from population-based German samples.

METHODS

Analyses are based on data of 456 individuals diagnosed with T2DM who participated in two population-based studies in southern Germany (KORA (Cooperative Health Research in the Region of Augsburg)-A: 1997/1998, n = 178; KORA-S4: 1999-2001, n = 278). We compared the participants' 10-year observed incidence of mortality, CV mortality, myocardial infarction (MI), and stroke with the predicted event rate of the UKPDS-OM2. The model's calibration was evaluated by Greenwood-Nam-D'Agostino tests and discrimination was evaluated by C-statistics.

RESULTS

Of the 456 participants with T2DM (mean age 65 years, mean diabetes duration 8 years, 56% male), over the 10-year follow-up time 129 died (61 due to CV events), 64 experienced an MI, and 46 a stroke. The UKPDS-OM2 significantly over-predicted mortality and CV mortality by 25% and 28%, respectively (Greenwood-Nam-D'Agostino tests: p < 0.01), but there was no significant difference between predicted and observed MI and stroke risk. The model poorly discriminated for death (C-statistic [95% confidence interval] = 0.64 [0.60-0.69]), CV death (0.64 [0.58-0.71]), and MI (0.58 [0.52-0.66]), and failed to discriminate for stroke (0.57 [0.47-0.66]).

CONCLUSIONS

The study results demonstrate acceptable calibration and poor discrimination of the UKPDS-OM2 for predicting death and CV events in this population-based German sample. Those limitations should be considered when using the UKPDS-OM2 for economic evaluations of healthcare strategies or using the risk equations for clinical decision-making.

Cost Effectiveness of Once-Weekly Semaglutide Versus Once-Weekly Dulaglutide in the Treatment of Type 2 Diabetes in Canada

OBJECTIVE

The aim of this study was to assess the cost effectiveness of semaglutide versus dulaglutide, as an add-on to metformin monotherapy, for the treatment of type 2 diabetes (T2D), from a Canadian societal perspective.

METHODS

The Swedish Institute for Health Economics Cohort Model of T2D was used to assess the cost effectiveness of once-weekly semaglutide (0.5 or 1.0 mg) versus once-weekly dulaglutide (0.75 or 1.5 mg) over a 40-year time horizon. Using data from the SUSTAIN 7 trial, which demonstrated comparatively greater reductions in glycated hemoglobin (HbA1c), body mass index and systolic blood pressure with semaglutide, compared with dulaglutide, a deterministic base-case and scenario simulation were conducted. The robustness of the results was evaluated with probabilistic sensitivity analyses and 15 deterministic sensitivity analyses.

RESULTS

The base-case analysis indicated that semaglutide is a dominant treatment option, compared with dulaglutide. Semaglutide was associated with lower total costs (Canadian dollars [CAN$]) versus dulaglutide for both low-dose (CAN$113,287 vs. CAN$113,690; cost-saving: CAN$403) and high-dose (CAN$112,983 vs. CAN$113,695; cost-saving: CAN$711) comparisons. Semaglutide resulted in increased quality-adjusted life-years (QALYs) and QALY gains, compared with dulaglutide, for both low-dose (11.10 vs. 11.07 QALYs; + 0.04 QALYs) and high-dose (11.12 vs. 11.07 QALYs; + 0.05 QALYs) comparisons. The probabilistic sensitivity analysis showed that for 66-73% of iterations, semaglutide was either dominant or was considered cost effective at a willingness-to-pay threshold of CAN$50,000.

CONCLUSIONS

From a Canadian societal perspective, semaglutide may be a cost-effective treatment option versus dulaglutide in patients with T2D who are inadequately controlled on metformin monotherapy.

TECH-VER: A Verification Checklist to Reduce Errors in Models and Improve Their Credibility

BACKGROUND

In health economic literature, checklists or best practice recommendations on model validation/credibility always declare verification of the programmed model as a fundamental step, such as 'is the model implemented correctly and does the implementation accurately represent the conceptual model?' However, to date, little operational guidance for the model verification process has been given. In this study, we aimed to create an operational checklist for model users or reviewers to verify the technical implementation of health economic decision analytical models and document their verification efforts.

METHODS

Literature on model validation, verification, programming errors and credibility was reviewed systematically from scientific databases. An initial beta version of the checklist was developed based on the checklists/tests identified from the literature and from authors' previous modeling/appraisal experience. Next, the first draft checklist was presented to a number of health economists on several occasions and was tested on different models (built in different software, developed by different stakeholders, including drug manufacturers, consultancies or academia), each time leading to an update of the checklist and culminating in the final version of the TECHnical VERification (TECH-VER) checklist, introduced in this paper.

RESULTS

The TECH-VER necessitates a model reviewer (preferably independent), an executable and transparent model, its input sources, and detailed documentation (e.g. technical report/scientific paper) in which the conceptual model, its implementation, programmed model inputs, and results are reported. The TECH-VER checklist consists of five domains: (1) input calculations; (2) event-state (patient flow) calculations; (3) result calculations; (4) uncertainty analysis calculations; and (5) other overall checks (e.g. validity or interface). The first four domains reflect the verification of the components of a typical health economic model. For these domains, as a prerequisite of verification tests, the reviewer should identify the relevant calculations in the electronic model and assess the provided justifications for the methods used in the identified calculations. For this purpose, we recommend completeness/consistency checks. Afterwards, the verification tests can be conducted for the calculations in each of these stages by checking the correctness of the implementation of these calculations. For this purpose, the following type of tests are recommended in consecutive order: (i) black-box tests, i.e. checking if model calculations are in line with a priori expectations; (ii) white-box testing, i.e. going through the program code details line by line, or cell by cell (recommended for some crucial calculations and if there are some unexpected results from the black-box tests); and (iii) model replication/parallel programming (recommended only in certain situations, and if the issues related to the identified unexpected results from black-box tests could not be resolved through white-box testing). To reduce the time burden of model verification, we suggest a hierarchical order in tests i-iii, where going to the next step is necessary when the previous step fails.

CONCLUSIONS

The TECH-VER checklist is a comprehensive checklist for the technical verification of decision analytical models, aiming to help identify model implementation errors and their root causes while improving the transparency and efficiency of the verification efforts. In addition to external reviews, we consider that the TECH-VER can be used as an internal training and quality control tool for new health economists, while developing their initial models. It is the authors' aim that the TECH-VER checklist transforms itself to an open-source living document, with possible future versions, or 'bolt-on' extensions for specific applications with additional 'fit-for-purpose' tests, as well as 'tips and tricks' and some demonstrative error examples. For this reason, the TECH-VER checklist and the list of black-box tests created in this paper and a few model verification examples is uploaded to an open access, online platform (github and the website of the institute), where other users will also be able to upload their original verification efforts and tests.

Pharmacological treatment for Type 2 diabetes integrating findings from cardiovascular outcome trials: an expert consensus in the UK

In people with Type 2 diabetes, cardiovascular disease is a leading cause of morbidity and mortality. Thus, as well as controlling glucose, reducing the risk of cardiovascular events is a key goal. The results of cardiovascular outcome trials have led to updates for many national and international guidelines. England, Wales and Northern Ireland remain exceptions, with the most recent update to the National Institute for Health and Care Excellence (NICE) guidelines published in 2015. We reviewed current national and international guidelines and recommendations on the management of people with Type 2 diabetes. This article shares our consensus on clinical recommendations for the use of sodium-glucose co-transporter 2 inhibitors (SGLT-2is) and glucagon-like peptide 1 receptor agonists (GLP-1RAs) in people with Type 2 diabetes and established or at very high risk of cardiovascular disease in the UK. We also consider cost-effectiveness for these therapies. We recommend considering each person's cardiovascular risk and using diabetes therapies with proven cardiovascular benefits when appropriate to improve long-term outcomes and cost-effectiveness.

Development and validation of a Health Policy Model of Type 2 diabetes in Chinese setting

Aim: Due to the difference in epidemiology and outcomes between eastern and western populations with Type 2 diabetes mellitus (T2DM), an important challenge is determining how useful the outcomes from diabetes models based on western populations are for eastern patients. Consequently, the principal aim of this study was to develop and validate a Health Policy Model (Chinese Outcomes Model for T2DM [COMT]) for supporting Chinese medical and health economic studies. Methods: The model is created to simulate a series of important complications of T2DM diabetes based on the latest Risk Equations for Complications of Type 2 Diabetes, which was adjusted by adding the adjustment regulator to the linear predictor within the risk equation. The validity of the model was conducted by using a total of 171 validation outcomes from seven studies in eastern populations and ten studies in western populations. The simulation cohorts in the COMT model were generated by copying each validation study's baseline characteristics. Concordance was tested by assessing the difference between the identity (45°) line and the best-fitting regression of the scatterplots for the predicted versus observed outcomes. Results: The slope coefficients of the best-fitting regression line between the predicted and corresponding observed actual outcomes was 0.9631 and the R² was 0.8701. There were major differences between western and eastern populations. The slope and R² of predictions were 0.9473 and 0.9272 in the eastern population and 1.0566 and 0.8863 in the western population, which showed more perfect agreement with the observed values in the eastern population than the western populations. The subset of macro-vascular and micro-vascular outcomes in the eastern population showed an identical tendency (the slope coefficient was close to 1), and mortality outcomes showed a slight tendency toward overestimation (the slope coefficient was close to 0.9208). Some degree of underprediction of macro-vascular and micro-vascular end points and overprediction of mortality end point was found in the western population. Conclusion: The COMT diabetes model simulated the long-term patient outcomes observed in eastern Asian T2DM patients with prediction accuracy. This study supports the COMT as a credible tool for Chinese healthcare decision makers. Further work is necessary to incorporate new local data to improve model validity and credibility.

Uncontrolled diabetes and health care utilisation: panel data evidence from Spain

Despite size and relevance of uncontrolled diabetes, robust evidence on its effects on health care utilisation is very limited, especially among European countries. We employed longitudinal administrative data from Spain (2004-2010) to explore the relationship between uncontrolled type 2 diabetes and health care utilisation. We used a biomarker (glycated haemoglobin, HbA1c) to detect the presence of uncontrolled diabetes and explore its effects on both primary and secondary health care. We estimated a range of panel count data models, including negative binomials with random effects, dynamic and hurdle specifications to account for unobserved heterogeneity, previous utilisation and selection. We found uncontrolled diabetes in between 27 and 30% of patients of both genders. Our estimates suggested that although women appeared to systematically consume more health care compared to men, their consumption levels did not seem to be influenced by uncontrolled diabetes. Conversely, among men uncontrolled diabetes increased the average number of GP visits per year by between 3 and 3.4%, specialist visits by 5.3-6.1%, depending on specifications, and also extended annual hospital length of stay by 15%. We also found some evidence of heterogeneity in utilisation based on the level of uncontrolled diabetes among male individuals. Overall, our results suggested the need for different diabetes management plans depending on gender and levels of glycaemic control.

Cost-Effectiveness Analysis of Canagliflozin 300 mg Versus Dapagliflozin 10 mg Added to Metformin in Patients with Type 2 Diabetes in the United States

INTRODUCTION

Agents that inhibit sodium glucose co-transporter 2 (SGLT2), including canagliflozin and dapagliflozin, are approved in the United States for the treatment of adults with type 2 diabetes mellitus (T2DM). SGLT2 inhibition lowers blood glucose by increasing urinary glucose excretion, which leads to a mild osmotic diuresis and a net loss of calories that are associated with reductions in body weight and blood pressure. This analysis evaluated the cost-effectiveness of canagliflozin 300 mg versus dapagliflozin 10 mg in patients with T2DM inadequately controlled with metformin in the United States.

METHODS

A 30-year cost-effectiveness analysis was performed using the validated Economic and Health Outcomes Model of T2DM (ECHO-T2DM) from the perspective of the third-party health care system in the United States. Patient demographics, biomarker values, and treatment effects for the ECHO-T2DM model were sourced primarily from a network meta-analysis (NMA) that included studies of canagliflozin and dapagliflozin in patients with T2DM on background metformin. Costs were derived from sources specific to the United States. Outcomes and costs were discounted at 3%. Sensitivity analyses that varied key model parameters were conducted.

RESULTS

Canagliflozin 300 mg dominated dapagliflozin 10 mg as an add-on to metformin over 30 years, with an estimated cost offset of $13,991 and a quality-adjusted life-year gain of 0.08 versus dapagliflozin 10 mg. Results were driven by the better HbA1c lowering achieved with canagliflozin, which translated to less need for insulin rescue therapy. Findings from sensitivity analyses were consistent with the base case.

CONCLUSION

These results suggest that canagliflozin 300 mg is likely to provide better health outcomes at a lower overall cost than dapagliflozin 10 mg in patients with T2DM inadequately controlled with metformin from the perspective of the United States health care system.

FUNDING

Janssen Scientific Affairs, LLC and Janssen Global Services, LLC.