Policy evaluation in diabetes prevention and treatment using a population-based macro simulation model: the MICADO model

A Systematic Review of Methodologies Used in Models of the Treatment of Diabetes Mellitus

BACKGROUND

Diabetes mellitus is a chronic and complex disease, increasing in prevalence and consequent health expenditure. Cost-effectiveness models with long time horizons are commonly used to perform economic evaluations of diabetes' treatments. As such, prediction accuracy and structural uncertainty are important features in cost-effectiveness models of chronic conditions.

OBJECTIVES

The aim of this systematic review is to identify and review published cost-effectiveness models of diabetes treatments developed between 2011 and 2022 regarding their methodological characteristics. Further, it also appraises the quality of the methods used, and discusses opportunities for further methodological research.

METHODS

A systematic literature review was conducted in MEDLINE and Embase to identify peer-reviewed papers reporting cost-effectiveness models of diabetes treatments, with time horizons of more than 5 years, published in English between 1 January 2011 and 31 of December 2022. Screening, full-text inclusion, data extraction, quality assessment and data synthesis using narrative synthesis were performed. The Philips checklist was used for quality assessment of the included studies. The study was registered in PROSPERO (CRD42021248999).

RESULTS

The literature search identified 30 studies presenting 29 unique cost-effectiveness models of type 1 and/or type 2 diabetes treatments. The review identified 26 type 2 diabetes mellitus (T2DM) models, 3 type 1 DM (T1DM) models and one model for both types of diabetes. Fifteen models were patient-level models, whereas 14 were at cohort level. Parameter uncertainty was assessed thoroughly in most of the models, whereas structural uncertainty was seldom addressed. All the models where validation was conducted performed well. The methodological quality of the models with respect to structure was high, whereas with respect to data modelling it was moderate.

CONCLUSIONS

Models developed in the past 12 years for health economic evaluations of diabetes treatments are of high-quality and make use of advanced methods. However, further developments are needed to improve the statistical modelling component of cost-effectiveness models and to provide better assessment of structural uncertainty.

Cost-Effectiveness of SGLT2 Inhibitors in a Real-World Population: A MICADO Model-Based Analysis Using Routine Data from a GP Registry

OBJECTIVE

Sodium-glucose cotransporter 2 inhibitors (SGLT2i) have been shown to reduce the risk of cardiovascular complications, which largely drive diabetes' health and economic burdens. Trial results indicated that SGLT2i are cost effective. However, these findings may not be generalizable to the real-world target population. This study aims to evaluate the cost effectiveness of SGLT2i in a routine care type 2 diabetes population that meets Dutch reimbursement criteria using the MICADO model.

METHODS

Individuals from the Hoorn Diabetes Care System cohort (N = 15,392) were filtered to satisfy trial inclusion criteria (including EMPA-REG, CANVAS, and DECLARE-TIMI58) or satisfy the current Dutch reimbursement criteria for SGLT2i. We validated a health economic model (MICADO) by comparing simulated and observed outcomes regarding the relative risks of events in the intervention and comparator arm from three trials, and used the validated model to evaluate the long-term health outcomes using the filtered cohorts' baseline characteristics and treatment effects from trials and a review of observational studies. The incremental cost-effectiveness ratio (ICER) of SGLT2i, compared with care-as-usual, was assessed from a third-party payer perspective, measured in euros (2021 price level), using a discount rate of 4% for costs and 1.5% for effects.

RESULTS

From Dutch individuals with diabetes in routine care, 15.8% qualify for the current Dutch reimbursement criteria for SGLT2i. Their characteristics were significantly different (lower HbA1c, higher age, and generally more preexisting complications) than trial populations. After validating the MICADO model, we found that lifetime ICERs of SGLT2i, when compared with usual care, were favorable (< €20,000/QALY) for all filtered cohorts, resulting in an ICER of €5440/QALY using trial-based treatment effect estimates in reimbursed population. Several pragmatic scenarios were tested, the ICERs remained favorable.

CONCLUSIONS

Although the Dutch reimbursement indications led to a target group that deviates from trial populations, SGLT2i are likely to be cost effective when compared with usual care.

A systematic review of whole disease models for informing healthcare resource allocation decisions

BACKGROUND

Whole disease models (WDM) are large-scale, system-level models which can evaluate multiple decision questions across an entire care pathway. Whilst this type of model can offer several advantages as a platform for undertaking economic analyses, the availability and quality of existing WDMs is unknown.

OBJECTIVES

This systematic review aimed to identify existing WDMs to explore which disease areas they cover, to critically assess the quality of these models and provide recommendations for future research.

METHODS

An electronic search was performed on multiple databases (MEDLINE, EMBASE, the NHS Economic Evaluation Database and the Health Technology Assessment database) on 23rd July 2023. Two independent reviewers selected studies for inclusion. Study quality was assessed using the National Institute for Health and Care Excellence (NICE) appraisal checklist for economic evaluations. Model characteristics were descriptively summarised.

RESULTS

Forty-four WDMs were identified, of which thirty-two were developed after 2010. The main disease areas covered by existing WDMs are heart disease, cancer, acquired immune deficiency syndrome and metabolic disease. The quality of included WDMs is generally low. Common limitations included failure to consider the harms and costs of adverse events (AEs) of interventions, lack of probabilistic sensitivity analysis (PSA) and poor reporting.

CONCLUSIONS

There has been an increase in the number of WDMs since 2010. However, their quality is generally low which means they may require significant modification before they could be re-used, such as modelling AEs of interventions and incorporation of PSA. Sufficient details of the WDMs need to be reported to allow future reuse/adaptation.

Prediction of complications in health economic models of type 2 diabetes: a review of methods used

AIM

Diabetes health economic (HE) models play important roles in decision making. For most HE models of diabetes 2 diabetes (T2D), the core model concerns the prediction of complications. However, reviews of HE models pay little attention to the incorporation of prediction models. The objective of the current review is to investigate how prediction models have been incorporated into HE models of T2D and to identify challenges and possible solutions.

METHODS

PubMed, Web of Science, Embase, and Cochrane were searched from January 1, 1997, to November 15, 2022, to identify published HE models for T2D. All models that participated in The Mount Hood Diabetes Simulation Modeling Database or previous challenges were manually searched. Data extraction was performed by two independent authors. Characteristics of HE models, their underlying prediction models, and methods of incorporating prediction models were investigated.

RESULTS

The scoping review identified 34 HE models, including a continuous-time object-oriented model (n = 1), discrete-time state transition models (n = 18), and discrete-time discrete event simulation models (n = 15). Published prediction models were often applied to simulate complication risks, such as the UKPDS (n = 20), Framingham (n = 7), BRAVO (n = 2), NDR (n = 2), and RECODe (n = 2). Four methods were identified to combine interdependent prediction models for different complications, including random order evaluation (n = 12), simultaneous evaluation (n = 4), the 'sunflower method' (n = 3), and pre-defined order (n = 1). The remaining studies did not consider interdependency or reported unclearly.

CONCLUSIONS

The methodology of integrating prediction models in HE models requires further attention, especially regarding how prediction models are selected, adjusted, and ordered.

The transferability and validity of a population-level simulation model for the economic evaluation of interventions in diabetes: the MICADO model

AIMS

Valid health economic models are essential to inform the adoption and reimbursement of therapies for diabetes mellitus. Often existing health economic models are applied in other countries and settings than those where they were developed. This practice requires assessing the transferability of a model developed from one setting to another. We evaluate the transferability of the MICADO model, developed for the Dutch 2007 setting, in two different settings using a range of adjustment steps. MICADO predicts micro- and macrovascular events at the population level.

METHODS

MICADO simulation results were compared to observed events in an Italian 2000-2015 cohort (Casale Monferrato Survey [CMS]) and in a Dutch 2008-2019 (Hoorn Diabetes Care Center [DCS]) cohort after adjusting the demographic characteristics. Additional adjustments were performed to: (1) risk factors prevalence at baseline, (2) prevalence of complications, and (3) all-cause mortality risks by age and sex. Model validity was assessed by mean average percentage error (MAPE) of cumulative incidences over 10 years of follow-up, where lower values mean better accuracy.

RESULTS

For mortality, MAPE was lower for CMS compared to DCS (0.38 vs. 0.70 following demographic adjustment) and adjustment step 3 improved it to 0.20 in CMS, whereas step 2 showed best results in DCS (0.65). MAPE for heart failure and stroke in DCS were 0.11 and 0.22, respectively, while for CMS was 0.42 and 0.41.

CONCLUSIONS

The transferability of the MICADO model varied by event and per cohort. Additional adjustments improved prediction of events for MICADO. To ensure a valid model in a new setting it is imperative to assess the impact of adjustments in terms of model accuracy, even when this involves the same country, but a new time period.

Modeling Chronic Kidney Disease in Type 2 Diabetes Mellitus: A Systematic Literature Review of Models, Data Sources, and Derivation Cohorts

INTRODUCTION

As novel therapies for chronic kidney disease (CKD) in type 2 diabetes mellitus (T2DM) become available, their long-term benefits should be evaluated using CKD progression models. Existing models offer different modeling approaches that could be reused, but it may be challenging for modelers to assess commonalities and differences between the many available models. Additionally, the data and underlying population characteristics informing model parameters may not always be evident. Therefore, this study reviewed and summarized existing modeling approaches and data sources for CKD in T2DM, as a reference for future model development.

METHODS

This systematic literature review included computer simulation models of CKD in T2DM populations. Searches were implemented in PubMed (including MEDLINE), Embase, and the Cochrane Library, up to October 2021. Models were classified as cohort state-transition models (cSTM) or individual patient simulation (IPS) models. Information was extracted on modeled kidney disease states, risk equations for CKD, data sources, and baseline characteristics of derivation cohorts in primary data sources.

RESULTS

The review identified 49 models (21 IPS, 28 cSTM). A five-state structure was standard among state-transition models, comprising one kidney disease-free state, three kidney disease states [frequently including albuminuria and end-stage kidney disease (ESKD)], and one death state. Five models captured CKD regression and three included cardiovascular disease (CVD). Risk equations most commonly predicted albuminuria and ESKD incidence, while the most predicted CKD sequelae were mortality and CVD. Most data sources were well-established registries, cohort studies, and clinical trials often initiated decades ago in predominantly White populations in high-income countries. Some recent models were developed from country-specific data, particularly for Asian countries, or from clinical outcomes trials.

CONCLUSION

Modeling CKD in T2DM is an active research area, with a trend towards IPS models developed from non-Western data and single data sources, primarily recent outcomes trials of novel renoprotective treatments.

Impact of Policies in Nutrition and Physical Activity on Diabetes and Its Risk Factors in the 28 Member States of the European Union

Since healthy eating and physically active lifestyles can reduce diabetes mellitus (DM) risk, these are often addressed by population-based interventions aiming to prevent DM. Our study examined the impact of nutritional and physical activity policies, national diabetes plans and national diabetes registers contribute to lower prevalence of DM in individuals in the member states of the European Union (EU), taking into account the demographic and socioeconomic status as well as lifestyle choices. Datasets on policy actions, plans and registers were retrieved from the World Cancer Research Fund International’s NOURISHING and MOVING policy databases and the European Coalition for Diabetes report. Individual-based data on DM, socioeconomic status and healthy behavior indicators were obtained via the European Health Interview Survey, 2014. Our results showed variation in types and numbers of implemented policies within the member states, additionally, the higher number of these actions were not associated with lower DM prevalence. Only weak correlation between the prevalence of DM and preventive policies was found. Thus, undoubtedly policies have an impact on reducing the prevalence of DM, its increasing burden could not be reversed which underlines the need for applying a network of preventive policies.

Health State Utility Values for Type 2 Diabetes and Related Complications in East and Southeast Asia: A Systematic Review and Meta-Analysis

OBJECTIVES

East and Southeast Asia has the greatest burden of diabetes in the world. We sought to derive a reference set of utility values for type 2 diabetes without complication and disutility (utility decrement) values for important diabetes-related complications to better inform economic evaluation.

METHODS

A systematic review to identify utility values for diabetes and related complications reported in East and Southeast Asia. We searched MEDLINE (OVID) from inception to May 26, 2020 for utility values elicited using direct and indirect methods. Identified studies were assessed for quality based on the National Institute of Health and Care Excellence guidelines. Utility and disutility estimates were pooled by meta-analyses with subgroup analyses to evaluate differences by nationality and valuation instrument. (PROSPERO: CRD42020191075).

RESULTS

We identified 17 studies for the systematic review from a total of 13 035 studies in the initial search, of which 13 studies met the quality criteria for inclusion in the meta-analyses. The pooled utility value for diabetes without complication was 0.88 (95% CI 0.83-0.93), with the pooled utility decrement for associated complications ranged from 0.00 (for excess BMI) to 0.18 (for amputation). The utility values were consistently more conservative than previous estimates derived in Western populations. Utility decrements were comparable for SF-6D and EQ-5D valuation instruments and for Chinese and other Asian groups.

CONCLUSIONS

A reference set of pooled disutility and utility values for type 2 diabetes and its complications in East and Southeast Asian populations yielded more conservative estimates than Western populations.

Economic Simulation Modeling in Type 2 Diabetes

PURPOSE OF REVIEW

This paper provides an overview of type 2 diabetes economic simulation modeling and reviews current topics of discussion and major challenges in the field.

RECENT FINDINGS

Important challenges in the field include increasing the generalizability of models and improving transparency in model reporting. To identify and address these issues, modeling groups have organized through the Mount Hood Diabetes Challenge meetings and developed tools (i.e., checklist, impact inventory) to standardize modeling methods and reporting of results. Accordingly, many newer diabetes models have begun utilizing these tools, allowing for improved comparability between diabetes models. In the last two decades, type 2 diabetes simulation models have improved considerably, due to the collaborative work performed through the Mount Hood Diabetes Challenge meetings. To continue to improve diabetes models, future work must focus on clarifying diabetes progression in racial/ethnic minorities and incorporating equity considerations into health economic analysis.

Modeling of Diabetes and Its Clinical Impact

Understanding all aspects of diabetes treatment is hindered by the complexity of this chronic disease and its multifaceted complications and comorbidities, including social and financial impacts. In vivo studies as well as clinical trials provided invaluable information for unraveling not only metabolic processes but also risk estimations of, for example, complications. These approaches are often time- and cost-consuming and have frequently been supported by simulation models. Simulation models provide the opportunity to investigate diabetes treatment from additional viewpoints and with alternative objectives. This review presents selected models focusing either on metabolic processes or risk estimations and financial outcomes to provide a basic insight into this complex subject. It also discusses opportunities and challenges of modeling diabetes.

A New Statistical Method to Determine the Degree of Validity of Health Economic Model Outcomes against Empirical Data

BACKGROUND

The validation of health economic (HE) model outcomes against empirical data is of key importance. Although statistical testing seems applicable, guidelines for the validation of HE models lack guidance on statistical validation, and actual validation efforts often present subjective judgment of graphs and point estimates.

OBJECTIVES

To discuss the applicability of existing validation techniques and to present a new method for quantifying the degrees of validity statistically, which is useful for decision makers.

METHODS

A new Bayesian method is proposed to determine how well HE model outcomes compare with empirical data. Validity is based on a pre-established accuracy interval in which the model outcomes should fall. The method uses the outcomes of a probabilistic sensitivity analysis and results in a posterior distribution around the probability that HE model outcomes can be regarded as valid.

RESULTS

We use a published diabetes model (Modelling Integrated Care for Diabetes based on Observational data) to validate the outcome "number of patients who are on dialysis or with end-stage renal disease." Results indicate that a high probability of a valid outcome is associated with relatively wide accuracy intervals. In particular, 25% deviation from the observed outcome implied approximately 60% expected validity.

CONCLUSIONS

Current practice in HE model validation can be improved by using an alternative method based on assessing whether the model outcomes fit to empirical data at a predefined level of accuracy. This method has the advantage of assessing both model bias and parameter uncertainty and resulting in a quantitative measure of the degree of validity that penalizes models predicting the mean of an outcome correctly but with overly wide credible intervals.