The impact of structural uncertainty on cost-effectiveness models for adjuvant endocrine breast cancer treatments: the need for disease-specific model standardization and improved guidance

Impact of Structural Differences on the Modeled Cost-Effectiveness of Noninvasive Prenatal Testing

BACKGROUND

Noninvasive prenatal testing (NIPT) was developed to improve the accuracy of prenatal screening to detect chromosomal abnormalities. Published economic analyses have yielded different incremental cost-effective ratios (ICERs), leading to conclusions of NIPT being dominant, cost-effective, and cost-ineffective. These analyses have used different model structures, and the extent to which these structural variations have contributed to differences in ICERs is unclear.

AIM

To assess the impact of different model structures on the cost-effectiveness of NIPT for the detection of trisomy 21 (T21; Down syndrome).

METHODS

A systematic review identified economic models comparing NIPT to conventional screening. The key variations in identified model structures were the number of health states and modeling approach. New models with different structures were developed in TreeAge and populated with consistent parameters to enable a comparison of the impact of selected structural variations on results.

RESULTS

The review identified 34 economic models. Based on these findings, demonstration models were developed: 1) a decision tree with 3 health states, 2) a decision tree with 5 health states, 3) a microsimulation with 3 health states, and 4) a microsimulation with 5 health states. The base-case ICER from each model was 1) USD$34,474 (2023)/quality-adjusted life-year (QALY), 2) USD$14,990 (2023)/QALY, (3) USD$54,983 (2023)/QALY, and (4) NIPT was dominated.

CONCLUSION

Model-structuring choices can have a large impact on the ICER and conclusions regarding cost-effectiveness, which may inadvertently affect policy decisions to support or not support funding for NIPT. The use of reference models could improve international consistency in health policy decision making for prenatal screening.

HIGHLIGHTS

NIPT is a clinical area in which a variety of modeling approaches have been published, with wide variation in reported cost-effectiveness.This study shows that when broader contextual factors are held constant, varying the model structure yields results that range from NIPT being less effective and more expensive than conventional screening (i.e., NIPT was dominated) through to NIPT being more effective and more expensive than conventional screening with an ICER of USD$54,983 (2023)/QALY.Model-structuring choices may inadvertently affect policy decisions to support or not support funding of NIPT. Reference models could improve international consistency in health policy decision making for prenatal screening.

A Systematic Review of Methodologies Used in Models of the Treatment of Diabetes Mellitus

BACKGROUND

Diabetes mellitus is a chronic and complex disease, increasing in prevalence and consequent health expenditure. Cost-effectiveness models with long time horizons are commonly used to perform economic evaluations of diabetes' treatments. As such, prediction accuracy and structural uncertainty are important features in cost-effectiveness models of chronic conditions.

OBJECTIVES

The aim of this systematic review is to identify and review published cost-effectiveness models of diabetes treatments developed between 2011 and 2022 regarding their methodological characteristics. Further, it also appraises the quality of the methods used, and discusses opportunities for further methodological research.

METHODS

A systematic literature review was conducted in MEDLINE and Embase to identify peer-reviewed papers reporting cost-effectiveness models of diabetes treatments, with time horizons of more than 5 years, published in English between 1 January 2011 and 31 of December 2022. Screening, full-text inclusion, data extraction, quality assessment and data synthesis using narrative synthesis were performed. The Philips checklist was used for quality assessment of the included studies. The study was registered in PROSPERO (CRD42021248999).

RESULTS

The literature search identified 30 studies presenting 29 unique cost-effectiveness models of type 1 and/or type 2 diabetes treatments. The review identified 26 type 2 diabetes mellitus (T2DM) models, 3 type 1 DM (T1DM) models and one model for both types of diabetes. Fifteen models were patient-level models, whereas 14 were at cohort level. Parameter uncertainty was assessed thoroughly in most of the models, whereas structural uncertainty was seldom addressed. All the models where validation was conducted performed well. The methodological quality of the models with respect to structure was high, whereas with respect to data modelling it was moderate.

CONCLUSIONS

Models developed in the past 12 years for health economic evaluations of diabetes treatments are of high-quality and make use of advanced methods. However, further developments are needed to improve the statistical modelling component of cost-effectiveness models and to provide better assessment of structural uncertainty.

Model parameters influencing the cost-effectiveness of sacubitril/valsartan in heart failure: evidence from a systematic literature review

OBJECTIVES

To summarize cost-effectiveness (CE) evidence of sacubitril/valsartan for the treatment of heart failure (HF) patients with reduced ejection fraction (HFrEF). The impact of different modeling approaches and parameters on the CE results is also described.

METHODS

We conducted a systematic literature review using multiple databases: Embase^®; MEDLINE^®; MEDLINE^®-In Process; NIHR CRD database including DARE, NHS EED, and HTA databases; and the Cost Effectiveness Analysis registry. We also reviewed HTA countries' websites to identify CE reports of sacubitril/valsartan, published up to 25-July-2021. Articles published in English as full-texts, conference-abstracts, or HTA reports were included.

RESULTS

We included 44 CE models [39 from 37 publications (22 full-texts; 15 conference-abstracts) and 5 HTAs; Europe, n = 20; North and South Americas, n = 14; Asia and Australia, n = 10]. Most models adopted a Markov structure with constant transition probabilities of events (n = 27) or a mix of Markov and regression-based models (n = 16), with variations in structural assumptions and chosen parameters. Study authors concluded sacubitril/valsartan to be a cost-effective therapy in 37/41 models in chronic HFrEF patients and 2/3 models in hospitalized patients stabilized after an acute decompensation for HF. CE models showing sacubitril/valsartan not to be a cost-effective treatment generally modeled a shorter time horizon. Effect of sacubitril/valsartan on cardiovascular and all-cause mortality, cost, duration of effect and time horizon was the main model drivers.

CONCLUSIONS

Most evidence indicated sacubitril/valsartan is cost-effective in HFrEF. The use of a lifetime horizon is recommended in future models as HF is a chronic disease. Data on the CE of sacubitril/valsartan in the inpatient setting were limited and further research is warranted.

How Sensitive is Sensitivity Analysis?: Evaluation of Pharmacoeconomic Submissions in Korea

Purpose: We aimed to describe the types of uncertainties examined in the economic evaluations submitted for reimbursement in Korea and their impact on the incremental cost-effectiveness ratio (ICER).

Method: Fifty dossiers were submitted by pharmaceutical companies to the economic subcommittee of the Pharmaceutical Benefit Coverage Advisory Committee (PBCAC) from January 2014 to December 2018. The types of uncertainties were categorized as structural and parametric, and the frequencies of the sensitivity analysis per variables were analyzed. The impact of uncertainties was measured by the percent variance of the ICER relative to that of the base case analysis.

Results: Of the 50 submissions, varying discount rate (44 submissions), followed by time horizon (38 submissions) and model assumptions (29 submissions), were most frequently used to examine structural uncertainty, while utility (42 submissions), resource use (41 submissions), and relative effectiveness (26 submissions) were used to examine parametric uncertainty. A total of 1,236 scenarios (a scenario corresponds to a case where a single variable is varied by a single range) were presented in the one-way sensitivity analyses, where parametric and structural sensitivity analyses comprised 679 and 557 scenarios, respectively. Varying drug prices had the highest impact on ICER (median variance 19.9%), followed by discount rate (12.2%), model assumptions (11.9%), extrapolation (11.8%), and time horizon (10.0%).

Conclusions: Variables related to long-term assumptions, such as model assumptions, time horizon, extrapolation, and discounting rate, were related to a high level of uncertainty. Caution should be exercised when using immature data.

Development of a conceptual model for evaluating new non-curative and curative therapies for sickle cell disease

BACKGROUND

Sickle cell disease (SCD) is a clinically heterogeneous disease with many acute and chronic complications driven by ongoing vaso-occlusion and hemolysis. It causes a disproportionate burden on Black and Hispanic communities. Our objective was to follow the SMDM/ISPOR Task Force recommendations for good practices and create a conceptual model of the progression of SCD under current clinical practice to inform cost-effectiveness analyses (CEA) of promising curative therapies in the pipeline over a lifetime horizon.

METHODS

We used consultations with experts, providers, and patients to identify acute events and chronic conditions in the conceptual model. We compared our model structure to previous CEA models of interventions for SCD, assessed the prevalence of the identified disease attributes in Medicaid and Medicare claims databases, and identified relevant outcomes following the 2nd Panel in CEA. We determined an appropriate modeling technique and relevant data sources for parameterizing the model.

RESULTS

The conceptual model structure included four dimensions of disease: chronic pain, acute events, chronic conditions, and treatment complications, spanning 26 disease attributes with significant impacts on health-related quality of life and resource. We modeled chronic pain separately to reflect its importance to patients and interaction with all other disease attributes. We identified additional data sources for health state utilities and non-medical costs and benefits of SCD. We will use a microsimulation model with age- and sex-specific transitions between health states predicted by patient demographic characteristics and disease history.

CONCLUSION

Developing the model structure through an explicit process of model conceptualization can increase the transparency and accuracy of results. We will populate the conceptual model with the data sources described and evaluate the cost-effectiveness of curative therapies.

Health variations among breast-cancer patients from different disease states: evidence from China

BACKGROUND

This study aimed to obtain health utility parameters among Chinese breast cancer patients in different disease states for subsequent health economics model. In addition, we aimed to explore the feasibility of establishing a breast cancer health utility mapping model in China.

METHODS

Multiple patient-reported health attributes were assessed, including quality of life, which was measured by the Functional Assessment of Cancer Therapy-Breast (FACT-B) instrument; health utility and self-rated health, which were measured by the EuroQol-5 Dimension-5 Level (EQ-5D-5L) questionnaire. Multivariate regression models, including a linear regression model, an ordinal logistic regression model and a Tobit model, were employed to analyze health differences among 446 breast cancer patients. Subgroup analyses were performed to examine differences in multiple dimensions of health derived from the FACT-B and EQ-5D-5L instruments. A mapping function was used to estimate health utility from quality of life. Rank correlation analyses were employed to examine the correlation between estimated and observed health utility values.

RESULTS

A total of 446 breast cancer patients with different disease states were analyzed. The health utility values of breast cancer patients in the P state (without cancer recurrence and metastasis), R state (with cancer recurrence within a year), S state (with primary and recurrent breast cancer for the second year and above), and M state (metastatic cancer) were 0.81 (SD ± 0.23), 0.90 (SD ± 0.12), 0.78 (SD ± 0.31), and 0.74 (SD ± 0.27), respectively. There were positive correlations between all scores, including every domain of the FACT-B instrument (p < 0.001). Results from multivariate analysis suggested that patients in the R and M states had lower scores for overall quality of life (R, β = - 9.45, p < 0.01; M, β = - 6.72, p < 0.05). Patients in the M state had lower health utility values than patients in the P state (β = - 0.11, p < 0.05). Estimated health utility values, which were derived from quality of life by using a mapping function, were significantly correlated with directly measured health utility values (p < 0.001).

CONCLUSIONS

We obtained the health utility and health-related quality of life (HRQoL) scores of Chinese breast cancer patients in different disease states. Mapping health utility values from quality of life using four disease states could be feasible in health economic modelling, but the mapping function may need further revision.

Structuring a conceptual model for cost-effectiveness analysis of frailty interventions

BACKGROUND

Frailty is a major health issue which impacts the life of older people, posing a significant challenge to the health system. One of the key emerging areas is the development of frailty interventions to halt or reverse the progression of the condition. In many countries, economic evidence is required to inform public funding decisions for such interventions, and cost-effectiveness models are needed to estimate long-term costs and effects. Such models should capture current clinical understanding of frailty, its progression and its health consequences. The objective of this paper is to present a conceptual model of frailty that can be used to inform the development of a cost-effectiveness model to evaluate frailty interventions.

METHODS

After critical analysis of the clinical and economic literature, a Delphi study consisting of experts from the disciplines of clinical medicine and epidemiology was undertaken to inform the key components of the conceptual model. We also identified relevant databases that can be used to populate and validate the model.

RESULTS

A list of significant health states/events for which frailty is a strong independent risk factor was identified (e.g., hip fracture, hospital admission, delirium, death). We also identified a list of important patient attributes that may influence disease progression (e.g., age, gender, previous hospital admissions, depression). A number of large-scale relevant databases were also identified to populate and validate the cost-effectiveness model. Face validity of model structure was confirmed by experts.

DISCUSSION AND CONCLUSIONS

The proposed conceptual model is being used as a basis for developing a new cost-effectiveness model to estimate lifetime costs and outcomes associated with a range of frailty interventions. Using an appropriate model structure, which more accurately reflects the natural history of frailty, will improve model transparency and accuracy. This will ultimately lead to better informed public funding decisions around interventions to manage frailty.

Systematic review of utility values used in the pharmacoeconomic evaluations for schizophrenia: implications on cost-effectiveness results

Background and Objectives: Utility elicitation studies for schizophrenia generate different utility values for the same health states. We reviewed utility values used in schizophrenia pharmacoeconomic evaluations and evaluated the impact of their selection on the incremental cost-effectiveness ratio (ICER). Methods: A systematic search was performed in Medline and Embase. Health state definitions, associated utility values, elicitation studies, and value selection processes were extracted. Sets of utility values for all schizophrenia health states were used in a cost-effectiveness model to evaluate the ICER. Results: Thirty-five cost-utility analyses (CUAs) referring to 11 utility elicitation studies were included. The most frequent health states were 'stable' (28 CUAs, 7 utility elicitation studies, 10 values, value range 0.650-0.919), 'relapse requiring hospitalisation' (18, 5, 7, 0.270-0.604), 'relapse not requiring hospitalisation' (18, 5, 10, 0.460-0.762), and 'relapse only' (10, 5, 6, 0.498-0.700). Seventeen sets of utility values were identified with difference in utility values between relapse and stable ranging from -0.358 to -0.050, resulting in ICERs ranging from -56.2% to +222.6% from average. Conclusion: The use of utility values for schizophrenia health states differs among CUAs and impacts on the ICER. More rigorous and transparent use of utility values and sensitivity analysis with different sets of utility values are suggested for future CUAs.

Multivariable and Structural Uncertainty Analyses for Cost-Effectiveness Estimates: Back to the Future

BACKGROUND

In this commentary, celebrating the 20th anniversary of the journal Value in Health, I present a brief overview and illustration of the evolution over the past 20 years of the methodological literature providing guidelines for multivariable and structural uncertainty analysis for cost-effectiveness estimates.

METHODS

To illustrate the impact of the guidelines for uncertainty analyses, I show how the inclusion of multivariable and structural uncertainty analyses in cost-effectiveness analyses published in Value in Health changed over the past 20 years using publications from 1999/2000, 2007 and 2017.

RESULTS

The commentary is organized in three sections: past, focusing on the development and use of methods for multivariable uncertainty analysis; present, focusing on the growing awareness of the need for structural uncertainty analysis, suggested frameworks for structural uncertainty analysis and how it is currently implemented; and future, considering different methods for combining multivariable and structural uncertainty analyses over the next decades.

CONCLUSIONS

I conclude by suggesting how the continued evolution of uncertainty analyses in published studies and health technology assessment submissions can best take into account an important goal of cost-effectiveness analyses: to provide useful information to decision makers.

Model Structuring for Economic Evaluations of New Health Technologies

In countries such as Australia, the UK and Canada, decisions on whether to fund new health technologies are commonly informed by decision analytic models. While the impact of making inappropriate structural choices/assumptions on model predictions is well noted, there is a lack of clarity about the definition of key structural aspects, the process of developing model structure (including the development of conceptual models) and uncertainty associated with the structuring process (structural uncertainty) in guidelines developed by national funding bodies. This forms the focus of this article. Building on the reports of good modelling practice, and recognising the fundamental role of model structuring within the model development process, we specified key structural choices and provided ideas about model structuring for the future direction. This will help to further standardise guidelines developed by national funding bodies, with potential impact on transparency, comprehensiveness and consistency of model structuring. We argue that the process of model structuring and structural sensitivity analysis should be documented in a more systematic and transparent way in submissions to national funding bodies. Within the decision-making process, the development of conceptual models and presentation of all key structural choices would mean that national funding bodies could be more confident of maximising value for money when making public funding decisions.

Critical Analysis of Markov Models Used for the Economic Evaluation of Colorectal Cancer Screening: A Systematic Review

BACKGROUND

The economic evaluation of colorectal cancer screening is challenging because of the need to model the underlying unobservable natural history of the disease.

OBJECTIVES

To describe the available Markov models and to critically analyze their main structural assumptions.

METHODS

A systematic search was performed in eight relevant databases (MEDLINE, Embase, Econlit, National Health Service Economic Evaluation Database, Health Economic Evaluations Database, Health Technology Assessment database, Cost-Effective Analysis Registry, and European Network of Health Economics Evaluation Databases), identifying 34 models that met the inclusion criteria. A comparative analysis of model structure and parameterization was conducted using two checklists and guidelines for cost-effectiveness screening models.

RESULTS

Two modeling techniques were identified. One strategy used a Markov model to reproduce the natural history of the disease and an overlaying model that reproduced the screening process, whereas the other used a single model to represent a screening program. Most of the studies included only adenoma-carcinoma sequences, a few included de novo cancer, and none included the serrated pathway. Parameterization of adenoma dwell time, sojourn time, and surveillance differed between studies, and there was a lack of validation and statistical calibration against local epidemiological data. Most of the studies analyzed failed to perform an adequate literature review and synthesis of diagnostic accuracy properties of the screening tests modeled.

CONCLUSIONS

Several strategies to model colorectal cancer screening have been developed, but many challenges remain to adequately represent the natural history of the disease and the screening process. Structural uncertainty analysis could be a useful strategy for understanding the impact of the assumptions of different models on cost-effectiveness results.

Disparities in model-based cost-effectiveness analyses of tuberculosis diagnosis: A systematic review

Background

Structural approach disparities were minimally addressed in past systematic reviews of model-based cost-effectiveness analyses addressing Tuberculosis management strategies. This review aimed to identify the structural approach disparities in model-based cost-effectiveness analysis studies addressing Tuberculosis diagnosis and describe potential hazards caused by those disparities.

Methods

A systematic search to identify studies published before October 2015 was performed in five electronic databases. After removal of duplication, studies’ titles and abstracts were screened based on predetermined criteria. The full texts of potentially relevant studies were subsequently screened and excluded when they did not address active pulmonary Tuberculosis diagnosis. Quality of the studies was assessed using the “Philips’ checklist.” Various data regarding general information, cost-effectiveness results, and disease modeling were extracted using standardized data extraction forms. Data pertaining to models’ structural approaches were compared and analyzed qualitatively for their applicability in various study settings, as well as their potential influence on main outcomes and cost-effectiveness conclusion.

Results

A total of 27 studies were included in the review. Most studies utilized a static model, which could underestimate the cost-effectiveness of the diagnostic tools strategies, due to the omission of indirect diagnosis effects, i.e. transmission reduction. A few structural assumption disparities were found in the dynamic models. Extensive disparities were found in the static models, consisting of varying structural assumptions regarding treatment outcomes, clinical diagnosis and empirical treatment, inpatient discharge decision, and re-diagnosis of false negative patients.

Conclusion

In cost-effectiveness analysis studies addressing active pulmonary Tuberculosis diagnosis, models showed numerous disparities in their structural approaches. Several structural approaches could be inapplicable in certain settings. Furthermore, they could contribute to under- or overestimation of the cost-effectiveness of the diagnosis tools or strategies. They could thus lead to ambiguities and difficulties when interpreting a study result. A set of recommendations is proposed to manage issues related to these structural disparities.

The Importance of Model Structure in the Cost-Effectiveness Analysis of Primary Care Interventions for the Management of Hypertension

BACKGROUND

Management of hypertension can lead to significant reductions in blood pressure, thereby reducing the risk of cardiovascular disease. Modeling the course of cardiovascular disease is not without complications, and uncertainty surrounding the structure of a model will almost always arise once a choice of a model structure is defined.

OBJECTIVES

To provide a practical illustration of the impact on the results of cost-effectiveness of changing or adapting model structures in a previously published cost-utility analysis of a primary care intervention for the management of hypertension Targets and Self-Management for the Control of Blood Pressure in Stroke and at Risk Groups (TASMIN-SR).

METHODS

The case study assessed the structural uncertainty arising from model structure and from the exclusion of secondary events. Four alternative model structures were implemented. Long-term cost-effectiveness was estimated and the results compared with those from the TASMIN-SR model.

RESULTS

The main cost-effectiveness results obtained in the TASMIN-SR study did not change with the implementation of alternative model structures. Choice of model type was limited to a cohort Markov model, and because of the lack of epidemiological data, only model 4 captured structural uncertainty arising from the exclusion of secondary events in the case study model.

CONCLUSIONS

The results of this study indicate that the main conclusions drawn from the TASMIN-SR model of cost-effectiveness were robust to changes in model structure and the inclusion of secondary events. Even though one of the models produced results that were different to those of TASMIN-SR, the fact that the main conclusions were identical suggests that a more parsimonious model may have sufficed.

Methodological Issues in Economic Evaluations Submitted to the Pan-Canadian Oncology Drug Review (pCODR)

BACKGROUND

Public drug plans are faced with increasingly difficult funding decisions. In Canada, the pan-Canadian Oncology Drug Review (pCODR) makes funding recommendations to the provincial and territorial drug plans responsible for cancer drugs. Assessments of the economic models submitted by pharmaceutical manufacturers are publicly reported.

OBJECTIVES

The main objective of this research was to identify recurring methodological issues in economic models submitted to pCODR for funding reviews. The secondary objective was to explore whether there exists any observed relationships between reported methodological issues and funding recommendations made by pCODR's expert review committee.

METHODS

Publicly available Economic Guidance Reports from July 2011 (inception) until June 2014 for drug reviews with a final funding recommendation (N = 34) were independently examined by two authors. Major methodological issues from each review were abstracted and grouped into nine main categories. Each issue was also categorized based on perception of the reviewer's actions to manage it.

RESULTS

The most commonly reported issues involved costing (59% of reviews), time horizon (56%), and model structure (36%). Several types of issues were identified that usually could not be resolved, such as quality of clinical data or uncertainty with indirect comparisons. Issues with costing or choice of utility estimates could usually be addressed or explored by reviewers. No statistically significant relationship was found between any methodological issue and funding recommendations from the expert review committee.

CONCLUSIONS

The findings provide insights that can be used by parties who submit or review economic evidence for continuous improvement and consistency in economic modeling, reporting, and decision making.

A systematic review of economic evaluation in pancreatic ductal adenocarcinoma

OBJECTIVES

The economic evaluation (EE) of healthcare interventions has become a necessity. However, high quality needs to be ensured in order to achieve validated results and help making informed decisions. Thus, the objective of the present study was to systematically identify and review published pancreatic ductal adenocarcinoma-related EEs and to assess their quality.

METHODS

Systematic literature research was conducted in PubMed and Cochrane to identify published EEs between 2000 and 2015. The quality of each selected EE was assessed by two independent reviewers, using the Drummond's checklist.

RESULTS

Our systematic review was based on 32 EEs and showed a wide variety of methodological approaches, including different perspectives, time horizon, and cost effectiveness analyses. Nearly two-thirds of EEs are full EEs (n = 21), and about one-third of EEs had a Drummond score ≥7, synonymous with 'high quality'. Close to 50% of full EEs had a Drummond score ≥7, whereas all of partial EEs had a Drummond score <7 (n = 11).

CONCLUSIONS

Over the past 15 years, a lot of interest has been evinced over the EE of pancreatic ductal adenocarcinoma (PDAC) and its direct impact on therapeutic advances in PDAC. To provide a framework for health care decision-making, to facilitate transferability and to lend credibility to health EEs, their quality must be improved. For the last 4 years, a tendency towards a quality improvement of these studies has been observed, probably coupled with a context of rational decision-making in health care, a better and wider spread of recommendations and thus, medical practitioners' full endorsement.

Using Parameter Constraints to Choose State Structures in Cost-Effectiveness Modelling

BACKGROUND

This article addresses the choice of state structure in a cost-effectiveness multi-state model. Key model outputs, such as treatment recommendations and prioritisation of future research, may be sensitive to state structure choice. For example, it may be uncertain whether to consider similar disease severities or similar clinical events as the same state or as separate states. Standard statistical methods for comparing models require a common reference dataset but merging states in a model aggregates the data, rendering these methods invalid.

METHODS

We propose a method that involves re-expressing a model with merged states as a model on the larger state space in which particular transition probabilities, costs and utilities are constrained to be equal between states. This produces a model that gives identical estimates of cost effectiveness to the model with merged states, while leaving the data unchanged. The comparison of state structures can be achieved by comparing maximised likelihoods or information criteria between constrained and unconstrained models. We can thus test whether the costs and/or health consequences for a patient in two states are the same, and hence if the states can be merged. We note that different structures can be used for rates, costs and utilities, as appropriate.

APPLICATION

We illustrate our method with applications to two recent models evaluating the cost effectiveness of prescribing anti-depressant medications by depression severity and the cost effectiveness of diagnostic tests for coronary artery disease.

CONCLUSIONS

State structures in cost-effectiveness models can be compared using standard methods to compare constrained and unconstrained models.

Reviewing the quality, health benefit and value for money of chemotherapy and targeted therapy for metastatic breast cancer

Purpose

To provide an overview of model characteristics and outcomes of model-based economic evaluations concerning chemotherapy and targeted therapy (TT) for metastatic breast cancer (MBC); to assess the quality of the studies; to analyse the association between model characteristics and study quality and outcomes.

Methods

PubMED and NHS EED were systematically searched. Inclusion criteria were as follows: English or Dutch language, model-based economic evaluation, chemotherapy or TT as intervention, population diagnosed with MBC, published between 2000 and 2014, reporting life years (LY) or quality-adjusted life-year (QALY) and an incremental cost-effectiveness ratio. General characteristics, model characteristics and outcomes of the studies were extracted. Quality of the studies was assessed through a checklist.

Results

24 studies were included, considering 50 comparisons (20 concerning chemotherapy and 30 TT). Seven comparisons were represented in multiple studies. A health state-transition model including the following health states: stable/progression-free disease, progression and death was used in 18 studies. Studies fulfilled on average 14 out of the 26 items of the quality checklist, mostly due to a lack of transparency in reporting. Thirty-one per cent of the incremental net monetary benefit was positive. TT led to higher iQALY gained, and industry-sponsored studies reported more favourable cost-effectiveness outcomes.

Conclusions

The development of a disease-specific reference model would improve the transparency and quality of model-based cost-effectiveness assessments for MBC treatments. Incremental health benefits increased over time, but were outweighed by the increased treatment costs. Consequently, increased health benefits led to lower value for money.

Electronic supplementary material

The online version of this article (doi:10.1007/s10549-017-4374-6) contains supplementary material, which is available to authorized users.

Cost-utility analyses of drug therapies in breast cancer: a systematic review

The economic evaluation (EE) of health care products has become a necessity. Their quality must be high in order to trust the results and make informed decisions. While cost-utility analyses (CUAs) should be preferred to cost-effectiveness analyses in the oncology area, the quality of breast cancer (BC)-related CUA has been given little attention so far. Thus, firstly, a systematic review of published CUA related to drug therapies for BC, gene expression profiling, and HER2 status testing was performed. Secondly, the quality of selected CUA was assessed and the factors associated with a high-quality CUA identified. The systematic literature search was conducted in PubMed, MEDLINE/EMBASE, and Cochrane to identify published CUA between 2000 and 2014. After screening and data extraction, the quality of each selected CUA was assessed by two independent reviewers, using the checklist proposed by Drummond et al. The analysis of factors associated with a high-quality CUA (defined as a Drummond score ≥7) was performed using a two-step approach. Our systematic review was based on 140 CUAs and showed a wide variety of methodological approaches, including differences in the perspective adopted, the time horizon, measurement of cost and effectiveness, and more specially health-state utility values (HSUVs). The median Drummond score was 7 [range 3-10]. Only one in two of the CUA (n = 74) had a Drummond score ≥7, synonymous of "high quality." The statistically significant predictors of a high-quality CUA were article with "gene expression profiling" topic (p = 0.001), consulting or pharmaceutical company as main location of first author (p = 0.004), and articles with both incremental cost-utility ratio and incremental cost-effectiveness ratio as outcomes of EE (p = 0.02). Our systematic review identified only 140 CUAs published over the past 15 years with one in two of high quality. It showed a wide variety of methodological approaches, especially focused on HSUVs. A critical appraisal of utility values is necessary to better understand one of the main difficulties encountered by authors and propose areas for improvement to increase the quality of CUA. Since the last 5 years, there is a tendency toward an improvement in the quality of these studies, probably coupled with economic context, a better and widely spreading of recommendations and thus appropriation by medical practitioners. That being said, there is an urgent need for mandatory use of European and international recommendations to ensure quality of such approaches and to allow easy comparison.

Exploring Uncertainty in Economic Evaluations of Drugs and Medical Devices: Lessons from the First Review of Manufacturers' Submissions to the French National Authority for Health

OBJECTIVES

The objective of this paper was to evaluate how uncertainty has been accounted for in the cost-effectiveness analyses (CEAs) submitted by manufacturers to the French National Authority for Health (HAS) and to identify recurring concerns in these submissions.

METHODS

We used a cross-sectional design to evaluate manufacturers' submissions from the beginning of the evaluation process in October 2013 to the end of May 2015 (n = 28). The sources of uncertainty attached to these CEAs were categorized and assessed. Relevant data were extracted independently by two assessors.

RESULTS

Adherence to the HAS reference case was generally considered to be acceptable. Methodological uncertainty and parameter uncertainty were the sources of uncertainty that were most frequently explored by manufacturers. The quality of reporting of deterministic sensitivity analysis and probabilistic sensitivity analysis varied substantially across submissions, with a frequent lack of justification of the plausible range of parameter point estimates in 12 submissions (43 %). Structural uncertainty was explored much less frequently. Concerns related to omission of either important clinical events or relevant health states or extrapolation of the effects of the technology beyond the time horizon of the clinical trials were identified in 16 submissions (57 %).

CONCLUSIONS

This study presented a characterization of the treatment of uncertainty for the first 28 manufacturers' submissions to the HAS. This work identified important concerns regarding the exploration of sources of uncertainty. The findings may help manufacturers to improve the quality of their submissions and may provide useful insights for extending guidelines on uncertainty analysis in CEAs submitted to the HAS.

Use of quality checklists and need for disease-specific guidance in economic evaluations: a meta-review

Economic evaluations have become an essential part of reimbursement decisions in a wide range of countries. To ensure high quality, a variety of checklists with different purposes have been developed and implemented enabling assessment of these evaluations. Three of these checklists are most frequently used and are recommended by the Cochrane Handbook for Systematic Reviews for critical appraisal (Drummond, CHEC and Philips). Every checklist is developed with a different purpose having, for example, a focus on reporting or conducting and on modeling or trial-based evaluations. This review outlines the heterogeneity in choice and implementation of these quality checklists in an incorrect manner. This ultimately results in under- and even possibly overestimation of quality of included economic evaluations. More guidance in selecting correct checklists suiting the purpose of the quality check is therefore of utmost importance. Moreover, it appears that current checklists are lacking detailed disease-specific guidance resulting in models not correctly reflecting disease progression. Therefore, outcomes indicate that the problem of the wide variability of methodological choices is prevalent in some other disease areas too, regardless of the availability of quality checklists. More international collaboration should therefore be initiated in developing and publishing standardized and open source disease-specific reference models to overcome this problem.

Structural Uncertainty of Markov Models for Advanced Breast Cancer

Objective. To examine the impact of structural uncertainty of Markov models in modeling cost-effectiveness for the treatment of advanced breast cancer (ABC). Methods. Four common Markov models for ABC were identified and examined. Markov models 1 and 2 have 4 health states (stable-disease, responding-to-therapy, disease-progressing, and death), and Markov models 3 and 4 only have 3 health states (stable-disease, disease-progressing, and death). In models 1 and 3, the possibility of death can occur in any health state, while in models 2 and 4, the chance of dying can only occur in the disease-progressing health state. A simulation was conducted to examine the impact of using different model structures on cost-effectiveness results in the context of a combination therapy of lapatinib and capecitabine for the treatment of HER2-positive ABC. Model averaging with an assumption of equal weights in all 4 models was used to account for structural uncertainty. Results. Markov model 3 yielded the lowest incremental cost-effectiveness ratio (ICER) of $303,909 per quality-adjusted life year (QALY), while Markov model 1 produced the highest ICER ($495,800/QALY). At a willingness-to-pay threshold of $150,000/QALY, the probabilities that the combination therapy is considered to be cost-effective for Markov models 1, 2, 3, and 4 were 14.5%, 14.1%, 21.6%, and 17.0%, respectively. When using model averaging to synthesize different model structures, the resulting ICER was $389,270/QALY. Conclusions. Our study shows that modeling ABC with different Markov model structures yielded a wide range of cost-effectiveness results, suggesting the need to investigate structural uncertainty in health economic evaluation. When applied in the context of HER2-positive ABC treatment, the combination therapy with lapatinib is not cost-effective, regardless of which model was used and whether uncertainties were accounted for.

Value of Information Analysis Applied to the Economic Evaluation of Interventions Aimed at Reducing Juvenile Delinquency: An Illustration

Objectives

To investigate whether a value of information analysis, commonly applied in health care evaluations, is feasible and meaningful in the field of crime prevention.

Methods

Interventions aimed at reducing juvenile delinquency are increasingly being evaluated according to their cost-effectiveness. Results of cost-effectiveness models are subject to uncertainty in their cost and effect estimates. Further research can reduce that parameter uncertainty. The value of such further research can be estimated using a value of information analysis, as illustrated in the current study. We built upon an earlier published cost-effectiveness model that demonstrated the comparison of two interventions aimed at reducing juvenile delinquency. Outcomes were presented as costs per criminal activity free year.

Results

At a societal willingness-to-pay of €71,700 per criminal activity free year, further research to eliminate parameter uncertainty was valued at €176 million. Therefore, in this illustrative analysis, the value of information analysis determined that society should be willing to spend a maximum of €176 million in reducing decision uncertainty in the cost-effectiveness of the two interventions. Moreover, the results suggest that reducing uncertainty in some specific model parameters might be more valuable than in others.

Conclusions

Using a value of information framework to assess the value of conducting further research in the field of crime prevention proved to be feasible. The results were meaningful and can be interpreted according to health care evaluation studies. This analysis can be helpful in justifying additional research funds to further inform the reimbursement decision in regard to interventions for juvenile delinquents.

Integrated Simulation Framework for Toxicity, Dose Intensity, Disease Progression, and Cost Effectiveness for Castration-Resistant Prostate Cancer Treatment With Eribulin

Quantitative model-based analyses are helpful to support decision-making in drug development. In oncology, disease progression/clinical outcome (DPCO) models have been used for early predictions of clinical outcome, but most of such approaches did not include adverse events or dose intensity. In addition, cost-effectiveness evaluations of investigational compounds are becoming increasingly important. Here, we developed an integrated model-based framework including relevant treatment effects for patients with castration-resistant prostate cancer treated with the anticancer agent eribulin. The framework included (i) a DPCO model relating prostate-specific antigen (PSA) dynamics to survival; (ii) models for adverse events including dose-limiting neutropenia and other graded toxicities; (iii) a model for Eastern Cooperative Oncology Group (ECOG) performance score; (iv) a model for dropout; (v) the consideration of cost effectiveness. The model allowed simulation of realistic treatment courses. Subsequently, simulations evaluating alternative treatment protocols or patient characteristics were performed in order to derive inferences on expected efficacy and cost effectiveness.

Exploring structural uncertainty in model-based economic evaluations

Given the inherent uncertainty in estimates produced by decision analytic models, the assessment of uncertainty in model-based evaluations is an essential part of the decision-making process. Although the impact of uncertainty around the choice of model structure and making incorrect structural assumptions on model predictions is noted, relatively little attention has been paid to characterising this type of uncertainty in guidelines developed by national funding bodies such as the Australian Pharmaceutical Benefits Advisory Committee (PBAC). The absence of a detailed description and evaluation of structural uncertainty can add further uncertainty to the decision-making process, with potential impact on the quality of funding decisions. This paper provides a summary of key elements of structural uncertainty describing why it matters and how it could be characterised. Five alternative approaches to characterising structural uncertainty are discussed, including scenario analysis, model selection, model averaging, parameterization and discrepancy. We argue that the potential effect of structural uncertainty on model predictions should be considered in submissions to national funding bodies; however, the characterisation of structural uncertainty is not well defined within the guidelines of these bodies. There has been little consideration of the forms of structural sensitivity analysis that might best inform applied decision-making processes, and empirical research in this area is required.

Decision-analytic models: current methodological challenges

Modelers seeking to help inform decisions about insurance (public or private) coverage of the cost of pharmaceuticals or other health care interventions face various methodological challenges. In this review, which is not meant to be comprehensive, we cover those that in our experience are most vexing. The biggest challenge is getting decision makers to trust the model. This is a major problem because most models undergo only cursory validation; our field has lacked the motivation, time, and data to properly validate models intended to inform health care decisions. Without documented, adequate validation, there is little basis for decision makers to have confidence that the model's results are credible and should be used in a health technology appraisal. A fundamental problem for validation is that the models are very artificial and lack sufficient depth to adequately represent the reality they are simulating. Typically, modelers assume that all resources have infinite capacity so any patient needing care receives it immediately; there are no waiting times or queues, contrary to the common experience in actual practice. Moreover, all the patients enter the model simultaneously at time zero rather than over time as happens in actuality; differences between patients are ignored or minimized and structural modeling choices that make little sense (e.g., using states to represent events) are forced by commitment to a technique (and even to specific spreadsheet software!). The resulting structural uncertainty is rarely addressed, because methods are lacking and even probabilistic analysis of parameter uncertainty suffers from weak consideration of correlation and arbitrary distribution choices. Stakeholders must see to it that models are fit for the stated purpose and provide the best possible estimates given available data-the decisions at stake deserve nothing less.