A Blueprint for Multi-use Disease Modeling in Health Economics: Results from Two Expert-Panel Consultations

BACKGROUND

The current use of health economic decision models in HTA is mostly confined to single use cases, which may be inefficient and result in little consistency over different treatment comparisons, and consequently inconsistent health policy decisions, for the same disorder. Multi-use disease models (MUDMs) (other terms: generic models, whole disease models, disease models) may offer a solution. However, much is uncertain about their definition and application. The current research aimed to develop a blueprint for the application of MUDMs.

METHODS

We elicited expert opinion using a two-round modified Delphi process. The panel consisted of experts and stakeholders in health economic modelling from various professional backgrounds. The first questionnaire concerned definition, terminology, potential applications, issues and recommendations for MUDMs and was based on an exploratory scoping review. In the second round, the panel members were asked to reconsider their input, based on feedback regarding first-round results, and to score issues and recommendations for priority. Finally, adding input from external advisors and policy makers in a structured way, an overview of issues and challenges was developed during two team consensus meetings.

RESULTS

In total, 54 respondents contributed to the panel results. The term 'multi-use disease models' was proposed and agreed upon, and a definition was provided. The panel prioritized 10 potential applications (with comparing alternative policies and supporting resource allocation decisions as the top 2), while 20 issues (with model transparency and stakeholders' roles as the top 2) were identified as challenges. Opinions on potential features concerning operationalization of multi-use models were given, with 11 of these subsequently receiving high priority scores (regular updates and revalidation after updates were the top 2).

CONCLUSIONS

MUDMs would improve on current decision support regarding cost-effectiveness information. Given feasibility challenges, this would be most relevant for diseases with multiple treatments, large burden of disease and requiring more complex models. The current overview offers policy makers a starting point to organize the development, use, and maintenance of MUDMs and to support choices concerning which diseases and policy decisions they will be helpful for.

A Prototype Software Framework for Transferable Computational Health Economic Models and Its Early Application in Youth Mental Health

We are developing an economic model to explore multiple topics in Australian youth mental health policy. To help make that model more readily transferable to other jurisdictions, we developed a software framework for authoring modular computational health economic models (CHEMs) (the software files that implement health economic models). We specified framework user requirements for: a simple programming syntax; a template CHEM module; tools for authoring new CHEM modules; search tools for finding existing CHEM modules; tools for supplying CHEM modules with data; reproducible analysis and reporting tools; and tools to help maintain a CHEM project website. We implemented the framework as six development version code libraries in the programming language R that integrate with online services for software development and research data archiving. We used the framework to author five development version R libraries of CHEM modules focussed on utility mapping in youth mental health. These modules provide tools for variable validation, dataset description, multi-attribute instrument scoring, construction of mapping models, reporting of mapping studies and making out of sample predictions. We assessed these CHEM module libraries as mostly meeting transparency, reusability and updatability criteria that we have previously developed, but requiring more detailed documentation and unit testing of individual modules. Our software framework has potential value as a prototype for future tools to support the development of transferable CHEMs.Code: Visit https://www.ready4-dev.com for more information about how to find, install and apply the prototype software framework.

Challenges in the Evaluation of Emerging Highly Specialised Technologies: Is There a Role for Living HTA?

There is currently deep uncertainty about the clinical benefits and cost effectiveness of highly specialised technologies (HSTs), like gene and cell therapies. These treatments are novel, typically have high upfront costs, the patient populations are small and heterogenous, there is minimal information on their long-term safety and effectiveness, and data are limited and often of poor quality. With the increasing number of these technologies and their high cost burden on governments and health care providers, policy makers are currently walking a decision tightrope. On the one hand, an unfavourable funding decision could potentially limit patient access to life-saving treatments, while on the other, a favourable decision could result in unsustainable budget impacts and perhaps poorer patient health outcomes. Health technology assessment (HTA) is meant to determine the value of a health technology in order to promote an equitable, efficient, and high-quality health system. However, standard HTA processes have failed to mitigate the deep uncertainties associated with these technologies. In this paper, we propose a Living HTA framework to address these challenges. This framework includes a one-off process for making explicit the societal values associated with HSTs. These would inform the decision-making approach, data collection and the development of disease-specific reference models to be used by industry sponsors as the basis for their submissions for public funding. Coverage with an evidence development mechanism is also proposed by which data can be collected in real time to update the reference model on a rolling basis, thereby allowing re-assessment of the clinical and cost effectiveness of individual HSTs. The HTA would be 'live' until the results indicate there is sufficient certainty for the funding decision to be confirmed, the price changed or the funding removed.

Four Aspects Affecting Health Economic Decision Models and Their Validation

Health care decision makers in many jurisdictions use cost-effectiveness analysis based on health economic decision models for policy decisions regarding coverage and price negotiation for medicines and medical devices. While validation of health economic decision models has always been considered important, many reviews of model-based cost-effectiveness studies report limitations regarding their validation. The current opinion paper discusses four aspects of current health economic decision modeling with relevance for future directions in model validation: increased use of complex models, international cooperation, open-source modeling, and stakeholder involvement. First, new, more complex clinical study designs and treatment strategies may require relatively complex model structures and/or input data analyses. Simultaneously, more widespread technical knowledge along with wider data availability have led to a broader range of model types. This puts extra requirements on model validation and transparency. Second, increased international cooperation of policy makers and, in particular, health technology assessment (HTA) authorities in performing model assessments is discussed in relation to the repeated use of health economic models (multi-use disease models). We argue such coordinated efforts may benefit model validity. Third, open-source modeling is discussed as one possible answer to increased transparency requirements. Finally, involvement of all relevant stakeholders throughout the whole decision process is an ongoing development that necessarily also includes health economic modeling. We argue this implies that model validity should be considered in a broader perspective, with more focus on conceptual modeling, model transparency, accuracy requirements, and choice of relevant model outcomes than previously.

Development of a conceptual model of childhood asthma to inform asthma prevention policies

Background

There is no definitive cure for asthma, as prevention remains a major goal. Decision analytic models are routinely used to evaluate the value-for-money proposition of interventions. Following best practice standards in decision-analytic modelling, the objective of this study was to solicit expert opinion to develop a concept map for a policy model for primary prevention of asthma.

Methods

We reviewed currently available decision analytic models for asthma prevention. A steering committee of economic modellers, allergists and respirologists was then convened to draft a conceptual model of paediatric asthma. A modified Delphi method was followed to define the context of the problem at hand (evaluation of asthma prevention strategies) and develop the concept map of the model.

Results

Consensus was achieved after three rounds of discussions, followed by concealed voting. In the final conceptual model, asthma diagnosis was based on three domains of lung function, atopy and their symptoms. The panel recommended several markers for each domain. These domains were in turn affected by several risk factors. The panel clustered all risk factors under three groups of ‘patient characteristic’, ‘family history’ and ‘environmental factors’. To be capable of modelling the interplay among risk factors, the panel recommended the use of microsimulation, with an open-population approach that would enable modelling phased implementation and gradual and incomplete uptake of the intervention.

Conclusions

Economic evaluation of childhood interventions for preventing asthma will require modelling of several codependent risk factors and multiple domains that affect the diagnosis. The conceptual model can inform the development and validation of a policy model for childhood asthma prevention.

Mapping of the World Health Organization's Disability Assessment Schedule 2.0 to disability weights using the Multi-Country Survey Study on Health and Responsiveness

OBJECTIVES

To develop and test an internationally applicable mapping function for converting WHODAS-2.0 scores to disability weights, thereby enabling WHODAS-2.0 to be used in cost-utility analyses and sectoral decision-making.

METHODS

Data from 14 countries were used from the WHO Multi-Country Survey Study on Health and Responsiveness, administered among nationally representative samples of respondents aged 18+ years who were non-institutionalized and living in private households. For the combined total of 92,006 respondents, available WHODAS-2.0 items (for both 36-item and 12-item versions) were mapped onto disability weight estimates using a machine learning approach, whereby data were split into separate training and test sets; cross-validation was used to compare the performance of different regression and penalized regression models. Sensitivity analyses considered different imputation strategies and compared overall model performance with that of country-specific models.

RESULTS

Mapping functions converted WHODAS-2.0 scores into disability weights; R-squared values of 0.700-0.754 were obtained for the test data set. Penalized regression models reached comparable performance to standard regression models but with fewer predictors. Imputation had little impact on model performance. Model performance of the generic model on country-specific test sets was comparable to model performance of country-specific models.

CONCLUSIONS

Disability weights can be generated with good accuracy using WHODAS 2.0 scores, including in national settings where health state valuations are not directly available, which signifies the utility of WHODAS as an outcome measure in evaluative studies that express intervention benefits in terms of QALYs gained.

Transparency in Decision Modelling: What, Why, Who and How?

Transparency in decision modelling is an evolving concept. Recently, discussion has moved from reporting standards to open-source implementation of decision analytic models. However, in the debate about the supposed advantages and disadvantages of greater transparency, there is a lack of definition. The purpose of this article is not to present a case for or against transparency, but rather to provide a more nuanced understanding of what transparency means in the context of decision modelling and how it could be addressed. To this end, we review and summarise the discourse to date, drawing on our collective experience. We outline a taxonomy of the different manifestations of transparency, including reporting standards, reference models, collaboration, model registration, peer review and open-source modelling. Further, we map out the role and incentives for the various stakeholders, including industry, research organisations, publishers and decision makers. We outline the anticipated advantages and disadvantages of greater transparency with respect to each manifestation, as well as the perceived barriers and facilitators to greater transparency. These are considered with respect to the different stakeholders and with reference to issues including intellectual property, legality, standards, quality assurance, code integrity, health technology assessment processes, incentives, funding, software, access and deployment options, data protection and stakeholder engagement. For each manifestation of transparency, we discuss the 'what', 'why', 'who' and 'how'. Specifically, their meaning, why the community might (or might not) wish to embrace them, whose engagement as stakeholders is required and how relevant objectives might be realised. We identify current initiatives aimed to improve transparency to exemplify efforts in current practice and for the future.

Structuring a conceptual model for cost-effectiveness analysis of frailty interventions

BACKGROUND

Frailty is a major health issue which impacts the life of older people, posing a significant challenge to the health system. One of the key emerging areas is the development of frailty interventions to halt or reverse the progression of the condition. In many countries, economic evidence is required to inform public funding decisions for such interventions, and cost-effectiveness models are needed to estimate long-term costs and effects. Such models should capture current clinical understanding of frailty, its progression and its health consequences. The objective of this paper is to present a conceptual model of frailty that can be used to inform the development of a cost-effectiveness model to evaluate frailty interventions.

METHODS

After critical analysis of the clinical and economic literature, a Delphi study consisting of experts from the disciplines of clinical medicine and epidemiology was undertaken to inform the key components of the conceptual model. We also identified relevant databases that can be used to populate and validate the model.

RESULTS

A list of significant health states/events for which frailty is a strong independent risk factor was identified (e.g., hip fracture, hospital admission, delirium, death). We also identified a list of important patient attributes that may influence disease progression (e.g., age, gender, previous hospital admissions, depression). A number of large-scale relevant databases were also identified to populate and validate the cost-effectiveness model. Face validity of model structure was confirmed by experts.

DISCUSSION AND CONCLUSIONS

The proposed conceptual model is being used as a basis for developing a new cost-effectiveness model to estimate lifetime costs and outcomes associated with a range of frailty interventions. Using an appropriate model structure, which more accurately reflects the natural history of frailty, will improve model transparency and accuracy. This will ultimately lead to better informed public funding decisions around interventions to manage frailty.

Multivariable and Structural Uncertainty Analyses for Cost-Effectiveness Estimates: Back to the Future

BACKGROUND

In this commentary, celebrating the 20th anniversary of the journal Value in Health, I present a brief overview and illustration of the evolution over the past 20 years of the methodological literature providing guidelines for multivariable and structural uncertainty analysis for cost-effectiveness estimates.

METHODS

To illustrate the impact of the guidelines for uncertainty analyses, I show how the inclusion of multivariable and structural uncertainty analyses in cost-effectiveness analyses published in Value in Health changed over the past 20 years using publications from 1999/2000, 2007 and 2017.

RESULTS

The commentary is organized in three sections: past, focusing on the development and use of methods for multivariable uncertainty analysis; present, focusing on the growing awareness of the need for structural uncertainty analysis, suggested frameworks for structural uncertainty analysis and how it is currently implemented; and future, considering different methods for combining multivariable and structural uncertainty analyses over the next decades.

CONCLUSIONS

I conclude by suggesting how the continued evolution of uncertainty analyses in published studies and health technology assessment submissions can best take into account an important goal of cost-effectiveness analyses: to provide useful information to decision makers.

Trusting the Results of Model-Based Economic Analyses: Is there a Pragmatic Validation Solution?

Models have become a nearly essential component of health technology assessment. This is because the efficacy and safety data available from clinical trials are insufficient to provide the required estimates of impact of new interventions over long periods of time and for other populations and subgroups. Despite more than five decades of use of these decision-analytic models, decision makers are still often presented with poorly validated models and thus trust in their results is impaired. Among the reasons for this vexing situation are the artificial nature of the models, impairing their validation against observable data, the complexity in their formulation and implementation, the lack of data against which to validate the model results, and the challenges of short timelines and insufficient resources. This article addresses this crucial problem of achieving models that produce results that can be trusted and the resulting requirements for validation and transparency, areas where our field is currently deficient. Based on their differing perspectives and experiences, the authors characterize the situation and outline the requirements for improvement and pragmatic solutions to the problem of inadequate validation.

The Importance of Model Structure in the Cost-Effectiveness Analysis of Primary Care Interventions for the Management of Hypertension

BACKGROUND

Management of hypertension can lead to significant reductions in blood pressure, thereby reducing the risk of cardiovascular disease. Modeling the course of cardiovascular disease is not without complications, and uncertainty surrounding the structure of a model will almost always arise once a choice of a model structure is defined.

OBJECTIVES

To provide a practical illustration of the impact on the results of cost-effectiveness of changing or adapting model structures in a previously published cost-utility analysis of a primary care intervention for the management of hypertension Targets and Self-Management for the Control of Blood Pressure in Stroke and at Risk Groups (TASMIN-SR).

METHODS

The case study assessed the structural uncertainty arising from model structure and from the exclusion of secondary events. Four alternative model structures were implemented. Long-term cost-effectiveness was estimated and the results compared with those from the TASMIN-SR model.

RESULTS

The main cost-effectiveness results obtained in the TASMIN-SR study did not change with the implementation of alternative model structures. Choice of model type was limited to a cohort Markov model, and because of the lack of epidemiological data, only model 4 captured structural uncertainty arising from the exclusion of secondary events in the case study model.

CONCLUSIONS

The results of this study indicate that the main conclusions drawn from the TASMIN-SR model of cost-effectiveness were robust to changes in model structure and the inclusion of secondary events. Even though one of the models produced results that were different to those of TASMIN-SR, the fact that the main conclusions were identical suggests that a more parsimonious model may have sufficed.

Methodological Issues in Economic Evaluations Submitted to the Pan-Canadian Oncology Drug Review (pCODR)

BACKGROUND

Public drug plans are faced with increasingly difficult funding decisions. In Canada, the pan-Canadian Oncology Drug Review (pCODR) makes funding recommendations to the provincial and territorial drug plans responsible for cancer drugs. Assessments of the economic models submitted by pharmaceutical manufacturers are publicly reported.

OBJECTIVES

The main objective of this research was to identify recurring methodological issues in economic models submitted to pCODR for funding reviews. The secondary objective was to explore whether there exists any observed relationships between reported methodological issues and funding recommendations made by pCODR's expert review committee.

METHODS

Publicly available Economic Guidance Reports from July 2011 (inception) until June 2014 for drug reviews with a final funding recommendation (N = 34) were independently examined by two authors. Major methodological issues from each review were abstracted and grouped into nine main categories. Each issue was also categorized based on perception of the reviewer's actions to manage it.

RESULTS

The most commonly reported issues involved costing (59% of reviews), time horizon (56%), and model structure (36%). Several types of issues were identified that usually could not be resolved, such as quality of clinical data or uncertainty with indirect comparisons. Issues with costing or choice of utility estimates could usually be addressed or explored by reviewers. No statistically significant relationship was found between any methodological issue and funding recommendations from the expert review committee.

CONCLUSIONS

The findings provide insights that can be used by parties who submit or review economic evidence for continuous improvement and consistency in economic modeling, reporting, and decision making.

Reviewing the quality, health benefit and value for money of chemotherapy and targeted therapy for metastatic breast cancer

Purpose

To provide an overview of model characteristics and outcomes of model-based economic evaluations concerning chemotherapy and targeted therapy (TT) for metastatic breast cancer (MBC); to assess the quality of the studies; to analyse the association between model characteristics and study quality and outcomes.

Methods

PubMED and NHS EED were systematically searched. Inclusion criteria were as follows: English or Dutch language, model-based economic evaluation, chemotherapy or TT as intervention, population diagnosed with MBC, published between 2000 and 2014, reporting life years (LY) or quality-adjusted life-year (QALY) and an incremental cost-effectiveness ratio. General characteristics, model characteristics and outcomes of the studies were extracted. Quality of the studies was assessed through a checklist.

Results

24 studies were included, considering 50 comparisons (20 concerning chemotherapy and 30 TT). Seven comparisons were represented in multiple studies. A health state-transition model including the following health states: stable/progression-free disease, progression and death was used in 18 studies. Studies fulfilled on average 14 out of the 26 items of the quality checklist, mostly due to a lack of transparency in reporting. Thirty-one per cent of the incremental net monetary benefit was positive. TT led to higher iQALY gained, and industry-sponsored studies reported more favourable cost-effectiveness outcomes.

Conclusions

The development of a disease-specific reference model would improve the transparency and quality of model-based cost-effectiveness assessments for MBC treatments. Incremental health benefits increased over time, but were outweighed by the increased treatment costs. Consequently, increased health benefits led to lower value for money.

Electronic supplementary material

The online version of this article (doi:10.1007/s10549-017-4374-6) contains supplementary material, which is available to authorized users.

Validation and calibration of structural models that combine information from multiple sources

INTRODUCTION

Mathematical models that attempt to capture structural relationships between their components and combine information from multiple sources are increasingly used in medicine. Areas covered: We provide an overview of methods for model validation and calibration and survey studies comparing alternative approaches. Expert commentary: Model validation entails a confrontation of models with data, background knowledge, and other models, and can inform judgments about model credibility. Calibration involves selecting parameter values to improve the agreement of model outputs with data. When the goal of modeling is quantitative inference on the effects of interventions or forecasting, calibration can be viewed as estimation. This view clarifies issues related to parameter identifiability and facilitates formal model validation and the examination of consistency among different sources of information. In contrast, when the goal of modeling is the generation of qualitative insights about the modeled phenomenon, calibration is a rather informal process for selecting inputs that result in model behavior that roughly reproduces select aspects of the modeled phenomenon and cannot be equated to an estimation procedure. Current empirical research on validation and calibration methods consists primarily of methodological appraisals or case-studies of alternative techniques and cannot address the numerous complex and multifaceted methodological decisions that modelers must make. Further research is needed on different approaches for developing and validating complex models that combine evidence from multiple sources.

Do Model-Based Studies in Chronic Obstructive Pulmonary Disease Measure Correct Values of Utility? A Meta-Analysis

BACKGROUND

Chronic obstructive pulmonary disease (COPD) is a progressive chronic disease that has considerable impact on utility-based health-related quality of life. Utility is a key input of many decision analytic models used for economic evaluations.

OBJECTIVE

To systematically review COPD-related utilities and to compare these with alternative values used in decision models.

METHODS

The literature review comprised studies that generated utilities for COPD-related stages based on EuroQol five-dimensional questionnaire surveys of patients and of decision models of COPD progression that have been used for economic evaluations. The utility values used in modeling studies and those from the meta-analysis of actual patient-level studies were compared and differences quantified.

RESULTS

Twenty decision modeling studies that used utility value as an input parameter were found. Within the same span of publication period, 13 studies involving patient-level utility data were identified and included in the meta-analysis. The estimated mean utility values ranged from 0.806 (95% confidence interval [CI] 0.747-0.866) for stage I to 0.616 (95% CI 0.556-0.676) for stage IV. The utility scores for comparable stages in modeling studies were different (significant difference 0.045 [95% CI 0.041-0.052] for stage III). Modeling studies consistently used higher utility values than the average reported patient-level data.

CONCLUSIONS

COPD decision analytic models are based on a limited range of utility values that are systematically different from average values estimated using a meta-analysis. A more systematic approach in the application of utility measures in economic evaluation is required to appropriately reflect current literature.

Use of quality checklists and need for disease-specific guidance in economic evaluations: a meta-review

Economic evaluations have become an essential part of reimbursement decisions in a wide range of countries. To ensure high quality, a variety of checklists with different purposes have been developed and implemented enabling assessment of these evaluations. Three of these checklists are most frequently used and are recommended by the Cochrane Handbook for Systematic Reviews for critical appraisal (Drummond, CHEC and Philips). Every checklist is developed with a different purpose having, for example, a focus on reporting or conducting and on modeling or trial-based evaluations. This review outlines the heterogeneity in choice and implementation of these quality checklists in an incorrect manner. This ultimately results in under- and even possibly overestimation of quality of included economic evaluations. More guidance in selecting correct checklists suiting the purpose of the quality check is therefore of utmost importance. Moreover, it appears that current checklists are lacking detailed disease-specific guidance resulting in models not correctly reflecting disease progression. Therefore, outcomes indicate that the problem of the wide variability of methodological choices is prevalent in some other disease areas too, regardless of the availability of quality checklists. More international collaboration should therefore be initiated in developing and publishing standardized and open source disease-specific reference models to overcome this problem.

Exploring structural uncertainty in model-based economic evaluations

Given the inherent uncertainty in estimates produced by decision analytic models, the assessment of uncertainty in model-based evaluations is an essential part of the decision-making process. Although the impact of uncertainty around the choice of model structure and making incorrect structural assumptions on model predictions is noted, relatively little attention has been paid to characterising this type of uncertainty in guidelines developed by national funding bodies such as the Australian Pharmaceutical Benefits Advisory Committee (PBAC). The absence of a detailed description and evaluation of structural uncertainty can add further uncertainty to the decision-making process, with potential impact on the quality of funding decisions. This paper provides a summary of key elements of structural uncertainty describing why it matters and how it could be characterised. Five alternative approaches to characterising structural uncertainty are discussed, including scenario analysis, model selection, model averaging, parameterization and discrepancy. We argue that the potential effect of structural uncertainty on model predictions should be considered in submissions to national funding bodies; however, the characterisation of structural uncertainty is not well defined within the guidelines of these bodies. There has been little consideration of the forms of structural sensitivity analysis that might best inform applied decision-making processes, and empirical research in this area is required.

Decision-analytic models: current methodological challenges

Modelers seeking to help inform decisions about insurance (public or private) coverage of the cost of pharmaceuticals or other health care interventions face various methodological challenges. In this review, which is not meant to be comprehensive, we cover those that in our experience are most vexing. The biggest challenge is getting decision makers to trust the model. This is a major problem because most models undergo only cursory validation; our field has lacked the motivation, time, and data to properly validate models intended to inform health care decisions. Without documented, adequate validation, there is little basis for decision makers to have confidence that the model's results are credible and should be used in a health technology appraisal. A fundamental problem for validation is that the models are very artificial and lack sufficient depth to adequately represent the reality they are simulating. Typically, modelers assume that all resources have infinite capacity so any patient needing care receives it immediately; there are no waiting times or queues, contrary to the common experience in actual practice. Moreover, all the patients enter the model simultaneously at time zero rather than over time as happens in actuality; differences between patients are ignored or minimized and structural modeling choices that make little sense (e.g., using states to represent events) are forced by commitment to a technique (and even to specific spreadsheet software!). The resulting structural uncertainty is rarely addressed, because methods are lacking and even probabilistic analysis of parameter uncertainty suffers from weak consideration of correlation and arbitrary distribution choices. Stakeholders must see to it that models are fit for the stated purpose and provide the best possible estimates given available data-the decisions at stake deserve nothing less.

Including adverse drug events in economic evaluations of anti-tumour necrosis factor-α drugs for adult rheumatoid arthritis: a systematic review of economic decision analytic models

BACKGROUND

Anti-tumour necrosis factor-α drugs (anti-TNFs) have revolutionised the treatment of rheumatoid arthritis (RA). More effective than standard non-biological disease-modifying anti-rheumatic drugs (nbDMARDs), anti-TNFs are also substantially more expensive. Consequently, a number of model-based economic evaluations have been conducted to establish the relative cost-effectiveness of anti-TNFs. However, anti-TNFs are associated with an increased risk of adverse drug events (ADEs) such as serious infections relative to nbDMARDs. Such ADEs will likely impact on both the costs and consequences of anti-TNFs, for example, through hospitalisations and forced withdrawal from treatment.

OBJECTIVE

The aim of this review was to identify and critically appraise if, and how, ADEs have been incorporated into model-based cost-effectiveness analyses of anti-TNFs for adult patients with RA.

METHODS

A systematic literature review was performed. Electronic databases (Ovid MEDLINE; Ovid EMBASE; Web of Science; NHS Economic Evaluations Database) were searched for literature published between January 1990 and October 2013 using electronic search strategies. The reference lists of retrieved studies were also hand searched. In addition, the National Institute for Health and Care Excellence technology appraisals were searched to identify economic models used to inform UK healthcare decision making. Only full economic evaluations that had used an economic model to evaluate biological DMARDs (bDMARDs) (including anti-TNFs) for adult patients with RA and had incorporated the direct costs and/or consequences of ADEs were critically appraised. To be included, studies also had to be available as a full text in English. Data extracted included general study characteristics and information concerning the methods used to incorporate ADEs and any associated assumptions made. The extracted data were synthesised using a tabular and narrative format.

RESULTS

A total of 43 model-based economic evaluations of bDMARDs for adult RA were identified from 2,483 initially identified studies (2,473 published; ten technology appraisals). Of these, nine studies had incorporated the incidence and costs of ADEs and were critically reviewed. One study also explicitly estimated the potential consequences for patient utility. There was a general lack of detail specifically reporting on how ADEs were included in the economic models. Furthermore, there was substantial heterogeneity amongst the nine studies concerning the (i) application of risk-related terminology; (ii) method of incorporating the incidence, costs and consequences of ADEs; and (iii) ADE-related assumptions.

CONCLUSIONS

Model-based economic evaluations have played an integral role in healthcare reimbursement and funding decisions relating to anti-TNFs for adult patients with RA. However, current economic models have not routinely or systematically considered the direct costs or consequences of ADEs, which may bias the estimates of the relative cost-effectiveness of anti-TNFs. Omitting information on relevant costs and consequences of interventions for RA will affect the validity of the associated recommendations for informed decision making. To improve current practice it is recommended that (i) greater efforts be made to provide appropriate long-term safety data on the use of anti-TNFs in adult RA; (ii) empirical research be undertaken to identify and quantify the impact of, and possible methods for, including ADEs in economic models to inform future good practice guidelines; and (iii) economic modelling guidelines and reference cases be updated to explicitly identify ADEs as an important treatment outcome and address how they might be incorporated into economic models. Improved consideration of the possible implications of ADEs in economic models will ensure that healthcare decision makers are provided with reliable and accurate information with which to make efficient reimbursement and financing decisions.

The impact of structural uncertainty on cost-effectiveness models for adjuvant endocrine breast cancer treatments: the need for disease-specific model standardization and improved guidance

INTRODUCTION

Structural uncertainty relates to differences in model structure and parameterization. For many published health economic analyses in oncology, substantial differences in model structure exist, leading to differences in analysis outcomes and potentially impacting decision-making processes. The objectives of this analysis were (1) to identify differences in model structure and parameterization for cost-effectiveness analyses (CEAs) comparing tamoxifen and anastrazole for adjuvant breast cancer (ABC) treatment; and (2) to quantify the impact of these differences on analysis outcome metrics.

METHODS

The analysis consisted of four steps: (1) review of the literature for identification of eligible CEAs; (2) definition and implementation of a base model structure, which included the core structural components for all identified CEAs; (3) definition and implementation of changes or additions in the base model structure or parameterization; and (4) quantification of the impact of changes in model structure or parameterizations on the analysis outcome metrics life-years gained (LYG), incremental costs (IC) and the incremental cost-effectiveness ratio (ICER).

RESULTS

Eleven CEA analyses comparing anastrazole and tamoxifen as ABC treatment were identified. The base model consisted of the following health states: (1) on treatment; (2) off treatment; (3) local recurrence; (4) metastatic disease; (5) death due to breast cancer; and (6) death due to other causes. The base model estimates of anastrazole versus tamoxifen for the LYG, IC and ICER were 0.263 years, €3,647 and €13,868/LYG, respectively. In the published models that were evaluated, differences in model structure included the addition of different recurrence health states, and associated transition rates were identified. Differences in parameterization were related to the incidences of recurrence, local recurrence to metastatic disease, and metastatic disease to death. The separate impact of these model components on the LYG ranged from 0.207 to 0.356 years, while incremental costs ranged from €3,490 to €3,714 and ICERs ranged from €9,804/LYG to €17,966/LYG. When we re-analyzed the published CEAs in our framework by including their respective model properties, the LYG ranged from 0.207 to 0.383 years, IC ranged from €3,556 to €3,731 and ICERs ranged from €9,683/LYG to €17,570/LYG.

CONCLUSION

Differences in model structure and parameterization lead to substantial differences in analysis outcome metrics. This analysis supports the need for more guidance regarding structural uncertainty and the use of standardized disease-specific models for health economic analyses of adjuvant endocrine breast cancer therapies. The developed approach in the current analysis could potentially serve as a template for further evaluations of structural uncertainty and development of disease-specific models.

Model performance evaluation (validation and calibration) in model-based studies of therapeutic interventions for cardiovascular diseases : a review and suggested reporting framework

Decision analytic models play an increasingly important role in the economic evaluation of health technologies. Given uncertainties around the assumptions used to develop such models, several guidelines have been published to identify and assess 'best practice' in the model development process, including general modelling approach (e.g., time horizon), model structure, input data and model performance evaluation. This paper focuses on model performance evaluation. In the absence of a sufficient level of detail around model performance evaluation, concerns regarding the accuracy of model outputs, and hence the credibility of such models, are frequently raised. Following presentation of its components, a review of the application and reporting of model performance evaluation is presented. Taking cardiovascular disease as an illustrative example, the review investigates the use of face validity, internal validity, external validity, and cross model validity. As a part of the performance evaluation process, model calibration is also discussed and its use in applied studies investigated. The review found that the application and reporting of model performance evaluation across 81 studies of treatment for cardiovascular disease was variable. Cross-model validation was reported in 55 % of the reviewed studies, though the level of detail provided varied considerably. We found that very few studies documented other types of validity, and only 6 % of the reviewed articles reported a calibration process. Considering the above findings, we propose a comprehensive model performance evaluation framework (checklist), informed by a review of best-practice guidelines. This framework provides a basis for more accurate and consistent documentation of model performance evaluation. This will improve the peer review process and the comparability of modelling studies. Recognising the fundamental role of decision analytic models in informing public funding decisions, the proposed framework should usefully inform guidelines for preparing submissions to reimbursement bodies.