Estimating the cost-effectiveness of an intervention in a clinical trial when partial cost information is available: a Bayesian approach

Handling Missing Data in Health Economics and Outcomes Research (HEOR): A Systematic Review and Practical Recommendations

BACKGROUND

Missing data in costs and/or health outcomes and in confounding variables can create bias in the inference of health economics and outcomes research studies, which in turn can lead to inappropriate policies. Most of the literature focuses on handling missing data in randomized controlled trials, which are not necessarily always the data used in health economics and outcomes research.

OBJECTIVES

We aimed to provide an overview on missing data issues and how to address incomplete data and report the findings of a systematic literature review of methods used to deal with missing data in health economics and outcomes research studies that focused on cost, utility, and patient-reported outcomes.

METHODS

A systematic search of papers published in English language until the end of the year 2020 was carried out in PubMed. Studies using statistical methods to handle missing data for analyses of cost, utility, or patient-reported outcome data were included, as were reviews and guidance papers on handling missing data for those outcomes. The data extraction was conducted with a focus on the context of the study, the type of missing data, and the methods used to tackle missing data.

RESULTS

From 1433 identified records, 40 papers were included. Thirteen studies were economic evaluations. Thirty studies used multiple imputation with 17 studies using multiple imputation by chained equation, while 15 studies used a complete-case analysis. Seventeen studies addressed missing cost data and 23 studies dealt with missing outcome data. Eleven studies reported a single method while 20 studies used multiple methods to address missing data.

CONCLUSIONS

Several health economics and outcomes research studies did not offer a justification of their approach of handling missing data and some used only a single method without a sensitivity analysis. This systematic literature review highlights the importance of considering the missingness mechanism and including sensitivity analyses when planning, analyzing, and reporting health economics and outcomes research studies.

Cost-effectiveness of long-term psychoanalytic psychotherapy for treatment-resistant depression: RCT evidence from the Tavistock Adult Depression Study (TADS)

BACKGROUND

Treatment-resistant depression (TRD) accounts for a large fraction of the burden of depression. The interventions currently used are mostly pharmacological and short-term psychotherapies, but their effectiveness is limited. The Tavistock Adult Depression Study found evidence for the effectiveness of long-term psychoanalytic psychotherapy (LTPP) plus treatment as usual (TAU), versus TAU alone, for TRD. Even after a 2-year follow-up, moderate effect sizes were sustained. This study assessed the cost-effectiveness of this LTPP + TAU.

METHODS

We conducted a within-trial economic evaluation using a Bayesian framework.

RESULTS

Quality-adjusted life years (QALYs) were 0.16 higher in the LTPP + TAU group compared with TAU. The direct cost of LTPP was £5500, with no substantial compensating savings elsewhere. Overall, average health and social care costs in the LTPP + TAU group were £5000 more than in the TAU group, employment rates were unchanged, and effects on other non-healthcare costs were uncertain. Accordingly, the incremental cost-effectiveness ratio was ≈£33,000/QALY; the probability that LTPP + TAU was cost-effective at a willingness to pay of £20,000/QALY was 18 %.

LIMITATIONS

The sample size of this study was relatively small, and the fraction of missing service-use data was approximately 50 % at all time points. The study was conducted at a single site, potentially reducing generalizability.

CONCLUSIONS

Although LTPP + TAU was found to be clinically effective for treating TRD, it was not found to be cost-effective compared with TAU. However, given the sustained effects over the follow-up period it is likely that the time horizon of this study was too short to capture all benefits of LTPP augmentation.

A scoping review of statistical methods for trial-based economic evaluations: The current state of play

The statistical quality of trial-based economic evaluations is often suboptimal, while a comprehensive overview of available statistical methods is lacking. Therefore, this review summarized and critically appraised available statistical methods for trial-based economic evaluations. A literature search was performed to identify studies on statistical methods for dealing with baseline imbalances, skewed costs and/or effects, correlated costs and effects, clustered data, longitudinal data, missing data and censoring in trial-based economic evaluations. Data was extracted on the statistical methods described, their advantages, disadvantages, relative performance and recommendations of the study. Sixty-eight studies were included. Of them, 27 (40%) assessed methods for baseline imbalances, 39 (57%) assessed methods for skewed costs and/or effects, 27 (40%) assessed methods for correlated costs and effects, 18 (26%) assessed methods for clustered data, 7 (10%) assessed methods for longitudinal data, 26 (38%) assessed methods for missing data and 10 (15%) assessed methods for censoring. All identified methods were narratively described. This review provides a comprehensive overview of available statistical methods for dealing with the most common statistical complexities in trial-based economic evaluations. Herewith, it can provide valuable input for researchers when deciding which statistical methods to use in a trial-based economic evaluation.

Comparing methods for handling missing cost and quality of life data in the Early Endovenous Ablation in Venous Ulceration trial

Objectives

This study compares methods for handling missing data to conduct cost-effectiveness analysis in the context of a clinical study.

Methods

Patients in the Early Endovenous Ablation in Venous Ulceration (EVRA) trial had between 1 year and 5.5 years (median 3 years) of follow-up under early or deferred endovenous ablation. This study compares complete-case-analysis (CCA), multiple imputation using linear regression (MILR) and using predictive mean matching (MIPMM), Bayesian parametric approach using the R package missingHE (BPA), repeated measures fixed effect (RMFE) and repeated measures mixed model (RMM). The outcomes were total mean costs and total mean quality-adjusted life years (QALYs) at different time horizons (1 year, 3 years and 5 years).

Results

All methods found no statistically significant difference in cost at the 5% level in all time horizons, and all methods found statistically significantly greater mean QALY at year 1. By year 3, only BPA showed a statistically significant difference in QALY between treatments. Standard errors differed substantially between the methods employed.

Conclusion

CCA can be biased if data are MAR and is wasteful of the data. Hence the results for CCA are likely to be inaccurate. Other methods coincide in suggesting that early intervention is cost-effective at a threshold of £30,000 per QALY 1, 3 and 5 years. However, the variation in the results across the methods does generate some additional methodological uncertainty, underlining the importance of conducting sensitivity analyses using alternative approaches.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12962-022-00351-6.

Flexible Bayesian longitudinal models for cost-effectiveness analyses with informative missing data

Cost-effectiveness analyses (CEA) are recommended to include sensitivity analyses which make a range of contextually plausible assumptions about missing data. However, with longitudinal data on, for example, patients' health-related quality of life (HRQoL), the missingness patterns can be complicated because data are often missing both at specific timepoints (interim missingness) and following loss to follow-up. Methods to handle these complex missing data patterns have not been developed for CEA, and must recognize that data may be missing not at random, while accommodating both the correlation between costs and health outcomes and the non-normal distribution of these endpoints. We develop flexible Bayesian longitudinal models that allow the impact of interim missingness and loss to follow-up to be disentangled. This modeling framework enables studies to undertake sensitivity analyses according to various contextually plausible missing data mechanisms, jointly model costs and outcomes using appropriate distributions, and recognize the correlation among these endpoints over time. We exemplify these models in the REFLUX study in which 52% of participants had HRQoL data missing for at least one timepoint over the 5-year follow-up period. We provide guidance for sensitivity analyses and accompanying code to help future studies handle these complex forms of missing data.

A full Bayesian model to handle structural ones and missingness in economic evaluations from individual-level data

Economic evaluations from individual-level data are an important component of the process of technology appraisal, with a view to informing resource allocation decisions. A critical problem in these analyses is that both effectiveness and cost data typically present some complexity (eg, nonnormality, spikes, and missingness) that should be addressed using appropriate methods. However, in routine analyses, standardised approaches are typically used, possibly leading to biassed inferences. We present a general Bayesian framework that can handle the complexity. We show the benefits of using our approach with a motivating example, the MenSS trial, for which there are spikes at one in the effectiveness and missingness in both outcomes. We contrast a set of increasingly complex models and perform sensitivity analysis to assess the robustness of the conclusions to a range of plausible missingness assumptions. We demonstrate the flexibility of our approach with a second example, the PBS trial, and extend the framework to accommodate the characteristics of the data in this study. This paper highlights the importance of adopting a comprehensive modelling approach to economic evaluations and the strategic advantages of building these complex models within a Bayesian framework.

A Bayesian framework for health economic evaluation in studies with missing data

Health economics studies with missing data are increasingly using approaches such as multiple imputation that assume that the data are "missing at random." This assumption is often questionable, as-even given the observed data-the probability that data are missing may reflect the true, unobserved outcomes, such as the patients' true health status. In these cases, methodological guidelines recommend sensitivity analyses to recognise data may be "missing not at random" (MNAR), and call for the development of practical, accessible approaches for exploring the robustness of conclusions to MNAR assumptions. Little attention has been paid to the problem that data may be MNAR in health economics in general and in cost-effectiveness analyses (CEA) in particular. In this paper, we propose a Bayesian framework for CEA where outcome or cost data are missing. Our framework includes a practical, accessible approach to sensitivity analysis that allows the analyst to draw on expert opinion. We illustrate the framework in a CEA comparing an endovascular strategy with open repair for patients with ruptured abdominal aortic aneurysm, and provide software tools to implement this approach.

Bayesian sample size determination for cost-effectiveness studies with censored data

Cost-effectiveness models are commonly utilized to determine the combined clinical and economic impact of one treatment compared to another. However, most methods for sample size determination of cost-effectiveness studies assume fully observed costs and effectiveness outcomes, which presents challenges for survival-based studies in which censoring exists. We propose a Bayesian method for the design and analysis of cost-effectiveness data in which costs and effectiveness may be censored, and the sample size is approximated for both power and assurance. We explore two parametric models and demonstrate the flexibility of the approach to accommodate a variety of modifications to study assumptions.

Handling Missing Data in Within-Trial Cost-Effectiveness Analysis: A Review with Future Recommendations

Cost-effectiveness analyses (CEAs) alongside randomised controlled trials (RCTs) are increasingly designed to collect resource use and preference-based health status data for the purpose of healthcare technology assessment. However, because of the way these measures are collected, they are prone to missing data, which can ultimately affect the decision of whether an intervention is good value for money. We examine how missing cost and effect outcome data are handled in RCT-based CEAs, complementing a previous review (covering 2003-2009, 88 articles) with a new systematic review (2009-2015, 81 articles) focussing on two different perspectives. First, we provide guidelines on how the information about missingness and related methods should be presented to improve the reporting and handling of missing data. We propose to address this issue by means of a quality evaluation scheme, providing a structured approach that can be used to guide the collection of information, elicitation of the assumptions, choice of methods and considerations of possible limitations of the given missingness problem. Second, we review the description of the missing data, the statistical methods used to deal with them and the quality of the judgement underpinning the choice of these methods. Our review shows that missing data in within-RCT CEAs are still often inadequately handled and the overall level of information provided to support the chosen methods is rarely satisfactory.

A guide to handling missing data in cost-effectiveness analysis conducted within randomised controlled trials

Missing data are a frequent problem in cost-effectiveness analysis (CEA) within a randomised controlled trial. Inappropriate methods to handle missing data can lead to misleading results and ultimately can affect the decision of whether an intervention is good value for money. This article provides practical guidance on how to handle missing data in within-trial CEAs following a principled approach: (i) the analysis should be based on a plausible assumption for the missing data mechanism, i.e. whether the probability that data are missing is independent of or dependent on the observed and/or unobserved values; (ii) the method chosen for the base-case should fit with the assumed mechanism; and (iii) sensitivity analysis should be conducted to explore to what extent the results change with the assumption made. This approach is implemented in three stages, which are described in detail: (1) descriptive analysis to inform the assumption on the missing data mechanism; (2) how to choose between alternative methods given their underlying assumptions; and (3) methods for sensitivity analysis. The case study illustrates how to apply this approach in practice, including software code. The article concludes with recommendations for practice and suggestions for future research.

Bayesian models for cost-effectiveness analysis in the presence of structural zero costs

Bayesian modelling for cost-effectiveness data has received much attention in both the health economics and the statistical literature, in recent years. Cost-effectiveness data are characterised by a relatively complex structure of relationships linking a suitable measure of clinical benefit (e.g. quality-adjusted life years) and the associated costs. Simplifying assumptions, such as (bivariate) normality of the underlying distributions, are usually not granted, particularly for the cost variable, which is characterised by markedly skewed distributions. In addition, individual-level data sets are often characterised by the presence of structural zeros in the cost variable. Hurdle models can be used to account for the presence of excess zeros in a distribution and have been applied in the context of cost data. We extend their application to cost-effectiveness data, defining a full Bayesian specification, which consists of a model for the individual probability of null costs, a marginal model for the costs and a conditional model for the measure of effectiveness (given the observed costs). We presented the model using a working example to describe its main features. © 2013 The Authors. Statistics in Medicine published by John Wiley & Sons, Ltd.

Multiple imputation methods for handling missing data in cost-effectiveness analyses that use data from hierarchical studies: an application to cluster randomized trials

PURPOSE

Multiple imputation (MI) has been proposed for handling missing data in cost-effectiveness analyses (CEAs). In CEAs that use cluster randomized trials (CRTs), the imputation model, like the analysis model, should recognize the hierarchical structure of the data. This paper contrasts a multilevel MI approach that recognizes clustering, with single-level MI and complete case analysis (CCA) in CEAs that use CRTs.

METHODS

We consider a multilevel MI approach compatible with multilevel analytical models for CEAs that use CRTs. We took fully observed data from a CEA that evaluated an intervention to improve diagnosis of active labor in primiparous women using a CRT (2078 patients, 14 clusters). We generated scenarios with missing costs and outcomes that differed, for example, according to the proportion with missing data (10%-50%), the covariates that predicted missing data (individual, cluster-level), and the missingness mechanism: missing completely at random (MCAR), missing at random (MAR), or missing not at random (MNAR). We estimated incremental net benefits (INBs) for each approach and compared them with the estimates from the fully observed data, the "true" INBs.

RESULTS

When costs and outcomes were assumed to be MCAR, the INBs for each approach were similar to the true estimates. When data were MAR, the point estimates from the CCA differed from the true estimates. Multilevel MI provided point estimates and standard errors closer to the true values than did single-level MI across all settings, including those in which a high proportion of observations had cost and outcome data MAR and when data were MNAR.

CONCLUSIONS

Multilevel MI accommodates the multilevel structure of the data in CEAs that use cluster trials and provides accurate cost-effectiveness estimates across the range of circumstances considered.

Confounding and missing data in cost-effectiveness analysis: comparing different methods

INTRODUCTION

Common approaches in cost-effectiveness analyses do not adjust for confounders. In nonrandomized studies this can result in biased results. Parametric models such as regression models are commonly applied to adjust for confounding, but there are several issues which need to be accounted for. The distribution of costs is often skewed and there can be a considerable proportion of observations of zero costs, which cannot be well handled using simple linear models. Associations between costs and effectiveness cannot usually be explained using observed background information alone, which also requires special attention in parametric modeling. Furthermore, in longitudinal panel data, missing observations are a growing problem also with nonparametric methods when cumulative outcome measures are used.

METHODS

We compare two methods, which can handle the aforementioned issues, in addition to the standard unadjusted bootstrap techniques for assessing cost-effectiveness in the Helsinki Psychotherapy Study based on five repeated measurements of the Global Severity Index (SCL-90-GSI) and direct costs during one year of follow-up in two groups defined by the Defence Style Questionnaire (DSQ) at baseline. The first method models cumulative costs and effectiveness using generalized linear models, multiple imputation and bootstrap techniques. The second method deals with repeated measurement data directly using a hierarchical two-part logistic and gamma regression model for costs, a hierarchical linear model for effectiveness, and Bayesian inference.

RESULTS

The adjustment for confounders mitigated the differences of the DSQ groups. Our method, based on Bayesian inference, revealed the unexplained association of costs and effectiveness. Furthermore, the method also demonstrated strong heteroscedasticity in positive costs.

CONCLUSIONS

Confounders should be accounted for in cost-effectiveness analyses, if the comparison groups are not randomized.

JEL CLASSIFICATION

C1; C3; I1.

Review of statistical methods for analysing healthcare resources and costs

We review statistical methods for analysing healthcare resource use and costs, their ability to address skewness, excess zeros, multimodality and heavy right tails, and their ease for general use. We aim to provide guidance on analysing resource use and costs focusing on randomised trials, although methods often have wider applicability. Twelve broad categories of methods were identified: (I) methods based on the normal distribution, (II) methods following transformation of data, (III) single-distribution generalized linear models (GLMs), (IV) parametric models based on skewed distributions outside the GLM family, (V) models based on mixtures of parametric distributions, (VI) two (or multi)-part and Tobit models, (VII) survival methods, (VIII) non-parametric methods, (IX) methods based on truncation or trimming of data, (X) data components models, (XI) methods based on averaging across models, and (XII) Markov chain methods. Based on this review, our recommendations are that, first, simple methods are preferred in large samples where the near-normality of sample means is assured. Second, in somewhat smaller samples, relatively simple methods, able to deal with one or two of above data characteristics, may be preferable but checking sensitivity to assumptions is necessary. Finally, some more complex methods hold promise, but are relatively untried; their implementation requires substantial expertise and they are not currently recommended for wider applied work.

Cost-efficient design of occupational exposure assessment strategies--a review

When designing a strategy for collecting occupational exposure data, both economic and statistical performance criteria should be considered. However, very few studies have addressed the trade-off between the cost of obtaining data and the precision/accuracy of the exposure estimate as a research issue. To highlight the need of providing cost-efficient designs for assessing exposure variables in occupational research, the present review explains and critically evaluates the concepts and analytical tools used in available cost efficiency studies. Nine studies were identified through a systematic search using two algorithms in the databases PubMed and ScienceDirect. Two main approaches could be identified in these studies: 'comparisons' of the cost efficiency associated with different measurement designs and 'optimizations' of resource allocation on the basis of functions describing cost and statistical efficiency. In either case, the reviewed studies use simplified analytical tools and insufficient economic analyses. More research is needed to understand whether these drawbacks jeopardize the guidance on cost-efficient exposure assessment provided by the studies, as well as to support theoretical results by empirical data from occupational life.

Improving costing methods in multicentre economic evaluation: the use of multiple imputation for unit costs

Economic evaluations must use appropriate costing methods. However, in multicentre cost-effectiveness analyses (CEA) a fundamental issue of how best to measure and analyse unit costs has been neglected. Multicentre CEA commonly take the mean unit cost from a national database, such as NHS reference costs. This approach does not recognise that unit costs vary across centres and are unavailable in some centres. This paper proposes the use of multiple imputation (MI) to predict those centre-specific unit costs that are not available, while recognising the statistical uncertainty surrounding this imputation.We illustrate MI with a CEA of a multicentre randomised trial (1014 patients, 60 centres), implemented using multilevel modelling. We use MI to derive centre-specific unit costs, based on centre characteristics including average casemix, and compare this to using mean NHS reference costs. In this case study, using MI unit costs rather than mean reference costs led to less heterogeneity across centres, more precise estimates of incremental cost, but similar estimates of incremental cost-effectiveness.We conclude that using MI to predict unit costs can preserve correlations, maximise the use of available data, and, when combined with multilevel modelling is an appropriate method for recognising the statistical uncertainty in multicentre CEA.

Bayesian modelling of healthcare resource use in multinational randomized clinical trials

BACKGROUND

Most cost-effectiveness analyses conducted alongside multinational randomized clinical trials (RCT) are carried out applying the unit costs from the country of interest to trial-wide resource use items with the objective of estimating total healthcare costs by treatment group. However, this approach could confound 'price effects' with 'country effects'. An alternative approach is to use multilevel modelling techniques to analyse healthcare resource use (HCRU) from the trial, and obtain country-specific total costs by applying country-specific unit costs to corresponding shrinkage estimates of differential HCRU.

METHODS

To illustrate the feasibility of this approach, we analysed data from twin multinational RCTs, which enrolled approximately 2000 individuals into three treatment arms for the management of patients with chronic respiratory disease. The models were implemented using Bayesian multilevel models, to reflect the hierarchical structure of the data while controlling for co-variates at the patient and country level.

RESULTS

This analysis showed that directly modelling the level of HCRU is a promising approach to facilitate cost-effectiveness analyses conducted alongside multinational RCTs, offering several advantages compared with the modelling of direct costs.

CONCLUSIONS

It is argued that modelling the level of HCRU within the Bayesian framework avoids confounding the price effects with the country effects and facilitates the estimation of costs for several countries represented in the trial.