A New Approach for Sampling Ordered Parameters in Probabilistic Sensitivity Analysis

A methodological guide for implementing and interpreting results of probabilistic analysis

INTRODUCTION

Probabilistic analysis, also referred to as probabilistic sensitivity analysis (PSA), is used extensively in cost-effectiveness evaluations of health technologies. We present methodological guidance for implementing probabilistic analysis and interpreting its results for policy and decision-making.

METHODS

We review the methodological issues related to common practices in probabilistic analysis, explore aspects that are currently not widely addressed in the health economics literature, and provide an overview of recent methodological developments.

RESULTS

We use examples to highlight the advantages and disadvantages of common tools used for presenting probabilistic analysis results, including the cost-effectiveness acceptability curve (CEAC), cost-effectiveness acceptability frontier (CEAF), and value of information (VOI) analysis. We raise and address issues related to using Monte Carlo standard error to determine the number of iterations required, the implications of large uncertainty, and the credibility and meaningfulness of small differences in quality-adjusted life-years (QALYs). We then discuss evolving methods in probabilistic analysis, cautious uses of probabilistic analysis, and factors impacting parameter uncertainty.

CONCLUSIONS

A deeper understanding of probabilistic analysis methods enables health economists and decision-makers to more effectively address and interpret parameter uncertainty in health economic evaluations, which is essential for making informed policy decisions.

Clinical and cost-effectiveness of left ventricular assist devices as destination therapy for advanced heart failure: systematic review and economic evaluation

Background

Selected patients with advanced heart failure ineligible for heart transplantation could benefit from left ventricular assist device therapy as 'destination therapy'. There is evidence of the efficacy of destination therapy; however, it is not currently commissioned within the United Kingdom National Health Service due to the lack of economic evidence.

Objective

What is the clinical and cost-effectiveness of a left ventricular assist device compared to medical management for patients with advanced heart failure ineligible for heart transplantation (destination therapy)?

Methods

A systematic review of evidence on the clinical and cost-effectiveness of left ventricular assist devices as destination therapy was undertaken including, where feasible, a network meta-analysis to provide an indirect estimate of the relative effectiveness of currently available left ventricular assist devices compared to medical management. For the systematic reviews, data sources searched (up to 11 January 2022) were Cochrane CENTRAL, MEDLINE and EMBASE via Ovid for primary studies, and Epistemonikos and Cochrane Database of Systematic Reviews for relevant systematic reviews. Trial registers were also searched, along with data and reports from intervention-specific registries. Economic studies were identified in EconLit, CEA registry and the NHS Economic Evaluation Database (NHS EED). The searches were supplemented by checking reference lists of included studies. An economic model (Markov) was developed to estimate the cost-effectiveness of left ventricular assist devices compared to medical management from the United Kingdom National Health Service/personal social service perspective. Deterministic and probabilistic sensitivity analyses were conducted to explore uncertainties. Where possible, all analyses focused on the only currently available left ventricular assist device (HeartMate 3TM, Abbott, Chicago, IL, USA) in the United Kingdom.

Results

The clinical effectiveness review included 134 studies (240 articles). There were no studies directly comparing HeartMate 3 and medical management (a randomised trial is ongoing). The currently available left ventricular assist device improves patient survival and reduces stroke rates and complications compared to earlier devices and relative to medical management. For example, survival at 24 months is 77% with the HeartMate 3 device compared to 59% with the HeartMate II (MOMENTUM 3 trial). An indirect comparison demonstrated a reduction in mortality compared to medical management [relative risk of death 0.25 (95% confidence interval 0.13 to 0.47); 24 months; this study]. The cost-effectiveness review included 5 cost analyses and 14 economic evaluations covering different generations of devices and with different perspectives. The reported incremental costs per quality-adjusted life-year gained compared to medical management were lower for later generations of devices [as low as £46,207 (2019 prices; United Kingdom perspective; time horizon at least 5 years)]. The economic evaluation used different approaches to obtain the relative effects of current left ventricular assist devices compared to medical management from the United Kingdom National Health Service/personal social service perspective. All gave similar incremental cost-effectiveness ratios of £53,496-58,244 per quality-adjusted life-year gained - lifetime horizon. Model outputs were sensitive to parameter estimates relating to medical management. The findings did not materially differ on exploratory subgroup analyses based on the severity of heart failure.

Limitations

There was no direct evidence comparing the clinical effectiveness of HeartMate 3 to medical management. Indirect comparisons made were based on limited data from heterogeneous studies regarding the severity of heart failure (Interagency Registry for Mechanically Assisted Circulatory Support score distribution) and possible for survival only. Furthermore, the cost of medical management of advanced heart failure in the United Kingdom is not clear.

Conclusions

Using cost-effectiveness criteria applied in the United Kingdom, left ventricular assist devices compared to medical management for patients with advanced heart failure ineligible for heart transplant may not be cost-effective. When available, data from the ongoing evaluation of HeartMate 3 compared to medical management can be used to update cost-effectiveness estimates. An audit of the costs of medical management in the United Kingdom is required to further decrease uncertainty in the economic evaluation.

Study registration

This study is registered as PROSPERO CRD42020158987.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: NIHR128996) and is published in full in Health Technology Assessment; Vol. 28, No. 38. See the NIHR Funding and Awards website for further award information.

Simplified Methods for Modelling Dependent Parameters in Health Economic Evaluations: A Tutorial

BACKGROUND

In health economic evaluations, model parameters are often dependent on other model parameters. Although methods exist to simulate multivariate normal (MVN) distribution data and estimate transition probabilities in Markov models while considering competing risks, they are technically challenging for health economic modellers to implement. This tutorial introduces easily implementable applications for handling dependent parameters in modelling.

METHODS

Analytical proofs and proposed simplified methods for handling dependent parameters in typical health economic modelling scenarios are provided, and implementation of these methods are illustrated in seven examples along with the SAS and R code.

RESULTS

Methods to quantify the covariance and correlation coefficients of correlated variables based on published summary statistics and generation of MVN distribution data are demonstrated using examples of physician visits data and cost component data. The use of univariate normal distribution data instead of MVN distribution data to capture population heterogeneity is illustrated based on the results from multiple regression models with linear predictors, and two examples are provided (linear fixed-effects model and Cox proportional hazards model). A conditional probability method is introduced to handle two or more state transitions in a single Markov model cycle and applied in examples of one- and two-way state transitions.

CONCLUSIONS

This tutorial proposes an extension of routinely used methods along with several examples. These simplified methods may be easily applied by health economic modellers with varied statistical backgrounds.

The Difference Method Approach for Sampling Order Constrained Parameters: An Improved Implementation and Important Limitations

A health economic model may include a set of related inputs whose true values are uncertain, but that can be assumed to follow a logical order. Various approaches are available for performing probabilistic sensitivity analysis while preserving the order constraint, one such approach is known as the difference method. The difference method approach appears to have many of the required properties, has been endorsed by good practice guidelines, and is likely to prove a popular approach. However, the proposed implementation of the difference method approach is cumbersome, requiring numerical estimation, which might present a barrier to its adoption. Furthermore, it is unclear whether the method can always be applied to three or more model inputs and whether it is unbiased across all possible input values. This study has investigated these three issues for ordered inputs bounded between 0 and 1. An analytic solution is given that allows for more straightforward and compact implementation. The difference method approach cannot always be applied to a set of three or more model inputs, and this depends on the relative size of the variances of the logit-transformed Beta distributions fitted to each variable. The approach can also produce samples with biased means and variances under certain combinations of input means and variances. It is recommended that the difference method approach be used where appropriate; however, an understanding of its limitations is necessary to identify such cases.

Behavioural activation therapy for post-stroke depression: the BEADS feasibility RCT

BACKGROUND

There is currently insufficient evidence for the clinical effectiveness and cost-effectiveness of psychological therapies for post-stroke depression.

OBJECTIVE

To evaluate the feasibility of undertaking a definitive trial to evaluate the clinical effectiveness and cost-effectiveness of behavioural activation (BA) compared with usual stroke care for treating post-stroke depression.

DESIGN

Parallel-group, feasibility, multicentre, randomised controlled trial with nested qualitative research and a health economic evaluation.

SETTING

Acute and community stroke services in three sites in England.

PARTICIPANTS

Community-dwelling adults 3 months to 5 years post stroke who are depressed, as determined by the Patient Health Questionnaire-9 (PHQ-9) or the Visual Analogue Mood Scales 'Sad' item. Exclusions: patients who are blind and/or deaf, have dementia, are unable to communicate in English, do not have mental capacity to consent, are receiving treatment for depression at the time of stroke onset or are currently receiving psychological intervention.

RANDOMISATION AND BLINDING

Participants were randomised (1 : 1 ratio) to BA or usual stroke care. Randomisation was conducted using a computer-generated list with random permuted blocks of varying sizes, stratified by site. Participants and therapists were aware of the allocation, but outcome assessors were blind.

INTERVENTIONS

The intervention arm received up to 15 sessions of BA over 4 months. BA aims to improve mood by increasing people's level of enjoyable or valued activities. The control arm received usual care only.

MAIN OUTCOME MEASURES

Primary feasibility outcomes concerned feasibility of recruitment to the main trial, acceptability of research procedures and measures, appropriateness of baseline and outcome measures, retention of participants and potential value of conducting the definitive trial. Secondary feasibility outcomes concerned the delivery of the intervention. The primary clinical outcome 6 months post randomisation was the PHQ-9. Secondary clinical outcomes were Stroke Aphasic Depression Questionnaire - Hospital version, Nottingham Leisure Questionnaire, Nottingham Extended Activities of Daily Living, Carer Strain Index, EuroQol-5 Dimensions, five-level version and health-care resource use questionnaire.

RESULTS

Forty-eight participants were recruited in 27 centre-months of recruitment, at a recruitment rate of 1.8 participants per centre per month. The 25 participants randomised to receive BA attended a mean of 8.5 therapy sessions [standard deviation (SD) 4.4 therapy sessions]; 23 participants were allocated to usual care. Outcome assessments were completed by 39 (81%) participants (BA, n = 18; usual care, n = 21). Mean PHQ-9 scores at 6-month follow-up were 10.1 points (SD 6.9 points) and 14.4 points (SD 5.1 points) in the BA and control groups, respectively, a difference of -3.8 (95% confidence interval -6.9 to -0.6) after adjusting for baseline PHQ-9 score and centre, representing a reduction in depression in the BA arm. Therapy was delivered as intended. BA was acceptable to participants, carers and therapists. Value-of-information analysis indicates that the benefits of conducting a definitive trial would be likely to outweigh the costs. It is estimated that a sample size of between 580 and 623 participants would be needed for a definitive trial.

LIMITATIONS

Target recruitment was not achieved, although we identified methods to improve recruitment.

CONCLUSIONS

The Behavioural Activation Therapy for Depression after Stroke trial was feasible with regard to the majority of outcomes. The outstanding issue is whether or not a sufficient number of participants could be recruited within a reasonable time frame for a definitive trial. Future work is required to identify whether or not there are sufficient sites that are able to deliver the services required for a definitive trial.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN12715175.

FUNDING

This project was funded by the NIHR Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 23, No. 47. See the NIHR Journals Library website for further project information.

Management of Active Surveillance-Eligible Prostate Cancer during Pretransplantation Workup of Patients with Kidney Failure: A Simulation Study

BACKGROUND AND OBJECTIVES

The general rule that every active malignancy is an absolute contraindication for kidney transplantation is challenged by kidney failure patients diagnosed with active surveillance-eligible prostate cancer during pretransplantation workup. Interdisciplinary treatment teams therefore often face the challenge of balancing the benefits of early kidney transplantation and the risk of metastatic progression. Hence, we compared the quality-adjusted life expectancy of different management strategies in kidney failure patients diagnosed with active surveillance-eligible prostate cancer during pretransplantation workup.

DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS

A discrete event simulation model was developed on the basis of a systematic literature search, clinical guidelines, and expert opinion. After model validation and calibration, we simulated four management strategies in a hypothetical cohort of 100,000 patients: Definitive treatment (surgery or radiation therapy) and listing after a waiting period of 2 years, definitive treatment and immediate listing, active surveillance and listing after a waiting period of 2 years, and active surveillance and immediate listing. Individual patient results (quality-adjusted life years; QALYs) were aggregated into strategy-specific means (± SEs).

RESULTS

Active surveillance and immediate listing yielded the highest amount of quality-adjusted life expectancy (6.97 ± 0.01 QALYs) followed by definitive treatment and immediate listing (6.75 ± 0.01 QALYs). These two strategies involving immediate listing not only outperformed those incorporating a waiting period of 2 years (definitive treatment: 6.32 ± 0.01 QALYs; active surveillance: 6.59 ± 0.01 QALYs) but also yielded a higher proportion of successfully performed transplantations (72% and 74% versus 56% and 59%), with less time on hemodialysis on average (4.02 and 3.81 years versus 4.80 and 4.65 years).

CONCLUSIONS

Among kidney failure patients diagnosed with active surveillance-eligible prostate cancer during pretransplantation workup, the active surveillance and immediate listing strategy outperformed the alternative management strategies from a quality of life expectancy perspective, followed by definitive treatment and immediate listing.