Calculating Partial Expected Value of Perfect Information via Monte Carlo Sampling Algorithms

Modeling First-Line Daratumumab Use for Newly Diagnosed, Transplant-Ineligible, Multiple Myeloma: A Cost-Effectiveness and Risk Analysis for Healthcare Payers

BACKGROUND AND OBJECTIVE

This study aimed to assess the cost-effectiveness of two regimens regarded as the standard of care for the treatment of newly diagnosed, transplant-ineligible multiple myeloma in Singapore: (1) daratumumab, lenalidomide, and dexamethasone and (2) bortezomib, lenalidomide, and dexamethasone. Additionally, it aimed to explore potential strategies to manage decision uncertainty and mitigate financial risk.

METHODS

A cost-effectiveness analysis from the healthcare system perspective was conducted using a partitioned survival model to estimate lifetime costs and quality-adjusted life years (QALYs) associated with daratumumab-based treatment and the bortezomib-based regimen. The analysis used data from the MAIA and SWOG S0777 trials and incorporated local real-world data where available. Sensitivity analyses were performed to evaluate the robustness of the findings, and a risk analysis was conducted to analyze various payer strategies in terms of their payer strategy and uncertainty burden (P-SUB), which account for the decision uncertainty and the additional cost of choosing a suboptimal intervention.

RESULTS

The incremental cost-effectiveness ratio (ICER) for daratumumab, lenalidomide, and dexamethasone (DRd) compared with bortezomib, lenalidomide, and dexamethasone (VRd) was US $90,364 per QALY gained. The results were sensitive to variations in survival for DRd, postprogression treatment costs, cost of hospice care, and hazard ratio for progression-free survival. The scenarios explored indicated that structural assumptions, such as the time horizon of the analysis, significantly influenced the results due to uncertainties arising from immature trial data and treatment efficacy over time. Among the various payer strategies compared, an upfront price discount for daratumumab emerged as the best approach with the lowest P-SUB at US $14,708.

CONCLUSION

In conclusion, this study finds that daratumumab as a first-line treatment for myeloma exceeds the cost-effectiveness threshold considered in this evaluation. An upfront price reduction is the recommended strategy to manage uncertainties and mitigate financial risks. These findings highlight the importance of targeted payer strategies to address specific types and sources of uncertainty.

Value of Information for Clinical Trial Design: The Importance of Considering All Relevant Comparators

Value of Information (VOI) analyses calculate the economic value that could be generated by obtaining further information to reduce uncertainty in a health economic decision model. VOI has been suggested as a tool for research prioritisation and trial design as it can highlight economically valuable avenues for future research. Recent methodological advances have made it increasingly feasible to use VOI in practice for research; however, there are critical differences between the VOI approach and the standard methods used to design research studies such as clinical trials. We aimed to highlight key differences between the research design approach based on VOI and standard clinical trial design methods, in particular the importance of considering the full decision context. We present two hypothetical examples to demonstrate that VOI methods are only accurate when (1) all feasible comparators are included in the decision model when designing research, and (2) all comparators are retained in the decision model once the data have been collected and a final treatment recommendation is made. Omitting comparators from either the design or analysis phase of research when using VOI methods can lead to incorrect trial designs and/or treatment recommendations. Overall, we conclude that incorrectly specifying the health economic model by ignoring potential comparators can lead to misleading VOI results and potentially waste scarce research resources.

Cost-utility and value of information analysis of tisagenlecleucel for relapsed/refractory diffuse large B-cell lymphoma in the Irish healthcare setting

BACKGROUND

The evidence base of tisagenlecleucel is uncertain.

OBJECTIVE

To evaluate the cost-effectiveness of tisagenlecleucel. To conduct expected value of perfect information (EVPI) and partial EVPI (EVPPI) analyses.

STUDY DESIGN

A three-state partitioned survival model. A short-term decision tree partitioned patients in the tisagenlecleucel arm according to infusion status. Survival was extrapolated to 5 years; general population mortality with a standardised mortality ratio was then applied. EVPI and EVPPI were scaled up to population according to the incidence of the decision.

SETTING

Irish healthcare payer.

PARTICIPANTS

Patients with relapsed/refractory diffuse large B-cell lymphoma (R/R DLBCL).

INTERVENTIONS

Tisagenlecleucel versus Salvage Chemotherapy (with or without haematopoietic stem cell transplant).

MAIN OUTCOME MEASURE

Incremental cost-effectiveness ratio (ICER). Population EVPI and EVPPI.

RESULTS

At list prices, the ICER was €119,509 per quality-adjusted life year (QALY) (incremental costs €218,092; incremental QALYs 1.82). Probability of cost-effectiveness, at a €45,000 per QALY threshold, was 0%. Population EVPI was €0.00. Population EVPI, at the price of tisagenlecleucel that reduced the ICER to €45,000 per QALY, was €3,989,438. Here, survival analysis had the highest population EVPPI (€1,128,053).

CONCLUSION

Tisagenlecleucel is not cost-effective, versus salvage chemotherapy (with or without haematopoietic stem cell transplant), for R/R DLBCL in Ireland. At list prices, further research to decrease decision uncertainty may not be of value.

Living HTA: Automating Health Economic Evaluation with R

Background: Requiring access to sensitive data can be a significant obstacle for the development of health models in the Health Economics & Outcomes Research (HEOR) setting. We demonstrate how health economic evaluation can be conducted with minimal transfer of data between parties, while automating reporting as new information becomes available. Methods: We developed an automated analysis and reporting pipeline for health economic modelling and made the source code openly available on a GitHub repository. The pipeline consists of three parts: An economic model is constructed by the consultant using pseudo data. On the data-owner side, an application programming interface (API) is hosted on a server. This API hosts all sensitive data, so that data does not have to be provided to the consultant. An automated workflow is created, which calls the API, retrieves results, and generates a report. Results: The application of modern data science tools and practices allows analyses of data without the need for direct access – negating the need to send sensitive data. In addition, the entire workflow can be largely automated: the analysis can be scheduled to run at defined time points (e.g. monthly), or when triggered by an event (e.g. an update to the underlying data or model code); results can be generated automatically and then be exported into a report. Documents no longer need to be revised manually. Conclusions: This example demonstrates that it is possible, within a HEOR setting, to separate the health economic model from the data, and automate the main steps of the analysis pipeline.

Living HTA: Automating Health Technology Assessment with R

Background: Requiring access to sensitive data can be a significant obstacle for the development of health models in the Health Economics & Outcomes Research (HEOR) setting. We demonstrate how health economic evaluation can be conducted with minimal transfer of data between parties, while automating reporting as new information becomes available. Methods: We developed an automated analysis and reporting pipeline for health economic modelling and made the source code openly available on a GitHub repository. The pipeline consists of three parts: An economic model is constructed by the consultant using pseudo data. On the data-owner side, an application programming interface (API) is hosted on a server. This API hosts all sensitive data, so that data does not have to be provided to the consultant. An automated workflow is created, which calls the API, retrieves results, and generates a report. Results: The application of modern data science tools and practices allows analyses of data without the need for direct access – negating the need to send sensitive data. In addition, the entire workflow can be largely automated: the analysis can be scheduled to run at defined time points (e.g. monthly), or when triggered by an event (e.g. an update to the underlying data or model code); results can be generated automatically and then be exported into a report. Documents no longer need to be revised manually. Conclusions: This example demonstrates that it is possible, within a HEOR setting, to separate the health economic model from the data, and automate the main steps of the analysis pipeline.

Tisagenlecleucel for relapsed/refractory acute lymphoblastic leukemia in the Irish healthcare setting: cost-effectiveness and value of information analysis

OBJECTIVES

This study evaluates the cost-effectiveness of tisagenlecleucel (a CAR T-cell therapy), versus blinatumomab, for the treatment of pediatric and young adult patients with relapsed/refractory acute lymphoblastic leukemia (R/R ALL) in the Irish healthcare setting. The value of conducting further research, to investigate the value of uncertainty associated with the decision problem, is assessed by means of expected value of perfect information (EVPI) and partial EVPI (EVPPI) analyses.

METHODS

A three-state partitioned survival model was developed. A short-term decision tree partitioned patients in the tisagenlecleucel arm according to infusion status. Survival was extrapolated to 60 months; general population mortality with a standardized mortality ratio was then applied. Estimated EVPI and EVPPI were scaled up to population according to the incidence of the decision.

RESULTS

At list prices, the incremental cost-effectiveness ratio was EUR 73,086 per quality-adjusted life year (QALY) (incremental costs EUR 156,928; incremental QALYs 2.15). The probability of cost-effectiveness, at the willingness-to-pay threshold of EUR 45,000 per QALY, was 16 percent. At this threshold, population EVPI was EUR 314,455; population EVPPI was below EUR 100,000 for each parameter category.

CONCLUSIONS

Tisagenlecleucel is not cost effective, versus blinatumomab, for the treatment of pediatric and young adult patients with R/R ALL in Ireland (at list prices). Further research to decrease decision (parameter) uncertainty, at the defined willingness-to-pay threshold, may not be of value. However, there is a high degree of uncertainty underpinning the analysis, which may not be captured by EVPI analysis.

A Conceptual Framework for Life-Cycle Health Technology Assessment

OBJECTIVES

Health technology assessment (HTA) uses evidence appraisal and synthesis with economic evaluation to inform adoption decisions. Standard HTA processes sometimes struggle to (1) support decisions that involve significant uncertainty and (2) encourage continued generation of and adaptation to new evidence. We propose the life-cycle (LC)-HTA framework, addressing these challenges by providing additional tools to decision makers and improving outcomes for all stakeholders.

METHODS

Under the LC-HTA framework, HTA processes align to LC management. LC-HTA introduces changes in HTA methods to minimize analytic time while optimizing decision certainty. Where decision uncertainty exists, we recommend risk-based pricing and research-oriented managed access (ROMA). Contractual procurement agreements define the terms of reassessment and provide additional decision options to HTA agencies. LC-HTA extends value-of-information methods to inform ROMA agreements, leveraging routine, administrative data, and registries to reduce uncertainty.

RESULTS

LC-HTA enables the adoption of high-value high-risk innovations while improving health system sustainability through risk-sharing and reducing uncertainty. Responsiveness to evolving evidence is improved through contractually embedded decision rules to simplify reassessment. ROMA allows conditional adoption to obtain additional information, with confidence that the net value of that adoption decision is positive.

CONCLUSIONS

The LC-HTA framework improves outcomes for patients, sponsors, and payers. Patients benefit through earlier access to new technologies. Payers increase the value of the technologies they invest in and gain mechanisms to review investments. Sponsors benefit through greater certainty in outcomes related to their investment, swifter access to markets, and greater opportunities to demonstrate value.

Methods for Extrapolating Survival Analyses for the Economic Evaluation of Advanced Therapy Medicinal Products

There are two significant challenges for analysts conducting economic evaluations of advanced therapy medicinal products (ATMPs): (i) estimating long-term treatment effects in the absence of mature clinical data, and (ii) capturing potentially complex hazard functions. This review identifies and critiques a variety of methods that can be used to overcome these challenges. The narrative review is informed by a rapid literature review of methods used for the extrapolation of survival analyses in the economic evaluation of ATMPs. There are several methods that are more suitable than traditional parametric survival modelling approaches for capturing complex hazard functions, including, cure-mixture models and restricted cubic spline models. In the absence of mature clinical data, analysts may augment clinical trial data with data from other sources to aid extrapolation, however, the relative merits of employing methods for including data from different sources is not well understood. Given the high and potentially irrecoverable costs of making incorrect decisions concerning the reimbursement or commissioning of ATMPs, it is important that economic evaluations are correctly specified, and that both parameter and structural uncertainty associated with survival extrapolations are considered. Value of information analyses allow for this uncertainty to be expressed explicitly, and in monetary terms.

Probabilistic threshold analysis by pairwise stochastic approximation for decision-making under uncertainty

The concept of probabilistic parameter threshold analysis has recently been introduced as a way of probabilistic sensitivity analysis for decision-making under uncertainty, in particular, for health economic evaluations which compare two or more alternative treatments with consideration of uncertainty on outcomes and costs. In this paper we formulate the probabilistic threshold analysis as a root-finding problem involving the conditional expectations, and propose a pairwise stochastic approximation algorithm to search for the threshold value below and above which the choice of conditionally optimal decision options changes. Numerical experiments for both a simple synthetic testcase and a chemotherapy Markov model illustrate the effectiveness of our proposed algorithm, without any need for accurate estimation or approximation of conditional expectations which the existing approaches rely upon. Moreover we introduce a new measure called decision switching probability for probabilistic sensitivity analysis in this paper.

Breathing Synchronised Hypoglossal Nerve Stimulation with Inspire for Untreated Severe Obstructive Sleep Apnoea/Hypopnoea Syndrome: A Simulated Cost-Utility Analysis from a National Health Service Perspective

BACKGROUND

Hypoglossal nerve stimulation (HNS) with Inspire is a novel treatment indicated for moderate or severe obstructive sleep apnoea/hypopnoea syndrome (OSAHS), intolerant to or unable to be treated with continuous positive airway pressure (CPAP).

OBJECTIVE

The aim of this study was to assess the cost effectiveness of treating moderate or severe OSAHS, in patients intolerant to CPAP, with HNS, compared with standard care, from a National Health Service (NHS) perspective.

METHODS

A cohort state transition model was developed to compare HNS with Inspire with no treatment in UK adult patients with moderate or severe OSAHS who have previously tried and have not responded to CPAP therapy. Published literature was applied in the model to estimate incremental cost-effectiveness ratios (ICERs; 2019 Great British pounds per quality-adjusted life-year [QALY] gained), from an NHS and personal social services (PSS) perspective, over a cohort's lifetime.

RESULTS

The model base-case predicts that patients undergoing HNS will incur lifetime costs of £65,026 compared with £36,727 among untreated patients. The HNS cohort would gain 12.72 QALYs compared with 11.15 QALYs in the no-treatment arm. The ICER of treating severe OSAHS with HNS is therefore estimated to be £17,989 per QALYs gained. Probabilistic sensitivity analysis found that at a threshold of £30,000/QALY, HNS has a 69% probability of being cost effective. Limitations of the model include uncertainty around the utility data that were not sourced directly from HNS clinical trials. There is further uncertainty in the relationship between change in the Apnoea-Hypopnoea Index (AHI) and reduction in ischaemic heart disease and stroke because of difficulty capturing the reduction in risk over a long time horizon in studies.

CONCLUSIONS

Over a patient's lifetime, HNS with Inspire is expected to be cost effective when compared with no treatment in patients with severe OSAHS who have tried and have not responded to CPAP, from an NHS perspective.

Cost-effective clinical trial design: Application of a Bayesian sequential model to the ProFHER pragmatic trial

Background/Aims:

There is growing interest in the use of adaptive designs to improve the efficiency of clinical trials. We apply a Bayesian decision-theoretic model of a sequential experiment using cost and outcome data from the ProFHER pragmatic trial. We assess the model’s potential for delivering value-based research.

Methods:

Using parameter values estimated from the ProFHER pragmatic trial, including the costs of carrying out the trial, we establish when the trial could have stopped, had the model’s value-based stopping rule been used. We use a bootstrap analysis and simulation study to assess a range of operating characteristics, which we compare with a fixed sample size design which does not allow for early stopping.

Results:

We estimate that application of the model could have stopped the ProFHER trial early, reducing the sample size by about 14%, saving about 5% of the research budget and resulting in a technology recommendation which was the same as that of the trial. The bootstrap analysis suggests that the expected sample size would have been 38% lower, saving around 13% of the research budget, with a probability of 0.92 of making the same technology recommendation decision. It also shows a large degree of variability in the trial’s sample size.

Conclusions:

Benefits to trial cost stewardship may be achieved by monitoring trial data as they accumulate and using a stopping rule which balances the benefit of obtaining more information through continued recruitment with the cost of obtaining that information. We present recommendations for further research investigating the application of value-based sequential designs.

Multilevel and Quasi Monte Carlo Methods for the Calculation of the Expected Value of Partial Perfect Information

The expected value of partial perfect information (EVPPI) provides an upper bound on the value of collecting further evidence on a set of inputs to a cost-effectiveness decision model. Standard Monte Carlo estimation of EVPPI is computationally expensive as it requires nested simulation. Alternatives based on regression approximations to the model have been developed but are not practicable when the number of uncertain parameters of interest is large and when parameter estimates are highly correlated. The error associated with the regression approximation is difficult to determine, while MC allows the bias and precision to be controlled. In this article, we explore the potential of quasi Monte Carlo (QMC) and multilevel Monte Carlo (MLMC) estimation to reduce the computational cost of estimating EVPPI by reducing the variance compared with MC while preserving accuracy. We also develop methods to apply QMC and MLMC to EVPPI, addressing particular challenges that arise where Markov chain Monte Carlo (MCMC) has been used to estimate input parameter distributions. We illustrate the methods using 2 examples: a simplified decision tree model for treatments for depression and a complex Markov model for treatments to prevent stroke in atrial fibrillation, both of which use MCMC inputs. We compare the performance of QMC and MLMC with MC and the approximation techniques of generalized additive model (GAM) regression, Gaussian process (GP) regression, and integrated nested Laplace approximations (INLA-GP). We found QMC and MLMC to offer substantial computational savings when parameter sets are large and correlated and when the EVPPI is large. We also found that GP and INLA-GP were biased in those situations, whereas GAM cannot estimate EVPPI for large parameter sets.

Prehospital continuous positive airway pressure for acute respiratory failure: the ACUTE feasibility RCT

BACKGROUND

Acute respiratory failure is a life-threatening emergency. Standard prehospital management involves controlled oxygen therapy. Continuous positive airway pressure is a potentially beneficial alternative treatment; however, it is uncertain whether or not this treatment could improve outcomes in NHS ambulance services.

OBJECTIVES

To assess the feasibility of a large-scale pragmatic trial and to update an existing economic model to determine cost-effectiveness and the value of further research.

DESIGN

(1) An open-label, individual patient randomised controlled external pilot trial. (2) Cost-effectiveness and value-of-information analyses, updating an existing economic model. (3) Ancillary substudies, comprising an acute respiratory failure incidence study, an acute respiratory failure diagnostic agreement study, clinicians perceptions of a continuous positive airway pressure mixed-methods study and an investigation of allocation concealment.

SETTING

Four West Midlands Ambulance Service hubs, recruiting between August 2017 and July 2018.

PARTICIPANTS

Adults with respiratory distress and peripheral oxygen saturations below the British Thoracic Society's target levels were included. Patients with limited potential to benefit from, or with contraindications to, continuous positive airway pressure were excluded.

INTERVENTIONS

Prehospital continuous positive airway pressure (O-Two system, O-Two Medical Technologies Inc., Brampton, ON, Canada) was compared with standard oxygen therapy, titrated to the British Thoracic Society's peripheral oxygen saturation targets. Interventions were provided in identical sealed boxes.

MAIN OUTCOME MEASURES

Feasibility objectives estimated the incidence of eligible patients, the proportion recruited and allocated to treatment appropriately, adherence to allocated treatment, and retention and data completeness. The primary clinical end point was 30-day mortality.

RESULTS

Seventy-seven patients were enrolled (target 120 patients), including seven patients with a diagnosis for which continuous positive airway pressure could be ineffective or harmful. Continuous positive airway pressure was fully delivered to 74% of participants (target 75%). There were no major protocol violations/non-compliances. Full data were available for all key outcomes (target ≥ 90%). Thirty-day mortality was 27.3%. Of the 21 deceased participants, 14 (68%) either did not have a respiratory condition or had ceiling-of-treatment decision implemented that excluded hospital non-invasive ventilation and critical care. The base-case economic evaluation indicated that standard oxygen therapy was probably cost-effective (incremental cost-effectiveness ratio £5685 per quality-adjusted life-year), but there was considerable uncertainty (population expected value of perfect information of £16.5M). Expected value of partial perfect information analyses indicated that effectiveness of prehospital continuous positive airway pressure was the only important variable. The incidence rate of acute respiratory failure was 17.4 (95% confidence interval 16.3 to 18.5) per 100,000 persons per year. There was moderate agreement between the primary prehospital and final hospital diagnoses (Gwet's AC1 coefficient 0.56, 95% confidence interval 0.43 to 0.69). Lack of hospital awareness of the Ambulance continuous positive airway pressure (CPAP): Use, Treatment Effect and economics (ACUTE) trial, limited time to complete trial training and a desire to provide continuous positive airway pressure treatment were highlighted as key challenges by participating clinicians.

LIMITATIONS

During week 10 of recruitment, the continuous positive airway pressure arm equipment boxes developed a 'rattle'. After repackaging and redistribution, no further concerns were noted. A total of 41.4% of ambulance service clinicians not participating in the ACUTE trial indicated a difference between the control and the intervention arm trial boxes (115/278); of these clinician 70.4% correctly identified box contents.

CONCLUSIONS

Recruitment rate was below target and feasibility was not demonstrated. The economic evaluation results suggested that a definitive trial could represent value for money. However, limited compliance with continuous positive airway pressure and difficulty in identifying patients who could benefit from continuous positive airway pressure indicate that prehospital continuous positive airway pressure is unlikely to materially reduce mortality.

FUTURE WORK

A definitive clinical effectiveness trial of continuous positive airway pressure in the NHS is not recommended.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN12048261.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 25, No. 7. See the NIHR Journals Library website for further project information.

Cost-effectiveness of out-of-hospital continuous positive airway pressure for acute respiratory failure: decision analytic modelling using data from a feasibility trial

Background

Standard prehospital management for Acute respiratory failure (ARF) involves controlled oxygen therapy. Continuous positive airway pressure (CPAP) is a potentially beneficial alternative treatment, however, it is uncertain whether this could improve outcomes and provide value for money. This study aimed to evaluate the cost-effectiveness of prehospital CPAP in ARF.

Methods

A cost-utility economic evaluation was performed using a probabilistic decision tree model synthesising available evidence. The model consisted of a hypothetical cohort of patients in a representative ambulance service with undifferentiated ARF, receiving standard oxygen therapy or prehospital CPAP. Costs and quality adjusted life years (QALYs) were estimated using methods recommended by NICE.

Results

In the base case analysis, using CPAP effectiveness estimates form the ACUTE trial, the mean expected costs of standard care and prehospital CPAP were £15,201 and £14,850 respectively and the corresponding mean expected QALYs were 1.190 and 1.128, respectively. The mean ICER estimated as standard oxygen therapy compared to prehospital CPAP was £5685 per QALY which indicated that standard oxygen therapy strategy was likely to be cost-effective at a threshold of £20,000 per QALY (67% probability). The scenario analysis, using effectiveness estimates from an updated meta-analysis, suggested that prehospital CPAP was more effective (mean incremental QALYs of 0.157), but also more expensive (mean incremental costs of £1522), than standard care. The mean ICER, estimated as prehospital CPAP compared to standard care, was £9712 per QALY. At the £20,000 per QALY prehospital CPAP was highly likely to be the most cost-effective strategy (94%).

Conclusions

Cost-effectiveness of prehospital CPAP depends upon the estimate of effectiveness. When based on a small pragmatic feasibility trial, standard oxygen therapy is cost-effective. When based on meta-analysis of heterogeneous trials, CPAP is cost-effective. Value of information analyses support commissioning of a large pragmatic effectiveness trial, providing feasibility and plausibility conditions are met.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12873-021-00404-8.

Probabilistic One-Way Sensitivity Analysis with Multiple Comparators: The Conditional Net Benefit Frontier

Although there have been substantial developments in the analysis of uncertainty in economic evaluations of health care programmes, the development of methods for one-way sensitivity analysis has been notably slower. Conditional incremental net benefit was recently proposed as an approach for implementing probabilistic one-way sensitivity analysis for economic evaluations comparing two strategies. In this paper, we generalise this approach to economic evaluations that compare three or more strategies. We find that 'conditional net benefit' may be used to conduct probabilistic one-way sensitivity analysis for economic evaluations comparing any number of strategies. We also propose the 'conditional net benefit frontier', which may be used to identify the most cost-effective of any number of strategies conditional upon the specific value of a parameter of interest.

Decision Modeling for Economic Evaluation in Otolaryngology-Head and Neck Surgery: Review of Techniques

Decision making in health care is complex, and substantial uncertainty can be involved. Structured, systematic approaches to the integration of available evidence, assessment of uncertainty, and determination of choice are of significant benefit in an era of "value-based care." This is especially true for otolaryngology-head and neck surgery, where technological advancements are frequent and applicable to an array of subspecialties. Decision analysis aims to achieve these goals through various modeling techniques, including (1) decision trees, (2) Markov process, (3) microsimulation, and (4) discrete event simulation. While decision models have been used for decades, many clinicians and researchers continue to have difficulty deciphering them. In this review, we present an overview of various decision analysis modeling techniques, their purposes, how they can be interpreted, and commonly used syntax to promote understanding and use of these approaches. Throughout, we provide a sample research question to facilitate discussion of the advantages and disadvantages of each technique.

A Computationally Efficient Method for Probabilistic Parameter Threshold Analysis for Health Economic Evaluations

Background. Threshold analysis is used to determine the threshold value of an input parameter at which a health care strategy becomes cost-effective. Typically, it is performed in a deterministic manner, in which inputs are varied one at a time while the remaining inputs are each fixed at their mean value. This approach will result in incorrect threshold values if the cost-effectiveness model is nonlinear or if inputs are correlated. Objective. To propose a probabilistic method for performing threshold analysis, which accounts for the joint uncertainty in all input parameters and makes no assumption about the linearity of the cost-effectiveness model. Methods. Three methods are compared: 1) deterministic threshold analysis (DTA); 2) a 2-level Monte Carlo approach, which is considered the gold standard; and 3) a regression-based method using a generalized additive model (GAM), which identifies threshold values directly from a probabilistic sensitivity analysis sample. Results. We applied the 3 methods to estimate the minimum probability of hospitalization for typhoid fever at which 3 different vaccination strategies become cost-effective in Uganda. The threshold probability of hospitalization at which routine vaccination at 9 months with catchup campaign to 5 years becomes cost-effective is estimated to be 0.060 and 0.061 (95% confidence interval [CI], 0.058–0.064), respectively, for 2-level and GAM. According to DTA, routine vaccination at 9 months with catchup campaign to 5 years would never become cost-effective. The threshold probability at which routine vaccination at 9 months with catchup campaign to 15 years becomes cost-effective is estimated to be 0.092 (DTA), 0.074 (2-level), and 0.072 (95% CI, 0.069–0.075) (GAM). GAM is 430 times faster than the 2-level approach. Conclusions. When the cost-effectiveness model is nonlinear, GAM provides similar threshold values to the 2-level Monte Carlo approach and is computationally more efficient. DTA provides incorrect results and should not be used.

Computing the Expected Value of Sample Information Efficiently: Practical Guidance and Recommendations for Four Model-Based Methods

Value of information (VOI) analyses can help policy makers make informed decisions about whether to conduct and how to design future studies. Historically a computationally expensive method to compute the expected value of sample information (EVSI) restricted the use of VOI to simple decision models and study designs. Recently, 4 EVSI approximation methods have made such analyses more feasible and accessible. Members of the Collaborative Network for Value of Information (ConVOI) compared the inputs, the analyst's expertise and skills, and the software required for the 4 recently developed EVSI approximation methods. Our report provides practical guidance and recommendations to help inform the choice between the 4 efficient EVSI estimation methods. More specifically, this report provides: (1) a step-by-step guide to the methods' use, (2) the expertise and skills required to implement the methods, and (3) method recommendations based on the features of decision-analytic problems.

Calculating the Expected Value of Sample Information in Practice: Considerations from 3 Case Studies

Background. Investing efficiently in future research to improve policy decisions is an important goal. Expected value of sample information (EVSI) can be used to select the specific design and sample size of a proposed study by assessing the benefit of a range of different studies. Estimating EVSI with the standard nested Monte Carlo algorithm has a notoriously high computational burden, especially when using a complex decision model or when optimizing over study sample sizes and designs. Recently, several more efficient EVSI approximation methods have been developed. However, these approximation methods have not been compared, and therefore their comparative performance across different examples has not been explored. Methods. We compared 4 EVSI methods using 3 previously published health economic models. The examples were chosen to represent a range of real-world contexts, including situations with multiple study outcomes, missing data, and data from an observational rather than a randomized study. The computational speed and accuracy of each method were compared. Results. In each example, the approximation methods took minutes or hours to achieve reasonably accurate EVSI estimates, whereas the traditional Monte Carlo method took weeks. Specific methods are particularly suited to problems where we wish to compare multiple proposed sample sizes, when the proposed sample size is large, or when the health economic model is computationally expensive. Conclusions. As all the evaluated methods gave estimates similar to those given by traditional Monte Carlo, we suggest that EVSI can now be efficiently computed with confidence in realistic examples. No systematically superior EVSI computation method exists as the properties of the different methods depend on the underlying health economic model, data generation process, and user expertise.

One-Way Sensitivity Analysis for Probabilistic Cost-Effectiveness Analysis: Conditional Expected Incremental Net Benefit

Although probabilistic analysis has become the accepted standard for decision analytic cost-effectiveness models, deterministic one-way sensitivity analysis continues to be used to meet the need of decision makers to understand the impact that changing the value taken by one specific parameter has on the results of the analysis. The value of a probabilistic form of one-way sensitivity analysis has been recognised, but the proposed methods are computationally intensive. Deterministic one-way sensitivity analysis provides decision makers with biased and incomplete information whereas, in contrast, probabilistic one-way sensitivity analysis (POSA) can overcome these limitations, an observation supported in this study by results obtained when these methods were applied to a previously published cost-effectiveness analysis to produce a conditional incremental expected net benefit curve. The application of POSA will provide decision makers with unbiased information on how the expected net benefit is affected by a parameter taking on a specific value and the probability that the specific value will be observed.

A Framework to Prioritise Health Research Proposals for Funding: Integrating Value for Money

When making funding decisions, research organisations largely consider the merits (e.g. scientific rigour and feasibility) of submitted research proposals; yet, there is often little or no reference to their value for money. This may be attributed to the challenges of assessing and integrating value of research into existing research prioritisation processes. We propose a framework that considers both the merits of research and its value for money to guide health research funding decisions. A practical framework is developed based on current processes followed by funding organizations for assessing investigator-initiated research proposals, and analytical methods for evaluating the expected value of research. We apply the analytical methods to estimate the expected return on investment of two real-world grant applications. The framework comprises four sequential steps: (1) initial screening of applications for eligibility and completeness; (2) merit assessment of eligible proposals; (3) estimating the expected value of research for the shortlisted proposals that pass the first two steps and ranking of proposals based on return on investment; and (4) selecting research proposals for funding. We demonstrate how the expected value for money can be efficiently estimated using certain information provided in funding applications. The proposed framework integrates value-for-money assessment into the existing research prioritisation processes. Considering value for money to inform research funding decisions is vital to achieve efficient utilisation of research budgets and maximise returns on research investments.

Estimating the Expected Value of Sample Information across Different Sample Sizes Using Moment Matching and Nonlinear Regression

Background. The expected value of sample information (EVSI) determines the economic value of any future study with a specific design aimed at reducing uncertainty about the parameters underlying a health economic model. This has potential as a tool for trial design; the cost and value of different designs could be compared to find the trial with the greatest net benefit. However, despite recent developments, EVSI analysis can be slow, especially when optimizing over a large number of different designs. Methods. This article develops a method to reduce the computation time required to calculate the EVSI across different sample sizes. Our method extends the moment-matching approach to EVSI estimation to optimize over different sample sizes for the underlying trial while retaining a similar computational cost to a single EVSI estimate. This extension calculates the posterior variance of the net monetary benefit across alternative sample sizes and then uses Bayesian nonlinear regression to estimate the EVSI across these sample sizes. Results. A health economic model developed to assess the cost-effectiveness of interventions for chronic pain demonstrates that this EVSI calculation method is fast and accurate for realistic models. This example also highlights how different trial designs can be compared using the EVSI. Conclusion. The proposed estimation method is fast and accurate when calculating the EVSI across different sample sizes. This will allow researchers to realize the potential of using the EVSI to determine an economically optimal trial design for reducing uncertainty in health economic models. Limitations. Our method involves rerunning the health economic model, which can be more computationally expensive than some recent alternatives, especially in complex models.

Early cost-effectiveness modeling for better decisions in public research investment of personalized medicine technologies

Millions of dollars are spent on the development of new personalized medicine technologies. While these research costs are often supported by public research funds, many diagnostic tests and biomarkers are not adopted by the healthcare system due to lack of evidence on their cost-effectiveness. We describe a stepwise approach to conducting cost-effectiveness analyses that are performed early in the technology's development process and can help mitigate the potential risks of investment. Decision analytic modeling can identify the key drivers of cost effectiveness and provide minimum criteria that the technology needs to meet for adoption by public and private healthcare systems. A value of information analysis can quantify the added value of conducting more research to provide further evidence for policy decisions. These steps will allow public research funders to make better decisions on their investments to maximize the health benefits and to minimize the number of suboptimal technologies.

Calculating the Expected Value of Sample Information Using Efficient Nested Monte Carlo: A Tutorial

OBJECTIVE

The expected value of sample information (EVSI) quantifies the economic benefit of reducing uncertainty in a health economic model by collecting additional information. This has the potential to improve the allocation of research budgets. Despite this, practical EVSI evaluations are limited partly due to the computational cost of estimating this value using the gold-standard nested simulation methods. Recently, however, Heath et al. developed an estimation procedure that reduces the number of simulations required for this gold-standard calculation. Up to this point, this new method has been presented in purely technical terms.

STUDY DESIGN

This study presents the practical application of this new method to aid its implementation. We use a worked example to illustrate the key steps of the EVSI estimation procedure before discussing its optimal implementation using a practical health economic model.

METHODS

The worked example is based on a three-parameter linear health economic model. The more realistic model evaluates the cost-effectiveness of a new chemotherapy treatment, which aims to reduce the number of side effects experienced by patients. We use a Markov model structure to evaluate the health economic profile of experiencing side effects.

RESULTS

This EVSI estimation method offers accurate estimation within a feasible computation time, seconds compared to days, even for more complex model structures. The EVSI estimation is more accurate if a greater number of nested samples are used, even for a fixed computational cost.

CONCLUSIONS

This new method reduces the computational cost of estimating the EVSI by nested simulation.

Value of Information Choices that Influence Estimates: A Systematic Review of Prevailing Considerations

Background. Although value of information (VOI) analyses are increasingly advocated and used for research prioritization and reimbursement decisions, the interpretation and usefulness of VOI outcomes depend critically on the underlying choices and assumptions used in the analysis. In this article, we present a structured overview of all items reported in literature to potentially influence VOI outcomes. Use of this overview increases awareness and transparency of choices and assumptions underpinning VOI outcomes. Methods. A systematic literature review was performed to identify aspects of VOI analyses that were found to potentially influence VOI outcomes. Identified aspects were grouped to develop a structured overview. Explanations were defined for all items included in the overview. Results. We retrieved 687 unique papers, of which 71 original papers and 8 reviews were included. In the full text of these 79 papers, 16 aspects were found that may influence VOI outcomes. These aspects related to the underlying evidence (bias, synthesis, heterogeneity, correlation), uncertainty (structural, future pricing), model (relevance, approach, population), choices in VOI calculation (estimation technique, implementation level, population size, perspective), and aspects specifically for assessing the value of future study designs (reversal costs, efficient estimator). These aspects were aggregated into 7 items to provide a structured overview. Conclusion. The developed overview should increase awareness of key choices underlying VOI analysis and facilitate structured reporting of such choices and interpretation of the ensuing VOI outcomes by researchers and policy makers. Use of this overview should improve prioritization and reimbursement decisions.

Decision analytic modeling for the economic analysis of proton radiotherapy for non-small cell lung cancer

Background

Although proton radiation treatments are more costly than photon/X-ray therapy, they may lower overall treatment costs through reducing rates of severe toxicities and the costly management of those toxicities. To study this issue, we created a decision-model comparing proton vs. X-ray radiotherapy for locally advanced non-small cell lung cancer patients.

Methods

An influence diagram was created to model for radiation delivery, associated 6-month pneumonitis/esophagitis rates, and overall costs (radiation plus toxicity costs). Pneumonitis (age, chemo type, V20, MLD) and esophagitis (V60) predictors were modeled to impact toxicity rates. We performed toxicity-adjusted, rate-adjusted, risk group-adjusted, and radiosensitivity analyses.

Results

Upfront proton treatment costs exceeded that of photons [$16,730.37 (3DCRT), $23,893.83 (IMRT), $41,061.80 (protons)]. Based upon expected population pneumonitis and esophagitis rates for each modality, protons would be expected to recover $1,065.62 and $1,139.63 of the cost difference compared to 3DCRT or IMRT. For patients treated with IMRT experiencing grade 4 pneumonitis or grade 4 esophagitis, costs exceeded patients treated with protons without this toxicity. 3DCRT patients with grade 4 esophagitis had higher costs than proton patients without this toxicity. For the risk group analysis, high risk patients (age >65, carboplatin/paclitaxel) benefited more from proton therapy. A biomarker may allow patient selection for proton therapy, although the AUC alone is not sufficient to determine if the biomarker is clinically useful.

Conclusions

The comparison between proton and photon/X-ray radiation therapy for NSCLC needs to consider both the up-front cost of treatment and the possible long term cost of complications. In our analysis, current costs favor X-ray therapy. However, relatively small reductions in the cost of proton therapy may result in a shift to the preference for proton therapy.

Cost-Effectiveness and Value of Information Analysis of Brief Interventions to Promote Physical Activity in Primary Care

BACKGROUND

Brief interventions (BIs) delivered in primary care have shown potential to increase physical activity levels and may be cost-effective, at least in the short-term, when compared with usual care. Nevertheless, there is limited evidence on their longer term costs and health benefits.

OBJECTIVES

To estimate the cost-effectiveness of BIs to promote physical activity in primary care and to guide future research priorities using value of information analysis.

METHODS

A decision model was used to compare the cost-effectiveness of three classes of BIs that have been used, or could be used, to promote physical activity in primary care: 1) pedometer interventions, 2) advice/counseling on physical activity, and (3) action planning interventions. Published risk equations and data from the available literature or routine data sources were used to inform model parameters. Uncertainty was investigated with probabilistic sensitivity analysis, and value of information analysis was conducted to estimate the value of undertaking further research.

RESULTS

In the base-case, pedometer interventions yielded the highest expected net benefit at a willingness to pay of £20,000 per quality-adjusted life-year. There was, however, a great deal of decision uncertainty: the expected value of perfect information surrounding the decision problem for the National Health Service Health Check population was estimated at £1.85 billion.

CONCLUSIONS

Our analysis suggests that the use of pedometer BIs is the most cost-effective strategy to promote physical activity in primary care, and that there is potential value in further research into the cost-effectiveness of brief (i.e., <30 minutes) and very brief (i.e., <5 minutes) pedometer interventions in this setting.

Cost Effectiveness of the Angiotensin Receptor Neprilysin Inhibitor Sacubitril/Valsartan for Patients with Chronic Heart Failure and Reduced Ejection Fraction in the Netherlands: A Country Adaptation Analysis Under the Former and Current Dutch Pharmacoeconomic Guidelines

OBJECTIVES

To describe the adaptation of a global health economic model to determine whether treatment with the angiotensin receptor neprilysin inhibitor LCZ696 is cost effective compared with the angiotensin-converting enzyme inhibitor enalapril in adult patients with chronic heart failure with reduced left ventricular ejection fraction in the Netherlands; and to explore the effect of performing the cost-effectiveness analyses according to the new pharmacoeconomic Dutch guidelines (updated during the submission process of LCZ696), which require a value-of-information analysis and the inclusion of indirect medical costs of life-years gained.

METHODS

We adapted a UK model to reflect the societal perspective in the Netherlands by including travel expenses, productivity loss, informal care costs, and indirect medical costs during the life-years gained and performed a preliminary value-of-information analysis.

RESULTS

The incremental cost-effectiveness ratio obtained was €17,600 per quality-adjusted life-year (QALY) gained. This was robust to changes in most structural assumptions and across different subgroups of patients. Probability sensitivity analysis results showed that the probability that LCZ696 is cost-effective at a €50,000 per QALY threshold is 99.8%, with a population expected value of perfect information of €297,128. On including indirect medical costs of life-years gained, the incremental cost-effectiveness ratio was €26,491 per QALY gained, and LCZ696 was 99.46% cost effective at €50,000 per QALY, with a population expected value of perfect information of €2,849,647.

CONCLUSIONS

LCZ696 is cost effective compared with enalapril under the former and current Dutch guidelines. However, the (monetary) consequences of making a wrong decision were considerably different in both scenarios.

Efficient Monte Carlo Estimation of the Expected Value of Sample Information Using Moment Matching

BACKGROUND

The Expected Value of Sample Information (EVSI) is used to calculate the economic value of a new research strategy. Although this value would be important to both researchers and funders, there are very few practical applications of the EVSI. This is due to computational difficulties associated with calculating the EVSI in practical health economic models using nested simulations.

METHODS

We present an approximation method for the EVSI that is framed in a Bayesian setting and is based on estimating the distribution of the posterior mean of the incremental net benefit across all possible future samples, known as the distribution of the preposterior mean. Specifically, this distribution is estimated using moment matching coupled with simulations that are available for probabilistic sensitivity analysis, which is typically mandatory in health economic evaluations.

RESULTS

This novel approximation method is applied to a health economic model that has previously been used to assess the performance of other EVSI estimators and accurately estimates the EVSI. The computational time for this method is competitive with other methods.

CONCLUSION

We have developed a new calculation method for the EVSI which is computationally efficient and accurate.

LIMITATIONS

This novel method relies on some additional simulation so can be expensive in models with a large computational cost.

Examining the Feasibility and Utility of Estimating Partial Expected Value of Perfect Information (via a Nonparametric Approach) as Part of the Reimbursement Decision-Making Process in Ireland: Application to Drugs for Cancer

BACKGROUND

In Ireland, all new drugs for which reimbursement by the healthcare payer is sought undergo a health technology assessment by the National Centre for Pharmacoeconomics. The National Centre for Pharmacoeconomics estimate expected value of perfect information but not partial expected value of perfect information (owing to computational expense associated with typical methodologies).

OBJECTIVE

The objective of this study was to examine the feasibility and utility of estimating partial expected value of perfect information via a computationally efficient, non-parametric regression approach.

METHODS

This was a retrospective analysis of evaluations on drugs for cancer that had been submitted to the National Centre for Pharmacoeconomics (January 2010 to December 2014 inclusive). Drugs were excluded if cost effective at the submitted price. Drugs were excluded if concerns existed regarding the validity of the applicants' submission or if cost-effectiveness model functionality did not allow required modifications to be made. For each included drug (n = 14), value of information was estimated at the final reimbursement price, at a threshold equivalent to the incremental cost-effectiveness ratio at that price. The expected value of perfect information was estimated from probabilistic analysis. Partial expected value of perfect information was estimated via a non-parametric approach. Input parameters with a population value at least €1 million were identified as potential targets for research.

RESULTS

All partial estimates were determined within minutes. Thirty parameters (across nine models) each had a value of at least €1 million. These were categorised. Collectively, survival analysis parameters were valued at €19.32 million, health state utility parameters at €15.81 million and parameters associated with the cost of treating adverse effects at €6.64 million. Those associated with drug acquisition costs and with the cost of care were valued at €6.51 million and €5.71 million, respectively.

CONCLUSION

This research demonstrates that the estimation of partial expected value of perfect information via this computationally inexpensive approach could be considered feasible as part of the health technology assessment process for reimbursement purposes within the Irish healthcare system. It might be a useful tool in prioritising future research to decrease decision uncertainty.

A Review of Methods for Analysis of the Expected Value of Information

In recent years, value-of-information analysis has become more widespread in health economic evaluations, specifically as a tool to guide further research and perform probabilistic sensitivity analysis. This is partly due to methodological advancements allowing for the fast computation of a typical summary known as the expected value of partial perfect information (EVPPI). A recent review discussed some approximation methods for calculating the EVPPI, but as the research has been active over the intervening years, that review does not discuss some key estimation methods. Therefore, this paper presents a comprehensive review of these new methods. We begin by providing the technical details of these computation methods. We then present two case studies in order to compare the estimation performance of these new methods. We conclude that a method based on nonparametric regression offers the best method for calculating the EVPPI in terms of accuracy, computational time, and ease of implementation. This means that the EVPPI can now be used practically in health economic evaluations, especially as all the methods are developed in parallel with R functions and a web app to aid practitioners.

The Head Injury Transportation Straight to Neurosurgery (HITS-NS) randomised trial: a feasibility study

BACKGROUND

Reconfiguration of trauma services, with direct transport of traumatic brain injury (TBI) patients to neuroscience centres (NCs), bypassing non-specialist acute hospitals (NSAHs), could potentially improve outcomes. However, delays in stabilisation of airway, breathing and circulation (ABC) and the difficulties in reliably identifying TBI at scene may make this practice deleterious compared with selective secondary transfer from nearest NSAH to NC. National Institute for Health and Care Excellence guidance and systematic reviews suggested equipoise and poor-quality evidence - with regard to 'early neurosurgery' in this cohort - which we sought to address.

METHODS

Pilot cluster randomised controlled trial of bypass to NC conducted in two ambulance services with the ambulance station (n = 74) as unit of cluster [Lancashire/Cumbria in the North West Ambulance Service (NWAS) and the North East Ambulance Service (NEAS)]. Adult patients with signs of isolated TBI [Glasgow Coma Scale (GCS) score of < 13 in NWAS, GCS score of < 14 in NEAS] and stable ABC, injured nearest to a NSAH were transported either to that hospital (control clusters) or bypassed to the nearest NC (intervention clusters).

PRIMARY OUTCOMES

recruitment rate, protocol compliance, selection bias as a result of non-compliance, accuracy of paramedic TBI identification (overtriage of study inclusion criteria) and pathway acceptability to patients, families and staff. 'Open-label' secondary outcomes: 30-day mortality, 6-month Extended Glasgow Outcome Scale (GOSE) and European Quality of Life-5 Dimensions.

RESULTS

Overall, 56 clusters recruited 293 (169 intervention, 124 control) patients in 12 months, demonstrating cluster randomised pre-hospital trials as viable for heath service evaluations. Overall compliance was 62%, but 90% was achieved in the control arm and when face-to-face paramedic training was possible. Non-compliance appeared to be driven by proximity of the nearest hospital and perceptions of injury severity and so occurred more frequently in the intervention arm, in which the perceived time to the NC was greater and severity of injury was lower. Fewer than 25% of recruited patients had TBI on computed tomography scan (n = 70), with 7% (n = 20) requiring neurosurgery (craniotomy, craniectomy or intracranial pressure monitoring) but a further 18 requiring admission to an intensive care unit. An intention-to-treat analysis revealed the two trial arms to be equivalent in terms of age, GCS and severity of injury. No significant 30-day mortality differences were found (8.8% vs. 9.1/%; p > 0.05) in the 273 (159/113) patients with data available. There were no apparent differences in staff and patient preferences for either pathway, with satisfaction high with both. Very low responses to invitations to consent for follow-up in the large number of mild head injury-enrolled patients meant that only 20% of patients had 6-month outcomes. The trial-based economic evaluation could not focus on early neurosurgery because of these low numbers but instead investigated the comparative cost-effectiveness of bypass compared with selective secondary transfer for eligible patients at the scene of injury.

CONCLUSIONS

Current NHS England practice of bypassing patients with suspected TBI to neuroscience centres gives overtriage ratios of 13 : 1 for neurosurgery and 4 : 1 for TBI. This important finding makes studying the impact of bypass to facilitate early neurosurgery not plausible using this study design. Future research should explore an efficient comparative effectiveness design for evaluating 'early neurosurgery through bypass' and address the challenge of reliable TBI diagnosis at the scene of injury.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN68087745.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 20, No. 1. See the NIHR Journals Library website for further project information.

Efficient Value of Information Calculation Using a Nonparametric Regression Approach: An Applied Perspective

BACKGROUND

Value-of-information (VOI) analysis provides an analytical framework to assess whether obtaining additional evidence is worthwhile to reduce decision uncertainty. The reporting of VOI measures, particularly the expected value of perfect parameter information (EVPPI) and the expected value of sample information (EVSI), is limited because of the computational burden associated with typical two-level Monte-Carlo-based solution. Recently, a nonparametric regression approach was proposed that allows the estimation of multiparameter EVPPI and EVSI directly from a probabilistic sensitivity analysis sample.

OBJECTIVES

To demonstrate the value of the nonparametric regression approach in calculating VOI measures in real-world cases and to compare its performance with the standard approach of the Monte-Carlo simulation.

METHODS

We used the regression approach to calculate EVPPI and EVSI in two models, and compared the results with the estimates obtained via the standard Monte-Carlo simulation.

RESULTS

The VOI values from the two approaches were very close; computation using the regression method, however, was faster.

CONCLUSION

The nonparametric regression approach provides an efficient and easy-to-implement alternative for EVPPI and EVSI calculation in economic models.

Decisions on Further Research for Predictive Biomarkers of High-Dose Alkylating Chemotherapy in Triple-Negative Breast Cancer: A Value of Information Analysis

OBJECTIVES

To inform decisions about the design and priority of further studies of emerging predictive biomarkers of high-dose alkylating chemotherapy (HDAC) in triple-negative breast cancer (TNBC) using value-of-information analysis.

METHODS

A state transition model compared treating women with TNBC with current clinical practice and four biomarker strategies to personalize HDAC: 1) BRCA1-like profile by array comparative genomic hybridization (aCGH) testing; 2) BRCA1-like profile by multiplex ligation-dependent probe amplification (MLPA) testing; 3) strategy 1 followed by X-inactive specific transcript gene (XIST) and tumor suppressor p53 binding protein (53BP1) testing; and 4) strategy 2 followed by XIST and 53BP1 testing, from a Dutch societal perspective and a 20-year time horizon. Input data came from literature and expert opinions. We assessed the expected value of partial perfect information, the expected value of sample information, and the expected net benefit of sampling for potential ancillary studies of an ongoing randomized controlled trial (RCT; NCT01057069).

RESULTS

The expected value of partial perfect information indicated that further research should be prioritized to the parameter group including "biomarkers' prevalence, positive predictive value (PPV), and treatment response rates (TRRs) in biomarker-negative patients and patients with TNBC" (€639 million), followed by utilities (€48 million), costs (€40 million), and transition probabilities (TPs) (€30 million). By setting up four ancillary studies to the ongoing RCT, data on 1) TP and MLPA prevalence, PPV, and TRR; 2) aCGH and aCGH/MLPA plus XIST and 53BP1 prevalence, PPV, and TRR; 3) utilities; and 4) costs could be simultaneously collected (optimal size = 3000).

CONCLUSIONS

Further research on predictive biomarkers for HDAC should focus on gathering data on TPs, prevalence, PPV, TRRs, utilities, and costs from the four ancillary studies to the ongoing RCT.

The management of Otitis Media with Effusion in children with cleft palate (mOMEnt): a feasibility study and economic evaluation

BACKGROUND

Cleft lip and palate are among the most common congenital malformations, with an incidence of around 1 in 700. Cleft palate (CP) results in impaired Eustachian tube function, and 90% of children with CP have otitis media with effusion (OME) histories. There are several approaches to management, including watchful waiting, the provision of hearing aids (HAs) and the insertion of ventilation tubes (VTs). However, the evidence underpinning these strategies is unclear and there is a need to determine which treatment is the most appropriate.

OBJECTIVES

To identify the optimum study design, increase understanding of the impact of OME, determine the value of future research and develop a core outcome set (COS) for use in future studies.

DESIGN

The management of Otitis Media with Effusion in children with cleft palate (mOMEnt) study had four key components: (i) a survey evaluation of current clinical practice in each cleft centre; (ii) economic modelling and value of information (VOI) analysis to determine if the extent of existing decision uncertainty justifies the cost of further research; (iii) qualitative research to capture patient and parent opinion regarding willingness to participate in a trial and important outcomes; and (iv) the development of a COS for use in future effectiveness trials of OME in children with CP.

SETTING

The survey was carried out by e-mail with cleft centres. The qualitative research interviews took place in patients' homes. The COS was developed with health professionals and parents using a web-based Delphi exercise and a consensus meeting.

PARTICIPANTS

Clinicians working in the UK cleft centres, and parents and patients affected by CP and identified through two cleft clinics in the UK, or through the Cleft Lip and Palate Association.

RESULTS

The clinician survey revealed that care was predominantly delivered via a 'hub-and-spoke' model; there was some uncertainty about treatment strategies; it is not current practice to insert VTs at the time of palate repair; centres were in a position to take part in a future study; and the response rate to the survey was not good, representing a potential concern about future co-operation. A COS reflecting the opinions of clinicians and parents was developed, which included nine core outcomes important to both health-care professionals and parents. The qualitative research suggested that a trial would have a 25% recruitment rate, and although hearing was a key outcome, this was likely to be due to its psychosocial consequences. The VOI analysis suggested that the current uncertainty justified the costs of future research.

CONCLUSIONS

There exists significant uncertainty regarding the best management strategy for persistent OME in children with clefts, reflecting a lack of high-quality evidence regarding the effectiveness of individual treatments. It is feasible, cost-effective and of significance to clinicians and parents to undertake a trial examining the effectiveness of VTs and HAs for children with CP. However, in view of concerns about recruitment rate and engagement with the clinicians, we recommend that a trial with an internal pilot is considered.

FUNDING

The National Institute for Health Research Health Technology Assessment programme. This study was part-funded by the Healing Foundation supported by the Vocational Training Charitable Trust who funded trial staff including the study co-ordinator, information systems developer, study statistician, administrator and supervisory staff.

Estimating the expected value of partial perfect information in health economic evaluations using integrated nested Laplace approximation

The Expected Value of Perfect Partial Information (EVPPI) is a decision‐theoretic measure of the ‘cost’ of parametric uncertainty in decision making used principally in health economic decision making. Despite this decision‐theoretic grounding, the uptake of EVPPI calculations in practice has been slow. This is in part due to the prohibitive computational time required to estimate the EVPPI via Monte Carlo simulations. However, recent developments have demonstrated that the EVPPI can be estimated by non‐parametric regression methods, which have significantly decreased the computation time required to approximate the EVPPI. Under certain circumstances, high‐dimensional Gaussian Process (GP) regression is suggested, but this can still be prohibitively expensive. Applying fast computation methods developed in spatial statistics using Integrated Nested Laplace Approximations (INLA) and projecting from a high‐dimensional into a low‐dimensional input space allows us to decrease the computation time for fitting these high‐dimensional GP, often substantially. We demonstrate that the EVPPI calculated using our method for GP regression is in line with the standard GP regression method and that despite the apparent methodological complexity of this new method, R functions are available in the package BCEA to implement it simply and efficiently. © 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

e-CBT (myCompass), Antidepressant Medication, and Face-to-Face Psychological Treatment for Depression in Australia: A Cost-Effectiveness Comparison

Background

The economic cost of depression is becoming an ever more important determinant for health policy and decision makers. Internet-based interventions with and without therapist support have been found to be effective options for the treatment of mild to moderate depression. With increasing demands on health resources and shortages of mental health care professionals, the integration of cost-effective treatment options such as Internet-based programs into primary health care could increase efficiency in terms of resource use and costs.

Objective

Our aim was to evaluate the cost-effectiveness of an Internet-based intervention (myCompass) for the treatment of mild-to-moderate depression compared to treatment as usual and cognitive behavior therapy in a stepped care model.

Methods

A decision model was constructed using a cost utility framework to show both costs and health outcomes. In accordance with current treatment guidelines, a stepped care model included myCompass as the first low-intervention step in care for a proportion of the model cohort, with participants beginning from a low-intensity intervention to increasing levels of treatment. Model parameters were based on data from the recent randomized controlled trial of myCompass, which showed that the intervention reduced symptoms of depression, anxiety, and stress and improved work and social functioning for people with symptoms in the mild-to-moderate range.

Results

The average net monetary benefit (NMB) was calculated, identifying myCompass as the strategy with the highest net benefit. The mean incremental NMB per individual for the myCompass group was AUD 1165.88 compared to treatment as usual and AUD 522.58 for the cognitive behavioral therapy model.

Conclusions

Internet-based interventions can provide cost-effective access to treatment when provided as part of a stepped care model. Widespread dissemination of Internet-based programs can potentially reduce demands on primary and tertiary services and reduce unmet need.

A Conjugate Class of Utility Functions for Sequential Decision Problems

The use of the conjugacy property for members of the exponential family of distributions is commonplace within Bayesian statistical analysis, allowing for tractable and simple solutions to problems of inference. However, despite a shared motivation, there has been little previous development of a similar property for using utility functions within a Bayesian decision analysis. As such, this article explores a class of utility functions that appear to be reasonable for modeling the preferences of a decisionmaker in many real-life situations, but that also permit a tractable and simple analysis within sequential decision problems.

Community IntraVenous Antibiotic Study (CIVAS): protocol for an evaluation of patient preferences for and cost-effectiveness of community intravenous antibiotic services

INTRODUCTION

Outpatient parenteral antimicrobial therapy (OPAT) is used to treat a wide range of infections, and is common practice in countries such as the USA and Australia. In the UK, national guidelines (standards of care) for OPAT services have been developed to act as a benchmark for clinical monitoring and quality. However, the availability of OPAT services in the UK is still patchy and until quite recently was available only in specialist centres. Over time, National Health Service (NHS) Trusts have developed OPAT services in response to local needs, which has resulted in different service configurations and models of care. However, there has been no robust examination comparing the cost-effectiveness of each service type, or any systematic examination of patient preferences for services on which to base any business case decision.

METHODS AND ANALYSIS

The study will use a mixed methods approach, to evaluate patient preferences for and the cost-effectiveness of OPAT service models. The study includes seven NHS Trusts located in four counties. There are five inter-related work packages: a systematic review of the published research on the safety, efficacy and cost-effectiveness of intravenous antibiotic delivery services; a qualitative study to explore existing OPAT services and perceived barriers to future development; an economic model to estimate the comparative value of four different community intravenous antibiotic services; a discrete choice experiment to assess patient preferences for services, and an expert panel to agree which service models may constitute the optimal service model(s) of community intravenous antibiotics delivery.

ETHICS AND DISSEMINATION

The study has been approved by the NRES Committee, South West-Frenchay using the Proportionate Review Service (ref 13/SW/0060). The results of the study will be disseminated at national and international conferences, and in international journals.

Estimating the Expected Value of Sample Information Using the Probabilistic Sensitivity Analysis Sample: A Fast, Nonparametric Regression-Based Method

Health economic decision-analytic models are used to estimate the expected net benefits of competing decision options. The true values of the input parameters of such models are rarely known with certainty, and it is often useful to quantify the value to the decision maker of reducing uncertainty through collecting new data. In the context of a particular decision problem, the value of a proposed research design can be quantified by its expected value of sample information (EVSI). EVSI is commonly estimated via a 2-level Monte Carlo procedure in which plausible data sets are generated in an outer loop, and then, conditional on these, the parameters of the decision model are updated via Bayes rule and sampled in an inner loop. At each iteration of the inner loop, the decision model is evaluated. This is computationally demanding and may be difficult if the posterior distribution of the model parameters conditional on sampled data is hard to sample from. We describe a fast nonparametric regression-based method for estimating per-patient EVSI that requires only the probabilistic sensitivity analysis sample (i.e., the set of samples drawn from the joint distribution of the parameters and the corresponding net benefits). The method avoids the need to sample from the posterior distributions of the parameters and avoids the need to rerun the model. The only requirement is that sample data sets can be generated. The method is applicable with a model of any complexity and with any specification of model parameter distribution. We demonstrate in a case study the superior efficiency of the regression method over the 2-level Monte Carlo method.

Determining the impact of modeling additional sources of uncertainty in value-of-information analysis

BACKGROUND

The conditional reimbursement policy for expensive medicines in The Netherlands requires data collection on actual use and cost-effectiveness after the initial decision to reimburse a drug. This introduces new sources of uncertainty (less important in a randomized controlled trial than in daily practice), which may affect priorities for further research.

OBJECTIVES

This article focuses on determining the impact of including these uncertainties at the time a decision is made, and whether more complex models are always needed to address prioritization of additional research.

METHODS

We constructed a typical decision model for chronic progressive diseases with four health states and parameters related to transition and exacerbation probabilities, costs, and utilities. Different scenarios are built on the basis of three additional uncertainties: persistence, compliance, and broadening of indication. Persistence refers to treatment duration. Compliance describes the fraction of treatment benefit obtained because of not taking the medication as prescribed. Broadening of indication reflects a shift in the severity distribution at treatment start. These uncertainties were parameterized in the model and included in the value-of-information analysis.

RESULTS

The most important parameters were transition probabilities. Broadening of indication had little impact on the overall uncertainty. Compliance and persistence were important when establishing priorities for further research. Major differences with respect to the reference scenario were due to the parameterization of compliance in the decision model.

CONCLUSIONS

The usual practice of modeling only randomized controlled trial data at the time the decision on conditional reimbursement is made can lead to wrong decisions. Additional uncertainties arising from outcomes studies should be anticipated at an early stage and included in the model because this can have a strong impact on the prioritization of further research.

Network vs. pairwise meta-analyses: a case study of the impact of an evidence-synthesis paradigm on value of information outcomes

OBJECTIVE

To evaluate the impact of using two evidence-synthesis paradigms, pairwise meta-analysis (PMA) vs. network meta-analysis (NMA), on the expected value of information (EVI) outcomes, using pharmacotherapy of chronic obstructive pulmonary disease as a case study.

METHODS

Bayesian random-effects PMAs were performed for each pharmacotherapy vs. placebo, and a Bayesian random-effects NMA was performed combining both placebo-controlled and head-to-head trials. Both provided comparative rate ratio (RR) estimates between each pharmacotherapy vs. placebo. A Markov model was developed to project costs and quality-adjusted life-years of five commonly used treatments for chronic obsructive pulmonary disorder. RRs for the treatment effect compared with placebo derived using PMA and NMA were used alongside values from the literature to populate the model. In addition to standard cost-effectiveness outputs, we calculated and compared the expected value of perfect information (EVPI) and the expected value of partial perfect information (EVPPI) for treatment effects, for comparisons that included all or a subset of treatments.

RESULTS

The network of evidence included five different treatments, compared in 19 randomized controlled trials (RCTs), which in total included 28,172 individuals. The cost-effectiveness outcomes were similar between the two evidence-synthesis paradigms. The individual EVPI for all treatments was Can$1,262 for PMA-based analyses and Can$572 for NMA-based analyses. For all comparisons involving two, three, or four treatments, the comparison with the highest EVPI was different between the two methods. Similarly, the choice of PMA or NMA had resulted in substantially different EVPPI rankings.

CONCLUSION

Our case study shows that the choice of PMA or NMA can have significant effects on the EVI results. Under comparable conditions, the incorporation of more evidence in the NMA most likely increases the precision of estimates and therefore is likely to result in lower EVI outcomes. As our study demonstrates, the difference in EVI outcomes can be substantial, potentially affecting the decision to conduct research and the design of future research.

The cost-effectiveness of a theory-based online health behaviour intervention for new university students: an economic evaluation

BACKGROUND

Too many young people engage in unhealthy behaviours such as eating unhealthily, being physically inactive, binge drinking and smoking. This study aimed to estimate the short-term and long-term cost-effectiveness of a theory-based online health behaviour intervention ("U@Uni") in comparison with control in young people starting university.

METHODS

A costing analysis was conducted to estimate the full cost of U@Uni and the cost of U@Uni roll-out. The short-term cost-effectiveness of U@Uni was estimated using statistical analysis of 6-month cost and health-related quality of life data from the U@Uni randomised controlled trial. An economic modelling analysis combined evidence from the trial with published evidence of the effect of health behaviours on mortality risk and general population data on health behaviours, to estimate the lifetime cost-effectiveness of U@Uni in terms of incremental cost per QALY. Costs and effects were discounted at 1.5% per annum. A full probabilistic sensitivity analysis was conducted to account for uncertainty in model inputs and provide an estimate of the value of information for groups of important parameters.

RESULTS

To implement U@Uni for the randomised controlled trial was estimated to cost £292 per participant, whereas roll-out to another university was estimated to cost £19.71, both giving a QALY gain of 0.0128 per participant. The short-term (6-month) analysis suggested that U@Uni would not be cost-effective at a willingness-to-pay threshold of £20,000 per QALY (incremental cost per QALY gained = £243,926). When a lifetime horizon was adopted the results suggest that the full implementation of U@Uni is unlikely to be cost-effective, whereas the roll-out of U@Uni to another university is extremely likely to be cost-effective. The value of information analysis suggests that the most important drivers of decision uncertainty are uncertainties in the effect of U@Uni on health behaviours.

CONCLUSIONS

The study provides the first estimate of the costs and cost-effectiveness of an online health behaviour intervention targeted at new university students. The results suggest that the roll-out, but not the full implementation, of U@Uni would be a cost-effective decision for the UK Department of Health, given a lifetime perspective and a willingness-to pay threshold of £20,000 per QALY.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN67684181.

The value of information for managing contaminated sediments

Effective management of contaminated sediments is important for long-term human and environmental health, but site-management decisions are often made under high uncertainty and without the help of structured decision support tools. Potential trade-offs between remedial costs, environmental effects, human health risks, and societal benefits, as well as fundamental differences in stakeholder priorities, complicate decision making. Formal decision-analytic tools such as multicriteria decision analysis (MCDA) move beyond ad hoc decision support to quantitatively and holistically rank management alternatives and add transparency and replicability to the evaluation process. However, even the best decisions made under uncertainty may be found suboptimal in hindsight, once additional scientific, social, economic, or other details become known. Value of information (VoI) analysis extends MCDA by systematically evaluating the impact of uncertainty on a decision. VoI prioritizes future research in terms of expected decision relevance by helping decision makers estimate the likelihood that additional information will improve decision confidence or change their selection of a management plan. In this study, VoI analysis evaluates uncertainty, estimates decision confidence, and prioritizes research to inform selection of a sediment capping strategy for the dibenzo-p-dioxin and -furan contaminated Grenland fjord system in southern Norway. The VoI model extends stochastic MCDA to model decisions with and without simulated new information and compares decision confidence across scenarios with different degrees of remaining uncertainty. Results highlight opportunities for decision makers to benefit from additional information by anticipating the improved decision confidence (or lack thereof) expected from reducing uncertainties for each criterion or combination of criteria. This case study demonstrates the usefulness of VoI analysis for environmental decisions by predicting when decisions can be made confidently, for prioritizing areas of research to pursue to improve decision confidence, and for differentiating between decision-relevant and decision-irrelevant differences in evaluation perspectives, all of which help guide meaningful deliberation toward effective consensus solutions.

Value of information analysis optimizing future trial design from a pilot study on catheter securement devices

Background:

Value of information analysis has been proposed as an alternative to the standard hypothesis testing approach, which is based on type I and type II errors, in determining sample sizes for randomized clinical trials. However, in addition to sample size calculation, value of information analysis can optimize other aspects of research design such as possible comparator arms and alternative follow-up times, by considering trial designs that maximize the expected net benefit of research, which is the difference between the expected cost of the trial and the expected value of additional information.

Purpose:

To apply value of information methods to the results of a pilot study on catheter securement devices to determine the optimal design of a future larger clinical trial.

Methods:

An economic evaluation was performed using data from a multi-arm randomized controlled pilot study comparing the efficacy of four types of catheter securement devices: standard polyurethane, tissue adhesive, bordered polyurethane and sutureless securement device. Probabilistic Monte Carlo simulation was used to characterize uncertainty surrounding the study results and to calculate the expected value of additional information. To guide the optimal future trial design, the expected costs and benefits of the alternative trial designs were estimated and compared.

Results:

Analysis of the value of further information indicated that a randomized controlled trial on catheter securement devices is potentially worthwhile. Among the possible designs for the future trial, a four-arm study with 220 patients/arm would provide the highest expected net benefit corresponding to 130% return-on-investment. The initially considered design of 388 patients/arm, based on hypothesis testing calculations, would provide lower net benefit with return-on-investment of 79%.

Limitations:

Cost-effectiveness and value of information analyses were based on the data from a single pilot trial which might affect the accuracy of our uncertainty estimation. Another limitation was that different follow-up durations for the larger trial were not evaluated.

Conclusion:

The value of information approach allows efficient trial design by maximizing the expected net benefit of additional research. This approach should be considered early in the design of randomized clinical trials.

Economic evaluation of surgical insertion of ventilation tubes for the management of persistent bilateral otitis media with effusion in children

BACKGROUND

The surgical insertion of Ventilation Tubes (VTs) for the management of persistent bilateral Otitis Media with Effusion (OME) in children remains a contentious issue due to the varying opinions regarding the risks and benefits of this procedure. The aim of this study was to evaluate the economic impact of VTs insertion for the management of persistent bilateral OME in children, providing an additional perspective on the management of one of the commonest medical conditions of childhood.

METHODS

A decision-tree model was constructed to assess the cost-effectiveness of VTs strategy compared with the Hearing Aids (HAs) alone and HAs plus VTs strategies. The model used data from published sources, and assumed a 2-year time horizon and UK NHS perspective for costs. Outcomes were computed as Quality-Adjusted Life-Years (QALYs) by attaching a utility value to the total potential gains in Hearing Level in decibels (dBHL) over 12 and 24 months. Modelling uncertainty in the specification of decision-tree probabilities and QALYs was performed through Monte Carlo simulation. Expected Value of Perfect Information (EVPI) and partial EVPI (EVPPI) analyses were conducted to estimate the potential value of future research and uncertainty associated with the key parameters.

RESULTS

The VTs strategy was more effective and less costly when compared with the HAs plus VTs strategy, while the incremental cost-effectiveness ratio for the VTs strategy compared with the HAs strategy was £ 5,086 per QALY gained. At the willingness-to-pay threshold of £ 20,000 per QALY, the probability that the VTs strategy is likely to be more cost-effective was 0.58. The EVPI value at population level of around £ 9.5 million at the willingness-to-pay threshold of £ 20,000 indicated that future research in this area is potentially worthwhile, while the EVPPI analysis indicated considerable uncertainty surrounding the parameters used for computing the QALYs for which more precise estimates would be most valuable.

CONCLUSIONS

The VTs strategy is a cost-effective option when compared with the HAs alone and HAs plus VTs strategies, but the need for additional information from future study is evident to inform this surgical treatment choice. Future studies of surgical and non-surgical treatment of OME in childhood should evaluate the economic impact of pertinent interventions to provide greater context.