Value of information and pricing new healthcare interventions

Globally optimal trial design and risk sharing arrangements are key to avoiding opportunity costs of delay and enabling equitable, feasible and effective global vaccine research and implementation in current or future pandemics

Global vaccination in the face of pandemics such as COVID-19 and new variants is a race against time. Avoiding the opportunity costs of delay and the associated health, social, and downstream economic impacts is a challenge and an imperative. Failures to address the global challenges posed by COVID-19 have become increasingly evident as waves of vaccine-evading mutations have emerged, facilitated by unequal vaccination coverage and diminishing immunity against new variants worldwide. To address these challenges, societal decision-makers (governments) and industry manufacturer interests must be better aligned for rapid, globally optimal trial design, ideally with research coverage, implementation, and accessibility of effective vaccines across joint research, implementation, and distribution cycles to address pandemic evolution in real time. Value of information (VoI) methods for optimal global trial design and risk-sharing arrangements align the research, distribution, and implementation interests and efforts globally to meet head-on the imperative of avoiding opportunity costs of delay and enabling consistent global solutions with maximizing local and global net benefits. They uniquely enable feasible early adoption of the most promising strategies in real time while the best globally translatable evidence is collected and interests are aligned for global distribution and implementation. Furthermore, these methods are generally shown to be imperative for feasible, fast, and optimal solutions across joint research, reimbursement, and regulatory processes for current and future pandemics and other global existential threats. Establishing pathways for globally optimal trial designs, risk-sharing agreements, and efficient translation to practice is urgent on many fronts.

An Efficient Method for Computing Expected Value of Sample Information for Survival Data from an Ongoing Trial

BACKGROUND

Decisions about new health technologies are increasingly being made while trials are still in an early stage, which may result in substantial uncertainty around key decision drivers such as estimates of life expectancy and time to disease progression. Additional data collection can reduce uncertainty, and its value can be quantified by computing the expected value of sample information (EVSI), which has typically been described in the context of designing a future trial. In this article, we develop new methods for computing the EVSI of extending an existing trial's follow-up, first for an assumed survival model and then extending to capture uncertainty about the true survival model.

METHODS

We developed a nested Markov Chain Monte Carlo procedure and a nonparametric regression-based method. We compared the methods by computing single-model and model-averaged EVSI for collecting additional follow-up data in 2 synthetic case studies.

RESULTS

There was good agreement between the 2 methods. The regression-based method was fast and straightforward to implement, and scales easily to include any number of candidate survival models in the model uncertainty case. The nested Monte Carlo procedure, on the other hand, was extremely computationally demanding when we included model uncertainty.

CONCLUSIONS

We present a straightforward regression-based method for computing the EVSI of extending an existing trial's follow-up, both where a single known survival model is assumed and where we are uncertain about the true survival model. EVSI for ongoing trials can help decision makers determine whether early patient access to a new technology can be justified on the basis of the current evidence or whether more mature evidence is needed.

HIGHLIGHTS

Decisions about new health technologies are increasingly being made while trials are still in an early stage, which may result in substantial uncertainty around key decision drivers such as estimates of life-expectancy and time to disease progression. Additional data collection can reduce uncertainty, and its value can be quantified by computing the expected value of sample information (EVSI), which has typically been described in the context of designing a future trial.In this article, we have developed new methods for computing the EVSI of extending a trial's follow-up, both where a single known survival model is assumed and where we are uncertain about the true survival model. We extend a previously described nonparametric regression-based method for computing EVSI, which we demonstrate in synthetic case studies is fast, straightforward to implement, and scales easily to include any number of candidate survival models in the EVSI calculations.The EVSI methods that we present in this article can quantify the need for collecting additional follow-up data before making an adoption decision given any decision-making context.

Estimating Value-Based Price and Quantifying Uncertainty around It in Health Technology Assessment: Frequentist and Bayesian Approaches

BACKGROUND

Although several statistical methods have been developed to inform decision making on reimbursement under uncertainty (e.g., expected net benefit, cost-effectiveness acceptability curves, and expected value of perfect information [EVPI]), those for value-based pricing are limited. This research develops methods for estimating the value-based price and quantifying the uncertainty around it in health technology assessment.

METHODS

We defined the value-based price of a medical product under assessment as the price at which the incremental cost-effectiveness ratio is just equal to a cost-effectiveness threshold. According to this definition, we derived an explicit form of the value-based price. Using this explicit form, we developed frequentist and Bayesian approaches to value-based pricing under uncertainty. Our proposed methods were illustrated via 2 hypothetical case studies.

RESULTS

The value-based price can be expressed explicitly using cost, effectiveness, and a cost-effectiveness threshold and is a linear function of a cost-effectiveness threshold. In the frequentist framework, point estimation, interval estimation, and hypothesis testing for the value-based price are available. In the Bayesian framework, the best estimate of the value-based price under uncertainty is the weighted median value-based price with the weight of the expected consumption volume of a medical product under assessment. This is based on the opportunity loss incurred by a decision error in value-based pricing. This opportunity loss also provides a basis for the calculation of EVPI associated with value-based pricing. These methods provided estimates of the value-based prices of medical products and the uncertainty around them in 2 hypothetical case studies.

CONCLUSIONS

Our developed methods can improve decision making on value-based pricing in health technology assessment.

Combining Model-Based Clinical Trial Simulation, Pharmacoeconomics, and Value of Information to Optimize Trial Design

The Bayesian decision‐analytic approach to trial design uses prior distributions for treatment effects, updated with likelihoods for proposed trial data. Prior distributions for treatment effects based on previous trial results risks sample selection bias and difficulties when a proposed trial differs in terms of patient characteristics, medication adherence, or treatment doses and regimens. The aim of this study was to demonstrate the utility of using pharmacometric‐based clinical trial simulation (CTS) to generate prior distributions for use in Bayesian decision‐theoretic trial design. The methods consisted of four principal stages: a CTS to predict the distribution of treatment response for a range of trial designs; Bayesian updating for a proposed sample size; a pharmacoeconomic model to represent the perspective of a reimbursement authority in which price is contingent on trial outcome; and a model of the pharmaceutical company return on investment linking drug prices to sales revenue. We used a case study of febuxostat versus allopurinol for the treatment of hyperuricemia in patients with gout. Trial design scenarios studied included alternative treatment doses, inclusion criteria, input uncertainty, and sample size. Optimal trial sample sizes varied depending on the uncertainty of model inputs, trial inclusion criteria, and treatment doses. This interdisciplinary framework for trial design and sample size calculation may have value in supporting decisions during later phases of drug development and in identifying costly sources of uncertainty, and thus inform future research and development strategies.

Practical help for specifying the target difference in sample size calculations for RCTs: the DELTA2 five-stage study, including a workshop

BACKGROUND

The randomised controlled trial is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to its design is a calculation of the number of participants needed (the sample size) for the trial. The sample size is typically calculated by specifying the magnitude of the difference in the primary outcome between the intervention effects for the population of interest. This difference is called the 'target difference' and should be appropriate for the principal estimand of interest and determined by the primary aim of the study. The target difference between treatments should be considered realistic and/or important by one or more key stakeholder groups.

OBJECTIVE

The objective of the report is to provide practical help on the choice of target difference used in the sample size calculation for a randomised controlled trial for researchers and funder representatives.

METHODS

The Difference ELicitation in TriAls2 (DELTA2) recommendations and advice were developed through a five-stage process, which included two literature reviews of existing funder guidance and recent methodological literature; a Delphi process to engage with a wider group of stakeholders; a 2-day workshop; and finalising the core document.

RESULTS

Advice is provided for definitive trials (Phase III/IV studies). Methods for choosing the target difference are reviewed. To aid those new to the topic, and to encourage better practice, 10 recommendations are made regarding choosing the target difference and undertaking a sample size calculation. Recommended reporting items for trial proposal, protocols and results papers under the conventional approach are also provided. Case studies reflecting different trial designs and covering different conditions are provided. Alternative trial designs and methods for choosing the sample size are also briefly considered.

CONCLUSIONS

Choosing an appropriate sample size is crucial if a study is to inform clinical practice. The number of patients recruited into the trial needs to be sufficient to answer the objectives; however, the number should not be higher than necessary to avoid unnecessary burden on patients and wasting precious resources. The choice of the target difference is a key part of this process under the conventional approach to sample size calculations. This document provides advice and recommendations to improve practice and reporting regarding this aspect of trial design. Future work could extend the work to address other less common approaches to the sample size calculations, particularly in terms of appropriate reporting items.

FUNDING

Funded by the Medical Research Council (MRC) UK and the National Institute for Health Research as part of the MRC-National Institute for Health Research Methodology Research programme.

Value of information methods to design a clinical trial in a small population to optimise a health economic utility function

Background

Most confirmatory randomised controlled clinical trials (RCTs) are designed with specified power, usually 80% or 90%, for a hypothesis test conducted at a given significance level, usually 2.5% for a one-sided test. Approval of the experimental treatment by regulatory agencies is then based on the result of such a significance test with other information to balance the risk of adverse events against the benefit of the treatment to future patients. In the setting of a rare disease, recruiting sufficient patients to achieve conventional error rates for clinically reasonable effect sizes may be infeasible, suggesting that the decision-making process should reflect the size of the target population.

Methods

We considered the use of a decision-theoretic value of information (VOI) method to obtain the optimal sample size and significance level for confirmatory RCTs in a range of settings. We assume the decision maker represents society. For simplicity we assume the primary endpoint to be normally distributed with unknown mean following some normal prior distribution representing information on the anticipated effectiveness of the therapy available before the trial. The method is illustrated by an application in an RCT in haemophilia A. We explicitly specify the utility in terms of improvement in primary outcome and compare this with the costs of treating patients, both financial and in terms of potential harm, during the trial and in the future.

Results

The optimal sample size for the clinical trial decreases as the size of the population decreases. For non-zero cost of treating future patients, either monetary or in terms of potential harmful effects, stronger evidence is required for approval as the population size increases, though this is not the case if the costs of treating future patients are ignored.

Conclusions

Decision-theoretic VOI methods offer a flexible approach with both type I error rate and power (or equivalently trial sample size) depending on the size of the future population for whom the treatment under investigation is intended. This might be particularly suitable for small populations when there is considerable information about the patient population.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0475-0) contains supplementary material, which is available to authorized users.

Prioritizing HIV comparative effectiveness trials based on value of information: generic versus brand-name ART in the US

BACKGROUND

Value of Information (VOI) analysis examines whether to acquire information before making a decision. We introduced VOI to the HIV audience, using the example of generic antiretroviral therapy (ART) in the US.

METHODS AND FINDINGS

We used a mathematical model and probabilistic sensitivity analysis (PSA) to generate probability distributions of survival (in quality-adjusted life years, QALYs) and cost for three potential first-line ART regimens: three-pill generic, two-pill generic, and single-pill branded. These served as input for a comparison of two hypothetical two-arm trials: three-pill generic versus single-pill branded; and two-pill generic versus single-pill branded. We modeled pre-trial uncertainty by defining probability distributions around key inputs, including 24-week HIV-RNA suppression and subsequent ART failure. We assumed that, without a trial, patients received the single-pill branded strategy. Post-trial, we assumed that patients received the most cost-effective strategy. For both trials, we quantified the probability of changing to a generic-based regimen upon trial completion and the expected VOI in terms of improved health outcomes and costs. Assuming a willingness to pay (WTP) threshold of $100 000/QALY, the three-pill trial led to more treatment changes (84%) than the two-pill trial (78%). Estimated VOI was $48 000 (three-pill trial) and $35 700 (two-pill trial) per future patient initiating ART.

CONCLUSIONS

A three-pill trial of generic ART is more likely to lead to post-trial treatment changes and to provide more value than a two-pill trial if policy decisions are based on cost-effectiveness. Value of Information analysis can identify trials likely to confer the greatest impact and value for HIV care.

Valuing Trial Designs from a Pharmaceutical Perspective Using Value-Based Pricing

Our aim was to adapt the traditional framework for expected net benefit of sampling (ENBS) to be more compatible with drug development trials from the pharmaceutical perspective. We modify the traditional framework for conducting ENBS and assume that the price of the drug is conditional on the trial outcomes. We use a value-based pricing (VBP) criterion to determine price conditional on trial data using Bayesian updating of cost-effectiveness (CE) model parameters. We assume that there is a threshold price below which the company would not market the new intervention. We present a case study in which a phase III trial sample size and trial duration are varied. For each trial design, we sampled 10,000 trial outcomes and estimated VBP using a CE model. The expected commercial net benefit is calculated as the expected profits minus the trial costs. A clinical trial with shorter follow-up, and larger sample size, generated the greatest expected commercial net benefit. Increasing the duration of follow-up had a modest impact on profit forecasts. Expected net benefit of sampling can be adapted to value clinical trials in the pharmaceutical industry to optimise the expected commercial net benefit. However, the analyses can be very time consuming for complex CE models.

Estimating the Expected Value of Sample Information Using the Probabilistic Sensitivity Analysis Sample: A Fast, Nonparametric Regression-Based Method

Health economic decision-analytic models are used to estimate the expected net benefits of competing decision options. The true values of the input parameters of such models are rarely known with certainty, and it is often useful to quantify the value to the decision maker of reducing uncertainty through collecting new data. In the context of a particular decision problem, the value of a proposed research design can be quantified by its expected value of sample information (EVSI). EVSI is commonly estimated via a 2-level Monte Carlo procedure in which plausible data sets are generated in an outer loop, and then, conditional on these, the parameters of the decision model are updated via Bayes rule and sampled in an inner loop. At each iteration of the inner loop, the decision model is evaluated. This is computationally demanding and may be difficult if the posterior distribution of the model parameters conditional on sampled data is hard to sample from. We describe a fast nonparametric regression-based method for estimating per-patient EVSI that requires only the probabilistic sensitivity analysis sample (i.e., the set of samples drawn from the joint distribution of the parameters and the corresponding net benefits). The method avoids the need to sample from the posterior distributions of the parameters and avoids the need to rerun the model. The only requirement is that sample data sets can be generated. The method is applicable with a model of any complexity and with any specification of model parameter distribution. We demonstrate in a case study the superior efficiency of the regression method over the 2-level Monte Carlo method.

Computing Expected Value of Partial Sample Information from Probabilistic Sensitivity Analysis Using Linear Regression Metamodeling

Decision makers often desire both guidance on the most cost-effective interventions given current knowledge and also the value of collecting additional information to improve the decisions made (i.e., from value of information [VOI] analysis). Unfortunately, VOI analysis remains underused due to the conceptual, mathematical, and computational challenges of implementing Bayesian decision-theoretic approaches in models of sufficient complexity for real-world decision making. In this study, we propose a novel practical approach for conducting VOI analysis using a combination of probabilistic sensitivity analysis, linear regression metamodeling, and unit normal loss integral function--a parametric approach to VOI analysis. We adopt a linear approximation and leverage a fundamental assumption of VOI analysis, which requires that all sources of prior uncertainties be accurately specified. We provide examples of the approach and show that the assumptions we make do not induce substantial bias but greatly reduce the computational time needed to perform VOI analysis. Our approach avoids the need to analytically solve or approximate joint Bayesian updating, requires only one set of probabilistic sensitivity analysis simulations, and can be applied in models with correlated input parameters.

Decision-theoretic designs for small trials and pilot studies: A review

Pilot studies and other small clinical trials are often conducted but serve a variety of purposes and there is little consensus on their design. One paradigm that has been suggested for the design of such studies is Bayesian decision theory. In this article, we review the literature with the aim of summarizing current methodological developments in this area. We find that decision-theoretic methods have been applied to the design of small clinical trials in a number of areas. We divide our discussion of published methods into those for trials conducted in a single stage, those for multi-stage trials in which decisions are made through the course of the trial at a number of interim analyses, and those that attempt to design a series of clinical trials or a drug development programme. In all three cases, a number of methods have been proposed, depending on the decision maker’s perspective being considered and the details of utility functions that are used to construct the optimal design.

Better informing decision making with multiple outcomes cost-effectiveness analysis under uncertainty in cost-disutility space

INTRODUCTION

Comparing multiple, diverse outcomes with cost-effectiveness analysis (CEA) is important, yet challenging in areas like palliative care where domains are unamenable to integration with survival. Generic multi-attribute utility values exclude important domains and non-health outcomes, while partial analyses-where outcomes are considered separately, with their joint relationship under uncertainty ignored-lead to incorrect inference regarding preferred strategies.

OBJECTIVE

The objective of this paper is to consider whether such decision making can be better informed with alternative presentation and summary measures, extending methods previously shown to have advantages in multiple strategy comparison.

METHODS

Multiple outcomes CEA of a home-based palliative care model (PEACH) relative to usual care is undertaken in cost disutility (CDU) space and compared with analysis on the cost-effectiveness plane. Summary measures developed for comparing strategies across potential threshold values for multiple outcomes include: expected net loss (ENL) planes quantifying differences in expected net benefit; the ENL contour identifying preferred strategies minimising ENL and their expected value of perfect information; and cost-effectiveness acceptability planes showing probability of strategies minimising ENL.

RESULTS

Conventional analysis suggests PEACH is cost-effective when the threshold value per additional day at home (𝕜1) exceeds $1,068 or dominated by usual care when only the proportion of home deaths is considered. In contrast, neither alternative dominate in CDU space where cost and outcomes are jointly considered, with the optimal strategy depending on threshold values. For example, PEACH minimises ENL when 𝕜1=$2,000 and 𝕜2=$2,000 (threshold value for dying at home), with a 51.6% chance of PEACH being cost-effective.

CONCLUSION

Comparison in CDU space and associated summary measures have distinct advantages to multiple domain comparisons, aiding transparent and robust joint comparison of costs and multiple effects under uncertainty across potential threshold values for effect, better informing net benefit assessment and related reimbursement and research decisions.

Value of information analysis in healthcare: a review of principles and applications

BACKGROUND

Economic evaluations are increasingly utilized to inform decisions in healthcare; however, decisions remain uncertain when they are not based on adequate evidence. Value of information (VOI) analysis has been proposed as a systematic approach to measure decision uncertainty and assess whether there is sufficient evidence to support new technologies.

SCOPE

The objective of this paper is to review the principles and applications of VOI analysis in healthcare. Relevant databases were systematically searched to identify VOI articles. The findings from the selected articles were summarized and narratively presented.

FINDINGS

Various VOI methods have been developed and applied to inform decision-making, optimally designing research studies and setting research priorities. However, the application of this approach in healthcare remains limited due to technical and policy challenges.

CONCLUSION

There is a need to create more awareness about VOI analysis, simplify its current methods, and align them with the needs of decision-making organizations.

Can the real opportunity cost stand up: displaced services, the straw man outside the room

In current literature, displaced services have been suggested to provide a basis for determining a threshold value for the effects of a new technology as part of a reimbursement process when budgets are fixed. We critically examine the conditions under which displaced services would represent an economically meaningful threshold value. We first show that if we assume that the least cost-effective services are displaced to finance a new technology, then the incremental cost-effectiveness ratio (ICER) of the displaced services (d) only coincides with that related to the opportunity cost of adopting that new technology, the ICER of the most cost-effective service in expansion (n), under highly restrictive conditions-namely, complete allocative efficiency in existing provision of health care interventions. More generally, reimbursement of new technology with a fixed budget comprises two actions; adoption and financing through displacement and the effect of reimbursement is the net effect of these two actions. In order for the reimbursement process to be a pathway to allocative efficiency within a fixed budget, the net effect of the strategy of reimbursement is compared with the most cost-effective alternative strategy for reimbursement: optimal reallocation, the health gain maximizing expansion of existing services financed by the health loss minimizing contraction. The shadow price of the health effects of a new technology, βc = (1/n + 1/d - 1/m)(-1), accounts for both imperfect displacement (the ICER of the displaced service, d < m, the ICER of the least cost-effective of the existing services in contraction) and the allocative inefficiency (n < m) characteristic of health systems.

Optimal global value of information trials: better aligning manufacturer and decision maker interests and enabling feasible risk sharing

Risk sharing arrangements relate to adjusting payments for new health technologies given evidence of their performance over time. Such arrangements rely on prospective information regarding the incremental net benefit of the new technology, and its use in practice. However, once the new technology has been adopted in a particular jurisdiction, randomized clinical trials within that jurisdiction are likely to be infeasible and unethical in the cases where they would be most helpful, i.e. with current evidence of positive while uncertain incremental health and net monetary benefit. Informed patients in these cases would likely be reluctant to participate in a trial, preferring instead to receive the new technology with certainty. Consequently, informing risk sharing arrangements within a jurisdiction is problematic given the infeasibility of collecting prospective trial data. To overcome such problems, we demonstrate that global trials facilitate trialling post adoption, leading to more complete and robust risk sharing arrangements that mitigate the impact of costs of reversal on expected value of information in jurisdictions who adopt while a global trial is undertaken. More generally, optimally designed global trials offer distinct advantages over locally optimal solutions for decision makers and manufacturers alike: avoiding opportunity costs of delay in jurisdictions that adopt; overcoming barriers to evidence collection; and improving levels of expected implementation. Further, the greater strength and translatability of evidence across jurisdictions inherent in optimal global trial design reduces barriers to translation across jurisdictions characteristic of local trials. Consequently, efficiently designed global trials better align the interests of decision makers and manufacturers, increasing the feasibility of risk sharing and the expected strength of evidence over local trials, up until the point that current evidence is globally sufficient.