A Comprehensive Algorithm for Approval of Health Technologies With, Without, or Only in Research: The Key Principles for Informing Coverage Decisions

It Takes 2 to Tango. Setting Out the Conditions in Which Performance-Based Risk-Sharing Arrangements Work for Both Parties

OBJECTIVES

Faster regulatory approval processes often fail to achieve faster patient access. We seek an approach, using performance-based risk-sharing arrangements, to address uncertainty for payers regarding the relative effectiveness and value for money of products launched through accelerated approval schemes. One important reason for risk sharing is to resolve differences of opinion between innovators and payers about a technology's underlying value. To date, there has been no formal attempt to set out the circumstances in which risk sharing can address these differences.

METHODS

We use a value of information framework to understand what a performance-based risk-sharing arrangements can, in principle, add to a reimbursement scheme, separating payer perspectives on cost-effectiveness and the value of research from those of the innovator. We find 16 scenarios, developing 5 rules to analyze these 16 scenarios, identifying cases in which risk sharing adds value for both parties.

RESULTS

We find that risk sharing provides an improved solution in 9 out of 16 combinations of payer and innovator expectations about treatment outcome and the value of further research. Among our assumptions, who pays for research and scheme administration costs are key.

CONCLUSIONS

Steps should be undertaken to make risk sharing more practical, ensuring that payers consider it an option. This requires additional costs to the health system falling on the innovator in an efficient way that aligns incentives for product development for global markets. Health systems benefits are earlier patient access to cost-effective treatments and payers with higher confidence of not wasting money. Innovators get greater returns while conducting research.

Making Drug Approval Decisions in the Face of Uncertainty: Cumulative Evidence versus Value of Information

BACKGROUND

The COVID-19 pandemic underscored the criticality and complexity of decision making for novel treatment approval and further research. Our study aims to assess potential decision-making methodologies, an evaluation vital for refining future public health crisis responses.

METHODS

We compared 4 decision-making approaches to drug approval and research: the Food and Drug Administration's policy decisions, cumulative meta-analysis, a prospective value-of-information (VOI) approach (using information available at the time of decision), and a reference standard (retrospective VOI analysis using information available in hindsight). Possible decisions were to reject, accept, provide emergency use authorization, or allow access to new therapies only in research settings. We used monoclonal antibodies provided to hospitalized COVID-19 patients as a case study, examining the evidence from September 2020 to December 2021 and focusing on each method's capacity to optimize health outcomes and resource allocation.

RESULTS

Our findings indicate a notable discrepancy between policy decisions and the reference standard retrospective VOI approach with expected losses up to $269 billion USD, suggesting suboptimal resource use during the wait for emergency use authorization. Relying solely on cumulative meta-analysis for decision making results in the largest expected loss, while the policy approach showed a loss up to $16 billion and the prospective VOI approach presented the least loss (up to $2 billion).

CONCLUSION

Our research suggests that incorporating VOI analysis may be particularly useful for research prioritization and treatment implementation decisions during pandemics. While the prospective VOI approach was favored in this case study, further studies should validate the ideal decision-making method across various contexts. This study's findings not only enhance our understanding of decision-making strategies during a health crisis but also provide a potential framework for future pandemic responses.

HIGHLIGHTS

This study reviews discrepancies between a reference standard (retrospective VOI, using hindsight information) and 3 conceivable real-time approaches to research-treatment decisions during a pandemic, suggesting suboptimal use of resources.Of all prospective decision-making approaches considered, VOI closely mirrored the reference standard, yielding the least expected value loss across our study timeline.This study illustrates the possible benefit of VOI results and the need for evidence accumulation accompanied by modeling in health technology assessment for emerging therapies.

Lessons learned from the Canadian Fabry Disease Initiative for future risk-sharing and managed access agreements for pharmaceutical and advanced therapies in Canada

Risk sharing agreements (RSAs) and managed access agreements have emerged as tools to overcome evidentiary uncertainty and contain costs of pharmaceuticals; however, Canada has relatively little experience with these health policy instruments. This article describes one of the few examples of national RSAs. Enzyme replacement therapies (ERT) were introduced in Canada to treat Fabry disease in the early 2000s through an RSA. Based on qualitative interviews with key participating actors, this article explains how this RSA ensured continuity of treatment for patients already on ERT, and collected robust real-world evidence to secure treatment for future Fabry patients. We show the importance of partnerships, collaborations, and active patient communities in establishing RSAs, as well as the critical role of robust registries for the collection, storage, and use of that real-world data. In doing so, this paper points to reasons that explain the relative dearth of RSAs in Canada, which can be resource (both human and finance) intensive and are difficult to broker in a federalist health system. Through these findings, policy lessons are developed concerning the need for technological and governance platforms on how RSA in Canada can be more effectively supported going forward in a broader move towards "social pharmaceutical innovation".

Surrogacy and the Valuation of ATMPs: Taking Our Place in the Evidence Generation/Assessment Continuum

Medical technology is advancing rapidly, but established methods for health technology assessment are struggling to keep up. This challenge is particularly stark for the assessment of advanced therapy medicinal products-therapies often launched on the basis of single-arm studies powered to a surrogate primary endpoint. The most robust surrogacy methods investigate trial-level correlations between the treatment effect on the surrogate and the outcome of ultimate interest. However, these methods are often impossible with the evidence usually available for advanced therapy medicinal products at the time of the launch (randomized controlled trials are necessary for these advanced methods). Additionally, these surrogacy relationships are usually considered to be technology specific, adding uncertainty for any approach that primarily relies on historic data to estimate the surrogacy relationship for novel interventions such as advanced therapy medicinal products. The literature has already highlighted the need for early dialogue, staged assessment processes, and pricing arrangements that responsibly share the risk between the manufacturer and payer. However, it is our view that in addition to these critical developments, the modeling methods employed could also improve. Currently, health technology assessment practitioners typically either ignore the surrogate and simply extrapolate the endpoint of greatest patient relevance irrespective of the degree of maturity or assume historic surrogate relationships apply to the novel technology. In this opinion piece, we outline an additional avenue. By drawing on the understanding of the mechanism of action and insights generated earlier in the evidence generation/assessment continuum, cost-effectiveness modelers can make better use of the wider data available. These efforts are expected to reduce uncertainty at the time of the initial launch of pharmaceutical products and increase the value of subsequent data collection efforts.

Estimating the Potential Benefits of Confirmatory Trials for Drugs with Accelerated Approval: A Comprehensive Value of Information Framework

BACKGROUND

The US Food and Drug Administration's Accelerated Approval (AA) policy provides a pathway for patients to access potentially life-saving drugs rapidly. However, the use of surrogate endpoints, single-arm designs, and small sample sizes in preliminary trials that support AAs can lead to uncertainty regarding the clinical benefits of such drugs. This study aims to develop a comprehensive value of information (VOI) framework for assessing the potential benefits of future confirmatory trials, accounting for the various uncertainties inherent in preliminary trials.

METHODS

I formulated an expected value of information from confirmatory trial (EVICT) metric, which evaluates the potential benefits of a confirmatory trial that would reduce those uncertainties by using a clinically meaningful endpoint, a randomized control, and increased sample size. The EVICT metric can quantify the expected benefits of a well-designed confirmatory trial or an inadequately designed one that continues to use surrogate endpoints or single-arm design. The framework was illustrated using a hypothetical AA drug for metastatic breast cancer.

RESULTS

The case study demonstrates that a highly uncertain preliminary trial of an AA drug was associated with a substantial EVICT. A confirmatory trial with an increased sample size for this AA drug, utilizing a clinically meaningful endpoint and randomized control, yielded a population-level EVICT of $12.6 million. Persistently using a surrogate endpoint and single-arm trial design would reduce the EVICT by 60%.

CONCLUSIONS

This framework can provide accurate VOI estimates to guide coverage policies, value-based pricing, and the design of confirmatory trials for AA drugs.

Toward Best Practices for Economic Evaluations of Tumor-Agnostic Therapies: A Review of Current Barriers and Solutions

OBJECTIVES

Cancer therapies targeting tumor-agnostic biomarkers are challenging traditional health technology assessment (HTA) frameworks. The high prevalence of nonrandomized single-arm trials, heterogeneity, and small benefiting populations are driving outcomes uncertainty, challenging healthcare decision making. We conducted a structured literature review to identify barriers and prioritize solutions to generating economic evidence for tumor-agnostic therapies.

METHODS

We searched MEDLINE and Embase for English-language studies conducting economic evaluations of tumor-agnostic treatments or exploring related challenges and solutions. We included studies published by December 2022 and supplemented our review with Canadian Agency for Drugs and Technologies in Health and National Institute for Health and Care Excellence technical reports for approved tumor-agnostic therapies. Three reviewers abstracted and summarized key methodological and empirical study characteristics. Challenges and solutions were identified through authors' statements and categorized using directed content analysis.

RESULTS

Twenty-six studies met our inclusion criteria. Studies spanned economic evaluations (n = 5), reimbursement reviews (n = 4), qualitative research (n = 1), methods validations (n = 3), and commentaries or literature reviews (n = 13). Challenges encountered related to (1) the treatment setting and clinical trial designs, (2) a lack of data or low-quality data on clinical and cost parameters, and (3) an inability to produce evidence that meets HTA guidelines. Although attempted solutions centered on analytic approaches for managing missing data, proposed solutions highlighted the need for real-world evidence combined with life-cycle HTA to reduce future evidentiary uncertainty.

CONCLUSIONS

Therapeutic innovation outpaces HTA evidence generation and the methods that support it. Existing HTA frameworks must be adapted for tumor-agnostic treatments to support future economic evaluations enabling timely patient access.

IMproving facial PRosthesis construction with contactlESs Scanning and Digital workflow (IMPRESSeD): study protocol for a feasibility crossover randomised controlled trial of digital versus conventional manufacture of facial prostheses in patients with orbital or nasal facial defects

BACKGROUND

Facial prostheses can have a profound impact on patients' appearance, function and quality of life. There has been increasing interest in the digital manufacturing of facial prostheses which may offer many benefits to patients and healthcare services compared with conventional manufacturing processes. Most facial prosthesis research has adopted observational study designs with very few randomised controlled trials (RCTs) documented. There is a clear need for a well-designed RCT to compare the clinical and cost-effectiveness of digitally manufactured facial prostheses versus conventionally manufactured facial prostheses. This study protocol describes the planned conduct of a feasibility RCT which aims to address this knowledge gap and determine whether it is feasible to conduct a future definitive RCT.

METHODS

The IMPRESSeD study is a multi-centre, 2-arm, crossover, feasibility RCT with early health technology assessment and qualitative research. Up to 30 participants with acquired orbital or nasal defects will be recruited from the Maxillofacial Prosthetic Departments of participating NHS hospitals. All trial participants will receive 2 new facial prostheses manufactured using digital and conventional manufacturing methods. The order of receiving the facial prostheses will be allocated centrally using minimisation. The 2 prostheses will be made in tandem and marked with a colour label to mask the manufacturing method to the participants. Participants will be reviewed 4 weeks following the delivery of the first prosthesis and 4 weeks following the delivery of the second prosthesis. Primary feasibility outcomes include eligibility, recruitment, conversion, and attrition rates. Data will also be collected on patient preference, quality of life and resource use from the healthcare perspective. A qualitative sub-study will evaluate patients' perception, lived experience and preference of the different manufacturing methods.

DISCUSSION

There is uncertainty regarding the best method of manufacturing facial prostheses in terms of clinical effectiveness, cost-effectiveness and patient acceptability. There is a need for a well-designed RCT to compare digital and conventional manufacturing of facial prostheses to better inform clinical practice. The feasibility study will evaluate key parameters needed to design a definitive trial and will incorporate early health technology assessment and a qualitative sub-study to identify the potential benefits of further research.

TRIAL REGISTRATION

ISRCTN ISRCTN10516986). Prospectively registered on 08 June 2021,  https://www.isrctn.com/ISRCTN10516986 .

A Comparison of the Content and Consistency of Methodological Quality and Transferability Checklists for Reviewing Model-Based Economic Evaluations

OBJECTIVES

The aim of this study was to examine whether and how the content of six checklists (Caro, Consensus on Health Economic Criteria [CHEC]-Extended, European Network of Health Economic Databases [EURONHEED], National Institute for Health and Care Excellence [NICE], Philips, Welte) affect the consistency in findings on methodological quality and transferability, using 10 model-based economic evaluations of genetic-guided pharmacotherapy for venous thromboembolism.

METHODS

Each checklist was categorised by domain (structure, data, consistency, etc.) and type of assessment (presence vs. appropriateness) and was applied to each study by two independent reviewers who agreed on ratings via consensus, and discussion with a third reviewer when necessary. Methodological quality scores and rankings were examined using Spearman correlation tests, with subgroup analyses for domains and types of assessment. We compared overall ratings of transferability qualitatively, including how content may affect what is considered 'transferable'.

RESULTS

The checklists had similar proportions of items judging presence and appropriateness, but varying proportions of items across domains. For methodological quality, ranking consistencies were the highest between CHEC-Extended-Philips, Philips-NICE and NICE-Caro, with similar consistencies for domains and type of assessment. For transferability, NICE and Caro identified the same study, which scored high on EURONHEED, as transferable to the UK, while Welte, which considered methodological quality, identified none as transferable.

CONCLUSIONS

We found that the choice of checklist can affect findings on study quality and decisions about whether study results are transferable, indicating that different checklists may shortlist different sets of studies in formulating policy recommendations, leading to different policy decisions. Our systematic approach for evaluating the content of methodological quality and transferability checklists of economic evaluations can be extended to other checklists.

Coverage with evidence development for medical devices in Europe: Can practice meet theory?

Health economists have written extensively on the design and implementation of coverage with evidence development (CED) schemes and have proposed theoretical frameworks based on cost-effectiveness modeling and value of information analysis. CED may aid decision-makers when there is uncertainty about the (cost-)effectiveness of a new health technology at the time of reimbursement. Medical devices are potential candidates for CED schemes, as regulatory regimes do not usually require the same level of efficacy and safety data normally needed for pharmaceuticals. The purpose of this research is to assess whether the actual practice of CED for medical devices in Europe meets the theoretical principles proposed by health economists and whether theory and practice can be more closely aligned. Based on decision-makers' perceptions of the challenges associated with CED schemes, plus examples from the schemes themselves, we discuss a series of proposals for assessing the desirability of schemes, their design, implementation, and evaluation. These proposals, while reflecting the practical challenges with developing CED programs, embody many of the principles suggested by economists and should support decision-makers in dealing with uncertainty about the real-world performance of devices.

Emerging Therapies for COVID-19: The Value of Information From More Clinical Trials

OBJECTIVES

The COVID-19 pandemic necessitates time-sensitive policy and implementation decisions regarding new therapies in the face of uncertainty. This study aimed to quantify consequences of approving therapies or pursuing further research: immediate approval, use only in research, approval with research (eg, emergency use authorization), or reject.

METHODS

Using a cohort state-transition model for hospitalized patients with COVID-19, we estimated quality-adjusted life-years (QALYs) and costs associated with the following interventions: hydroxychloroquine, remdesivir, casirivimab-imdevimab, dexamethasone, baricitinib-remdesivir, tocilizumab, lopinavir-ritonavir, interferon beta-1a, and usual care. We used the model outcomes to conduct cost-effectiveness and value of information analyses from a US healthcare perspective and a lifetime horizon.

RESULTS

Assuming a $100 000-per-QALY willingness-to-pay threshold, only remdesivir, casirivimab-imdevimab, dexamethasone, baricitinib-remdesivir, and tocilizumab were (cost-) effective (incremental net health benefit 0.252, 0.164, 0.545, 0.668, and 0.524 QALYs and incremental net monetary benefit $25 249, $16 375, $54 526, $66 826, and $52 378). Our value of information analyses suggest that most value can be obtained if these 5 therapies are approved for immediate use rather than requiring additional randomized controlled trials (RCTs) (net value $20.6 billion, $13.4 billion, $7.4 billion, $54.6 billion, and $7.1 billion), hydroxychloroquine (net value $198 million) is only used in further RCTs if seeking to demonstrate decremental cost-effectiveness and otherwise rejected, and interferon beta-1a and lopinavir-ritonavir are rejected (ie, neither approved nor additional RCTs).

CONCLUSIONS

Estimating the real-time value of collecting additional evidence during the pandemic can inform policy makers and clinicians about the optimal moment to implement therapies and whether to perform further research.

A Conceptual Framework for Life-Cycle Health Technology Assessment

OBJECTIVES

Health technology assessment (HTA) uses evidence appraisal and synthesis with economic evaluation to inform adoption decisions. Standard HTA processes sometimes struggle to (1) support decisions that involve significant uncertainty and (2) encourage continued generation of and adaptation to new evidence. We propose the life-cycle (LC)-HTA framework, addressing these challenges by providing additional tools to decision makers and improving outcomes for all stakeholders.

METHODS

Under the LC-HTA framework, HTA processes align to LC management. LC-HTA introduces changes in HTA methods to minimize analytic time while optimizing decision certainty. Where decision uncertainty exists, we recommend risk-based pricing and research-oriented managed access (ROMA). Contractual procurement agreements define the terms of reassessment and provide additional decision options to HTA agencies. LC-HTA extends value-of-information methods to inform ROMA agreements, leveraging routine, administrative data, and registries to reduce uncertainty.

RESULTS

LC-HTA enables the adoption of high-value high-risk innovations while improving health system sustainability through risk-sharing and reducing uncertainty. Responsiveness to evolving evidence is improved through contractually embedded decision rules to simplify reassessment. ROMA allows conditional adoption to obtain additional information, with confidence that the net value of that adoption decision is positive.

CONCLUSIONS

The LC-HTA framework improves outcomes for patients, sponsors, and payers. Patients benefit through earlier access to new technologies. Payers increase the value of the technologies they invest in and gain mechanisms to review investments. Sponsors benefit through greater certainty in outcomes related to their investment, swifter access to markets, and greater opportunities to demonstrate value.

An Efficient Method for Computing Expected Value of Sample Information for Survival Data from an Ongoing Trial

BACKGROUND

Decisions about new health technologies are increasingly being made while trials are still in an early stage, which may result in substantial uncertainty around key decision drivers such as estimates of life expectancy and time to disease progression. Additional data collection can reduce uncertainty, and its value can be quantified by computing the expected value of sample information (EVSI), which has typically been described in the context of designing a future trial. In this article, we develop new methods for computing the EVSI of extending an existing trial's follow-up, first for an assumed survival model and then extending to capture uncertainty about the true survival model.

METHODS

We developed a nested Markov Chain Monte Carlo procedure and a nonparametric regression-based method. We compared the methods by computing single-model and model-averaged EVSI for collecting additional follow-up data in 2 synthetic case studies.

RESULTS

There was good agreement between the 2 methods. The regression-based method was fast and straightforward to implement, and scales easily to include any number of candidate survival models in the model uncertainty case. The nested Monte Carlo procedure, on the other hand, was extremely computationally demanding when we included model uncertainty.

CONCLUSIONS

We present a straightforward regression-based method for computing the EVSI of extending an existing trial's follow-up, both where a single known survival model is assumed and where we are uncertain about the true survival model. EVSI for ongoing trials can help decision makers determine whether early patient access to a new technology can be justified on the basis of the current evidence or whether more mature evidence is needed.

HIGHLIGHTS

Decisions about new health technologies are increasingly being made while trials are still in an early stage, which may result in substantial uncertainty around key decision drivers such as estimates of life-expectancy and time to disease progression. Additional data collection can reduce uncertainty, and its value can be quantified by computing the expected value of sample information (EVSI), which has typically been described in the context of designing a future trial.In this article, we have developed new methods for computing the EVSI of extending a trial's follow-up, both where a single known survival model is assumed and where we are uncertain about the true survival model. We extend a previously described nonparametric regression-based method for computing EVSI, which we demonstrate in synthetic case studies is fast, straightforward to implement, and scales easily to include any number of candidate survival models in the EVSI calculations.The EVSI methods that we present in this article can quantify the need for collecting additional follow-up data before making an adoption decision given any decision-making context.

Modelling approaches for histology-independent cancer drugs to inform NICE appraisals: a systematic review and decision-framework

BACKGROUND

The first histology-independent marketing authorisation in Europe was granted in 2019. This was the first time that a cancer treatment was approved based on a common biomarker rather than the location in the body at which the tumour originated. This research aims to explore the implications for National Institute for Health and Care Excellence appraisals.

METHODS

Targeted reviews were undertaken to determine the type of evidence that is likely to be available at the point of marketing authorisation and the analyses required to support National Institute for Health and Care Excellence appraisals. Several challenges were identified concerning the design and conduct of trials for histology-independent products, the greater levels of heterogeneity within the licensed population and the use of surrogate end points. We identified approaches to address these challenges by reviewing key statistical literature that focuses on the design and analysis of histology-independent trials and by undertaking a systematic review to evaluate the use of response end points as surrogate outcomes for survival end points. We developed a decision framework to help to inform approval and research policies for histology-independent products. The framework explored the uncertainties and risks associated with different approval policies, including the role of further data collection, pricing schemes and stratified decision-making.

RESULTS

We found that the potential for heterogeneity in treatment effects, across tumour types or other characteristics, is likely to be a central issue for National Institute for Health and Care Excellence appraisals. Bayesian hierarchical methods may serve as a useful vehicle to assess the level of heterogeneity across tumours and to estimate the pooled treatment effects for each tumour, which can inform whether or not the assumption of homogeneity is reasonable. Our review suggests that response end points may not be reliable surrogates for survival end points. However, a surrogate-based modelling approach, which captures all relevant uncertainty, may be preferable to the use of immature survival data. Several additional sources of heterogeneity were identified as presenting potential challenges to National Institute for Health and Care Excellence appraisal, including the cost of testing, baseline risk, quality of life and routine management costs. We concluded that a range of alternative approaches will be required to address different sources of heterogeneity to support National Institute for Health and Care Excellence appraisals. An exemplar case study was developed to illustrate the nature of the assessments that may be required.

CONCLUSIONS

Adequately designed and analysed basket studies that assess the homogeneity of outcomes and allow borrowing of information across baskets, where appropriate, are recommended. Where there is evidence of heterogeneity in treatment effects and estimates of cost-effectiveness, consideration should be given to optimised recommendations. Routine presentation of the scale of the consequences of heterogeneity and decision uncertainty may provide an important additional approach to the assessments specified in the current National Institute for Health and Care Excellence methods guide.

FURTHER RESEARCH

Further exploration of Bayesian hierarchical methods could help to inform decision-makers on whether or not there is sufficient evidence of homogeneity to support pooled analyses. Further research is also required to determine the appropriate basis for apportioning genomic testing costs where there are multiple targets and to address the challenges of uncontrolled Phase II studies, including the role and use of surrogate end points.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Evidence Synthesis programme and will be published in full in Health Technology Assessment; Vol. 25, No. 76. See the NIHR Journals Library website for further project information.

Program Evaluation of Population- and System-Level Policies: Evidence for Decision Making

BACKGROUND

Policy evaluations often focus on ex post estimation of causal effects on short-term surrogate outcomes. The value of such information is limited for decision making, as the failure to reflect policy-relevant outcomes and disregard for opportunity costs prohibits the assessment of value for money. Further, these evaluations do not always consider all relevant evidence, other courses of action, or decision uncertainty.

METHODS

In this article, we explore how policy evaluation could better meet the needs of decision making. We begin by defining the evidence required to inform decision making. We then conduct a literature review of challenges in evaluating policies. Finally, we highlight potential methods available to help address these challenges.

RESULTS

The evidence required to inform decision making includes the impacts on the policy-relevant outcomes, the costs and associated opportunity costs, and the consequences of uncertainty. Challenges in evaluating health policies are described using 8 categories: 1) valuation space; 2) comparators; 3) time of evaluation; 4) mechanisms of action; 5) effects; 6) resources, constraints, and opportunity costs; 7) fidelity, adaptation, and level of implementation; and 8) generalizability and external validity. Methods from a broad set of disciplines are available to improve policy evaluation, relating to causal inference, decision-analytic modeling, theory of change, realist evaluation, and structured expert elicitation.

LIMITATIONS

The targeted review may not identify all possible challenges, and the methods covered are not exhaustive.

CONCLUSIONS

Evaluations should provide appropriate evidence to inform decision making. There are challenges in evaluating policies, but methods from multiple disciplines are available to address these challenges.

IMPLICATIONS

Evaluators need to carefully consider the decision being informed, the necessary evidence to inform it, and the appropriate methods.[Box: see text].

HTA methodology and value frameworks for evaluation and policy making for cell and gene therapies

This last decade has been marked by significant advances in the development of cell and gene (C&G) therapies, such as gene targeting or stem cell-based therapies. C&G therapies offer transformative benefits to patients but present a challenge to current health technology decision-making systems because they are typically reviewed when clinical efficacy data are very limited and when there is uncertainty about the long-term durability of outcomes. These challenges are not unique to C&G therapies, but they face more of these barriers, reflecting the need for adapting existing value assessment frameworks. Still, C&G therapies have the potential to be cost-effective even at very high price points. The impact on healthcare budgets will depend on the success rate of pipeline assets and on the extent to which C&G therapies will expand to wider pathologies beyond rare or ultra-rare diseases. Getting pricing and reimbursement models right is important for incentivising research and development investment while not jeopardising the sustainability of healthcare systems. Payers and manufacturers therefore need to acknowledge each other's constraints-limitations in the evidence generation on the manufacturer side, budget considerations on the payer side-and embrace innovative thinking and approaches to ensure timely delivery of therapies to patients. Several experts in health technology assessment and clinical experts have worked together to produce this publication and identify methodological and policy options to improve the assessment of C&G therapies, and make it happen better, faster and sustainably in the coming years.

"Stick or Twist?" Negotiating Price and Data in an Era of Conditional Approval

BACKGROUND

Changes in the regulatory context enable faster approval of transformative medicines. They also lead to health technology assessment (HTA) agencies having to make decisions with less evidence. In response, HTA agencies have also initiated forms of conditional approval. When the evidence base for a new oncology treatment leaves substantial uncertainty, the new Cancer Drugs Fund allows the National Institute for Heath and Care Excellence to give the manufacturer two options: (1) offer a low price based on conservative assumptions and obtain immediate approval ("stick") or (2) wait until the evidence base has further matured before finalizing a potentially higher agreed price ("twist").

OBJECTIVES

The purpose of this article is to explain how, using the theoretical framework of the expected value of sample information, simulation methods can help inform a manufacturer's decisions when faced with the option to stick or twist.

METHODS

We first summarize a general model to help frame the manufacturer's negotiating strategy. We then use a motivating case study, based on a hypothetical immunotherapy, to illustrate how manufacturers can use simulation methods to robustly characterize the uncertainty inherent to further data collection and incorporate this uncertainty within their decision making.

RESULTS

Our approach allows us to estimate the commercial value of generating additional data (the difference between the estimated net present value of stick and twist). We test the sensitivity of the results to different assumptions via scenario analyses.

CONCLUSIONS

This article shows that simulation methods can be used to help pharmaceutical managers make informed strategic decisions in contexts of uncertainty.

Are There Different Evidence Thresholds for Genomic Versus Clinical Precision Medicine? A Value of Information-Based Framework Applied to Antiplatelet Drug Therapy

BACKGROUND

The threshold of sufficient evidence for adoption of clinically- and genomically-guided precision medicine (PM) has been unclear.

OBJECTIVE

To evaluate evidence thresholds for clinically guided PM versus genomically guided PM.

METHODS

We develop an "evidence threshold criterion" (ETC), which is the time-weighted difference between expected value of perfect information and incremental net health benefit minus the cost of research, and use it as a measure of evidence threshold that is proportional to the upper bound of disutility to a risk-averse decision maker for adopting a new intervention under decision uncertainty. A larger (more negative) ETC value indicates that only decision makers with low risk aversion would adopt new intervention. We evaluated the ETC plus cost of research (ETCc), assuming the same cost of research for both interventions, over time for a pharmacogenomic (PGx) testing intervention and avoidance of a drug-drug interaction (aDDI) intervention for acute coronary syndrome patients indicated for antiplatelet therapy. We then examined how the ETC may explain incongruous decision making across different national decision-making bodies.

RESULTS

The ETCc for PGx increased over time, whereas the ETCc for aDDI decreased to a negative value over time, indicating that decision makers with even low risk aversion will have doubts in adopting PGx, whereas decision makers who are highly risk-averse will continue to have doubts about adopting aDDI. National recommendation bodies appear to be consistent over time within their own decision making, but had different levels of risk aversion.

CONCLUSION

The ETC may be a useful metric for assessing policy makers' risk preferences and, in particular, understanding differences in policy recommendations for genomic versus clinical PM.

Incorporating Affordability Concerns Within Cost-Effectiveness Analysis for Health Technology Assessment

BACKGROUND

Recent policy developments and journal articles have emphasized a divergence: when interventions are found to be cost-effective but unaffordable. This apparent paradox reflects a conventional practice of cost-effectiveness analysis that does not properly evaluate the opportunity costs of an intervention that imposes non-marginal costs on the healthcare system.

OBJECTIVE

Taking the perspective of an exogenously resource constrained decision maker, this paper presents a framework by which concerns for affordability can be appropriately incorporated within cost-effectiveness analysis.

METHODS

A net benefit framework is proposed where health opportunity costs are estimated for each simulation iteration within each time period. The framework is applied to a hypothetical case study based on the recent experience of the English NHS with new hepatitis C drugs.

RESULTS

Under the proposed framework, but not under conventional cost-effectiveness analysis, estimates of health opportunity costs differ between scenarios involving different profiles of budget impact even when their net present value, or expected value, are the same.

CONCLUSIONS

The framework presented here reflects the importance of the scale of budget impacts along with their uncertainty distribution and time profile. In doing so it resolves issues with the conduct of conventional cost-effectiveness analysis where affordability concerns are not explicitly incorporated.

Uncertainty and Coverage With Evidence Development: Does Practice Meet Theory?

OBJECTIVES

In theory, a successful coverage with evidence development (CED) scheme is one that addresses the most important uncertainties in a given assessment. We investigated the following: (1) which uncertainties were present during the initial assessment of 3 Dutch CED cases, (2) how these uncertainties were integrated in the initial assessments, (3) whether CED research plans included the identified uncertainties, and (4) issues with managing uncertainty in CED research and ways forward from these issues.

METHODS

Three CED initial assessment dossiers were analyzed and 16 stakeholders were interviewed. Uncertainties were identified in interviews and dossiers and were categorized in different causes: unavailability, indirectness, and imprecision of evidence. Identified uncertainties could be mentioned, described, and explored. Issues and ways forward to address uncertainty in CED schemes were discussed during the interviews.

RESULTS

Forty-two uncertainties were identified. Thirteen (31%) were caused by unavailability, 17 (40%) by indirectness, and 12 (29%) by imprecision. Thirty-four uncertainties (81%) were only mentioned, 19 (45%) were described, and the impact of 3 (7%) uncertainties on the results was explored in the assessment dossiers. Seventeen uncertainties (40%) were included in the CED research plans. According to stakeholders, research did not address the identified uncertainty, but CED research should be designed to focus on these.

CONCLUSIONS

In practice, uncertainties were neither systematically nor completely identified in the analyzed CED schemes. A framework would help to systematically identify uncertainty, and this process should involve all stakeholders. Value of information analysis, and the uncertainties that are not included in this analysis should inform CED research design.

Value of information analysis in telehealth for chronic heart failure management

OBJECTIVES

Value of information (VOI) analysis provides information on opportunity cost of a decision in healthcare by estimating the cost of reducing parametric uncertainty and quantifying the value of generating additional evidence. This study is an application of the VOI methodology to the problem of choosing between home telemonitoring and nurse telephone support over usual care in chronic heart failure management in the Netherlands.

METHODS

The expected value of perfect information (EVPI) and the expected value of partially perfect information (EVPPI) analyses were based on an informal threshold of €20K per quality-adjusted life-year. These VOI-analyses were applied to a probabilistic Markov model comparing the 20-year costs and effects in three interventions. The EVPPI explored the value of decision uncertainty caused by the following group of parameters: treatment-specific transition probabilities between New York Heart Association (NYHA) defined disease states, utilities associated with the disease states, number of hospitalizations and ER visits, health state specific costs, and the distribution of patients per NYHA group. We performed the analysis for two population sizes in the Netherlands-patients in all NYHA classes of severity, and patients in NYHA IV class only.

RESULTS

The population EVPI for an effective population of 2,841,567 CHF patients in All NYHA classes of severity over the next 20 years is more than €4.5B, implying that further research is highly cost-effective. In the NYHA IV only analysis, for the effective population of 208,003 patients over next 20 years, the population EVPI at the same informal threshold is approx. €590M. The EVPPI analysis showed that the only relevant group of parameters that contribute to the overall decision uncertainty are transition probabilities, in both All NYHA and NYHA IV analyses.

CONCLUSIONS

Results of our VOI exercise show that the cost of uncertainty regarding the decision on reimbursement of telehealth interventions for chronic heart failure patients is high in the Netherlands, and that future research is needed, mainly on the transition probabilities.

Analytic Considerations in Applying a General Economic Evaluation Reference Case to Gene Therapy

The concept of a reference case, first proposed by the US Panel on Cost-Effectiveness in Health and Medicine, has been used to specify the required methodological features of economic evaluations of healthcare interventions. In the case of gene therapy, there is a difference of opinion on whether a specific methodological reference case is required. The aim of this article was to provide a more detailed analysis of the characteristics of gene therapy and the extent to which these characteristics warrant modifications to the methods suggested in general reference cases for economic evaluation. We argue that a completely new reference case is not required, but propose a tailored checklist that can be used by analysts and decision makers to determine which aspects of economic evaluation should be considered further, given the unique nature of gene therapy.

Which Costs Matter? Costs Included in Economic Evaluation and their Impact on Decision Uncertainty for Stable Coronary Artery Disease

BACKGROUND

Variation exists in the resource categories included in economic evaluations, and National Institute for Health and Care Excellence (NICE) guidance suggests the inclusion only of costs related to the index condition or intervention. However, there is a growing consensus that all healthcare costs should be included in economic evaluations for Health Technology Assessments (HTAs), particularly those related to extended years of life.

OBJECTIVE AND METHODS

We aimed to quantify the impact of a range of cost categories on the adoption decision about a hypothetical intervention, and uncertainty around that decision, for stable coronary artery disease (SCAD) based on a dataset comprising 94,966 patients. Three costing scenarios were considered: coronary heart disease (CHD) costs only, cardiovascular disease (CVD) costs and all costs. The first two illustrate different interpretations of what might be regarded as related costs.

RESULTS

Employing a 20-year time horizon, the highest mean expected incremental cost was when all costs were included (£2468) and the lowest when CVD costs only were included (£2377). The probability of the treatment being cost effective, estimating health opportunity costs using a ratio of £30,000 per quality-adjusted life-year (QALY), was different for each of the CHD (70%) costs, CVD costs (73%) and all costs (56%) scenarios. The results concern a hypothetical intervention and are illustrative only, as such they cannot necessarily be generalised to all interventions and diseases.

CONCLUSIONS

Cost categories included in an economic evaluation of SCAD impact on estimates of both cost effectiveness and decision uncertainty. With an aging and co-morbid population, the inclusion of all healthcare costs may have important ramifications for the selection of healthcare provision on economic grounds.

Highly Innovative Drug Program in the Czech Republic: Description and Pharmacoeconomic Results-Cost-Effectiveness and Budget Impact Analyses

BACKGROUND

Highly innovative drugs (HIDs) can be granted 2 to 3 years of temporary reimbursement (TR) to provide timely patient access and to collect real-world evidence through registries in the Czech Republic. A TR applicant does not need to comply with cost-effectiveness (CE) requirements and the willingness-to-pay threshold. It is only when mandatory transition to permanent reimbursement (PR) status occurs does the drug need to comply with CE and willingness-to-pay requirements.

OBJECTIVES

To describe and evaluate the HID program in the Czech Republic by analyzing the pharmacoeconomic results when a drug starts with TR status and transitions to PR status.

METHODS

The study was a retrospective analysis of reimbursement decisions of HIDs. All drugs approved for TR (valid from January 2008 to January 2018) were identified. A description of the HIDs and their pharmacoeconomic results were analyzed.

RESULTS

Fifty TR drugs were identified. Most (68%) were oncology drugs and 44% were orphan drugs. After the expiration of their TR status, 83% were successfully transitioned to PR status. Cost-utility analysis was used to support CE results in 42% of the TR drugs. The mean incremental cost-effectiveness ratio (cost/quality-adjusted life-year) of drugs that entered TR status was €97,868. When the time came for transition to PR status, the mean incremental cost-effectiveness ratio was €34,086 (lower by 65%). Net budget impact increased by 3% and decreased by 25% in the first and fifth years, respectively, after applying for PR.

CONCLUSIONS

This analysis provides better insight into the HID program for costly innovative drugs over a 10-year follow-up. A successful transition to PR status was observed for most of the HIDs (83%).

Smaller clinical trials for decision making; a case study to show p-values are costly

Background: Clinical trials might be larger than needed because arbitrary levels of statistical confidence are sought in the results. Traditional sample size calculations ignore the marginal value of the information collected for decision making. The statistical hypothesis testing objective is misaligned with the goal of generating information necessary for decision-making. The aim of the present study was to show that for a case study clinical trial designed to test a prior hypothesis against an arbitrary threshold of confidence more participants were recruited than needed to make a good decision about adoption.

Methods: We used data from a recent RCT powered for traditional rules of statistical significance. The data were also used for an economic analysis to show the intervention led to cost-savings and improved health outcomes. Adoption represented a sensible investment for decision-makers. We examined the effect of reducing the trial’s sample size on the results of the statistical hypothesis-testing analysis and the conclusions that would be drawn by decision-makers reading the economic analysis.

Results: As the sample size reduced it became more likely that the null hypothesis of no difference in the primary outcome between groups would fail to be rejected. For decision-makers reading the economic analysis, reducing the sample size had little effect on the conclusion about whether to adopt the intervention. There was always high probability the intervention reduced costs and improved health.

Conclusions: Decision makers managing health services are largely invariant to the sample size of the primary trial and the arbitrary p-value of 0.05. If the goal is to make a good decision about whether the intervention should be adopted widely, then that could have been achieved with a much smaller trial. It is plausible that hundreds of millions of research dollars are wasted each year recruiting more participants than required for RCTs.

The Australian Managed Entry Scheme: Are We Getting it Right?

In 2010, the Australian Government introduced the managed entry scheme (MES) to improve patient access to subsidised drugs on the Pharmaceutical Benefits Scheme and enhance the quality of evidence provided to decision makers. The aim of this paper was to critically review the Australian MES experience. We performed a comprehensive review of publicly available Pharmaceutical Benefits Advisory Committee online documents from January 2010 to July 2017. Relevant information on each MES agreement was systematically extracted, including its rationale, the conditions that guided its implementation and its policy outcomes. We identified 11 drugs where an MES was considered. Most of the identified drugs (75%) were antineoplastic agents and the main uncertainty was the overall survival benefit. More than half of the MES proposals were made by sponsors and most of the schemes were considered after previous rejected/deferred submissions for reimbursement. An MES was not established in 8 of 11 drugs (73%) despite the high evidence uncertainty. Nevertheless, six of these eight drugs were listed after the sponsors reduced their prices. Three MESs were established and implemented by Deeds of Agreement. The three cases were concluded and the required data were submitted within the agreed time frames. The need for feasibility and value of an MES should be carefully considered by stakeholders before embarking on such an agreement. It is essential to engage major stakeholders, including patient representatives, in this process. The conditions governing MESs should be clear, transparent and balanced to address the expectations of various stakeholders.

Decision uncertainty and value of further research: a case-study in fenestrated endovascular aneurysm repair for complex abdominal aortic aneurysms

Background

Fenestrated endovascular aneurysm repair (fEVAR) is a new approach for complex abdominal aortic aneurysms, limited to a few specialist centers, with limited evidence base. We developed a cost-effectiveness decision model of fEVAR compared to open surgical repair (OSR) to investigate the likely direction of costs and benefits and inform further research projects on this technology.

Methods

A systematic review with meta-analysis and a four-state Markov model were used to estimate the cost-effectiveness of fEVAR versus OSR. We used a recent coverage with evidence development framework to characterize the main sources of uncertainty and inform decisions about the type of further research that would be most worthwhile and feasible.

Results

Seven observational comparative studies were identified, of which four presented odds ratios adjusted for confounders. The odds ratios for operative mortality varied widely between studies. Assuming a central estimate of the odds ratio of 0.54 (95% CI 0.05–6.24), the decision model estimated that the incremental cost per quality adjusted life year (QALY) was £74,580/QALY with a probability of 9 and 16% of being cost-effective at standard cost-effectiveness thresholds of £20,000/QALY and £30,000/QALY, respectively. The Expected Value of Perfect Information over 10 years at a threshold of £20,000/QALY was £11.2 million. Operative mortality contributed to most of the uncertainty in the decision model.

Conclusions

In the case of “maturing technologies”, decision modelling indicates the likely direction of costs and benefits and guides the development of further research projects. In our analysis of fEVAR versus OSR, decision uncertainty, particularly around operative mortality, might be effectively resolved by a short-term RCT, or possibly a well-conducted comparative observational study. Decision makers may consider that a conditional coverage decision is warranted with assessments required to make this type of recommendation depending on local priorities and circumstances.

Electronic supplementary material

The online version of this article (10.1186/s12962-018-0098-7) contains supplementary material, which is available to authorized users.

Ustekinumab for Treating Moderately to Severely Active Crohn's Disease after Prior Therapy: An Evidence Review Group Perspective of a NICE Single Technology Appraisal

As part of the single technology appraisal (STA) process, the National Institute for Health and Care Excellence (NICE) invited Janssen to submit evidence on the clinical and cost effectiveness of their drug ustekinumab, an interleukin-12/23 inhibitor, for treating moderate-to-severe active Crohn's disease (CD). The Centre for Reviews and Dissemination (CRD) and Centre for Health Economics (CHE) Technology Appraisal Group at the University of York was commissioned to act as the independent Evidence Review Group (ERG). This article provides a description of the Company's submission, the ERG's critical review of submitted evidence, and the resulting NICE guidance. The main supporting clinical evidence was derived from four well conducted, randomised controlled trials, comparing ustekinumab with placebo in two sub-populations (conventional care failure and anti-TNFα failure patients) of adults with moderate-to-severe CD. Three trials assessed treatment induction over 8 weeks, while the fourth recruited successfully induced patients into a maintenance trial for 1 year. These trials showed ustekinumab to be more effective than placebo in terms of its ability to induce and maintain clinical response and remission. In the absence of any direct head-to-head data, the Company conducted a network meta-analysis (NMA), which synthesised induction trial data on ustekinumab and relevant comparators (vedolizumab, adalimumab and infliximab) using placebo data as a common comparator. This analysis found ustekinumab to be of comparable efficacy to previously approved biologics in treatment induction. A 'treatment sequence analysis' compared long-term treatment efficacy, finding ustekinumab to be comparable in maintaining treatment response and remission to the three other biologic therapies. However, the ERG had identified many limitations and potential bias in this analysis, and urged caution when interpreting the results. The Company's economic model estimated ustekinumab to be dominant in both sub-populations compared with conventional care; however, the ERG's preferred base-case estimated an incremental cost-effectiveness ratio of £109,279 in the conventional care failure sub-population, and £110,967 in the anti-TNFα failure sub-population when compared with conventional care. However, the ERG identified significant failings in both the model structure and data inputs, which could not be addressed without complete restructuring. The ERG considered that the economic analysis presented by the Company failed to adequately address the decision problem specified in NICE's scope. The NICE Appraisal Committee recommended ustekinumab within its market authorisation, on the grounds of sufficiently similar efficacy and costs to previously recommended biologic therapies. However, the ERG's analyses demonstrated that all currently recommended biologics are unlikely to be cost effective relative to conventional care, raising broader questions regarding the appropriateness of cost-comparison exercises for decision making.

The Impact of Decision Makers' Constraints on the Outcome of Value of Information Analysis

BACKGROUND

When proven effective, decision making regarding reimbursement of new health technology typically involves ethical, social, legal, and health economic aspects and constraints. Nevertheless, when applying standard value of information (VOI) analysis, the value of collecting additional evidence is typically estimated assuming that only cost-effectiveness outcomes guide such decisions.

OBJECTIVES

To illustrate how decision makers' constraints can be incorporated into VOI analyses and how these may influence VOI outcomes.

METHODS

A simulation study was performed to estimate the cost-effectiveness of a new hypothetical technology compared with usual care. Constraints were defined for the new technology on 1) the maximum acceptable rate of complications and 2) the maximum acceptable additional budget. The expected value of perfect information (EVPI) for the new technology was estimated in various scenarios, both with and without incorporating these constraints.

RESULTS

For a willingness-to-pay threshold of €20,000 per quality-adjusted life-year, the probability that the new technology was cost-effective equaled 57%, with an EVPI of €1868 per patient. Applying the complication rate constraint reduced the EVPI to €1137. Similarly, the EVPI reduced to €770 when applying the budget constraint. Applying both constraints simultaneously further reduced the EVPI to €318.

CONCLUSIONS

When decision makers explicitly apply additional constraints, beyond a willingness-to-pay threshold, to reimbursement decisions, these constraints can and should be incorporated into VOI analysis as well, because they may influence VOI outcomes. This requires continuous interaction between VOI analysts and decision makers and is expected to improve both the relevance and the acceptance of VOI outcomes.

How to design the cost-effectiveness appraisal process of new healthcare technologies to maximise population health: A conceptual framework

This paper presents a conceptual framework to analyse the design of the cost-effectiveness appraisal process of new healthcare technologies. The framework characterises the appraisal processes as a diagnostic test aimed at identifying cost-effective (true positive) and non-cost-effective (true negative) technologies. Using the framework, factors that influence the value of operating an appraisal process, in terms of net gain to population health, are identified. The framework is used to gain insight into current policy questions including (a) how rigorous the process should be, (b) who should have the burden of proof, and (c) how optimal design changes when allowing for appeals, price reductions, resubmissions, and re-evaluations. The paper demonstrates that there is no one optimal appraisal process and the process should be adapted over time and to the specific technology under assessment. Optimal design depends on country-specific features of (future) technologies, for example, effect, price, and size of the patient population, which might explain the difference in appraisal processes across countries. It is shown that burden of proof should be placed on the producers and that the impact of price reductions and patient access schemes on the producer's price setting should be considered when designing the appraisal process.

Levers for addressing medical underuse and overuse: achieving high-value health care

The preceding papers in this Series have outlined how underuse and overuse of health-care services occur within a complex system of health-care production, with a multiplicity of causes. Because poor care is ubiquitous and has considerable consequences for the health and wellbeing of billions of people around the world, remedying this problem is a morally and politically urgent task. Universal health coverage is a key step towards achieving the right care. Therefore, full consideration of potential levers of change must include an upstream perspective-ie, an understanding of the system-level factors that drive overuse and underuse, as well as the various incentives at work during a clinical encounter. One example of a system-level factor is the allocation of resources (eg, hospital beds and clinicians) to meet the needs of a local population to minimise underuse or overuse. Another example is priority setting using tools such as health technology assessment to guide the optimum diffusion of safe, effective, and cost-effective health-care services. In this Series paper we investigate a range of levers for eliminating medical underuse and overuse. Some levers could operate effectively (and be politically viable) across many different health and political systems (eg, increase patient activation with decision support) whereas other levers must be tailored to local contexts (eg, basing coverage decisions on a particular cost-effectiveness ratio). Ideally, policies must move beyond the purely incremental; that is, policies that merely tinker at the policy edges after underuse or overuse arises. In this regard, efforts to increase public awareness, mobilisation, and empowerment hold promise as universal methods to reset all other contexts and thereby enhance all other efforts to promote the right care.

Characterising Uncertainty in the Assessment of Medical Devices and Determining Future Research Needs

Decisions about the adoption of medical interventions are informed by evidence on their costs and effects. For a range of reasons, evidence relating to medical devices may be limited. The decision to adopt a device early in its life cycle when the evidence base is least mature may impact on the prospects of acquiring further evidence to reduce uncertainties. Equally, rejecting a device will result in no uptake in practice and hence no chance to learn about performance. Decision options such as 'only in research' or 'approval with research' can overcome these issues by allowing patients early access to promising new technologies while limiting the risks associated with making incorrect decisions until more evidence or learning is established. In this paper, we set out the issues relating to uncertainty and the value of research specific to devices: learning curve effects, incremental device innovation, investment and irrecoverable costs, and dynamic pricing. We show the circumstances under which an only in research or approval with research scheme may be an appropriate policy choice. We also consider how the value of additional research might be shared between the manufacturer and health sector to help inform who might reasonably be expected to conduct the research needed. © 2017 The Authors. Health Economics published by John Wiley & Sons, Ltd.

The role of imperfect surrogate endpoint information in drug approval and reimbursement decisions

Approval of new drugs is increasingly reliant on "surrogate endpoints," which correlate with but imperfectly predict clinical benefits. Proponents argue surrogate endpoints allow for faster approval, but critics charge they provide inadequate evidence. We develop an economic framework that addresses the value of improvement in the predictive power, or "quality," of surrogate endpoints, and clarifies how quality can influence decisions by regulators, payers, and manufacturers. For example, the framework shows how lower-quality surrogates lead to greater misalignment of incentives between payers and regulators, resulting in more drugs that are approved for use but not covered by payers. Efficient price-negotiation in the marketplace can help align payer incentives for granting access based on surrogates. Higher-quality surrogates increase manufacturer profits and social surplus from early access to new drugs. Since the return on better quality is shared between manufacturers and payers, private incentives to invest in higher-quality surrogates are inefficiently low.