Evidence-Based Decision Making: When Should We Wait For More Information?

Exploring the misalignment on the value of further research between payers and manufacturers. A case study on a novel total artificial heart

Payers and manufacturers can disagree on the appropriate level of evidence that is required for new medical devices, resulting in high societal costs due to decisions taken with sub-optimal information. A cost-effectiveness model of a hypothetical total artificial heart was built using data from the literature and the (simulated) results of a pivotal study. The expected value of perfect information (EVPI) was calculated from both the payer and manufacturer perspectives, using net monetary benefit and the company's return on investment respectively. A function was also defined, linking effectiveness to market shares. Additional constraints such as a minimum clinical difference or maximum budget impact were introduced into the company's decisions to simulate additional barriers to adoption. The difference in the EVPI between manufacturers and payers varied greatly depending on the underlying decision rules and constraints. The manufacturer's EVPI depends on the probability of being reimbursed, the uncertainty on the (cost-)effectiveness of the technology, as well as other parameters relating to initial investments, operating costs and market dynamics. The use of Value of information for both perspectives can outline potential misalignments and can be particularly useful to inform early dialogs between manufacturers and payers, or negotiations on conditional reimbursement schemes.

State of the ART? Two New Tools for Risk Communication in Health Technology Assessments

PURPOSE

Outcomes of health technology assessments (HTA) are uncertain, and decision-making is associated with a risk. This risk, consisting of the probability of making a wrong decision and its impact, is rarely considered in HTA. This hampers transparent and consistent risk assessment and management. The aim of this study was to develop risk communication tools in the context of health technology decision-making under uncertainty.

METHODS

We performed a scoping review of tools for uncertainty and risk communication within HTA using citation pearl-growing. We developed two tools, drawing on existing publications on risk and uncertainty communication for inspiration. Individual semi-structured interviews with HTA stakeholders were performed to identify potential improvements in usefulness, user-friendliness, and information adequacy. Tools were amended and further evaluated in a real-world HTA and workshop with HTA stakeholders.

RESULTS

The identified risk communication tools did not include non-quantified uncertainties, and did not link to risk management strategies. We developed two tools: the Assessment of Risk Table (ART), for a summary of quantified and non-quantified uncertainties and the resulting risk assessment, and the Appraisal of Risk Chart (ARCH), for linking net benefit and risk outcomes to appropriate risk management strategies. Stakeholders appreciated the usefulness of the tools. They also highlighted that more information on local policy options was required for optimal risk management use, and HTA processes may need adapting.

CONCLUSION

The risk communication tools presented here can help assess risk, facilitate communication between analysts and decision-makers, and guide the appropriate use of available risk management strategies.

Implementation Barriers to Value of Information Analysis in Health Technology Decision Making: Results From a Process Evaluation

OBJECTIVES

Value of information (VOI) analysis can support health technology assessment decision making, but it is a long way from being standard use. The objective of this study was to understand barriers to the implementation of VOI analysis and propose actions to overcome these.

METHODS

We performed a process evaluation of VOI analysis use within decision making on tomosynthesis versus digital mammography for use in the Dutch breast cancer population screening. Based on steering committee meeting attendance and regular meetings with analysts, we developed a list of barriers to VOI use, which were analyzed using an established diffusion model. We proposed actions to address these barriers. Barriers and actions were discussed and validated in a workshop with stakeholders representing patients, clinicians, regulators, policy advisors, researchers, and the industry.

RESULTS

Consensus was reached on groups of barriers, which included characteristics of VOI analysis itself, stakeholder's attitudes, analysts' and policy makers' skills and knowledge, system readiness, and implementation in the organization. Observed barriers did not only pertain to VOI analysis itself but also to formulating the objective of the assessment, economic modeling, and broader aspects of uncertainty assessment. Actions to overcome these barriers related to organizational changes, knowledge transfer, cultural change, and tools.

CONCLUSIONS

This in-depth analysis of barriers to implementation of VOI analysis and resulting actions and tools may be useful to health technology assessment organizations that wish to implement VOI analysis in technology assessment and research prioritization. Further research should focus on application and evaluation of the proposed actions in real-world assessment processes.

Broadening the Concept of Value: A Scoping Review on the Option Value of Medical Technologies

OBJECTIVES

A recent debate in health economics and outcomes research community identified option value as one of the elements warranting consideration in the assessment of medical technologies. To conduct a scoping review of contributions on option value in the healthcare sector and identify relevant conceptual aspects and methods used to incorporate it in standard economic evaluations.

METHODS

A systematic search was conducted up to July 2020 to identify contributions from electronic bibliographic database and gray literature. Data on the proposed definitions of option value, theoretical implications of its use in economic evaluations, and methods used to estimate it were extracted and analyzed.

RESULTS

We found 57 eligible studies. Three different definitions emerged: insurance value, real option value, and option value of survival. Focusing on the latter (24 studies), we analyzed in depth 8 empirical applications across 7 therapeutic areas. The most relevant methodological challenges were on the perspective used in economic evaluations and how to robustly manage forecasting uncertainty, update cost-effectiveness thresholds, and avoid double-counting issues. For empirical studies assessing the total value of the technology, including option value, estimates ranged from +7% to +469% of its conventional value.

CONCLUSIONS

This review synthesizes theoretical and empirical aspects on option value of healthcare technologies and proposes a terminology to distinguish 3 different concepts identified. Future work should focus primarily on agreeing on whether option value should be included in economic evaluations and, if so, on developing and validating reliable methods for its ex-ante estimation.

Building a trusted framework for uncertainty assessment in rare diseases: suggestions for improvement (Response to "TRUST4RD: tool for reducing uncertainties in the evidence generation for specialised treatments for rare diseases")

The aim of this letter to the editor is to provide a comprehensive summary of uncertainty assessment in Health Technology Assessment, with a focus on transferability to the setting of rare diseases. The authors of "TRUST4RD: tool for reducing uncertainties in the evidence generation for specialised treatments for rare diseases" presented recommendations for reducing uncertainty in rare diseases. Their article is of great importance but unfortunately suffers from a lack of references to the wider uncertainty in Health Technology Assessment and research prioritisation literature and consequently fails to provide a trusted framework for decision-making in rare diseases. In this letter to the editor we critique the authors' tool and provide pointers as to how their proposal can be strengthened. We present references to the literature, including our own tool for uncertainty assessment (TRUST; unrelated to the authors' research), apply TRUST to two assessments of orphan drugs in rare diseases and provide a broader perspective on uncertainty and risk management in rare diseases, including a detailed research agenda.

Long-term cost-effectiveness of a disease management program for patients with atrial fibrillation compared to standard care - a multi-state survival model based on a randomized controlled trial

AIM

To assess the long-term cost-effectiveness of an atrial fibrillation disease management program (i.e. the SAFETY program) from the Australian healthcare system perspective.

METHODS

A multistate Markov model was developed based on patient-level data from the SAFETY randomized controlled trial. Predicted long-term survival, dependent on hospital admission history, was estimated by extrapolating parametric survival models. Quality-adjusted life years (QALY) and life years (LY) were the primary and secondary outcome measures used to estimate the incremental cost-utility/effectiveness ratio (ICUR/ICER). Both deterministic and probabilistic sensitivity analyses (PSA) were undertaken.

RESULTS

The SAFETY program was associated with both higher costs ($94,953 vs. $78,433) and benefits [QALY (3.99 vs 3.60); LY (5.86 vs 5.24)], with an ICUR of $42,513/QALY or ICER of $26,356/LY, compared to standard care. Due to the extended survival, the SAFETY was associated with a greater number of hospitalizations (14.85 vs 11.65) and higher costs for medications ($25,084 vs $22,402) and outpatient care ($12,904 vs $11,524). The cost per hospitalization for an average length of stay, analytical time horizon, and cost of medication are key determinants of ICUR. The PSA showed that the intervention has a 70.4% probability of being cost-effective at a threshold of $50,000/QALY.

CONCLUSIONS

The SAFETY program has a high probability of being cost-effective for patients with atrial fibrillation. It is associated with uncertainty that further research could potentially eliminate; implementation with further evidence collection is recommended.

Value of Information Analysis for Research Decisions-An Introduction: Report 1 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force

Healthcare resource allocation decisions made under conditions of uncertainty may turn out to be suboptimal. In a resource constrained system in which there is a fixed budget, these suboptimal decisions will result in health loss. Consequently, there may be value in reducing uncertainty, through the collection of new evidence, to make better resource allocation decisions. This value can be quantified using a value of information (VOI) analysis. This report, from the ISPOR VOI Task Force, introduces VOI analysis, defines key concepts and terminology, and outlines the role of VOI for supporting decision making, including the steps involved in undertaking and interpreting VOI analyses. The report is specifically aimed at those tasked with making decisions about the adoption of healthcare or the funding of healthcare research. The report provides a number of recommendations for good practice when planning, undertaking, or reviewing the results of VOI analyses.

Is There Something Else Beyond Cost-Effectiveness Analysis for Public Health Decision Making?

Healthcare costs are a concern for the sustainability of health systems in both rich and poor countries. Achieving a balance between the aspirations of payers and the manufacturers of new technologies is a challenge for democratic societies. Evidence about the efficacy and effectiveness of a new intervention is a fundamental aspect for its inclusion, but additional information about organization, implementation, and feasibility is required. Economic evaluations, especially cost-effectiveness analyses (CEA), help inform the choice of a particular health intervention, but they are not the only input for decision making (DM). Use of CEA is relatively recent but has quickly become widespread. CEA techniques have evolved into increasingly complex and sophisticated methods intended to reflect reality closely but, at the same time, their results have become more difficult to verify and validate. In developed countries, CEA results have generated intense debates, but in developing countries, these reflections are still weak due to lack of technical capacity. Competing perspectives on CEAs exist and can heavily influence the DM process. The use of CEAs and the interpretation of their results requires critical analysis, especially when public funds are to be invested. Here, we present a perspective on the use of CEAs for DM that arises from our experience of its use in developing countries and requires the consideration of other rationalities, in addition to the economic one, for DM.

Pembrolizumab for Treating Relapsed or Refractory Classical Hodgkin Lymphoma: An Evidence Review Group Perspective of a NICE Single Technology Appraisal

As part of its Single Technology Appraisal (STA) process, the National Institute for Health and Care Excellence (NICE) invited the manufacturer (Merck Sharp & Dohme; MSD) of pembrolizumab (Keytruda®) to submit evidence of its clinical and cost effectiveness for the treatment of patients with relapsed or refractory classical Hodgkin lymphoma (RRcHL) who did not respond to treatment with brentuximab vedotin. Kleijnen Systematic Reviews Ltd, in collaboration with Maastricht University Medical Centre+, was commissioned to act as the independent Evidence Review Group (ERG). The ERG produced a detailed review of the evidence for the clinical and cost effectiveness of the technology, based on the company's submission to NICE. According to the NICE scope, pembrolizumab was compared with single or combination chemotherapy. Comparisons were undertaken in two populations: patients who did and did not receive prior autologous stem cell transplant (autoSCT; populations 1 and 2, respectively). Despite it having been recommended by NICE in population 1 at the time the ERG received the company submission, nivolumab was not included as a comparator. No studies directly comparing pembrolizumab and its comparators were identified. One ongoing, single-arm study of the efficacy and safety of pembrolizumab (KEYNOTE-087) and one comparative observational study (Cheah et al., 2016) were used to inform the comparative effectiveness of pembrolizumab and standard of care (SoC), using indirect comparisons in both populations. Almost all analyses showed significant PFS and overall response rate benefits for pembrolizumab versus SoC, but due to being based on indirect comparison, were likely to contain systematic error. The economic evaluation therefore suffered from substantial uncertainty in any estimates of cost effectiveness. Furthermore, there was a lack of evidence on the uptake and timing of allogeneic stem cell transplant, and alternative assumptions had a significant impact on cost effectiveness. Immature survival data from KEYNOTE-087 exacerbated this issue and necessitated the use of alternative data sources for longer-term extrapolation of survival. Some issues identified in the company's analyses were amended by the ERG. The revised ERG deterministic base-case incremental cost-effectiveness ratios based on the company's second Appraisal Consultation Document response for pembrolizumab versus SoC (with a commercial access agreement) for populations 1 and 2 were £54,325 and £62,527 per quality-adjusted life-year gained, respectively. There was substantial uncertainty around these ICERs, especially in population 2. NICE did not recommend pembrolizumab as an option for treating RRcHL in population 1, but recommended pembrolizumab for use within the Cancer Drugs Fund in population 2.

Uncertainty and Coverage With Evidence Development: Does Practice Meet Theory?

OBJECTIVES

In theory, a successful coverage with evidence development (CED) scheme is one that addresses the most important uncertainties in a given assessment. We investigated the following: (1) which uncertainties were present during the initial assessment of 3 Dutch CED cases, (2) how these uncertainties were integrated in the initial assessments, (3) whether CED research plans included the identified uncertainties, and (4) issues with managing uncertainty in CED research and ways forward from these issues.

METHODS

Three CED initial assessment dossiers were analyzed and 16 stakeholders were interviewed. Uncertainties were identified in interviews and dossiers and were categorized in different causes: unavailability, indirectness, and imprecision of evidence. Identified uncertainties could be mentioned, described, and explored. Issues and ways forward to address uncertainty in CED schemes were discussed during the interviews.

RESULTS

Forty-two uncertainties were identified. Thirteen (31%) were caused by unavailability, 17 (40%) by indirectness, and 12 (29%) by imprecision. Thirty-four uncertainties (81%) were only mentioned, 19 (45%) were described, and the impact of 3 (7%) uncertainties on the results was explored in the assessment dossiers. Seventeen uncertainties (40%) were included in the CED research plans. According to stakeholders, research did not address the identified uncertainty, but CED research should be designed to focus on these.

CONCLUSIONS

In practice, uncertainties were neither systematically nor completely identified in the analyzed CED schemes. A framework would help to systematically identify uncertainty, and this process should involve all stakeholders. Value of information analysis, and the uncertainties that are not included in this analysis should inform CED research design.

Health technology assessment in Australia: a role for clinical registries?

Objective Health technology assessment (HTA) is a process of assessing evidence to inform policy decisions about public subsidy of new drugs and medical procedures. Where evidence is uncertain but the technology itself is promising, funders may recommend funding on an interim basis. It is unknown whether evidence from clinical registries is used to resolve uncertainties identified in interim-funded decisions made by Australian HTA bodies. Therefore, the present study evaluated the role of evidence from clinical registries in resolving evidence uncertainties identified by the Medical Services Advisory Committee (MSAC). Methods All HTAs considered by MSAC between 1998 and 2015 were reviewed and assessments that recommended interim funding were identified. The MSAC website was searched to identify reassessments of these recommendations and sources of evidence used to resolve the uncertainties were identified. Results Of 173 HTA reports considered by MSAC, 17 (10%) contained an interim funding recommendation. Eight recommendations cited uncertainty around safety, 15 cited uncertainty around clinical effectiveness and 13 cited uncertainty around economics (cost-effectiveness and/or budget impact). Of the 17 interim funding recommendations, 11 (65%) have been reassessed. Only two reassessments relied on clinical registry evidence to resolve evidence gaps identified at the time of the interim funding recommendation. Conclusions Clinical registries are underused as a source of evidence for resolving uncertainties around promising new health technologies in Australia. An open dialogue between stakeholders on the role of registries in this context is needed. What is known about the topic? HTA is a process of assessing the evidence to inform policy decisions about public subsidy of new health technologies (e.g. pharmaceuticals, diagnostic tests, medical procedures). Where evidence is uncertain but the technology under evaluation is promising, funders may recommend the funding of the technology on a temporary basis while additional evidence is collected. Clinical registries have been suggested as a means of collecting additional evidence in these situations. What is does this paper add? It is currently unknown whether evidence from clinical registries is used to resolve uncertainties identified at the time that temporary (interim) funding decisions are made by Australia’s HTA bodies, in particular MSAC. The present study found that MSAC rarely relies on the interim funding mechanism (17/173 assessments). Of the 11 subsequent reassessments of interim recommendations, two relied on registry evidence to provide Australian-specific data for addressing uncertainties around long-term safety, effectiveness and cost-effectiveness. These findings suggest that clinical registries, although a feasible source of evidence for HTAs, are rarely used for this purpose. What are the implications for practitioners? Given the registries’ ability to resolve both a wider range of questions than those typically addressed by randomised control trials and applicability to a wider group of patients (and, hence, providing estimates of outcomes that are more generalisable), the potential of clinical registries to resolve HTA issues needs more attention from both researchers and decision makers. Stakeholder collaboration to define the evidence requirements for new technologies early in their development phase would be valuable to determine the potential role for clinical registries.

A Comprehensive Algorithm for Approval of Health Technologies With, Without, or Only in Research: The Key Principles for Informing Coverage Decisions

BACKGROUND

The value of evidence about the performance of a technology and the value of access to a technology are central to policy decisions regarding coverage with, without, or only in research and managed entry (or risk-sharing) agreements.

OBJECTIVES

We aim to outline the key principles of what assessments are needed to inform "only in research" (OIR) or "approval with research" (AWR) recommendations, in addition to approval or rejection.

METHODS

We developed a comprehensive algorithm to inform the sequence of assessments and judgments that lead to different types of guidance: OIR, AWR, Approve, or Reject. This algorithm identifies the order in which assessments might be made, how similar guidance might be arrived at through different combinations of considerations, and when guidance might change.

RESULTS

The key principles are whether the technology is expected to be cost-effective; whether the technology has significant irrecoverable costs; whether additional research is needed; whether research is possible with approval and whether there are opportunity costs that once committed by approval cannot be recovered; and whether there are effective price reductions. Determining expected cost-effectiveness is only a first step. In addition to AWR for technologies expected to be cost-effective and OIR for those not expected to be cost-effective, there are other important circumstances when OIR should be considered.

CONCLUSIONS

These principles demonstrate that cost-effectiveness is a necessary but not sufficient condition for approval. Even when research is possible with approval, OIR may be appropriate when a technology is expected to be cost-effective due to significant irrecoverable costs.

Value of Information Analysis Informing Adoption and Research Decisions in a Portfolio of Health Care Interventions

Background: Value of information (VOI) analysis quantifies the value of additional research in reducing decision uncertainty. It addresses adoption and research decisions simultaneously by comparing the expected benefits and costs of research studies. Nevertheless, the application of this approach in practice remains limited. Objectives: To apply VOI analysis in health care interventions to guide adoption decisions, optimize trial design, and prioritize research. Methods: The analysis was from the perspective of Queensland Health, Australia. It included four interventions: clinically indicated catheter replacement, tissue adhesive for securing catheters, negative pressure wound therapy (NPWT) in caesarean sections, and nutritional support for preventing pressure ulcers. For each intervention, cost-effectiveness analysis was performed, decision uncertainty characterized, and VOI calculated using Monte Carlo simulations. The benefits and costs of additional research were considered together with the costs and consequences of acting now versus waiting for more information. All values are reported in 2014 Australian dollars (AU$). Results: All interventions were cost-effective, but with various levels of decision uncertainty. The current evidence is sufficient to support the adoption of clinically indicated catheter replacement. For the tissue adhesive, an additional study before adoption is worthwhile with a four-arm trial of 220 patients per arm. Additional research on NPWT before adoption is worthwhile with a two-arm trial of 200 patients per arm. Nutritional support should be adopted with a two-arm trial of 1200 patients per arm. Based on the expected net monetary benefits, the studies were ranked as follows: 1) NPWT (AU$1.2 million), 2) tissue adhesive (AU$0.3 milliion), and 3) nutritional support (AU$0.1 million). Conclusions: VOI analysis is a useful and practical approach to inform adoption and research decisions. Efforts should be focused on facilitating its integration into decision making frameworks.

Financing prevention: opportunities for economic analysis across the translational research cycle

Prevention advocates often make the case that preventive intervention not only improves public health and welfare but also can save public resources. Increasingly, evidence-based policy efforts considering prevention are focusing on how programs can save taxpayer resources from reduced burden on health, criminal justice, and social service systems. Evidence of prevention's return has begun to draw substantial investments from the public and private sector. Yet, translating prevention effectiveness into economic impact requires specific economic analyses to be employed across the stages of translational research. This work discusses the role of economic analysis in prevention science and presents key translational research opportunities to meet growing demand for estimates of prevention's economic and fiscal impact.

Evaluation of patient registries supporting reimbursement decisions: the case of oxaliplatin for treatment of stage III colon cancer

BACKGROUND

Access with evidence development has been established for expensive intramural drugs in The Netherlands. The procedure involves a 4-year period of conditional reimbursement. During this period, additional evidence has to be gathered usually through a patient registry. Given the costs and time involved in gathering the data, it is important to carefully evaluate the registry.

OBJECTIVES

This study aimed to develop a model for the regular evaluation of patient registries during an access with evidence development process and find the optimal length of the registry period.

METHODS

We used data from a recent registry in The Netherlands on oxaliplatin as a treatment option for stage III colon cancer. We added simulated follow-up data to the empirical data available and applied value of information analysis to balance the gains of extending the period and amount of data gathering against the costs of registering patients.

RESULTS

We show that given the assumptions on cohort size, follow-up time, and purpose of the registry, the current (partly simulated) registry was not very efficient. Notably, the observation period could have been stopped to make a definite reimbursement decision after a maximum of 2 years rather than the fixed 4-year period.

CONCLUSIONS

Patient registries may be an efficient way to gather data on new medical treatments, but they need to be carefully designed and evaluated, in particular regarding their follow-up time. For each purpose, data gathering can be tailored to make sure decisions are taken at the moment that sufficient data are available.

Examining the role of carbon capture and storage through an ethical lens

The risk posed by anthropogenic climate change is generally accepted, and the challenge we face to reduce greenhouse gas (GHG) emissions to a tolerable limit cannot be underestimated. Reducing GHG emissions can be achieved either by producing less GHG to begin with or by emitting less GHG into the atmosphere. One carbon mitigation technology with large potential for capturing carbon dioxide at the point source of emissions is carbon capture and storage (CCS). However, the merits of CCS have been questioned, both on practical and ethical grounds. While the practical concerns have already received substantial attention, the ethical concerns still demand further consideration. This article aims to respond to this deficit by reviewing the critical ethical challenges raised by CCS as a possible tool in a climate mitigation strategy and argues that the urgency stemming from climate change underpins many of the concerns raised by CCS.

Optimal decision rules for HTA under uncertainty: a wider, dynamic perspective

We present a two-period framework, which combines real option and decision-theoretic approaches to health technology assessment under uncertainty. By viewing adoption, treatment and research decisions as a single economic project, we illustrate how their key dimensions affect optimal rules. We consider the results in relation to the existing literature and argue that developments in this direction could contribute substantially to efficiency gains in resource allocation.

Refining drug safety in rheumatology

This issue explores several important safety concerns that currently plague the rheumatologist and health care providers who care for patients with rheumatic diseases. Weighing safety against efficacy can be a complex task that is best alleviated by understanding the issues, nature, and breadth of problems associated with drug use. Therapeutic decision-making must be evidence-based, judicious, and appropriate for the patient and situation. Understanding drug safety is paramount to ensuring both success of therapy and benefit to the patient. Similarly, it is important not to underestimate the impact of uncontrolled disease activity in decision-making. Drug safety must be weighed against the severity and risks of the disease under treatment. Clearly the benefit/risk ratio has improved for many of the therapies discussed in this book. The use of both conventional and novel therapies mandates an understanding of the mechanisms of action, unique toxicities, screening and monitoring measures, and rules for drug avoidance.

Politics and its intersection with coverage with evidence development: a qualitative analysis from expert interviews

BACKGROUND

Pressures on health care budgets have led policy makers to discuss how to balance the provision of costly technologies to populations in need and making coverage decisions under uncertainty. Coverage with evidence development (CED) is being employed to meet these challenges.

METHODS

Twenty-four interviews were carried out between June 2009 and December 2010 with researchers, decision makers and policy makers from Australia, Canada, United Kingdom and United States. Three phases of coding occurred, the first being manual coding where the interviews were read and notes were taken and nodes were extracted and imputed. NVIVO coding was applied to the interview transcripts, with both broad general searches for word usages and imputed nodes.

RESULTS

Four overarching thematic areas emerged out of contextual analysis of the interviews - (1) what constitutes CED; (2) the lack of a systematic approach/governance structure; (3) the role of the pharmaceutical industry and overt political considerations in CED; and (4) alternatives and barriers to CED. We explore these themes and then use concrete examples of CED projects in each of the four countries to illustrate the political issues that our interviewees raised.

CONCLUSION

Until the underlying political nature of CED is recognized then fundamental questions about its usefulness and operation will remain unresolved.

When does NICE recommend the use of health technologies within a programme of evidence development? : a systematic review of NICE guidance

BACKGROUND

There is growing interest internationally in linking reimbursement decisions with recommendations for further research. In the UK, the National Institute for Health and Clinical Excellence (NICE) can issue guidance to approve the routine use of a health intervention, reject routine use or recommend use within a research programme. These latter recommendations have restricted use to 'only in research' (OIR) or have recommended further research alongside routine use ('approval with research' or AWR). However, it is not currently clear when such recommendations are likely to be made.

OBJECTIVES

This study aims to identify NICE technology appraisals where OIR or AWR recommendations were made and to examine the key considerations that led to those decisions.

METHODS

Draft and final guidance including OIR/AWR recommendations were identified. The documents were reviewed to establish the characteristics of the technology appraisal, the cost effectiveness of the technologies, the key considerations that led to the recommendations and the types of research required.

RESULTS

In total, 29 final and 31 draft guidance documents included OIR/AWR recommendations up to January 2010. Overall, 86 % of final guidance included OIR recommendations. Of these, the majority were for technologies considered to be cost ineffective (83 %) and the majority of final guidance (66 %) specified the need for further evidence on relative effectiveness. The use of OIR/AWR recommendations is decreasing over time and they have rarely been used in appraisals conducted through the single technology appraisal process.

CONCLUSION

NICE has used its ability to recommend technologies within research programmes, although predominantly within the multiple technology appraisal process. OIR recommendations have been most frequently issued for technologies considered cost ineffective and the most frequently cited consideration is uncertainty related to relative effectiveness. Key considerations cited for most AWR recommendations and some OIR recommendations included a need for further evidence on long-term outcomes and adverse effects of treatment.

Comparative Effectiveness Research: The Experience of the National Institute for Health and Clinical Excellence

Purpose

To assess the relevance of the experience of the National Institute for Health and Clinical Excellence (NICE) in the United Kingdom to the comparative effectiveness research (CER) initiative in the United States.

Methods

The activities of NICE were reviewed to assess its experience in analytic methods, engagement with stakeholders, communication of findings, and implementation of recommendations.

Results

The main lessons for the United States from the experience of NICE relate to how the institute has gathered, synthesized, and used information on the clinical and cost effectiveness of health care interventions. The experience of NICE suggests that ways will have to be found to reconcile the differing stakeholder perspectives on the value of health care. Given the emphasis in the United States on being patient centered, there will be situations where patients' expectations for the provision of care far exceed that which payers feel should be made available on grounds of value for money. Explicit restrictions on access to care based on CER like those found in the United Kingdom are unlikely, but alternative solutions, such as value-based reimbursement, will need to be pursued if unnecessary expenditures are to be avoided. It will also be important that the CER initiative show some impact on the use of health care resources. The longer that NICE has been in existence in the United Kingdom, questions about its impact have been more frequently asked, given the resources devoted to its activities.

Conclusion

Although there are distinct differences between the health systems of the United Kingdom and United States, lessons can be learned from examining the successes and challenges experienced by NICE.

Economics of cancer biomarkers

Cancer accounts for approximately 13% of all deaths worldwide, and in 2010 the estimated total cost of cancer in the USA was more than US$263 billion. Biomarker use for screening, monitoring, diagnosis and treatment optimization has the potential to improve patient outcomes and reduce costs associated with inappropriate (or suboptimal) therapeutic regimens. Since a new technology may have additional initial cost, a policy question arises regarding whether the improvement in outcomes is attained at a ‘reasonable’ additional cost compared with existing technology. This paper presents an overview of health economic issues surrounding biomarkers in general, with a focus on cancer care and treatment optimization in particular. While this article is not a systematic review of the literature, it includes relevant examples to provide a real-world perspective.

Value-based purchasing of medical devices

BACKGROUND

Health care in the United States is known for its continued innovation and production of new devices and techniques. While the intention of these devices is to improve the delivery and outcome of patient care, they do not always achieve this goal. As new technologies enter the market, hospitals and physicians must determine which of these new devices to incorporate into practice, and it is important these devices bring value to patient care. We provide a model of a physician-engaged process to decrease cost and increase review of physician preference items.

QUESTIONS/PURPOSES

We describe the challenges, implementation, and outcomes of cost reduction and product stabilization of a value-based process for purchasing medical devices at a major academic medical center.

METHODS

We implemented a physician-driven committee that standardized and utilized evidence-based, clinically sound, and financially responsible methods for introducing or consolidating new supplies, devices, and technology for patient care. This committee worked with institutional finance and administrative leaders to accomplish its goals.

RESULTS

Utilizing this physician-driven committee, we provided access to new products, standardized some products, decreased costs of physician preference items 11% to 26% across service lines, and achieved savings of greater than $8 million per year.

CONCLUSIONS

The implementation of a facility-based technology assessment committee that critically evaluates new technology can decrease hospital costs on implants and standardize some product lines.

Cost-effectiveness of involving nurse specialists for adult patients with urinary incontinence in primary care compared to care-as-usual: an economic evaluation alongside a pragmatic randomized controlled trial

AIMS

To determine the 12-month, societal cost-effectiveness of involving urinary incontinence (UI) nurse specialists in primary care compared to care-as-usual by general practitioners (GPs).

METHODS

From 2005 until 2008 an economic evaluation was performed alongside a pragmatic multicenter randomized controlled trial comparing UI patients receiving care by nurse specialists with patients receiving care-as-usual by GPs in the Netherlands. One hundred eighty-six adult patients with stress, urgency, or mixed UI were randomly allocated to the intervention and 198 to care-as-usual; they were followed for 1 year. Main outcome measures were Quality Adjusted Life Year (QALY(societal) ) based on societal preferences for health outcomes (EuroQol-5D), QALY(patient) based on patient preferences for health outcomes (EuroQol VAS), and Incontinence Severity weighted Life Year (ISLY) based on patient-reported severity and impact of UI (ICIQ-UI SF). Health care resource use, patient and family costs, and productivity costs were assessed. Data were collected by three monthly questionnaires. Incremental cost-effectiveness ratios were calculated. Uncertainty was assessed using bootstrap simulation, and the expected value of perfect information was calculated (EVPI).

RESULTS

Compared to care-as-usual, nurse specialist involvement costs € 16,742/QALY(societal) gained. Both QALY(patient) and ISLY yield slightly more favorable cost-effectiveness results. At a threshold of € 40,000/QALY(societal,) the probability that the intervention is cost-effective is 58%. The EVPI amounts to € 78 million.

CONCLUSIONS

Based on these results, we recommend adopting the nurse specialist intervention in primary care, while conducting more research through careful monitoring of the effectiveness and costs of the intervention in routine practice.

Reflections on the evolution of health technology assessment in Europe

Health technology assessment (HTA) has assumed an increasing role in health systems in recent years, with many countries establishing agencies or programmes to evaluate health technology and other interventions to inform policy decisions and clinical practice. This paper reflects upon its development and evolution in Europe over the last decade, with a focus on England, France, Germany and Sweden. In particular, we explore how HTA has evolved over time as well as its impact on policy and practice. While countries share many of the same objectives, there are differences in the way HTA agencies and programmes are organised, operate, and influence decision making. Despite these differences, all systems are faced with opportunities and challenges related to stakeholder involvement and acceptance, the suitability and transparency of assessment requirements and methods, balancing evidence and values in decision making, and demonstrating impact.

The role of the expected value of individualized care in cost-effectiveness analyses and decision making

OBJECTIVE

To explore the feasibility and potential role of the expected value of individualized care (EVIC) framework.

METHODS

The EVIC quantifies how much benefits are forgone when a treatment decision is based on the best-expected outcomes in the population rather than in the individual patient. We have reviewed which types of patient-level attributes contribute to the EVIC and how they affect the interpretation of the outcomes. In addition, we have applied the EVIC framework to the outcomes of a microsimulation-based cost-effectiveness analysis for glaucoma treatment.

RESULTS

For EVIC outcomes to inform decisions about clinical practice, we need to calculate the parameter-specific EVIC of known or knowable patient-level attributes and compare it with the real costs of implementing individualized care. In the case study, the total EVIC was €580 per patient, but patient-level attributes known at treatment decision had minimal impact. A subgroup policy based on individual disease progression could be worthwhile if a predictive test for glaucoma progression could be developed and implemented for less than €130 per patient.

CONCLUSIONS

The EVIC framework is feasible in cost-effectiveness analyses and can be informative for decision making. The EVIC outcomes are particularly informative when they are (close to) zero. When the EVIC has a high value, implications depend on the type of patient-level attribute. EVIC can be a useful tool to identify opportunities to improve efficiency in health care by individualization of care and to quantify the maximal investment opportunities for implementing subgroup policy.

When to wait for more evidence? Real options analysis in proton therapy

PURPOSE

Trends suggest that cancer spending growth will accelerate. One method for controlling costs is to examine whether the benefits of new technologies are worth the extra costs. However, especially new and emerging technologies are often more costly, while limited clinical evidence of superiority is available. In that situation it is often unclear whether to adopt the new technology now, with the risk of investing in a suboptimal therapy, or to wait for more evidence, with the risk of withholding patients their optimal treatment. This trade-off is especially difficult when it is costly to reverse the decision to adopt a technology, as is the case for proton therapy. Real options analysis, a technique originating from financial economics, assists in making this trade-off.

METHODS

We examined whether to adopt proton therapy, as compared to stereotactic body radiotherapy, in the treatment of inoperable stage I non-small cell lung cancer. Three options are available: adopt without further research; adopt and undertake a trial; or delay adoption and undertake a trial. The decision depends on the expected net gain of each option, calculated by subtracting its total costs from its expected benefits.

RESULTS

In The Netherlands, adopt and trial was found to be the preferred option, with an optimal sample size of 200 patients. Increase of treatment costs abroad and costs of reversal altered the preferred option.

CONCLUSION

We have shown that real options analysis provides a transparent method of weighing the costs and benefits of adopting and/or further researching new and expensive technologies.

Breaking up is hard to do: the economic impact of provisional funding contingent upon evidence development

Funding contingent upon evidence development (FED) has recently been the subject of some considerable debate in the literature but relatively little has been made of its economic impact. We argue that FED has the potential to shorten the lag between innovation and access but may also (i) crowd-out more valuable interventions in situations in which there is a fixed dedicated budget; or (ii) lead to a de facto increase in the funding threshold and increased expenditure growth in situations in which the programme budget is open-ended. Although FED would typically entail periodic review of provisional or interim listings, it may prove difficult to withdraw funding even at cost/QALY ratios well in excess of current listing thresholds. Further consideration of the design and implementation of FED processes is therefore required to ensure that its introduction yields net benefits over existing processes.

A method for addressing research gaps in HTA, developed whilst evaluating robotic-assisted surgery: a proposal

Background

When evaluating health technologies with insufficient scientific evidence, only innovative potentials can be assessed. A Regional policy initiative linking the governance of health innovations to the development of clinical research has been launched by the Region of Emilia Romagna Healthcare Authority. This program, aimed at enhancing the research capacity of health organizations, encourages the development of adoption plans that combine use in clinical practice along with experimental use producing better knowledge. Following the launch of this program we developed and propose a method that, by evaluating and ranking scientific uncertainty, identifies the moment (during the stages of the technology's development) where it would be sensible to invest in research resources and capacity to further its evaluation. The method was developed and tested during a research project evaluating robotic surgery.

Methods

A multidisciplinary panel carried out a 5-step evaluation process: 1) definition of the technology's evidence profile and of all relevant clinical outcomes; 2) systematic review of scientific literature and outline of the uncertainty profile differentiating research results into steady, plausible, uncertain and unknown results; 3) definition of the acceptable level of uncertainty for investing research resources; 4) analysis of local context; 5) identification of clinical indications with promising clinical return.

Results

Outputs for each step of the evaluation process are: 1) evidence profile of the technology and systematic review; 2) uncertainty profile for each clinical indication; 3) exclusion of clinical indications not fulfilling the criteria of maximum acceptable risk; 4) mapping of local context; 5) recommendations for research.

Outputs of the evaluation process for robotic surgery are described in the paper.

Conclusions

This method attempts to rank levels of uncertainty in order to distinguish promising from hazardous clinical use and to outline a research course of action. Decision makers wishing to tie coverage policies to the development of scientific evidence could find this method a useful aid to the governance of innovations.

Coverage policy development for personalized medicine: private payer perspectives on developing policy for the 21-gene assay

PURPOSE

Personalized medicine is changing oncology practice and challenging decision making. A key challenge is the limited clinical evidence for many personalized medicine technologies. We describe the strategies private payers employed to develop coverage policy for personalized medicine using the example of the 21-gene assay in breast cancer.

METHODS

We examined the coverage policies of six private payers for the 21-gene assay. We then interviewed senior executives (n = 7) from these payers to elucidate factors informing coverage decisions. We additionally focused on the timing of payer decisions compared with the timing of evidence development, measured by publication of primary studies and relevant clinical guidelines.

RESULTS

The 21-gene assay became commercially available in 2004. The interviewed payers granted coverage between 2005 and 2008. Their policies varied in structure (eg, whether prior authorization was required). All payers reported clinical evidence as the most important factor in decision making, but all used some health care system factors (eg, physician adoption or medical society endorsement) to inform decision making as well. Payers had different perceptions about the strength of clinical evidence at the time of the coverage decision.

CONCLUSION

Coverage of the 21-gene assay is currently widespread, but policies differ in timing and structure. A key approach private payers use to develop coverage policies for novel technologies is considering both clinical evidence and health care system factors. Policy variation may emerge from the range of factors used and perception of the evidence. Future research should examine the role of health care system factors in policy development and related policy variations.

Funding the unfundable: mechanisms for managing uncertainty in decisions on the introduction of new and innovative technologies into healthcare systems

As tensions between payers, responsible for ensuring prudent and principled use of scarce resources, and both providers and patients, who legitimately want access to technologies from which they could benefit, continue to mount, interest in approaches to managing the uncertainty surrounding the introduction of new health technologies has heightened. The purpose of this project was to compile an inventory of various types of 'access with evidence development' (AED) schemes, examining characteristics of the technologies to which they have been applied, the uncertainty they sought to address, the terms of arrangements of each scheme, and the policy outcomes. It also aimed to identify issues related to such schemes, including advantages and disadvantages from the perspectives of various stakeholder groups. A comprehensive search, review and appraisal of peer-reviewed and 'grey' literature were performed, followed by a facilitated workshop of academics and decision makers with expertise in AED schemes. Information was extracted and compiled in tabular form to identify patterns or trends. To enhance the validity of interpretations made, member checking was performed. Although the concept of AED is not new, evaluative data are sparse. Despite varying opinions on the 'right' answers to some of the questions raised, there appears to be consensus on a 'way forward'--development of methodological guidelines. All stakeholders seemed to share the view that AEDs offer the potential to facilitate patient access to promising new technologies and encourage innovation while ensuring effective use of scarce healthcare resources. There is no agreement on what constitutes 'sufficient evidence', and it depends on the specific uncertainty in question. There is agreement on the need for 'best practice' guidelines around the implementation and evaluation of AED schemes. This is the first attempt at a comprehensive analysis of methods that have been used to address uncertainty concerning a new drug or other technology. The analysis reveals that, although various approaches have been experimented with, many of them have not achieved the ostensible goal of the approach. This article outlines challenges related to AED schemes and issues that remain unresolved.

Using comparative effectiveness research to inform policy and practice in the UK HHS: past, present and future

Health systems that have fixed budgets and a coherent organizational structure generally have found it valuable to have a dedicated primary research capacity to answer decision-oriented value-for-money questions of particular importance to the system. The UK NHS is one example of such a system. Here, we review the historical evolution of building comparative effectiveness research (CER) capacity in the NHS, describe the current situation, with a focus on how this research is used to inform decisions, and discuss present and emerging challenges. We draw some possible lessons for the US, which is currently considering using CER to inform healthcare policy and practice decisions.

Access with evidence development in the UK: past experience, current initiatives and future potential

Access with evidence development (AED) describes the general approach of linking some form of access to the healthcare market with the generation of additional evidence relating to the value of the healthcare intervention under evaluation, with an explicit aim of aiding future decision making. A number of health systems around the world are interested in the potential for such schemes. This article looks in detail at the potential for some form of AED in the UK, focusing on the two major decision-making bodies: the Scottish Medicines Consortium in Scotland and the National Institute for Health and Clinical Excellence in England and Wales. We consider past experience with these approaches and current initiatives that are exploring their potential, and speculate as to how these schemes might develop in the future.

Feasibility and challenges of independent research on drugs: the Italian medicines agency (AIFA) experience

Key points * National Health Service (NHS) is becoming increasingly aware of the need to support independent research to answer some important questions for patient care in areas of scant commercial interest. * This article reports the main features and strategies of the independent research programme on drugs launched by the Italian Medicines Agency (AIFA) in 2005. * In the three bids launched between 2005 and 2007, a total of 151 studies have been approved for funding for a total of about 78 million Euro. * In this article we describe the Italian legislative framework under which the programme was launched, the types of research funded and discuss how the supported studies could contribute, in an international framework, to the knowledge needed on drug efficacy, effectiveness and safety.

Access with evidence development schemes: a framework for description and evaluation

There is an inevitable tension between robust reimbursement processes and providing speedy access to new and novel technologies, given uncertainties about key pieces of evidence and subsequent concerns regarding their overall efficiency. The public perception of these treatments as 'breakthrough', combined with substantial clinical pressure, has led to healthcare payers looking for schemes that allow the new technology to be made available to (some) patients, while (at least partially) protecting the principles of their reimbursement decision-making processes. Current literature on these schemes is almost completely descriptive and provides little help in planning future schemes. We propose a framework for evaluating current schemes and informing the design of future schemes. We examine the value of the framework using the UK Multiple Sclerosis Risk-Sharing Scheme as a case study.

Expensive cancer drugs: a comparison between the United States and the United Kingdom

CONTEXT

This article compares the United Kingdom's and the United States' experiences with expensive cancer drugs to illustrate the challenges posed by new, extremely costly, medical technologies.

METHODS

This article describes British and American coverage, access, and cost-sharing policies with regard to expensive cancer drugs and then compares the costs of eleven such drugs to British patients, American Medicare beneficiaries, and American patients purchasing the drugs in the retail market. Three questions posed by these comparisons are then examined: First, which system is fairer? In which system are cancer patients better off? Assuming that no system can sustainably provide to everyone at least some expensive cancer drugs for some clinical indications, what challenges does each system face in making these difficult determinations?

FINDINGS

In both the British and American health care systems, not all patients who might benefit from or desire access to expensive cancer drugs have access to them. The popular characterization of the United States, where all cancer drugs are available for all to access as and when needed, and that of the British NHS, where top-down population rationing poses insurmountable obstacles to British patients' access, are far from the reality in both countries.

CONCLUSIONS

Key elements of the British system are fairer than the American system, and the British system is better structured to deal with difficult decisions about expensive end-of-life cancer drugs. Both systems face common ethical, financial, organizational, and priority-setting challenges in making these decisions.

NICE at 10 years: new challenges ahead

NICE has existed for 10 years. During this period, its methodological approach to developing guidance and assessing the value (cost-effectiveness) of healthcare interventions has received considerable national and international interest. At the same time, individual decisions have generated enormous controversy. This 10th anniversary provides an opportunity to look back at how the institute has responded and adapted to the various challenges and controversies. Following Lord Darzi's review of the National Health Service (UK) in 2008, the institute took on further responsibilities for setting clinical standards for the National Health Service. This article explores these developments and speculates on future trends.

Comparative effectiveness research and evidence-based health policy: experience from four countries

CONTEXT

The discussion about improving the efficiency, quality, and long-term sustainability of the U.S. health care system is increasingly focusing on the need to provide better evidence for decision making through comparative effectiveness research (CER). In recent years, several other countries have established agencies to evaluate health technologies and broader management strategies to inform health care policy decisions. This article reviews experiences from Britain, France, Australia, and Germany.

METHODS

This article draws on the experience of senior technical and administrative staff in setting up and running the CER entities studied. Besides reviewing the agencies' websites, legal framework documents, and informal interviews with key stakeholders, this analysis was informed by a workshop bringing together U.S. and international experts.

FINDINGS

This article builds a matrix of features identified from the international models studied that offer insights into near-term decisions about the location, design, and function of a U.S.-based CER entity. While each country has developed a CER capacity unique to its health system, elements such as the inclusiveness of relevant stakeholders, transparency in operation, independence of the central government and other interests, and adaptability to a changing environment are prerequisites for these entities' successful operation.

CONCLUSIONS

While the CER entities evolved separately and have different responsibilities, they have adopted a set of core structural, technical, and procedural principles, including mechanisms for engaging with stakeholders, governance and oversight arrangements, and explicit methodologies for analyzing evidence, to ensure a high-quality product that is relevant to their system.

The policy debate over public investment in comparative effectiveness research

BACKGROUND

Policy makers across the political spectrum, as well as many clinicians and physician professional associations, have proposed that better information on comparative clinical effectiveness should be a key element of any solution to the US health-care cost crisis. This superficial consensus hides intense disagreements over critical issues essential to any new public effort to promote more comparative effectiveness research (CER).

METHODS AND RESULTS

This article reviews the background for these disputes, summarizes the different perspectives represented by policy makers and advocates, and offers a framework to aid both practicing and academic internists in understanding the key elements of the emerging debate. Regarding the fundamental question of "what is CER," disagreements rage over whether value or cost effectiveness should be a consideration, and how specific patient perspectives should be reflected in the development and the use of such research. The question of how to pay for CER invokes controversies over the role of the market in producing such information and the private (e.g., insurers and employers) versus public responsibility for its production. The financing debate further highlights the high stakes of comparative effectiveness research, and the risks of stakeholder interests subverting any public process. Accordingly there are a range of proposals for the federal government's role in prioritization, development, and dissemination of CER.

CONCLUSION

The internal medicine community, with its long history of commitment to scientific medical practice and its leadership in evidence-based medicine, should have a strong interest and play an active role in this debate.