Making a decision to wait for more evidence: when the National Institute for Health and Clinical Excellence recommends a technology only in the context of research

Unifying Research and Reimbursement Decisions: Case Studies Demonstrating the Sequence of Assessment and Judgments Required

BACKGROUND

The key principles regarding what assessments lead to different types of guidance about the use of health technologies (Only in Research, Approval with Research, Approve, or Reject) provide an explicit and transparent framework for technology appraisal.

OBJECTIVE

We aim to demonstrate how these principles and assessments can be applied in practice through the use of a seven-point checklist of assessment.

METHODS

The value of access to a technology and the value of additional evidence are explored through the application of the checklist to the case studies of enhanced external counterpulsation for chronic stable angina and clopidogrel for the management of patients with non-ST-segment elevation acute coronary syndromes.

RESULTS

The case studies demonstrate the importance of considering 1) the expected cost-effectiveness and population net health effects; 2) the need for evidence and whether the type of research required can be conducted once a technology is approved for widespread use; 3) whether there are sources of uncertainty that cannot be resolved by research but only over time; and 4) whether there are significant (opportunity) costs that once committed by approval cannot be recovered.

CONCLUSIONS

The checklist demonstrates that cost-effectiveness is a necessary but not sufficient condition for approval. Only in Research may be appropriate when a technology is expected to be cost-effective due to significant irrecoverable costs. It is only approval that can be ruled out if a technology is not expected to be cost-effective. Lack of cost-effectiveness is not a necessary or sufficient condition for rejection.

When does NICE recommend the use of health technologies within a programme of evidence development? : a systematic review of NICE guidance

BACKGROUND

There is growing interest internationally in linking reimbursement decisions with recommendations for further research. In the UK, the National Institute for Health and Clinical Excellence (NICE) can issue guidance to approve the routine use of a health intervention, reject routine use or recommend use within a research programme. These latter recommendations have restricted use to 'only in research' (OIR) or have recommended further research alongside routine use ('approval with research' or AWR). However, it is not currently clear when such recommendations are likely to be made.

OBJECTIVES

This study aims to identify NICE technology appraisals where OIR or AWR recommendations were made and to examine the key considerations that led to those decisions.

METHODS

Draft and final guidance including OIR/AWR recommendations were identified. The documents were reviewed to establish the characteristics of the technology appraisal, the cost effectiveness of the technologies, the key considerations that led to the recommendations and the types of research required.

RESULTS

In total, 29 final and 31 draft guidance documents included OIR/AWR recommendations up to January 2010. Overall, 86 % of final guidance included OIR recommendations. Of these, the majority were for technologies considered to be cost ineffective (83 %) and the majority of final guidance (66 %) specified the need for further evidence on relative effectiveness. The use of OIR/AWR recommendations is decreasing over time and they have rarely been used in appraisals conducted through the single technology appraisal process.

CONCLUSION

NICE has used its ability to recommend technologies within research programmes, although predominantly within the multiple technology appraisal process. OIR recommendations have been most frequently issued for technologies considered cost ineffective and the most frequently cited consideration is uncertainty related to relative effectiveness. Key considerations cited for most AWR recommendations and some OIR recommendations included a need for further evidence on long-term outcomes and adverse effects of treatment.

Dangerous omissions: the consequences of ignoring decision uncertainty

Institutions with the responsibility for making adoption (reimbursement) decisions in health care often lack the remit to demand or commission further research: adoption decisions are their only policy instrument. The decision to adopt a technology also influences the prospects of acquiring further evidence because the incentives to conduct research are reduced and the ethical basis of further clinical trials maybe undermined. In these circumstances the decision maker must consider whether the benefits of immediate access to a technology exceeds the value of the evidence which maybe forgone for future patients. We outline how these expected opportunity losses can be established from the perspective of a societal decision maker with and without the remit to commission research, and demonstrate how these considerations change the appropriate decision rules in cost-effectiveness analysis. Importantly, we identify those circumstances in which the approval of a technology that is expected to be cost-effective should be withheld, i.e. when an 'only in research' recommendation should be made. We demonstrate that a sufficient condition for immediate adoption of a technology can provide incentives for manufacturers to reduce the price or provide additional supporting evidence. However, decisions based solely on expected net benefit provide no such incentives, may undermine the evidence base for future clinical practice and reduce expected net health benefits for the patient population.

Linking payment to health outcomes: a taxonomy and examination of performance-based reimbursement schemes between healthcare payers and manufacturers

OBJECTIVE

To identify, categorize and examine performance-based health outcomes reimbursement schemes for medical technology.

METHODS

We performed a review of performance-based health outcomes reimbursement schemes over the past 10 years (7/98-010/09) using publicly available databases, web and grey literature searches, and input from healthcare reimbursement experts. We developed a taxonomy of scheme types by inductively organizing the schemes identified according to the timing, execution, and health outcomes measured in the schemes.

RESULTS

Our search yielded 34 coverage with evidence development schemes, 10 conditional treatment continuation schemes, and 14 performance-linked reimbursement schemes. The majority of schemes are in Europe and Australia, with an increasing number in Canada and the U.S.

CONCLUSION

These schemes have the potential to alter the reimbursement and pricing landscape for medical technology, but significant challenges, including high transaction costs and insufficient information systems, may limit their long-term impact. Future studies regarding experiences and outcomes of implemented schemes are necessary.

Funding the unfundable: mechanisms for managing uncertainty in decisions on the introduction of new and innovative technologies into healthcare systems

As tensions between payers, responsible for ensuring prudent and principled use of scarce resources, and both providers and patients, who legitimately want access to technologies from which they could benefit, continue to mount, interest in approaches to managing the uncertainty surrounding the introduction of new health technologies has heightened. The purpose of this project was to compile an inventory of various types of 'access with evidence development' (AED) schemes, examining characteristics of the technologies to which they have been applied, the uncertainty they sought to address, the terms of arrangements of each scheme, and the policy outcomes. It also aimed to identify issues related to such schemes, including advantages and disadvantages from the perspectives of various stakeholder groups. A comprehensive search, review and appraisal of peer-reviewed and 'grey' literature were performed, followed by a facilitated workshop of academics and decision makers with expertise in AED schemes. Information was extracted and compiled in tabular form to identify patterns or trends. To enhance the validity of interpretations made, member checking was performed. Although the concept of AED is not new, evaluative data are sparse. Despite varying opinions on the 'right' answers to some of the questions raised, there appears to be consensus on a 'way forward'--development of methodological guidelines. All stakeholders seemed to share the view that AEDs offer the potential to facilitate patient access to promising new technologies and encourage innovation while ensuring effective use of scarce healthcare resources. There is no agreement on what constitutes 'sufficient evidence', and it depends on the specific uncertainty in question. There is agreement on the need for 'best practice' guidelines around the implementation and evaluation of AED schemes. This is the first attempt at a comprehensive analysis of methods that have been used to address uncertainty concerning a new drug or other technology. The analysis reveals that, although various approaches have been experimented with, many of them have not achieved the ostensible goal of the approach. This article outlines challenges related to AED schemes and issues that remain unresolved.

Using comparative effectiveness research to inform policy and practice in the UK HHS: past, present and future

Health systems that have fixed budgets and a coherent organizational structure generally have found it valuable to have a dedicated primary research capacity to answer decision-oriented value-for-money questions of particular importance to the system. The UK NHS is one example of such a system. Here, we review the historical evolution of building comparative effectiveness research (CER) capacity in the NHS, describe the current situation, with a focus on how this research is used to inform decisions, and discuss present and emerging challenges. We draw some possible lessons for the US, which is currently considering using CER to inform healthcare policy and practice decisions.

Access with evidence development in the UK: past experience, current initiatives and future potential

Access with evidence development (AED) describes the general approach of linking some form of access to the healthcare market with the generation of additional evidence relating to the value of the healthcare intervention under evaluation, with an explicit aim of aiding future decision making. A number of health systems around the world are interested in the potential for such schemes. This article looks in detail at the potential for some form of AED in the UK, focusing on the two major decision-making bodies: the Scottish Medicines Consortium in Scotland and the National Institute for Health and Clinical Excellence in England and Wales. We consider past experience with these approaches and current initiatives that are exploring their potential, and speculate as to how these schemes might develop in the future.

SUPPORT Tools for evidence-informed health Policymaking (STP) 17: Dealing with insufficient research evidence

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers.

In this article, we address the issue of decision making in situations in which there is insufficient evidence at hand. Policymakers often have insufficient evidence to know with certainty what the impacts of a health policy or programme option will be, but they must still make decisions. We suggest four questions that can be considered when there may be insufficient evidence to be confident about the impacts of implementing an option. These are: 1. Is there a systematic review of the impacts of the option? 2. Has inconclusive evidence been misinterpreted as evidence of no effect? 3. Is it possible to be confident about a decision despite a lack of evidence? 4. Is the option potentially harmful, ineffective or not worth the cost?

Perspectives on the National Institute for Health and Clinical Excellence's recommendations to use health technologies only in research

Background:The concept of using public funds to pay for healthcare interventions only when provided in the context of ongoing research is receiving increasing attention worldwide. Nevertheless, these decisions are often controversial and implementation can be problematic.

Objectives:The aim of this study was to investigate the views of United Kingdom stakeholders on the current arrangements for implementing “only in research” (OIR) decisions and to investigate how improvements might be made.

Methods:After an internal review of previous OIR decisions issued by the National Institute for Health and Clinical Excellence (NICE), deliberations by NICE's Citizens Council, and an international workshop convened by NICE and the United States Agency for Healthcare Research and Quality, thirteen key stakeholders and experts from academia, industry, government, and the National Health Service (NHS) were interviewed using a semistructured interview guide. Interview transcripts were subjected to a framework-based analysis using computer-assisted qualitative data analysis software.

Results:All interviewees endorsed the use of the OIR option. There was a high degree of consensus for several suggestions regarding how the use of the OIR option might be improved. For example, there was universal agreement that a formal process should be established to prioritize research needs arising from OIR decisions and that funds for publicly funded research projects should be channeled in a manner that would better motivate healthcare providers to participate in OIR-related research.

Conclusions:The findings of this study suggest several potential modifications of the OIR pathway in the United Kingdom and may also be helpful to health technology assessment agencies in other countries that already use or are considering using an OIR-like option to reduce the uncertainty inherent in health technology assessment.

Health technology assessment and primary data collection for reducing uncertainty in decision making

BACKGROUND

Health care expenditures continue to escalate, and pressures for increased spending will continue. Health care decision makers from publicly financed systems, private insurance companies, or even from individual health care institutions, will continue to be faced with making difficult purchasing, access, and reimbursement decisions. As a result, decision makers are increasingly turning to evidence-based platforms to help control costs and make the most efficient use of existing resources. Most tools used to assist with evidence-based decision making focus on clinical outcomes.

HEALTH TECHNOLOGY ASSESSMENT

Health technology assessment (HTA) is increasing in popularity because it also considers other factors important for decision making, such as cost, social and ethical values, legal issues, and factors such as the feasibility of implementation. In some jurisdictions, HTAs have also been supplemented with primary data collection to help address uncertainty that may still exist after conducting a traditional HTA.

ROLE OF PRIMARY DATA COLLECTION

The HTA process adopted in Ontario, Canada, is unique in that assessments are also made to determine what primary data research should be conducted and what should be collected in these studies. In this article, concerns with the traditional HTA process are discussed, followed by a description of the HTA process that has been established in Ontario, with a particular focus on the data collection program followed by the Programs for Assessment of Technology in Health Research Institute. An illustrative example is used to show how the Ontario HTA process works and the role value of information analyses plays in addressing decision uncertainty, determining research feasibility, and determining study data collection needs.

Exploring uncertainty in cost-effectiveness analysis

This paper describes the key principles of why an assessment of uncertainty and its consequences are critical for the types of decisions that a body such as the UK National Institute for Health and Clinical Excellence (NICE) has to make. In doing so, it poses the question of whether formal methods may be useful to NICE and its advisory committees in making such assessments. Broadly, these include the following: (i) should probabilistic sensitivity analysis continue to be recommended as a means to characterize parameter uncertainty; (ii) which methods should be used to represent other sources of uncertainty; (iii) when can computationally expensive models be justified and is computation expense a sufficient justification for failing to express uncertainty; (iv) which summary measures of uncertainty should be used to present the results to decision makers; and (v) should formal methods be recommended to inform the assessment of the need for evidence and the consequences of an uncertain decision for the UK NHS?