Are the UK systems of innovation and evaluation of medical devices compatible? The role of NICE's Medical Technologies Evaluation Programme (MTEP)

Health technology assessment of medical devices: current landscape, challenges, and a way forward

Background

Health Technology Assessment (HTA) has been widely recognized as informing healthcare decision-making, and interest in HTA of medical devices has been steadily increasing. How does the assessment of medical devices differ from that of drug therapies, and what innovations can be adopted to overcome the inherent challenges in medical device HTA?

Method

HTA Accelerator Database was used to describe the landscape of HTA reports for medical devices from HTA bodies, and a literature search was conducted to understand the growth trend of relevant HTA publications in four case studies. Another literature review was conducted for a narrative synthesis of the characteristic differences and challenges of HTA in medical devices. We further conducted a focused Internet search of guidelines and a narrative review of methodologies specific to the HTA of medical devices.

Main body

The evidence of HTA reports and journal publications on medical devices around the world has been growing. The challenges in assessing medical devices include scarcity of well-designed randomized controlled trials, inconsistent real-world evidence data sources and methods, device-user interaction, short product lifecycles, inexplicit target population, and a lack of direct medical outcomes. Practical solutions in terms of methodological advancement of HTA for medical devices were also discussed in some HTA guidelines and literature.

Conclusion

To better conduct HTA on medical devices, we recommend considering multi-source evidence such as real-world evidence; standardizing HTA processes, methodologies, and criteria; and integrating HTA into decision-making.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12962-022-00389-6.

Current status of medical device malfunction reporting: using end user experience to identify current problems

INTRODUCTION

The current under-reporting of medical device malfunctions, difficulties with the current system and absence of continuous good-quality data has removed the possibility for constant data interrogation and trend recognition to identify evolving issues. This research used end user experiences aiming to understand causes for the lack of data and knowledge on device performance and associated patient risks. This approach was used to identify existing barriers and methods for improvement.

METHODS

This is a qualitative study involving semistructured interviews and surveys with clinicians (15 interviews, 39 surveys) and manufacturers (13 interview participants, 5 surveys). Multiple sources of recruiting were used. Data collected were thematically analysed. Interview results were used to design the surveys. Standards for Reporting Qualitative Research was used.

RESULTS

Medical device use is based on personal experience rather than evidence which is scarce. Multiple barriers to reporting were identified alongside patient safety and system related aspects. Furthermore, the acceptable level of error was variable as were effects on working practice. Many workarounds have been developed to overcome problems and have become normalised in daily work. These factors were found to have a limiting impact on improvements and learning. Greater system transparency, feedback on submitted reports, a more efficient system of reporting and better communication with manufacturers were reported as some of the required improvements.

CONCLUSIONS

This study has identified numerous complex issues affecting reporting of medical device performance and their subsequent effect on patient safety and clinical staff. The focus on incidents has created many limitations to learning and development. The rich experience of end users should be appropriately used to identify system weaknesses and seek improvement methods. Better communication methods should be developed between healthcare and MedTech (Medical Technologies) industry.

Broadening the application of health technology assessment in the Netherlands: a worthwhile destination but not an easy ride?

Currently, reimbursement decisions based on health technology assessments (HTA) in the Netherlands mostly concern outpatient pharmaceuticals. The Dutch government aspires to broaden the systematic application of full HTA towards other types of health care in order to optimise the content of the basic benefit package. This paper identifies important challenges for broadening the scope of full HTA to other types of health care. Based on a description of the Dutch reimbursement decision-making process, five important characteristics of outpatient pharmaceuticals were identified, which are all relevant to the successful application of HTA: (i) closed reimbursement system, (ii) absence of alternative policy measures, (iii) existence of marketing authorisation, (iv) identifiable and accountable counterparty, and (v) product characteristics. For a selection of other types of health care, which may be subject to HTA more frequently in the future, deviations from these characteristics of outpatient pharmaceuticals are discussed. The implications of such deviations for performing HTA and the decision-making process are highlighted. It is concluded that broadening the application of HTA will require policy makers to meet both important policy-related and methodological challenges. These challenges differ per health care domain, which may inform policy makers which expansions of the current use of HTA are most feasible.

Assessment of Devices, Diagnostics and Digital Technologies: A Review of NICE Medical Technologies Guidance

BACKGROUND

The Medical Technologies Evaluation Programme (MTEP) of NICE in England aims to evaluate medical devices that are deemed to be cost-saving or cost-neutral and produce Medical Technology Guidance (MTG) to encourage their adoption.

OBJECTIVE

To review the MTGs since MTEP's inception in 2009 until February 2017.

METHODS

One researcher assessed all published MTGs and extracted data on the clinical and economic evidence supporting each technology. The NICE Committee's decision outcome for each assessment was also recorded. A qualitative analysis was performed on technologies that were not supported for adoption to identify the main drivers of the decision.

RESULTS

Thirty-one MTGs were reviewed. The committee fully supported the medical devices in 14 MTGs, 11 were partially supported and six were not supported. Of the MTGs, 58% had no RCT data available and the main source of evidence came from non-experimental studies. There was no statistically significant difference in the average number of RCTs and non-experimental studies between the fully-supported, partially-supported, and not-supported technologies. Whilst all the fully-supported MTGs demonstrated cost-saving results, only 50% of the not-supported MTGs did. The sponsor estimated a higher average cost-saving than the EAC in most of the cases (20/31). The qualitative evaluation suggests that the main drivers for negative decisions were the quantity or quality of studies, and costs incurred in the economic evaluation results.

CONCLUSIONS

The main drivers of the decision-making process are the quality and quantity of the submitted evidence supporting the technologies, as well as the economic evaluation results.

A Review of NICE Methods and Processes Across Health Technology Assessment Programmes: Why the Differences and What is the Impact?

BACKGROUND

Decisions made by the National Institute for Health and Care Excellence (NICE) exert an influence on the allocation of resources within 'fixed' National Health Service budgets. Yet guidance for different types of health interventions is handled via different 'programmes' within NICE, which follow different methods and processes.

OBJECTIVE

The objective of this research was to identify differences in the processes and methods of NICE health technology assessment programmes and to explore how these could impact on allocative efficiency within the National Health Service.

METHODS

Data were extracted from the NICE technology appraisal programme, medical technologies guidance, diagnostic assessment programme, highly specialised technologies programme, and clinical guidelines process and methods manuals to undertake a systematic comparison. Five qualitative interviews were carried out with NICE members of staff and committee members to explore the reasons for the differences found.

RESULTS

The main differences identified were in the required evidence review period, or lack thereof, mandatory funding status, the provision of a reference case for economic evaluation, the requirement for and the type of economic analysis undertaken, and the decision making criteria used for appraisal.

CONCLUSION

Many of the differences found can be justified on grounds of practicality and relevance to the health technologies under assessment. Nevertheless, from a strict utilitarian view, there are several potential areas of inefficiency that could lead to the misallocation of resources within the National Health Service, although some of these might be eliminated or reduced if an egalitarian view is taken. The challenge is determining where society is willing to trade health gains between different people.

Linking the Regulatory and Reimbursement Processes for Medical Devices: The Need for Integrated Assessments

Much criticism has been directed at the licencing requirements for medical devices (MDs) as they often result in a lack of robust evidence to inform health technology assessment (HTA) decisions. To better understand the current international decisional framework on MD technologies, we undertook three linked research studies: a review of the device regulatory procedures, a survey of current HTA practices and an empirical comparison of HTA reports of drugs versus MDs. Our review confirms that current device regulatory processes across the globe are substantially less stringent than drugs. As a result, international HTA agencies report that they face a number of challenges when assessing MDs, including reliance on suboptimal data to make clinical and cost-effectiveness decisions. Whilst many HTA agencies have adapted their processes and procedures to handle MD technology submissions, in our comparison of HTA reports we found little evidence of the application of methodologies that take account of device-specific issues, such as incremental development. Overall, our research reinforces the need for better linkage between licencing and HTA and the development and application of innovative HTA methodologies with the objective of securing faster patient access for those technologies that can be shown to represent good value for money. © 2017 The Authors. Health Economics Published by John Wiley & Sons, Ltd.

Assessing the Expected Value of Research Studies in Reducing Uncertainty and Improving Implementation Dynamics

BACKGROUND

With low implementation of cost-effective health technologies being a problem in many health systems, it is worth considering the potential effects of research on implementation at the time of health technology assessment. Meaningful and realistic implementation estimates must be of dynamic nature.

OBJECTIVE

To extend existing methods for assessing the value of research studies in terms of both reduction of uncertainty and improvement in implementation by considering diffusion based on expert beliefs with and without further research conditional on the strength of evidence.

METHODS

We use expected value of sample information and expected value of specific implementation measure concepts accounting for the effects of specific research studies on implementation and the reduction of uncertainty. Diffusion theory and elicitation of expert beliefs about the shape of diffusion curves inform implementation dynamics. We illustrate use of the resulting dynamic expected value of research in a preterm birth screening technology and results are compared with those from a static analysis.

RESULTS

Allowing for diffusion based on expert beliefs had a significant impact on the expected value of research in the case study, suggesting that mistakes are made where static implementation levels are assumed. Incorporating the effects of research on implementation resulted in an increase in the expected value of research compared to the expected value of sample information alone.

CONCLUSIONS

Assessing the expected value of research in reducing uncertainty and improving implementation dynamics has the potential to complement currently used analyses in health technology assessments, especially in recommendations for further research. The combination of expected value of research, diffusion theory, and elicitation described in this article is an important addition to the existing methods of health technology assessment.

Review of Economic Submissions to NICE Medical Technologies Evaluation Programme

The economic evaluation of medical devices is increasingly used to inform decision making on adopting new or novel technologies; however, challenges are inevitable due to the unique characteristics of devices. Cost-consequence analyses are recommended and employed by the English National Institute for Health and Care Excellence (NICE) Medical Technologies Evaluation Programme (MTEP) to help address these challenges. The aim of this work was to review the critiques raised for previous MTEP submissions and explore if there were common problems across submissions. We reviewed a sample of 12 economic submissions to MTEP representing 50 % of 24 sets of guidance issued to July 2015. For each submission, we reviewed the External Assessment Centre's (EAC) report and the guidance document produced by NICE. We identified the main problems raised by the EAC's assessments and the committee's considerations for each submission, and explored strategies for improvement. We found that the identification and measurement of costs and consequences are the main shortcomings within economic submissions to MTEP. Together, these shortcomings accounted for 42 % of criticisms by the EACs among the reviewed submissions. In certain circumstances problems with these shortcomings may be unavoidable, for example, if there is a limited evidence base for the device being appraised. Nevertheless, strategies can often be adopted to improve submissions, including the use of more appropriate time horizons, whilst cost and resource use information should be taken, where possible, from nationally representative sources.

When Future Change Matters: Modeling Future Price and Diffusion in Health Technology Assessments of Medical Devices

BACKGROUND

Health technology assessments (HTAs) that take account of future price changes have been examined in the literature, but the important issue of price reductions that are generated by the reimbursement decision has been ignored.

OBJECTIVES

To explore the impact of future price reductions caused by increasing uptake on HTAs and decision making for medical devices.

METHODS

We demonstrate the use of a two-stage modeling approach to derive estimates of technology price as a consequence of changes in technology uptake over future periods on the basis of existing theory and supported by empirical studies. We explore the impact on cost-effectiveness and expected value of information analysis in an illustrative example on the basis of a technology in development for preterm birth screening.

RESULTS

The application of our approach to the case study technology generates smaller incremental cost-effectiveness ratios compared with the commonly used single cohort approach. The extent of this reduction in the incremental cost-effectiveness ratio depends on the magnitude of the modeled price reduction, the speed of diffusion, and the length of the assumed technology life horizon. Results of value of information analysis are affected through changes in the expected net benefit calculation, the addition of uncertain parameters, and the diffusion-adjusted estimate of the affected patient population.

CONCLUSIONS

Because modeling future changes in price and uptake has the potential to affect HTA outcomes, modeling techniques that can address such changes should be considered for medical devices that may otherwise be rejected.

PROMISE AND PLAUSIBILITY: HEALTH TECHNOLOGY ADOPTION DECISIONS WITH LIMITED EVIDENCE

Background: The adoption of new medical devices and diagnostics is often hampered by lack of published evidence which makes conventional health technology assessment (HTA) difficult. We now have 5 years’ experience of the Medical Technologies Advisory Committee of the National Institute for Health and Care Excellence (NICE) in the United Kingdom, addressing this problem. This committee assesses devices and diagnostics against claims of advantage, to produce guidance on adoption for the health service.

Methods: We have reflected on the practical, technical, and intellectual processes we have used in developing guidance for the health service.

Results: When scientific and clinical evidence is sparse, promise and plausibility play an increased part in decision-making. Drivers of promise include a clear design and mechanism of action, the possibility of radical improvement in care and/or resource use, and improving health outcomes for large numbers of patients. Plausibility relates to judgements about the whether the promise is likely to be delivered in a “real world” setting. Promise and plausibility need to be balanced against the amount of evidence available. We examine the influence they may have on decision-making compared with other factors such as risk and cost.

Conclusions: Decisions about adoption of new devices and diagnostics with little evidence are influenced by judgements of their promise and the plausibility of claims that they will provide benefits in a real-world setting. This kind of decision making needs to be transparent and this article explains how these influences can be balanced against the use of more familiar criteria.

A review of the economic tools for assessing new medical devices

Whereas the economic evaluation of pharmaceuticals is an established practice within international health technology assessment (HTA) and is often produced with the support of comprehensive methodological guidance, the equivalent procedure for medical devices is less developed. Medical devices, including diagnostic products, are a rapidly growing market in healthcare, with over 10,000 medical technology patent applications filed in Europe in 2012-nearly double the number filed for pharmaceuticals. This increase in the market place, in combination with the limited, or constricting, budgets that healthcare decision makers face, has led to a greater level of examination with respect to the economic evaluation of medical devices. However, methodological questions that arise due to the unique characteristics of medical devices have yet to be addressed fully. This review of journal publications and HTA guidance identified these characteristics and the challenges they may subsequently pose from an economic evaluation perspective. These unique features of devices can be grouped into four categories: (1) data quality issues; (2) learning curve; (3) measuring long-term outcomes from diagnostic devices; and (4) wider impact from organisational change. We review the current evaluation toolbox available to researchers and explore potential future approaches to improve the economic evaluation of medical devices.