A review of the economic tools for assessing new medical devices

Are Medical Device Characteristics Included in HTA Methods Guidelines and Reports? A Brief Review

It is well accepted that medical devices (MDs) and procedures have several unique characteristics compared with pharmaceuticals, such as learning curve (LC), incremental innovation (II), dynamic pricing (DP), and organizational impact (OI). The objective of this study was to determine the extent to which these MD characteristics are routinely assessed by health technology assessment (HTA) agencies and incorporated in their guidelines and reports. Three approaches were taken. First, a review of the most recent HTA methods guidelines from 14 selected HTA agencies and 5 HTA networks was undertaken. Next, HTA reports from these agencies were reviewed for inclusion of MD-specific characteristics for 16 selected MDs. Finally, a narrative literature review on this topic was conducted. A total of 13 of the included HTA organizations, and some HTA networks (2/5), have published either general or MD-specific method guidelines, whilst several addressed MD-specific characteristics. NICE included all four MD characteristics in their guidelines, but this did not equate to their inclusion in published HTA evaluations. European Network HTA (EUnetHTA) described the inclusion of LC (within patient safety) and OI within their guidance. The results highlight a lack of consistency. For the narrative review, 10/149 articles identified were reviewed. Most provided recommendations on challenges faced by HTAs, proposed steps to address uncertainties around MD characteristics and reported a lack of methodological guidance for evaluating MDs. A lack of inclusion of MD characteristics in HTA is a complex interplay of several important factors. For these characteristics to become a formal part of HTA of MDs in the future, clear guidance and frameworks are required to enable manufacturers to develop appropriate evidence, and HTA practitioners to assess their impact more broadly.

Proposing a health-specific organizational impact framework to evaluate organizational impacts in health technology assessments

OBJECTIVES

Health technology assessments (HTAs) have traditionally included clinical and cost-effectiveness evaluation of new health technologies (HTs). However, new HTs can generate important organizational impacts (OIs) that influence their overall value. OIs are currently not clearly identified and evaluated in HTA procedures and tools are limited. To address this issue, a comprehensive framework that allows to assess OIs of new HTs in HTAs is proposed.

METHODS

A working and methodological group identified the Oslo Manual 2018, 4th edition, OECD/Eurostat, on the objectives and outcomes of commercial innovations as the basis for the OIs framework for HTAs. The Oslo Manual was translated to the healthcare sector and adapted to HTA procedures through a three-step process.

RESULTS

The framework is composed of three main parts. Part I tackles the context of the evaluation, Part II the categories of impacts and the specific impacts - in total, 16 OIs were identified - and Part III the stakeholders involved. The central part of the framework is Part II, and consists of three categories of impacts: (i) on the care process, (ii) on the stakeholders' capabilities and skills, and (iii) on society or the community.

CONCLUSIONS

This framework provides a comprehensive and structured basis to document OIs of new HTs. It thus contributes to the extension of HTA evaluation criteria to other dimensions than clinical and economic aspects, that is, organizational aspects. Some of its intrinsic limitations and the questions they raise in the field for policy-makers, practitioners, and researchers are discussed.

Health technology assessment of medical devices: current landscape, challenges, and a way forward

Background

Health Technology Assessment (HTA) has been widely recognized as informing healthcare decision-making, and interest in HTA of medical devices has been steadily increasing. How does the assessment of medical devices differ from that of drug therapies, and what innovations can be adopted to overcome the inherent challenges in medical device HTA?

Method

HTA Accelerator Database was used to describe the landscape of HTA reports for medical devices from HTA bodies, and a literature search was conducted to understand the growth trend of relevant HTA publications in four case studies. Another literature review was conducted for a narrative synthesis of the characteristic differences and challenges of HTA in medical devices. We further conducted a focused Internet search of guidelines and a narrative review of methodologies specific to the HTA of medical devices.

Main body

The evidence of HTA reports and journal publications on medical devices around the world has been growing. The challenges in assessing medical devices include scarcity of well-designed randomized controlled trials, inconsistent real-world evidence data sources and methods, device-user interaction, short product lifecycles, inexplicit target population, and a lack of direct medical outcomes. Practical solutions in terms of methodological advancement of HTA for medical devices were also discussed in some HTA guidelines and literature.

Conclusion

To better conduct HTA on medical devices, we recommend considering multi-source evidence such as real-world evidence; standardizing HTA processes, methodologies, and criteria; and integrating HTA into decision-making.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12962-022-00389-6.

Integrating organizational impacts into health technology assessment (HTA): an analysis of the content and use of existing evaluation frameworks

Objective

The French health technology assessment (HTA) agency initiated a research between 2018 and 2019 with the aim of determining whether other HTA organizations (agencies, bodies, institutes, and expert networks) and researchers had developed an evaluation framework of organizational impacts (OIs).

Methods

Three types of investigation were carried out: (i) an analysis of documents published by selected HTA organizations, (ii) a rapid review on the OI issues, (iii) a questionnaire survey to experts of the International Network of Agencies for Health Technology Assessment.

Results

The analyses highlight six key points: (i) there is no explicit conceptual definition of OIs; (ii) OIs are often not included in a specific dimension of the evaluation or in the same dimensions; (iii) three recurring categories emerge from the assessment of OIs: processes, structure, and culture; (iv) despite its limitations, the European Network for Health Technology Assessment framework (Core Model) is the most mature assessment model to date; (v) the question of the scope of OIs to be considered is unresolved (micro-meso-macro); and (vi) the delineation between OI assessment and economic assessment must be addressed.

Conclusions

Although the issue of considering OI in HTA has been raised for many years, it remains largely unresolved. Defining the concept of OI is a prerequisite for taking the next step toward an evaluation framework. As the question of the impact of innovation goes beyond the health sector, extensive research on how to define and take into account these OIs may be relevant.

Searching for medical devices - practical guidance

Systematic reviews of medical devices have generally adopted the same methodology for the conduct of their reviews as reviews of other clinical interventions, in particular, medicines. It has been suggested that these methods may need to be developed to account for the challenges of reviewing the evidence for medical devices when compared to reviews of medicines. The purpose of this research note is to focus on the methods of searching for medical devices. Our aim is to set out guidance on 'how to search' for medical devices. This includes: Defining what you are searching for; How to design a search strategy; Searching bibliographic databases; Searching beyond bibliographic databases; and Search reporting. The research note is written by three experienced searchers/researchers with experience of critically appraising Medical Technologies Evaluation Programme (MTEP) submissions, undertaking systematic searches and/or reviews of medical devices, or developing guidance for searching for studies for systematic reviews of the effectiveness of interventions, including medical devices. The research note includes a worked example of a search for a fictitious medical device. This article is protected by copyright. All rights reserved.

Quality of economic evaluations of drug-coated balloons and drug-eluting stents in peripheral artery disease: a systematic review

OBJECTIVE

We aimed to perform a systematic review of economic evaluations of drug-coated balloons (DCBs) and drug-eluting stents (DESs) in peripheral artery disease (PAD) and to assess the level of evidence of relevant studies. The purpose was not to present economic findings.

METHODS

A systematic review was performed using four electronic databases to identify health economic evaluation studies reporting on the use of DCBs and DESs in PAD. The methodological and reporting quality of the studies was assessed using three different tools, the Drummond, Cooper, and CHEERS (Consolidated Health Economic Evaluation Reporting Standards) checklists.

RESULTS

Six articles were included in this review of the 1,728 publications identified. Four studies were cost-effectiveness analyses and two cost-utility analyses. According to the Cooper hierarchy scale, the studies used good-quality data sources. The level of evidence used for clinical effect sizes, safety data, baseline clinical data, and costs was of high quality in general. In contrast, an evaluation of the reporting quality suggested that essential information was lacking.

CONCLUSION

The present study demonstrates that clinical data used in economic evaluations of DCBs and DESs in PAD are from clinical studies of high quality in general. However, the quality of reporting represents a concern when interpreting the results provided by these economic studies.

The Use of Decision Analytic Modeling in the Evaluation of Surgical Innovations: A Scoping Review

OBJECTIVES

The main objective of this review was to map how decision analytic models are used in surgical innovation (in which research phase, with what aim) and to understand how challenges related to the assessment of surgical interventions are incorporated.

METHODS

We systematically searched PubMed, Embase, and the Cochrane Library for studies published in 2018. We included original articles using a decision analytic model to compare surgical strategies. We included modeling studies of surgical innovations. General, innovation, and modeling characteristics were extracted, as were outcomes, recommendations, and handling of challenges related to the assessment of surgical interventions (learning curve, incremental innovation, dynamic pricing, quality variation, organizational impact).

RESULTS

We included 46 studies. The number of studies increased with each research phase, from 4% (n = 2) in the preclinical phase to 40% (n = 20) in phase 3 studies. Eighty-one studies were excluded because they investigated established surgical procedures, indicating that modeling is predominantly applied after the innovation process. Regardless of the research stage, the aim to determine cost-effectiveness was most frequently identified (n = 40, 87%), whereas exploratory aims (eg, exploring when a strategy becomes cost-effective) were less common (n = 9, 20%). Most challenges related to the assessment of surgical interventions were rarely incorporated in models (eg, learning curve [n = 1, 2%], organizational impact [n = 2, 4%], and incremental innovation [n = 1, 2%]), except for dynamic pricing (n = 10, 22%) and quality variation (n = 6, 13%).

CONCLUSIONS

In surgical innovation, modeling is predominantly used in later research stages to assess cost-effectiveness. The exploratory use of modeling seems still largely overlooked in surgery; therefore, the opportunity to inform research and development may not be optimally used.

A Web-Based Communication Platform to Improve Home Care Services in Norway (DigiHelse): Pilot Study

Background

Home care service in Norway is struggling to meet the increasing demand for health care under restricted budget constraints, although one-fourth of municipal budgets are dedicated to health services. The integration of Web-based technology in at-home care is expected to enhance communication and patient involvement, increase efficiency and reduce cost. DigiHelse is a Web-based platform designed to reinforce home care service in Norway and is currently undergoing a development process to meet the predefined needs of the country’s municipalities. Some of the main features of the platform are digital messages between residents and the home care service, highlighting information on planned and completed visits, the opportunity to cancel visits, and notifications for completed visits.

Objective

This study aimed to test the usability and economic feasibility of adopting DigiHelse in four districts in Oslo by applying registry and behavioral data collected throughout a one-year pilot study. Early health technology assessment was used to estimate the potential future value of DigiHelse, including the predictive value of behavior data.

Methods

Outcome measures identified by stakeholder insights and scenario drafting in the project’s concept phase were used to assess potential socioeconomic benefits. Aggregated data were collected to assess changes in health consumption at baseline, and then 15 and 52 weeks after DigiHelse was implemented. The present value calculation was updated with data from four intervention groups and one control group. A quasi-experimental difference-in-difference design was applied to estimate the causal effect. Descriptive behavioral data from the digital platform was applied to assess the usability of the platform.

Results

Over the total study period (52 weeks), rates increased for all outcome estimates: the number of visits (rate ratio=1.04; P=.10), unnecessary trips (rate ratio=1.37; P=.26), and phone calls (rate ratio=1.24; P=.08). A significant gap was found between the estimated value of DigiHelse in the concept phase and after the one-year pilot. In the present pilot assessment, costs are expected to exceed potential savings by €67 million (US $75 million) over ten years, as compared to the corresponding concept estimates of a potential gain of €172.6 million (US $193.6 million). Interestingly, behavioral data from the digital platform revealed that only 3.55% (121/3405) of recipients actively used the platform after one year.

Conclusions

Behavioral data provides a valuable source for assessing usability. In this pilot study, the low adoption rate may, at least in part, explain the inability of DigiHelse to perform as expected. This study points to an early assessment of behavioral data as an opportunity to identify inefficiencies and direct digital development. For DigiHelse, insight into why the recipients in Oslo have not made greater use of the Web-based platform seems to be the next step in ensuring the right improvement measures for the home care service.

Critical Review of European Health-Economic Guidelines for the Health Technology Assessment of Medical Devices

Background: Health-technology assessment (HTA) is a recognized mechanism to determine the relative benefits of innovative medical technologies. One aspect is their health-economic impact. While the process and methodology for pharmaceuticals is well-established, guidance for medical devices is sparse.

Aim: To provide an overview of the health-economic aspect in current European HTA guidelines concerning medical devices and identifying issues raised and potential improvements proposed in recent literature.

Methodology: Available guidelines by European agencies were each reviewed and summarized. To complement this, a full systematic review of current literature concerning potential improvements to existing HTA practices for medical devices, from PubMed and EMBASE, was conducted; the focus was on health economics. Authors could only review documents in English, French, or German. The systematic review yielded 518 unique articles concerning HTA for medical devices, 32 of which were considered for full-text review after screening of all abstracts.

Results: There is very limited consensus in—and mostly a complete lack of—guidance specific to medical devices in official HTA guidelines, for both clinical and economic analyses. Twenty two of 41 European countries had published official HTA guidance in English, French, or German. Among these only 4 (England, France, the Netherlands, and Sweden) dedicated a chapter or separate document to medical devices. In the literature, there is sufficient evidence to suggest medical devices need to be addressed separately from pharmaceuticals. However, mostly challenges are discussed rather than implementable solutions offered. We present the following set of frequently discussed issues and summarize any solutions that pertain to them: a weak evidence base, learning-curve effects, organizational impact, incremental innovation, diversity of devices, dynamic pricing, and transferability. We further combine reviewed information to suggest a set of possible best practices for health-economic assessment of medical devices.

Conclusion: For greater efficiency in medical-device innovation, European agencies should look to (re-)address the specific requirements of medical devices in their HTA guidelines. When both the health-economic and data requirements for the HTA of medical devices are defined, the development of practical solutions will likely follow.

Early assessment of innovation in a healthcare setting

OBJECTIVES

Early assessment can assist in allocating resources for innovation effectively and produce the most beneficial technology for an institution. The aim of the present study was to identify methods and discuss the analytical approaches applied for the early assessment of innovation in a healthcare setting.

METHODS

Knowledge synthesis based on a structured search (using the MEDLINE, Embase, and Cochrane databases) and thematic analysis was conducted. An analytical framework based on the stage of innovation (developmental, introduction, or early diffusion) was applied to assess whether methods vary according to stage. Themes (type of innovation, study, analysis, study design, method, and main target audience) were then decided among the authors. Identified methods and analysis were discussed according to the innovation stage.

RESULTS

A total of 1,064 articles matched the search strategy. Overall, thirty-nine articles matched the inclusion criteria. The use of methods has a tendency to change according to the stage of innovation. Stakeholder analysis was a prominent method in the innovation stages and particularly in the developmental stage, as the introduction and early diffusion stage has more availability of data and may apply more complex methods. Barriers to the identified methods were also discussed as all of the innovation stages suffered from lack of data and substantial uncertainty.

CONCLUSIONS

Although this review has identified applicable approaches for early assessment in different innovation stages, research is required regarding the value of the available data and methods and tools to enhance interactions between different parties at different stages of innovation.

ASSESSING THE VALUE OF INNOVATIVE MEDICAL DEVICES AND DIAGNOSTICS: THE IMPORTANCE OF CLEAR AND RELEVANT CLAIMS OF BENEFIT

Objectives:

Large numbers of new medical devices and diagnostics are developed and health services need to identify which ones offer real advantages. The National Institute for Health and Care Excellence (NICE) has introduced a system for assessing technologies that are often notified by companies, based on claims made for their benefits to patients, the National Health Service, and the environment.

Methods:

Detailed scrutiny of claims made for the benefits of products and the corresponding evidence, seeking associations between these and the selection of products for full evaluation to produce NICE guidance.

Results:

Between 2009 and 2015 a NICE committee considered 169 technologies, of which it selected 74 (44 percent) for full evaluation, based on the claims of benefit and the evidence available. An average of 7.5 claims were made per technology; the total number did not influence selection but presence of studies supporting all the claims (p < .001) or any of the claims (p < .05) had a positive influence, as did claims for quicker patient recovery (p < .001). A greater number of studies to support the claims made selection more likely (p < .001), as did cohort studies (p < .05) and surveys (p < .05) but, unexpectedly, not randomized trials. The Medical Device Directive class had no influence.

Conclusions:

This study presents categories of claims that may be useful to those developing new products and to others engaged in health technology assessment. It illustrates the importance of relevant evidence and of having a clear vision of the place of new products in care pathways from an early stage.

A Scoping Review of Economic Evaluations Alongside Randomised Controlled Trials of Home Monitoring in Chronic Disease Management

Many countries have considered telemedicine and home monitoring of patients as a solution to the demographic challenges that health-care systems face. However, reviews of economic evaluations of telemedicine have identified methodological problems in many studies as they do not comply with guidelines. The aim of this study was to examine economic evaluations alongside randomised controlled trials of home monitoring in chronic disease management and hereby to explore the resources included in the programme costs, the types of health-care utilisation that change as a result of home monitoring and discuss the value of economic evaluation alongside randomised controlled trials of home monitoring on the basis of the studies identified. A scoping review of economic evaluations of home monitoring of patients with chronic disease based on randomised controlled trials and including information on the programme costs and the costs of equipment was carried out based on a Medline (PubMed) search. Nine studies met the inclusion criteria. All studies include both costs of equipment and use of staff, but there is large variation in the types of equipment and types of tasks for the staff included in the costs. Equipment costs constituted 16-73% of the total programme costs. In six of the nine studies, home monitoring resulted in a reduction in primary care or emergency contacts. However, in total, home monitoring resulted in increased average costs per patient in six studies and reduced costs in three of the nine studies. The review is limited by the small number of studies found and the restriction to randomised controlled trials, which can be problematic in this area due to lack of blinding of patients and healthcare professionals and the difficulty of implementing organisational changes in hospital departments for the limited period of a trial. Furthermore, our results may be based on assessments of older telemedicine interventions.

Designing medical technology for resilience: integrating health economics and human factors approaches

INTRODUCTION

The slow adoption of innovation into healthcare calls into question the manner of evidence generation for medical technology. This paper identifies potential reasons for this including a lack of attention to human factors, poor evaluation of economic benefits, lack of understanding of the existing healthcare system and a failure to recognise the need to generate resilient products. Areas covered: Recognising a cross-disciplinary need to enhance evidence generation early in a technology's life cycle, the present paper proposes a new approach that integrates human factors and health economic evaluation as part of a wider systems approach to the design of technology. This approach (Human and Economic Resilience Design for Medical Technology or HERD MedTech) supports early stages of product development and is based on the recent experiences of the National Institute for Health Research London Diagnostic Evidence Co-operative in the UK. Expert commentary: HERD MedTech i) proposes a shift from design for usability to design for resilience, ii) aspires to reduce the need for service adaptation to technological constraints iii) ensures value of innovation at the time of product development, and iv) aims to stimulate discussion around the integration of pre- and post-market methods of assessment of medical technology.

Existing reporting guidelines for clinical trials are not completely relevant for implantable medical devices: a systematic review

OBJECTIVES

The aim of this study was to determine relevant items for reporting clinical trials on implantable medical devices (IMDs) and to identify reporting guidelines which include these items.

STUDY DESIGN AND SETTING

A panel of experts identified the most relevant items for evaluating IMDs from an initial list based on reference papers. We then conducted a systematic review of articles indexed in MEDLINE. We retrieved reporting guidelines from the EQUATOR network's library for health research reporting. Finally, we screened these reporting guidelines to find those using our set of reporting items.

RESULTS

Seven relevant reporting items were selected that related to four topics: randomization, learning curve, surgical setting, and device information. A total of 348 reporting guidelines were identified, among which 26 met our inclusion criteria. However, none of the 26 reporting guidelines presented all seven items together. The most frequently reported item was timing of randomization (65%). On the contrary, device information and learning curve effects were poorly specified.

CONCLUSION

To our knowledge, this study is the first to identify specific items related to IMDs in reporting guidelines for clinical trials. We have shown that no existing reporting guideline is totally suitable for these devices.

Review of Economic Submissions to NICE Medical Technologies Evaluation Programme

The economic evaluation of medical devices is increasingly used to inform decision making on adopting new or novel technologies; however, challenges are inevitable due to the unique characteristics of devices. Cost-consequence analyses are recommended and employed by the English National Institute for Health and Care Excellence (NICE) Medical Technologies Evaluation Programme (MTEP) to help address these challenges. The aim of this work was to review the critiques raised for previous MTEP submissions and explore if there were common problems across submissions. We reviewed a sample of 12 economic submissions to MTEP representing 50 % of 24 sets of guidance issued to July 2015. For each submission, we reviewed the External Assessment Centre's (EAC) report and the guidance document produced by NICE. We identified the main problems raised by the EAC's assessments and the committee's considerations for each submission, and explored strategies for improvement. We found that the identification and measurement of costs and consequences are the main shortcomings within economic submissions to MTEP. Together, these shortcomings accounted for 42 % of criticisms by the EACs among the reviewed submissions. In certain circumstances problems with these shortcomings may be unavoidable, for example, if there is a limited evidence base for the device being appraised. Nevertheless, strategies can often be adopted to improve submissions, including the use of more appropriate time horizons, whilst cost and resource use information should be taken, where possible, from nationally representative sources.

HTA of medical devices: Challenges and ideas for the future from a European perspective

OBJECTIVES

Health Technology Assessment (HTA) of Medical devices (MDs) and MD-based procedures can be challenging due to the unique features and particularities of this group of technologies, such as device-operator interaction. The aim of this study was to (1) clarify, and supplement earlier findings on European HTA institutions' structural, procedural and methodological characteristics with regard to the assessment of MDs and to (2) capture the institutions' perceptions regarding challenges and future trends.

METHODS

Semi-structured telephone interviews with 16 representatives from leading European HTA institutions were performed between April and July 2015. Summative and directed content analysis was used for the analysis, which is reported according to the COREQ checklist.

RESULTS

Findings from the analysis of the interviews were manifold and partially confirmed what has been noted in the literature (e.g. scarce evidence; identifying relevant studies challenging due to more incremental innovations). Additional themes emerged that can be important for future considerations by HTA institutions and policy-makers alike (e.g. areas for future research; need for specific tools).

CONCLUSIONS

The collective opinion of 16 European HTA institutions on MD evaluation could provide ideas to ameliorate the current regulatory situation beyond the modified EU regulation and start broader, more in-depth methodological discussions around the issue. Interviewed experts seem to agree that new approaches such as coverage with evidence development as some countries already introduced could help to overcome the problem with scarce evidence.