A Systematic Review of the Level of Evidence in Economic Evaluations of Medical Devices: The Example of Vertebroplasty and Kyphoplasty

Trends and Themes in the Study of Value in Orthopedic Surgery: A Systematic Review

Background

The study of value in orthopedic surgery aims to maximize health outcomes gained per unit cost through various health economic tools but is fragmented across various subspecialties and geographies. Therefore, it is difficult to ascertain whether this research methodology is being used to its full potential across all orthopedic subspecialties and geographies.

Purpose

We sought to assess the distribution of prior health economics literature in orthopedic surgery across subspecialties and geographies. The secondary aim was to identify pertinent methodologic trends that may affect the conclusions drawn.

Methods

A systematic review utilizing 3 electronic databases (Medline, Embase, and Web of Science) was performed. Inclusion criteria included prior systematic reviews assessing economic analyses across all orthopedic surgery subspecialities published between 2010 and April 24, 2021. The quality of evidence was assessed using the Assessment of Multiple Systematic Review tool. Data were qualitatively analyzed.

Results

In the 44 studies included, arthroplasty (36.4%) and spine (31.8%) were the most represented subspecialties. Almost half of studies originated from the United States (45.5%), followed by the United Kingdom (18.2%). Health economic models were most commonly from the perspective of the health care or hospital system (40.5%), followed by the societal perspective (23.5%), and the payer perspective (14.8%).

Conclusions

The study of value in orthopedic surgery is not uniformly leveraged across all subspecialties and geographies. Methodologically, the societal perspective was inadequately represented, despite orthopedic pathologies often incurring significant indirect costs (eg, time off work, rehabilitation expenses).

A pragmatic review of cost-effectiveness evaluations of reimbursed medical devices conducted for Japan

AIMS

The aim of this study is to examine the characteristics and quality of economic evaluations for reimbursed medical devices in Japan.

MATERIALS AND METHODS

A pragmatic review of cost-effectiveness evaluations (CEA) and other economic evaluations for medical devices reimbursed in Japan published between January 2010 and December 2023 was conducted using the PubMed (Medline), Japan Medical Abstracts Society (ICHUSHI Web), and the Cost-Effectiveness Analysis Registry of the Tufts Medical Center databases-as well as Google Scholar. Evaluations for devices assessed under the health technology assessment system in Japan were also reviewed.

RESULTS

Thirty-nine published studies were included with 20 studies (51%) for devices used to treat cardiovascular conditions, 7 studies (18%) for devices used to treat neurological/neurovascular conditions and orthopedic conditions, respectively, and 5 studies (13%) for devices used to treat other types of conditions. The number of published studies for reimbursed medical devices increased from 2017. Nearly 60% of the studies were cost-utility analyses and many were cost-consequence analyses (26%). Although the quality of the studies conducted were good, lack of data was mentioned as a key limitation of nearly all studies-with limited data in general (33%), lack of long-term outcomes data (33%), and lack data for Japan (21%) being key issues. Moreover, lack of cost data was mentioned as a limitation for nearly half (49%) of studies.

LIMITATIONS

As this was not a systematic literature review, some relevant studies may have been excluded. Moreover, some databases that are known to cover other journals were not used.

CONCLUSIONS

Despite concerns about lack of sufficient outcomes data, good quality CEAs have been published for reimbursed devices in Japan. However, lack of data may still be an issue and the impact of the learning curve effect on cost-effectiveness may need to be considered more.

The Use of Evidence-Informed Deliberative Processes for Health Benefit Package Design in Kazakhstan

Kazakhstan strives to obtain Universal Health Coverage (UHC) by using health technology assessment (HTA) for determining their health benefit package. This paper reports on employing evidence-informed deliberative processes (EDPs), a practical and stepwise approach to enhance legitimate health benefit package design in Kazakhstan.

METHODS

The Ministry of Health of Kazakhstan approved the operationalization and application of EDPs during March 2019 and December 2020. We used a combination of desk research, conducting HTA, online surveys as well as a face-to-face workshop in Nur-Sultan, Kazakhstan, and two online workshops to prioritize 25 selected health technologies. During the latter, we tested two alternative approaches to prioritization: quantitative multicriteria decision analysis (MCDA) and the use of decision rules.

RESULTS

For each of the HTA reports, evidence summaries were developed according to the decision criteria (safety, social priority disease, severity of disease, effectiveness, cost-effectiveness, level of evidence, financial risk protection and budget impact). When appraising the evidence, the advisory committee preferred using quantitative MCDA, and only when this would result in any controversy could decision rules be applied.

CONCLUSIONS

Despite several challenges, including a partial disruption because of the COVID-19 pandemic, implementation of the process will likely play a key role in determining an evidence-informed and transparent health benefit package.

Current State of Health Economic Analyses for Low-Grade Glioma Management: A Systematic Review

BACKGROUND

Health economic analyses help determine the value of a medical intervention by assessing the costs and outcomes associated with it. The objective of this study was to assess the level of evidence in economic evaluations for low-grade glioma (LGG) management.

METHODS

Following the PRISMA guidelines, we conducted a systematic review of English articles in Medline, Embase, The Central Registration Depository, EconPapers, and EconLit. The results were screened, and data were extracted by 2 independent reviewers for studies reporting economic evaluations for LGG. The quality of each study was evaluated using the CHEERS (Consolidated Health Economic Evaluation Reporting Standards) checklist, the hierarchy scale developed by Cooper et al. (2005), and the Quality of Health Economic Studies instrument.

RESULTS

Three studies met our inclusion criteria. The adjusted incremental cost-effectiveness ratio (ICER) values for the included studies ranged from $3934 to $9936, but each evaluated a different aspect of LGG management. All had a good quality of reporting per the CHEERS checklist. Based on the Cooper et al. hierarchy scale, the quality of data use was lacking most for utilities. The quality of study design was scored as 82, 92, and 100 for each study using the Quality of Health Economic Studies instrument.

CONCLUSIONS

Although a limited number of economic evaluations were identified, the studies evaluated here were well designed. The interventions assessed were all considered cost-effective, but pooled analysis was not possible because of heterogeneity in the interventions assessed. Given the importance of value and cost-effectiveness in medical care, more evidence is needed in this area.

Cost-Effectiveness of Hip Arthroscopy for Treatment of Femoroacetabular Impingement Syndrome and Labral Tears: A Systematic Review

Background:

Hip arthroscopy has frequently been shown to produce successful outcomes as a treatment for femoroacetabular impingement (FAI) and labral tears. However, there is less literature on whether the favorable results of hip arthroscopy can justify the costs, especially when compared with a nonoperative treatment.

Purpose:

To systematically review the cost-effectiveness of hip arthroscopy for treating FAI and labral tears.

Study Design:

Systematic review; Level of evidence, 3.

Methods:

PubMed/MEDLINE, Embase, and Cochrane Library databases, and the Tufts University Cost-Effectiveness Analysis Registry were searched to identify articles that reported the cost per quality-adjusted life-year (QALY) generated by hip arthroscopy. The key terms used were “hip arthroscopy,” “cost,” “utility,” and “economic evaluation.” The threshold for cost-effectiveness was set at $50,000/QALY. The Methodological Index for Non-Randomized Studies instrument and Quality of Health Economic Studies (QHES) score were used to determine the quality of the studies. This study was prospectively registered on PROSPERO (CRD42020172991).

Results:

Six studies that reported the cost-effectiveness of hip arthroscopy were identified, and 5 of these studies compared hip arthroscopy to a nonoperative comparator. These studies were found to have a mean QHES score of 85.2 and a mean cohort age that ranged from 33-37 years. From both a health care system perspective and a societal perspective, 4 studies reported that hip arthroscopy was more costly but resulted in far greater gains than did nonoperative treatment. The preferred treatment strategy was most sensitive to duration of benefit, preoperative osteoarthritis, cost of the arthroscopy, and the improvement in QALYs with hip arthroscopy.

Conclusion:

In the majority of the studies, hip arthroscopy had a higher initial cost but provided greater gain in QALYs than did a nonoperative treatment. In certain cases, hip arthroscopy can be cost-effective given a long enough duration of benefit and appropriate patient selection. However, there is further need for literature to analyze willingness-to-pay thresholds.

Quality of economic evaluations of ventricular assist devices: A systematic review

OBJECTIVE

Because of a lack of suitable heart donors, alternatives to transplantation are required. These alternatives can have high costs. The aim of this study was to perform a systematic review of cost-effectiveness studies of ventricular assist devices (VADs) and to assess the level of evidence of relevant studies. The purpose was not to present economic findings.

METHODS

A systematic review was performed using four electronic databases to identify health economic evaluation studies dealing with VADs. The methodological quality and reporting quality of the studies was assessed using three different tools, the Drummond, Cooper, and CHEERS (Consolidated Health Economic Evaluation Reporting Standards) checklists.

RESULTS

Of the 1,258 publications identified, thirteen articles were included in this review. Twelve studies were cost-utility analyses and one was a cost-effectiveness analysis. According to the Cooper hierarchy scale, the quality of the data used was heterogeneous. The level of evidence used for clinical effect sizes, safety data, and baseline clinical data was of poor quality. In contrast, cost data were of high quality in most studies. Quality of reporting varied between studies, with an average score of 17.4 (range 15-19) according to the CHEERS checklist.

CONCLUSION

The current study shows that the quality of clinical data used in economic evaluations of VADs is rather poor in general. This is a concern that deserves greater attention in the process of health technology assessment of medical devices.

Percutaneous Vertebroplasty: A History of Procedure, Technology, Culture, Specialty, and Economics

Percutaneous vertebroplasty (VP) progressed from a virtually unknown procedure to one performed on hundreds of thousands of patients annually. The development of VP provides a historically exciting case study into a rapidly adopted procedure. VP was the synthesis of information gained from spinal biopsy developments, the inception of biomaterials used in medicine, and the unique health care climate in France during the 1980s. It was designed as a revolutionary technique to treat vertebral body fractures with minimal side effects and was rapidly adopted and marketed in the United States. The impact of percutaneous vertebroplasty on spine surgery was profound.

Level of Evidence in Economic Evaluations of Left Atrial Appendage Closure Devices: A Systematic Review

OBJECTIVES

The objective of the present work was to assess the level of evidence in economic evaluations of percutaneous left atrial appendage closure devices, and to test the complementarity of three different tools for assessing the quality of economic evaluations.

METHODS

We conducted a systematic review of articles in English or French listed in MEDLINE, Embase, Cochrane, the Cost-Effectiveness Analysis registry and the National Health Service Economic Evaluation Database. We included only economic evaluations concerning left atrial appendage closure devices. Data were extracted from articles by two authors working independently and using three analysis grids to measure the quality of economic evaluations [the British Medical Journal (BMJ) checklist, the hierarchy scale developed by Cooper et al. (J Health Serv Res Policy 10:245-50, 2005) and the Quality of Health Economic Studies (QHES) instrument].

RESULTS

Seven economic evaluations met our inclusion criteria. All were published between 2013 and 2016. All were cost-utility analyses, and fully complied with the BMJ checklist. According to the hierarchy scale developed by Cooper et al., the quality of data used was heterogeneous. Finally, the mean score for the seven economic studies was 90/100 with the QHES instrument.

CONCLUSIONS

Despite the recent development of left atrial appendage closure devices, most economic evaluations conducted here were well-designed studies. Furthermore, different tools used to assess the quality of these studies were complementary, but none gave a global vision of the quality of economic studies.

Designing medical technology for resilience: integrating health economics and human factors approaches

INTRODUCTION

The slow adoption of innovation into healthcare calls into question the manner of evidence generation for medical technology. This paper identifies potential reasons for this including a lack of attention to human factors, poor evaluation of economic benefits, lack of understanding of the existing healthcare system and a failure to recognise the need to generate resilient products. Areas covered: Recognising a cross-disciplinary need to enhance evidence generation early in a technology's life cycle, the present paper proposes a new approach that integrates human factors and health economic evaluation as part of a wider systems approach to the design of technology. This approach (Human and Economic Resilience Design for Medical Technology or HERD MedTech) supports early stages of product development and is based on the recent experiences of the National Institute for Health Research London Diagnostic Evidence Co-operative in the UK. Expert commentary: HERD MedTech i) proposes a shift from design for usability to design for resilience, ii) aspires to reduce the need for service adaptation to technological constraints iii) ensures value of innovation at the time of product development, and iv) aims to stimulate discussion around the integration of pre- and post-market methods of assessment of medical technology.

Existing reporting guidelines for clinical trials are not completely relevant for implantable medical devices: a systematic review

OBJECTIVES

The aim of this study was to determine relevant items for reporting clinical trials on implantable medical devices (IMDs) and to identify reporting guidelines which include these items.

STUDY DESIGN AND SETTING

A panel of experts identified the most relevant items for evaluating IMDs from an initial list based on reference papers. We then conducted a systematic review of articles indexed in MEDLINE. We retrieved reporting guidelines from the EQUATOR network's library for health research reporting. Finally, we screened these reporting guidelines to find those using our set of reporting items.

RESULTS

Seven relevant reporting items were selected that related to four topics: randomization, learning curve, surgical setting, and device information. A total of 348 reporting guidelines were identified, among which 26 met our inclusion criteria. However, none of the 26 reporting guidelines presented all seven items together. The most frequently reported item was timing of randomization (65%). On the contrary, device information and learning curve effects were poorly specified.

CONCLUSION

To our knowledge, this study is the first to identify specific items related to IMDs in reporting guidelines for clinical trials. We have shown that no existing reporting guideline is totally suitable for these devices.

HTA of medical devices: Challenges and ideas for the future from a European perspective

OBJECTIVES

Health Technology Assessment (HTA) of Medical devices (MDs) and MD-based procedures can be challenging due to the unique features and particularities of this group of technologies, such as device-operator interaction. The aim of this study was to (1) clarify, and supplement earlier findings on European HTA institutions' structural, procedural and methodological characteristics with regard to the assessment of MDs and to (2) capture the institutions' perceptions regarding challenges and future trends.

METHODS

Semi-structured telephone interviews with 16 representatives from leading European HTA institutions were performed between April and July 2015. Summative and directed content analysis was used for the analysis, which is reported according to the COREQ checklist.

RESULTS

Findings from the analysis of the interviews were manifold and partially confirmed what has been noted in the literature (e.g. scarce evidence; identifying relevant studies challenging due to more incremental innovations). Additional themes emerged that can be important for future considerations by HTA institutions and policy-makers alike (e.g. areas for future research; need for specific tools).

CONCLUSIONS

The collective opinion of 16 European HTA institutions on MD evaluation could provide ideas to ameliorate the current regulatory situation beyond the modified EU regulation and start broader, more in-depth methodological discussions around the issue. Interviewed experts seem to agree that new approaches such as coverage with evidence development as some countries already introduced could help to overcome the problem with scarce evidence.