Contrasting clinical evidence for market authorisation of cardio-vascular devices in Europe and the USA: a systematic analysis of 10 devices based on Austrian pre-reimbursement assessments

Lifecycle evaluation of medical devices: supporting or jeopardizing patient outcomes? A comparative analysis of evaluation models

OBJECTIVES

Lack of evidence regarding safety and effectiveness at market entry is driving the need to consider adopting a lifecycle approach to evaluating medical devices, but it is unclear what lifecycle evaluation means. This research sought to explore the tacit meanings of "lifecycle" and "lifecycle evaluation" as embodied within evaluation models/frameworks used for medical devices.

METHODS

Drawing on qualitative evidence synthesis methods and using an inductive approach, novel methods were developed to identify, appraise, analyze, and synthesize lifecycle evaluation models used for medical devices. Data was extracted (including purpose; audience; characterization; outputs; timing; and type of model) from key texts for coding, categorization, and comparison, exploring embodied meaning across four broad perspectives.

RESULTS

Fifty-two models were included in the synthesis. They demonstrated significant heterogeneity of meaning, form, scope, timing, and purpose. The "lifecycle" may represent a single stage, a series of stages, a cycle of innovation, or a system. "Lifecycle evaluation" focuses on the overarching goal of the stakeholder group, and may use a single or repeated evaluation to inform decision-making regarding the adoption of health technologies (Healthcare), resource allocation (Policymaking), investment in new product development or marketing (Trade and Industry), or market regulation (Regulation). The adoption of a lifecycle approach by regulators has resulted in the deferral of evidence generation to the post-market phase.

CONCLUSIONS

Using a "lifecycle evaluation" approach to inform reimbursement decision-making must not be allowed to further jeopardize evidence generation and patient safety by accepting inadequate evidence of safety and effectiveness for reimbursement decisions.

Utilization of innovative medical technologies in German inpatient care: does evidence matter?

BACKGROUND

The reimbursement of new technologies in inpatient care is not always linked to a requirement for evidence-based evaluation of patient benefit. In Germany, every new technology approved for market was until recently eligible for reimbursement in inpatient care unless explicitly excluded. The aim of this work was (1) to investigate the type of evidence that was available at the time of introduction of 25 innovative technologies and how this evidence evolved over time, and (2) to explore the relationship between clinical evidence and utilization for these technologies in German inpatient care.

METHODS

This study combined different methods. A systematic search for evidence published between 2003 and 2017 was conducted in four bibliographic databases, clinical trial registries, resources for clinical guidelines, and health technology assessment-databases. Information was also collected on funding mechanisms and safety notices. Utilization was measured by hospital procedures captured in claims data. The body of evidence, funding and safety notices per technology were analyzed descriptively. The relationship between utilization and evidence was explored empirically using a multilevel regression analysis.

RESULTS

The number of included publications per technology ranges from two to 498. For all technologies, non-comparative studies form the bulk of the evidence. The number of randomized controlled clinical trials per technology ranges from zero to 19. Some technologies were utilized for several years without an adequate evidence base. A relationship between evidence and utilization could be shown for several but not all technologies.

CONCLUSIONS

This study reveals a mixed picture regarding the evidence available for new technologies, and the relationship between the development of evidence and the use of technologies over time. Although the influence of funding and safety notices requires further investigation, these results re-emphasize the need for strengthening market approval standards and HTA pathways as well as approaches such as coverage with evidence development.

Gaps in the evidence underpinning high-risk medical devices in Europe at market entry, and potential solutions

AIM

To determine the level of evidence for innovative high-risk medical devices at market entry.

METHODS

We reviewed all Belgian healthcare payer (RIZIV-INAMI) assessor reports on novel implants or invasive medical devices (n = 18, Class IIb-III) available between 2018 to mid-2019 on applications submitted for inclusion on their reimbursement list. We also conducted a review of the literature on evidence gaps and an analysis of relevant legal and ethical frameworks within the European context.

FINDINGS

Conformity assessment of medical devices is based on performance, safety, and an acceptable risk-benefit balance. Information submitted for obtaining CE marking is confidential and legally protected, limiting access to clinical evidence. Seven out of the 18 RIZIV-INAMI assessor reports (39%) included a randomized controlled trial (RCT) using the novel device, whilst 2 applications (11%) referred to an RCT that used a different device. The population included was inappropriate or unclear for 3 devices (17%). Only half of the applications presented evidence on quality of life or functioning and 2 (11%) presented overall survival data. Four applications (22%) included no data beyond twelve months. The findings from the literature demonstrated similar problems with the study design and the clinical evidence.

DISCUSSION AND CONCLUSIONS

CE marking does not indicate that a device is effective, only that it complies with the law. The lack of transparency hampers evidence-based decision making. Despite greater emphasis on clinical benefit for the patient, the provisions of the European Medical Device Regulation (MDR) are not yet fully aligned with international ethical standards for clinical research. The MDR fails to address key issues, such as the lack of access to data submitted for CE marking and a failure to require evidence of clinical effectiveness. Indeed, a first report shows no improvement in the clinical evidence for implantable devices generated under the MDR. Thus, patients may continue to be exposed to ineffective or unsafe novel devices. The Health Technology Assessment Regulation plans for Joint Scientific Consultations for specific high-risk devices before companies begin their pivotal clinical investigations. The demanded comparative evidence should facilitate payer decisions. Nevertheless, there is also a need for legislation requiring comparative RCTs assessing patient-relevant outcomes for high-risk devices to ensure implementation, including development and implementation of common specifications for study designs.

Reimbursement decisions for medical services in Austria: an analysis of influencing factors for the hospital individual services catalogue between 2008 and 2020

OBJECTIVES

To (1) describe the (evidence-based) reimbursement process of hospital individual services, (2) evaluate the accordance between evidence-based recommendations and reimbursement decision of individual services and (3) elaborate potential aspects that play a role in the decision-making process in Austria.

METHOD

The reimbursement process is described based on selected relevant sources such as official documents. Evidence-based recommendations and subsequent reimbursement decisions for the annual maintenance of the hospital individual service catalogue in Austria between 2008 and 2020 were analysed using a mixed methods approach, encompassing descriptive statistics and a focus group with Austrian decision makers.

RESULTS

118 evidence-based recommendations were analysed. There were 93 (78.8%) negative and 25 (21.2%) positive evidence-based recommendations. In total, 107 out of 118 evidence-based recommendations (90.1%) did not lead to a deviating reimbursement decision. We identified six aspects that may have played a role in the decision-making process for the annual maintenance of the hospital individual service catalogue, with clinical evidence being the most notable. Further aspects included quality assurance/organisational aspects (i.e., structural quality assurance), costs (if comparable to already existing medical services, not: cost-effectiveness), procedural aspects (e.g., if certain criteria for adoption have not been met formally through the proposals), "other countries" (i.e., taking into account how other countries decided) and situational aspects (such as the COVID-19 pandemic).

CONCLUSIONS

There is good accordance between evidence-based recommendations and reimbursement decisions regarding hospital individual services in Austria. Beyond clinical evidence, organisational aspects seem to be considered often with regard to quality assurance but costs do not appear to play a major role. The Austrian system has mechanisms in place that can restrict widespread adoption of novel hospital individual services with uncertain clinical benefits. Future studies could investigate how well these mechanisms work and how they compare to other health systems in Europe.

Premarket evaluation of medical devices: a cross-sectional analysis of clinical studies submitted to a German ethics committee

OBJECTIVE

To assess the methodological quality of pre-market clinical studies performed on medical devices (MDs), including in-vitro diagnostic (IVD) MDs, in Europe.

DESIGN

Observational cross-sectional study.

SETTING

A large German ethics committee.

MATERIALS

From the consecutive sample of study applications between March 2010 and December 2013, we selected MD study applications requiring approval by an ethics committee and the competent federal authority. These included pre-market studies on devices that had not yet received a CE (Conformité Européenne) mark or had previously been CE marked for a different indication. Also included were post-CE studies requiring federal authority approval because the study entailed additional invasive or otherwise burdensome components.

PRIMARY AND SECONDARY OUTCOME MEASURES

Besides the design of the studies, we assessed the planned sample size, study duration and other aspects.

RESULTS

122 study applications were analysed: 98 (80%) concerned therapeutic rather than diagnostic devices and 84 (69%) were pre-market studies. The proportion of studies on class I, IIa, IIb and III devices was 10%, 15%, 28% and 39%, respectively. 10 studies (8%) investigated IVD MDs. A randomised controlled trial (RCT) was planned in 70 (57%) of the 122 applications; studies with non-randomised control groups (n=23; 19%) or without controls (n=29; 24%) were less common. In the sub-group of pre-market studies on therapeutic devices, the proportion of RCTs was 66% (43/65). The median sample size was 120 participants or samples (IQR 53-229). The median study duration was 24 (14-38) months. 87 studies (71%) considered at least one patient-relevant outcome. 12 (17%) and 37 (53%) of the 70 RCTs applied a fully or partially blinded design, respectively.

CONCLUSION

A large proportion of MD studies in Germany apply a randomised controlled design, thus contradicting the industry argument that RCTs on MDs are commonly infeasible.

Pre-reimbursement: early assessment for coverage decisions

Background

In the past decade, the Ludwig Boltzmann Institute for Health Technology Assessment (LBI-HTA) has introduced two programs: “Horizon Scanning in Oncology” (HSO) and extra medical services (“MELs”), which are to facilitate coverage decisions based on early assessments. This article aims to outline the general process and methods within these two programs.

Methods

A narrative-descriptive synthesis of the literature was performed to outline the general and LBI-HTA-specific processes and methods of early assessments.

Results

In total, 79 HSO assessments (2009–2018) and 95 MELs (2008–2018) have been conducted by the LBI-HTA. Recently, additional methods that contribute to European applicability have been introduced into these programs.

Conclusions

Overall, pre-coverage decisions based on early assessment reports are dependent on the existing evidence. However, the organisation of the health care system and the cross-linking between decision-makers and HTA institutions can have an impact.

The HTA Core Model®-10 Years of Developing an International Framework to Share Multidimensional Value Assessment

BACKGROUND AND OBJECTIVES

The HTA Core Model^® as a science-based framework for assessing dimensions of value was developed as a part of the European network for Health Technology Assessment project in the period 2006 to 2008 to facilitate production and sharing of health technology assessment (HTA) information, such as evidence on efficacy and effectiveness and patient aspects, to inform decisions.

METHODS

It covers clinical value as well as organizational, economic, and patient aspects of technologies and has been field-tested in two consecutive joint actions in the period 2010 to 2016. A large number of HTA institutions were involved in the work.

RESULTS

The model has undergone revisions and improvement after iterations of piloting and can be used in a local, national, or international context to produce structured HTA information that can be taken forward by users into their own frameworks to fit their specific needs when informing decisions on technology. The model has a broad scope and offers a common ground to various stakeholders through offering a standard structure and a transparent set of proposed HTA questions. It consists of three main components: 1) the HTA ontology, 2) methodological guidance, and 3) a common reporting structure. It covers domains such as effectiveness, safety, and economics, and also includes domains covering organizational, patient, social, and legal aspects. There is a full model and a focused rapid relative effectiveness assessment model, and a third joint action is to continue till 2020.

CONCLUSION

The HTA Core Model is now available for everyone around the world as a framework for assessing value.

Review times and adverse events for cardiovascular devices

Shorter regulatory review times for high-risk cardiovascular devices correlate with the likelihood of reports of adverse events.

Drugs and Devices: Comparison of European and U.S. Approval Processes

The regulation of medical drugs and devices involves competing goals of assuring safety and efficacy while providing rapid movement of innovative therapies through the investigative and regulatory processes as quickly as possible. The United States and the European Union approach these challenges in different ways. Whereas the United States has always relied on a strictly centralized process through 1 agency, the Food and Drug Administration (FDA), the European Commission synchronized the regulations of 28 different countries as they combined to create the European Union. The FDA historically developed as a consumer protection agency, whereas the regulations from the European Commission arose out of a need to harmonize inter-state commercial interests while preserving national "autonomy." Thus, whereas the FDA has the advantages of centralization and common rules, the European Union regulates medical drug and device approvals through a network of centralized and decentralized agencies throughout its member states. This study explores some of the similarities and differences in European and U.S. regulation of drugs and devices, and discusses challenges facing each.

Comparison of rates of safety issues and reporting of trial outcomes for medical devices approved in the European Union and United States: cohort study

OBJECTIVE

 To evaluate safety alerts and recalls, publication of key trial outcomes, and subsequent US approval of high profile medical devices introduced in the European Union.

DESIGN

 Cohort study.

SETTING

 Novel cardiovascular, orthopedic, and neurologic devices approved in the EU through Conformité Européenne marking between 2005 and 2010.

DATA SOURCES

 Public and commercial databases searched up to January 2016 for press releases and announcements of approvals; public Food and Drug Administration and European regulatory authority databases for US approvals and safety alerts and recalls; and Medline, Embase, and Web of Science for peer reviewed publications.

MAIN OUTCOME MEASURES

 We categorized the novelty of the devices in the study sample as a "major innovation" or an "other change," and extracted descriptive data about the devices and information on any safety alerts and withdrawals. Linear regression models examined factors associated with differential EU and US approvals. Cox proportional hazards regression models were used to evaluate factors associated with safety alerts and recalls and the publication of trial outcomes for devices categorized as major innovations. Models controlled for time, therapeutic category, regulatory pathway, size of sponsoring company, and indicator variables for devices approved first in the EU and devices approved only in the EU.

RESULTS

 67% (206/309) of devices identified were approved in both the US and the EU, of which 63% (129/206) were approved first in the EU. The unadjusted rate of safety alerts and recalls for devices approved first in the EU was 27% (62/232) compared with 14% (11/77) for devices approved first in the US. The adjusted hazard ratio for safety alerts and recalls was 2.9 (95% confidence interval 1.4 to 6.2) for devices approved first in the EU. The results of pivotal trials were published for 49% (37/75) of devices categorized as major innovations, with an overall publication rate of 37% five years after approval.

CONCLUSIONS

 Devices approved first in the EU are associated with an increased risk of post-marketing safety alerts and recalls. Poor trial publication rates mean that patients and clinicians need greater regulatory transparency to make informed decisions about treatment.

Health technology assessment of medical devices: What is different? An overview of three European projects

BACKGROUND

With the growing use and importance of health technology assessment (HTA) in decision making during recent years, health technology assessors, decision makers and stakeholders are confronted with methodological challenges due to specific characteristics of health technologies (e. g., pharmaceuticals, diagnostic tests, screening programs), their developmental environment, and their regulation process. Being aware of the necessity to use HTA as a policy instrument for sustainable health care systems in a regulatory environment of decentralized Conformité Européenne (CE) marking, the European Union (EU) is increasingly supporting the development of methods for the assessment of medical devices (MD) on different levels: within the scope of European research projects and within joint assessment activities of the member states of the European network for Health Technology Assessment (EUnetHTA).

OBJECTIVE

First, this article describes three projects: MedtecHTA-Methods for Health Technology Assessment of Medical Devices, a European Perspective Work Package 3 (WP3), Comparative Effectiveness of Medical Devices led by the University for Health Sciences, Medical Informatics and Technology (UMIT). Second, we discuss the experiences of the Ludwig Boltzmann Institute Health Technology Assessment (LBI HTA) with the joint production of rapid assessments of medical devices by several European HTA agencies within EUnetHTA. Third, a brief outline is given of the framework of joint methodological guideline elaboration by the EUnetHTA partner organizations because a guideline for therapeutic MD is also being developed here.

METHODS

We will describe aims, methods and some preliminary results of MedtecHTA and EUnetHTA Joint Action 2 Work Package 5 Strand B (WP5B) applying the HTA Core Model for Rapid Assessment for national adaptation and reporting, and give an overview of the development process of methodological guidelines within WP 7 of EUnetHTA Joint Action 2.

RESULTS

Based on a literature review in MedtecHTA WP3 incremental development, context dependency and the physical mode of action of MD were identified as those characteristics making therapeutic MD different from drugs with regard to evaluation methods. In addition, regulation does not stipulate clinical trials. These characteristics were also identified as challenges for the production of joint assessments of MD within the HTA network EUnetHTA. Furthermore, adequate timing of assessment production, the variety of involved manufacturers, the non-transparent regulation process of MD in Europe and the often poor evidence base pose a challenge to EUnetHTA assessors. As a consequence, processes and methods for the joint production of rapid assessments must be continuously adapted and improved.

DISCUSSION

Research on HTA methods for the assessment of MD tries to provide tools to deal with rapidly developing devices during evidence generation, dependence of clinical effectiveness of MD on user experience and context factors. There are also tools to integrate evidence from different sources adjusting for different levels of validity, but these methods are not established and need high epidemiological and statistical expertise. A framework for deciding whether additional evidence is needed to reduce uncertainty regarding safety, clinical effectiveness and cost-effectiveness will be adapted to MD. The whole process of evidence generation before and after market access has to be considered to provide an environment for conclusive HTA recommendations informing health care decision making. In Joint Action 2, EUnetHTA develops transparent processes for the early dialogue with stakeholders and fosters dissemination of appropriate HTA methods. In the case of MD, there are special accumulated needs for such efforts.