Real World Data in Health Technology Assessment of Complex Health Technologies

How can health technology assessment be improved to optimise access to medicines? Results from a Delphi study in Europe : Better access to medicines through HTA

INTRODUCTION

Access to medicines is a shared goal across healthcare stakeholders. Since health technology assessment (HTA) informs funding decisions, it shapes access to medicines. Despite its wide implementation, significant access variations due to HTA are observed across Europe. This paper elicited the opinions of European stakeholders on how HTA can be improved to facilitate access.

METHODS

A scoping review identified HTA features that influence access to medicines within markets and areas for improvement, while three access dimensions were identified (availability, affordability, timeliness). Using the Delphi method, we elicited the opinions of European stakeholders to validate the literature findings.

RESULTS

Nineteen participants from 14 countries participated in the Delphi panel. Thirteen HTA features that could be improved to optimise access to medicines in Europe were identified. Of these, 11 recorded a positive impact on at least one of the three access dimensions. HTA features had mostly a positive impact on timeliness and a less clear impact on affordability. 'Early scientific advice' and 'clarity in evidentiary requirements' showed a positive impact on all access dimensions. 'Established ways to deal with uncertainty during HTA' could improve medicines' availability and timeliness, while more 'reliance on real-world evidence' could expedite time to market access.

CONCLUSIONS

Our results reiterate that increased transparency during HTA and the decision-making processes is essential; the use of and reliance on new evidence generation such as real-world evidence can optimise the availability of medicines; and better collaborations between regulatory institutions within and between countries are paramount for better access to medicines.

Evidence Quality and Health Technology Assessment Outcomes in Reappraisals of Drugs for Rare Diseases in Germany

OBJECTIVES

Evidence on reappraisals of health technologies in Germany is limited, and for rare disease treatments (RDTs), the Federal Joint Committee follows different processes (limited or regular), depending on whether an annual revenue threshold has been exceeded. Our objective is to better understand (re)appraisal processes and their outcomes for RDTs in Germany.

METHODS

We analyzed appraisal documents of 55 RDT indications for which an initial appraisal and a reappraisal were conducted between 2011 and 2023. We extracted information for the type of evidence, the risk of bias, the availability of additional evidence, and the change in the maturity of survival data as proxies for evidence quality. Specifically, we reviewed the reasons for conducting reappraisals, examined how evidence quality and the clinical benefit rating (CBR) differed between initial appraisals and reappraisals, and explored the association between evidence quality and (1) the CBR and (2) the change in the CBR after reappraisal.

RESULTS

Most reappraisals were conducted because the annual revenue threshold was exceeded or the initial appraisal resolution was time limited. Almost all initial appraisals used the limited process, whereas the majority of reappraisals used the regular process. The CBR increased in only 9 and decreased in 21 of 55 reappraisals. There was some evidence that reappraisals with an accepted randomized controlled trial were significantly more likely to achieve a higher CBR.

CONCLUSIONS

Findings confirmed that reasons and processes for conducting reappraisals of RDTs in Germany differ. Further, high CBRs in reappraisals were not common and evidence quality in initial appraisals and reappraisals was limited.

Suitability of the Current Health Technology Assessment of Innovative Artificial Intelligence-Based Medical Devices: Scoping Literature Review

BACKGROUND

Artificial intelligence (AI)-based medical devices have garnered attention due to their ability to revolutionize medicine. Their health technology assessment framework is lacking.

OBJECTIVE

This study aims to analyze the suitability of each health technology assessment (HTA) domain for the assessment of AI-based medical devices.

METHODS

We conducted a scoping literature review following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology. We searched databases (PubMed, Embase, and Cochrane Library), gray literature, and HTA agency websites.

RESULTS

A total of 10.1% (78/775) of the references were included. Data quality and integration are vital aspects to consider when describing and assessing the technical characteristics of AI-based medical devices during an HTA process. When it comes to implementing specialized HTA for AI-based medical devices, several practical challenges and potential barriers could be highlighted and should be taken into account (AI technological evolution timeline, data requirements, complexity and transparency, clinical validation and safety requirements, regulatory and ethical considerations, and economic evaluation).

CONCLUSIONS

The adaptation of the HTA process through a methodological framework for AI-based medical devices enhances the comparability of results across different evaluations and jurisdictions. By defining the necessary expertise, the framework supports the development of a skilled workforce capable of conducting robust and reliable HTAs of AI-based medical devices. A comprehensive adapted HTA framework for AI-based medical devices can provide valuable insights into the effectiveness, cost-effectiveness, and societal impact of AI-based medical devices, guiding their responsible implementation and maximizing their benefits for patients and health care systems.

Acceptability of Using Real-World Data to Estimate Relative Treatment Effects in Health Technology Assessments: Barriers and Future Steps

OBJECTIVES

Evidence about the comparative effects of new treatments is typically collected in randomized controlled trials (RCTs). In some instances, RCTs are not possible, or their value is limited by an inability to capture treatment effects over the longer term or in all relevant population subgroups. In these cases, nonrandomized studies (NRS) using real-world data (RWD) are increasingly used to complement trial evidence on treatment effects for health technology assessment (HTA). However, there have been concerns over a lack of acceptability of this evidence by HTA agencies. This article aims to identify the barriers to the acceptance of NRS and steps that may facilitate increases in the acceptability of NRS in the future.

METHODS

Opinions of the authorship team based on their experience in real-world evidence research in academic, HTA, and industry settings, supported by a critical assessment of existing studies.

RESULTS

Barriers were identified that are applicable to key stakeholder groups, including HTA agencies (eg, the lack of comprehensive methodological guidelines for using RWD), evidence generators (eg, avoidable deviations from best practices), and external stakeholders (eg, data controllers providing timely access to high-quality RWD). Future steps that may facilitate future acceptability of NRS include improvements in the quality, integration, and accessibility of RWD, wider use of demonstration projects to highlight the value and applicability of nonrandomized designs, living, and more detailed HTA guidelines, and improvements in HTA infrastructure relating to RWD.

CONCLUSION

NRS can represent a crucial source of evidence on treatment effects for use in HTA when RCT evidence is limited.

Cross-sectional analysis of use of real-world data in single technology appraisals of oncological medicine by the National Institute for Health and Care Excellence in 2011-2021

OBJECTIVES

This study aims to identify how real-world data (RWD) have been used in single technology appraisals (STAs) of cancer drugs by the National Institute for Health and Care Excellence (NICE).

DESIGN

Cross-sectional study of NICE technology appraisals of cancer drugs for which guidance was issued between January 2011 and December 2021 (n=229). The appraisals were reviewed following a published protocol to extract the data about the use of RWD. The use of RWD was analysed by reviewing the specific ways in which RWD were used and by identifying different patterns of use.

PRIMARY OUTCOME MEASURE

The number of appraisals where RWD are used in the economic modelling.

RESULTS

Most appraisals used RWD in their economic models. The parametric use of RWD was commonly made in the economic models (76% of the included appraisals), whereas non-parametric use was less common (41%). Despite widespread use of RWD, there was no dominant pattern of use. Three sources of RWD (registries, administrative data, chart reviews) were found across the three important parts of the economic model (choice of comparators, overall survival and volume of treatment).

CONCLUSIONS

NICE has had a long-standing interest in the use of RWD in STAs. A systematic review of oncology appraisals suggests that RWD have been widely used in diverse parts of the economic models. Between 2011 and 2021, parametric use was more commonly found in economic models than non-parametric use. Nonetheless, there was no clear pattern in the way these data were used. As each appraisal involves a different decision problem and the ability of RWD to provide the information required for the economic modelling varies, appraisals will continue to differ with respect to their use of RWD.

Real-world data: a comprehensive literature review on the barriers, challenges, and opportunities associated with their inclusion in the health technology assessment process

Objective: This review aimed to assess the current use and acceptance of real-world data (RWD) and real-world evidence (RWE) in health technology assessment (HTA) process. It additionally aimed to discern stakeholders' viewpoints concerning RWD and RWE in HTA and illuminate the obstacles, difficulties, prospects, and consequences associated with the incorporation of RWD and RWE into the realm of HTA. Methods: A comprehensive PRISMA-based systematic review was performed in July 2022 in PubMed/Medline, Scopus, IDEAS-RePEc, International HTA database, and Centre for Reviews and Dissemination with ad hoc supplementary search in Google Scholar and international organization websites. The review included pre-determined inclusion criteria while the selection of eligible studies, the data extraction process and quality assessment were carried out using standardized and transparent methods. Results: Twenty-nine (n = 29) studies were included in the review out of 2,115 studies identified by the search strategy. In various global contexts, disparities in RWD utilization were evident, with randomized controlled trials (RCTs) serving as the primary evidence source. RWD and RWE played pivotal roles, surpassing relative effectiveness assessments (REAs) and significantly influencing decision-making and cost-effectiveness analyses. Identified challenges impeding RWD integration into HTA encompassed limited local data access, complexities in non-randomized trial design, data quality, privacy, and fragmentation. Addressing these is imperative for optimal RWD utilization. Incorporating RWD/RWE in HTA yields multifaceted advantages, enhancing understanding of treatment efficacy, resource utilization, and cost analysis, particularly via patient registries. RWE complements assessments of advanced therapy medicinal products (ATMPs) and rare diseases. Local data utilization strengthens HTA, bridging gaps when RCT data is lacking. RWD aids medical device decision-making, cancer drug reassessment, and indirect treatment comparisons. Challenges include data availability, stakeholder acceptance, expertise, and privacy. However, standardization, training, collaboration, and guidance can surmount these barriers, fostering enhanced RWD utilization in HTA. Conclusion: This study highlights the intricate global landscape of RWD and RWE acceptance in HTA. Recognizing regional nuances, addressing methodological challenges, and promoting collaboration are pivotal, among others, for leveraging RWD and RWE effectively in healthcare decision-making.

Use of Healthcare Claims Data to Generate Real-World Evidence on Patients With Drug-Resistant Epilepsy: Practical Considerations for Research

Objectives: Regulatory bodies, health technology assessment agencies, payers, physicians, and other decision-makers increasingly recognize the importance of real-world evidence (RWE) to provide important and relevant insights on treatment patterns, burden/cost of illness, product safety, and long-term and comparative effectiveness. However, RWE generation requires a careful approach to ensure rigorous analysis and interpretation. There are limited examples of comprehensive methodology for the generation of RWE on patients who have undergone neuromodulation for drug-resistant epilepsy (DRE). This is likely due, at least in part, to the many challenges inherent in using real-world data to define DRE, neuromodulation (including type implanted), and related outcomes of interest. We sought to provide recommendations to enable generation of robust RWE that can increase knowledge of "real-world" patients with DRE and help inform the difficult decisions regarding treatment choices and reimbursement for this particularly vulnerable population. Methods: We drew upon our collective decades of experience in RWE generation and relevant disciplines (epidemiology, health economics, and biostatistics) to describe challenges inherent to this therapeutic area and to provide potential solutions thereto within healthcare claims databases. Several examples were provided from our experiences in DRE to further illustrate our recommendations for generation of robust RWE in this therapeutic area. Results: Our recommendations focus on considerations for the selection of an appropriate data source, development of a study timeline, exposure allotment (specifically, neuromodulation implantation for patients with DRE), and ascertainment of relevant outcomes. Conclusions: The need for RWE to inform healthcare decisions has never been greater and continues to grow in importance to regulators, payers, physicians, and other key stakeholders. However, as real-world data sources used to generate RWE are typically generated for reasons other than research, rigorous methodology is required to minimize bias and fully unlock their value.

Tools for assessing quality of studies investigating health interventions using real-world data: a literature review and content analysis

OBJECTIVES

We aimed to identify existing appraisal tools for non-randomised studies of interventions (NRSIs) and to compare the criteria that the tools provide at the quality-item level.

DESIGN

Literature review through three approaches: systematic search of journal articles, snowballing search of reviews on appraisal tools and grey literature search on websites of health technology assessment (HTA) agencies.

DATA SOURCES

Systematic search: Medline; Snowballing: starting from three articles (D'Andrea et al, Quigley et al and Faria et al); Grey literature: websites of European HTA agencies listed by the International Network of Agencies for Health Technology Assessment. Appraisal tools were searched through April 2022.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

We included a tool, if it addressed quality concerns of NRSIs and was published in English (unless from grey literature). A tool was excluded, if it was only for diagnostic, prognostic, qualitative or secondary studies.

DATA EXTRACTION AND SYNTHESIS

Two independent researchers searched, screened and reviewed all included studies and tools, summarised quality items and scored whether and to what extent a quality item was described by a tool, for either methodological quality or reporting.

RESULTS

Forty-nine tools met inclusion criteria and were included for the content analysis. Concerns regarding the quality of NRSI were categorised into 4 domains and 26 items. The Research Triangle Institute Item Bank (RTI Item Bank) and STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) were the most comprehensive tools for methodological quality and reporting, respectively, as they addressed (n=20; 17) and sufficiently described (n=18; 13) the highest number of items. However, none of the tools covered all items.

CONCLUSION

Most of the tools have their own strengths, but none of them could address all quality concerns relevant to NRSIs. Even the most comprehensive tools can be complemented by several items. We suggest decision-makers, researchers and tool developers consider the quality-item level heterogeneity, when selecting a tool or identifying a research gap.

OSF REGISTRATION NUMBER

OSF registration DOI (https://doi.org/10.17605/OSF.IO/KCSGX).

Transporting Comparative Effectiveness Evidence Between Countries: Considerations for Health Technology Assessments

Internal validity is often the primary concern for health technology assessment agencies when assessing comparative effectiveness evidence. However, the increasing use of real-world data from countries other than a health technology assessment agency's target population in effectiveness research has increased concerns over the external validity, or "transportability", of this evidence, and has led to a preference for local data. Methods have been developed to enable a lack of transportability to be addressed, for example by accounting for cross-country differences in disease characteristics, but their consideration in health technology assessments is limited. This may be because of limited knowledge of the methods and/or uncertainties in how best to utilise them within existing health technology assessment frameworks. This article aims to provide an introduction to transportability, including a summary of its assumptions and the methods available for identifying and adjusting for a lack of transportability, before discussing important considerations relating to their use in health technology assessment settings, including guidance on the identification of effect modifiers, guidance on the choice of target population, estimand, study sample and methods, and how evaluations of transportability can be integrated into health technology assessment submission and decision processes.

Advancing the use of real world evidence in health technology assessment: insights from a multi-stakeholder workshop

Introduction: Real-world evidence (RWE) in health technology assessment (HTA) holds significant potential for informing healthcare decision-making. A multistakeholder workshop was organised by the European Health Data and Evidence Network (EHDEN) and the GetReal Institute to explore the status, challenges, and opportunities in incorporating RWE into HTA, with a focus on learning from regulatory initiatives such as the European Medicines Agency (EMA) Data Analysis and Real World Interrogation Network (DARWIN EU®). Methods: The workshop gathered key stakeholders from regulatory agencies, HTA organizations, academia, and industry for three panel discussions on RWE and HTA integration. Insights and recommendations were collected through panel discussions and audience polls. The workshop outcomes were reviewed by authors to identify key themes, challenges, and recommendations. Results: The workshop discussions revealed several important findings relating to the use of RWE in HTA. Compared with regulatory processes, its adoption in HTA to date has been slow. Barriers include limited trust in RWE, data quality concerns, and uncertainty about best practices. Facilitators include multidisciplinary training, educational initiatives, and stakeholder collaboration, which could be facilitated by initiatives like EHDEN and the GetReal Institute. Demonstrating the impact of "driver projects" could promote RWE adoption in HTA. Conclusion: To enhance the integration of RWE in HTA, it is crucial to address known barriers through comprehensive training, stakeholder collaboration, and impactful exemplar research projects. By upskilling users and beneficiaries of RWE and those that generate it, promoting collaboration, and conducting "driver projects," can strengthen the HTA evidence base for more informed healthcare decisions.

Comparative effectiveness and cost-effectiveness of natalizumab and fingolimod in rapidly evolving severe relapsing-remitting multiple sclerosis in the United Kingdom

AIM

To evaluate the real-world comparative effectiveness and the cost-effectiveness, from a UK National Health Service perspective, of natalizumab versus fingolimod in patients with rapidly evolving severe relapsing-remitting multiple sclerosis (RES-RRMS).

METHODS

Real-world data from the MSBase Registry were obtained for patients with RES-RRMS who were previously either naive to disease-modifying therapies or had been treated with interferon-based therapies, glatiramer acetate, dimethyl fumarate, or teriflunomide (collectively known as BRACETD). Matched cohorts were selected by 3-way multinomial propensity score matching, and the annualized relapse rate (ARR) and 6-month-confirmed disability worsening (CDW6M) and improvement (CDI6M) were compared between treatment groups. Comparative effectiveness results were used in a cost-effectiveness model comparing natalizumab and fingolimod, using an established Markov structure over a lifetime horizon with health states based on the Expanded Disability Status Scale. Additional model data sources included the UK MS Survey 2015, published literature, and publicly available sources.

RESULTS

In the comparative effectiveness analysis, we found a significantly lower ARR for patients starting natalizumab compared with fingolimod (rate ratio [RR] = 0.65; 95% confidence interval [CI], 0.57-0.73) or BRACETD (RR = 0.46; 95% CI, 0.42-0.53). Similarly, CDI6M was higher for patients starting natalizumab compared with fingolimod (hazard ratio [HR] = 1.25; 95% CI, 1.01-1.55) and BRACETD (HR = 1.46; 95% CI, 1.16-1.85). In patients starting fingolimod, we found a lower ARR (RR = 0.72; 95% CI, 0.65-0.80) compared with starting BRACETD, but no difference in CDI6M (HR = 1.17; 95% CI, 0.91-1.50). Differences in CDW6M were not found between the treatment groups. In the base-case cost-effectiveness analysis, natalizumab dominated fingolimod (0.302 higher quality-adjusted life-years [QALYs] and £17,141 lower predicted lifetime costs). Similar cost-effectiveness results were observed across sensitivity analyses.

CONCLUSIONS

This MSBase Registry analysis suggests that natalizumab improves clinical outcomes when compared with fingolimod, which translates to higher QALYs and lower costs in UK patients with RES-RRMS.

Integration of real-world evidence from different data sources in health technology assessment

Real-world evidence (RWE) is being increasingly used by a wide range of stakeholders involved in the therapeutic product lifecycle but remains underutilized in the health technology assessment (HTA) process. RWE aims to fill the current evidence gaps, reduce the uncertainty around the benefits of medical technologies, and better understand the long-term impact of health technologies in real-world conditions. Despite the minimal use of RWE in some elements of HTA, there has been a larger push to further utilize RWE in the HTA processes. HTA bodies, as other stakeholders, work towards developing more robust means to leverage RWE from various data sources in the HTA processes. However, these agencies need to overcome important challenges before the broader incorporation of RWE into their routine practice. This paper aims to explore the extensive integration of RWE utilizing diverse sources of RWD. We discuss the utilization of RWE in HTA processes, considering aspects such as when, where, and how RWE can be effectively applied. Additionally, we seek the potential challenges and barriers associated with the utilization of different data sources.

Guidance on using real-world evidence from Western Europe in Central and Eastern European health policy decision making

Aim: Real-world data and real-world evidence (RWE) are becoming more important for healthcare decision making and health technology assessment. We aimed to propose solutions to overcome barriers preventing Central and Eastern European (CEE) countries from using RWE generated in Western Europe. Materials & methods: To achieve this, following a scoping review and a webinar, the most important barriers were selected through a survey. A workshop was held with CEE experts to discuss proposed solutions. Results: Based on survey results, we selected the nine most important barriers. Multiple solutions were proposed, for example, the need for a European consensus, and building trust in using RWE. Conclusion: Through collaboration with regional stakeholders, we proposed a list of solutions to overcome barriers on transferring RWE from Western Europe to CEE countries.

R WE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 10

In this latest update we discuss the transportability of comparative effectiveness evidence across countries. We highlight results of a survey indicating that European HTA agencies are reluctant to accept real-world data from other countries, review recent benefit assessments indicating a potential softening of a requirement for the use of local real-world data in Germany, and outline a recent review presenting approaches that can correct for a lack of transportability.

Transferability of real-world data across borders for regulatory and health technology assessment decision-making

Recently, there has been increased consideration of real-world data (RWD) and real-world evidence (RWE) in regulatory and health technology assessment (HTA) decision-making. Due to challenges in identifying high-quality and relevant RWD sources, researchers and regulatory/HTA bodies may turn to RWD generated in locales outside of the locale of interest (referred to as "transferring RWD"). We therefore performed a review of stakeholder guidance as well as selected case studies to identify themes for researchers to consider when transferring RWD from one jurisdiction to another. Our review highlighted that there is limited consensus on defining decision-grade, transferred RWD; certain stakeholders have issued relevant guidance, but the recommendations are high-level and additional effort is needed to generate comprehensive guidance. Additionally, the case studies revealed that RWD transferability has not been a consistent concern for regulatory/HTA bodies and that more focus has been put on the evaluation of internal validity. To help develop transferability best practices (alongside internal validity best practices), we suggest that researchers address the following considerations in their justification for transferring RWD: treatment pathways, nature of the healthcare system, incidence/prevalence of indication, and patient demographics. We also recommend that RWD transferability should garner more attention as the use of imported RWD could open doors to high-quality data sources and potentially reduce methodological issues that often arise in the use of local RWD; we thus hope this review provides a foundation for further dialogue around the suitability and utility of transferred RWD in the regulatory/HTA decision-making space.