Organized structure of real-world evidence best practices: moving from fragmented recommendations to comprehensive guidance

Real world data on digital remote refraction in a healthy population of 14,680 eyes

Refractive errors are the leading cause of preventable visual impairment, to which web-based remote refraction could contribute. We report real-world 2021-2022 data of the underlying algorithm and validated these to conventional prescriptions among healthy individuals (high visual acuity and satisfactied current refraction). Participants were 18-45 years with a spherical (S) error between -3.50 + 2.00S to -2.00 Diopter Cylinder (DC), reported as Spherical Equivalent (SEQ) in mean differences and 95% Limits of agreement. Consecutive measurements (n = 14,680) were assessed of which n = 6386 selected for validation. The mean difference was 0.01D(SD 0.69) and -0.73D(SD 0.92) for myopes and hyperopes respectively. This algorithm shows variation, nonetheless, 67% and 82% of myopes were within ±0.5 and ±0.75D. The test underestimates hyperopes (34% and 50% within ±0.5D and ±0.75D) and had inconsistencies distinguishing hyperopia. This proof-of-concept shows home testing has the potency to increase accessibility to care by delivering a valuable alternative for uncomplicated refractive assessments.

Next-generation sequencing in oncology: challenges in economic evaluations

INTRODUCTION

Next-generation sequencing (NGS) identifies genetic variants to inform personalized treatment plans. Insufficient evidence of cost-effectiveness impedes the integration of NGS into routine cancer care. The complexity of personalized treatment challenges conventional economic evaluation. Clearly delineating challenges informs future cost-effectiveness analyses to better value and contextualize health, preference-, and equity-based outcomes.

AREAS COVERED

We conducted a scoping review to characterize the applied methods and outcomes of economic evaluations of NGS in oncology and identify existing challenges. We included 27 articles published since 2016 from a search of PubMed, Embase, and Web of Science. Identified challenges included defining the evaluative scope, managing evidentiary limitations including lack of causal evidence, incorporating preference-based utility, and assessing distributional and equity-based impacts. These challenges reflect the difficulty of generating high-quality clinical effectiveness and real-world evidence (RWE) for NGS-guided interventions.

EXPERT OPINION

Adapting methodological approaches and developing life-cycle health technology assessment (HTA) guidance using RWE is crucial for implementing NGS in oncology. Healthcare systems, decision-makers, and HTA organizations are facing a pivotal opportunity to adapt to an evolving clinical paradigm and create innovative regulatory and reimbursement processes that will enable more sustainable, equitable, and patient-oriented healthcare.

Use of transportability methods for real-world evidence generation: a review of current applications

Aim: To evaluate how transportability methods are currently used for real-world evidence (RWE) generation to inform good practices and support adoption and acceptance of these methods in the RWE context. Methods: We conducted a targeted literature review to identify studies that transported an effect estimate of the clinical effectiveness or safety of a biomedical exposure to a target real-world population. Records were identified from PubMed-indexed articles published any time before 25 July 2023 (inclusive). Two reviewers screened abstracts/titles and reviewed the full text of candidate studies to identify the final set of articles. Data on the therapeutic area, exposure(s), outcome(s), original and target populations and details of the transportability analysis (e.g., analytic method used, estimate transported, stated assumptions) were abstracted from each article. Results: Of 458 unique records identified, six were retained in the final review. Articles were published during 2021-2023, focused on the US/Canada context, and covered a range of therapeutic areas. Four studies transported an RCT effect estimate, while two transported effect estimates derived from real-world data. Almost all articles used weighting methods to transport estimates. Two studies discussed all transportability assumptions, and one evaluated the likelihood of meeting all assumptions and the impact of potential violations. Conclusion: The use of transportability methods for RWE generation is an emerging and promising area of research to address evidence gaps in settings with limited data and infrastructure. More transparent and rigorous reporting of methods, assumptions and limitations may increase the use and acceptability of transportability for producing robust evidence on treatment effectiveness and safety.

FOUNTAIN: a modular research platform for integrated real-world evidence generation

BACKGROUND

Real-world evidence (RWE) plays a key role in regulatory and healthcare decision-making, but the potentially fragmentated nature of generated evidence may limit its utility for clinical decision-making. Heterogeneity and a lack of reproducibility in RWE resulting from inconsistent application of methodologies across data sources should be minimized through harmonization.

METHODS

This paper's aim is to describe and reflect upon a multidisciplinary research platform (FOUNTAIN; FinerenOne mUlti-database NeTwork for evidence generAtIoN) with coordinated studies using diverse RWE generation approaches and explore the platform's strengths and limitations. With guidance from an executive advisory committee of multidisciplinary experts and patient representatives, the goal of the FOUNTAIN platform is to harmonize RWE generation across a portfolio of research projects, including research partner collaborations and a common data model (CDM)-based program. FOUNTAIN's overarching objectives as a research platform are to establish long-term collaborations among pharmacoepidemiology research partners and experts and to integrate diverse approaches for RWE generation, including global protocol execution by research partners in local data sources and common protocol execution in multiple data sources through federated data networks, while ensuring harmonization of medical definitions, methodology, and reproducible artifacts across all studies. Specifically, the aim of the multiple studies run within the frame of FOUNTAIN is to provide insight into the real-world utilization, effectiveness, and safety of finerenone across its life-cycle.

RESULTS

Currently, the FOUNTAIN platform includes 9 research partner collaborations and 8 CDM-mapped data sources from 7 countries (United States, United Kingdom, China, Japan, The Netherlands, Spain, and Denmark). These databases and research partners were selected after a feasibility fit-for-purpose evaluation. Six multicountry, multidatabase, cohort studies are ongoing to describe patient populations, current standard of care, comorbidity profiles, healthcare resource use, and treatment effectiveness and safety in different patient populations with chronic kidney disease and type 2 diabetes. Strengths and potential limitations of FOUNTAIN are described in the context of valid RWE generation.

CONCLUSION

The establishment of the FOUNTAIN platform has allowed harmonized execution of multiple studies, promoting consistency both within individual studies that employ multiple data sources and across all studies run within the platform's framework. FOUNTAIN presents a proposal to efficiently improve the consistency and generalizability of RWE on finerenone.

The maze of real-world evidence frameworks: from a desert to a jungle! An environmental scan and comparison across regulatory and health technology assessment agencies

Aim: Regulatory and health technology assessment (HTA) agencies have increasingly published frameworks, guidelines, and recommendations for the use of real-world evidence (RWE) in healthcare decision-making. Variations in the scope and content of these documents, with updates running in parallel, may create challenges for their implementation especially during the market authorization and reimbursement phases of a medicine's life cycle. This environmental scan aimed to comprehensively identify and summarize the guidance documents for RWE developed by most well-established regulatory and reimbursement agencies, as well as other organizations focused on healthcare decision-making, and present their similarities and differences. Methods: RWE guidance documents, including white papers from regulatory and HTA agencies, were reviewed in March 2024. Data on scope and recommendations from each body were extracted by two reviewers and similarities and differences were summarized across four topics: study planning, choosing fit-for-purpose data, study conduct, and reporting. Post-authorization or non-pharmacological guidance was excluded. Results: Forty-six documents were identified across multiple agencies; US FDA produced the most RWE-related guidance. All agencies addressed specific and often similar methodological issues related to study design, data fitness-for-purpose, reliability, and reproducibility, although inconsistency in terminologies on these topics was noted. Two HTA bodies (National Institute for Health and Care Excellence [NICE] and Canada's Drug Agency) each centralized all related RWE guidance under a unified framework. RWE quality tools and checklists were not consistently named and some differences in preferences were noted. European Medicines Agency, NICE, Haute Autorité de Santé, and the Institute for Quality and Efficiency in Health Care included specific recommendations on the use of analytical approaches to address RWE complexities and increase trust in its findings. Conclusion: Similarities in agencies' expectations on RWE studies design, quality elements, and reporting will facilitate evidence generation strategy and activities for manufacturers facing multiple, including global, regulatory and reimbursement submissions and re-submissions. A strong preference by decision-making bodies for local real-world data generation may hinder opportunities for data sharing and outputs from international federated data networks. Closer collaboration between decision-making agencies towards a harmonized RWE roadmap, which can be centrally preserved in a living mode, will provide manufacturers and researchers clarity on minimum acceptance requirements and expectations, especially as novel methodologies for RWE generation are rapidly emerging.

UK Stakeholder Perspectives on Surrogate Endpoints in Cancer, and the Potential for UK Real-World Datasets to Validate Their Use in Decision-Making

Duration of overall survival in patients with cancer has lengthened due to earlier detection and improved treatments. However, these improvements have created challenges in assessing the impact of newer treatments, particularly those used early in the treatment pathway. As overall survival remains most decision-makers' preferred primary endpoint, therapeutic innovations may take a long time to be introduced into clinical practice. Moreover, it is difficult to extrapolate findings to heterogeneous populations and address the concerns of patients wishing to evaluate everyday quality and extension of life. There is growing interest in the use of surrogate or interim endpoints to demonstrate robust treatment effects sooner than is possible with measurement of overall survival. It is hoped that they could speed up patients' access to new drugs, combinations, and sequences, and inform treatment decision-making. However, while surrogate endpoints have been used by regulators for drug approvals, this has occurred on a case-by-case basis. Evidence standards are yet to be clearly defined for acceptability in health technology appraisals or to shape clinical practice. This article considers the relevance of the use of surrogate endpoints in cancer in the UK context, and explores whether collection and analysis of real-world UK data and evidence might contribute to validation.

Real-World Evidence to Inform Regulatory Decision Making: A Scoping Review

Real-world evidence (RWE) is increasingly considered in regulatory decision making. When, and to which extent, RWE is considered relevant by regulators likely depends on many factors. This review aimed to identify factors that make RWE necessary or desirable to inform regulatory decision making. A scoping review was conducted using literature databases (PubMed, Embase, Emcare, Web of Science, and Cochrane Library) and websites of regulatory agencies, health technology assessment agencies, research institutes, and professional organizations involved with RWE. Articles were included if: (1) they discussed factors or contexts that impact whether RWE could be necessary or desirable in regulatory decision making; (2) focused on pharmacological or biological interventions in humans; and (3) considered decision making in Europe or North America, or without a focus on a specific region. We included 118 articles in the scoping review. Two major themes and six subthemes were identified. The first theme concerns questions addressable with RWE, with subthemes epidemiology and benefit-risk assessment. The second theme concerns contextual factors, with subthemes feasibility, ethical considerations, limitations of available evidence, and disease and treatment-specific aspects. Collectively, these themes encompassed 43 factors influencing the need for RWE in regulatory decisions. Although single factors may not make RWE fully necessary, their cumulative influence could make RWE essential and pivotal in regulatory decision making. This overview contributes to ongoing discussions emphasizing the nuanced interplay of factors influencing the necessity or desirability of RWE to inform regulatory decision making.

Acceptability of Using Real-World Data to Estimate Relative Treatment Effects in Health Technology Assessments: Barriers and Future Steps

OBJECTIVES

Evidence about the comparative effects of new treatments is typically collected in randomized controlled trials (RCTs). In some instances, RCTs are not possible, or their value is limited by an inability to capture treatment effects over the longer term or in all relevant population subgroups. In these cases, nonrandomized studies (NRS) using real-world data (RWD) are increasingly used to complement trial evidence on treatment effects for health technology assessment (HTA). However, there have been concerns over a lack of acceptability of this evidence by HTA agencies. This article aims to identify the barriers to the acceptance of NRS and steps that may facilitate increases in the acceptability of NRS in the future.

METHODS

Opinions of the authorship team based on their experience in real-world evidence research in academic, HTA, and industry settings, supported by a critical assessment of existing studies.

RESULTS

Barriers were identified that are applicable to key stakeholder groups, including HTA agencies (eg, the lack of comprehensive methodological guidelines for using RWD), evidence generators (eg, avoidable deviations from best practices), and external stakeholders (eg, data controllers providing timely access to high-quality RWD). Future steps that may facilitate future acceptability of NRS include improvements in the quality, integration, and accessibility of RWD, wider use of demonstration projects to highlight the value and applicability of nonrandomized designs, living, and more detailed HTA guidelines, and improvements in HTA infrastructure relating to RWD.

CONCLUSION

NRS can represent a crucial source of evidence on treatment effects for use in HTA when RCT evidence is limited.

Real-world evidence for regulatory decision-making: updated guidance from around the world

Leveraging the value of real-world evidence (RWE) to make informed regulatory decisions in the field of health care continues to gain momentum. Improving clinical evidence generation by evaluating the outcomes and patient experiences at the point-of-care would help achieve the ultimate aim of ensuring that effective and safe treatments are rapidly approved for patient use. In our previous publication, we assessed the global regulatory landscape with respect to RWE and provided a review of the regional availability of frameworks and guidance through May 2021 on the basis of 3 key regulatory elements: regulatory RWE frameworks, data quality guidance, and study methods guidance. In the current review, we have updated and elaborated upon recent developments in the regulatory RWE environment from a regional perspective under the same 3 regulatory elements stated above. In addition, we have also included a new category on procedural guidance. The review also discusses the perceived gaps and potential opportunities for future development and harmonization in this field to support framework establishment in regions without pre-existing RWE policies. Additionally, the article reviews current developments of health technology assessment (HTA) bodies pertaining to RWE and discusses the status of evidentiary alignment among regulators and HTA agencies.

Regulatory and HTA Considerations for Development of Real-World Data Derived External Controls

Regulators and Health Technology Assessment (HTA) bodies are increasingly familiar with, and publishing guidance on, external controls derived from real-world data (RWD) to generate real-world evidence (RWE). We recently conducted a systematic literature review (SLR) evaluating publicly available information on the use of RWD-derived external controls to contextualize outcomes from uncontrolled trials submitted to the European Medicines Agency (EMA), the US Food and Drug Administration (FDA), and/or select HTA bodies. The review identified several key operational and methodological aspects for which more detailed guidance and alignment within and between regulatory agencies and HTA bodies is necessary. This paper builds on the SLR findings by delineating a set of key takeaways for the responsible generation of fit-for-purpose RWE. Practical methodological and operational guidelines for designing, conducting, and reporting RWD-derived external control studies are explored and discussed. These considerations include: (i) early engagement with regulators and HTA bodies during the study planning phase; (ii) consideration of the appropriateness and comparability of external controls across multiple dimensions, including eligibility criteria, temporality, population representation, and clinical evaluation; (iii) ensuring adequate sample sizes, including hypothesis testing considerations; (iv) implementation of a clear and transparent strategy for assessing and addressing data quality, including data missingness across trials and RWD; (v) selection of comparable and meaningful endpoints that are operationalized and analyzed using appropriate analytic methods; and (vi) conduct of sensitivity analyses to assess the robustness of findings in the context of uncertainty and sources of potential bias.

Effectively Leveraging RWD for External Controls: A Systematic Literature Review of Regulatory and HTA Decisions

Real-world data (RWD)-derived external controls can be used to contextualize efficacy findings for investigational therapies evaluated in uncontrolled trials. As the number of submissions to regulatory and health technology assessment (HTA) bodies using external controls rises, and in light of recent regulatory and HTA guidance on the appropriate use of RWD, there is a need to address the operational and methodological challenges impeding the quality of real-world evidence (RWE) generation and the consistency in evaluation of RWE across agencies. This systematic review summarizes publicly available information on the use of external controls to contextualize outcomes from uncontrolled trials for all indications from January 1, 2015, through August 20, 2021, that were submitted to the European Medicines Agency, the US Food and Drug Administration, and/or select major HTA bodies (National Institute for Health and Care Excellence (NICE), Haute Autorité de Santé (HAS), Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG), and Gemeinsamer Bundesausschuss (G-BA)). By systematically reviewing submissions to regulatory and HTA bodies in the context of recent guidance, this study provides quantitative and qualitative insights into how external control design and analytic choices may be viewed by different agencies in practice. The primary operational and methodological aspects identified for discussion include, but are not limited to, engagement of regulators and HTA bodies, approaches to handling missing data (a component of data quality), and selection of real-world endpoints. Continued collaboration and guidance to address these and other aspects will inform and assist stakeholders attempting to generate evidence using external controls.

Using Real-World Data in the Health Technology Assessment of Pharmaceuticals: Strengths, Difficulties, and a Pragmatic Way Forward

In the past decade, there have been increasing calls for greater use of real-world evidence (RWE) and data (RWD), with the explicit goal of enabling faster provision of effective medicines to patients in need. The push for decision makers to accept RWE is especially noticeable in the pursuit of regulatory approval, but RWE, particularly when used to estimate the relative effectiveness of interventions, is not always readily accepted by agencies responsible for reimbursement and pricing of new pharmaceuticals and, to a varying degree, is not accepted across jurisdictions. This lack of trust hampers the use of RWE despite a very large and growing literature base on the principles of how RWE should be used. In this article, we suggest an important part of the explanation of why this situation has arisen and make suggestions for its alleviation. Given that problems commonly arise that are particular to the question being asked and the data sources being used, general guidance on the principles of how to use RWD cannot cover all eventualities. Therefore, we are suggesting the creation of an archive, or repository, to record uses of RWD in support of decisions by funding bodies or their advisors. This article introduces a proposed, structured classification of decision types using RWE, around which evidence can be assembled in a curated source (RWD/RWE taxonomy) and thus facilitate judgments on when evidence is "good enough." This article is part of a series in a special issue of this journal that looks at the barriers to optimal use of RWE in health technology assessment and how to overcome them. We begin significantly to populate our "taxonomy" with examples in an accompanying article. We also propose recommendations for international standards of evaluating the acceptability of RWD governance practices.

Structure and Content of a Taxonomy to Support the Use of Real-World Evidence by Health Technology Assessment Practitioners and Healthcare Decision Makers

This is one of a series of articles that consider the barriers to optimal use of real-world evidence (RWE) in health technology assessment and how to overcome them. The work was performed as part of EUreccA 2025, in particular with the RWE workstream embodied within that collaboration. Elsewhere in this issue we described the reasoning and process that led us to develop practical tools to support RWE use, including this taxonomy and explained the methods used to do so. The taxonomy classifies questions that are typically addressed using real-world data in health technology assessment and the data sources typically used to address these questions. In this article, we describe the taxonomy itself. For as many of the pairings as possible, we have provided links to advice and methods on how to address the associated question using those data. We have also provided links to examples of RWE use in practical decision making to answer the questions posed. Our work is not complete, but we believe it is sufficient to demonstrate the value of such a taxonomy and information source if it is completed and curated as a "wiki" by the community that would use it.

What, Where, and How to Collect Real-World Data and Generate Real-World Evidence to Support Drug Reimbursement Decision-Making in Asia: A reflection Into the Past and A Way Forward

BACKGROUND

Globally, there is increasing interest in the use of real-world data (RWD) and real-world evidence (RWE) to inform health technology assessment (HTA) and reimbursement decision-making. Using current practices and case studies shared by eleven health systems in Asia, a non-binding guidance that seeks to align practices for generating and using RWD/RWE for decision-making in Asia was developed by the REAL World Data In ASia for HEalth Technology Assessment in Reimbursement (REALISE) Working Group, addressing a current gap and needs among HTA users and generators.

METHODS

The guidance document was developed over two face-to-face workshops, in addition to an online survey, a face-to-face interview and pragmatic search of literature. The specific focus was on what, where and how to collect RWD/ RWE.

RESULTS

All 11 REALISE member jurisdictions participated in the online survey and the first in-person workshop, 10 participated in the second in-person workshop, and 8 participated in the in-depth face-to-face interviews. The guidance document was iteratively reviewed by all working group members and the International Advisory Panel. There was substantial variation in: (a) sources and types of RWD being used in HTA, and (b) the relative importance and prioritization of RWE being used for policy-making. A list of national-level databases and other sources of RWD available in each country was compiled. A list of useful guidance on data collection, quality assurance and study design were also compiled.

CONCLUSION

The REALISE guidance document serves to align the collection of better quality RWD and generation of reliable RWE to ultimately inform HTA in Asia.

A Structured Process to Identify Fit-for-Purpose Study Design and Data to Generate Valid and Transparent Real-World Evidence for Regulatory Uses

Generating evidence from real-world data requires fit-for-purpose study design and data. In addition to validity, decision makers require transparency in the reasoning that underlies study design and data source decisions. The 2019 Structured Preapproval and Postapproval Comparative Study Design Framework to Generate Valid and Transparent Real-World Evidence (SPACE) and the 2021 Structured Process to Identify Fit-For-Purpose Data (SPIFD)-intended to be used together-provide a step-by-step guide to identify decision grade, fit-for-purpose study design and data. In this update (referred to as "SPIFD2" to encompass both the design and data aspects) we provide an update to these frameworks that combines the templates into one, more explicitly calls for articulation of the hypothetical target trial and sources of bias that may arise in the real-world emulation, and provides explicit references to the Structured Template and Reporting Tool for Real-World Evidence (STaRT-RWE) tables that we suggest using immediately after invoking the SPIFD2 framework. Following the steps recommended in the SPIFD2 process requires due diligence on the part of the researcher to ensure that every aspect of study design and data selection is rationalized and supported by evidence. The resulting stepwise documentation enables reproducibility and clear communication with decision makers, and it increases the likelihood that the evidence generated is valid, fit-for-purpose, and sufficient to support healthcare and regulatory decisions.

Real-world data in oncology: a questionnaire-based analysis of the academic research landscape examining the policies and experiences of the cancer cooperative groups

BACKGROUND

Real-world data (RWD) have quickly emerged as an important source of information to address uncertainties about new treatments, including novel anticancer therapies. Many stakeholders are using such data and the evidence derived therefrom to answer the questions that remain about the safety and effectiveness of antitumor medicines after their approval by regulators. Our objective was to investigate the academic RWD study landscape and explore to what extent RWD are being integrated into investigator-initiated clinical research.

MATERIALS AND METHODS

We designed an online survey that was distributed between May and August 2022 to representatives of cancer cooperative groups active in Europe, North America, South America, Asia, and/or Oceania.

RESULTS

In total, 125 cooperative groups operating in 58 different countries and conducting research across 13 distinct cancer domains participated in the survey. While most of the responders (67.2%) did not have a formal policy in place to gather and utilize RWD, a majority (68.0%) had carried out studies involving the analysis of such data before, both for exploratory and confirmatory purposes. The groups that were experienced in capturing and interpreting RWD had mainly worked with observational RWD that were not predominantly prospective or retrospective in nature and which originated from disease registries, electronic health records, and patient questionnaires. They perceived the low costs and the large scale of RWD research to be its most significant benefits, and viewed the accompanying methodological and operational challenges as its biggest constraints. However, they did not have a common understanding of what RWD were. Despite their experience with analyzing RWD, their research portfolio still primarily comprised traditional clinical trials; 62.5% of the groups that had never undertaken any RWD studies were nonetheless planning to initiate them in the future.

CONCLUSIONS

Cancer cooperative groups are already incorporating RWD studies into their research agendas, but still lack knowledge and expertise in this regard, and do not agree on what RWD are. The conduct of conventional clinical trials continues to be their priority.

An introduction to causal inference for pharmacometricians

As formal causal inference begins to play a greater role in disciplines that intersect with pharmacometrics, such as biostatistics, epidemiology, and artificial intelligence/machine learning, pharmacometricians may increasingly benefit from a basic fluency in foundational causal inference concepts. This tutorial seeks to orient pharmacometricians to three such fundamental concepts: potential outcomes, g-formula, and directed acyclic graphs (DAGs).

Transferability of real-world data across borders for regulatory and health technology assessment decision-making

Recently, there has been increased consideration of real-world data (RWD) and real-world evidence (RWE) in regulatory and health technology assessment (HTA) decision-making. Due to challenges in identifying high-quality and relevant RWD sources, researchers and regulatory/HTA bodies may turn to RWD generated in locales outside of the locale of interest (referred to as "transferring RWD"). We therefore performed a review of stakeholder guidance as well as selected case studies to identify themes for researchers to consider when transferring RWD from one jurisdiction to another. Our review highlighted that there is limited consensus on defining decision-grade, transferred RWD; certain stakeholders have issued relevant guidance, but the recommendations are high-level and additional effort is needed to generate comprehensive guidance. Additionally, the case studies revealed that RWD transferability has not been a consistent concern for regulatory/HTA bodies and that more focus has been put on the evaluation of internal validity. To help develop transferability best practices (alongside internal validity best practices), we suggest that researchers address the following considerations in their justification for transferring RWD: treatment pathways, nature of the healthcare system, incidence/prevalence of indication, and patient demographics. We also recommend that RWD transferability should garner more attention as the use of imported RWD could open doors to high-quality data sources and potentially reduce methodological issues that often arise in the use of local RWD; we thus hope this review provides a foundation for further dialogue around the suitability and utility of transferred RWD in the regulatory/HTA decision-making space.

HARmonized Protocol Template to Enhance Reproducibility of Hypothesis Evaluating Real-World Evidence Studies on Treatment Effects: A Good Practices Report of a Joint ISPE/ISPOR Task Force

OBJECTIVES

Ambiguity in communication of key study parameters limits the utility of real-world evidence (RWE) studies in healthcare decision-making. Clear communication about data provenance, design, analysis, and implementation is needed. This would facilitate reproducibility, replication in independent data, and assessment of potential sources of bias.

METHODS

The International Society for Pharmacoepidemiology (ISPE) and ISPOR-The Professional Society for Health Economics and Outcomes Research (ISPOR) convened a joint task force, including representation from key international stakeholders, to create a harmonized protocol template for RWE studies that evaluate a treatment effect and are intended to inform decision-making. The template builds on existing efforts to improve transparency and incorporates recent insights regarding the level of detail needed to enable RWE study reproducibility. The over-arching principle was to reach for sufficient clarity regarding data, design, analysis, and implementation to achieve 3 main goals. One, to help investigators thoroughly consider, then document their choices and rationale for key study parameters that define the causal question (e.g., target estimand), two, to facilitate decision-making by enabling reviewers to readily assess potential for biases related to these choices, and three, to facilitate reproducibility.

STRATEGIES TO DISSEMINATE AND FACILITATE USE

Recognizing that the impact of this harmonized template relies on uptake, we have outlined a plan to introduce and pilot the template with key international stakeholders over the next 2 years.

CONCLUSION

The HARmonized Protocol Template to Enhance Reproducibility (HARPER) helps to create a shared understanding of intended scientific decisions through a common text, tabular and visual structure. The template provides a set of core recommendations for clear and reproducible RWE study protocols and is intended to be used as a backbone throughout the research process from developing a valid study protocol, to registration, through implementation and reporting on those implementation decisions.

The relevance of real-world data for the evaluation of neuropathic pain treatments

Treatment of neuropathic pain (NP) is challenging. Interest in real-world evidence (RWE) for benefit-risk assessments of NP treatments increases given the paucity of drugs showing efficacy in randomized controlled trials and restricted labels of available medicines. To provide further context, a literature review regarding regulatory use of RWE and a clinical trial registry search for randomized controlled trials over the last 10 years was carried out. Taken together, and especially for available NP treatments, there is increasing support to consider RWE when evaluating their benefit-risk profile. Examples are provided in which RWE could be used effectively for updating the product label and informing treatment recommendations. Collected and analyzed according to state-of-the-art standards, RWE can inform treatment recommendations and product label decisions.

R WE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 8

In this latest update we highlight the publication of a draft real-world evidence framework by the UK National Institute for Health and Care Excellence, and describe a press release from Germany's Institute for Quality and Efficiency in HealthCare outlining how real-world evidence submissions are not following their guidelines. We also discuss whether the lack of adherence to guidelines is due to ignorance of these guidelines on the part of manufacturers, the difficulty in achieving best practices or a failure of guidelines to comprehensively describe best practices.

Treatment effectiveness in a rare oncology indication: Lessons from an external control cohort study

Real-world data (RWD) reflecting patient treatment in routine clinical practice can be used to develop external control groups for single-arm trials. External controls can provide valuable benchmark results on potential comparator drug effectiveness, particularly in rare indications when randomized controlled trials are either infeasible or unethical. This paper describes lessons learned from a descriptive real-world external control cohort study conducted to provide benchmark data for a single-arm clinical trial in a rare oncology biomarker driven disease. Conducting external control cohort studies to evaluate treatment effectiveness in rare indications likely will present with data and analysis challenges as seen in the example study. However, there are mitigating measures that can be applied in the study design, identification of RWD sources, and data analysis. The lessons learned and reported here with a proposal of an external control study framework can provide guidance for future research in this area, and may be applicable as well in other rare indications. Taking these learnings into consideration, the use of real-world external controls to contextualize treatment effectiveness in rare indications is a valuable approach and warrants further application in the future.

KEY LEARNINGS FROM ICER'S REAL WORLD EVIDENCE REASSESSMENT PILOT

Health technology assessment (HTA) agencies are considering adopting a lifecycle approach to assessments to address uncertainties in the evidence base at launch and to revisit the clinical and economic value of therapies in a dynamic clinical landscape. For reassessments of therapies post launch, HTA agencies are looking to real-world evidence (RWE) to enhance the clinical and economic evidence base, though challenges and concerns in using RWE in decision-making exists. Stakeholders are embarking on demonstration projects to address the challenges and concerns and to further define when and how RWE can be used in HTA decision making. The Institute for Clinical and Economic Review piloted a 24-month observational RWE reassessment. Key learnings from this pilot include identifying the benefits and challenges with using RWE in reassessments and considerations on prioritizing and selecting topics relevant for RWE updates.

The value of anticancer drugs - a regulatory view

The high prices of new anticancer drugs and the marginal added benefit perceived by some stakeholders have fuelled a debate on the value of anticancer drugs in the European Union, even though an agreed definition of what constitutes a drug's value does not exist. In this Perspective, we discuss the value of drugs from different viewpoints and objectives of decision makers: for regulators, assessment of the benefit-risk balance of a drug is a cornerstone for approval; payers rely on cost-effectiveness analyses carried out by health technology assessment agencies for reimbursement decisions; for patients, treatment choices are based on personal preferences and attitudes to risk; and clinicians can use several scales (such as the ESMO Magnitude of Clinical Benefit Scale (ESMO-MCBS)) that have been developed as an attempt to measure value objectively. Although a unique definition that fully captures the concept of value is unlikely to emerge, herein we discuss the importance of understanding different perspectives, and how regulators can help to inform different decision makers.

Can we use existing guidance to support the development of robust real-world evidence for health technology assessment/payer decision-making?

Advances in the digitization of health systems and expedited regulatory approvals of innovative treatments have led to increased potential for the use of real-world data (RWD) to generate real-world evidence (RWE) to complement evidence from clinical trials. However, health technology assessment (HTA) bodies and payers have concerns about the ability to generate RWE of sufficient quality to be pivotal evidence of relative treatment effectiveness. Consequently, there is a growing need for HTA bodies and payers to develop guidance for the industry and other stakeholders about the use of RWD/RWE to support access, reimbursement, and pricing. We therefore sought to (i) understand barriers to the use of RWD/RWE by HTA bodies and payers; (ii) review potential solutions in the form of published guidance; and (iii) review findings with selected HTA/payer bodies. Four themes considered key to shaping the generation of robust RWE for HTA bodies and payers were identified as: (i) data (availability, governance, and quality); (ii) methodology (design and analytics); (iii) trust (transparency and reproducibility); and (iv) policy and partnerships. A range of guidance documents were found from trusted sources that could address these themes. These were discussed with HTA experts. This commentary summarizes the potential guidance solutions available to help resolve issues faced by HTA decision-makers in the adoption of RWD/RWE. It shows that there is alignment among stakeholders about the areas that need improvement in the development of RWE and that the key priority to move forward is better collaboration to make data usable for multiple purposes.

The Structured Process to Identify Fit-for-purpose Data (SPIFD): A data feasibility assessment framework

To complement real‐world evidence (RWE) guidelines, the 2019 Structured Preapproval and Postapproval Comparative study design framework to generate valid and transparent real‐world Evidence (SPACE) framework elucidated a process for designing valid and transparent real‐world studies. As an extension to SPACE, here, we provide a structured framework for conducting feasibility assessments—a step‐by‐step guide to identify decision grade, fit‐for‐purpose data, which complements the United States Food and Drug Administration (FDA)’s framework for a RWE program. The process was informed by our collective experience conducting systematic feasibility assessments of existing data sources for pharmacoepidemiology studies to support regulatory decisions. Used with the SPACE framework, the Structured Process to Identify Fit‐For‐Purpose Data (SPIFD) provides a systematic process for conducting feasibility assessments to determine if a data source is fit for decision making, helping ensure justification and transparency throughout study development, from articulation of a specific and meaningful research question to identification of fit‐for‐purpose data and study design.

R WE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 4

In this month's round up, we discuss a number of recent publications and guidelines addressing the use of real-world evidence to evaluate the clinical benefit of health technology assessments and what the publications mean practically for manufacturers.