HARmonized Protocol Template to Enhance Reproducibility of hypothesis evaluating real-world evidence studies on treatment effects: A good practices report of a joint ISPE/ISPOR task force

Real-World Evidence to Reinforce Clinical Trial Evidence in Health Technology Assessment: A Critical Review of Real-World Evidence Requirements from Seven Countries and Recommendations to Improve Acceptance

BACKGROUND

Real-world evidence (RWE) can reinforce clinical trial evidence in health technology assessment (HTA).

OBJECTIVES

Review HTA bodies' (HTAbs) requirements for RWE, real uses, and acceptance across seven countries (Brazil, Canada, France, Germany, Italy, Spain, and the United Kingdom) and outline recommendations that may improve acceptance of RWE in efficacy/effectiveness assessments and appraisals processes.

METHODS

RWE requirements were summarized based on HTAbs' guidelines. Acceptance by HTAbs was evaluated based on industry experience and case studies.

RESULTS

As of June 2022, RWE methodological guidelines were in place in three of the seven countries. HTAbs typically requested analyses based on local data sources, but the preferred study design and data sources differed. HTAbs had individual submission, assessment, and appraisal processes; some allowed early meetings for the protocol and/or results validation, though few involved external experts or medical societies to provide input to assessment and appraisal. The extent of submission, assessment, and appraisal requirements did not necessarily reflect the degree of acceptance.

CONCLUSION

All the countries reviewed face common challenges regarding the use of RWE. Our proposals address the need to facilitate collaboration and communication with industry and regulatory agencies and the need for specific guidelines describing RWE design and criteria of acceptance throughout the assessment and appraisal processes.

Comparing major and mild cognitive impairment risks in older type-2 diabetic patients: a Danish register-based study on dipeptidyl peptidase-4 inhibitors vs. glucagon-like peptide-1 analogues

INTRODUCTION

The prevalence of major and mild cognitive impairment (CI) in type-2 diabetes older patients is 15-25% and 30-60%, respectively, thus affecting quality of life and health outcomes. There is, therefore, the need of head-to-head studies aiming at identifying the optimal treatment for individuals with type-2 diabetes at increased risk of mild and major CI. This study focuses on the risk of developing mild and major CI in Danish patients treated with dipeptidyl peptidase-4 inhibitors (DPP-4i) and glucagon-like peptide-1 analogues (GLP-1a) using administrative and healthcare registers.

METHODS

An active comparator design with a 3-year follow-up period was used. The main outcome was the hospital admission with a diagnosis of mild CI or major CI. Multivariate Cox Regression analysis was performed using the high-dimensional propensity score to obtain adjusted Hazard Ratio (HR) estimates. Inverse probability of treatment weighting (IPTW) and marginal structured model were used to calculate risk differences while accounting for the variations of confounders throughout the follow-up period.

RESULTS

Our results show a significant higher risk of major CI between DPP-4i and GLP-1a in unadjusted [HR (95% CI) = 3.13 (2.45-4.00), p < 0.001] and adjusted analyses [HR (95% CI) = 1.58 (1.22-2.06), p = 0.001]. No statistically significant differences were observed for mild CI. IPTW resulted stable throughout the follow-up period. Marginal structure modeling (β (95% CI) = 0.022 (0.020-0.024), p < 0.001) resulted in a higher risk of major CI for DPP-4i when compared to GLP-1a.

DISCUSSION

DPP-4i was associated with an increased risk of developing major CI when compared to GLP-1a among older individuals with type-2 diabetes.

Real-world evidence in the reassessment of oncology therapies: payer perceptions from five countries

Aim: This study explored the perceived value of real-world evidence (RWE) in the reassessment of oncology therapies by collecting the perspectives of health technology assessment/payer decision-makers. Materials & methods: A web-based survey was conducted using the Market Access Transformation Rapid Payer Response online portal. 30 participants from France, Germany, Spain, the UK and the USA were recruited based on their expertise. Results: Participants agreed that the most common uses of RWE are to confirm efficacy and safety results from randomized controlled trials and to reevaluate the projected utilization of an oncology therapy. We found variability in other reported uses of RWE. Conclusion: The organizations developing RWE should ensure that their plans recognize the heterogeneity in payer perceptions.

Biases in Electronic Health Records Data for Generating Real-World Evidence: An Overview

Electronic Health Records (EHR) are increasingly being perceived as a unique source of data for clinical research as they provide unprecedentedly large volumes of real-time data from real-world settings. In this review of the secondary uses of EHR, we identify the anticipated breadth of opportunities, pointing out the data deficiencies and potential biases that are likely to limit the search for true causal relationships. This paper provides a comprehensive overview of the types of biases that arise along the pathways that generate real-world evidence and the sources of these biases. We distinguish between two levels in the production of EHR data where biases are likely to arise: (i) at the healthcare system level, where the principal source of bias resides in access to, and provision of, medical care, and in the acquisition and documentation of medical and administrative data; and (ii) at the research level, where biases arise from the processes of extracting, analyzing, and interpreting these data. Due to the plethora of biases, mainly in the form of selection and information bias, we conclude with advising extreme caution about making causal inferences based on secondary uses of EHRs.

The FAIRification of research in real-world evidence: A practical introduction to reproducible analytic workflows using Git and R

Transparency and reproducibility are major prerequisites for conducting meaningful real-world evidence (RWE) studies that are fit for decision-making. Many advances have been made in the documentation and reporting of study protocols and results, but the principles for version control and sharing of analytic code in RWE are not yet as established as in other quantitative disciplines like computational biology and health informatics. In this practical tutorial, we aim to give an introduction to distributed version control systems (VCS) tailored toward the FAIR (Findable, Accessible, Interoperable, and Reproducible) implementation of RWE studies. To ease adoption, we provide detailed step-by-step instructions with practical examples on how the Git VCS and R programming language can be implemented into RWE study workflows to facilitate reproducible analyzes. We further discuss and showcase how these tools can be used to track changes, collaborate, disseminate, and archive RWE studies through dedicated project repositories that maintain a complete audit trail of all relevant study documents.

R WE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 14

In this latest update we highlight: a publication from the US FDA regarding the definitions of real-world data (RWD) and real-world evidence (RWE); a publication from academic researchers on a demonstration project for target trial emulation; a publication from the National Institute of Health and Care Excellence (NICE) on the 1 year anniversary of their RWE framework; and a publication from NICE and Flatiron Health on the utility of US RWD for initial UK health technology assessment decision making.

Building trust in real world evidence (RWE): moving transparency in RWE towards the randomized controlled trial standard

The increasing use of RWE in regulatory and reimbursement decision-making indicates the significant progress that has been made in building trust in RWE through greater transparency. This review of the published literature and key online sources was conducted to provide an update on progress towards greater transparency in RWE, based on four key barriers to trust identified in a 2016 paper and applying learnings from transparency initiatives established for RCTs, such as the US FDA Amendments Act (FDAAA) 2007 Final Rule. Multiple initiatives from the US FDA, EMA and organizations such as the ISPOR-ISPE Joint Task Force have yielded new guidance documents and tools that enable greater transparency in RWE study methodology (STaRT-RWE and HARPER templates), data source selection and quality (SPIFD2 framework, REQueST tool), strategy (the Center for Open Science RWE Study Registry), and will therefore improve transparency in RWE study reporting. Programs such as the REPEAT Initiative and RWE DUPLICATE are investigating reproducibility of RWE studies and improving understanding of the circumstances when valid inference on treatment effects can be obtained from RWE studies. Further work is needed to embed and to implement the new tools and guidance that are available, and thus raise standards for RWE transparency towards the levels expected for RCTs.

Improving Food and Drug Administration-Centers for Medicare and Medicaid Services Coordination for Drugs Granted Accelerated Approval

Policy Points The increasing number of drugs granted accelerated approval by the Food and Drug Administration (FDA) has challenged the Medicare program, which often pays for expensive therapies despite substantial uncertainty about benefits and risks to Medicare beneficiaries. We recommend several administrative and legislative approaches for improving FDA-Centers for Medicare and Medicaid Services (CMS) coordination around accelerated-approval drugs, including promoting earlier discussions among the FDA, the CMS, and drug companies; strengthening Medicare's coverage with evidence development program; linking Medicare payment to evidence generation milestones; and ensuring that the CMS has adequate staffing and resources to evaluate new therapies. These activities can help improve the integrity; transparency; and efficiency of approval, coverage, and payment processes for drugs granted accelerated approval.

CONTEXT

The Food and Drug Administration (FDA)'s accelerated-approval pathway expedites patient access to promising treatments. However, increasing use of this pathway has challenged the Medicare program, which often pays for expensive therapies despite substantial uncertainty about benefits and risks to Medicare beneficiaries. We examined approaches to improve coordination between the FDA and Centers for Medicare and Medicaid Services (CMS) for drugs granted accelerated approval.

METHODS

We argue that policymakers have focused on expedited pathways at the FDA without sufficient attention to complementary policies at the CMS. Although differences between the FDA and CMS decisions are to be expected given the agencies' different missions and statutory obligations, procedural improvements can ensure that Medicare beneficiaries have timely access to novel therapies that are likely to improve health outcomes. To inform policy options and recommendations, we conducted semistructured interviews with stakeholders to capture diverse perspectives on the topic.

FINDINGS

We recommend ten areas for consideration: clarifying the FDA's evidentiary standards; strengthening FDA authorities; promoting earlier discussions among the FDA, the CMS, and drug companies; improving Medicare's coverage with evidence development program; tying Medicare payment for accelerated-approval drugs to evidence generation milestones; issuing CMS guidance on real-world evidence; clarifying Medicare's "reasonable and necessary" criteria; adopting lessons from international regulatory-reimbursement harmonization efforts; ensuring that the CMS has adequate staffing and expertise; and emphasizing equity.

CONCLUSIONS

Better coordination between the FDA and CMS could improve the transparency and predictability of drug approval and coverage around accelerated-approval drugs, with important implications for patient outcomes, health spending, and evidence generation processes. Improved coordination will require reforms at both the FDA and CMS, with special attention to honoring the agencies' distinct authorities. It will require administrative and legislative actions, new resources, and strong leadership at both agencies.

Principles for Good Practice in the Conduct of Non-interventional Studies: The View of Industry Researchers

This reflection paper presents a consolidated view of EFPIA on the need for principles for good practice in the generation and use of non-interventional studies (NIS), including overarching principles such as the registration of hypothesis evaluating treatment effect (HETE) studies. We first define NIS and the important adjacencies to clinical trials and relationship with real-world evidence (RWE). We then outline the principles for good practice with respect to appropriate research design, study protocol, fit-for-purpose variables and data quality, analytical methods, bias reduction, transparency in conduct and use, privacy management and ethics review. We conclude with recommendations for action for the research community to promote trust and credibility in the use of NIS.

SURF: A Screening Tool (for Sponsors) to Evaluate Whether Using Real-World Data to Support an Effectiveness Claim in an FDA Application Has Regulatory Feasibility

Based on recent guidance and publicly available approvals, the US Food and Drug Administration (FDA) has demonstrated its openness to considering evidence of effectiveness from real-world data (RWD) and nonrandomized studies (or "real-world evidence (RWE)") in its decision making. Through analysis of the FDA approvals, several authors have identified methodologic issues commonly discussed by FDA reviewers. However, in our analysis of FDA guidance and use cases, acceptability of RWE also critically depends on whether the characteristics of the clinical development program align with circumstances in which the FDA may have flexibility in considering evidence from real-world study designs relative to more robust designs. Successful use of RWD requires advance planning to allocate the necessary resources and time to undertake principled epidemiology approaches to study design, data selection, and implementation of analyses as well as address regulatory feedback. Thus, sponsors benefit from early identification of programs in which successful RWD use is more likely, to ensure efficient use of resources required for the next steps of scientific feasibility assessment. We developed SURF, a screening tool intended to help a sponsor decide whether to prioritize resources for further exploring the feasibility of using an RWD approach to meet the FDA's effectiveness evidentiary standards for a particular clinical development program. Here, we provide an analysis of FDA guidance, highlighting the circumstances in which RWD approaches may be most acceptable, and demonstrate the use of this screening tool, including the rationale for its construction, types of evidence needed, and application to publicly available approvals as support of concept.

R WE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 13

In this latest update we highlight a report from the European Medicines Agency on their use of real-world evidence (RWE) in decision making, RWE reporting guidance from the Canadian Agency for Drugs and Technologies in Health and highlight some new data demonstrating the value medicines for spinal muscular atrophy have brought patients.

Drug survival and safety of biosimilars and originator adalimumab in the treatment of psoriasis: a multinational cohort study

INTRODUCTION

Psoriasis is a chronic inflammatory skin disease. Adalimumab is an effective but previously expensive biological treatment for psoriasis. The introduction of biosimilars following the patent expiry of the originator adalimumab Humira has reduced the unit cost of treatment. However, the long-term effectiveness and safety of adalimumab biosimilars for treating psoriasis in real-world settings are uncertain and may be a barrier to widespread usage.

METHODS AND ANALYSIS

This study aims to compare the drug survival and safety of adalimumab biosimilars to adalimumab originator for the treatment of psoriasis. We will use both routinely collected healthcare databases and dedicated pharmacovigilance registries from the PsoNet initiative, including data from the UK, France and Spain. We will conduct a cohort study using a prevalent new user design. We will match patients on previous adalimumab exposure time to create two equal-sized cohorts of biosimilar and originator users. The coprimary outcomes are drug survival, defined by the time from cohort entry to discontinuation of the drug of interest; and risk of serious adverse events, defined by adverse events leading to hospitalisation or death. Cox proportional hazards models will be fitted to calculate HRs as the effect estimate for the outcomes.

ETHICS AND DISSEMINATION

The participating registries agree with the Declaration of Helsinki and received approval from local ethics committees. The results of the study will be published in scientific journals and presented at international dermatology conferences by the end of 2023.

R WE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 11

In this latest update we highlight a study from the REPEAT initiative that evaluates the reproducibility of real-world data studies, the publication of the HARPER Protocol Template developed by a joint ISPE/ISPOR taskforce, and discuss recent US FDA guidance on external control arms.

A Structured Process to Identify Fit-for-Purpose Study Design and Data to Generate Valid and Transparent Real-World Evidence for Regulatory Uses

Generating evidence from real-world data requires fit-for-purpose study design and data. In addition to validity, decision makers require transparency in the reasoning that underlies study design and data source decisions. The 2019 Structured Preapproval and Postapproval Comparative Study Design Framework to Generate Valid and Transparent Real-World Evidence (SPACE) and the 2021 Structured Process to Identify Fit-For-Purpose Data (SPIFD)-intended to be used together-provide a step-by-step guide to identify decision grade, fit-for-purpose study design and data. In this update (referred to as "SPIFD2" to encompass both the design and data aspects) we provide an update to these frameworks that combines the templates into one, more explicitly calls for articulation of the hypothetical target trial and sources of bias that may arise in the real-world emulation, and provides explicit references to the Structured Template and Reporting Tool for Real-World Evidence (STaRT-RWE) tables that we suggest using immediately after invoking the SPIFD2 framework. Following the steps recommended in the SPIFD2 process requires due diligence on the part of the researcher to ensure that every aspect of study design and data selection is rationalized and supported by evidence. The resulting stepwise documentation enables reproducibility and clear communication with decision makers, and it increases the likelihood that the evidence generated is valid, fit-for-purpose, and sufficient to support healthcare and regulatory decisions.

Real-world data for precision cancer medicine-A European perspective

Leveraging real-world data (RWD) for drug access is necessary to overcome a key challenge of modern precision oncology: tackling numerous low-prevalence oncogenic mutations across cancers. Withholding a potentially active medication in patients with rare mutations for the sake of control chemotherapy or "best" supportive care is neither practicable nor ethically justifiable anymore, particularly as RWD could meanwhile be used instead, according to scientific principles outlined by the US Food and Drug Administration, European Medicines Agency and other stakeholders. However, practical implementation varies, with occasionally opposite recommendations based on the same evidence in different countries. In the face of growing need for precision drugs, more transparency of evaluation, a priori availability of guidance for the academia and industry, as well as a harmonized framework for health technology assessment across the European Union (EU) are imperative. These could in turn trigger infrastructural changes in national and pan-European registries, cancer management guidelines (e.g., frequency of routine radiologic restaging, inclusion of patient-reported outcomes), and the health data space, to ensure conformity with declared standards and facilitate extraction of RWD sets (including patient-level data) suitable for approval and pricing with minimal effort. For an EU-wide unification of precision cancer medicine, collective negotiation of drug supply contracts and funding solidarity would additionally be required to handle the financial burden. According to experience from pivotal European programs, off-label use could potentially also be harmonized across EU-states to accelerate availability of novel drugs, streamline collection of valuable RWD, and mitigate related costs through wider partnerships with pharmaceutical companies.

Early access for innovative oncology medicines: a different story in each nation

BACKGROUND

As innovative oncology medicines are rapidly developed, there is increasing pressure on payers to offer patients timely access to life-saving therapies. The uncertainty surrounding these therapies when phase III clinical trials are pending has necessitated new, adapted pathways to market access, with timelines that greatly vary by country. Understanding differences between pathways may identify opportunities to expedite patient access universally.

OBJECTIVES

To describe early access pathways for new oncology medicines among selected countries with established health technology assessment (HTA) frameworks and publicly funded health systems, with a special focus on real-world evidence (RWE).

METHODS

We reviewed the HTA agency websites of the selected OECD countries: National Institute for Health and Care Excellence (NICE) for England and Wales; Haute Autorité de Santé (HAS) for France; IQWiG and G-BA for Germany; Agenzia Italiana del Farmaco (AIFA) for Italy; Pharmaceutical Benefits Advisory Committee (PBAC) for Australia; and CADTH and Institut National d'Excellence en Santé et Services Sociaux (INESSS) for Canada as the primary source of evidence.

RESULTS

Processes for early patient access to innovative oncology therapies varied across selected countries; however, most countries have an established pathway for publicly funded early access (England and Wales, France, Germany, Italy, and Australia). The utilization of RWE to support earlier access (coverage with evidence) also varied by country, with some HTA organizations being actively engaged in these agreements (NICE, AIFA, and HAS) and others having no established processes in place (G-BA and CADTH/INESSS).

CONCLUSIONS

This review of early access pathways for novel oncology medicines found substantial variability between countries of interest. Coverage with evidence frameworks may provide a unique opportunity for industry and payers to collaborate on earlier access to innovative cancer therapies with life-saving potential.