Real-World Evidence: Useful in the Real World of US Payer Decision Making? How? When? And What Studies?

Clinical Evidence Supporting FDA Approval of Gene and RNA Therapies for Rare Inherited Conditions

BACKGROUND

Gene and RNA therapies have potential to transform the treatment of rare inherited diseases, but there are concerns about the evidence supporting their use and high costs.

OBJECTIVE

We analyze the evidence supporting Food and Drug Administration (FDA) approval of gene and RNA therapies for rare inherited diseases and discuss implications for clinical practice and policy.

METHODS

We conducted a qualitative analysis of FDA documents outlining the basis of approval for gene and RNA therapies approved for rare inherited diseases between 2016 and 2023. For each drug, we gathered five characteristics of the evidence supporting FDA approval (no phase 3 trial, nonrandomized, no clinical endpoint, lack of demonstrated benefit, and significant protocol deviation) and four characteristics of the FDA approval process (prior rejection or complete response, negative committee vote, discrepancy between label and trial population, and boxed warning). The main outcome was the number of drugs with each characteristic.

RESULTS

Between 2016 and 2023, 19 gene and RNA therapies received FDA approval to treat rare inherited diseases. The most common limitations in the evidence supporting approval of these drugs were nonrandomized studies (8/19, 42%), no clinical endpoint (7/19, 37%), lack of demonstrated benefit or inconsistent results (4/19, 21%), and no phase 3 trial (4/19, 21%). Half (3/6) of accelerated approvals and 57% (5/9) of drugs with breakthrough designation had nonrandomized trials, and gene therapies with one-time dosing were overrepresented (5/7, 71%) among the drugs with nonrandomized trials. Five of six accelerated approvals (83%) and five of nine pediatric drugs (56%), most of which were indicated for Duchenne muscular dystrophy, had no clinical endpoint. Four of nine (44%) pediatric drugs and four of six (67%) accelerated approvals failed to demonstrate benefit compared with none of the nonpediatric drugs and none of the traditional approvals. Five drugs, which all had different indications and represented a mix of RNA and gene therapies, did not have any of these evidence characteristics. Among drugs that received prior rejections or negative committee opinions, all four had nonrandomized trials and lacked a clinical endpoint, and 75% (3/4) lacked demonstrated benefit. Five of nine (56%) pediatric drugs were indicated for broader age groups according to the drug label compared with the trial populations. Of the three drugs with boxed warnings, two had pediatric indications and nonrandomized studies, and one had no phase 3 trial.

CONCLUSIONS

Issues related to trial design, outcome, and data integrity in the evidence supporting FDA approval of rare inherited disease gene and RNA therapies raise questions about whether this evidence is adequate to inform prescribing decisions. Gene and RNA therapies with accelerated approval and pediatric indications were overrepresented among drugs lacking clinical endpoints or demonstrated benefit and should be the focus of efforts to reduce uncertainty in the evidence.

Medication use evaluation of tocilizumab implementation in COVID-19 treatment guidelines: A causal inference approach

PURPOSE

Introduction of new medications to health-system formularies is often not accompanied by assessments of their clinical impact on the local patient population. The growing availability of electronic health record (EHR) data and advancements in pharmacoepidemiology methods offer institutions the opportunity to monitor the medication implementation process and assess clinical effectiveness in the local clinical context. In this study, we applied novel causal inference methods to evaluate the effects of a formulary policy introducing tocilizumab therapy for critically ill patients with coronavirus disease 2019 (COVID-19).

METHODS

We conducted a medication use evaluation utilizing EHR data from patients admitted to a large medical center during the 6 months before and after implementation of a formulary policy endorsing the use of tocilizumab for treatment of COVID-19. The impact of tocilizumab on 28-day all-cause mortality was assessed using a difference-in-differences analysis, with ineligible patients serving as a nonequivalent control group, and a matched analysis guided by a target trial emulation framework. Safety endpoints assessed included the incidence of secondary infections and liver enzyme elevations. Our findings were benchmarked against clinical trials, an observational study, and a meta-analysis.

RESULTS

Following guideline modification, tocilizumab was administered to 69% of eligible patients. This implementation was associated with a 3.1% absolute risk reduction in 28-day mortality (odds ratio, 0.86; number needed to treat to prevent one death, 32) attributable to the inclusion of tocilizumab in the guidelines and an additional 8.6% absolute risk reduction (odds ratio, 0.65; number needed to treat to prevent one death, 12) linked to its administration. These findings were consistent with estimates from published literature, although the effect estimates from the difference-in-differences analysis exhibited imprecision.

CONCLUSION

Evaluating formulary management decisions through novel causal inference approaches offers valuable estimates of clinical effectiveness and the potential to optimize the impact of new medications on population outcomes.

FOUNTAIN: a modular research platform for integrated real-world evidence generation

BACKGROUND

Real-world evidence (RWE) plays a key role in regulatory and healthcare decision-making, but the potentially fragmentated nature of generated evidence may limit its utility for clinical decision-making. Heterogeneity and a lack of reproducibility in RWE resulting from inconsistent application of methodologies across data sources should be minimized through harmonization.

METHODS

This paper's aim is to describe and reflect upon a multidisciplinary research platform (FOUNTAIN; FinerenOne mUlti-database NeTwork for evidence generAtIoN) with coordinated studies using diverse RWE generation approaches and explore the platform's strengths and limitations. With guidance from an executive advisory committee of multidisciplinary experts and patient representatives, the goal of the FOUNTAIN platform is to harmonize RWE generation across a portfolio of research projects, including research partner collaborations and a common data model (CDM)-based program. FOUNTAIN's overarching objectives as a research platform are to establish long-term collaborations among pharmacoepidemiology research partners and experts and to integrate diverse approaches for RWE generation, including global protocol execution by research partners in local data sources and common protocol execution in multiple data sources through federated data networks, while ensuring harmonization of medical definitions, methodology, and reproducible artifacts across all studies. Specifically, the aim of the multiple studies run within the frame of FOUNTAIN is to provide insight into the real-world utilization, effectiveness, and safety of finerenone across its life-cycle.

RESULTS

Currently, the FOUNTAIN platform includes 9 research partner collaborations and 8 CDM-mapped data sources from 7 countries (United States, United Kingdom, China, Japan, The Netherlands, Spain, and Denmark). These databases and research partners were selected after a feasibility fit-for-purpose evaluation. Six multicountry, multidatabase, cohort studies are ongoing to describe patient populations, current standard of care, comorbidity profiles, healthcare resource use, and treatment effectiveness and safety in different patient populations with chronic kidney disease and type 2 diabetes. Strengths and potential limitations of FOUNTAIN are described in the context of valid RWE generation.

CONCLUSION

The establishment of the FOUNTAIN platform has allowed harmonized execution of multiple studies, promoting consistency both within individual studies that employ multiple data sources and across all studies run within the platform's framework. FOUNTAIN presents a proposal to efficiently improve the consistency and generalizability of RWE on finerenone.

Construction and application of nasopharyngeal carcinoma-specific big data platform based on electronic health records

OBJECTIVE

To establish a nasopharyngeal carcinoma-specific big data platform based on electronic health records (EHRs) to provide data support for real-world study of nasopharyngeal carcinoma.

METHODS

A multidisciplinary expert team was established for this project. Based on industry standards and practical feasibility, the team designed the nasopharyngeal carcinoma data element standards including 14 modules and 640 fields. Data from patients diagnosed with nasopharyngeal carcinoma who visited Southern Hospital after 1999 were extracted from 15 EHRs systems and were cleaned, structured, and standardized using information technologies such as machine learning and natural language processing. In addition, a series of measures such as quality control and data encryption were taken to ensure data quality and patient privacy. At the platform application level, 10 functional modules were designed according to the needs of nasopharyngeal carcinoma research.

RESULTS

As of 1 October 2022, the Big Data platform has included 11,617patients, of whom 8228 (70.83 %) were male and 3389 (29.17 %) were female, with a median age of 48 years (interquartile range, 40 years). The data in the platform were validated to have a high level of completeness and accuracy, especially for key variables such as social demographics, laboratory tests and vital signs. Currently, six projects involving risk factors, early diagnosis, treatment efficacy and prevention of treatment-related toxic reactions have been conducted on the platform.

CONCLUSIONS

We have established a high-quality NPC-specific big data platform by integrating heterogeneous data from multiple sources in the EHR. The platform provides an effective tool and strong data support for real-world studies of nasopharyngeal carcinoma, which helps to improve research efficiency, reduce costs, and improve the quality of research results. We expect to promote multicenter nasopharyngeal carcinoma data sharing in the future to facilitate the generation of high-quality real-world evidence in nasopharyngeal carcinoma. This article may provide some reference value for other comprehensive hospitals to establish a big data platform for nasopharyngeal carcinoma.

Real-world evidence in the reassessment of oncology therapies: payer perceptions from five countries

Aim: This study explored the perceived value of real-world evidence (RWE) in the reassessment of oncology therapies by collecting the perspectives of health technology assessment/payer decision-makers.Materials & methods: A web-based survey was conducted using the Market Access Transformation Rapid Payer Response online portal. 30 participants from France, Germany, Spain, the UK and the USA were recruited based on their expertise.Results: Participants agreed that the most common uses of RWE are to confirm efficacy and safety results from randomized controlled trials and to reevaluate the projected utilization of an oncology therapy. We found variability in other reported uses of RWE.Conclusion: The organizations developing RWE should ensure that their plans recognize the heterogeneity in payer perceptions.

Generating and using real-world data: A worthwhile uphill battle

The precision oncology paradigm challenges the feasibility and data generalizability of traditional clinical trials. Consequently, an unmet need exists for practical approaches to test many subgroups, evaluate real-world drug value, and gather comprehensive, accessible datasets to validate novel biomarkers. Real-world data (RWD) are increasingly recognized to have the potential to fill this gap in research methodology. Established applications of RWD include informing disease epidemiology, pharmacovigilance, and healthcare quality assessment. Currently, concerns regarding RWD quality and comprehensiveness, privacy, and biases hamper their broader application. Nonetheless, RWD may play a pivotal role in supplementing clinical trials, enabling conditional reimbursement and accelerated drug access, and innovating trial conduct. Moreover, purpose-built RWD repositories may support the extension or refinement of drug indications and facilitate the discovery and validation of new biomarkers. This perspective explores the potential of leveraging RWD to advance oncology, highlights its benefits and challenges, and suggests a path forward in this evolving field.

Clinical comprehensive evaluation of direct oral anticoagulants for patients with atrial fibrillation in China

BACKGROUND

Direct oral anticoagulants (DOACs) are increasingly recommended over warfarin in stroke prevention for patients with non-valvular atrial fibrillation (AF). However, there is an important evidence gap in choosing the most appropriate DOAC for Chinese patients in clinical practice.

METHODS

A multi-criteria decision analysis (MCDA) was adopted to build a scoring framework. Attributes and criteria were identified and determined by a scoping literature review, two rounds of Delphi surveys, and a consensus meeting. Weights of each attribute and criterion in the framework were determined using analytic hierarchy process (AHP). Evidence was collected based on the domestic or at least Asian data. Scoring methods for each criterion were developed depended on their characteristics and determined with an expert consensus meeting. Comprehensive scores of each DOAC were calculated based on the utility scores of each criterion and their corresponding weights.

RESULTS

A total of 5 attributes, including safety, efficacy, costs/cost-effectiveness, suitability, and accessibility, were determined, and 16 criteria were under the 5 attributes. The safety and efficacy were ranked as the top two important attributes with the weights of 38.8% and 35.9%, respectively, while the suitability received the lowest weight of 7.9%. The comprehensive score for edoxaban was the highest (72.3), followed by dabigatran (49.7), rivaroxaban (37.9), and apixaban (35.8).

CONCLUSIONS

This study provided a scoring framework developed for comprehensive evaluation of DOACs in China. The ranking of DOACs could help to support the decision-making in clinical practice. The framework could provide a reference for comprehensive evaluation of other drugs.

Concepts of Designing and Implementing Pharmacoepidemiology Studies on the Safety of Systemic Treatments in Dermatology Practice

The U.S. Food and Drug Administration and clinical guidelines use evidence from pharmacoepidemiology studies to inform prescribing decisions and fill evidence gaps left by randomized controlled trials (RCTs). The long-term safety and infrequent adverse reactions are not well-understood when RCTs are short and involve few patients, as is the case for most systemic immunomodulating drugs in dermatology. A better understanding of the design and implementation of pharmacoepidemiology studies will help practitioners assess the accuracy of etiologic findings and use them with confidence in clinical practice. Conducting pharmacoepidemiology studies follows a structured approach, which we discuss in this article: (i) a design layer connects the research question with the appropriate study design, and considering which hypothetical RCT one ideally would want to conduct reduces inadvertent investigator errors; (ii) a measurement layer transforms longitudinal patient-level data into variables that identify the study population, patient characteristics, treatment, and outcomes; and (iii) the analysis focuses on the causal treatment effect estimation. The review and interpretation of pharmacoepidemiology studies should consider issues beyond a typical review of RCTs, chiefly the lack of baseline randomization and the use of secondary data. Well-designed and well-conducted pharmacoepidemiologic studies complement dermatology practice with critical information on prescribing systemic medications.

Evaluation of reporting quality of cohort studies using real-world data based on RECORD: systematic review

OBJECTIVE

Real-world data (RWD) and real-world evidence (RWE) have been paid more and more attention in recent years. We aimed to evaluate the reporting quality of cohort studies using real-world data (RWD) published between 2013 and 2021 and analyze the possible factors.

METHODS

We conducted a comprehensive search in Medline and Embase through the OVID interface for cohort studies published from 2013 to 2021 on April 29, 2022. Studies aimed at comparing the effectiveness or safety of exposure factors in the real-world setting were included. The evaluation was based on the REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. Agreement for inclusion and evaluation was calculated using Cohen's kappa. Pearson chi-square test or Fisher's exact test and Mann-Whitney U test were used to analyze the possible factors, including the release of RECORD, journal IFs, and article citations. Bonferroni's correction was conducted for multiple comparisons. Interrupted time series analysis was performed to display the changes in report quality over time.

RESULTS

187 articles were finally included. The mean ± SD of the percentage of adequately reported items in the 187 articles was 44.7 ± 14.3 with a range of 11.1-87%. Of 23 items, the adequate reporting rate of 10 items reached 50%, and the reporting rate of some vital items was inadequate. After Bonferroni's correction, the reporting of only one item significantly improved after the release of RECORD and there was no significant improvement in the overall report quality. For interrupted time series analysis, there were no significant changes in the slope (p = 0.42) and level (p = 0.12) of adequate reporting rate. The journal IFs and citations were respectively related to 2 areas and the former significantly higher in high-reporting quality articles.

CONCLUSION

The endorsement of the RECORD cheklist was generally inadequate in cohort studies using RWD and has not improved in recent years. We encourage researchers to endorse relevant guidelines when utilizing RWD for research.

Developing Patient-Centered Real-World Evidence: Emerging Methods Recommendations From a Consensus Process

OBJECTIVES

The Joint ISPOR-ISPE Special Task Force on Real-World Evidence included patient/stakeholder engagement as a recommended good procedural practice when designing, conducting, and disseminating real-world evidence (RWE). However, there are no guidelines describing how patient experience data (PED) can be applied when designing real-world data (RWD) studies. This article describes development of consensus recommendations to guide researchers in applying PED to develop patient-centered RWE.

METHODS

A multidisciplinary advisory board, identified through recommendations of collaborators, was established to guide development of recommendations. Semistructured interviews were conducted to identify how experienced RWD researchers (n = 15) would apply PED when designing a hypothetical RWD study. Transcripts were analyzed and emerging themes developed into preliminary methods recommendations. An eDelphi survey (n = 26) was conducted to refine/develop consensus on the draft recommendations.

RESULTS

We identified 13 recommendations for incorporating PED throughout the design, conduct, and translation of RWE. The recommendations encompass themes related to the development of a patient-centered research question, designing a study, disseminating RWE, and general considerations. For example, consider how patient input can inform population/subgroups, comparators, and study period. Researchers can leverage existing information describing PED and may be able to apply those insights to studies relying on traditional RWD sources and/or patient registries.

CONCLUSIONS

Applying these emerging recommendations may improve the patient centricity of RWE through improved relevance of RWE to patient communities of interest and foster greater multidisciplinary participation and transparency in RWD research. As researchers gather experience by applying the methods recommendations, further refinement of these consensus recommendations may lead to "best practices."

Value-based evidence across health care sectors: a push for transparent real-world studies, data, and evidence dissemination

There is currently a heightened need for transparency in pharmaceutical sectors. The inclusion of real-world (RW) evidence, in addition to clinical trial evidence, in decision-making processes, was an important step forward toward a more inclusive established value proposition. This advance has introduced new transparency challenges. Increasing transparency is a critical step toward accelerating improvement in type, quality, and access to data, regardless of whether these originate from clinical trials or from RW studies. However, so far, advances in transparency have been relatively restricted to clinical trials, and there remains a lack of similar expectations or standards of transparency concerning the generation and reporting of RW data. This perspective paper aims to highlight the need for transparency concerning RW studies, data, and evidence across health care sectors, to identify areas for improvement, and provide concrete recommendations and practices for the future. Specific issues are discussed from different stakeholder perspectives, culminating in recommended actions, from individual stakeholder perspectives, for improved RW study, data, and evidence transparency. Furthermore, a list of potential guidelines for consideration by stakeholders is proposed. While recommendations from different stakeholder perspectives are made, true transparency in the processes involved in the generation, reporting, and use of RW evidence will require a concerted effort from all stakeholders across health care sectors.

Reproducibility of real-world evidence studies using clinical practice data to inform regulatory and coverage decisions

Studies that generate real-world evidence on the effects of medical products through analysis of digital data collected in clinical practice provide key insights for regulators, payers, and other healthcare decision-makers. Ensuring reproducibility of such findings is fundamental to effective evidence-based decision-making. We reproduce results for 150 studies published in peer-reviewed journals using the same healthcare databases as original investigators and evaluate the completeness of reporting for 250. Original and reproduction effect sizes were positively correlated (Pearson's correlation = 0.85), a strong relationship with some room for improvement. The median and interquartile range for the relative magnitude of effect (e.g., hazard ratiooriginal/hazard ratioreproduction) is 1.0 [0.9, 1.1], range [0.3, 2.1]. While the majority of results are closely reproduced, a subset are not. The latter can be explained by incomplete reporting and updated data. Greater methodological transparency aligned with new guidance may further improve reproducibility and validity assessment, thus facilitating evidence-based decision-making. Study registration number: EUPAS19636.

How do cancer clinicians perceive real-world data and the evidence derived therefrom? Findings from an international survey of the European Organisation for Research and Treatment of Cancer

Background: The role of real-world evidence (RWE) in the development of anticancer therapies has been gradually growing over time. Regulators, payers and health technology assessment agencies, spurred by the rise of the precision medicine model, are increasingly incorporating RWE into their decision-making regarding the authorization and reimbursement of novel antineoplastic treatments. However, it remains unclear how this trend is viewed by clinicians in the field. This study aimed to investigate the opinions of these stakeholders with respect to RWE and its suitability for informing regulatory, reimbursement-related and clinical decisions in oncology.

Methods: An online survey was disseminated to clinicians belonging to the network of the European Organisation for Research and Treatment of Cancer between May and July 2021.

Results: In total, 557 clinicians across 30 different countries participated in the survey, representing 13 distinct cancer domains. Despite seeing the methodological challenges associated with its interpretation as difficult to overcome, the respondents mostly (75.0%) perceived RWE positively, and believed such evidence could be relatively strong, depending on the designs and data sources of the studies from which it is produced. Few (4.6%) saw a future expansion of its influence on decision-makers as a negative evolution. Furthermore, nearly all (94.0%) participants were open to the idea of sharing anonymized or pseudonymized electronic health data of their patients with external parties for research purposes. Nevertheless, most clinicians (77.0%) still considered randomized controlled trials (RCTs) to be the gold standard for generating clinical evidence in oncology, and a plurality (49.2%) thought that RWE cannot fully address the knowledge gaps that remain after a new antitumor intervention has entered the market. Moreover, a majority of respondents (50.7%) expressed that they relied more heavily on RCT-derived evidence than on RWE for their own decision-making.

Conclusion: While cancer clinicians have positive opinions about RWE and want to contribute to its generation, they also continue to hold RCTs in high regard as sources of actionable evidence.

Using real-world evidence in healthcare from Western to Central and Eastern Europe: a review of existing barriers

As part of the HTx (Next Generation Health Technology Assessment) project, this study was aimed at identifying the main barriers for application of real-world evidence (RWE) for the purposes of health technology assessment in the Central and Eastern European countries. A mixed methods approach was employed to identify the main barriers: a scoping review of the literature and a series of discussions with stakeholders. Based on the applied approaches, we attempted to summarize the main barriers and challenges related to transferability of RWE in five main groups: technical, regulatory, clinical, scientific and perceptional barriers. Further research should pursue the development of detailed, consensus-based guidelines to improve the harmonization and standardization of RWE.

Acute lower respiratory infections: real-world evidence of antibiotic prescription pattern and costs from a large administrative Italian database

OBJECTIVE

This observational retrospective analysis aimed to describe antibiotic prescription pattern in patients with acute exacerbation of chronic obstructive pulmonary disease (AECOPD) and community-acquired pneumonia (CAP) and their costs, from the Italian National Health Service perspective.

METHODS

From the ReS database, a cross-linkage of Italian healthcare administrative databases through a unique anonymous code allowed to select subjects aged ≥12 years, supplied with at least an antibacterial for systemic use (ATC code: J01) from 01/01/2017 to 12/31/2017 and evaluable until the end of 2018. Prescriptions of different antibiotics on the same date were excluded. The prescription pattern was assessed for patients with an AECOPD (aged ≥50) or a CAP event (aged ≥12) in 2017. A 30-day cost analysis after the antibacterial supply and according to absence/presence (15 days before/after the supply) of AECOPD/CAP hospitalization was performed.

RESULTS

In 2017, among patients aged ≥12 (~5 million), 1,845,268 were supplied with ≥1 antibacterial (37.2%). Antibacterial prescriptions potentially related to AECOPD were 39,940 and 4,059 to CAP: quinolones were the most prescribed (37.2% and 39.0%, respectively), followed by third-generation cephalosporins (25.5%; 27.5%), penicillins (15.4%; 14.9%), and macrolides (14.4%; 11.3%); the 30-day mean cost was €709 and €2,889. An association AECOPD/CAP-antibacterial supply costed more when the hospitalization occurred 15 days after the antibiotic supply (€5,006 and €4,966, respectively).

CONCLUSIONS

Findings confirmed the very high use of antimicrobials in Italy and highlighted the urgent need of improving current prescribing practices and developing new molecules, to stop the incessant spread of antimicrobial resistance and related socioeconomic impacts.

Assessing Payers' Preferences for Real-World Evidence in the United States: A Discrete Choice Experiment

OBJECTIVES

To rank the US payers' preferences for attributes of real-world evidence (RWE) studies in the context of chronic disease and to quantify trade-offs among them.

METHODS

We conducted a discrete choice experiment in which 180 employees from payer organizations were tasked to choose between 2 RWE studies assuming they were assessing evidence to inform formulary decisions for chronic disease treatment. Each RWE study was characterized by 7 attributes with 3 levels each: very informative, moderately informative, and not measured. We used a D-optimal main-effects design. Survey data were fitted to a conditional logit model to obtain a relative measure of the ranking of importance for each attribute.

RESULTS

Clinical outcomes were the most preferred attribute. It was 4.68 times as important as productivity outcomes-the least preferred attribute. It was followed by health-related quality of life (2.78), methodologic rigor (2.09), resource utilization (1.71), and external validity (1.56).

CONCLUSIONS

This study provides a quantification of the value payers place on key RWE attributes. Across attributes, payers have higher preferences for clinical and health-related quality of life outcomes than the other attributes. Between attributes' levels, payers prefer high levels of information in clinical outcomes and methodologic rigor but are indifferent in other attributes. Our results bridge the gap between the information that payers seek and the attributes that RWE studies prioritize and effectively guide future research design.

Can we use existing guidance to support the development of robust real-world evidence for health technology assessment/payer decision-making?

Advances in the digitization of health systems and expedited regulatory approvals of innovative treatments have led to increased potential for the use of real-world data (RWD) to generate real-world evidence (RWE) to complement evidence from clinical trials. However, health technology assessment (HTA) bodies and payers have concerns about the ability to generate RWE of sufficient quality to be pivotal evidence of relative treatment effectiveness. Consequently, there is a growing need for HTA bodies and payers to develop guidance for the industry and other stakeholders about the use of RWD/RWE to support access, reimbursement, and pricing. We therefore sought to (i) understand barriers to the use of RWD/RWE by HTA bodies and payers; (ii) review potential solutions in the form of published guidance; and (iii) review findings with selected HTA/payer bodies. Four themes considered key to shaping the generation of robust RWE for HTA bodies and payers were identified as: (i) data (availability, governance, and quality); (ii) methodology (design and analytics); (iii) trust (transparency and reproducibility); and (iv) policy and partnerships. A range of guidance documents were found from trusted sources that could address these themes. These were discussed with HTA experts. This commentary summarizes the potential guidance solutions available to help resolve issues faced by HTA decision-makers in the adoption of RWD/RWE. It shows that there is alignment among stakeholders about the areas that need improvement in the development of RWE and that the key priority to move forward is better collaboration to make data usable for multiple purposes.

Use of RWE to Inform Regulatory, Public Health Policy, and Intervention Priorities for the Developing World

For low‐ and middle‐income countries (LMICs) to benefit from real‐world evidence (RWE)/real‐world data (RWD) in both product registration (“regulatory”) decision making and in product utilization policy (“policy”) decision making, they need to overcome several challenges. They need to deploy more electronic health records systems (EHRs), adjust for confounder variables, build trust between stakeholders, and create laws and regulations for local generation of data that are assented for secondary use. The role of procurers and their use of RWE/RWD in the LMIC context likewise is in a state of ongoing development. Procurers of health products are strong players currently in the “access” chain as LMICs continue to work on strengthening governmental health technology assessment (HTA) bodies. Procurers’ use of RWE is presently at an early stage and is mostly indirect, leveraging RWE results that are produced by researchers in high‐income countries (HICs), often under considerably different regulatory and policy objectives and constraints compared to LMICs’ epidemiology and priorities. Pending wider deployment of EHRs and other RWE sources, stakeholders must realize that populations from HIC RWE (i) can be devised to closely resemble phenotypic patterns in LMIC populations and (ii) can be analyzed to align with LMICs’ unmet needs.

Assessment of Alectinib vs Ceritinib in ALK-Positive Non-Small Cell Lung Cancer in Phase 2 Trials and in Real-world Data

IMPORTANCE

Quantitative assessment of bias from unmeasured confounding and missing data can help evaluate uncertainty in findings from indirect comparisons using real-world data (RWD).

OBJECTIVE

To compare the effectiveness of alectinib vs ceritinib in terms of overall survival (OS) in patients with ALK-positive, crizotinib-refractory, non-small cell lung cancer (NSCLC) and to assess the sensitivity of these findings to unmeasured confounding and missing data assumptions.

DESIGN, SETTING, AND PARTICIPANTS

This comparative effectiveness research study compared patients from 2 phase 2 alectinib trials and real-world patients. Patients were monitored from June 2013 to March 2020. Comparisons of interest were between alectinib trial data vs ceritinib RWD and alectinib RWD vs ceritinib RWD. RWD treatment groups were selected from nationally representative cancer data from US cancer clinics, the majority from community centers. Participants were ALK-positive patients aged 18 years or older with advanced NSCLC, prior exposure to crizotinib, and Eastern Cooperative Oncology Group Performance Status (PS) of 0 to 2. Data analysis was performed from October 2020 to March 2021.

EXPOSURES

Initiation of alectinib or ceritinib therapy.

MAIN OUTCOMES AND MEASURES

The main outcome was OS.

RESULTS

In total, there were 355 patients: 183 (85 men [46.4%]) in the alectinib trial, 91 (43 men [47.3%]) in the ceritinib RWD group, and 81 (38 men [46.9%]) in the alectinib RWD group. Patients in the alectinib trial were younger (mean [SD] age, 52.53 [11.18] vs 57.97 [11.71] years), more heavily pretreated (mean [SD] number of prior therapy lines, 1.95 [0.72] vs 1.47 [0.81]), and had more favorable baseline ECOG PS (ECOG PS of 0 or 1, 165 patients [90.2%] vs 37 patients [77.1%]) than those in the ceritinib RWD group. The alectinib RWD group (mean [SD] age, 58.69 [11.26] years) had more patients with favorable ECOG PS (ECOG PS of 0 or 1, 49 patients [92.4%] vs 37 patients [77.1%]) and more White patients (56 patients [72.7%] vs 53 patients [62.4%]) compared with the ceritinib group. Compared with ceritinib RWD, alectinib-exposed patients had significantly longer OS in alectinib trials (adjusted hazard ratio [HR], 0.59; 95% CI, 0.44-0.75; P < .001) and alectinib RWD (HR, 0.46; 95% CI, 0.29-0.63; P < .001) after adjustment for baseline confounders. For the worst-case HR estimate of 0.59, residual confounding by a hypothetical confounder associated with mortality and treatment by a risk ratio greater than 2.24 was required to reverse the findings. Conclusions were robust to plausible deviations from random missingness for missing ECOG PS and underrecorded comorbidities and central nervous system metastases in RWD.

CONCLUSIONS AND RELEVANCE

Alectinib exposure was associated with longer OS compared with ceritinib in patients with ALK-positive NSCLC, and only substantial levels of bias examined reversed the findings. These findings suggest that quantitative bias analysis can be a useful tool to address uncertainty of findings for decision-makers considering RWD.

Conducting Real-world Evidence Studies on the Clinical Outcomes of Diabetes Treatments

Real-world evidence (RWE), the understanding of treatment effectiveness in clinical practice generated from longitudinal patient-level data from the routine operation of the healthcare system, is thought to complement evidence on the efficacy of medications from randomized controlled trials (RCTs). RWE studies follow a structured approach. (1) A design layer decides on the study design, which is driven by the study question and refined by a medically informed target population, patient-informed outcomes, and biologically informed effect windows. Imagining the randomized trial we would ideally perform before designing an RWE study in its likeness reduces bias; the new-user active comparator cohort design has proven useful in many RWE studies of diabetes treatments. (2) A measurement layer transforms the longitudinal patient-level data stream into variables that identify the study population, the pre-exposure patient characteristics, the treatment, and the treatment-emergent outcomes. Working with secondary data increases the measurement complexity compared to primary data collection that we find in most RCTs. (3) An analysis layer focuses on the causal treatment effect estimation. Propensity score analyses have gained in popularity to minimize confounding in healthcare database analyses. Well-understood investigator errors, like immortal time bias, adjustment for causal intermediates, or reverse causation, should be avoided. To increase reproducibility of RWE findings, studies require full implementation transparency. This article integrates state-of-the-art knowledge on how to conduct and review RWE studies on diabetes treatments to maximize study validity and ultimately increased confidence in RWE-based decision making.

Variation in health plan coverage of ESAs for anemia due to chronic kidney disease

BACKGROUND: Because health plans each issue their own policies, drug coverage can vary. This variation can result in patients having unequal access to treatment. In this study, we evaluate commercial health plans' coverage policies for erythropoiesis-stimulating agents (ESAs) for patients with anemia resulting from chronic kidney disease (CKD). OBJECTIVES: To assess how a set of US commercial health plans cover ESAs for patients with anemia due to CKD. Our second objective was to examine the evidence that the plans reviewed when formulating their coverage policies. METHODS: We used the Tufts Medical Center Specialty Drug and Evidence and Coverage Database to identify coverage policies issued by 17 of the largest US commercial health plans for ESAs. The following drugs were indicated for anemia due to CKD: darbepoetin alfa, methoxy polyethylene glycol-epoetin beta, epoetin alfa (available as two brands), and epoetin alfa-epbx. Coverage policies were current as of May 2019. We determined whether the health plans applied any restrictions, such as step therapy protocols or patient subgroup restrictions, in their coverage policies. We categorized the evidence that plans cited to support their policies into seven categories: randomized controlled trials (RCTs), real-world evidence (RWE) studies (studies based on data collected in a real-world setting), other clinical studies (eg, single arm trials), systematic reviews and/or meta-analyses, clinical or treatment guidelines, health technology assessments, and economic evaluations. RESULTS: We categorized 72.5% of coverage policies (58/80 policies) as equivalent to the FDA label and 27.5% (22/80 policies) as more restrictive. In restricted policies, plans most often applied step therapy protocols (18/22 policies), followed by prescriber requirements (4/22 policies), and patient subgroup restrictions (3/22 policies). Five health plans applied restrictions in at least half of their coverage policies; seven plans did not apply restrictions in any policy. Plans that cited evidence reviewed an average of 10 citations across their ESA coverage policies, ranging from one to 29 studies. Plans varied with respect to the types of cited studies: at least 50% of evidence cited by five health plans was RCTs, while half or more of the evidence cited by four health plans was clinical or treatment guidelines. CONCLUSIONS: Health plans varied in how they covered ESAs for patients with anemia due to CKD and in the evidence cited in their coverage policies. Inconsistencies in plans' coverage policies may have implications for patients' access to ESAs. DISCLOSURES: This study was funded by Otsuka Pharmaceutical Development and Commercialization. Sanon, Redmond, and Mogahadam are employed by Otsuka Pharmaceutical. Michalopoulos was employed by Otsuka Pharmaceutical at the time of this study. Margaretos, Panzer, and Chambers are employed by Tufts Medical Center, Institute for Clinical Research and Health Policy Studies, Center for the Evaluation of Value and Risk in Health. Lai was with Tufts Medical Center, Institute for Clinical Research and Health Policy Studies, Center for the Evaluation of Value and Risk in Health at the time of this study.

Payer perceptions of the use of real-world evidence in oncology-based decision making

BACKGROUND: Randomized controlled trials (RCTs), the gold standard of safety and efficacy evidence, are conducted in select patients that may not mirror real-world populations. As a result, healthcare decision makers may have limited information when making formulary decisions, especially in oncology, given accelerated regulatory approvals and niche patient populations. Real-world evidence (RWE) studies may help address these knowledge gaps and help inform oncology formulary decision making. OBJECTIVE: To assess US payer perceptions regarding the use and relevance of RWE in informing oncology formulary decisionmaking. METHODS: A national survey containing single-answer, multiple-answer, and free-response questions evaluated 4 key areas: (1) the value of RWE, (2) barriers to RWE, (3) sources of RWE, and (4) use of RWE in outcomes-based contracting. The survey was distributed to 221 US payers through the Academy of Managed Care Pharmacy (AMCP) Market Insights program in February 2020. Ten additional respondents were invited to discuss the survey results. The survey results were presented primarily as frequencies of responses and were evaluated by the respondent's plan size, type, and geography (regional vs national). Differences in responses for categorical data were compared using a Pearson Chi-Square or a Fisher's Exact test. Two-tailed values are reported and a level of ≤ 0.05 was used to indicate statistical significance. RESULTS: The national survey had a 45.9% response rate, with 106 payers responding. Most were from managed care organizations (MCOs; 47.5%) and pharmacy benefit managers (PBMs; 37.4%), with 54.5% from large plans (≥ 1 million lives) and 45.5% from small plans (< 1 million lives). Respondents were largely pharmacists (89.9%), with 55.6% overall indicating their job was a pharmacy administrator. Most (84.9%) used RWE to inform formulary decisions in oncology to support comparative effectiveness in the absence of head-to-head clinical trials (4.1 on a scale of 1 = Not At All Useful to 5 = Extremely Useful) and validation of National Comprehensive Cancer Network (NCCN) recommendations (4.0). Almost half (41.5%) used RWE results to inform off-label usage decisions. Payers valued RWE pre-launch to inform formulary and contracting decisions and desired real-world comparative effectiveness data post-launch to validate coverage decisions. However, the majority of payers (54.7%) did not conduct their own real-world studies. Commonly considered RWE sources included claims data (79.2%), medical records (68.9%), prospective cohort studies (60.4%), patient registries (36.8%), and patient outcome surveys (33.0%). Barriers to conducting internal RWE studies included the lack of resources and personnel, analytic capabilities, appropriate in-house data, and perceived value in conducting analyses. Payers expressed interest in using outcomes-based contracting in oncology; few have direct experience, and operationalizing through value measurement is challenging. CONCLUSIONS: RWE providing comparative treatment data, validation of NCCN treatment recommendations, and information on off-label usage are appreciated pre launch with post launch validation. DISCLOSURES: Pfizer provided funding for this research, and employees of Pfizer led the development of the survey and contributed to the manuscript as authors. Arondekar and Niyazov are employees of Pfizer; Oderda, Biskupiak, and Brixner are managers of Millcreek Outcomes Group and were paid as consultants on this project. Burgoyne was a consultant for Pfizer on this project. Malone was paid by Millcreek Outcomes as a consultant on this project.

Real-World Evidence: Bridging Gaps in Evidence to Guide Payer Decisions

Randomized controlled trials (RCTs) are preferred by payers for health technology assessments and coverage decisions. However, the inclusion of a highly selective patient population and the rigorously controlled conditions in RCTs may not be reflective of real-world clinical practice. Real-world evidence (RWE) obtained from an analysis of real-world data (RWD) from observational studies can bridge gaps in evidence not addressed by RCTs and is thus valuable to public and private payers for decision-making. Through a broad literature search to obtain insights into payers' experience, we found that payers have concerns about real-world studies with respect to data quality, poor internal validity, potential bias, and lack of meaningful endpoints. However, they valued RWE to fill evidence gaps not addressed by RCTs, such as high-quality, real-world, long-term effectiveness and safety data; head-to-head drug comparisons; cost analyses for tiering formulary placement; medication use and adherence patterns; identification of relevant responder and non-responder patient subpopulations; and patient-reported outcomes (PROs). RWE can be used to assess clinically meaningful endpoints and gauge the impact of interventions on the quality of healthcare. Here, we review how payers use or can use RWD on the comparative effectiveness and safety of treatments, PROs, medication adherence and persistence, prescribing patterns, healthcare resource utilization, and patient characteristics and/or biomarkers associated with treatment response when making health technology assessments and payer coverage decisions across therapeutic areas.

Accelerating Rare Disease Drug Development: Lessons Learned from Muscular Dystrophy Patient Advocacy Groups

With scientific and molecular advancements related to disease pathogenesis, advances in gene and stem cell therapies, and the promise of lucrative markets for biopharmaceutical companies, there has been a rapid expansion in the number of potential new muscular dystrophy (MD) treatments. The first champion for a newly diagnosed MD patient and their caregivers is typically an MD-specific patient advocacy group (PAG). Muscular dystrophy PAGs have been among the most active in the rare disease drug development space. Notable achievements in the last decade include promulgating the first U.S. clinical research guidance, setting up registries and natural history studies, and investing in companies-some of which have brought potentially disease-modifying products to the market. This paper will discuss five key strategies that have been successfully employed by MD PAGs to advance treatments: (1) creating a national registry, (2) understanding the barriers to identifying patients with certain subtypes of muscular dystrophy to participate in clinical trials, (3) partnering with the biopharmaceutical industry, (4) collaborating with the regulators, and (5) incorporating market access and use insights early in clinical development. While clearly helpful within the MD community, these tactics could also be employed by PAGs representing other types of rare diseases.

Real-World Evidence: A Primer

SUMMARY

The insights that real-world data (RWD) can provide, beyond what can be learned within the traditional clinical trial setting, have gained enormous traction in recent years. RWD, which are increasingly available and accessible, can further our understanding of disease, disease progression, and safety and effectiveness of treatments with the speed and accuracy required by the health care environment and patients today. Over the decades since RWD were first recognized, innovation has evolved to take real-world research beyond finding ways to identify, store, and analyze large volumes of data. The research community has developed strong methods to address challenges of using RWD and as a result has increased the acceptance of RWD in research, practice, and policy. Historic concerns about RWD relate to data quality, privacy, and transparency; however, new tools, methods, and approaches mitigate these challenges and expand the utility of RWD to new applications. Specific guidelines for RWD use have been developed and published by numerous groups, including regulatory authorities. These and other efforts have shown that the more RWD are used and understood and the more the tools for handling it are refined, the more useful it will be.

Cost-effectiveness of GLP-1 receptor agonists versus insulin for the treatment of type 2 diabetes: a real-world study and systematic review

BACKGROUND

To conduct a real-word-study-based cost-effectiveness analysis of a GLP-1 receptor agonist (GLP-1RA) versus insulin among type 2 diabetes patients requiring intensified injection therapy and a systematic review of cost-effectiveness studies of GLP-1RAs versus insulin.

METHODS

Individual-level analyses incorporating real-world effectiveness and cost data were conducted for a cohort of 1022 propensity-score-matched pairs of GLP-1RA and insulin users from Taiwan's National Health Insurance Research Database, 2007-2016. Study outcomes included the number needed to treat (NNT) to prevent one case of clinical events, healthcare costs, and cost per case of event prevented. Costs were in 2019 US dollars. Analyses were performed from a third-party payer and healthcare sector perspectives. Structured systematic review procedures were conducted to synthesize updated evidence on the cost-effectiveness of GLP-1RAs versus insulin.

RESULTS

Over a mean follow-up of 2.3 years, the NNT using a GLP-1RA versus insulin to prevent one case of all-cause mortality and hospitalized hypoglycemia was 57 and 30, respectively. Using GLP-1RAs instead of insulin cost US$54,851 and US$29,115 per case of all-cause mortality and hospitalized hypoglycemia prevented, respectively, from the payer perspective, and saved US$19,391 and US$10,293, respectively, from the healthcare sector perspective. Sensitivity analyses showed that the probability of using GLP-1RAs versus insulin being cost-effective for preventing one case of all-cause mortality or hospitalized hypoglycemia ranged from 60 to 100%. The systematic review revealed a cost-effective profile of using GLP-1RAs versus insulin.

CONCLUSIONS

Using GLP-1RAs versus insulin for type 2 diabetes patients requiring intensified injection therapy in clinical practice is cost-effective.

Online tools to synthesize real-world evidence of comparative effectiveness research to enhance formulary decision making

Results of randomized controlled trials (RCTs) provide valuable comparisons of 2 or more interventions to inform health care decision making; however, many more comparisons are required than available time and resources to conduct them. Moreover, RCTs have limited generalizability. Comparative effectiveness research (CER) using real-world evidence (RWE) can increase generalizability and is important for decision making, but use of nonrandomized designs makes their evaluation challenging. Several tools are available to assist. In this study, we comparatively characterize 5 tools used to evaluate RWE studies in the context of making health care adoption decision making: (1) Good Research for Comparative Effectiveness (GRACE) Checklist, (2) IMI GetReal RWE Navigator (Navigator), (3) Center for Medical Technology Policy (CMTP) RWE Decoder, (4) CER Collaborative tool, and (5) Real World Evidence Assessments and Needs Guidance (REAdi) tool. We describe each and then compare their features along 8 domains: (1) objective/user/context, (2) development/scope, (3) platform/presentation, (4) user design, (5) study-level internal/external validity of evidence, (6) summarizing body of evidence, (7) assisting in decision making, and (8) sharing results/making improvements. Our summary suggests that the GRACE Checklist aids stakeholders in evaluation of the quality and applicability of individual CER studies. Navigator is a collection of educational resources to guide demonstration of effectiveness, a guidance tool to support development of medicines, and a directory of authoritative resources for RWE. The CMTP RWE Decoder aids in the assessment of relevance and rigor of RWE. The CER Collaborative tool aids in the assessment of credibility and relevance. The REAdi tool aids in refinement of the research question, study retrieval, quality assessment, grading the body of evidence, and prompts with questions to facilitate coverage decisions. All tools specify a framework, were designed with stakeholder input, assess internal validity, are available online, and are easy to use. They vary in their complexity and comprehensiveness. The RWE Decoder, CER Collaborative tool, and REAdi tool synthesize evidence and were specifically designed to aid formulary decision making. This study adds clarity on what the tools provide so that the user can determine which best fits a given purpose. DISCLOSURES: This work was supported by the Health Tech Fund, which was provided to the University of Washington School of Pharmacy by its Corporate Advisory Board. This consortium of pharmaceutical and biotech companies supports the research program of the University of Washington School of Pharmacy across the competitive space. The sponsors seeded the idea for the project and contributed to study design and improvement. The authors had full control of all content development, manuscript drafting, and submission for publication. The REAdi tool was developed by the authors. Chen, Bansal, Barthold, Carlson, Veenstra, Basu, Devine, Yun, Ta, and Beal were supported by a training grant from the University of Washington-Allergan Fellowship, unrelated to this work. Basu reports personal fees from Salutis Consulting, unrelated to this work. Graff is an employee of the National Pharmaceutical Council, which was a partner in the development of the CER Collaborative and funding partner for the CMTP RWE Decoder and the GRACE Checklist. A previous version of this work was presented as an invited workshop at AMCP Nexus 2018; October 22-25, 2018; Orlando, FL.

Use of real-world evidence in economic assessments of pharmaceuticals in the United States

BACKGROUND: Despite the increasing interest in expanding the use of real-world evidence (RWE) in economic assessments of pharmaceuticals, decision makers face uncertainty about how RWE should be used. OBJECTIVE: To assess the use of RWE in economic assessments of drugs by the Institute for Clinical and Economic Review (ICER). METHODS: We reviewed cost-effectiveness and budget impact analyses in final evidence reports of pharmaceuticals published by ICER. We calculated the total number of RWE uses and the proportion of model inputs informed by RWE per report. We classified model inputs into 15 categories based on their attributes and then examined what category each RWE informed to classify the reason for RWE use. Finally, we characterized RWE by study design, data source, and sponsor type. RESULTS: We identified 33 reports, all of which used RWE; the mean RWE use per report was 12 (range = 4-26). The average proportion of model inputs informed by RWE per report was 32.7%, but this proportion had a wide range (range = 4.1%-76.9%). RWE was most commonly used for disease progression inputs (28.7%) and health care resource utilization and costs (21.1%), but was rarely used for drug-specific clinical outcomes such as effectiveness (1.5%), adverse drug event rates (0.5%), and discontinuation rates (1.2%). The most frequently used study design was a retrospective cohort (56.6%), and the most frequently used data source was registry data (41.4%). About a third (30.2%) of RWE was industry-sponsored. CONCLUSIONS: RWE has been commonly used to inform pharmaceutical value assessments conducted by ICER. However, there has been relatively limited use of RWE to inform drug-specific effectiveness, despite calls for greater inclusion of RWE in value assessments for real-world drug effectiveness. DISCLOSURES: This study was funded by the University of Washington School of Pharmacy Corporate Advisory Board Health Tech Fund (HTF). The funder had no role in management, analysis, and interpretation of the data; preparation, review, and approval of the manuscript; and the decision to submit the manuscript for publication. All authors were employed by the CHOICE Institute, University of Washington School of Pharmacy, at the time of the study. Carlson reports grants from the Institute for Clinical and Economic Review during the conduct of the study and personal fees from Bayer, Adaptive Biotechnologies, Allergan, Galderma, and ViFor Pharma, unrelated to this study. Veenstra reports personal fees from several manufacturers unrelated to this study. The other authors have nothing to disclose.

The use of real-world evidence in ICER's scoping process and clinical evidence assessments

BACKGROUND: There has been growing interest in using real-world evidence (RWE) for health technology assessment (HTA) in the United States. The Institute for Clinical and Economic Review (ICER) is an independent U.S.-based HTA organization that focuses primarily on pharmaceuticals. RWE is used to inform ICER's scoping and comparative clinical effectiveness (CCE) assessments, but the extent to which it is used has not been quantified. OBJECTIVE: To systematically evaluate use of RWE in the scoping and CCE assessment sections of the ICER HTA reports on pharmaceuticals. METHODS: We reviewed all ICER reports of pharmaceuticals published between January 2014 and June 2019. We examined the average number of instances and the proportion of RWE use in the scoping documents to inform the population, intervention, comparator, outcome, setting, or timing (PICOTS) elements of the appraisal. We also examined the average number of instances and the proportion of RWE use in the CCE assessments to inform effectiveness, safety, or treatment patterns. Finally, we evaluated use of RWE in clinical guidelines that were cited in the CCE assessments. RESULTS: In ICER scoping documents, the mean (SD) number of instances of RWE use was 3.8 (3.7) per document (55% for outcomes, 20% for population, 14% for comparator, 11% for intervention, and 0% for timing and setting). In ICER CCE assessments, the mean (SD) number of instances per assessment was 0.7 (0.5) per drug (53% for effectiveness, 44% for safety, and 3% for treatment patterns). In clinical guidelines used in ICER reports, the mean (SD) number of instances of RWE use was 1.6 (2.3) per drug per guideline (41% for effectiveness, 30% for safety, and 29% for treatment patterns). CONCLUSIONS: RWE was frequently used in the ICER scoping process, particularly to inform selection of outcomes. RWE was used infrequently in ICER CCE assessments, while more often used to inform effectiveness, safety, and treatment patterns in relevant clinical guidelines. There are opportunities to increase the use of RWE in U.S. HTA processes. DISCLOSURES: This study was supported by the Health Tech Fund, University of Washington School of Pharmacy, which was created through unrestricted support from several health care industry companies. Veenstra and Carlson report grant support from the Institute for Clinical and Economic Review outside the submitted work. Carlson reports personal fees from Bayer, Allergan, and Galderma outside the submitted work. Jiao, Lee, and Devine report no support outside the submitted work.

Use of real-world evidence in cancer drug funding decisions in Canada: a qualitative study of stakeholders' perspectives

BACKGROUND

Real-world evidence (RWE) can provide postmarket data to inform whether funded cancer drugs yield expected outcomes and value for money, but it is unclear how to incorporate RWE into Canadian cancer drug funding decisions. As part of the Canadian Real-World Evidence Value for Cancer Drugs (CanREValue) Collaboration, this study aimed to explore stakeholder perspectives on the current state of RWE in Canada to inform a Canadian framework for use of RWE in cancer drug funding decisions.

METHODS

This was a qualitative descriptive study. Qualitative semistructured interviews were conducted from April to July 2018. Participants were Canadian and international stakeholders who had experience with RWE and drug funding decision-making. Thematic analysis was used to analyze data.

RESULTS

Thirty stakeholders participated in the study. Five themes were identified. Stakeholders indicated that RWE had value in cancer drug funding decisions. However, a cultural shift is needed to adopt RWE in decision-making. Further, the Canadian infrastructure for real-world data is currently inadequate for decision-making, and there is a need for committed investment in building capacity to collect and analyze RWE. Finally, there is a need for increased collaboration among key stakeholders.

INTERPRETATION

The findings of this study suggest that if RWE is to be used in drug funding decisions, there is a need for a cultural shift, improved data infrastructure, committed investment in capacity building and increased stakeholder collaboration. Together with local stakeholder engagement, application of these findings may contribute to optimizing implementation of RWE.

Improving transparency to build trust in real-world secondary data studies for hypothesis testing-Why, what, and how: recommendations and a road map from the real-world evidence transparency initiative

Real-world data (RWD) and the derivations of these data into real-world evidence (RWE) are rapidly expanding from informing healthcare decisions at the patient and health system level to influencing major health policy decisions, including regulatory approvals and coverage. Recent examples include the approval of palbociclib in combination with endocrine therapy for male breast cancer and the inclusion of RWE in the label of paliperidone palmitate for schizophrenia. This interest has created an urgency to develop processes that promote trust in the evidence-generation process. Key stakeholders and decision-makers include patients and their healthcare providers; learning health systems; health technology assessment bodies and payers; pharmacoepidemiologists and other clinical reseachers, and policy makers interested in bioethical and regulatory issues. A key to optimal uptake of RWE is transparency of the research process to enable decision-makers to evaluate the quality of the methods used and the applicability of the evidence that results from the RWE studies. Registration of RWE studies-particularly for hypothesis evaluating treatment effectiveness (HETE) studies-has been proposed to improve transparency, trust, and research replicability. Although registration would not guarantee better RWE studies would be conducted, it would encourage the prospective disclosure of study plans, timing, and rationale for modifications. A joint task force of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the International Society for Pharmacoepidemiology (ISPE) recommended that investigators preregister their RWE studies and post their study protocols in a publicly available forum before starting studies to reduce publication bias and improve the transparency of research methods. Recognizing that published recommendations alone are insufficient, especially without accessible registration options and with no incentives, a group of experts gathered on February 25 and 26, 2019, in National Harbor, Maryland, to explore the structural and practical challenges to the successful implementation of the recommendations of the ISPOR/ISPE task force for preregistration. This positioning article describes a plan for making registration of HETE RWE studies routine. The plan includes specifying the rationale for registering HETE RWE studies, the studies that should be registered, where and when these studies should be registered, how and when analytic deviations from protocols should be reported, how and when to publish results, and incentives to encourage registration. Table 1 summarizes the rationale, goals, and potential solutions that increase transparency, in addition to unique concerns about secondary data studies. Definitions of terms used throughout this report are provided in Table 2.

Patterns of prescription, hospitalizations and costs of herpes zoster in patients at risk, from a large Italian claims database

Purpose

This observational study aimed to investigate the incidence of herpes zoster (HZ) among at-risk subjects aged ≥50 years, characterize them and assess annual healthcare utilization and costs from the Italian National Health System (NHS) perspective.

Methods

Records of reimbursed drug prescriptions, hospitalizations and outpatient specialist care from the Fondazione ReS database were linked to identify patients aged ≥50 years at HZ risk (i.e. cardiovascular disease/chronic obstructive pulmonary disease/diabetes/immunosuppression, according to the Italian National Vaccine Prevention Plan - PNPV 2017-2019) in 2013. New HZ events (incidence per 1,000) were researched in 2 years, and subjects with HZ in the previous year were excluded. Antiviral and pain therapy consumptions, hospitalizations for HZ and costs paid by NHS were assessed annually.

Results

From 12,562,609 inhabitants in 2013, a total of 1,004,705 patients (18.5% aged ≥50 years) at risk without a previous event were selected. The 2-year incidence of HZ was 5.9 per 1,000 (mean age 74 ± 10 years; 54.3% female). Patients aged 80-89 (7.2 per 1,000), females (6.7 per 1,000) and immunosuppressed subjects (6.9 per 1,000) had the highest incidence rates. One year after the new HZ episode, 82.2% were treated with specific antivirals (79.3% brivudine), generating an annual average cost/treated of €106; 8.0% were hospitalized for HZ, with an average cost/hospitalized of €3,927; the overall mean cost/incident patient was €402.

Conclusions

This analysis provided HZ incidence in subjects aged ≥50 years considered at risk by the PNPV and its burden from the NHS perspective. Our findings can help health governance to improve clinical decisions and economic positioning concerning zoster vaccine plan.

Improving Transparency to Build Trust in Real-World Secondary Data Studies for Hypothesis Testing-Why, What, and How: Recommendations and a Road Map from the Real-World Evidence Transparency Initiative

Real-world data (RWD) and the derivations of these data into real-world evidence (RWE) are rapidly expanding from informing healthcare decisions at the patient and health system level to influencing major health policy decisions, including regulatory approvals and coverage. Recent examples include the approval of palbociclib in combination with endocrine therapy for male breast cancer and the inclusion of RWE in the label of paliperidone palmitate for schizophrenia. This interest has created an urgency to develop processes that promote trust in the evidence-generation process. Key stakeholders and decision-makers include patients and their healthcare providers; learning health systems; health technology assessment bodies and payers; pharmacoepidemiologists and other clinical reseachers, and policy makers interested in bioethical and regulatory issues. A key to optimal uptake of RWE is transparency of the research process to enable decision-makers to evaluate the quality of the methods used and the applicability of the evidence that results from the RWE studies. Registration of RWE studies-particularly for hypothesis evaluating treatment effectiveness (HETE) studies-has been proposed to improve transparency, trust, and research replicability. Although registration would not guarantee better RWE studies would be conducted, it would encourage the prospective disclosure of study plans, timing, and rationale for modifications. A joint task force of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the International Society for Pharmacoepidemiology (ISPE) recommended that investigators preregister their RWE studies and post their study protocols in a publicly available forum before starting studies to reduce publication bias and improve the transparency of research methods. Recognizing that published recommendations alone are insufficient, especially without accessible registration options and with no incentives, a group of experts gathered on February 25 and 26, 2019, in National Harbor, Maryland, to explore the structural and practical challenges to the successful implementation of the recommendations of the ISPOR/ISPE task force for preregistration. This positioning article describes a plan for making registration of HETE RWE studies routine. The plan includes specifying the rationale for registering HETE RWE studies, the studies that should be registered, where and when these studies should be registered, how and when analytic deviations from protocols should be reported, how and when to publish results, and incentives to encourage registration. Table 1 summarizes the rationale, goals, and potential solutions that increase transparency, in addition to unique concerns about secondary data studies. Definitions of terms used throughout this report are provided in Table 2.

Use of Real-World Evidence in US Payer Coverage Decision-Making for Next-Generation Sequencing-Based Tests: Challenges, Opportunities, and Potential Solutions

OBJECTIVES

Given the potential of real-world evidence (RWE) to inform understanding of the risk-benefit profile of next-generation sequencing (NGS)-based testing, we undertook a study to describe the current landscape of whether and how payers use RWE as part of their coverage decision making and potential solutions for overcoming barriers.

METHODS

We performed a scoping literature review of existing RWE evidentiary frameworks for evaluating new technologies and identified barriers to clinical integration and evidence gaps for NGS. We synthesized findings as potential solutions for improving the relevance and utility of RWE for payer decision-making.

RESULTS

Payers require evidence of clinical utility to inform coverage decisions, yet we found a relatively small number of published RWE studies, and these are predominately focused on oncology, pharmacogenomics, and perinatal/pediatric testing. We identified 3 categories of innovation that may help address the current undersupply of RWE studies for NGS: (1) increasing use of RWE to inform outcomes-based contracting for new technologies, (2) precision medicine initiatives that integrate clinical and genomic data and enable data sharing, and (3) Food and Drug Administration reforms to encourage the use of RWE. Potential solutions include development of data and evidence review standards, payer engagement in RWE study design, use of incentives and partnerships to lower the barriers to RWE generation, education of payers and providers concerning the use of RWE and NGS, and frameworks for conducting outcomes-based contracting for NGS.

CONCLUSIONS

We provide numerous suggestions to overcome the data, methodologic, infrastructure, and policy challenges constraining greater integration of RWE in assessments of NGS.

Transparency in real-world evidence (RWE) studies to build confidence for decision-making: Reporting RWE research in diabetes

Transparency of real-world evidence (RWE) studies is critical to understanding how findings of a specific study were derived and is a necessary foundation to assessing validity and determination of whether decisions should be informed by the findings. In the present paper, we lay out strategies to improve clarity in the reporting of comparative effectiveness studies using real-world data that were generated by the routine operation of a healthcare system. This may include claims data, electronic health records, wearable devices, patient-reported outcomes or patient registries. These recommendations were discussed with multiple stakeholders, including regulators, payers, academics and journal editors, and endorsed by two professional societies that focus on RWE. We remind readers interested in diabetes research of the utility of conceptualizing a target trial that is then emulated by a RWE study when planning and communicating about RWE study implementation. We recommend the use of a graphical representation showcasing temporality of key longitudinal study design choices. We highlight study elements that should be reported to provide the clarity necessary to make a study reproducible. Finally, we suggest registering study protocols to increase process transparency. With these tools the readership of diabetes RWE studies will be able to more efficiently understand each study and be more able to assess a study's validity with reasonably high confidence before making decisions based on its findings.

What types of real-world evidence studies do U.S. commercial health plans cite in their specialty drug coverage decisions?

PURPOSE

To examine the RWE U.S. commercial health plans cite in their specialty drug coverage decisions.

METHODS

We used the Tufts Medical Center Specialty Drug Evidence and Coverage Database to identify specialty drug coverage decisions (n = 7267) issued by 17 large commercial health plans. We categorized the clinical evidence plans cited in these coverage decisions (n = 5227) as randomized controlled trials (RCTs), RWE studies, and other clinical studies (studies other than RCT or RWE study). We categorized RWE studies with respect to study type, for example, case series, studies based on medical records, and so on. We compared the frequency that plans cited different categories of RWE, cited RWE for different diseases, and cited RWE for drugs on the market for different time periods.

RESULTS

RWE comprised 16% of cited clinical studies. Health plans cited RWE with different frequencies (5%-31% of the cited clinical evidence). Overall, plans cited RWE categorized as medical records most often (26% of cited RWE studies). Plans varied in the frequency they cited different RWE categories. Plans most frequently cited RWE for gastroenterological diseases (35% of clinical study citations) and least frequently for respiratory diseases (11% of clinical study citations). Plans cited RWE more for drugs that have long been on the market.

CONCLUSIONS

Health plans varied with respect to the number and types of RWE studies they cited in their specialty drug coverage decisions. Plans cited RWE more often for some diseases than others, and cited more RWE for older drugs.

Real-world evidence to support regulatory decision-making for medicines: Considerations for external control arms

Randomized clinical trials (RCTs) are the gold standard in producing clinical evidence of efficacy and safety of medical interventions. More recently, a new paradigm is emerging-specifically within the context of preauthorization regulatory decision-making-for some novel uses of real-world evidence (RWE) from a variety of real-world data (RWD) sources to answer certain clinical questions. Traditionally reserved for rare diseases and other special circumstances, external controls (eg, historical controls) are recognized as a possible type of control arm for single-arm trials. However, creating and analyzing an external control arm using RWD can be challenging since design and analytics may not fully control for all systematic differences (biases). Nonetheless, certain biases can be attenuated using appropriate design and analytical approaches. The main objective of this paper is to improve the scientific rigor in the generation of external control arms using RWD. Here we (a) discuss the rationale and regulatory circumstances appropriate for external control arms, (b) define different types of external control arms, and (c) describe study design elements and approaches to mitigate certain biases in external control arms. This manuscript received endorsement from the International Society for Pharmacoepidemiology (ISPE).

Application of a Graphical Depiction of Longitudinal Study Designs to Managed Care Pharmacy Research

Managed care pharmacists apply real-world evidence (RWE) to support activities such as pipeline forecasting, clinical policy development, and contracting for pharmaceutical products. Managed care pharmacy researchers strive to produce studies that can be applied in practice. While asking the right research question is necessary, it is not sufficient. As with all studies, consumers of RWE look for internal and external validity, as well as sources of bias, to determine how the study findings can be applied in their work. To date, however, some of the safeguards that exist for clinical trials-such as public registration of study protocols-are lacking for RWE. Several leading professional organizations have initiatives dedicated to improving the credibility and reliability of such research. One component common to these initiatives is enhanced transparency and completeness of methodologic reporting. Graphical representations of study designs can improve the reporting and design of research conducted in health care databases, specifically by enhancing the transparency and clarity of often complex studies. As such, Schneeweiss et al. (2019) proposed a graphical framework for longitudinal study designs in health care databases. Herein, we apply this framework to 2 studies published in the Journal of Managed Care & Specialty Pharmacy that represent common research designs and report how application of the framework revealed deficiencies in reporting. We advocate for adoption of this framework in the effort to increase the usability of RWE studies using health care databases by managed care pharmacy. DISCLOSURES: No funding was provided for this work. Gatwood has received research funding from Merck & Co., AstraZeneca, and GlaxoSmithKline, unrelated to this work. Schneeweiss is a consultant to Aetion, of which he also owns equity. He is the principal investigator of investigator-initiated grants to the Brigham and Women's Hospital from Bayer, Genentech, Boehringer Ingelheim, and Vertex. Wang reports support from investigator-initiated grants from Novartis, Boehringer Ingelheim, and Johnson & Johnson to Brigham and Women's Hospital, unrelated to this work. Happe and Brown have nothing to disclose.

Actionable Real-World Evidence to Improve Health Outcomes and Reduce Medical Spending Among Risk-Stratified Patients with Diabetes

BACKGROUND

Type 2 diabetes mellitus (T2DM) is a chronic condition with a high economic burden as well as drug treatments that have not all demonstrated effects on longevity. Managed care organizations want to improve health outcomes in these complex patients but lack actionable evidence to make informed decisions on which therapies are most effective among their members and may also control total health care spending.

OBJECTIVE

To produce actionable evidence by identifying antidiabetic treatments that are effective and may reduce total cost of care in various risk groups of patients with T2DM, using insurance claims data that includes medical claims and pharmacy dispensing data among members of Horizon Blue Cross Blue Shield of New Jersey with T2DM.

METHODS

We identified patients with T2DM in longitudinal claims data from Horizon between 2014 and 2017 with demographic and enrollment information, inpatient and outpatient diagnoses and procedures, and pharmacy dispensing. Outcomes included myocardial infarction, heart failure (HF), stroke, percutaneous revascularization, health care services utilization, and plan costs (i.e., medical, pharmacy, and total cost of care). After propensity score decile adjustment on over 20 covariates, we evaluated the effectiveness and safety of second-line antidiabetic treatment that included sodium-glucose co-transporter-2 (SGLT-2) inhibitors, sulfonylureas (SUs), dipeptidyl peptidase-4 (DPP-4) inhibitors, and glucagon-like peptide-1 (GLP-1) receptor agonists.

RESULTS

Among 115,308 members with T2DM, the most common comorbidities were cardiovascular risk factors, including hyperlipidemia (56%), hypertension (50%), and existing cardiovascular disease (CVD; 55%). Among members receiving dual antidiabetic treatment (n = 20,204), the most prevalent treatments were metformin plus the following second-line medications: SUs (42%), DPP-4 inhibitors (29%), SGLT-2 inhibitors (10%), or GLP-1 receptor agonists (3%). Approximately 20% of members accounted for 79% of total cost of care, with an average of $9,605 per member per year (PMPY). Compared with SU initiation and after propensity score decile adjustment, new users of SGLT-2 inhibitors had a reduced risk for HF hospitalization (HR = 0.35, 95% CI = 0.13-0.89), hypoglycemia, albuminuria, microvascular disease, and metabolic failure. Among SGLT-2 inhibitor initiators with established CVD, the savings in total cost of care compared with SU initiators was $5,520 per member over an average treatment duration of 6 months and an approximate savings of $11,000 PMPY if patients persisted on treatment for 12 months.

CONCLUSIONS

In the Horizon membership, we confirmed that SGLT-2 inhibitors reduce HF hospitalizations, resulting in reduced medical spending and savings in total cost of care. Regulatory-grade analytics of local data provided the confidence to encourage increased SGLT-2 inhibitor use to produce better outcomes and save total cost of care despite higher pharmacy spending.

DISCLOSURES

This research did not receive outside funding; however, Aetion has since begun a contractual relationship with Horizon Blue Cross Blue Shield of New Jersey. Garry, Petruski-Ivleva, Cheever, and Rajan are employees of and have stock options in Aetion, a company that makes software for the analysis of real-world data. Eapen was an employee of Aetion during the implementation of this study. Rassen is an employee of and has ownership interest in Aetion. Murk is a consultant to Aetion of which he owns equity. Schneeweiss is a consultant to WHISCON and to Aetion, of which he also owns equity. He is the principal investigator of investigator-initiated grants to the Brigham and Women's Hospital from Bayer, Genentech, Boehringer Ingelheim, and Vertex. Gambino is an employee and officer at Horizon Blue Cross and Blue Shield of New Jersey. He was recently appointed to a board observer position at Aetion, as Horizon has small equity interest in Aetion. Jan is an employee of Rutgers State University and Horizon Blue Cross Blue Shield of New Jersey and has no conflict of interest or association with Aetion or any pharmaceutical company. Jang and Rubin are employees of Horizon Blue Cross and Blue Shield of New Jersey and have no conflict of interest or association with Aetion. This work was presented as a poster at AMCP Nexus 2018, October 22-25, 2018, in Orlando, FL; as part of a continuing education session at the AMCP Managed Care & Specialty Pharmacy 2019 Annual Meeting in San Diego, CA, March 25-28, 2019; as invited podium presenter at the Blue Cross Blue Shield 2019 National Summit conference in Grapevine, TX, April 29-May 2, 2019; and was accepted for a podium presentation at the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) 2019 annual conference in New Orleans, LA, May 18-22, 2019, where it won an award for Best Podium Presentation.

Choosing Among Common Data Models for Real-World Data Analyses Fit for Making Decisions About the Effectiveness of Medical Products

Many real-world data analyses use common data models (CDMs) to standardize terminologies for medication use, medical events and procedures, data structures, and interpretations of data to facilitate analyses across data sources. For decision makers, key aspects that influence the choice of a CDM may include (i) adaptability to a specific question; (ii) transparency to reproduce findings, assess validity, and instill confidence in findings; and (iii) ease and speed of use. Organizing CDMs preserve the original information from a data source and have maximum adaptability. Full mapping data models, or preconfigured rules systems, are easy to use, since all raw codes are mapped to medical constructs. Adaptive rule systems grow libraries of reusable measures that can easily adjust to preserve adaptability, expedite analyses, and ensure study-specific transparency.

Real-World Evidence, Causal Inference, and Machine Learning

The current focus on real world evidence (RWE) is occurring at a time when at least two major trends are converging. First, is the progress made in observational research design and methods over the past decade. Second, the development of numerous large observational healthcare databases around the world is creating repositories of improved data assets to support observational research. OBJECTIVE: This paper examines the implications of the improvements in observational methods and research design, as well as the growing availability of real world data for the quality of RWE. These developments have been very positive. On the other hand, unstructured data, such as medical notes, and the sparcity of data created by merging multiple data assets are not easily handled by traditional health services research statistical methods. In response, machine learning methods are gaining increased traction as potential tools for analyzing massive, complex datasets. CONCLUSIONS: Machine learning methods have traditionally been used for classification and prediction, rather than causal inference. The prediction capabilities of machine learning are valuable by themselves. However, using machine learning for causal inference is still evolving. Machine learning can be used for hypothesis generation, followed by the application of traditional causal methods. But relatively recent developments, such as targeted maximum likelihood methods, are directly integrating machine learning with causal inference.

Graphical Depiction of Longitudinal Study Designs in Health Care Databases

Pharmacoepidemiologic and pharmacoeconomic analysis of health care databases has become a vital source of evidence to support health care decision making and efficient management of health care organizations. However, decision makers often consider studies done in nonrandomized health care databases more difficult to review than randomized trials because many design choices need to be considered. This is perceived as an important barrier to decision making about the effectiveness and safety of medical products. Design flaws in longitudinal database studies are avoidable but can be unintentionally obscured in the convoluted prose of methods sections, which often lack specificity. We propose a simple framework of graphical representation that visualizes study design implementations in a comprehensive, unambiguous, and intuitive way; contains a level of detail that enables reproduction of key study design variables; and uses standardized structure and terminology to simplify review and communication to a broad audience of decision makers. Visualization of design details will make database studies more reproducible, quicker to review, and easier to communicate to a broad audience of decision makers.

A framework to guide the optimal development and use of real-world evidence for drug coverage and formulary decisions

AIM

To provide a framework for optimizing the development and use of real-world evidence (RWE) in drug coverage decisions. Materials & methods: The Institute for Clinical and Economic Review convened a Policy Summit with representatives from 23 payer and life science companies that compose the Institute for Clinical and Economic Review membership.

RESULTS

Summit participants helped refine a new conceptual framework that emphasizes the central role of contextual considerations and the evidentiary argument that the RWE is intended to support in designing the process for the development and interpretation of RWE.

CONCLUSION

This framework may provide a structured way for pharmaceutical manufacturers and payers to develop a shared understanding of the best way to develop RWE that will ultimately be useful in informing coverage and formulary decisions.

Interpretation and Impact of Real-World Clinical Data for the Practicing Clinician

Real-world studies have become increasingly important in providing evidence of treatment effectiveness in clinical practice. While randomized clinical trials (RCTs) are the “gold standard” for evaluating the safety and efficacy of new therapeutic agents, necessarily strict inclusion and exclusion criteria mean that trial populations are often not representative of the patient populations encountered in clinical practice. Real-world studies may use information from electronic health and claims databases, which provide large datasets from diverse patient populations, and/or may be observational, collecting prospective or retrospective data over a long period of time. They can therefore provide information on the long-term safety, particularly pertaining to rare events, and effectiveness of drugs in large heterogeneous populations, as well as information on utilization patterns and health and economic outcomes. This review focuses on how evidence from real-world studies can be utilized to complement data from RCTs to gain a more complete picture of the advantages and disadvantages of medications as they are used in practice.

Funding: Sanofi US, Inc.

Interpretation and Impact of Real-World Clinical Data for the Practicing Clinician

Real-world studies have become increasingly important in providing evidence of treatment effectiveness in clinical practice. While randomized clinical trials (RCTs) are the "gold standard" for evaluating the safety and efficacy of new therapeutic agents, necessarily strict inclusion and exclusion criteria mean that trial populations are often not representative of the patient populations encountered in clinical practice. Real-world studies may use information from electronic health and claims databases, which provide large datasets from diverse patient populations, and/or may be observational, collecting prospective or retrospective data over a long period of time. They can therefore provide information on the long-term safety, particularly pertaining to rare events, and effectiveness of drugs in large heterogeneous populations, as well as information on utilization patterns and health and economic outcomes. This review focuses on how evidence from real-world studies can be utilized to complement data from RCTs to gain a more complete picture of the advantages and disadvantages of medications as they are used in practice.Funding: Sanofi US, Inc.