Examining the Use of Real-World Evidence in the Regulatory Process

A Two-Step Framework for Validating Causal Effect Estimates

BACKGROUND

Comparing causal effect estimates obtained using observational data to those obtained from the gold standard (i.e., randomized controlled trials [RCTs]) helps assess the validity of these estimates. However, comparisons are challenging due to differences between observational data and RCT generated data. The unknown treatment assignment mechanism in the observational data and varying sampling mechanisms between the RCT and the observational data can lead to confounding and sampling bias, respectively.

AIMS

The objective of this study is to propose a two-step framework to validate causal effect estimates obtained from observational data by adjusting for both mechanisms.

MATERIALS AND METHODS

An estimator of causal effects related to the two mechanisms is constructed. A two-step framework for comparing causal effect estimates is derived from the estimator. An R package RCTrep is developed to implement the framework in practice.

RESULTS

A simulation study is conducted to show that using our framework observational data can produce causal effect estimates similar to those of an RCT. A real-world application of the framework to validate treatment effects of adjuvant chemotherapy obtained from registry data is demonstrated.

CONCLUSION

This  study constructs a framework for comparing causal effect estimates between observational data and RCT data, facilitating the assessment of the validity of causal effect estimates obtained from observational data.

Exploring the discrepancies between clinical trials and real-world data: A small-cell lung cancer study

The potential of real-world data to inform clinical trial design and supplement control arms has gained much interest in recent years. The most common approach relies on reproducing control arm outcomes by matching real-world patient cohorts to clinical trial baseline populations. However, recent studies pointed out that there is a lack of replicability, generalisability, and consensus. In this article, we propose a novel approach that aims to explore and examine these discrepancies by concomitantly investigating the impact of selection criteria and operations on the measurements of outcomes from the patient data. We tested the approach on a dataset consisting of small-cell lung cancer patients receiving platinum-based chemotherapy regimens from a real-world data cohort (n = 223) and six clinical trial control arms (n = 1224). The results showed that the discrepancy between real-world and clinical trial data potentially depends on differences in both patient populations and operational conditions (e.g., frequency of assessments, and censoring), for which further investigation is required. Discovering and accounting for confounders, including hidden effects of differences in operations related to the treatment process and clinical trial study protocol, would potentially allow for improved translation between clinical trials and real-world data. Continued development of the method presented here to systematically explore and account for these differences could pave the way for transferring learning across clinical studies and developing mutual translation between the real-world and clinical trials to inform clinical study design.

Machine Learning Methods to Estimate Individualized Treatment Effects for Use in Health Technology Assessment

BACKGROUND

Recent developments in causal inference and machine learning (ML) allow for the estimation of individualized treatment effects (ITEs), which reveal whether treatment effectiveness varies according to patients' observed covariates. ITEs can be used to stratify health policy decisions according to individual characteristics and potentially achieve greater population health. Little is known about the appropriateness of available ML methods for use in health technology assessment.

METHODS

In this scoping review, we evaluate ML methods available for estimating ITEs, aiming to help practitioners assess their suitability in health technology assessment. We present a taxonomy of ML approaches, categorized by key challenges in health technology assessment using observational data, including handling time-varying confounding and time-to event data and quantifying uncertainty.

RESULTS

We found a wide range of algorithms for simpler settings with baseline confounding and continuous or binary outcomes. Not many ML algorithms can handle time-varying or unobserved confounding, and at the time of writing, no ML algorithm was capable of estimating ITEs for time-to-event outcomes while accounting for time-varying confounding. Many of the ML algorithms that estimate ITEs in longitudinal settings do not formally quantify uncertainty around the point estimates.

LIMITATIONS

This scoping review may not cover all relevant ML methods and algorithms as they are continuously evolving.

CONCLUSIONS

Existing ML methods available for ITE estimation are limited in handling important challenges posed by observational data when used for cost-effectiveness analysis, such as time-to-event outcomes, time-varying and hidden confounding, or the need to estimate sampling uncertainty around the estimates.

IMPLICATIONS

ML methods are promising but need further development before they can be used to estimate ITEs for health technology assessments.

HIGHLIGHTS

Estimating individualized treatment effects (ITEs) using observational data and machine learning (ML) can support personalized treatment advice and help deliver more customized information on the effectiveness and cost-effectiveness of health technologies.ML methods for ITE estimation are mostly designed for handling confounding at baseline but not time-varying or unobserved confounding. The few models that account for time-varying confounding are designed for continuous or binary outcomes, not time-to-event outcomes.Not all ML methods for estimating ITEs can quantify the uncertainty of their predictions.Future work on developing ML that addresses the concerns summarized in this review is needed before these methods can be widely used in clinical and health technology assessment-like decision making.

Critical Appraisal and Future Challenges of Artificial Intelligence and Anticancer Drug Development

The conventional rules for anti-cancer drug development are no longer sufficient given the relatively limited number of patients available for therapeutic trials. It is thus a real challenge to better design trials in the context of new drug approval for anti-cancer treatment. Artificial intelligence (AI)-based in silico trials can incorporate far fewer but more informative patients and could be conducted faster and at a lower cost. AI can be integrated into in silico clinical trials to improve data analysis, modeling and simulation, personalized medicine approaches, trial design optimization, and virtual patient generation. Health authorities are encouraged to thoroughly review the rules for setting up clinical trials, incorporating AI and in silico methodology once they have been appropriately validated. This article also aims to highlight the limits and challenges related to AI and machine learning.

Disease progression strikingly differs in research and real-world Parkinson's populations

Characterization of Parkinson's disease (PD) progression using real-world evidence could guide clinical trial design and identify subpopulations. Efforts to curate research populations, the increasing availability of real-world data, and advances in natural language processing, particularly large language models, allow for a more granular comparison of populations than previously possible. This study includes two research populations and two real-world data-derived (RWD) populations. The research populations are the Harvard Biomarkers Study (HBS, N = 935), a longitudinal biomarkers cohort study with in-person structured study visits; and Fox Insights (N = 36,660), an online self-survey-based research study of the Michael J. Fox Foundation. Real-world cohorts are the Optum Integrated Claims-electronic health records (N = 157,475), representing wide-scale linked medical and claims data and de-identified data from Mass General Brigham (MGB, N = 22,949), an academic hospital system. Structured, de-identified electronic health records data at MGB are supplemented using a manually validated natural language processing with a large language model to extract measurements of PD progression. Motor and cognitive progression scores change more rapidly in MGB than HBS (median survival until H&Y 3: 5.6 years vs. >10, p < 0.001; mini-mental state exam median decline 0.28 vs. 0.11, p < 0.001; and clinically recognized cognitive decline, p = 0.001). In real-world populations, patients are diagnosed more than eleven years later (RWD mean of 72.2 vs. research mean of 60.4, p < 0.001). After diagnosis, in real-world cohorts, treatment with PD medications has initiated an average of 2.3 years later (95% CI: [2.1-2.4]; p < 0.001). This study provides a detailed characterization of Parkinson's progression in diverse populations. It delineates systemic divergences in the patient populations enrolled in research settings vs. patients in the real-world. These divergences are likely due to a combination of selection bias and real population differences, but exact attribution of the causes is challenging. This study emphasizes a need to utilize multiple data sources and to diligently consider potential biases when planning, choosing data sources, and performing downstream tasks and analyses.

The Digital Therapeutics Real-World Evidence Framework: An Approach for Guiding Evidence-Based Digital Therapeutics Design, Development, Testing, and Monitoring

Digital therapeutics (DTx) are a promising way to provide safe, effective, accessible, sustainable, scalable, and equitable approaches to advance individual and population health. However, developing and deploying DTx is inherently complex in that DTx includes multiple interacting components, such as tools to support activities like medication adherence, health behavior goal-setting or self-monitoring, and algorithms that adapt the provision of these according to individual needs that may change over time. While myriad frameworks exist for different phases of DTx development, no single framework exists to guide evidence production for DTx across its full life cycle, from initial DTx development to long-term use. To fill this gap, we propose the DTx real-world evidence (RWE) framework as a pragmatic, iterative, milestone-driven approach for developing DTx. The DTx RWE framework is derived from the 4-phase development model used for behavioral interventions, but it includes key adaptations that are specific to the unique characteristics of DTx. To ensure the highest level of fidelity to the needs of users, the framework also incorporates real-world data (RWD) across the entire life cycle of DTx development and use. The DTx RWE framework is intended for any group interested in developing and deploying DTx in real-world contexts, including those in industry, health care, public health, and academia. Moreover, entities that fund research that supports the development of DTx and agencies that regulate DTx might find the DTx RWE framework useful as they endeavor to improve how DTxcan advance individual and population health.

Transforaminal epidural steroid injection for radiculopathy and the evolution to surgical treatment: a pragmatic prospective observational multicenter study

Aim: The aim of this study is to analyze the real-world outcomes of transforaminal epidural steroid injections (TFESIs) in all patients with radiculopathy and their long-term outcomes. Methods: Patients with radiculopathy and failure of conservative treatment were included in a prospective, multicenter, observational cohort study. Results: In total, 117 patients were treated with one or two TFESIs. The mean duration of follow-up was 116 (±14) weeks. In total 19,6% (95% CI: 12.9-28.0%) patients were treated with surgery after insufficient symptom improvement. The evolution to surgery was not associated with etiology, symptom duration or previous spine surgery. Conclusion: Real-world data confirms that TFESIs is an effective treatment with satisfactory results in about 80% of patients for a period of 2 years.

Disease progression strikingly differs in research and real-world Parkinson's populations

Characterization of Parkinson's disease (PD) progression using real-world evidence could guide clinical trial design and identify subpopulations. Efforts to curate research populations, the increasing availability of real-world data and recent advances in natural language processing, particularly large language models, allow for a more granular comparison of populations and the methods of data collection describing these populations than previously possible. This study includes two research populations and two real-world data derived (RWD) populations. The research populations are the Harvard Biomarkers Study (HBS, N = 935), a longitudinal biomarkers cohort study with in-person structured study visits; and Fox Insights (N = 36,660), an online self-survey-based research study of the Michael J. Fox Foundation. Real-world cohorts are the Optum Integrated Claims-electronic health records (N = 157,475), representing wide-scale linked medical and claims data and de-identified data from Mass General Brigham (MGB, N = 22,949), an academic hospital system. Structured, de-identified electronic health records data at MGB are supplemented using natural language processing with a large language model to extract measurements of PD progression. This extraction process is manually validated for accuracy. Motor and cognitive progression scores change more rapidly in MGB than HBS (median survival until H&Y 3: 5.6 years vs. >10, p<0.001; mini-mental state exam median decline 0.28 vs. 0.11, p<0.001; and clinically recognized cognitive decline, p=0.001). In the real-world populations, patients are diagnosed more than eleven years later (RWD mean of 72.2 vs. research mean of 60.4, p<0.001). After diagnosis, in real-world cohorts, treatment with PD medications is initiated 2.3 years later on average (95% CI: [2.1-2.4]; p<0.001). This study provides a detailed characterization of Parkinson's progression in diverse populations. It delineates systemic divergences in the patient populations enrolled in research settings vs. patients in the real world. These divergences are likely due to a combination of selection bias and real population differences, but exact attribution of the causes is challenging using existing data. This study emphasizes a need to utilize multiple data sources and to diligently consider potential biases when planning, choosing data sources, and performing downstream tasks and analyses.

Effectiveness of bDMARDs in ankylosing spondylitis patients by biologic use: experience from the CorEvitas PsA/SpA Registry

OBJECTIVE

To describe bDMARD initiators by biologic experience among ankylosing spondylitis (AS) patients and change in disease activity and patient-reported outcomes (PROs) in real-world US patients.

METHODS

We included patients ≥18 years with AS based on physician diagnosis enrolled between 3/2013 and 11/2019 in the CorEvitas Psoriatic Arthritis (PSA)/Spondyloarthritis Registry (NCT02530268). Patients concurrently diagnosed with PSA were excluded. Baseline (bDMARD initiation) demographics, comorbidities, disease characteristics, treatment, and PROs were collected. Response rates and changes in disease activity and PROs between baseline and 6- and 12- month follow-up visits were calculated.

RESULTS

Of the 489 AS patients in the PsA/SpA Registry, 254 AS (52.0%) patients initiated a bDMARD at enrollment or during follow-up (total initiations: AS = 313). Of the 313 AS initiations, 179 (57.2%) had a 6-month follow-up, 122 (39.0%) had a 12-month follow-up, and 94 (30.0%) had a 6- and 12-month follow-up visit. For those AS initiators with a 6-month follow-up, the mean age was 49.1 years, 44.4% were female, and 70.4%, 47.5%, 96.1%, and 46.9% had never used cDMARDs, TNFis, non-TNFis, and bDMARDs, respectively. Of these 179 AS initiators, 20.1% and 14.0% achieved ASAS20/40, respectively. Further, only 34% achieved low disease activity (ASDAS <2.1). When stratified by biologic-naivete and biologic-experience, the ASAS 20/40 achievement rates were 26.2% and 14.7%, and 21.4% and 7.4%, respectively, for this cohort.

CONCLUSION

Although AS patients initiate bDMARDs, many do not achieve optimal treatment responses. Future research is needed to investigate the aspects associated with inadequate improvement and treatment response to bDMARDs.

Effectiveness of Single-Tablet Combination Therapy in Improving Adherence and Persistence and the Relation to Clinical and Economic Outcomes

Background: Single-tablet combination therapies (STCTs) combine multiple drugs into one formulation, making drug administration more convenient for patients. STCTs were developed to address concerns with treatment adherence and persistence, but the impact of STCT use is not fully understood across indications. Objectives: We conducted a systematic literature review (SLR) to examine STCT-associated outcomes across 4 evidence domains: clinical trials, real-world evidence (RWE), health-related quality of life (HRQoL) studies, and economic evaluations. Methods: Four SLRs were conducted across the aforementioned domains. Included studies compared STCTs as well as fixed-dose combinations ([FDCs] of non-tablet formulations) with the equivalent active compounds and doses in loose-dose combinations (LDCs). Original research articles were included; case reports, case series, and non-English-language sources were excluded. Databases searched included EconLit, Embase, and Ovid MEDLINE® ALL. Two independent reviewers assessed relevant studies and extracted data. Conflicts were resolved with a third reviewer or consensus-based discussion. Results: In all, 109 studies were identified; 27 studies were identified in more than one SLR. Treatment adherence was significantly higher in patients receiving FDCs vs LDCs in 12 of 13 RWE studies and 3 of 13 clinical trials. All 18 RWE studies reported higher persistence with FDCs. In RWE studies examining clinical outcomes (n = 17), 14 reported positive findings with FDCs, including a reduced need for add-on medication, blood pressure control, and improved hemoglobin A1C. HRQoL studies generally reported numerical improvements with STCTs or similarities between STCTs and LDCs. Economic outcomes favored STCT use. All 6 cost-effectiveness or cost-utility analyses found FDCs were less expensive and more efficacious than LDCs. Four budget impact models found that STCTs were associated with cost savings. Medical costs and healthcare resource use were generally lower with FDCs than with LDCs. Discussion: Evidence from RWE and economic studies strongly favored STCT use, while clinical trials and HRQoL studies primarily reported similarity between STCTs and LDCs. This may be due to clinical trial procedures aimed at maximizing adherence and HRQoL measures that are not designed to evaluate drug administration. Conclusions: Our findings highlight the value of STCTs for improving patient adherence, persistence, and clinical outcomes while also offering economic advantages.

Effectiveness of pharmacogenomics on the response and remission of treatment-resistant depression: a meta-analysis of randomised controlled trials

Background

Pharmacogenomics (PGx) is a promising tool to realise tailored drug therapy for depression.

Aims

To investigate the treatment efficacy of PGx for treatment-resistant depression (TRD) compared with treatment as usual.

Methods

A systematic search was conducted in PubMed, Embase, the Cochrane Library, Web of Science and PsycINFO to identify relevant studies published from inception to 15 April 2023. Two-arm randomised controlled trials (RCTs) exploring the efficacy of PGx-guided versus unguided treatment for TRD were included. The risk of bias in the included studies was evaluated using the Cochrane risk of bias assessment tool. The overall quality of evidence was assessed using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach.

Results

Seven RCTs (n=3003) comparing PGx-guided (n=1492) and unguided (n=1511) groups were identified and analysed. PGx-guided treatment was superior to treatment as usual in response (relative risk (RR)=1.31; 95% confidence interval (95% CI): 1.15 to 1.49; p<0.001) and remission (RR=1.40; 95% CI: 1.09 to 1.80; p=0.009) improvements. Effect sizes for acceptability (RR=0.90; 95% CI: 0.80 to 1.02; p=0.100) and side effect burden (RR=0.58; 95% CI: 0.29 to 1.15; p=0.120) between the two groups were not statistically different. The overall quality of evidence was rated from 'very low' (25%) to 'low' (75%) based on the GRADE criteria.

Conclusions

PGx-guided treatment has shown a small overall effect in improving the response and remission rates for patients with TRD. However, these results should be interpreted cautiously because of the few included studies and the low quality of evidence. Further high-quality clinical trials are warranted to confirm the findings.

PROSPERO registration number

CRD42022340182.

Efficacy and safety of combination therapy with pirfenidone and nintedanib in patients with idiopathic pulmonary fibrosis

Background: Recent studies have suggested that combination therapy with pirfenidone and nintedanib is safe and tolerable in patients with idiopathic pulmonary fibrosis (IPF). However, data from real-world practice are limited. Thus, we aimed to investigate the safety and efficacy of this combination therapy in patients with IPF in a real-world setting. Methods: A multicenter retrospective cohort study was conducted to investigate the safety and efficacy of combination therapy with pirfenidone and nintedanib in 45 patients with IPF. Incidences of adverse events and rates of lung function decline were compared before and after the combination therapy. Propensity score matching was performed to compare the outcomes between the combination and monotherapy groups. Results: The mean age of the patients was 68.8 years, and 82.2% of them were male. The median follow-up duration after combination therapy was 12.1 months. The majority of the patients (97.8%) received nintedanib as an add-on to pirfenidone. The most common adverse events after the combination therapy were diarrhea and anorexia. Pirfenidone or nintedanib was stopped in 12 patients owing to gastrointestinal AEs, lung transplantation, or financial problems. In patients with serial lung function data, the rate of decline in the forced vital capacity was significantly reduced after the combination therapy. In the matched analysis, the combination group had a higher incidence of diarrhea than the monotherapy group without an increase in serious adverse events; however, the two groups had similar changes in forced vital capacity (FVC). Conclusion: The combination of pirfenidone and nintedanib in patients with IPF has the potential to reduce the rate of FVC decline. However, in the matched analysis, FVC decline was comparable between the patients on combination therapy and those on monotherapy. The incidence of certain adverse events, particularly diarrhea, was higher with combination therapy, but serious adverse events were similar between the groups.

Development of a definition for real-world evidence using the Jandhyala method for observing consensus opinion among medical affairs pharmaceutical physicians

OBJECTIVE

Low rates of inclusion of real-world evidence (RWE) during regulation may arise from lack of clarity and consensus on its definition. A conceptually mature definition of RWE may have pragmatic utility, increasing its inclusion during regulation. The aim was to develop a definition of RWE to promote inclusion in regulatory submissions and assess its conceptual maturity.

METHODS

Thirteen medical affairs pharmaceutical physicians completed two qualitative online surveys to generate items needed in a definition of RWE. Items that reached a consensus index of > 50% (CI > = 0.51) were retained in the final definition. The maturity of the definition was assessed using concept analysis.

RESULTS

After attrition, 11 participants completed the study and generated 18 items to be included in a definition of RWE. All items reached the consensus threshold and were included. The definition was conceptually mature on three of the four dimensions: the potential for a consensual definition across stakeholders, a description of its characteristics and clear preconditions and outcomes. Further research is needed to delineate the boundaries of RWE.

CONCLUSIONS

A definition of RWE was generated that may increase its inclusion during medicines regulation, especially with further refinement from regulators and other stakeholders.

Tumor-derived systems as novel biomedical tools-turning the enemy into an ally

Cancer is a complex illness that presents significant challenges in its understanding and treatment. The classic definition, "a group of diseases characterized by the uncontrolled growth and spread of abnormal cells in the body," fails to convey the intricate interaction between the many entities involved in cancer. Recent advancements in the field of cancer research have shed light on the role played by individual cancer cells and the tumor microenvironment as a whole in tumor development and progression. This breakthrough enables the utilization of the tumor and its components as biological tools, opening new possibilities. This article delves deeply into the concept of "tumor-derived systems", an umbrella term for tools sourced from the tumor that aid in combatting it. It includes cancer cell membrane-coated nanoparticles (for tumor theranostics), extracellular vesicles (for tumor diagnosis/therapy), tumor cell lysates (for cancer vaccine development), and engineered cancer cells/organoids (for cancer research). This review seeks to offer a complete overview of the tumor-derived materials that are utilized in cancer research, as well as their current stages of development and implementation. It is aimed primarily at researchers working at the interface of cancer biology and biomedical engineering, and it provides vital insights into this fast-growing topic.

Clouds across the new dawn for clinical, diagnostic and biological data: accelerating the development, delivery and uptake of personalized medicine

Growing awareness of the genetic basis of disease is transforming the opportunities for improving patient care by accelerating the development, delivery and uptake of personalised medicine and diseases diagnostics. This can mean more precise treatments reaching the right patients at the right time at the right cost. But it will be possible only with a coherent European Union (EU) approach to regulation. For clinical and biological data, on which the EU is now legislating with its planned European Health Data Space (EHDS), it is crucial that the design of this new system respects the constraints also implicit in the testing which generates data. The current EHDS proposal may fail to meet this requirement. It risks being over-ambitious, while taking insufficient account of the demanding realities of data access in daily practice and current economics/business models. It is marred by imprecision and ambiguity, by overlaps with other EU legislation, and by lack of clarity on funding. This paper identifies key issues where legislators should ensure that the opportunities are not squandered by the adoption of over-hasty or ill-considered provisions that jeopardise the gains that could be made in improved healthcare.

Escitalopram and Sertraline Population Pharmacokinetic Analysis in Pediatric Patients

BACKGROUND AND OBJECTIVE

Escitalopram and sertraline are commonly prescribed for anxiety and depressive disorders in children and adolescents. The pharmacokinetics (PK) of these medications have been evaluated in adults and demonstrate extensive variability, but studies in pediatric patients are limited. Therefore, we performed a population PK analysis for escitalopram and sertraline in children and adolescents to characterize the effects of demographic, clinical, and pharmacogenetic factors on drug exposure.

METHODS

A PK dataset was generated by extracting data from the electronic health record and opportunistic sampling of escitalopram- and sertraline-treated psychiatrically hospitalized pediatric patients aged 5-18 years. A population PK analysis of escitalopram and sertraline was performed using NONMEM. Concentration-time profiles were simulated using MwPharm++ to evaluate how covariates included in the final models influence medication exposure and compared to adult therapeutic reference ranges.

RESULTS

The final escitalopram cohort consisted of 315 samples from 288 patients, and the sertraline cohort consisted of 265 samples from 255 patients. A one-compartment model with a proportional residual error model best described the data for both medications. For escitalopram, CYP2C19 phenotype and concomitant CYP2C19 inhibitors affected apparent clearance (CL/F), and normalizing CL/F and apparent volume of distribution (V/F) to body surface area (BSA) improved estimations. The final escitalopram model estimated CL/F and V/F at 14.2 L/h/1.73 m2 and 428 L/1.73 m2, respectively. For sertraline, CYP2C19 phenotype and concomitant CYP2C19 inhibitors influenced CL/F, and empirical allometric scaling of patient body weight on CL/F and V/F was significant. The final sertraline model estimated CL/F and V/F at 124 L/h/70 kg and 4320 L/70 kg, respectively. Normalized trough concentrations (Ctrough) for CYP2C19 poor metabolizers taking escitalopram were 3.98-fold higher compared to normal metabolizers (151.1 ng/mL vs 38.0 ng/mL, p < 0.0001), and normalized Ctrough for CYP2C19 poor metabolizers taking sertraline were 3.23-fold higher compared to normal, rapid, and ultrarapid metabolizers combined (121.7 ng/mL vs 37.68 ng/mL, p < 0.0001). Escitalopram- and sertraline-treated poor metabolizers may benefit from a dose reduction of 50-75% and 25-50%, respectively, to normalize exposure to other phenotypes.

CONCLUSION

To our knowledge, this is the largest population PK analysis of escitalopram and sertraline in pediatric patients. Significant PK variability for both medications was observed and was largely explained by CYP2C19 phenotype. Slower CYP2C19 metabolizers taking escitalopram or sertraline may benefit from dose reductions given increased exposure.

Real-World Data: Applications and Relevance to Cancer Clinical Trials

Randomized controlled trials (RCTs) are the gold standard for comparative-effectiveness research (CER). Since the 1980s, there has been a rise in the creation and utilization of large national cancer databases to provide readily accessible "real-world data" (RWD). This review article discusses the role of RCTs in oncology, and the role of RWD from the national cancer database in CER. RCTs remain the preferred study type for CER because they minimize confounding and bias. RCTs have challenges to conduct, including extensive time and resources, but these factors do not impact the internal validity of the result. Generalizability and external validity are potential limitations of RCTs. RWD is ideal for studying cancer epidemiology, patterns of care, disparities in care delivery, quality-of-care evaluation, and applicability of RCT data in specific populations excluded from RCTs. However, retrospective databases with RWD have limitations in CER due to unmeasured confounders and are often suboptimal in identifying causal treatment effects.

Safety and Effectiveness of Regdanvimab for COVID-19 Treatment: A Phase 4 Post-marketing Surveillance Study Conducted in South Korea

INTRODUCTION

Regdanvimab, a neutralising monoclonal antibody (mAb) against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), received approval for the treatment of coronavirus disease 2019 (COVID-19) in South Korea in 2021. The Ministry of Food and Drug Safety in South Korea mandate that new medications be re-examined for safety and effectiveness post-approval in at least 3000 individuals. This post-marketing surveillance (PMS) study was used to evaluate the safety and effectiveness of regdanvimab in real-world clinical care.

METHODS

This prospective, multicentre, phase 4 PMS study was conducted between February 2021 and March 2022 in South Korea. Eligible patients were aged ≥ 18 years with confirmed mild COVID-19 at high risk of disease progression or moderate COVID-19. Patients were hospitalised and treated with regdanvimab (40 mg/kg, day 1) and then monitored until discharge, with a follow-up call on day 28. Adverse events (AEs) were documented, and the COVID-19 disease progression rate was used to measure effectiveness.

RESULTS

Of the 3123 patients with COVID-19 infection identified, 3036 were eligible for inclusion. Approximately 80% and 5% of the eligible patients were diagnosed with COVID-19 during the delta- and omicron-dominant periods, respectively. Median (range) age was 57 (18-95) years, and 50.6% of patients were male. COVID-19 severity was assessed before treatment, and high-risk mild and moderate COVID-19 was diagnosed in 1030 (33.9%) and 2006 (66.1%) patients, respectively. AEs and adverse drug reactions (ADRs) were experienced by 684 (22.5%) and 363 (12.0%) patients, respectively. The most common ADR was increased liver function test (n = 62, 2.0%). Nine (0.3%) patients discontinued regdanvimab due to ADRs. Overall, 378 (12.5%) patients experienced disease progression after regdanvimab infusion, with extended hospitalisation/re-admission (n = 300, 9.9%) as the most common reason. Supplemental oxygen was required by 282 (9.3%) patients. Ten (0.3%) patients required intensive care monitoring and 3 (0.1%) died due to COVID-19.

CONCLUSION

This large-scale PMS study demonstrated that regdanvimab was effective against COVID-19 progression and had an acceptable safety profile when used in real-world clinical practice.

A methodology to assess the population size and estimate the needed resources for new licensed medications by combining clinical and administrative databases: The example of glycated haemoglobin in type 2 diabetes

PURPOSE

To develop and validate a model to estimate glycated haemoglobin (HbA1c) values in patients with type 2 diabetes mellitus (T2DM) using a clinical data source, with the aim to apply this equation to administrative databases.

METHODS

Using a primary care and administrative Italian databases, namely the Health Search database (HSD) and the ReS (Ricerca e Salute) database, we selected all patients aged 18 years or older on 31 December 2018 being diagnosed with T2DM and without prior prescription of sodium-glucose cotransporter-2 (SGLT-2) inhibitors. We included patients prescribed with and adherent to metformin. HSD was used to develop and test (using 2019 data as well) the algorithm imputing HbA1c values ≥7% according to a series of covariates. The algorithm was gathered by combining beta-coefficients being estimated by logistic regression models using complete case (excluding missing values) and imputed (after multiple imputation) dataset. The final algorithm was applied to ReS database using the same covariates.

RESULTS

The tested algorithms were able to explain 17%-18% variation in assessing HbA1c values. Good discrimination (70%) and calibration were obtained as well. The best algorithm (three) cut-offs, namely those providing correct classifications ranging 66%-70% was therefore calculated and applied to ReS database. By doing so, from 52 999 (27.9, 95% CI: 27.7%-28.1%) to 74 250 (40.1%, 95% CI: 38.9%-39.3%) patients were estimated with HbA1c ≥7%.

CONCLUSION

Through this methodology, healthcare authorities should be able to quantify the population eligible to a new licensed medication, such as SGLT-2 inhibitors, and to simulate scenarios to assess reimbursement criteria according to precise estimates.

Predicting non-improvement of symptoms in daily mental healthcare practice using routinely collected patient-level data: a machine learning approach

Objectives

Anxiety and mood disorders greatly affect the quality of life for individuals worldwide. A substantial proportion of patients do not sufficiently improve during evidence-based treatments in mental healthcare. It remains challenging to predict which patients will or will not benefit. Moreover, the limited research available on predictors of treatment outcomes comes from efficacy RCTs with strict selection criteria which may limit generalizability to a real-world context. The current study evaluates the performance of different machine learning (ML) models in predicting non-improvement in an observational sample of patients treated in routine specialized mental healthcare.

Methods

In the current longitudinal exploratory prediction study diagnosis-related, sociodemographic, clinical and routinely collected patient-reported quantitative outcome measures were acquired during treatment as usual of 755 patients with a primary anxiety, depressive, obsessive compulsive or trauma-related disorder in a specialized outpatient mental healthcare center. ML algorithms were trained to predict non-response (< 0.5 standard deviation improvement) in symptomatic distress 6 months after baseline. Different models were trained, including models with and without early change scores in psychopathology and well-being and models with a trimmed set of predictor variables. Performance of trained models was evaluated in a hold-out sample (30%) as a proxy for unseen data.

Results

ML models without early change scores performed poorly in predicting six-month non-response in the hold-out sample with Area Under the Curves (AUCs) < 0.63. Including early change scores slightly improved the models' performance (AUC range: 0.68-0.73). Computationally-intensive ML models did not significantly outperform logistic regression (AUC: 0.69). Reduced prediction models performed similar to the full prediction models in both the models without (AUC: 0.58-0.62 vs. 0.58-0.63) and models with early change scores (AUC: 0.69-0.73 vs. 0.68-0.71). Across different ML algorithms, early change scores in psychopathology and well-being consistently emerged as important predictors for non-improvement.

Conclusion

Accurately predicting treatment outcomes in a mental healthcare context remains challenging. While advanced ML algorithms offer flexibility, they showed limited additional value compared to traditional logistic regression in this study. The current study confirmed the importance of taking early change scores in both psychopathology and well-being into account for predicting longer-term outcomes in symptomatic distress.

Paving the way for patient centricity in real-world evidence (RWE): Qualitative interviews to identify considerations for wider implementation of patient-reported outcomes in RWE generation

Objectives

Real-world evidence (RWE) generation can be enhanced by including patient-reported outcomes (PROs). Methods for collecting and using PRO data in the real-world setting are currently underdeveloped and there is no international guidance specific to its use in this context. This study explored stakeholders' perspectives and needs for using PROs in RWE generation. Barriers, facilitators, and opportunities for wider use of PROs in real-world studies were also investigated.

Methods

Online semi-structured interviews were conducted with international stakeholders: patients, patient advocates, regulators, payers, clinicians, academic researchers, and industry experts. Interviews were recorded, transcribed verbatim and analysed using NVivo 20. Thematic analysis was conducted based on the updated Consolidated Framework for Implementation Research (CFIR).

Results

Twenty-three interviews were conducted. Participants confirmed that the use of PROs in RWE generation is not yet well established. Participants expressed a mixed level of confidence in the value of PROs collected in a real-world setting. Operational challenges associated with collecting routine PRO data to inform care delivery at the individual level (e.g., setting up infrastructure) need to be addressed. Methodological and other challenges (e.g., financing research) associated with collecting prospective de novo data in a real-world setting should be considered to facilitate PRO utilisation in real-world studies.

Conclusions

Several opportunities and challenges were identified regarding the broader use of PROs in RWE research. Joint efforts from different stakeholders are needed to maximise PRO implementation, with consideration given to each stakeholders' specific needs (e.g., by developing good practices).

Regulatory and HTA Considerations for Development of Real-World Data Derived External Controls

Regulators and Health Technology Assessment (HTA) bodies are increasingly familiar with, and publishing guidance on, external controls derived from real-world data (RWD) to generate real-world evidence (RWE). We recently conducted a systematic literature review (SLR) evaluating publicly available information on the use of RWD-derived external controls to contextualize outcomes from uncontrolled trials submitted to the European Medicines Agency (EMA), the US Food and Drug Administration (FDA), and/or select HTA bodies. The review identified several key operational and methodological aspects for which more detailed guidance and alignment within and between regulatory agencies and HTA bodies is necessary. This paper builds on the SLR findings by delineating a set of key takeaways for the responsible generation of fit-for-purpose RWE. Practical methodological and operational guidelines for designing, conducting, and reporting RWD-derived external control studies are explored and discussed. These considerations include: (i) early engagement with regulators and HTA bodies during the study planning phase; (ii) consideration of the appropriateness and comparability of external controls across multiple dimensions, including eligibility criteria, temporality, population representation, and clinical evaluation; (iii) ensuring adequate sample sizes, including hypothesis testing considerations; (iv) implementation of a clear and transparent strategy for assessing and addressing data quality, including data missingness across trials and RWD; (v) selection of comparable and meaningful endpoints that are operationalized and analyzed using appropriate analytic methods; and (vi) conduct of sensitivity analyses to assess the robustness of findings in the context of uncertainty and sources of potential bias.

Effectively Leveraging RWD for External Controls: A Systematic Literature Review of Regulatory and HTA Decisions

Real-world data (RWD)-derived external controls can be used to contextualize efficacy findings for investigational therapies evaluated in uncontrolled trials. As the number of submissions to regulatory and health technology assessment (HTA) bodies using external controls rises, and in light of recent regulatory and HTA guidance on the appropriate use of RWD, there is a need to address the operational and methodological challenges impeding the quality of real-world evidence (RWE) generation and the consistency in evaluation of RWE across agencies. This systematic review summarizes publicly available information on the use of external controls to contextualize outcomes from uncontrolled trials for all indications from January 1, 2015, through August 20, 2021, that were submitted to the European Medicines Agency, the US Food and Drug Administration, and/or select major HTA bodies (National Institute for Health and Care Excellence (NICE), Haute Autorité de Santé (HAS), Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG), and Gemeinsamer Bundesausschuss (G-BA)). By systematically reviewing submissions to regulatory and HTA bodies in the context of recent guidance, this study provides quantitative and qualitative insights into how external control design and analytic choices may be viewed by different agencies in practice. The primary operational and methodological aspects identified for discussion include, but are not limited to, engagement of regulators and HTA bodies, approaches to handling missing data (a component of data quality), and selection of real-world endpoints. Continued collaboration and guidance to address these and other aspects will inform and assist stakeholders attempting to generate evidence using external controls.

Generating fit-for-purpose real-world evidence in Asia: How far are we from closing the gaps?

Evidence generated by randomized controlled trials (RCTs) does not often represent the patient journey and clinical outcomes in the real world due to limited external validity or generalizability. Studies based on real-world data are intended to generalize results to the broader population; however, if the influence of external factors or confounders is not effectively managed, the cause-and-effect relationship and internal validity may be challenged, resulting in flawed results. The collection of quality real-world evidence (RWE) is crucial in Asia as there is often an underrepresentation of Asian populations in RCTs. In addition, few countries in Asia are catching up with the Western world in issuing practical foundational principles and guidance for conducting and adopting evidence for regulatory and reimbursement decisions. However, privacy and data protection laws are generally lagging behind technological developments in electronic medical records. While leveraging RWE in clinical and regulatory decision-making holds excellent potential, collective efforts across industry, governments, and research institutions are required for generating standardized practices and building capabilities for developing fit-for-purpose RWE in Asia.

Survival outcomes of patients with metastatic non-small cell lung cancer receiving chemotherapy or immunotherapy as first-line in a real-life setting

Treatment of metastatic non-small cell lung cancer (mNSCLC) has been modified due to the development of immunotherapy. We assessed survival outcomes (overall [OS] and progression-free [rwPFS] survivals, time-to-next-treatment [TNT]) in mNSCLC patients after first-line immunotherapy and chemotherapy in real-life settings. Association between rwPFS and TNT, two candidate surrogate endpoints (SE), with OS was assessed. This retrospective multi-center study uses data from patients included in the Epidemio-Strategy Medico-Economic program with mNSCLC over 2015-2019. The impact of treatment on rwPFS/OS was evaluated with Cox models. Individual-level associations between SE and OS were estimated with an iterative multiple imputation approach and joint survival models. The population included 5294 patients (63 years median age). Median OS in immunotherapy group was 16.4 months (95%CI [14.1-NR]) and was higher than in chemotherapy group (11.6 months; 95%CI [11.0-12.2]). Improved OS was observed for the immunotherapy group after 3 months for subjects with performance status 0-1 (HR = 0.59; 95%CI [0.42-0.83], p < 0.01). The associations between rwPFS and TNT with OS were close ([Formula: see text]=0.57). Results emphasized a survival improvement with immunotherapy for patients in good health condition. There was moderate evidence of individual-level association between candidate SE and OS.

Building the observational medical outcomes partnership's T-MSIS Analytic File common data model

Objectives

This effort used Databricks to create an Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) for Transformed MSIS Analytic File (TAF) Medicaid records.

Materials and methods

Our process included data volume and content assessment of TAF, translation mapping of TAF concepts to OMOP concepts and the creation of Extract Transform and Load (ETL) code.

Results

The final CDM contained 119,048,562 individuals and 24,806,828,121 clinical observations from 2014 through 2018.

Discussion

The transformation of TAF into OMOP can support the generation of evidence with special attention to low-income patients on public insurance. Such patients are perhaps underrepresented in academic medical center patient populations.

Conclusion

Our effort successfully used Databricks to transform TAF records into OMOP CDM. Our CDM can be used to generate evidence for OMOP network studies.

Commercializing Personal Health Information: A Critical Qualitative Content Analysis of Documents Describing Proprietary Primary Care Databases in Canada

BACKGROUND

Commercial data brokers have amassed large collections of primary care patient data in proprietary databases. Our study objective was to critically analyze how entities involved in the collection and use of these records construct the value of these proprietary databases. We also discuss the implications of the collection and use of these databases.

METHODS

We conducted a critical qualitative content analysis using publicly available documents describing the creation and use of proprietary databases containing Canadian primary care patient data. We identified relevant commercial data brokers, as well as entities involved in collecting data or in using data from these databases. We sampled documents associated with these entities that described any aspect of the collection, processing, and use of the proprietary databases. We extracted data from each document using a structured data tool. We conducted an interpretive thematic content analysis by inductively coding documents and the extracted data.

RESULTS

We analyzed 25 documents produced between 2013 and 2021. These documents were largely directed at the pharmaceutical industry, as well as shareholders, academics, and governments. The documents constructed the value of the proprietary databases by describing extensive, intimate, detailed patient-level data holdings. They provided examples of how the databases could be used by pharmaceutical companies for regulatory approval, marketing and understanding physician behaviour. The documents constructed the value of these data more broadly by claiming to improve health for patients, while also addressing risks to privacy. Some documents referred to the trade-offs between patient privacy and data utility, which suggests these considerations may be in tension.

CONCLUSION

Documents in our analysis positioned the proprietary databases as socially legitimate and valuable, particularly to pharmaceutical companies. The databases, however, may pose risks to patient privacy and contribute to problematic drug promotion. Solutions include expanding public data repositories with appropriate governance and external regulatory oversight.

Real-World Efficacy and Safety of Extended-Release Methylphenidate (PRC-063) in the Treatment of ADHD in Pediatric and Adult Subjects: Results of a Phase IV Multicenter Comparison With Lisdexamfetamine Dimesylate

OBJECTIVE

To evaluate the real-world efficacy, safety, and functional outcomes of PRC-063 (multilayer-release methylphenidate) versus lisdexamfetamine (LDX) in ADHD subjects in a phase IV, open-label study.

METHOD

The primary endpoint was the change in the ADHD-DSM-5 Rating Scale (ADHD-5-RS) total score from baseline to Month 4. Secondary endpoints included a non-inferiority comparison between PRC-063 and LDX and measures of functioning and evening behavior.

RESULTS

One hundred forty-three pediatric and 112 adult subjects were enrolled. Mean ADHD-5-RS scores (standard deviation) were reduced in pediatric (-16.6 [10.4]) and adult (-14.8 [10.6]) subjects treated with PRC-063 (p < .001). PRC-063 was non-inferior to LDX in the pediatric population but not in the adult population. Significant improvements were demonstrated in quality of life and functionality. Both medications were well-tolerated; more adverse events led to study discontinuation in pediatric subjects treated with LDX versus PRC-063.

CONCLUSION

PRC-063 and LDX significantly improved ADHD symptomatology and functioning and were well-tolerated.

Confounding adjustment in the analysis of augmented randomized controlled trial with hybrid control arm

The augmented randomized controlled trial (RCT) with hybrid control arm includes a randomized treatment group (RT), a smaller randomized control group (RC), and a large synthetic control (SC) group from real-world data. This kind of trial is useful when there is logistics and ethics hurdle to conduct a fully powered RCT with equal allocation, or when it is necessary to increase the power of the RCT by incorporating real-world data. A difficulty in the analysis of augmented RCT is that the SC and RC may be systematically different in the distribution of observed and unmeasured confounding factors, causing bias when the two control groups are analyzed together as hybrid controls. We propose to use propensity score (PS) analysis to balance the observed confounders between SC and RC. The possible bias caused by unmeasured confounders can be estimated and tested by analyzing propensity score adjusted outcomes from SC and RC. We also propose a partial bias correction (PBC) procedure to reduce bias from unmeasured confounding. Extensive simulation studies show that the proposed PS + PBC procedures can improve the efficiency and statistical power by effectively incorporating the SC into the RCT data analysis, while still control the estimation bias and Type I error inflation that might arise from unmeasured confounding. We illustrate the proposed statistical procedures with data from an augmented RCT in oncology.

Effectiveness of the Lilly Connected Care Program in Improving Glycemic Management Among Patients With Type 2 Diabetes in China: Retrospective Real-world Study

BACKGROUND

Type 2 diabetes mellitus (T2DM) is a worldwide public health concern. Mobile health management platforms could be a potential way to achieve effective glycemic control.

OBJECTIVE

This study aimed to evaluate the real-world effectiveness of the Lilly Connected Care Program (LCCP) platform in glycemic control among patients with T2DM in China.

METHODS

This retrospective study included Chinese patients with T2DM (aged ≥18 years) from April 1, 2017, to January 31, 2020, for the LCCP group and from January 1, 2015, to January 31, 2020, for the non-LCCP group. Propensity score matching was used to match the LCCP and non-LCCP groups to reduce confounding, with covariates including age, sex, the duration of diabetes, baseline hemoglobin A_1c (HbA_1c), and the number of oral antidiabetic medication classes. HbA_1c reduction over 4 months, the proportions of patients achieving an HbA_1c reduction of ≥0.5% or ≥1%, and the proportions of patients reaching to target HbA_1c level of ≤6.5% or <7% were compared between the LCCP and non-LCCP groups. Multivariate linear regression was used to assess factors associated with HbA_1c reduction.

RESULTS

A total of 923 patients were included, among whom 303 pairs of patients were well matched after propensity score matching. HbA_1c reduction during the 4-month follow-up was significantly larger in the LCCP group than the non-LCCP group (mean 2.21%, SD 2.37% vs mean 1.65%, SD 2.29%; P=.003). The LCCP group had a higher proportion of patients with an HbA_1c reduction of ≥1% (209/303, 69% vs 174/303, 57.4%; P=.003) and ≥0.5% (229/303, 75.6% vs 206/303, 68%; P=.04). The proportions of patients reaching the target HbA_1c level of ≤6.5% were significantly different between the LCCP and non-LCCP groups (88/303, 29% vs 61/303, 20.1%; P=.01), whereas the difference in the proportions of patients reaching the target HbA_1c level of <7% was not statistically significant (LCCP vs non-LCCP: 128/303, 42.2% vs 109/303, 36%; P=.11). LCCP participation and higher baseline HbA_1c were associated with a larger HbA_1c reduction, whereas older age, longer diabetes duration, and higher baseline dose of premixed insulin analogue were associated with a smaller HbA_1c reduction.

CONCLUSIONS

The LCCP mobile platform was effective in glycemic control among patients with T2DM in China in the real world.

Mimicking Clinical Trials Using Real-World Data: A Novel Method and Applications

INTRODUCTION

Simulating individual-level trial data when only summary data are available is often useful for meta-analysis, forming external control arms and calibrating trial results to real-world data (RWD). The joint distribution of baseline characteristics in a trial is usually simulated by combining its summary data with RWD's correlations. However, RWD correlations may not be a perfect proxy for the trial. A misspecified correlation structure could bias any analysis in which the outcomes generating models are nonlinear or include effect modifiers.

METHODS

We developed an iterative algorithm using copula and resampling, which was based on the estimated propensity score for the likelihood of enrollment in a trial given participants' characteristics. Validation was performed using Monte Carlo simulations under different scenarios in which the marginal and joint distributions of covariates differ between trial samples and RWD. Two applications were illustrated using an actual trial and the Surveillance, Epidemiology, and End Results-Medicare data. We calculated the standardized mean difference (SMD) to assess the generalizability of the trial and explored the feasibility of generating an external control by applying a parametric Weibull model trained in RWD to predict survival in the simulated trial cohort.

RESULTS

Across all scenarios, approximated correlations derived from the algorithm were closer to the true correlations than the RWD's correlations. The algorithm also successfully reproduced the joint distribution of characteristics for the actual trial. A similar SMD was observed using simulated data and individual-level trial data. The 95% confidence intervals were overlapped between adjusted survival estimates from the simulated trial and actual trial Kaplan-Meier estimates.

CONCLUSIONS

The algorithm could be a feasible way to simulate individual-level data when only summary data are available. Further research is needed to validate our approach with larger sample sizes.

HIGHLIGHTS

The correlation structure is crucial to building the joint distribution of patient characteristics, and a misspecified correlation structure could potentially influence predicted outcomes.An iterative algorithm was developed to approximate a trial's correlation structure using published summary trial data and real-world data.The algorithm could be a feasible way to simulate individual-level trial data when only trial summary data are available.

Structure and Content of a Taxonomy to Support the Use of Real-World Evidence by Health Technology Assessment Practitioners and Healthcare Decision Makers

This is one of a series of articles that consider the barriers to optimal use of real-world evidence (RWE) in health technology assessment and how to overcome them. The work was performed as part of EUreccA 2025, in particular with the RWE workstream embodied within that collaboration. Elsewhere in this issue we described the reasoning and process that led us to develop practical tools to support RWE use, including this taxonomy and explained the methods used to do so. The taxonomy classifies questions that are typically addressed using real-world data in health technology assessment and the data sources typically used to address these questions. In this article, we describe the taxonomy itself. For as many of the pairings as possible, we have provided links to advice and methods on how to address the associated question using those data. We have also provided links to examples of RWE use in practical decision making to answer the questions posed. Our work is not complete, but we believe it is sufficient to demonstrate the value of such a taxonomy and information source if it is completed and curated as a "wiki" by the community that would use it.

Data Governance for Real-World Data Management: A Proposal for a Checklist to Support Decision Making

OBJECTIVES

Real-world data (RWD) and real-world evidence (RWE) can provide extensive information on healthcare for use in health technology assessment and decision making. Nevertheless, there is a lack of consensus surrounding the appropriate data governance (DG) practices for RWD/RWE. Data sharing is also a large concern, especially considering evolving data protection regulations. Our objective is to propose recommendations for international standards of evaluating the acceptability of RWD governance practices.

METHODS

After reviewing the literature, we created a checklist targeting DG practices for RWD/RWE. We then carried out a 3-round Delphi panel, including European policy makers, health technology assessment experts, and hospital managers. The consensus for each statement was measured and the checklist adjusted accordingly.

RESULTS

The literature review identified the main topics regarding RWD/RWE DG practices: data privacy and security, data management and linkage, data access management, and the generation and use of RWE. Members of the Delphi panel (21 experts/25 invited) were presented a total of 24 statements related to each of the topics. Experts demonstrated a progressive level of consensus and importance ratings in all topics and to most statements. We suggest a refined checklist in which the statements rated less important or with less consensus have been removed.

CONCLUSIONS

This study suggests how the DG of RWD/RWE could be qualitatively evaluated. We propose checklists that could be used by all RWD/RWE users to help ensure the quality and integrity of RWD/RWE governance and complement data protection law.

Recording type 2 diabetes mellitus in a standardised central Saudi database: a retrospective validation study

OBJECTIVES

This study was conducted to assess the validity of recording (and the original diagnostic practice) of type 2 diabetes mellitus at a hospital whose records were integrated to a centralised database (the standardised common data model (CDM) of the Saudi National Pharmacoepidemiologic Database (NPED)).

DESIGN

A retrospective single-centre validation study.

SETTINGS

Data of the study participants were extracted from the CDM of the NPED (only records of one tertiary care hospital were integrated at the time of the study) between 1 January 2013 and 1 July 2018.

PARTICIPANTS

A random sample of patients with type 2 diabetes mellitus (≥18 years old and with a code of type 2 diabetes mellitus) matched with a control group (patients without diabetes) based on age and sex.

OUTCOME MEASURES

The standardised coding of type 2 diabetes in the CDM was validated by comparing the presence of diabetes in the CDM versus the original electronic records at the hospital, the recording in paper-based medical records, and the physician re-assessment of diabetes in the included cases and controls, respectively. Sensitivity, specificity, positive predictive value and negative predictive value were estimated for each pairwise comparison using RStudio V.1.4.1103.

RESULTS

A total of 437 random sample of patients with type 2 diabetes mellitus was identified and matched with 437 controls. Only 190 of 437 (43.0%) had paper-based medical records. All estimates were above 90% except for sensitivity and specificity of CDM versus paper-based records (54%; 95% CI 47% to 61% and 68%; 95% CI 62% to 73%, respectively).

CONCLUSIONS

This study provided an assessment to the extent of which only type 2 diabetes mellitus code can be used to identify patients with this disease at a Saudi centralised database. A future multi-centre study would help adding more emphasis to the study findings.

Real-world data in oncology: a questionnaire-based analysis of the academic research landscape examining the policies and experiences of the cancer cooperative groups

BACKGROUND

Real-world data (RWD) have quickly emerged as an important source of information to address uncertainties about new treatments, including novel anticancer therapies. Many stakeholders are using such data and the evidence derived therefrom to answer the questions that remain about the safety and effectiveness of antitumor medicines after their approval by regulators. Our objective was to investigate the academic RWD study landscape and explore to what extent RWD are being integrated into investigator-initiated clinical research.

MATERIALS AND METHODS

We designed an online survey that was distributed between May and August 2022 to representatives of cancer cooperative groups active in Europe, North America, South America, Asia, and/or Oceania.

RESULTS

In total, 125 cooperative groups operating in 58 different countries and conducting research across 13 distinct cancer domains participated in the survey. While most of the responders (67.2%) did not have a formal policy in place to gather and utilize RWD, a majority (68.0%) had carried out studies involving the analysis of such data before, both for exploratory and confirmatory purposes. The groups that were experienced in capturing and interpreting RWD had mainly worked with observational RWD that were not predominantly prospective or retrospective in nature and which originated from disease registries, electronic health records, and patient questionnaires. They perceived the low costs and the large scale of RWD research to be its most significant benefits, and viewed the accompanying methodological and operational challenges as its biggest constraints. However, they did not have a common understanding of what RWD were. Despite their experience with analyzing RWD, their research portfolio still primarily comprised traditional clinical trials; 62.5% of the groups that had never undertaken any RWD studies were nonetheless planning to initiate them in the future.

CONCLUSIONS

Cancer cooperative groups are already incorporating RWD studies into their research agendas, but still lack knowledge and expertise in this regard, and do not agree on what RWD are. The conduct of conventional clinical trials continues to be their priority.

An Attempt to Replicate Randomized Trials of Diabetes Treatments Using a Japanese Administrative Claims and Health Checkup Database: A Feasibility Study

BACKGROUND

Use of real-world evidence (RWE) has been limited for evaluating effectiveness because of the lack of confidence in its reliability. Examining whether a rigorously designed observational study using real-world data (RWD) can reproduce the results of a randomized controlled trial (RCT) will provide insights into the implementation of high-quality RWE studies that can produce valid conclusions.

OBJECTIVE

We aimed to replicate published RCTs using a Japanese claims and health checkup database and examine whether the emulated RWE studies' results agree with those of the original RCTs.

METHODS

We selected three RCTs on diabetes medications for replication in patients with type 2 diabetes. The study outcome was either the change or percentage change in HbA1c levels from baseline. We designed three observational studies using the RWD to mimic the critical study elements of the respective RCTs as closely as possible. We performed 1:1 propensity score nearest-neighbor matching to balance the groups for potential confounders. The differences in outcomes between the groups and their 95% confidence intervals (CIs) were calculated in each RWE study, and the results were compared with those of the RCT.

RESULTS

Patient characteristics, such as age, sex, and duration of diabetes, differed between the RWE studies and RCTs. In Trial 1 emulation, the percentage changes in HbA1c levels were larger in the treatment group than in the comparator group (difference -6.21, 95% confidence interval (CI) -11.01 to -1.40). In Trial 2, the change in HbA1c level was larger in the treatment group (difference -0.01; 95% CI -0.25 to 0.23), and in Trial 3, it was smaller in the treatment group (difference 0.46; 95% CI -0.01 to 0.94). These results did not show regulatory or estimate agreement with the RCTs.

CONCLUSIONS

None of the three emulated RWE studies using this claims and health checkup database reproduced the same conclusions as the RCTs. These discrepancies could largely be attributed to design differences between RWE studies and RCTs, primarily due to the lack of necessary data in the database. This particular RWD source may not be the best fit for evaluating treatment effects using laboratory data as the study outcome.

Advancing evidence-based treatment of infectious diseases in children with real-world data: Opportunities and challenges

There is an increased interest in utilizing real-world data (RWD) for pharmaceutical research and regulatory decision-making. The development and use of pediatric medicines could benefit greatly from real-world data studies given nearly half of drugs prescribed to children are "off-label", meaning there is a lack of pediatric-specific evidence from controlled trials, while there is an abundance of data from routine clinical practice. Currently, the use of real-world data, such as data from electronic health records, is lacking in pediatric research, especially within infectious diseases. Here, we discuss opportunities and challenges for real-world data to generate evidence on the optimal treatment and management of infectious diseases in children.

Augmenting randomized clinical trial data with historical control data: Precision medicine applications

As precision medicine becomes more precise, the sizes of the molecularly targeted subpopulations become increasingly smaller. This can make it challenging to conduct randomized clinical trials of the targeted therapies in a timely manner. To help with this problem of a small patient subpopulation, a study design that is frequently proposed is to conduct a small randomized clinical trial (RCT) with the intent of augmenting the RCT control arm data with historical data from a set of patients who have received the control treatment outside the RCT (historical control data). In particular, strategies have been developed that compare the treatment outcomes across the cohorts of patients treated with the standard (control) treatment to guide the use of the historical data in the analysis; this can lessen the potential well-known biases of using historical controls without any randomization. Using some simple examples and completed studies, we demonstrate in this commentary that these strategies are unlikely to be useful in precision medicine applications.

iGlarLixi versus basal plus Rapid-Acting insulin in adults with type 2 diabetes advancing from basal insulin therapy: The SoliSimplify Real-World study

AIM

For people with suboptimally controlled type 2 diabetes (T2D) on basal insulin (BI), guidelines recommend several treatment advancement options. This study compared the clinical effectiveness of once-daily iGlarLixi versus a multiple-injection BI + rapid acting insulin (RAI) regimen in adults with T2D advancing from BI therapy in real-world clinical practice.

MATERIALS AND METHODS

Electronic medical records from the Observational Medical Outcomes Partnership (OMOP) database were analysed retrospectively using propensity score matching to compare therapy advancement with iGlarLixi or BI + RAI in US adults ≥18 years with T2D on BI who had ≥1 valid glycated haemoglobin (HbA1c) value at baseline and at the 6-month follow-up. The primary objective was non-inferiority of iGlarLixi to BI + RAI in HbA1c change from baseline to 6 months (margin 0.3%).

RESULTS

Propensity score matching generated cohorts with balanced baseline characteristics (N = 814 in each group). HbA1c reduction from baseline to 6 months with iGlarLixi was non-inferior to BI + RAI [mean difference (95% confidence interval): 0.1 (-0.1, 0.2)%; one-sided p = .0032]. At 6 months, weight gain was significantly lower with iGlarLixi than with BI + RAI [-0.8 (-1.3, -0.2) kg; two-sided p = .0069]. Achievement of HbA1c <7% without hypoglycaemia and weight gain were similar between groups [odds ratio (95% confidence interval): 1.15 (0.81, 1.63); p = .4280]. Hypoglycaemia was low in both groups, probably because of underreporting.

CONCLUSIONS

In real-world clinical practice, glycaemic outcomes 6 months after treatment advancement from BI are similar for people with T2D using iGlarLixi versus BI + RAI, with iGlarLixi leading to less weight gain.

Identification of cancer chemotherapy regimens and patient cohorts in administrative claims: challenges, opportunities, and a proposed algorithm

BACKGROUND

Real-world evidence is a valuable source of information in healthcare. This study describes the challenges and successes during algorithm development to identify cancer cohorts and multi-agent chemotherapy regimens from claims data to perform a comparative effectiveness analysis of granulocyte colony stimulating factor (G-CSF) use.

METHODS

Using the Biologics and Biosimilars Collective Intelligence Consortium's Distributed Research Network, we iteratively developed and tested a de novo algorithm to accurately identify patients by cancer diagnosis, then extract chemotherapy and G-CSF administrations for a retrospective study of prophylactic G-CSF.

RESULTS

After identifying patients with cancer and subsequent chemotherapy exposures, we observed only 12% of patients with cancer received chemotherapy, which is fewer than expected based on prior analyses. Therefore, we reversed the initial inclusion criteria to identify chemotherapy receipt, then prior cancer diagnosis, which increased the number of patients from 2,814 to 3,645, or 68% of patients receiving chemotherapy had diagnoses of interest. Additionally, we excluded patients with cancer diagnoses that differed from those of interest in the 183 days before the index date of G-CSF receipt, including early-stage cancers without G-CSF or chemotherapy exposure. By removing this criterion, we retained 77 patients who were previously excluded. Finally, we incorporated a 5-day window to identify all chemotherapy drugs administered (excluding oral prednisone and methotrexate, as these medications may be used for other non-malignant conditions) as patients may fill oral prescriptions days to weeks prior to infusion. This increased the number of patients with chemotherapy exposures of interest to 6,010. The final cohort of included patients, based on G-CSF exposure, increased from 420 from the initial algorithm to 886 using the final algorithm.

CONCLUSIONS

Medications used for multiple indications, sensitivity and specificity of administrative codes, and relative timing of medication exposure must all be evaluated to identify patient cohorts receiving chemotherapy from claims data.

Safety and effectiveness of vortioxetine for major depressive disorder: Real-world evidence from a population-based study in South Korea

Background

A post-marketing surveillance study was conducted to assess the real-world safety and effectiveness of vortioxetine for the treatment of major depressive disorder (MDD) in South Korea.

Methods

Adult patients aged 19-94 years receiving vortioxetine for MDD at 72 hospitals and clinics in South Korea between 19^th August 2014 and 18^th August 2020 were included. Patients were followed for up to 24±2 weeks, at up to three visits. Adverse events (AEs) and effectiveness, assessed by both clinician and patient-reported measures, were analyzed.

Results

A total of 3,263 patients (mean age: 51.28 years) were included in the safety set; 1,095 were aged ≥65 years. The majority of the safety set (61.97%) were female. The overall rate of any AEs and serious AEs were 17.13 and 1.56%, respectively. The majority of AEs were mild (88.32%). The rates of AEs did not differ statistically by age (≥65 years: 16.89% [185/1,095] versus <65 years: 17.25% [374/2,168)], p=0.7989), sex (male: 15.95% [198/1,241] versus female: 17.85% [361/2,022], p=0.1623), or liver impairment (with liver impairment: 20.90% [14/67] versus without liver impairment: 17.05% [545/3,196], p=0.4087). Effectiveness was assessed in 1,918 patients. By 24±2 weeks, there were significant clinical improvements from baseline, assessed by change in Montgomery-Asberg Depression Rating Scale total score (mean±standard deviation [SD]: -10.49±9.42 points, p <0.0001), the proportion of patients with improved symptoms using the Clinical Global Impression - Improvement scores (79.29%), and in both patient-reported measures, with a significant improvement in the Korean Version of the Perceived Deficits Questionnaire-Depression (mean±SD: -6.06±13.23, p <0.0001) and Digit Symbol Substitution Test (mean±SD: 4.83±9.81, p <0.0001) total scores from baseline. Similar to the safety profiles, the proportions of patients with improved symptoms compared with baseline using the Clinical Global Impression - Improvement scores did not differ by age (≥65 years: 82.09% versus <65 years: 78.32%, p=0.0511), sex (male: 77.45% versus female: 81.01%, p=0.0587), or liver impairment (with liver impairment: 67.57% versus without liver impairment: 79.85%, p=0.0663).

Conclusion

Vortioxetine appears to be well-tolerated and effective for treating MDD patients in the real-world setting in South Korea, irrespective of age, sex, and liver impairment, reflecting the known profile of vortioxetine based on studies worldwide.

Target Trial Emulation: A Design Tool for Cancer Clinical Trials

PURPOSE

To apply target trial emulation to explore the potential impact of eligibility criteria on the primary outcome of a randomized controlled trial.

METHODS

Simulations of a real-world explanatory trial of transarterial radioembolization for advanced unresectable hepatocellular carcinoma with portal vein invasion were performed to examine the effects of cohort specification on survival outcomes and patient sample size. Simulations comprised 24 different permutations of the trial varied on three disease nonspecific eligibility parameters. Treatment and control arms for these emulated trials were drawn from the National Cancer Database and matched by treatment propensity. Target trial emulation served as the causal framework for this analysis, allowing the architecture of a true controlled experiment to address forms of bias routinely encountered in comparative effectiveness studies involving real-world observational data.

RESULTS

Twenty-four propensity score-matched cohorts comprising a wider clinical spectrum of patients than specified by the original target trial were successfully generated using the National Cancer Database. The arms for each of the emulated trials demonstrated exchangeability across all eligibility criteria and other clinical covariates. Significant treatment benefits were associated with only a narrow range of eligibility criteria, indicating that the original target trial was well specified.

CONCLUSION

The impact of patient selection on treatment outcomes can be studied using target trial emulation. This analytical framework can furthermore serve to leverage existing real-world data to inform the task of cohort specification for a randomized controlled trial, facilitating a more data-driven approach for this important step in clinical trial design.

Emulate randomized clinical trials using heterogeneous treatment effect estimation for personalized treatments: Methodology review and benchmark

Big data and (deep) machine learning have been ambitious tools in digital medicine, but these tools focus mainly on association. Intervention in medicine is about the causal effects. The average treatment effect has long been studied as a measure of causal effect, assuming that all populations have the same effect size. However, no "one-size-fits-all" treatment seems to work in some complex diseases. Treatment effects may vary by patient. Estimating heterogeneous treatment effects (HTE) may have a high impact on developing personalized treatment. Lots of advanced machine learning models for estimating HTE have emerged in recent years, but there has been limited translational research into the real-world healthcare domain. To fill the gap, we reviewed and compared eleven recent HTE estimation methodologies, including meta-learner, representation learning models, and tree-based models. We performed a comprehensive benchmark experiment based on nationwide healthcare claim data with application to Alzheimer's disease drug repurposing. We provided some challenges and opportunities in HTE estimation analysis in the healthcare domain to close the gap between innovative HTE models and deployment to real-world healthcare problems.

Post-marketing safety of immunomodulatory drugs in multiple myeloma: A pharmacovigilance investigation based on the FDA adverse event reporting system

Objective: In recent years, the emergence of immunomodulatory drugs (IMiDs) has significantly improved clinical outcomes in patients with multiple myeloma (MM); however, serious adverse events (AEs) have hindered their safe clinical application. This study aimed to characterize the safety profiles and differences in IMiDs through a disproportionality analysis using the U.S. Food and Drug Administration Adverse Event Reporting System (FAERS), a post-marketing surveillance database. Methods: This study filtered reports of thalidomide, lenalidomide, and pomalidomide as primary suspect drugs in FAERS files from January 2013 to December 2021. AEs in the reports were retrieved according to the preferred terms (PTs) of the Medical Dictionary for Regulatory Activities. Furthermore, we detected safety signals using the reporting odds ratio (ROR), proportional reporting ratio (PRR), and Bayesian belief propagation neural network (BCPNN). When all three algorithms showed an association between the target drug and the AE, a positive signal was generated. Results: We extracted 9,968 thalidomide, 231,926 lenalidomide, and 55,066 pomalidomide AE reports. AEs were more common in male patients and in those >44 years old. Important safety signals were detected based on the system organ classes (SOC), including thalidomide (cardiac disorders: ROR, 2.87; PRR, 2.79; IC 1.22), lenalidomide (gastrointestinal disorders: ROR, 2.38; PRR, 2.27; IC 0.75), and pomalidomide (respiratory, thoracic, and mediastinal disorders: ROR, 2.14; PRR, 2.09; IC 0.85). Within the PT level, we identified novel risk signals: the thalidomide-induced second primary malignancy (SPM) signal was significant; lenalidomide reduced the success rate of hematopoietic stem cell collection; and three IMiDs may cause human chorionic gonadotropin increase, but this needs to be proven by clinical data. Pneumonia, sepsis, and renal failure are common risk factors for death due to IMiDs. Compared with thalidomide and lenalidomide, pomalidomide has a lower risk of venous thromboembolism (VTE) and is beneficial to patients with renal insufficiency. Conclusion: Mining data from FAERS resulted in novel AE signals, including adenocarcinoma of colon, harvest failure of blood stem cells, and increased levels of human chorionic gonadotropin. Further investigation is required to verify the significance of these signals. Moreover, IMiDs showed differences in safety reports, which should be emphasized by clinicians.

Real-world evidence of biological agents in dermatology: A review of its applications, advantages, and limitations

Randomized controlled trials (RCTs) are regarded as the gold standard of evidence-based medicine. However, the disadvantages of RCTs have been well-documented for a lengthy period of time. Due to the carefully controlled conditions, a small group of patients is studied, resulting in a lack of external validity and generalizability. To address this issue, real-world evidence (RWE) has grown in importance as a complement to randomized controlled trials (RCTs). We introduce and describe the databases used by RWE in this post. Applications of RWE are described in following aspects with examples of patients in the use of biological agents with skin diseases: (1) support the safety and efficacy outcomes of RCTs; (2) reveal real-world situations of adverse event management; (3) optimal treatment plans; (4) treatment's real-world cost burden. We also focus on its superiority of generalizability, external validity, and time and economic efficiencies as well as its deficiencies in data resources, study design, and statistical methodologies.

The application, value, and impact of outcomes research in animal health and veterinary medicine

Outcomes research is a relatively recent field of study in animal health and veterinary medicine despite being well-established in human medicine. As the field of animal health is broad-ranging in terms of animal species, objectives, research methodologies, design, analysis, values, and outcomes, there is inherent versatility in the application and impact of the discipline of outcomes research to a variety of stakeholders. The major themes of outcomes relevant to the animal health industry have been distilled down to include, but are not limited to, health, production, economics, and marketing. An outcomes research approach considers an element of value along with an outcome of interest, setting it apart from traditional research approaches. Elements of value are determined by the stakeholders' use of products and/or services that meet or exceed functional, emotional, life-changing, and/or societal needs. Stakeholder perception of value depends on many factors such as the purpose of the animal (e.g., companion vs. food production) and the stakeholder's role (e.g., veterinarian, client, pet-owner, producer, consumer, government official, industry representative, policy holder). Key areas of application of outcomes research principles include comparative medicine, veterinary product development, and post-licensure evaluation of veterinary pharmaceuticals and/or biologics. Topics currently trending in human healthcare outcomes research, such as drug pricing, precision medicine, or the use of real-world evidence, offer novel and interesting perspectives for addressing themes common to the animal health sector. An approach that evaluates the benefits of practices and interventions to veterinary patients and society while maximizing outcomes is paramount to combating many current and future scientific challenges where feeding the world, caring for our aging companion animals, and implementing novel technologies in companion animal medicine and in production animal agriculture are at the forefront of our industry goals.

Real-world experience in treating pediatric relapsed/refractory or therapy-related myeloid malignancies with decitabine, vorinostat, and FLAG therapy based on a phase 1 study run by the TACL consortium

Current therapies for relapsed/refractory (R/R) pediatric myeloid neoplasms are inadequately effective. Real-world data (RWD) can improve care by augmenting traditional studies and include individuals not eligible for clinical trials. The Therapeutic Advances in Childhood Leukemia and Lymphoma (TACL) consortium recently completed T2016-003, a phase 1 study of decitabine, vorinostat, fludarabine, cytarabine, and granulocyte colony-stimulating factor (G-CSF) in R/R acute myeloid leukemia (AML), which added epigenetic drugs to a cytotoxic backbone. We report results of RWD from six centers that treated 28 pediatric patients (26 with AML, two with other myeloid neoplasms) identically to the TACL study but who were not enrolled. This allowed unique analyses and the ability to compare data with the 35 TACL study patients. The overall response rate (ORR) (complete response [CR] plus CR with incomplete count recovery) among 26 RWD evaluable patients was 65%. The ORR of 13 patients with relapsed AML with epigenetic alterations was 69% (T2016-003 + RWD: 68%, n = 25), of eight patients with refractory AML was 38% (T2016-003 + RWD: 41%, n = 17) and of five patients with therapy-related AML (t-AML) was 80% (T2016-003 + RWD: 75%, n = 8). The mean number of Grade 3/4 toxicities experienced by the T2016-003-eligible RWD population (n = 22) (one per patient-cycle) was not meaningfully different than those (n = 6) who would have been TACL study-ineligible secondary to comorbidities (two per patient-cycle). Overall, this therapy was well tolerated and effective in pediatric patients with R/R myeloid neoplasms, particularly those with epigenetic alterations, t-AML, and refractory disease.

Intravenous edaravone treatment in ALS and survival: An exploratory, retrospective, administrative claims analysis

BACKGROUND

We aimed to evaluate overall survival in US patients with amyotrophic lateral sclerosis (ALS) treated with intravenous (IV) edaravone compared with those not treated with IV edaravone in a real-world setting.

METHODS

This exploratory retrospective comparative effectiveness observational analysis included patients with ALS who were enrolled in an administrative claims database from 8 August 2017 to 31 March 2020. Propensity score matching identified IV edaravone-treated patients (cases) and non-edaravone-treated patients (controls) matched for covariates: age, race, geographic region, sex, pre-index disease duration, insurance, history of cardiovascular disease, riluzole prescription, gastrostomy tube placement, artificial nutrition, noninvasive ventilation, and all-cause hospitalisation. For cases, the index date was the date of the first claim for IV edaravone. For controls, it was the date IV edaravone was available (8 August 2017). The effect of IV edaravone on all-cause mortality was estimated with shared frailty Cox regression analysis.

FINDINGS

318 cases were matched to 318 controls. In both groups, 208 patients (65.4%) had a history of riluzole prescription. As of 31 March 2021, there were 155 deaths (48.7%) among the cases and 196 among the controls (61.6%). Median overall survival time was 29.5 months with edaravone and 23.5 months without, respectively, and the risk of death was 27% lower in cases than in controls (HR, 0.73; 95% CI, 0.59-0.91; p=0.005).

INTERPRETATION

In this real-world analysis, IV edaravone treatment in a large predominantly riluzole-treated US cohort was associated with prolonged overall survival compared with not using IV edaravone. Data from adequately powered RCTs are needed to support this finding.

FUNDING

Funded by Mitsubishi Tanabe Pharma America.