Regulatory-grade clinical trial design using real-world data

Taipei Medical University Clinical Research Database: a collaborative hospital EHR database aligned with international common data standards

OBJECTIVE

The objective of this paper is to provide a comprehensive overview of the development and features of the Taipei Medical University Clinical Research Database (TMUCRD), a repository of real-world data (RWD) derived from electronic health records (EHRs) and other sources.

METHODS

TMUCRD was developed by integrating EHRs from three affiliated hospitals, including Taipei Medical University Hospital, Wan-Fang Hospital and Shuang-Ho Hospital. The data cover over 15 years and include diverse patient care information. The database was converted to the Observational Medical Outcomes Partnership Common Data Model (OMOP CDM) for standardisation.

RESULTS

TMUCRD comprises 89 tables (eg, 29 tables for each hospital and 2 linked tables), including demographics, diagnoses, medications, procedures and measurements, among others. It encompasses data from more than 4.15 million patients with various medical records, spanning from the year 2004 to 2021. The dataset offers insights into disease prevalence, medication usage, laboratory tests and patient characteristics.

DISCUSSION

TMUCRD stands out due to its unique advantages, including diverse data types, comprehensive patient information, linked mortality and cancer registry data, regular updates and a swift application process. Its compatibility with the OMOP CDM enhances its usability and interoperability.

CONCLUSION

TMUCRD serves as a valuable resource for researchers and scholars interested in leveraging RWD for clinical research. Its availability and integration of diverse healthcare data contribute to a collaborative and data-driven approach to advancing medical knowledge and practice.

Exploring Breast Cancer Systemic Drug Therapy Patterns in Real-World Data

PURPOSE

To explore medications and their administration patterns in real-world patients with breast cancer.

METHODS

A retrospective study was performed using TriNetX, a federated network of deidentified, Health Insurance Portability and Accountability Act-compliant data from 21 health care organizations across North America. Patients diagnosed with breast cancer between January 1, 2013, and May 31, 2022, were included. We investigated a rule-based and unsupervised learning algorithm to extract medications and their administration patterns. To group similar administration patterns, we used three features in k-means clustering: total number of administrations, median number of days between administrations, and standard deviation of the days between administrations. We explored the first three lines of therapy for patients classified into six groups on the basis of their stage at diagnosis (early as stages I-III v late as stage IV) and the sensitivity of the tumor's receptors to targeted therapies: hormone receptor-positive/human epidermal growth factor 2-negative (HR+/ERBB2-), ERBB2-positive (ERBB2+/HR±), or triple-negative (TN; HR-/ERBB2-). To add credence to the derived regimens, we compared them to the National Comprehensive Cancer Network (NCCN): Breast Cancer (version 2.2023) recommendations.

RESULTS

In early-stage HR+/ERBB2- and TN groups, the most common regimens were (1) cyclophosphamide and docetaxel, administered once every 3 weeks for three to six cycles and (2) cyclophosphamide and doxorubicin, administered once every 2 weeks for four cycles, followed by paclitaxel administered once every week for 12 cycles. In the early-stage ERBB2+/HR± group, most patients were administered carboplatin and docetaxel with or without pertuzumab and with trastuzumab (for six or more cycles). Medications most commonly administered in our data set (7,798 patients) agreed with recommendations from the NCCN in terms of medications (regimens), number of administrations (cycles), and days between administrations (cycle length).

CONCLUSION

Although there is a general agreement with the NCCN Guidelines, real-world medication data exhibit variability in the medications and their administration patterns.

Regulatory and HTA Considerations for Development of Real-World Data Derived External Controls

Regulators and Health Technology Assessment (HTA) bodies are increasingly familiar with, and publishing guidance on, external controls derived from real-world data (RWD) to generate real-world evidence (RWE). We recently conducted a systematic literature review (SLR) evaluating publicly available information on the use of RWD-derived external controls to contextualize outcomes from uncontrolled trials submitted to the European Medicines Agency (EMA), the US Food and Drug Administration (FDA), and/or select HTA bodies. The review identified several key operational and methodological aspects for which more detailed guidance and alignment within and between regulatory agencies and HTA bodies is necessary. This paper builds on the SLR findings by delineating a set of key takeaways for the responsible generation of fit-for-purpose RWE. Practical methodological and operational guidelines for designing, conducting, and reporting RWD-derived external control studies are explored and discussed. These considerations include: (i) early engagement with regulators and HTA bodies during the study planning phase; (ii) consideration of the appropriateness and comparability of external controls across multiple dimensions, including eligibility criteria, temporality, population representation, and clinical evaluation; (iii) ensuring adequate sample sizes, including hypothesis testing considerations; (iv) implementation of a clear and transparent strategy for assessing and addressing data quality, including data missingness across trials and RWD; (v) selection of comparable and meaningful endpoints that are operationalized and analyzed using appropriate analytic methods; and (vi) conduct of sensitivity analyses to assess the robustness of findings in the context of uncertainty and sources of potential bias.

Using artificial intelligence to identify patients with migraine and associated symptoms and conditions within electronic health records

BACKGROUND

Real-world evidence (RWE)-based on information obtained from sources such as electronic health records (EHRs), claims and billing databases, product and disease registries, and personal devices and health applications-is increasingly used to support healthcare decision making. There is variability in the collection of EHR data, which includes "structured data" in predefined fields (e.g., problem list, open claims, medication list, etc.) and "unstructured data" as free text or narrative. Healthcare providers are likely to provide more complete information as free text, but extracting meaning from these fields requires newer technologies and a rigorous methodology to generate higher-quality evidence. Herein, an approach to identify concepts associated with the presence and progression of migraine was developed and validated using the complete patient record in EHR data, including both the structured and unstructured portions.

METHODS

"Traditional RWE" approaches (i.e., capture from structured EHR fields and extraction using structured queries) and "Advanced RWE" approaches (i.e., capture from unstructured EHR data and processing by artificial intelligence [AI] technology, including natural language processing and AI-based inference) were evaluated against a manual chart abstraction reference standard for data collected from a tertiary care setting. The primary endpoint was recall; differences were compared using chi square.

RESULTS

Compared with manual chart abstraction, recall for migraine and headache were 66.6% and 29.6%, respectively, for Traditional RWE, and 96.8% and 92.9% for Advanced RWE; differences were statistically significant (absolute differences, 30.2% and 63.3%; P < 0.001). Recall of 6 migraine-associated symptoms favored Advanced RWE over Traditional RWE to a greater extent (absolute differences, 71.5-88.8%; P < 0.001). The difference between traditional and advanced techniques for recall of migraine medications was less pronounced, approximately 80% for Traditional RWE and ≥ 98% for Advanced RWE (P < 0.001).

CONCLUSION

Unstructured EHR data, processed using AI technologies, provides a more credible approach to enable RWE in migraine than using structured EHR and claims data alone. An algorithm was developed that could be used to further study and validate the use of RWE to support diagnosis and management of patients with migraine.

A nutritional supplement based on a synbiotic combination of Bacillus subtilis DSM 32315 and L-alanyl-L-glutamine improves glucose metabolism in healthy prediabetic subjects - A real-life post-marketing study

Introduction

Impaired glucose homeostasis is a significant risk factor for cardiometabolic diseases, whereas the efficacy of available standard therapies is limited, mainly because of poor adherence. This post-marketing study assessed the glucose-lowering potential of a synbiotic-based formulation.

Methods

One hundred ninety-two participants were enrolled in a digital nutrition program with continuous glucose monitoring (CGM) and received a study product comprising Bacillus subtilis DSM 32315 and L-alanyl-L-glutamine. Participants underwent a first sensor phase without supplementation, followed by a 14-day supplementation phase without sensor, and completed by a second sensor phase while continuing supplementation. Fasting glucose levels were determined before and after supplementation by CGM. In addition, the postprandial glycemic response to an oral glucose challenge, body weight, HbA1c concentrations, and BMI was analyzed. Subgroup analyses of subjects with elevated glucose and HbA1c levels vs. normoglycemic subjects were performed.

Results

Supplementation with the study product resulted in significant improvements in glucose parameters (delta values: fasting glucose -2,13% ± 8.86; iAUC_0-120 -4.91% ± 78.87; HbA1c: -1.20% ± 4.72) accompanied by a significant weight reduction (-1.07 kg ± 2.30) in the study population. Subgroup analyses revealed that the improvements were mainly attributed to a prediabetic subgroup with elevated fasting glucose and HbA1c values before supplementation (delta values: fasting glucose -6.10% 4± 7.89; iAUC_0-120 -6.28% ± 115.85; HbA1c -3.31% ± 4.36; weight: -1.47 kg ± 2.82).

Conclusion

This study indicates that the synbiotic composition is an effective and convenient approach to counteract hyperglycemia. Further placebo-controlled studies are warranted to test its efficacy in the treatment of cardiometabolic diseases.

Sex-related differences in spontaneous intracerebral hemorrhage outcomes: A prognostic study based on 111,112 medical records

Objective

To identify sex-related differences in the outcome of hospitalized patients with spontaneous intracerebral hemorrhage (SICH), and to identify potential causal pathways between sex and SICH outcome.

Methods

A total of 111,112 medical records of in-hospital patients with SICH were collected. Data- and expert-driven techniques were applied, such as a multivariate logistic regression model and causal mediation analysis. These analyses were used to determine the confounders and mediators, estimate the true effect of sex on the SICH outcome, and estimate the average causal mediation effect for each mediator.

Results

(1) Failure (disability or death) rates in women with SICH were significantly lower than in men with SICH. On the day of discharge, the odds ratio (OR) of failure between women and men was 0.9137 [95% confidence interval (CI), 0.8879–0.9402], while the odds ratio at 90 days post-discharge was 0.9353 (95% confidence interval, 0.9121–0.9591). (2) The sex-related difference in SICH outcome decreased with increasing age and disappeared after 75 years. (3) Deep coma, brainstem hemorrhage, and an infratentorial hemorrhage volume of >10 ml accounted for 62.76% (p < 0.001), 33.46% (p < 0.001), and 11.56% (p < 0.001) of the overall effect on the day of discharge, and for 52.28% (p < 0.001), 27.65% (p < 0.001), and 10.86% (p < 0.001) of the overall effect at the 90-day post-discharge.

Conclusion

Men have a higher failure risk than women, which may be partially mediated by a higher risk for deep coma, brainstem hemorrhage, and an infratentorial hemorrhage volume of >10 ml. Future work should explore the biological mechanisms underlying this difference.

Digital Health and Big Data Analytics: Implications of Real-World Evidence for Clinicians and Policymakers

Real world data (RWD) and real-world evidence (RWE) plays an increasingly important role in clinical research since scientific knowledge is obtained during routine clinical large-scale practice and not experimentally as occurs in the highly controlled traditional clinical trials. Particularly, the electronic health records (EHRs) are a relevant source of data. Nevertheless, there are also significant challenges in the correct use and interpretation of EHRs data, such as bias, heterogeneity of the population, and missing or non-standardized data formats. Despite the RWD and RWE recognized difficulties, these are easily outweighed by the benefits of ensuring the efficacy, safety, and cost-effectiveness in complement to the gold standards of the randomized controlled trial (RCT), namely by providing a complete picture regarding factors and variables that can guide robust clinical decisions. Their relevance can be even further evident as healthcare units develop more accurate EHRs always in the respect for the privacy of patient data. This editorial is an overview of the RWD and RWE major aspects of the state of the art and supports the Special Issue on "Digital Health and Big Data Analytics: Implications of Real-World Evidence for Clinicians and Policymakers" aimed to explore all the potential and the utility of RWD and RWE in offering insights on diseases in a broad spectrum.

Real-World Data as External Controls: Practical Experience from Notable Marketing Applications of New Therapies

INTRODUCTION

Real-world data (RWD) can contextualize findings from single-arm trials when randomized comparative trials are unethical or unfeasible. Findings from single-arm trials alone are difficult to interpret and a comparison, when feasible and meaningful, to patient-level information from RWD facilitates the evaluation. As such, there have been several recent regulatory applications including RWD or other external data to support the product's efficacy and safety. This paper summarizes some lessons learned from such contextualization from 20 notable new drug or biologic licensing applications in oncology and rare diseases.

METHODS

This review focuses on 20 notable new drug or biologic licensing applications that included patient-level RWD or other external data for contextualization of trial results. Publicly available regulatory documents including clinical and statistical reviews, advisory committee briefing materials and minutes, and approved product labeling were retrieved for each application. The authors conducted independent assessments of these documents focusing on the regulatory evaluation, in each case. Three examples are presented in detail to illustrate the salient issues and themes identified across applications.

RESULTS

Regulatory decisions were strongly influenced by the quality and usability of the RWD. Comparability of cohort attributes such as endpoints, populations, follow-up, index and censoring criteria, as well as data completeness and accuracy of key variables appeared to be essential to ensure the quality and relevance of the RWD. Given adequate sample size of the clinical trials or external control, the use of appropriate analytic methods to properly account for confounding, such as regression or matching, and pre-specification of these methods while blinded to patient outcomes seemed good strategies to address baseline differences.

DISCUSSION

Contextualizing single-arm trials with patient-level RWD appears to be an advance in regulatory science; however, challenges remain. Statisticians and epidemiologists have long focused on analytical methods for comparative effectiveness but hurdles in use of RWD have often occurred upstream of the analyses. More specifically, we noted hurdles in evaluating data quality, justifying cohort selection or initiation of follow-up, and demonstrating comparability of cohorts and endpoints.

A Road Map for Remote Digital Health Technology for Motor Neuron Disease

Despite recent and potent technological advances, the real-world implementation of remote digital health technology in the care and monitoring of patients with motor neuron disease has not yet been realized. Digital health technology may increase the accessibility to and personalization of care, whereas remote biosensors could optimize the collection of vital clinical parameters, irrespective of patients’ ability to visit the clinic. To facilitate the wide-scale adoption of digital health care technology and to align current initiatives, we outline a road map that will identify clinically relevant digital parameters; mediate the development of benefit-to-burden criteria for innovative technology; and direct the validation, harmonization, and adoption of digital health care technology in real-world settings. We define two key end products of the road map: (1) a set of reliable digital parameters to capture data collected under free-living conditions that reflect patient-centric measures and facilitate clinical decision making and (2) an integrated, open-source system that provides personalized feedback to patients, health care providers, clinical researchers, and caregivers and is linked to a flexible and adaptable platform that integrates patient data in real time. Given the ever-changing care needs of patients and the relentless progression rate of motor neuron disease, the adoption of digital health care technology will significantly benefit the delivery of care and accelerate the development of effective treatments.

Looking in the medicine cabinet: methods for using real-world data to assess the impact of measles, mumps and rubella (MMR) and recombinant adjuvanted varicella-zoster vaccines on coronavirus disease 2019 (COVID-19) prevention and case fatality

Background: Analysis of real-world data can be used to identify promising leads and dead ends among products being repurposed for clinical practice for coronavirus disease 2019 (COVID-19).  This paper uses real-world data from Cerner Labs collected from 90 source institutions in the United States to assess the potential impact of two viral vaccines on COVID-19 case fatality rates. Methods: We identified 373,032 polymerase chase reaction (PCR)-positive COVID-19 cases in the Cerner Labs database between 01-MAR-2020 and 31-DEC-2020 and identified patients that had received measles, mumps and rubella (MMR) or a recombinant adjuvanted varicella-zoster vaccine within the previous 5 years. We calculated heterogeneity scores to support interpretation of results across institutions, and used stepwise forward variable selection to construct covariable-based propensity scores. These scores were used to match cases and control for biasing and confounding issues inherent in observational data. Results: Neither the recombinant adjuvanted varicella-zoster vaccine nor MMR showed significant efficacy in prevention of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection. We could not derive clinically significant results on the impact of MMR for case fatality rates due to persistently high rates of heterogeneity between institutions. However, we were able to achieve acceptable levels of heterogeneity for the analysis of the recombinant adjuvanted varicella-zoster vaccine, and found a clinically meaningful benefit of reduced case fatality rate, with an odds ratio of 0.43 (95% confidence interval [CI]: 0.38 – 0.48). Conclusions: Using propensity score matching and heterogeneity statistics can help guide our interpretation of real-world data, and rigorous statistical methods are needed to reduce bias or disparities in data interpretation. Applying these methods to the impact of viral vaccines on COVID-19 case fatalities yields actionable findings for further analysis.

Power prior for borrowing the real-world data in bioequivalence test with a parallel design

Recently, real-world study has attracted wide attention for drug development. In bioequivalence study, the reference drug often has been marketed for many years and accumulated abundant real-world data. It is therefore appealing to incorporate these data in the design to improve trial efficiency. In this paper, we propose a Bayesian method to include real-world data of the reference drug in a current bioequivalence trial, with the aim to increase the power of analysis and reduce sample size for long half-life drugs. We adopt the power prior method for incorporating real-world data and use the average bioequivalence posterior probability to evaluate the bioequivalence between the test drug and the reference drug. Simulations were conducted to investigate the performance of the proposed method in different scenarios. The simulation results show that the proposed design has higher power than the traditional design without borrowing real-world data, while controlling the type I error. Moreover, the proposed method saves sample size and reduces costs for the trial.