Contemporary use of real-world data for clinical trial conduct in the United States: a scoping review

Protocol for a phase 2 study of bosutinib for amyotrophic lateral sclerosis using real-world data: induced pluripotent stem cell-based drug repurposing for amyotrophic lateral sclerosis medicine (iDReAM) study

INTRODUCTION

Amyotrophic lateral sclerosis (ALS) is a progressive, severe neurodegenerative disease caused by motor neuron death. Development of a medicine for ALS is urgently needed, and induced pluripotent cell-based drug repurposing identified a Src/c-Abl inhibitor, bosutinib, as a candidate for molecular targeted therapy of ALS. A phase 1 study confirmed the safety and tolerability of bosutinib in a 12-week treatment of ALS patients. The objectives of this study are to evaluate the efficacy and longer-term safety of bosutinib in ALS patients.

METHODS AND ANALYSIS

An open-label, multicentre phase 2 study was designed. The study consisted of a 12-week observation period, a 1-week transitional period, a 24-week study treatment period and a 4-week follow-up period. Following the transitional period, patients whose total Revised ALS Functional Rating Scale (ALSFRS-R) score declined by 1 to 4 points during the 12-week observation period were to receive bosutinib for 24 weeks. In this study, 25 ALS patients will be enrolled; patients will be randomly assigned to the following groups: 12 patients in the 200 mg quaque die (QD) group and 13 patients in the 300 mg QD group of bosutinib. The safety and exploratory efficacy of bosutinib in ALS patients for 24 weeks will be assessed. Efficacy using the ALSFRS-R score will be compared with the external published data from an edaravone study (MCI186-19) and registry data from a multicentre ALS cohort study, the Japanese Consortium for Amyotrophic Lateral Sclerosis Research.

ETHICS AND DISSEMINATION

This study was approved by the ethics committees of Kyoto University, Tokushima University, Kitasato University, Tottori University, Nara Medical University School of Medicine, Toho University and Hiroshima University. The findings will be disseminated in peer-reviewed journals and at scientific conferences.

TRIAL REGISTRATION NUMBER

jRCT2051220002; Pre-results, NCT04744532; Pre-results.

The use of linked administrative data in Australian randomised controlled trials: A scoping review

BACKGROUND/AIMS

The demand for simplified data collection within trials to increase efficiency and reduce costs has led to broader interest in repurposing routinely collected administrative data for use in clinical trials research. The aim of this scoping review is to describe how and why administrative data have been used in Australian randomised controlled trial conduct and analyses, specifically the advantages and limitations of their use as well as barriers and enablers to accessing administrative data for use alongside randomised controlled trials.

METHODS

Databases were searched to November 2022. Randomised controlled trials were included if they accessed one or more Australian administrative data sets, where some or all trial participants were enrolled in Australia, and where the article was published between January 2000 and November 2022. Titles and abstracts were independently screened by two reviewers, and the full texts of selected studies were assessed against the eligibility criteria by two independent reviewers. Data were extracted from included articles by two reviewers using a data extraction tool.

RESULTS

Forty-one articles from 36 randomised controlled trials were included. Trial characteristics, including the sample size, disease area, population, and intervention, were varied; however, randomised controlled trials most commonly linked to government reimbursed claims data sets, hospital admissions data sets and birth/death registries, and the most common reason for linkage was to ascertain disease outcomes or survival status, and to track health service use. The majority of randomised controlled trials were able to achieve linkage in over 90% of trial participants; however, consent and participant withdrawals were common limitations to participant linkage. Reported advantages were the reliability and accuracy of the data, the ease of long term follow-up, and the use of established data linkage units. Common reported limitations were locating participants who had moved outside the jurisdictional area, missing data where consent was not provided, and unavailability of certain healthcare data.

CONCLUSIONS

As linked administrative data are not intended for research purposes, detailed knowledge of the data sets is required by researchers, and the time delay in receiving the data is viewed as a barrier to its use. The lack of access to primary care data sets is viewed as a barrier to administrative data use; however, work to expand the number of healthcare data sets that can be linked has made it easier for researchers to access and use these data, which may have implications on how randomised controlled trials will be run in future.

Development and utility of a clinical research informatics application for participant recruitment and workflow management for a return of results pilot trial in familial hypercholesterolemia in the Million Veteran Program

Objective

The development of clinical research informatics tools and workflow processes associated with re-engaging biobank participants has become necessary as genomic repositories increasingly consider the return of actionable research results.

Materials and Methods

Here we describe the development and utility of an informatics application for participant recruitment and enrollment management for the Veterans Affairs Million Veteran Program Return Of Actionable Results Study, a randomized controlled pilot trial returning individual genetic results associated with familial hypercholesterolemia.

Results

The application is developed in Python-Flask and was placed into production in November 2021. The application includes modules for chart review, medication reconciliation, participant contact and biospecimen logging, survey recording, randomization, and documentation of genetic counseling and result disclosure. Three primary users, a genetic counselor and two research coordinators, and 326 Veteran participants have been integrated into the system as of February 23, 2023. The application has successfully handled 3367 task requests involving greater than 95 000 structured data points. Specifically, application users have recorded 326 chart reviews, 867 recruitment telephone calls, 158 telephone-based surveys, and 61 return of results genetic counseling sessions, among other available study tasks.

Conclusion

The development of usable, customizable, and secure informatics tools will become increasingly important as large genomic repositories begin to return research results at scale. Our work provides a proof-of-concept for developing and using such tools to aid in managing the return of results process within a national biobank.

Effectiveness of IT-supported patient recruitment: study protocol for an interrupted time series study at ten German university hospitals

BACKGROUND

As part of the German Medical Informatics Initiative, the MIRACUM project establishes data integration centers across ten German university hospitals. The embedded MIRACUM Use Case "Alerting in Care - IT Support for Patient Recruitment", aims to support the recruitment into clinical trials by automatically querying the repositories for patients satisfying eligibility criteria and presenting them as screening candidates. The objective of this study is to investigate whether the developed recruitment tool has a positive effect on study recruitment within a multi-center environment by increasing the number of participants. Its secondary objective is the measurement of organizational burden and user satisfaction of the provided IT solution.

METHODS

The study uses an Interrupted Time Series Design with a duration of 15 months. All trials start in the control phase of randomized length with regular recruitment and change to the intervention phase with additional IT support. The intervention consists of the application of a recruitment-support system which uses patient data collected in general care for screening according to specific criteria. The inclusion and exclusion criteria of all selected trials are translated into a machine-readable format using the OHDSI ATLAS tool. All patient data from the data integration centers is regularly checked against these criteria. The primary outcome is the number of participants recruited per trial and week standardized by the targeted number of participants per week and the expected recruitment duration of the specific trial. Secondary outcomes are usability, usefulness, and efficacy of the recruitment support. Sample size calculation based on simple parallel group assumption can demonstrate an effect size of d=0.57 on a significance level of 5% and a power of 80% with a total number of 100 trials (10 per site). Data describing the included trials and the recruitment process is collected at each site. The primary analysis will be conducted using linear mixed models with the actual recruitment number per week and trial standardized by the expected recruitment number per week and trial as the dependent variable.

DISCUSSION

The application of an IT-supported recruitment solution developed in the MIRACUM consortium leads to an increased number of recruited participants in studies at German university hospitals. It supports employees engaged in the recruitment of trial participants and is easy to integrate in their daily work.

A comparative study of clinical trial and real-world data in patients with diabetic kidney disease

A growing body of research is focusing on real-world data (RWD) to supplement or replace randomized controlled trials (RCTs). However, due to the disparities in data generation mechanisms, differences are likely and necessitate scrutiny to validate the merging of these datasets. We compared the characteristics of RCT data from 5734 diabetic kidney disease patients with corresponding RWD from electronic health records (EHRs) of 23,523 patients. Demographics, diagnoses, medications, laboratory measurements, and vital signs were analyzed using visualization, statistical comparison, and cluster analysis. RCT and RWD sets exhibited significant differences in prevalence, longitudinality, completeness, and sampling density. The cluster analysis revealed distinct patient subgroups within both RCT and RWD sets, as well as clusters containing patients from both sets. We stress the importance of validation to verify the feasibility of combining RCT and RWD, for instance, in building an external control arm. Our results highlight general differences between RCT and RWD sets, which should be considered during the planning stages of an RCT-RWD study. If they are, RWD has the potential to enrich RCT data by providing first-hand baseline data, filling in missing data or by subgrouping or matching individuals, which calls for advanced methods to mitigate the differences between datasets.

Real-world Application of Racial and Ethnic Imputation and Cohort Balancing Techniques to Deliver Equitable Clinical Trial Recruitment

Enhancing diversity and inclusion in clinical trial recruitment, especially for historically marginalized populations including Black, Indigenous, and People of Color individuals, is essential. This practice ensures that generalizable trial results are achieved to deliver safe, effective, and equitable health and healthcare. However, recruitment is limited by two inextricably linked barriers - the inability to recruit and retain enough trial participants, and the lack of diversity amongst trial populations whereby racial and ethnic groups are underrepresented when compared to national composition. To overcome these barriers, this study describes and evaluates a framework that combines 1) probabilistic and machine learning models to accurately impute missing race and ethnicity fields in real-world data including medical and pharmacy claims for the identification of eligible trial participants, 2) randomized controlled trial experimentation to deliver an optimal patient outreach strategy, and 3) stratified sampling techniques to effectively balance cohorts to continuously improve engagement and recruitment metrics.

The Roles of Electronic Health Records for Clinical Trials in Low- and Middle-Income Countries: Scoping Review

BACKGROUND

Clinical trials are a crucial element in advancing medical knowledge and developing new treatments by establishing the evidence base for safety and therapeutic efficacy. However, the success of these trials depends on various factors, including trial design, project planning, research staff training, and adequate sample size. It is also crucial to recruit participants efficiently and retain them throughout the trial to ensure timely completion.

OBJECTIVE

There is an increasing interest in using electronic health records (EHRs)-a widely adopted tool in clinical practice-for clinical trials. This scoping review aims to understand the use of EHR in supporting the conduct of clinical trials in low- and middle-income countries (LMICs) and to identify its strengths and limitations.

METHODS

A comprehensive search was performed using 5 databases: MEDLINE, Embase, Scopus, Cochrane Library, and the Cumulative Index to Nursing and Allied Health Literature. We followed the latest version of the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guideline to conduct this review. We included clinical trials that used EHR at any step, conducted a narrative synthesis of the included studies, and mapped the roles of EHRs into the life cycle of a clinical trial.

RESULTS

A total of 30 studies met the inclusion criteria: 13 were randomized controlled trials, 3 were cluster randomized controlled trials, 12 were quasi-experimental studies, and 2 were feasibility pilot studies. Most of the studies addressed infectious diseases (15/30, 50%), with 80% (12/15) of them about HIV or AIDS and another 40% (12/30) focused on noncommunicable diseases. Our synthesis divided the roles of EHRs into 7 major categories: participant identification and recruitment (12/30, 40%), baseline information collection (6/30, 20%), intervention (8/30, 27%), fidelity assessment (2/30, 7%), primary outcome assessment (24/30, 80%), nonprimary outcome assessment (13/30, 43%), and extended follow-up (2/30, 7%). None of the studies used EHR for participant consent and randomization.

CONCLUSIONS

Despite the enormous potential of EHRs to increase the effectiveness and efficiency of conducting clinical trials in LMICs, challenges remain. Continued exploration of the appropriate uses of EHRs by navigating their strengths and limitations to ensure fitness for use is necessary to better understand the most optimal uses of EHRs for conducting clinical trials in LMICs.

The Stanford Medicine data science ecosystem for clinical and translational research

Objective

To describe the infrastructure, tools, and services developed at Stanford Medicine to maintain its data science ecosystem and research patient data repository for clinical and translational research.

Materials and Methods

The data science ecosystem, dubbed the Stanford Data Science Resources (SDSR), includes infrastructure and tools to create, search, retrieve, and analyze patient data, as well as services for data deidentification, linkage, and processing to extract high-value information from healthcare IT systems. Data are made available via self-service and concierge access, on HIPAA compliant secure computing infrastructure supported by in-depth user training.

Results

The Stanford Medicine Research Data Repository (STARR) functions as the SDSR data integration point, and includes electronic medical records, clinical images, text, bedside monitoring data and HL7 messages. SDSR tools include tools for electronic phenotyping, cohort building, and a search engine for patient timelines. The SDSR supports patient data collection, reproducible research, and teaching using healthcare data, and facilitates industry collaborations and large-scale observational studies.

Discussion

Research patient data repositories and their underlying data science infrastructure are essential to realizing a learning health system and advancing the mission of academic medical centers. Challenges to maintaining the SDSR include ensuring sufficient financial support while providing researchers and clinicians with maximal access to data and digital infrastructure, balancing tool development with user training, and supporting the diverse needs of users.

Conclusion

Our experience maintaining the SDSR offers a case study for academic medical centers developing data science and research informatics infrastructure.

Integrated Real-World Study Databases in 3 Diverse Asian Health Care Systems in Taiwan, India, and Thailand: Scoping Review

BACKGROUND

The use of real-world data (RWD) warehouses for research in Asia is on the rise, but current trends remain largely unexplored. Given the varied economic and health care landscapes in different Asian countries, understanding these trends can offer valuable insights.

OBJECTIVE

We sought to discern the contemporary landscape of linked RWD warehouses and explore their trends and patterns in 3 Asian countries with contrasting economies and health care systems: Taiwan, India, and Thailand.

METHODS

Using a systematic scoping review methodology, we conducted an exhaustive literature search on PubMed with filters for the English language and the past 5 years. The search combined Medical Subject Heading terms and specific keywords. Studies were screened against strict eligibility criteria to identify eligible studies using RWD databases from more than one health care facility in at least 1 of the 3 target countries.

RESULTS

Our search yielded 2277 studies, of which 833 (36.6%) met our criteria. Overall, single-country studies (SCS) dominated at 89.4% (n=745), with cross-country collaboration studies (CCCS) being at 10.6% (n=88). However, the country-wise breakdown showed that of all the SCS, 623 (83.6%) were from Taiwan, 81 (10.9%) from India, and 41 (5.5%) from Thailand. Among the total studies conducted in each country, India at 39.1% (n=133) and Thailand at 43.1% (n=72) had a significantly higher percentage of CCCS compared to Taiwan at 7.6% (n=51). Over a 5-year span from 2017 to 2022, India and Thailand experienced an annual increase in RWD studies by approximately 18.2% and 13.8%, respectively, while Taiwan's contributions remained consistent. Comparative effectiveness research (CER) was predominant in Taiwan (n=410, or 65.8% of SCS) but less common in India (n=12, or 14.8% of SCS) and Thailand (n=11, or 26.8% of SCS). CER percentages in CCCS were similar across the 3 countries, ranging from 19.2% (n=10) to 29% (n=9). The type of RWD source also varied significantly across countries, with India demonstrating a high reliance on electronic medical records or electronic health records at 55.6% (n=45) of SCS and Taiwan showing an increasing trend in their use over the period. Registries were used in 26 (83.9%) CCCS and 31 (75.6%) SCS from Thailand but in <50% of SCS from Taiwan and India. Health insurance/administrative claims data were used in most of the SCS from Taiwan (n=458, 73.5%). There was a consistent predominant focus on cardiology/metabolic disorders in all studies, with a noticeable increase in oncology and infectious disease research from 2017 to 2022.

CONCLUSIONS

This review provides a comprehensive understanding of the evolving landscape of RWD research in Taiwan, India, and Thailand. The observed differences and trends emphasize the unique economic, clinical, and research settings in each country, advocating for tailored strategies for leveraging RWD for future health care research and decision-making.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/43741.

Optimizing data integration in trials that use EHR data: lessons learned from a multi-center randomized clinical trial

BACKGROUND

Despite great promise, trials that ascertain patient clinical data from electronic health records (EHR), referred to here as "EHR-sourced" trials, are limited by uncertainty about how existing trial sites and infrastructure can be best used to operationalize study goals. Evidence is needed to support the practical use of EHRs in contemporary clinical trial settings.

MAIN TEXT

We describe a demonstration project that used EHR data to complement data collected for a contemporary multi-center pharmaceutical industry outcomes trial, and how a central coordinating center supported participating sites through the technical, governance, and operational aspects of this type of activity. We discuss operational considerations related to site selection, data extraction, site performance, and data transfer and quality review, and we outline challenges and lessons learned. We surveyed potential sites and used their responses to assess feasibility, determine the potential capabilities of sites and choose an appropriate data extraction strategy. We designed a flexible, multimodal approach for data extraction, enabling each site to either leverage an existing data source, create a new research datamart, or send all data to the central coordinating center to produce the requisite data elements. We evaluated site performance, as reflected by the speed of contracting and IRB approval, total patients enrolled, enrollment yield, data quality, and compared performance by data collection strategy.

CONCLUSION

While broadening the type of sites able to participate in EHR-sourced trials may lead to greater generalizability and improved enrollment, sites with fewer technical resources may require additional support to participate. Central coordinating center support is essential to facilitate the execution of operational processes. Future work should focus on sharing lessons learned and creating reusable tools to facilitate participation of heterogeneous trial sites.

Randomized-controlled trials are methodologically inappropriate in adolescent transgender healthcare

Background

Despite multiple rigorous observational studies documenting the association between positive mental health outcomes and access to puberty blockers, hormone therapy, and transition-related surgeries among adolescents, some jurisdictions have banned or are attempting to ban gender-affirming medical interventions for minors due to an absence of randomized-controlled trials (RCTs) proving their mental health benefits.

Methods

This article critically reviews whether RCTs are methodologically appropriate for studying the association between adolescent gender-affirming care and mental health outcomes.

Results

The scientific value of RCTs is severely impeded when studying the impact of gender-affirming care on the mental health of trans adolescent. Gender-affirming interventions have physiologically evident effects and are highly desired by participants, giving rise to concerns over adherence, drop-out, response bias, and generalizability. Complementary and well-designed observational studies can instead be used to ground reliable recommendations for clinical practice and policymaking in adolescent trans healthcare, without the need for RCTs.

Conclusion

The lack of RCTs on the mental health impacts of gender-affirming care for trans adolescents does not entail that gender-affirming interventions are based on insufficient evidence. Given the methodological limitations of RCTs, complementary and well-designed observational studies offer more reliable scientific evidence than RCTs and should be considered of sufficient quality to guide clinical practice and policymaking.

Generate Analysis-Ready Data for Real-world Evidence: Tutorial for Harnessing Electronic Health Records With Advanced Informatic Technologies

Although randomized controlled trials (RCTs) are the gold standard for establishing the efficacy and safety of a medical treatment, real-world evidence (RWE) generated from real-world data has been vital in postapproval monitoring and is being promoted for the regulatory process of experimental therapies. An emerging source of real-world data is electronic health records (EHRs), which contain detailed information on patient care in both structured (eg, diagnosis codes) and unstructured (eg, clinical notes and images) forms. Despite the granularity of the data available in EHRs, the critical variables required to reliably assess the relationship between a treatment and clinical outcome are challenging to extract. To address this fundamental challenge and accelerate the reliable use of EHRs for RWE, we introduce an integrated data curation and modeling pipeline consisting of 4 modules that leverage recent advances in natural language processing, computational phenotyping, and causal modeling techniques with noisy data. Module 1 consists of techniques for data harmonization. We use natural language processing to recognize clinical variables from RCT design documents and map the extracted variables to EHR features with description matching and knowledge networks. Module 2 then develops techniques for cohort construction using advanced phenotyping algorithms to both identify patients with diseases of interest and define the treatment arms. Module 3 introduces methods for variable curation, including a list of existing tools to extract baseline variables from different sources (eg, codified, free text, and medical imaging) and end points of various types (eg, death, binary, temporal, and numerical). Finally, module 4 presents validation and robust modeling methods, and we propose a strategy to create gold-standard labels for EHR variables of interest to validate data curation quality and perform subsequent causal modeling for RWE. In addition to the workflow proposed in our pipeline, we also develop a reporting guideline for RWE that covers the necessary information to facilitate transparent reporting and reproducibility of results. Moreover, our pipeline is highly data driven, enhancing study data with a rich variety of publicly available information and knowledge sources. We also showcase our pipeline and provide guidance on the deployment of relevant tools by revisiting the emulation of the Clinical Outcomes of Surgical Therapy Study Group Trial on laparoscopy-assisted colectomy versus open colectomy in patients with early-stage colon cancer. We also draw on existing literature on EHR emulation of RCTs together with our own studies with the Mass General Brigham EHR.

Data Challenges for Externally Controlled Trials: Viewpoint

The preferred evidence of a large randomized controlled trial is difficult to adopt in scenarios, such as rare conditions or clinical subgroups with high unmet needs, and evidence from external sources, including real-world data, is being increasingly considered by decision makers. Real-world data originate from many sources, and identifying suitable real-world data that can be used to contextualize a single-arm trial, as an external control arm, has several challenges. In this viewpoint article, we provide an overview of the technical challenges raised by regulatory and health reimbursement agencies when evaluating comparative efficacy, such as identification, outcome, and time selection challenges. By breaking down these challenges, we provide practical solutions for researchers to consider through the approaches of detailed planning, collection, and record linkage to analyze external data for comparative efficacy.

Evolving a national clinical trials learning health system

Clinical trials generate key evidence to inform decision making, and also benefit participants directly. However, clinical trials frequently fail, often struggle to enroll participants, and are expensive. Part of the problem with trial conduct may be the disconnected nature of clinical trials, preventing rapid data sharing, generation of insights and targeted improvement interventions, and identification of knowledge gaps. In other areas of healthcare, a learning health system (LHS) has been proposed as a model to facilitate continuous learning and improvement. We propose that an LHS approach could greatly benefit clinical trials, allowing for continuous improvements to trial conduct and efficiency. A robust trial data sharing system, continuous analysis of trial enrollment and other success metrics, and development of targeted trial improvement interventions are potentially key components of a Trials LHS reflecting the learning cycle and allowing for continuous trial improvement. Through the development and use of a Trials LHS, clinical trials could be treated as a system, producing benefits to patients, advancing care, and decreasing costs for stakeholders.

Machine learning approaches for electronic health records phenotyping: a methodical review

OBJECTIVE

Accurate and rapid phenotyping is a prerequisite to leveraging electronic health records for biomedical research. While early phenotyping relied on rule-based algorithms curated by experts, machine learning (ML) approaches have emerged as an alternative to improve scalability across phenotypes and healthcare settings. This study evaluates ML-based phenotyping with respect to (1) the data sources used, (2) the phenotypes considered, (3) the methods applied, and (4) the reporting and evaluation methods used.

MATERIALS AND METHODS

We searched PubMed and Web of Science for articles published between 2018 and 2022. After screening 850 articles, we recorded 37 variables on 100 studies.

RESULTS

Most studies utilized data from a single institution and included information in clinical notes. Although chronic conditions were most commonly considered, ML also enabled the characterization of nuanced phenotypes such as social determinants of health. Supervised deep learning was the most popular ML paradigm, while semi-supervised and weakly supervised learning were applied to expedite algorithm development and unsupervised learning to facilitate phenotype discovery. ML approaches did not uniformly outperform rule-based algorithms, but deep learning offered a marginal improvement over traditional ML for many conditions.

DISCUSSION

Despite the progress in ML-based phenotyping, most articles focused on binary phenotypes and few articles evaluated external validity or used multi-institution data. Study settings were infrequently reported and analytic code was rarely released.

CONCLUSION

Continued research in ML-based phenotyping is warranted, with emphasis on characterizing nuanced phenotypes, establishing reporting and evaluation standards, and developing methods to accommodate misclassified phenotypes due to algorithm errors in downstream applications.

Contemporary Databases in Real-world Studies Regarding the Diverse Health Care Systems of India, Thailand, and Taiwan: Protocol for a Scoping Review

BACKGROUND

Real-world data (RWD) related to patient health status or health care delivery can be broadly defined as data collected outside of conventional clinical trials, including those from databases, treatment and disease registries, electronic medical records, insurance claims, and information directly contributed by health care professionals or patients. RWD are used to generate real-world evidence (RWE), which is increasingly relevant to policy makers in Asia, who use RWE to support decision-making in several areas, including public health policy, regulatory health technology assessment, and reimbursement; set priorities; or inform clinical practice.

OBJECTIVE

To support the achievement of the benefits of RWE in Asian health care strategies and policies, we sought to identify the linked contemporary databases used in real-world studies from three representative countries-India, Thailand, and Taiwan-and explore variations in results based on these countries' economies and health care reimbursement systems by performing a systematic scoping review. Herein, we describe the protocol and preliminary findings of our scoping review.

METHODS

The PubMed search strategy covered 3 concepts. Concept 1 was designed to identify potential RWE and RWD studies by applying various Medical Subject Headings (MeSH) terms ("Treatment Outcome," "Evidence-Based Medicine," "Retrospective Studies," and "Time Factors") and related keywords (eg, "real-world," "actual life," and "actual practice"). Concept 2 introduced the three countries-India, Taiwan, and Thailand. Concept 3 focused on data types, using a combination of MeSH terms ("Electronic Health Records," "Insurance, Health," "Registries," "Databases, Pharmaceutical," and "Pharmaceutical Services") and related keywords (eg, "electronic medical record," "electronic healthcare record," "EMR," "EHR," "administrative database," and "registry"). These searches were conducted with filters for language (English) and publication date (publications in the last 5 years before the search). The retrieved articles will undergo 2 screening phases (phase 1: review of titles and abstracts; phase 2: review of full texts) to identify relevant and eligible articles for data extraction. The data to be extracted from eligible studies will include the characteristics of databases, the regions covered, and the patient populations.

RESULTS

The literature search was conducted on September 27, 2022. We retrieved 3,172,434, 1,094,125, and 672,794 articles for concepts 1, 2, and 3, respectively. After applying all 3 concepts and the language and publication date filters, 2277 articles were identified. These will be further screened to identify eligible studies. Based on phase 1 screening and our progress to date, approximately 44% (1003/2277) of articles have undergone phase 2 screening to judge their eligibility. Around 800 studies will be used for data extraction.

CONCLUSIONS

Our research will be crucial for nurturing advancement in RWD generation within Asia by identifying linked clinical RWD databases and new avenues for public-private partnerships and multiple collaborations for expanding the scope and spectrum of high-quality, robust RWE generation in Asia.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/43741.

Performance of EHR classifiers for patient eligibility in a clinical trial of precision screening

BACKGROUND

Validated computable eligibility criteria use real-world data and facilitate the conduct of clinical trials. The Genomic Medicine at VA (GenoVA) Study is a pragmatic trial of polygenic risk score testing enrolling patients without known diagnoses of 6 common diseases: atrial fibrillation, coronary artery disease, type 2 diabetes, breast cancer, colorectal cancer, and prostate cancer. We describe the validation of computable disease classifiers as eligibility criteria and their performance in the first 16 months of trial enrollment.

METHODS

We identified well-performing published computable classifiers for the 6 target diseases and validated these in the target population using blinded physician review. If needed, classifiers were refined and then underwent a subsequent round of blinded review until true positive and true negative rates ≥80% were achieved. The optimized classifiers were then implemented as pre-screening exclusion criteria; telephone screens enabled an assessment of their real-world negative predictive value (NPV-RW).

RESULTS

Published classifiers for type 2 diabetes and breast and prostate cancer achieved desired performance in blinded chart review without modification; the classifier for atrial fibrillation required two rounds of refinement before achieving desired performance. Among the 1077 potential participants screened in the first 16 months of enrollment, NPV-RW of the classifiers ranged from 98.4% for coronary artery disease to 99.9% for colorectal cancer. Performance did not differ by gender or race/ethnicity.

CONCLUSIONS

Computable disease classifiers can serve as efficient and accurate pre-screening classifiers for clinical trials, although performance will depend on the trial objectives and diseases under study.

Leveraging electronic health record data for clinical trial planning by assessing eligibility criteria's impact on patient count and safety

OBJECTIVE

To present an approach on using electronic health record (EHR) data that assesses how different eligibility criteria, either individually or in combination, can impact patient count and safety (exemplified by all-cause hospitalization risk) and further assist with criteria selection for prospective clinical trials.

MATERIALS AND METHODS

Trials in three disease domains - relapsed/refractory (r/r) lymphoma/leukemia; hepatitis C virus (HCV); stages 3 and 4 chronic kidney disease (CKD) - were analyzed as case studies for this approach. For each disease domain, criteria were identified and all criteria combinations were used to create EHR cohorts. Per combination, two values were derived: (1) number of eligible patients meeting the selected criteria; (2) hospitalization risk, measured as the hazard ratio between those that qualified and those that did not. From these values, k-means clustering was applied to derive which criteria combinations maximized patient counts but minimized hospitalization risk.

RESULTS

Criteria combinations that reduced hospitalization risk without substantial reductions on patient counts were as follows: for r/r lymphoma/leukemia (23 trials; 9 criteria; 623 patients), applying no infection and adequate absolute neutrophil count while forgoing no prior malignancy; for HCV (15; 7; 751), applying no human immunodeficiency virus and no hepatocellular carcinoma while forgoing no decompensated liver disease/cirrhosis; for CKD (10; 9; 23893), applying no congestive heart failure.

CONCLUSIONS

Within each disease domain, the more drastic effects were generally driven by a few criteria. Similar criteria across different disease domains introduce different changes. Although results are contingent on the trial sample and the EHR data used, this approach demonstrates how EHR data can inform the impact on safety and available patients when exploring different criteria combinations for designing clinical trials.

BigMouth: development and maintenance of a successful dental data repository

Few clinical datasets exist in dentistry to conduct secondary research. Hence, a novel dental data repository called BigMouth was developed, which has grown to include 11 academic institutions contributing Electronic Health Record data on over 4.5 million patients. The primary purpose for BigMouth is to serve as a high-quality resource for rapidly conducting oral health-related research. BigMouth allows for assessing the oral health status of a diverse US patient population; provides rationale and evidence for new oral health care delivery modes; and embraces the specific oral health research education mission. A data governance framework that encouraged data sharing while controlling contributed data was initially developed. This transformed over time into a mature framework, including a fee schedule for data requests and allowing access to researchers from noncontributing institutions. Adoption of BigMouth helps to foster new collaborations between clinical, epidemiological, statistical, and informatics experts and provides an additional venue for professional development.

Use of Real-World Evidence to Drive Drug Development Strategy and Inform Clinical Trial Design

Interest in real-world data (RWD) and real-world evidence (RWE) to expedite and enrich the development of new biopharmaceutical products has proliferated in recent years, spurred by the 21st Century Cures Act in the United States and similar policy efforts in other countries, willingness by regulators to consider RWE in their decisions, demands from third-party payers, and growing concerns about the limitations of traditional clinical trials. Although much of the recent literature on RWE has focused on potential regulatory uses (e.g., product approvals in oncology or rare diseases based on single-arm trials with external control arms), this article reviews how biopharmaceutical companies can leverage RWE to inform internal decisions made throughout the product development process. Specifically, this article will review use of RWD to guide pipeline and portfolio strategy; use of novel sources of RWD to inform product development, use of RWD to inform clinical development, use of advanced analytics to harness "big" RWD, and considerations when using RWD to inform internal decisions. Topics discussed will include the use of molecular, clinicogenomic, medical imaging, radiomic, and patient-derived xenograft data to augment traditional sources of RWE, the use of RWD to inform clinical trial eligibility criteria, enrich trial population based on predicted response, select endpoints, estimate sample size, understand disease progression, and enhance diversity of participants, the growing use of data tokenization and advanced analytical techniques based on artificial intelligence in RWE, as well as the importance of data quality and methodological transparency in RWE.

A Road Map for Remote Digital Health Technology for Motor Neuron Disease

Despite recent and potent technological advances, the real-world implementation of remote digital health technology in the care and monitoring of patients with motor neuron disease has not yet been realized. Digital health technology may increase the accessibility to and personalization of care, whereas remote biosensors could optimize the collection of vital clinical parameters, irrespective of patients’ ability to visit the clinic. To facilitate the wide-scale adoption of digital health care technology and to align current initiatives, we outline a road map that will identify clinically relevant digital parameters; mediate the development of benefit-to-burden criteria for innovative technology; and direct the validation, harmonization, and adoption of digital health care technology in real-world settings. We define two key end products of the road map: (1) a set of reliable digital parameters to capture data collected under free-living conditions that reflect patient-centric measures and facilitate clinical decision making and (2) an integrated, open-source system that provides personalized feedback to patients, health care providers, clinical researchers, and caregivers and is linked to a flexible and adaptable platform that integrates patient data in real time. Given the ever-changing care needs of patients and the relentless progression rate of motor neuron disease, the adoption of digital health care technology will significantly benefit the delivery of care and accelerate the development of effective treatments.

Clinical comparison between trial participants and potentially eligible patients using electronic health record data: A generalizability assessment method

OBJECTIVE

To present a generalizability assessment method that compares baseline clinical characteristics of trial participants (TP) to potentially eligible (PE) patients as presented in their electronic health record (EHR) data while controlling for clinical setting and recruitment period.

METHODS

For each clinical trial, a clinical event was defined to identify patients of interest using available EHR data from one clinical setting during the trial's recruitment timeframe. The trial's eligibility criteria were then applied and patients were separated into two mutually exclusive groups: (1) TP, which were patients that participated in the trial per trial enrollment data; (2) PE, the remaining patients. The primary outcome was standardized differences in clinical characteristics between TP and PE per trial. A standardized difference was considered prominent if its absolute value was greater than or equal to 0.1. The secondary outcome was the difference in mean propensity scores (PS) between TP and PE per trial, in which the PS represented prediction for a patient to be in the trial. Three diverse trials were selected for illustration: one focused on hepatitis C virus (HCV) patients receiving a liver transplantation; one focused on leukemia patients and lymphoma patients; and one focused on appendicitis patients.

RESULTS

For the HCV trial, 43 TP and 83 PE were found, with 61 characteristics evaluated. Prominent differences were found among 69% of characteristics, with a mean PS difference of 0.13. For the leukemia/lymphoma trial, 23 TP and 23 PE were found, with 39 characteristics evaluated. Prominent differences were found among 82% of characteristics, with a mean PS difference of 0.76. For the appendicitis trial, 123 TP and 242 PE were found, with 52 characteristics evaluated. Prominent differences were found among 52% of characteristics, with a mean PS difference of 0.15.

CONCLUSIONS

Differences in clinical characteristics were observed between TP and PE among all three trials. In two of the three trials, not all of the differences necessarily compromised trial generalizability and subsets of PE could be considered similar to their corresponding TP. In the remaining trial, lack of generalizability appeared present, but may be a result of other factors such as small sample size or site recruitment strategy. These inconsistent findings suggest eligibility criteria alone are sometimes insufficient in defining a target group to generalize to. With caveats in limited scalability, EHR data quality, and lack of patient perspective on trial participation, this generalizability assessment method that incorporates control for temporality and clinical setting promise to better pinpoint clinical patterns and trial considerations.

Comparison of Clinical Characteristics Between Clinical Trial Participants and Nonparticipants Using Electronic Health Record Data

IMPORTANCE

Assessing generalizability of clinical trials is important to ensure appropriate application of interventions, but most assessments provide minimal granularity on comparisons of clinical characteristics.

OBJECTIVE

To assess the extent of underlying clinical differences between clinical trial participants and nonparticipants by using a combination of electronic health record and trial enrollment data.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study used data obtained from a single academic medical center between September 1996 and January 2019 to identify 1645 clinical trial participants from a diverse set of 202 available trials conducted at the center. Using an aggregated resampling procedure, nonparticipants were matched to participants 1:1 based on trial conditions, number of recent visits to a health care professional, and calendar time.

EXPOSURES

Clinical trial enrollment vs no enrollment.

MAIN OUTCOMES AND MEASURES

The primary outcome was standardized differences in clinical characteristics between participants and nonparticipants in clinical trials stratified into the 4 most common disease domains.

RESULTS

This cross-sectional study included 1645 participants from 202 trials (929 [56.5%] male; mean [SD] age, 54.65 [21.38] years) and an aggregated set of 1645 nonparticipants (855 [52.0%] male; mean [SD] age, 57.24 [21.91] years). The most common disease domains for the selected trials were neoplastic disease (86 trials; 737 participants), disorders of the digestive system (31 trials; 321 participants), inflammatory disorders (28 trials; 276 participants), and disorders of the cardiovascular system (27 trials; 319 participants); trials could qualify for multiple disease domains. Among 31 conditions, the percentage of conditions for which the prevalence was lower among participants than among nonparticipants per standardized differences was 64.5% (20 conditions) for neoplastic disease trials, 61.3% (19) for digestive system trials, 58.1% (18) for inflammatory disorder trials, and 38.7% (12) for cardiovascular system trials. Among 17 medications, the percentage of medications for which use was less among participants than among nonparticipants per standardized differences was 64.7% (11) for neoplastic disease trials, 58.8% (10) for digestive system trials, 88.2% (15) for inflammatory disorder trials, and 52.9% (9) for cardiovascular system trials.

CONCLUSIONS AND RELEVANCE

Using a combination of electronic health record and trial enrollment data, this study found that clinical trial participants had fewer comorbidities and less use of medication than nonparticipants across a variety of disease domains. Combining trial enrollment data with electronic health record data may be useful for better understanding of the generalizability of trial results.