Clinical Trial Generalizability Assessment in the Big Data Era: A Review

Utilization of geospatial distribution in the measurement of study cohort representativeness

OBJECTIVE

The ability to apply results from a study to a broader population remains a primary objective in translational science. Distinct from intrinsic elements of scientific rigor, the extrinsic concept of generalization requires there be alignment between a study cohort and population in which results are expected to be applied. Widespread efforts have been made to quantify representativeness of a study cohorts. These techniques, however, often consider the study and target cohorts as monolithic collections that can be directly compared. Overlooking known impacts to health from socio-demographic and environmental factors tied to individual's geographical location, and potentially obfuscating misalignment in underrepresented population subgroups. This manuscript introduces several measures to account for geographic information in the measurement of cohort representation.

METHODS

Metrics were defined across two themes. First, measures of recruitment, to assess a study cohort is drawn at an expected rate and in an expected geographical pattern with respect to individuals in a reference cohort. Quantifying the coverage and spread across the distinct geographic regions comprising the target population. Second, measures of individual characteristics, to assess if the study cohort accurately reflects the sociodemographic, clinical, and geographic diversity observed across a reference cohort. Employing intra-individual measures of distance and aggregate measures of alignment designed to account for geospatial proximity of individuals.

RESULTS

As an empirical demonstration, methods are applied to an active clinical study examining asthma in Black and African American patients at a US Midwestern pediatric hospital. Results illustrate how areas of over- and under-recruitment can be identified and contextualized in light of study recruitment patterns. At an individual-level, highlighting the ability to identify a subset of features for which the study cohort closely resembled the broader population. In addition to an opportunity to dive deeper into misalignments, to identify study cohort members that are in some way distinct from the communities for which they are expected to represent.

CONCLUSION

Together, these metrics provide a comprehensive spatial assessment of a study cohort with respect to a broader target population. Such an approach offers researchers a toolset by which to target expected generalization of results derived from a given study.

Comparison of Demographics: National Amyotrophic Lateral Sclerosis Registry and Clinical Trials Data

OBJECTIVE

To characterize the participant demographics in the Pooled Resource Open-Access ALS Clinical Trials (PRO-ACT) database compared with the web-portal National Amyotrophic Lateral Sclerosis (ALS) Registry (the Registry).

METHODS

Demographics and ALS symptom information were compared between the self-reported registrant data in the Registry web portal (2010-2021) and the latest available PRO-ACT data (updated August 2022), which is a collection of clinical trials data.

RESULTS

Greater percentages of younger (≤ 59 years old) but smaller percentages of older (60 + years old) participants were represented in PRO-ACT compared to Registry. Enrollment for minority race groups was greater in the Registry portal data, but race information was largely missing/unknown in PRO-ACT database. Median age at the time of diagnosis and age at the time of symptom onset were significantly higher for Registry enrollees compared to the participants of PRO-ACT. Symptom onset sites were similarly reported, but duration between self-noted symptom onset and diagnosis was slight, but significantly longer for the Registry enrollees (11 vs. 9 months). Hispanic were as likely as non-Hispanic to participate in research studies, based on the Registry data.

CONCLUSION

There was a notable difference in the age distribution and minority representation of enrollees between the PRO-ACT and Registry study populations. Age distribution in the PRO-ACT database skewed to a younger and less diverse cohort. Despite the clinical heterogeneity and complex disease mechanism of ALS, identifying the underrepresented demographic niche in the PRO-ACT and Registry study populations can help improve patient participation and criteria for patient selection to enhance generalizability.

Hasty generalizations and generics in medical research: A systematic review

It is unknown to what extent medical researchers generalize study findings beyond their samples when their sample size, sample diversity, or knowledge of conditions that support external validity do not warrant it. It is also unknown to what extent medical researchers describe their results with precise quantifications or unquantified generalizations, i.e., generics, that can obscure variations between individuals. We therefore systematically reviewed all prospective studies (n = 533) published in the top four highest ranking medical journals, Lancet, New England Journal of Medicine (NEJM), Journal of the American Medical Association (JAMA), and the British Medical Journal (BMJ), from January 2022 to May 2023. We additionally reviewed all NEJM Journal Watch clinical research summaries (n = 143) published during the same time. Of all research articles reporting prospective studies, 52.5% included generalizations beyond specific national study populations, with the numbers of articles with generics varying significantly between journals (JAMA = 12%; Lancet = 77%) (p < 0.001, V = 0.48). There was no evidence that articles containing broader generalizations or generics were correlated with larger or more nationally diverse samples. Moreover, only 10.2% of articles with generalizations beyond specific national populations reported external validity strengthening factors that could potentially support such extrapolations. There was no evidence that original research articles and NEJM Journal Watch summaries intended for practitioners differed in their use of broad generalizations, including generics. Finally, from the journal with the highest citation impact, articles containing broader conclusions were correlated with more citations. Since there was no evidence that studies with generalizations beyond specific national study populations or with generics were associated with larger, more nationally diverse samples, or with reports of population similarity that may permit extensions of conclusions, our findings suggest that the generalizations in many articles were insufficiently supported. Caution against overly broad generalizations in medical research is warranted.

Incidence and determinants of seizures in multiple sclerosis: a meta-analysis of randomised clinical trials

BACKGROUND

Seizures are reported to be more prevalent in individuals with multiple sclerosis (MS) compared with the general population. Existing data predominantly originate from population-based studies, which introduce variability in methodologies and are vulnerable to selection and reporting biases.

METHODS

This meta-analysis aims to assess the incidence of seizures in patients participating in randomised clinical trials and to identify potential contributing factors. Data were extracted from 60 articles published from 1993 to 2022. The pooled effect size, representing the incidence rate of seizure events, was estimated using a random-effect model. Metaregression was employed to explore factors influencing the pooled effect size.

RESULTS

The meta-analysis included data from 53 535 patients and 120 seizure events in a median follow-up of 2 years. The pooled incidence rate of seizures was 68.0 per 100 000 patient-years, significantly higher than the general population rate of 34.6. Generalised tonic-clonic seizures were the most common type reported, although there was a high risk of misclassification for focal seizures with secondary generalisation. Disease progression, longer disease duration, higher disability levels and lower brain volume were associated with a higher incidence of seizures. Particularly, sphingosine-1-phosphate receptor (S1PR) modulators exhibited a 2.45-fold increased risk of seizures compared with placebo or comparators, with a risk difference of 20.5 events per 100 000 patient-years.

CONCLUSIONS

Patients with MS face a nearly twofold higher seizure risk compared with the general population. This risk appears to be associated not only with disease burden but also with S1PR modulators. Our findings underscore epilepsy as a significant comorbidity in MS and emphasise the necessity for further research into its triggers, preventive measures and treatment strategies.

Long-term follow-up of outcomes including progression-free survival 2 in patients with transplant-ineligible multiple myeloma in the real-world practice: A multi-institutional report from the Canadian Myeloma Research Group (CMRG) database

Multiple myeloma remains an incurable cancer mostly affecting older adults and is characterized by a series of remission inductions and relapses. This study aims to evaluate the outcomes in newly diagnosed transplant-ineligible patients using bortezomib/lenalidomide-based regimens in the Canadian real world as well as their outcomes in the second line. The Canadian Myeloma Research Group Database (CMRG-DB) is a national database with input from multiple Canadian Centres with now up to 8000 patients entered. A total of 1980 transplant ineligible patients were identified in the CMRG-DB between the years of 2007-2021. The four most commonly used induction regimens are bortezomib/melphalan/prednisone (VMP) (23%), cyclophosphamide/bortezomib/dexamethasone (CyBorD) (47%), lenalidomide/dexamethasone (Rd) (24%), and bortezomib/lenalidomide/dexamethasone (VRd) (6%). After a median follow-up of 30.46 months (0.89-168.42), the median progression-free survival (mPFS) and median overall survival (mOS) of each cohort are 23.5, 22.9, 34.0 months, and not reached (NR) and 64.1, 51.1, 61.5 months, and NR respectively. At the time of data cut-off, 1128 patients had gone on to second-line therapy. The mPFS2 based on first-line therapy, VMP, CyBorD, Rd, and VRd is 53.3, 48.4, 62.7 months, and NR respectively. The most common second-line regimens are Rd (47.4%), DRd (12.9%), CyBorD (10.3%), and RVd (8.9%) with a mPFS and a mOS of 17.0, 31.1, 15.4, and 14.0 months and 34.7, NR, 47.6, 33.4 months, respectively. This study represents the real-world outcomes in newly diagnosed transplant-ineligible myeloma patients in Canada. The spectra of therapy presented here reflect the regimens still widely used around the world. While this is sure to change with anti-CD38 monoclonal antibodies now reflecting a new standard of care in frontline therapy, this cohort is reflective of the type of multiple myeloma patient currently experiencing relapse in the real-world setting.

Generalizability of Randomized Clinical Trial Outcomes for Diabetes Control Resulting From Bariatric Surgery

Objective

To assess the external validity of randomized controlled trials (RCTs) of bariatric surgical treatment on diabetes control.

Background

Multisite RCTs provide the strongest evidence supporting clinical treatments and have the greatest internal validity. However, characteristics of trial participants may not be representative of patients receiving treatment in the real world. There is a need to assess how the results of RCTs generalize to all contemporary patient populations undergoing treatments.

Methods

All patients undergoing sleeve gastrectomy at University of California Los Angeles (UCLA) between January 8, 2018 and May 19, 2023 had their baseline characteristics, weight change, and diabetes control compared with those enrolled in the surgical treatment and medications potentially eradicate diabetes efficiently (STAMPEDE) and diabetes surgery study (DSS) RCTs of bariatric surgery's effect on diabetes control. Weight loss and diabetes control were compared between UCLA patients who did and did not fit the entry criteria for these RCTs.

Results

Only 65 (17%) of 387 patients with diabetes fulfilled the eligibility criteria for STAMPEDE, and 29 (7.5%) fulfilled the criteria for DSS due to being older, having higher body mass index, and lower HbA1c. UCLA patients experienced slightly less weight loss than patients in the RCTs but had similar diabetes control. The 313 (81%) patients not eligible for study entry into either RCT had similar long-term diabetes control as those who were eligible for the RCTs.

Conclusions

Even though only a very small proportion of patients undergoing bariatric surgery met the eligibility criteria for the 2 major RCTs, most patients in this contemporary cohort had similar outcomes. Diabetes outcomes from STAMPEDE and DSS generalize to most patients undergoing bariatric surgery for diabetes control.

The Logic of Generalization From Systematic Reviews and Meta-Analyses of Impact Evaluations

Systematic reviews and meta-analyses are viewed as potent tools for generalized causal inference. These reviews are routinely used to inform decision makers about expected effects of interventions. However, the logic of generalization from research reviews to diverse policy and practice contexts is not well developed. Building on sampling theory, concerns about epistemic uncertainty, and principles of generalized causal inference, this article presents a pragmatic approach to generalizability assessment for use with systematic reviews and meta-analyses. This approach is applied to two systematic reviews and meta-analyses of effects of "evidence-based" psychosocial interventions for youth and families. Evaluations included in systematic reviews are not necessarily representative of populations and treatments of interest. Generalizability of results is limited by high risks of bias, uncertain estimates, and insufficient descriptive data from impact evaluations. Systematic reviews and meta-analyses can be used to test generalizability claims, explore heterogeneity, and identify potential moderators of effects. These reviews can also produce pooled estimates that are not representative of any larger sets of studies, programs, or people. Further work is needed to improve the conduct and reporting of impact evaluations and systematic reviews, and to develop practical approaches to generalizability assessment and guide applications of interventions in diverse policy and practice contexts.

Impact of birth weight to placental weight ratio and other perinatal risk factors on left ventricular dimensions in newborns: a prospective cohort analysis

OBJECTIVES

To investigate the association between birth weight to placental weight (BW/PW) ratio, and echocardiographic left ventricle (LV) morphology at birth, while accounting for other relevant perinatal factors.

METHODS

A prospective cohort study was conducted on neonates at NewYork-Presbyterian Brooklyn Methodist Hospital from 2014 to 2018, categorized by their BW/PW percentile. Missing data were imputed with principal component analysis. Chi-squared and one-way analysis of variance were used to compare BW/PW groups and the best regression model was selected using a genetic and backward stepwise algorithm.

RESULTS

We analyzed 827 neonates in three BW/PW groups: small (n=16), normal (n=488), and large (n=323). Placental thickness and smallest diameter were positively correlated with several LV parameters, including inter-ventricular septal thickness during diastole (IVSd) (p=0.002, p<0.001) and systole (IVSs) (p=0.001, p<0.001), LV posterior wall thickness at end of diastole (LVPWd) (p=0.003, p<0.001) and systole (LVPWs) (p<0.001, p<0.001), LV mass (p=0.017, p<0.001), and LV mass/volume (p=0.011, p<0.001). The BW/PW ratio correlated with an increased shortening fraction (estimate=0.29, 95 % CI 0.03-0.55, p=0.027). PW correlated with IVSs (p=0.019), while the longest placental diameter was linked to a decrease in LV internal dimension during diastole (LVIDd) (estimate=-0.07, p=0.039), LV mass (estimate=-0.11, p=0.024), and LV mass/volume (estimate=-0.55, p=0.005).

CONCLUSIONS

This study found that several placental factors, including the BW/PW ratio, can independently affect LV dimension and morphology, highlighting the importance of fetal growth and placental health in the physiological adaptation of the fetal heart. More research is needed to establish causation and inform newborn prevention strategies.

Quantitative evaluation of the impact of relaxing eligibility criteria on the risk-benefit profile of drugs for lung cancer based on real-world data

INTRODUCTION

Restrictive eligibility criteria in cancer drug trials result in low enrollment rates and limited population diversity. Relaxed eligibility criteria (REC) based on solid evidence is becoming necessary for stakeholders worldwide. However, the absence of high-quality, favorable evidence remains a major challenge. This study presents a protocol to quantitatively evaluate the impact of relaxing eligibility criteria in common non-small cell lung cancer (NSCLC) protocols in China, on the risk-benefit profile. This involves a detailed explanation of the rationale, framework, and design of REC.

METHODS

To evaluate our REC in NSCLC drug trials, we will first construct a structured, cross-dimensional real-world NSCLC database using deep learning methods. We will then establish randomized virtual cohorts and perform benefit-risk assessment using Monte Carlo simulation and propensity matching. Shapley value will be utilized to quantitatively measure the effect of the change of each eligibility criterion on patient volume, clinical efficacy and safety.

DISCUSSION

This study is one of the few that focuses on the problem of overly stringent eligibility criteria cancer drug clinical trials, providing quantitative evaluation of the effect of relaxing each NSCLC eligibility criterion. This study will not only provide scientific evidence for the rational design of population inclusion in lung cancer clinical trials, but also establish a data governance system, as well as a REC evaluation framework that can be generalized to other cancer studies.

Population Neuroscience: Understanding Concepts of Generalizability and Transportability and Their Application to Improving the Public's Health

In population neuroscience, samples are not often selected with equal or known probability from an underlying population of interest; in other words, samples are not often formally representative of a specified underlying population. This chapter provides an overview of an epidemiological approach to considering the implications of selective participation on the value of our results for population health. We discuss definitions of generalizability and transportability, given the growing recognition that generalizability and transportability are central for interpreting data that are aiming to be population-based. We provide evidence that differences in the prevalence of effect measure modifiers between a study sample and a target population will lead to a lack of generalizability and transportability. We provide an example of an association between a poly-genetic risk score and depression, showing how an internally valid association can differ based on the prevalence of effect measure modifiers. We show that when estimating associations, inferences from a study sample to a population can depend on clearly defining a target population. Given that representative sampling from explicitly defined target populations may not be feasible or realistic in many situations, especially given the sample sizes needed for statistical power for many exposures of interest (and especially when interactions are being tested), researchers should be well versed in tools available to enhance the interpretability of samples regarding target populations.

Representation of Real-World Adults With Chronic Kidney Disease in Clinical Trials Supporting Blood Pressure Treatment Targets

BACKGROUND

Little is known about how well trial participants with chronic kidney disease (CKD) represent real-world adults with CKD. We assessed the population representativeness of clinical trials supporting the 2021 Kidney Disease: Improving Global Outcomes blood pressure (BP) guidelines in real-world adults with CKD.

METHODS AND RESULTS

Using a cross-sectional analysis, we identified patients with CKD who met the guideline definition of hypertension based on use of antihypertensive medications or sustained systolic BP ≥120 mm Hg in 2019 in the Veterans Affairs and Kaiser Permanente of Southern California. We applied the eligibility criteria from 3 BP target trials, SPRINT (Systolic Pressure Intervention Trial), ACCORD (Action to Control Cardiovascular Risk in Diabetes), and AASK (African American Study of Kidney Disease), to estimate the proportion of adults with a systolic BP above the guideline-recommended target and the proportion who met eligibility criteria for ≥1 trial. We identified 503 480 adults in the Veterans Affairs and 73 412 adults in Kaiser Permanente of Southern California with CKD and hypertension in 2019. We estimated 79.7% in the Veterans Affairs and 87.3% in the Kaiser Permanente of Southern California populations had a systolic BP ≥120 mm Hg; only 23.8% [23.7%-24.0%] in the Veterans Affairs and 20.8% [20.5%-21.1%] in Kaiser Permanente of Southern California were trial-eligible. Among trial-ineligible patients, >50% met >1 exclusion criteria.

CONCLUSIONS

Major BP target trials were representative of fewer than 1 in 4 real-world adults with CKD and hypertension. A large proportion of adults who are at risk for cardiovascular morbidity from hypertension and susceptible to adverse treatment effects lack relevant treatment information.

Longitudinal Effects of Elexacaftor/Tezacaftor/Ivacaftor: Multidimensional Assessment of Neuropsychological Side Effects and Physical and Mental Health Outcomes in Adolescents and Adults

BACKGROUND

Italy initiated elexacaftor/tezacaftor/ivacaftor (ETI) for people with cystic fibrosis (pwCF) in July 2021. It has led to dramatic improvements in lung function, BMI, sweat chloride, and respiratory symptoms. However, few data are available on side effects or effects on a broad range of outcomes.

RESEARCH QUESTION

How does ETI affect mental health, cognitive processing, neuropsychological side effects, GI symptoms, and health-related quality of life over time?

STUDY DESIGN AND METHODS

This was a prospective, "real-world" longitudinal study. Participants were recruited consecutively and evaluated at initiation (T0) and after 1 month, 3 months, and 6 months of starting treatment. Assessments included depression (nine-item Patient Health Questionnaire), anxiety (seven-item Generalized Anxiety Disorder), cognition (Symbol Digit Modalities Test), GI Symptom Tracker, and health-related quality of life (Cystic Fibrosis Questionnaire-Revised). Based on literature, an ad hoc questionnaire was developed to assess side effects: insomnia, headache, memory problems, "brain fog," and concentration problems. Following descriptive analyses, longitudinal data were analyzed by using mixed models for repeated measures, controlling for age and sex when appropriate.

RESULTS

Ninety-two consecutive pwCF (female/male, 46/46; mean age, 25.4 years) participated. FEV1 increased initially and then remained stable. BMI also increased significantly from T0 to 6 months (P < .01). Depression improved from T0 to 1 month (P < .001); however, no changes in anxiety were found. Cognitive processing improved from T0 to subsequent assessments. Positive changes were reported on the GI Symptom Tracker for stools and adherence challenges, although no changes were found for abdominal pain and digestion. Side effects occurred in 10% to 29%, with no reduction over time; insomnia increased significantly across time. Female participants reported more side effects than male participants (ie, insomnia, headache, concentration problems, brain fog).

INTERPRETATION

This prospective study evaluated the effects of ETI using multiple measures. Significant improvements were found in many domains; however, side effects were reported by a substantial proportion of pwCF, with no improvements over time. Female participants reported more side effects than male participants. pwCF should be followed up systematically to assess the frequency of side effects after starting this new modulator.

A Novel Digital Twin Strategy to Examine the Implications of Randomized Control Trials for Real-World Populations

Randomized clinical trials (RCTs) are essential to guide medical practice; however, their generalizability to a given population is often uncertain. We developed a statistically informed Generative Adversarial Network (GAN) model, RCT-Twin-GAN, that leverages relationships between covariates and outcomes and generates a digital twin of an RCT (RCT-Twin) conditioned on covariate distributions from a second patient population. We used RCT-Twin-GAN to reproduce treatment effect outcomes of the Systolic Blood Pressure Intervention Trial (SPRINT) and the Action to Control Cardiovascular Risk in Diabetes (ACCORD) Blood Pressure Trial, which tested the same intervention but had different treatment effect results. To demonstrate treatment effect estimates of each RCT conditioned on the other RCT patient population, we evaluated the cardiovascular event-free survival of SPRINT digital twins conditioned on the ACCORD cohort and vice versa (SPRINT-conditioned ACCORD twins). The conditioned digital twins were balanced by the intervention arm (mean absolute standardized mean difference (MASMD) of covariates between treatment arms 0.019 (SD 0.018), and the conditioned covariates of the SPRINT-Twin on ACCORD were more similar to ACCORD than a sprint (MASMD 0.0082 SD 0.016 vs. 0.46 SD 0.20). Most importantly, across iterations, SPRINT conditioned ACCORD-Twin datasets reproduced the overall non-significant effect size seen in ACCORD (5-year cardiovascular outcome hazard ratio (95% confidence interval) of 0.88 (0.73-1.06) in ACCORD vs median 0.87 (0.68-1.13) in the SPRINT conditioned ACCORD-Twin), while the ACCORD conditioned SPRINT-Twins reproduced the significant effect size seen in SPRINT (0.75 (0.64-0.89) vs median 0.79 (0.72-0.86)) in ACCORD conditioned SPRINT-Twin). Finally, we describe the translation of this approach to real-world populations by conditioning the trials on an electronic health record population. Therefore, RCT-Twin-GAN simulates the direct translation of RCT-derived treatment effects across various patient populations with varying covariate distributions.

A scoping review of fair machine learning techniques when using real-world data

OBJECTIVE

The integration of artificial intelligence (AI) and machine learning (ML) in health care to aid clinical decisions is widespread. However, as AI and ML take important roles in health care, there are concerns about AI and ML associated fairness and bias. That is, an AI tool may have a disparate impact, with its benefits and drawbacks unevenly distributed across societal strata and subpopulations, potentially exacerbating existing health inequities. Thus, the objectives of this scoping review were to summarize existing literature and identify gaps in the topic of tackling algorithmic bias and optimizing fairness in AI/ML models using real-world data (RWD) in health care domains.

METHODS

We conducted a thorough review of techniques for assessing and optimizing AI/ML model fairness in health care when using RWD in health care domains. The focus lies on appraising different quantification metrics for accessing fairness, publicly accessible datasets for ML fairness research, and bias mitigation approaches.

RESULTS

We identified 11 papers that are focused on optimizing model fairness in health care applications. The current research on mitigating bias issues in RWD is limited, both in terms of disease variety and health care applications, as well as the accessibility of public datasets for ML fairness research. Existing studies often indicate positive outcomes when using pre-processing techniques to address algorithmic bias. There remain unresolved questions within the field that require further research, which includes pinpointing the root causes of bias in ML models, broadening fairness research in AI/ML with the use of RWD and exploring its implications in healthcare settings, and evaluating and addressing bias in multi-modal data.

CONCLUSION

This paper provides useful reference material and insights to researchers regarding AI/ML fairness in real-world health care data and reveals the gaps in the field. Fair AI/ML in health care is a burgeoning field that requires a heightened research focus to cover diverse applications and different types of RWD.

Overcoming combination fatigue: Addressing high-dimensional effect measure modification and interaction in clinical, biomedical, and epidemiologic research using multilevel analysis of individual heterogeneity and discriminatory accuracy (MAIHDA)

Growing interest in precision medicine, gene-environment interactions, health equity, expanding diversity in research, and the generalizability results, requires researchers to evaluate how the effects of treatments or exposures differ across numerous subgroups. Evaluating combination complexity, in the form of effect measure modification and interaction, is therefore a common study aim in the biomedical, clinical, and epidemiologic sciences. There is also substantial interest in expanding the combinations of factors analyzed to include complex treatment protocols (e.g., multiple study arms or factorial randomization), comorbid medical conditions or risk factors, and sociodemographic and other subgroup identifiers. However, expanding the number of subgroup category combinations creates combination fatigue problems, including concerns over small sample size, reduced power, multiple testing, spurious results, and design and analytic complexity. Creative new approaches for managing combination fatigue and evaluating high-dimensional effect measure modification and interaction are needed. Intersectional MAIHDA (multilevel analysis of individual heterogeneity and discriminatory accuracy) has already attracted substantial interest in social epidemiology, and has been hailed as the new gold standard for investigating health inequities across complex intersections of social identity. Leveraging the inherent advantages of multilevel models, a more general multicategorical MAIHDA can be used to study statistical interactions and predict effects across high-dimensional combinations of conditions, with important advantages over alternative approaches. Though it has primarily been used thus far as an analytic approach, MAIHDA should also be used as a framework for study design. In this article, I introduce MAIHDA to the broader health sciences research community, discuss its advantages over conventional approaches, and provide an overview of potential applications in clinical, biomedical, and epidemiologic research.

Parsing Clinical Trial Eligibility Criteria for Cohort Query by a Multi-Input Multi-Output Sequence Labeling Model

To enable electronic screening of eligible patients for clinical trials, free-text clinical trial eligibility criteria should be translated to a computable format. Natural language processing (NLP) techniques have the potential to automate this process. In this study, we explored a supervised multi-input multi-output (MIMO) sequence labelling model to parse eligibility criteria into combinations of fact and condition tuples. Our experiments on a small manually annotated training dataset showed that that the performance of the MIMO framework with a BERT-based encoder using all the input sequences achieved an overall lenient-level AUROC of 0.61. Although the performance is suboptimal, representing eligibility criteria into logical and semantically clear tuples can potentially make subsequent translation of these tuples into database queries more reliable.

Early surgery versus conservative treatment in patients with traumatic intracerebral hematoma: a CENTER-TBI study

PURPOSE

Evidence regarding the effect of surgery in traumatic intracerebral hematoma (t-ICH) is limited and relies on the STITCH(Trauma) trial. This study is aimed at comparing the effectiveness of early surgery to conservative treatment in patients with a t-ICH.

METHODS

In a prospective cohort, we included patients with a large t-ICH (< 48 h of injury). Primary outcome was the Glasgow Outcome Scale Extended (GOSE) at 6 months, analyzed with multivariable proportional odds logistic regression. Subgroups included injury severity and isolated vs. non-isolated t-ICH.

RESULTS

A total of 367 patients with a large t-ICH were included, of whom 160 received early surgery and 207 received conservative treatment. Patients receiving early surgery were younger (median age 54 vs. 58 years) and more severely injured (median Glasgow Coma Scale 7 vs. 10) compared to those treated conservatively. In the overall cohort, early surgery was not associated with better functional outcome (adjusted odds ratio (AOR) 1.1, (95% CI, 0.6-1.7)) compared to conservative treatment. Early surgery was associated with better outcome for patients with moderate TBI and isolated t-ICH (AOR 1.5 (95% CI, 1.1-2.0); P value for interaction 0.71, and AOR 1.8 (95% CI, 1.3-2.5); P value for interaction 0.004). Conversely, in mild TBI and those with a smaller t-ICH (< 33 cc), conservative treatment was associated with better outcome (AOR 0.6 (95% CI, 0.4-0.9); P value for interaction 0.71, and AOR 0.8 (95% CI, 0.5-1.0); P value for interaction 0.32).

CONCLUSIONS

Early surgery in t-ICH might benefit those with moderate TBI and isolated t-ICH, comparable with results of the STITCH(Trauma) trial.

Results from a global multicenter registry of 6193 patients to refine endoscopic anatomical enucleation of the prostate (REAP) by evaluating trends and outcomes and nuances of prostate enucleation in a real-world setting

PURPOSE

To collect a multicentric, global database to assess current preferences and outcomes for endoscopic enucleation of the prostate (EEP).

METHODS

Endourologists experienced in EEP from across the globe were invited to participate in the creation of this retrospective registry. Surgical procedures were performed between January 2020 and August 2022.

INCLUSION CRITERIA

lower urinary tract symptoms not responding to or worsening despite medical therapy and absolute indication for surgery.

EXCLUSION CRITERIA

prostate cancer, concomitant lower urinary tract surgery, previous prostate/urethral surgery, pelvic radiotherapy.

RESULTS

Ten centers from 7 countries, involving 13 surgeons enrolled 6193 patients. Median age was 68 [62-74] years. 2326 (37.8%) patients had large prostates (> 80 cc). The most popular energy modality was the Holmium laser. The most common technique used for enucleation was the 2-lobe (48.8%). 86.2% of the procedures were performed under spinal anesthesia. Median operation time was 67 [50-95] minutes. Median postoperative catheter time was 2 [1, 3] days. Urinary tract infections were the most reported complications (4.7%) followed by acute urinary retention (4.1%). Post-operative bleeding needing additional intervention was reported in 0.9% of cases. 3 and 12-month follow-up visits showed improvement in symptoms and micturition parameters. Only 8 patients (1.4%) required redo surgery for residual adenoma. Stress urinary incontinence was reported in 53.9% of patients and after 3 months was found to persist in 16.2% of the cohort.

CONCLUSION

Our database contributes real-world data to support EEP as a truly well-established global, safe minimally invasive intervention and provides insights for further research.

Are we enrolling the right patients? A scoping review of external validity and generalizability of clinical trials in bloodstream infections

BACKGROUND

Having a representative population in randomized clinical trials (RCTs) improves external validity and generalizability of trial results. There are limited data examining differences between RCT-enrolled and real-world populations in bloodstream infections (BSI).

OBJECTIVES

We conducted a scoping review aiming to review studies assessing generalizability of BSI RCT populations, to identify sub-groups that have been systematically under-represented and to explore approaches to improve external validity of future RCTs.

SOURCES

MEDLINE, Embase, and Cochrane Library databases were searched for terms related to external validity or generalizability, BSI, and clinical trials in papers published up to 1 August 2023. Studies comparing enrolled versus nonenrolled patients, or papers discussing external validity or generalizability in the context of BSI RCTs were included.

CONTENT

Sixteen papers were included in the final review. Five compared RCT-enrolled and nonenrolled participants from the same source population. There were significant differences between the two groups in all studies, with nonenrolled patients having a greater comorbidity burden and consistently worse outcomes including mortality. We identified several barriers to improving generalizability of RCT populations and outlined potential approaches to reduce these barriers, such as alternative/simplified consent processes, streamlining eligibility criteria and follow-up procedures, quota-based sampling techniques, and ensuring diversity in site and study team selection.

IMPLICATIONS

Study cohorts in BSI RCTs are not representative of the general BSI patient population. As we increasingly adopt large pragmatic trials in infectious diseases, it is important to recognize the importance of maximizing generalizability to ensure that our research findings are of direct relevance to our patients.

Predicting risk factors for pediatric mortality in clinical trial research: A retrospective, cross-sectional study using a Healthcare Cost and Utilization Project database

Introduction

Incorporating real-world data using "big data" analysis in healthcare are useful to extract specific information for healthcare delivery system improvement. All-cause mortality is an essential measure to enhance patient safety in clinical trial research, especially for underrepresented pediatric participants.

Objective

This study aimed to determine the associations between pediatric mortality and patient-specific factors using the Healthcare Cost and Utilization Project (HCUP) database.

Methods

Data from the 2019 the HCUP Kids' Inpatient Database (KID) were used to conduct a logistic regression analysis to determine associations between pediatric patients' the chance of survival and their demographic and socioeconomic background, discharge records, and hospital information.

Results

Total number of diagnoses (OR = 0.84), total number of procedures (OR = 0.86), length of stay (OR = 1.04), age intervals greater than 1 year (OR > 1.0), transfer into the hospital from a different acute care (OR = 0.34), major diagnoses of multiple significant trauma (OR = 0.03) or hepatobiliary system and pancreas (OR = 0.10), region of hospital - west and midwest (OR > 1.0), and medium or larger hospital bed size (OR > 1.0) were all significantly associated with the chance of survival for patients participating in pediatric clinical trials (p < 0.05).

Conclusion

Real-world clinical trial data analysis showed the potential improvement area including reallocating trial resources to promote trial quality and safe participation for pediatric patients. Pediatric trials need tools that are developed using user-centered design approaches to satisfy the unique needs and requirements of pediatric patients and their caregivers. Safe intrahospital transfer procedures and active dissemination of successful trial best practices are crucial to trial management, adherence, quality, and safety.

Unraveling the Predictive Potential of Rapid Scoring in Pleural Infection: A Critical Review

Pleural infection, or pleural empyema, is a severe medical condition associated with high morbidity and mortality rates. Timely and accurate prognostication is crucial for optimizing patient outcomes and resource allocation. Rapid scoring systems have emerged as promising tools in pleural infection prognostication, integrating various clinical and laboratory parameters to assess disease severity and quantitatively predict short-term and long-term outcomes. This review article critically evaluates existing rapid scoring systems, including CURB-65 (confusion, uremia, respiratory rate, blood pressure, age ≥ 65 years), A-DROP (age (male >70 years, female >75 years), dehydration, respiratory failure, orientation disturbance, and low blood pressure), and APACHE II (acute physiology and chronic health evaluation II), assessing their predictive accuracy and limitations. Our analysis highlights the potential clinical implications of rapid scoring, including risk stratification, treatment tailoring, and follow-up planning. We discuss practical considerations and challenges in implementing rapid scoring such as data accessibility and potential sources of bias. Furthermore, we emphasize the importance of validation, transparency, and multidisciplinary collaboration to refine and enhance the clinical applicability of these scoring systems. The prospects for rapid scoring in pleural infection management are promising, with ongoing research and data science advances offering improvement opportunities. Ultimately, the successful integration of rapid scoring into clinical practice can potentially improve patient care and outcomes in pleural infection management.

Perinatal factors impacting echocardiographic left ventricular measurement in small for gestational age infants: a prospective cohort study

INTRODUCTION

Infants born small for gestational age (SGA) have an increased risk of developing various cardiovascular complications. While many influencing factors can be adjusted or adapt over time, congenital factors also have a significant role. This study, therefore, seeks to explore the effect of perinatal factors on the left ventricular (LV) parameters in SGA infants, as assessed immediately after birth.

METHODS AND MATERIALS

This single-center prospective cohort study, conducted between 2014 and 2018, involved healthy SGA newborns born > 35 weeks' gestation, delivered at New York-Presbyterian Brooklyn Methodist Hospital, and a gestational age (GA)-matched control group of appropriate for gestational age (AGA) infants. Data analysis was performed using multivariate linear regression in STATA.

RESULTS

The study enrolled 528 neonates, 114 SGA and 414 AGA. SGA infants exhibited a mean GA of 38.05 weeks (vs. 38.54), higher male representation (69.3% vs. 51.5%), lower birth weight (BW) (2318g vs 3381g), lower Apgar scores at birth, and a higher rate of neonatal intensive care unit admission compared to AGA infants (41.2% vs.18.9%; p<0.001). Furthermore, SGA infants were more likely to be born to nulliparous women (63.16% vs. 38.16%; p<0.001), with lower body mass index (BMI) (29.8 vs. 31.7; p=0.004), a lower prevalence of gestational maternal diabetes (GDM) (14.9 % vs. 35.5%; p<0.001), and a higher prevalence of preeclampsia (18.4 % vs. 6.52%; p<0.001). BW was identified as the most significant predictor affecting most LV parameters in this study (p<0.001), except shortening fraction, asymmetric interventricular septal hypertrophy and Inter-ventricular septal thickness/LV posterior wall ratio (IVS/LVPW). Lower GA (coefficient = -0.09, p=0.002), insulin use in GDM (coefficient = 0.39, p=0.014), and low APGAR scores at 1 minute (coefficient = -0.07, p<0.001) were significant predictors of IVS during diastole (R-squared [R2]=0.24). High maternal BMI is marginally associated with LVPW during systole (R2=0.27, coefficient = 0.01, p=0.050), while male sex was a significant predictor of LV internal dimension during diastole (R2=0.29, p=0.033).

CONCLUSION

This study highlights the significant influence of perinatal factors on LV parameters in SGA infants, with BW being the most influential factor. Although LV morphology alone may not predict future cardiovascular risk in the SGA population, further research is needed to develop effective strategies for long-term cardiovascular health management in this population.

PRO Hair Safe Study: The Patient's Perspective on the Effects of Scalp Cooling on Hair Preservation

Purpose

Alopecia has been reported a distressing side-effect of chemotherapy for breast cancer patients (BCP) that is highly relevant for quality of life during treatment. For the prevention of chemotherapy-induced alopecia, scalp cooling (SC) has been reported to be an effective and safe intervention. However, data on the patient's perspective on effectiveness and applicability of SC in a clinical routine setting are scarce. In this comparative study, we aimed at a longitudinal assessment of patient-reported outcome (PRO) data on the effect of SC on alopecia and its effect on symptoms and functional health when applied in clinical routine in BCP receiving taxane or anthracycline-based chemotherapy.

Patients and Methods

Study participants were allocated either to the intervention group receiving SC or to the control group based on patient preference (non-randomized study). All patients completed PRO-measures on hair preservation (EORTC Item Library items on hair loss), symptom and functional health measures (EORTC QLQ-C30 and -BR23) and the Body Image Scale (BIS). Outcomes were assessed at chemotherapy start (baseline), mid-chemotherapy, last chemotherapy cycle, 3 months follow-up and 6-9 months follow-up.

Results

Overall, we included 113 patients: 75 patients underwent SC (mean age = 51.3 years, 52.7% premenopausal); 38 patients standard care (mean age = 55.6 years, 39.5% premenopausal). A total of 53 patients (70.7%) discontinued SC, with 39 patients (73.5%) stating alopecia as the primary reason. On average, BCP stayed on treatment with the cooling cap for 40.2% of the duration of their chemotherapy (SD 25.3%). In an intention-to-treat analysis, we found no difference between the SC group and the control group with regard to their patient-reported hair loss (p=0.831) across the observation period, overall QOL (p=0.627), emotional functioning (p=0.737), social functioning (p=0.635) and body image (p=0.463) did not differ between groups.

Conclusion

We found a high rate of SC-decliners and no beneficial effects of SC for patient-reported hair loss, symptoms and functional health. The efficacy and tolerability of SC applied in a clinical routine setting hence appeared to be limited. The further determination and up-front definition of criteria prognostic for effectiveness of SC may be helpful to identify patient subgroups that may experience a treatment benefit.

Machine learning enabled subgroup analysis with real-world data to inform clinical trial eligibility criteria design

Overly restrictive eligibility criteria for clinical trials may limit the generalizability of the trial results to their target real-world patient populations. We developed a novel machine learning approach using large collections of real-world data (RWD) to better inform clinical trial eligibility criteria design. We extracted patients' clinical events from electronic health records (EHRs), which include demographics, diagnoses, and drugs, and assumed certain compositions of these clinical events within an individual's EHRs can determine the subphenotypes-homogeneous clusters of patients, where patients within each subgroup share similar clinical characteristics. We introduced an outcome-guided probabilistic model to identify those subphenotypes, such that the patients within the same subgroup not only share similar clinical characteristics but also at similar risk levels of encountering severe adverse events (SAEs). We evaluated our algorithm on two previously conducted clinical trials with EHRs from the OneFlorida+ Clinical Research Consortium. Our model can clearly identify the patient subgroups who are more likely to suffer or not suffer from SAEs as subphenotypes in a transparent and interpretable way. Our approach identified a set of clinical topics and derived novel patient representations based on them. Each clinical topic represents a certain clinical event composition pattern learned from the patient EHRs. Tested on both trials, patient subgroup (#SAE=0) and patient subgroup (#SAE>0) can be well-separated by k-means clustering using the inferred topics. The inferred topics characterized as likely to align with the patient subgroup (#SAE>0) revealed meaningful combinations of clinical features and can provide data-driven recommendations for refining the exclusion criteria of clinical trials. The proposed supervised topic modeling approach can infer the clinical topics from the subphenotypes with or without SAEs. The potential rules for describing the patient subgroups with SAEs can be further derived to inform the design of clinical trial eligibility criteria.

Adverse events in MS patients fulfilling or not inclusion criteria of the respective clinical trial - The problem of generalizability

BACKGROUND

The aim of this study was to evaluate how many MS patients treated with an approved DMD in routine care would have fulfilled the inclusion and exclusion criteria of phase III clinical trial and would therefore be eligible for the respective drug trial. Further, adverse events and disease progression for these patients were compared.

METHODS

A comparison of patients fulfilling phase III clinical trial inclusion and exclusion criteria and those who do not with regard to sociodemographic and clinical characteristics, adverse events and disease progression. Database was the REGIMS register, a national, prospective, observational, clinical multicentre registry. 1248 MS Patients were included.

RESULTS

27.2% patients would have been eligible for inclusion into a phase III clinical trial of their indication. Patients who did not meet the criterion age are more likely to have a serious adverse event (SAE), whereas patients who did not fulfil the criterion relapse had a significant lower occurrence of an adverse event (AE). Non-fulfilment of other inclusion criteria (EDSS Score; medication history and MS type) did not show any significant differences in drug safety variables, AE and SAE.

CONCLUSION

Our results suggest that a low transferability of phase III clinical trial criteria, to patients in routine care with the exception of age, does not imply a higher risk with regard to adverse and serious adverse events.

Analysis of the distribution of trial sites in South Korea using social network analysis

Location of trial sites can be a potential source of study bias. Considering that clinical trials have been mostly conducted in urban areas, the distribution of trial sites need to be evaluated. We analyzed clinical trial approval data using social network analysis to quantitatively assess the site-by-site connections. The approval list of clinical trials from the Ministry of Food and Drug Safety database between 2014 and 2021 was analyzed. The number of clinical trials per trial site was counted according to the approval year and study phase and evaluated for distribution using empirical cumulative distribution function plots. Trial sites and conducts of a clinical trial were mapped into nodes and edges in the social network analysis, and basic network parameters were obtained. The clinical trials were concentrated at several trial sites. Forty-nine to 60.6% of phase 1 and up to 30% of the other study phases of clinical trials were at the top 5 trial sites. The annual distribution of the number of clinical trials per site was comparable across the study period. Connections among the trial sites in the metropolitan area were prominent. Graph size and density were higher in phase 3 trials than in the other phases. We demonstrated that the conduct of clinical trials was concentrated in the Seoul Metropolitan Area in both number of trials and connections using social network analysis.

Evaluating Reporting Completeness of Patient-Reported Outcomes in Esophageal Motility Disorders: A Cross-Sectional Analysis of Randomized Controlled Trials

Esophageal motility disorders (EMD) can have significant effects on quality of life. Patient-reported outcomes (PROs) provide valuable insight into the patient's perspective on their treatment and are becoming increasingly used in randomized controlled trials (RCTs). Thus, our investigation aims to evaluate the completeness of reporting of PROs in RCTs pertaining to EMDs. We searched MEDLINE, Embase, and Cochrane Central Register of Controlled Trials for published RCTs focused on EMDs. Included RCTs were published between 2006 and 2020, reported a primary outcome related to an EMDs, and listed at least one PRO measure as a primary or secondary outcome. Investigators screened and extracted data in a masked, duplicate fashion. Data extraction was carried out using both the CONSORT-PRO adaptation and Cochrane Collaboration Risk of Bias 2.0 tool. We assessed overall mean percent completion of the CONSORT-PRO adaptation and a bivariate regression analysis was used to assess relationships between trial characteristics and completeness of reporting. The overall mean percent completion of the CONSORT-PRO checklist adaptation was 43.86% (SD = 17.03). RCTs with a primary PRO had a mean completeness of 47.73% (SD = 17.32) and RCTs with a secondary PRO was 35.36% (SD = 13.52). RCTs with a conflict of interest statement were 18.15% (SE = 6.5) more complete (t = 2.79, P = .009) than trials lacking a statement. No additional significant associations between trial characteristics and completeness of reporting were found. PRO reporting completeness in RCTs focused on EMDs was inadequate. We urge EMD researchers to prioritize complete PRO reporting to foster patient-centered research for future RCTs on EMDs.

Effect of electronic reminders on patients' compliance during clear aligner treatment: an interrupted time series study

Patient compliance is relevant to achieving therapeutic goals during clear aligner therapy (CAT). The aim of this study was to evaluate the efficacy of remote electronic (e-)reminders and e-feedback on compliance during CAT using an interrupted time series (ITS) analysis. We used routinely collected mobile application data from a German healthtech company (PlusDental, Berlin). Our primary outcome was self-reported compliance (aligner wear time min. 22 h on 75% of their aligners were classified as fully compliant, min. 22 h on 50–74.9% of their aligners: fairly compliant; min. 22 h on < 50% of their aligners: poorly compliant). E-reminders and e-feedback were introduced in the 1st quarter of 2020. Compliance was assessed at semi-monthly intervals from June-December 2019 (n = 1899) and June-December 2020 (n = 5486), resulting in a pre- and post-intervention group. ITS and segmented regression modelling were used to estimate the effect on the change in levels and trends of poor compliance. Pre-intervention, poor compliance was at 24.47% (95% CI: 22.59% to 26.46%). After the introduction of e-reminders and e-feedback (i.e., post-intervention), the percentage of poorly compliant patients decreased substantially, levelling off at 9.32% (95% CI: 8.31% to 10.45%). E-reminders and e-feedback were effective for increasing compliance in CAT patients.

Clinical Significance: Orthodontists and dentists may consider digital monitoring and e-reminders to improve compliance and increase treatment success.

Implementation of Web-Based Psychosocial Interventions for Adults With Acquired Brain Injury and Their Caregivers: Systematic Review

BACKGROUND

More than 135 million people worldwide live with acquired brain injury (ABI) and its many psychosocial sequelae. This growing global burden necessitates scalable rehabilitation services. Despite demonstrated potential to increase the accessibility and scalability of psychosocial supports, digital health interventions are challenging to implement and sustain. The Nonadoption, Abandonment, Scale-Up, Spread, and Sustainability (NASSS) framework can offer developers and researchers a comprehensive overview of considerations to implement, scale, and sustain digital health interventions.

OBJECTIVE

This systematic review identified published, peer-reviewed primary evidence of implementation outcomes, strategies, and factors for web-based psychosocial interventions targeting either adults with ABI or their formal or informal caregivers; evaluated and summarized this evidence; synthesized qualitative and quantitative implementation data according to the NASSS framework; and provided recommendations for future implementation. Results were compared with 3 hypotheses which state that complexity (dynamic, unpredictable, and poorly characterized factors) in most or all NASSS domains increases likelihood of implementation failure; success is achievable, but difficult with many complicated domains (containing multiple interacting factors); and simplicity (straightforward, predictable, and few factors) in most or all domains increases the likelihood of success.

METHODS

From a comprehensive search of MEDLINE, EMBASE, PsycINFO, CINAHL, Scopus, speechBITE, and neuroBITE, we reviewed primary implementation evidence from January 2008 to June 2020. For web-based psychosocial interventions delivered via standard desktop computer, mobile phone, tablet, television, and virtual reality devices to adults with ABI or their formal or informal caregivers, we extracted intervention characteristics, stakeholder involvement, implementation scope and outcomes, study design and quality, and implementation data. Implementation data were both narratively synthesized and descriptively quantified across all 7 domains (condition, technology, value proposition, adopters, organization, wider system, and their interaction over time) and all subdomains of the NASSS framework. Study quality and risk of bias were assessed using the 2018 Mixed Methods Appraisal Tool.

RESULTS

We identified 60 peer-reviewed studies from 12 countries, including 5723 adults with ABI, 1920 carers, and 50 health care staff. The findings aligned with all 3 hypotheses.

CONCLUSIONS

Although studies were of low methodological quality and insufficient number to statistically test relationships, the results appeared consistent with recommendations to reduce complexity as much as possible to facilitate implementation. Although studies excluded individuals with a range of comorbidities and sociocultural challenges, such simplification of NASSS domain 1 may have been necessary to advance intervention value propositions (domain 3). However, to create equitable digital health solutions that can be successfully implemented in real-world settings, it is recommended that developers involve people with ABI, their close others, and health care staff in addressing complexities in domains 2 to 7 from the earliest intervention design stages.

TRIAL REGISTRATION

PROSPERO International Prospective Register of Systematic Reviews CRD42020186387; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42020186387.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1177/20552076211035988.

Application of non-negative matrix factorization in oncology: one approach for establishing precision medicine

The increase in the expectations of artificial intelligence (AI) technology has led to machine learning technology being actively used in the medical field. Non-negative matrix factorization (NMF) is a machine learning technique used for image analysis, speech recognition, and language processing; recently, it is being applied to medical research. Precision medicine, wherein important information is extracted from large-scale medical data to provide optimal medical care for every individual, is considered important in medical policies globally, and the application of machine learning techniques to this end is being handled in several ways. NMF is also introduced differently because of the characteristics of its algorithms. In this review, the importance of NMF in the field of medicine, with a focus on the field of oncology, is described by explaining the mathematical science of NMF and the characteristics of the algorithm, providing examples of how NMF can be used to establish precision medicine, and presenting the challenges of NMF. Finally, the direction regarding the effective use of NMF in the field of oncology is also discussed.

Standardizing, Harmonizing, and Protecting Data Collection to Broaden the Impact of COVID-19 Research: The Rapid Acceleration of Diagnostics-Underserved Populations (RADx-up) Initiative

Objective

The Rapid Acceleration of Diagnostics-Underserved Populations (RADx-UP) program is a consortium of community-engaged research projects with the goal of increasing access to Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) tests in underserved populations. To accelerate clinical research, common data elements (CDEs) were selected and refined to standardize data collection and enhance cross-consortium analysis.

Materials and Methods

The RADx-UP consortium began with more than 700 CDEs from the National Institutes of Health (NIH) CDE Repository, Disaster Research Response (DR2) guidelines, and the PHENotypes and eXposures (PhenX) Toolkit. Following a review of initial CDEs, we made selections and further refinements through an iterative process that included live forums, consultations, and surveys completed by the first 69 RADx-UP projects.

Results

Following a multistep CDE development process, we decreased the number of CDEs, modified the question types, and changed the CDE wording. Most research projects were willing to collect and share demographic NIH Tier 1 CDEs, with the top exception reason being a lack of CDE applicability to the project. The NIH RADx-UP Tier 1 CDE with the lowest frequency of collection and sharing was sexual orientation.

Discussion

We engaged a wide range of projects and solicited bidirectional input to create CDEs. These RADx-UP CDEs could serve as the foundation for a patient-centered informatics architecture allowing the integration of disease-specific databases to support hypothesis-driven clinical research in underserved populations.

Conclusion

A community-engaged approach using bidirectional feedback can lead to the better development and implementation of CDEs in underserved populations during public health emergencies.

Evolution of eligibility criteria for non-transplant randomized controlled trials in adults with acute myeloid leukemia

Eligibility criteria for clinical trials are intended to select suitable study subjects but can limit trial participation and generalization of results. While reported for other cancers, non-enrollment rates and evolution of eligibility criteria over time have so far not been studied for randomized controlled trials (RCTs) involving adults with acute myeloid leukemia (AML). Among 3698 studies published between 2010 and 2020, including 447 involving prospective clinical trials, we identified 75 phase three RCTs testing non-transplant therapies for adults with AML. Only 31 studies (41%) provided information on non-enrollment; in these studies, the median non-enrollment rate was 23%, primarily attributed to restrictive eligibility criteria. In 95% of trials, eligibility criteria were reported with the total number per trial increasing over time (P < 0.001), particularly in industry-funded trials. A total of 27 eligibility criteria were used across trials, mostly concerning comorbidities or performance status, with eight of them becoming more common over time. The concordance with recent ASCO - Friends of Cancer Research eligibility criteria recommendations greatly varied, from 35% to 99%. Together, our analyses suggest that the ability to generalize results from non-transplant RCTs may be increasingly limited because of high non-enrollment rates and increasingly restrictive eligibility criteria.

Classifying Drug Ratings Using User Reviews with Transformer-Based Language Models

Drug review websites such as Drugs.com provide users' textual reviews and numeric ratings of drugs. These reviews along with the ratings are used for the consumers for choosing a drug. However, the numeric ratings may not always be consistent with text reviews and purely relying on the rating score for finding positive/negative reviews may not be reliable. Automatic classification of user ratings based on textual review can create a more reliable rating for drugs. In this project, we built classification models to classify drug review ratings using textual reviews with traditional machine learning and deep learning models. Traditional machine learning models including Random Forest and Naive Bayesian classifiers were built using TF-IDF features as input. Also, transformer-based neural network models including BERT, Bio_ClinicalBERT, RoBERTa, XLNet, ELECTRA, and ALBERT were built using the raw text as input. Overall, Bio_ClinicalBERT model outperformed the other models with an overall accuracy of 87%. We further identified concepts of the Unified Medical Language System (UMLS) from the postings and analyzed their semantic types stratified by class types. This research demonstrated that transformer-based models can be used to classify drug reviews based solely on textual reviews.

Quality and clinical generalizability of feasibility outcomes in exercise prehabilitation before colorectal cancer surgery - A systematic review

Suboptimal quality of feasibility assessments might partially explain inconsistencies observed in the effectiveness of exercise prehabilitation before colorectal cancer (CRC) surgery. This systematic review aimed to assess the reporting quality and clinical generalizability of feasibility outcomes in feasibility studies addressing exercise prehabilitation before CRC surgery. PubMed/Medline, Embase, Cochrane, and CINAHL were searched to identify all feasibility studies focussing on exercise prehabilitation in CRC surgery. Reporting quality was assessed using the Thabane et al. checklist and the Consolidated Standards of Reporting Trials extension for feasibility studies. Clinical generalizability was evaluated by appraising patient participation in all steps of the study and intervention. Twelve studies were included. The main feasibility outcome in all studies was adherence to the intervention by the study sample. Based on adherence, 10 studies (83%) concluded exercise prehabilitation to be feasible. Six studies (50%) reported all details to assess patient participation showing retention rates between 18.4% and 58.2%, which was caused by non-participation and drop-out. Three feasibility studies (25%) discussed patient-reported barriers to participation and five additional studies (41%) described potential selection bias. Four studies (33%) reported lessons learned to solve issues hampering feasibility and clinical generalizability. Results suggest that true feasibility of exercise prehabilitation before CRC surgery remains questionable due to poor reporting quality, insufficient clarity regarding the representativeness of the study sample for the target population, and limited attention for clinical generalizability. Feasibility of exercise prehabilitation might be improved by offering supervised community- or home-based interventions tailored to the physical and mental abilities of the patient.

Validation of Real-World Data-based Endpoint Measures of Cancer Treatment Outcomes

Recently, there has been a growing interest in using real-world data (RWD) to generate real-world evidence that complements clinical trials. To quantify treatment effects, it is important to develop meaningful RWD-based endpoints. In cancer trials, two real-world endpoints are of particular interest: real-world overall survival (rwOS) and real-world time to next treatment (rwTTNT). In this work, we identified ways to calculate these real-world endpoints with structured electronic health record (EHR) data and validate these endpoints against the gold-standard measurements of these endpoints derived from linked EHR and tumor registry (TR) data. In addition, we examined and reported data quality issues, especially inconsistencies between the EHR and TR data. Using a survival model, we show that the presence of next treatment was not significantly associated with rwOS, but patients who had longer rwTTNT had longer rwOS, validating the use of rwTTNT as a real-world surrogate marker for measuring cancer endpoints.

Temporal Subtyping of Alzheimer's Disease Using Medical Conditions Preceding Alzheimer's Disease Onset in Electronic Health Records

Subtyping of Alzheimer's disease (AD) can facilitate diagnosis, treatment, prognosis and disease management. It can also support the testing of new prevention and treatment strategies through clinical trials. In this study, we employed spectral clustering to cluster 29,922 AD patients in the OneFlorida Data Trust using their longitudinal EHR data of diagnosis and conditions into four subtypes. These subtypes exhibit different patterns of progression of other conditions prior to the first AD diagnosis. In addition, according to the results of various statistical tests, these subtypes are also significantly different with respect to demographics, mortality, and prescription medications after the AD diagnosis. This study could potentially facilitate early detection and personalized treatment of AD as well as data-driven generalizability assessment of clinical trials for AD.

Construction of Financial Management Early Warning Model Based on Improved Ant Colony Neural Network

With the advent of the era of economic globalization, the world capital market is also facing financial risks. It is necessary to have a corresponding financial management early warning model to reduce economic losses. This paper uses the combination of ant colony algorithm and neural network algorithm to build a neural network improved by ant colony algorithm model. By setting relevant assumptions, the financial statements and annual report texts are predicted and analyzed and compared with the original static data forecasting model. Compared with traditional methods, the time series sequencing analysis used in this paper makes the result prediction more accurate. This allows one year's data to be used to predict the data for the next two years. This research can provide a corresponding reference for the optimization of financial management early warning system.

Research on Practical Intelligent Mode of Digital Image Economy Based on Improved Genetic Multilayer Neural Network

In the context of economic globalization and digitization, the current financial field is in an unprecedented complex situation. The methods and means to deal with this complexity are developing towards image intelligence. This paper takes financial prediction as the starting point, selects the artificial neural network in the intelligent algorithm and optimizes the algorithm, forecasts through the improved multilayer neural network, and compares it with the traditional neural network. Through comparison, it is found that the prediction success rate of the improved genetic multilayer neural network increases with the increase of the dimension of the input image data. This shows that, by adding more technical indicators as the input of the combined network, the prediction efficiency of the improved genetic multilayer neural network can be further improved and the advantage of computing speed can be maintained.

Construction of Community Life Service in the Sharing Economy Based on Deep Neural Network

Currently, the development of sharing economy and interconnection also has a profound impact on community life services. This study is based on the deep neural network theory, combined with the evolution mechanism of the commercial network of the community life service industry, link prediction theory, and the latest deep neural network algorithm, referring to the evolution model of merger and stripping, and the network structure is optimized on this basis. Through simulation experiments and result analysis, the model is used to deeply study the evolution trend and dynamics of the community life service business network from the perspective of quantitative analysis. Then the business network structure is optimized and development is promoted at the same time. At the same time, it can also upgrade those old scattered industries and provide theoretical and decision-making guidance for the future transformation and upgrading of the innovative community life service industry.

Health economic studies of colorectal cancer and the contribution of administrative data: A systematic review

INTRODUCTION

Several forces are contributing to an increase in the number of people living with and surviving colorectal cancer (CRC). However, due to the lack of available data, little is known about the implications of these forces. In recent years, the use of administrative records to inform research has been increasing. The aim of this paper is to investigate the potential contribution that administrative data could have on the health economic research of CRC.

METHODS

To achieve this aim, we conducted a systematic review of the health economic CRC literature published in the United Kingdom and Europe within the last decade (2009-2019).

RESULTS

Thirty-seven relevant studies were identified and divided into economic evaluations, cost of illness studies and cost consequence analyses.

CONCLUSIONS

The use of administrative data, including cancer registry, screening and hospital records, within the health economic research of CRC is commonplace. However, we found that this data often come from regional databases, which reduces the generalisability of results. Further, administrative data appear less able to contribute towards understanding the wider and indirect costs associated with the disease. We explore several ways in which various sources of administrative data could enhance future research in this area.

Quantifying representativeness in randomized clinical trials using machine learning fairness metrics

OBJECTIVE

We help identify subpopulations underrepresented in randomized clinical trials (RCTs) cohorts with respect to national, community-based or health system target populations by formulating population representativeness of RCTs as a machine learning (ML) fairness problem, deriving new representation metrics, and deploying them in easy-to-understand interactive visualization tools.

MATERIALS AND METHODS

We represent RCT cohort enrollment as random binary classification fairness problems, and then show how ML fairness metrics based on enrollment fraction can be efficiently calculated using easily computed rates of subpopulations in RCT cohorts and target populations. We propose standardized versions of these metrics and deploy them in an interactive tool to analyze 3 RCTs with respect to type 2 diabetes and hypertension target populations in the National Health and Nutrition Examination Survey.

RESULTS

We demonstrate how the proposed metrics and associated statistics enable users to rapidly examine representativeness of all subpopulations in the RCT defined by a set of categorical traits (eg, gender, race, ethnicity, smoking status, and blood pressure) with respect to target populations.

DISCUSSION

The normalized metrics provide an intuitive standardized scale for evaluating representation across subgroups, which may have vastly different enrollment fractions and rates in RCT study cohorts. The metrics are beneficial complements to other approaches (eg, enrollment fractions) used to identify generalizability and health equity of RCTs.

CONCLUSION

By quantifying the gaps between RCT and target populations, the proposed methods can support generalizability evaluation of existing RCT cohorts. The interactive visualization tool can be readily applied to identified underrepresented subgroups with respect to any desired source or target populations.

Contemporary use of real-world data for clinical trial conduct in the United States: a scoping review

OBJECTIVE

Real-world data (RWD), defined as routinely collected healthcare data, can be a potential catalyst for addressing challenges faced in clinical trials. We performed a scoping review of database-specific RWD applications within clinical trial contexts, synthesizing prominent uses and themes.

MATERIALS AND METHODS

Querying 3 biomedical literature databases, research articles using electronic health records, administrative claims databases, or clinical registries either within a clinical trial or in tandem with methodology related to clinical trials were included. Articles were required to use at least 1 US RWD source. All abstract screening, full-text screening, and data extraction was performed by 1 reviewer. Two reviewers independently verified all decisions.

RESULTS

Of 2020 screened articles, 89 qualified: 59 articles used electronic health records, 29 used administrative claims, and 26 used registries. Our synthesis was driven by the general life cycle of a clinical trial, culminating into 3 major themes: trial process tasks (51 articles); dissemination strategies (6); and generalizability assessments (34). Despite a diverse set of diseases studied, <10% of trials using RWD for trial process tasks evaluated medications or procedures (5/51). All articles highlighted data-related challenges, such as missing values.

DISCUSSION

Database-specific RWD have been occasionally leveraged for various clinical trial tasks. We observed underuse of RWD within conducted medication or procedure trials, though it is subject to the confounder of implicit report of RWD use.

CONCLUSION

Enhanced incorporation of RWD should be further explored for medication or procedure trials, including better understanding of how to handle related data quality issues to facilitate RWD use.

Clinical comparison between trial participants and potentially eligible patients using electronic health record data: A generalizability assessment method

OBJECTIVE

To present a generalizability assessment method that compares baseline clinical characteristics of trial participants (TP) to potentially eligible (PE) patients as presented in their electronic health record (EHR) data while controlling for clinical setting and recruitment period.

METHODS

For each clinical trial, a clinical event was defined to identify patients of interest using available EHR data from one clinical setting during the trial's recruitment timeframe. The trial's eligibility criteria were then applied and patients were separated into two mutually exclusive groups: (1) TP, which were patients that participated in the trial per trial enrollment data; (2) PE, the remaining patients. The primary outcome was standardized differences in clinical characteristics between TP and PE per trial. A standardized difference was considered prominent if its absolute value was greater than or equal to 0.1. The secondary outcome was the difference in mean propensity scores (PS) between TP and PE per trial, in which the PS represented prediction for a patient to be in the trial. Three diverse trials were selected for illustration: one focused on hepatitis C virus (HCV) patients receiving a liver transplantation; one focused on leukemia patients and lymphoma patients; and one focused on appendicitis patients.

RESULTS

For the HCV trial, 43 TP and 83 PE were found, with 61 characteristics evaluated. Prominent differences were found among 69% of characteristics, with a mean PS difference of 0.13. For the leukemia/lymphoma trial, 23 TP and 23 PE were found, with 39 characteristics evaluated. Prominent differences were found among 82% of characteristics, with a mean PS difference of 0.76. For the appendicitis trial, 123 TP and 242 PE were found, with 52 characteristics evaluated. Prominent differences were found among 52% of characteristics, with a mean PS difference of 0.15.

CONCLUSIONS

Differences in clinical characteristics were observed between TP and PE among all three trials. In two of the three trials, not all of the differences necessarily compromised trial generalizability and subsets of PE could be considered similar to their corresponding TP. In the remaining trial, lack of generalizability appeared present, but may be a result of other factors such as small sample size or site recruitment strategy. These inconsistent findings suggest eligibility criteria alone are sometimes insufficient in defining a target group to generalize to. With caveats in limited scalability, EHR data quality, and lack of patient perspective on trial participation, this generalizability assessment method that incorporates control for temporality and clinical setting promise to better pinpoint clinical patterns and trial considerations.

Applicability of evidence from randomized controlled trials and systematic reviews to clinical practice: A conceptual review

BACKGROUND

The value of randomized controlled trials is dependent on the applicability of their findings to clinical decision-making. The aim of this study is to determine a definition and principles for the applicability of evidence from randomized controlled trials and systematic reviews.

METHODS

This narrative review searched studies from PubMed and Web of Science databases using Cochrane Collaboration's Qualitative Evidence Syntheses guidance. Empirical studies were excluded. Based on the included studies, a definition for the concept and propositions for principles of applicability were formulated.

RESULTS

A definition and 11 propositions are presented, 6 propositions having additional sub-propositions. Low risk of bias, ability to answer to specific questions, documentation of the details of how randomized controlled trials turned out, reporting of favourable and adverse outcomes, and systematic comparison of randomized controlled trials and clinical data were considered important. Biomedical randomized controlled trials have the widest applicability, while heterogeneity in study characteristics, human perception, behaviour, environmental, equity factors, and health economic issues lessen applicability. Obtaining applicable evidence is a gradual process. Methodological and substance expertise is necessary for assessing applicability.

DISCUSSION

A definition of applicability and requirements for applicable evidence from randomized controlled trials to real-world contexts are presented. Propositions are suggested for any assessment of applicability of findings from randomized controlled trials, systematic reviews and meta-analyses.

Eligibility in Cancer Clinical Research: The Intersection of Discovery, Generalizability, Beneficence, and Justice

Eligibility criteria in clinical trials limit the study population for safety and scientific purposes. The American Society of Clinical Oncology and The Friends of Cancer Research collaboration reconsidered common eligibility criteria in cancer trials and found many to be unnecessarily restrictive. The current recommendations further their efforts to facilitate accrual and improve the generalizability of research results to practice.See related articles, p. 2394, 2400, 2408, 2416, 2424, and 2430.

Semantic categorization of Chinese eligibility criteria in clinical trials using machine learning methods

BACKGROUND

Semantic categorization analysis of clinical trials eligibility criteria based on natural language processing technology is crucial for the task of optimizing clinical trials design and building automated patient recruitment system. However, most of related researches focused on English eligibility criteria, and to the best of our knowledge, there are no researches studied the Chinese eligibility criteria. Thus in this study, we aimed to explore the semantic categories of Chinese eligibility criteria.

METHODS

We downloaded the clinical trials registration files from the website of Chinese Clinical Trial Registry (ChiCTR) and extracted both the Chinese eligibility criteria and corresponding English eligibility criteria. We represented the criteria sentences based on the Unified Medical Language System semantic types and conducted the hierarchical clustering algorithm for the induction of semantic categories. Furthermore, in order to explore the classification performance of Chinese eligibility criteria with our developed semantic categories, we implemented multiple classification algorithms, include four baseline machine learning algorithms (LR, NB, kNN, SVM), three deep learning algorithms (CNN, RNN, FastText) and two pre-trained language models (BERT, ERNIE).

RESULTS

We totally developed 44 types of semantic categories, summarized 8 topic groups, and investigated the average incidence and prevalence in 272 hepatocellular carcinoma related Chinese clinical trials. Compared with the previous proposed categories in English eligibility criteria, 13 novel categories are identified in Chinese eligibility criteria. The classification result shows that most of semantic categories performed quite well, the pre-trained language model ERNIE achieved best performance with macro-average F1 score of 0.7980 and micro-average F1 score of 0.8484.

CONCLUSION

As a pilot study of Chinese eligibility criteria analysis, we developed the 44 semantic categories by hierarchical clustering algorithms for the first times, and validated the classification capacity with multiple classification algorithms.

Comparison of Clinical Characteristics Between Clinical Trial Participants and Nonparticipants Using Electronic Health Record Data

IMPORTANCE

Assessing generalizability of clinical trials is important to ensure appropriate application of interventions, but most assessments provide minimal granularity on comparisons of clinical characteristics.

OBJECTIVE

To assess the extent of underlying clinical differences between clinical trial participants and nonparticipants by using a combination of electronic health record and trial enrollment data.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study used data obtained from a single academic medical center between September 1996 and January 2019 to identify 1645 clinical trial participants from a diverse set of 202 available trials conducted at the center. Using an aggregated resampling procedure, nonparticipants were matched to participants 1:1 based on trial conditions, number of recent visits to a health care professional, and calendar time.

EXPOSURES

Clinical trial enrollment vs no enrollment.

MAIN OUTCOMES AND MEASURES

The primary outcome was standardized differences in clinical characteristics between participants and nonparticipants in clinical trials stratified into the 4 most common disease domains.

RESULTS

This cross-sectional study included 1645 participants from 202 trials (929 [56.5%] male; mean [SD] age, 54.65 [21.38] years) and an aggregated set of 1645 nonparticipants (855 [52.0%] male; mean [SD] age, 57.24 [21.91] years). The most common disease domains for the selected trials were neoplastic disease (86 trials; 737 participants), disorders of the digestive system (31 trials; 321 participants), inflammatory disorders (28 trials; 276 participants), and disorders of the cardiovascular system (27 trials; 319 participants); trials could qualify for multiple disease domains. Among 31 conditions, the percentage of conditions for which the prevalence was lower among participants than among nonparticipants per standardized differences was 64.5% (20 conditions) for neoplastic disease trials, 61.3% (19) for digestive system trials, 58.1% (18) for inflammatory disorder trials, and 38.7% (12) for cardiovascular system trials. Among 17 medications, the percentage of medications for which use was less among participants than among nonparticipants per standardized differences was 64.7% (11) for neoplastic disease trials, 58.8% (10) for digestive system trials, 88.2% (15) for inflammatory disorder trials, and 52.9% (9) for cardiovascular system trials.

CONCLUSIONS AND RELEVANCE

Using a combination of electronic health record and trial enrollment data, this study found that clinical trial participants had fewer comorbidities and less use of medication than nonparticipants across a variety of disease domains. Combining trial enrollment data with electronic health record data may be useful for better understanding of the generalizability of trial results.

How the clinical research community responded to the COVID-19 pandemic: an analysis of the COVID-19 clinical studies in ClinicalTrials.gov

OBJECTIVE

In the past few months, a large number of clinical studies on the novel coronavirus disease (COVID-19) have been initiated worldwide to find effective therapeutics, vaccines, and preventive strategies for COVID-19. In this study, we aim to understand the landscape of COVID-19 clinical research and identify the issues that may cause recruitment difficulty or reduce study generalizability.

METHODS

We analyzed 3765 COVID-19 studies registered in the largest public registry-ClinicalTrials.gov, leveraging natural language processing (NLP) and using descriptive, association, and clustering analyses. We first characterized COVID-19 studies by study features such as phase and tested intervention. We then took a deep dive and analyzed their eligibility criteria to understand whether these studies: (1) considered the reported underlying health conditions that may lead to severe illnesses, and (2) excluded older adults, either explicitly or implicitly, which may reduce the generalizability of these studies to the older adults population.

RESULTS

Our analysis included 2295 interventional studies and 1470 observational studies. Most trials did not explicitly exclude older adults with common chronic conditions. However, known risk factors such as diabetes and hypertension were considered by less than 5% of trials based on their trial description. Pregnant women were excluded by 34.9% of the studies.

CONCLUSIONS

Most COVID-19 clinical studies included both genders and older adults. However, risk factors such as diabetes, hypertension, and pregnancy were under-represented, likely skewing the population that was sampled. A careful examination of existing COVID-19 studies can inform future COVID-19 trial design towards balanced internal validity and generalizability.