Quantifying representativeness in randomized clinical trials using machine learning fairness metrics

Enhancing neuro-oncology care through equity-driven applications of artificial intelligence

The disease course and clinical outcome for brain tumor patients depend not only on the molecular and histological features of the tumor but also on the patient's demographics and social determinants of health. While current investigations in neuro-oncology have broadly utilized artificial intelligence (AI) to enrich tumor diagnosis and more accurately predict treatment response, postoperative complications, and survival, equity-driven applications of AI have been limited. However, AI applications to advance health equity in the broader medical field have the potential to serve as practical blueprints to address known disparities in neuro-oncologic care. In this consensus review, we will describe current applications of AI in neuro-oncology, postulate viable AI solutions for the most pressing inequities in neuro-oncology based on broader literature, propose a framework for the effective integration of equity into AI-based neuro-oncology research, and close with the limitations of AI.

Quantitative evaluation of the impact of relaxing eligibility criteria on the risk-benefit profile of drugs for lung cancer based on real-world data

INTRODUCTION

Restrictive eligibility criteria in cancer drug trials result in low enrollment rates and limited population diversity. Relaxed eligibility criteria (REC) based on solid evidence is becoming necessary for stakeholders worldwide. However, the absence of high-quality, favorable evidence remains a major challenge. This study presents a protocol to quantitatively evaluate the impact of relaxing eligibility criteria in common non-small cell lung cancer (NSCLC) protocols in China, on the risk-benefit profile. This involves a detailed explanation of the rationale, framework, and design of REC.

METHODS

To evaluate our REC in NSCLC drug trials, we will first construct a structured, cross-dimensional real-world NSCLC database using deep learning methods. We will then establish randomized virtual cohorts and perform benefit-risk assessment using Monte Carlo simulation and propensity matching. Shapley value will be utilized to quantitatively measure the effect of the change of each eligibility criterion on patient volume, clinical efficacy and safety.

DISCUSSION

This study is one of the few that focuses on the problem of overly stringent eligibility criteria cancer drug clinical trials, providing quantitative evaluation of the effect of relaxing each NSCLC eligibility criterion. This study will not only provide scientific evidence for the rational design of population inclusion in lung cancer clinical trials, but also establish a data governance system, as well as a REC evaluation framework that can be generalized to other cancer studies.

A novel approach to assessing disparity in representativeness of clinical trial participants within a large midwestern healthcare system

Background

Representativeness in clinical trials (CT) serves as a metric of access to healthcare and reflects differences that may determine differential efficacy of medical interventions; thus, quantifying representativeness in CT participation is critical.

Methods

This retrospective, descriptive study utilized patient demographic data extracted from the largest Midwestern non-profit healthcare system. Using data between January 1, 2019 and December 31, 2021, a CT Participant Sample of 4,537 system patients who were active CT participants was compared to a CT Patient Population of 195,726 system patients receiving care by the PI of active CTs, which represented the target population. Chi-square goodness-of-fit tests were used to test differences in distributions of demographic variables between groups, indicating disparity in CT participation. Two metrics adapted from literature - participation incidence disparity (PID) and participation incidence ratio (PIR) - were calculated to quantify absolute and relative disparity in representativeness proportions, respectively. Descriptive approaches to assessing representativeness are also provided.

Results

Results showed significant differences by race/ethnicity (χ2 = 50.64; p < 0.0001), age categories (χ2 = 56.64; p < 0.0001), and insurance (χ2 = 41.29; p < 0.0001). PID and PIR metrics revealed reduced CT participation among non-White racial/ethnic groups and increased CT participation among White Non-Hispanic patients. Further, CT participants ≥80 or Worker's Compensation were underrepresented while those with Self-Pay insurance were overrepresented as CT participants.

Conclusions

Despite progress, continued efforts to not only enroll participants into CTs that are representative of the healthcare system and region, but also to better assess representativeness quantitatively are still needed.

Framework for Research in Equitable Synthetic Control Arms

Randomized Clinical Trials (RCTs) measure an intervention's efficacy, but they may not be generalizable to a desired target population if the RCT is not equitable. Thus, representativeness of RCTs has become a national priority. Synthetic Controls (SCs) that incorporate observational data into RCTs have shown great potential to produce more efficient studies, but their equity is rarely considered. Here, we examine how to improve treatment effect estimation and equity of a trial by augmenting "on-trial" concurrent controls with SCs to form a Hybrid Control Arm (HCA). We introduce FRESCA - a framework to evaluate HCA construction methods using RCT simulations. FRESCA shows that doing propensity and equity adjustment when constructing the HCA leads to accurate population treatment effect estimates while meeting equity goals with potentially less "on-trial" patients. This work represents the first investigation of equity in HCA design that provides definitions, metrics, compelling questions, and resources for future work.

Fair Canonical Correlation Analysis

This paper investigates fairness and bias in Canonical Correlation Analysis (CCA), a widely used statistical technique for examining the relationship between two sets of variables. We present a framework that alleviates unfairness by minimizing the correlation disparity error associated with protected attributes. Our approach enables CCA to learn global projection matrices from all data points while ensuring that these matrices yield comparable correlation levels to group-specific projection matrices. Experimental evaluation on both synthetic and real-world datasets demonstrates the efficacy of our method in reducing correlation disparity error without compromising CCA accuracy.

Representativeness in health research studies: an audit of Greater Manchester Clinical Research Network studies between 2016 and 2021

BACKGROUND

There are increasing concerns that participants in health research in the UK are not representative of the UK population, risking widening health inequities. However, detailed information on the magnitude of the problem is limited. Therefore, we evaluated if the health research conducted in the Greater Manchester region was broadly representative of its diverse population.

METHODS

We conducted an audit of all health  research studies conducted exclusively in Greater Manchester, using data from a national research network. Two researchers selected studies that were (1) an interventional or observational study of a health outcome; (2) 'closed' for recruitment between May 2016 and May 2021 and (3) human research. They extracted study information (dates, contacts, sample recruited, clinical speciality). Participant characteristics were sourced from published and unpublished manuscripts and requested directly from principal investigators and named study contacts. Data were extracted, summarised and compared to the Greater Manchester population for the following metrics: ethnicity, sex, age, deprivation and smoking status. A weighted mean age estimate was calculated to account for variation in age reporting. Too few studies provided patient-level deprivation data so, using the area code of the recruitment site, the area level multiple deprivation, health deprivation and disability index and decile was derived. These data were geo-mapped using QGIS 3.26.

RESULTS

Overall, 145/153 (95%) studies met inclusion criteria and participant information was sourced for 85/145 (59%) studies, representing 21,797 participants. Participant information was incomplete for all metrics. Where ethnicity (N = 10,259) data were available and compared to Greater Manchester estimates there was evidence that ethnic minorities were under-represented (6% versus 16%). Most of the recruitment occurred in central Manchester (50%) and with NHS hospital settings (74%).

CONCLUSIONS

Greater Manchester health research in 2016-2021 was centralised and under-represented ethnic minorities. We could not report which ethnic minority group was least represented because sourcing detailed participant information was challenging. Recommendations to improve the reporting of key participant characteristics with which to monitor representativeness in health research are discussed.

Positive Psychology Interventions in Medical Populations: Critical Issues in Intervention Development, Testing, and Implementation

Positive psychological well-being is prospectively associated with superior health outcomes. Positive psychology interventions have promise as a potentially feasible and effective means of increasing well-being and health in those with medical illness, and several initial studies have shown the potential of such programs in medical populations. At the same time, numerous key issues in the existing positive psychology literature must be addressed to ensure that these interventions are optimally effective. These include (1) assessing the nature and scope of PPWB as part of intervention development and application; (2) identifying and utilizing theoretical models that can clearly outline potential mechanisms by which positive psychology interventions may affect health outcomes; (3) determining consistent, realistic targets for positive psychology interventions; (4) developing consistent approaches to the promotion of positive psychological well-being; (5) emphasizing the inclusion of diverse samples in treatment development and testing; and (6) considering implementation and scalability from the start of intervention development to ensure effective real-world application. Attention to these six domains could greatly facilitate the generation of effective, replicable, and easily adopted positive psychology programs for medical populations with the potential to have an important impact on public health.

Comparing risk prediction models aimed at predicting hospitalizations for adverse drug events in community dwelling older adults: a protocol paper

Background

The objective of this paper is to describe the creation, validation, and comparison of two risk prediction modeling approaches for community-dwelling older adults to identify individuals at highest risk for adverse drug event-related hospitalizations. One approach will use traditional statistical methods, the second will use a machine learning approach.

Methods

We will construct medication, clinical, health care utilization, and other variables known to be associated with adverse drug event-related hospitalizations. To create the cohort, we will include older adults (≥ 65 years of age) empaneled to a primary care physician within the Cedars-Sinai Health System primary care clinics with polypharmacy (≥ 5 medications) or at least 1 medication commonly implicated in ADEs (certain oral hypoglycemics, anti-coagulants, anti-platelets, and insulins). We will use a Fine-Gray Cox proportional hazards model for one risk modeling approach and DataRobot, a data science and analytics platform, to run and compare several widely used supervised machine learning algorithms, including Random Forest, Support Vector Machine, Extreme Gradient Boosting (XGBoost), Decision Tree, Naïve Bayes, and K-Nearest Neighbors. We will use a variety of metrics to compare model performance and to assess the risk of algorithmic bias.

Discussion

In conclusion, we hope to develop a pragmatic model that can be implemented in the primary care setting to risk stratify older adults to further optimize medication management.

Artificial Intelligence Applied to clinical trials: opportunities and challenges

Background

Clinical Trials (CTs) remain the foundation of safe and effective drug development. Given the evolving data-driven and personalized medicine approach in healthcare, it is imperative for companies and regulators to utilize tailored Artificial Intelligence (AI) solutions that enable expeditious and streamlined clinical research. In this paper, we identified opportunities, challenges, and potential implications of AI in CTs.

Methods

Following an extensive search in relevant databases and websites, we gathered publications tackling the use of AI and Machine Learning (ML) in CTs from the past 5 years in the US and Europe, including Regulatory Authorities' documents.

Results

Documented applications of AI commonly concern the oncology field and are mostly being applied in the area of recruitment. Main opportunities discussed aim to create efficiencies across CT activities, including the ability to reduce sample sizes, improve enrollment and conduct faster, more optimized adaptive CTs. While AI is an area of enthusiastic development, the identified challenges are ethical in nature and relate to data availability, standards, and most importantly, lack of regulatory guidance hindering the acceptance of AI tools in drug development. However, future implications are significant and are anticipated to improve the probability of success, reduce trial burden and overall, speed up research and regulatory approval.

Conclusion

The use of AI in CTs is in its relative infancy; however, it is a fast-evolving field. As regulators provide more guidance on the acceptability of AI in specific areas, we anticipate the scope of use to broaden and the volume of implementation to increase rapidly.

Recommendations on compiling test datasets for evaluating artificial intelligence solutions in pathology

Artificial intelligence (AI) solutions that automatically extract information from digital histology images have shown great promise for improving pathological diagnosis. Prior to routine use, it is important to evaluate their predictive performance and obtain regulatory approval. This assessment requires appropriate test datasets. However, compiling such datasets is challenging and specific recommendations are missing. A committee of various stakeholders, including commercial AI developers, pathologists, and researchers, discussed key aspects and conducted extensive literature reviews on test datasets in pathology. Here, we summarize the results and derive general recommendations on compiling test datasets. We address several questions: Which and how many images are needed? How to deal with low-prevalence subsets? How can potential bias be detected? How should datasets be reported? What are the regulatory requirements in different countries? The recommendations are intended to help AI developers demonstrate the utility of their products and to help pathologists and regulatory agencies verify reported performance measures. Further research is needed to formulate criteria for sufficiently representative test datasets so that AI solutions can operate with less user intervention and better support diagnostic workflows in the future.