Screening intervention to identify eligible patients and improve accrual to phase II-IV oncology clinical trials

[Development of an artificial intelligence system to improve cancer clinical trial eligibility screening]

INTRODUCTION

The recruitment step of all clinical trials is time consuming, harsh and generate extra costs. Artificial intelligence tools could improve recruitment in order to shorten inclusion phase. The objective was to assess the performance of an artificial intelligence driven tool (text mining, machine learning, classification…) for the screening and detection of patients, potentially eligible for recruitment in one of the clinical trials open at the "Institut de Cancérologie de Lorraine".

METHODS

Computerized clinical data during the first medical consultation among patients managed in an anticancer center over the 2019-2023 period were used to study the performances of an artificial intelligence tool (SAS® Viya). Recall, precision and F1-score were used to determine the artificial intelligence algorithm effectiveness. Time saved on screening was determined by the difference between the time taken using the artificial intelligence-assisted method and that taken using the standard method in clinical trial participant screening.

RESULTS

Out of 9876 patients included in the study, the artificial intelligence algorithm obtained the following scores: precision of 96 %, recall of 94 % and a 0.95 F1-score to detect patients with breast cancer (n=2039) and potentially eligible for inclusion in a clinical trial. The screening of 258 potentially eligible patient's files took 20s per file vs. 5min and 6s with standard method.

DISCUSSION

This study suggests that artificial intelligence could yield sizable improvements over standard practices in several aspects of the patient screening process, as well as in approaches to feasibility, site selection, and trial selection.

Strategies for increasing accrual in cancer clinical trials: What is the evidence?

INTRODUCTION

Despite the importance of clinical trial participation among cancer patients, few participate-and even fewer patients from ethnic and racial minoritized groups. It is unclear whether suggested approaches to increase accrual are successful. We conducted a scoping review to identify evidence-based approaches to increase participation in cancer treatment clinical trials that demonstrated clear increases in accrual. Notably, more stringent than other published reviews, only those studies with comparison data to measure a difference in accrual rates were included.

METHODS

We searched PubMed/MEDLINE, Embase, CINAHL, and Web of Science for English-language articles published from January 1, 2012, to August 8, 2022. Studies were included if they were conducted in the United States, described single or multicomponent interventions, and provided data to measure accrual relative to baseline levels or that compared accrual rates with other interventions.

RESULTS

Sixteen articles were included: six with interventions addressing patient barriers, two addressing provider barriers, seven describing institutional change, and one describing policy change. Key themes emerged, such as a focus on patient education, cultural competency, and building the capacity of clinics. Few studies provide comparative accrual data, making it difficult to identify with certainty any effective, evidence-based approaches for increasing accrual. Some patient- and system-level interventions studies showed modest increases in accrual primarily through pre-post measurement.

CONCLUSION

Despite an extensive body of literature about the barriers that impede cancer treatment trial accrual, along with numerous recommendations for how to overcome these barriers, results reveal surprisingly little evidence published in the last 10 years on interventions that increase accrual relative to baseline levels or compared with other interventions. As clinical trials are a primary vehicle through which we improve cancer care, it is critical that evidence-based approaches are used to inform all efforts to increase accrual. Strategies for increasing participation in cancer clinical trials must be developed and rigorously evaluated so that these strategies can be disseminated, participation in trials can increase and become more equitable, and trial results can become more generalizable.

Evaluation of an artificial intelligence-based clinical trial matching system in Chinese patients with hepatocellular carcinoma: a retrospective study

BACKGROUND

Artificial intelligence (AI)-assisted clinical trial screening is a promising prospect, although previous matching systems were developed in English, and relevant studies have only been conducted in Western countries. Therefore, we evaluated an AI-based clinical trial matching system (CTMS) that extracts medical data from the electronic health record system and matches them to clinical trials automatically.

METHODS

This study included 1,053 consecutive inpatients primarily diagnosed with hepatocellular carcinoma who were referred to the liver tumor center of an academic medical center in China between January and December 2019. The eligibility criteria extracted from two clinical trials, patient attributes, and gold standard were decided manually. We evaluated the performance of the CTMS against the established gold standard by measuring the accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and run time required.

RESULTS

The manual reviewers demonstrated acceptable interrater reliability (Cohen's kappa 0.65-0.88). The performance results for the CTMS were as follows: accuracy, 92.9-98.0%; sensitivity, 51.9-83.5%; specificity, 99.0-99.1%; PPV, 75.7-85.1%; and NPV, 97.4-98.9%. The time required for eligibility determination by the CTMS and manual reviewers was 2 and 150 h, respectively.

CONCLUSIONS

We found that the CTMS is particularly reliable in excluding ineligible patients in a significantly reduced amount of time. The CTMS excluded ineligible patients for clinical trials with good performance, reducing 98.7% of the work time. Thus, such AI-based systems with natural language processing and machine learning have potential utility in Chinese clinical trials.

Brief Report: Implementation of a Universal Prescreening Protocol to Increase Recruitment to Lung Cancer Studies at a Veterans Affairs Cancer Center

Introduction

The oncology clinical trial recruitment process is time, labor, and resource intensive, and poor accrual rates are common. We describe the VA Connecticut Cancer Center experience of implementing a standardized, universal prescreening protocol and its impact on thoracic oncology research recruitment.

Methods

Research coordinators prescreened potentially eligible patients with confirmed or suspected cancer from multiple clinical sources and entered relevant patient and research study information into a centralized electronic database. The database provided real-time lists of potential studies for each patient. This enabled the research team to alert the patient's oncologist in advance of clinic visits and to prepare documents needed for enrollment. Clinicians could ensure sufficient time and attention in clinic to the informed consent process, therefore maximizing enrollment opportunities. Patients were also monitored on waitlists for future studies.

Results

From March 2017 to December 2020, a total of 1518 patients with lung nodules and suspected or confirmed lung cancers were prescreened. Of these, 379 patients were enrolled to a study, 103 patients declined participation, and 639 were monitored for future studies. Our prescreening protocol identified all new patients with lung cancer who were ultimately added to the cancer registry. We found a substantial increase in study enrollment after prescreening implementation.

Conclusions

Universal prescreening was associated with improved patient enrollment to thoracic oncology studies. The protocol was integral in our VA becoming the top accruing VA site for National Cancer Institute's National Clinical Trials Network studies for 2019 to 2021.

Strategies to Advance Equity in Cancer Clinical Trials

Cancer clinical trials are critical for testing new treatments, yet less than 5% of patients with cancer enroll in these trials. Minority groups, elderly individuals, and rural populations are particularly underrepresented in cancer treatment trials. Strategies for advancing equity in cancer clinical trials for these populations include (1) optimizing clinical trial matching by broadening eligibility criteria, screening all patients for trial eligibility, expanding the number of trials against which patients are screened, and following up on all patient matches with an enrollment invitation; (2) conducting site self-assessments to identify clinical-, patient-, provider-, and system-level barriers that contribute to low rates of clinical trial screening and enrollment; (3) creating a quality improvement plan that addresses the barriers to enrollment and incorporates the use of tools and strategies such as clinical trial checklists; workforce development and trainings to improve cultural competence and reduce unconscious bias; guides to promote community education, outreach and engagement with cancer clinical trials; screening and accrual logs designed to measure participation by demographics; models of informed consent that improve understanding; clinical trial designs that reduce accessibility barriers; use of cancer clinical trial patient navigators; and programs to eliminate barriers to participation and out-of-pocket expenses; and (4) working with stakeholders to develop both protocols that are inclusive of diverse populations' geographic locations, and strategies to access those trials. These actions will support greater access for populations that have remained underrepresented in cancer clinical trials and thereby increase the generalizability and efficiency of cancer research.

A systematic review on natural language processing systems for eligibility prescreening in clinical research

OBJECTIVE

We conducted a systematic review to assess the effect of natural language processing (NLP) systems in improving the accuracy and efficiency of eligibility prescreening during the clinical research recruitment process.

MATERIALS AND METHODS

Guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards of quality for reporting systematic reviews, a protocol for study eligibility was developed a priori and registered in the PROSPERO database. Using predetermined inclusion criteria, studies published from database inception through February 2021 were identified from 5 databases. The Joanna Briggs Institute Critical Appraisal Checklist for Quasi-experimental Studies was adapted to determine the study quality and the risk of bias of the included articles.

RESULTS

Eleven studies representing 8 unique NLP systems met the inclusion criteria. These studies demonstrated moderate study quality and exhibited heterogeneity in the study design, setting, and intervention type. All 11 studies evaluated the NLP system's performance for identifying eligible participants; 7 studies evaluated the system's impact on time efficiency; 4 studies evaluated the system's impact on workload; and 2 studies evaluated the system's impact on recruitment.

DISCUSSION

NLP systems in clinical research eligibility prescreening are an understudied but promising field that requires further research to assess its impact on real-world adoption. Future studies should be centered on continuing to develop and evaluate relevant NLP systems to improve enrollment into clinical studies.

CONCLUSION

Understanding the role of NLP systems in improving eligibility prescreening is critical to the advancement of clinical research recruitment.

The Impact of an Online Training Program About Cancer Clinical Trials on Primary Care Physicians' Knowledge, Attitudes and Beliefs, and Behavior

Participation in cancer clinical trials (CCTs) is critical to improving cancer treatments and quality of care. However, rates of patient participation remain low. Research has shown that a trusted physician recommendation is an important influence on patients' decisions to enroll in a CCT. Improving primary care providers' (PCPs') knowledge, attitudes, and beliefs about CCTs is a promising potential path for improving CCT participation. The aim of this pilot study was to test the effect of an online educational course for PCPs about clinical trials on primary care providers' knowledge, attitudes and beliefs, and behavior. Forty-one PCPs in the New York City area participated in a 1-h online training session on cancer clinical trials. These PCPs had self-selected to complete the training in a previous survey. The objectives of the training module were to (1) educate the PCPs about clinical trials, with a focus on overcoming misconceptions; and (2) discuss roles of PCPs in partnering with oncologists to help patients gain access to clinical trials. The training module included didactics, audio excerpts, and case descriptions. Participants completed a pre-test immediately before taking the course, a post-test immediately after taking the course, and a 3-month post-course survey. All three assessments included a general T/F knowledge test, a 7-item attitude/belief scale, and a knowledge test focused specifically on local resources and access for clinical trials. Forty-one PCPs completed the module and the pre-post course surveys. Eighty percent (33/41) also completed the 3-month post-course survey. General knowledge and local knowledge increased significantly (p < .05) from pre- to post-course. At 3 months post-training, both general and local knowledge scores remained significantly increased from baseline. For those who completed the 3-month post-course survey, attitudes and beliefs increased significantly from pre- to post-course, but this change was not sustained at 3 months post-training. At 3 months post-training, 52% of the PCPs who had an interaction with a recently diagnosed cancer patient reported speaking with patients about CCTs as a result of the training. A brief online course showed significant and sustained improvement in PCPs' general and local knowledge about cancer clinical trials, which translated into self-reported behavior change. Future dissemination of the course and further research into its impact are important next steps.

Accuracy of an Artificial Intelligence System for Cancer Clinical Trial Eligibility Screening: Retrospective Pilot Study

Background

Screening patients for eligibility for clinical trials is labor intensive. It requires abstraction of data elements from multiple components of the longitudinal health record and matching them to inclusion and exclusion criteria for each trial. Artificial intelligence (AI) systems have been developed to improve the efficiency and accuracy of this process.

Objective

This study aims to evaluate the ability of an AI clinical decision support system (CDSS) to identify eligible patients for a set of clinical trials.

Methods

This study included the deidentified data from a cohort of patients with breast cancer seen at the medical oncology clinic of an academic medical center between May and July 2017 and assessed patient eligibility for 4 breast cancer clinical trials. CDSS eligibility screening performance was validated against manual screening. Accuracy, sensitivity, specificity, positive predictive value, and negative predictive value for eligibility determinations were calculated. Disagreements between manual screeners and the CDSS were examined to identify sources of discrepancies. Interrater reliability between manual reviewers was analyzed using Cohen (pairwise) and Fleiss (three-way) κ, and the significance of differences was determined by Wilcoxon signed-rank test.

Results

In total, 318 patients with breast cancer were included. Interrater reliability for manual screening ranged from 0.60-0.77, indicating substantial agreement. The overall accuracy of breast cancer trial eligibility determinations by the CDSS was 87.6%. CDSS sensitivity was 81.1% and specificity was 89%.

Conclusions

The AI CDSS in this study demonstrated accuracy, sensitivity, and specificity of greater than 80% in determining the eligibility of patients for breast cancer clinical trials. CDSSs can accurately exclude ineligible patients for clinical trials and offer the potential to increase screening efficiency and accuracy. Additional research is needed to explore whether increased efficiency in screening and trial matching translates to improvements in trial enrollment, accruals, feasibility assessments, and cost.

Artificial Intelligence Tool for Optimizing Eligibility Screening for Clinical Trials in a Large Community Cancer Center

PURPOSE

Less than 5% of patients with cancer enroll in clinical trials, and 1 in 5 trials are stopped for poor accrual. We evaluated an automated clinical trial matching system that uses natural language processing to extract patient and trial characteristics from unstructured sources and machine learning to match patients to clinical trials.

PATIENTS AND METHODS

Medical records from 997 patients with breast cancer were assessed for trial eligibility at Highlands Oncology Group between May and August 2016. System and manual attribute extraction and eligibility determinations were compared using the percentage of agreement for 239 patients and 4 trials. Sensitivity and specificity of system-generated eligibility determinations were measured, and the time required for manual review and system-assisted eligibility determinations were compared.

RESULTS

Agreement between system and manual attribute extraction ranged from 64.3% to 94.0%. Agreement between system and manual eligibility determinations was 81%-96%. System eligibility determinations demonstrated specificities between 76% and 99%, with sensitivities between 91% and 95% for 3 trials and 46.7% for the 4th. Manual eligibility screening of 90 patients for 3 trials took 110 minutes; system-assisted eligibility determinations of the same patients for the same trials required 24 minutes.

CONCLUSION

In this study, the clinical trial matching system displayed a promising performance in screening patients with breast cancer for trial eligibility. System-assisted trial eligibility determinations were substantially faster than manual review, and the system reliably excluded ineligible patients for all trials and identified eligible patients for most trials.

Strategies designed to help healthcare professionals to recruit participants to research studies

BACKGROUND

Identifying and approaching eligible participants for recruitment to research studies usually relies on healthcare professionals. This process is sometimes hampered by deliberate or inadvertent gatekeeping that can introduce bias into patient selection.

OBJECTIVES

Our primary objective was to identify and assess the effect of strategies designed to help healthcare professionals to recruit participants to research studies.

SEARCH METHODS

We performed searches on 5 January 2015 in the following electronic databases: Cochrane Methodology Register, CENTRAL, MEDLINE, EMBASE, CINAHL, British Nursing Index, PsycINFO, ASSIA and Web of Science (SSCI, SCI-EXPANDED) from 1985 onwards. We checked the reference lists of all included studies and relevant review articles and did citation tracking through Web of Science for all included studies.

SELECTION CRITERIA

We selected all studies that evaluated a strategy to identify and recruit participants for research via healthcare professionals and provided pre-post comparison data on recruitment rates.

DATA COLLECTION AND ANALYSIS

Two review authors independently screened search results for potential eligibility, read full papers, applied the selection criteria and extracted data. We calculated risk ratios for each study to indicate the effect of each strategy.

MAIN RESULTS

Eleven studies met our eligibility criteria and all were at medium or high risk of bias. Only five studies gave the total number of participants (totalling 7372 participants). Three studies used a randomised design, with the others using pre-post comparisons. Several different strategies were investigated. Four studies examined the impact of additional visits or information for the study site, with no increases in recruitment demonstrated. Increased recruitment rates were reported in two studies that used a dedicated clinical recruiter, and five studies that introduced an automated alert system for identifying eligible participants. The studies were embedded into trials evaluating care in oncology mainly but also in emergency departments, diabetes and lower back pain.

AUTHORS' CONCLUSIONS

There is no strong evidence for any single strategy to help healthcare professionals to recruit participants in research studies. Additional visits or information did not appear to increase recruitment by healthcare professionals. The most promising strategies appear to be those with a dedicated resource (e.g. a clinical recruiter or automated alert system) for identifying suitable participants that reduced the demand on healthcare professionals, but these were assessed in studies at high risk of bias.

The National Cancer Institute-American Society of Clinical Oncology Cancer Trial Accrual Symposium: summary and recommendations

INTRODUCTION

Many challenges to clinical trial accrual exist, resulting in studies with inadequate enrollment and potentially delaying answers to important scientific and clinical questions.

METHODS

The National Cancer Institute (NCI) and the American Society of Clinical Oncology (ASCO) cosponsored the Cancer Trial Accrual Symposium: Science and Solutions on April 29-30, 2010 to examine the state of accrual science related to patient/community, physician/provider, and site/organizational influences, and identify new interventions to facilitate clinical trial enrollment. The symposium featured breakout sessions, plenary sessions, and a poster session including 100 abstracts. Among the 358 attendees were clinical investigators, researchers of accrual strategies, research administrators, nurses, research coordinators, patient advocates, and educators. A bibliography of the accrual literature in these three major areas was provided to participants in advance of the meeting. After the symposium, the literature in these areas was revisited to determine if the symposium recommendations remained relevant within the context of the current literature.

RESULTS

Few rigorously conducted studies have tested interventions to address challenges to clinical trials accrual. Attendees developed recommendations for improving accrual and identified priority areas for future accrual research at the patient/community, physician/provider, and site/organizational levels. Current literature continues to support the symposium recommendations.

CONCLUSIONS

A combination of approaches addressing both the multifactorial nature of accrual challenges and the characteristics of the target population may be needed to improve accrual to cancer clinical trials. Recommendations for best practices and for future research developed from the symposium are provided.