Evaluation of an artificial intelligence clinical trial matching system in Australian lung cancer patients

Artificial intelligence for optimizing recruitment and retention in clinical trials: a scoping review

OBJECTIVE

The objective of our research is to conduct a comprehensive review that aims to systematically map, describe, and summarize the current utilization of artificial intelligence (AI) in the recruitment and retention of participants in clinical trials.

MATERIALS AND METHODS

A comprehensive electronic search was conducted using the search strategy developed by the authors. The search encompassed research published in English, without any time limitations, which utilizes AI in the recruitment process of clinical trials. Data extraction was performed using a data charting table, which included publication details, study design, and specific outcomes/results.

RESULTS

The search yielded 5731 articles, of which 51 were included. All the studies were designed specifically for optimizing recruitment in clinical trials and were published between 2004 and 2023. Oncology was the most covered clinical area. Applying AI to recruitment in clinical trials has demonstrated several positive outcomes, such as increasing efficiency, cost savings, improving recruitment, accuracy, patient satisfaction, and creating user-friendly interfaces. It also raises various technical and ethical issues, such as limited quantity and quality of sample size, privacy, data security, transparency, discrimination, and selection bias.

DISCUSSION AND CONCLUSION

While AI holds promise for optimizing recruitment in clinical trials, its effectiveness requires further validation. Future research should focus on using valid and standardized outcome measures, methodologically improving the rigor of the research carried out.

CriteriaMapper: establishing the automatic identification of clinical trial cohorts from electronic health records by matching normalized eligibility criteria and patient clinical characteristics

The use of electronic health records (EHRs) holds the potential to enhance clinical trial activities. However, the identification of eligible patients within EHRs presents considerable challenges. We aimed to develop a CriteriaMapper system for phenotyping eligibility criteria, enabling the identification of patients from EHRs with clinical characteristics that match those criteria. We utilized clinical trial eligibility criteria and patient EHRs from the Mount Sinai Database. The CriteriaMapper system was developed to normalize the criteria using national standard terminologies and in-house databases, facilitating computability and queryability to bridge clinical trial criteria and EHRs. The system employed rule-based pattern recognition and manual annotation. Our system normalized 367 out of 640 unique eligibility criteria attributes, covering various medical conditions including non-small cell lung cancer, small cell lung cancer, prostate cancer, breast cancer, multiple myeloma, ulcerative colitis, Crohn's disease, non-alcoholic steatohepatitis, and sickle cell anemia. About 174 criteria were encoded with standard terminologies and 193 were normalized using the in-house reference tables. The agreement between automated and manual normalization was high (Cohen's Kappa = 0.82), and patient matching demonstrated a 0.94 F1 score. Our system has proven effective on EHRs from multiple institutions, showing broad applicability and promising improved clinical trial processes, leading to better patient selection, and enhanced clinical research outcomes.

Accelerating drug discovery, development, and clinical trials by artificial intelligence

Artificial intelligence (AI) has profoundly advanced the field of biomedical research, which also demonstrates transformative capacity for innovation in drug development. This paper aims to deliver a comprehensive analysis of the progress in AI-assisted drug development, particularly focusing on small molecules, RNA, and antibodies. Moreover, this paper elucidates the current integration of AI methodologies within the industrial drug development framework. This encompasses a detailed examination of the industry-standard drug development process, supplemented by a review of medications presently undergoing clinical trials. Conclusively, the paper tackles a predominant obstacle within the AI pharmaceutical sector: the absence of AI-conceived drugs receiving approval. This paper also advocates for the adoption of large language models and diffusion models as a viable strategy to surmount this challenge. This review not only underscores the significant potential of AI in drug discovery but also deliberates on the challenges and prospects within this dynamically progressing field.

Implementation and evaluation of an additional GPT-4-based reviewer in PRISMA-based medical systematic literature reviews

BACKGROUND

PRISMA-based literature reviews require meticulous scrutiny of extensive textual data by multiple reviewers, which is associated with considerable human effort.

OBJECTIVE

To evaluate feasibility and reliability of using GPT-4 API as a complementary reviewer in systematic literature reviews based on the PRISMA framework.

METHODOLOGY

A systematic literature review on the role of natural language processing and Large Language Models (LLMs) in automatic patient-trial matching was conducted using human reviewers and an AI-based reviewer (GPT-4 API). A RAG methodology with LangChain integration was used to process full-text articles. Agreement levels between two human reviewers and GPT-4 API for abstract screening and between a single reviewer and GPT-4 API for full-text parameter extraction were evaluated.

RESULTS

An almost perfect GPT-human reviewer agreement in the abstract screening process (Cohen's kappa > 0.9) and a lower agreement in the full-text parameter extraction were observed.

CONCLUSION

As GPT-4 has performed on a par with human reviewers in abstract screening, we conclude that GPT-4 has an exciting potential of being used as a main screening tool for systematic literature reviews, replacing at least one of the human reviewers.

Potential application of artificial intelligence in cancer therapy

PURPOSE OF REVIEW

This review underscores the critical role and challenges associated with the widespread adoption of artificial intelligence in cancer care to enhance disease management, streamline clinical processes, optimize data retrieval of health information, and generate and synthesize evidence.

RECENT FINDINGS

Advancements in artificial intelligence models and the development of digital biomarkers and diagnostics are applicable across the cancer continuum from early detection to survivorship care. Additionally, generative artificial intelligence has promised to streamline clinical documentation and patient communications, generate structured data for clinical trial matching, automate cancer registries, and facilitate advanced clinical decision support. Widespread adoption of artificial intelligence has been slow because of concerns about data diversity and data shift, model reliability and algorithm bias, legal oversight, and high information technology and infrastructure costs.

SUMMARY

Artificial intelligence models have significant potential to transform cancer care. Efforts are underway to deploy artificial intelligence models in the cancer practice, evaluate their clinical impact, and enhance their fairness and explainability. Standardized guidelines for the ethical integration of artificial intelligence models in cancer care pathways and clinical operations are needed. Clear governance and oversight will be necessary to gain trust in artificial intelligence-assisted cancer care by clinicians, scientists, and patients.

Use of Natural Language Processing to Infer Sites of Metastatic Disease From Radiology Reports at Scale

PURPOSE

To evaluate natural language processing (NLP) methods to infer metastatic sites from radiology reports.

METHODS

A set of 4,522 computed tomography (CT) reports of 550 patients with 14 types of cancer was used to fine-tune four clinical large language models (LLMs) for multilabel classification of metastatic sites. We also developed an NLP information extraction (IE) system (on the basis of named entity recognition, assertion status detection, and relation extraction) for comparison. Model performances were measured by F1 scores on test and three external validation sets. The best model was used to facilitate analysis of metastatic frequencies in a cohort study of 6,555 patients with 53,838 CT reports.

RESULTS

The RadBERT, BioBERT, GatorTron-base, and GatorTron-medium LLMs achieved F1 scores of 0.84, 0.87, 0.89, and 0.91, respectively, on the test set. The IE system performed best, achieving an F1 score of 0.93. F1 scores of the IE system by individual cancer type ranged from 0.89 to 0.96. The IE system attained F1 scores of 0.89, 0.83, and 0.81, respectively, on external validation sets including additional cancer types, positron emission tomography-CT ,and magnetic resonance imaging scans, respectively. In our cohort study, we found that for colorectal cancer, liver-only metastases were higher in de novo stage IV versus recurrent patients (29.7% v 12.2%; P < .001). Conversely, lung-only metastases were more frequent in recurrent versus de novo stage IV patients (17.2% v 7.3%; P < .001).

CONCLUSION

We developed an IE system that accurately infers metastatic sites in multiple primary cancers from radiology reports. It has explainable methods and performs better than some clinical LLMs. The inferred metastatic phenotypes could enhance cancer research databases and clinical trial matching, and identify potential patients for oligometastatic interventions.

[Development of an artificial intelligence system to improve cancer clinical trial eligibility screening]

INTRODUCTION

The recruitment step of all clinical trials is time consuming, harsh and generate extra costs. Artificial intelligence tools could improve recruitment in order to shorten inclusion phase. The objective was to assess the performance of an artificial intelligence driven tool (text mining, machine learning, classification…) for the screening and detection of patients, potentially eligible for recruitment in one of the clinical trials open at the "Institut de Cancérologie de Lorraine".

METHODS

Computerized clinical data during the first medical consultation among patients managed in an anticancer center over the 2019-2023 period were used to study the performances of an artificial intelligence tool (SAS® Viya). Recall, precision and F1-score were used to determine the artificial intelligence algorithm effectiveness. Time saved on screening was determined by the difference between the time taken using the artificial intelligence-assisted method and that taken using the standard method in clinical trial participant screening.

RESULTS

Out of 9876 patients included in the study, the artificial intelligence algorithm obtained the following scores: precision of 96 %, recall of 94 % and a 0.95 F1-score to detect patients with breast cancer (n=2039) and potentially eligible for inclusion in a clinical trial. The screening of 258 potentially eligible patient's files took 20s per file vs. 5min and 6s with standard method.

DISCUSSION

This study suggests that artificial intelligence could yield sizable improvements over standard practices in several aspects of the patient screening process, as well as in approaches to feasibility, site selection, and trial selection.

Precision Hypertension

Hypertension affects >1 billion people worldwide. Complications of hypertension include stroke, renal failure, cardiac hypertrophy, myocardial infarction, and cardiac failure. Despite the development of various antihypertensive drugs, the number of people with uncontrolled hypertension continues to rise. While the lack of compliance associated with frequent side effects to medication is a contributory issue, there has been a failure to consider the diverse nature of hypertensive populations. Instead, we propose that hypertension can only be truly managed by precision. A precision medicine approach would consider each patient's unique factors. In this review, we discuss the progress toward precision medicine for hypertension with more predictiveness and individualization of treatment. We will highlight the advances in data science, omics (genomics, metabolomics, proteomics, etc), artificial intelligence, gene therapy, and gene editing and their application to precision hypertension.

Artificial intelligence tools for optimising recruitment and retention in clinical trials: a scoping review protocol

INTRODUCTION

In recent years, the influence of artificial intelligence technology on clinical trials has been steadily increasing. It has brought about significant improvements in the efficiency and cost reduction of clinical trials. The objective of this scoping review is to systematically map, describe and summarise the current utilisation of artificial intelligence in recruitment and retention process of clinical trials that has been reported in research. Additionally, the review aims to identify benefits and drawbacks, as well as barriers and facilitators associated with the application of artificial intelligence in optimising recruitment and retention in clinical trials. The findings of this review will provide insights and recommendations for future development of artificial intelligence in the context of clinical trials.

METHODS AND ANALYSIS

The review of relevant literature will follow the methodological framework for scoping studies provided by the Joanna Briggs Institute. A comprehensive electronic search will be conducted using the search strategy developed by the authors. Leading medical and computer science databases such as PubMed, Embase, Scopus, IEEE Xplore and Web of Science Core Collection will be searched. The search will encompass analytical observational studies, descriptive observational studies, experimental and quasi-experimental studies published in all languages, without any time limitations, which use artificial intelligence tools in the recruitment and retention process of clinical trials. The review team will screen the identified studies and import them into a dedicated electronic library specifically created for this review. Data extraction will be performed using a data charting table.

ETHICS AND DISSEMINATION

Secondary data will be attained in this scoping review; therefore, no ethical approval is required. The results of the final review will be published in a peer-reviewed journal. It is expected that results will inform future artificial intelligence and clinical trials research.

Evaluation of an artificial intelligence-based clinical trial matching system in Chinese patients with hepatocellular carcinoma: a retrospective study

BACKGROUND

Artificial intelligence (AI)-assisted clinical trial screening is a promising prospect, although previous matching systems were developed in English, and relevant studies have only been conducted in Western countries. Therefore, we evaluated an AI-based clinical trial matching system (CTMS) that extracts medical data from the electronic health record system and matches them to clinical trials automatically.

METHODS

This study included 1,053 consecutive inpatients primarily diagnosed with hepatocellular carcinoma who were referred to the liver tumor center of an academic medical center in China between January and December 2019. The eligibility criteria extracted from two clinical trials, patient attributes, and gold standard were decided manually. We evaluated the performance of the CTMS against the established gold standard by measuring the accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and run time required.

RESULTS

The manual reviewers demonstrated acceptable interrater reliability (Cohen's kappa 0.65-0.88). The performance results for the CTMS were as follows: accuracy, 92.9-98.0%; sensitivity, 51.9-83.5%; specificity, 99.0-99.1%; PPV, 75.7-85.1%; and NPV, 97.4-98.9%. The time required for eligibility determination by the CTMS and manual reviewers was 2 and 150 h, respectively.

CONCLUSIONS

We found that the CTMS is particularly reliable in excluding ineligible patients in a significantly reduced amount of time. The CTMS excluded ineligible patients for clinical trials with good performance, reducing 98.7% of the work time. Thus, such AI-based systems with natural language processing and machine learning have potential utility in Chinese clinical trials.

AI-Driven Models for Diagnosing and Predicting Outcomes in Lung Cancer: A Systematic Review and Meta-Analysis

(1) Background: Lung cancer's high mortality due to late diagnosis highlights a need for early detection strategies. Artificial intelligence (AI) in healthcare, particularly for lung cancer, offers promise by analyzing medical data for early identification and personalized treatment. This systematic review evaluates AI's performance in early lung cancer detection, analyzing its techniques, strengths, limitations, and comparative edge over traditional methods. (2) Methods: This systematic review and meta-analysis followed the PRISMA guidelines rigorously, outlining a comprehensive protocol and employing tailored search strategies across diverse databases. Two reviewers independently screened studies based on predefined criteria, ensuring the selection of high-quality data relevant to AI's role in lung cancer detection. The extraction of key study details and performance metrics, followed by quality assessment, facilitated a robust analysis using R software (Version 4.3.0). The process, depicted via a PRISMA flow diagram, allowed for the meticulous evaluation and synthesis of the findings in this review. (3) Results: From 1024 records, 39 studies met the inclusion criteria, showcasing diverse AI model applications for lung cancer detection, emphasizing varying strengths among the studies. These findings underscore AI's potential for early lung cancer diagnosis but highlight the need for standardization amidst study variations. The results demonstrate promising pooled sensitivity and specificity of 0.87, signifying AI's accuracy in identifying true positives and negatives, despite the observed heterogeneity attributed to diverse study parameters. (4) Conclusions: AI demonstrates promise in early lung cancer detection, showing high accuracy levels in this systematic review. However, study variations underline the need for standardized protocols to fully leverage AI's potential in revolutionizing early diagnosis, ultimately benefiting patients and healthcare professionals. As the field progresses, validated AI models from large-scale perspective studies will greatly benefit clinical practice and patient care in the future.

Matching Patients to Accelerate Clinical Trials (MPACT): Enabling Technology for Oncology Clinical Trial Workflow

Clinical trial enrollment is impeded by the significant time burden placed on research coordinators screening eligible patients. With 50,000 new cancer cases every year, the Veterans Health Administration (VHA) has made increased access for Veterans to high-quality clinical trials a priority. To aid in this effort, we worked with research coordinators to build the MPACT (Matching Patients to Accelerate Clinical Trials) platform with a goal of improving efficiency in the screening process. MPACT supports both a trial prescreening workflow and a screening workflow, employing Natural Language Processing and Data Science methods to produce reliable phenotypes of trial eligibility criteria. MPACT also has a functionality to track a patient's eligibility status over time. Qualitative feedback has been promising with users reporting a reduction in time spent on identifying eligible patients.

Large Language Models for Healthcare Data Augmentation: An Example on Patient-Trial Matching

The process of matching patients with suitable clinical trials is essential for advancing medical research and providing optimal care. However, current approaches face challenges such as data standardization, ethical considerations, and a lack of interoperability between Electronic Health Records (EHRs) and clinical trial criteria. In this paper, we explore the potential of large language models (LLMs) to address these challenges by leveraging their advanced natural language generation capabilities to improve compatibility between EHRs and clinical trial descriptions. We propose an innovative privacy-aware data augmentation approach for LLM-based patient-trial matching (LLM-PTM), which balances the benefits of LLMs while ensuring the security and confidentiality of sensitive patient data. Our experiments demonstrate a 7.32% average improvement in performance using the proposed LLM-PTM method, and the generalizability to new data is improved by 12.12%. Additionally, we present case studies to further illustrate the effectiveness of our approach and provide a deeper understanding of its underlying principles.

Telemedicine and Cancer Clinical Research: Opportunities for Transformation

ABSTRACT

Telemedicine represents an established mode of patient care delivery that has and will continue to transform cancer clinical research. Through telemedicine, opportunities exist to improve patient care, enhance access to novel therapies, streamline data collection and monitoring, support communication, and increase trial efficiency. Potential challenges include disparities in technology access and literacy, physical examination performance, biospecimen collection, privacy and security concerns, coverage of services by insurance, and regulatory considerations. Coupled with artificial intelligence, telemedicine may offer ways to reach geographically dispersed candidates for narrowly focused cancer clinical trials, such as those targeting rare genomic subsets. Collaboration among clinical trial staff, clinicians, regulators, professional societies, patients, and their advocates is critical to optimize the benefits of telemedicine for clinical cancer research.

An artificial intelligence-assisted diagnostic system for the prediction of benignity and malignancy of pulmonary nodules and its practical value for patients with different clinical characteristics

Objectives

This study aimed to explore the value of an artificial intelligence (AI)-assisted diagnostic system in the prediction of pulmonary nodules.

Methods

The AI system was able to make predictions of benign or malignant nodules. 260 cases of solitary pulmonary nodules (SPNs) were divided into 173 malignant cases and 87 benign cases based on the surgical pathological diagnosis. A stratified data analysis was applied to compare the diagnostic effectiveness of the AI system to distinguish between the subgroups with different clinical characteristics.

Results

The accuracy of AI system in judging benignity and malignancy of the nodules was 75.77% (p < 0.05). We created an ROC curve by calculating the true positive rate (TPR) and the false positive rate (FPR) at different threshold values, and the AUC was 0.755. Results of the stratified analysis were as follows. (1) By nodule position: the AUC was 0.677, 0.758, 0.744, 0.982, and 0.725, respectively, for the nodules in the left upper lobe, left lower lobe, right upper lobe, right middle lobe, and right lower lobe. (2) By nodule size: the AUC was 0.778, 0.771, and 0.686, respectively, for the nodules measuring 5-10, 10-20, and 20-30 mm in diameter. (3) The predictive accuracy was higher for the subsolid pulmonary nodules than for the solid ones (80.54 vs. 66.67%).

Conclusion

The AI system can be applied to assist in the prediction of benign and malignant pulmonary nodules. It can provide a valuable reference, especially for the diagnosis of subsolid nodules and small nodules measuring 5-10 mm in diameter.

Harnessing artificial intelligence to improve clinical trial design

Zhang et al. discuss how artificial intelligence (AI) can be used to optimize clinical trial design and potentially boost the success rate of clinical trials. AI has unparalleled potential to leverage real-world data and unlock valuable insights for innovative trial design.

The Utility of Artificial Intelligence in the Diagnosis and Management of Pancreatic Cancer

Artificial intelligence (AI) has made significant advancements in the medical domain in recent years. AI, an expansive field comprising Machine Learning (ML) and, within it, Deep Learning (DL), seeks to emulate the intricate operations of the human brain. It examines vast amounts of data and plays a crucial role in decision-making, overcoming limitations related to human evaluation. DL utilizes complex algorithms to analyze data. ML and DL are subsets of AI that utilize hard statistical techniques that help machines consistently improve at tasks with experience. Pancreatic cancer is more common in developed countries and is one of the leading causes of cancer-related mortality worldwide. Managing pancreatic cancer remains a challenge despite significant advancements in diagnosis and treatment. AI has secured an almost ubiquitous presence in the field of oncological workup and management, especially in gastroenterology malignancies. AI is particularly useful for various investigations of pancreatic carcinoma because it has specific radiological features that enable diagnostic procedures without the requirement of a histological study. However, interpreting and evaluating resulting images is not always simple since images vary as the disease progresses. Secondly, a number of factors may impact prognosis and response to the treatment process. Currently, AI models have been created for diagnosing, grading, staging, and predicting prognosis and treatment response. This review presents the most up-to-date knowledge on the use of AI in the diagnosis and treatment of pancreatic carcinoma.

Innovative Technologies in CNS Trials: Promises and Pitfalls for Recruitment, Retention, and Representativeness

Objective

Recruitment of a sufficiently large and representative patient sample and its retention during central nervous system (CNS) trials presents major challenges for study sponsors. Technological advances are reshaping clinical trial operations to meet these challenges, and the COVID-19 pandemic further accelerated this development.

Method of Research

The International Society for CNS Clinical Trials and Methodology (ISCTM; www.isctm.org) Innovative Technologies for CNS Trials Working Group surveyed the state of technological innovations for improved recruitment and retention and assessed their promises and pitfalls.

Results

Online advertisement and electronic patient registries can enhance recruitment, but challenges with sample representativeness, conversion rates from eligible prescreening to enrolled patients, data privacy and security, and patient identification remain hurdles for optimal use of these technologies. Electronic medical records (EMR) mining with artificial intelligence (AI)/machine learning (ML) methods is promising but awaits translation into trials. During the study treatment phase, technological innovations increasingly support participant retention, including adherence with the investigational treatment. Digital tools for adherence and retention support take many forms, including patient-centric communication channels between researchers and participants, real-time study reminders, and digital behavioral interventions to increase study compliance. However, such tools add technical complexities to trials, and their impact on the generalizability of results are largely unknown.

Conclusion

Overall, the group found a scarcity of systematic data directly assessing the impact of technological innovations on study recruitment and retention in CNS trials, even for strategies with already high adoption, such as online recruitment. Given the added complexity and costs associated with most technological innovations, such data is needed to fully harness technologies for CNS trials and drive further adoption.

Automated Matching of Patients to Clinical Trials: A Patient-Centric Natural Language Processing Approach for Pediatric Leukemia

PURPOSE

Matching patients to clinical trials is cumbersome and costly. Attempts have been made to automate the matching process; however, most have used a trial-centric approach, which focuses on a single trial. In this study, we developed a patient-centric matching tool that matches patient-specific demographic and clinical information with free-text clinical trial inclusion and exclusion criteria extracted using natural language processing to return a list of relevant clinical trials ordered by the patient's likelihood of eligibility.

MATERIALS AND METHODS

Records from pediatric leukemia clinical trials were downloaded from ClinicalTrials.gov. Regular expressions were used to discretize and extract individual trial criteria. A multilabel support vector machine (SVM) was trained to classify sentence embeddings of criteria into relevant clinical categories. Labeled criteria were parsed using regular expressions to extract numbers, comparators, and relationships. In the validation phase, a patient-trial match score was generated for each trial and returned in the form of a ranked list for each patient.

RESULTS

In total, 5,251 discretized criteria were extracted from 216 protocols. The most frequent criterion was previous chemotherapy/biologics (17%). The multilabel SVM demonstrated a pooled accuracy of 75%. The text processing pipeline was able to automatically extract 68% of eligibility criteria rules, as compared with 80% in a manual version of the tool. Automated matching was accomplished in approximately 4 seconds, as compared with several hours using manual derivation.

CONCLUSION

To our knowledge, this project represents the first open-source attempt to generate a patient-centric clinical trial matching tool. The tool demonstrated acceptable performance when compared with a manual version, and it has potential to save time and money when matching patients to trials.

Machine Learning in Clinical Trials: A Primer with Applications to Neurology

We reviewed foundational concepts in artificial intelligence (AI) and machine learning (ML) and discussed ways in which these methodologies may be employed to enhance progress in clinical trials and research, with particular attention to applications in the design, conduct, and interpretation of clinical trials for neurologic diseases. We discussed ways in which ML may help to accelerate the pace of subject recruitment, provide realistic simulation of medical interventions, and enhance remote trial administration via novel digital biomarkers and therapeutics. Lastly, we provide a brief overview of the technical, administrative, and regulatory challenges that must be addressed as ML achieves greater integration into clinical trial workflows.

Medical imaging and multimodal artificial intelligence models for streamlining and enhancing cancer care: opportunities and challenges

INTRODUCTION

Artificial intelligence (AI) has the potential to transform oncologic care. There have been significant developments in AI applications in medical imaging and increasing interest in multimodal models. These are likely to enable improved oncologic care through more precise diagnosis, increasingly in a more personalized and less invasive manner. In this review, we provide an overview of the current state and challenges that clinicians, administrative personnel and policy makers need to be aware of and mitigate for the technology to reach its full potential.

AREAS COVERED

The article provides a brief targeted overview of AI, a high-level review of the current state and future potential AI applications in diagnostic radiology and to a lesser extent digital pathology, focusing on oncologic applications. This is followed by a discussion of emerging approaches, including multimodal models. The article concludes with a discussion of technical, regulatory challenges and infrastructure needs for AI to realize its full potential.

EXPERT OPINION

There is a large volume of promising research, and steadily increasing commercially available tools using AI. For the most advanced and promising precision diagnostic applications of AI to be used clinically, robust and comprehensive quality monitoring systems and informatics platforms will likely be required.

The role of artificial intelligence in hastening time to recruitment in clinical trials

Novel and developing artificial intelligence (AI) systems can be integrated into healthcare settings in numerous ways. For example, in the case of automated image classification and natural language processing, AI systems are beginning to demonstrate near expert level performance in detecting abnormalities such as seizure activity. This paper, however, focuses on AI integration into clinical trials. During the clinical trial recruitment process, considerable labor and time is spent sifting through electronic health record and interviewing patients. With the advancement of deep learning techniques such as natural language processing, intricate electronic health record data can be efficiently processed. This provides utility to workflows such as recruitment for clinical trials. Studies are starting to show promise in shortening the time to recruitment and reducing workload for those involved in clinical trial design. Additionally, numerous guidelines are being constructed to encourage integration of AI into the healthcare setting with meaningful impact. The goal would be to improve the clinical trial process by reducing bias in patient composition, improving retention of participants, and lowering costs and labor.

Use of artificial intelligence for cancer clinical trial enrollment: a systematic review and meta-analysis

BACKGROUND

The aim of this study is to provide a comprehensive understanding of the current landscape of artificial intelligence (AI) for cancer clinical trial enrollment and its predictive accuracy in identifying eligible patients for inclusion in such trials.

METHODS

Databases of PubMed, Embase, and Cochrane CENTRAL were searched until June 2022. Articles were included if they reported on AI actively being used in the clinical trial enrollment process. Narrative synthesis was conducted among all extracted data: accuracy, sensitivity, specificity, positive predictive value, and negative predictive value. For studies where the 2x2 contingency table could be calculated or supplied by authors, a meta-analysis to calculate summary statistics was conducted using the hierarchical summary receiver operating characteristics curve model.

RESULTS

Ten articles reporting on more than 50 000 patients in 19 datasets were included. Accuracy, sensitivity, and specificity exceeded 80% in all but 1 dataset. Positive predictive value exceeded 80% in 5 of 17 datasets. Negative predictive value exceeded 80% in all datasets. Summary sensitivity was 90.5% (95% confidence interval [CI] = 70.9% to 97.4%); summary specificity was 99.3% (95% CI = 81.8% to 99.9%).

CONCLUSIONS

AI demonstrated comparable, if not superior, performance to manual screening for patient enrollment into cancer clinical trials. As well, AI is highly efficient, requiring less time and human resources to screen patients. AI should be further investigated and implemented for patient recruitment into cancer clinical trials. Future research should validate the use of AI for clinical trials enrollment in less resource-rich regions and ensure broad inclusion for generalizability to all sexes, ages, and ethnicities.

Patient attrition in Molecular Tumour Boards: a systematic review

BACKGROUND

Molecular Tumour Boards (MTBs) were created with the purpose of supporting clinical decision-making within precision medicine. Though in use globally, reporting on these meetings often focuses on the small percentages of patients that receive treatment via this process and are less likely to report on, and assess, patients who do not receive treatment.

METHODS

A literature review was performed to understand patient attrition within MTBs and barriers to patients receiving treatment. A total of 51 papers were reviewed spanning a 6-year period from 11 different countries.

RESULTS

In total, 20% of patients received treatment through the MTB process. Of those that did not receive treatment, the main reasons were no mutations identified (27%), no actionable mutations (22%) and clinical deterioration (15%). However, data were often incomplete due to inconsistent reporting of MTBs with only 55% reporting on patients having no mutations, 55% reporting on the presence of actionable mutations with no treatment options and 59% reporting on clinical deterioration.

DISCUSSION

As patient attrition in MTBs is an issue which is very rarely alluded to in reporting, more transparent reporting is needed to understand barriers to treatment and integration of new technologies is required to process increasing omic and treatment data.

Digital Health Applications in Oncology: An Opportunity to Seize

Digital health advances have transformed many clinical areas including psychiatric and cardiovascular care. However, digital health innovation is relatively nascent in cancer care, which represents the fastest growing area of health-care spending. Opportunities for digital health innovation in oncology include patient-facing technologies that improve patient experience, safety, and patient-clinician interactions; clinician-facing technologies that improve their ability to diagnose pathology and predict adverse events; and quality of care and research infrastructure to improve clinical workflows, documentation, decision support, and clinical trial monitoring. The COVID-19 pandemic and associated shifts of care to the home and community dramatically accelerated the integration of digital health technologies into virtually every aspect of oncology care. However, the pandemic has also exposed potential flaws in the digital health ecosystem, namely in clinical integration strategies; data access, quality, and security; and regulatory oversight and reimbursement for digital health technologies. Stemming from the proceedings of a 2020 workshop convened by the National Cancer Policy Forum of the National Academies of Sciences, Engineering, and Medicine, this article summarizes the current state of digital health technologies in medical practice and strategies to improve clinical utility and integration. These recommendations, with calls to action for clinicians, health systems, technology innovators, and policy makers, will facilitate efficient yet safe integration of digital health technologies into cancer care.

Scoping review of the current landscape of AI-based applications in clinical trials

Background

Clinical trials are essential for bringing new drugs, technologies and procedures to the market and clinical practice. Considering the design and the four-phase development, only 10% of them complete the entire process, partly due to the increasing costs and complexity of clinical trials. This low completion rate has a huge negative impact in terms of population health, quality of care and health economics and sustainability. Automating some of the process' tasks with artificial intelligence (AI) tools could optimize some of the most burdensome ones, like patient selection, matching and enrollment; better patient selection could also reduce harmful treatment side effects. Although the pharmaceutical industry is embracing artificial AI tools, there is little evidence in the literature of their application in clinical trials.

Methods

To address this issue, we performed a scoping review. Following the PRISMA-ScR guidelines, we performed a search on PubMed for articles on the implementation of AI in the development of clinical trials.

Results

The search yielded 772 articles, of which 15 were included. The articles were published between 2019 and 2022 and the results were presented descriptively. About half of the studies addressed the topic of patient recruitment; 12 articles reported specific examples of AI applications; five studies presented a quantitative estimate of the effectiveness of these tools.

Conclusion

All studies present encouraging results on the implementation of AI-based applications to the development of clinical trials. AI-based applications have a lot of potential, but more studies are needed to validate these tools and facilitate their adoption.

Natural Language Processing of Radiology Reports to Detect Complications of Ischemic Stroke

BACKGROUND

Abstraction of critical data from unstructured radiologic reports using natural language processing (NLP) is a powerful tool to automate the detection of important clinical features and enhance research efforts. We present a set of NLP approaches to identify critical findings in patients with acute ischemic stroke from radiology reports of computed tomography (CT) and magnetic resonance imaging (MRI).

METHODS

We trained machine learning classifiers to identify categorical outcomes of edema, midline shift (MLS), hemorrhagic transformation, and parenchymal hematoma, as well as rule-based systems (RBS) to identify intraventricular hemorrhage (IVH) and continuous MLS measurements within CT/MRI reports. Using a derivation cohort of 2289 reports from 550 individuals with acute middle cerebral artery territory ischemic strokes, we externally validated our models on reports from a separate institution as well as from patients with ischemic strokes in any vascular territory.

RESULTS

In all data sets, a deep neural network with pretrained biomedical word embeddings (BioClinicalBERT) achieved the highest discrimination performance for binary prediction of edema (area under precision recall curve [AUPRC] > 0.94), MLS (AUPRC > 0.98), hemorrhagic conversion (AUPRC > 0.89), and parenchymal hematoma (AUPRC > 0.76). BioClinicalBERT outperformed lasso regression (p < 0.001) for all outcomes except parenchymal hematoma (p = 0.755). Tailored RBS for IVH and continuous MLS outperformed BioClinicalBERT (p < 0.001) and linear regression, respectively (p < 0.001).

CONCLUSIONS

Our study demonstrates robust performance and external validity of a core NLP tool kit for identifying both categorical and continuous outcomes of ischemic stroke from unstructured radiographic text data. Medically tailored NLP methods have multiple important big data applications, including scalable electronic phenotyping, augmentation of clinical risk prediction models, and facilitation of automatic alert systems in the hospital setting.

A systematic review on natural language processing systems for eligibility prescreening in clinical research

OBJECTIVE

We conducted a systematic review to assess the effect of natural language processing (NLP) systems in improving the accuracy and efficiency of eligibility prescreening during the clinical research recruitment process.

MATERIALS AND METHODS

Guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards of quality for reporting systematic reviews, a protocol for study eligibility was developed a priori and registered in the PROSPERO database. Using predetermined inclusion criteria, studies published from database inception through February 2021 were identified from 5 databases. The Joanna Briggs Institute Critical Appraisal Checklist for Quasi-experimental Studies was adapted to determine the study quality and the risk of bias of the included articles.

RESULTS

Eleven studies representing 8 unique NLP systems met the inclusion criteria. These studies demonstrated moderate study quality and exhibited heterogeneity in the study design, setting, and intervention type. All 11 studies evaluated the NLP system's performance for identifying eligible participants; 7 studies evaluated the system's impact on time efficiency; 4 studies evaluated the system's impact on workload; and 2 studies evaluated the system's impact on recruitment.

DISCUSSION

NLP systems in clinical research eligibility prescreening are an understudied but promising field that requires further research to assess its impact on real-world adoption. Future studies should be centered on continuing to develop and evaluate relevant NLP systems to improve enrollment into clinical studies.

CONCLUSION

Understanding the role of NLP systems in improving eligibility prescreening is critical to the advancement of clinical research recruitment.

Molecular-based precision oncology clinical decision making augmented by artificial intelligence

The rapid growth and decreasing cost of Next-generation sequencing (NGS) technologies have made it possible to conduct routine large panel genomic sequencing in many disease settings, especially in the oncology domain. Furthermore, it is now known that optimal disease management of patients depends on individualized cancer treatment guided by comprehensive molecular testing. However, translating results from molecular sequencing reports into actionable clinical insights remains a challenge to most clinicians. In this review, we discuss about some representative systems that leverage artificial intelligence (AI) to facilitate some processes of clinicians' decision making based upon molecular data, focusing on their application in precision oncology. Some limitations and pitfalls of the current application of AI in clinical decision making are also discussed.

Artificial intelligence in oncology: From bench to clinic

In the past few years, Artificial Intelligence (AI) techniques have been applied to almost every facet of oncology, from basic research to drug development and clinical care. In the clinical arena where AI has perhaps received the most attention, AI is showing promise in enhancing and automating image-based diagnostic approaches in fields such as radiology and pathology. Robust AI applications, which retain high performance and reproducibility over multiple datasets, extend from predicting indications for drug development to improving clinical decision support using electronic health record data. In this article, we review some of these advances. We also introduce common concepts and fundamentals of AI and its various uses, along with its caveats, to provide an overview of the opportunities and challenges in the field of oncology. Leveraging AI techniques productively to provide better care throughout a patient's medical journey can fuel the predictive promise of precision medicine.

Accuracy of an Artificial Intelligence System for Cancer Clinical Trial Eligibility Screening: Retrospective Pilot Study

Background

Screening patients for eligibility for clinical trials is labor intensive. It requires abstraction of data elements from multiple components of the longitudinal health record and matching them to inclusion and exclusion criteria for each trial. Artificial intelligence (AI) systems have been developed to improve the efficiency and accuracy of this process.

Objective

This study aims to evaluate the ability of an AI clinical decision support system (CDSS) to identify eligible patients for a set of clinical trials.

Methods

This study included the deidentified data from a cohort of patients with breast cancer seen at the medical oncology clinic of an academic medical center between May and July 2017 and assessed patient eligibility for 4 breast cancer clinical trials. CDSS eligibility screening performance was validated against manual screening. Accuracy, sensitivity, specificity, positive predictive value, and negative predictive value for eligibility determinations were calculated. Disagreements between manual screeners and the CDSS were examined to identify sources of discrepancies. Interrater reliability between manual reviewers was analyzed using Cohen (pairwise) and Fleiss (three-way) κ, and the significance of differences was determined by Wilcoxon signed-rank test.

Results

In total, 318 patients with breast cancer were included. Interrater reliability for manual screening ranged from 0.60-0.77, indicating substantial agreement. The overall accuracy of breast cancer trial eligibility determinations by the CDSS was 87.6%. CDSS sensitivity was 81.1% and specificity was 89%.

Conclusions

The AI CDSS in this study demonstrated accuracy, sensitivity, and specificity of greater than 80% in determining the eligibility of patients for breast cancer clinical trials. CDSSs can accurately exclude ineligible patients for clinical trials and offer the potential to increase screening efficiency and accuracy. Additional research is needed to explore whether increased efficiency in screening and trial matching translates to improvements in trial enrollment, accruals, feasibility assessments, and cost.