Digital ECMT Cancer Trial Matching Tool: an Open Source Research Application to Support Oncologists in the Identification of Precision Medicine Clinical Trials

Implementation and evaluation of an additional GPT-4-based reviewer in PRISMA-based medical systematic literature reviews

BACKGROUND

PRISMA-based literature reviews require meticulous scrutiny of extensive textual data by multiple reviewers, which is associated with considerable human effort.

OBJECTIVE

To evaluate feasibility and reliability of using GPT-4 API as a complementary reviewer in systematic literature reviews based on the PRISMA framework.

METHODOLOGY

A systematic literature review on the role of natural language processing and Large Language Models (LLMs) in automatic patient-trial matching was conducted using human reviewers and an AI-based reviewer (GPT-4 API). A RAG methodology with LangChain integration was used to process full-text articles. Agreement levels between two human reviewers and GPT-4 API for abstract screening and between a single reviewer and GPT-4 API for full-text parameter extraction were evaluated.

RESULTS

An almost perfect GPT-human reviewer agreement in the abstract screening process (Cohen's kappa > 0.9) and a lower agreement in the full-text parameter extraction were observed.

CONCLUSION

As GPT-4 has performed on a par with human reviewers in abstract screening, we conclude that GPT-4 has an exciting potential of being used as a main screening tool for systematic literature reviews, replacing at least one of the human reviewers.

A call to action to advance patient-focused and decentralized clinical trials

This commentary is a call to action for a concerted commitment and effort to transform clinical trials and enable people with cancer to participate in clinical trials closer to home. Three key strategies are identified to address major barriers: confront challenges with the interpretation of US Food and Drug Administration Form 1572 requirements (Statement of Investigator); broaden acceptance of local laboratories and imaging centers; and invest in the creation of effective, sustainable partnerships between research centers and local providers.