Exploring Capabilities of Large Language Models such as ChatGPT in Radiation Oncology

Purpose

Technological progress of machine learning and natural language processing has led to the development of large language models (LLMs), capable of producing well-formed text responses and providing natural language access to knowledge. Modern conversational LLMs such as ChatGPT have shown remarkable capabilities across a variety of fields, including medicine. These models may assess even highly specialized medical knowledge within specific disciplines, such as radiation therapy. We conducted an exploratory study to examine the capabilities of ChatGPT to answer questions in radiation therapy.

Methods and Materials

A set of multiple-choice questions about clinical, physics, and biology general knowledge in radiation oncology as well as a set of open-ended questions were created. These were given as prompts to the LLM ChatGPT, and the answers were collected and analyzed. For the multiple-choice questions, it was checked how many of the answers of the model could be clearly assigned to one of the allowed multiple-choice-answers, and the proportion of correct answers was determined. For the open-ended questions, independent blinded radiation oncologists evaluated the quality of the answers regarding correctness and usefulness on a 5-point Likert scale. Furthermore, the evaluators were asked to provide suggestions for improving the quality of the answers.

Results

For 70 multiple-choice questions, ChatGPT gave valid answers in 66 cases (94.3%). In 60.61% of the valid answers, the selected answer was correct (50.0% of clinical questions, 78.6% of physics questions, and 58.3% of biology questions). For 25 open-ended questions, 12 answers of ChatGPT were considered as "acceptable," "good," or "very good" regarding both correctness and helpfulness by all 6 participating radiation oncologists. Overall, the answers were considered "very good" in 29.3% and 28%, "good" in 28% and 29.3%, "acceptable" in 19.3% and 19.3%, "bad" in 9.3% and 9.3%, and "very bad" in 14% and 14% regarding correctness/helpfulness.

Conclusions

Modern conversational LLMs such as ChatGPT can provide satisfying answers to many relevant questions in radiation therapy. As they still fall short of consistently providing correct information, it is problematic to use them for obtaining medical information. As LLMs will further improve in the future, they are expected to have an increasing impact not only on general society, but also on clinical practice, including radiation oncology.

Current Research and Development in Hyperthermic Intraperitoneal Chemotherapy (HIPEC)-A Cross-Sectional Analysis of Clinical Trials Registered on ClinicalTrials.gov

INTRODUCTION

Over the past two decades, cytoreductive surgery and HIPEC has improved outcomes for selected patients with peritoneal metastasis from various origins. This is a cross-sectional study with descriptive analyses of HIPEC trials registered on ClinicalTrials.gov. This study aimed to characterize clinical trials on HIPEC registered on ClinicalTrials.gov with the primary objective of identifying a trial focus and to examine whether trial results were published.

METHODS

The search included trials registered from 1 January 2001 to 14 March 2022. We examined the associations of exposure variables and other trial features with two primary outcomes: therapeutic focus and results reporting.

RESULTS

In total, 234 clinical trials were identified; 26 (11%) were already published, and 15 (6%) trials have reported their results but have not been published as full papers. Among ongoing nonpublished trials, 81 (39%) were randomized, 30 (14%) were blinded, n = 39 (20%) were later phase trials (i.e., phases 3 and 4), n = 152 (73%) were from a single institution, and 91 (44%) had parallel groups. Most of the trials were recruiting at the time of this analysis (75, 36%), and 39 (20%) were completed but had yet to publish results. In total, 68% of the trials focused on treatment strategies, and 53% investigated the oncological outcome. The most studied neoplasms for HIPEC trials were peritoneally metastasized colorectal cancer (32%), gastric cancer (29%), and ovarian cancer (26%). Twenty different drugs were analyzed in these clinical trials.

CONCLUSIONS

Many study results are awaited from ongoing HIPEC trials. Most HIPEC trials focused on gastric, colorectal, or ovarian cancer. Many clinical trials were identified involving multiple entities and chemotherapeutic agents.

Role of Postoperative Radiotherapy in the Management for Resected NSCLC - Decision Criteria in Clinical Routine Pre- and Post-LungART

BACKGROUND

The role of postoperative radiation therapy (PORT) in stage III N2 NSCLC is controversial. We analyzed decision-making for PORT among European radiation oncology experts in lung cancer.

METHODS

Twenty-two experts were asked before and after presentation of the results of the LungART trial to describe their decision criteria for PORT in the management of pN+ NSCLC patients. Treatment strategies were subsequently converted into decision trees and analyzed.

RESULTS

Following decision criteria were identified: extracapsular nodal extension, incomplete lymph node resection, multistation lymph nodes, high nodal tumor load, poor response to induction chemotherapy, ineligibility to receive adjuvant chemotherapy, performance status, resection margin, lung function and cardiopulmonary comorbidities. The LungART results had impact on decision-making and reduced the number of recommendations for PORT. The only clear indication for PORT was a R1/2 resection. Six experts out of ten who initially recommended PORT for all R0 resected pN2 patients no longer used PORT routinely for these patients, while four still recommended PORT for all patients with pN2. Fourteen experts used PORT only for patients with risk factors, compared to eleven before the presentation of the LungART trial. Four experts stated that PORT was never recommended in R0 resected pN2 patients regardless of risk factors.

CONCLUSION

After presentation of the LungART trial results at ESMO 2020, 82% of our experts still used PORT for stage III pN2 NSCLC patients with risk factors. The recommendation for PORT decreased, especially for patients without risk factors. Cardiopulmonary comorbidities became more relevant in the decision-making for PORT.

Pressurized Intraperitoneal Aerosol Chemotherapy-Related Clinical Trials in the Treatment of Peritoneal Metastases

INTRODUCTION

Pressurized intraperitoneal aerosol chemotherapy (PIPAC) is a treatment option for patients with peritoneal metastases. We evaluated the current status of ongoing prospective clinical trials investigating PIPAC to provide an overview and predict trends in this field.

METHODS

All 367,494 records of clinical trials registered at ClinicalTrials.gov were searched for trials dealing with PIPAC. Active or unpublished trials were further analyzed.

RESULTS

In total, 22 clinical trials were identified and selected for further analyses. Most trials had a single-arm design and were phase I or II. No phase III trials were registered. Academic centers were recorded as primary sponsors in the majority of trials (63.6%). Every year, between 2 and 5 new trials were initiated. In 17 trials (81.8%), PIPAC was used in a palliative setting only, 2 trials performed PIPAC in a neoadjuvant setting, and 2 trials performed PIPAC in an adjuvant setting. Six different drugs (doxorubicin, cisplatin, oxaliplatin, nab-paclitaxel, 5-fluorouracil, and docetaxel) were used in these clinical trials. Most trials investigated the efficacy (n = 15) or safety (n = 7) of PIPAC therapies.

CONCLUSIONS

The results of ongoing clinical trials will bring specific information on indications for PIPAC as well as the impact of PIPAC on quality of life and overall survival.

Radiotherapy for thymic epithelial tumours: a review

Thymic epithelial tumours (TETs) represent a rare disease, yet they are the most common tumours of the anterior mediastinum. Due to the rare occurrence of TETs, evidence on optimal treatment is limited. Surgery is the treatment of choice in the management of TETs, while the role of postoperative radiotherapy (PORT) remains unresolved. PORT remains debated for thymomas, especially in completely resected stage II tumours, for which PORT may be more likely to benefit in the presence of aggressive histology (WHO subtype B2, B3) or extensive transcapsular invasion (Masaoka-Koga stage IIB). For stage III thymoma, evidence suggests an overall survival (OS) benefit for PORT after complete resection. For incompletely resected thymomas stage II or higher PORT is recommended. Thymic carcinomas at any stage with positive resection margins should be offered PORT. Radiotherapy plays an important role in the management of unresectable locally advanced TETs. Induction therapy (chemotherapy or chemoradiation) followed by surgery may be useful for locally advanced thymic malignancies initially considered as unresectable. Chemotherapy only is offered in patients with unresectable, metastatic tumours in palliative intent, checkpoint inhibitors may be promising for refractory diseases. Due to the lack of high-level evidence and the importance of a multidisciplinary approach, TETs should be discussed within a multidisciplinary team and the final recommendation should reflect individual patient preferences.

Role of radiotherapy in the management of brain metastases of NSCLC - Decision criteria in clinical routine

BACKGROUND

Whole brain radiotherapy (WBRT) is a common treatment option for brain metastases secondary to non-small cell lung cancer (NSCLC). Data from the QUARTZ trial suggest that WBRT can be omitted in selected patients and treated with optimal supportive care alone. Nevertheless, WBRT is still widely used to treat brain metastases secondary to NSCLC. We analysed decision criteria influencing the selection for WBRT among European radiation oncology experts.

METHODS

Twenty-two European radiation oncology experts in lung cancer as selected by the European Society for Therapeutic Radiation Oncology (ESTRO) for previous projects and by the Advisory Committee on Radiation Oncology Practice (ACROP) for lung cancer were asked to describe their strategies in the management of brain metastases of NSCLC. Treatment strategies were subsequently converted into decision trees and analysed for agreement and discrepancies.

RESULTS

Eight decision criteria (suitability for SRS, performance status, symptoms, eligibility for targeted therapy, extra-cranial tumour control, age, prognostic scores and "Zugzwang" (the compulsion to treat)) were identified. WBRT was recommended by a majority of the European experts for symptomatic patients not suitable for radiosurgery or fractionated stereotactic radiotherapy. There was also a tendency to use WBRT in the ALK/EGFR/ROS1 negative NSCLC setting.

CONCLUSION

Despite the results of the QUARTZ trial WBRT is still widely used among European radiation oncology experts.