|
1
|
Guckenberger M, Andratschke N, Ahmadsei M, Christ SM, Heusel AE, Kamal S, Kroese TE, Looman EL, Reichl S, Vlaskou Badra E, von der Grün J, Willmann J, Tanadini-Lang S, Mayinger M. Potential of ChatGPT in facilitating research in radiation oncology? Radiother Oncol 2023; 188:109894. [PMID: 37659658 DOI: 10.1016/j.radonc.2023.109894] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2023] [Revised: 08/20/2023] [Accepted: 08/25/2023] [Indexed: 09/04/2023]
Abstract
PURPOSE To evaluate the potential of the artificial intelligence (AI) chatbot ChatGPT in supporting young clinical scientists with scientific tasks in radio oncological research. MATERIALS AND METHODS Seven scientific tasks were to be completed in 3 h by 8 radiation oncologists with different scientific experience working at a university hospital: creation of a scientific synopsis, creation of a research question and corresponding clinical trial hypotheses, writing of the first paragraph of a manuscript introduction, clinical trial sample size calculation, and clinical data analyses (multivariate analysis, boxplot and survival curve). No participant had prior experience with an AI chatbot. All participants were instructed in ChatGPT v3.5 and its use was provided for all tasks. Answers were scored independently by two blinded experts. The subjective value of ChatGPT was rated by each participant. Data were analyzed with regression-, t-test and Spearman correlation (p < 0.05). RESULTS Participants completed tasks 1-3 with an average score of 50% and 4-7 with 56%. Scientific experience, number of original publications and of first/last authorships showed a positive correlation with overall scoring (p = 0.01-0.04). Participants with little to moderate scientific experience scored ChatGPT to be more helpful in solving tasks 4-7 compared to more experienced participants (p = 0.04), with simultaneously presenting lower scorings (p = 0.03). CONCLUSIONS ChatGPT did not compensate for differences in scientific experience of young clinical scientists, with less experienced researchers believing false AI-generated scientific results.
Collapse
Affiliation(s)
- Matthias Guckenberger
- Department of Radiation Oncology, University Hospital of Zurich, University of Zurich, Zurich, Switzerland
| | - Nicolaus Andratschke
- Department of Radiation Oncology, University Hospital of Zurich, University of Zurich, Zurich, Switzerland
| | - Maiwand Ahmadsei
- Department of Radiation Oncology, University Hospital of Zurich, University of Zurich, Zurich, Switzerland
| | - Sebastian Matthias Christ
- Department of Radiation Oncology, University Hospital of Zurich, University of Zurich, Zurich, Switzerland
| | - Astrid Elisabeth Heusel
- Department of Radiation Oncology, University Hospital of Zurich, University of Zurich, Zurich, Switzerland
| | - Sandeep Kamal
- Department of Radiation Oncology, University Hospital of Zurich, University of Zurich, Zurich, Switzerland
| | - Tiuri Ewout Kroese
- Department of Radiation Oncology, University Hospital of Zurich, University of Zurich, Zurich, Switzerland
| | - Esmée Lauren Looman
- Department of Radiation Oncology, University Hospital of Zurich, University of Zurich, Zurich, Switzerland
| | - Sabrina Reichl
- Department of Radiation Oncology, University Hospital of Zurich, University of Zurich, Zurich, Switzerland
| | - Eugenia Vlaskou Badra
- Department of Radiation Oncology, University Hospital of Zurich, University of Zurich, Zurich, Switzerland
| | - Jens von der Grün
- Department of Radiation Oncology, University Hospital of Zurich, University of Zurich, Zurich, Switzerland
| | - Jonas Willmann
- Department of Radiation Oncology, University Hospital of Zurich, University of Zurich, Zurich, Switzerland
| | - Stephanie Tanadini-Lang
- Department of Radiation Oncology, University Hospital of Zurich, University of Zurich, Zurich, Switzerland
| | - Michael Mayinger
- Department of Radiation Oncology, University Hospital of Zurich, University of Zurich, Zurich, Switzerland.
| |
Collapse
|