1
|
Hurkmans C, Bibault JE, Brock KK, van Elmpt W, Feng M, David Fuller C, Jereczek-Fossa BA, Korreman S, Landry G, Madesta F, Mayo C, McWilliam A, Moura F, Muren LP, El Naqa I, Seuntjens J, Valentini V, Velec M. A joint ESTRO and AAPM guideline for development, clinical validation and reporting of artificial intelligence models in radiation therapy. Radiother Oncol 2024; 197:110345. [PMID: 38838989 DOI: 10.1016/j.radonc.2024.110345] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2024] [Accepted: 05/23/2024] [Indexed: 06/07/2024]
Abstract
BACKGROUND AND PURPOSE Artificial Intelligence (AI) models in radiation therapy are being developed with increasing pace. Despite this, the radiation therapy community has not widely adopted these models in clinical practice. A cohesive guideline on how to develop, report and clinically validate AI algorithms might help bridge this gap. METHODS AND MATERIALS A Delphi process with all co-authors was followed to determine which topics should be addressed in this comprehensive guideline. Separate sections of the guideline, including Statements, were written by subgroups of the authors and discussed with the whole group at several meetings. Statements were formulated and scored as highly recommended or recommended. RESULTS The following topics were found most relevant: Decision making, image analysis, volume segmentation, treatment planning, patient specific quality assurance of treatment delivery, adaptive treatment, outcome prediction, training, validation and testing of AI model parameters, model availability for others to verify, model quality assurance/updates and upgrades, ethics. Key references were given together with an outlook on current hurdles and possibilities to overcome these. 19 Statements were formulated. CONCLUSION A cohesive guideline has been written which addresses main topics regarding AI in radiation therapy. It will help to guide development, as well as transparent and consistent reporting and validation of new AI tools and facilitate adoption.
Collapse
Affiliation(s)
- Coen Hurkmans
- Department of Radiation Oncology, Catharina Hospital, Eindhoven, the Netherlands; Department of Electrical Engineering, Technical University Eindhoven, Eindhoven, the Netherlands.
| | | | - Kristy K Brock
- Departments of Imaging Physics and Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Wouter van Elmpt
- Department of Radiation Oncology (MAASTRO), GROW - School for Oncology and Reproduction, Maastricht University Medical Centre+, Maastricht, the Netherlands
| | - Mary Feng
- University of California San Francisco, San Francisco, CA, USA
| | - Clifton David Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer, Houston, TX
| | - Barbara A Jereczek-Fossa
- Dept. of Oncology and Hemato-oncology, University of Milan, Milan, Italy; Dept. of Radiation Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Stine Korreman
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark; Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
| | - Guillaume Landry
- Department of Radiation Oncology, LMU University Hospital, LMU Munich, Munich, Germany; German Cancer Consortium (DKTK), Partner Site Munich, a Partnership between DKFZ and LMU University Hospital Munich, Germany; Bavarian Cancer Research Center (BZKF), Partner Site Munich, Munich, Germany
| | - Frederic Madesta
- Department of Computational Neuroscience, University Medical Center Hamburg-Eppendorf, Hamburg, Germany; Institute for Applied Medical Informatics, University Medical Center Hamburg-Eppendorf, Hamburg, Germany; Center for Biomedical Artificial Intelligence (bAIome), University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Chuck Mayo
- Institute for Healthcare Policy and Innovation, University of Michigan, USA
| | - Alan McWilliam
- Division of Cancer Sciences, The University of Manchester, Manchester, UK
| | - Filipe Moura
- CrossI&D Lisbon Research Center, Portuguese Red Cross Higher Health School Lisbon, Portugal
| | - Ludvig P Muren
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark; Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
| | - Issam El Naqa
- Department of Machine Learning, Moffitt Cancer Center, Tampa, FL 33612, USA
| | - Jan Seuntjens
- Princess Margaret Cancer Centre, Radiation Medicine Program, University Health Network & Departments of Radiation Oncology and Medical Biophysics, University of Toronto, Toronto, Canada
| | - Vincenzo Valentini
- Department of Diagnostic Imaging, Oncological Radiotherapy and Hematology, Fondazione Policlinico Universitario "Agostino Gemelli" IRCCS, Rome, Italy; Università Cattolica del Sacro Cuore, Rome, Italy
| | - Michael Velec
- Radiation Medicine Program, Princess Margaret Cancer Centre and Department of Radiation Oncology, University of Toronto, Toronto, Canada
| |
Collapse
|
2
|
Richardson SL, Bosch WR, Mayo CS, McNutt TR, Moran JM, Popple RA, Xiao Y, Covington EL. Order From Chaos: The Benefits of Standardized Nomenclature in Radiation Oncology. Pract Radiat Oncol 2024:S1879-8500(24)00080-8. [PMID: 38636586 DOI: 10.1016/j.prro.2024.04.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Revised: 02/28/2024] [Accepted: 04/01/2024] [Indexed: 04/20/2024]
Abstract
Although standardization has been shown to improve patient safety and improve the efficiency of workflows, implementation of standards can take considerable effort and requires the engagement of all clinical stakeholders. Engaging team members includes increasing awareness of the proposed benefit of the standard, a clear implementation plan, monitoring for improvements, and open communication to support successful implementation. The benefits of standardization often focus on large institutions to improve research endeavors, yet all clinics can benefit from standardization to increase quality and implement more efficient or automated workflow. The benefits of nomenclature standardization for all team members and institution sizes, including success stories, are discussed with practical implementation guides to facilitate the adoption of standardized nomenclature in radiation oncology.
Collapse
Affiliation(s)
- Susan L Richardson
- Department of Radiation Oncology, Swedish Medical Center-Tumor Institute, Seattle, Washington.
| | - Walter R Bosch
- Department of Radiation Oncology, Washington University, Saint Louis, Missouri
| | - Charles S Mayo
- Department of Radiation Oncology, University of Michigan, Ann Arbor, Michigan
| | - Todd R McNutt
- Department of Radiation Oncology and Molecular Radiation Sciences, Johns Hopkins University, Baltimore, Maryland
| | - Jean M Moran
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York, New York
| | - Richard A Popple
- Department of Radiation Oncology, University of Alabama at Birmingham, Birmingham, Alabama
| | - Ying Xiao
- Department of Radiation Oncology, University of Pennsylvania, Philadelphia, Pennsylvania
| | | |
Collapse
|
3
|
Suh Y, Jeong J, Park SM, Heo KN, Lee MY, Ah YM, Kim JW, Kim KI, Lee JY. Development of a claims-based risk-scoring model to predict emergency department visits in older patients receiving anti-neoplastic therapy. Sci Rep 2024; 14:1485. [PMID: 38233529 PMCID: PMC10794170 DOI: 10.1038/s41598-024-51981-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2023] [Accepted: 01/11/2024] [Indexed: 01/19/2024] Open
Abstract
This study developed and validated a risk-scoring model, with a particular emphasis on medication-related factors, to predict emergency department (ED) visits among older Korean adults (aged 65 and older) undergoing anti-neoplastic therapy. Utilizing national claims data, we constructed two cohorts: the development cohort (2016-2018) with 34,642 patients and validation cohort (2019) with 10,902 patients. The model included a comprehensive set of predictors: demographics, cancer type, comorbid conditions, ED visit history, and medication use variables. We employed the least absolute shrinkage and selection operator (LASSO) regression to refine and select the most relevant predictors. Out of 120 predictor variables, 12 were integral to the final model, including seven related to medication use. The model demonstrated acceptable predictive performance in the validation cohort with a C-statistic of 0.76 (95% CI 0.74-0.77), indicating reasonable calibration. This risk-scoring model, after further clinical validation, has the potential to assist healthcare providers in the effective management and care of older patients receiving anti-neoplastic therapy.
Collapse
Affiliation(s)
- Yewon Suh
- College of Pharmacy and Research Institute of Pharmaceutical Sciences, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, Republic of Korea
- Department of Pharmacy, Seoul National University Bundang Hospital, Seongnam, Gyeonggi-do, Republic of Korea
| | - Jonghyun Jeong
- College of Pharmacy and Research Institute of Pharmaceutical Sciences, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, Republic of Korea
| | - Soh Mee Park
- College of Pharmacy and Research Institute of Pharmaceutical Sciences, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, Republic of Korea
- Department of Pharmacy, Seoul National University Bundang Hospital, Seongnam, Gyeonggi-do, Republic of Korea
| | - Kyu-Nam Heo
- College of Pharmacy and Research Institute of Pharmaceutical Sciences, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, Republic of Korea
| | - Mee Yeon Lee
- College of Pharmacy and Research Institute of Pharmaceutical Sciences, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, Republic of Korea
| | - Young-Mi Ah
- College of Pharmacy, Yeungnam University, Gyeongsan, Gyeongbuk, Republic of Korea
| | - Jin Won Kim
- Division of Hematology and Medical Oncology, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Gyeonggi-do, Republic of Korea
- Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Kwang-Il Kim
- Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea
- Division of Geriatrics, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Gyeonggi-do, Republic of Korea
| | - Ju-Yeun Lee
- College of Pharmacy and Research Institute of Pharmaceutical Sciences, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, Republic of Korea.
- Department of Pharmacy, Seoul National University Bundang Hospital, Seongnam, Gyeonggi-do, Republic of Korea.
| |
Collapse
|
4
|
Gebrael G, Sahu KK, Chigarira B, Tripathi N, Mathew Thomas V, Sayegh N, Maughan BL, Agarwal N, Swami U, Li H. Enhancing Triage Efficiency and Accuracy in Emergency Rooms for Patients with Metastatic Prostate Cancer: A Retrospective Analysis of Artificial Intelligence-Assisted Triage Using ChatGPT 4.0. Cancers (Basel) 2023; 15:3717. [PMID: 37509379 PMCID: PMC10378202 DOI: 10.3390/cancers15143717] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Revised: 07/19/2023] [Accepted: 07/20/2023] [Indexed: 07/30/2023] Open
Abstract
BACKGROUND Accurate and efficient triage is crucial for prioritizing care and managing resources in emergency rooms. This study investigates the effectiveness of ChatGPT, an advanced artificial intelligence system, in assisting health providers with decision-making for patients presenting with metastatic prostate cancer, focusing on the potential to improve both patient outcomes and resource allocation. METHODS Clinical data from patients with metastatic prostate cancer who presented to the emergency room between 1 May 2022 and 30 April 2023 were retrospectively collected. The primary outcome was the sensitivity and specificity of ChatGPT in determining whether a patient required admission or discharge. The secondary outcomes included the agreement between ChatGPT and emergency medicine physicians, the comprehensiveness of diagnoses, the accuracy of treatment plans proposed by both parties, and the length of medical decision making. RESULTS Of the 147 patients screened, 56 met the inclusion criteria. ChatGPT had a sensitivity of 95.7% in determining admission and a specificity of 18.2% in discharging patients. In 87.5% of cases, ChatGPT made the same primary diagnoses as physicians, with more accurate terminology use (42.9% vs. 21.4%, p = 0.02) and more comprehensive diagnostic lists (median number of diagnoses: 3 vs. 2, p < 0.001). Emergency Severity Index scores calculated by ChatGPT were not associated with admission (p = 0.12), hospital stay length (p = 0.91) or ICU admission (p = 0.54). Despite shorter mean word count (169 ± 66 vs. 272 ± 105, p < 0.001), ChatGPT was more likely to give additional treatment recommendations than physicians (94.3% vs. 73.5%, p < 0.001). CONCLUSIONS Our hypothesis-generating data demonstrated that ChatGPT is associated with a high sensitivity in determining the admission of patients with metastatic prostate cancer in the emergency room. It also provides accurate and comprehensive diagnoses. These findings suggest that ChatGPT has the potential to assist health providers in improving patient triage in emergency settings, and may enhance both efficiency and quality of care provided by the physicians.
Collapse
Affiliation(s)
- Georges Gebrael
- Department of Oncology, Huntsman Cancer Institute, University of Utah, Salt Lake City, UT 84112, USA
| | - Kamal Kant Sahu
- Department of Oncology, Huntsman Cancer Institute, University of Utah, Salt Lake City, UT 84112, USA
| | - Beverly Chigarira
- Department of Oncology, Huntsman Cancer Institute, University of Utah, Salt Lake City, UT 84112, USA
| | - Nishita Tripathi
- Department of Oncology, Huntsman Cancer Institute, University of Utah, Salt Lake City, UT 84112, USA
| | - Vinay Mathew Thomas
- Department of Oncology, Huntsman Cancer Institute, University of Utah, Salt Lake City, UT 84112, USA
| | - Nicolas Sayegh
- Department of Oncology, Huntsman Cancer Institute, University of Utah, Salt Lake City, UT 84112, USA
| | - Benjamin L Maughan
- Department of Oncology, Huntsman Cancer Institute, University of Utah, Salt Lake City, UT 84112, USA
| | - Neeraj Agarwal
- Department of Oncology, Huntsman Cancer Institute, University of Utah, Salt Lake City, UT 84112, USA
| | - Umang Swami
- Department of Oncology, Huntsman Cancer Institute, University of Utah, Salt Lake City, UT 84112, USA
| | - Haoran Li
- Division of Medical Oncology, University of Kansas Cancer Center, Westwood, KS 66205, USA
| |
Collapse
|