1
|
Talcott W, Covington E, Bazan J, Wright JL. The Future of Safety and Quality in Radiation Oncology. Semin Radiat Oncol 2024; 34:433-440. [PMID: 39271278 DOI: 10.1016/j.semradonc.2024.07.008] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/15/2024]
Abstract
The increasing complexity of radiation therapy treatment presents new potentials for error and suboptimal care. High-performing programs thus not only require adherence to, but also ongoing improvement of, key safety and quality practices. In this article, we review these practices including standardization, risk analysis, peer review, and maintenance of strong safety culture, while also describing recent innovations and promising future directions. We specifically highlight the growing role of artificial intelligence in radiation oncology, both as a tool to deliver safe, high-quality care and as a potential new source of safety challenges.
Collapse
Affiliation(s)
- Wesley Talcott
- Northwell Health Department of Radiation Oncology, New York, NY
| | | | - Jose Bazan
- City of Hope Comprehensive Cancer Center, Department of Radiation Oncology, Duarte, CA
| | - Jean L Wright
- Department of Radiation Oncology, Johns Hopkins University, Baltimore, MD.
| |
Collapse
|
2
|
Holmes J, Zhang L, Ding Y, Feng H, Liu Z, Liu T, Wong WW, Vora SA, Ashman JB, Liu W. Benchmarking a Foundation Large Language Model on its Ability to Relabel Structure Names in Accordance With the American Association of Physicists in Medicine Task Group-263 Report. Pract Radiat Oncol 2024:S1879-8500(24)00098-5. [PMID: 39243241 DOI: 10.1016/j.prro.2024.04.017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2023] [Revised: 04/11/2024] [Accepted: 04/16/2024] [Indexed: 09/09/2024]
Abstract
PURPOSE To introduce the concept of using large language models (LLMs) to relabel structure names in accordance with the American Association of Physicists in Medicine Task Group-263 standard and to establish a benchmark for future studies to reference. METHODS AND MATERIALS Generative Pretrained Transformer (GPT)-4 was implemented within a Digital Imaging and Communications in Medicine server. Upon receiving a structure-set Digital Imaging and Communications in Medicine file, the server prompts GPT-4 to relabel the structure names according to the American Association of Physicists in Medicine Task Group-263 report. The results were evaluated for 3 disease sites: prostate, head and neck, and thorax. For each disease site, 150 patients were randomly selected for manually tuning the instructions prompt (in batches of 50), and 50 patients were randomly selected for evaluation. Structure names considered were those that were most likely to be relevant for studies using structure contours for many patients. RESULTS The per-patient accuracy was 97.2%, 98.3%, and 97.1% for prostate, head and neck, and thorax disease sites, respectively. On a per-structure basis, the clinical target volume was relabeled correctly in 100%, 95.3%, and 92.9% of cases, respectively. CONCLUSIONS Given the accuracy of GPT-4 in relabeling structure names as presented in this work, LLMs are poised to become an important method for standardizing structure names in radiation oncology, especially considering the rapid advancements in LLM capabilities that are likely to continue.
Collapse
Affiliation(s)
- Jason Holmes
- Department of Radiation Oncology, Mayo Clinic, Phoenix, Arizona.
| | - Lian Zhang
- Department of Radiation Oncology, Mayo Clinic, Phoenix, Arizona
| | - Yuzhen Ding
- Department of Radiation Oncology, Mayo Clinic, Phoenix, Arizona
| | - Hongying Feng
- Department of Radiation Oncology, Mayo Clinic, Phoenix, Arizona
| | - Zhengliang Liu
- School of Computing, University of Georgia, Athens, Georgia
| | - Tianming Liu
- School of Computing, University of Georgia, Athens, Georgia
| | - William W Wong
- Department of Radiation Oncology, Mayo Clinic, Phoenix, Arizona
| | - Sujay A Vora
- Department of Radiation Oncology, Mayo Clinic, Phoenix, Arizona
| | | | - Wei Liu
- Department of Radiation Oncology, Mayo Clinic, Phoenix, Arizona
| |
Collapse
|
3
|
Covington EL, Suresh K, Anderson BM, Barker M, Dess K, Price JG, Moncion A, Vaccarelli MJ, Santanam L, Xiao Y, Mayo C. Perceptions on and roadblocks to implementation of standardized nomenclature in radiation oncology: A survey from TG-263U1. J Appl Clin Med Phys 2024; 25:e14359. [PMID: 38689502 PMCID: PMC11163509 DOI: 10.1002/acm2.14359] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2023] [Revised: 02/02/2024] [Accepted: 03/25/2024] [Indexed: 05/02/2024] Open
Abstract
PURPOSE AAPM Task Group No. 263U1 (Update to Report No. 263 - Standardizing Nomenclatures in Radiation Oncology) disseminated a survey to receive feedback on utilization, gaps, and means to facilitate further adoption. METHODS The survey was created by TG-263U1 members to solicit feedback from physicists, dosimetrists, and physicians working in radiation oncology. Questions on the adoption of the TG-263 standard were coupled with demographic information, such as clinical role, place of primary employment (e.g., private hospital, academic center), and size of institution. The survey was emailed to all AAPM, AAMD, and ASTRO members. RESULTS The survey received 463 responses with 310 completed survey responses used for analysis, of whom most had the clinical role of medical physicist (73%) and the majority were from the United States (83%). There were 83% of respondents who indicated that they believe that having a nomenclature standard is important or very important and 61% had adopted all or portions of TG-263 in their clinics. For those yet to adopt TG-263, the staffing and implementation efforts were the main cause for delaying adoption. Fewer respondents had trouble adopting TG-263 for organs at risk (29%) versus target (44%) nomenclature. Common themes in written feedback were lack of physician support and available resources, especially in vendor systems, to facilitate adoption. CONCLUSIONS While there is strong support and belief in the benefit of standardized nomenclature, the widespread adoption of TG-263 has been hindered by the effort needed by staff for implementation. Feedback from the survey is being utilized to drive the focus of the update efforts and create tools to facilitate easier adoption of TG-263.
Collapse
Affiliation(s)
| | - Krithika Suresh
- Department of Radiation OncologyMichigan MedicineAnn ArborMichiganUSA
| | - Brian M. Anderson
- Department of Radiation OncologyUniversity of North Carolina at Chapel HillChapel HillNorth CarolinaUSA
| | | | - Kathryn Dess
- Department of Radiation OncologyMichigan MedicineAnn ArborMichiganUSA
| | - Jeremy G. Price
- Department of Radiation OncologyFox Chase Cancer CenterPhiladelphiaPennsylvaniaUSA
| | - Alexander Moncion
- Department of Radiation OncologyMichigan MedicineAnn ArborMichiganUSA
| | | | - Lakshmi Santanam
- Medical Physics DepartmentMemorial Sloan‐Kettering Cancer CenterNew YorkNew YorkUSA
| | - Ying Xiao
- Department of Radiation OncologyUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - Charles Mayo
- Department of Radiation OncologyMichigan MedicineAnn ArborMichiganUSA
| |
Collapse
|
4
|
Richardson SL, Bosch WR, Mayo CS, McNutt TR, Moran JM, Popple RA, Xiao Y, Covington EL. Order From Chaos: The Benefits of Standardized Nomenclature in Radiation Oncology. Pract Radiat Oncol 2024:S1879-8500(24)00080-8. [PMID: 38636586 DOI: 10.1016/j.prro.2024.04.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Revised: 02/28/2024] [Accepted: 04/01/2024] [Indexed: 04/20/2024]
Abstract
Although standardization has been shown to improve patient safety and improve the efficiency of workflows, implementation of standards can take considerable effort and requires the engagement of all clinical stakeholders. Engaging team members includes increasing awareness of the proposed benefit of the standard, a clear implementation plan, monitoring for improvements, and open communication to support successful implementation. The benefits of standardization often focus on large institutions to improve research endeavors, yet all clinics can benefit from standardization to increase quality and implement more efficient or automated workflow. The benefits of nomenclature standardization for all team members and institution sizes, including success stories, are discussed with practical implementation guides to facilitate the adoption of standardized nomenclature in radiation oncology.
Collapse
Affiliation(s)
- Susan L Richardson
- Department of Radiation Oncology, Swedish Medical Center-Tumor Institute, Seattle, Washington.
| | - Walter R Bosch
- Department of Radiation Oncology, Washington University, Saint Louis, Missouri
| | - Charles S Mayo
- Department of Radiation Oncology, University of Michigan, Ann Arbor, Michigan
| | - Todd R McNutt
- Department of Radiation Oncology and Molecular Radiation Sciences, Johns Hopkins University, Baltimore, Maryland
| | - Jean M Moran
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York, New York
| | - Richard A Popple
- Department of Radiation Oncology, University of Alabama at Birmingham, Birmingham, Alabama
| | - Ying Xiao
- Department of Radiation Oncology, University of Pennsylvania, Philadelphia, Pennsylvania
| | | |
Collapse
|
5
|
Kapoor R, Sleeman WC, Ghosh P, Palta J. Infrastructure tools to support an effective Radiation Oncology Learning Health System. J Appl Clin Med Phys 2023; 24:e14127. [PMID: 37624227 PMCID: PMC10562037 DOI: 10.1002/acm2.14127] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2023] [Revised: 07/17/2023] [Accepted: 07/19/2023] [Indexed: 08/26/2023] Open
Abstract
PURPOSE Radiation Oncology Learning Health System (RO-LHS) is a promising approach to improve the quality of care by integrating clinical, dosimetry, treatment delivery, research data in real-time. This paper describes a novel set of tools to support the development of a RO-LHS and the current challenges they can address. METHODS We present a knowledge graph-based approach to map radiotherapy data from clinical databases to an ontology-based data repository using FAIR concepts. This strategy ensures that the data are easily discoverable, accessible, and can be used by other clinical decision support systems. It allows for visualization, presentation, and data analyses of valuable information to identify trends and patterns in patient outcomes. We designed a search engine that utilizes ontology-based keyword searching, synonym-based term matching that leverages the hierarchical nature of ontologies to retrieve patient records based on parent and children classes, connects to the Bioportal database for relevant clinical attributes retrieval. To identify similar patients, a method involving text corpus creation and vector embedding models (Word2Vec, Doc2Vec, GloVe, and FastText) are employed, using cosine similarity and distance metrics. RESULTS The data pipeline and tool were tested with 1660 patient clinical and dosimetry records resulting in 504 180 RDF (Resource Description Framework) tuples and visualized data relationships using graph-based representations. Patient similarity analysis using embedding models showed that the Word2Vec model had the highest mean cosine similarity, while the GloVe model exhibited more compact embeddings with lower Euclidean and Manhattan distances. CONCLUSIONS The framework and tools described support the development of a RO-LHS. By integrating diverse data sources and facilitating data discovery and analysis, they contribute to continuous learning and improvement in patient care. The tools enhance the quality of care by enabling the identification of cohorts, clinical decision support, and the development of clinical studies and machine learning programs in radiation oncology.
Collapse
Affiliation(s)
- Rishabh Kapoor
- Department of Radiation OncologyVirginia Commonwealth UniversityRichmondVirginiaUSA
| | - William C Sleeman
- Department of Radiation OncologyVirginia Commonwealth UniversityRichmondVirginiaUSA
| | - Preetam Ghosh
- Department of Radiation OncologyVirginia Commonwealth UniversityRichmondVirginiaUSA
| | - Jatinder Palta
- Department of Radiation OncologyVirginia Commonwealth UniversityRichmondVirginiaUSA
| |
Collapse
|
6
|
Lempart M, Scherman J, Nilsson MP, Jamtheim Gustafsson C. Deep learning-based classification of organs at risk and delineation guideline in pelvic cancer radiation therapy. J Appl Clin Med Phys 2023; 24:e14022. [PMID: 37177830 PMCID: PMC10476996 DOI: 10.1002/acm2.14022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Revised: 04/13/2023] [Accepted: 04/19/2023] [Indexed: 05/15/2023] Open
Abstract
Deep learning (DL) models for radiation therapy (RT) image segmentation require accurately annotated training data. Multiple organ delineation guidelines exist; however, information on the used guideline is not provided with the delineation. Extraction of training data with coherent guidelines can therefore be challenging. We present a supervised classification method for pelvis structure delineations where bowel cavity, femoral heads, bladder, and rectum data, with two guidelines, were classified. The impact on DL-based segmentation quality using mixed guideline training data was also demonstrated. Bowel cavity was manually delineated on CT images for anal cancer patients (n = 170) according to guidelines Devisetty and RTOG. The DL segmentation quality from using training data with coherent or mixed guidelines was investigated. A supervised 3D squeeze-and-excite SENet-154 model was trained to classify two bowel cavity delineation guidelines. In addition, a pelvis CT dataset with manual delineations from prostate cancer patients (n = 1854) was used where data with an alternative guideline for femoral heads, rectum, and bladder were generated using commercial software. The model was evaluated on internal (n = 200) and external test data (n = 99). By using mixed, compared to coherent, delineation guideline training data mean DICE score decreased 3% units, mean Hausdorff distance (95%) increased 5 mm and mean surface distance (MSD) increased 1 mm. The classification of bowel cavity test data achieved 99.8% unweighted classification accuracy, 99.9% macro average precision, 97.2% macro average recall, and 98.5% macro average F1. Corresponding metrics for the pelvis internal test data were all 99% or above and for the external pelvis test data they were 96.3%, 96.6%, 93.3%, and 94.6%. Impaired segmentation performance was observed for training data with mixed guidelines. The DL delineation classification models achieved excellent results on internal and external test data. This can facilitate automated guideline-specific data extraction while avoiding the need for consistent and correct structure labels.
Collapse
Affiliation(s)
- Michael Lempart
- Radiation Physics, Department of HematologyOncology, and Radiation PhysicsSkåne University HospitalLundSweden
- Department of Translational MedicineMedical Radiation PhysicsLund UniversityMalmöSweden
| | - Jonas Scherman
- Radiation Physics, Department of HematologyOncology, and Radiation PhysicsSkåne University HospitalLundSweden
| | - Martin P. Nilsson
- Department of HematologyOncology, and Radiation PhysicsSkåne University HospitalLundSweden
| | - Christian Jamtheim Gustafsson
- Radiation Physics, Department of HematologyOncology, and Radiation PhysicsSkåne University HospitalLundSweden
- Department of Translational MedicineMedical Radiation PhysicsLund UniversityMalmöSweden
| |
Collapse
|
7
|
Haidar A, Field M, Batumalai V, Cloak K, Al Mouiee D, Chlap P, Huang X, Chin V, Aly F, Carolan M, Sykes J, Vinod SK, Delaney GP, Holloway L. Standardising Breast Radiotherapy Structure Naming Conventions: A Machine Learning Approach. Cancers (Basel) 2023; 15:cancers15030564. [PMID: 36765523 PMCID: PMC9913464 DOI: 10.3390/cancers15030564] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Revised: 01/01/2023] [Accepted: 01/11/2023] [Indexed: 01/18/2023] Open
Abstract
In progressing the use of big data in health systems, standardised nomenclature is required to enable data pooling and analyses. In many radiotherapy planning systems and their data archives, target volumes (TV) and organ-at-risk (OAR) structure nomenclature has not been standardised. Machine learning (ML) has been utilised to standardise volumes nomenclature in retrospective datasets. However, only subsets of the structures have been targeted. Within this paper, we proposed a new approach for standardising all the structures nomenclature by using multi-modal artificial neural networks. A cohort consisting of 1613 breast cancer patients treated with radiotherapy was identified from Liverpool & Macarthur Cancer Therapy Centres, NSW, Australia. Four types of volume characteristics were generated to represent each target and OAR volume: textual features, geometric features, dosimetry features, and imaging data. Five datasets were created from the original cohort, the first four represented different subsets of volumes and the last one represented the whole list of volumes. For each dataset, 15 sets of combinations of features were generated to investigate the effect of using different characteristics on the standardisation performance. The best model reported 99.416% classification accuracy over the hold-out sample when used to standardise all the nomenclatures in a breast cancer radiotherapy plan into 21 classes. Our results showed that ML based automation methods can be used for standardising naming conventions in a radiotherapy plan taking into consideration the inclusion of multiple modalities to better represent each volume.
Collapse
Affiliation(s)
- Ali Haidar
- Ingham Institute for Applied Medical Research, Liverpool, NSW 2170, Australia
- Liverpool and Macarthur Cancer Therapy Centres, Liverpool, NSW 2170, Australia
- South Western Sydney Clinical School, University of New South Wales, Liverpool, NSW 2170, Australia
- Correspondence: or
| | - Matthew Field
- Ingham Institute for Applied Medical Research, Liverpool, NSW 2170, Australia
- Liverpool and Macarthur Cancer Therapy Centres, Liverpool, NSW 2170, Australia
- South Western Sydney Clinical School, University of New South Wales, Liverpool, NSW 2170, Australia
| | - Vikneswary Batumalai
- South Western Sydney Clinical School, University of New South Wales, Liverpool, NSW 2170, Australia
- GenesisCare, Alexandria, NSW 2015, Australia
| | - Kirrily Cloak
- Ingham Institute for Applied Medical Research, Liverpool, NSW 2170, Australia
- Liverpool and Macarthur Cancer Therapy Centres, Liverpool, NSW 2170, Australia
- South Western Sydney Clinical School, University of New South Wales, Liverpool, NSW 2170, Australia
| | - Daniel Al Mouiee
- Ingham Institute for Applied Medical Research, Liverpool, NSW 2170, Australia
- Liverpool and Macarthur Cancer Therapy Centres, Liverpool, NSW 2170, Australia
- South Western Sydney Clinical School, University of New South Wales, Liverpool, NSW 2170, Australia
| | - Phillip Chlap
- Ingham Institute for Applied Medical Research, Liverpool, NSW 2170, Australia
- Liverpool and Macarthur Cancer Therapy Centres, Liverpool, NSW 2170, Australia
- South Western Sydney Clinical School, University of New South Wales, Liverpool, NSW 2170, Australia
| | - Xiaoshui Huang
- Ingham Institute for Applied Medical Research, Liverpool, NSW 2170, Australia
- Liverpool and Macarthur Cancer Therapy Centres, Liverpool, NSW 2170, Australia
- University of Sydney, Camperdown, NSW 2006, Australia
| | - Vicky Chin
- Ingham Institute for Applied Medical Research, Liverpool, NSW 2170, Australia
- Liverpool and Macarthur Cancer Therapy Centres, Liverpool, NSW 2170, Australia
- South Western Sydney Clinical School, University of New South Wales, Liverpool, NSW 2170, Australia
| | - Farhannah Aly
- Ingham Institute for Applied Medical Research, Liverpool, NSW 2170, Australia
- Liverpool and Macarthur Cancer Therapy Centres, Liverpool, NSW 2170, Australia
- South Western Sydney Clinical School, University of New South Wales, Liverpool, NSW 2170, Australia
| | - Martin Carolan
- Illawarra Cancer Care Center, Wollongong, NSW 2522, Australia
- University of Wollongong, Wollongong, NSW 2522, Australia
| | - Jonathan Sykes
- University of Sydney, Camperdown, NSW 2006, Australia
- Blacktown Hospital, Blacktown, NSW 2148, Australia
| | - Shalini K. Vinod
- Ingham Institute for Applied Medical Research, Liverpool, NSW 2170, Australia
- Liverpool and Macarthur Cancer Therapy Centres, Liverpool, NSW 2170, Australia
- South Western Sydney Clinical School, University of New South Wales, Liverpool, NSW 2170, Australia
| | - Geoffrey P. Delaney
- Ingham Institute for Applied Medical Research, Liverpool, NSW 2170, Australia
- Liverpool and Macarthur Cancer Therapy Centres, Liverpool, NSW 2170, Australia
- South Western Sydney Clinical School, University of New South Wales, Liverpool, NSW 2170, Australia
| | - Lois Holloway
- Ingham Institute for Applied Medical Research, Liverpool, NSW 2170, Australia
- Liverpool and Macarthur Cancer Therapy Centres, Liverpool, NSW 2170, Australia
- South Western Sydney Clinical School, University of New South Wales, Liverpool, NSW 2170, Australia
- University of Sydney, Camperdown, NSW 2006, Australia
| |
Collapse
|
8
|
Recent Applications of Artificial Intelligence in Radiotherapy: Where We Are and Beyond. APPLIED SCIENCES-BASEL 2022. [DOI: 10.3390/app12073223] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
In recent decades, artificial intelligence (AI) tools have been applied in many medical fields, opening the possibility of finding novel solutions for managing very complex and multifactorial problems, such as those commonly encountered in radiotherapy (RT). We conducted a PubMed and Scopus search to identify the AI application field in RT limited to the last four years. In total, 1824 original papers were identified, and 921 were analyzed by considering the phase of the RT workflow according to the applied AI approaches. AI permits the processing of large quantities of information, data, and images stored in RT oncology information systems, a process that is not manageable for individuals or groups. AI allows the iterative application of complex tasks in large datasets (e.g., delineating normal tissues or finding optimal planning solutions) and might support the entire community working in the various sectors of RT, as summarized in this overview. AI-based tools are now on the roadmap for RT and have been applied to the entire workflow, mainly for segmentation, the generation of synthetic images, and outcome prediction. Several concerns were raised, including the need for harmonization while overcoming ethical, legal, and skill barriers.
Collapse
|
9
|
Jamtheim Gustafsson C, Lempart M, Swärd J, Persson E, Nyholm T, Thellenberg Karlsson C, Scherman J. Deep learning-based classification and structure name standardization for organ at risk and target delineations in prostate cancer radiotherapy. J Appl Clin Med Phys 2021; 22:51-63. [PMID: 34623738 PMCID: PMC8664152 DOI: 10.1002/acm2.13446] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2021] [Revised: 09/16/2021] [Accepted: 09/24/2021] [Indexed: 11/12/2022] Open
Abstract
Radiotherapy (RT) datasets can suffer from variations in annotation of organ at risk (OAR) and target structures. Annotation standards exist, but their description for prostate targets is limited. This restricts the use of such data for supervised machine learning purposes as it requires properly annotated data. The aim of this work was to develop a modality independent deep learning (DL) model for automatic classification and annotation of prostate RT DICOM structures. Delineated prostate organs at risk (OAR), support- and target structures (gross tumor volume [GTV]/clinical target volume [CTV]/planning target volume [PTV]), along with or without separate vesicles and/or lymph nodes, were extracted as binary masks from 1854 patients. An image modality independent 2D InceptionResNetV2 classification network was trained with varying amounts of training data using four image input channels. Channel 1-3 consisted of orthogonal 2D projections from each individual binary structure. The fourth channel contained a summation of the other available binary structure masks. Structure classification performance was assessed in independent CT (n = 200 pat) and magnetic resonance imaging (MRI) (n = 40 pat) test datasets and an external CT (n = 99 pat) dataset from another clinic. A weighted classification accuracy of 99.4% was achieved during training. The unweighted classification accuracy and the weighted average F1 score among different structures in the CT test dataset were 98.8% and 98.4% and 98.6% and 98.5% for the MRI test dataset, respectively. The external CT dataset yielded the corresponding results 98.4% and 98.7% when analyzed for trained structures only, and results from the full dataset yielded 79.6% and 75.2%. Most misclassifications in the external CT dataset occurred due to multiple CTVs and PTVs being fused together, which was not included in the training data. Our proposed DL-based method for automated renaming and standardization of prostate radiotherapy annotations shows great potential. Clinic specific contouring standards however need to be represented in the training data for successful use. Source code is available at https://github.com/jamtheim/DicomRTStructRenamerPublic.
Collapse
Affiliation(s)
- Christian Jamtheim Gustafsson
- Department of Hematology Oncology and Radiation Physics, Skåne University Hospital, Lund, Sweden.,Department of Translational Sciences, Medical Radiation Physics, Lund University, Malmö, Sweden
| | - Michael Lempart
- Department of Hematology Oncology and Radiation Physics, Skåne University Hospital, Lund, Sweden.,Department of Translational Sciences, Medical Radiation Physics, Lund University, Malmö, Sweden
| | - Johan Swärd
- Centre for Mathematical Sciences, Mathematical Statistics, Lund University, Lund, Sweden
| | - Emilia Persson
- Department of Hematology Oncology and Radiation Physics, Skåne University Hospital, Lund, Sweden.,Department of Translational Sciences, Medical Radiation Physics, Lund University, Malmö, Sweden
| | - Tufve Nyholm
- Department of Radiation Sciences, Radiation Physics, Umeå University, Umeå, Sweden
| | | | - Jonas Scherman
- Department of Hematology Oncology and Radiation Physics, Skåne University Hospital, Lund, Sweden
| |
Collapse
|
10
|
Kapoor R, Sleeman WC, Nalluri JJ, Turner P, Bose P, Cherevko A, Srinivasan S, Syed K, Ghosh P, Hagan M, Palta JR. Automated data abstraction for quality surveillance and outcome assessment in radiation oncology. J Appl Clin Med Phys 2021; 22:177-187. [PMID: 34101349 PMCID: PMC8292697 DOI: 10.1002/acm2.13308] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2021] [Revised: 04/22/2021] [Accepted: 05/10/2021] [Indexed: 11/24/2022] Open
Abstract
Rigorous radiotherapy quality surveillance and comprehensive outcome assessment require electronic capture and automatic abstraction of clinical, radiation treatment planning, and delivery data. We present the design and implementation framework of an integrated data abstraction, aggregation, and storage, curation, and analytics software: the Health Information Gateway and Exchange (HINGE), which collates data for cancer patients receiving radiotherapy. The HINGE software abstracts structured DICOM‐RT data from the treatment planning system (TPS), treatment data from the treatment management system (TMS), and clinical data from the electronic health records (EHRs). HINGE software has disease site‐specific “Smart” templates that facilitate the entry of relevant clinical information by physicians and clinical staff in a discrete manner as part of the routine clinical documentation. Radiotherapy data abstracted from these disparate sources and the smart templates are processed for quality and outcome assessment. The predictive data analyses are done on using well‐defined clinical and dosimetry quality measures defined by disease site experts in radiation oncology. HINGE application software connects seamlessly to the local IT/medical infrastructure via interfaces and cloud services and performs data extraction and aggregation functions without human intervention. It provides tools to assess variations in radiation oncology practices and outcomes and determines gaps in radiotherapy quality delivered by each provider.
Collapse
Affiliation(s)
- Rishabh Kapoor
- Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA, USA.,National Radiation Oncology Program, US Veterans Healthcare Administration, Richmond, VA, USA
| | - William C Sleeman
- Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA, USA.,National Radiation Oncology Program, US Veterans Healthcare Administration, Richmond, VA, USA
| | - Joseph J Nalluri
- Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA, USA.,National Radiation Oncology Program, US Veterans Healthcare Administration, Richmond, VA, USA
| | - Paul Turner
- Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA, USA.,National Radiation Oncology Program, US Veterans Healthcare Administration, Richmond, VA, USA
| | - Priyankar Bose
- Department of Computer Science, Virginia Commonwealth University, Richmond, VA, USA
| | - Andrii Cherevko
- Department of Computer Science, Virginia Commonwealth University, Richmond, VA, USA
| | - Sriram Srinivasan
- Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA, USA.,National Radiation Oncology Program, US Veterans Healthcare Administration, Richmond, VA, USA
| | - Khajamoinuddin Syed
- Department of Computer Science, Virginia Commonwealth University, Richmond, VA, USA
| | - Preetam Ghosh
- Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA, USA.,Department of Computer Science, Virginia Commonwealth University, Richmond, VA, USA
| | - Michael Hagan
- Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA, USA.,National Radiation Oncology Program, US Veterans Healthcare Administration, Richmond, VA, USA
| | - Jatinder R Palta
- Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA, USA.,National Radiation Oncology Program, US Veterans Healthcare Administration, Richmond, VA, USA
| |
Collapse
|
11
|
Syed K, Sleeman WC, Hagan M, Palta J, Kapoor R, Ghosh P. Multi-View Data Integration Methods for Radiotherapy Structure Name Standardization. Cancers (Basel) 2021; 13:cancers13081796. [PMID: 33918716 PMCID: PMC8070367 DOI: 10.3390/cancers13081796] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2021] [Revised: 03/28/2021] [Accepted: 04/05/2021] [Indexed: 11/24/2022] Open
Abstract
Simple Summary Structure names associated with radiotherapy treatments need standardization to develop data pipelines enabling personalized treatment plans. Automatic classification of structure names based on the currently available TG-263 nomenclature can help with data aggregation from both retrospective and future data sources. The aim of our proposed machine learning-based data integration methods is to achieve highly accurate structure name classification to automate the data aggregation process. Our multi-view models can overcome the challenges of integrating different data types associated with radiotherapy structures, such as the physician-given text labels and geometric or image data. The models exhibited high accuracy when tested on multi-center and multi-institutional lung and prostate cancer patients data and outperformed the models built on any single data type. This highlights the importance of combining different types of data in building generalizable models for structure name standardization. Abstract Standardization of radiotherapy structure names is essential for developing data-driven personalized radiotherapy treatment plans. Different types of data are associated with radiotherapy structures, such as the physician-given text labels, geometric (image) data, and Dose-Volume Histograms (DVH). Prior work on structure name standardization used just one type of data. We present novel approaches to integrate complementary types (views) of structure data to build better-performing machine learning models. We present two methods, namely (a) intermediate integration and (b) late integration, to combine physician-given textual structure name features and geometric information of structures. The dataset consisted of 709 prostate cancer and 752 lung cancer patients across 40 radiotherapy centers administered by the U.S. Veterans Health Administration (VA) and the Department of Radiation Oncology, Virginia Commonwealth University (VCU). We used randomly selected data from 30 centers for training and ten centers for testing. We also used the VCU data for testing. We observed that the intermediate integration approach outperformed the models with a single view of the dataset, while late integration showed comparable performance with single-view results. Thus, we demonstrate that combining different views (types of data) helps build better models for structure name standardization to enable big data analytics in radiation oncology.
Collapse
Affiliation(s)
- Khajamoinuddin Syed
- Department of Computer Science, Virginia Commonwealth University, Richmond, VA 23284, USA; (W.C.S.IV); (P.G.)
- Correspondence:
| | - William C. Sleeman
- Department of Computer Science, Virginia Commonwealth University, Richmond, VA 23284, USA; (W.C.S.IV); (P.G.)
- Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA 23298, USA; (M.H.); (J.P.); (R.K.)
| | - Michael Hagan
- Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA 23298, USA; (M.H.); (J.P.); (R.K.)
- National Radiation Oncology Program, Department of Veteran Affairs, Richmond, VA 23249, USA
| | - Jatinder Palta
- Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA 23298, USA; (M.H.); (J.P.); (R.K.)
- National Radiation Oncology Program, Department of Veteran Affairs, Richmond, VA 23249, USA
| | - Rishabh Kapoor
- Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA 23298, USA; (M.H.); (J.P.); (R.K.)
- National Radiation Oncology Program, Department of Veteran Affairs, Richmond, VA 23249, USA
| | - Preetam Ghosh
- Department of Computer Science, Virginia Commonwealth University, Richmond, VA 23284, USA; (W.C.S.IV); (P.G.)
| |
Collapse
|