1
|
Bordigoni B, Trivellato S, Pellegrini R, Meregalli S, Bonetto E, Belmonte M, Castellano M, Panizza D, Arcangeli S, De Ponti E. Automated segmentation in pelvic radiotherapy: A comprehensive evaluation of ATLAS-, machine learning-, and deep learning-based models. Phys Med 2024; 125:104486. [PMID: 39098106 DOI: 10.1016/j.ejmp.2024.104486] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/16/2023] [Revised: 06/20/2024] [Accepted: 07/17/2024] [Indexed: 08/06/2024] Open
Abstract
Artificial intelligence can standardize and automatize highly demanding procedures, such as manual segmentation, especially in an anatomical site as common as the pelvis. This study investigated four automated segmentation tools on computed tomography (CT) images in female and male pelvic radiotherapy (RT) starting from simpler and well-known atlas-based methods to the most recent neural networks-based algorithms. The evaluation included quantitative, qualitative and time efficiency assessments. A mono-institutional consecutive series of 40 cervical cancer and 40 prostate cancer structure sets were retrospectively selected. After a preparatory phase, the remaining 20 testing sets per each site were auto-segmented by the atlas-based model STAPLE, a Random Forest-based model, and two Deep Learning-based tools (DL), MVision and LimbusAI. Setting manual segmentation as the Ground Truth, 200 structure sets were compared in terms of Dice Similarity Coefficient (DSC), Hausdorff Distance (HD), and Distance-to-Agreement Portion (DAP). Automated segmentation and manual correction durations were recorded. Expert clinicians performed a qualitative evaluation. In cervical cancer CTs, DL outperformed the other tools with higher quantitative metrics, qualitative scores, and shorter correction times. On the other hand, in prostate cancer CTs, the performance across all the analyzed tools was comparable in terms of both quantitative and qualitative metrics. Such discrepancy in performance outcome could be explained by the wide range of anatomical variability in cervical cancer with respect to the strict bladder and rectum filling preparation in prostate Stereotactic Body Radiation Therapy (SBRT). Decreasing segmentation times can reduce the burden of pelvic radiation therapy routine in an automated workflow.
Collapse
Affiliation(s)
- B Bordigoni
- Medical Physics, Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy
| | - S Trivellato
- Medical Physics, Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy
| | | | - S Meregalli
- Radiation Oncology, Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy
| | - E Bonetto
- Radiation Oncology, Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy
| | - M Belmonte
- School of Medicine and Surgery, University of Milano Bicocca, Milano, Italy; Radiation Oncology, Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy
| | - M Castellano
- School of Medicine and Surgery, University of Milano Bicocca, Milano, Italy; Radiation Oncology, Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy
| | - D Panizza
- Medical Physics, Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy; School of Medicine and Surgery, University of Milano Bicocca, Milano, Italy
| | - S Arcangeli
- School of Medicine and Surgery, University of Milano Bicocca, Milano, Italy; Radiation Oncology, Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy.
| | - E De Ponti
- Medical Physics, Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy; School of Medicine and Surgery, University of Milano Bicocca, Milano, Italy
| |
Collapse
|
2
|
Bakx N, Van der Sangen M, Theuws J, Bluemink J, Hurkmans C. Comparison of the use of a clinically implemented deep learning segmentation model with the simulated study setting for breast cancer patients receiving radiotherapy. Acta Oncol 2024; 63:477-481. [PMID: 38899395 PMCID: PMC11332522 DOI: 10.2340/1651-226x.2024.34986] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Accepted: 04/24/2024] [Indexed: 06/21/2024]
Abstract
BACKGROUND Deep learning (DL) models for auto-segmentation in radiotherapy have been extensively studied in retrospective and pilot settings. However, these studies might not reflect the clinical setting. This study compares the use of a clinically implemented in-house trained DL segmentation model for breast cancer to a previously performed pilot study to assess possible differences in performance or acceptability. MATERIAL AND METHODS Sixty patients with whole breast radiotherapy, with or without an indication for locoregional radiotherapy were included. Structures were qualitatively scored by radiotherapy technologists and radiation oncologists. Quantitative evaluation was performed using dice-similarity coefficient (DSC), 95th percentile of Hausdorff Distance (95%HD) and surface DSC (sDSC), and time needed for generating, checking, and correcting structures was measured. RESULTS Ninety-three percent of all contours in clinic were scored as clinically acceptable or usable as a starting point, comparable to 92% achieved in the pilot study. Compared to the pilot study, no significant changes in time reduction were achieved for organs at risks (OARs). For target volumes, significantly more time was needed compared to the pilot study for patients including lymph node levels 1-4, although time reduction was still 33% compared to manual segmentation. Almost all contours have better DSC and 95%HD than inter-observer variations. Only CTVn4 scored worse for both metrics, and the thyroid had a higher 95%HD value. INTERPRETATION The use of the DL model in clinical practice is comparable to the pilot study, showing high acceptability rates and time reduction.
Collapse
Affiliation(s)
- Nienke Bakx
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, The Netherlands.
| | | | - Jacqueline Theuws
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, The Netherlands
| | - Johanna Bluemink
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, The Netherlands
| | - Coen Hurkmans
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, The Netherlands; Technical University Eindhoven, Departments of Applied Physics and Electrical Engineering, Eindhoven, The Netherlands
| |
Collapse
|
3
|
Kraus AC, Iqbal Z, Cardan RA, Popple RA, Stanley DN, Shen S, Pogue JA, Wu X, Lee K, Marcrom S, Cardenas CE. Prospective Evaluation of Automated Contouring for CT-Based Brachytherapy for Gynecologic Malignancies. Adv Radiat Oncol 2024; 9:101417. [PMID: 38435965 PMCID: PMC10906166 DOI: 10.1016/j.adro.2023.101417] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Accepted: 11/30/2023] [Indexed: 03/05/2024] Open
Abstract
Purpose The use of deep learning to auto-contour organs at risk (OARs) in gynecologic radiation treatment is well established. Yet, there is limited data investigating the prospective use of auto-contouring in clinical practice. In this study, we assess the accuracy and efficiency of auto-contouring OARs for computed tomography-based brachytherapy treatment planning of gynecologic malignancies. Methods and Materials An inhouse contouring tool automatically delineated 5 OARs in gynecologic radiation treatment planning: the bladder, small bowel, sigmoid, rectum, and urethra. Accuracy of each auto-contour was evaluated using a 5-point Likert scale: a score of 5 indicated the contour could be used without edits, while a score of 1 indicated the contour was unusable. During scoring, automated contours were edited and subsequently used for treatment planning. Dice similarity coefficient, mean surface distance, 95% Hausdorff distance, Hausdorff distance, and dosimetric changes between original and edited contours were calculated. Contour approval time and total planning time of a prospective auto-contoured (AC) cohort were compared with times from a retrospective manually contoured (MC) cohort. Results Thirty AC cases from January 2022 to July 2022 and 31 MC cases from July 2021 to January 2022 were included. The mean (±SD) Likert score for each OAR was the following: bladder 4.77 (±0.58), small bowel 3.96 (±0.91), sigmoid colon 3.92 (±0.81), rectum 4.6 (±0.71), and urethra 4.27 (±0.78). No ACs required major edits. All OARs had a mean Dice similarity coefficient > 0.86, mean surface distance < 0.48 mm, 95% Hausdorff distance < 3.2 mm, and Hausdorff distance < 10.32 mm between original and edited contours. There was no significant difference in dose-volume histogram metrics (D2.0 cc/D0.1 cc) between original and edited contours (P values > .05). The average time to plan approval in the AC cohort was 19% less than the MC cohort. (AC vs MC, 117.0 + 18.0 minutes vs 144.9 ± 64.5 minutes, P = .045). Conclusions Automated contouring is useful and accurate in clinical practice. Auto-contouring OARs streamlines radiation treatment workflows and decreases time required to design and approve gynecologic brachytherapy plans.
Collapse
Affiliation(s)
- Abigayle C. Kraus
- Heersink School of Medicine, University of Alabama at Birmingham, Birmingham, Alabama
| | - Zohaib Iqbal
- Department of Radiation Oncology, University of Alabama at Birmingham, Birmingham, Alabama
| | - Rex A. Cardan
- Department of Radiation Oncology, University of Alabama at Birmingham, Birmingham, Alabama
| | - Richard A. Popple
- Department of Radiation Oncology, University of Alabama at Birmingham, Birmingham, Alabama
| | - Dennis N. Stanley
- Department of Radiation Oncology, University of Alabama at Birmingham, Birmingham, Alabama
| | - Sui Shen
- Department of Radiation Oncology, University of Alabama at Birmingham, Birmingham, Alabama
| | - Joel A. Pogue
- Department of Radiation Oncology, University of Alabama at Birmingham, Birmingham, Alabama
| | - Xingen Wu
- Department of Radiation Oncology, University of Alabama at Birmingham, Birmingham, Alabama
| | - Kevin Lee
- Department of Radiation Oncology, University of Alabama at Birmingham, Birmingham, Alabama
| | - Samuel Marcrom
- Department of Radiation Oncology, University of Alabama at Birmingham, Birmingham, Alabama
| | - Carlos E. Cardenas
- Department of Radiation Oncology, University of Alabama at Birmingham, Birmingham, Alabama
| |
Collapse
|
4
|
Dei D, Lambri N, Crespi L, Brioso RC, Loiacono D, Clerici E, Bellu L, De Philippis C, Navarria P, Bramanti S, Carlo-Stella C, Rusconi R, Reggiori G, Tomatis S, Scorsetti M, Mancosu P. Deep learning and atlas-based models to streamline the segmentation workflow of total marrow and lymphoid irradiation. LA RADIOLOGIA MEDICA 2024; 129:515-523. [PMID: 38308062 DOI: 10.1007/s11547-024-01760-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Accepted: 01/03/2024] [Indexed: 02/04/2024]
Abstract
PURPOSE To improve the workflow of total marrow and lymphoid irradiation (TMLI) by enhancing the delineation of organs at risk (OARs) and clinical target volume (CTV) using deep learning (DL) and atlas-based (AB) segmentation models. MATERIALS AND METHODS Ninety-five TMLI plans optimized in our institute were analyzed. Two commercial DL software were tested for segmenting 18 OARs. An AB model for lymph node CTV (CTV_LN) delineation was built using 20 TMLI patients. The AB model was evaluated on 20 independent patients, and a semiautomatic approach was tested by correcting the automatic contours. The generated OARs and CTV_LN contours were compared to manual contours in terms of topological agreement, dose statistics, and time workload. A clinical decision tree was developed to define a specific contouring strategy for each OAR. RESULTS The two DL models achieved a median [interquartile range] dice similarity coefficient (DSC) of 0.84 [0.71;0.93] and 0.85 [0.70;0.93] across the OARs. The absolute median Dmean difference between manual and the two DL models was 2.0 [0.7;6.6]% and 2.4 [0.9;7.1]%. The AB model achieved a median DSC of 0.70 [0.66;0.74] for CTV_LN delineation, increasing to 0.94 [0.94;0.95] after manual revision, with minimal Dmean differences. Since September 2022, our institution has implemented DL and AB models for all TMLI patients, reducing from 5 to 2 h the time required to complete the entire segmentation process. CONCLUSION DL models can streamline the TMLI contouring process of OARs. Manual revision is still necessary for lymph node delineation using AB models.
Collapse
Affiliation(s)
- Damiano Dei
- Department of Biomedical Sciences, Humanitas University, Via Rita Levi Montalcini 4, 20072, Pieve Emanuele, Milan, Italy
- Department of Radiotherapy and Radiosurgery, IRCCS Humanitas Research Hospital, Via Manzoni 56, 20089, Rozzano, Milan, Italy
| | - Nicola Lambri
- Department of Biomedical Sciences, Humanitas University, Via Rita Levi Montalcini 4, 20072, Pieve Emanuele, Milan, Italy.
- Department of Radiotherapy and Radiosurgery, IRCCS Humanitas Research Hospital, Via Manzoni 56, 20089, Rozzano, Milan, Italy.
| | - Leonardo Crespi
- Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milan, Italy
- Health Data Science Centre, Human Technopole, Milan, Italy
| | - Ricardo Coimbra Brioso
- Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milan, Italy
| | - Daniele Loiacono
- Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milan, Italy
| | - Elena Clerici
- Department of Radiotherapy and Radiosurgery, IRCCS Humanitas Research Hospital, Via Manzoni 56, 20089, Rozzano, Milan, Italy
| | - Luisa Bellu
- Department of Biomedical Sciences, Humanitas University, Via Rita Levi Montalcini 4, 20072, Pieve Emanuele, Milan, Italy
- Department of Radiotherapy and Radiosurgery, IRCCS Humanitas Research Hospital, Via Manzoni 56, 20089, Rozzano, Milan, Italy
| | - Chiara De Philippis
- Department of Oncology and Hematology, IRCCS Humanitas Research Hospital, Via Manzoni 56, 20089, Rozzano, Milan, Italy
| | - Pierina Navarria
- Department of Radiotherapy and Radiosurgery, IRCCS Humanitas Research Hospital, Via Manzoni 56, 20089, Rozzano, Milan, Italy
| | - Stefania Bramanti
- Department of Oncology and Hematology, IRCCS Humanitas Research Hospital, Via Manzoni 56, 20089, Rozzano, Milan, Italy
| | - Carmelo Carlo-Stella
- Department of Biomedical Sciences, Humanitas University, Via Rita Levi Montalcini 4, 20072, Pieve Emanuele, Milan, Italy
- Department of Oncology and Hematology, IRCCS Humanitas Research Hospital, Via Manzoni 56, 20089, Rozzano, Milan, Italy
| | - Roberto Rusconi
- Department of Biomedical Sciences, Humanitas University, Via Rita Levi Montalcini 4, 20072, Pieve Emanuele, Milan, Italy
- Department of Radiotherapy and Radiosurgery, IRCCS Humanitas Research Hospital, Via Manzoni 56, 20089, Rozzano, Milan, Italy
| | - Giacomo Reggiori
- Department of Biomedical Sciences, Humanitas University, Via Rita Levi Montalcini 4, 20072, Pieve Emanuele, Milan, Italy
- Department of Radiotherapy and Radiosurgery, IRCCS Humanitas Research Hospital, Via Manzoni 56, 20089, Rozzano, Milan, Italy
| | - Stefano Tomatis
- Department of Radiotherapy and Radiosurgery, IRCCS Humanitas Research Hospital, Via Manzoni 56, 20089, Rozzano, Milan, Italy
| | - Marta Scorsetti
- Department of Biomedical Sciences, Humanitas University, Via Rita Levi Montalcini 4, 20072, Pieve Emanuele, Milan, Italy
- Department of Radiotherapy and Radiosurgery, IRCCS Humanitas Research Hospital, Via Manzoni 56, 20089, Rozzano, Milan, Italy
| | - Pietro Mancosu
- Department of Radiotherapy and Radiosurgery, IRCCS Humanitas Research Hospital, Via Manzoni 56, 20089, Rozzano, Milan, Italy
| |
Collapse
|
5
|
Court LE, Aggarwal A, Jhingran A, Naidoo K, Netherton T, Olanrewaju A, Peterson C, Parkes J, Simonds H, Trauernicht C, Zhang L, Beadle BM. Artificial Intelligence-Based Radiotherapy Contouring and Planning to Improve Global Access to Cancer Care. JCO Glob Oncol 2024; 10:e2300376. [PMID: 38484191 PMCID: PMC10954080 DOI: 10.1200/go.23.00376] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2023] [Revised: 11/22/2023] [Accepted: 01/22/2024] [Indexed: 03/19/2024] Open
Abstract
PURPOSE Increased automation has been identified as one approach to improving global cancer care. The Radiation Planning Assistant (RPA) is a web-based tool offering automated radiotherapy (RT) contouring and planning to low-resource clinics. In this study, the RPA workflow and clinical acceptability were assessed by physicians around the world. METHODS The RPA output for 75 cases was reviewed by at least three physicians; 31 radiation oncologists at 16 institutions in six countries on five continents reviewed RPA contours and plans for clinical acceptability using a 5-point Likert scale. RESULTS For cervical cancer, RPA plans using bony landmarks were scored as usable as-is in 81% (with minor edits 93%); using soft tissue contours, plans were scored as usable as-is in 79% (with minor edits 96%). For postmastectomy breast cancer, RPA plans were scored as usable as-is in 44% (with minor edits 91%). For whole-brain treatment, RPA plans were scored as usable as-is in 67% (with minor edits 99%). For head/neck cancer, the normal tissue autocontours were acceptable as-is in 89% (with minor edits 97%). The clinical target volumes (CTVs) were acceptable as-is in 40% (with minor edits 93%). The volumetric-modulated arc therapy (VMAT) plans were acceptable as-is in 87% (with minor edits 96%). For cervical cancer, the normal tissue autocontours were acceptable as-is in 92% (with minor edits 99%). The CTVs for cervical cancer were scored as acceptable as-is in 83% (with minor edits 92%). The VMAT plans for cervical cancer were acceptable as-is in 99% (with minor edits 100%). CONCLUSION The RPA, a web-based tool designed to improve access to high-quality RT in low-resource settings, has high rates of clinical acceptability by practicing clinicians around the world. It has significant potential for successful implementation in low-resource clinics.
Collapse
Affiliation(s)
| | - Ajay Aggarwal
- Guy's and St Thomas Hospitals, London, United Kingdom
| | - Anuja Jhingran
- University of Texas MD Anderson Cancer Center, Houston, TX
| | | | | | | | | | | | | | | | - Lifei Zhang
- University of Texas MD Anderson Cancer Center, Houston, TX
| | | |
Collapse
|
6
|
Brooks J, Tryggestad E, Anand A, Beltran C, Foote R, Lucido JJ, Laack NN, Routman D, Patel SH, Seetamsetty S, Moseley D. Knowledge-based quality assurance of a comprehensive set of organ at risk contours for head and neck radiotherapy. Front Oncol 2024; 14:1295251. [PMID: 38487718 PMCID: PMC10937434 DOI: 10.3389/fonc.2024.1295251] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2023] [Accepted: 02/05/2024] [Indexed: 03/17/2024] Open
Abstract
Introduction Manual review of organ at risk (OAR) contours is crucial for creating safe radiotherapy plans but can be time-consuming and error prone. Statistical and deep learning models show the potential to automatically detect improper contours by identifying outliers using large sets of acceptable data (knowledge-based outlier detection) and may be able to assist human reviewers during review of OAR contours. Methods This study developed an automated knowledge-based outlier detection method and assessed its ability to detect erroneous contours for all common head and neck (HN) OAR types used clinically at our institution. We utilized 490 accurate CT-based HN structure sets from unique patients, each with forty-two HN OAR contours when anatomically present. The structure sets were distributed as 80% for training, 10% for validation, and 10% for testing. In addition, 190 and 37 simulated contours containing errors were added to the validation and test sets, respectively. Single-contour features, including location, shape, orientation, volume, and CT number, were used to train three single-contour feature models (z-score, Mahalanobis distance [MD], and autoencoder [AE]). Additionally, a novel contour-to-contour relationship (CCR) model was trained using the minimum distance and volumetric overlap between pairs of OAR contours to quantify overlap and separation. Inferences from single-contour feature models were combined with the CCR model inferences and inferences evaluating the number of disconnected parts in a single contour and then compared. Results In the test dataset, before combination with the CCR model, the area under the curve values were 0.922/0.939/0.939 for the z-score, MD, and AE models respectively for all contours. After combination with CCR model inferences, the z-score, MD, and AE had sensitivities of 0.838/0.892/0.865, specificities of 0.922/0.907/0.887, and balanced accuracies (BA) of 0.880/0.900/0.876 respectively. In the validation dataset, with similar overall performance and no signs of overfitting, model performance for individual OAR types was assessed. The combined AE model demonstrated minimum, median, and maximum BAs of 0.729, 0.908, and 0.980 across OAR types. Discussion Our novel knowledge-based method combines models utilizing single-contour and CCR features to effectively detect erroneous OAR contours across a comprehensive set of 42 clinically used OAR types for HN radiotherapy.
Collapse
Affiliation(s)
- Jamison Brooks
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - Erik Tryggestad
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - Aman Anand
- Department of Radiation Oncology, Mayo Clinic Arizona, Phoenix, AZ, United States
| | - Chris Beltran
- Department of Radiation Oncology, Mayo Clinic Florida, Jacksonville, FL, United States
| | - Robert Foote
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - J. John Lucido
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - Nadia N. Laack
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - David Routman
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - Samir H. Patel
- Department of Radiation Oncology, Mayo Clinic Arizona, Phoenix, AZ, United States
| | - Srinivas Seetamsetty
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| | - Douglas Moseley
- Department of Radiation Oncology, Mayo Clinic Rochester, Rochester, MN, United States
| |
Collapse
|
7
|
Choi MS, Chang JS, Kim K, Kim JH, Kim TH, Kim S, Cha H, Cho O, Choi JH, Kim M, Kim J, Kim TG, Yeo SG, Chang AR, Ahn SJ, Choi J, Kang KM, Kwon J, Koo T, Kim MY, Choi SH, Jeong BK, Jang BS, Jo IY, Lee H, Kim N, Park HJ, Im JH, Lee SW, Cho Y, Lee SY, Chang JH, Chun J, Lee EM, Kim JS, Shin KH, Kim YB. Assessment of deep learning-based auto-contouring on interobserver consistency in target volume and organs-at-risk delineation for breast cancer: Implications for RTQA program in a multi-institutional study. Breast 2024; 73:103599. [PMID: 37992527 PMCID: PMC10700624 DOI: 10.1016/j.breast.2023.103599] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Revised: 10/30/2023] [Accepted: 11/09/2023] [Indexed: 11/24/2023] Open
Abstract
PURPOSE To quantify interobserver variation (IOV) in target volume and organs-at-risk (OAR) contouring across 31 institutions in breast cancer cases and to explore the clinical utility of deep learning (DL)-based auto-contouring in reducing potential IOV. METHODS AND MATERIALS In phase 1, two breast cancer cases were randomly selected and distributed to multiple institutions for contouring six clinical target volumes (CTVs) and eight OAR. In Phase 2, auto-contour sets were generated using a previously published DL Breast segmentation model and were made available for all participants. The difference in IOV of submitted contours in phases 1 and 2 was investigated quantitatively using the Dice similarity coefficient (DSC) and Hausdorff distance (HD). The qualitative analysis involved using contour heat maps to visualize the extent and location of these variations and the required modification. RESULTS Over 800 pairwise comparisons were analysed for each structure in each case. Quantitative phase 2 metrics showed significant improvement in the mean DSC (from 0.69 to 0.77) and HD (from 34.9 to 17.9 mm). Quantitative analysis showed increased interobserver agreement in phase 2, specifically for CTV structures (5-19 %), leading to fewer manual adjustments. Underlying IOV differences causes were reported using a questionnaire and hierarchical clustering analysis based on the volume of CTVs. CONCLUSION DL-based auto-contours improved the contour agreement for OARs and CTVs significantly, both qualitatively and quantitatively, suggesting its potential role in minimizing radiation therapy protocol deviation.
Collapse
Affiliation(s)
- Min Seo Choi
- Department of Radiation Oncology, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Jee Suk Chang
- Department of Radiation Oncology, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Kyubo Kim
- Department of Radiation Oncology, Ewha Womans University College of Medicine, Seoul, Republic of Korea; Department of Radiation Oncology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seongnam, Republic of Korea
| | - Jin Hee Kim
- Department of Radiation Oncology, Dongsan Medical Center, Keimyung University School of Medicine, Daegu, Republic of Korea
| | - Tae Hyung Kim
- Department of Radiation Oncology, Nowon Eulji Medical Center, Eulji University School of Medicine, Seoul, Republic of Korea
| | - Sungmin Kim
- Department of Radiation Oncology, Dong-A University Hospital, Dong-A University College of Medicine, Busan, Republic of Korea
| | - Hyejung Cha
- Department of Radiation Oncology, Yonsei University Wonju College of Medicine, Wonju, Republic of Korea
| | - Oyeon Cho
- Department of Radiation Oncology, Ajou University School of Medicine, Suwon, Republic of Korea
| | - Jin Hwa Choi
- Department of Radiation Oncology, Chung-Ang University Hospital, Seoul, Republic of Korea
| | - Myungsoo Kim
- Department of Radiation Oncology, Incheon St Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Republic of Korea
| | - Juree Kim
- Department of Radiation Oncology, Ilsan CHA Medical Center, CHA University School of Medicine, Goyang, Republic of Korea
| | - Tae Gyu Kim
- Department of Radiation Oncology, Samsung Changwon Hospital, Sungkyunkwan University School of Medicine, Changwon, Republic of Korea
| | - Seung-Gu Yeo
- Department of Radiation Oncology, Soonchunhyang University College of Medicine, Soonchunhyang University Hospital, Bucheon, Republic of Korea
| | - Ah Ram Chang
- Department of Radiation Oncology, Soonchunhyang University College of Medicine, Seoul, Republic of Korea
| | - Sung-Ja Ahn
- Department of Radiation Oncology, Chonnam National University Medical School, Gwangju, Republic of Korea
| | - Jinhyun Choi
- Department of Radiation Oncology, Jeju National University Hospital, Jeju University College of Medicine, Republic of Korea
| | - Ki Mun Kang
- Gyeongsang National University Changwon Hospital, Gyeongsang National University College of Medicine, Jinju, Republic of Korea
| | - Jeanny Kwon
- Department of Radiation Oncology, Chungnam National University School of Medicine, Daejeon, Republic of Korea
| | - Taeryool Koo
- Department of Radiation Oncology, Hallym University Sacred Heart Hospital, Hallym University College of Medicine, Anyang, Republic of Korea
| | - Mi Young Kim
- Department of Radiation Oncology, Kyungpook National University Chilgok Hospital, Daegu, Republic of Korea
| | - Seo Hee Choi
- Department of Radiation Oncology, Yongin Severance Hospital, Yonsei University College of Medicine, Yongin, Republic of Korea
| | - Bae Kwon Jeong
- Department of Radiation Oncology, Gyeongsang National University Hospital, Gyeongsang National University College of Medicine, Jinju, Republic of Korea
| | - Bum-Sup Jang
- Department of Radiation Oncology, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - In Young Jo
- Department of Radiation Oncology, Soonchunhyang University Hospital, Cheonan, Republic of Korea
| | - Hyebin Lee
- Department of Radiation Oncology, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
| | - Nalee Kim
- Department of Radiation Oncology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
| | - Hae Jin Park
- Department of Radiation Oncology, Hanyang University College of Medicine, Seoul, Republic of Korea
| | - Jung Ho Im
- Department of Radiation Oncology, CHA Bundang Medical Center, CHA University School of Medicine, Seongnam, Republic of Korea
| | - Sea-Won Lee
- Department of Radiation Oncology, Eunpyeong St. Mary's Hospital, Catholic University of Korea College of Medicine, Seoul, Republic of Korea
| | - Yeona Cho
- Department of Radiation Oncology, Gangnam Severance Hospital, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Sun Young Lee
- Department of Radiation Oncology, Chonbuk National University Hospital, Jeonju, Republic of Korea
| | - Ji Hyun Chang
- Department of Radiation Oncology, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Jaehee Chun
- Department of Radiation Oncology, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Eung Man Lee
- Department of Radiation Oncology, Ewha Womans University College of Medicine, Seoul, Republic of Korea
| | - Jin Sung Kim
- Department of Radiation Oncology, Yonsei University College of Medicine, Seoul, Republic of Korea.
| | - Kyung Hwan Shin
- Department of Radiation Oncology, Seoul National University College of Medicine, Seoul, Republic of Korea.
| | - Yong Bae Kim
- Department of Radiation Oncology, Yonsei University College of Medicine, Seoul, Republic of Korea
| |
Collapse
|
8
|
Vivancos Bargalló H, Stick LB, Korreman SS, Kronborg C, Nielsen MM, Borgen AC, Offersen BV, Nørrevang O, Kallehauge JF. Classification of laterality and mastectomy/lumpectomy for breast cancer patients for improved performance of deep learning auto segmentation. Acta Oncol 2023; 62:1546-1550. [PMID: 37584197 DOI: 10.1080/0284186x.2023.2245965] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2023] [Accepted: 08/03/2023] [Indexed: 08/17/2023]
Affiliation(s)
- Helena Vivancos Bargalló
- Medical Physics department, Hospital de la Santa Creu i Sant Pau, Barcelona, Spain
- Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
| | | | - Stine Sofia Korreman
- Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Camilla Kronborg
- Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Mathias M Nielsen
- Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
| | | | - Birgitte Vrou Offersen
- Department of Experimental Clinical Oncologyy, Aarhus University Hospital, Aarhus, Denmark
- Department of Experimental Clinical Oncology, Aarhus University Hospital, Aarhus, Denmark
| | - Ole Nørrevang
- Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
| | - Jesper F Kallehauge
- Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| |
Collapse
|
9
|
Mikalsen SG, Skjøtskift T, Flote VG, Hämäläinen NP, Heydari M, Rydén-Eilertsen K. Extensive clinical testing of Deep Learning Segmentation models for thorax and breast cancer radiotherapy planning. Acta Oncol 2023; 62:1184-1193. [PMID: 37883678 DOI: 10.1080/0284186x.2023.2270152] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2023] [Accepted: 10/04/2023] [Indexed: 10/28/2023]
Abstract
BACKGROUND The performance of deep learning segmentation (DLS) models for automatic organ extraction from CT images in the thorax and breast regions was investigated. Furthermore, the readiness and feasibility of integrating DLS into clinical practice were addressed by measuring the potential time savings and dosimetric impact. MATERIAL AND METHODS Thirty patients referred to radiotherapy for breast cancer were prospectively included. A total of 23 clinically relevant left- and right-sided organs were contoured manually on CT images according to ESTRO guidelines. Next, auto-segmentation was executed, and the geometric agreement between the auto-segmented and manually contoured organs was qualitatively assessed applying a scale in the range [0-not acceptable, 3-no corrections]. A quantitative validation was carried out by calculating Dice coefficients (DSC) and the 95% percentile of Hausdorff distances (HD95). The dosimetric impact of optimizing the treatment plans on the uncorrected DLS contours, was investigated from a dose coverage analysis using DVH values of the manually delineated contours as references. RESULTS The qualitative analysis showed that 93% of the DLS generated OAR contours did not need corrections, except for the heart where 67% of the contours needed corrections. The majority of DLS generated CTVs needed corrections, whereas a minority were deemed not acceptable. Still, using the DLS-model for CTV and heart delineation is on average 14 minutes faster. An average DSC=0.91 and H95=9.8 mm were found for the left and right breasts, respectively. Likewise, and average DSC in the range [0.66, 0.76]mm and HD95 in the range [7.04, 12.05]mm were found for the lymph nodes. CONCLUSION The validation showed that the DLS generated OAR contours can be used clinically. Corrections were required to most of the DLS generated CTVs, and therefore warrants more attention before possibly implementing the DLS models clinically.
Collapse
Affiliation(s)
| | | | | | | | - Mojgan Heydari
- Department of Medical Physics, Oslo University Hospital, Oslo, Norway
| | | |
Collapse
|
10
|
McQuinlan Y, Brouwer CL, Lin Z, Gan Y, Sung Kim J, van Elmpt W, Gooding MJ. An investigation into the risk of population bias in deep learning autocontouring. Radiother Oncol 2023; 186:109747. [PMID: 37330053 DOI: 10.1016/j.radonc.2023.109747] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2023] [Revised: 05/30/2023] [Accepted: 06/08/2023] [Indexed: 06/19/2023]
Abstract
BACKGROUND AND PURPOSE To date, data used in the development of Deep Learning-based automatic contouring (DLC) algorithms have been largely sourced from single geographic populations. This study aimed to evaluate the risk of population-based bias by determining whether the performance of an autocontouring system is impacted by geographic population. MATERIALS AND METHODS 80 Head Neck CT deidentified scans were collected from four clinics in Europe (n = 2) and Asia (n = 2). A single observer manually delineated 16 organs-at-risk in each. Subsequently, the data was contoured using a DLC solution, and trained using single institution (European) data. Autocontours were compared to manual delineations using quantitative measures. A Kruskal-Wallis test was used to test for any difference between populations. Clinical acceptability of automatic and manual contours to observers from each participating institution was assessed using a blinded subjective evaluation. RESULTS Seven organs showed a significant difference in volume between groups. Four organs showed statistical differences in quantitative similarity measures. The qualitative test showed greater variation in acceptance of contouring between observers than between data from different origins, with greater acceptance by the South Korean observers. CONCLUSION Much of the statistical difference in quantitative performance could be explained by the difference in organ volume impacting the contour similarity measures and the small sample size. However, the qualitative assessment suggests that observer perception bias has a greater impact on the apparent clinical acceptability than quantitatively observed differences. This investigation of potential geographic bias should extend to more patients, populations, and anatomical regions in the future.
Collapse
Affiliation(s)
| | - Charlotte L Brouwer
- University of Groningen, University Medical Center Groningen, Department of Radiation Oncology, Groningen, The Netherlands.
| | - Zhixiong Lin
- Shantou University Medical Centre, Guangdong, China.
| | - Yong Gan
- Shantou University Medical Centre, Guangdong, China.
| | - Jin Sung Kim
- Yonsei University Health System, Seoul, Republic of Korea.
| | - Wouter van Elmpt
- Department of Radiation Oncology (MAASTRO), GROW - School for Oncology and Reproduction, Maastricht University Medical Centre+, Maastricht, The Netherlands.
| | - Mark J Gooding
- Mirada Medical Ltd, Oxford, United Kingdom; Inpictura Ltd, Oxford, United Kingdom.
| |
Collapse
|
11
|
Magrabi F, Lyell D, Coiera E. Automation in Contemporary Clinical Information Systems: a Survey of AI in Healthcare Settings. Yearb Med Inform 2023; 32:115-126. [PMID: 38147855 PMCID: PMC10751141 DOI: 10.1055/s-0043-1768733] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2023] Open
Abstract
AIMS AND OBJECTIVES To examine the nature and use of automation in contemporary clinical information systems by reviewing studies reporting the implementation and evaluation of artificial intelligence (AI) technologies in healthcare settings. METHOD PubMed/MEDLINE, Web of Science, EMBASE, the tables of contents of major informatics journals, and the bibliographies of articles were searched for studies reporting evaluation of AI in clinical settings from January 2021 to December 2022. We documented the clinical application areas and tasks supported, and the level of system autonomy. Reported effects on user experience, decision-making, care delivery and outcomes were summarised. RESULTS AI technologies are being applied in a wide variety of clinical areas. Most contemporary systems utilise deep learning, use routinely collected data, support diagnosis and triage, are assistive (requiring users to confirm or approve AI provided information or decisions), and are used by doctors in acute care settings in high-income nations. AI systems are integrated and used within existing clinical information systems including electronic medical records. There is limited support for One Health goals. Evaluation is largely based on quantitative methods measuring effects on decision-making. CONCLUSION AI systems are being implemented and evaluated in many clinical areas. There remain many opportunities to understand patterns of routine use and evaluate effects on decision-making, care delivery and patient outcomes using mixed-methods. Support for One Health including integrating data about environmental factors and social determinants needs further exploration.
Collapse
Affiliation(s)
- Farah Magrabi
- Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia
| | - David Lyell
- Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia
| | - Enrico Coiera
- Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia
| |
Collapse
|
12
|
Bakx N, Rijkaart D, van der Sangen M, Theuws J, van der Toorn PP, Verrijssen AS, van der Leer J, Mutsaers J, van Nunen T, Reinders M, Schuengel I, Smits J, Hagelaar E, van Gruijthuijsen D, Bluemink H, Hurkmans C. Clinical evaluation of a deep learning segmentation model including manual adjustments afterwards for locally advanced breast cancer. Tech Innov Patient Support Radiat Oncol 2023; 26:100211. [PMID: 37229460 PMCID: PMC10205480 DOI: 10.1016/j.tipsro.2023.100211] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 04/23/2023] [Accepted: 05/09/2023] [Indexed: 05/27/2023] Open
Abstract
Introduction Deep learning (DL) models are increasingly developed for auto-segmentation in radiotherapy. Qualitative analysis is of great importance for clinical implementation, next to quantitative. This study evaluates a DL segmentation model for left- and right-sided locally advanced breast cancer both quantitatively and qualitatively. Methods For each side a DL model was trained, including primary breast CTV (CTVp), lymph node levels 1-4, heart, lungs, humeral head, thyroid and esophagus. For evaluation, both automatic segmentation, including correction of contours when needed, and manual delineation was performed and both processes were timed. Quantitative scoring with dice-similarity coefficient (DSC), 95% Hausdorff Distance (95%HD) and surface DSC (sDSC) was used to compare both the automatic (not-corrected) and corrected contours with the manual contours. Qualitative scoring was performed by five radiotherapy technologists and five radiation oncologists using a 3-point Likert scale. Results Time reduction was achieved using auto-segmentation in 95% of the cases, including correction. The time reduction (mean ± std) was 42.4% ± 26.5% and 58.5% ± 19.1% for OARs and CTVs, respectively, corresponding to an absolute mean reduction (hh:mm:ss) of 00:08:51 and 00:25:38. Good quantitative results were achieved before correction, e.g. mean DSC for the right-sided CTVp was 0.92 ± 0.06, whereas correction statistically significantly improved this contour by only 0.02 ± 0.05, respectively. In 92% of the cases, auto-contours were scored as clinically acceptable, with or without corrections. Conclusions A DL segmentation model was trained and was shown to be a time-efficient way to generate clinically acceptable contours for locally advanced breast cancer.
Collapse
Affiliation(s)
- Nienke Bakx
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | - Dorien Rijkaart
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | | | - Jacqueline Theuws
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | | | - An-Sofie Verrijssen
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | - Jorien van der Leer
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | - Joline Mutsaers
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | - Thérèse van Nunen
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | - Marjon Reinders
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | - Inge Schuengel
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | - Julia Smits
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | - Els Hagelaar
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | | | - Hanneke Bluemink
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
| | - Coen Hurkmans
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, the Netherlands
- Technical University Eindhoven, Faculties of Physics and Electrical Engineering, Eindhoven, the Netherlands
| |
Collapse
|
13
|
Shen J, Gu P, Wang Y, Wang Z. Advances in automatic delineation of target volume and cardiac substructure in breast cancer radiotherapy (Review). Oncol Lett 2023; 25:110. [PMID: 36817059 PMCID: PMC9932716 DOI: 10.3892/ol.2023.13697] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2022] [Accepted: 01/06/2023] [Indexed: 02/05/2023] Open
Abstract
Postoperative adjuvant radiotherapy plays an important role in the treatment of patients with breast cancer. With the continuous development of radiotherapeutic technologies, the requirements for radiotherapeutic accuracy are increasingly high. The accuracy of target volume and organ at risk delineation significantly affects the effect of radiotherapy. Automatic delineation software has been continuously developed for the automatic delineation of target areas and organs at risk. Automatic segmentation based on an atlas and deep learning is a hot topic in current clinical research. Automatic delineation can not only reduce the workload and delineation times, but also establish a uniform delineation standard and reduce inter-observer and intra-observer differences. In patients with breast cancer, especially in patients who undergo left breast radiotherapy, the protection of the heart is particularly important. Treating the whole heart as an organ at risk cannot meet the clinical needs, and it is necessary to limit the dose to specific cardiac substructures. The present review discusses the importance of automatic delineation of target volume and cardiac substructure in radiotherapy for patients with breast cancer.
Collapse
Affiliation(s)
- Jingjing Shen
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200438, P.R. China
| | - Peihua Gu
- Department of Oncology and Radiotherapy, Shidong Hospital Affiliated to University of Shanghai for Science and Technology, Shanghai 200438, P.R. China
| | - Yun Wang
- Department of Oncology and Radiotherapy, Shidong Hospital Affiliated to University of Shanghai for Science and Technology, Shanghai 200438, P.R. China
| | - Zhongming Wang
- Department of Oncology and Radiotherapy, Shidong Hospital Affiliated to University of Shanghai for Science and Technology, Shanghai 200438, P.R. China
| |
Collapse
|
14
|
Mackay K, Bernstein D, Glocker B, Kamnitsas K, Taylor A. A Review of the Metrics Used to Assess Auto-Contouring Systems in Radiotherapy. Clin Oncol (R Coll Radiol) 2023; 35:354-369. [PMID: 36803407 DOI: 10.1016/j.clon.2023.01.016] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2022] [Revised: 12/05/2022] [Accepted: 01/23/2023] [Indexed: 02/01/2023]
Abstract
Auto-contouring could revolutionise future planning of radiotherapy treatment. The lack of consensus on how to assess and validate auto-contouring systems currently limits clinical use. This review formally quantifies the assessment metrics used in studies published during one calendar year and assesses the need for standardised practice. A PubMed literature search was undertaken for papers evaluating radiotherapy auto-contouring published during 2021. Papers were assessed for types of metric and the methodology used to generate ground-truth comparators. Our PubMed search identified 212 studies, of which 117 met the criteria for clinical review. Geometric assessment metrics were used in 116 of 117 studies (99.1%). This includes the Dice Similarity Coefficient used in 113 (96.6%) studies. Clinically relevant metrics, such as qualitative, dosimetric and time-saving metrics, were less frequently used in 22 (18.8%), 27 (23.1%) and 18 (15.4%) of 117 studies, respectively. There was heterogeneity within each category of metric. Over 90 different names for geometric measures were used. Methods for qualitative assessment were different in all but two papers. Variation existed in the methods used to generate radiotherapy plans for dosimetric assessment. Consideration of editing time was only given in 11 (9.4%) papers. A single manual contour as a ground-truth comparator was used in 65 (55.6%) studies. Only 31 (26.5%) studies compared auto-contours to usual inter- and/or intra-observer variation. In conclusion, significant variation exists in how research papers currently assess the accuracy of automatically generated contours. Geometric measures are the most popular, however their clinical utility is unknown. There is heterogeneity in the methods used to perform clinical assessment. Considering the different stages of system implementation may provide a framework to decide the most appropriate metrics. This analysis supports the need for a consensus on the clinical implementation of auto-contouring.
Collapse
Affiliation(s)
- K Mackay
- The Institute of Cancer Research, London, UK; The Royal Marsden Hospital, London, UK.
| | - D Bernstein
- The Institute of Cancer Research, London, UK; The Royal Marsden Hospital, London, UK
| | - B Glocker
- Department of Computing, Imperial College London, South Kensington Campus, London, UK
| | - K Kamnitsas
- Department of Computing, Imperial College London, South Kensington Campus, London, UK; Department of Engineering Science, University of Oxford, Oxford, UK
| | - A Taylor
- The Institute of Cancer Research, London, UK; The Royal Marsden Hospital, London, UK
| |
Collapse
|
15
|
Strolin S, Santoro M, Paolani G, Ammendolia I, Arcelli A, Benini A, Bisello S, Cardano R, Cavallini L, Deraco E, Donati CM, Galietta E, Galuppi A, Guido A, Ferioli M, Laghi V, Medici F, Ntreta M, Razganiayeva N, Siepe G, Tolento G, Vallerossa D, Zamagni A, Morganti AG, Strigari L. How smart is artificial intelligence in organs delineation? Testing a CE and FDA-approved Deep-Learning tool using multiple expert contours delineated on planning CT images. Front Oncol 2023; 13:1089807. [PMID: 36937399 PMCID: PMC10019504 DOI: 10.3389/fonc.2023.1089807] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Accepted: 01/19/2023] [Indexed: 03/06/2023] Open
Abstract
Background A CE- and FDA-approved cloud-based Deep learning (DL)-tool for automatic organs at risk (OARs) and clinical target volumes segmentation on computer tomography images is available. Before its implementation in the clinical practice, an independent external validation was conducted. Methods At least a senior and two in training Radiation Oncologists (ROs) manually contoured the volumes of interest (VOIs) for 6 tumoral sites. The auto-segmented contours were retrieved from the DL-tool and, if needed, manually corrected by ROs. The level of ROs satisfaction and the duration of contouring were registered. Relative volume differences, similarity indices, satisfactory grades, and time saved were analyzed using a semi-automatic tool. Results Seven thousand seven hundred sixty-five VOIs were delineated on the CT images of 111 representative patients. The median (range) time for manual VOIs delineation, DL-based segmentation, and subsequent manual corrections were 25.0 (8.0-115.0), 2.3 (1.2-8) and 10.0 minutes (0.3-46.3), respectively. The overall time for VOIs retrieving and modification was statistically significantly lower than for manual contouring (p<0.001). The DL-tool was generally appreciated by ROs, with 44% of vote 4 (well done) and 43% of vote 5 (very well done), correlated with the saved time (p<0.001). The relative volume differences and similarity indexes suggested a better inter-agreement of manually adjusted DL-based VOIs than manually segmented ones. Conclusions The application of the DL-tool resulted satisfactory, especially in complex delineation cases, improving the ROs inter-agreement of delineated VOIs and saving time.
Collapse
Affiliation(s)
- Silvia Strolin
- Department of Medical Physics, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Miriam Santoro
- Department of Medical Physics, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
- Medical Physics Specialization School, Alma Mater Studiorum, University of Bologna, Bologna, Italy
| | - Giulia Paolani
- Department of Medical Physics, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
- Medical Physics Specialization School, Alma Mater Studiorum, University of Bologna, Bologna, Italy
- *Correspondence: Lidia Strigari, ; Giulia Paolani,
| | - Ilario Ammendolia
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Alessandra Arcelli
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Anna Benini
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Silvia Bisello
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Raffaele Cardano
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Letizia Cavallini
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Elisa Deraco
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Costanza Maria Donati
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Erika Galietta
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Andrea Galuppi
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Alessandra Guido
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Martina Ferioli
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Viola Laghi
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Federica Medici
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Maria Ntreta
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Natalya Razganiayeva
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Giambattista Siepe
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Giorgio Tolento
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Daria Vallerossa
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Alice Zamagni
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Alessio Giuseppe Morganti
- Department of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
- Department of Medical and Surgical Sciences (DIMEC), Alma Mater Studiorum, University of Bologna, Bologna, Italy
| | - Lidia Strigari
- Department of Medical Physics, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
- *Correspondence: Lidia Strigari, ; Giulia Paolani,
| |
Collapse
|
16
|
Cè M, Caloro E, Pellegrino ME, Basile M, Sorce A, Fazzini D, Oliva G, Cellina M. Artificial intelligence in breast cancer imaging: risk stratification, lesion detection and classification, treatment planning and prognosis-a narrative review. EXPLORATION OF TARGETED ANTI-TUMOR THERAPY 2022; 3:795-816. [PMID: 36654817 PMCID: PMC9834285 DOI: 10.37349/etat.2022.00113] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2022] [Accepted: 09/28/2022] [Indexed: 12/28/2022] Open
Abstract
The advent of artificial intelligence (AI) represents a real game changer in today's landscape of breast cancer imaging. Several innovative AI-based tools have been developed and validated in recent years that promise to accelerate the goal of real patient-tailored management. Numerous studies confirm that proper integration of AI into existing clinical workflows could bring significant benefits to women, radiologists, and healthcare systems. The AI-based approach has proved particularly useful for developing new risk prediction models that integrate multi-data streams for planning individualized screening protocols. Furthermore, AI models could help radiologists in the pre-screening and lesion detection phase, increasing diagnostic accuracy, while reducing workload and complications related to overdiagnosis. Radiomics and radiogenomics approaches could extrapolate the so-called imaging signature of the tumor to plan a targeted treatment. The main challenges to the development of AI tools are the huge amounts of high-quality data required to train and validate these models and the need for a multidisciplinary team with solid machine-learning skills. The purpose of this article is to present a summary of the most important AI applications in breast cancer imaging, analyzing possible challenges and new perspectives related to the widespread adoption of these new tools.
Collapse
Affiliation(s)
- Maurizio Cè
- Postgraduate School in Diagnostic and Interventional Radiology, University of Milan, 20122 Milan, Italy
| | - Elena Caloro
- Postgraduate School in Diagnostic and Interventional Radiology, University of Milan, 20122 Milan, Italy
| | - Maria E. Pellegrino
- Postgraduate School in Diagnostic and Interventional Radiology, University of Milan, 20122 Milan, Italy
| | - Mariachiara Basile
- Postgraduate School in Diagnostic and Interventional Radiology, University of Milan, 20122 Milan, Italy
| | - Adriana Sorce
- Postgraduate School in Diagnostic and Interventional Radiology, University of Milan, 20122 Milan, Italy
| | | | - Giancarlo Oliva
- Department of Radiology, ASST Fatebenefratelli Sacco, 20121 Milan, Italy
| | - Michaela Cellina
- Department of Radiology, ASST Fatebenefratelli Sacco, 20121 Milan, Italy
| |
Collapse
|
17
|
Song JY, Chie EK, Kang SH, Jeon YJ, Ko YA, Kim DY, Kang HC. Dosimetric evaluation of magnetic resonance imaging-guided adaptive radiation therapy in pancreatic cancer by extent of re-contouring of organs-at-risk. Radiat Oncol J 2022; 40:242-250. [PMID: 36606301 PMCID: PMC9830039 DOI: 10.3857/roj.2022.00332] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Accepted: 11/14/2022] [Indexed: 12/27/2022] Open
Abstract
PURPOSE The safety of online contouring and planning for adaptive radiotherapy is unknown. This study aimed to evaluate the dosimetric difference of the organ-at-risk (OAR) according to the extent of contouring in stereotactic magnetic resonance image-guided adaptive RT (SMART) for pancreatic cancer. MATERIALS AND METHODS We reviewed the treatment plan data used for SMART in patients with pancreatic cancer. For the online contouring and planning, OARs within 2 cm from the planning target volume (PTV) in the craniocaudal direction were re-controlled daily at the attending physician's discretion. The entire OARs were re-contoured retrospectively for data analysis. We termed the two contouring methods the Rough OAR and the Full OAR, respectively. The proportion of dose constraint violation and other dosimetric parameters was analyzed. RESULTS Nineteen patients with 94 fractions of SMART were included in the analysis. The dose constraint was violated in 10.6% and 43.6% of the fractions in Rough OAR and Full OAR methods, respectively (p = 0.075). Patients with a large tumor, a short distance from gross tumor volume (GTV) to OAR, and a tumor in the body or tail were associated with more occult dose constraint violations-large tumor (p = 0.027), short distance from GTV to OAR (p = 0.061), tumor in body or tail (p = 0.054). No dose constraint violation occurred outside 2 cm from the PTV. CONCLUSION More occult dose constraint violations can be found by the Full OAR method in patients with pancreatic cancer with some clinical factors in the online re-planning for SMART. Re-contouring all the OARs would be helpful to detect occult dose constraint violations in SMART planning. Since the dosimetric profile of SMART cannot be represented by a single fraction, patient selection for the Full OAR method should be weighted between the clinical usefulness and the time and workforce required.
Collapse
Affiliation(s)
- Jun Yeong Song
- Department of Radiation Oncology, Seoul National University Hospital, Seoul, Korea,Department of Radiation Oncology, Seoul National University College of Medicine, Seoul, Korea
| | - Eui Kyu Chie
- Department of Radiation Oncology, Seoul National University Hospital, Seoul, Korea,Department of Radiation Oncology, Seoul National University College of Medicine, Seoul, Korea
| | - Seong-Hee Kang
- Department of Radiation Oncology, Seoul National University Hospital, Seoul, Korea
| | - Yeon-Jun Jeon
- Seoul National University College of Medicine, Seoul, Korea
| | - Yoon-Ah Ko
- Seoul National University College of Medicine, Seoul, Korea
| | - Dong-Yun Kim
- Department of Radiation Oncology, Seoul National University College of Medicine, Seoul, Korea,Department of Radiation Oncology, Kyung Hee University Hospital at Gangdong, Seoul, Korea
| | - Hyun-Cheol Kang
- Department of Radiation Oncology, Seoul National University Hospital, Seoul, Korea,Department of Radiation Oncology, Seoul National University College of Medicine, Seoul, Korea,Correspondence: Hyun-Cheol Kang Department of Radiation Oncology, Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea. Tel: +82-2-2072-2526 E-mail:
| |
Collapse
|
18
|
Kim JS, Kim JH, Chang JH, Kim DW, Shin KH. Prediction of breast cancer-related lymphedema risk after postoperative radiotherapy via multivariable logistic regression analysis. Front Oncol 2022; 12:1026043. [PMID: 36387231 PMCID: PMC9643832 DOI: 10.3389/fonc.2022.1026043] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2022] [Accepted: 10/04/2022] [Indexed: 08/04/2023] Open
Abstract
PURPOSE We identified novel clinical and dosimetric prognostic factors affecting breast cancer-related lymphedema after postoperative radiotherapy (RT) and developed a multivariable logistic regression model to predict lymphedema in these patients. METHODS AND MATERIALS In total, 580 patients with unilateral breast cancer were retrospectively reviewed. All patients underwent breast surgery and postoperative RT with or without systemic treatment in 2015. Among the 580 patients, 532 with available RT plan data were randomly divided into training (n=372) and test (n=160) cohorts at a 7:3 ratio to generate and validate the lymphedema prediction models, respectively. An area under the curve (AUC) value was estimated to compare models. RESULTS The median follow-up duration was 5.4 years. In total, 104 (17.9%) patients experienced lymphedema with a cumulative incidence as follows: 1 year, 10.5%; 3 years, 16.4%; and 5 years, 17.6%. Multivariate analysis showed that body mass index ≥25 kg/m2 (hazard ratio [HR] 1.845), dissected lymph nodes ≥7 (HR 1.789), and taxane-base chemotherapy (HR 4.200) were significantly associated with increased lymphedema risk. Conversely, receipt of RT at least 1 month after surgery reduced the risk of lymphedema (HR 0.638). A multivariable logistic regression model using the above factors, as well as the minimum dose of axillary level I and supraclavicular lymph node, was created with an AUC of 0.761 and 0.794 in the training and test cohorts, respectively. CONCLUSIONS Our study demonstrated that a shorter interval from surgery to RT and other established clinical factors were associated with increased lymphedema risk. By combining these factors with two dosimetric parameters, we propose a multivariable logistic regression model for breast cancer-related lymphedema prediction after RT.
Collapse
Affiliation(s)
- Jae Sik Kim
- Department of Radiation Oncology, Seoul National University College of Medicine, Seoul, South Korea
- Department of Radiation Oncology, Soonchunhyang University Seoul Hospital, Seoul, South Korea
| | - Jin Ho Kim
- Department of Radiation Oncology, Seoul National University College of Medicine, Seoul, South Korea
- Department of Radiation Oncology, Seoul National University Hospital, Seoul, South Korea
| | - Ji Hyun Chang
- Department of Radiation Oncology, Seoul National University College of Medicine, Seoul, South Korea
- Department of Radiation Oncology, Seoul National University Hospital, Seoul, South Korea
| | - Do Wook Kim
- Department of Radiation Oncology, Seoul National University College of Medicine, Seoul, South Korea
- Department of Radiation Oncology, Seoul National University Hospital, Seoul, South Korea
| | - Kyung Hwan Shin
- Department of Radiation Oncology, Seoul National University College of Medicine, Seoul, South Korea
- Department of Radiation Oncology, Seoul National University Hospital, Seoul, South Korea
| |
Collapse
|
19
|
Retrospective Analysis for Dose Reduction to Organs at Risk with New Personalized Breast Holder (PERSBRA) in Left Breast IMRT. J Pers Med 2022; 12:jpm12091368. [PMID: 36143153 PMCID: PMC9505458 DOI: 10.3390/jpm12091368] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2022] [Revised: 08/21/2022] [Accepted: 08/22/2022] [Indexed: 11/17/2022] Open
Abstract
This study evaluated dose differences in normal organs at risk, such as the lungs, heart, left anterior descending artery (LAD), right coronary artery, left ventricle, and right breast under personalized breast holder (PERSBRA), when using intensity-modulated radiation therapy (IMRT). This study evaluated the radiation protection offered by PERSBRA in left breast cancer radiation therapy. Here, we retrospectively collected data from 24 patients with left breast cancer who underwent breast-conserving surgery as well as IMRT radiotherapy. We compared the dose differences in target coverage and organs at risk with and without PERSBRA. For target coverage, tumor prescribed dose 95% coverage, conformity index, and homogeneity index were evaluated. For organs at risk, we compared the mean heart dose, mean left ventricle dose, LAD maximum and mean dose, mean left lung receiving 20 Gy, 10 Gy, and 5 Gy of left lung volume, maximum and mean coronary artery of the right, maximum of right breast, and mean dose. Good target coverage was achieved with and without PERSBRA. When PERSBRA was used with IMRT, the mean dose of the heart decreased by 42%, the maximum dose of LAD decreased by 26.4%, and the mean dose of LAD decreased by 47.0%. The mean dose of the left ventricle decreased by 54.1%, the volume (V20) of the left lung that received 20 Gy decreased by 22.8%, the volume (V10) of the left lung that received 10 Gy decreased by 19.8%, the volume (V5) of the left lung that received 5 Gy decreased by 15.7%, and the mean dose of the left lung decreased by 23.3%. Using PERSBRA with IMRT greatly decreases the dose to organs at risk (left lung, heart, left ventricle, and LAD). This study found that PERSBRA with IMRT can achieve results similar to deep inspiration breath-hold radiotherapy (DIBH) in terms of reducing the heart radiation dose and the risk of developing heart disease in patients with left breast cancer who cannot undergo DIBH.
Collapse
|
20
|
Almberg SS, Lervåg C, Frengen J, Eidem M, Abramova T, Nordstrand C, Alsaker M, Tøndel H, Raj SX, Wanderås AD. Training, validation, and clinical implementation of a deep-learning segmentation model for radiotherapy of loco-regional breast cancer. Radiother Oncol 2022; 173:62-68. [DOI: 10.1016/j.radonc.2022.05.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2022] [Revised: 05/07/2022] [Accepted: 05/18/2022] [Indexed: 11/29/2022]
|