1
|
Talcott W, Covington E, Bazan J, Wright JL. The Future of Safety and Quality in Radiation Oncology. Semin Radiat Oncol 2024; 34:433-440. [PMID: 39271278 DOI: 10.1016/j.semradonc.2024.07.008] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/15/2024]
Abstract
The increasing complexity of radiation therapy treatment presents new potentials for error and suboptimal care. High-performing programs thus not only require adherence to, but also ongoing improvement of, key safety and quality practices. In this article, we review these practices including standardization, risk analysis, peer review, and maintenance of strong safety culture, while also describing recent innovations and promising future directions. We specifically highlight the growing role of artificial intelligence in radiation oncology, both as a tool to deliver safe, high-quality care and as a potential new source of safety challenges.
Collapse
Affiliation(s)
- Wesley Talcott
- Northwell Health Department of Radiation Oncology, New York, NY
| | | | - Jose Bazan
- City of Hope Comprehensive Cancer Center, Department of Radiation Oncology, Duarte, CA
| | - Jean L Wright
- Department of Radiation Oncology, Johns Hopkins University, Baltimore, MD.
| |
Collapse
|
2
|
de Boisredon d'Assier MA, Portafaix A, Vorontsov E, Le WT, Kadoury S. Image-level supervision and self-training for transformer-based cross-modality tumor segmentation. Med Image Anal 2024; 97:103287. [PMID: 39111265 DOI: 10.1016/j.media.2024.103287] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 06/20/2024] [Accepted: 07/24/2024] [Indexed: 08/30/2024]
Abstract
Deep neural networks are commonly used for automated medical image segmentation, but models will frequently struggle to generalize well across different imaging modalities. This issue is particularly problematic due to the limited availability of annotated data, both in the target as well as the source modality, making it difficult to deploy these models on a larger scale. To overcome these challenges, we propose a new semi-supervised training strategy called MoDATTS. Our approach is designed for accurate cross-modality 3D tumor segmentation on unpaired bi-modal datasets. An image-to-image translation strategy between modalities is used to produce synthetic but annotated images and labels in the desired modality and improve generalization to the unannotated target modality. We also use powerful vision transformer architectures for both image translation (TransUNet) and segmentation (Medformer) tasks and introduce an iterative self-training procedure in the later task to further close the domain gap between modalities, thus also training on unlabeled images in the target modality. MoDATTS additionally allows the possibility to exploit image-level labels with a semi-supervised objective that encourages the model to disentangle tumors from the background. This semi-supervised methodology helps in particular to maintain downstream segmentation performance when pixel-level label scarcity is also present in the source modality dataset, or when the source dataset contains healthy controls. The proposed model achieves superior performance compared to other methods from participating teams in the CrossMoDA 2022 vestibular schwannoma (VS) segmentation challenge, as evidenced by its reported top Dice score of 0.87±0.04 for the VS segmentation. MoDATTS also yields consistent improvements in Dice scores over baselines on a cross-modality adult brain gliomas segmentation task composed of four different contrasts from the BraTS 2020 challenge dataset, where 95% of a target supervised model performance is reached when no target modality annotations are available. We report that 99% and 100% of this maximum performance can be attained if 20% and 50% of the target data is additionally annotated, which further demonstrates that MoDATTS can be leveraged to reduce the annotation burden.
Collapse
Affiliation(s)
| | - Aloys Portafaix
- Polytechnique Montreal, Montreal, QC, Canada; Centre de Recherche du Centre Hospitalier de l'Université de Montréal, Montreal, QC, Canada
| | | | - William Trung Le
- Polytechnique Montreal, Montreal, QC, Canada; Centre de Recherche du Centre Hospitalier de l'Université de Montréal, Montreal, QC, Canada
| | - Samuel Kadoury
- Polytechnique Montreal, Montreal, QC, Canada; Centre de Recherche du Centre Hospitalier de l'Université de Montréal, Montreal, QC, Canada.
| |
Collapse
|
3
|
Rydygier M, Skóra T, Kisielewicz K, Spaleniak A, Garbacz M, Lipa M, Foltyńska G, Góra E, Gajewski J, Krzempek D, Kopeć R, Ruciński A. Proton Therapy Adaptation of Perisinusoidal and Brain Areas in the Cyclotron Centre Bronowice in Krakow: A Dosimetric Analysis. Cancers (Basel) 2024; 16:3128. [PMID: 39335100 PMCID: PMC11430589 DOI: 10.3390/cancers16183128] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2024] [Revised: 09/06/2024] [Accepted: 09/09/2024] [Indexed: 09/30/2024] Open
Abstract
Applying a proton beam in radiotherapy enables precise irradiation of the tumor volume, but only for continuous assessment of changes in patient anatomy. Proton beam range uncertainties in the treatment process may originate not only from physical beam properties but also from patient-specific factors such as tumor shrinkage, edema formation and sinus filling, which are not incorporated in tumor volume safety margins. In this paper, we evaluated variations in dose distribution in proton therapy resulting from the differences observed in the control tomographic images and the dosimetric influence of applied adaptive treatment. The data from weekly computed tomography (CT) control scans of 21 patients, which serve as the basis for adaptive radiotherapy, were used for this study. Dosimetric analysis of adaptive proton therapy (APT) was performed on patients with head and neck (H&N) area tumors who were divided into two groups: patients with tumors in the sinus/nasal area and patients with tumors in the brain area. For this analysis, the reference treatment plans were forward-calculated using weekly control CT scans. A comparative evaluation of organ at risk (OAR) dose-volume histogram (DVH) parameters, as well as conformity and homogeneity indices, was conducted between the initial and recalculated dose distributions to assess the necessity of the adaptation process in terms of dosimetric parameters. Changes in PTV volume after replanning were observed in seventeen patient cases, showing a discrepancy of over 1 cm3 in ten cases. In these cases, tumor progression occurred in 30% of patients, while regression was observed in 70%. The statistical analysis indicates that the use of the adaptive planning procedure results in a statistically significant improvement in dose distribution, particularly in the PTV area. The findings led to the conclusion that the adaptive procedure provides significant advantages in terms of dose distribution within the treated volume. However, when considering the entire patient group, APT did not result in a statistically significant dose reduction in OARs (α = 0.05).
Collapse
Affiliation(s)
- Marzena Rydygier
- Cyclotron Centre Bronowice, Institute of Nuclear Physics Polish Academy of Sciences, PL31342 Kraków, Poland; (M.R.)
| | - Tomasz Skóra
- National Oncology Institute—National Research Institute, Krakow Branch, PL31115 Kraków, Poland
| | - Kamil Kisielewicz
- National Oncology Institute—National Research Institute, Krakow Branch, PL31115 Kraków, Poland
| | - Anna Spaleniak
- Cyclotron Centre Bronowice, Institute of Nuclear Physics Polish Academy of Sciences, PL31342 Kraków, Poland; (M.R.)
| | - Magdalena Garbacz
- Cyclotron Centre Bronowice, Institute of Nuclear Physics Polish Academy of Sciences, PL31342 Kraków, Poland; (M.R.)
| | - Monika Lipa
- Cyclotron Centre Bronowice, Institute of Nuclear Physics Polish Academy of Sciences, PL31342 Kraków, Poland; (M.R.)
| | - Gabriela Foltyńska
- Cyclotron Centre Bronowice, Institute of Nuclear Physics Polish Academy of Sciences, PL31342 Kraków, Poland; (M.R.)
| | - Eleonora Góra
- National Oncology Institute—National Research Institute, Krakow Branch, PL31115 Kraków, Poland
| | - Jan Gajewski
- Cyclotron Centre Bronowice, Institute of Nuclear Physics Polish Academy of Sciences, PL31342 Kraków, Poland; (M.R.)
| | - Dawid Krzempek
- Cyclotron Centre Bronowice, Institute of Nuclear Physics Polish Academy of Sciences, PL31342 Kraków, Poland; (M.R.)
| | - Renata Kopeć
- Cyclotron Centre Bronowice, Institute of Nuclear Physics Polish Academy of Sciences, PL31342 Kraków, Poland; (M.R.)
| | - Antoni Ruciński
- Cyclotron Centre Bronowice, Institute of Nuclear Physics Polish Academy of Sciences, PL31342 Kraków, Poland; (M.R.)
| |
Collapse
|
4
|
Zhang Y, Amjad A, Ding J, Sarosiek C, Zarenia M, Conlin R, Hall WA, Erickson B, Paulson E. Comprehensive Clinical Usability-Oriented Contour Quality Evaluation for Deep Learning Auto-segmentation: Combining Multiple Quantitative Metrics Through Machine Learning. Pract Radiat Oncol 2024:S1879-8500(24)00204-2. [PMID: 39233005 DOI: 10.1016/j.prro.2024.07.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2024] [Revised: 06/07/2024] [Accepted: 07/30/2024] [Indexed: 09/06/2024]
Abstract
PURPOSE The current commonly used metrics for evaluating the quality of auto-segmented contours have limitations and do not always reflect the clinical usefulness of the contours. This work aims to develop a novel contour quality classification (CQC) method by combining multiple quantitative metrics for clinical usability-oriented contour quality evaluation for deep learning-based auto-segmentation (DLAS). METHODS AND MATERIALS The CQC was designed to categorize contours on slices as acceptable, minor edit, or major edit based on the expected editing effort/time with supervised ensemble tree classification models using 7 quantitative metrics. Organ-specific models were trained for 5 abdominal organs (pancreas, duodenum, stomach, small, and large bowels) using 50 magnetic resonance imaging (MRI) data sets. Twenty additional MRI and 9 computed tomography (CT) data sets were employed for testing. Interobserver variation (IOV) was assessed among 6 observers and consensus labels were established through majority vote for evaluation. The CQC was also compared with a threshold-based baseline approach. RESULTS For the 5 organs, the average area under the curve was 0.982 ± 0.01 and 0.979 ± 0.01, the mean accuracy was 95.8% ± 1.7% and 94.3% ± 2.1%, and the mean risk rate was 0.8% ± 0.4% and 0.7% ± 0.5% for MRI and CT testing data set, respectively. The CQC results closely matched the IOV results (mean accuracy of 94.2% ± 0.8% and 94.8% ± 1.7%) and were significantly higher than those obtained using the threshold-based method (mean accuracy of 80.0% ± 4.7%, 83.8% ± 5.2%, and 77.3% ± 6.6% using 1, 2, and 3 metrics). CONCLUSIONS The CQC models demonstrated high performance in classifying the quality of contour slices. This method can address the limitations of existing metrics and offers an intuitive and comprehensive solution for clinically oriented evaluation and comparison of DLAS systems.
Collapse
Affiliation(s)
- Ying Zhang
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin; Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, Texas.
| | - Asma Amjad
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin
| | - Jie Ding
- Department of Radiation Oncology, Emory University School of Medicine, Atlanta, Georgia
| | - Christina Sarosiek
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin
| | - Mohammad Zarenia
- Department of Radiation Medicine, MedStar Georgetown University Hospital, Washington, District of Columbia
| | - Renae Conlin
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin
| | - William A Hall
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin
| | - Beth Erickson
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin
| | - Eric Paulson
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin
| |
Collapse
|
5
|
Huang Y, Khodabakhshi Z, Gomaa A, Schmidt M, Fietkau R, Guckenberger M, Andratschke N, Bert C, Tanadini-Lang S, Putz F. Multicenter privacy-preserving model training for deep learning brain metastases autosegmentation. Radiother Oncol 2024; 198:110419. [PMID: 38969106 DOI: 10.1016/j.radonc.2024.110419] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2024] [Revised: 06/19/2024] [Accepted: 06/21/2024] [Indexed: 07/07/2024]
Abstract
OBJECTIVES This work aims to explore the impact of multicenter data heterogeneity on deep learning brain metastases (BM) autosegmentation performance, and assess the efficacy of an incremental transfer learning technique, namely learning without forgetting (LWF), to improve model generalizability without sharing raw data. MATERIALS AND METHODS A total of six BM datasets from University Hospital Erlangen (UKER), University Hospital Zurich (USZ), Stanford, UCSF, New York University (NYU), and BraTS Challenge 2023 were used. First, the performance of the DeepMedic network for BM autosegmentation was established for exclusive single-center training and mixed multicenter training, respectively. Subsequently privacy-preserving bilateral collaboration was evaluated, where a pretrained model is shared to another center for further training using transfer learning (TL) either with or without LWF. RESULTS For single-center training, average F1 scores of BM detection range from 0.625 (NYU) to 0.876 (UKER) on respective single-center test data. Mixed multicenter training notably improves F1 scores at Stanford and NYU, with negligible improvement at other centers. When the UKER pretrained model is applied to USZ, LWF achieves a higher average F1 score (0.839) than naive TL (0.570) and single-center training (0.688) on combined UKER and USZ test data. Naive TL improves sensitivity and contouring accuracy, but compromises precision. Conversely, LWF demonstrates commendable sensitivity, precision and contouring accuracy. When applied to Stanford, similar performance was observed. CONCLUSION Data heterogeneity (e.g., variations in metastases density, spatial distribution, and image spatial resolution across centers) results in varying performance in BM autosegmentation, posing challenges to model generalizability. LWF is a promising approach to peer-to-peer privacy-preserving model training.
Collapse
Affiliation(s)
- Yixing Huang
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany; Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany; Bavarian Cancer Research Center (BZKF), Erlangen, Germany.
| | - Zahra Khodabakhshi
- Department of Radiation Oncology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Ahmed Gomaa
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany; Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany; Bavarian Cancer Research Center (BZKF), Erlangen, Germany
| | - Manuel Schmidt
- Department of Neuroradiology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Rainer Fietkau
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany; Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany; Bavarian Cancer Research Center (BZKF), Erlangen, Germany
| | - Matthias Guckenberger
- Department of Radiation Oncology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Nicolaus Andratschke
- Department of Radiation Oncology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Christoph Bert
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany; Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany; Bavarian Cancer Research Center (BZKF), Erlangen, Germany
| | - Stephanie Tanadini-Lang
- Department of Radiation Oncology, University Hospital Zurich, University of Zurich, Zurich, Switzerland.
| | - Florian Putz
- Department of Radiation Oncology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany; Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN), Erlangen, Germany; Bavarian Cancer Research Center (BZKF), Erlangen, Germany
| |
Collapse
|
6
|
Kim YW, Biggs S, Claridge Mackonis E. Investigation on performance of multiple AI-based auto-contouring systems in organs at risks (OARs) delineation. Phys Eng Sci Med 2024; 47:1123-1140. [PMID: 39222214 PMCID: PMC11408550 DOI: 10.1007/s13246-024-01434-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Accepted: 04/30/2024] [Indexed: 09/04/2024]
Abstract
Manual contouring of organs at risk (OAR) is time-consuming and subject to inter-observer variability. AI-based auto-contouring is proposed as a solution to these problems if it can produce clinically acceptable results. This study investigated the performance of multiple AI-based auto-contouring systems in different OAR segmentations. The auto-contouring was performed using seven different AI-based segmentation systems (Radiotherapy AI, Limbus AI version 1.5 and 1.6, Therapanacea, MIM, Siemens AI-Rad Companion and RadFormation) on a total of 42 clinical cases with varying anatomical sites. Volumetric and surface dice similarity coefficients and maximum Hausdorff distance (HD) between the expert's contours and automated contours were calculated to evaluate their performance. Radiotherapy AI has shown better performance than other software in most tested structures considered in the head and neck, and brain cases. No specific software had shown overall superior performance over other software in lung, breast, pelvis and abdomen cases. Each tested AI system was able to produce comparable contours to the experts' contours of organs at risk which can potentially be used for clinical use. A reduced performance of AI systems in the case of small and complex anatomical structures was found and reported, showing that it is still essential to review each contour produced by AI systems for clinical uses. This study has also demonstrated a method of comparing contouring software options which could be replicated in clinics or used for ongoing quality assurance of purchased systems.
Collapse
Affiliation(s)
- Young Woo Kim
- Department of Radiation Oncology, Chris O'Brien Lifehouse, Sydney, NSW, Australia
| | | | - Elizabeth Claridge Mackonis
- Department of Radiation Oncology, Chris O'Brien Lifehouse, Sydney, NSW, Australia.
- Institute of Medical Physics, School of Physics, University of Sydney, Sydney, NSW, Australia.
| |
Collapse
|
7
|
Bordigoni B, Trivellato S, Pellegrini R, Meregalli S, Bonetto E, Belmonte M, Castellano M, Panizza D, Arcangeli S, De Ponti E. Automated segmentation in pelvic radiotherapy: A comprehensive evaluation of ATLAS-, machine learning-, and deep learning-based models. Phys Med 2024; 125:104486. [PMID: 39098106 DOI: 10.1016/j.ejmp.2024.104486] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/16/2023] [Revised: 06/20/2024] [Accepted: 07/17/2024] [Indexed: 08/06/2024] Open
Abstract
Artificial intelligence can standardize and automatize highly demanding procedures, such as manual segmentation, especially in an anatomical site as common as the pelvis. This study investigated four automated segmentation tools on computed tomography (CT) images in female and male pelvic radiotherapy (RT) starting from simpler and well-known atlas-based methods to the most recent neural networks-based algorithms. The evaluation included quantitative, qualitative and time efficiency assessments. A mono-institutional consecutive series of 40 cervical cancer and 40 prostate cancer structure sets were retrospectively selected. After a preparatory phase, the remaining 20 testing sets per each site were auto-segmented by the atlas-based model STAPLE, a Random Forest-based model, and two Deep Learning-based tools (DL), MVision and LimbusAI. Setting manual segmentation as the Ground Truth, 200 structure sets were compared in terms of Dice Similarity Coefficient (DSC), Hausdorff Distance (HD), and Distance-to-Agreement Portion (DAP). Automated segmentation and manual correction durations were recorded. Expert clinicians performed a qualitative evaluation. In cervical cancer CTs, DL outperformed the other tools with higher quantitative metrics, qualitative scores, and shorter correction times. On the other hand, in prostate cancer CTs, the performance across all the analyzed tools was comparable in terms of both quantitative and qualitative metrics. Such discrepancy in performance outcome could be explained by the wide range of anatomical variability in cervical cancer with respect to the strict bladder and rectum filling preparation in prostate Stereotactic Body Radiation Therapy (SBRT). Decreasing segmentation times can reduce the burden of pelvic radiation therapy routine in an automated workflow.
Collapse
Affiliation(s)
- B Bordigoni
- Medical Physics, Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy
| | - S Trivellato
- Medical Physics, Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy
| | | | - S Meregalli
- Radiation Oncology, Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy
| | - E Bonetto
- Radiation Oncology, Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy
| | - M Belmonte
- School of Medicine and Surgery, University of Milano Bicocca, Milano, Italy; Radiation Oncology, Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy
| | - M Castellano
- School of Medicine and Surgery, University of Milano Bicocca, Milano, Italy; Radiation Oncology, Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy
| | - D Panizza
- Medical Physics, Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy; School of Medicine and Surgery, University of Milano Bicocca, Milano, Italy
| | - S Arcangeli
- School of Medicine and Surgery, University of Milano Bicocca, Milano, Italy; Radiation Oncology, Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy.
| | - E De Ponti
- Medical Physics, Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy; School of Medicine and Surgery, University of Milano Bicocca, Milano, Italy
| |
Collapse
|
8
|
Podobnik G, Ibragimov B, Tappeiner E, Lee C, Kim JS, Mesbah Z, Modzelewski R, Ma Y, Yang F, Rudecki M, Wodziński M, Peterlin P, Strojan P, Vrtovec T. HaN-Seg: The head and neck organ-at-risk CT and MR segmentation challenge. Radiother Oncol 2024; 198:110410. [PMID: 38917883 DOI: 10.1016/j.radonc.2024.110410] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2024] [Revised: 06/12/2024] [Accepted: 06/15/2024] [Indexed: 06/27/2024]
Abstract
BACKGROUND AND PURPOSE To promote the development of auto-segmentation methods for head and neck (HaN) radiation treatment (RT) planning that exploit the information of computed tomography (CT) and magnetic resonance (MR) imaging modalities, we organized HaN-Seg: The Head and Neck Organ-at-Risk CT and MR Segmentation Challenge. MATERIALS AND METHODS The challenge task was to automatically segment 30 organs-at-risk (OARs) of the HaN region in 14 withheld test cases given the availability of 42 publicly available training cases. Each case consisted of one contrast-enhanced CT and one T1-weighted MR image of the HaN region of the same patient, with up to 30 corresponding reference OAR delineation masks. The performance was evaluated in terms of the Dice similarity coefficient (DSC) and 95-percentile Hausdorff distance (HD95), and statistical ranking was applied for each metric by pairwise comparison of the submitted methods using the Wilcoxon signed-rank test. RESULTS While 23 teams registered for the challenge, only seven submitted their methods for the final phase. The top-performing team achieved a DSC of 76.9 % and a HD95 of 3.5 mm. All participating teams utilized architectures based on U-Net, with the winning team leveraging rigid MR to CT registration combined with network entry-level concatenation of both modalities. CONCLUSION This challenge simulated a real-world clinical scenario by providing non-registered MR and CT images with varying fields-of-view and voxel sizes. Remarkably, the top-performing teams achieved segmentation performance surpassing the inter-observer agreement on the same dataset. These results set a benchmark for future research on this publicly available dataset and on paired multi-modal image segmentation in general.
Collapse
Affiliation(s)
- Gašper Podobnik
- University of Ljubljana, Faculty Electrical Engineering, Tržaška cesta 25, Ljubljana 1000, Slovenia.
| | - Bulat Ibragimov
- University of Ljubljana, Faculty Electrical Engineering, Tržaška cesta 25, Ljubljana 1000, Slovenia; University of Copenhagen, Department of Computer Science, Universitetsparken 1, Copenhagen 2100, Denmark
| | - Elias Tappeiner
- UMIT Tirol - Private University for Health Sciences and Health Technology, Eduard-Wallnöfer-Zentrum 1, Hall in Tirol 6060, Austria
| | - Chanwoong Lee
- Yonsei University, College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul 03722, South Korea; Yonsei Cancer Center, Department of RadiationOncology, 50-1 Yonsei-Ro, Seodaemun-gu, Seoul 03722, South Korea
| | - Jin Sung Kim
- Yonsei University, College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul 03722, South Korea; Yonsei Cancer Center, Department of RadiationOncology, 50-1 Yonsei-Ro, Seodaemun-gu, Seoul 03722, South Korea; Oncosoft Inc, 37 Myeongmul-gil, Seodaemun-gu, Seoul 03722, South Korea
| | - Zacharia Mesbah
- Henri Becquerel Cancer Center, 1 Rue d'Amiens, Rouen 76000, France; Siemens Healthineers, 6 Rue du Général Audran, CS20146, Courbevoie 92412, France
| | - Romain Modzelewski
- Henri Becquerel Cancer Center, 1 Rue d'Amiens, Rouen 76000, France; Litis UR 4108, 684 Av. de l'Université, Saint- Étienne-du-Rouvray 76800, France
| | - Yihao Ma
- Guizhou Medical University, School of Biology & Engineering, 9FW8+2P3, Ankang Avenue, Gui'an New Area, Guiyang, Guizhou Province 561113, China
| | - Fan Yang
- Guizhou Medical University, School of Biology & Engineering, 9FW8+2P3, Ankang Avenue, Gui'an New Area, Guiyang, Guizhou Province 561113, China
| | - Mikołaj Rudecki
- AGH University of Kraków, Department of Measurement and Electronicsal, Mickiewicza 30, Kraków 30-059, Poland
| | - Marek Wodziński
- AGH University of Kraków, Department of Measurement and Electronicsal, Mickiewicza 30, Kraków 30-059, Poland; University of Applied Sciences Western Switzerland, Information Systems Institute, Rue de la Plaine 2, Sierre 3960, Switzerland
| | - Primož Peterlin
- Institute of Oncology, Ljubljana, Zaloška cesta 2, Ljubljana 1000, Slovenia
| | - Primož Strojan
- Institute of Oncology, Ljubljana, Zaloška cesta 2, Ljubljana 1000, Slovenia
| | - Tomaž Vrtovec
- University of Ljubljana, Faculty Electrical Engineering, Tržaška cesta 25, Ljubljana 1000, Slovenia
| |
Collapse
|
9
|
Huynh BN, Groendahl AR, Tomic O, Liland KH, Knudtsen IS, Hoebers F, van Elmpt W, Dale E, Malinen E, Futsaether CM. Deep learning with uncertainty estimation for automatic tumor segmentation in PET/CT of head and neck cancers: impact of model complexity, image processing and augmentation. Biomed Phys Eng Express 2024; 10:055038. [PMID: 39127060 DOI: 10.1088/2057-1976/ad6dcd] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2024] [Accepted: 08/09/2024] [Indexed: 08/12/2024]
Abstract
Objective.Target volumes for radiotherapy are usually contoured manually, which can be time-consuming and prone to inter- and intra-observer variability. Automatic contouring by convolutional neural networks (CNN) can be fast and consistent but may produce unrealistic contours or miss relevant structures. We evaluate approaches for increasing the quality and assessing the uncertainty of CNN-generated contours of head and neck cancers with PET/CT as input.Approach.Two patient cohorts with head and neck squamous cell carcinoma and baseline18F-fluorodeoxyglucose positron emission tomography and computed tomography images (FDG-PET/CT) were collected retrospectively from two centers. The union of manual contours of the gross primary tumor and involved nodes was used to train CNN models for generating automatic contours. The impact of image preprocessing, image augmentation, transfer learning and CNN complexity, architecture, and dimension (2D or 3D) on model performance and generalizability across centers was evaluated. A Monte Carlo dropout technique was used to quantify and visualize the uncertainty of the automatic contours.Main results. CNN models provided contours with good overlap with the manually contoured ground truth (median Dice Similarity Coefficient: 0.75-0.77), consistent with reported inter-observer variations and previous auto-contouring studies. Image augmentation and model dimension, rather than model complexity, architecture, or advanced image preprocessing, had the largest impact on model performance and cross-center generalizability. Transfer learning on a limited number of patients from a separate center increased model generalizability without decreasing model performance on the original training cohort. High model uncertainty was associated with false positive and false negative voxels as well as low Dice coefficients.Significance.High quality automatic contours can be obtained using deep learning architectures that are not overly complex. Uncertainty estimation of the predicted contours shows potential for highlighting regions of the contour requiring manual revision or flagging segmentations requiring manual inspection and intervention.
Collapse
Affiliation(s)
- Bao Ngoc Huynh
- Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway
| | - Aurora Rosvoll Groendahl
- Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway
- Section of Oncology, Vestre Viken Hospital Trust, Drammen, Norway
| | - Oliver Tomic
- Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway
| | - Kristian Hovde Liland
- Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway
| | - Ingerid Skjei Knudtsen
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Trondheim, Norway
| | - Frank Hoebers
- Department of Radiation Oncology (MAASTRO), GROW School for Oncology and Reproduction, Maastricht, Netherlands
| | - Wouter van Elmpt
- Department of Radiation Oncology (MAASTRO), GROW School for Oncology and Reproduction, Maastricht, Netherlands
| | - Einar Dale
- Department of Oncology, Oslo University Hospital, Oslo, Norway
| | - Eirik Malinen
- Department of Medical Physics, Oslo University Hospital, Oslo, Norway
- Department of Physics, University of Oslo, Oslo, Norway
| | | |
Collapse
|
10
|
Akramova R, Watanabe Y. Radiomics as a measure superior to common similarity metrics for tumor segmentation performance evaluation. J Appl Clin Med Phys 2024; 25:e14442. [PMID: 38922790 PMCID: PMC11302798 DOI: 10.1002/acm2.14442] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2024] [Revised: 05/04/2024] [Accepted: 05/20/2024] [Indexed: 06/28/2024] Open
Abstract
PURPOSE To propose radiomics features as a superior measure for evaluating the segmentation ability of physicians and auto-segmentation tools and to compare its performance with the most commonly used metrics: Dice similarity coefficient (DSC), surface Dice similarity coefficient (sDSC), and Hausdorff distance (HD). MATERIALS/METHODS The data of 10 lung cancer patients' CT images with nine tumor segmentations per tumor were downloaded from the RIDER (Reference Database to Evaluate Response) database. Radiomics features of 90 segmented tumors were extracted using the PyRadiomics program. The intraclass correlation coefficient (ICC) of radiomics features were used to evaluate the segmentation similarity and compare their performance with DSC, sDSC, and HD. We calculated one ICC per radiomics feature and per tumor for nine segmentations and 36 ICCs per radiomics feature for 36 pairs of nine segmentations. Meanwhile, there were 360 DSC, sDSC, and HD values calculated for 36 pairs for 10 tumors. RESULTS The ICC of radiomics features exhibited greater sensitivity to segmentation changes than DSC and sDSC. The ICCs of the wavelet-LLL first order Maximum, wavelet-LLL glcm MCC, wavelet-LLL glcm Cluster Shade features ranged from 0.130 to 0.997, 0.033 to 0.978, and 0.160 to 0.998, respectively. On the other hand, all DSC and sDSC were larger than 0.778 and 0.700, respectively, while HD varied from 0 to 1.9 mm. The results indicated that the radiomics features could capture subtle variations in tumor segmentation characteristics, which could not be easily detected by DSC and sDSC. CONCLUSIONS This study demonstrates the superiority of radiomics features with ICC as a measure for evaluating a physician's tumor segmentation ability and the performance of auto-segmentation tools. Radiomics features offer a more sensitive and comprehensive evaluation, providing valuable insights into tumor characteristics. Therefore, the new metrics can be used to evaluate new auto-segmentation methods and enhance trainees' segmentation skills in medical training and education.
Collapse
Affiliation(s)
- Rukhsora Akramova
- Department of Radiation OncologyUniversity of MinnesotaMinneapolisMinnesotaUSA
| | - Yoichi Watanabe
- Department of Radiation OncologyUniversity of MinnesotaMinneapolisMinnesotaUSA
| |
Collapse
|
11
|
Dot G, Chaurasia A, Dubois G, Savoldelli C, Haghighat S, Azimian S, Taramsari AR, Sivaramakrishnan G, Issa J, Dubey A, Schouman T, Gajny L. DentalSegmentator: Robust open source deep learning-based CT and CBCT image segmentation. J Dent 2024; 147:105130. [PMID: 38878813 DOI: 10.1016/j.jdent.2024.105130] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2024] [Revised: 06/08/2024] [Accepted: 06/12/2024] [Indexed: 06/30/2024] Open
Abstract
OBJECTIVES Segmentation of anatomical structures on dento-maxillo-facial (DMF) computed tomography (CT) or cone beam computed tomography (CBCT) scans is increasingly needed in digital dentistry. The main aim of this research was to propose and evaluate a novel open source tool called DentalSegmentator for fully automatic segmentation of five anatomical structures on DMF CT and CBCT scans: maxilla/upper skull, mandible, upper teeth, lower teeth, and the mandibular canal. METHODS A retrospective sample of 470 CT and CBCT scans was used as a training/validation set. The performance and generalizability of the tool was evaluated by comparing segmentations provided by experts and automatic segmentations in two hold-out test datasets: an internal dataset of 133 CT and CBCT scans acquired before orthognathic surgery and an external dataset of 123 CBCT scans randomly sampled from routine examinations in 5 institutions. RESULTS The mean overall results in the internal test dataset (n = 133) were a Dice similarity coefficient (DSC) of 92.2 ± 6.3 % and a normalised surface distance (NSD) of 98.2 ± 2.2 %. The mean overall results on the external test dataset (n = 123) were a DSC of 94.2 ± 7.4 % and a NSD of 98.4 ± 3.6 %. CONCLUSIONS The results obtained from this highly diverse dataset demonstrate that this tool can provide fully automatic and robust multiclass segmentation for DMF CT and CBCT scans. To encourage the clinical deployment of DentalSegmentator, the pre-trained nnU-Net model has been made publicly available along with an extension for the 3D Slicer software. CLINICAL SIGNIFICANCE DentalSegmentator open source 3D Slicer extension provides a free, robust, and easy-to-use approach to obtaining patient-specific three-dimensional models from CT and CBCT scans. These models serve various purposes in a digital dentistry workflow, such as visualization, treatment planning, intervention, and follow-up.
Collapse
Affiliation(s)
- Gauthier Dot
- UFR Odontologie, Universite Paris Cité, Paris, France; Service de Medecine Bucco-Dentaire, AP-HP, Hopital Pitie-Salpetriere, Paris, France; Institut de Biomecanique Humaine Georges Charpak, Arts et Metiers Institute of Technology, Paris, France.
| | - Akhilanand Chaurasia
- Department of Oral Medicine and Radiology, Faculty of Dental Sciences, King George Medical University, Lucknow, Uttar Pradesh, India
| | - Guillaume Dubois
- Institut de Biomecanique Humaine Georges Charpak, Arts et Metiers Institute of Technology, Paris, France; Materialise France, Malakoff, France
| | - Charles Savoldelli
- Department of Oral and Maxillofacial Surgery, Head and Neck Institute, University Hospital of Nice, France
| | - Sara Haghighat
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI On Health, Berlin, Germany
| | - Sarina Azimian
- Research Committee, School of Dentistry, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | | | | | - Julien Issa
- Department of Diagnostics, Chair of Practical Clinical Dentistry, Poznan University of Medical Sciences, Poznan, Poland; Doctoral School, Poznan University of Medical Sciences, Poznan, Poland
| | - Abhishek Dubey
- Department of Oral Medicine and Radiology, Maharana Pratap Dental College, Kanpur, India
| | - Thomas Schouman
- Institut de Biomecanique Humaine Georges Charpak, Arts et Metiers Institute of Technology, Paris, France; AP-HP, Hopital Pitie-Salpetriere, Service de Chirurgie Maxillo-Faciale, Medecine Sorbonne Universite, Paris, France
| | - Laurent Gajny
- Institut de Biomecanique Humaine Georges Charpak, Arts et Metiers Institute of Technology, Paris, France
| |
Collapse
|
12
|
Nielsen CP, Lorenzen EL, Jensen K, Eriksen JG, Johansen J, Gyldenkerne N, Zukauskaite R, Kjellgren M, Maare C, Lønkvist CK, Nowicka-Matus K, Szejniuk WM, Farhadi M, Ujmajuridze Z, Marienhagen K, Johansen TS, Friborg J, Overgaard J, Hansen CR. Interobserver variation in organs at risk contouring in head and neck cancer according to the DAHANCA guidelines. Radiother Oncol 2024; 197:110337. [PMID: 38772479 DOI: 10.1016/j.radonc.2024.110337] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2024] [Revised: 04/24/2024] [Accepted: 05/14/2024] [Indexed: 05/23/2024]
Affiliation(s)
- Camilla Panduro Nielsen
- Laboratory of Radiation Physics, Odense University Hospital, Odense, Denmark; Department of Clinical Research, University of Southern Denmark, Odense, Denmark.
| | - Ebbe L Lorenzen
- Laboratory of Radiation Physics, Odense University Hospital, Odense, Denmark; Department of Clinical Research, University of Southern Denmark, Odense, Denmark
| | - Kenneth Jensen
- Danish Centre for Particle Therapy, Aarhus University Hospital, Denmark
| | - Jesper Grau Eriksen
- Department of Oncology, Aarhus University Hospital, Denmark; Department of Experimental Clinical Oncology, Aarhus University Hospital, Denmark
| | - Jørgen Johansen
- Department of Clinical Research, University of Southern Denmark, Odense, Denmark; Danish Centre for Particle Therapy, Aarhus University Hospital, Denmark; Department of Oncology, Odense University Hospital, Denmark
| | | | - Ruta Zukauskaite
- Department of Clinical Research, University of Southern Denmark, Odense, Denmark; Department of Oncology, Odense University Hospital, Denmark
| | - Martin Kjellgren
- Laboratory of Radiation Physics, Odense University Hospital, Odense, Denmark
| | - Christian Maare
- Department of Oncology, Copenhagen University Hospital Herlev, Denmark
| | | | - Kinga Nowicka-Matus
- Department of Oncology & Clinical Cancer Research Center, Aalborg University Hospital, Denmark
| | - Weronika Maria Szejniuk
- Danish Centre for Particle Therapy, Aarhus University Hospital, Denmark; Department of Oncology & Clinical Cancer Research Center, Aalborg University Hospital, Denmark; Department of Clinical Medicine, Aalborg University, Denmark
| | - Mohammad Farhadi
- Department of Oncology, Zealand University Hospital Næstved, Denmark
| | - Zaza Ujmajuridze
- Department of Oncology, Zealand University Hospital Næstved, Denmark
| | | | - Tanja Stagaard Johansen
- Danish Centre for Particle Therapy, Aarhus University Hospital, Denmark; Department of Oncology, Rigshospitalet, Denmark
| | | | - Jens Overgaard
- Department of Experimental Clinical Oncology, Aarhus University Hospital, Denmark
| | - Christian Rønn Hansen
- Laboratory of Radiation Physics, Odense University Hospital, Odense, Denmark; Department of Clinical Research, University of Southern Denmark, Odense, Denmark; Danish Centre for Particle Therapy, Aarhus University Hospital, Denmark
| |
Collapse
|
13
|
Sjogreen C, Netherton TJ, Lee A, Soliman M, Gay SS, Nguyen C, Mumme R, Vazquez I, Rhee DJ, Cardenas CE, Martel MK, Beadle BM, Court LE. Landmark-based auto-contouring of clinical target volumes for radiotherapy of nasopharyngeal cancer. J Appl Clin Med Phys 2024:e14474. [PMID: 39074490 DOI: 10.1002/acm2.14474] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2024] [Revised: 07/01/2024] [Accepted: 07/09/2024] [Indexed: 07/31/2024] Open
Abstract
BACKGROUND The delineation of clinical target volumes (CTVs) for radiotherapy for nasopharyngeal cancer is complex and varies based on the location and extent of disease. PURPOSE The current study aimed to develop an auto-contouring solution following one protocol guidelines (NRG-HN001) that can be adjusted to meet other guidelines, such as RTOG-0225 and the 2018 International guidelines. METHODS The study used 2-channel 3-dimensional U-Net and nnU-Net framework to auto-contour 27 normal structures in the head and neck (H&N) region that are used to define CTVs in the protocol. To define the CTV-Expansion (CTV1 and CTV2) and CTV-Overall (the outer envelope of all the CTV contours), we used adjustable morphological geometric landmarks and mimicked physician interpretation of the protocol rules by partially or fully including select anatomic structures. The results were evaluated quantitatively using the dice similarity coefficient (DSC) and mean surface distance (MSD) and qualitatively by independent reviews by two H&N radiation oncologists. RESULTS The auto-contouring tool showed high accuracy for nasopharyngeal CTVs. Comparison between auto-contours and clinical contours for 19 patients with cancers of various stages showed a DSC of 0.94 ± 0.02 and MSD of 0.4 ± 0.4 mm for CTV-Expansion and a DSC of 0.83 ± 0.02 and MSD of 2.4 ± 0.5 mm for CTV-Overall. Upon independent review, two H&N physicians found the auto-contours to be usable without edits in 85% and 75% of cases. In 15% of cases, minor edits were required by both physicians. Thus, one physician rated 100% of the auto-contours as usable (use as is, or after minor edits), while the other physician rated 90% as usable. The second physician required major edits in 10% of cases. CONCLUSIONS The study demonstrates the ability of an auto-contouring tool to reliably delineate nasopharyngeal CTVs based on protocol guidelines. The tool was found to be clinically acceptable by two H&N radiation oncology physicians in at least 90% of the cases.
Collapse
Affiliation(s)
- Carlos Sjogreen
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Tucker J Netherton
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Anna Lee
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Moaaz Soliman
- Department of Radiation Oncology, Karmanos Cancer Institute, Detroit, Michigan, USA
| | - Skylar S Gay
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Callistus Nguyen
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Raymond Mumme
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Ivan Vazquez
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Dong Joo Rhee
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Carlos E Cardenas
- Department of Radiation Oncology, University of Alabama Birmingham, Birmingham, Alabama, USA
| | - Mary K Martel
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Beth M Beadle
- Department of Radiation Oncology, Stanford University, Stanford, California, USA
| | - Laurence Edward Court
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| |
Collapse
|
14
|
Wen F, Zhou J, Chen Z, Dou M, Yao Y, Wang X, Xu F, Shen Y. Efficient application of deep learning-based elective lymph node regions delineation for pelvic malignancies. Med Phys 2024. [PMID: 39072765 DOI: 10.1002/mp.17330] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2023] [Revised: 07/02/2024] [Accepted: 07/13/2024] [Indexed: 07/30/2024] Open
Abstract
BACKGROUND While there are established international consensuses on the delineation of pelvic lymph node regions (LNRs), significant inter- and intra-observer variabilities persist. Contouring these clinical target volumes for irradiation in pelvic malignancies is both time-consuming and labor-intensive. PURPOSE The purpose of this study was to develop a deep learning model of pelvic LNRs delineation for patients with pelvic cancers. METHODS Planning computed tomography (CT) studies of 160 patients with pelvic primary malignancies (including rectal, prostate, and cervical cancer) were retrospectively collected and divided into training set (n = 120) and testing set (n = 40). Six pelvic LNRs, including abdominal presacral, pelvic presacral, internal iliac nodes, external iliac nodes, obturator nodes, and inguinal nodes were delineated by two radiation oncologists as ground truth (Gt) contours. The cascaded multi-heads U-net (CMU-net) was constructed based on the Gt contours from training cohort, which was subsequently verified in the testing cohort. The automatic delineation of six LNRs (Auto) was evaluated using dice similarity coefficient (DSC), average surface distance (ASD), 95th percentile Hausdorff distance (HD95), and a 7-point scale score. RESULTS In the testing set, the DSC of six pelvic LNRs by CMU-net model varied from 0.851 to 0.942, ASD from 0.381 to 1.037 mm, and HD95 from 2.025 to 3.697 mm. No significant differences were founded in these three parameters between postoperative and preoperative cases. 95.9% and 96.2% of auto delineations by CMU-net model got a score of 1-3 by two expert radiation oncologists, respectively, meaning only minor edits needed. CONCLUSIONS The CMU-net was successfully developed for automated delineation of pelvic LNRs for pelvic malignancies radiotherapy with improved contouring efficiency and highly consistent, which might justify its implementation in radiotherapy work flow.
Collapse
Affiliation(s)
- Feng Wen
- Division of Abdominal Tumor Multimodality Treatment, Department of Radiation Oncology, Cancer Center, West China Hospital, Sichuan University, Chengdu, China
- Med-X Center for Informatics, Sichuan University, Chengdu, China
| | - Jie Zhou
- Lung Cancer Center, West China Hospital, Sichuan University, Chengdu, China
| | - Zhebin Chen
- Chengdu Institute of Compute Application, Chinese Academy of Sciences, Chengdu, China
| | - Meng Dou
- Chengdu Institute of Compute Application, Chinese Academy of Sciences, Chengdu, China
| | - Yu Yao
- Chengdu Institute of Compute Application, Chinese Academy of Sciences, Chengdu, China
| | - Xin Wang
- Division of Abdominal Tumor Multimodality Treatment, Department of Radiation Oncology, Cancer Center, West China Hospital, Sichuan University, Chengdu, China
| | - Feng Xu
- Lung Cancer Center, West China Hospital, Sichuan University, Chengdu, China
| | - Yali Shen
- Division of Abdominal Tumor Multimodality Treatment, Department of Radiation Oncology, Cancer Center, West China Hospital, Sichuan University, Chengdu, China
| |
Collapse
|
15
|
Liu W, Zhang B, Liu T, Jiang J, Liu Y. Artificial Intelligence in Pancreatic Image Analysis: A Review. SENSORS (BASEL, SWITZERLAND) 2024; 24:4749. [PMID: 39066145 PMCID: PMC11280964 DOI: 10.3390/s24144749] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/05/2024] [Revised: 07/15/2024] [Accepted: 07/16/2024] [Indexed: 07/28/2024]
Abstract
Pancreatic cancer is a highly lethal disease with a poor prognosis. Its early diagnosis and accurate treatment mainly rely on medical imaging, so accurate medical image analysis is especially vital for pancreatic cancer patients. However, medical image analysis of pancreatic cancer is facing challenges due to ambiguous symptoms, high misdiagnosis rates, and significant financial costs. Artificial intelligence (AI) offers a promising solution by relieving medical personnel's workload, improving clinical decision-making, and reducing patient costs. This study focuses on AI applications such as segmentation, classification, object detection, and prognosis prediction across five types of medical imaging: CT, MRI, EUS, PET, and pathological images, as well as integrating these imaging modalities to boost diagnostic accuracy and treatment efficiency. In addition, this study discusses current hot topics and future directions aimed at overcoming the challenges in AI-enabled automated pancreatic cancer diagnosis algorithms.
Collapse
Affiliation(s)
- Weixuan Liu
- Sydney Smart Technology College, Northeastern University at Qinhuangdao, Qinhuangdao 066004, China; (W.L.); (B.Z.)
| | - Bairui Zhang
- Sydney Smart Technology College, Northeastern University at Qinhuangdao, Qinhuangdao 066004, China; (W.L.); (B.Z.)
| | - Tao Liu
- School of Mathematics and Statistics, Northeastern University at Qinhuangdao, Qinhuangdao 066004, China;
| | - Juntao Jiang
- College of Control Science and Engineering, Zhejiang University, Hangzhou 310058, China
| | - Yong Liu
- College of Control Science and Engineering, Zhejiang University, Hangzhou 310058, China
| |
Collapse
|
16
|
Johnson CL, Press RH, Simone CB, Shen B, Tsai P, Hu L, Yu F, Apinorasethkul C, Ackerman C, Zhai H, Lin H, Huang S. Clinical validation of commercial deep-learning based auto-segmentation models for organs at risk in the head and neck region: a single institution study. Front Oncol 2024; 14:1375096. [PMID: 39055552 PMCID: PMC11269179 DOI: 10.3389/fonc.2024.1375096] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2024] [Accepted: 06/20/2024] [Indexed: 07/27/2024] Open
Abstract
Purpose To evaluate organ at risk (OAR) auto-segmentation in the head and neck region of computed tomography images using two different commercially available deep-learning-based auto-segmentation (DLAS) tools in a single institutional clinical applications. Methods Twenty-two OARs were manually contoured by clinicians according to published guidelines on planning computed tomography (pCT) images for 40 clinical head and neck cancer (HNC) cases. Automatic contours were generated for each patient using two deep-learning-based auto-segmentation models-Manteia AccuContour and MIM ProtégéAI. The accuracy and integrity of autocontours (ACs) were then compared to expert contours (ECs) using the Sørensen-Dice similarity coefficient (DSC) and Mean Distance (MD) metrics. Results ACs were generated for 22 OARs using AccuContour and 17 OARs using ProtégéAI with average contour generation time of 1 min/patient and 5 min/patient respectively. EC and AC agreement was highest for the mandible (DSC 0.90 ± 0.16) and (DSC 0.91 ± 0.03), and lowest for the chiasm (DSC 0.28 ± 0.14) and (DSC 0.30 ± 0.14) for AccuContour and ProtégéAI respectively. Using AccuContour, the average MD was<1mm for 10 of the 22 OARs contoured, 1-2mm for 6 OARs, and 2-3mm for 6 OARs. For ProtégéAI, the average mean distance was<1mm for 8 out of 17 OARs, 1-2mm for 6 OARs, and 2-3mm for 3 OARs. Conclusions Both DLAS programs were proven to be valuable tools to significantly reduce the time required to generate large amounts of OAR contours in the head and neck region, even though manual editing of ACs is likely needed prior to implementation into treatment planning. The DSCs and MDs achieved were similar to those reported in other studies that evaluated various other DLAS solutions. Still, small volume structures with nonideal contrast in CT images, such as nerves, are very challenging and will require additional solutions to achieve sufficient results.
Collapse
Affiliation(s)
| | | | | | - Brian Shen
- New York Proton Center, New York, NY, United States
| | | | - Lei Hu
- New York Proton Center, New York, NY, United States
| | - Francis Yu
- New York Proton Center, New York, NY, United States
| | | | | | - Huifang Zhai
- New York Proton Center, New York, NY, United States
| | - Haibo Lin
- New York Proton Center, New York, NY, United States
| | - Sheng Huang
- New York Proton Center, New York, NY, United States
- National Clinical Research Center for Cancer, Tianjin’s Clinical Research Center for Cancer, Tianjin Medical University Cancer Institute & Hospital, Tianjin, China
| |
Collapse
|
17
|
Skett S, Patel T, Duprez D, Gupta S, Netherton T, Trauernicht C, Aldridge S, Eaton D, Cardenas C, Court LE, Smith D, Aggarwal A. Autocontouring of primary lung lesions and nodal disease for radiotherapy based only on computed tomography images. Phys Imaging Radiat Oncol 2024; 31:100637. [PMID: 39297080 PMCID: PMC11408859 DOI: 10.1016/j.phro.2024.100637] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2024] [Revised: 08/21/2024] [Accepted: 08/21/2024] [Indexed: 09/21/2024] Open
Abstract
Background and purpose In many clinics, positron-emission tomography is unavailable and clinician time extremely limited. Here we describe a deep-learning model for autocontouring gross disease for patients undergoing palliative radiotherapy for primary lung lesions and/or hilar/mediastinal nodal disease, based only on computed tomography (CT) images. Materials and methods An autocontouring model (nnU-Net) was trained to contour gross disease in 379 cases (352 training, 27 test); 11 further test cases from an external centre were also included. Anchor-point-based post-processing was applied to remove extraneous autocontoured regions. The autocontours were evaluated quantitatively in terms of volume similarity (Dice similarity coefficient [DSC], surface Dice coefficient, 95th percentile Hausdorff distance [HD95], and mean surface distance), and scored for usability by two consultant oncologists. The magnitude of treatment margin needed to account for geometric discrepancies was also assessed. Results The anchor point process successfully removed all erroneous regions from the autocontoured disease, and identified two cases to be excluded from further analysis due to 'missed' disease. The average DSC and HD95 were 0.8 ± 0.1 and 10.5 ± 7.3 mm, respectively. A 10-mm uniform margin-distance applied to the autocontoured region was found to yield "full coverage" (sensitivity > 0.99) of the clinical contour for 64 % of cases. Ninety-seven percent of evaluated autocontours were scored by both clinicians as requiring no or minor edits. Conclusions Our autocontouring model was shown to produce clinically usable disease outlines, based on CT alone, for approximately two-thirds of patients undergoing lung radiotherapy. Further work is necessary to improve this before clinical implementation.
Collapse
Affiliation(s)
- Stephen Skett
- Guy's and St. Thomas' NHS Foundation Trust, London, United Kingdom
| | - Tina Patel
- Guy's and St. Thomas' NHS Foundation Trust, London, United Kingdom
| | - Didier Duprez
- Stellenbosch University Faculty of Medicine and Health Sciences, Tygerberg Hospital, Cape Town, South Africa
| | - Sunnia Gupta
- Guy's and St. Thomas' NHS Foundation Trust, London, United Kingdom
| | - Tucker Netherton
- The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Christoph Trauernicht
- Stellenbosch University Faculty of Medicine and Health Sciences, Tygerberg Hospital, Cape Town, South Africa
| | - Sarah Aldridge
- Guy's and St. Thomas' NHS Foundation Trust, London, United Kingdom
| | - David Eaton
- Guy's and St. Thomas' NHS Foundation Trust, London, United Kingdom
| | - Carlos Cardenas
- University of Alabama at Birmingham Hazelrig-Salter Radiation Oncology Center, Birmingham, AL, United States
| | - Laurence E Court
- The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Daniel Smith
- Guy's and St. Thomas' NHS Foundation Trust, London, United Kingdom
| | - Ajay Aggarwal
- Guy's and St. Thomas' NHS Foundation Trust, London, United Kingdom
| |
Collapse
|
18
|
Li C, Mao Y, Liang S, Li J, Wang Y, Guo Y. Deep causal learning for pancreatic cancer segmentation in CT sequences. Neural Netw 2024; 175:106294. [PMID: 38657562 DOI: 10.1016/j.neunet.2024.106294] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2023] [Revised: 03/19/2024] [Accepted: 04/05/2024] [Indexed: 04/26/2024]
Abstract
Segmenting the irregular pancreas and inconspicuous tumor simultaneously is an essential but challenging step in diagnosing pancreatic cancer. Current deep-learning (DL) methods usually segment the pancreas or tumor independently using mixed image features, which are disrupted by surrounding complex and low-contrast background tissues. Here, we proposed a deep causal learning framework named CausegNet for pancreas and tumor co-segmentation in 3D CT sequences. Specifically, a causality-aware module and a counterfactual loss are employed to enhance the DL network's comprehension of the anatomical causal relationship between the foreground elements (pancreas and tumor) and the background. By integrating causality into CausegNet, the network focuses solely on extracting intrinsic foreground causal features while effectively learning the potential causality between the pancreas and the tumor. Then based on the extracted causal features, CausegNet applies a counterfactual inference to significantly reduce the background interference and sequentially search for pancreas and tumor from the foreground. Consequently, our approach can handle deformable pancreas and obscure tumors, resulting in superior co-segmentation performance in both public and real clinical datasets, achieving the highest pancreas/tumor Dice coefficients of 86.67%/84.28%. The visualized features and anti-noise experiments further demonstrate the causal interpretability and stability of our method. Furthermore, our approach improves the accuracy and sensitivity of downstream pancreatic cancer risk assessment task by 12.50% and 50.00%, respectively, compared to experienced clinicians, indicating promising clinical applications.
Collapse
Affiliation(s)
- Chengkang Li
- School of Information Science and Technology of Fudan University, Shanghai 200433, China; Key Laboratory of Medical Imaging Computing and Computer Assisted Intervention (MICCAI) of Shanghai, Shanghai 200032, China
| | - Yishen Mao
- Department of Pancreatic Surgery, Pancreatic Disease Institute, Huashan Hospital, Shanghai Medical College, Fudan University, Shanghai 200040, China
| | - Shuyu Liang
- School of Information Science and Technology of Fudan University, Shanghai 200433, China; Key Laboratory of Medical Imaging Computing and Computer Assisted Intervention (MICCAI) of Shanghai, Shanghai 200032, China
| | - Ji Li
- Department of Pancreatic Surgery, Pancreatic Disease Institute, Huashan Hospital, Shanghai Medical College, Fudan University, Shanghai 200040, China.
| | - Yuanyuan Wang
- School of Information Science and Technology of Fudan University, Shanghai 200433, China; Key Laboratory of Medical Imaging Computing and Computer Assisted Intervention (MICCAI) of Shanghai, Shanghai 200032, China.
| | - Yi Guo
- School of Information Science and Technology of Fudan University, Shanghai 200433, China; Key Laboratory of Medical Imaging Computing and Computer Assisted Intervention (MICCAI) of Shanghai, Shanghai 200032, China.
| |
Collapse
|
19
|
Zeverino M, Piccolo C, Marguet M, Jeanneret-Sozzi W, Bourhis J, Bochud F, Moeckli R. Sensitivity of automated and manual treatment planning approaches to contouring variation in early-breast cancer treatment. Phys Med 2024; 123:103402. [PMID: 38875932 DOI: 10.1016/j.ejmp.2024.103402] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Revised: 05/24/2024] [Accepted: 06/05/2024] [Indexed: 06/16/2024] Open
Abstract
PURPOSE One of the advantages of integrating automated processes in treatment planning is the reduction of manual planning variability. This study aims to assess whether a deep-learning-based auto-planning solution can also reduce the contouring variation-related impact on the planned dose for early-breast cancer treatment. METHODS Auto- and manual plans were optimized for 20 patients using both auto- and manual OARs, including both lungs, right breast, heart, and left-anterior-descending (LAD) artery. Differences in terms of recalculated dose (ΔDrcM,ΔDrcA) and reoptimized dose (ΔDroM,ΔDroA) for manual (M) and auto (A)-plans, were evaluated on manual structures. The correlation between several geometric similarities and dose differences was also explored (Spearman's test). RESULTS Auto-contours were found slightly smaller in size than manual contours for right breast and heart and more than twice larger for LAD. Recalculated dose differences were found negligible for both planning approaches except for heart (ΔDrcM=-0.4 Gy, ΔDrcA=-0.3 Gy) and right breast (ΔDrcM=-1.2 Gy, ΔDrcA=-1.3 Gy) maximum dose. Re-optimized dose differences were considered equivalent to recalculated ones for both lungs and LAD, while they were significantly smaller for heart (ΔDroM=-0.2 Gy, ΔDroA=-0.2 Gy) and right breast (ΔDroM =-0.3 Gy, ΔDroA=-0.9 Gy) maximum dose. Twenty-one correlations were found for ΔDrcM,A (M=8,A=13) that reduced to four for ΔDroM,A (M=3,A=1). CONCLUSIONS The sensitivity of auto-planning to contouring variation was found not relevant when compared to manual planning, regardless of the method used to calculate the dose differences. Nonetheless, the method employed to define the dose differences strongly affected the correlation analysis resulting highly reduced when dose was reoptimized, regardless of the planning approach.
Collapse
Affiliation(s)
- Michele Zeverino
- Institute of Radiation Physics, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Consiglia Piccolo
- Institute of Radiation Physics, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Maud Marguet
- Institute of Radiation Physics, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Wendy Jeanneret-Sozzi
- Radiation Oncology Department, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Jean Bourhis
- Radiation Oncology Department, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Francois Bochud
- Institute of Radiation Physics, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Raphaël Moeckli
- Institute of Radiation Physics, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland.
| |
Collapse
|
20
|
Chen J, Zhou M, Wu W, Zhang J, Li Y, Li D. STimage-1K4M: A histopathology image-gene expression dataset for spatial transcriptomics. ARXIV 2024:arXiv:2406.06393v2. [PMID: 38947920 PMCID: PMC11213178] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Subscribe] [Scholar Register] [Indexed: 07/02/2024]
Abstract
Recent advances in multi-modal algorithms have driven and been driven by the increasing availability of large image-text datasets, leading to significant strides in various fields, including computational pathology. However, in most existing medical image-text datasets, the text typically provides high-level summaries that may not sufficiently describe sub-tile regions within a large pathology image. For example, an image might cover an extensive tissue area containing cancerous and healthy regions, but the accompanying text might only specify that this image is a cancer slide, lacking the nuanced details needed for in-depth analysis. In this study, we introduce STimage-1K4M, a novel dataset designed to bridge this gap by providing genomic features for sub-tile images. STimage-1K4M contains 1,149 images derived from spatial transcriptomics data, which captures gene expression information at the level of individual spatial spots within a pathology image. Specifically, each image in the dataset is broken down into smaller sub-image tiles, with each tile paired with 15,000 - 30,000 dimensional gene expressions. With 4,293,195 pairs of sub-tile images and gene expressions, STimage-1K4M offers unprecedented granularity, paving the way for a wide range of advanced research in multi-modal data analysis an innovative applications in computational pathology, and beyond.
Collapse
Affiliation(s)
| | | | - Wenrong Wu
- University of North Carolina at Chapel Hill
| | | | - Yun Li
- University of North Carolina at Chapel Hill
| | - Didong Li
- University of North Carolina at Chapel Hill
| |
Collapse
|
21
|
Bakx N, Van der Sangen M, Theuws J, Bluemink J, Hurkmans C. Comparison of the use of a clinically implemented deep learning segmentation model with the simulated study setting for breast cancer patients receiving radiotherapy. Acta Oncol 2024; 63:477-481. [PMID: 38899395 PMCID: PMC11332522 DOI: 10.2340/1651-226x.2024.34986] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Accepted: 04/24/2024] [Indexed: 06/21/2024]
Abstract
BACKGROUND Deep learning (DL) models for auto-segmentation in radiotherapy have been extensively studied in retrospective and pilot settings. However, these studies might not reflect the clinical setting. This study compares the use of a clinically implemented in-house trained DL segmentation model for breast cancer to a previously performed pilot study to assess possible differences in performance or acceptability. MATERIAL AND METHODS Sixty patients with whole breast radiotherapy, with or without an indication for locoregional radiotherapy were included. Structures were qualitatively scored by radiotherapy technologists and radiation oncologists. Quantitative evaluation was performed using dice-similarity coefficient (DSC), 95th percentile of Hausdorff Distance (95%HD) and surface DSC (sDSC), and time needed for generating, checking, and correcting structures was measured. RESULTS Ninety-three percent of all contours in clinic were scored as clinically acceptable or usable as a starting point, comparable to 92% achieved in the pilot study. Compared to the pilot study, no significant changes in time reduction were achieved for organs at risks (OARs). For target volumes, significantly more time was needed compared to the pilot study for patients including lymph node levels 1-4, although time reduction was still 33% compared to manual segmentation. Almost all contours have better DSC and 95%HD than inter-observer variations. Only CTVn4 scored worse for both metrics, and the thyroid had a higher 95%HD value. INTERPRETATION The use of the DL model in clinical practice is comparable to the pilot study, showing high acceptability rates and time reduction.
Collapse
Affiliation(s)
- Nienke Bakx
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, The Netherlands.
| | | | - Jacqueline Theuws
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, The Netherlands
| | - Johanna Bluemink
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, The Netherlands
| | - Coen Hurkmans
- Catharina Hospital, Department of Radiation Oncology, Eindhoven, The Netherlands; Technical University Eindhoven, Departments of Applied Physics and Electrical Engineering, Eindhoven, The Netherlands
| |
Collapse
|
22
|
He D, Udupa JK, Tong Y, Torigian DA. Predicting the effort required to manually mend auto-segmentations. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2024:2024.06.12.24308779. [PMID: 38947045 PMCID: PMC11213037 DOI: 10.1101/2024.06.12.24308779] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/02/2024]
Abstract
Auto-segmentation is one of the critical and foundational steps for medical image analysis. The quality of auto-segmentation techniques influences the efficiency of precision radiology and radiation oncology since high- quality auto-segmentations usually require limited manual correction. Segmentation metrics are necessary and important to evaluate auto-segmentation results and guide the development of auto-segmentation techniques. Currently widely applied segmentation metrics usually compare the auto-segmentation with the ground truth in terms of the overlapping area (e.g., Dice Coefficient (DC)) or the distance between boundaries (e.g., Hausdorff Distance (HD)). However, these metrics may not well indicate the manual mending effort required when observing the auto-segmentation results in clinical practice. In this article, we study different segmentation metrics to explore the appropriate way of evaluating auto-segmentations with clinical demands. The mending time for correcting auto-segmentations by experts is recorded to indicate the required mending effort. Five well-defined metrics, the overlapping area-based metric DC, the segmentation boundary distance-based metric HD, the segmentation boundary length-based metrics surface DC (surDC) and added path length (APL), and a newly proposed hybrid metric Mendability Index (MI) are discussed in the correlation analysis experiment and regression experiment. In addition to these explicitly defined metrics, we also preliminarily explore the feasibility of using deep learning models to predict the mending effort, which takes segmentation masks and the original images as the input. Experiments are conducted using datasets of 7 objects from three different institutions, which contain the original computed tomography (CT) images, the ground truth segmentations, the auto-segmentations, the corrected segmentations, and the recorded mending time. According to the correlation analysis and regression experiments for the five well-defined metrics, the variety of MI shows the best performance to indicate the mending effort for sparse objects, while the variety of HD works best when assessing the mending effort for non-sparse objects. Moreover, the deep learning models could well predict efforts required to mend auto-segmentations, even without the need of ground truth segmentations, demonstrating the potential of a novel and easy way to evaluate and boost auto-segmentation techniques.
Collapse
Affiliation(s)
- Da He
- Medical Image Processing Group, 602 Goddard building, 3710 Hamilton Walk, Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104, United States
- University of Michigan-Shanghai Jiao Tong University Joint Institute, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Jayaram K. Udupa
- Medical Image Processing Group, 602 Goddard building, 3710 Hamilton Walk, Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Yubing Tong
- Medical Image Processing Group, 602 Goddard building, 3710 Hamilton Walk, Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Drew A. Torigian
- Medical Image Processing Group, 602 Goddard building, 3710 Hamilton Walk, Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104, United States
| |
Collapse
|
23
|
Sahlsten J, Jaskari J, Wahid KA, Ahmed S, Glerean E, He R, Kann BH, Mäkitie A, Fuller CD, Naser MA, Kaski K. Application of simultaneous uncertainty quantification and segmentation for oropharyngeal cancer use-case with Bayesian deep learning. COMMUNICATIONS MEDICINE 2024; 4:110. [PMID: 38851837 PMCID: PMC11162474 DOI: 10.1038/s43856-024-00528-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 05/16/2024] [Indexed: 06/10/2024] Open
Abstract
BACKGROUND Radiotherapy is a core treatment modality for oropharyngeal cancer (OPC), where the primary gross tumor volume (GTVp) is manually segmented with high interobserver variability. This calls for reliable and trustworthy automated tools in clinician workflow. Therefore, accurate uncertainty quantification and its downstream utilization is critical. METHODS Here we propose uncertainty-aware deep learning for OPC GTVp segmentation, and illustrate the utility of uncertainty in multiple applications. We examine two Bayesian deep learning (BDL) models and eight uncertainty measures, and utilize a large multi-institute dataset of 292 PET/CT scans to systematically analyze our approach. RESULTS We show that our uncertainty-based approach accurately predicts the quality of the deep learning segmentation in 86.6% of cases, identifies low performance cases for semi-automated correction, and visualizes regions of the scans where the segmentations likely fail. CONCLUSIONS Our BDL-based analysis provides a first-step towards more widespread implementation of uncertainty quantification in OPC GTVp segmentation.
Collapse
Affiliation(s)
- Jaakko Sahlsten
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Joel Jaskari
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Kareem A Wahid
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Sara Ahmed
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Enrico Glerean
- Department of Neuroscience and Biomedical Engineering, Aalto University School of Science, Espoo, Finland
| | - Renjie He
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Benjamin H Kann
- Artificial Intelligence in Medicine Program, Brigham and Women's Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA, USA
| | - Antti Mäkitie
- Department of Otorhinolaryngology, Head and Neck Surgery, University of Helsinki and Helsinki University Hospital, Helsinki, Finland
- Research Program in Systems Oncology, University of Helsinki, Helsinki, Finland
| | - Clifton D Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Mohamed A Naser
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA.
| | - Kimmo Kaski
- Department of Computer Science, Aalto University School of Science, Espoo, Finland.
| |
Collapse
|
24
|
Takeya A, Watanabe K, Haga A. Fine structural human phantom in dentistry and instance tooth segmentation. Sci Rep 2024; 14:12630. [PMID: 38824210 PMCID: PMC11144222 DOI: 10.1038/s41598-024-63319-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/01/2024] [Accepted: 05/28/2024] [Indexed: 06/03/2024] Open
Abstract
In this study, we present the development of a fine structural human phantom designed specifically for applications in dentistry. This research focused on assessing the viability of applying medical computer vision techniques to the task of segmenting individual teeth within a phantom. Using a virtual cone-beam computed tomography (CBCT) system, we generated over 170,000 training datasets. These datasets were produced by varying the elemental densities and tooth sizes within the human phantom, as well as varying the X-ray spectrum, noise intensity, and projection cutoff intensity in the virtual CBCT system. The deep-learning (DL) based tooth segmentation model was trained using the generated datasets. The results demonstrate an agreement with manual contouring when applied to clinical CBCT data. Specifically, the Dice similarity coefficient exceeded 0.87, indicating the robust performance of the developed segmentation model even when virtual imaging was used. The present results show the practical utility of virtual imaging techniques in dentistry and highlight the potential of medical computer vision for enhancing precision and efficiency in dental imaging processes.
Collapse
Affiliation(s)
- Atsushi Takeya
- Graduate School of Biomedical Sciences, Tokushima University, 3-18-15 Kuramoto-cho, Tokushima, 770-8503, Japan
| | - Keiichiro Watanabe
- Graduate School of Biomedical Sciences, Tokushima University, 3-18-15 Kuramoto-cho, Tokushima, 770-8503, Japan
| | - Akihiro Haga
- Graduate School of Biomedical Sciences, Tokushima University, 3-18-15 Kuramoto-cho, Tokushima, 770-8503, Japan.
| |
Collapse
|
25
|
Wahid KA, Sahin O, Kundu S, Lin D, Alanis A, Tehami S, Kamel S, Duke S, Sherer MV, Rasmussen M, Korreman S, Fuentes D, Cislo M, Nelms BE, Christodouleas JP, Murphy JD, Mohamed AS, He R, Naser MA, Gillespie EF, Fuller CD. Associations Between Radiation Oncologist Demographic Factors and Segmentation Similarity Benchmarks: Insights From a Crowd-Sourced Challenge Using Bayesian Estimation. JCO Clin Cancer Inform 2024; 8:e2300174. [PMID: 38870441 PMCID: PMC11214868 DOI: 10.1200/cci.23.00174] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Revised: 01/08/2024] [Accepted: 04/03/2024] [Indexed: 06/15/2024] Open
Abstract
PURPOSE The quality of radiotherapy auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of clinician-derived segmentations are poorly understood; our study aims to quantify these factors. METHODS Organ at risk (OAR) and tumor-related segmentations provided by radiation oncologists from the Contouring Collaborative for Consensus in Radiation Oncology data set were used. Segmentations were derived from five disease sites: breast, sarcoma, head and neck (H&N), gynecologic (GYN), and GI. Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus, which served as a reference standard benchmark. The Dice similarity coefficient (DSC) was primarily used as a metric for the comparisons. DSC was stratified into binary groups on the basis of structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Bayesian estimation were used to investigate the association between demographic variables and the binarized DSC for each disease site. Variables with a highest density interval excluding zero were considered to substantially affect the outcome measure. RESULTS Five hundred seventy-four, 110, 452, 112, and 48 segmentations were used for the breast, sarcoma, H&N, GYN, and GI cases, respectively. The median percentage of segmentations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumors, respectively. Regression analysis revealed that the structure being tumor-related had a substantial negative impact on binarized DSC for the breast, sarcoma, H&N, and GI cases. There were no recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations. CONCLUSION Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality relative to benchmarks.
Collapse
Affiliation(s)
- Kareem A. Wahid
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Onur Sahin
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Suprateek Kundu
- Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Diana Lin
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY
| | - Anthony Alanis
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Salik Tehami
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Serageldin Kamel
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Simon Duke
- Department of Radiation Oncology, Cambridge University Hospitals, Cambridge, United Kingdom
| | - Michael V. Sherer
- Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, CA
| | - Mathis Rasmussen
- Department of Oncology, Aarhus University Hospital, Aarhus, Denmark
| | - Stine Korreman
- Department of Oncology, Aarhus University Hospital, Aarhus, Denmark
| | - David Fuentes
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Michael Cislo
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY
| | | | - John P. Christodouleas
- Department of Radiation Oncology, The University of Pennsylvania Cancer Center, Philadelphia, PA
- Elekta, Atlanta, GA
| | - James D. Murphy
- Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, CA
| | - Abdallah S.R. Mohamed
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Renjie He
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Mohammed A. Naser
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Erin F. Gillespie
- Department of Radiation Oncology, University of Washington Fred Hutchinson Cancer Center, Seattle, WA
| | - Clifton D. Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| |
Collapse
|
26
|
Alrashdi I. Fog-based deep learning framework for real-time pandemic screening in smart cities from multi-site tomographies. BMC Med Imaging 2024; 24:123. [PMID: 38797827 PMCID: PMC11129417 DOI: 10.1186/s12880-024-01302-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2024] [Accepted: 05/20/2024] [Indexed: 05/29/2024] Open
Abstract
The quick proliferation of pandemic diseases has been imposing many concerns on the international health infrastructure. To combat pandemic diseases in smart cities, Artificial Intelligence of Things (AIoT) technology, based on the integration of artificial intelligence (AI) with the Internet of Things (IoT), is commonly used to promote efficient control and diagnosis during the outbreak, thereby minimizing possible losses. However, the presence of multi-source institutional data remains one of the major challenges hindering the practical usage of AIoT solutions for pandemic disease diagnosis. This paper presents a novel framework that utilizes multi-site data fusion to boost the accurateness of pandemic disease diagnosis. In particular, we focus on a case study of COVID-19 lesion segmentation, a crucial task for understanding disease progression and optimizing treatment strategies. In this study, we propose a novel multi-decoder segmentation network for efficient segmentation of infections from cross-domain CT scans in smart cities. The multi-decoder segmentation network leverages data from heterogeneous domains and utilizes strong learning representations to accurately segment infections. Performance evaluation of the multi-decoder segmentation network was conducted on three publicly accessible datasets, demonstrating robust results with an average dice score of 89.9% and an average surface dice of 86.87%. To address scalability and latency issues associated with centralized cloud systems, fog computing (FC) emerges as a viable solution. FC brings resources closer to the operator, offering low latency and energy-efficient data management and processing. In this context, we propose a unique FC technique called PANDFOG to deploy the multi-decoder segmentation network on edge nodes for practical and clinical applications of automated COVID-19 pneumonia analysis. The results of this study highlight the efficacy of the multi-decoder segmentation network in accurately segmenting infections from cross-domain CT scans. Moreover, the proposed PANDFOG system demonstrates the practical deployment of the multi-decoder segmentation network on edge nodes, providing real-time access to COVID-19 segmentation findings for improved patient monitoring and clinical decision-making.
Collapse
Affiliation(s)
- Ibrahim Alrashdi
- Department of Computer Science, College of Computer and Information Sciences, Jouf University, 72388, Sakaka, Aljouf, Saudi Arabia.
| |
Collapse
|
27
|
Koitka S, Baldini G, Kroll L, van Landeghem N, Pollok OB, Haubold J, Pelka O, Kim M, Kleesiek J, Nensa F, Hosch R. SAROS: A dataset for whole-body region and organ segmentation in CT imaging. Sci Data 2024; 11:483. [PMID: 38729970 PMCID: PMC11087485 DOI: 10.1038/s41597-024-03337-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2023] [Accepted: 05/01/2024] [Indexed: 05/12/2024] Open
Abstract
The Sparsely Annotated Region and Organ Segmentation (SAROS) dataset was created using data from The Cancer Imaging Archive (TCIA) to provide a large open-access CT dataset with high-quality annotations of body landmarks. In-house segmentation models were employed to generate annotation proposals on randomly selected cases from TCIA. The dataset includes 13 semantic body region labels (abdominal/thoracic cavity, bones, brain, breast implant, mediastinum, muscle, parotid/submandibular/thyroid glands, pericardium, spinal cord, subcutaneous tissue) and six body part labels (left/right arm/leg, head, torso). Case selection was based on the DICOM series description, gender, and imaging protocol, resulting in 882 patients (438 female) for a total of 900 CTs. Manual review and correction of proposals were conducted in a continuous quality control cycle. Only every fifth axial slice was annotated, yielding 20150 annotated slices from 28 data collections. For the reproducibility on downstream tasks, five cross-validation folds and a test set were pre-defined. The SAROS dataset serves as an open-access resource for training and evaluating novel segmentation models, covering various scanner vendors and diseases.
Collapse
Affiliation(s)
- Sven Koitka
- Institute of Interventional and Diagnostic Radiology and Neuroradiology, University Hospital Essen, Essen, Germany
- Institute for Artificial Intelligence in Medicine, University Hospital Essen, Essen, Germany
| | - Giulia Baldini
- Institute of Interventional and Diagnostic Radiology and Neuroradiology, University Hospital Essen, Essen, Germany
- Institute for Artificial Intelligence in Medicine, University Hospital Essen, Essen, Germany
| | - Lennard Kroll
- Institute of Interventional and Diagnostic Radiology and Neuroradiology, University Hospital Essen, Essen, Germany
| | - Natalie van Landeghem
- Institute of Interventional and Diagnostic Radiology and Neuroradiology, University Hospital Essen, Essen, Germany
| | - Olivia B Pollok
- Institute for Artificial Intelligence in Medicine, University Hospital Essen, Essen, Germany
| | - Johannes Haubold
- Institute of Interventional and Diagnostic Radiology and Neuroradiology, University Hospital Essen, Essen, Germany
| | - Obioma Pelka
- Institute for Artificial Intelligence in Medicine, University Hospital Essen, Essen, Germany
- Data Integration Center, Central IT Department, University Hospital Essen, Essen, Germany
| | - Moon Kim
- Institute for Artificial Intelligence in Medicine, University Hospital Essen, Essen, Germany
| | - Jens Kleesiek
- Institute for Artificial Intelligence in Medicine, University Hospital Essen, Essen, Germany
| | - Felix Nensa
- Institute of Interventional and Diagnostic Radiology and Neuroradiology, University Hospital Essen, Essen, Germany
- Institute for Artificial Intelligence in Medicine, University Hospital Essen, Essen, Germany
| | - René Hosch
- Institute of Interventional and Diagnostic Radiology and Neuroradiology, University Hospital Essen, Essen, Germany.
- Institute for Artificial Intelligence in Medicine, University Hospital Essen, Essen, Germany.
| |
Collapse
|
28
|
Rong Y, Chen Q, Fu Y, Yang X, Al-Hallaq HA, Wu QJ, Yuan L, Xiao Y, Cai B, Latifi K, Benedict SH, Buchsbaum JC, Qi XS. NRG Oncology Assessment of Artificial Intelligence Deep Learning-Based Auto-segmentation for Radiation Therapy: Current Developments, Clinical Considerations, and Future Directions. Int J Radiat Oncol Biol Phys 2024; 119:261-280. [PMID: 37972715 PMCID: PMC11023777 DOI: 10.1016/j.ijrobp.2023.10.033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Revised: 09/16/2023] [Accepted: 10/14/2023] [Indexed: 11/19/2023]
Abstract
Deep learning neural networks (DLNN) in Artificial intelligence (AI) have been extensively explored for automatic segmentation in radiotherapy (RT). In contrast to traditional model-based methods, data-driven AI-based models for auto-segmentation have shown high accuracy in early studies in research settings and controlled environment (single institution). Vendor-provided commercial AI models are made available as part of the integrated treatment planning system (TPS) or as a stand-alone tool that provides streamlined workflow interacting with the main TPS. These commercial tools have drawn clinics' attention thanks to their significant benefit in reducing the workload from manual contouring and shortening the duration of treatment planning. However, challenges occur when applying these commercial AI-based segmentation models to diverse clinical scenarios, particularly in uncontrolled environments. Contouring nomenclature and guideline standardization has been the main task undertaken by the NRG Oncology. AI auto-segmentation holds the potential clinical trial participants to reduce interobserver variations, nomenclature non-compliance, and contouring guideline deviations. Meanwhile, trial reviewers could use AI tools to verify contour accuracy and compliance of those submitted datasets. In recognizing the growing clinical utilization and potential of these commercial AI auto-segmentation tools, NRG Oncology has formed a working group to evaluate the clinical utilization and potential of commercial AI auto-segmentation tools. The group will assess in-house and commercially available AI models, evaluation metrics, clinical challenges, and limitations, as well as future developments in addressing these challenges. General recommendations are made in terms of the implementation of these commercial AI models, as well as precautions in recognizing the challenges and limitations.
Collapse
Affiliation(s)
- Yi Rong
- Mayo Clinic Arizona, Phoenix, AZ
| | - Quan Chen
- City of Hope Comprehensive Cancer Center Duarte, CA
| | - Yabo Fu
- Memorial Sloan Kettering Cancer Center, Commack, NY
| | | | | | | | - Lulin Yuan
- Virginia Commonwealth University, Richmond, VA
| | - Ying Xiao
- University of Pennsylvania/Abramson Cancer Center, Philadelphia, PA
| | - Bin Cai
- The University of Texas Southwestern Medical Center, Dallas, TX
| | | | - Stanley H Benedict
- University of California Davis Comprehensive Cancer Center, Sacramento, CA
| | | | - X Sharon Qi
- University of California Los Angeles, Los Angeles, CA
| |
Collapse
|
29
|
Tada DK, Teng P, Vyapari K, Banola A, Foster G, Diaz E, Kim GHJ, Goldin JG, Abtin F, McNitt-Gray M, Brown MS. Quantifying lung fissure integrity using a three-dimensional patch-based convolutional neural network on CT images for emphysema treatment planning. J Med Imaging (Bellingham) 2024; 11:034502. [PMID: 38817711 PMCID: PMC11135203 DOI: 10.1117/1.jmi.11.3.034502] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2023] [Revised: 04/19/2024] [Accepted: 05/03/2024] [Indexed: 06/01/2024] Open
Abstract
Purpose Evaluation of lung fissure integrity is required to determine whether emphysema patients have complete fissures and are candidates for endobronchial valve (EBV) therapy. We propose a deep learning (DL) approach to segment fissures using a three-dimensional patch-based convolutional neural network (CNN) and quantitatively assess fissure integrity on CT to evaluate it in subjects with severe emphysema. Approach From an anonymized image database of patients with severe emphysema, 129 CT scans were used. Lung lobe segmentations were performed to identify lobar regions, and the boundaries among these regions were used to construct approximate interlobar regions of interest (ROIs). The interlobar ROIs were annotated by expert image analysts to identify voxels where the fissure was present and create a reference ROI that excluded non-fissure voxels (where the fissure is incomplete). A CNN configured by nnU-Net was trained using 86 CT scans and their corresponding reference ROIs to segment the ROIs of left oblique fissure (LOF), right oblique fissure (ROF), and right horizontal fissure (RHF). For an independent test set of 43 cases, fissure integrity was quantified by mapping the segmented fissure ROI along the interlobar ROI. A fissure integrity score (FIS) was then calculated as the percentage of labeled fissure voxels divided by total voxels in the interlobar ROI. Predicted FIS (p-FIS) was quantified from the CNN output, and statistical analyses were performed comparing p-FIS and reference FIS (r-FIS). Results The absolute percent error mean (±SD) between r-FIS and p-FIS for the test set was 4.0% (± 4.1 % ), 6.0% (± 9.3 % ), and 12.2% (± 12.5 % ) for the LOF, ROF, and RHF, respectively. Conclusions A DL approach was developed to segment lung fissures on CT images and accurately quantify FIS. It has potential to assist in the identification of emphysema patients who would benefit from EBV treatment.
Collapse
Affiliation(s)
- Dallas K. Tada
- The University of California, Los Angeles (UCLA), David Geffen School of Medicine at UCLA, Center for Computer Vision and Imaging Biomarkers, Department of Radiological Sciences, Los Angeles, California, United States
| | - Pangyu Teng
- The University of California, Los Angeles (UCLA), David Geffen School of Medicine at UCLA, Center for Computer Vision and Imaging Biomarkers, Department of Radiological Sciences, Los Angeles, California, United States
| | - Kalyani Vyapari
- The University of California, Los Angeles (UCLA), David Geffen School of Medicine at UCLA, Center for Computer Vision and Imaging Biomarkers, Department of Radiological Sciences, Los Angeles, California, United States
| | - Ashley Banola
- The University of California, Los Angeles (UCLA), David Geffen School of Medicine at UCLA, Center for Computer Vision and Imaging Biomarkers, Department of Radiological Sciences, Los Angeles, California, United States
| | - George Foster
- The University of California, Los Angeles (UCLA), David Geffen School of Medicine at UCLA, Center for Computer Vision and Imaging Biomarkers, Department of Radiological Sciences, Los Angeles, California, United States
| | - Esteban Diaz
- The University of California, Los Angeles (UCLA), David Geffen School of Medicine at UCLA, Center for Computer Vision and Imaging Biomarkers, Department of Radiological Sciences, Los Angeles, California, United States
| | - Grace Hyun J. Kim
- The University of California, Los Angeles (UCLA), David Geffen School of Medicine at UCLA, Center for Computer Vision and Imaging Biomarkers, Department of Radiological Sciences, Los Angeles, California, United States
| | - Jonathan G. Goldin
- The University of California, Los Angeles (UCLA), David Geffen School of Medicine at UCLA, Center for Computer Vision and Imaging Biomarkers, Department of Radiological Sciences, Los Angeles, California, United States
| | - Fereidoun Abtin
- The University of California, Los Angeles (UCLA), David Geffen School of Medicine at UCLA, Center for Computer Vision and Imaging Biomarkers, Department of Radiological Sciences, Los Angeles, California, United States
| | - Michael McNitt-Gray
- The University of California, Los Angeles (UCLA), David Geffen School of Medicine at UCLA, Center for Computer Vision and Imaging Biomarkers, Department of Radiological Sciences, Los Angeles, California, United States
| | - Matthew S. Brown
- The University of California, Los Angeles (UCLA), David Geffen School of Medicine at UCLA, Center for Computer Vision and Imaging Biomarkers, Department of Radiological Sciences, Los Angeles, California, United States
| |
Collapse
|
30
|
Strasberg HR, Jackson GP, Bakken SR, Boxwala A, Richardson JE, Morrow JD. Perspectives on the role of industry in informatics research and authorship. J Am Med Inform Assoc 2024; 31:1206-1210. [PMID: 38531679 PMCID: PMC11031207 DOI: 10.1093/jamia/ocae063] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2023] [Revised: 01/23/2024] [Accepted: 03/06/2024] [Indexed: 03/28/2024] Open
Abstract
OBJECTIVES Advances in informatics research come from academic, nonprofit, and for-profit industry organizations, and from academic-industry partnerships. While scientific studies of commercial products may offer critical lessons for the field, manuscripts authored by industry scientists are sometimes categorically rejected. We review historical context, community perceptions, and guidelines on informatics authorship. PROCESS We convened an expert panel at the American Medical Informatics Association 2022 Annual Symposium to explore the role of industry in informatics research and authorship with community input. The panel summarized session themes and prepared recommendations. CONCLUSIONS Authorship for informatics research, regardless of affiliation, should be determined by International Committee of Medical Journal Editors uniform requirements for authorship. All authors meeting criteria should be included, and categorical rejection based on author affiliation is unethical. Informatics research should be evaluated based on its scientific rigor; all sources of bias and conflicts of interest should be addressed through disclosure and, when possible, methodological mitigation.
Collapse
Affiliation(s)
- Howard R Strasberg
- Clinical Effectiveness, Wolters Kluwer Health, Waltham, MA 02451, United States
| | - Gretchen Purcell Jackson
- Intuitive Surgical, Sunnyvale, CA 94086, United States
- Department of Pediatric Surgery, Vanderbilt University Medical Center, Nashville, TN 37232, United States
| | - Suzanne R Bakken
- School of Nursing, Department of Biomedical Informatics, and Data Science Institute, Columbia University, New York, NY 10032, United States
| | - Aziz Boxwala
- Elimu Informatics, La Jolla, CA 92037, United States
| | - Joshua E Richardson
- Center for Informatics, RTI International, Berkeley, CA 94704, United States
| | - Jon D Morrow
- Department of Obstetrics and Gynecology, New York University School of Medicine, New York, NY 10016, United States
| |
Collapse
|
31
|
Mody P, Huiskes M, Chaves-de-Plaza NF, Onderwater A, Lamsma R, Hildebrandt K, Hoekstra N, Astreinidou E, Staring M, Dankers F. Large-scale dose evaluation of deep learning organ contours in head-and-neck radiotherapy by leveraging existing plans. Phys Imaging Radiat Oncol 2024; 30:100572. [PMID: 38633281 PMCID: PMC11021837 DOI: 10.1016/j.phro.2024.100572] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2023] [Revised: 03/21/2024] [Accepted: 03/21/2024] [Indexed: 04/19/2024] Open
Abstract
Background and purpose Retrospective dose evaluation for organ-at-risk auto-contours has previously used small cohorts due to additional manual effort required for treatment planning on auto-contours. We aimed to do this at large scale, by a) proposing and assessing an automated plan optimization workflow that used existing clinical plan parameters and b) using it for head-and-neck auto-contour dose evaluation. Materials and methods Our automated workflow emulated our clinic's treatment planning protocol and reused existing clinical plan optimization parameters. This workflow recreated the original clinical plan (P OG ) with manual contours (P MC ) and evaluated the dose effect (P OG - P MC ) on 70 photon and 30 proton plans of head-and-neck patients. As a use-case, the same workflow (and parameters) created a plan using auto-contours (P AC ) of eight head-and-neck organs-at-risk from a commercial tool and evaluated their dose effect (P MC - P AC ). Results For plan recreation (P OG - P MC ), our workflow had a median impact of 1.0% and 1.5% across dose metrics of auto-contours, for photon and proton respectively. Computer time of automated planning was 25% (photon) and 42% (proton) of manual planning time. For auto-contour evaluation (P MC - P AC ), we noticed an impact of 2.0% and 2.6% for photon and proton radiotherapy. All evaluations had a median Δ NTCP (Normal Tissue Complication Probability) less than 0.3%. Conclusions The plan replication capability of our automated program provides a blueprint for other clinics to perform auto-contour dose evaluation with large patient cohorts. Finally, despite geometric differences, auto-contours had a minimal median dose impact, hence inspiring confidence in their utility and facilitating their clinical adoption.
Collapse
Affiliation(s)
- Prerak Mody
- Division of Image Processing (LKEB), Department of Radiology, Leiden University Medical Center, Leiden 2333 ZA, The Netherlands
- HollandPTC consortium – Erasmus Medical Center, Rotterdam, Holland Proton Therapy Centre, Delft, Leiden University Medical Center (LUMC), Leiden and Delft University of Technology, Delft, The Netherlands
| | - Merle Huiskes
- Department of Radiation Oncology, Leiden University Medical Center, Leiden 2333 ZA, The Netherlands
| | - Nicolas F. Chaves-de-Plaza
- HollandPTC consortium – Erasmus Medical Center, Rotterdam, Holland Proton Therapy Centre, Delft, Leiden University Medical Center (LUMC), Leiden and Delft University of Technology, Delft, The Netherlands
- Computer Graphics and Visualization Group, EEMCS, TU Delft, Delft 2628 CD, The Netherlands
| | - Alice Onderwater
- Department of Radiation Oncology, Leiden University Medical Center, Leiden 2333 ZA, The Netherlands
| | - Rense Lamsma
- Department of Radiation Oncology, Leiden University Medical Center, Leiden 2333 ZA, The Netherlands
| | - Klaus Hildebrandt
- Computer Graphics and Visualization Group, EEMCS, TU Delft, Delft 2628 CD, The Netherlands
| | - Nienke Hoekstra
- Department of Radiation Oncology, Leiden University Medical Center, Leiden 2333 ZA, The Netherlands
| | - Eleftheria Astreinidou
- Department of Radiation Oncology, Leiden University Medical Center, Leiden 2333 ZA, The Netherlands
| | - Marius Staring
- Division of Image Processing (LKEB), Department of Radiology, Leiden University Medical Center, Leiden 2333 ZA, The Netherlands
- Department of Radiation Oncology, Leiden University Medical Center, Leiden 2333 ZA, The Netherlands
| | - Frank Dankers
- Department of Radiation Oncology, Leiden University Medical Center, Leiden 2333 ZA, The Netherlands
| |
Collapse
|
32
|
Welch ML, Kim S, Hope AJ, Huang SH, Lu Z, Marsilla J, Kazmierski M, Rey-McIntyre K, Patel T, O'Sullivan B, Waldron J, Bratman S, Haibe-Kains B, Tadic T. RADCURE: An open-source head and neck cancer CT dataset for clinical radiation therapy insights. Med Phys 2024; 51:3101-3109. [PMID: 38362943 DOI: 10.1002/mp.16972] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2023] [Revised: 01/17/2024] [Accepted: 01/17/2024] [Indexed: 02/17/2024] Open
Abstract
PURPOSE This manuscript presents RADCURE, one of the most extensive head and neck cancer (HNC) imaging datasets accessible to the public. Initially collected for clinical radiation therapy (RT) treatment planning, this dataset has been retrospectively reconstructed for use in imaging research. ACQUISITION AND VALIDATION METHODS RADCURE encompasses data from 3346 patients, featuring computed tomography (CT) RT simulation images with corresponding target and organ-at-risk contours. These CT scans were collected using systems from three different manufacturers. Standard clinical imaging protocols were followed, and contours were manually generated and reviewed at weekly RT quality assurance rounds. RADCURE imaging and structure set data was extracted from our institution's radiation treatment planning and oncology information systems using a custom-built data mining and processing system. Furthermore, images were linked to our clinical anthology of outcomes data for each patient and includes demographic, clinical and treatment information based on the 7th edition TNM staging system (Tumor-Node-Metastasis Classification System of Malignant Tumors). The median patient age is 63, with the final dataset including 80% males. Half of the cohort is diagnosed with oropharyngeal cancer, while laryngeal, nasopharyngeal, and hypopharyngeal cancers account for 25%, 12%, and 5% of cases, respectively. The median duration of follow-up is five years, with 60% of the cohort surviving until the last follow-up point. DATA FORMAT AND USAGE NOTES The dataset provides images and contours in DICOM CT and RT-STRUCT formats, respectively. We have standardized the nomenclature for individual contours-such as the gross primary tumor, gross nodal volumes, and 19 organs-at-risk-to enhance the RT-STRUCT files' utility. Accompanying demographic, clinical, and treatment data are supplied in a comma-separated values (CSV) file format. This comprehensive dataset is publicly accessible via The Cancer Imaging Archive. POTENTIAL APPLICATIONS RADCURE's amalgamation of imaging, clinical, demographic, and treatment data renders it an invaluable resource for a broad spectrum of radiomics image analysis research endeavors. Researchers can utilize this dataset to advance routine clinical procedures using machine learning or artificial intelligence, to identify new non-invasive biomarkers, or to forge prognostic models.
Collapse
Affiliation(s)
- Mattea L Welch
- Princess Margaret Cancer Centre, Toronto, ON, Canada
- Cancer Digital Intelligence Program, Toronto, ON, Canada
| | - Sejin Kim
- Princess Margaret Cancer Centre, Toronto, ON, Canada
- Cancer Digital Intelligence Program, Toronto, ON, Canada
- Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada
| | - Andrew J Hope
- Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, ON, Canada
- Department of Radiation Oncology, University of Toronto, Toronto, ON, Canada
| | - Shao Hui Huang
- Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, ON, Canada
- Department of Radiation Oncology, University of Toronto, Toronto, ON, Canada
| | - Zhibin Lu
- Princess Margaret Cancer Centre, Toronto, ON, Canada
| | - Joseph Marsilla
- Princess Margaret Cancer Centre, Toronto, ON, Canada
- Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada
| | - Michal Kazmierski
- Princess Margaret Cancer Centre, Toronto, ON, Canada
- Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada
| | - Katrina Rey-McIntyre
- Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, ON, Canada
| | - Tirth Patel
- Cancer Digital Intelligence Program, Toronto, ON, Canada
- Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, ON, Canada
- TECHNA Institute, University Health Network, Toronto, ON, Canada
| | - Brian O'Sullivan
- Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, ON, Canada
- Department of Radiation Oncology, University of Toronto, Toronto, ON, Canada
| | - John Waldron
- Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, ON, Canada
- Department of Radiation Oncology, University of Toronto, Toronto, ON, Canada
| | - Scott Bratman
- Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada
| | - Benjamin Haibe-Kains
- Princess Margaret Cancer Centre, Toronto, ON, Canada
- Cancer Digital Intelligence Program, Toronto, ON, Canada
- Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada
- TECHNA Institute, University Health Network, Toronto, ON, Canada
| | - Tony Tadic
- Cancer Digital Intelligence Program, Toronto, ON, Canada
- Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, ON, Canada
- Department of Radiation Oncology, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
33
|
Huang Y, Yang J, Sun Q, Yuan Y, Li H, Hou Y. Multi-residual 2D network integrating spatial correlation for whole heart segmentation. Comput Biol Med 2024; 172:108261. [PMID: 38508056 DOI: 10.1016/j.compbiomed.2024.108261] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2023] [Revised: 02/21/2024] [Accepted: 03/06/2024] [Indexed: 03/22/2024]
Abstract
Whole heart segmentation (WHS) has significant clinical value for cardiac anatomy, modeling, and analysis of cardiac function. This study aims to address the WHS accuracy on cardiac CT images, as well as the fast inference speed and low graphics processing unit (GPU) memory consumption required by practical clinical applications. Thus, we propose a multi-residual two-dimensional (2D) network integrating spatial correlation for WHS. The network performs slice-by-slice segmentation on three-dimensional cardiac CT images in a 2D encoder-decoder manner. In the network, a convolutional long short-term memory skip connection module is designed to perform spatial correlation feature extraction on the feature maps at different resolutions extracted by the sub-modules of the pre-trained ResNet-based encoder. Moreover, a decoder based on the multi-residual module is designed to analyze the extracted features from the perspectives of multi-scale and channel attention, thereby accurately delineating the various substructures of the heart. The proposed method is verified on a dataset of the multi-modality WHS challenge, an in-house WHS dataset, and a dataset of the abdominal organ segmentation challenge. The dice, Jaccard, average symmetric surface distance, Hausdorff distance, inference time, and maximum GPU memory of the WHS are 0.914, 0.843, 1.066 mm, 15.778 mm, 9.535 s, and 1905 MB, respectively. The proposed network has high accuracy, fast inference speed, minimal GPU memory consumption, strong robustness, and good generalization. It can be deployed to clinical practical applications for WHS and can be effectively extended and applied to other multi-organ segmentation fields. The source code is publicly available at https://github.com/nancy1984yan/MultiResNet-SC.
Collapse
Affiliation(s)
- Yan Huang
- Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, Liaoning, China; School of Computer Science and Engineering, Northeastern University, Shenyang, Liaoning, China
| | - Jinzhu Yang
- Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, Liaoning, China; School of Computer Science and Engineering, Northeastern University, Shenyang, Liaoning, China; National Frontiers Science Center for Industrial Intelligence and Systems Optimization, Shenyang, Liaoning, China.
| | - Qi Sun
- Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, Liaoning, China; School of Computer Science and Engineering, Northeastern University, Shenyang, Liaoning, China
| | - Yuliang Yuan
- Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, Liaoning, China; School of Computer Science and Engineering, Northeastern University, Shenyang, Liaoning, China
| | - Honghe Li
- Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, Liaoning, China; School of Computer Science and Engineering, Northeastern University, Shenyang, Liaoning, China
| | - Yang Hou
- Department of Radiology, Shengjing Hospital of China Medical University, Shenyang, Liaoning, China
| |
Collapse
|
34
|
Podobnik G, Ibragimov B, Peterlin P, Strojan P, Vrtovec T. vOARiability: Interobserver and intermodality variability analysis in OAR contouring from head and neck CT and MR images. Med Phys 2024; 51:2175-2186. [PMID: 38230752 DOI: 10.1002/mp.16924] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Revised: 10/31/2023] [Accepted: 12/13/2023] [Indexed: 01/18/2024] Open
Abstract
BACKGROUND Accurate and consistent contouring of organs-at-risk (OARs) from medical images is a key step of radiotherapy (RT) cancer treatment planning. Most contouring approaches rely on computed tomography (CT) images, but the integration of complementary magnetic resonance (MR) modality is highly recommended, especially from the perspective of OAR contouring, synthetic CT and MR image generation for MR-only RT, and MR-guided RT. Although MR has been recognized as valuable for contouring OARs in the head and neck (HaN) region, the accuracy and consistency of the resulting contours have not been yet objectively evaluated. PURPOSE To analyze the interobserver and intermodality variability in contouring OARs in the HaN region, performed by observers with different level of experience from CT and MR images of the same patients. METHODS In the final cohort of 27 CT and MR images of the same patients, contours of up to 31 OARs were obtained by a radiation oncology resident (junior observer, JO) and a board-certified radiation oncologist (senior observer, SO). The resulting contours were then evaluated in terms of interobserver variability, characterized as the agreement among different observers (JO and SO) when contouring OARs in a selected modality (CT or MR), and intermodality variability, characterized as the agreement among different modalities (CT and MR) when OARs were contoured by a selected observer (JO or SO), both by the Dice coefficient (DC) and 95-percentile Hausdorff distance (HD95 $_{95}$ ). RESULTS The mean (±standard deviation) interobserver variability was 69.0 ± 20.2% and 5.1 ± 4.1 mm, while the mean intermodality variability was 61.6 ± 19.0% and 6.1 ± 4.3 mm in terms of DC and HD95 $_{95}$ , respectively, across all OARs. Statistically significant differences were only found for specific OARs. The performed MR to CT image registration resulted in a mean target registration error of 1.7 ± 0.5 mm, which was considered as valid for the analysis of intermodality variability. CONCLUSIONS The contouring variability was, in general, similar for both image modalities, and experience did not considerably affect the contouring performance. However, the results indicate that an OAR is difficult to contour regardless of whether it is contoured in the CT or MR image, and that observer experience may be an important factor for OARs that are deemed difficult to contour. Several of the differences in the resulting variability can be also attributed to adherence to guidelines, especially for OARs with poor visibility or without distinctive boundaries in either CT or MR images. Although considerable contouring differences were observed for specific OARs, it can be concluded that almost all OARs can be contoured with a similar degree of variability in either the CT or MR modality, which works in favor of MR images from the perspective of MR-only and MR-guided RT.
Collapse
Affiliation(s)
- Gašper Podobnik
- Faculty of Electrical Engineering, University of Ljubljana, Ljubljana, Slovenia
| | - Bulat Ibragimov
- Faculty of Electrical Engineering, University of Ljubljana, Ljubljana, Slovenia
- Department of Computer Science, University of Copenhagen, Copenhagen, Denmark
| | | | | | - Tomaž Vrtovec
- Faculty of Electrical Engineering, University of Ljubljana, Ljubljana, Slovenia
| |
Collapse
|
35
|
Koo J, Caudell J, Latifi K, Moros EG, Feygelman V. Essentially unedited deep-learning-based OARs are suitable for rigorous oropharyngeal and laryngeal cancer treatment planning. J Appl Clin Med Phys 2024; 25:e14202. [PMID: 37942993 DOI: 10.1002/acm2.14202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2023] [Revised: 10/19/2023] [Accepted: 10/25/2023] [Indexed: 11/10/2023] Open
Abstract
Quality of organ at risk (OAR) autosegmentation is often judged by concordance metrics against the human-generated gold standard. However, the ultimate goal is the ability to use unedited autosegmented OARs in treatment planning, while maintaining the plan quality. We tested this approach with head and neck (HN) OARs generated by a prototype deep-learning (DL) model on patients previously treated for oropharyngeal and laryngeal cancer. Forty patients were selected, with all structures delineated by an experienced physician. For each patient, a set of 13 OARs were generated by the DL model. Each patient was re-planned based on original targets and unedited DL-produced OARs. The new dose distributions were then applied back to the manually delineated structures. The target coverage was evaluated with inhomogeneity index (II) and the relative volume of regret. For the OARs, Dice similarity coefficient (DSC) of areas under the DVH curves, individual DVH objectives, and composite continuous plan quality metric (PQM) were compared. The nearly identical primary target coverage for the original and re-generated plans was achieved, with the same II and relative volume of regret values. The average DSC of the areas under the corresponding pairs of DVH curves was 0.97 ± 0.06. The number of critical DVH points which met the clinical objectives with the dose optimized on autosegmented structures but failed when evaluated on the manual ones was 5 of 896 (0.6%). The average OAR PQM score with the re-planned dose distributions was essentially the same when evaluated either on the autosegmented or manual OARs. Thus, rigorous HN treatment planning is possible with OARs segmented by a prototype DL algorithm with minimal, if any, manual editing.
Collapse
Affiliation(s)
- Jihye Koo
- Department of Radiation Oncology, Moffitt Cancer Center, Tampa, Florida, USA
- Department of Physics, University of South Florida, Tampa, Florida, USA
| | - Jimmy Caudell
- Department of Radiation Oncology, Moffitt Cancer Center, Tampa, Florida, USA
| | - Kujtim Latifi
- Department of Radiation Oncology, Moffitt Cancer Center, Tampa, Florida, USA
| | - Eduardo G Moros
- Department of Radiation Oncology, Moffitt Cancer Center, Tampa, Florida, USA
| | - Vladimir Feygelman
- Department of Radiation Oncology, Moffitt Cancer Center, Tampa, Florida, USA
| |
Collapse
|
36
|
REINKE ANNIKA, TIZABI MINUD, BAUMGARTNER MICHAEL, EISENMANN MATTHIAS, HECKMANN-NÖTZEL DOREEN, KAVUR AEMRE, RÄDSCH TIM, SUDRE CAROLEH, ACION LAURA, ANTONELLI MICHELA, ARBEL TAL, BAKAS SPYRIDON, BENIS ARRIEL, BLASCHKO MATTHEWB, BUETTNER FLORIAN, CARDOSO MJORGE, CHEPLYGINA VERONIKA, CHEN JIANXU, CHRISTODOULOU EVANGELIA, CIMINI BETHA, COLLINS GARYS, FARAHANI KEYVAN, FERRER LUCIANA, GALDRAN ADRIAN, VAN GINNEKEN BRAM, GLOCKER BEN, GODAU PATRICK, HAASE ROBERT, HASHIMOTO DANIELA, HOFFMAN MICHAELM, HUISMAN MEREL, ISENSEE FABIAN, JANNIN PIERRE, KAHN CHARLESE, KAINMUELLER DAGMAR, KAINZ BERNHARD, KARARGYRIS ALEXANDROS, KARTHIKESALINGAM ALAN, KENNGOTT HANNES, KLEESIEK JENS, KOFLER FLORIAN, KOOI THIJS, KOPP-SCHNEIDER ANNETTE, KOZUBEK MICHAL, KRESHUK ANNA, KURC TAHSIN, LANDMAN BENNETTA, LITJENS GEERT, MADANI AMIN, MAIER-HEIN KLAUS, MARTEL ANNEL, MATTSON PETER, MEIJERING ERIK, MENZE BJOERN, MOONS KARELG, MÜLLER HENNING, NICHYPORUK BRENNAN, NICKEL FELIX, PETERSEN JENS, RAFELSKI SUSANNEM, RAJPOOT NASIR, REYES MAURICIO, RIEGLER MICHAELA, RIEKE NICOLA, SAEZ-RODRIGUEZ JULIO, SÁNCHEZ CLARAI, SHETTY SHRAVYA, SUMMERS RONALDM, TAHA ABDELA, TIULPIN ALEKSEI, TSAFTARIS SOTIRIOSA, VAN CALSTER BEN, VAROQUAUX GAËL, YANIV ZIVR, JÄGER PAULF, MAIER-HEIN LENA. Understanding metric-related pitfalls in image analysis validation. ARXIV 2024:arXiv:2302.01790v4. [PMID: 36945687 PMCID: PMC10029046] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 03/23/2023]
Abstract
Validation metrics are key for the reliable tracking of scientific progress and for bridging the current chasm between artificial intelligence (AI) research and its translation into practice. However, increasing evidence shows that particularly in image analysis, metrics are often chosen inadequately in relation to the underlying research problem. This could be attributed to a lack of accessibility of metric-related knowledge: While taking into account the individual strengths, weaknesses, and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multi-stage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides the first reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Focusing on biomedical image analysis but with the potential of transfer to other fields, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. To facilitate comprehension, illustrations and specific examples accompany each pitfall. As a structured body of information accessible to researchers of all levels of expertise, this work enhances global comprehension of a key topic in image analysis validation.
Collapse
Affiliation(s)
- ANNIKA REINKE
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems and HI Helmholtz Imaging, Germany and Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
| | - MINU D. TIZABI
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Germany and National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, Germany
| | - MICHAEL BAUMGARTNER
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Germany
| | - MATTHIAS EISENMANN
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Germany
| | - DOREEN HECKMANN-NÖTZEL
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Germany and National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, Germany
| | - A. EMRE KAVUR
- HI Applied Computer Vision Lab, Division of Medical Image Computing; German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Germany
| | - TIM RÄDSCH
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems and HI Helmholtz Imaging, Germany
| | - CAROLE H. SUDRE
- MRC Unit for Lifelong Health and Ageing at UCL and Centre for Medical Image Computing, Department of Computer Science, University College London, London, UK and School of Biomedical Engineering and Imaging Science, King’s College London, London, UK
| | - LAURA ACION
- Instituto de Cálculo, CONICET – Universidad de Buenos Aires, Buenos Aires, Argentina
| | - MICHELA ANTONELLI
- School of Biomedical Engineering and Imaging Science, King’s College London, London, UK and Centre for Medical Image Computing, University College London, London, UK
| | - TAL ARBEL
- Centre for Intelligent Machines and MILA (Quebec Artificial Intelligence Institute), McGill University, Montreal, Canada
| | - SPYRIDON BAKAS
- Division of Computational Pathology, Dept of Pathology & Laboratory Medicine, Indiana University School of Medicine, IU Health Information and Translational Sciences Building, Indianapolis, USA and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Richards Medical Research Laboratories FL7, Philadelphia, PA, USA
| | - ARRIEL BENIS
- Department of Digital Medical Technologies, Holon Institute of Technology, Holon, Israel and European Federation for Medical Informatics, Le Mont-sur-Lausanne, Switzerland
| | - MATTHEW B. BLASCHKO
- Center for Processing Speech and Images, Department of Electrical Engineering, KU Leuven, Leuven, Belgium
| | - FLORIAN BUETTNER
- German Cancer Consortium (DKTK), partner site Frankfurt/Mainz, a partnership between DKFZ and UCT Frankfurt-Marburg, Germany, German Cancer Research Center (DKFZ) Heidelberg, Germany, Goethe University Frankfurt, Department of Medicine, Germany, Goethe University Frankfurt, Department of Informatics, Germany, and Frankfurt Cancer Insititute, Germany
| | - M. JORGE CARDOSO
- School of Biomedical Engineering and Imaging Science, King’s College London, London, UK
| | - VERONIKA CHEPLYGINA
- Department of Computer Science, IT University of Copenhagen, Copenhagen, Denmark
| | - JIANXU CHEN
- Leibniz-Institut für Analytische Wissenschaften – ISAS – e.V., Dortmund, Germany
| | - EVANGELIA CHRISTODOULOU
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Germany
| | - BETH A. CIMINI
- Imaging Platform, Broad Institute of MIT and Harvard, Cambridge, Massachusetts, USA
| | - GARY S. COLLINS
- Centre for Statistics in Medicine, University of Oxford, Oxford, UK
| | - KEYVAN FARAHANI
- Center for Biomedical Informatics and Information Technology, National Cancer Institute, Bethesda, MD, USA
| | - LUCIANA FERRER
- Instituto de Investigación en Ciencias de la Computación (ICC), CONICET-UBA, Ciudad Universitaria, Ciudad Autónoma de Buenos Aires, Argentina
| | - ADRIAN GALDRAN
- Universitat Pompeu Fabra, Barcelona, Spain and University of Adelaide, Adelaide, Australia
| | - BRAM VAN GINNEKEN
- Fraunhofer MEVIS, Bremen, Germany and Radboud Institute for Health Sciences, Radboud University Medical Center, Nijmegen, The Netherlands
| | - BEN GLOCKER
- Department of Computing, Imperial College London, London, UK
| | - PATRICK GODAU
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Germany, Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany, and National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, Germany
| | - ROBERT HAASE
- Now with: Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI), Leipzig University, Leipzig, Germany, DFG Cluster of Excellence “Physics of Life”, Technische Universität (TU) Dresden, Dresden, Germany, and Center for Systems Biology , Dresden, Germany
| | - DANIEL A. HASHIMOTO
- Department of Surgery, Perelman School of Medicine, Philadelphia, PA, USA and General Robotics Automation Sensing and Perception Laboratory, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA, USA
| | - MICHAEL M. HOFFMAN
- Princess Margaret Cancer Centre, University Health Network, Toronto, Canada, Department of Medical Biophysics, University of Toronto, Toronto, Canada, Department of Computer Science, University of Toronto, Toronto, Canada, and Vector Institute for Artificial Intelligence, Toronto, Canada
| | - MEREL HUISMAN
- Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, The Netherlands
| | - FABIAN ISENSEE
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing and HI Applied Computer Vision Lab, Germany
| | - PIERRE JANNIN
- Laboratoire Traitement du Signal et de l’Image – UMR_S 1099, Université de Rennes 1, Rennes, France and INSERM, Paris Cedex, France
| | - CHARLES E. KAHN
- Department of Radiology and Institute for Biomedical Informatics, University of Pennsylvania, Philadelphia, PA, USA
| | - DAGMAR KAINMUELLER
- Max-Delbrück Center for Molecular Medicine in the Helmholtz Association (MDC), Biomedical Image Analysis and HI Helmholtz Imaging, Berlin, Germany and University of Potsdam, Digital Engineering Faculty, Potsdam, Germany
| | - BERNHARD KAINZ
- Department of Computing, Faculty of Engineering, Imperial College London, London, UK and Department AIBE, Friedrich-Alexander-Universität (FAU), Erlangen-Nürnberg, Germany
| | | | | | - HANNES KENNGOTT
- Department of General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - JENS KLEESIEK
- Translational Image-guided Oncology (TIO), Institute for AI in Medicine (IKIM), University Medicine Essen, Essen, Germany
| | | | | | | | - MICHAL KOZUBEK
- Centre for Biomedical Image Analysis and Faculty of Informatics, Masaryk University, Brno, Czech Republic
| | - ANNA KRESHUK
- Cell Biology and Biophysics Unit, European Molecular Biology Laboratory (EMBL), Heidelberg, Germany
| | - TAHSIN KURC
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, USA
| | | | - GEERT LITJENS
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - AMIN MADANI
- Department of Surgery, University Health Network, Philadelphia, PA, Canada
| | - KLAUS MAIER-HEIN
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing and HI Helmholtz Imaging, Germany and Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany
| | - ANNE L. MARTEL
- Physical Sciences, Sunnybrook Research Institute, Toronto, Canada and Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada
| | | | - ERIK MEIJERING
- School of Computer Science and Engineering, University of New South Wales, Sydney, Australia
| | - BJOERN MENZE
- Department of Quantitative Biomedicine, University of Zurich, Zurich, Switzerland
| | - KAREL G.M. MOONS
- Julius Center for Health Sciences and Primary Care, UMC Utrecht, Utrecht University, Utrecht, The Netherlands
| | - HENNING MÜLLER
- Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO), Sierre, Switzerland and Medical Faculty, University of Geneva, Geneva, Switzerland
| | | | - FELIX NICKEL
- Department of General, Visceral and Thoracic Surgery, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - JENS PETERSEN
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Germany
| | | | - NASIR RAJPOOT
- Tissue Image Analytics Laboratory, Department of Computer Science, University of Warwick, Coventry, UK
| | - MAURICIO REYES
- ARTORG Center for Biomedical Engineering Research, University of Bern, Bern, Switzerland and Department of Radiation Oncology, University Hospital Bern, University of Bern, Bern, Switzerland
| | - MICHAEL A. RIEGLER
- Simula Metropolitan Center for Digital Engineering, Oslo, Norway and UiT The Arctic University of Norway, Tromsø, Norway
| | | | - JULIO SAEZ-RODRIGUEZ
- Institute for Computational Biomedicine, Heidelberg University, Heidelberg. Germany and Faculty of Medicine, Heidelberg University Hospital, Heidelberg, Germany
| | - CLARA I. SÁNCHEZ
- Informatics Institute, Faculty of Science, University of Amsterdam, Amsterdam, The Netherlands
| | | | | | - ABDEL A. TAHA
- Institute of Information Systems Engineering, TU Wien, Vienna, Austria
| | - ALEKSEI TIULPIN
- Research Unit of Health Sciences and Technology, Faculty of Medicine, University of Oulu, Oulu, Finland and Neurocenter Oulu, Oulu University Hospital, Oulu, Finland
| | | | - BEN VAN CALSTER
- Department of Development and Regeneration and EPI-centre, KU Leuven, Leuven, Belgium and Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands
| | - GAËL VAROQUAUX
- Parietal project team, INRIA Saclay-Île de France, Palaiseau, France
| | - ZIV R. YANIV
- National Institute of Allergy and Infectious Diseases, National Institutes of Health, Bethesda, MD, USA
| | - PAUL F. JÄGER
- German Cancer Research Center (DKFZ) Heidelberg, Interactive Machine Learning Group and HI Helmholtz Imaging, Germany
| | - LENA MAIER-HEIN
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems and HI Helmholtz Imaging, Germany, Faculty of Mathematics and Computer Science and Medical Faculty, Heidelberg University, Heidelberg, Germany, and National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, Germany
| |
Collapse
|
37
|
He D, Udupa JK, Tong Y, Torigian DA. Predicting human effort needed to correct auto-segmentations. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2024; 12931:129310D. [PMID: 38957573 PMCID: PMC11218903 DOI: 10.1117/12.3006471] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/04/2024]
Abstract
Medical image auto-segmentation techniques are basic and critical for numerous image-based analysis applications that play an important role in developing advanced and personalized medicine. Compared with manual segmentations, auto-segmentations are expected to contribute to a more efficient clinical routine and workflow by requiring fewer human interventions or revisions to auto-segmentations. However, current auto-segmentation methods are usually developed with the help of some popular segmentation metrics that do not directly consider human correction behavior. Dice Coefficient (DC) focuses on the truly-segmented areas, while Hausdorff Distance (HD) only measures the maximal distance between the auto-segmentation boundary with the ground truth boundary. Boundary length-based metrics such as surface DC (surDC) and Added Path Length (APL) try to distinguish truly-predicted boundary pixels and wrong ones. It is uncertain if these metrics can reliably indicate the required manual mending effort for application in segmentation research. Therefore, in this paper, the potential use of the above four metrics, as well as a novel metric called Mendability Index (MI), to predict the human correction effort is studied with linear and support vector regression models. 265 3D computed tomography (CT) samples for 3 objects of interest from 3 institutions with corresponding auto-segmentations and ground truth segmentations are utilized to train and test the prediction models. The five-fold cross-validation experiments demonstrate that meaningful human effort prediction can be achieved using segmentation metrics with varying prediction errors for different objects. The improved variant of MI, called MIhd, generally shows the best prediction performance, suggesting its potential to indicate reliably the clinical value of auto-segmentations.
Collapse
Affiliation(s)
- Da He
- Medical Image Processing Group, 602 Goddard building, 3710 Hamilton Walk, Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104, United States
- University of Michigan-Shanghai Jiao Tong University Joint Institute, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Jayaram K Udupa
- Medical Image Processing Group, 602 Goddard building, 3710 Hamilton Walk, Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Yubing Tong
- Medical Image Processing Group, 602 Goddard building, 3710 Hamilton Walk, Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Drew A Torigian
- Medical Image Processing Group, 602 Goddard building, 3710 Hamilton Walk, Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104, United States
| |
Collapse
|
38
|
Walter A, Hoegen-Saßmannshausen P, Stanic G, Rodrigues JP, Adeberg S, Jäkel O, Frank M, Giske K. Segmentation of 71 Anatomical Structures Necessary for the Evaluation of Guideline-Conforming Clinical Target Volumes in Head and Neck Cancers. Cancers (Basel) 2024; 16:415. [PMID: 38254904 PMCID: PMC11154560 DOI: 10.3390/cancers16020415] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Revised: 12/28/2023] [Accepted: 01/08/2024] [Indexed: 01/24/2024] Open
Abstract
The delineation of the clinical target volumes (CTVs) for radiation therapy is time-consuming, requires intensive training and shows high inter-observer variability. Supervised deep-learning methods depend heavily on consistent training data; thus, State-of-the-Art research focuses on making CTV labels more homogeneous and strictly bounding them to current standards. International consensus expert guidelines standardize CTV delineation by conditioning the extension of the clinical target volume on the surrounding anatomical structures. Training strategies that directly follow the construction rules given in the expert guidelines or the possibility of quantifying the conformance of manually drawn contours to the guidelines are still missing. Seventy-one anatomical structures that are relevant to CTV delineation in head- and neck-cancer patients, according to the expert guidelines, were segmented on 104 computed tomography scans, to assess the possibility of automating their segmentation by State-of-the-Art deep learning methods. All 71 anatomical structures were subdivided into three subsets of non-overlapping structures, and a 3D nnU-Net model with five-fold cross-validation was trained for each subset, to automatically segment the structures on planning computed tomography scans. We report the DICE, Hausdorff distance and surface DICE for 71 + 5 anatomical structures, for most of which no previous segmentation accuracies have been reported. For those structures for which prediction values have been reported, our segmentation accuracy matched or exceeded the reported values. The predictions from our models were always better than those predicted by the TotalSegmentator. The sDICE with 2 mm margin was larger than 80% for almost all the structures. Individual structures with decreased segmentation accuracy are analyzed and discussed with respect to their impact on the CTV delineation following the expert guidelines. No deviation is expected to affect the rule-based automation of the CTV delineation.
Collapse
Affiliation(s)
- Alexandra Walter
- Department of Medical Physics in Radiation Oncology, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 280, 69120 Heidelberg, Germany; (G.S.); (J.P.R.); (O.J.); (K.G.)
- Heidelberg Institute of Radiation Oncology (HIRO), National Center for Radiation Research in Oncology (NCRO), 69120 Heidelberg, Germany;
- Karlsruhe Institute of Technology (KIT), Scientific Computing Center, Zirkel 2, 76131 Karlsruhe, Germany;
| | - Philipp Hoegen-Saßmannshausen
- Heidelberg Institute of Radiation Oncology (HIRO), National Center for Radiation Research in Oncology (NCRO), 69120 Heidelberg, Germany;
- Department of Radiation Oncology, Heidelberg University Hospital, 69120 Heidelberg, Germany
- Clinical Cooperation Unit Radiation Oncology, German Cancer Research Center (DKFZ), 69120 Heidelberg, Germany
- National Center for Tumor Diseases (NCT), NCT Heidelberg, 69120 Heidelberg, Germany
| | - Goran Stanic
- Department of Medical Physics in Radiation Oncology, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 280, 69120 Heidelberg, Germany; (G.S.); (J.P.R.); (O.J.); (K.G.)
- Heidelberg Institute of Radiation Oncology (HIRO), National Center for Radiation Research in Oncology (NCRO), 69120 Heidelberg, Germany;
- Faculty of Physics and Astronomy, University of Heidelberg, 69120 Heidelberg, Germany
| | - Joao Pedro Rodrigues
- Department of Medical Physics in Radiation Oncology, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 280, 69120 Heidelberg, Germany; (G.S.); (J.P.R.); (O.J.); (K.G.)
- Heidelberg Institute of Radiation Oncology (HIRO), National Center for Radiation Research in Oncology (NCRO), 69120 Heidelberg, Germany;
| | - Sebastian Adeberg
- Department of Radiotherapy and Radiation Oncology, Marburg University Hospital, 35043 Marburg, Germany;
- Marburg Ion-Beam Therapy Center (MIT), 35043 Marburg, Germany
- Universitäres Centrum für Tumorerkrankungen (UCT), 35033 Marburg, Germany
| | - Oliver Jäkel
- Department of Medical Physics in Radiation Oncology, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 280, 69120 Heidelberg, Germany; (G.S.); (J.P.R.); (O.J.); (K.G.)
- Heidelberg Institute of Radiation Oncology (HIRO), National Center for Radiation Research in Oncology (NCRO), 69120 Heidelberg, Germany;
- Heidelberg Ion-Beam Therapy Center (HIT), 69120 Heidelberg, Germany
| | - Martin Frank
- Karlsruhe Institute of Technology (KIT), Scientific Computing Center, Zirkel 2, 76131 Karlsruhe, Germany;
| | - Kristina Giske
- Department of Medical Physics in Radiation Oncology, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 280, 69120 Heidelberg, Germany; (G.S.); (J.P.R.); (O.J.); (K.G.)
- Heidelberg Institute of Radiation Oncology (HIRO), National Center for Radiation Research in Oncology (NCRO), 69120 Heidelberg, Germany;
| |
Collapse
|
39
|
Kehayias CE, Yan Y, Bontempi D, Quirk S, Bitterman DS, Bredfeldt JS, Aerts HJWL, Mak RH, Guthier CV. Prospective deployment of an automated implementation solution for artificial intelligence translation to clinical radiation oncology. Front Oncol 2024; 13:1305511. [PMID: 38239639 PMCID: PMC10794768 DOI: 10.3389/fonc.2023.1305511] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2023] [Accepted: 12/11/2023] [Indexed: 01/22/2024] Open
Abstract
Introduction Artificial intelligence (AI)-based technologies embody countless solutions in radiation oncology, yet translation of AI-assisted software tools to actual clinical environments remains unrealized. We present the Deep Learning On-Demand Assistant (DL-ODA), a fully automated, end-to-end clinical platform that enables AI interventions for any disease site featuring an automated model-training pipeline, auto-segmentations, and QA reporting. Materials and methods We developed, tested, and prospectively deployed the DL-ODA system at a large university affiliated hospital center. Medical professionals activate the DL-ODA via two pathways (1): On-Demand, used for immediate AI decision support for a patient-specific treatment plan, and (2) Ambient, in which QA is provided for all daily radiotherapy (RT) plans by comparing DL segmentations with manual delineations and calculating the dosimetric impact. To demonstrate the implementation of a new anatomy segmentation, we used the model-training pipeline to generate a breast segmentation model based on a large clinical dataset. Additionally, the contour QA functionality of existing models was assessed using a retrospective cohort of 3,399 lung and 885 spine RT cases. Ambient QA was performed for various disease sites including spine RT and heart for dosimetric sparing. Results Successful training of the breast model was completed in less than a day and resulted in clinically viable whole breast contours. For the retrospective analysis, we evaluated manual-versus-AI similarity for the ten most common structures. The DL-ODA detected high similarities in heart, lung, liver, and kidney delineations but lower for esophagus, trachea, stomach, and small bowel due largely to incomplete manual contouring. The deployed Ambient QAs for heart and spine sites have prospectively processed over 2,500 cases and 230 cases over 9 months and 5 months, respectively, automatically alerting the RT personnel. Discussion The DL-ODA capabilities in providing universal AI interventions were demonstrated for On-Demand contour QA, DL segmentations, and automated model training, and confirmed successful integration of the system into a large academic radiotherapy department. The novelty of deploying the DL-ODA as a multi-modal, fully automated end-to-end AI clinical implementation solution marks a significant step towards a generalizable framework that leverages AI to improve the efficiency and reliability of RT systems.
Collapse
Affiliation(s)
- Christopher E. Kehayias
- Department of Radiation Oncology, Brigham and Women’s Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA, United States
| | - Yujie Yan
- Department of Radiation Oncology, Brigham and Women’s Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA, United States
| | - Dennis Bontempi
- Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, MA, United States
- Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, Netherlands
| | - Sarah Quirk
- Department of Radiation Oncology, Brigham and Women’s Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA, United States
| | - Danielle S. Bitterman
- Department of Radiation Oncology, Brigham and Women’s Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA, United States
| | - Jeremy S. Bredfeldt
- Department of Radiation Oncology, Brigham and Women’s Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA, United States
| | - Hugo J. W. L. Aerts
- Department of Radiation Oncology, Brigham and Women’s Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA, United States
- Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, MA, United States
- Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, Netherlands
| | - Raymond H. Mak
- Department of Radiation Oncology, Brigham and Women’s Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA, United States
- Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, MA, United States
| | - Christian V. Guthier
- Department of Radiation Oncology, Brigham and Women’s Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA, United States
- Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, MA, United States
| |
Collapse
|
40
|
Maroongroge S, Mohamed ASR, Nguyen C, Guma De la Vega J, Frank SJ, Garden AS, Gunn BG, Lee A, Mayo L, Moreno A, Morrison WH, Phan J, Spiotto MT, Court LE, Fuller CD, Rosenthal DI, Netherton TJ. Clinical acceptability of automatically generated lymph node levels and structures of deglutition and mastication for head and neck radiation therapy. Phys Imaging Radiat Oncol 2024; 29:100540. [PMID: 38356692 PMCID: PMC10864833 DOI: 10.1016/j.phro.2024.100540] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Revised: 01/22/2024] [Accepted: 01/24/2024] [Indexed: 02/16/2024] Open
Abstract
Background and Purpose Auto-contouring of complex anatomy in computed tomography (CT) scans is a highly anticipated solution to many problems in radiotherapy. In this study, artificial intelligence (AI)-based auto-contouring models were clinically validated for lymph node levels and structures of swallowing and chewing in the head and neck. Materials and Methods CT scans of 145 head and neck radiotherapy patients were retrospectively curated. One cohort (n = 47) was used to analyze seven lymph node levels and the other (n = 98) used to analyze 17 swallowing and chewing structures. Separate nnUnet models were trained and validated using the separate cohorts. For the lymph node levels, preference and clinical acceptability of AI vs human contours were scored. For the swallowing and chewing structures, clinical acceptability was scored. Quantitative analyses of the test sets were performed for AI vs human contours for all structures using overlap and distance metrics. Results Median Dice Similarity Coefficient ranged from 0.77 to 0.89 for lymph node levels and 0.86 to 0.96 for chewing and swallowing structures. The AI contours were superior to or equally preferred to the manual contours at rates ranging from 75% to 91%; there was not a significant difference in clinical acceptability for nodal levels I-V for manual versus AI contours. Across all AI-generated lymph node level contours, 92% were rated as usable with stylistic to no edits. Of the 340 contours in the chewing and swallowing cohort, 4% required minor edits. Conclusions An accurate approach was developed to auto-contour lymph node levels and chewing and swallowing structures on CT images for patients with intact nodal anatomy. Only a small portion of test set auto-contours required minor edits.
Collapse
Affiliation(s)
- Sean Maroongroge
- Department of Radiation Oncology, Division of Radiation Oncology, University of Texas MD Anderson Cancer Center, United States
| | - Abdallah SR. Mohamed
- Department of Radiation Oncology, Division of Radiation Oncology, University of Texas MD Anderson Cancer Center, United States
| | - Callistus Nguyen
- Department of Radiation Physics, Division of Radiation Oncology, University of Texas MD Anderson Cancer Center, United States
| | - Jean Guma De la Vega
- Department of Radiation Physics, Division of Radiation Oncology, University of Texas MD Anderson Cancer Center, United States
| | - Steven J. Frank
- Department of Radiation Oncology, Division of Radiation Oncology, University of Texas MD Anderson Cancer Center, United States
| | - Adam S. Garden
- Department of Radiation Oncology, Division of Radiation Oncology, University of Texas MD Anderson Cancer Center, United States
| | - Brandon G. Gunn
- Department of Radiation Oncology, Division of Radiation Oncology, University of Texas MD Anderson Cancer Center, United States
| | - Anna Lee
- Department of Radiation Oncology, Division of Radiation Oncology, University of Texas MD Anderson Cancer Center, United States
| | - Lauren Mayo
- Department of Radiation Oncology, Division of Radiation Oncology, University of Texas MD Anderson Cancer Center, United States
| | - Amy Moreno
- Department of Radiation Oncology, Division of Radiation Oncology, University of Texas MD Anderson Cancer Center, United States
| | - William H. Morrison
- Department of Radiation Oncology, Division of Radiation Oncology, University of Texas MD Anderson Cancer Center, United States
| | - Jack Phan
- Department of Radiation Oncology, Division of Radiation Oncology, University of Texas MD Anderson Cancer Center, United States
| | - Michael T. Spiotto
- Department of Radiation Oncology, Division of Radiation Oncology, University of Texas MD Anderson Cancer Center, United States
| | - Laurence E. Court
- Department of Radiation Physics, Division of Radiation Oncology, University of Texas MD Anderson Cancer Center, United States
| | - Clifton D. Fuller
- Department of Radiation Oncology, Division of Radiation Oncology, University of Texas MD Anderson Cancer Center, United States
| | - David I. Rosenthal
- Department of Radiation Oncology, Division of Radiation Oncology, University of Texas MD Anderson Cancer Center, United States
| | - Tucker J. Netherton
- Department of Radiation Physics, Division of Radiation Oncology, University of Texas MD Anderson Cancer Center, United States
| |
Collapse
|
41
|
Fernandes MG, Bussink J, Wijsman R, Stam B, Monshouwer R. Estimating how contouring differences affect normal tissue complication probability modelling. Phys Imaging Radiat Oncol 2024; 29:100533. [PMID: 38292649 PMCID: PMC10825684 DOI: 10.1016/j.phro.2024.100533] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Revised: 11/15/2023] [Accepted: 12/30/2023] [Indexed: 02/01/2024] Open
Abstract
Background and purpose Normal tissue complication probability (NTCP) models are developed from large retrospective datasets where automatic contouring is often used to contour the organs at risk. This study proposes a methodology to estimate how discrepancies between two sets of contours are reflected on NTCP model performance. We apply this methodology to heart contours within a dataset of non-small cell lung cancer (NSCLC) patients. Materials and methods One of the contour sets is designated the ground truth and a dosimetric parameter derived from it is used to simulate outcomes via a predefined NTCP relationship. For each simulated outcome, the selected dosimetric parameters associated with each contour set are individually used to fit a toxicity model and their performance is compared. Our dataset comprised 605 stage IIA-IIIB NSCLC patients. Manual, deep learning, and atlas-based heart contours were available. Results How contour differences were reflected in NTCP model performance depended on the slope of the predefined model, the dosimetric parameter utilized, and the size of the cohort. The impact of contour differences on NTCP model performance increased with steeper NTCP curves. In our dataset, parameters on the low range of the dose-volume histogram were more robust to contour differences. Conclusions Our methodology can be used to estimate whether a given contouring model is fit for NTCP model development. For the heart in comparable datasets, average Dice should be at least as high as between our manual and deep learning contours for shallow NTCP relationships (88.5 ± 4.5 %) and higher for steep relationships.
Collapse
Affiliation(s)
| | - Johan Bussink
- Department of Radiation Oncology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Robin Wijsman
- Department of Radiation Oncology, University Medical Center Groningen, Groningen, The Netherlands
| | - Barbara Stam
- Department of Radiation Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - René Monshouwer
- Department of Radiation Oncology, Radboud University Medical Center, Nijmegen, The Netherlands
| |
Collapse
|
42
|
Chen Y, Pahlavian SH, Jacobs P, Neupane T, Forghani-Arani F, Castillo E, Castillo R, Vinogradskiy Y. Systematic Evaluation of the Impact of Lung Segmentation Methods on 4-Dimensional Computed Tomography Ventilation Imaging Using a Large Patient Database. Int J Radiat Oncol Biol Phys 2024; 118:242-252. [PMID: 37607642 PMCID: PMC10842520 DOI: 10.1016/j.ijrobp.2023.08.017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 08/04/2023] [Accepted: 08/08/2023] [Indexed: 08/24/2023]
Abstract
PURPOSE A novel form of lung functional imaging applied for functional avoidance radiation therapy has been developed that uses 4-dimensional computed tomography (4DCT) data and image processing techniques to calculate lung ventilation (4DCT-ventilation). Lung segmentation is a common step to define a region of interest for 4DCT-ventilation generation. The purpose of this study was to quantitatively evaluate the sensitivity of 4DCT-ventilation imaging using different lung segmentation methods. METHODS AND MATERIALS The 4DCT data of 350 patients from 2 institutions were used. Lung contours were generated using 3 methods: (1) reference segmentations that removed airways and pulmonary vasculature manually (Lung-Manual), (2) standard lung contours used for planning (Lung-RadOnc), and (3) artificial intelligence (AI)-based contours that removed the airways and pulmonary vasculature (Lung-AI). The AI model was based on a residual 3-dimensional U-Net and was trained using the Lung-Manual contours of 279 patients. We compared the Lung-RadOnc or Lung-AI with Lung-Manual contours for the entire 4DCT-ventilation functional avoidance process including lung segmentation (surface Dice similarity coefficient [Surface DSC]), 4DCT-ventilation generation (correlation), and subanalysis of 10 patients on a dosimetric endpoint (percentage of high functional volume of lung receiving ≥20 Gy [fV20{%}]). RESULTS Surface DSC comparing Lung-Manual/Lung-RadOnc and Lung-Manual/Lung-AI contours was 0.40 ± 0.06 and 0.86 ± 0.04, respectively. The correlation between 4DCT-ventilation images generated with Lung-Manual/Lung-RadOnc and Lung-Manual/Lung-AI were 0.48 ± 0.14 and 0.85 ± 0.14, respectively. The difference in fV20[%] between 4DCT-ventilation generated with Lung-Manual/Lung-RadOnc and Lung-Manual/Lung-AI was 2.5% ± 4.1% and 0.3% ± 0.5%, respectively. CONCLUSIONS Our work showed that using standard planning lung contours can result in significantly variable 4DCT-ventilation images. The study demonstrated that AI-based segmentations generate lung contours and 4DCT-ventilation images that are similar to those generated using manual methods. The significance of the study is that it characterizes the lung segmentation sensitivity of the 4DCT-ventilation process and develops methods that can facilitate the integration of this novel imaging in busy clinics.
Collapse
Affiliation(s)
- Yingxuan Chen
- Department of Radiation Oncology, Thomas Jefferson University, Philadelphia, Pennsylvania
| | | | | | - Taindra Neupane
- Department of Radiation Oncology, Thomas Jefferson University, Philadelphia, Pennsylvania
| | | | - Edward Castillo
- Department of Biomedical Engineering, University of Texas at Austin, Austin, Texas
| | - Richard Castillo
- Department of Radiation Oncology, Emory University, Atlanta, Georgia
| | - Yevgeniy Vinogradskiy
- Department of Radiation Oncology, Thomas Jefferson University, Philadelphia, Pennsylvania.
| |
Collapse
|
43
|
Paudyal R, Jiang J, Han J, Diplas BH, Riaz N, Hatzoglou V, Lee N, Deasy JO, Veeraraghavan H, Shukla-Dave A. Auto-segmentation of neck nodal metastases using self-distilled masked image transformer on longitudinal MR images. BJR ARTIFICIAL INTELLIGENCE 2024; 1:ubae004. [PMID: 38476956 PMCID: PMC10928808 DOI: 10.1093/bjrai/ubae004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Revised: 01/23/2024] [Accepted: 01/24/2024] [Indexed: 03/14/2024]
Abstract
Objectives Auto-segmentation promises greater speed and lower inter-reader variability than manual segmentations in radiation oncology clinical practice. This study aims to implement and evaluate the accuracy of the auto-segmentation algorithm, "Masked Image modeling using the vision Transformers (SMIT)," for neck nodal metastases on longitudinal T2-weighted (T2w) MR images in oropharyngeal squamous cell carcinoma (OPSCC) patients. Methods This prospective clinical trial study included 123 human papillomaviruses (HPV-positive [+]) related OSPCC patients who received concurrent chemoradiotherapy. T2w MR images were acquired on 3 T at pre-treatment (Tx), week 0, and intra-Tx weeks (1-3). Manual delineations of metastatic neck nodes from 123 OPSCC patients were used for the SMIT auto-segmentation, and total tumor volumes were calculated. Standard statistical analyses compared contour volumes from SMIT vs manual segmentation (Wilcoxon signed-rank test [WSRT]), and Spearman's rank correlation coefficients (ρ) were computed. Segmentation accuracy was evaluated on the test data set using the dice similarity coefficient (DSC) metric value. P-values <0.05 were considered significant. Results No significant difference in manual and SMIT delineated tumor volume at pre-Tx (8.68 ± 7.15 vs 8.38 ± 7.01 cm3, P = 0.26 [WSRT]), and the Bland-Altman method established the limits of agreement as -1.71 to 2.31 cm3, with a mean difference of 0.30 cm3. SMIT model and manually delineated tumor volume estimates were highly correlated (ρ = 0.84-0.96, P < 0.001). The mean DSC metric values were 0.86, 0.85, 0.77, and 0.79 at the pre-Tx and intra-Tx weeks (1-3), respectively. Conclusions The SMIT algorithm provides sufficient segmentation accuracy for oncological applications in HPV+ OPSCC. Advances in knowledge First evaluation of auto-segmentation with SMIT using longitudinal T2w MRI in HPV+ OPSCC.
Collapse
Affiliation(s)
- Ramesh Paudyal
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York, NY 10065, United States
| | - Jue Jiang
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York, NY 10065, United States
| | - James Han
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY 10065, United States
| | - Bill H Diplas
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY 10065, United States
| | - Nadeem Riaz
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY 10065, United States
| | - Vaios Hatzoglou
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York, NY 10065, United States
| | - Nancy Lee
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY 10065, United States
| | - Joseph O Deasy
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York, NY 10065, United States
| | - Harini Veeraraghavan
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York, NY 10065, United States
| | - Amita Shukla-Dave
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York, NY 10065, United States
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York, NY 10065, United States
| |
Collapse
|
44
|
Li B, Zhang J, Wang Q, Li H, Wang Q. Three-dimensional spine reconstruction from biplane radiographs using convolutional neural networks. Med Eng Phys 2024; 123:104088. [PMID: 38365341 DOI: 10.1016/j.medengphy.2023.104088] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2023] [Revised: 12/04/2023] [Accepted: 12/10/2023] [Indexed: 02/18/2024]
Abstract
PURPOSE The purpose of this study was to develop and evaluate a deep learning network for three-dimensional reconstruction of the spine from biplanar radiographs. METHODS The proposed approach focused on extracting similar features and multiscale features of bone tissue in biplanar radiographs. Bone tissue features were reconstructed for feature representation across dimensions to generate three-dimensional volumes. The number of feature mappings was gradually reduced in the reconstruction to transform the high-dimensional features into the three-dimensional image domain. We produced and made eight public datasets to train and test the proposed network. Two evaluation metrics were proposed and combined with four classical evaluation metrics to measure the performance of the method. RESULTS In comparative experiments, the reconstruction results of this method achieved a Hausdorff distance of 1.85 mm, a surface overlap of 0.2 mm, a volume overlap of 0.9664, and an offset distance of only 0.21 mm from the vertebral body centroid. The results of this study indicate that the proposed method is reliable.
Collapse
Affiliation(s)
- Bo Li
- Department of Electronic Engineering, Yunnan University, Kunming, China
| | - Junhua Zhang
- Department of Electronic Engineering, Yunnan University, Kunming, China.
| | - Qian Wang
- Department of Electronic Engineering, Yunnan University, Kunming, China
| | - Hongjian Li
- The First People's Hospital of Yunnan Province, China
| | - Qiyang Wang
- The First People's Hospital of Yunnan Province, China
| |
Collapse
|
45
|
Gay SS, Cardenas CE, Nguyen C, Netherton TJ, Yu C, Zhao Y, Skett S, Patel T, Adjogatse D, Guerrero Urbano T, Naidoo K, Beadle BM, Yang J, Aggarwal A, Court LE. Fully-automated, CT-only GTV contouring for palliative head and neck radiotherapy. Sci Rep 2023; 13:21797. [PMID: 38066074 PMCID: PMC10709623 DOI: 10.1038/s41598-023-48944-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Accepted: 12/01/2023] [Indexed: 12/18/2023] Open
Abstract
Planning for palliative radiotherapy is performed without the advantage of MR or PET imaging in many clinics. Here, we investigated CT-only GTV delineation for palliative treatment of head and neck cancer. Two multi-institutional datasets of palliative-intent treatment plans were retrospectively acquired: a set of 102 non-contrast-enhanced CTs and a set of 96 contrast-enhanced CTs. The nnU-Net auto-segmentation network was chosen for its strength in medical image segmentation, and five approaches separately trained: (1) heuristic-cropped, non-contrast images with a single GTV channel, (2) cropping around a manually-placed point in the tumor center for non-contrast images with a single GTV channel, (3) contrast-enhanced images with a single GTV channel, (4) contrast-enhanced images with separate primary and nodal GTV channels, and (5) contrast-enhanced images along with synthetic MR images with separate primary and nodal GTV channels. Median Dice similarity coefficient ranged from 0.6 to 0.7, surface Dice from 0.30 to 0.56, and 95th Hausdorff distance from 14.7 to 19.7 mm across the five approaches. Only surface Dice exhibited statistically-significant difference across these five approaches using a two-tailed Wilcoxon Rank-Sum test (p ≤ 0.05). Our CT-only results met or exceeded published values for head and neck GTV autocontouring using multi-modality images. However, significant edits would be necessary before clinical use in palliative radiotherapy.
Collapse
Affiliation(s)
- Skylar S Gay
- Unit 1472, Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, 77030, USA.
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX, USA.
| | - Carlos E Cardenas
- Department of Radiation Oncology, The University of Alabama at Birmingham, Birmingham, AL, USA
| | - Callistus Nguyen
- Unit 1472, Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, 77030, USA
| | - Tucker J Netherton
- Unit 1472, Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, 77030, USA
| | - Cenji Yu
- Unit 1472, Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX, USA
| | - Yao Zhao
- Unit 1472, Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, 77030, USA
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX, USA
| | | | | | | | | | | | | | - Jinzhong Yang
- Unit 1472, Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, 77030, USA
| | | | - Laurence E Court
- Unit 1472, Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, 77030, USA
| |
Collapse
|
46
|
Kovacs B, Netzer N, Baumgartner M, Schrader A, Isensee F, Weißer C, Wolf I, Görtz M, Jaeger PF, Schütz V, Floca R, Gnirs R, Stenzinger A, Hohenfellner M, Schlemmer HP, Bonekamp D, Maier-Hein KH. Addressing image misalignments in multi-parametric prostate MRI for enhanced computer-aided diagnosis of prostate cancer. Sci Rep 2023; 13:19805. [PMID: 37957250 PMCID: PMC10643562 DOI: 10.1038/s41598-023-46747-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2022] [Accepted: 11/04/2023] [Indexed: 11/15/2023] Open
Abstract
Prostate cancer (PCa) diagnosis on multi-parametric magnetic resonance images (MRI) requires radiologists with a high level of expertise. Misalignments between the MRI sequences can be caused by patient movement, elastic soft-tissue deformations, and imaging artifacts. They further increase the complexity of the task prompting radiologists to interpret the images. Recently, computer-aided diagnosis (CAD) tools have demonstrated potential for PCa diagnosis typically relying on complex co-registration of the input modalities. However, there is no consensus among research groups on whether CAD systems profit from using registration. Furthermore, alternative strategies to handle multi-modal misalignments have not been explored so far. Our study introduces and compares different strategies to cope with image misalignments and evaluates them regarding to their direct effect on diagnostic accuracy of PCa. In addition to established registration algorithms, we propose 'misalignment augmentation' as a concept to increase CAD robustness. As the results demonstrate, misalignment augmentations can not only compensate for a complete lack of registration, but if used in conjunction with registration, also improve the overall performance on an independent test set.
Collapse
Affiliation(s)
- Balint Kovacs
- Division of Medical Image Computing, German Cancer Research Center (DKFZ) Heidelberg, Im Neuenheimer Feld 223, 69120, Heidelberg, Germany.
- Division of Radiology, German Cancer Research Center (DKFZ) Heidelberg, Heidelberg, Germany.
- Medical Faculty Heidelberg, Heidelberg University, Heidelberg, Germany.
| | - Nils Netzer
- Division of Radiology, German Cancer Research Center (DKFZ) Heidelberg, Heidelberg, Germany
- Medical Faculty Heidelberg, Heidelberg University, Heidelberg, Germany
| | - Michael Baumgartner
- Division of Medical Image Computing, German Cancer Research Center (DKFZ) Heidelberg, Im Neuenheimer Feld 223, 69120, Heidelberg, Germany
- Helmholtz Imaging, German Cancer Research Center (DKFZ) Heidelberg, Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
| | - Adrian Schrader
- Division of Radiology, German Cancer Research Center (DKFZ) Heidelberg, Heidelberg, Germany
- Medical Faculty Heidelberg, Heidelberg University, Heidelberg, Germany
| | - Fabian Isensee
- Division of Medical Image Computing, German Cancer Research Center (DKFZ) Heidelberg, Im Neuenheimer Feld 223, 69120, Heidelberg, Germany
- Helmholtz Imaging, German Cancer Research Center (DKFZ) Heidelberg, Heidelberg, Germany
| | - Cedric Weißer
- Division of Radiology, German Cancer Research Center (DKFZ) Heidelberg, Heidelberg, Germany
- Medical Faculty Heidelberg, Heidelberg University, Heidelberg, Germany
| | - Ivo Wolf
- Mannheim University of Applied Sciences, Mannheim, Germany
| | - Magdalena Görtz
- Junior Clinical Cooperation Unit 'Multiparametric Methods for Early Detection of Prostate Cancer', German Cancer Research Center (DKFZ) Heidelberg, Heidelberg, Germany
- Department of Urology, University of Heidelberg Medical Center, Heidelberg, Germany
| | - Paul F Jaeger
- Helmholtz Imaging, German Cancer Research Center (DKFZ) Heidelberg, Heidelberg, Germany
- Interactive Machine Learning Group, German Cancer Research Center (DKFZ) Heidelberg, Heidelberg, Germany
| | - Victoria Schütz
- Department of Urology, University of Heidelberg Medical Center, Heidelberg, Germany
| | - Ralf Floca
- Division of Medical Image Computing, German Cancer Research Center (DKFZ) Heidelberg, Im Neuenheimer Feld 223, 69120, Heidelberg, Germany
| | - Regula Gnirs
- Division of Radiology, German Cancer Research Center (DKFZ) Heidelberg, Heidelberg, Germany
| | - Albrecht Stenzinger
- Institute of Pathology, University of Heidelberg Medical Center, Heidelberg, Germany
| | - Markus Hohenfellner
- Department of Urology, University of Heidelberg Medical Center, Heidelberg, Germany
| | - Heinz-Peter Schlemmer
- Division of Radiology, German Cancer Research Center (DKFZ) Heidelberg, Heidelberg, Germany
- German Cancer Consortium (DKTK), DKFZ, Core Center Heidelberg, Heidelberg, Germany
| | - David Bonekamp
- Division of Radiology, German Cancer Research Center (DKFZ) Heidelberg, Heidelberg, Germany
- Medical Faculty Heidelberg, Heidelberg University, Heidelberg, Germany
- German Cancer Consortium (DKTK), DKFZ, Core Center Heidelberg, Heidelberg, Germany
| | - Klaus H Maier-Hein
- Division of Medical Image Computing, German Cancer Research Center (DKFZ) Heidelberg, Im Neuenheimer Feld 223, 69120, Heidelberg, Germany
- Helmholtz Imaging, German Cancer Research Center (DKFZ) Heidelberg, Heidelberg, Germany
- German Cancer Consortium (DKTK), DKFZ, Core Center Heidelberg, Heidelberg, Germany
- Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany
| |
Collapse
|
47
|
Davey A, Pan S, Bryce-Atkinson A, Mandeville H, Janssens GO, Kelly SM, Hol M, Tang V, Davies LSC, SIOP-Europe Radiation Oncology Working Group, Aznar M. The need for consensus on delineation and dose constraints of dentofacial structures in paediatric radiotherapy: Outcomes of a SIOP Europe survey. Clin Transl Radiat Oncol 2023; 43:100681. [PMID: 37790584 PMCID: PMC10543782 DOI: 10.1016/j.ctro.2023.100681] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Accepted: 09/21/2023] [Indexed: 10/05/2023] Open
Abstract
Background and purpose Children receiving radiotherapy for head-and-neck tumours often experience severe dentofacial side effects. Despite this, recommendations for contouring and dose constraints to dentofacial structures are lacking in clinical practice. We report on a survey aiming to understand current practice in contouring and dose assessment to dentofacial structures. Methods A digital survey was distributed to European Society for Paediatric Oncology members of the Radiation Oncology Working Group, and member-affiliated centres in Europe, Australia, and New Zealand. The questions focused on clinical practice and aimed to establish areas for future development. Results Results from 52 paediatric radiotherapy centres across 27 countries are reported. Only 29/52 centres routinely delineated some dentofacial structures, with the most common being the mandible (25 centres), temporo-mandibular joint (22), dentition (13), orbit (10) and maxillary bone (eight). For most bones contoured, an 'As Low As Reasonably Achievable' dose objective was implemented. Only four centres reported age-adapted dose constraints.The largest barrier to clinical implementation of dose constraints was firstly, the lack of contouring guidance (49/52, 94%) and secondly, that delineation is time-consuming (33/52, 63%). Most respondents who routinely contour dentofacial structures (25/27, 90%) agreed a contouring atlas would aid delineation. Conclusion Routine delineation of dentofacial structures is infrequent in paediatric radiotherapy. Based on survey findings, we aim to 1) define a consensus-contouring atlas for dentofacial structures, 2) develop auto-contouring solutions for dentofacial structures to aid clinical implementation, and 3) carry out treatment planning studies to investigate the importance of delineation of these structures for planning optimisation.
Collapse
Affiliation(s)
- Angela Davey
- Division of Cancer Sciences, Faculty of Biology, Medicine and Health, The University of Manchester, Manchester, UK
| | - Shermaine Pan
- Department of Proton Therapy, The Christie NHS Foundation Trust, Manchester, UK
| | - Abigail Bryce-Atkinson
- Division of Cancer Sciences, Faculty of Biology, Medicine and Health, The University of Manchester, Manchester, UK
| | | | - Geert O. Janssens
- Princess Maxima Center for Paediatric Oncology, Utrecht, The Netherlands
- Department of Radiation Oncology, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Sarah M. Kelly
- European Society for Paediatric Oncology (SIOP Europe), Clos Chapelle-aux-Champs 30, Brussels, Belgium
- The European Organisation for Research and Treatment of Cancer (EORTC) Headquarters, Avenue E. Mounier 83, Brussels, Belgium
- Faculty of Medicine and Health Sciences, Ghent University, Ghent, Belgium
| | - Marinka Hol
- Princess Maxima Center for Paediatric Oncology, Utrecht, The Netherlands
- Department of Otorhinolaryngology, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Vivian Tang
- Paediatric Radiology, Royal Manchester Children’s Hospital, Manchester, UK
| | | | | | - Marianne Aznar
- Division of Cancer Sciences, Faculty of Biology, Medicine and Health, The University of Manchester, Manchester, UK
| |
Collapse
|
48
|
Zhong NN, Wang HQ, Huang XY, Li ZZ, Cao LM, Huo FY, Liu B, Bu LL. Enhancing head and neck tumor management with artificial intelligence: Integration and perspectives. Semin Cancer Biol 2023; 95:52-74. [PMID: 37473825 DOI: 10.1016/j.semcancer.2023.07.002] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2023] [Revised: 07/11/2023] [Accepted: 07/15/2023] [Indexed: 07/22/2023]
Abstract
Head and neck tumors (HNTs) constitute a multifaceted ensemble of pathologies that primarily involve regions such as the oral cavity, pharynx, and nasal cavity. The intricate anatomical structure of these regions poses considerable challenges to efficacious treatment strategies. Despite the availability of myriad treatment modalities, the overall therapeutic efficacy for HNTs continues to remain subdued. In recent years, the deployment of artificial intelligence (AI) in healthcare practices has garnered noteworthy attention. AI modalities, inclusive of machine learning (ML), neural networks (NNs), and deep learning (DL), when amalgamated into the holistic management of HNTs, promise to augment the precision, safety, and efficacy of treatment regimens. The integration of AI within HNT management is intricately intertwined with domains such as medical imaging, bioinformatics, and medical robotics. This article intends to scrutinize the cutting-edge advancements and prospective applications of AI in the realm of HNTs, elucidating AI's indispensable role in prevention, diagnosis, treatment, prognostication, research, and inter-sectoral integration. The overarching objective is to stimulate scholarly discourse and invigorate insights among medical practitioners and researchers to propel further exploration, thereby facilitating superior therapeutic alternatives for patients.
Collapse
Affiliation(s)
- Nian-Nian Zhong
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Han-Qi Wang
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Xin-Yue Huang
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Zi-Zhan Li
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Lei-Ming Cao
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Fang-Yi Huo
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Bing Liu
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China; Department of Oral & Maxillofacial - Head Neck Oncology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China.
| | - Lin-Lin Bu
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China; Department of Oral & Maxillofacial - Head Neck Oncology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China.
| |
Collapse
|
49
|
De Kerf G, Claessens M, Raouassi F, Mercier C, Stas D, Ost P, Dirix P, Verellen D. A geometry and dose-volume based performance monitoring of artificial intelligence models in radiotherapy treatment planning for prostate cancer. Phys Imaging Radiat Oncol 2023; 28:100494. [PMID: 37809056 PMCID: PMC10550805 DOI: 10.1016/j.phro.2023.100494] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2023] [Revised: 09/20/2023] [Accepted: 09/20/2023] [Indexed: 10/10/2023] Open
Abstract
Background and Purpose Clinical Artificial Intelligence (AI) implementations lack ground-truth when applied on real-world data. This study investigated how combined geometrical and dose-volume metrics can be used as performance monitoring tools to detect clinically relevant candidates for model retraining. Materials and Methods Fifty patients were analyzed for both AI-segmentation and planning. For AI-segmentation, geometrical (Standard Surface Dice 3 mm and Local Surface Dice 3 mm) and dose-volume based parameters were calculated for two organs (bladder and anorectum) to compare AI output against the clinically corrected structure. A Local Surface Dice was introduced to detect geometrical changes in the vicinity of the target volumes, while an Absolute Dose Difference (ADD) evaluation increased focus on dose-volume related changes. AI-planning performance was evaluated using clinical goal analysis in combination with volume and target overlap metrics. Results The Local Surface Dice reported equal or lower values compared to the Standard Surface Dice (anorectum: (0.93 ± 0.11) vs (0.98 ± 0.04); bladder: (0.97 ± 0.06) vs (0.98 ± 0.04)). The ADD metric showed a difference of (0.9 ± 0.8)Gy for the anorectum D 1 cm 3 . The bladder D 5cm 3 reported a difference of (0.7 ± 1.5)Gy. Mandatory clinical goals were fulfilled in 90 % of the DLP plans. Conclusions Combining dose-volume and geometrical metrics allowed detection of clinically relevant changes, applied to both auto-segmentation and auto-planning output and the Local Surface Dice was more sensitive to local changes compared to the Standard Surface Dice. This monitoring is able to evaluate AI behavior in clinical practice and allows candidate selection for active learning.
Collapse
Affiliation(s)
- Geert De Kerf
- Department of Radiation Oncology, Iridium Netwerk, Wilrijk (Antwerp), Belgium
| | - Michaël Claessens
- Department of Radiation Oncology, Iridium Netwerk, Wilrijk (Antwerp), Belgium
- Centre for Oncological Research (CORE), Integrated Personalized and Precision Oncology Network (IPPON), University of Antwerp, Antwerp, Belgium
| | - Fadoua Raouassi
- Department of Radiation Oncology, Iridium Netwerk, Wilrijk (Antwerp), Belgium
| | - Carole Mercier
- Department of Radiation Oncology, Iridium Netwerk, Wilrijk (Antwerp), Belgium
- Centre for Oncological Research (CORE), Integrated Personalized and Precision Oncology Network (IPPON), University of Antwerp, Antwerp, Belgium
| | - Daan Stas
- Department of Radiation Oncology, Iridium Netwerk, Wilrijk (Antwerp), Belgium
- Faculty of Medicine and Health Sciences, University of Antwerp, Antwerp, Belgium
| | - Piet Ost
- Department of Radiation Oncology, Iridium Netwerk, Wilrijk (Antwerp), Belgium
- Centre for Oncological Research (CORE), Integrated Personalized and Precision Oncology Network (IPPON), University of Antwerp, Antwerp, Belgium
| | - Piet Dirix
- Department of Radiation Oncology, Iridium Netwerk, Wilrijk (Antwerp), Belgium
- Centre for Oncological Research (CORE), Integrated Personalized and Precision Oncology Network (IPPON), University of Antwerp, Antwerp, Belgium
| | - Dirk Verellen
- Department of Radiation Oncology, Iridium Netwerk, Wilrijk (Antwerp), Belgium
- Centre for Oncological Research (CORE), Integrated Personalized and Precision Oncology Network (IPPON), University of Antwerp, Antwerp, Belgium
| |
Collapse
|
50
|
Heilemann G, Buschmann M, Lechner W, Dick V, Eckert F, Heilmann M, Herrmann H, Moll M, Knoth J, Konrad S, Simek IM, Thiele C, Zaharie A, Georg D, Widder J, Trnkova P. Clinical Implementation and Evaluation of Auto-Segmentation Tools for Multi-Site Contouring in Radiotherapy. Phys Imaging Radiat Oncol 2023; 28:100515. [PMID: 38111502 PMCID: PMC10726238 DOI: 10.1016/j.phro.2023.100515] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2023] [Revised: 11/09/2023] [Accepted: 11/09/2023] [Indexed: 12/20/2023] Open
Abstract
Background and purpose Tools for auto-segmentation in radiotherapy are widely available, but guidelines for clinical implementation are missing. The goal was to develop a workflow for performance evaluation of three commercial auto-segmentation tools to select one candidate for clinical implementation. Materials and Methods One hundred patients with six treatment sites (brain, head-and-neck, thorax, abdomen, and pelvis) were included. Three sets of AI-based contours for organs-at-risk (OAR) generated by three software tools and manually drawn expert contours were blindly rated for contouring accuracy. The dice similarity coefficient (DSC), the Hausdorff distance, and a dose/volume evaluation based on the recalculation of the original treatment plan were assessed. Statistically significant differences were tested using the Kruskal-Wallis test and the post-hoc Dunn Test with Bonferroni correction. Results The mean DSC scores compared to expert contours for all OARs combined were 0.80 ± 0.10, 0.75 ± 0.10, and 0.74 ± 0.11 for the three software tools. Physicians' rating identified equivalent or superior performance of some AI-based contours in head (eye, lens, optic nerve, brain, chiasm), thorax (e.g., heart and lungs), and pelvis and abdomen (e.g., kidney, femoral head) compared to manual contours. For some OARs, the AI models provided results requiring only minor corrections. Bowel-bag and stomach were not fit for direct use. During the interdisciplinary discussion, the physicians' rating was considered the most relevant. Conclusion A comprehensive method for evaluation and clinical implementation of commercially available auto-segmentation software was developed. The in-depth analysis yielded clear instructions for clinical use within the radiotherapy department.
Collapse
Affiliation(s)
- Gerd Heilemann
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Martin Buschmann
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Wolfgang Lechner
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Vincent Dick
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Franziska Eckert
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Martin Heilmann
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Harald Herrmann
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Matthias Moll
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Johannes Knoth
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Stefan Konrad
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Inga-Malin Simek
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Christopher Thiele
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Alexandru Zaharie
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Dietmar Georg
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Joachim Widder
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| | - Petra Trnkova
- Department of Radiation Oncology, Comprehensive Cancer Center Vienna, Medical University Vienna, Vienna, Austria
| |
Collapse
|