Comprehensive clinical evaluation of deep learning-based auto-segmentation for radiotherapy in patients with cervical cancer

Artificial intelligence contouring in radiotherapy for organs-at-risk and lymph node areas

INTRODUCTION

The delineation of organs-at-risk and lymph node areas is a crucial step in radiotherapy, but it is time-consuming and associated with substantial user-dependent variability in contouring. Artificial intelligence (AI) appears to be the solution to facilitate and standardize this work. The objective of this study is to compare eight available AI software programs in terms of technical aspects and accuracy for contouring organs-at-risk and lymph node areas with current international contouring recommendations.

MATERIAL AND METHODS

From January-July 2023, we performed a blinded study of the contour scoring of the organs-at-risk and lymph node areas by eight self-contouring AI programs by 20 radiation oncologists. It was a single-center study conducted in radiation department at the Lorraine Cancer Institute. A qualitative analysis of technical characteristics of the different AI programs was also performed. Three adults (two women and one man) and three children (one girl and two boys) provided six whole-body anonymized CT scans, along with two other adult brain MRI scans. Using a scoring scale from 1 to 3 (best score), radiation oncologists blindly assessed the quality of contouring of organs-at-risk and lymph node areas of all scans and MRI data by the eight AI programs. We have chosen to define the threshold of an average score equal to or greater than 2 to characterize a high-performing AI software, meaning an AI with minimal to moderate corrections but usable in clinical routine.

RESULTS

For adults CT scans: There were two AI programs for which the overall average quality score (that is, all areas tested for OARs and lymph nodes) was higher than 2.0: Limbus (overall average score = 2.03 (0.16)) and MVision (overall average score = 2.13 (0.19)). If we only consider OARs for adults, only Limbus, Therapanacea, MVision and Radformation have an average score above 2. For children CT scan, MVision was the only program to have a average score higher than 2 with overall average score = 2.07 (0.19). If we only consider OARs for children, only Limbus and MVision have an average score above 2. For brain MRIs: TheraPanacea was the only program with an average score over 2, for both brain delineation (2.75 (0.35)) and OARs (2.09 (0.19)). The comparative analysis of the technical aspects highlights the similarities and differences between the software. There is no difference in between senior radiation oncologist and residents for OARs contouring.

CONCLUSION

For adult CT-scan, two AI programs on the market, MVision and Limbus, delineate most OARs and lymph nodes areas that are useful in clinical routine. For children CT-scan, only one IA, MVision, program is efficient. For adult brain MRI, Therapancea,only one AI program is efficient.

TRIAL REGISTRATION

CNIL-MR0004 Number HDH434.

Automated Organ Segmentation for Radiation Therapy: A Comparative Analysis of AI-Based Tools Versus Manual Contouring in Korean Cancer Patients

BACKGROUND

Accurate delineation of tumors and organs at risk (OARs) is crucial for intensity-modulated radiation therapy. This study aimed to evaluate the performance of OncoStudio, an AI-based auto-segmentation tool developed for Korean patients, compared with Protégé AI, a globally developed tool that uses data from Korean cancer patients.

METHODS

A retrospective analysis of 1200 Korean cancer patients treated with radiotherapy was conducted. Auto-contours generated via OncoStudio and Protégé AI were compared with manual contours across the head and neck and thoracic, abdominal, and pelvic organs. Accuracy was assessed using the Dice similarity coefficient (DSC), mean surface distance (MSD), and 95% Hausdorff distance (HD). Feedback was obtained from 10 participants, including radiation oncologists, residents, and radiation therapists, via an online survey with a Turing test component.

RESULTS

OncoStudio outperformed Protégé AI in 85% of the evaluated OARs (p < 0.001). For head and neck organs, OncoStudio achieved a similar DSC (0.70 vs. 0.70, p = 0.637) but significantly lower MSD and 95% HD values (p < 0.001). In thoracic organs, OncoStudio performed excellently in 90% of cases, with a significantly greater DSC (male: 0.87 vs. 0.82, p < 0.001; female: 0.95 vs. 0.87, p < 0.001). OncoStudio also demonstrated superior accuracy in abdominal (DSC 0.88 vs. 0.81, p < 0.001) and pelvic organs (male: DSC 0.95 vs. 0.85, p < 0.001; female: DSC 0.82 vs. 0.73, p < 0.001). Clinicians favored OncoStudio in 70% of cases, with 90% endorsing its clinical suitability for Korean patients.

CONCLUSIONS

OncoStudio, which is tailored for Korean patients, demonstrated superior segmentation accuracy across multiple anatomical regions, suggesting its suitability for radiotherapy planning in this population.

A quantitative analysis of artificial intelligence research in cervical cancer: a bibliometric approach utilizing CiteSpace and VOSviewer

Background

Cervical cancer, a severe threat to women's health, is experiencing a global increase in incidence, notably among younger demographics. With artificial intelligence (AI) making strides, its integration into medical research is expanding, particularly in cervical cancer studies. This bibliometric study aims to evaluate AI's role, highlighting research trends and potential future directions in the field.

Methods

This study systematically retrieved literature from the Web of Science Core Collection (WoSCC), employing VOSviewer and CiteSpace for analysis. This included examining collaborations and keyword co-occurrences, with a focus on the relationship between citing and cited journals and authors. A burst ranking analysis identified research hotspots based on citation frequency.

Results

The study analyzed 927 articles from 2008 to 2024 by 5,299 authors across 81 regions. China, the U.S., and India were the top contributors, with key institutions like the Chinese Academy of Sciences and the NIH leading in publications. Schiffman, Mark, featured among the top authors, while Jemal, A, was the most cited. 'Diagnostics' and 'IEEE Access' stood out for publication volume and citation impact, respectively. Keywords such as 'cervical cancer,' 'deep learning,' 'classification,' and 'machine learning' were dominant. The most cited article was by Berner, ES; et al., published in 2008.

Conclusions

AI's application in cervical cancer research is expanding, with a growing scholarly community. The study suggests that AI, especially deep learning and machine learning, will remain a key research area, focusing on improving diagnostics and treatment. There is a need for increased international collaboration to maximize AI's potential in advancing cervical cancer research and patient care.

Automated segmentation in pelvic radiotherapy: A comprehensive evaluation of ATLAS-, machine learning-, and deep learning-based models

Artificial intelligence can standardize and automatize highly demanding procedures, such as manual segmentation, especially in an anatomical site as common as the pelvis. This study investigated four automated segmentation tools on computed tomography (CT) images in female and male pelvic radiotherapy (RT) starting from simpler and well-known atlas-based methods to the most recent neural networks-based algorithms. The evaluation included quantitative, qualitative and time efficiency assessments. A mono-institutional consecutive series of 40 cervical cancer and 40 prostate cancer structure sets were retrospectively selected. After a preparatory phase, the remaining 20 testing sets per each site were auto-segmented by the atlas-based model STAPLE, a Random Forest-based model, and two Deep Learning-based tools (DL), MVision and LimbusAI. Setting manual segmentation as the Ground Truth, 200 structure sets were compared in terms of Dice Similarity Coefficient (DSC), Hausdorff Distance (HD), and Distance-to-Agreement Portion (DAP). Automated segmentation and manual correction durations were recorded. Expert clinicians performed a qualitative evaluation. In cervical cancer CTs, DL outperformed the other tools with higher quantitative metrics, qualitative scores, and shorter correction times. On the other hand, in prostate cancer CTs, the performance across all the analyzed tools was comparable in terms of both quantitative and qualitative metrics. Such discrepancy in performance outcome could be explained by the wide range of anatomical variability in cervical cancer with respect to the strict bladder and rectum filling preparation in prostate Stereotactic Body Radiation Therapy (SBRT). Decreasing segmentation times can reduce the burden of pelvic radiation therapy routine in an automated workflow.

A deep learning-based 3D Prompt-nnUnet model for automatic segmentation in brachytherapy of postoperative endometrial carcinoma

PURPOSE

To create and evaluate a three-dimensional (3D) Prompt-nnUnet module that utilizes the prompts-based model combined with 3D nnUnet for producing the rapid and consistent autosegmentation of high-risk clinical target volume (HR CTV) and organ at risk (OAR) in high-dose-rate brachytherapy (HDR BT) for patients with postoperative endometrial carcinoma (EC).

METHODS AND MATERIALS

On two experimental batches, a total of 321 computed tomography (CT) scans were obtained for HR CTV segmentation from 321 patients with EC, and 125 CT scans for OARs segmentation from 125 patients. The numbers of training/validation/test were 257/32/32 and 87/13/25 for HR CTV and OARs respectively. A novel comparison of the deep learning neural network 3D Prompt-nnUnet and 3D nnUnet was applied for HR CTV and OARs segmentation. Three-fold cross validation and several quantitative metrics were employed, including Dice similarity coefficient (DSC), Hausdorff distance (HD), 95th percentile of Hausdorff distance (HD95%), and intersection over union (IoU).

RESULTS

The Prompt-nnUnet included two forms of parameters Predict-Prompt (PP) and Label-Prompt (LP), with the LP performing most similarly to the experienced radiation oncologist and outperforming the less experienced ones. During the testing phase, the mean DSC values for the LP were 0.96 ± 0.02, 0.91 ± 0.02, and 0.83 ± 0.07 for HR CTV, rectum and urethra, respectively. The mean HD values (mm) were 2.73 ± 0.95, 8.18 ± 4.84, and 2.11 ± 0.50, respectively. The mean HD95% values (mm) were 1.66 ± 1.11, 3.07 ± 0.94, and 1.35 ± 0.55, respectively. The mean IoUs were 0.92 ± 0.04, 0.84 ± 0.03, and 0.71 ± 0.09, respectively. A delineation time < 2.35 s per structure in the new model was observed, which was available to save clinician time.

CONCLUSION

The Prompt-nnUnet architecture, particularly the LP, was highly consistent with ground truth (GT) in HR CTV or OAR autosegmentation, reducing interobserver variability and shortening treatment time.

Cervical cancer segmentation based on medical images: a literature review

Background and Objective

Cervical cancer clinical target volume (CTV) outlining and organs at risk segmentation are crucial steps in the diagnosis and treatment of cervical cancer. Manual segmentation is inefficient and subjective, leading to the development of automated or semi-automated methods. However, limitation of image quality, organ motion, and individual differences still pose significant challenges. Apart from numbers of studies on the medical images' segmentation, a comprehensive review within the field is lacking. The purpose of this paper is to comprehensively review the literatures on different types of medical image segmentation regarding cervical cancer and discuss the current level and challenges in segmentation process.

Methods

As of May 31, 2023, we conducted a comprehensive literature search on Google Scholar, PubMed, and Web of Science using the following term combinations: "cervical cancer images", "segmentation", and "outline". The included studies focused on the segmentation of cervical cancer utilizing computed tomography (CT), magnetic resonance (MR), and positron emission tomography (PET) images, with screening for eligibility by two independent investigators.

Key Content and Findings

This paper reviews representative papers on CTV and organs at risk segmentation in cervical cancer and classifies the methods into three categories based on image modalities. The traditional or deep learning methods are comprehensively described. The similarities and differences of related methods are analyzed, and their advantages and limitations are discussed in-depth. We have also included experimental results by using our private datasets to verify the performance of selected methods. The results indicate that the residual module and squeeze-and-excitation blocks module can significantly improve the performance of the model. Additionally, the segmentation method based on improved level set demonstrates better segmentation accuracy than other methods.

Conclusions

The paper provides valuable insights into the current state-of-the-art in cervical cancer CTV outlining and organs at risk segmentation, highlighting areas for future research.

Clinical Implementation and Evaluation of Auto-Segmentation Tools for Multi-Site Contouring in Radiotherapy

Background and purpose

Tools for auto-segmentation in radiotherapy are widely available, but guidelines for clinical implementation are missing. The goal was to develop a workflow for performance evaluation of three commercial auto-segmentation tools to select one candidate for clinical implementation.

Materials and Methods

One hundred patients with six treatment sites (brain, head-and-neck, thorax, abdomen, and pelvis) were included. Three sets of AI-based contours for organs-at-risk (OAR) generated by three software tools and manually drawn expert contours were blindly rated for contouring accuracy. The dice similarity coefficient (DSC), the Hausdorff distance, and a dose/volume evaluation based on the recalculation of the original treatment plan were assessed. Statistically significant differences were tested using the Kruskal-Wallis test and the post-hoc Dunn Test with Bonferroni correction.

Results

The mean DSC scores compared to expert contours for all OARs combined were 0.80 ± 0.10, 0.75 ± 0.10, and 0.74 ± 0.11 for the three software tools. Physicians' rating identified equivalent or superior performance of some AI-based contours in head (eye, lens, optic nerve, brain, chiasm), thorax (e.g., heart and lungs), and pelvis and abdomen (e.g., kidney, femoral head) compared to manual contours. For some OARs, the AI models provided results requiring only minor corrections. Bowel-bag and stomach were not fit for direct use. During the interdisciplinary discussion, the physicians' rating was considered the most relevant.

Conclusion

A comprehensive method for evaluation and clinical implementation of commercially available auto-segmentation software was developed. The in-depth analysis yielded clear instructions for clinical use within the radiotherapy department.

Feasibility of artificial intelligence-driven interfractional monitoring of organ changes by mega-voltage computed tomography in intensity-modulated radiotherapy of prostate cancer

PURPOSE

High-dose radiotherapy (RT) for localized prostate cancer requires careful consideration of target position changes and adjacent organs-at-risk (OARs), such as the rectum and bladder. Therefore, daily monitoring of target position and OAR changes is crucial in minimizing interfractional dosimetric uncertainties. For efficient monitoring of the internal condition of patients, we assessed the feasibility of an auto-segmentation of OARs on the daily acquired images, such as megavoltage computed tomography (MVCT), via a commercial artificial intelligence (AI)-based solution in this study.

MATERIALS AND METHODS

We collected MVCT images weekly during the entire course of RT for 100 prostate cancer patients treated with the helical TomoTherapy system. Based on the manually contoured body outline, the bladder including prostate area, and rectal balloon regions for the 100 MVCT images, we trained the commercially available fully convolutional (FC)-DenseNet model and tested its auto-contouring performance.

RESULTS

Based on the optimally determined hyperparameters, the FC-DenseNet model successfully auto-contoured all regions of interest showing high dice similarity coefficient (DSC) over 0.8 and a small mean surface distance (MSD) within 1.43 mm in reference to the manually contoured data. With this well-trained AI model, we have efficiently monitored the patient's internal condition through six MVCT scans, analyzing DSC, MSD, centroid, and volume differences.

CONCLUSION

We have verified the feasibility of utilizing a commercial AI-based model for auto-segmentation with low-quality daily MVCT images. In the future, we will establish a fast and accurate auto-segmentation and internal organ monitoring system for efficiently determining the time for adaptive replanning.