Evaluation of atlas-based auto-segmentation software in prostate cancer patients

Impact of Artificial Intelligence-Based Autosegmentation of Organs at Risk in Low- and Middle-Income Countries

Purpose

Radiation therapy (RT) processes require significant human resources and expertise, creating a barrier to rapid RT deployment in low- and middle-income countries (LMICs). Accurate segmentation of tumor targets and organs at risk (OARs) is crucial for optimal RT. This study assessed the impact of artificial intelligence (AI)-based autosegmentation of OARs in 2 LMICs.

Methods and Materials

Ten patients, comprising 5 head and neck (HN) cancer patients and 5 prostate cancer patients, were randomly selected. Planning computed tomography images were subjected to autosegmentation using an Food and Drug Administration-approved AI software tool and manual segmentation by experienced radiation oncologists from 2 LMIC RT clinics. The control data, obtained from a large academic institution in the United States, consisted of contours obtained by an experienced radiation oncologist. The segmentation time, DICE similarity coefficient (DSC), Hausdorff distance, and mean surface distance were evaluated.

Results

AI significantly reduced segmentation time, averaging 2 minutes per patient, compared with 57 to 84 minutes for manual contouring in LMICs. Compared with the control data, the AI pelvic contours provided better agreement than did the LMIC manual contours (mean DSC of 0.834 vs 0.807 in LMIC1 and 0.844 vs 0.801 in LMIC2). For HN contours, AI provided better agreement for the majority of OAR contours than manual contours in LMIC1 (mean DSC: 0.823 vs 0.821) or LMIC2 (mean DSC: 0.792 vs 0.748). Neither the AI nor LMIC manual contours had good agreement with the control data (DSC < 0.600) for the optic nerves, chiasm, and cochlea.

Conclusions

AI-based autosegmentation generates OAR contours of comparable quality to manual segmentation for both pelvic and HN cancer patients in LMICs, with substantial time savings.

Efficient application of deep learning-based elective lymph node regions delineation for pelvic malignancies

BACKGROUND

While there are established international consensuses on the delineation of pelvic lymph node regions (LNRs), significant inter- and intra-observer variabilities persist. Contouring these clinical target volumes for irradiation in pelvic malignancies is both time-consuming and labor-intensive.

PURPOSE

The purpose of this study was to develop a deep learning model of pelvic LNRs delineation for patients with pelvic cancers.

METHODS

Planning computed tomography (CT) studies of 160 patients with pelvic primary malignancies (including rectal, prostate, and cervical cancer) were retrospectively collected and divided into training set (n = 120) and testing set (n = 40). Six pelvic LNRs, including abdominal presacral, pelvic presacral, internal iliac nodes, external iliac nodes, obturator nodes, and inguinal nodes were delineated by two radiation oncologists as ground truth (Gt) contours. The cascaded multi-heads U-net (CMU-net) was constructed based on the Gt contours from training cohort, which was subsequently verified in the testing cohort. The automatic delineation of six LNRs (Auto) was evaluated using dice similarity coefficient (DSC), average surface distance (ASD), 95th percentile Hausdorff distance (HD95), and a 7-point scale score.

RESULTS

In the testing set, the DSC of six pelvic LNRs by CMU-net model varied from 0.851 to 0.942, ASD from 0.381 to 1.037 mm, and HD95 from 2.025 to 3.697 mm. No significant differences were founded in these three parameters between postoperative and preoperative cases. 95.9% and 96.2% of auto delineations by CMU-net model got a score of 1-3 by two expert radiation oncologists, respectively, meaning only minor edits needed.

CONCLUSIONS

The CMU-net was successfully developed for automated delineation of pelvic LNRs for pelvic malignancies radiotherapy with improved contouring efficiency and highly consistent, which might justify its implementation in radiotherapy work flow.

Automated segmentation in pelvic radiotherapy: A comprehensive evaluation of ATLAS-, machine learning-, and deep learning-based models

Artificial intelligence can standardize and automatize highly demanding procedures, such as manual segmentation, especially in an anatomical site as common as the pelvis. This study investigated four automated segmentation tools on computed tomography (CT) images in female and male pelvic radiotherapy (RT) starting from simpler and well-known atlas-based methods to the most recent neural networks-based algorithms. The evaluation included quantitative, qualitative and time efficiency assessments. A mono-institutional consecutive series of 40 cervical cancer and 40 prostate cancer structure sets were retrospectively selected. After a preparatory phase, the remaining 20 testing sets per each site were auto-segmented by the atlas-based model STAPLE, a Random Forest-based model, and two Deep Learning-based tools (DL), MVision and LimbusAI. Setting manual segmentation as the Ground Truth, 200 structure sets were compared in terms of Dice Similarity Coefficient (DSC), Hausdorff Distance (HD), and Distance-to-Agreement Portion (DAP). Automated segmentation and manual correction durations were recorded. Expert clinicians performed a qualitative evaluation. In cervical cancer CTs, DL outperformed the other tools with higher quantitative metrics, qualitative scores, and shorter correction times. On the other hand, in prostate cancer CTs, the performance across all the analyzed tools was comparable in terms of both quantitative and qualitative metrics. Such discrepancy in performance outcome could be explained by the wide range of anatomical variability in cervical cancer with respect to the strict bladder and rectum filling preparation in prostate Stereotactic Body Radiation Therapy (SBRT). Decreasing segmentation times can reduce the burden of pelvic radiation therapy routine in an automated workflow.

Comparison of commercial atlas-based automatic segmentation software for prostate radiotherapy treatment planning

This study aims to assess the accuracy of automatic atlas-based contours for various key anatomical structures in prostate radiotherapy treatment planning. The evaluated structures include the bladder, rectum, prostate, seminal vesicles, femoral heads and penile bulb. CT images from 20 patients who underwent intensity-modulated radiotherapy were randomly chosen to create an atlas library. Atlas contours of the seven anatomical structures were generated using four software packages: ABAS, Eclipse, MIM, and RayStation. These contours were then compared to manual delineations performed by oncologists, which served as the ground truth. Evaluation metrics such as dice similarity coefficient (DSC), mean distance to agreement (MDA), and volume ratio (VR) were calculated to assess the accuracy of the contours. Additionally, the time taken by each software to generate the atlas contour was recorded. The mean DSC values for the bladder exhibited strong agreement (>0.8) with manual delineations for all software except for Eclipse and RayStation. Similarly, the femoral heads showed significant similarity between the atlas contours and ground truth across all software, with mean DSC values exceeding 0.9 and MDA values close to zero. On the other hand, the penile bulb displayed only moderate agreement with the ground truth, with mean DSC values ranging from 0.5 to 0.7 for all software. A similar trend was observed in the prostate atlas contours, except for MIM, which achieved a mean DSC of over 0.8. For the rectum, both ABAS and MIM atlases demonstrated strong agreement with the ground truth, resulting in mean DSC values of more than 0.8. Overall, MIM and ABAS outperformed Eclipse and RayStation in both DSC and MDA. These results indicate that the atlas-based segmentation employed in this study produces acceptable contours for the anatomical structures of interest in prostate radiotherapy treatment planning.

Evaluation of multiple-vendor AI autocontouring solutions

BACKGROUND

Multiple artificial intelligence (AI)-based autocontouring solutions have become available, each promising high accuracy and time savings compared with manual contouring. Before implementing AI-driven autocontouring into clinical practice, three commercially available CT-based solutions were evaluated.

MATERIALS AND METHODS

The following solutions were evaluated in this work: MIM-ProtégéAI+ (MIM), Radformation-AutoContour (RAD), and Siemens-DirectORGANS (SIE). Sixteen organs were identified that could be contoured by all solutions. For each organ, ten patients that had manually generated contours approved by the treating physician (AP) were identified, totaling forty-seven different patients. CT scans in the supine position were acquired using a Siemens-SOMATOMgo 64-slice helical scanner and used to generate autocontours. Physician scoring of contour accuracy was performed by at least three physicians using a five-point Likert scale. Dice similarity coefficient (DSC), Hausdorff distance (HD) and mean distance to agreement (MDA) were calculated comparing AI contours to "ground truth" AP contours.

RESULTS

The average physician score ranged from 1.00, indicating that all physicians reviewed the contour as clinically acceptable with no modifications necessary, to 3.70, indicating changes are required and that the time taken to modify the structures would likely take as long or longer than manually generating the contour. When averaged across all sixteen structures, the AP contours had a physician score of 2.02, MIM 2.07, RAD 1.96 and SIE 1.99. DSC ranged from 0.37 to 0.98, with 41/48 (85.4%) contours having an average DSC ≥ 0.7. Average HD ranged from 2.9 to 43.3 mm. Average MDA ranged from 0.6 to 26.1 mm.

CONCLUSIONS

The results of our comparison demonstrate that each vendor's AI contouring solution exhibited capabilities similar to those of manual contouring. There were a small number of cases where unusual anatomy led to poor scores with one or more of the solutions. The consistency and comparable performance of all three vendors' solutions suggest that radiation oncology centers can confidently choose any of the evaluated solutions based on individual preferences, resource availability, and compatibility with their existing clinical workflows. Although AI-based contouring may result in high-quality contours for the majority of patients, a minority of patients require manual contouring and more in-depth physician review.

Feasibility evaluation of novel AI-based deep-learning contouring algorithm for radiotherapy

PURPOSE

To evaluate the clinical feasibility of the Siemens Healthineers AI-Rad Companion Organs RT VA30A (Organs-RT) auto-contouring algorithm for organs at risk (OARs) of the pelvis, thorax, and head and neck (H&N).

METHODS

Computed tomography (CT) datasets from 30 patients (10 pelvis, 10 thorax, and 10 H&N) were collected. Four sets of OARs were generated on each scan, one set by Organs-RT and the others by three experienced users independently. A physician (expert) then evaluated each contour by assigning a score from the following scale: 1-Must Redo, 2-Major Edits, 3-Minor Edits, 4-Clinically usable. Using the highest-scored OAR from the human users as a reference, the contours generated by Organs-RT were evaluated via Dice Similarity Coefficient (DSC), Hausdorff Distance (HDD), Mean Distance to Agreement (mDTA), Volume comparison, and visual inspection. Additionally, each human user recorded the time to delineate each structure set and time-saving efficiency was measured.

RESULTS

The average DSC obtained for the pelvic OARs ranged between (0.81 ± 0.06)Rectum and (0.94 ± 0.03)Bladder . (0.75 ± 0.09)Esophagus to( 0.96 ± 0.02 ) Rt . Lung ${( {0.96 \pm 0.02} )}_{{\mathrm{Rt}}.{\mathrm{\ Lung}}}$ for the thoracic OARs and (0.66 ± 0.07)Lips to (0.83 ± 0.04)Brainstem for the H&N. The average HDD in cm for the pelvis cohort ranged between (0.95 ± 0.35)Bladder to (3.62 ± 2.50)Rectum , (0.42 ± 0.06)SpinalCord to (2.09 ± 2.00)Esophagus for the thoracic set and( 0.53 ± 0.22 ) Cerv _ SpinalCord ${( {0.53 \pm 0.22} )}_{{\mathrm{Cerv}}\_{\mathrm{SpinalCord}}}$ to (1.50 ± 0.50)Mandible for the H&N region. The time-saving efficiency was 67% for H&N, 83% for pelvis, and 84% for thorax. 72.5%, 82%, and 50% of the pelvis, thorax, and H&N OARs were scored as clinically usable by the expert, respectively.

CONCLUSIONS

The highest agreement registered between OARs generated by Organs-RT and their respective references was for the bladder, heart, lungs, and femoral heads, with an overall DSC≥0.92. The poorest agreement was for the rectum, esophagus, and lips, with an overall DSC⩽0.81. Nonetheless, Organs-RT serves as a reliable auto-contouring tool by minimizing overall contouring time and increasing time-saving efficiency in radiotherapy treatment planning.

Real-world validation of Artificial Intelligence-based Computed Tomography auto-contouring for prostate cancer radiotherapy planning

Background and purpose

Artificial Intelligence (AI)-based auto-contouring for treatment planning in radiotherapy needs extensive clinical validation, including the impact of editing after automatic segmentation. The aims of this study were to assess the performance of a commercial system for Clinical Target Volumes (CTVs) (prostate/seminal vesicles) and selected Organs at Risk (OARs) (rectum/bladder/femoral heads + femurs), evaluating also inter-observer variability (manual vs automatic + editing) and the reduction of contouring time.

Materials and methods

Two expert observers contoured CTVs/OARs of 20 patients in our Treatment Planning System (TPS). Computed Tomography (CT) images were sent to the automatic contouring workstation: automatic contours were generated and sent back to TPS, where observers could edit them if necessary. Inter- and intra-observer consistency was estimated using Dice Similarity Coefficients (DSC). Radiation oncologists were also asked to score the quality of automatic contours, ranging from 1 (complete re-contouring) to 5 (no editing). Contouring times (manual vs automatic + edit) were compared.

Results

DSCs (manual vs automatic only) were consistent with inter-observer variability (between 0.65 for seminal vesicles and 0.94 for bladder); editing further improved performances (range: 0.76-0.94). The median clinical score was 4 (little editing) and it was <4 in 3/2 patients for the two observers respectively. Inter-observer variability of automatic + editing contours improved significantly, being lower than manual contouring (e.g.: seminal vesicles: 0.83vs0.73; prostate: 0.86vs0.83; rectum: 0.96vs0.81). Oncologist contouring time reduced from 17 to 24 min of manual contouring time to 3-7 min of editing time for the two observers (p < 0.01).

Conclusion

Automatic contouring with a commercial AI-based system followed by editing can replace manual contouring, resulting in significantly reduced time for segmentation and better consistency between operators.

Evaluating the Effectiveness of Deep Learning Contouring across Multiple Radiotherapy Centres

Background and purpose

Deep learning contouring (DLC) has the potential to decrease contouring time and variability of organ contours. This work evaluates the effectiveness of DLC for prostate and head and neck across four radiotherapy centres using a commercial system.

Materials and methods

Computed tomography scans of 123 prostate and 310 head and neck patients were evaluated. Besides one head and neck model, generic DLC models were used. Contouring time using centres' existing clinical methods and contour editing time after DLC were compared. Timing was evaluated using paired and non-paired studies. Commercial software or in-house scripts assessed dice similarity coefficient (DSC) and distance to agreement (DTA). One centre assessed head and neck inter-observer variability.

Results

The mean contouring time saved for prostate structures using DLC compared to the existing clinical method was 5.9 ± 3.5 min. The best agreement was shown for the femoral heads (median DSC 0.92 ± 0.03, median DTA 1.5 ± 0.3 mm) and the worst for the rectum (median DSC 0.68 ± 0.04, median DTA 4.6 ± 0.6 mm). The mean contouring time saved for head and neck structures using DLC was 16.2 ± 8.6 min. For one centre there was no DLC time-saving compared to an atlas-based method. DLC contours reduced inter-observer variability compared to manual contours for the brainstem, left parotid gland and left submandibular gland.

Conclusions

Generic prostate and head and neck DLC models can provide time-savings which can be assessed with paired or non-paired studies to integrate with clinical workload. Reducing inter-observer variability potential has been shown.

Clinical evaluation of deep learning and atlas-based auto-segmentation for critical organs at risk in radiation therapy

INTRODUCTION

Contouring organs at risk (OARs) is a time-intensive task that is a critical part of radiation therapy. Atlas-based automatic segmentation has shown some success at reducing this time burden on practitioners; however, this method often requires significant manual editing to reach a clinically accurate standard. Deep learning (DL) auto-segmentation has recently emerged as a promising solution. This study compares the accuracy of DL and atlas-based auto-segmentation in relation to clinical 'gold standard' reference contours.

METHODS

Ninety CT datasets (30 head and neck, 30 thoracic, 30 pelvic) were automatically contoured using both atlas and DL segmentation techniques. Sixteen critical OARs were then quantitatively measured for accuracy using the Dice similarity coefficient (DSC) and Hausdorff distance (HD). Qualitative analysis was performed to visually classify the accuracy of each structure into one of four explicitly defined categories. Additionally, the time to edit atlas and DL contours to a clinically acceptable level was recorded for a subset of 9 OARs.

RESULTS

Of the 16 OARs analysed, DL delivered statistically significant improvements over atlas segmentation in 13 OARs measured with DSC, 12 OARs measured with HD, and 12 OARs measured qualitatively. The mean editing time for the subset of DL contours was 50%, 23% and 61% faster (all P < 0.05) than that of atlas segmentation for the head and neck, thorax, and pelvis respectively.

CONCLUSIONS

Deep learning segmentation comprehensively outperformed atlas-based contouring for the majority of evaluated OARs. Improvements were observed in geometric accuracy and visual acceptability, while editing time was reduced leading to increased workflow efficiency.

The clinical evaluation of atlas-based auto-segmentation for automatic contouring during cervical cancer radiotherapy

Purpose

Our purpose was to investigate the influence of atlas library size and CT cross-slice number on the accuracy and efficiency of the atlas-based auto-segmentation (ABAS) method for the automatic contouring of clinical treatment volume (CTV) and organs at risk (OARs) during cervical cancer radiotherapy.

Methods

Of 140 cervical cancer patients, contours from 20, 40, 60, 80, 100, and 120 patients were selected incrementally to create six atlas library groups in ABAS. Another 20 tested patients were automatically contoured with the ABAS method and manually contoured by the same professional oncologist. Contours included CTV, bladder, rectum, femoral head-L, femoral head-R, and spinal cord. The CT cross-slice numbers of the 20 tested patients included 61, 65, 72, 75, 81, and 84. The index of dice similarity coefficients (DSCs) and Hausdorff distance (HD) were used to assess the consistency between ABAS automatic contouring and manual contouring. The randomized block analysis of variance and paired t-test were used for statistical analysis.

Results

The mean DSC values of “CTV, bladder, femoral head, and spinal cord” were all larger than 0.8. The femoral head and spinal cord showed a high degree of agreement between ABAS automatic contouring and manual contouring, with a mean DC >0.80 and HD <1 cm in all atlas library groups. A post-hoc least significant difference comparison indicated that no significant difference had been found between different atlas library sizes with DSC and HD values. For ABAS efficiency, the atlas library size had no effect on the time of ABAS automatic contouring. The time of automatic contouring increased slightly with the increase in CT cross-slice numbers, which were 99.9, 106.8, 114.0, 120.6, 127.9, and 134.8 s with CT cross-slices of 61, 65, 72, 75, 81, and 84, respectively.

Conclusion

A total of 20 atlas library sizes and a minimum CT cross-slice number including CTV and OARs are enough for ensuring the accuracy and efficiency of ABAS automatic contouring during cervical cancer radiotherapy.

Convolutional Neural Networks Enable Robust Automatic Segmentation of the Rat Hippocampus in MRI After Traumatic Brain Injury

Registration-based methods are commonly used in the automatic segmentation of magnetic resonance (MR) brain images. However, these methods are not robust to the presence of gross pathologies that can alter the brain anatomy and affect the alignment of the atlas image with the target image. In this work, we develop a robust algorithm, MU-Net-R, for automatic segmentation of the normal and injured rat hippocampus based on an ensemble of U-net-like Convolutional Neural Networks (CNNs). MU-Net-R was trained on manually segmented MR images of sham-operated rats and rats with traumatic brain injury (TBI) by lateral fluid percussion. The performance of MU-Net-R was quantitatively compared with methods based on single and multi-atlas registration using MR images from two large preclinical cohorts. Automatic segmentations using MU-Net-R and multi-atlas registration were of excellent quality, achieving cross-validated Dice scores above 0.90 despite the presence of brain lesions, atrophy, and ventricular enlargement. In contrast, the performance of single-atlas segmentation was unsatisfactory (cross-validated Dice scores below 0.85). Interestingly, the registration-based methods were better at segmenting the contralateral than the ipsilateral hippocampus, whereas MU-Net-R segmented the contralateral and ipsilateral hippocampus equally well. We assessed the progression of hippocampal damage after TBI by using our automatic segmentation tool. Our data show that the presence of TBI, time after TBI, and whether the hippocampus was ipsilateral or contralateral to the injury were the parameters that explained hippocampal volume.

Metrics to evaluate the performance of auto-segmentation for radiation treatment planning: A critical review

Advances in artificial intelligence-based methods have led to the development and publication of numerous systems for auto-segmentation in radiotherapy. These systems have the potential to decrease contour variability, which has been associated with poor clinical outcomes and increased efficiency in the treatment planning workflow. However, there are no uniform standards for evaluating auto-segmentation platforms to assess their efficacy at meeting these goals. Here, we review the most frequently used evaluation techniques which include geometric overlap, dosimetric parameters, time spent contouring, and clinical rating scales. These data suggest that many of the most commonly used geometric indices, such as the Dice Similarity Coefficient, are not well correlated with clinically meaningful endpoints. As such, a multi-domain evaluation, including composite geometric and/or dosimetric metrics with physician-reported assessment, is necessary to gauge the clinical readiness of auto-segmentation for radiation treatment planning.

An evaluation of MR based deep learning auto-contouring for planning head and neck radiotherapy

INTRODUCTION

Auto contouring models help consistently define volumes and reduce clinical workload. This study aimed to evaluate the cross acquisition of a Magnetic Resonance (MR) deep learning auto contouring model for organ at risk (OAR) delineation in head and neck radiotherapy.

METHODS

Two auto contouring models were evaluated using deep learning contouring expert (DLCExpert) for OAR delineation: a CT model (modelCT) and an MR model (modelMRI). Models were trained to generate auto contours for the bilateral parotid glands and submandibular glands. Auto-contours for modelMRI were trained on diagnostic images and tested on 10 diagnostic, 10 MR radiotherapy planning (RTP), eight MR-Linac (MRL) scans and, by modelCT, on 10 CT planning scans. Goodness of fit scores, dice similarity coefficient (DSC) and distance to agreement (DTA) were calculated for comparison.

RESULTS

ModelMRI contours improved the mean DSC and DTA compared with manual contours for the bilateral parotid glands and submandibular glands on the diagnostic and RTP MRs compared with the MRL sequence. There were statistically significant differences seen for modelMRI compared to modelCT for the left parotid (mean DTA 2.3 v 2.8 mm), right parotid (mean DTA 1.9 v 2.7 mm), left submandibular gland (mean DTA 2.2 v 2.4 mm) and right submandibular gland (mean DTA 1.6 v 3.2 mm).

CONCLUSION

A deep learning MR auto-contouring model shows promise for OAR auto-contouring with statistically improved performance vs a CT based model. Performance is affected by the method of MR acquisition and further work is needed to improve its use with MRL images.

MR-Guided Radiotherapy for Prostate Cancer

External beam radiotherapy remains the primary treatment modality for localized prostate cancer. The radiobiology of prostate carcinoma lends itself to hypofractionation, with recent studies showing good outcomes with shorter treatment schedules. However, the ability to accurately deliver hypofractionated treatment is limited by current image-guided techniques. Magnetic resonance imaging is the main diagnostic tool for localized prostate cancer and its use in the therapeutic setting offers anatomical information to improve organ delineation. MR-guided radiotherapy, with daily re-planning, has shown early promise in the accurate delivery of radiotherapy. In this article, we discuss the shortcomings of current image-guidance strategies and the potential benefits and limitations of MR-guided treatment for prostate cancer. We also recount present experiences of MR-linac workflow and the opportunities afforded by this technology.

Methodological approach to create an atlas using a commercial auto-contouring software

PURPOSE

The aim of this work was to establish a methodological approach for creation and optimization of an atlas for auto-contouring, using the commercial software MIM MAESTRO (MIM Software Inc. Cleveland OH).

METHODS

A computed tomography (CT) male pelvis atlas was created and optimized to evaluate how different tools and options impact on the accuracy of automatic segmentation. Pelvic lymph nodes (PLN), rectum, bladder, and femurs of 55 subjects were reviewed for consistency by a senior consultant radiation oncologist with 15 yr of experience. Several atlas and workflow options were tuned to optimize the accuracy of auto-contours. The deformable image registration (DIR), the finalization method, the k number of atlas best matching subjects, and several post-processing options were studied. To test our atlas performances, automatic and reference manual contours of 20 test subjects were statistically compared based on dice similarity coefficient (DSC) and mean distance to agreement (MDA) indices. The effect of field of view (FOV) reduction on auto-contouring time was also investigated.

RESULTS

With the optimized atlas and workflow, DSC and MDA median values of bladder, rectum, PLN, and femurs were 0.91 and 1.6 mm, 0.85 and 1.6 mm, 0.85 and 1.8 mm, and 0.96 and 0.5 mm, respectively. Auto-contouring time was more than halved by strictly cropping the FOV of the subject to be contoured to the pelvic region.

CONCLUSION

A statistically significant improvement of auto-contours accuracy was obtained using our atlas and optimized workflow instead of the MIM Software pelvic atlas.

Automatic segmentation of pelvic organs-at-risk using a fusion network model based on limited training samples

Background: Efficient and accurate methods are needed to automatically segmenting organs-at-risk (OAR) to accelerate the radiotherapy workflow and decrease the treatment wait time. We developed and evaluated the use of a fused model Dense V-Network for its ability to accurately segment pelvic OAR.Material and methods: We combined two network models, Dense Net and V-Net, to establish the Dense V-Network algorithm. For the training model, we adopted 100 kV computed tomography (CT) images of patients with cervical cancer, including 80 randomly selected as training sets, by which to adjust parameters of the automatic segmentation model, and the remaining 20 as test sets to evaluate the performance of the convolutional neural network model. Three representative parameters were used to evaluate the segmentation results quantitatively.Results: Clinical results revealed that Dice similarity coefficient values of the bladder, small intestine, rectum, femoral head and spinal cord were all above 0.87 mm; and Jaccard distance was within 2.3 mm. Except for the small intestine, the Hausdorff distance of other organs was less than 9.0 mm. Comparison of our approaches with those of the Atlas and other studies demonstrated that the Dense V-Network had more accurate and efficient performance and faster speed.Conclusions: The Dense V-Network algorithm can be used to automatically segment pelvic OARs accurately and efficiently, while shortening patients' waiting time and accelerating radiotherapy workflow.

Clinical Evaluation of Deep Learning and Atlas-Based Auto-Contouring of Bladder and Rectum for Prostate Radiation Therapy

PURPOSE

Auto-contouring may reduce workload, interobserver variation, and time associated with manual contouring of organs at risk. Manual contouring remains the standard due in part to uncertainty around the time and workload savings after accounting for the review and editing of auto-contours. This preliminary study compares a standard manual contouring workflow with 2 auto-contouring workflows (atlas and deep learning) for contouring the bladder and rectum in patients with prostate cancer.

METHODS AND MATERIALS

Three contouring workflows were defined based on the initial contour-generation method including manual (MAN), atlas-based auto-contour (ATLAS), and deep-learning auto-contour (DEEP). For each workflow, initial contour generation was retrospectively performed on 15 patients with prostate cancer. Then, radiation oncologists (ROs) edited each contour while blinded to the manner in which the initial contour was generated. Workflows were compared by time (both in initial contour generation and in RO editing), contour similarity, and dosimetric evaluation.

RESULTS

Mean durations for initial contour generation were 10.9 min, 1.4 min, and 1.2 min for MAN, DEEP, and ATLAS, respectively. Initial DEEP contours were more geometrically similar to initial MAN contours. Mean durations of the RO editing steps for MAN, DEEP, and ATLAS contours were 4.1 min, 4.7 min, and 10.2 min, respectively. The geometric extent of RO edits was consistently larger for ATLAS contours compared with MAN and DEEP. No differences in clinically relevant dose-volume metrics were observed between workflows.

CONCLUSION

Auto-contouring software affords time savings for initial contour generation; however, it is important to also quantify workload changes at the RO editing step. Using deep-learning auto-contouring for bladder and rectum contour generation reduced contouring time without negatively affecting RO editing times, contour geometry, or clinically relevant dose-volume metrics. This work contributes to growing evidence that deep-learning methods are a clinically viable solution for organ-at-risk contouring in radiation therapy.

Atlas-based auto-segmentation for postoperative radiotherapy planning in endometrial and cervical cancers

Background

Since intensity-modulated radiation therapy (IMRT) has become popular for the treatment of gynecologic cancers, the contouring process has become more critical. This study evaluated the feasibility of atlas-based auto-segmentation (ABAS) for contouring in patients with endometrial and cervical cancers.

Methods

A total of 75 sets of planning CT images from 75 patients were collected. Contours for the pelvic nodal clinical target volume (CTV), femur, and bladder were carefully generated by two skilled radiation oncologists. Of 75 patients, 60 were randomly registered in three different atlas libraries for ABAS in groups of 20, 40, or 60. ABAS was conducted in 15 patients, followed by manual correction (ABAS_c). The time required to generate all contours was recorded, and the accuracy of segmentation was assessed using Dice’s coefficient (DC) and the Hausdorff distance (HD) and compared to those of manually delineated contours.

Results

For ABAS-CTV, the best results were achieved with groups of 60 patients (DC, 0.79; HD, 19.7 mm) and the worst results with groups of 20 patients (DC, 0.75; p = 0.012; HD, 21.3 mm; p = 0.002). ABAS_c-CTV performed better than ABAS-CTV in terms of both HD and DC (ABAS_c [n = 60]; DC, 0.84; HD, 15.6 mm; all p < 0.017). ABAS required an average of 45.1 s, whereas ABAS_c required 191.1 s; both methods required less time than the manual methods (p < 0.001). Both ABAS-Femur and simultaneous ABAS-Bilateral-femurs showed satisfactory performance, regardless of the atlas library used (DC > 0.9 and HD ≤10.0 mm), with significant time reduction compared to that needed for manual delineation (p < 0.001). However, ABAS-Bladder did not prove to be feasible, with inferior results regardless of library size (DC < 0.6 and HD > 40 mm). Furthermore, ABAS_c-Bladder required a longer processing time than manual contouring to achieve the same accuracy.

Conclusions

ABAS could help physicians to delineate the CTV and organs-at-risk (e.g., femurs) in IMRT planning considering its consistency, efficacy, and accuracy.

Implementing user-defined atlas-based auto-segmentation for a large multi-centre organisation: the Australian Experience

INTRODUCTION

Contouring has become an increasingly important aspect of radiation therapy due to inverse planning, and yet is extremely time-consuming. To improve contouring efficiency and reduce potential inter-observer variation, the atlas-based auto-segmentation (ABAS) function in Velocity was introduced to ICON cancer centres (ICC) throughout Australia as a solution for automatic contouring.

METHODS

This paper described the implementation process of the ABAS function and the construction of user-defined atlas sets and compared the contouring efficiency before and after the introduction of ABAS.

RESULTS

The results indicate that the main limitation to the ABAS performance was Velocity's sub-optimal atlas selection method. Three user-defined atlas sets were constructed. Results suggested that the introduction of the ABAS saved at least 5 minutes of manual contouring time (P < 0.05), although further verification was required due to limitations in the data collection method. The pilot rollout adopting a 'champion' approach was successful and provided an opportunity to improve the user-defined atlases prior to the national implementation.

CONCLUSION

The implementation of user-defined ABAS for head and neck (H&N) and female thorax patients at ICCs was successful, which achieved at least 5 minutes of efficiency gain.

Impact of CT reconstruction algorithm on auto-segmentation performance

Model‐based iterative reconstruction (MBIR) reduces CT imaging dose while maintaining image quality. However, MBIR reduces noise while preserving edges which may impact intensity‐based tasks such as auto‐segmentation. This work evaluates the sensitivity of an auto‐contouring prostate atlas across multiple MBIR reconstruction protocols and benchmarks the results against filtered back projection (FBP). Images were created from raw projection data for 11 prostate cancer cases using FBP and nine different MBIR reconstructions (3 protocols/3 noise reduction levels) yielding 10 reconstructions/patient. Five bony structures, bladder, rectum, prostate, and seminal vesicles (SVs) were segmented using an auto‐segmentation pipeline that renders 3D binary masks for analysis. Performance was evaluated for volume percent difference (VPD) and Dice similarity coefficient (DSC), using FBP as the gold standard. Nonparametric Friedman tests plus post hoc all pairwise comparisons were employed to test for significant differences (P < 0.05) for soft tissue organs and protocol/level combinations. A physician performed qualitative grading of 396 MBIR contours across the prostate, bladder, SVs, and rectum in comparison to FBP using a six‐point scale. MBIR contours agreed with FBP for bony anatomy (DSC ≥ 0.98), bladder (DSC ≥ 0.94, VPD < 8.5%), and prostate (DSC = 0.94 ± 0.03, VPD = 4.50 ± 4.77% (range: 0.07–26.39%). Increased variability was observed for rectum (VPD = 7.50 ± 7.56% and DSC = 0.90 ± 0.08) and SVs (VPD and DSC of 8.23 ± 9.86% range (0.00–35.80%) and 0.87 ± 0.11, respectively). Over the all protocol/level comparisons, a significant difference was observed for the prostate VPD between BSPL1 and BSTL2 (adjusted P‐value = 0.039). Nevertheless, 300 of 396 (75.8%) of the four soft tissue structures using MBIR were graded as equivalent or better than FBP, suggesting that MBIR offered potential improvements in auto‐segmentation performance when compared to FBP. Future work may involve tuning organ‐specific MBIR parameters to further improve auto‐segmentation performance. Running title: Impact of CT Reconstruction Algorithm on Auto‐segmentation Performance.

Improving fiducial and prostate capsule visualization for radiotherapy planning using MRI

BACKGROUND AND PURPOSE

Intraprostatic fiducial markers (FM) improve the accuracy of radiotherapy (RT) delivery. Here we assess geometric integrity and contouring consistency using a T2*-weighted (T2*W) sequence alone, which allows visualization of the FM.

MATERIAL AND METHODS

Ten patients scanned within the Prostate Advances in Comparative Evidence (PACE) trial (NCT01584258) had prostate images acquired with computed tomography (CT) and Magnetic Resonance (MR) Imaging: T2-weighted (T2W) and T2*W sequences. The prostate was contoured independently on each imaging dataset by three clinicians. Interobserver variability was assessed using comparison indices with Monaco ADMIRE (research version 2.0, Elekta AB) and examined for statistical differences between imaging sets. CT and MR images of two test objects were acquired to assess geometric distortion and accuracy of marker positioning. The first was a linear test object comprising straight tubes in three orthogonal directions, the second was a smaller test object with markers suspended in gel.

RESULTS

Interobserver variability for prostate contouring was lower for both T2W and T2*W compared to CT, this was statistically significant when comparing CT and T2*W images. All markers are visible in T2*W images with 29/30 correctly identified, only 3/30 are visible in T2W images. Assessment of geometric distortion revealed in-plane displacements were under 0.375 mm in MRI, and through plane displacements could not be detected. The signal loss in the MR images is symmetric in relation to the true marker position shown in CT images.

CONCLUSION

Prostate T2*W images are geometrically accurate, and yield consistent prostate contours. This single sequence can be used to identify FM and for prostate delineation in a mixed MR-CT workflow.

[Siriade 2.0: An e-learning platform for radiation oncology contouring]

PURPOSE

In 2008, the French national society of radiation oncology (SFRO) and the association for radiation oncology continued education (AFCOR) created Siriade, an e-learning website dedicated to contouring.

MATERIAL AND METHODS

Between 2015 and 2017, this platform was updated using the latest digital online tools available. Two main sections were needed: a theoretical part and another section of online workshops.

RESULTS

Teaching courses are available as online commented videos, available on demand. The practical section of the website is an online contouring workshop that automatically generates a report quantifying the quality of the user's delineation compared with the experts'.

CONCLUSION

Siriade 2.0 is an innovating digital tool for radiation oncology initial and continuous education.

Comparative evaluation of autocontouring in clinical practice: A practical method using the Turing test

PURPOSE

Automated techniques for estimating the contours of organs and structures in medical images have become more widespread and a variety of measures are available for assessing their quality. Quantitative measures of geometric agreement, for example, overlap with a gold-standard delineation, are popular but may not predict the level of clinical acceptance for the contouring method. Therefore, surrogate measures that relate more directly to the clinical judgment of contours, and to the way they are used in routine workflows, need to be developed. The purpose of this study is to propose a method (inspired by the Turing Test) for providing contour quality measures that directly draw upon practitioners' assessments of manual and automatic contours. This approach assumes that an inability to distinguish automatically produced contours from those of clinical experts would indicate that the contours are of sufficient quality for clinical use. In turn, it is anticipated that such contours would receive less manual editing prior to being accepted for clinical use. In this study, an initial assessment of this approach is performed with radiation oncologists and therapists.

METHODS

Eight clinical observers were presented with thoracic organ-at-risk contours through a web interface and were asked to determine if they were automatically generated or manually delineated. The accuracy of the visual determination was assessed, and the proportion of contours for which the source was misclassified recorded. Contours of six different organs in a clinical workflow were for 20 patient cases. The time required to edit autocontours to a clinically acceptable standard was also measured, as a gold standard of clinical utility. Established quantitative measures of autocontouring performance, such as Dice similarity coefficient with respect to the original clinical contour and the misclassification rate accessed with the proposed framework, were evaluated as surrogates of the editing time measured.

RESULTS

The misclassification rates for each organ were: esophagus 30.0%, heart 22.9%, left lung 51.2%, right lung 58.5%, mediastinum envelope 43.9%, and spinal cord 46.8%. The time savings resulting from editing the autocontours compared to the standard clinical workflow were 12%, 25%, 43%, 77%, 46%, and 50%, respectively, for these organs. The median Dice similarity coefficients between the clinical contours and the autocontours were 0.46, 0.90, 0.98, 0.98, 0.94, and 0.86, respectively, for these organs.

CONCLUSIONS

A better correspondence with time saving was observed for the misclassification rate than the quantitative contour measures explored. From this, we conclude that the inability to accurately judge the source of a contour indicates a reduced need for editing and therefore a greater time saving overall. Hence, task-based assessments of contouring performance may be considered as an additional way of evaluating the clinical utility of autosegmentation methods.

Autosegmentation for thoracic radiation treatment planning: A grand challenge at AAPM 2017

PURPOSE

This report presents the methods and results of the Thoracic Auto-Segmentation Challenge organized at the 2017 Annual Meeting of American Association of Physicists in Medicine. The purpose of the challenge was to provide a benchmark dataset and platform for evaluating performance of autosegmentation methods of organs at risk (OARs) in thoracic CT images.

METHODS

Sixty thoracic CT scans provided by three different institutions were separated into 36 training, 12 offline testing, and 12 online testing scans. Eleven participants completed the offline challenge, and seven completed the online challenge. The OARs were left and right lungs, heart, esophagus, and spinal cord. Clinical contours used for treatment planning were quality checked and edited to adhere to the RTOG 1106 contouring guidelines. Algorithms were evaluated using the Dice coefficient, Hausdorff distance, and mean surface distance. A consolidated score was computed by normalizing the metrics against interrater variability and averaging over all patients and structures.

RESULTS

The interrater study revealed highest variability in Dice for the esophagus and spinal cord, and in surface distances for lungs and heart. Five out of seven algorithms that participated in the online challenge employed deep-learning methods. Although the top three participants using deep learning produced the best segmentation for all structures, there was no significant difference in the performance among them. The fourth place participant used a multi-atlas-based approach. The highest Dice scores were produced for lungs, with averages ranging from 0.95 to 0.98, while the lowest Dice scores were produced for esophagus, with a range of 0.55-0.72.

CONCLUSION

The results of the challenge showed that the lungs and heart can be segmented fairly accurately by various algorithms, while deep-learning methods performed better on the esophagus. Our dataset together with the manual contours for all training cases continues to be available publicly as an ongoing benchmarking resource.

Magnetic Resonance Imaging-Guided Adaptive Radiation Therapy: A "Game Changer" for Prostate Treatment?

Radiation therapy to the prostate involves increasingly sophisticated delivery techniques and changing fractionation schedules. With a low estimated α/β ratio, a larger dose per fraction would be beneficial, with moderate fractionation schedules rapidly becoming a standard of care. The integration of a magnetic resonance imaging (MRI) scanner and linear accelerator allows for accurate soft tissue tracking with the capacity to replan for the anatomy of the day. Extreme hypofractionation schedules become a possibility using the potentially automated steps of autosegmentation, MRI-only workflow, and real-time adaptive planning. The present report reviews the steps involved in hypofractionated adaptive MRI-guided prostate radiation therapy and addresses the challenges for implementation.

Automatic Intracranial Segmentation: Is the Clinician Still Needed?

INTRODUCTION

Stereotactic hypofractionated radiotherapy is an effective treatment for brain metastases in oligometastatic patients. Its planning is however time-consuming because of the number of organs at risk to be manually segmented. This study evaluates 2 automated segmentation commercial software.

METHODS

Patients were scanned in the treatment position. The computed tomography scan was registered on a magnetic resonance imaging and volumes were manually segmented by a clinician. Then 2 automated segmentations were performed (with iPlan and Smart Segmentation). RT STRUCT files were compared with Aquilab's Artistruct segment comparison module. We selected common segmented volume ratio as the main judging criterion. Secondary criteria were Dice-Sørensen coefficients, overlap ratio, and additional segmented volume.

RESULTS

Twenty consecutive patients were included. Agreement between manual and automated contouring was poor. Common segmented volumes ranged from 7.71% to 82.54%, Dice-Sørensen coefficient ranged from 0.0745 to 0.8398, overlap ratio ranged from 0.0414 to 0.7275, and additional segmented volume ranged from 9.80% to 92.25%. Each software outperformed the other on some organs while performing worse on others.

CONCLUSION

No software seemed clearly better than the other. Common segmented volumes were much too low for routine use in stereotactic hypofractionated brain radiotherapy. Manual editing is still needed.