Benefits of deep learning for delineation of organs at risk in head and neck cancer

Is full-automation in radiotherapy treatment planning ready for take off?

Radiotherapy treatment planning is undergoing a transformation with the increasing integration of automation. This transition draws parallels with the aviation industry, which has a long-standing history of addressing challenges and opportunities introduced by automated systems. Both fields witness a shift from manual operations to systems capable of operating independently, raising questions about the risks and evolving role of humans within automated workflows. In response to this shift, a working group assembled during the ESTRO Physics Workshop 2023, reflected on parallels to draw lessons for radiotherapy. A taxonomy is proposed, leveraging insights from aviation, that outlines the observed levels of automation within the context of radiotherapy and their corresponding implications for human involvement. Among the common identified risks associated with automation integration are complacency, overreliance, attention tunneling, data overload, a lack of transparency and training. These risks require mitigation strategies. Such strategies include ensuring role complementarity, introducing checklists and safety requirements for human-automation interaction and using automation for cognitive unload and workflow management. Focusing on already automated processes, such as dose calculation and auto-contouring as examples, we have translated lessons learned from aviation. It remains crucial to strike a balance between automation and human involvement. While automation offers the potential for increased efficiency and accuracy, it must be complemented by human oversight, expertise, and critical decision-making. The irreplaceable value of human judgment remains, particularly in complex clinical situations. Learning from aviation, we identify a need for human factors engineering research in radiation oncology and a continued requirement for proactive incident learning.

Quantifying and visualising uncertainty in deep learning-based segmentation for radiation therapy treatment planning: What do radiation oncologists and therapists want?

BACKGROUND AND PURPOSE

During the ESTRO 2023 physics workshop on "AI for the fully automated radiotherapy treatment chain", the topic of deep learning (DL) segmentation was discussed. Despite its widespread use in radiotherapy, the time needed to evaluate and correct DL segmentations remains burdensome. While segmentation uncertainty could be beneficial for clinicians, there is a lack of understanding on what information should be presented to ease their task. This study aimed to gather insights from clinicians on uncertainty visualisation options.

MATERIALS AND METHODS

Two sessions of structured interviews were conducted across four institutions already using DL segmentation clinically. The first session focused on the main problems hindering the clinical use of DL. In the second session, ten visualisation options displaying uncertainty information at different levels (structure, slice, or voxel) with binary or continuous values were presented. Dosimetric information was also present in some visualisations. For each case, sixteen clinicians (radiation oncologists and radiation therapists) were asked to grade an overall score, the usability, the training required, and the expected time gain.

RESULTS

Participants preferred the binary voxel-level uncertainty visualisation, followed by binary structure-level uncertainty visualisation. Combining structure-level and voxel-level visualisation methods has been proposed as a promising approach. The benefits of dosimetric information were perceived diversely among participants since it complexifies the display but could be useful for the online adaptive workflow.

CONCLUSION

Preferences for uncertainty visualisation methods were assessed within a multi-institutional experienced group of clinicians. Further refinement of preferences may help in selecting the best options for clinical implementation.

A Simulation-Free Replacement Solution for Radiation Therapy Immobilization Devices Using Computer Numerical Control (CNC) -Milled Polystyrene Molds

Purpose

In radiation therapy (RT), if an immobilization device is lost or damaged, the patient may need to be brought back for resimulation, device fabrication, and treatment planning, causing additional imaging radiation exposure, inconvenience, cost, and delay. We describe a simulation-free method for replacing lost or damaged RT immobilization devices.

Methods and Materials

Replacement immobilization devices were fabricated using existing simulation scans as design templates by computer numerical control (CNC) milling of molds made from extruded polystyrene (XPS). XPS material attenuation and bolusing properties were evaluated, a standard workflow was established, and 12 patients were treated. Setup reproducibility was analyzed postfacto using Dice similarity coefficient (DSC) and mean distance to agreement (MDA) calculations comparing onboard treatment imaging with computed tomography (CT) simulations.

Results

Results showed that XPS foam material had less dosimetric impact (attenuation and bolusing) than materials used for our standard immobilization devices. The average direct cost to produce each replacement mold was $242.17, compared with over $2000 for standard resimulation. Hands-on time to manufacture was 86.3 minutes, whereas molds were delivered in as little as 4 hours and mostly within 24 hours, compared with a week or more required for standard resimulation. Each mold was optically scanned after production and was measured to be within 2-mm tolerance (pointwise displacement) of design input. All patients were successfully treated using the CNC-milled foam mold replacements, and pretreatment imaging verified satisfactory clinical setup reproduction for each case. The external body contours from the setup cone beam CT and the original CT simulation with matching superior-inferior extent were compared by calculating the DSC and MDA. DSC average was 0.966 (SD, 0.011), and MDA average was 2.694 mm (SD, 0.986).

Conclusions

CNC milling of XPS foam is a quicker and more convenient solution than traditional resimulation for replacing lost or damaged RT immobilization devices. Satisfactory patient immobilization, low dosimetric impact compared with standard immobilization devices, and strong correlation of onboard contours with CT simulations are shown. We share our clinical experience, workflow, and manufacturing guide to help other clinicians who may want to adopt this solution.

Analyzing the Relationship between Dose and Geometric Agreement Metrics for Auto-Contouring in Head and Neck Normal Tissues

This study aimed to determine the relationship between geometric and dosimetric agreement metrics in head and neck (H&N) cancer radiotherapy plans. A total 287 plans were retrospectively analyzed, comparing auto-contoured and clinically used contours using a Dice similarity coefficient (DSC), surface DSC (sDSC), and Hausdorff distance (HD). Organs-at-risk (OARs) with ≥200 cGy dose differences from the clinical contour in terms of Dmax (D0.01cc) and Dmean were further examined against proximity to the planning target volume (PTV). A secondary set of 91 plans from multiple institutions validated these findings. For 4995 contour pairs across 19 OARs, 90% had a DSC, sDSC, and HD of at least 0.75, 0.86, and less than 7.65 mm, respectively. Dosimetrically, the absolute difference between the two contour sets was <200 cGy for 95% of OARs in terms of Dmax and 96% in terms of Dmean. In total, 97% of OARs exhibiting significant dose differences between the clinically edited contour and auto-contour were within 2.5 cm PTV regardless of geometric agreement. There was an approximately linear trend between geometric agreement and identifying at least 200 cGy dose differences, with higher geometric agreement corresponding to a lower fraction of cases being identified. Analysis of the secondary dataset validated these findings. Geometric indices are approximate indicators of contour quality and identify contours exhibiting significant dosimetric discordance. For a small subset of OARs within 2.5 cm of the PTV, geometric agreement metrics can be misleading in terms of contour quality.

Gross failure rates and failure modes for a commercial AI-based auto-segmentation algorithm in head and neck cancer patients

PURPOSE

Artificial intelligence (AI) based commercial software can be used to automatically delineate organs at risk (OAR), with potential for efficiency savings in the radiotherapy treatment planning pathway, and reduction of inter- and intra-observer variability. There has been little research investigating gross failure rates and failure modes of such systems.

METHOD

50 head and neck (H&N) patient data sets with "gold standard" contours were compared to AI-generated contours to produce expected mean and standard deviation values for the Dice Similarity Coefficient (DSC), for four common H&N OARs (brainstem, mandible, left and right parotid). An AI-based commercial system was applied to 500 H&N patients. AI-generated contours were compared to manual contours, outlined by an expert human, and a gross failure was set at three standard deviations below the expected mean DSC. Failures were inspected to assess reason for failure of the AI-based system with failures relating to suboptimal manual contouring censored. True failures were classified into 4 sub-types (setup position, anatomy, image artefacts and unknown).

RESULTS

There were 24 true failures of the AI-based commercial software, a gross failure rate of 1.2%. Fifteen failures were due to patient anatomy, four were due to dental image artefacts, three were due to patient position and two were unknown. True failure rates by OAR were 0.4% (brainstem), 2.2% (mandible), 1.4% (left parotid) and 0.8% (right parotid).

CONCLUSION

True failures of the AI-based system were predominantly associated with a non-standard element within the CT scan. It is likely that these non-standard elements were the reason for the gross failure, and suggests that patient datasets used to train the AI model did not contain sufficient heterogeneity of data. Regardless of the reasons for failure, the true failure rate for the AI-based system in the H&N region for the OARs investigated was low (∼1%).

Evaluation of Deep Learning Clinical Target Volumes Auto-Contouring for Magnetic Resonance Imaging-Guided Online Adaptive Treatment of Rectal Cancer

Purpose

Segmentation of clinical target volumes (CTV) on medical images can be time-consuming and is prone to interobserver variation (IOV). This is a problem for online adaptive radiation therapy, where CTV segmentation must be performed every treatment fraction, leading to longer treatment times and logistic challenges. Deep learning (DL)-based auto-contouring has the potential to speed up CTV contouring, but its current clinical use is limited. One reason for this is that it can be time-consuming to verify the accuracy of CTV contours produced using auto-contouring, and there is a risk of bias being introduced. To be accepted by clinicians, auto-contouring must be trustworthy. Therefore, there is a need for a comprehensive commissioning framework when introducing DL-based auto-contouring in clinical practice. We present such a framework and apply it to an in-house developed DL model for auto-contouring of the CTV in rectal cancer patients treated with MRI-guided online adaptive radiation therapy.

Methods and Materials

The framework for evaluating DL-based auto-contouring consisted of 3 steps: (1) Quantitative evaluation of the model's performance and comparison with IOV; (2) Expert observations and corrections; and (3) Evaluation of the impact on expected volumetric target coverage. These steps were performed on independent data sets. The framework was applied to an in-house trained nnU-Net model, using the data of 44 rectal cancer patients treated at our institution.

Results

The framework established that the model's performance after expert corrections was comparable to IOV, and although the model introduced a bias, this had no relevant impact on clinical practice. Additionally, we found a substantial time gain without reducing quality as determined by volumetric target coverage.

Conclusions

Our framework provides a comprehensive evaluation of the performance and clinical usability of target auto-contouring models. Based on the results, we conclude that the model is eligible for clinical use.

Evaluation of multiple-vendor AI autocontouring solutions

BACKGROUND

Multiple artificial intelligence (AI)-based autocontouring solutions have become available, each promising high accuracy and time savings compared with manual contouring. Before implementing AI-driven autocontouring into clinical practice, three commercially available CT-based solutions were evaluated.

MATERIALS AND METHODS

The following solutions were evaluated in this work: MIM-ProtégéAI+ (MIM), Radformation-AutoContour (RAD), and Siemens-DirectORGANS (SIE). Sixteen organs were identified that could be contoured by all solutions. For each organ, ten patients that had manually generated contours approved by the treating physician (AP) were identified, totaling forty-seven different patients. CT scans in the supine position were acquired using a Siemens-SOMATOMgo 64-slice helical scanner and used to generate autocontours. Physician scoring of contour accuracy was performed by at least three physicians using a five-point Likert scale. Dice similarity coefficient (DSC), Hausdorff distance (HD) and mean distance to agreement (MDA) were calculated comparing AI contours to "ground truth" AP contours.

RESULTS

The average physician score ranged from 1.00, indicating that all physicians reviewed the contour as clinically acceptable with no modifications necessary, to 3.70, indicating changes are required and that the time taken to modify the structures would likely take as long or longer than manually generating the contour. When averaged across all sixteen structures, the AP contours had a physician score of 2.02, MIM 2.07, RAD 1.96 and SIE 1.99. DSC ranged from 0.37 to 0.98, with 41/48 (85.4%) contours having an average DSC ≥ 0.7. Average HD ranged from 2.9 to 43.3 mm. Average MDA ranged from 0.6 to 26.1 mm.

CONCLUSIONS

The results of our comparison demonstrate that each vendor's AI contouring solution exhibited capabilities similar to those of manual contouring. There were a small number of cases where unusual anatomy led to poor scores with one or more of the solutions. The consistency and comparable performance of all three vendors' solutions suggest that radiation oncology centers can confidently choose any of the evaluated solutions based on individual preferences, resource availability, and compatibility with their existing clinical workflows. Although AI-based contouring may result in high-quality contours for the majority of patients, a minority of patients require manual contouring and more in-depth physician review.

Order From Chaos: The Benefits of Standardized Nomenclature in Radiation Oncology

Although standardization has been shown to improve patient safety and improve the efficiency of workflows, implementation of standards can take considerable effort and requires the engagement of all clinical stakeholders. Engaging team members includes increasing awareness of the proposed benefit of the standard, a clear implementation plan, monitoring for improvements, and open communication to support successful implementation. The benefits of standardization often focus on large institutions to improve research endeavors, yet all clinics can benefit from standardization to increase quality and implement more efficient or automated workflow. The benefits of nomenclature standardization for all team members and institution sizes, including success stories, are discussed with practical implementation guides to facilitate the adoption of standardized nomenclature in radiation oncology.

Segmentation of 71 Anatomical Structures Necessary for the Evaluation of Guideline-Conforming Clinical Target Volumes in Head and Neck Cancers

The delineation of the clinical target volumes (CTVs) for radiation therapy is time-consuming, requires intensive training and shows high inter-observer variability. Supervised deep-learning methods depend heavily on consistent training data; thus, State-of-the-Art research focuses on making CTV labels more homogeneous and strictly bounding them to current standards. International consensus expert guidelines standardize CTV delineation by conditioning the extension of the clinical target volume on the surrounding anatomical structures. Training strategies that directly follow the construction rules given in the expert guidelines or the possibility of quantifying the conformance of manually drawn contours to the guidelines are still missing. Seventy-one anatomical structures that are relevant to CTV delineation in head- and neck-cancer patients, according to the expert guidelines, were segmented on 104 computed tomography scans, to assess the possibility of automating their segmentation by State-of-the-Art deep learning methods. All 71 anatomical structures were subdivided into three subsets of non-overlapping structures, and a 3D nnU-Net model with five-fold cross-validation was trained for each subset, to automatically segment the structures on planning computed tomography scans. We report the DICE, Hausdorff distance and surface DICE for 71 + 5 anatomical structures, for most of which no previous segmentation accuracies have been reported. For those structures for which prediction values have been reported, our segmentation accuracy matched or exceeded the reported values. The predictions from our models were always better than those predicted by the TotalSegmentator. The sDICE with 2 mm margin was larger than 80% for almost all the structures. Individual structures with decreased segmentation accuracy are analyzed and discussed with respect to their impact on the CTV delineation following the expert guidelines. No deviation is expected to affect the rule-based automation of the CTV delineation.

Clinical acceptability of automatically generated lymph node levels and structures of deglutition and mastication for head and neck radiation therapy

Background and Purpose

Auto-contouring of complex anatomy in computed tomography (CT) scans is a highly anticipated solution to many problems in radiotherapy. In this study, artificial intelligence (AI)-based auto-contouring models were clinically validated for lymph node levels and structures of swallowing and chewing in the head and neck.

Materials and Methods

CT scans of 145 head and neck radiotherapy patients were retrospectively curated. One cohort (n = 47) was used to analyze seven lymph node levels and the other (n = 98) used to analyze 17 swallowing and chewing structures. Separate nnUnet models were trained and validated using the separate cohorts. For the lymph node levels, preference and clinical acceptability of AI vs human contours were scored. For the swallowing and chewing structures, clinical acceptability was scored. Quantitative analyses of the test sets were performed for AI vs human contours for all structures using overlap and distance metrics.

Results

Median Dice Similarity Coefficient ranged from 0.77 to 0.89 for lymph node levels and 0.86 to 0.96 for chewing and swallowing structures. The AI contours were superior to or equally preferred to the manual contours at rates ranging from 75% to 91%; there was not a significant difference in clinical acceptability for nodal levels I-V for manual versus AI contours. Across all AI-generated lymph node level contours, 92% were rated as usable with stylistic to no edits. Of the 340 contours in the chewing and swallowing cohort, 4% required minor edits.

Conclusions

An accurate approach was developed to auto-contour lymph node levels and chewing and swallowing structures on CT images for patients with intact nodal anatomy. Only a small portion of test set auto-contours required minor edits.

Comparing multi-image and image augmentation strategies for deep learning-based prostate segmentation

During MR-Linac-based adaptive radiotherapy, multiple images are acquired per patient. These can be applied in training deep learning networks to reduce annotation efforts. This study examined the advantage of using multiple versus single images for prostate treatment segmentation. Findings indicate minimal improvement in DICE and Hausdorff 95% metrics with multiple images. Maximum difference was seen for the rectum in the low data regime, training with images from five patients. Utilizing a 2D U-net resulted in DICE values of 0.80/0.83 when including 1/5 images per patient, respectively. Including more patients in training reduced the difference. Standard augmentation methods remained more effective.

Consistency in contouring of organs at risk by artificial intelligence vs oncologists in head and neck cancer patients

BACKGROUND

In the Danish Head and Neck Cancer Group (DAHANCA) 35 trial, patients are selected for proton treatment based on simulated reductions of Normal Tissue Complication Probability (NTCP) for proton compared to photon treatment at the referring departments. After inclusion in the trial, immobilization, scanning, contouring and planning are repeated at the national proton centre. The new contours could result in reduced expected NTCP gain of the proton plan, resulting in a loss of validity in the selection process. The present study evaluates if contour consistency can be improved by having access to AI (Artificial Intelligence) based contours.

MATERIALS AND METHODS

The 63 patients in the DAHANCA 35 pilot trial had a CT from the local DAHANCA centre and one from the proton centre. A nationally validated convolutional neural network, based on nnU-Net, was used to contour OARs on both scans for each patient. Using deformable image registration, local AI and oncologist contours were transferred to the proton centre scans for comparison. Consistency was calculated with the Dice Similarity Coefficient (DSC) and Mean Surface Distance (MSD), comparing contours from AI to AI and oncologist to oncologist, respectively. Two NTCP models were applied to calculate NTCP for xerostomia and dysphagia.

RESULTS

The AI contours showed significantly better consistency than the contours by oncologists. The median and interquartile range of DSC was 0.85 [0.78 - 0.90] and 0.68 [0.51 - 0.80] for AI and oncologist contours, respectively. The median and interquartile range of MSD was 0.9 mm [0.7 - 1.1] mm and 1.9 mm [1.5 - 2.6] mm for AI and oncologist contours, respectively. There was no significant difference in Δ NTCP.

CONCLUSIONS

The study showed that OAR contours made by the AI algorithm were more consistent than those made by oncologists. No significant impact on the Δ NTCP calculations could be discerned.

Dosimetrically triggered adaptive radiotherapy for head and neck cancer: Considerations for the implementation of clinical protocols

PURPOSE

Defining dosimetric rules to automatically detect patients requiring adaptive radiotherapy (ART) is not straightforward, and most centres perform ad-hoc ART with no specific protocol. This study aims to propose and analyse different steps to design a protocol for dosimetrically triggered ART of head and neck (H&N) cancer. As a proof-of-concept, the designed protocol was applied to patients treated in TomoTherapy units, using their available software for daily MVCT image and dose accumulation.

METHODS

An initial protocol was designed by a multidisciplinary team, with a set of flagging criteria based only on dose-volume metrics, including two action levels: (1) surveillance (orange flag), and (2) immediate verification (red flag). This protocol was adapted to the clinical needs following an iterative process. First, the protocol was applied to 38 H&N patients with daily imaging. Automatic software generated the daily contours, recomputed the daily dose and flagged the dosimetric differences with respect to the planning dose. Second, these results were compared, by a sensitivity/specificity test, to the answers of a physician. Third, the physician, supported by the multidisciplinary team, performed a self-analysis of the provided answers and translated them into mathematical rules in order to upgrade the protocol. The upgraded protocol was applied to different definitions of the target volume (i.e. deformed CTV + 0, 2 and 4 mm), in order to quantify how the number of flags decreases when reducing the CTV-to-PTV margin.

RESULTS

The sensitivity of the initial protocol was very low, specifically for the orange flags. The best values were 0.84 for red and 0.15 for orange flags. After the review and upgrade process, the sensitivity of the upgraded protocol increased to 0.96 for red and 0.84 for orange flags. The number of patients flagged per week with the final (upgraded) protocol decreased in median by 26% and 18% for red and orange flags, respectively, when reducing the CTV-to-PTV margin from 4 to 2 mm. This resulted in only one patient flagged at the last fraction for both red and orange flags.

CONCLUSION

Our results demonstrate the value of iterative protocol design with retrospective data, and shows the feasibility of automatically-triggered ART using simple dosimetric rules to mimic the physician's decisions. Using a proper target volume definition is important and influences the flagging rate, particularly when decreasing the CTV-to-PTV margin.

Extensive clinical testing of Deep Learning Segmentation models for thorax and breast cancer radiotherapy planning

BACKGROUND

The performance of deep learning segmentation (DLS) models for automatic organ extraction from CT images in the thorax and breast regions was investigated. Furthermore, the readiness and feasibility of integrating DLS into clinical practice were addressed by measuring the potential time savings and dosimetric impact.

MATERIAL AND METHODS

Thirty patients referred to radiotherapy for breast cancer were prospectively included. A total of 23 clinically relevant left- and right-sided organs were contoured manually on CT images according to ESTRO guidelines. Next, auto-segmentation was executed, and the geometric agreement between the auto-segmented and manually contoured organs was qualitatively assessed applying a scale in the range [0-not acceptable, 3-no corrections]. A quantitative validation was carried out by calculating Dice coefficients (DSC) and the 95% percentile of Hausdorff distances (HD95). The dosimetric impact of optimizing the treatment plans on the uncorrected DLS contours, was investigated from a dose coverage analysis using DVH values of the manually delineated contours as references.

RESULTS

The qualitative analysis showed that 93% of the DLS generated OAR contours did not need corrections, except for the heart where 67% of the contours needed corrections. The majority of DLS generated CTVs needed corrections, whereas a minority were deemed not acceptable. Still, using the DLS-model for CTV and heart delineation is on average 14 minutes faster. An average DSC=0.91 and H95=9.8 mm were found for the left and right breasts, respectively. Likewise, and average DSC in the range [0.66, 0.76]mm and HD95 in the range [7.04, 12.05]mm were found for the lymph nodes.

CONCLUSION

The validation showed that the DLS generated OAR contours can be used clinically. Corrections were required to most of the DLS generated CTVs, and therefore warrants more attention before possibly implementing the DLS models clinically.

Impact of imperfection in medical imaging data on deep learning-based segmentation performance: An experimental study using synthesized data

BACKGROUND

Clinical data used to train deep learning models are often not clean data. They can contain imperfections in both the imaging data and the corresponding segmentations.

PURPOSE

This study investigates the influence of data imperfections on the performance of deep learning models for parotid gland segmentation. This was done in a controlled manner by using synthesized data. The insights this study provides may be used to make deep learning models better and more reliable.

METHODS

The data were synthesized by using the clinical segmentations, creating a pseudo ground-truth in the process. Three kinds of imperfections were simulated: incorrect segmentations, low image contrast, and artifacts in the imaging data. The severity of each imperfection was varied in five levels. Models resulting from training sets from each of the five levels were cross-evaluated with test sets from each of the five levels.

RESULTS

Using synthesized data led to almost perfect parotid gland segmentation when no error was added. Lowering the quality of the parotid gland segmentations used for training substantially lowered the model performance. Additionally, lowering the image quality of the training data by decreasing the contrast or introducing artifacts made the resulting models more robust to data containing those respective kinds of data imperfection.

CONCLUSION

This study demonstrated the importance of good-quality segmentations for deep learning training and it shows that using low-quality imaging data for training can enhance the robustness of the resulting models.

Deep learning-based classification of organs at risk and delineation guideline in pelvic cancer radiation therapy

Deep learning (DL) models for radiation therapy (RT) image segmentation require accurately annotated training data. Multiple organ delineation guidelines exist; however, information on the used guideline is not provided with the delineation. Extraction of training data with coherent guidelines can therefore be challenging. We present a supervised classification method for pelvis structure delineations where bowel cavity, femoral heads, bladder, and rectum data, with two guidelines, were classified. The impact on DL-based segmentation quality using mixed guideline training data was also demonstrated. Bowel cavity was manually delineated on CT images for anal cancer patients (n = 170) according to guidelines Devisetty and RTOG. The DL segmentation quality from using training data with coherent or mixed guidelines was investigated. A supervised 3D squeeze-and-excite SENet-154 model was trained to classify two bowel cavity delineation guidelines. In addition, a pelvis CT dataset with manual delineations from prostate cancer patients (n = 1854) was used where data with an alternative guideline for femoral heads, rectum, and bladder were generated using commercial software. The model was evaluated on internal (n = 200) and external test data (n = 99). By using mixed, compared to coherent, delineation guideline training data mean DICE score decreased 3% units, mean Hausdorff distance (95%) increased 5 mm and mean surface distance (MSD) increased 1 mm. The classification of bowel cavity test data achieved 99.8% unweighted classification accuracy, 99.9% macro average precision, 97.2% macro average recall, and 98.5% macro average F1. Corresponding metrics for the pelvis internal test data were all 99% or above and for the external pelvis test data they were 96.3%, 96.6%, 93.3%, and 94.6%. Impaired segmentation performance was observed for training data with mixed guidelines. The DL delineation classification models achieved excellent results on internal and external test data. This can facilitate automated guideline-specific data extraction while avoiding the need for consistent and correct structure labels.

Dosimetric comparison of autocontouring techniques for online adaptive proton therapy

Objective.Anatomical and daily set-up uncertainties impede high precision delivery of proton therapy. With online adaptation, the daily plan is reoptimized on an image taken shortly before the treatment, reducing these uncertainties and, hence, allowing a more accurate delivery. This reoptimization requires target and organs-at-risk (OAR) contours on the daily image, which need to be delineated automatically since manual contouring is too slow. Whereas multiple methods for autocontouring exist, none of them are fully accurate, which affects the daily dose. This work aims to quantify the magnitude of this dosimetric effect for four contouring techniques.Approach.Plans reoptimized on automatic contours are compared with plans reoptimized on manual contours. The methods include rigid and deformable registration (DIR), deep-learning based segmentation and patient-specific segmentation.Main results.It was found that independently of the contouring method, the dosimetric influence of usingautomaticOARcontoursis small (<5% prescribed dose in most cases), with DIR yielding the best results. Contrarily, the dosimetric effect of using theautomatic target contourwas larger (>5% prescribed dose in most cases), indicating that manual verification of that contour remains necessary. However, when compared to non-adaptive therapy, the dose differences caused by automatically contouring the target were small and target coverage was improved, especially for DIR.Significance.The results show that manual adjustment of OARs is rarely necessary and that several autocontouring techniques are directly usable. Contrarily, manual adjustment of the target is important. This allows prioritizing tasks during time-critical online adaptive proton therapy and therefore supports its further clinical implementation.

A clinical evaluation of the performance of five commercial artificial intelligence contouring systems for radiotherapy

Purpose/objectives

Auto-segmentation with artificial intelligence (AI) offers an opportunity to reduce inter- and intra-observer variability in contouring, to improve the quality of contours, as well as to reduce the time taken to conduct this manual task. In this work we benchmark the AI auto-segmentation contours produced by five commercial vendors against a common dataset.

Methods and materials

The organ at risk (OAR) contours generated by five commercial AI auto-segmentation solutions (Mirada (Mir), MVision (MV), Radformation (Rad), RayStation (Ray) and TheraPanacea (Ther)) were compared to manually-drawn expert contours from 20 breast, 20 head and neck, 20 lung and 20 prostate patients. Comparisons were made using geometric similarity metrics including volumetric and surface Dice similarity coefficient (vDSC and sDSC), Hausdorff distance (HD) and Added Path Length (APL). To assess the time saved, the time taken to manually draw the expert contours, as well as the time to correct the AI contours, were recorded.

Results

There are differences in the number of CT contours offered by each AI auto-segmentation solution at the time of the study (Mir 99; MV 143; Rad 83; Ray 67; Ther 86), with all offering contours of some lymph node levels as well as OARs. Averaged across all structures, the median vDSCs were good for all systems and compared favorably with existing literature: Mir 0.82; MV 0.88; Rad 0.86; Ray 0.87; Ther 0.88. All systems offer substantial time savings, ranging between: breast 14-20 mins; head and neck 74-93 mins; lung 20-26 mins; prostate 35-42 mins. The time saved, averaged across all structures, was similar for all systems: Mir 39.8 mins; MV 43.6 mins; Rad 36.6 min; Ray 43.2 mins; Ther 45.2 mins.

Conclusions

All five commercial AI auto-segmentation solutions evaluated in this work offer high quality contours in significantly reduced time compared to manual contouring, and could be used to render the radiotherapy workflow more efficient and standardized.

Deep-learning magnetic resonance imaging-based automatic segmentation for organs-at-risk in the brain: Accuracy and impact on dose distribution

Background and purpose

Normal tissue sparing in radiotherapy relies on proper delineation. While manual contouring is time consuming and subject to inter-observer variability, auto-contouring could optimize workflows and harmonize practice. We assessed the accuracy of a commercial, deep-learning, MRI-based tool for brain organs-at-risk delineation.

Materials and methods

Thirty adult brain tumor patients were retrospectively manually recontoured. Two additional structure sets were obtained: AI (artificial intelligence) and AIedit (manually corrected auto-contours). For 15 selected cases, identical plans were optimized for each structure set. We used Dice Similarity Coefficient (DSC) and mean surface-distance (MSD) for geometric comparison and gamma analysis and dose-volume-histogram comparison for dose metrics evaluation. Wilcoxon signed-ranks test was used for paired data, Spearman coefficient(ρ) for correlations and Bland-Altman plots to assess level of agreement.

Results

Auto-contouring was significantly faster than manual (1.1/20 min, p < 0.01). Median DSC and MSD were 0.7/0.9 mm for AI and 0.8/0.5 mm for AIedit. DSC was significantly correlated with structure size (ρ = 0.76, p < 0.01), with higher DSC for large structures. Median gamma pass rate was 74% (71-81%) for Plan_AI and 82% (75-86%) for Plan_AIedit, with no correlation with DSC or MSD. Differences between Dmean_AI and Dmean_Ref were ≤ 0.2 Gy (p < 0.05). The dose difference was moderately correlated with DSC. Bland Altman plot showed minimal discrepancy (0.1/0) between AI and reference Dmean/Dmax.

Conclusions

The AI-model showed good accuracy for large structures, but developments are required for smaller ones. Auto-segmentation was significantly faster, with minor differences in dose distribution caused by geometric variations.

Enhancing Radiotherapy Workflow for Head and Neck Cancer with Artificial Intelligence: A Systematic Review

BACKGROUND

Head and neck cancer (HNC) is characterized by complex-shaped tumors and numerous organs at risk (OARs), inducing challenging radiotherapy (RT) planning, optimization, and delivery. In this review, we provided a thorough description of the applications of artificial intelligence (AI) tools in the HNC RT process.

METHODS

The PubMed database was queried, and a total of 168 articles (2016-2022) were screened by a group of experts in radiation oncology. The group selected 62 articles, which were subdivided into three categories, representing the whole RT workflow: (i) target and OAR contouring, (ii) planning, and (iii) delivery.

RESULTS

The majority of the selected studies focused on the OARs segmentation process. Overall, the performance of AI models was evaluated using standard metrics, while limited research was found on how the introduction of AI could impact clinical outcomes. Additionally, papers usually lacked information about the confidence level associated with the predictions made by the AI models.

CONCLUSIONS

AI represents a promising tool to automate the RT workflow for the complex field of HNC treatment. To ensure that the development of AI technologies in RT is effectively aligned with clinical needs, we suggest conducting future studies within interdisciplinary groups, including clinicians and computer scientists.

Validation of clinical acceptability of deep-learning-based automated segmentation of organs-at-risk for head-and-neck radiotherapy treatment planning

Introduction

Organ-at-risk segmentation for head and neck cancer radiation therapy is a complex and time-consuming process (requiring up to 42 individual structure, and may delay start of treatment or even limit access to function-preserving care. Feasibility of using a deep learning (DL) based autosegmentation model to reduce contouring time without compromising contour accuracy is assessed through a blinded randomized trial of radiation oncologists (ROs) using retrospective, de-identified patient data.

Methods

Two head and neck expert ROs used dedicated time to create gold standard (GS) contours on computed tomography (CT) images. 445 CTs were used to train a custom 3D U-Net DL model covering 42 organs-at-risk, with an additional 20 CTs were held out for the randomized trial. For each held-out patient dataset, one of the eight participant ROs was randomly allocated to review and revise the contours produced by the DL model, while another reviewed contours produced by a medical dosimetry assistant (MDA), both blinded to their origin. Time required for MDAs and ROs to contour was recorded, and the unrevised DL contours, as well as the RO-revised contours by the MDAs and DL model were compared to the GS for that patient.

Results

Mean time for initial MDA contouring was 2.3 hours (range 1.6-3.8 hours) and RO-revision took 1.1 hours (range, 0.4-4.4 hours), compared to 0.7 hours (range 0.1-2.0 hours) for the RO-revisions to DL contours. Total time reduced by 76% (95%-Confidence Interval: 65%-88%) and RO-revision time reduced by 35% (95%-CI,-39%-91%). All geometric and dosimetric metrics computed, agreement with GS was equivalent or significantly greater (p<0.05) for RO-revised DL contours compared to the RO-revised MDA contours, including volumetric Dice similarity coefficient (VDSC), surface DSC, added path length, and the 95%-Hausdorff distance. 32 OARs (76%) had mean VDSC greater than 0.8 for the RO-revised DL contours, compared to 20 (48%) for RO-revised MDA contours, and 34 (81%) for the unrevised DL OARs.

Conclusion

DL autosegmentation demonstrated significant time-savings for organ-at-risk contouring while improving agreement with the institutional GS, indicating comparable accuracy of DL model. Integration into the clinical practice with a prospective evaluation is currently underway.

A simple single-cycle interactive strategy to improve deep learning-based segmentation of organs-at-risk in head-and-neck cancer

Background and purpose

Interactive segmentation seeks to incorporate human knowledge into segmentation models and thereby reducing the total amount of editing of auto-segmentations. By performing only interactions which provide new information, segmentation performance may increase cost-effectively. The aim of this study was to develop, evaluate and test feasibility of a deep learning-based single-cycle interactive segmentation model with the input being computer tomography (CT) and a small amount of information rich contours.

Methods and Materials

A single-cycle interactive segmentation model, which took CT and the most cranial and caudal contour slices for each of 16 organs-at-risk for head-and-neck cancer as input, was developed. A CT-only model served as control. The models were evaluated with Dice similarity coefficient, Hausdorff Distance 95th percentile and average symmetric surface distance. A subset of 8 organs-at-risk were selected for a feasibility test. In this, a designated radiation oncologist used both single-cycle interactive segmentation and atlas-based auto-contouring for three cases. Contouring time and added path length were recorded.

Results

The medians of Dice coefficients increased with single-cycle interactive segmentation in the range of 0.004 (Brain)-0.90 (EyeBack_merged) when compared to CT-only. In the feasibility test, contouring time and added path length were reduced for all three cases as compared to editing atlas-based auto-segmentations.

Conclusion

Single-cycle interactive segmentation improved segmentation metrics when compared to the CT-only model and was clinically feasible from a technical and usability point of view. The study suggests that it may be cost-effective to add a small amount of contouring input to deep learning-based segmentation models.

Automated Contouring and Planning in Radiation Therapy: What Is 'Clinically Acceptable'?

Developers and users of artificial-intelligence-based tools for automatic contouring and treatment planning in radiotherapy are expected to assess clinical acceptability of these tools. However, what is 'clinical acceptability'? Quantitative and qualitative approaches have been used to assess this ill-defined concept, all of which have advantages and disadvantages or limitations. The approach chosen may depend on the goal of the study as well as on available resources. In this paper, we discuss various aspects of 'clinical acceptability' and how they can move us toward a standard for defining clinical acceptability of new autocontouring and planning tools.

Automatic gross tumor segmentation of canine head and neck cancer using deep learning and cross-species transfer learning

Background

Radiotherapy (RT) is increasingly being used on dogs with spontaneous head and neck cancer (HNC), which account for a large percentage of veterinary patients treated with RT. Accurate definition of the gross tumor volume (GTV) is a vital part of RT planning, ensuring adequate dose coverage of the tumor while limiting the radiation dose to surrounding tissues. Currently the GTV is contoured manually in medical images, which is a time-consuming and challenging task.

Purpose

The purpose of this study was to evaluate the applicability of deep learning-based automatic segmentation of the GTV in canine patients with HNC.

Materials and methods

Contrast-enhanced computed tomography (CT) images and corresponding manual GTV contours of 36 canine HNC patients and 197 human HNC patients were included. A 3D U-Net convolutional neural network (CNN) was trained to automatically segment the GTV in canine patients using two main approaches: (i) training models from scratch based solely on canine CT images, and (ii) using cross-species transfer learning where models were pretrained on CT images of human patients and then fine-tuned on CT images of canine patients. For the canine patients, automatic segmentations were assessed using the Dice similarity coefficient (Dice), the positive predictive value, the true positive rate, and surface distance metrics, calculated from a four-fold cross-validation strategy where each fold was used as a validation set and test set once in independent model runs.

Results

CNN models trained from scratch on canine data or by using transfer learning obtained mean test set Dice scores of 0.55 and 0.52, respectively, indicating acceptable auto-segmentations, similar to the mean Dice performances reported for CT-based automatic segmentation in human HNC studies. Automatic segmentation of nasal cavity tumors appeared particularly promising, resulting in mean test set Dice scores of 0.69 for both approaches.

Conclusion

In conclusion, deep learning-based automatic segmentation of the GTV using CNN models based on canine data only or a cross-species transfer learning approach shows promise for future application in RT of canine HNC patients.

A clinical and time savings evaluation of a deep learning automatic contouring algorithm

Automatic contouring algorithms may streamline clinical workflows by reducing normal organ-at-risk (OAR) contouring time. Here we report the first comprehensive quantitative and qualitative evaluation, along with time savings assessment for a prototype deep learning segmentation algorithm from Siemens Healthineers. The accuracy of contours generated by the prototype were evaluated quantitatively using the Sorensen-Dice coefficient (Dice), Jaccard index (JC), and Hausdorff distance (Haus). Normal pelvic and head and neck OAR contours were evaluated retrospectively comparing the automatic and manual clinical contours in 100 patient cases. Contouring performance outliers were investigated. To quantify the time savings, a certified medical dosimetrist manually contoured de novo and, separately, edited the generated OARs for 10 head and neck and 10 pelvic patients. The automatic, edited, and manually generated contours were visually evaluated and scored by a practicing radiation oncologist on a scale of 1-4, where a higher score indicated better performance. The quantitative comparison revealed high (> 0.8) Dice and JC performance for relatively large organs such as the lungs, brain, femurs, and kidneys. Smaller elongated structures that had relatively low Dice and JC values tended to have low Hausdorff distances. Poor performing outlier cases revealed common anatomical inconsistencies including overestimation of the bladder and incorrect superior-inferior truncation of the spinal cord and femur contours. In all cases, editing contours was faster than manual contouring with an average time saving of 43.4% or 11.8 minutes per patient. The physician scored 240 structures with > 95% of structures receiving a score of 3 or 4. Of the structures reviewed, only 11 structures needed major revision or to be redone entirely. Our results indicate the evaluated auto-contouring solution has the potential to reduce clinical contouring time. The algorithm's performance is promising, but human review and some editing is required prior to clinical use.

Comparison of atlas-based and deep learning methods for organs at risk delineation on head-and-neck CT images using an automated treatment planning system

BACKGROUND AND PURPOSE

To investigate the performance of head-and-neck (HN) organs-at-risk (OAR) automatic segmentation (AS) using four atlas-based (ABAS) and two deep learning (DL) solutions.

MATERIAL AND METHODS

All patients underwent iodine contrast-enhanced planning CT. Fourteen OAR were manually delineated. DL.1 and DL.2 solutions were trained with 63 mono-centric patients and > 1000 multi-centric patients, respectively. Ten and 15 patients with varied anatomies were selected for the atlas library and for testing, respectively. The evaluation was based on geometric indices (DICE coefficient and 95th percentile-Hausdorff Distance (HD_95%)), time needed for manual corrections and clinical dosimetric endpoints obtained using automated treatment planning.

RESULTS

Both DICE and HD_95% results indicated that DL algorithms generally performed better compared with ABAS algorithms for automatic segmentation of HN OAR. However, the hybrid-ABAS (ABAS.3) algorithm sometimes provided the highest agreement to the reference contours compared with the 2 DL. Compared with DL.2 and ABAS.3, DL.1 contours were the fastest to correct. For the 3 solutions, the differences in dose distributions obtained using AS contours and AS + manually corrected contours were not statistically significant. High dose differences could be observed when OAR contours were at short distances to the targets. However, this was not always interrelated.

CONCLUSION

DL methods generally showed higher delineation accuracy compared with ABAS methods for AS segmentation of HN OAR. Most ABAS contours had high conformity to the reference but were more time consuming than DL algorithms, especially when considering the computing time and the time spent on manual corrections.

Impact of random outliers in auto-segmented targets on radiotherapy treatment plans for glioblastoma

AIMS

To save time and have more consistent contours, fully automatic segmentation of targets and organs at risk (OAR) is a valuable asset in radiotherapy. Though current deep learning (DL) based models are on par with manual contouring, they are not perfect and typical errors, as false positives, occur frequently and unpredictably. While it is possible to solve this for OARs, it is far from straightforward for target structures. In order to tackle this problem, in this study, we analyzed the occurrence and the possible dose effects of automated delineation outliers.

METHODS

First, a set of controlled experiments on synthetically generated outliers on the CT of a glioblastoma (GBM) patient was performed. We analyzed the dosimetric impact on outliers with different location, shape, absolute size and relative size to the main target, resulting in 61 simulated scenarios. Second, multiple segmentation models where trained on a U-Net network based on 80 training sets consisting of GBM cases with annotated gross tumor volume (GTV) and edema structures. On 20 test cases, 5 different trained models and a majority voting method were used to predict the GTV and edema. The amount of outliers on the predictions were determined, as well as their size and distance from the actual target.

RESULTS

We found that plans containing outliers result in an increased dose to healthy brain tissue. The extent of the dose effect is dependent on the relative size, location and the distance to the main targets and involved OARs. Generally, the larger the absolute outlier volume and the distance to the target the higher the potential dose effect. For 120 predicted GTV and edema structures, we found 1887 outliers. After construction of the planning treatment volume (PTV), 137 outliers remained with a mean distance to the target of 38.5 ± 5.0 mm and a mean size of 1010.8 ± 95.6 mm3. We also found that majority voting of DL results is capable to reduce outliers.

CONCLUSIONS

This study shows that there is a severe risk of false positive outliers in current DL predictions of target structures. Additionally, these errors will have an evident detrimental impact on the dose and therefore could affect treatment outcome.

Independent validation of a dysphagia dose response model for the selection of head and neck cancer patients to proton therapy

Background and purpose

The model based approach involves the use of normal tissue complication models for selection of head and neck cancer patients to proton therapy. Our goal was to validate the clinical utility of the related dysphagia model using an independent patient cohort.

Materials and Methods

A dataset of 277 head and neck cancer (pharynx and larynx) patients treated with (chemo)radiotherapy between 2019 and 2021 was acquired. For the evaluation of the model discrimination we used statistical metrics such as the sensitivity, specificity and the area under the receiver operating characteristic curve. After the validation we evaluated if the dysphagia model can be improved using the closed testing procedure, the Brier and the Hosmer-Lemeshow score.

Results

The performance of the original normal tissue complication probability model for dysphagia grade II-IV at 6 months was good (AUC = 0.80). According to the graphical calibration assessment, the original model showed underestimated dysphagia risk predictions. The closed testing procedure indicated that the model had to be updated and selected a revised model with new predictor coefficients as an optimal model. The revised model had also satisfactory discrimination (AUC = 0.83) with improved calibration.

Conclusion

The validation of the normal tissue complication probability model for grade II-IV dysphagia was successful in our independent validation cohort. However, the closed testing procedure indicated that the model should be updated with new coefficients.

Can Generative Adversarial Networks help to overcome the limited data problem in segmentation?

PURPOSE

For image translational tasks, the application of deep learning methods showed that Generative Adversarial Network (GAN) architectures outperform the traditional U-Net networks, when using the same training data size. This study investigates whether this performance boost can also be expected for segmentation tasks with small training dataset size.

MATERIALS/METHODS

Two models were trained on varying training dataset sizes ranging from 1-100 patients: a) U-Net and b) U-Net with patch discriminator (conditional GAN). The performance of both models to segment the male pelvis on CT-data was evaluated (Dice similarity coefficient, Hausdorff) with respect to training data size.

RESULTS

No significant differences were observed between the U-Net and cGAN when the models were trained with the same training sizes up to 100 patients. The training dataset size had a significant impact on the models' performances, with vast improvements when increasing dataset sizes from 1 to 20 patients.

CONCLUSION

When introducing GANs for the segmentation task no significant performance boost was observed in our experiments, even in segmentation models developed on small datasets.

Impact of Denoising on Deep-Learning-Based Automatic Segmentation Framework for Breast Cancer Radiotherapy Planning

Objective: This study aimed to investigate the segmentation accuracy of organs at risk (OARs) when denoised computed tomography (CT) images are used as input data for a deep-learning-based auto-segmentation framework. Methods: We used non-contrast enhanced planning CT scans from 40 patients with breast cancer. The heart, lungs, esophagus, spinal cord, and liver were manually delineated by two experienced radiation oncologists in a double-blind manner. The denoised CT images were used as input data for the AccuContourTM segmentation software to increase the signal difference between structures of interest and unwanted noise in non-contrast CT. The accuracy of the segmentation was assessed using the Dice similarity coefficient (DSC), and the results were compared with those of conventional deep-learning-based auto-segmentation without denoising. Results: The average DSC outcomes were higher than 0.80 for all OARs except for the esophagus. AccuContourTM-based and denoising-based auto-segmentation demonstrated comparable performance for the lungs and spinal cord but showed limited performance for the esophagus. Denoising-based auto-segmentation for the liver was minimal but had statistically significantly better DSC than AccuContourTM-based auto-segmentation (p < 0.05). Conclusions: Denoising-based auto-segmentation demonstrated satisfactory performance in automatic liver segmentation from non-contrast enhanced CT scans. Further external validation studies with larger cohorts are needed to verify the usefulness of denoising-based auto-segmentation.

Development and evaluation of an automated EPTN-consensus based organ at risk atlas in the brain on MRI

BACKGROUND AND PURPOSE

During radiotherapy treatment planning, avoidance of organs at risk (OARs) is important. An international consensus-based delineation guideline was recently published with 34 OARs in the brain. We developed an MR-based OAR autosegmentation atlas and evaluated its performance compared to manual delineation.

MATERIALS AND METHODS

Anonymized cerebral T1-weighted MR scans (voxel size 0.9x0.9x0.9mm 3) were available. OARs were manually delineated according to international consensus. Fifty MR scans were used to develop the autosegmentation atlas in a commercially available treatment planning system (Raystation®). The performance of this atlas was tested on another 40 MR scans by automatically delineating 34 OARs, as defined by the 2018 EPTN consensus. Spatial overlap between manual and automated delineations was determined by calculating the Dice similarity coefficient (DSC). Two radiation oncologists determined the quality of each automatically delineated OAR. The time needed to delineate all OARs manually or to adjust automatically delineated OARs was determined.

RESULTS

DSC was ≥0.75 in 31 (91%) out of 34 automated OAR delineations. Delineations were rated by radiation oncologists as excellent or good in 29 (85%) out 34 OAR delineations, while 4 were rated fair (12%) and 1 was rated poor (3%). Interobserver agreement between the radiation oncologists ranged from 77-100% per OAR. The time to manually delineate all OARs was 88.5 minutes, while the time needed to adjust automatically delineated OARs was 15.8 minutes.

CONCLUSION

Autosegmentation of OARs enables high-quality contouring within a limited time. Accurate OAR delineation helps to define OAR constraints to mitigate serious complications and helps with the development of NTCP models.

Towards a safe and efficient clinical implementation of machine learning in radiation oncology by exploring model interpretability, explainability and data-model dependency

The interest in machine learning (ML) has grown tremendously in recent years, partly due to the performance leap that occurred with new techniques of deep learning, convolutional neural networks for images, increased computational power, and wider availability of large datasets. Most fields of medicine follow that popular trend and, notably, radiation oncology is one of those that are at the forefront, with already a long tradition in using digital images and fully computerized workflows. ML models are driven by data, and in contrast with many statistical or physical models, they can be very large and complex, with countless generic parameters. This inevitably raises two questions, namely, the tight dependence between the models and the datasets that feed them, and the interpretability of the models, which scales with its complexity. Any problems in the data used to train the model will be later reflected in their performance. This, together with the low interpretability of ML models, makes their implementation into the clinical workflow particularly difficult. Building tools for risk assessment and quality assurance of ML models must involve then two main points: interpretability and data-model dependency. After a joint introduction of both radiation oncology and ML, this paper reviews the main risks and current solutions when applying the latter to workflows in the former. Risks associated with data and models, as well as their interaction, are detailed. Next, the core concepts of interpretability, explainability, and data-model dependency are formally defined and illustrated with examples. Afterwards, a broad discussion goes through key applications of ML in workflows of radiation oncology as well as vendors' perspectives for the clinical implementation of ML.

Clinical Evaluation of a Deep Learning Model for Segmentation of Target Volumes in Breast Cancer Radiotherapy

PURPOSE/OBJECTIVE(S)

Precise segmentation of clinical target volumes (CTV) in breast cancer is indispensable for state-of-the art radiotherapy. Despite international guidelines, significant intra- and interobserver variability exists, negatively impacting treatment outcomes. The aim of this study is to evaluate the performance and efficiency of segmentation of CTVs in planning CT images of breast cancer patients using a 3D convolutional neural network (CNN) compared to the manual process.

MATERIALS/METHODS

An expert radiation oncologist (RO) segmented all CTVs separately according to international guidelines in 150 breast cancer patients. This data was used to create, train and validate a 3D CNN. The network's performance was additionally evaluated in a test set of 20 patients. Primary endpoints are quantitative and qualitative analysis of the segmentation data generated by the CNN for each level specifically as well as for the total PTV to be irradiated. The secondary endpoint is the evaluation of time efficiency.

RESULTS

In the test set, segmentation performance was best for the contralateral breast and the breast CTV and worst for Rotter's space and the internal mammary nodal (IMN) level. Analysis of impact on PTV resulted in non-significant over-segmentation of the primary PTV and significant under-segmentation of the nodal PTV, resulting in slight variations of overlap with OARs. Guideline consistency improved from 77.14% to 90.71% in favor of CNN segmentation while saving on average 24 minutes per patient with a median time of 35 minutes for pure manual segmentation.

CONCLUSION

3D CNN based delineation for breast cancer radiotherapy is feasible and performant, as scored by quantitative and qualitative metrics.

Artificial Intelligence Radiotherapy Planning: Automatic Segmentation of Human Organs in CT Images Based on a Modified Convolutional Neural Network

Objective:

Precise segmentation of human organs and anatomic structures (especially organs at risk, OARs) is the basis and prerequisite for the treatment planning of radiation therapy. In order to ensure rapid and accurate design of radiotherapy treatment planning, an automatic organ segmentation technique was investigated based on deep learning convolutional neural network.

Method

A deep learning convolutional neural network (CNN) algorithm called BCDU-Net has been modified and developed further by us. Twenty two thousand CT images and the corresponding organ contours of 17 types delineated manually by experienced physicians from 329 patients were used to train and validate the algorithm. The CT images randomly selected were employed to test the modified BCDU-Net algorithm. The weight parameters of the algorithm model were acquired from the training of the convolutional neural network.

Result

The average Dice similarity coefficient (DSC) of the automatic segmentation and manual segmentation of the human organs of 17 types reached 0.8376, and the best coefficient reached up to 0.9676. It took 1.5–2 s and about 1 h to automatically segment the contours of an organ in an image of the CT dataset for a patient and the 17 organs for the CT dataset with the method developed by us, respectively.

Conclusion

The modified deep neural network algorithm could be used to automatically segment human organs of 17 types quickly and accurately. The accuracy and speed of the method meet the requirements of its application in radiotherapy.

Real-world analysis of manual editing of deep learning contouring in the thorax region

Background and purpose

User-adjustments after deep-learning (DL) contouring in radiotherapy were evaluated to get insight in real-world editing during clinical practice. This study assessed the amount, type and spatial regions of editing of auto-contouring for organs-at-risk (OARs) in routine clinical workflow for patients in the thorax region.

Materials and methods

A total of 350 lung cancer and 362 breast cancer patients, contoured between March 2020 and March 2021 using a commercial DL-contouring method followed by manual adjustments were retrospectively analyzed. Subsampling was performed for some OARs, using an inter-slice gap of 1-3 slices. Commonly-used whole-organ contouring assessment measures were calculated, and all cases were registered to a common reference shape per OAR to identify regions of manual adjustment. Results were expressed as the median, 10th-90th percentile of adjustment and visualized using 3D renderings.

Results

Per OAR, the median amount of editing was below 1 mm. However, large adjustments were found in some locations for most OARs. In general, enlarging of the auto-contours was needed. Subsampling DL-contours showed less adjustments were made in the interpolated slices compared to simulated no-subsampling for these OARs.

Conclusion

The real-world performance of automatic DL-contouring software was evaluated and proven useful in clinical practice. Specific regions-of-adjustment were identified per OAR in the thorax region, and separate models were found to be necessary for specific clinical indications different from training data. This analysis showed the need to perform routine clinical analysis especially when procedures or acquisition protocols change to have the best configuration of the workflow.

General and custom deep learning autosegmentation models for organs in head and neck, abdomen, and male pelvis

PURPOSE

To reduce workload and inconsistencies in organ segmentation for radiation treatment planning, we developed and evaluated general and custom autosegmentation models on computed tomography (CT) for three major tumor sites using a well-established deep convolutional neural network (DCNN).

METHODS

Five CT-based autosegmentation models for 42 organs at risk (OARs) in head and neck (HN), abdomen (ABD), and male pelvis (MP) were developed using a full three-dimensional (3D) DCNN architecture. Two types of deep learning (DL) models were separately trained using either general diversified multi-institutional datasets or custom well-controlled single-institution datasets. To improve segmentation accuracy, an adaptive spatial resolution approach for small and/or narrow OARs and a pseudo scan extension approach, when CT scan length is too short to cover entire organs, were implemented. The performance of the obtained models was evaluated based on accuracy and clinical applicability of the autosegmented contours using qualitative visual inspection and quantitative calculation of dice similarity coefficient (DSC), mean distance to agreement (MDA), and time efficiency.

RESULTS

The five DL autosegmentation models developed for the three anatomical sites were found to have high accuracy (DSC ranging from 0.8 to 0.98) for 74% OARs and marginally acceptable for 26% OARs. The custom models performed slightly better than the general models, even with smaller custom datasets used for the custom model training. The organ-based approaches improved autosegmentation accuracy for small or complex organs (e.g., eye lens, optic nerves, inner ears, and bowels). Compared with traditional manual contouring times, the autosegmentation times, including subsequent manual editing, if necessary, were substantially reduced by 88% for MP, 80% for HN, and 65% for ABD models.

CONCLUSIONS

The obtained autosegmentation models, incorporating organ-based approaches, were found to be effective and accurate for most OARs in the male pelvis, head and neck, and abdomen. We have demonstrated that our multianatomical DL autosegmentation models are clinically useful for radiation treatment planning.

Development of explanatory movies for the delineation of new organs at risk in neuro-oncology

•

Accurate and uniform OAR delineation is essential to gather consistent toxicity data.

•

New OARs were introduced in the 2021 update of EPTN Neurological Contouring Atlas.

•

We developed explanatory movies for the delineation of these OARs.

•

This aims to facilitate the training of delineation professionals.

Ten new organs at risk (OARs) were recently introduced in the updated European Particle Therapy Network neurological contouring atlas. Despite the use of the illustrated atlas and descriptive text, interindividual contouring variations may persist. To further facilitate the contouring of these OARs, educational films were developed and published on www.cancerdata.org.

Inter-observer variability of organ contouring for preclinical studies with cone beam Computed Tomography imaging

Background and purpose

In preclinical radiation studies, there is great interest in quantifying the radiation response of healthy tissues. Manual contouring has significant impact on the treatment-planning because of variation introduced by human interpretation. This results in inconsistencies when assessing normal tissue volumes. Evaluation of these discrepancies can provide a better understanding on the limitations of the current preclinical radiation workflow. In the present work, interobserver variability (IOV) in manual contouring of rodent normal tissues on cone-beam Computed Tomography, in head and thorax regions was evaluated.

Materials and methods

Two animal technicians performed manually (assisted) contouring of normal tissues located within the thorax and head regions of rodents, 20 cases per body site. Mean surface distance (MSD), displacement of center of mass (ΔCoM), DICE similarity coefficient (DSC) and the 95th percentile Hausdorff distance (HD₉₅) were calculated between the contours of the two observers to evaluate the IOV.

Results

For the thorax organs, right lung had the lowest IOV (ΔCoM: 0.08 ± 0.04 mm, DSC: 0.96 ± 0.01, MSD:0.07 ± 0.01 mm, HD₉₅:0.20 ± 0.03 mm) while spinal cord, the highest IOV (ΔCoM:0.5 ± 0.3 mm, DSC:0.81 ± 0.05, MSD:0.14 ± 0.03 mm, HD₉₅:0.8 ± 0.2 mm). Regarding head organs, right eye demonstrated the lowest IOV (ΔCoM:0.12 ± 0.08 mm, DSC: 0.93 ± 0.02, MSD: 0.15 ± 0.04 mm, HD₉₅: 0.29 ± 0.07 mm) while complete brain, the highest IOV (ΔCoM: 0.2 ± 0.1 mm, DSC: 0.94 ± 0.02, MSD: 0.3 ± 0.1 mm, HD₉₅: 0.5 ± 0.1 mm).

Conclusions

Our findings reveal small IOV, within the sub-mm range, for thorax and head normal tissues in rodents. The set of contours can serve as a basis for developing an automated delineation method for e.g., treatment planning.

Application of deep learning to auto-delineation of target volumes and organs at risk in radiotherapy

The technological advancement heralded the arrival of precision radiotherapy (RT), thereby increasing the therapeutic ratio and decreasing the side effects from treatment. Contour of target volumes (TV) and organs at risk (OARs) in RT is a complicated process. In recent years, automatic contouring of TV and OARs has rapidly developed due to the advances in deep learning (DL). This technology has the potential to save time and to reduce intra- or inter-observer variability. In this paper, the authors provide an overview of RT, introduce the concept of DL, summarize the data characteristics of the included literature, summarize the possible challenges for DL in the future, and discuss the possible research directions.

Deep learning-based classification and structure name standardization for organ at risk and target delineations in prostate cancer radiotherapy

Radiotherapy (RT) datasets can suffer from variations in annotation of organ at risk (OAR) and target structures. Annotation standards exist, but their description for prostate targets is limited. This restricts the use of such data for supervised machine learning purposes as it requires properly annotated data. The aim of this work was to develop a modality independent deep learning (DL) model for automatic classification and annotation of prostate RT DICOM structures. Delineated prostate organs at risk (OAR), support- and target structures (gross tumor volume [GTV]/clinical target volume [CTV]/planning target volume [PTV]), along with or without separate vesicles and/or lymph nodes, were extracted as binary masks from 1854 patients. An image modality independent 2D InceptionResNetV2 classification network was trained with varying amounts of training data using four image input channels. Channel 1-3 consisted of orthogonal 2D projections from each individual binary structure. The fourth channel contained a summation of the other available binary structure masks. Structure classification performance was assessed in independent CT (n = 200 pat) and magnetic resonance imaging (MRI) (n = 40 pat) test datasets and an external CT (n = 99 pat) dataset from another clinic. A weighted classification accuracy of 99.4% was achieved during training. The unweighted classification accuracy and the weighted average F1 score among different structures in the CT test dataset were 98.8% and 98.4% and 98.6% and 98.5% for the MRI test dataset, respectively. The external CT dataset yielded the corresponding results 98.4% and 98.7% when analyzed for trained structures only, and results from the full dataset yielded 79.6% and 75.2%. Most misclassifications in the external CT dataset occurred due to multiple CTVs and PTVs being fused together, which was not included in the training data. Our proposed DL-based method for automated renaming and standardization of prostate radiotherapy annotations shows great potential. Clinic specific contouring standards however need to be represented in the training data for successful use. Source code is available at https://github.com/jamtheim/DicomRTStructRenamerPublic.

Automated delineation of head and neck organs at risk using synthetic MRI-aided mask scoring regional convolutional neural network

PURPOSE

Auto-segmentation algorithms offer a potential solution to eliminate the labor-intensive, time-consuming, and observer-dependent manual delineation of organs-at-risk (OARs) in radiotherapy treatment planning. This study aimed to develop a deep learning-based automated OAR delineation method to tackle the current challenges remaining in achieving reliable expert performance with the state-of-the-art auto-delineation algorithms.

METHODS

The accuracy of OAR delineation is expected to be improved by utilizing the complementary contrasts provided by computed tomography (CT) (bony-structure contrast) and magnetic resonance imaging (MRI) (soft-tissue contrast). Given CT images, synthetic MR images were firstly generated by a pre-trained cycle-consistent generative adversarial network. The features of CT and synthetic MRI were then extracted and combined for the final delineation of organs using mask scoring regional convolutional neural network. Both in-house and public datasets containing CT scans from head-and-neck (HN) cancer patients were adopted to quantitatively evaluate the performance of the proposed method against current state-of-the-art algorithms in metrics including Dice similarity coefficient (DSC), 95th percentile Hausdorff distance (HD95), mean surface distance (MSD), and residual mean square distance (RMS).

RESULTS

Across all of 18 OARs in our in-house dataset, the proposed method achieved an average DSC, HD95, MSD, and RMS of 0.77 (0.58-0.90), 2.90 mm (1.32-7.63 mm), 0.89 mm (0.42-1.85 mm), and 1.44 mm (0.71-3.15 mm), respectively, outperforming the current state-of-the-art algorithms by 6%, 16%, 25%, and 36%, respectively. On public datasets, for all nine OARs, an average DSC of 0.86 (0.73-0.97) were achieved, 6% better than the competing methods.

CONCLUSION

We demonstrated the feasibility of a synthetic MRI-aided deep learning framework for automated delineation of OARs in HN radiotherapy treatment planning. The proposed method could be adopted into routine HN cancer radiotherapy treatment planning to rapidly contour OARs with high accuracy.

An Adversarial Deep-Learning-Based Model for Cervical Cancer CTV Segmentation With Multicenter Blinded Randomized Controlled Validation

Purpose

To propose a novel deep-learning-based auto-segmentation model for CTV delineation in cervical cancer and to evaluate whether it can perform comparably well to manual delineation by a three-stage multicenter evaluation framework.

Methods

An adversarial deep-learning-based auto-segmentation model was trained and configured for cervical cancer CTV contouring using CT data from 237 patients. Then CT scans of additional 20 consecutive patients with locally advanced cervical cancer were collected to perform a three-stage multicenter randomized controlled evaluation involving nine oncologists from six medical centers. This evaluation system is a combination of objective performance metrics, radiation oncologist assessment, and finally the head-to-head Turing imitation test. Accuracy and effectiveness were evaluated step by step. The intra-observer consistency of each oncologist was also tested.

Results

In stage-1 evaluation, the mean DSC and the 95HD value of the proposed model were 0.88 and 3.46 mm, respectively. In stage-2, the oncologist grading evaluation showed the majority of AI contours were comparable to the GT contours. The average CTV scores for AI and GT were 2.68 vs. 2.71 in week 0 (P = .206), and 2.62 vs. 2.63 in week 2 (P = .552), with no significant statistical differences. In stage-3, the Turing imitation test showed that the percentage of AI contours, which were judged to be better than GT contours by ≥5 oncologists, was 60.0% in week 0 and 42.5% in week 2. Most oncologists demonstrated good consistency between the 2 weeks (P > 0.05).

Conclusions

The tested AI model was demonstrated to be accurate and comparable to the manual CTV segmentation in cervical cancer patients when assessed by our three-stage evaluation framework.

The impact of organ-at-risk contour variations on automatically generated treatment plans for NSCLC

BACKGROUND AND PURPOSE

Quality of automatic contouring is generally assessed by comparison with manual delineations, but the effect of contour differences on the resulting dose distribution remains unknown. This study evaluated dosimetric differences between treatment plans optimized using various organ-at-risk (OAR) contouring methods.

MATERIALS AND METHODS

OARs of twenty lung cancer patients were manually and automatically contoured, after which user-adjustments were made. For each contour set, an automated treatment plan was generated. The dosimetric effect of intra-observer contour variation and the influence of contour variations on treatment plan evaluation and generation were studied using dose-volume histogram (DVH)-parameters for thoracic OARs.

RESULTS

Dosimetric effect of intra-observer contour variability was highest for Heart D_max (3.4 ± 6.8 Gy) and lowest for Lungs-GTV D_mean (0.3 ± 0.4 Gy). The effect of contour variation on treatment plan evaluation was highest for Heart D_max (6.0 ± 13.4 Gy) and Esophagus D_max (8.7 ± 17.2 Gy). Dose differences for the various treatment plans, evaluated on the reference (manual) contour, were on average below 1 Gy/1%. For Heart D_mean, higher dose differences were found for overlap with PTV (median 0.2 Gy, 95% 1.7 Gy) vs. no PTV overlap (median 0 Gy, 95% 0.5 Gy). For D_max-parameters, largest dose difference was found between 0-1 cm distance to PTV (median 1.5 Gy, 95% 4.7 Gy).

CONCLUSION

Dose differences arising from automatic contour variations were of the same magnitude or lower than intra-observer contour variability. For Heart D_mean, we recommend delineation errors to be corrected when the heart overlaps with the PTV. For D_max-parameters, we recommend checking contours if the distance is close to PTV (<5 cm). For the lungs, only obvious large errors need to be adjusted.

Deep learning for segmentation in radiation therapy planning: a review

Segmentation of organs and structures, as either targets or organs-at-risk, has a significant influence on the success of radiation therapy. Manual segmentation is a tedious and time-consuming task for clinicians, and inter-observer variability can affect the outcomes of radiation therapy. The recent hype over deep neural networks has added many powerful auto-segmentation methods as variations of convolutional neural networks (CNN). This paper presents a descriptive review of the literature on deep learning techniques for segmentation in radiation therapy planning. The most common CNN architecture across the four clinical sub sites considered was U-net, with the majority of deep learning segmentation articles focussed on head and neck normal tissue structures. The most common data sets were CT images from an inhouse source, along with some public data sets. N-fold cross-validation was commonly employed; however, not all work separated training, test and validation data sets. This area of research is expanding rapidly. To facilitate comparisons of proposed methods and benchmarking, consistent use of appropriate metrics and independent validation should be carefully considered.

Effects of sample size and data augmentation on U-Net-based automatic segmentation of various organs

Deep learning has demonstrated high efficacy for automatic segmentation in contour delineation, which is crucial in radiation therapy planning. However, the collection, labeling, and management of medical imaging data can be challenging. This study aims to elucidate the effects of sample size and data augmentation on the automatic segmentation of computed tomography images using U-Net, a deep learning method. For the chest and pelvic regions, 232 and 556 cases are evaluated, respectively. We investigate multiple conditions by changing the sum of the training and validation datasets across a broad range of values: 10-200 and 10-500 cases for the chest and pelvic regions, respectively. A U-Net is constructed, and horizontal-flip data augmentation, which produces left and right inverse images resulting in twice the number of images, is compared with no augmentation for each training session. All lung cases and more than 100 prostate, bladder, and rectum cases indicate that adding horizontal-flip data augmentation is almost as effective as doubling the number of cases. The slope of the Dice similarity coefficient (DSC) in all organs decreases rapidly until approximately 100 cases, stabilizes after 200 cases, and shows minimal changes as the number of cases is increased further. The DSCs stabilize at a smaller sample size with the incorporation of data augmentation in all organs except the heart. This finding is applicable to the automation of radiation therapy for rare cancers, where large datasets may be difficult to obtain.

Metrics to evaluate the performance of auto-segmentation for radiation treatment planning: A critical review

Advances in artificial intelligence-based methods have led to the development and publication of numerous systems for auto-segmentation in radiotherapy. These systems have the potential to decrease contour variability, which has been associated with poor clinical outcomes and increased efficiency in the treatment planning workflow. However, there are no uniform standards for evaluating auto-segmentation platforms to assess their efficacy at meeting these goals. Here, we review the most frequently used evaluation techniques which include geometric overlap, dosimetric parameters, time spent contouring, and clinical rating scales. These data suggest that many of the most commonly used geometric indices, such as the Dice Similarity Coefficient, are not well correlated with clinically meaningful endpoints. As such, a multi-domain evaluation, including composite geometric and/or dosimetric metrics with physician-reported assessment, is necessary to gauge the clinical readiness of auto-segmentation for radiation treatment planning.