Deep learning-based automatic segmentation of cardiac substructures for lung cancers

PURPOSE

Accurate and comprehensive segmentation of cardiac substructures is crucial for minimizing the risk of radiation-induced heart disease in lung cancer radiotherapy. We sought to develop and validate deep learning-based auto-segmentation models for cardiac substructures.

MATERIALS AND METHODS

Nineteen cardiac substructures (whole heart, 4 heart chambers, 6 great vessels, 4 valves, and 4 coronary arteries) in 100 patients treated for non-small cell lung cancer were manually delineated by two radiation oncologists. The valves and coronary arteries were delineated as planning risk volumes. An nnU-Net auto-segmentation model was trained, validated, and tested on this dataset with a split ratio of 75:5:20. The auto-segmented contours were evaluated by comparing them with manually drawn contours in terms of Dice similarity coefficient (DSC) and dose metrics extracted from clinical plans. An independent dataset of 42 patients was used for subjective evaluation of the auto-segmentation model by 4 physicians.

RESULTS

The average DSCs were 0.95 (+/- 0.01) for the whole heart, 0.91 (+/- 0.02) for 4 chambers, 0.86 (+/- 0.09) for 6 great vessels, 0.81 (+/- 0.09) for 4 valves, and 0.60 (+/- 0.14) for 4 coronary arteries. The average absolute errors in mean/max doses to all substructures were 1.04 (+/- 1.99) Gy and 2.20 (+/- 4.37) Gy. The subjective evaluation revealed that 94% of the auto-segmented contours were clinically acceptable.

CONCLUSION

We demonstrated the effectiveness of our nnU-Net model for delineating cardiac substructures, including coronary arteries. Our results indicate that this model has promise for studies regarding radiation dose to cardiac substructures.

Comparison of deep learning networks for fully automated head and neck tumor delineation on multi-centric PET/CT images

OBJECTIVES

Deep learning-based auto-segmentation of head and neck cancer (HNC) tumors is expected to have better reproducibility than manual delineation. Positron emission tomography (PET) and computed tomography (CT) are commonly used in tumor segmentation. However, current methods still face challenges in handling whole-body scans where a manual selection of a bounding box may be required. Moreover, different institutions might still apply different guidelines for tumor delineation. This study aimed at exploring the auto-localization and segmentation of HNC tumors from entire PET/CT scans and investigating the transferability of trained baseline models to external real world cohorts.

METHODS

We employed 2D Retina Unet to find HNC tumors from whole-body PET/CT and utilized a regular Unet to segment the union of the tumor and involved lymph nodes. In comparison, 2D/3D Retina Unets were also implemented to localize and segment the same target in an end-to-end manner. The segmentation performance was evaluated via Dice similarity coefficient (DSC) and Hausdorff distance 95th percentile (HD95). Delineated PET/CT scans from the HECKTOR challenge were used to train the baseline models by 5-fold cross-validation. Another 271 delineated PET/CTs from three different institutions (MAASTRO, CRO, BERLIN) were used for external testing. Finally, facility-specific transfer learning was applied to investigate the improvement of segmentation performance against baseline models.

RESULTS

Encouraging localization results were observed, achieving a maximum omnidirectional tumor center difference lower than 6.8 cm for external testing. The three baseline models yielded similar averaged cross-validation (CV) results with a DSC in a range of 0.71-0.75, while the averaged CV HD95 was 8.6, 10.7 and 9.8 mm for the regular Unet, 2D and 3D Retina Unets, respectively. More than a 10% drop in DSC and a 40% increase in HD95 were observed if the baseline models were tested on the three external cohorts directly. After the facility-specific training, an improvement in external testing was observed for all models. The regular Unet had the best DSC (0.70) for the MAASTRO cohort, and the best HD95 (7.8 and 7.9 mm) in the MAASTRO and CRO cohorts. The 2D Retina Unet had the best DSC (0.76 and 0.67) for the CRO and BERLIN cohorts, and the best HD95 (12.4 mm) for the BERLIN cohort.

CONCLUSION

The regular Unet outperformed the other two baseline models in CV and most external testing cohorts. Facility-specific transfer learning can potentially improve HNC segmentation performance for individual institutions, where the 2D Retina Unets could achieve comparable or even better results than the regular Unet.

Validation of a deep-learning segmentation model for adult and pediatric head and neck radiotherapy in different patient positions

Background and purpose

Autocontouring for radiotherapy has the potential to significantly save time and reduce interobserver variability. We aimed to assess the performance of a commercial autocontouring model for head and neck (H&N) patients in eight orientations relevant to particle therapy with fixed beam lines, focusing on validation and implementation for routine clinical use.

Materials and methods

Autocontouring was performed on sixteen organs at risk (OARs) for 98 adult and pediatric patients with 137 H&N CT scans in eight orientations. A geometric comparison of the autocontours and manual segmentations was performed using the Hausdorff Distance 95th percentile, Dice Similarity Coefficient (DSC) and surface DSC and compared to interobserver variability where available. Additional qualitative scoring and dose-volume-histogram (DVH) parameters analyses were performed for twenty patients in two positions, consisting of scoring on a 0-3 scale based on clinical usability and comparing the mean (Dmean) and near-maximum (D2%) dose, respectively.

Results

For the geometric analysis, the model performance in head-first-supine straight and hyperextended orientations was in the same range as the interobserver variability. HD95, DSC and surface DSC was heterogeneous in other orientations. No significant geometric differences were found between pediatric and adult autocontours. The qualitative scoring yielded a median score of ≥ 2 for 13/16 OARs while 7/32 DVH parameters were significantly different.

Conclusions

For head-first-supine straight and hyperextended scans, we found that 13/16 OAR autocontours were suited for use in daily clinical practice and subsequently implemented. Further development is needed for other patient orientations before implementation.

A radiomics based approach using adrenal gland and periadrenal fat CT images to allocate COVID-19 health care resources fairly

BACKGROUND

The value of radiomics features from the adrenal gland and periadrenal fat CT images for predicting disease progression in patients with COVID-19 has not been studied extensively. We assess the value of radiomics features from the adrenal gland and periadrenal fat CT images in predicting COVID-19 disease exacerbation.

METHODS

A total of 1,245 patients (685 moderate and 560 severe patients) were enrolled in a retrospective study. We proposed a 3D V-net to segment adrenal glands in onset CT images automatically, and periadrenal fat was obtained using inflation operation around the adrenal gland. Next, we built a clinical model (CM), three radiomics models (adrenal gland model [AM], periadrenal fat model [PM], and fusion of adrenal gland and periadrenal fat model [FM]), and radiomics nomogram (RN) after radiomics features extracted.

RESULTS

The auto-segmentation framework yielded a dice value 0.79 in the training set. CM, AM, PM, FM, and RN obtained AUCs of 0.717, 0.716, 0.736, 0.760, and 0.833 in the validation set. FM and RN had better predictive efficacy than CM (P < 0.0001) in the training set. RN showed that there was no significant difference in the validation set (mean absolute error [MAE] = 0.04) and test set (MAE = 0.075) between predictive and actual results. Decision curve analysis showed that if the threshold probability was between 0.4 and 0.8 in the validation set or between 0.3 and 0.7 in the test set, it could gain more net benefits using RN than FM and CM.

CONCLUSIONS

Radiomics features extracted from the adrenal gland and periadrenal fat CT images are related to disease exacerbation in patients with COVID-19.

Efficient segmentation using domain adaptation for MRI-guided and CBCT-guided online adaptive radiotherapy

BACKGROUND

Delineation of regions of interest (ROIs) is important for adaptive radiotherapy (ART) but it is also time consuming and labor intensive.

AIM

This study aims to develop efficient segmentation methods for magnetic resonance imaging-guided ART (MRIgART) and cone-beam computed tomography-guided ART (CBCTgART).

MATERIALS AND METHODS

MRIgART and CBCTgART studies enrolled 242 prostate cancer patients and 530 nasopharyngeal carcinoma patients, respectively. A public dataset of CBCT from 35 pancreatic cancer patients was adopted to test the framework. We designed two domain adaption methods to learn and adapt the features from planning computed tomography (pCT) to MRI or CBCT modalities. The pCT was transformed to synthetic MRI (sMRI) for MRIgART, while CBCT was transformed to synthetic CT (sCT) for CBCTgART. Generalized segmentation models were trained with large popular data in which the inputs were sMRI for MRIgART and pCT for CBCTgART. Finally, the personalized models for each patient were established by fine-tuning the generalized model with the contours on pCT of that patient. The proposed method was compared with deformable image registration (DIR), a regular deep learning (DL) model trained on the same modality (DL-regular), and a generalized model in our framework (DL-generalized).

RESULTS

The proposed method achieved better or comparable performance. For MRIgART of the prostate cancer patients, the mean dice similarity coefficient (DSC) of four ROIs was 87.2%, 83.75%, 85.36%, and 92.20% for the DIR, DL-regular, DL-generalized, and proposed method, respectively. For CBCTgART of the nasopharyngeal carcinoma patients, the mean DSC of two target volumes were 90.81% and 91.18%, 75.17% and 58.30%, for the DIR, DL-regular, DL-generalized, and the proposed method, respectively. For CBCTgART of the pancreatic cancer patients, the mean DSC of two ROIs were 61.94% and 61.44%, 63.94% and 81.56%, for the DIR, DL-regular, DL-generalized, and the proposed method, respectively.

CONCLUSION

The proposed method utilizing personalized modeling improved the segmentation accuracy of ART.

Prior information guided auto-segmentation of clinical target volume of tumor bed in postoperative breast cancer radiotherapy

BACKGROUND

Accurate delineation of clinical target volume of tumor bed (CTV-TB) is important but it is also challenging due to surgical effects and soft tissue contrast. Recently a few auto-segmentation methods were developed to improve the process. However, those methods had comparatively low segmentation accuracy. In this study the prior information was introduced to aid auto-segmentation of CTV-TB based on a deep-learning model.

METHODS

To aid the delineation of CTV-TB, the tumor contour on preoperative CT was transformed onto postoperative CT via deformable image registration. Both original and transformed tumor contours were used for prior information in training an auto-segmentation model. Then, the CTV-TB contour on postoperative CT was predicted by the model. 110 pairs of preoperative and postoperative CT images were used with a 5-fold cross-validation strategy. The predicted contour was compared with the clinically approved contour for accuracy evaluation using dice similarity coefficient (DSC) and Hausdorff distance.

RESULTS

The average DSC of the deep-learning model with prior information was improved than the one without prior information (0.808 vs. 0.734, P < 0.05). The average DSC of the deep-learning model with prior information was higher than that of the traditional method (0.808 vs. 0.622, P < 0.05).

CONCLUSIONS

The introduction of prior information in deep-learning model can improve segmentation accuracy of CTV-TB. The proposed method provided an effective way to automatically delineate CTV-TB in postoperative breast cancer radiotherapy.

Prior knowledge based deep learning auto-segmentation in magnetic resonance imaging-guided radiotherapy of prostate cancer

Background and purpose

Automation is desirable for organ segmentation in radiotherapy. This study compared deep learning methods for auto-segmentation of organs-at-risk (OARs) and clinical target volume (CTV) in prostate cancer patients undergoing fractionated magnetic resonance (MR)-guided adaptive radiation therapy. Models predicting dense displacement fields (DDFMs) between planning and fraction images were compared to patient-specific (PSM) and baseline (BM) segmentation models.

Materials and methods

A dataset of 92 patients with planning and fraction MR images (MRIs) from two institutions were used. DDFMs were trained to predict dense displacement fields (DDFs) between the planning and fraction images, which were subsequently used to propagate the planning contours of the bladder, rectum, and CTV to the daily MRI. The training was performed either with true planning-fraction image pairs or with planning images and their counterparts deformed by known DDFs. The BMs were trained on 53 planning images, while to generate PSMs, the BMs were fine-tuned using the planning image of a given single patient. The evaluation included Dice similarity coefficient (DSC), the average (HDavg) and the 95th percentile (HD95) Hausdorff distance (HD).

Results

The DDFMs with DSCs for bladder/rectum of 0.76/0.76 performed worse than PSMs (0.91/0.90) and BMs (0.89/0.88). The same trend was observed for HDs. For CTV, DDFM and PSM performed similarly yielding DSCs of 0.87 and 0.84, respectively.

Conclusions

DDFMs were found suitable for CTV delineation after rigid alignment. However, for OARs they were outperformed by PSMs, as they predicted only limited deformations even in the presence of substantial anatomical changes.

Clinical Implementation and Evaluation of Auto-Segmentation Tools for Multi-Site Contouring in Radiotherapy

Background and purpose

Tools for auto-segmentation in radiotherapy are widely available, but guidelines for clinical implementation are missing. The goal was to develop a workflow for performance evaluation of three commercial auto-segmentation tools to select one candidate for clinical implementation.

Materials and Methods

One hundred patients with six treatment sites (brain, head-and-neck, thorax, abdomen, and pelvis) were included. Three sets of AI-based contours for organs-at-risk (OAR) generated by three software tools and manually drawn expert contours were blindly rated for contouring accuracy. The dice similarity coefficient (DSC), the Hausdorff distance, and a dose/volume evaluation based on the recalculation of the original treatment plan were assessed. Statistically significant differences were tested using the Kruskal-Wallis test and the post-hoc Dunn Test with Bonferroni correction.

Results

The mean DSC scores compared to expert contours for all OARs combined were 0.80 ± 0.10, 0.75 ± 0.10, and 0.74 ± 0.11 for the three software tools. Physicians' rating identified equivalent or superior performance of some AI-based contours in head (eye, lens, optic nerve, brain, chiasm), thorax (e.g., heart and lungs), and pelvis and abdomen (e.g., kidney, femoral head) compared to manual contours. For some OARs, the AI models provided results requiring only minor corrections. Bowel-bag and stomach were not fit for direct use. During the interdisciplinary discussion, the physicians' rating was considered the most relevant.

Conclusion

A comprehensive method for evaluation and clinical implementation of commercially available auto-segmentation software was developed. The in-depth analysis yielded clear instructions for clinical use within the radiotherapy department.

Assembling High-Quality Lymph Node Clinical Target Volumes for Cervical Cancer Radiotherapy Using a Deep Learning-Based Approach

AIM

The study aimed to explore an approach for accurately assembling high-quality lymph node clinical target volumes (CTV) on CT images in cervical cancer radiotherapy with the encoder-decoder 3D network.

METHODS

216 cases of CT images treated at our center between 2017 and 2020 were included as a sample, which were divided into two cohorts, including 152 cases and 64 controls, respectively. Para-aortic lymph node, common iliac, external iliac, internal iliac, obturator, presacral, and groin nodal regions were delineated as sub-CTV manually in the cohort including 152 cases. Then, the 152 cases were randomly divided into training (96 cases), validation (36 cases), and test (20 cases) groups for the training process. Each structure was individually trained and optimized through a deep learning model. An additional 64 cases with 6 different clinical conditions were taken as examples to verify the feasibility of CTV generation based on our model. Dice similarity coefficient (DSC) and Hausdorff distance (HD) metrics were both used for quantitative evaluation.

RESULTS

Comparing auto-segmentation results to ground truth, the mean DSC value/HD was 0.838/7.7mm, 0.853/4.7mm, 0.855/4.7mm, 0.844/4.7mm, 0.784/5.2mm, 0.826/4.8mm and 0.874/4.8mm for CTV_PAN, CTV_common iliac, CTV_internal iliac, CTV_external iliac, CTV_obturator, CTV_presacral, and CTV_groin, respectively. The similarity comparison results of six different clinical situations were 0.877/4.4mm, 0.879/4.6mm, 0.881/4.2mm, 0.882/4.3mm, 0.872/6.0mm, and 0.875/4.9mm for DSC value/HD, respectively.

CONCLUSION

We have developed a deep learning-based approach to segmenting lymph node sub-regions automatically and assembling high-quality CTVs according to clinical needs in cervical cancer radiotherapy. This work can increase the efficiency of the process of cervical cancer detection and treatment.

Deep learning based automatic segmentation of organs-at-risk for 0.35 T MRgRT of lung tumors

BACKGROUND AND PURPOSE

Magnetic resonance imaging guided radiotherapy (MRgRT) offers treatment plan adaptation to the anatomy of the day. In the current MRgRT workflow, this requires the time consuming and repetitive task of manual delineation of organs-at-risk (OARs), which is also prone to inter- and intra-observer variability. Therefore, deep learning autosegmentation (DLAS) is becoming increasingly attractive. No investigation of its application to OARs in thoracic magnetic resonance images (MRIs) from MRgRT has been done so far. This study aimed to fill this gap.

MATERIALS AND METHODS

122 planning MRIs from patients treated at a 0.35 T MR-Linac were retrospectively collected. Using an 80/19/23 (training/validation/test) split, individual 3D U-Nets for segmentation of the left lung, right lung, heart, aorta, spinal canal and esophagus were trained. These were compared to the clinically used contours based on Dice similarity coefficient (DSC) and Hausdorff distance (HD). They were also graded on their clinical usability by a radiation oncologist.

RESULTS

Median DSC was 0.96, 0.96, 0.94, 0.90, 0.88 and 0.78 for left lung, right lung, heart, aorta, spinal canal and esophagus, respectively. Median 95th percentile values of the HD were 3.9, 5.3, 5.8, 3.0, 2.6 and 3.5 mm, respectively. The physician preferred the network generated contours over the clinical contours, deeming 85 out of 129 to not require any correction, 25 immediately usable for treatment planning, 15 requiring minor and 4 requiring major corrections.

CONCLUSIONS

We trained 3D U-Nets on clinical MRI planning data which produced accurate delineations in the thoracic region. DLAS contours were preferred over the clinical contours.

Clinical evaluation of a deep learning segmentation model including manual adjustments afterwards for locally advanced breast cancer

Introduction

Deep learning (DL) models are increasingly developed for auto-segmentation in radiotherapy. Qualitative analysis is of great importance for clinical implementation, next to quantitative. This study evaluates a DL segmentation model for left- and right-sided locally advanced breast cancer both quantitatively and qualitatively.

Methods

For each side a DL model was trained, including primary breast CTV (CTVp), lymph node levels 1-4, heart, lungs, humeral head, thyroid and esophagus. For evaluation, both automatic segmentation, including correction of contours when needed, and manual delineation was performed and both processes were timed. Quantitative scoring with dice-similarity coefficient (DSC), 95% Hausdorff Distance (95%HD) and surface DSC (sDSC) was used to compare both the automatic (not-corrected) and corrected contours with the manual contours. Qualitative scoring was performed by five radiotherapy technologists and five radiation oncologists using a 3-point Likert scale.

Results

Time reduction was achieved using auto-segmentation in 95% of the cases, including correction. The time reduction (mean ± std) was 42.4% ± 26.5% and 58.5% ± 19.1% for OARs and CTVs, respectively, corresponding to an absolute mean reduction (hh:mm:ss) of 00:08:51 and 00:25:38. Good quantitative results were achieved before correction, e.g. mean DSC for the right-sided CTVp was 0.92 ± 0.06, whereas correction statistically significantly improved this contour by only 0.02 ± 0.05, respectively. In 92% of the cases, auto-contours were scored as clinically acceptable, with or without corrections.

Conclusions

A DL segmentation model was trained and was shown to be a time-efficient way to generate clinically acceptable contours for locally advanced breast cancer.

Comparison of the output of a deep learning segmentation model for locoregional breast cancer radiotherapy trained on 2 different datasets

Introduction

The development of deep learning (DL) models for auto-segmentation is increasing and more models become commercially available. Mostly, commercial models are trained on external data. To study the effect of using a model trained on external data, compared to the same model trained on in-house collected data, the performance of these two DL models was evaluated.

Methods

The evaluation was performed using in-house collected data of 30 breast cancer patients. Quantitative analysis was performed using Dice similarity coefficient (DSC), surface DSC (sDSC) and 95th percentile of Hausdorff Distance (95% HD). These values were compared with previously reported inter-observer variations (IOV).

Results

For a number of structures, statistically significant differences were found between the two models. For organs at risk, mean values for DSC ranged from 0.63 to 0.98 and 0.71 to 0.96 for the in-house and external model, respectively. For target volumes, mean DSC values of 0.57 to 0.94 and 0.33 to 0.92 were found. The difference of 95% HD values ranged 0.08 to 3.23 mm between the two models, except for CTVn4 with 9.95 mm. For the external model, both DSC and 95% HD are outside the range of IOV for CTVn4, whereas this is the case for the DSC found for the thyroid of the in-house model.

Conclusions

Statistically significant differences were found between both models, which were mostly within published inter-observer variations, showing clinical usefulness of both models. Our findings could encourage discussion and revision of existing guidelines, to further decrease inter-observer, but also inter-institute variability.

Artificial Intelligence for Radiotherapy Auto-Contouring: Current Use, Perceptions of and Barriers to Implementation

AIMS

Artificial intelligence has the potential to transform the radiotherapy workflow, resulting in improved quality, safety, accuracy and timeliness of radiotherapy delivery. Several commercially available artificial intelligence-based auto-contouring tools have emerged in recent years. Their clinical deployment raises important considerations for clinical oncologists, including quality assurance and validation, education, training and job planning. Despite this, there is little in the literature capturing the views of clinical oncologists with respect to these factors.

MATERIALS AND METHODS

The Royal College of Radiologists realises the transformational impact artificial intelligence is set to have on our specialty and has appointed the Artificial Intelligence for Clinical Oncology working group. The aim of this work was to survey clinical oncologists with regards to perceptions, current use of and barriers to using artificial intelligence-based auto-contouring for radiotherapy. Here we share our findings with the wider clinical and radiation oncology communities. We hope to use these insights in developing support, guidance and educational resources for the deployment of auto-contouring for clinical use, to help develop the case for wider access to artificial intelligence-based auto-contouring across the UK and to share practice from early-adopters.

RESULTS

In total, 78% of clinical oncologists surveyed felt that artificial intelligence would have a positive impact on radiotherapy. Attitudes to risk were more varied, but 49% felt that artificial intelligence will decrease risk for patients. There is a marked appetite for urgent guidance, education and training on the safe use of such tools in clinical practice. Furthermore, there is a concern that the adoption and implementation of such tools is not equitable, which risks exacerbating existing inequalities across the country.

CONCLUSION

Careful coordination is required to ensure that all radiotherapy departments, and the patients they serve, may enjoy the benefits of artificial intelligence in radiotherapy. Professional organisations, such as the Royal College of Radiologists, have a key role to play in delivering this.

PyRaDiSe: A Python package for DICOM-RT-based auto-segmentation pipeline construction and DICOM-RT data conversion

BACKGROUND AND OBJECTIVE

Despite fast evolution cycles in deep learning methodologies for medical imaging in radiotherapy, auto-segmentation solutions rarely run in clinics due to the lack of open-source frameworks feasible for processing DICOM RT Structure Sets. Besides this shortage, available open-source DICOM RT Structure Set converters rely exclusively on 2D reconstruction approaches leading to pixelated contours with potentially low acceptance by healthcare professionals. PyRaDiSe, an open-source, deep learning framework independent Python package, addresses these issues by providing a framework for building auto-segmentation solutions feasible to operate directly on DICOM data. In addition, PyRaDiSe provides profound DICOM RT Structure Set conversion and processing capabilities; thus, it applies also to auto-segmentation-related tasks, such as dataset construction for deep learning model training.

METHODS

The PyRaDiSe package follows a holistic approach and provides DICOM data handling, deep learning model inference, pre-processing, and post-processing functionalities. The DICOM data handling allows for highly automated and flexible handling of DICOM image series, DICOM RT Structure Sets, and DICOM registrations, including 2D-based and 3D-based conversion from and to DICOM RT Structure Sets. For deep learning model inference, extending given skeleton classes is straightforwardly achieved, allowing for employing any deep learning framework. Furthermore, a profound set of pre-processing and post-processing routines is included that incorporate partial invertibility for restoring spatial properties, such as image origin or orientation.

RESULTS

The PyRaDiSe package, characterized by its flexibility and automated routines, allows for fast deployment and prototyping, reducing efforts for auto-segmentation pipeline implementation. Furthermore, while deep learning model inference is independent of the deep learning framework, it can easily be integrated into famous deep learning frameworks such as PyTorch or Tensorflow. The developed package has successfully demonstrated its capabilities in a research project at our institution for organs-at-risk segmentation in brain tumor patients. Furthermore, PyRaDiSe has shown its conversion performance for dataset construction.

CONCLUSIONS

The PyRaDiSe package closes the gap between data science and clinical radiotherapy by enabling deep learning segmentation models to be easily transferred into clinical research practice. PyRaDiSe is available on https://github.com/ubern-mia/pyradise and can be installed directly from the Python Package Index using pip install pyradise.

Auto-segmentation of the tibia and femur from knee MR images via deep learning and its application to cartilage strain and recovery

The ability to efficiently and reproducibly generate subject-specific 3D models of bone and soft tissue is important to many areas of musculoskeletal research. However, methodologies requiring such models have largely been limited by lengthy manual segmentation times. Recently, machine learning, and more specifically, convolutional neural networks, have shown potential to alleviate this bottleneck in research throughput. Thus, the purpose of this work was to develop a modified version of the convolutional neural network architecture U-Net to automate segmentation of the tibia and femur from double echo steady state knee magnetic resonance (MR) images. Our model was trained on a dataset of over 4,000 MR images from 34 subjects, segmented by three experienced researchers, and reviewed by a musculoskeletal radiologist. For our validation and testing sets, we achieved dice coefficients of 0.985 and 0.984, respectively. As further testing, we applied our trained model to a prior study of tibial cartilage strain and recovery. In this analysis, across all subjects, there were no statistically significant differences in cartilage strain between the machine learning and ground truth bone models, with a mean difference of 0.2 ± 0.7 % (mean ± 95 % confidence interval). This difference is within the measurement resolution of previous cartilage strain studies from our lab using manual segmentation. In summary, we successfully trained, validated, and tested a machine learning model capable of segmenting MR images of the knee, achieving results that are comparable to trained human segmenters.

Mendability Index: A new metric for estimating the effort required for manually editing auto-segmentations of objects of interest

Auto-segmentation of medical images is critical to boost precision radiology and radiation oncology efficiency, thereby improving medical quality for both health care practitioners and patients. An appropriate metric to evaluate auto-segmentation results is one of the significant tools necessary for building an effective, robust, and practical auto-segmentation technique. However, by comparing the predicted segmentation with the ground truth, currently widely-used metrics usually focus on the overlapping area (Dice Coefficient) or the most severe shifting of the boundary (Hausdorff Distance), which seem inconsistent with human reader behaviors. Human readers usually verify and correct auto-segmentation contours and then apply the modified segmentation masks to guide clinical application in diagnosis or treatment. A metric called Mendability Index (MI) is proposed to better estimate the effort required for manually editing the auto-segmentations of objects of interest in medical images so that the segmentations become acceptable for the application at hand. Considering different human behaviors for different errors, MI classifies auto-segmented errors into three types with different quantitative behaviors. The fluctuation of human subjective delineation is also considered in MI. 505 3D computed tomography (CT) auto-segmentations consisting of 6 objects from 3 institutions with the corresponding ground truth and the recorded manual mending time needed by experts are used to validate the performance of the proposed MI. The correlation between the time for editing with the segmentation metrics demonstrates that MI is generally more suitable for indicating mending efforts than Dice Coefficient or Hausdorff Distance, suggesting that MI may be an effective metric to quantify the clinical value of auto-segmentations.

A Review of the Metrics Used to Assess Auto-Contouring Systems in Radiotherapy

Auto-contouring could revolutionise future planning of radiotherapy treatment. The lack of consensus on how to assess and validate auto-contouring systems currently limits clinical use. This review formally quantifies the assessment metrics used in studies published during one calendar year and assesses the need for standardised practice. A PubMed literature search was undertaken for papers evaluating radiotherapy auto-contouring published during 2021. Papers were assessed for types of metric and the methodology used to generate ground-truth comparators. Our PubMed search identified 212 studies, of which 117 met the criteria for clinical review. Geometric assessment metrics were used in 116 of 117 studies (99.1%). This includes the Dice Similarity Coefficient used in 113 (96.6%) studies. Clinically relevant metrics, such as qualitative, dosimetric and time-saving metrics, were less frequently used in 22 (18.8%), 27 (23.1%) and 18 (15.4%) of 117 studies, respectively. There was heterogeneity within each category of metric. Over 90 different names for geometric measures were used. Methods for qualitative assessment were different in all but two papers. Variation existed in the methods used to generate radiotherapy plans for dosimetric assessment. Consideration of editing time was only given in 11 (9.4%) papers. A single manual contour as a ground-truth comparator was used in 65 (55.6%) studies. Only 31 (26.5%) studies compared auto-contours to usual inter- and/or intra-observer variation. In conclusion, significant variation exists in how research papers currently assess the accuracy of automatically generated contours. Geometric measures are the most popular, however their clinical utility is unknown. There is heterogeneity in the methods used to perform clinical assessment. Considering the different stages of system implementation may provide a framework to decide the most appropriate metrics. This analysis supports the need for a consensus on the clinical implementation of auto-contouring.

Improved accuracy of auto-segmentation of organs at risk in radiotherapy planning for nasopharyngeal carcinoma based on fully convolutional neural network deep learning

OBJECTIVE

We examined a modified encoder-decoder architecture-based fully convolutional neural network, OrganNet, for simultaneous auto-segmentation of 24 organs at risk (OARs) in the head and neck, followed by validation tests and evaluation of clinical application.

MATERIALS AND METHODS

Computed tomography (CT) images from 310 radiotherapy plans were used as the experimental data set, of which 260 and 50 were used as the training and test sets, respectively. An improved U-Net architecture was established by introducing a batch normalization layer, residual squeeze-and-excitation layer, and unique organ-specific loss function for deep learning training. The performance of the trained network model was evaluated by comparing the manual-delineation and the STAPLE contour of 10 physicians from different centers.

RESULTS

Our model achieved good segmentation in all 24 OARs in nasopharyngeal cancer radiotherapy plan CT images, with an average Dice similarity coefficient of 83.75%. Specifically, the mean Dice coefficients in large-volume organs (brainstem, spinal cord, left/right parotid glands, left/right temporal lobes, and left/right mandibles) were 84.97% - 95.00%, and in small-volume organs (pituitary, lens, optic nerve, and optic chiasma) were 55.46% - 91.56%. respectively. Using the STAPLE contours as standard contour, the OrganNet achieved comparable or better DICE in organ segmentation then that of the manual-delineation as well.

CONCLUSION

The established OrganNet enables simultaneous automatic segmentation of multiple targets on CT images of the head and neck radiotherapy plans, effectively improves the accuracy of U-Net based segmentation for OARs, especially for small-volume organs.

Patient-specific daily updated deep learning auto-segmentation for MRI-guided adaptive radiotherapy

BACKGROUND AND PURPOSE

Deep Learning (DL) technique has shown great potential but still has limited success in online contouring for MR-guided adaptive radiotherapy (MRgART). This study proposed a patient-specific DL auto-segmentation (DLAS) strategy using the patient's previous images and contours to update the model and improve segmentation accuracy and efficiency for MRgART.

METHODS AND MATERIALS

A prototype model was trained for each patient using the first set of MRI and corresponding contours as inputs. The patient-specific model was updated after each fraction with all the available fractional MRIs/contours, and then used to predict the segmentation for the next fraction. During model training, a variant was fitted under consistency constraints, limiting the differences in the volume, length and centroid between the predictions for the latest MRI within a reasonable range. The model performance was evaluated for both organ-at-risks and tumors auto-segmentation for a total of 6 abdominal/pelvic cases (each with at least 8 sets of MRIs/contours) underwent MRgART through Dice Similarity Coefficient (DSC) and 95% Hausdorff Distance (HD95), and was compared with deformable image registration (DIR) and frozen DL model (no updating after pre-training). The contouring time was also recorded and analyzed.

RESULTS

The proposed model achieved superior performance with higher mean DSC (0.90, 95 % CI: 0.88-0.95), as compared to DIR (0.63, 95 %CI: 0.59-0.68) and frozen DL models (0.74, 95 % CI: 0.71-0.79). As for tumors, the proposed method yielded a median DSC of 0.95, 95 % CI: 0.94-0.97, and a median HD95 of 1.63 mm, 95 % CI: 1.22 mm-2.06 mm. The contouring time was reduced significantly (p < 0.05) using the proposed method (73.4 ± 6.5 secs) compared to the manual process (12 ∼ 22 mins). The online ART time was reduced to 1650 ± 274 seconds with the proposed method, as compared to 3251.8 ± 447 seconds using the original workflow.

CONCLUSION

The proposed patient-specific DLAS method can significantly improve the segmentation accuracy and efficiency for longitudinal MRIs, thereby facilitating the routine practice of MRgART.

A deep learning-based self-adapting ensemble method for segmentation in gynecological brachytherapy

Purpose

Fast and accurate outlining of the organs at risk (OARs) and high-risk clinical tumor volume (HRCTV) is especially important in high-dose-rate brachytherapy due to the highly time-intensive online treatment planning process and the high dose gradient around the HRCTV. This study aims to apply a self-configured ensemble method for fast and reproducible auto-segmentation of OARs and HRCTVs in gynecological cancer.

Materials and methods

We applied nnU-Net (no new U-Net), an automatically adapted deep convolutional neural network based on U-Net, to segment the bladder, rectum and HRCTV on CT images in gynecological cancer. In nnU-Net, three architectures, including 2D U-Net, 3D U-Net and 3D-Cascade U-Net, were trained and finally ensembled. 207 cases were randomly chosen for training, and 30 for testing. Quantitative evaluation used well-established image segmentation metrics, including dice similarity coefficient (DSC), 95% Hausdorff distance (HD95%), and average surface distance (ASD). Qualitative analysis of automated segmentation results was performed visually by two radiation oncologists. The dosimetric evaluation was performed by comparing the dose-volume parameters of both predicted segmentation and human contouring.

Results

nnU-Net obtained high qualitative and quantitative segmentation accuracy on the test dataset and performed better than previously reported methods in bladder and rectum segmentation. In quantitative evaluation, 3D-Cascade achieved the best performance in the bladder (DSC: 0.936 ± 0.051, HD95%: 3.503 ± 1.956, ASD: 0.944 ± 0.503), rectum (DSC: 0.831 ± 0.074, HD95%: 7.579 ± 5.857, ASD: 3.6 ± 3.485), and HRCTV (DSC: 0.836 ± 0.07, HD95%: 7.42 ± 5.023, ASD: 2.094 ± 1.311). According to the qualitative evaluation, over 76% of the test data set had no or minor visually detectable errors in segmentation.

Conclusion

This work showed nnU-Net’s superiority in segmenting OARs and HRCTV in gynecological brachytherapy cases in our center, among which 3D-Cascade shows the highest accuracy in segmentation across different applicators and patient anatomy.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13014-022-02121-3.

Artificial intelligence in radiotherapy

Radiotherapy is a discipline closely integrated with computer science. Artificial intelligence (AI) has developed rapidly over the past few years. With the explosive growth of medical big data, AI promises to revolutionize the field of radiotherapy through highly automated workflow, enhanced quality assurance, improved regional balances of expert experiences, and individualized treatment guided by multi-omics. In addition to independent researchers, the increasing number of large databases, biobanks, and open challenges significantly facilitated AI studies on radiation oncology. This article reviews the latest research, clinical applications, and challenges of AI in each part of radiotherapy including image processing, contouring, planning, quality assurance, motion management, and outcome prediction. By summarizing cutting-edge findings and challenges, we aim to inspire researchers to explore more future possibilities and accelerate the arrival of AI radiotherapy.

Clinical evaluation of deep learning-based clinical target volume three-channel auto-segmentation algorithm for adaptive radiotherapy in cervical cancer

Objectives

Accurate contouring of the clinical target volume (CTV) is a key element of radiotherapy in cervical cancer. We validated a novel deep learning (DL)-based auto-segmentation algorithm for CTVs in cervical cancer called the three-channel adaptive auto-segmentation network (TCAS).

Methods

A total of 107 cases were collected and contoured by senior radiation oncologists (ROs). Each case consisted of the following: (1) contrast-enhanced CT scan for positioning, (2) the related CTV, (3) multiple plain CT scans during treatment and (4) the related CTV. After registration between (1) and (3) for the same patient, the aligned image and CTV were generated. Method 1 is rigid registration, method 2 is deformable registration, and the aligned CTV is seen as the result. Method 3 is rigid registration and TCAS, method 4 is deformable registration and TCAS, and the result is generated by a DL-based method.

Results

From the 107 cases, 15 pairs were selected as the test set. The dice similarity coefficient (DSC) of method 1 was 0.8155 ± 0.0368; the DSC of method 2 was 0.8277 ± 0.0315; the DSCs of method 3 and 4 were 0.8914 ± 0.0294 and 0.8921 ± 0.0231, respectively. The mean surface distance and Hausdorff distance of methods 3 and 4 were markedly better than those of method 1 and 2.

Conclusions

The TCAS achieved comparable accuracy to the manual delineation performed by senior ROs and was significantly better than direct registration.

Machine Learning for Auto-Segmentation in Radiotherapy Planning

Manual segmentation of target structures and organs at risk is a crucial step in the radiotherapy workflow. It has the disadvantages that it can require several hours of clinician time per patient and is prone to inter- and intra-observer variability. Automatic segmentation (auto-segmentation), using computer algorithms, seeks to address these issues. Advances in machine learning and computer vision have led to the development of methods for accurate and efficient auto-segmentation. This review surveys auto-segmentation techniques and applications in radiotherapy planning. It provides an overview of traditional approaches to auto-segmentation, including intensity analysis, shape modelling and atlas-based methods. The focus, though, is on uses of machine learning and deep learning, including convolutional neural networks. Finally, the future of machine-learning-driven auto-segmentation in clinical settings is considered, and the barriers that must be overcome for it to be widely accepted into routine practice are highlighted.

Deep learning-based pancreas volume assessment in individuals with type 1 diabetes

Pancreas volume is reduced in individuals with diabetes and in autoantibody positive individuals at high risk for developing type 1 diabetes (T1D). Studies investigating pancreas volume are underway to assess pancreas volume in large clinical databases and studies, but manual pancreas annotation is time-consuming and subjective, preventing extension to large studies and databases. This study develops deep learning for automated pancreas volume measurement in individuals with diabetes. A convolutional neural network was trained using manual pancreas annotation on 160 abdominal magnetic resonance imaging (MRI) scans from individuals with T1D, controls, or a combination thereof. Models trained using each cohort were then tested on scans of 25 individuals with T1D. Deep learning and manual segmentations of the pancreas displayed high overlap (Dice coefficient = 0.81) and excellent correlation of pancreas volume measurements (R² = 0.94). Correlation was highest when training data included individuals both with and without T1D. The pancreas of individuals with T1D can be automatically segmented to measure pancreas volume. This algorithm can be applied to large imaging datasets to quantify the spectrum of human pancreas volume.

A novel semi auto-segmentation method for accurate dose and NTCP evaluation in adaptive head and neck radiotherapy

BACKGROUND AND PURPOSE

Accurate segmentation of organs-at-risk (OARs) is crucial but tedious and time-consuming in adaptive radiotherapy (ART). The purpose of this work was to automate head and neck OAR-segmentation on repeat CT (rCT) by an optimal combination of human and auto-segmentation for accurate prediction of Normal Tissue Complication Probability (NTCP).

MATERIALS AND METHODS

Human segmentation (HS) of 3 observers, deformable image registration (DIR) based contour propagation and deep learning contouring (DLC) were carried out to segment 15 OARs on 15 rCTs. The original treatment plan was re-calculated on rCT to obtain mean dose (D_mean) and consequent NTCP-predictions. The average D_mean and NTCP-predictions of the three observers were referred to as the gold standard to calculate the absolute difference of D_mean and NTCP-predictions (|ΔD_mean| and |ΔNTCP|).

RESULTS

The average |ΔD_mean| of parotid glands in HS was 1.40 Gy, lower than that obtained with DIR and DLC (3.64 Gy, p < 0.001 and 3.72 Gy, p < 0.001, respectively). DLC showed the highest |ΔD_mean| in middle Pharyngeal Constrictor Muscle (PCM) (5.13 Gy, p = 0.01). DIR showed second highest |ΔD_mean| in the cricopharyngeal inlet (2.85 Gy, p = 0.01). The semi auto-segmentation (SAS) adopted HS, DIR and DLC for segmentation of parotid glands, PCM and all other OARs, respectively. The 90th percentile |ΔNTCP|was 2.19%, 2.24%, 1.10% and 1.50% for DIR, DLC, HS and SAS respectively.

CONCLUSIONS

Human segmentation of the parotid glands remains necessary for accurate interpretation of mean dose and NTCP during ART. Proposed semi auto-segmentation allows NTCP-predictions within 1.5% accuracy for 90% of the cases.

Diagnosis of obstructive sleep apnea with prediction of flow characteristics according to airway morphology automatically extracted from medical images: Computational fluid dynamics and artificial intelligence approach

BACKGROUND

Obstructive sleep apnea syndrome (OSAS) is being observed in an increasing number of cases. It can be diagnosed using several methods such as polysomnography.

OBJECTIVES

To overcome the challenges of time and cost faced by conventional diagnostic methods, this paper proposes computational fluid dynamics (CFD) and machine-learning approaches that are derived from the upper-airway morphology with automatic segmentation using deep learning.

METHOD

We adopted a 3D UNet deep-learning model to perform medical image segmentation. 3D UNet prevents the feature-extraction loss that may occur by concatenating layers and extracts the anteroposterior coordination and width of the airway morphology. To create flow characteristics of the upper airway training data, we analyzed the changes in flow characteristics according to the upper-airway morphology using CFD. A multivariate Gaussian process regression (MVGPR) model was used to train the flow characteristic values. The trained MVGPR enables the prompt prediction of the aerodynamic features of the upper airway without simulation. Unlike conventional regression methods, MVGPR can be trained by considering the correlation between the flow characteristics. As a diagnostic step, a support vector machine (SVM) with predicted aerodynamic and biometric features was used in this study to classify patients as healthy or suffering from moderate OSAS. SVM is beneficial as it is easy to learn even with a small dataset, and it can diagnose various flow characteristics as factors while enhancing the feature via the kernel function. As the patient dataset is small, the Monte Carlo cross-validation was used to validate the trained model. Furthermore, to overcome the imbalanced data problem, the oversampling method was applied.

RESULT

The segmented upper-airway results of the high-resolution and low-resolution models present overall average dice coefficients of 0.76±0.041 and 0.74±0.052, respectively. Furthermore, the classification accuracy, sensitivity, specificity, and F1-score of the diagnosis algorithm were 81.5%, 89.3%, 86.2%, and 87.6%, respectively.

CONCLUSION

The convenience and accuracy of sleep apnea diagnosis are improved using deep learning and machine learning. Further, the proposed method can aid clinicians in making appropriate decisions to evaluate the possible applications of OSAS.

Prospects for daily online adaptive radiotherapy via ethos for prostate cancer patients without nodal involvement using unedited CBCT auto-segmentation

Purpose

 Implementing new online adaptive radiation therapy technologies is challenging because extra clinical resources are required particularly expert contour review. Here, we provide the first assessment of Varian's Ethos™ adaptive platform for prostate cancer using no manual edits after auto‐segmentation to minimize this impact on clinical efficiency.

Methods

Twenty‐five prostate patients previously treated at our clinic were re‐planned using an Ethos™ emulator. Clinical target volumes (CTV) included intact prostate and proximal seminal vesicles. The following clinical margins were used: 3 mm posterior, 5 mm left/right/anterior, and 7 mm superior/inferior. Adapted plans were calculated for 10 fractions per patient using Ethos's auto‐segmentation and auto‐planning workflow without manual contouring edits. Doses and auto‐segmented structures were exported to our clinical treatment planning system where contours were modified as needed for all 250 CTVs and organs‐at‐risk. Dose metrics from adapted plans were compared to unadapted plans to evaluate CTV and OAR dose changes.

Results

Overall 96% of fractions required auto‐segmentation edits, although corrections were generally minor (<10% of the volume for 70% of CTVs, 88% of bladders, and 90% of rectums). However, for one patient the auto‐segmented CTV failed to include the superior portion of prostate that extended into the bladder at all 10 fractions resulting in under‐contouring of the CTV by 31.3% ± 6.7%. For the 24 patients with minor auto‐segmentation corrections, adaptation improved CTV D98% by 2.9% ± 5.3%. For non‐adapted fractions where bladder or rectum V90% exceeded clinical thresholds, adaptation reduced them by 13.1% ± 1.0% and 6.5% ± 7.3%, respectively.

Conclusion

 For most patients, Ethos's online adaptive radiation therapy workflow improved CTV D98% and reduced normal tissue dose when structures would otherwise exceed clinical thresholds, even without time‐consuming manual edits. However, for one in 25 patients, large contour edits were required and thus scrutiny of the daily auto‐segmentation is necessary and not all patients will be good candidates for adaptation.

The dosimetric impact of deep learning-based auto-segmentation of organs at risk on nasopharyngeal and rectal cancer

Purpose

To investigate the dosimetric impact of deep learning-based auto-segmentation of organs at risk (OARs) on nasopharyngeal and rectal cancer.

Methods and materials

Twenty patients, including ten nasopharyngeal carcinoma (NPC) patients and ten rectal cancer patients, who received radiotherapy in our department were enrolled in this study. Two deep learning-based auto-segmentation systems, including an in-house developed system (FD) and a commercial product (UIH), were used to generate two auto-segmented OARs sets (OAR_FD and OAR_UIH). Treatment plans based on auto-segmented OARs and following our clinical requirements were generated for each patient on each OARs sets (Plan_FD and Plan_UIH). Geometric metrics (Hausdorff distance (HD), mean distance to agreement (MDA), the Dice similarity coefficient (DICE) and the Jaccard index) were calculated for geometric evaluation. The dosimetric impact was evaluated by comparing Plan_FD and Plan_UIH to original clinically approved plans (Plan_Manual) with dose-volume metrics and 3D gamma analysis. Spearman’s correlation analysis was performed to investigate the correlation between dosimetric difference and geometric metrics.

Results

FD and UIH could provide similar geometric performance in parotids, temporal lobes, lens, and eyes (DICE, p > 0.05). OAR_FD had better geometric performance in the optic nerves, oral cavity, larynx, and femoral heads (DICE, p < 0.05). OAR_UIH had better geometric performance in the bladder (DICE, p < 0.05). In dosimetric analysis, both Plan_FD and Plan_UIH had nonsignificant dosimetric differences compared to Plan_Manual for most PTV and OARs dose-volume metrics. The only significant dosimetric difference was the max dose of the left temporal lobe for Plan_FD vs. Plan_Manual (p = 0.05). Only one significant correlation was found between the mean dose of the femoral head and its HD index (R = 0.4, p = 0.01), there is no OARs showed strong correlation between its dosimetric difference and all of four geometric metrics.

Conclusions

Deep learning-based OARs auto-segmentation for NPC and rectal cancer has a nonsignificant impact on most PTV and OARs dose-volume metrics. Correlations between the auto-segmentation geometric metric and dosimetric difference were not observed for most OARs.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13014-021-01837-y.

A novel specific grading standard study of auto-segmentation of organs at risk in thorax: subjective-objective-combined grading standard

BACKGROUND

To develop a novel subjective-objective-combined (SOC) grading standard for auto-segmentation for each organ at risk (OAR) in the thorax.

METHODS

A radiation oncologist manually delineated 13 thoracic OARs from computed tomography (CT) images of 40 patients. OAR auto-segmentation accuracy was graded by five geometric objective indexes, including the Dice similarity coefficient (DSC), the difference of the Euclidean distance between centers of mass (ΔCMD), the difference of volume (ΔV), maximum Hausdorff distance (MHD), and average Hausdorff distance (AHD). The grading results were compared with those of the corresponding geometric indexes obtained by geometric objective methods in the other two centers. OAR auto-segmentation accuracy was also graded by our subjective evaluation standard. These grading results were compared with those of DSC. Based on the subjective evaluation standard and the five geometric indexes, the correspondence between the subjective evaluation level and the geometric index range was established for each OAR.

RESULTS

For ΔCMD, ΔV, and MHD, the grading results of the geometric objective evaluation methods at our center and the other two centers were inconsistent. For DSC and AHD, the grading results of three centers were consistent. Seven OARs' grading results in the subjective evaluation standard were inconsistent with those of DSC. Six OARs' grading results in the subjective evaluation standard were consistent with those of DSC. Finally, we proposed a new evaluation method that combined the subjective evaluation level of those OARs with the range of corresponding DSC to determine the grading standard. If the DSC ranges between the adjacent levels did not overlap, the DSC range was used as the grading standard. Otherwise, the mean value of DSC was used as the grading standard.

CONCLUSIONS

A novel OAR-specific SOC grading standard in thorax was developed. The SOC grading standard provides a possible alternative for evaluation of the auto-segmentation accuracy for thoracic OARs.

Metrics to evaluate the performance of auto-segmentation for radiation treatment planning: A critical review

Advances in artificial intelligence-based methods have led to the development and publication of numerous systems for auto-segmentation in radiotherapy. These systems have the potential to decrease contour variability, which has been associated with poor clinical outcomes and increased efficiency in the treatment planning workflow. However, there are no uniform standards for evaluating auto-segmentation platforms to assess their efficacy at meeting these goals. Here, we review the most frequently used evaluation techniques which include geometric overlap, dosimetric parameters, time spent contouring, and clinical rating scales. These data suggest that many of the most commonly used geometric indices, such as the Dice Similarity Coefficient, are not well correlated with clinically meaningful endpoints. As such, a multi-domain evaluation, including composite geometric and/or dosimetric metrics with physician-reported assessment, is necessary to gauge the clinical readiness of auto-segmentation for radiation treatment planning.

Evaluation on Auto-segmentation of the Clinical Target Volume (CTV) for Graves' Ophthalmopathy (GO) with a Fully Convolutional Network (FCN) on CT Images

CDATA[Purpose: The aim of this study is to evaluate the accuracy and dosimetric effects for auto- segmentation of the CTV for GO in CT images based on FCN.

METHODS

An FCN-8s network architecture for auto-segmentation was built based on Caffe. CT images of 121 patients with GO who have received radiotherapy at the West China Hospital of Sichuan University were randomly selected for training and testing. Two methods were used to segment the CTV of GO: treating the two-part CTV as a whole anatomical region or considering the two parts of CTV as two independent regions. Dice Similarity Coefficient (DSC) and Hausdorff Distance (HD) were used as evaluation criteria. The auto-segmented contours were imported into the original treatment plan to analyse the dosimetric characteristics.

RESULTS

The similarity comparison between manual contours and auto-segmental contours showed an average DSC value of up to 0.83. The max HD values for segmenting two parts of CTV separately was a little bit smaller than treating CTV with one label (8.23±2.80 vs. 9.03±2.78). The dosimetric comparison between manual contours and auto-segmental contours showed there was a significant difference (p<0.05) with the lack of dose for auto-segmental CTV.

CONCLUSION

Based on deep learning architecture, the automatic segmentation model for small target areas can carry out auto contouring tasks well. Treating separate parts of one target as different anatomic regions can help to improve the auto-contouring quality. The dosimetric evaluation can provide us with different perspectives for further exploration of automatic sketching tools.

Training deep-learning segmentation models from severely limited data

PURPOSE

To enable generation of high-quality deep learning segmentation models from severely limited contoured cases (e.g., ~10 cases).

METHODS

Thirty head and neck computed tomography (CT) scans with well-defined contours were deformably registered to 200 CT scans of the same anatomic site without contours. Acquired deformation vector fields were used to train a principal component analysis (PCA) model for each of the 30 contoured CT scans by capturing the mean deformation and most prominent variations. Each PCA model can produce an infinite number of synthetic CT scans and corresponding contours by applying random deformations. We used 300, 600, 1000, and 2000 synthetic CT scans and contours generated from one PCA model to train V-Net, a 3D convolutional neural network architecture, to segment parotid and submandibular glands. We repeated the training using same numbers of training cases generated from 7, 10, 20, and 30 PCA models, with the data distributed evenly between each PCA model. Performance of the segmentation models was evaluated with Dice similarity coefficients between auto-generated contours and physician-drawn contours on 162 test CT scans for parotid glands and another 21 test CT scans for submandibular glands.

RESULTS

Dice values varied with the number of synthetic CT scans and the number of PCA models used to train the network. By using 2000 synthetic CT scans generated from 10 PCA models, we achieved Dice values of 82.8% ± 6.8% for right parotid, 82.0% ± 6.9% for left parotid, and 74.2% ± 6.8% for submandibular glands. These results are comparable with those obtained from state-of-the-art auto-contouring approaches, including a deep learning network trained from more than 1000 contoured patients and a multi-atlas algorithm from 12 well-contoured atlases. Improvement was marginal when >10 PCA models or >2000 synthetic CT scans were used.

CONCLUSIONS

We demonstrated an effective data augmentation approach to train high-quality deep learning segmentation models from a limited number of well-contoured patient cases.

Clinical feasibility of deep learning-based auto-segmentation of target volumes and organs-at-risk in breast cancer patients after breast-conserving surgery

Background

In breast cancer patients receiving radiotherapy (RT), accurate target delineation and reduction of radiation doses to the nearby normal organs is important. However, manual clinical target volume (CTV) and organs-at-risk (OARs) segmentation for treatment planning increases physicians’ workload and inter-physician variability considerably. In this study, we evaluated the potential benefits of deep learning-based auto-segmented contours by comparing them to manually delineated contours for breast cancer patients.

Methods

CTVs for bilateral breasts, regional lymph nodes, and OARs (including the heart, lungs, esophagus, spinal cord, and thyroid) were manually delineated on planning computed tomography scans of 111 breast cancer patients who received breast-conserving surgery. Subsequently, a two-stage convolutional neural network algorithm was used. Quantitative metrics, including the Dice similarity coefficient (DSC) and 95% Hausdorff distance, and qualitative scoring by two panels from 10 institutions were used for analysis. Inter-observer variability and delineation time were assessed; furthermore, dose-volume histograms and dosimetric parameters were also analyzed using another set of patient data.

Results

The correlation between the auto-segmented and manual contours was acceptable for OARs, with a mean DSC higher than 0.80 for all OARs. In addition, the CTVs showed favorable results, with mean DSCs higher than 0.70 for all breast and regional lymph node CTVs. Furthermore, qualitative subjective scoring showed that the results were acceptable for all CTVs and OARs, with a median score of at least 8 (possible range: 0–10) for (1) the differences between manual and auto-segmented contours and (2) the extent to which auto-segmentation would assist physicians in clinical practice. The differences in dosimetric parameters between the auto-segmented and manual contours were minimal.

Conclusions

The feasibility of deep learning-based auto-segmentation in breast RT planning was demonstrated. Although deep learning-based auto-segmentation cannot be a substitute for radiation oncologists, it is a useful tool with excellent potential in assisting radiation oncologists in the future.

Trial registration Retrospectively registered.

atlas-based models for fast auto-segmentation of the masticatory muscles on head and neck CT images

BACKGROUND

Impaired function of masticatory muscles will lead to trismus. Routine delineation of these muscles during planning may improve dose tracking and facilitate dose reduction resulting in decreased radiation-related trismus. This study aimed to compare a deep learning model with a commercial atlas-based model for fast auto-segmentation of the masticatory muscles on head and neck computed tomography (CT) images.

MATERIAL AND METHODS

Paired masseter (M), temporalis (T), medial and lateral pterygoid (MP, LP) muscles were manually segmented on 56 CT images. CT images were randomly divided into training (n = 27) and validation (n = 29) cohorts. Two methods were used for automatic delineation of masticatory muscles (MMs): Deep learning auto-segmentation (DLAS) and atlas-based auto-segmentation (ABAS). The automatic algorithms were evaluated using Dice similarity coefficient (DSC), recall, precision, Hausdorff distance (HD), HD95, and mean surface distance (MSD). A consolidated score was calculated by normalizing the metrics against interobserver variability and averaging over all patients. Differences in dose (∆Dose) to MMs for DLAS and ABAS segmentations were assessed. A paired t-test was used to compare the geometric and dosimetric difference between DLAS and ABAS methods.

RESULTS

DLAS outperformed ABAS in delineating all MMs (p < 0.05). The DLAS mean DSC for M, T, MP, and LP ranged from 0.83 ± 0.03 to 0.89 ± 0.02, the ABAS mean DSC ranged from 0.79 ± 0.05 to 0.85 ± 0.04. The mean value for recall, HD, HD95, MSD also improved with DLAS for auto-segmentation. Interobserver variation revealed the highest variability in DSC and MSD for both T and MP, and the highest scores were achieved for T by both automatic algorithms. With few exceptions, the mean ∆D98%, ∆D95%, ∆D50%, and ∆D2% for all structures were below 10% for DLAS and ABAS and had no detectable statistical difference (P > 0.05). DLAS based contours had dose endpoints more closely matched with that of the manually segmented when compared with ABAS.

CONCLUSIONS

DLAS auto-segmentation of masticatory muscles for the head and neck radiotherapy had improved segmentation accuracy compared with ABAS with no qualitative difference in dosimetric endpoints compared to manually segmented contours.

Validation of automated magnetic resonance image segmentation for radiation therapy planning in prostate cancer

Background and purpose

Magnetic resonance imaging (MRI) is increasingly used in radiation therapy planning of prostate cancer (PC) to reduce target volume delineation uncertainty. This study aimed to assess and validate the performance of a fully automated segmentation tool (AST) in MRI based radiation therapy planning of PC.

Material and methods

Pelvic structures of 65 PC patients delineated in an MRI-only workflow according to established guidelines were included in the analysis. Automatic vs manual segmentation by an experienced oncologist was compared with geometrical parameters, such as the dice similarity coefficient (DSC). Fifteen patients had a second MRI within 15 days to assess repeatability of the AST for prostate and seminal vesicles. Furthermore, we investigated whether hormonal therapy or body mass index (BMI) affected the AST results.

Results

The AST showed high agreement with manual segmentation expressed as DSC (mean, SD) for delineating prostate (0.84, 0.04), bladder (0.92, 0.04) and rectum (0.86, 0.04). For seminal vesicles (0.56, 0.17) and penile bulb (0.69, 0.12) the respective agreement was moderate. Performance of AST was not influenced by neoadjuvant hormonal therapy, although those on treatment had significantly smaller prostates than the hormone-naïve patients (p < 0.0001). In repeat assessment, consistency of prostate delineation resulted in mean DSC of 0.89, (SD 0.03) between the paired MRI scans for AST, while mean DSC of manual delineation was 0.82, (SD 0.05).

Conclusion

Fully automated MRI segmentation tool showed good agreement and repeatability compared with manual segmentation and was found clinically robust in patients with PC. However, manual review and adjustment of some structures in individual cases remain important in clinical use.

Winter is over: The use of Artificial Intelligence to individualise radiation therapy for breast cancer

Artificial intelligence demonstrated its value for automated contouring of organs at risk and target volumes as well as for auto-planning of radiation dose distributions in terms of saving time, increasing consistency, and improving dose-volumes parameters. Future developments include incorporating dose/outcome data to optimise dose distributions with optimal coverage of the high-risk areas, while at the same time limiting doses to low-risk areas. An infinite gradient of volumes and doses to deliver spatially-adjusted radiation can be generated, allowing to avoid unnecessary radiation to organs at risk. Therefore, data about patient-, tumour-, and treatment-related factors have to be combined with dose distributions and outcome-containing databases.

Impact of 90Y PET gradient-based tumor segmentation on voxel-level dosimetry in liver radioembolization

BACKGROUND

The purpose was to validate ⁹⁰Y PET gradient-based tumor segmentation in phantoms and to evaluate the impact of the segmentation method on reported tumor absorbed dose (AD) and biological effective dose (BED) in ⁹⁰Y microsphere radioembolization (RE) patients. A semi-automated gradient-based method was applied to phantoms and patient tumors on the ⁹⁰Y PET with the initial bounding volume for gradient detection determined from a registered diagnostic CT or MR; this PET-based segmentation (PS) was compared with radiologist-defined morphologic segmentation (MS) on CT or MRI. AD and BED volume histogram metrics (D90, D70, mean) were calculated using both segmentations and concordance/correlations were investigated. Spatial concordance was assessed using Dice similarity coefficient (DSC) and mean distance to agreement (MDA). PS was repeated to assess intra-observer variability.

RESULTS

In phantoms, PS demonstrated high accuracy in lesion volumes (within 15%), AD metrics (within 11%), high spatial concordance relative to morphologic segmentation (DSC > 0.86 and MDA < 1.5 mm), and low intra-observer variability (DSC > 0.99, MDA < 0.2 mm, AD/BED metrics within 2%). For patients (58 lesions), spatial concordance between PS and MS was degraded compared to in-phantom (average DSC = 0.54, average MDA = 4.8 mm); the average mean tumor AD was 226 ± 153 and 197 ± 138 Gy, respectively for PS and MS. For patient AD metrics, the best Pearson correlation (r) and concordance correlation coefficient (ccc) between segmentation methods was found for mean AD (r = 0.94, ccc = 0.92), but worsened as the metric approached the minimum dose (for D90, r = 0.77, ccc = 0.69); BED metrics exhibited a similar trend. Patient PS showed low intra-observer variability (average DSC = 0.81, average MDA = 2.2 mm, average AD/BED metrics within 3.0%).

CONCLUSIONS

⁹⁰Y PET gradient-based segmentation led to accurate/robust results in phantoms, and showed high concordance with MS for reporting mean tumor AD/BED in patients. However, tumor coverage metrics such as D90 exhibited worse concordance between segmentation methods, highlighting the need to standardize segmentation methods when reporting AD/BED metrics from post-therapy ⁹⁰Y PET. Estimated differences in reported AD/BED metrics due to segmentation method will be useful for interpreting RE dosimetry results in the literature including tumor response data.