Vision 20/20: perspectives on automated image segmentation for radiotherapy

The dosimetric impact of deep learning-based auto-segmentation of organs at risk on nasopharyngeal and rectal cancer

Purpose

To investigate the dosimetric impact of deep learning-based auto-segmentation of organs at risk (OARs) on nasopharyngeal and rectal cancer.

Methods and materials

Twenty patients, including ten nasopharyngeal carcinoma (NPC) patients and ten rectal cancer patients, who received radiotherapy in our department were enrolled in this study. Two deep learning-based auto-segmentation systems, including an in-house developed system (FD) and a commercial product (UIH), were used to generate two auto-segmented OARs sets (OAR_FD and OAR_UIH). Treatment plans based on auto-segmented OARs and following our clinical requirements were generated for each patient on each OARs sets (Plan_FD and Plan_UIH). Geometric metrics (Hausdorff distance (HD), mean distance to agreement (MDA), the Dice similarity coefficient (DICE) and the Jaccard index) were calculated for geometric evaluation. The dosimetric impact was evaluated by comparing Plan_FD and Plan_UIH to original clinically approved plans (Plan_Manual) with dose-volume metrics and 3D gamma analysis. Spearman’s correlation analysis was performed to investigate the correlation between dosimetric difference and geometric metrics.

Results

FD and UIH could provide similar geometric performance in parotids, temporal lobes, lens, and eyes (DICE, p > 0.05). OAR_FD had better geometric performance in the optic nerves, oral cavity, larynx, and femoral heads (DICE, p < 0.05). OAR_UIH had better geometric performance in the bladder (DICE, p < 0.05). In dosimetric analysis, both Plan_FD and Plan_UIH had nonsignificant dosimetric differences compared to Plan_Manual for most PTV and OARs dose-volume metrics. The only significant dosimetric difference was the max dose of the left temporal lobe for Plan_FD vs. Plan_Manual (p = 0.05). Only one significant correlation was found between the mean dose of the femoral head and its HD index (R = 0.4, p = 0.01), there is no OARs showed strong correlation between its dosimetric difference and all of four geometric metrics.

Conclusions

Deep learning-based OARs auto-segmentation for NPC and rectal cancer has a nonsignificant impact on most PTV and OARs dose-volume metrics. Correlations between the auto-segmentation geometric metric and dosimetric difference were not observed for most OARs.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13014-021-01837-y.

Investigation of clinical target volume segmentation for whole breast irradiation using three-dimensional convolutional neural networks with gradient-weighted class activation mapping

This study aims to implement three-dimensional convolutional neural networks (3D-CNN) for clinical target volume (CTV) segmentation for whole breast irradiation and investigate the focus of 3D-CNNs during decision-making using gradient-weighted class activation mapping (Grad-CAM). A 3D-UNet CNN was adopted to conduct automatic segmentation of the CTV for breast cancer. The 3D-UNet was trained using three datasets of left-, right-, and both left- and right-sided breast cancer patients. Segmentation accuracy was evaluated using the Dice similarity coefficient (DSC). Grad-CAM was applied to trained CNNs. The DSCs for the datasets of the left-, right-, and both left- and right-sided breasts were on an average 0.88, 0.89, and 0.85, respectively. The Grad-CAM heatmaps showed that the 3D-UNet used for segmentation determined the CTV region from the target-side breast tissue and by referring to the opposite-side breast. Although the size of the dataset was limited, DSC ≥ 0.85 was achieved for the segmentation of breast CTV using the 3D-UNet. Grad-CAM indicates the applicable scope and limitations of using a CNN by indicating the focus of such networks during decision-making.

A novel specific grading standard study of auto-segmentation of organs at risk in thorax: subjective-objective-combined grading standard

BACKGROUND

To develop a novel subjective-objective-combined (SOC) grading standard for auto-segmentation for each organ at risk (OAR) in the thorax.

METHODS

A radiation oncologist manually delineated 13 thoracic OARs from computed tomography (CT) images of 40 patients. OAR auto-segmentation accuracy was graded by five geometric objective indexes, including the Dice similarity coefficient (DSC), the difference of the Euclidean distance between centers of mass (ΔCMD), the difference of volume (ΔV), maximum Hausdorff distance (MHD), and average Hausdorff distance (AHD). The grading results were compared with those of the corresponding geometric indexes obtained by geometric objective methods in the other two centers. OAR auto-segmentation accuracy was also graded by our subjective evaluation standard. These grading results were compared with those of DSC. Based on the subjective evaluation standard and the five geometric indexes, the correspondence between the subjective evaluation level and the geometric index range was established for each OAR.

RESULTS

For ΔCMD, ΔV, and MHD, the grading results of the geometric objective evaluation methods at our center and the other two centers were inconsistent. For DSC and AHD, the grading results of three centers were consistent. Seven OARs' grading results in the subjective evaluation standard were inconsistent with those of DSC. Six OARs' grading results in the subjective evaluation standard were consistent with those of DSC. Finally, we proposed a new evaluation method that combined the subjective evaluation level of those OARs with the range of corresponding DSC to determine the grading standard. If the DSC ranges between the adjacent levels did not overlap, the DSC range was used as the grading standard. Otherwise, the mean value of DSC was used as the grading standard.

CONCLUSIONS

A novel OAR-specific SOC grading standard in thorax was developed. The SOC grading standard provides a possible alternative for evaluation of the auto-segmentation accuracy for thoracic OARs.

Head and neck multi-organ segmentation on dual-energy CT using dual pyramid convolutional neural networks

Organ delineation is crucial to diagnosis and therapy, while it is also labor-intensive and observer-dependent. Dual energy CT (DECT) provides additional image contrast than conventional single energy CT (SECT), which may facilitate automatic organ segmentation. This work aims to develop an automatic multi-organ segmentation approach using deep learning for head-and-neck region on DECT. We proposed a mask scoring regional convolutional neural network (R-CNN) where comprehensive features are firstly learnt from two independent pyramid networks and are then combined via deep attention strategy to highlight the informative ones extracted from both two channels of low and high energy CT. To perform multi-organ segmentation and avoid misclassification, a mask scoring subnetwork was integrated into the Mask R-CNN framework to build the correlation between the class of potential detected organ's region-of-interest (ROI) and the shape of that organ's segmentation within that ROI. We evaluated our model on DECT images from 127 head-and-neck cancer patients (66 training, 61 testing) with manual contours of 19 organs as training target and ground truth. For large- and mid-sized organs such as brain and parotid, the proposed method successfully achieved average Dice similarity coefficient (DSC) larger than 0.8. For small-sized organs with very low contrast such as chiasm, cochlea, lens and optic nerves, the DSCs ranged between around 0.5 and 0.8. With the proposed method, using DECT images outperforms using SECT in almost all 19 organs with statistical significance in DSC (p<0.05). Meanwhile, by using the DECT, the proposed method is also significantly superior to a recently developed FCN-based method in most of organs in terms of DSC and the 95th percentile Hausdorff distance. Quantitative results demonstrated the feasibility of the proposed method, the superiority of using DECT to SECT, and the advantage of the proposed R-CNN over FCN on the head-and-neck patient study. The proposed method has the potential to facilitate the current head-and-neck cancer radiation therapy workflow in treatment planning.

Automated atlas-based segmentation for skull base surgical planning

PURPOSE

Computational surgical planning tools could help develop novel skull base surgical approaches that improve safety and patient outcomes. This defines a need for automated skull base segmentation to improve the usability of surgical planning software. The objective of this work was to design and validate an algorithm for atlas-based automated segmentation of skull base structures in individual image sets for skull base surgical planning.

METHODS

Advanced Normalization Tools software was used to construct a synthetic CT template from 6 subjects, and skull base structures were manually segmented to create a reference atlas. Landmark registration followed by Elastix deformable registration was applied to the template to register it to each of the 30 trusted reference image sets. Dice coefficient, average Hausdorff distance, and clinical usability scoring were used to compare the atlas segmentations to those of the trusted reference image sets.

RESULTS

The mean for average Hausdorff distance for all structures was less than 2 mm (mean for 95th percentile Hausdorff distance was less than 5 mm). For structures greater than 2.5 mL in volume, the average Dice coefficient was 0.73 (range 0.59-0.82), and for structures less than 2.5 mL in volume the Dice coefficient was less than 0.7. The usability scoring survey was completed by three experts, and all structures met the criteria for acceptable effort except for the foramen spinosum, rotundum, and carotid artery, which required more than minor corrections.

CONCLUSION

Currently available open-source algorithms, such as the Elastix deformable algorithm, can be used for automated atlas-based segmentation of skull base structures with acceptable clinical accuracy and minimal corrections with the use of the proposed atlas. The first publicly available CT template and anterior skull base segmentation atlas being released (available at this link: http://hdl.handle.net/1773/46259 ) with this paper will allow for general use of automated atlas-based segmentation of the skull base.

A Preliminary Experience of Implementing Deep-Learning Based Auto-Segmentation in Head and Neck Cancer: A Study on Real-World Clinical Cases

Purpose

While artificial intelligence has shown great promise in organs-at-risk (OARs) auto segmentation for head and neck cancer (HNC) radiotherapy, to reach the level of clinical acceptance of this technology in real-world routine practice is still a challenge. The purpose of this study was to validate a U-net-based full convolutional neural network (CNN) for the automatic delineation of OARs of HNC, focusing on clinical implementation and evaluation.

Methods

In the first phase, the CNN was trained on 364 clinical HNC patients’ CT images with annotated contouring from routine clinical cases by different oncologists. The automated delineation accuracy was quantified using the Dice similarity coefficient (DSC) and 95% Hausdorff distance (HD). To assess efficiency, the time required to edit the auto-contours to a clinically acceptable standard was evaluated by a questionnaire. For subjective evaluation, expert oncologists (more than 10 years’ experience) were randomly presented with automated delineations or manual contours of 15 OARs for 30 patient cases. In the second phase, the network was retrained with an additional 300 patients, which were generated by pre-trained CNN and edited by oncologists until to meet clinical acceptance.

Results

Based on DSC, the CNN performed best for the spinal cord, brainstem, temporal lobe, eyes, optic nerve, parotid glands and larynx (DSC >0.7). Higher conformity for the OARs delineation was achieved by retraining our architecture, largest DSC improvement on oral cavity (0.53 to 0.93). Compared with the manual delineation time, after using auto-contouring, this duration was significantly shortened from hours to minutes. In the subjective evaluation, two observes showed an apparent inclination on automatic OARs contouring, even for relatively low DSC values. Most of the automated OARs segmentation can reach the clinical acceptance level compared to manual delineations.

Conclusions

After retraining, the CNN developed for OARs automated delineation in HNC was proved to be more robust, efficiency and consistency in clinical practice. Deep learning-based auto-segmentation shows great potential to alleviate the labor-intensive contouring of OAR for radiotherapy treatment planning.

Artificial intelligence in radiation oncology

Artificial intelligence (AI) is a computer science that tries to mimic human-like intelligence in machines that use computer software and algorithms to perform specific tasks without direct human input. Machine learning (ML) is a subunit of AI that uses data-driven algorithms that learn to imitate human behavior based on a previous example or experience. Deep learning is an ML technique that uses deep neural networks to create a model. The growth and sharing of data, increasing computing power, and developments in AI have initiated a transformation in healthcare. Advances in radiation oncology have produced a significant amount of data that must be integrated with computed tomography imaging, dosimetry, and imaging performed before each fraction. Of the many algorithms used in radiation oncology, has advantages and limitations with different computational power requirements. The aim of this review is to summarize the radiotherapy (RT) process in workflow order by identifying specific areas in which quality and efficiency can be improved by ML. The RT stage is divided into seven stages: patient evaluation, simulation, contouring, planning, quality control, treatment application, and patient follow-up. A systematic evaluation of the applicability, limitations, and advantages of AI algorithms has been done for each stage.

Male pelvic CT multi-organ segmentation using synthetic MRI-aided dual pyramid networks

The delineation of the prostate and organs-at-risk (OARs) is fundamental to prostate radiation treatment planning, but is currently labor-intensive and observer-dependent. We aimed to develop an automated computed tomography (CT)-based multi-organ (bladder, prostate, rectum, left and right femoral heads (RFHs)) segmentation method for prostate radiation therapy treatment planning. The proposed method uses synthetic MRIs (sMRIs) to offer superior soft-tissue information for male pelvic CT images. Cycle-consistent adversarial networks (CycleGAN) were used to generate CT-based sMRIs. Dual pyramid networks (DPNs) extracted features from both CTs and sMRIs. A deep attention strategy was integrated into the DPNs to select the most relevant features from both CTs and sMRIs to identify organ boundaries. The CT-based sMRI generated from our previously trained CycleGAN and its corresponding CT images were inputted to the proposed DPNs to provide complementary information for pelvic multi-organ segmentation. The proposed method was trained and evaluated using datasets from 140 patients with prostate cancer, and were then compared against state-of-art methods. The Dice similarity coefficients and mean surface distances between our results and ground truth were 0.95 ± 0.05, 1.16 ± 0.70 mm; 0.88 ± 0.08, 1.64 ± 1.26 mm; 0.90 ± 0.04, 1.27 ± 0.48 mm; 0.95 ± 0.04, 1.08 ± 1.29 mm; and 0.95 ± 0.04, 1.11 ± 1.49 mm for bladder, prostate, rectum, left and RFHs, respectively. Mean center of mass distances was within 3 mm for all organs. Our results performed significantly better than those of competing methods in most evaluation metrics. We demonstrated the feasibility of sMRI-aided DPNs for multi-organ segmentation on pelvic CT images, and its superiority over other networks. The proposed method could be used in routine prostate cancer radiotherapy treatment planning to rapidly segment the prostate and standard OARs.

Segmentation evaluation with sparse ground truth data: Simulating true segmentations as perfect/imperfect as those generated by humans

Fully annotated data sets play important roles in medical image segmentation and evaluation. Expense and imprecision are the two main issues in generating ground truth (GT) segmentations. In this paper, in an attempt to overcome these two issues jointly, we propose a method, named SparseGT, which exploit variability among human segmenters to maximally save manual workload in GT generation for evaluating actual segmentations by algorithms. Pseudo ground truth (p-GT) segmentations are created by only a small fraction of workload and with human-level perfection/imperfection, and they can be used in practice as a substitute for fully manual GT in evaluating segmentation algorithms at the same precision. p-GT segmentations are generated by first selecting slices sparsely, where manual contouring is conducted only on these sparse slices, and subsequently filling segmentations on other slices automatically. By creating p-GT with different levels of sparseness, we determine the largest workload reduction achievable for each considered object, where the variability of the generated p-GT is statistically indistinguishable from inter-segmenter differences in full manual GT segmentations for that object. Furthermore, we investigate the segmentation evaluation errors introduced by variability in manual GT by applying p-GT in evaluation of actual segmentations by an algorithm. Experiments are conducted on ∼500 computed tomography (CT) studies involving six objects in two body regions, Head & Neck and Thorax, where optimal sparseness and corresponding evaluation errors are determined for each object and each strategy. Our results indicate that creating p-GT by the concatenated strategy of uniformly selecting sparse slices and filling segmentations via deep-learning (DL) network show highest manual workload reduction by ∼80-96% without sacrificing evaluation accuracy compared to fully manual GT. Nevertheless, other strategies also have obvious contributions in different situations. A non-uniform strategy for slice selection shows its advantage for objects with irregular shape change from slice to slice. An interpolation strategy for filling segmentations can achieve ∼60-90% of workload reduction in simulating human-level GT without the need of an actual training stage and shows potential in enlarging data sets for training p-GT generation networks. We conclude that not only over 90% reduction in workload is feasible without sacrificing evaluation accuracy but also the suitable strategy and the optimal sparseness level achievable for creating p-GT are object- and application-specific.

Ethical evaluation of artificial intelligence applications in radiotherapy using the Four Topics Approach

Artificial Intelligence is the capability of a machine to imitate intelligent human behavior. An important impact can be expected from Artificial Intelligence throughout the workflow of radiotherapy (such as automated organ segmentation, treatment planning, prediction of outcome and quality assurance). However, ethical concerns regarding the binding agreement between the patient and the physician have followed the introduction of artificial intelligence. Through the recording of personal and social moral values in addition to the usual demographics and the implementation of these as distinctive inputs to matching algorithms, ethical concerns such as consistency, applicability and relevance can be solved. In the meantime, physicians' awareness of the ethical dimension in their decision-making should be challenged, so that they prioritize treating their patients and not diseases, remain vigilant to preserve patient safety, avoid unintended harm and establish institutional policies on these issues.

Automated approach for segmenting gross tumor volumes for lung cancer stereotactic body radiation therapy using CT-based dense V-networks

The aim of this study was to develop an automated segmentation approach for small gross tumor volumes (GTVs) in 3D planning computed tomography (CT) images using dense V-networks (DVNs) that offer more advantages in segmenting smaller structures than conventional V-networks. Regions of interest (ROI) with dimensions of 50 × 50 × 6-72 pixels in the planning CT images were cropped based on the GTV centroids when applying stereotactic body radiotherapy (SBRT) to patients. Segmentation accuracy of GTV contours for 192 lung cancer patients [with the following tumor types: 118 solid, 53 part-solid types and 21 pure ground-glass opacity (pure GGO)], who underwent SBRT, were evaluated based on a 10-fold cross-validation test using Dice's similarity coefficient (DSC) and Hausdorff distance (HD). For each case, 11 segmented GTVs consisting of three single outputs, four logical AND outputs, and four logical OR outputs from combinations of two or three outputs from DVNs were obtained by three runs with different initial weights. The AND output (combination of three outputs) achieved the highest values of average 3D-DSC (0.832 ± 0.074) and HD (4.57 ± 2.44 mm). The average 3D DSCs from the AND output for solid, part-solid and pure GGO types were 0.838 ± 0.074, 0.822 ± 0.078 and 0.819 ± 0.059, respectively. This study suggests that the proposed approach could be useful in segmenting GTVs for planning lung cancer SBRT.

Head and neck cancer patient images for determining auto-segmentation accuracy in T2-weighted magnetic resonance imaging through expert manual segmentations

PURPOSE

The use of magnetic resonance imaging (MRI) in radiotherapy treatment planning has rapidly increased due to its ability to evaluate patient's anatomy without the use of ionizing radiation and due to its high soft tissue contrast. For these reasons, MRI has become the modality of choice for longitudinal and adaptive treatment studies. Automatic segmentation could offer many benefits for these studies. In this work, we describe a T2-weighted MRI dataset of head and neck cancer patients that can be used to evaluate the accuracy of head and neck normal tissue auto-segmentation systems through comparisons to available expert manual segmentations.

ACQUISITION AND VALIDATION METHODS

T2-weighted MRI images were acquired for 55 head and neck cancer patients. These scans were collected after radiotherapy computed tomography (CT) simulation scans using a thermoplastic mask to replicate patient treatment position. All scans were acquired on a single 1.5 T Siemens MAGNETOM Aera MRI with two large four-channel flex phased-array coils. The scans covered the region encompassing the nasopharynx region cranially and supraclavicular lymph node region caudally, when possible, in the superior-inferior direction. Manual contours were created for the left/right submandibular gland, left/right parotids, left/right lymph node level II, and left/right lymph node level III. These contours underwent quality assurance to ensure adherence to predefined guidelines, and were corrected if edits were necessary.

DATA FORMAT AND USAGE NOTES

The T2-weighted images and RTSTRUCT files are available in DICOM format. The regions of interest are named based on AAPM's Task Group 263 nomenclature recommendations (Glnd_Submand_L, Glnd_Submand_R, LN_Neck_II_L, Parotid_L, Parotid_R, LN_Neck_II_R, LN_Neck_III_L, LN_Neck_III_R). This dataset is available on The Cancer Imaging Archive (TCIA) by the National Cancer Institute under the collection "AAPM RT-MAC Grand Challenge 2019" (https://doi.org/10.7937/tcia.2019.bcfjqfqb).

POTENTIAL APPLICATIONS

This dataset provides head and neck patient MRI scans to evaluate auto-segmentation systems on T2-weighted images. Additional anatomies could be provided at a later time to enhance the existing library of contours.

Spatiotemporal Free-Form Registration Method Assisted by a Minimum Spanning Tree During Discontinuous Transformations

The sliding motion along the boundaries of discontinuous regions has been actively studied in B-spline free-form deformation framework. This study focusses on the sliding motion for a velocity field-based 3D+t registration. The discontinuity of the tangent direction guides the deformation of the object region, and a separate control of two regions provides a better registration accuracy. The sliding motion under the velocity field-based transformation is conducted under the [Formula: see text]-Rényi entropy estimator using a minimum spanning tree (MST) topology. Moreover, a new topology changing method of the MST is proposed. The topology change is performed as follows: inserting random noise, constructing the MST, and removing random noise while preserving a local connection consistency of the MST. This random noise process (RNP) prevents the [Formula: see text]-Rényi entropy-based registration from degrading in sliding motion, because the RNP creates a small disturbance around special locations. Experiments were performed using two publicly available datasets: the DIR-Lab dataset, which consists of 4D pulmonary computed tomography (CT) images, and a benchmarking framework dataset for cardiac 3D ultrasound. For the 4D pulmonary CT images, RNP produced a significantly improved result for the original MST with sliding motion (p<0.05). For the cardiac 3D ultrasound dataset, only a discontinuity-based registration indicated activity of the RNP. In contrast, the single MST without sliding motion did not show any improvement. These experiments proved the effectiveness of the RNP for sliding motion.

A slice classification model-facilitated 3D encoder-decoder network for segmenting organs at risk in head and neck cancer

For deep learning networks used to segment organs at risk (OARs) in head and neck (H&N) cancers, the class-imbalance problem between small volume OARs and whole computed tomography (CT) images results in delineation with serious false-positives on irrelevant slices and unnecessary time-consuming calculations. To alleviate this problem, a slice classification model-facilitated 3D encoder-decoder network was developed and validated. In the developed two-step segmentation model, a slice classification model was firstly utilized to classify CT slices into six categories in the craniocaudal direction. Then the target categories for different OARs were pushed to the different 3D encoder-decoder segmentation networks, respectively. All the patients were divided into training (n = 120), validation (n = 30) and testing (n = 20) datasets. The average accuracy of the slice classification model was 95.99%. The Dice similarity coefficient and 95% Hausdorff distance, respectively, for each OAR were as follows: right eye (0.88 ± 0.03 and 1.57 ± 0.92 mm), left eye (0.89 ± 0.03 and 1.35 ± 0.43 mm), right optic nerve (0.72 ± 0.09 and 1.79 ± 1.01 mm), left optic nerve (0.73 ± 0.09 and 1.60 ± 0.71 mm), brainstem (0.87 ± 0.04 and 2.28 ± 0.99 mm), right temporal lobe (0.81 ± 0.12 and 3.28 ± 2.27 mm), left temporal lobe (0.82 ± 0.09 and 3.73 ± 2.08 mm), right temporomandibular joint (0.70 ± 0.13 and 1.79 ± 0.79 mm), left temporomandibular joint (0.70 ± 0.16 and 1.98 ± 1.48 mm), mandible (0.89 ± 0.02 and 1.66 ± 0.51 mm), right parotid (0.77 ± 0.07 and 7.30 ± 4.19 mm) and left parotid (0.71 ± 0.12 and 8.41 ± 4.84 mm). The total segmentation time was 40.13 s. The 3D encoder-decoder network facilitated by the slice classification model demonstrated superior performance in accuracy and efficiency in segmenting OARs in H&N CT images. This may significantly reduce the workload for radiation oncologists.

Deforming to Best Practice: Key considerations for deformable image registration in radiotherapy

Image registration is a process that underlies many new techniques in radiation oncology - from multimodal imaging and contour propagation in treatment planning to dose accumulation throughout treatment. Deformable image registration (DIR) is a subset of image registration subject to high levels of complexity in process and validation. A need for local guidance to assist in high-quality utilisation and best practice was identified within the Australian community, leading to collaborative activity and workshops. This report communicates the current limitations and best practice advice from early adopters to help guide those implementing DIR in the clinic at this early stage. They are based on the state of image registration applications in radiotherapy in Australia and New Zealand (ANZ), and consensus discussions made at the 'Deforming to Best Practice' workshops in 2018. The current status of clinical application use cases is presented, including multimodal imaging, automatic segmentation, adaptive radiotherapy, retreatment, dose accumulation and response assessment, along with uptake, accuracy and limitations. Key areas of concern and preliminary suggestions for commissioning, quality assurance, education and training, and the use of automation are also reported. Many questions remain, and the radiotherapy community will benefit from continued research in this area. However, DIR is available to clinics and this report is intended to aid departments using or about to use DIR tools now.

Evaluation of a cycle-generative adversarial network-based cone-beam CT to synthetic CT conversion algorithm for adaptive radiation therapy

PURPOSE

Image-guided radiation therapy could benefit from implementing adaptive radiation therapy (ART) techniques. A cycle-generative adversarial network (cycle-GAN)-based cone-beam computed tomography (CBCT)-to-synthetic CT (sCT) conversion algorithm was evaluated regarding image quality, image segmentation and dosimetric accuracy for head and neck (H&N), thoracic and pelvic body regions.

METHODS

Using a cycle-GAN, three body site-specific models were priorly trained with independent paired CT and CBCT datasets of a kV imaging system (XVI, Elekta). sCT were generated based on first-fraction CBCT for 15 patients of each body region. Mean errors (ME) and mean absolute errors (MAE) were analyzed for the sCT. On the sCT, manually delineated structures were compared to deformed structures from the planning CT (pCT) and evaluated with standard segmentation metrics. Treatment plans were recalculated on sCT. A comparison of clinically relevant dose-volume parameters (D₉₈, D₅₀ and D₂ of the target volume) and 3D-gamma (3%/3mm) analysis were performed.

RESULTS

The mean ME and MAE were 1.4, 29.6, 5.4 Hounsfield units (HU) and 77.2, 94.2, 41.8 HU for H&N, thoracic and pelvic region, respectively. Dice similarity coefficients varied between 66.7 ± 8.3% (seminal vesicles) and 94.9 ± 2.0% (lungs). Maximum mean surface distances were 6.3 mm (heart), followed by 3.5 mm (brainstem). The mean dosimetric differences of the target volumes did not exceed 1.7%. Mean 3D gamma pass rates greater than 97.8% were achieved in all cases.

CONCLUSIONS

The presented method generates sCT images with a quality close to pCT and yielded clinically acceptable dosimetric deviations. Thus, an important prerequisite towards clinical implementation of CBCT-based ART is fulfilled.

Methodological approach to create an atlas using a commercial auto-contouring software

PURPOSE

The aim of this work was to establish a methodological approach for creation and optimization of an atlas for auto-contouring, using the commercial software MIM MAESTRO (MIM Software Inc. Cleveland OH).

METHODS

A computed tomography (CT) male pelvis atlas was created and optimized to evaluate how different tools and options impact on the accuracy of automatic segmentation. Pelvic lymph nodes (PLN), rectum, bladder, and femurs of 55 subjects were reviewed for consistency by a senior consultant radiation oncologist with 15 yr of experience. Several atlas and workflow options were tuned to optimize the accuracy of auto-contours. The deformable image registration (DIR), the finalization method, the k number of atlas best matching subjects, and several post-processing options were studied. To test our atlas performances, automatic and reference manual contours of 20 test subjects were statistically compared based on dice similarity coefficient (DSC) and mean distance to agreement (MDA) indices. The effect of field of view (FOV) reduction on auto-contouring time was also investigated.

RESULTS

With the optimized atlas and workflow, DSC and MDA median values of bladder, rectum, PLN, and femurs were 0.91 and 1.6 mm, 0.85 and 1.6 mm, 0.85 and 1.8 mm, and 0.96 and 0.5 mm, respectively. Auto-contouring time was more than halved by strictly cropping the FOV of the subject to be contoured to the pelvic region.

CONCLUSION

A statistically significant improvement of auto-contours accuracy was obtained using our atlas and optimized workflow instead of the MIM Software pelvic atlas.

Atlas Sampling for Prone Breast Automatic Segmentation of Organs at Risk: The Importance of Patients' Body Mass Index and Breast Cup Size for an Optimized Contouring of the Heart and the Coronary Vessels

Objective:

Delineation of organs at risk is a time-consuming task. This study evaluates the benefits of using single-subject atlas-based automatic segmentation of organs at risk in patients with breast cancer treated in prone position, with 2 different criteria for choosing the atlas subject. Together with laterality (left/right), the criteria used were either (1) breast volume or (2) body mass index and breast cup size.

Methods:

An atlas supporting different selection criteria for automatic segmentation was generated from contours drawn by a senior radiation oncologist (RO_A). Atlas organs at risk included heart, left anterior descending artery, and right coronary artery. Manual contours drawn by RO_A and automatic segmentation contours of organs at risk and breast clinical target volume were created for 27 nonatlas patients. A second radiation oncologist (RO_B) manually contoured (M_B) the breast clinical target volume and the heart. Contouring times were recorded and the reliability of the automatic segmentation was assessed in the context of 3-D planning.

Results:

Accounting for body mass index and breast cup size improved automatic segmentation results compared to breast volume-based sampling, especially for the heart (mean similarity indexes >0.9 for automatic segmentation organs at risk and clinical target volume after RO_A editing). Mean similarity indexes for the left anterior descending artery and the right coronary artery edited by RO_A expanded by 1 cm were ≥0.8. Using automatic segmentation reduced contouring time by 40%. For each parameter analyzed (eg, D_2%), the difference in dose, averaged over all patients, between automatic segmentation structures edited by RO_A and the same structure manually drawn by RO_A was <1.5% of the prescribed dose. The mean heart dose was reliable for the unedited heart segmentation, and for right-sided treatments, automatic segmentation was adequate for treatment planning with 3-D conformal tangential fields.

Conclusions:

Automatic segmentation for prone breast radiotherapy stratified by body mass index and breast cup size improved segmentation accuracy for the heart and coronary vessels compared to breast volume sampling. A significant reduction in contouring time can be achieved by using automatic segmentation.

A Deep Learning-Based Automated CT Segmentation of Prostate Cancer Anatomy for Radiation Therapy Planning-A Retrospective Multicenter Study

A commercial deep learning (DL)-based automated segmentation tool (AST) for computed tomography (CT) is evaluated for accuracy and efficiency gain within prostate cancer patients. Thirty patients from six clinics were reviewed with manual- (MC), automated- (AC) and automated and edited (AEC) contouring methods. In the AEC group, created contours (prostate, seminal vesicles, bladder, rectum, femoral heads and penile bulb) were edited, whereas the MC group included empty datasets for MC. In one clinic, lymph node CTV delineations were evaluated for interobserver variability. Compared to MC, the mean time saved using the AST was 12 min for the whole data set (46%) and 12 min for the lymph node CTV (60%), respectively. The delineation consistency between MC and AEC groups according to the Dice similarity coefficient (DSC) improved from 0.78 to 0.94 for the whole data set and from 0.76 to 0.91 for the lymph nodes. The mean DSCs between MC and AC for all six clinics were 0.82 for prostate, 0.72 for seminal vesicles, 0.93 for bladder, 0.84 for rectum, 0.69 for femoral heads and 0.51 for penile bulb. This study proves that using a general DL-based AST for CT images saves time and improves consistency.

An Automated Workflow to Improve Efficiency in Radiation Therapy Treatment Planning by Prioritizing Organs at Risk

PURPOSE

Manual delineation (MD) of organs at risk (OAR) is time and labor intensive. Auto-delineation (AD) can reduce the need for MD, but because current algorithms are imperfect, manual review and modification is still typically used. Recognizing that many OARs are sufficiently far from important dose levels that they do not pose a realistic risk, we hypothesize that some OARs can be excluded from MD and manual review with no clinical effect. The purpose of this study was to develop a method that automatically identifies these OARs and enables more efficient workflows that incorporate AD without degrading clinical quality.

METHODS AND MATERIALS

Preliminary dose map estimates were generated for n = 10 patients with head and neck cancers using only prescription and target-volume information. Conservative estimates of clinical OAR objectives were computed using AD structures with spatial expansion buffers to account for potential delineation uncertainties. OARs with estimated dose metrics below clinical tolerances were deemed low priority and excluded from MD and/or manual review. Final plans were then optimized using high-priority MD OARs and low-priority AD OARs and compared with reference plans generated using all MD OARs. Multiple different spatial buffers were used to accommodate different potential delineation uncertainties.

RESULTS

Sixty-seven out of 201 total OARs were identified as low-priority using the proposed methodology, which permitted a 33% reduction in structures requiring manual delineation/review. Plans optimized using low-priority AD OARs without review or modification met all planning objectives that were met when all MD OARs were used, indicating clinical equivalence.

CONCLUSIONS

Prioritizing OARs using estimated dose distributions allowed a substantial reduction in required MD and review without affecting clinically relevant dosimetry.

Analysis of Geometric Performance and Dosimetric Impact of Using Automatic Contour Segmentation for Radiotherapy Planning

Purpose: To analyze geometric discrepancy and dosimetric impact in using contours generated by auto-segmentation (AS) against manually segmented (MS) clinical contours. Methods: A 48-subject prostate atlas was created and another 15 patients were used for testing. Contours were generated using a commercial atlas-based segmentation tool and compared to their clinical MS counterparts. The geometric correlation was evaluated using the Dice similarity coefficient (DSC) and Hausdorff distance (HD). Dosimetric relevance was evaluated for a subset of patients by assessing the DVH differences derived by optimizing plan dose using the AS and MS contours, respectively, and evaluating with respect to each. A paired t-test was employed for statistical comparison. The discrepancy in plan quality with respect to clinical dosimetric endpoints was evaluated. The analysis was repeated for head/neck (HN) with a 31-subject atlas and 15 test cases. Results: Dice agreement between AS and MS differed significantly across structures: from (L:0.92/R: 0.91) for the femoral heads to seminal vesical of 0.38 in the prostate cohort, and from 0.98 for the brain, to 0.36 for the chiasm of the HN group. Despite the geometric disagreement, the paired t-tests showed the lack of statistical evidence for systematic differences in dosimetric plan quality yielded by the AS and MS approach for the prostate cohort. In HN cases, statistically significant differences in dosimetric endpoints were observed in structures with small volumes or elongated shapes such as cord (p = 0.01) and esophagus (p = 0.04). The largest absolute dose difference of 11 Gy was seen in the mean pharynx dose. Conclusion: Varying AS performance among structures suggests a differential approach of using AS on a subset of structures and focus MS on the rest. The discrepancy between geometric and dosimetric-end-point driven evaluation also indicates the clinical utility of AS contours in optimization and evaluating plan quality despite of suboptimal geometrical accuracy.

Technical note: Atlas-based Auto-segmentation of masticatory muscles for head and neck cancer radiotherapy

PURPOSE

The study aimed to use quantitative geometric and dosimetric metrics to assess the accuracy of atlas-based auto-segmentation of masticatory muscles (MMs) compared to manual drawn contours for head and neck cancer (HNC) radiotherapy (RT).

MATERIALS AND METHODS

Fifty-eight patients with HNC treated with RT were analyzed. Paired MMs (masseter, temporalis, and medial and lateral pterygoids) were manually delineated on planning computed tomography (CT) images for all patients. Twenty-nine patients were used to generate the MM atlas. Using this atlas, automatic segmentation of the MMs was performed for the remaining 29 patients without manual correction. Auto-segmentation accuracy for MMs was compared using dice similarity coefficients (DSCs), Hausdorff distance (HD), HD95, and variation in the center of mass (∆COM). The dosimetric impact on MMs was calculated (∆dose) using dosimetric parameters (D99%, D95%, D50%, and D1%), and compared with the geometric indices to test correlation.

RESULTS

DSC_mean ranges from 0.79 ± 0.04 to 0.85 ± 0.04, HD_mean from 0.43 ± 0.08 to 0.82 ± 0.26 cm, HD95_mean from 0.32 ± 0.08 to 0.42 ± 0.16 cm, and ∆COM_mean from 0.18 ± 0.11 to 0.33 ± 0.23 cm. The mean MM volume difference was < 15%. The correlation coefficient (r) of geometric and dosimetric indices for the four MMs ranges between -0.456 and 0.300.

CONCLUSIONS

Atlas-based auto-segmentation for masticatory muscles provides geometrically accurate contours compared to manual drawn contours. Dose obtained from those auto-segmented contours is comparable to that from manual drawn contours. Atlas-based auto-segmentation strategy for MM in HN radiotherapy is readily availalbe for clinical implementation.

Automatic segmentation and applicator reconstruction for CT-based brachytherapy of cervical cancer using 3D convolutional neural networks

In this study, we present deep learning-based approaches to automatic segmentation and applicator reconstruction with high accuracy and efficiency in the planning computed tomography (CT) for cervical cancer brachytherapy (BT). A novel three-dimensional (3D) convolutional neural network (CNN) architecture was proposed and referred to as DSD-UNET. The dataset of 91 patients received CT-based BT of cervical cancer was used to train and test DSD-UNET model for auto-segmentation of high-risk clinical target volume (HR-CTV) and organs at risk (OARs). Automatic applicator reconstruction was achieved with DSD-UNET-based segmentation of applicator components followed by 3D skeletonization and polynomial curve fitting. Digitization of the channel paths for tandem and ovoid applicator in the planning CT was evaluated utilizing the data from 32 patients. Dice similarity coefficient (DSC), Jaccard Index (JI), and Hausdorff distance (HD) were used to quantitatively evaluate the accuracy. The segmentation performance of DSD-UNET was compared with that of 3D U-Net. Results showed that DSD-UNET method outperformed 3D U-Net on segmentations of all the structures. The mean DSC values of DSD-UNET method were 86.9%, 82.9%, and 82.1% for bladder, HR-CTV, and rectum, respectively. For the performance of automatic applicator reconstruction, outstanding segmentation accuracy was first achieved for the intrauterine and ovoid tubes (average DSC value of 92.1%, average HD value of 2.3 mm). Finally, HDs between the channel paths determined automatically and manually were 0.88 ± 0.12 mm, 0.95 ± 0.16 mm, and 0.96 ± 0.15 mm for the intrauterine, left ovoid, and right ovoid tubes, respectively. The proposed DSD-UNET method outperformed the 3D U-Net and could segment HR-CTV, bladder, and rectum with relatively good accuracy. Accurate digitization of the channel paths could be achieved with the DSD-UNET-based method. The proposed approaches could be useful to improve the efficiency and consistency of treatment planning for cervical cancer BT.

Dose-escalated volumetric modulated arc therapy for total marrow irradiation: A feasibility dosimetric study with 4DCT planning and simultaneous integrated boost

PURPOSE

To evaluate the planning feasibility of dose-escalated total marrow irradiation (TMI) with simultaneous integrated boost (SIB) to the active bone marrow (ABM) using volumetric modulated arc therapy (VMAT), and to assess the impact of using planning organs at risk (OAR) volumes (PRV) accounting for breathing motion in the optimization.

METHODS

Five patients underwent whole-body CT and thoraco-abdominal 4DCT. A planning target volume (PTV) including all bones and ABM was contoured on each whole-body CT. PRV of selected OAR (liver, heart, kidneys, lungs, spleen, stomach) were determined with 4DCT. Planning consisted of 9-10 full 6 MV photon VMAT arcs. Four plans were created for each patient with 12 Gy prescribed to the PTV, with or without an additional 4 Gy SIB to the ABM. Planning dose constraints were set on the OAR or on the PRV. Planning objective was a PTV D_mean < 110% of the prescribed dose, a PTV V_110% < 50%, and OAR D_mean ≤ 50-60%.

RESULTS

PTV D_mean < 110% was accomplished for most plans (n = 18/20), while all achieved V_110%<50%. SIB plans succeeded to optimally cover the boost volume (median ABM D_mean = 16.3 Gy) and resulted in similar OAR sparing compared to plans without SIB (median OAR D_mean = 40-54% of the ABM prescribed dose). No statistically significant differences between plans optimized with constraints on OAR or PRV were found.

CONCLUSIONS

Adding a 4 Gy SIB to the ABM for TMI is feasible with VMAT technique, and results in OAR sparing similar to plans without SIB. Setting dose constraints on PRV does not impair PTV dosimetric parameters.

Assessment of manual adjustment performed in clinical practice following deep learning contouring for head and neck organs at risk in radiotherapy

BACKGROUND AND PURPOSE

Auto-contouring performance has been widely studied in development and commissioning studies in radiotherapy, and its impact on clinical workflow assessed in that context. This study aimed to evaluate the manual adjustment of auto-contouring in routine clinical practice and to identify improvements regarding the auto-contouring model and clinical user interaction, to improve the efficiency of auto-contouring.

MATERIALS AND METHODS

A total of 103 clinical head and neck cancer cases, contoured using a commercial deep-learning contouring system and subsequently checked and edited for clinical use were retrospectively taken from clinical data over a twelve-month period (April 2019-April 2020). The amount of adjustment performed was calculated, and all cases were registered to a common reference frame for assessment purposes. The median, 10th and 90th percentile of adjustment were calculated and displayed using 3D renderings of structures to visually assess systematic and random adjustment. Results were also compared to inter-observer variation reported previously. Assessment was performed for both the whole structures and for regional sub-structures, and according to the radiation therapy technologist (RTT) who edited the contour.

RESULTS

The median amount of adjustment was low for all structures (<2 mm), although large local adjustment was observed for some structures. The median was systematically greater or equal to zero, indicating that the auto-contouring tends to under-segment the desired contour.

CONCLUSION

Auto-contouring performance assessment in routine clinical practice has identified systematic improvements required technically, but also highlighted the need for continued RTT training to ensure adherence to guidelines.

[Multi-scale 3D convolutional neural network-based segmentation of head and neck organs at risk]

OBJECTIVE

To establish an algorithm based on 3D convolution neural network to segment the organs at risk (OARs) in the head and neck on CT images.

METHODS

We propose an automatic segmentation algorithm of head and neck OARs based on V-Net. To enhance the feature expression ability of the 3D neural network, we combined the squeeze and exception (SE) module with the residual convolution module in V-Net to increase the weight of the features that has greater contributions to the segmentation task. Using a multi-scale strategy, we completed organ segmentation using two cascade models for location and fine segmentation, and the input image was resampled to different resolutions during preprocessing to allow the two models to focus on the extraction of global location information and local detail features respectively.

RESULTS

Our experiments on segmentation of 22 OARs in the head and neck indicated that compared with the existing methods, the proposed method achieved better segmentation accuracy and efficiency, and the average segmentation accuracy was improved by 9%. At the same time, the average test time was reduced from 33.82 s to 2.79 s.

CONCLUSIONS

The 3D convolution neural network based on multi-scale strategy can effectively and efficiently improve the accuracy of organ segmentation and can be potentially used in clinical setting for segmentation of other organs to improve the efficiency of clinical treatment.

[Using stacked neural network to improve the auto-segmentation accuracy of Graves' ophthalmopathy target volumes for radiotherapy]

Compared with the previous automatic segmentation neural network for the target area which considered the target area as an independent area, a stacked neural network which uses the position and shape information of the organs around the target area to regulate the shape and position of the target area through the superposition of multiple networks and fusion of spatial position information to improve the segmentation accuracy on medical images was proposed in this paper. Taking the Graves' ophthalmopathy disease as an example, the left and right radiotherapy target areas were segmented by the stacked neural network based on the fully convolutional neural network. The volume Dice similarity coefficient (DSC) and bidirectional Hausdorff distance (HD) were calculated based on the target area manually drawn by the doctor. Compared with the full convolutional neural network, the stacked neural network segmentation results can increase the volume DSC on the left and right sides by 1.7% and 3.4% respectively, while the two-way HD on the left and right sides decrease by 0.6. The results show that the stacked neural network improves the degree of coincidence between the automatic segmentation result and the doctor's delineation of the target area, while reducing the segmentation error of small areas. The stacked neural network can effectively improve the accuracy of the automatic delineation of the radiotherapy target area of Graves' ophthalmopathy.

Auto-segmentation of organs at risk for head and neck radiotherapy planning: From atlas-based to deep learning methods

Radiotherapy (RT) is one of the basic treatment modalities for cancer of the head and neck (H&N), which requires a precise spatial description of the target volumes and organs at risk (OARs) to deliver a highly conformal radiation dose to the tumor cells while sparing the healthy tissues. For this purpose, target volumes and OARs have to be delineated and segmented from medical images. As manual delineation is a tedious and time-consuming task subjected to intra/interobserver variability, computerized auto-segmentation has been developed as an alternative. The field of medical imaging and RT planning has experienced an increased interest in the past decade, with new emerging trends that shifted the field of H&N OAR auto-segmentation from atlas-based to deep learning-based approaches. In this review, we systematically analyzed 78 relevant publications on auto-segmentation of OARs in the H&N region from 2008 to date, and provided critical discussions and recommendations from various perspectives: image modality - both computed tomography and magnetic resonance image modalities are being exploited, but the potential of the latter should be explored more in the future; OAR - the spinal cord, brainstem, and major salivary glands are the most studied OARs, but additional experiments should be conducted for several less studied soft tissue structures; image database - several image databases with the corresponding ground truth are currently available for methodology evaluation, but should be augmented with data from multiple observers and multiple institutions; methodology - current methods have shifted from atlas-based to deep learning auto-segmentation, which is expected to become even more sophisticated; ground truth - delineation guidelines should be followed and participation of multiple experts from multiple institutions is recommended; performance metrics - the Dice coefficient as the standard volumetric overlap metrics should be accompanied with at least one distance metrics, and combined with clinical acceptability scores and risk assessments; segmentation performance - the best performing methods achieve clinically acceptable auto-segmentation for several OARs, however, the dosimetric impact should be also studied to provide clinically relevant endpoints for RT planning.

Reinventing radiation therapy with machine learning and imaging bio-markers (radiomics): State-of-the-art, challenges and perspectives

Radiation therapy is a pivotal cancer treatment that has significantly progressed over the last decade due to numerous technological breakthroughs. Imaging is now playing a critical role on deployment of the clinical workflow, both for treatment planning and treatment delivery. Machine-learning analysis of predefined features extracted from medical images, i.e. radiomics, has emerged as a promising clinical tool for a wide range of clinical problems addressing drug development, clinical diagnosis, treatment selection and implementation as well as prognosis. Radiomics denotes a paradigm shift redefining medical images as a quantitative asset for data-driven precision medicine. The adoption of machine-learning in a clinical setting and in particular of radiomics features requires the selection of robust, representative and clinically interpretable biomarkers that are properly evaluated on a representative clinical data set. To be clinically relevant, radiomics must not only improve patients' management with great accuracy but also be reproducible and generalizable. Hence, this review explores the existing literature and exposes its potential technical caveats, such as the lack of quality control, standardization, sufficient sample size, type of data collection, and external validation. Based upon the analysis of 165 original research studies based on PET, CT-scan, and MRI, this review provides an overview of new concepts, and hypotheses generating findings that should be validated. In particular, it describes evolving research trends to enhance several clinical tasks such as prognostication, treatment planning, response assessment, prediction of recurrence/relapse, and prediction of toxicity. Perspectives regarding the implementation of an AI-based radiotherapy workflow are presented.

Voxel based evaluation of sequential radiotherapy treatment plans with different dose fractionation schemes

OBJECTIVE

This study presents a methodology for voxel-based evaluation of two phase sequential radiotherapy treatment plans having conventional dose scheme in the first phase and subsequent hypofractionation dose scheme in the second phase based upon different priority [planning target volume (PTV), clinical target volume (CTV) and organs at risk (OAR)] of display modes.

METHODS

A case of carcinoma prostate was selected for demonstration. Varian Eclipse treatment planning system (TPS) was used for contouring and planning. In the first phase, a dose of 52 Gy in 26 fractions to the PTV and in the second phase, a dose of 19.5 Gy in 3 fractions to the PTV Boost was planned on the same CT data set. Both the plans (Phase 1 and Phase 2) were exported and processed using "Voxel-based radiobiology display (VRb) tool". Plan Sum for Biologically effective dose (BED)-Cube and equivalent dose of 2Gy (EQD₂)-Cube was reconstructed using a combination of linear quadratic (LQ) and linear quadratic-linear (LQ-L) radiobiological models. Tumor control probability (TCP) and normal tissue complication probability (NTCP) for different target volumes and organs were also calculated using EQD₂-volume histograms of the Plan Sum.

RESULTS

An in-house graphical user interface (GUI) is developed to present the qualitative and quantitative evaluation of the multiphase treatment plans with different display modes and dose regimens. The voxel based TCP obtained for the combined target volume was 90.56%. NTCP for the bladder and rectum was calculated from the Plan Sum histograms and found to be 0.33% and ~0.0% respectively.

CONCLUSION

The proposed methodology using the VRb tool offers superior plan evaluation for multiphase sequential radiotherapy treatment plans over the existing methods.

ADVANCES IN KNOWLEDGE

PTV, CTV and OAR priority based display modes in VRb tool offers better understanding of radiobiological evaluation of sequential radiotherapy treatment plans.

External validation of deep learning-based contouring of head and neck organs at risk

BACKGROUND AND PURPOSE

Head and neck (HN) radiotherapy can benefit from automatic delineation of tumor and surrounding organs because of the complex anatomy and the regular need for adaptation. The aim of this study was to assess the performance of a commercially available deep learning contouring (DLC) model on an external validation set.

MATERIALS AND METHODS

The CT-based DLC model, trained at the University Medical Center Groningen (UMCG), was applied to an independent set of 58 patients from the Radboud University Medical Center (RUMC). DLC results were compared to the RUMC manual reference using the Dice similarity coefficient (DSC) and 95th percentile of Hausdorff distance (HD95). Craniocaudal spatial information was added by calculating binned measures. In addition, a qualitative evaluation compared the acceptance of manual and DLC contours in both groups of observers.

RESULTS

Good correspondence was shown for the mandible (DSC 0.90; HD95 3.6 mm). Performance was reasonable for the glandular OARs, brainstem and oral cavity (DSC 0.78-0.85, HD95 3.7-7.3 mm). The other aerodigestive tract OARs showed only moderate agreement (DSC 0.53-0.65, HD95 around 9 mm). The binned measures displayed the largest deviations caudally and/or cranially.

CONCLUSIONS

This study demonstrates that the DLC model can provide a reasonable starting point for delineation when applied to an independent patient cohort. The qualitative evaluation did not reveal large differences in the interpretation of contouring guidelines between RUMC and UMCG observers.

Automation of data analysis in molecular cancer imaging and its potential impact on future clinical practice

Digitalization, especially the use of machine learning and computational intelligence, is considered to dramatically shape medical procedures in the near future. In the field of cancer diagnostics, radiomics, the extraction of multiple quantitative image features and their clustered analysis, is gaining increasing attention to obtain more detailed, reproducible, and meaningful information about the disease entity, its prognosis and the ideal therapeutic option. In this context, automation of diagnostic procedures can improve the entire pipeline, which comprises patient registration, planning and performing an imaging examination at the scanner, image reconstruction, image analysis, and feeding the diagnostic information from various sources into decision support systems. With a focus on cancer diagnostics, this review article reports and discusses how computer-assistance can be integrated into diagnostic procedures and which benefits and challenges arise from it. Besides a strong view on classical imaging modalities like x-ray, CT, MRI, ultrasound, PET, SPECT and hybrid imaging devices thereof, it is outlined how imaging data can be combined with data deriving from patient anamnesis, clinical chemistry, pathology, and different omics. In this context, the article also discusses IT infrastructures that are required to realize this integration in the clinical routine. Although there are still many challenges to comprehensively implement automated and integrated data analysis in molecular cancer imaging, the authors conclude that we are entering a new era of medical diagnostics and precision medicine.

Clinical Evaluation of Deep Learning and Atlas-Based Auto-Contouring of Bladder and Rectum for Prostate Radiation Therapy

PURPOSE

Auto-contouring may reduce workload, interobserver variation, and time associated with manual contouring of organs at risk. Manual contouring remains the standard due in part to uncertainty around the time and workload savings after accounting for the review and editing of auto-contours. This preliminary study compares a standard manual contouring workflow with 2 auto-contouring workflows (atlas and deep learning) for contouring the bladder and rectum in patients with prostate cancer.

METHODS AND MATERIALS

Three contouring workflows were defined based on the initial contour-generation method including manual (MAN), atlas-based auto-contour (ATLAS), and deep-learning auto-contour (DEEP). For each workflow, initial contour generation was retrospectively performed on 15 patients with prostate cancer. Then, radiation oncologists (ROs) edited each contour while blinded to the manner in which the initial contour was generated. Workflows were compared by time (both in initial contour generation and in RO editing), contour similarity, and dosimetric evaluation.

RESULTS

Mean durations for initial contour generation were 10.9 min, 1.4 min, and 1.2 min for MAN, DEEP, and ATLAS, respectively. Initial DEEP contours were more geometrically similar to initial MAN contours. Mean durations of the RO editing steps for MAN, DEEP, and ATLAS contours were 4.1 min, 4.7 min, and 10.2 min, respectively. The geometric extent of RO edits was consistently larger for ATLAS contours compared with MAN and DEEP. No differences in clinically relevant dose-volume metrics were observed between workflows.

CONCLUSION

Auto-contouring software affords time savings for initial contour generation; however, it is important to also quantify workload changes at the RO editing step. Using deep-learning auto-contouring for bladder and rectum contour generation reduced contouring time without negatively affecting RO editing times, contour geometry, or clinically relevant dose-volume metrics. This work contributes to growing evidence that deep-learning methods are a clinically viable solution for organ-at-risk contouring in radiation therapy.

Deep convolutional neural networks for automatic segmentation of thoracic organs-at-risk in radiation oncology - use of non-domain transfer learning

PURPOSE

Segmentation of organs-at-risk (OARs) is an essential component of the radiation oncology workflow. Commonly segmented thoracic OARs include the heart, esophagus, spinal cord, and lungs. This study evaluated a convolutional neural network (CNN) for automatic segmentation of these OARs.

METHODS

The dataset was created retrospectively from consecutive radiotherapy plans containing all five OARs of interest, including 22,411 CT slices from 168 patients. Patients were divided into training, validation, and test datasets according to a 66%/17%/17% split. We trained a modified U-Net, applying transfer learning from a VGG16 image classification model trained on ImageNet. The Dice coefficient and 95% Hausdorff distance on the test set for each organ was compared to a commercial atlas-based segmentation model using the Wilcoxon signed-rank test.

RESULTS

On the test dataset, the median Dice coefficients for the CNN model vs. the multi-atlas model were 71% vs. 67% for the spinal cord, 96% vs. 94% for the right lung, 96%vs. 94% for the left lung, 91% vs. 85% for the heart, and 63% vs. 37% for the esophagus. The median 95% Hausdorff distances were 9.5  mm vs. 25.3 mm, 5.1  mm vs. 8.1 mm, 4.0  mm vs. 8.0 mm, 9.8  mm vs. 15.8 mm, and 9.2 mm vs. 20.0 mm for the respective organs. The results all favored the CNN model (P < 0.05).

CONCLUSIONS

A 2D CNN can achieve superior results to commercial atlas-based software for OAR segmentation utilizing non-domain transfer learning, which has potential utility for quality assurance and expediting patient care.

Radiotherapy dose distribution prediction for breast cancer using deformable image registration

BACKGROUND

Radiotherapy treatment planning dose prediction can be used to ensure plan quality and guide automatic plan. One of the dose prediction methods is incorporating historical treatment planning data into algorithms to estimate the dose-volume histogram (DVH) of organ for new patients. Although DVH is used extensively in treatment plan quality and radiotherapy prognosis evaluation, three-dimensional dose distribution can describe the radiation effects more explicitly. The purpose of this retrospective study was to predict the dose distribution of breast cancer radiotherapy by means of deformable registration into atlas images with historical treatment planning data that were considered to achieve expert level. The atlas cohort comprised 20 patients with left-sided breast cancer, previously treated by volumetric-modulated arc radiotherapy. The registration-based prediction technique was applied to 20 patients outside the atlas cohort. This study evaluated and compared three different approaches: registration to the most similar image from a dataset of individual atlas images (SIM), registration to all images from a database of individual atlas images with the average method (WEI_A), and the weighted method (WEI_F). The dose prediction performance of each strategy was quantified using nine metrics, including the region of interest dose error, 80% and 100% prescription area dice similarity coefficients (DSCs), and γ metrics. A Friedman test and a nonparametric exact Wilcoxon signed rank test were performed to compare the differences among groups. The clinical doses of all cases served as the gold standard.

RESULTS

The WEI_F method could achieve superior dose prediction results to those of WEI_A. WEI_F outperformed SIM in the organ-at-risk mean absolute difference (MAD). When using the WEI_F method, the MAD values for the ipsilateral lung, heart, and whole lung were 197.9 ± 42.9, 166 ± 55.1, 122.3 ± 25.5, and 55.3 ± 42.2 cGy, respectively. Moreover, SIM exhibited superior prediction in the DSC and γ metrics. When using the SIM method, the means of the 80% and 100% prescription area DSCs, 33γ metric, and 55γ metric were 0.85 ± 0.05, 0.84 ± 0.05, 0.64 ± 0.13, and 0.84 ± 0.10, respectively. The plan target volume and spinal cord MAD when using SIM and WEI were 235.6 ± 158.4 cGy versus 227.4 ± 144.0 cGy ([Formula: see text]) and 61.4 ± 44.9 cGy versus 55.3 ± 42.2 cGy ([Formula: see text]), respectively.

CONCLUSIONS

A predicted dose distribution that approximated the clinical plan could be generated using the methods presented in this study.

Atlas-based auto-segmentation for postoperative radiotherapy planning in endometrial and cervical cancers

Background

Since intensity-modulated radiation therapy (IMRT) has become popular for the treatment of gynecologic cancers, the contouring process has become more critical. This study evaluated the feasibility of atlas-based auto-segmentation (ABAS) for contouring in patients with endometrial and cervical cancers.

Methods

A total of 75 sets of planning CT images from 75 patients were collected. Contours for the pelvic nodal clinical target volume (CTV), femur, and bladder were carefully generated by two skilled radiation oncologists. Of 75 patients, 60 were randomly registered in three different atlas libraries for ABAS in groups of 20, 40, or 60. ABAS was conducted in 15 patients, followed by manual correction (ABAS_c). The time required to generate all contours was recorded, and the accuracy of segmentation was assessed using Dice’s coefficient (DC) and the Hausdorff distance (HD) and compared to those of manually delineated contours.

Results

For ABAS-CTV, the best results were achieved with groups of 60 patients (DC, 0.79; HD, 19.7 mm) and the worst results with groups of 20 patients (DC, 0.75; p = 0.012; HD, 21.3 mm; p = 0.002). ABAS_c-CTV performed better than ABAS-CTV in terms of both HD and DC (ABAS_c [n = 60]; DC, 0.84; HD, 15.6 mm; all p < 0.017). ABAS required an average of 45.1 s, whereas ABAS_c required 191.1 s; both methods required less time than the manual methods (p < 0.001). Both ABAS-Femur and simultaneous ABAS-Bilateral-femurs showed satisfactory performance, regardless of the atlas library used (DC > 0.9 and HD ≤10.0 mm), with significant time reduction compared to that needed for manual delineation (p < 0.001). However, ABAS-Bladder did not prove to be feasible, with inferior results regardless of library size (DC < 0.6 and HD > 40 mm). Furthermore, ABAS_c-Bladder required a longer processing time than manual contouring to achieve the same accuracy.

Conclusions

ABAS could help physicians to delineate the CTV and organs-at-risk (e.g., femurs) in IMRT planning considering its consistency, efficacy, and accuracy.

J. Kotte AN, Bol GH, T. van den Berg CA. Clinical implementation of MRI-based organs-at-risk auto-segmentation with convolutional networks for prostate radiotherapy

BACKGROUND

Structure delineation is a necessary, yet time-consuming manual procedure in radiotherapy. Recently, convolutional neural networks have been proposed to speed-up and automatise this procedure, obtaining promising results. With the advent of magnetic resonance imaging (MRI)-guided radiotherapy, MR-based segmentation is becoming increasingly relevant. However, the majority of the studies investigated automatic contouring based on computed tomography (CT).

PURPOSE

In this study, we investigate the feasibility of clinical use of deep learning-based automatic OARs delineation on MRI.

MATERIALS AND METHODS

We included 150 patients diagnosed with prostate cancer who underwent MR-only radiotherapy. A three-dimensional (3D) T1-weighted dual spoiled gradient-recalled echo sequence was acquired with 3T MRI for the generation of the synthetic-CT. The first 48 patients were included in a feasibility study training two 3D convolutional networks called DeepMedic and dense V-net (dV-net) to segment bladder, rectum and femurs. A research version of an atlas-based software was considered for comparison. Dice similarity coefficient, 95% Hausdorff distances (HD95), and mean distances were calculated against clinical delineations. For eight patients, an expert RTT scored the quality of the contouring for all the three methods. A choice among the three approaches was made, and the chosen approach was retrained on 97 patients and implemented for automatic use in the clinical workflow. For the successive 53 patients, Dice, HD95 and mean distances were calculated against the clinically used delineations.

RESULTS

DeepMedic, dV-net and the atlas-based software generated contours in 60 s, 4 s and 10-15 min, respectively. Performances were higher for both the networks compared to the atlas-based software. The qualitative analysis demonstrated that delineation from DeepMedic required fewer adaptations, followed by dV-net and the atlas-based software. DeepMedic was clinically implemented. After retraining DeepMedic and testing on the successive patients, the performances slightly improved.

CONCLUSION

High conformality for OARs delineation was achieved with two in-house trained networks, obtaining a significant speed-up of the delineation procedure. Comparison of different approaches has been performed leading to the succesful adoption of one of the neural networks, DeepMedic, in the clinical workflow. DeepMedic maintained in a clinical setting the accuracy obtained in the feasibility study.

Artificial Intelligence in radiotherapy: state of the art and future directions

Recent advances in computing capability allowed the development of sophisticated predictive models to assess complex relationships within observational data, described as Artificial Intelligence. Medicine is one of the several fields of application and Radiation oncology could benefit from these approaches, particularly in patients' medical records, imaging, baseline pathology, planning or instrumental data. Artificial Intelligence systems could simplify many steps of the complex workflow of radiotherapy such as segmentation, planning or delivery. However, Artificial Intelligence could be considered as a "black box" in which human operator may only understand input and output predictions and its application to the clinical practice remains a challenge. The low transparency of the overall system is questionable from manifold points of view (ethical included). Given the complexity of this issue, we collected the basic definitions to help the clinician to understand current literature, and overviewed experiences regarding implementation of AI within radiotherapy clinical workflow, aiming to describe this field from the clinician perspective.

A method of rapid quantification of patient-specific organ doses for CT using deep-learning-based multi-organ segmentation and GPU-accelerated Monte Carlo dose computing

PURPOSE

One technical barrier to patient-specific computed tomography (CT) dosimetry has been the lack of computational tools for the automatic patient-specific multi-organ segmentation of CT images and rapid organ dose quantification. When previous CT images are available for the same body region of the patient, the ability to obtain patient-specific organ doses for CT - in a similar manner as radiation therapy treatment planning - will open the door to personalized and prospective CT scan protocols. This study aims to demonstrate the feasibility of combining deep-learning algorithms for automatic segmentation of multiple radiosensitive organs from CT images with the GPU-based Monte Carlo rapid organ dose calculation.

METHODS

A deep convolutional neural network (CNN) based on the U-Net for organ segmentation is developed and trained to automatically delineate multiple radiosensitive organs from CT images. Two databases are used: The lung CT segmentation challenge 2017 (LCTSC) dataset that contains 60 thoracic CT scan patients, each consisting of five segmented organs, and the Pancreas-CT (PCT) dataset, which contains 43 abdominal CT scan patients each consisting of eight segmented organs. A fivefold cross-validation method is performed on both sets of data. Dice similarity coefficients (DSCs) are used to evaluate the segmentation performance against the ground truth. A GPU-based Monte Carlo dose code, ARCHER, is used to calculate patient-specific CT organ doses. The proposed method is evaluated in terms of relative dose errors (RDEs). To demonstrate the potential improvement of the new method, organ dose results are compared against those obtained for population-average patient phantoms used in an off-line dose reporting software, VirtualDose, at Massachusetts General Hospital.

RESULTS

The median DSCs are found to be 0.97 (right lung), 0.96 (left lung), 0.92 (heart), 0.86 (spinal cord), 0.76 (esophagus) for the LCTSC dataset, along with 0.96 (spleen), 0.96 (liver), 0.95 (left kidney), 0.90 (stomach), 0.87 (gall bladder), 0.80 (pancreas), 0.75 (esophagus), and 0.61 (duodenum) for the PCT dataset. Comparing with organ dose results from population-averaged phantoms, the new patient-specific method achieved smaller absolute RDEs (mean ± standard deviation) for all organs: 1.8% ± 1.4% (vs 16.0% ± 11.8%) for the lung, 0.8% ± 0.7% (vs 34.0% ± 31.1%) for the heart, 1.6% ± 1.7% (vs 45.7% ± 29.3%) for the esophagus, 0.6% ± 1.2% (vs 15.8% ± 12.7%) for the spleen, 1.2% ± 1.0% (vs 18.1% ± 15.7%) for the pancreas, 0.9% ± 0.6% (vs 20.0% ± 15.2%) for the left kidney, 1.7% ± 3.1% (vs 19.1% ± 9.8%) for the gallbladder, 0.3% ± 0.3% (vs 24.2% ± 18.7%) for the liver, and 1.6% ± 1.7% (vs 19.3% ± 13.6%) for the stomach. The trained automatic segmentation tool takes <5 s per patient for all 103 patients in the dataset. The Monte Carlo radiation dose calculations performed in parallel to the segmentation process using the GPU-accelerated ARCHER code take <4 s per patient to achieve <0.5% statistical uncertainty in all organ doses for all 103 patients in the database.

CONCLUSION

This work shows the feasibility to perform combined automatic patient-specific multi-organ segmentation of CT images and rapid GPU-based Monte Carlo dose quantification with clinically acceptable accuracy and efficiency.

Improving accuracy and robustness of deep convolutional neural network based thoracic OAR segmentation

Deep convolutional neural network (DCNN) has shown great success in various medical image segmentation tasks, including organ-at-risk (OAR) segmentation from computed tomography (CT) images. However, most studies use the dataset from the same source(s) for training and testing so that the ability of a trained DCNN to generalize to a different dataset is not well studied, as well as the strategy to address the issue of performance drop on a different dataset. In this study we investigated the performance of a well-trained DCNN model from a public dataset for thoracic OAR segmentation on a local dataset and explored the systematic differences between the datasets. We observed that a subtle shift of organs inside patient body due to the abdominal compression technique during image acquisition caused significantly worse performance on the local dataset. Furthermore, we developed an optimal strategy via incorporating different numbers of new cases from the local institution and using transfer learning to improve the accuracy and robustness of the trained DCNN model. We found that by adding as few as 10 cases from the local institution, the performance can reach the same level as in the original dataset. With transfer learning, the training time can be significantly shortened with slightly worse performance for heart segmentation.

CT images with expert manual contours of thoracic cancer for benchmarking auto-segmentation accuracy

PURPOSE

Automatic segmentation offers many benefits for radiotherapy treatment planning; however, the lack of publicly available benchmark datasets limits the clinical use of automatic segmentation. In this work, we present a well-curated computed tomography (CT) dataset of high-quality manually drawn contours from patients with thoracic cancer that can be used to evaluate the accuracy of thoracic normal tissue auto-segmentation systems.

ACQUISITION AND VALIDATION METHODS

Computed tomography scans of 60 patients undergoing treatment simulation for thoracic radiotherapy were acquired from three institutions: MD Anderson Cancer Center, Memorial Sloan Kettering Cancer Center, and the MAASTRO clinic. Each institution provided CT scans from 20 patients, including mean intensity projection four-dimensional CT (4D CT), exhale phase (4D CT), or free-breathing CT scans depending on their clinical practice. All CT scans covered the entire thoracic region with a 50-cm field of view and slice spacing of 1, 2.5, or 3 mm. Manual contours of left/right lungs, esophagus, heart, and spinal cord were retrieved from the clinical treatment plans. These contours were checked for quality and edited if necessary to ensure adherence to RTOG 1106 contouring guidelines.

DATA FORMAT AND USAGE NOTES

The CT images and RTSTRUCT files are available in DICOM format. The regions of interest were named according to the nomenclature recommended by American Association of Physicists in Medicine Task Group 263 as Lung_L, Lung_R, Esophagus, Heart, and SpinalCord. This dataset is available on The Cancer Imaging Archive (funded by the National Cancer Institute) under Lung CT Segmentation Challenge 2017 (http://doi.org/10.7937/K9/TCIA.2017.3r3fvz08).

POTENTIAL APPLICATIONS

This dataset provides CT scans with well-delineated manually drawn contours from patients with thoracic cancer that can be used to evaluate auto-segmentation systems. Additional anatomies could be supplied in the future to enhance the existing library of contours.