A combined learning algorithm for prostate segmentation on 3D CT images

Deep learning based automatic segmentation of the Internal Pudendal Artery in definitive radiotherapy treatment planning of localized prostate cancer

Background and purpose

Radiation-induced erectile dysfunction (RiED) commonly affects prostate cancer patients, prompting clinical trials across institutions to explore dose-sparing to internal-pudendal-arteries (IPA) for preserving sexual potency. IPA, challenging to segment, isn't conventionally considered an organ-at-risk (OAR). This study proposes a deep learning (DL) auto-segmentation model for IPA, using Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) or CT alone to accommodate varied clinical practices.

Materials and methods

A total of 86 patients with CT and MRI images and noisy IPA labels were recruited in this study. We split the data into 42/14/30 for model training, testing, and a clinical observer study, respectively. There were three major innovations in this model: 1) we designed an architecture with squeeze-and-excite blocks and modality attention for effective feature extraction and production of accurate segmentation, 2) a novel loss function was used for training the model effectively with noisy labels, and 3) modality dropout strategy was used for making the model capable of segmentation in the absence of MRI.

Results

Test dataset metrics were DSC 61.71 ± 7.7 %, ASD 2.5 ± .87 mm, and HD95 7.0 ± 2.3 mm. AI segmented contours showed dosimetric similarity to expert physician's contours. Observer study indicated higher scores for AI contours (mean = 3.7) compared to inexperienced physicians' contours (mean = 3.1). Inexperienced physicians improved scores to 3.7 when starting with AI contours.

Conclusion

The proposed model achieved good quality IPA contours to improve uniformity of segmentation and to facilitate introduction of standardized IPA segmentation into clinical trials and practice.

Research related to the diagnosis of prostate cancer based on machine learning medical images: A review

BACKGROUND

Prostate cancer is currently the second most prevalent cancer among men. Accurate diagnosis of prostate cancer can provide effective treatment for patients and greatly reduce mortality. The current medical imaging tools for screening prostate cancer are mainly MRI, CT and ultrasound. In the past 20 years, these medical imaging methods have made great progress with machine learning, especially the rise of deep learning has led to a wider application of artificial intelligence in the use of image-assisted diagnosis of prostate cancer.

METHOD

This review collected medical image processing methods, prostate and prostate cancer on MR images, CT images, and ultrasound images through search engines such as web of science, PubMed, and Google Scholar, including image pre-processing methods, segmentation of prostate gland on medical images, registration between prostate gland on different modal images, detection of prostate cancer lesions on the prostate.

CONCLUSION

Through these collated papers, it is found that the current research on the diagnosis and staging of prostate cancer using machine learning and deep learning is in its infancy, and most of the existing studies are on the diagnosis of prostate cancer and classification of lesions, and the accuracy is low, with the best results having an accuracy of less than 0.95. There are fewer studies on staging. The research is mainly focused on MR images and much less on CT images, ultrasound images.

DISCUSSION

Machine learning and deep learning combined with medical imaging have a broad application prospect for the diagnosis and staging of prostate cancer, but the research in this area still has more room for development.

Role of Machine Learning-Based CT Body Composition in Risk Prediction and Prognostication: Current State and Future Directions

CT body composition analysis has been shown to play an important role in predicting health and has the potential to improve patient outcomes if implemented clinically. Recent advances in artificial intelligence and machine learning have led to high speed and accuracy for extracting body composition metrics from CT scans. These may inform preoperative interventions and guide treatment planning. This review aims to discuss the clinical applications of CT body composition in clinical practice, as it moves towards widespread clinical implementation.

PSA-Net: Deep learning-based physician style-aware segmentation network for postoperative prostate cancer clinical target volumes

PURPOSE

Automatic segmentation of medical images with deep learning (DL) algorithms has proven highly successful in recent times. With most of these automation networks, inter-observer variation is an acknowledged problem that leads to suboptimal results. This problem is even more significant in segmenting postoperative clinical target volumes (CTV) because they lack a macroscopic visible tumor in the image. This study, using postoperative prostate CTV segmentation as the test case, tries to determine 1) whether physician styles are consistent and learnable, 2) whether physician style affects treatment outcome and toxicity, and 3) how to explicitly deal with different physician styles in DL-assisted CTV segmentation to facilitate its clinical acceptance.

METHODS

A dataset of 373 postoperative prostate cancer patients from UT Southwestern Medical Center was used for this study. We used another 83 patients from Mayo Clinic to validate the developed model and its adaptability. To determine whether physician styles are consistent and learnable, we trained a 3D convolutional neural network classifier to identify which physician had contoured a CTV from just the contour and the corresponding CT scan. Next, we evaluated whether adapting automatic segmentation to specific physician styles would be clinically feasible based on a lack of difference between outcomes. Here, biochemical progression-free survival (BCFS) and grade 3+ genitourinary and gastrointestinal toxicity were estimated with the Kaplan-Meier method and compared between physician styles with the log rank test and subsequently with a multivariate Cox regression. When we found no statistically significant differences in outcome or toxicity between contouring styles, we proposed a concept called physician style-aware (PSA) segmentation by developing an encoder-multidecoder network with perceptual loss to model different physician styles of CTV segmentation.

RESULTS

The classification network captured the different physician styles with 87% accuracy. Subsequent outcome analysis showed no differences in BCFS and grade 3+ toxicity among physicians. With the proposed physician style-aware network (PSA-Net), Dice similarity coefficient (DSC) accuracy for all physicians was 3.4% higher on average than with a general model that does not differentiate physician styles. We show that these stylistic contouring variations also exist between institutions that follow the same segmentation guidelines, and we show the proposed method's effectiveness in adapting to new institutional styles. We observed an accuracy improvement of 5% in terms of DSC when adapting to the style of a separate institution.

CONCLUSION

The performance of the classification network established that physician styles are learnable, and the lack of difference between outcomes among physicians shows that the network can feasibly adapt to different styles in the clinic. Therefore, we developed a novel PSA-Net model that can produce contours specific to the treating physician, thus improving segmentation accuracy and avoiding the need to train multiple models to achieve different style segmentations. We successfully validated this model on data from a separate institution, thus supporting the model's generalizability to diverse datasets.

HF-UNet: Learning Hierarchically Inter-Task Relevance in Multi-Task U-Net for Accurate Prostate Segmentation in CT Images

Accurate segmentation of the prostate is a key step in external beam radiation therapy treatments. In this paper, we tackle the challenging task of prostate segmentation in CT images by a two-stage network with 1) the first stage to fast localize, and 2) the second stage to accurately segment the prostate. To precisely segment the prostate in the second stage, we formulate prostate segmentation into a multi-task learning framework, which includes a main task to segment the prostate, and an auxiliary task to delineate the prostate boundary. Here, the second task is applied to provide additional guidance of unclear prostate boundary in CT images. Besides, the conventional multi-task deep networks typically share most of the parameters (i.e., feature representations) across all tasks, which may limit their data fitting ability, as the specificity of different tasks are inevitably ignored. By contrast, we solve them by a hierarchically-fused U-Net structure, namely HF-UNet. The HF-UNet has two complementary branches for two tasks, with the novel proposed attention-based task consistency learning block to communicate at each level between the two decoding branches. Therefore, HF-UNet endows the ability to learn hierarchically the shared representations for different tasks, and preserve the specificity of learned representations for different tasks simultaneously. We did extensive evaluations of the proposed method on a large planning CT image dataset and a benchmark prostate zonal dataset. The experimental results show HF-UNet outperforms the conventional multi-task network architectures and the state-of-the-art methods.

MetricUNet: Synergistic image- and voxel-level learning for precise prostate segmentation via online sampling

Fully convolutional networks (FCNs), including UNet and VNet, are widely-used network architectures for semantic segmentation in recent studies. However, conventional FCN is typically trained by the cross-entropy or Dice loss, which only calculates the error between predictions and ground-truth labels for pixels individually. This often results in non-smooth neighborhoods in the predicted segmentation. This problem becomes more serious in CT prostate segmentation as CT images are usually of low tissue contrast. To address this problem, we propose a two-stage framework, with the first stage to quickly localize the prostate region, and the second stage to precisely segment the prostate by a multi-task UNet architecture. We introduce a novel online metric learning module through voxel-wise sampling in the multi-task network. Therefore, the proposed network has a dual-branch architecture that tackles two tasks: (1) a segmentation sub-network aiming to generate the prostate segmentation, and (2) a voxel-metric learning sub-network aiming to improve the quality of the learned feature space supervised by a metric loss. Specifically, the voxel-metric learning sub-network samples tuples (including triplets and pairs) in voxel-level through the intermediate feature maps. Unlike conventional deep metric learning methods that generate triplets or pairs in image-level before the training phase, our proposed voxel-wise tuples are sampled in an online manner and operated in an end-to-end fashion via multi-task learning. To evaluate the proposed method, we implement extensive experiments on a real CT image dataset consisting 339 patients. The ablation studies show that our method can effectively learn more representative voxel-level features compared with the conventional learning methods with cross-entropy or Dice loss. And the comparisons show that the proposed method outperforms the state-of-the-art methods by a reasonable margin.

CT Male Pelvic Organ Segmentation via Hybrid Loss Network With Incomplete Annotation

Sufficient data with complete annotation is essential for training deep models to perform automatic and accurate segmentation of CT male pelvic organs, especially when such data is with great challenges such as low contrast and large shape variation. However, manual annotation is expensive in terms of both finance and human effort, which usually results in insufficient completely annotated data in real applications. To this end, we propose a novel deep framework to segment male pelvic organs in CT images with incomplete annotation delineated in a very user-friendly manner. Specifically, we design a hybrid loss network derived from both voxel classification and boundary regression, to jointly improve the organ segmentation performance in an iterative way. Moreover, we introduce a label completion strategy to complete the labels of the rich unannotated voxels and then embed them into the training data to enhance the model capability. To reduce the computation complexity and improve segmentation performance, we locate the pelvic region based on salient bone structures to focus on the candidate segmentation organs. Experimental results on a large planning CT pelvic organ dataset show that our proposed method with incomplete annotation achieves comparable segmentation performance to the state-of-the-art methods with complete annotation. Moreover, our proposed method requires much less effort of manual contouring from medical professionals such that an institutional specific model can be more easily established.

Deep learning-based three-dimensional segmentation of the prostate on computed tomography images

Segmentation of the prostate in computed tomography (CT) is used for planning and guidance of prostate treatment procedures. However, due to the low soft-tissue contrast of the images, manual delineation of the prostate on CT is a time-consuming task with high interobserver variability. We developed an automatic, three-dimensional (3-D) prostate segmentation algorithm based on a customized U-Net architecture. Our dataset contained 92 3-D abdominal CT scans from 92 patients, of which 69 images were used for training and validation and the remaining for testing the convolutional neural network model. Compared to manual segmentation by an expert radiologist, our method achieved 83 % ± 6 % for Dice similarity coefficient (DSC), 2.3 ± 0.6    mm for mean absolute distance (MAD), and 1.9 ± 4.0    cm 3 for signed volume difference ( Δ V ). The average recorded interexpert difference measured on the same test dataset was 92% (DSC), 1.1 mm (MAD), and 2.1    cm 3 ( Δ V ). The proposed algorithm is fast, accurate, and robust for 3-D segmentation of the prostate on CT images.

CT male pelvic organ segmentation using fully convolutional networks with boundary sensitive representation

Accurate segmentation of the prostate and organs at risk (e.g., bladder and rectum) in CT images is a crucial step for radiation therapy in the treatment of prostate cancer. However, it is a very challenging task due to unclear boundaries, large intra- and inter-patient shape variability, and uncertain existence of bowel gases and fiducial markers. In this paper, we propose a novel automatic segmentation framework using fully convolutional networks with boundary sensitive representation to address this challenging problem. Our novel segmentation framework contains three modules. First, an organ localization model is designed to focus on the candidate segmentation region of each organ for better performance. Then, a boundary sensitive representation model based on multi-task learning is proposed to represent the semantic boundary information in a more robust and accurate manner. Finally, a multi-label cross-entropy loss function combining boundary sensitive representation is introduced to train a fully convolutional network for the organ segmentation. The proposed method is evaluated on a large and diverse planning CT dataset with 313 images from 313 prostate cancer patients. Experimental results show that the performance of our proposed method outperforms the baseline fully convolutional networks, as well as other state-of-the-art methods in CT male pelvic organ segmentation.

Automatic Segmentation of the Prostate on CT Images Using Deep Neural Networks (DNN)

PURPOSE

Recent advances in deep neural networks (DNNs) have unlocked opportunities for their application for automatic image segmentation. We have evaluated a DNN-based algorithm for automatic segmentation of the prostate gland on a large cohort of patient images.

METHODS AND MATERIALS

Planning-CT data sets for 1114 patients with prostate cancer were retrospectively selected and divided into 2 groups. Group A contained 1104 data sets, with 1 physician-generated prostate gland contour for each data set. Among these image sets, 771 were used for training, 193 for validation, and 140 for testing. Group B contained 10 data sets, each including prostate contours delineated by 5 independent physicians and a consensus contour generated using the STAPLE method in the CERR software package. All images were resampled to a spatial resolution of 1 × 1 × 1.5 mm. A region (128 × 128 × 64 voxels) containing the prostate was selected to train a DNN. The best-performing model on the validation data sets was used to segment the prostate on all testing images. Results were compared between DNN and physician-generated contours using the Dice similarity coefficient, Hausdorff distances, regional contour distances, and center-of-mass distances.

RESULTS

The mean Dice similarity coefficients between DNN-based prostate segmentation and physician-generated contours for test data in Group A, Group B, and Group B-consensus were 0.85 ± 0.06 (range, 0.65-0.93), 0.85 ± 0.04 (range, 0.80-0.91), and 0.88 ± 0.03 (range, 0.82-0.92), respectively. The Hausdorff distance was 7.0 ± 3.5 mm, 7.3 ± 2.0 mm, and 6.3 ± 2.0 mm for Group A, Group B, and Group B-consensus, respectively. The mean center-of-mass distances for all 3 data set groups were within 5 mm.

CONCLUSIONS

A DNN-based algorithm was used to automatically segment the prostate for a large cohort of patients with prostate cancer. DNN-based prostate segmentations were compared to the consensus contour for a smaller group of patients; the agreement between DNN segmentations and consensus contour was similar to the agreement reported in a previous study. Clinical use of DNNs is promising, but further investigation is warranted.

A semiautomatic algorithm for three-dimensional segmentation of the prostate on CT images using shape and local texture characteristics

Prostate segmentation in computed tomography (CT) images is useful for planning and guidance of the diagnostic and therapeutic procedures. However, the low soft-tissue contrast of CT images makes the manual prostate segmentation a time-consuming task with high inter-observer variation. We developed a semi-automatic, three-dimensional (3D) prostate segmentation algorithm using shape and texture analysis and have evaluated the method against manual reference segmentations. In a training data set we defined an inter-subject correspondence between surface points in the spherical coordinate system. We applied this correspondence to model the globular and smoothly curved shape of the prostate with 86, well-distributed surface points using a point distribution model that captures prostate shape variation. We also studied the local texture difference between prostate and non-prostate tissues close to the prostate surface. For segmentation, we used the learned shape and texture characteristics of the prostate in CT images and we used a set of user inputs for prostate localization. We trained our algorithm using 23 CT images and tested it on 10 images. We evaluated the results compared with those of two experts' manual reference segmentations using different error metrics. The average measured Dice similarity coefficient (DSC) and mean absolute distance (MAD) were 88 ± 2% and 1.9 ± 0.5 mm, respectively. The averaged inter-expert difference measured on the same dataset was 91 ± 4% (DSC) and 1.3 ± 0.6 mm (MAD). With no prior intra-patient information, the proposed algorithm showed a fast, robust and accurate performance for 3D CT segmentation.