Patient specific prostate segmentation in 3-d magnetic resonance images

Label-set impact on deep learning-based prostate segmentation on MRI

BACKGROUND

Prostate segmentation is an essential step in computer-aided detection and diagnosis systems for prostate cancer. Deep learning (DL)-based methods provide good performance for prostate gland and zones segmentation, but little is known about the impact of manual segmentation (that is, label) selection on their performance. In this work, we investigated these effects by obtaining two different expert label-sets for the PROSTATEx I challenge training dataset (n = 198) and using them, in addition to an in-house dataset (n = 233), to assess the effect on segmentation performance. The automatic segmentation method we used was nnU-Net.

RESULTS

The selection of training/testing label-set had a significant (p < 0.001) impact on model performance. Furthermore, it was found that model performance was significantly (p < 0.001) higher when the model was trained and tested with the same label-set. Moreover, the results showed that agreement between automatic segmentations was significantly (p < 0.0001) higher than agreement between manual segmentations and that the models were able to outperform the human label-sets used to train them.

CONCLUSIONS

We investigated the impact of label-set selection on the performance of a DL-based prostate segmentation model. We found that the use of different sets of manual prostate gland and zone segmentations has a measurable impact on model performance. Nevertheless, DL-based segmentation appeared to have a greater inter-reader agreement than manual segmentation. More thought should be given to the label-set, with a focus on multicenter manual segmentation and agreement on common procedures.

CRITICAL RELEVANCE STATEMENT

Label-set selection significantly impacts the performance of a deep learning-based prostate segmentation model. Models using different label-set showed higher agreement than manual segmentations.

KEY POINTS

• Label-set selection has a significant impact on the performance of automatic segmentation models. • Deep learning-based models demonstrated true learning rather than simply mimicking the label-set. • Automatic segmentation appears to have a greater inter-reader agreement than manual segmentation.

Diagnosis of Prostate Cancer Using GLCM Enabled KNN Technique by Analyzing MRI Images

Cancer has a disproportionately large influence on the death rate of adults. A patient needs to get a diagnosis of their condition as quickly as is humanly feasible in order to have the greatest chance of surviving their sickness. Skilled medical professionals use medical imaging and other traditional diagnostic methods to search for clues that may indicate the presence of malignant tendencies inside the body. Nevertheless, manual diagnosis may be time-consuming and subjective owing to the wide range of interobserver variability induced by the enormous number of medical imaging data. This variability is caused by the fact that medical imaging data are collected. Because of this, the process of accurately diagnosing a patient could become more difficult. To execute jobs that included machine learning and the interpretation of complicated imagery, cutting-edge computer technology was necessary. Since the 1980s, researchers have been working on developing a computer-aided diagnostic system that would help medical professionals in the early diagnosis of various malignancies. According to the most recent projections, prostate cancer will be discovered in the body of one out of every seven men at some time throughout the course of their life. It is unacceptable how many men are being told that they have prostate cancer, and the condition is responsible for the deaths of a rising number of men every year. Because of the high quality and multidimensionality of the MRI pictures, you will also need a powerful diagnosis system in addition to the CAD tools. Since it has been shown that CAD technology is beneficial, researchers are looking at methods to improve the accuracy, precision, and speed of the systems that use it. The effectiveness of CAD technology has been shown. This research proposes a strategy that is both effective and efficient for the processing of images and the extraction of features as well as for machine learning. This work makes use of MRI scans and machine learning in an effort to detect prostate cancer at an early stage. Histogram equalization is used while doing the preliminary processing on photographs. The image's overall quality is elevated as a result. The fuzzy C means approach is used in order to segment the images. Using a Gray Level Cooccurrence Matrix (GLCM), it is feasible to extract features from a dataset. The KNN, random forest, and AdaBoost classification algorithms are used in the classification process.

CAN3D: Fast 3D medical image segmentation via compact context aggregation

Direct automatic segmentation of objects in 3D medical imaging, such as magnetic resonance (MR) imaging, is challenging as it often involves accurately identifying multiple individual structures with complex geometries within a large volume under investigation. Most deep learning approaches address these challenges by enhancing their learning capability through a substantial increase in trainable parameters within their models. An increased model complexity will incur high computational costs and large memory requirements unsuitable for real-time implementation on standard clinical workstations, as clinical imaging systems typically have low-end computer hardware with limited memory and CPU resources only. This paper presents a compact convolutional neural network (CAN3D) designed specifically for clinical workstations and allows the segmentation of large 3D Magnetic Resonance (MR) images in real-time. The proposed CAN3D has a shallow memory footprint to reduce the number of model parameters and computer memory required for state-of-the-art performance and maintain data integrity by directly processing large full-size 3D image input volumes with no patches required. The proposed architecture significantly reduces computational costs, especially for inference using the CPU. We also develop a novel loss function with extra shape constraints to improve segmentation accuracy for imbalanced classes in 3D MR images. Compared to state-of-the-art approaches (U-Net3D, improved U-Net3D and V-Net), CAN3D reduced the number of parameters up to two orders of magnitude and achieved much faster inference, up to 5 times when predicting with a standard commercial CPU (instead of GPU). For the open-access OAI-ZIB knee MR dataset, in comparison with manual segmentation, CAN3D achieved Dice coefficient values of (mean = 0.87 ± 0.02 and 0.85 ± 0.04) with mean surface distance errors (mean = 0.36 ± 0.32 mm and 0.29 ± 0.10 mm) for imbalanced classes such as (femoral and tibial) cartilage volumes respectively when training volume-wise under only 12G video memory. Similarly, CAN3D demonstrated high accuracy and efficiency on a pelvis 3D MR imaging dataset for prostate cancer consisting of 211 examinations with expert manual semantic labels (bladder, body, bone, rectum, prostate) now released publicly for scientific use as part of this work.

A new data augmentation method based on local image warping for medical image segmentation

PURPOSE

The segmentation accuracy of medical images was improved by increasing the number of training samples using a local image warping technique. The performance of the proposed method was evaluated in the segmentation of breast masses, prostate and brain tumors, and lung nodules.

METHODS

We propose a simple data augmentation method which is called stochastic evolution (SE). Specifically, the idea of SE stems from our thinking about the deterioration of the diseased tissue and the healing process. In order to simulate this natural process, we implement it according to the local distortion algorithm in image warping. In other words, the irregular deterioration and healing processes of the diseased tissue is simulated according to the direction of the local distortion, thereby producing a natural sample that is indistinguishable by humans.

RESULTS

The proposed method is evaluated on four segmentation tasks of breast masses, prostate, brain tumors, and lung nodules. Comparing the experimental results of four segmentation methods based on the UNet segmentation architecture without adding any expanded data during training, the accuracy and the Hausdorff distance obtained in our approach remain almost the same as other methods. However, the dice similarity coefficient (DSC) and sensitivity (SEN) have both improved to some extent. Among them, DSC is increased by 5.2%, 2.8%, 1.0%, and 3.2%, respectively; SEN is increased by 6.9%, 4.3%, 1.2%, and 4.5%, respectively.

CONCLUSIONS

Experimental results show that the proposed SE data augmentation method could improve the segmentation accuracy of breast masses, prostate, brain tumors, and lung nodules. The method also shows the robustness with different image datasets and imaging modalities.

Multi-Modal Siamese Network for Diagnostically Similar Lesion Retrieval in Prostate MRI

Multi-parametric prostate MRI (mpMRI) is a powerful tool to diagnose prostate cancer, though difficult to interpret even for experienced radiologists. A common radiological procedure is to compare a magnetic resonance image with similarly diagnosed cases. To assist the radiological image interpretation process, computerized Content-Based Image Retrieval systems (CBIRs) can therefore be employed to improve the reporting workflow and increase its accuracy. In this article, we propose a new, supervised siamese deep learning architecture able to handle multi-modal and multi-view MR images with similar PIRADS score. An experimental comparison with well-established deep learning-based CBIRs (namely standard siamese networks and autoencoders) showed significantly improved performance with respect to both diagnostic (ROC-AUC), and information retrieval metrics (Precision-Recall, Discounted Cumulative Gain and Mean Average Precision). Finally, the new proposed multi-view siamese network is general in design, facilitating a broad use in diagnostic medical imaging retrieval.

2D to 3D Evolutionary Deep Convolutional Neural Networks for Medical Image Segmentation

Developing a Deep Convolutional Neural Network (DCNN) is a challenging task that involves deep learning with significant effort required to configure the network topology. The design of a 3D DCNN not only requires a good complicated structure but also a considerable number of appropriate parameters to run effectively. Evolutionary computation is an effective approach that can find an optimum network structure and/or its parameters automatically. Note that the Neuroevolution approach is computationally costly, even for developing 2D networks. As it is expected that it will require even more massive computation to develop 3D Neuroevolutionary networks, this research topic has not been investigated until now. In this article, in addition to developing 3D networks, we investigate the possibility of using 2D images and 2D Neuroevolutionary networks to develop 3D networks for 3D volume segmentation. In doing so, we propose to first establish new evolutionary 2D deep networks for medical image segmentation and then convert the 2D networks to 3D networks in order to obtain optimal evolutionary 3D deep convolutional neural networks. The proposed approach results in a massive saving in computational and processing time to develop 3D networks, while achieved high accuracy for 3D medical image segmentation of nine various datasets.

Boundary Coding Representation for Organ Segmentation in Prostate Cancer Radiotherapy

Accurate segmentation of the prostate and organs at risk (OARs, e.g., bladder and rectum) in male pelvic CT images is a critical step for prostate cancer radiotherapy. Unfortunately, the unclear organ boundary and large shape variation make the segmentation task very challenging. Previous studies usually used representations defined directly on unclear boundaries as context information to guide segmentation. Those boundary representations may not be so discriminative, resulting in limited performance improvement. To this end, we propose a novel boundary coding network (BCnet) to learn a discriminative representation for organ boundary and use it as the context information to guide the segmentation. Specifically, we design a two-stage learning strategy in the proposed BCnet: 1) Boundary coding representation learning. Two sub-networks under the supervision of the dilation and erosion masks transformed from the manually delineated organ mask are first separately trained to learn the spatial-semantic context near the organ boundary. Then we encode the organ boundary based on the predictions of these two sub-networks and design a multi-atlas based refinement strategy by transferring the knowledge from training data to inference. 2) Organ segmentation. The boundary coding representation as context information, in addition to the image patches, are used to train the final segmentation network. Experimental results on a large and diverse male pelvic CT dataset show that our method achieves superior performance compared with several state-of-the-art methods.

Automatic segmentation of prostate magnetic resonance imaging using generative adversarial networks

BACKGROUND

Automatic and detailed segmentation of the prostate using magnetic resonance imaging (MRI) plays an essential role in prostate imaging diagnosis. Traditionally, prostate gland was manually delineated by the clinician in a time-consuming process that requires professional experience of the observer. Thus, we proposed an automatic prostate segmentation method, called SegDGAN, which is based on a classic generative adversarial network model.

MATERIAL AND METHODS

The proposed method comprises a fully convolutional generation network of densely con- nected blocks and a critic network with multi-scale feature extraction. In these computations, the objective function is optimized using mean absolute error and the Dice coefficient, leading to improved accuracy of segmentation results and correspondence with the ground truth. The common and similar medical image segmentation networks U-Net, FCN, and SegAN were selected for qualitative and quantitative comparisons with SegDGAN using a 220-patient dataset and the public datasets. The commonly used segmentation evaluation metrics DSC, VOE, ASD, and HD were used to compare the accuracy of segmentation between these methods.

RESULTS

SegDGAN achieved the highest DSC value of 91.66%, the lowest VOE value of 15.28%, the lowest ASD values of 0.51 mm and the lowest HD value of 11.58 mm with the clinical dataset. In addition, the highest DSC value, and the lowest VOE, ASD and HD values obtained with the public data set PROMISE12 were 86.24%, 23.60%, 1.02 mm and 7.57 mm, respectively.

CONCLUSIONS

Our experimental results show that the SegDGAN model have the potential to improve the accuracy of MRI-based prostate gland segmentation. Code has been made available at: https://github.com/w3user/SegDGAN.

A Quality Control System for Automated Prostate Segmentation on T2-Weighted MRI

Computer-aided detection and diagnosis (CAD) systems have the potential to improve robustness and efficiency compared to traditional radiological reading of magnetic resonance imaging (MRI). Fully automated segmentation of the prostate is a crucial step of CAD for prostate cancer, but visual inspection is still required to detect poorly segmented cases. The aim of this work was therefore to establish a fully automated quality control (QC) system for prostate segmentation based on T2-weighted MRI. Four different deep learning-based segmentation methods were used to segment the prostate for 585 patients. First order, shape and textural radiomics features were extracted from the segmented prostate masks. A reference quality score (QS) was calculated for each automated segmentation in comparison to a manual segmentation. A least absolute shrinkage and selection operator (LASSO) was trained and optimized on a randomly assigned training dataset (N = 1756, 439 cases from each segmentation method) to build a generalizable linear regression model based on the radiomics features that best estimated the reference QS. Subsequently, the model was used to estimate the QSs for an independent testing dataset (N = 584, 146 cases from each segmentation method). The mean ± standard deviation absolute error between the estimated and reference QSs was 5.47 ± 6.33 on a scale from 0 to 100. In addition, we found a strong correlation between the estimated and reference QSs (rho = 0.70). In conclusion, we developed an automated QC system that may be helpful for evaluating the quality of automated prostate segmentations.

Optimisation of 2D U-Net Model Components for Automatic Prostate Segmentation on MRI

In this paper, we develop an optimised state-of-the-art 2D U-Net model by studying the effects of the individual deep learning model components in performing prostate segmentation. We found that for upsampling, the combination of interpolation and convolution is better than the use of transposed convolution. For combining feature maps in each convolution block, it is only beneficial if a skip connection with concatenation is used. With respect to pooling, average pooling is better than strided-convolution, max, RMS or L2 pooling. Introducing a batch normalisation layer before the activation layer gives further performance improvement. The optimisation is based on a private dataset as it has a fixed 2D resolution and voxel size for every image which mitigates the need of a resizing operation in the data preparation process. Non-enhancing data preprocessing was applied and five-fold cross-validation was used to evaluate the fully automatic segmentation approach. We show it outperforms the traditional methods that were previously applied on the private dataset, as well as outperforming other comparable state-of-the-art 2D models on the public dataset PROMISE12.

Radiomics and radiogenomics of prostate cancer

Radiomics and radiogenomics are attractive research topics in prostate cancer. Radiomics mainly focuses on extraction of quantitative information from medical imaging, whereas radiogenomics aims to correlate these imaging features to genomic data. The purpose of this review is to provide a brief overview summarizing recent progress in the application of radiomics-based approaches in prostate cancer and to discuss the potential role of radiogenomics in prostate cancer.

Deeply supervised 3D fully convolutional networks with group dilated convolution for automatic MRI prostate segmentation

PURPOSE

Reliable automated segmentation of the prostate is indispensable for image-guided prostate interventions. However, the segmentation task is challenging due to inhomogeneous intensity distributions, variation in prostate anatomy, among other problems. Manual segmentation can be time-consuming and is subject to inter- and intraobserver variation. We developed an automated deep learning-based method to address this technical challenge.

METHODS

We propose a three-dimensional (3D) fully convolutional networks (FCN) with deep supervision and group dilated convolution to segment the prostate on magnetic resonance imaging (MRI). In this method, a deeply supervised mechanism was introduced into a 3D FCN to effectively alleviate the common exploding or vanishing gradients problems in training deep models, which forces the update process of the hidden layer filters to favor highly discriminative features. A group dilated convolution which aggregates multiscale contextual information for dense prediction was proposed to enlarge the effective receptive field of convolutional neural networks, which improve the prediction accuracy of prostate boundary. In addition, we introduced a combined loss function including cosine and cross entropy, which measures similarity and dissimilarity between segmented and manual contours, to further improve the segmentation accuracy. Prostate volumes manually segmented by experienced physicians were used as a gold standard against which our segmentation accuracy was measured.

RESULTS

The proposed method was evaluated on an internal dataset comprising 40 T2-weighted prostate MR volumes. Our method achieved a Dice similarity coefficient (DSC) of 0.86 ± 0.04, a mean surface distance (MSD) of 1.79 ± 0.46 mm, 95% Hausdorff distance (95%HD) of 7.98 ± 2.91 mm, and absolute relative volume difference (aRVD) of 15.65 ± 10.82. A public dataset (PROMISE12) including 50 T2-weighted prostate MR volumes was also employed to evaluate our approach. Our method yielded a DSC of 0.88 ± 0.05, MSD of 1.02 ± 0.35 mm, 95% HD of 9.50 ± 5.11 mm, and aRVD of 8.93 ± 7.56.

CONCLUSION

We developed a novel deeply supervised deep learning-based approach with a group dilated convolution to automatically segment the MRI prostate, demonstrated its clinical feasibility, and validated its accuracy against manual segmentation. The proposed technique could be a useful tool for image-guided interventions in prostate cancer.

Rotationally resliced 3D prostate segmentation of MR images using Bhattacharyya similarity and active band theory

PURPOSE

In this article, we propose a novel, semi-automatic segmentation method to process 3D MR images of the prostate using the Bhattacharyya coefficient and active band theory with the goal of providing technical support for computer-aided diagnosis and surgery of the prostate.

METHODS

Our method consecutively segments a stack of rotationally resectioned 2D slices of a prostate MR image by assessing the similarity of the shape and intensity distribution in neighboring slices. 2D segmentation is first performed on an initial slice by manually selecting several points on the prostate boundary, after which the segmentation results are propagated consecutively to neighboring slices. A framework of iterative graph cuts is used to optimize the energy function, which contains a global term for the Bhattacharyya coefficient with the help of an auxiliary function. Our method does not require previously segmented data for training or for building statistical models, and manual intervention can be applied flexibly and intuitively, indicating the potential utility of this method in the clinic.

RESULTS

We tested our method on 3D T2-weighted MR images from the ISBI dataset and PROMISE12 dataset of 129 patients, and the Dice similarity coefficients were 90.34 ± 2.21% and 89.32 ± 3.08%, respectively. The comparison was performed with several state-of-the-art methods, and the results demonstrate that the proposed method is robust and accurate, achieving similar or higher accuracy than other methods without requiring training.

CONCLUSION

The proposed algorithm for segmenting 3D MR images of the prostate is accurate, robust, and readily applicable to a clinical environment for computer-aided surgery or diagnosis.

Comparative study of algorithms for synthetic CT generation from MRI: Consequences for MRI-guided radiation planning in the pelvic region

PURPOSE

Magnetic resonance imaging (MRI)-guided radiation therapy (RT) treatment planning is limited by the fact that the electron density distribution required for dose calculation is not readily provided by MR imaging. We compare a selection of novel synthetic CT generation algorithms recently reported in the literature, including segmentation-based, atlas-based and machine learning techniques, using the same cohort of patients and quantitative evaluation metrics.

METHODS

Six MRI-guided synthetic CT generation algorithms were evaluated: one segmentation technique into a single tissue class (water-only), four atlas-based techniques, namely, median value of atlas images (ALMedian), atlas-based local weighted voting (ALWV), bone enhanced atlas-based local weighted voting (ALWV-Bone), iterative atlas-based local weighted voting (ALWV-Iter), and a machine learning technique using deep convolution neural network (DCNN).

RESULTS

Organ auto-contouring from MR images was evaluated for bladder, rectum, bones, and body boundary. Overall, DCNN exhibited higher segmentation accuracy resulting in Dice indices (DSC) of 0.93 ± 0.17, 0.90 ± 0.04, and 0.93 ± 0.02 for bladder, rectum, and bones, respectively. On the other hand, ALMedian showed the lowest accuracy with DSC of 0.82 ± 0.20, 0.81 ± 0.08, and 0.88 ± 0.04, respectively. DCNN reached the best performance in terms of accurate derivation of synthetic CT values within each organ, with a mean absolute error within the body contour of 32.7 ± 7.9 HU, followed by the advanced atlas-based methods (ALWV: 40.5 ± 8.2 HU, ALWV-Iter: 42.4 ± 8.1 HU, ALWV-Bone: 44.0 ± 8.9 HU). ALMedian led to the highest error (52.1 ± 11.1 HU). Considering the dosimetric evaluation results, ALWV-Iter, ALWV, DCNN and ALWV-Bone led to similar mean dose estimation within each organ at risk and target volume with less than 1% dose discrepancy. However, the two-dimensional gamma analysis demonstrated higher pass rates for ALWV-Bone, DCNN, ALMedian and ALWV-Iter at 1%/1 mm criterion with 94.99 ± 5.15%, 94.59 ± 5.65%, 93.68 ± 5.53% and 93.10 ± 5.99% success, respectively, while ALWV and water-only resulted in 86.91 ± 13.50% and 80.77 ± 12.10%, respectively.

CONCLUSIONS

Overall, machine learning and advanced atlas-based methods exhibited promising performance by achieving reliable organ segmentation and synthetic CT generation. DCNN appears to have slightly better performance by achieving accurate automated organ segmentation and relatively small dosimetric errors (followed closely by advanced atlas-based methods, which in some cases achieved similar performance). However, the DCNN approach showed higher vulnerability to anatomical variation, where a greater number of outliers was observed with this method. Considering the dosimetric results obtained from the evaluated methods, the challenge of electron density estimation from MR images can be resolved with a clinically tolerable error.

Automatic prostate segmentation on MR images with deep network and graph model

Automated prostate diagnoses and treatments have gained much attention due to the high mortality rate of prostate cancer. In particular, unsupervised (automatic) prostate segmentation is an active and challenging research. Most conventional works usually utilize handcrafted (low-level) features for prostate segmentation; however they often fail to extract the intrinsic structure of the prostate, especially on images with blurred boundaries. In this paper, we propose a novel automated prostate segmentation model with learned features from deep network. Specifically, we first generate a set of prostate proposals in transverse plane via recognizing the position and coarse estimate of the shape of the prostate on the global prostate image and using the deep network to extract highly effective features for the boundary refinement in a finer scale. With consideration of the correlations among different sequential images, we then construct a graph to select the best prostate proposals from proposal set for its use in 3D prostate segmentation. Experimental evaluation demonstrates that our proposed deep network and graph based method is superior to state-of-the-art couterparts, in terms of both dice similarity coefficient and Hausdorff distance, on public dataset.

Multi-atlas and unsupervised learning approach to perirectal space segmentation in CT images

Perirectal space segmentation in computed tomography images aids in quantifying radiation dose received by healthy tissues and toxicity during the course of radiation therapy treatment of the prostate. Radiation dose normalised by tissue volume facilitates predicting outcomes or possible harmful side effects of radiation therapy treatment. Manual segmentation of the perirectal space is time consuming and challenging in the presence of inter-patient anatomical variability and may suffer from inter- and intra-observer variabilities. However automatic or semi-automatic segmentation of the perirectal space in CT images is a challenging task due to inter patient anatomical variability, contrast variability and imaging artifacts. In the model presented here, a volume of interest is obtained in a multi-atlas based segmentation approach. Un-supervised learning in the volume of interest with a Gaussian-mixture-modeling based clustering approach is adopted to achieve a soft segmentation of the perirectal space. Probabilities from soft clustering are further refined by rigid registration of the multi-atlas mask in a probabilistic domain. A maximum a posteriori approach is adopted to achieve a binary segmentation from the refined probabilities. A mean volume similarity value of 97% and a mean surface difference of 3.06 ± 0.51 mm is achieved in a leave-one-patient-out validation framework with a subset of a clinical trial dataset. Qualitative results show a good approximation of the perirectal space volume compared to the ground truth.

Fast automated segmentation of multiple objects via spatially weighted shape learning

Active shape models (ASMs) have proved successful in automatic segmentation by using shape and appearance priors in a number of areas such as prostate segmentation, where accurate contouring is important in treatment planning for prostate cancer. The ASM approach however, is heavily reliant on a good initialisation for achieving high segmentation quality. This initialisation often requires algorithms with high computational complexity, such as three dimensional (3D) image registration. In this work, we present a fast, self-initialised ASM approach that simultaneously fits multiple objects hierarchically controlled by spatially weighted shape learning. Prominent objects are targeted initially and spatial weights are progressively adjusted so that the next (more difficult, less visible) object is simultaneously initialised using a series of weighted shape models. The scheme was validated and compared to a multi-atlas approach on 3D magnetic resonance (MR) images of 38 cancer patients and had the same (mean, median, inter-rater) Dice's similarity coefficients of (0.79, 0.81, 0.85), while having no registration error and a computational time of 12-15 min, nearly an order of magnitude faster than the multi-atlas approach.

Deformable MR Prostate Segmentation via Deep Feature Learning and Sparse Patch Matching

Automatic and reliable segmentation of the prostate is an important but difficult task for various clinical applications such as prostate cancer radiotherapy. The main challenges for accurate MR prostate localization lie in two aspects: (1) inhomogeneous and inconsistent appearance around prostate boundary, and (2) the large shape variation across different patients. To tackle these two problems, we propose a new deformable MR prostate segmentation method by unifying deep feature learning with the sparse patch matching. First, instead of directly using handcrafted features, we propose to learn the latent feature representation from prostate MR images by the stacked sparse auto-encoder (SSAE). Since the deep learning algorithm learns the feature hierarchy from the data, the learned features are often more concise and effective than the handcrafted features in describing the underlying data. To improve the discriminability of learned features, we further refine the feature representation in a supervised fashion. Second, based on the learned features, a sparse patch matching method is proposed to infer a prostate likelihood map by transferring the prostate labels from multiple atlases to the new prostate MR image. Finally, a deformable segmentation is used to integrate a sparse shape model with the prostate likelihood map for achieving the final segmentation. The proposed method has been extensively evaluated on the dataset that contains 66 T2-wighted prostate MR images. Experimental results show that the deep-learned features are more effective than the handcrafted features in guiding MR prostate segmentation. Moreover, our method shows superior performance than other state-of-the-art segmentation methods.

Superpixel-Based Segmentation for 3D Prostate MR Images

This paper proposes a method for segmenting the prostate on magnetic resonance (MR) images. A superpixel-based 3D graph cut algorithm is proposed to obtain the prostate surface. Instead of pixels, superpixels are considered as the basic processing units to construct a 3D superpixel-based graph. The superpixels are labeled as the prostate or background by minimizing an energy function using graph cut based on the 3D superpixel-based graph. To construct the energy function, we proposed a superpixel-based shape data term, an appearance data term, and two superpixel-based smoothness terms. The proposed superpixel-based terms provide the effectiveness and robustness for the segmentation of the prostate. The segmentation result of graph cuts is used as an initialization of a 3D active contour model to overcome the drawback of the graph cut. The result of 3D active contour model is then used to update the shape model and appearance model of the graph cut. Iterations of the 3D graph cut and 3D active contour model have the ability to jump out of local minima and obtain a smooth prostate surface. On our 43 MR volumes, the proposed method yields a mean Dice ratio of 89.3 ±1.9%. On PROMISE12 test data set, our method was ranked at the second place; the mean Dice ratio and standard deviation is 87.0±3.2%. The experimental results show that the proposed method outperforms several state-of-the-art prostate MRI segmentation methods.

Visual saliency-based active learning for prostate magnetic resonance imaging segmentation

We propose an active learning (AL) approach for prostate segmentation from magnetic resonance images. Our label query strategy is inspired from the principles of visual saliency that have similar considerations for choosing the most salient region. These similarities are encoded in a graph using classification maps and low-level features. Random walks are used to identify the most informative node, which is equivalent to the label query sample in AL. To reduce computation time, a volume of interest (VOI) is identified and all subsequent analysis, such as probability map generation using semisupervised random forest classifiers and label query, is restricted to this VOI. The negative log-likelihood of the probability maps serves as the penalty cost in a second-order Markov random field cost function, which is optimized using graph cuts for prostate segmentation. Experimental results on the Medical Image Computing and Computer Assisted Intervention (MICCAI) 2012 prostate segmentation challenge show the superior performance of our approach to conventional methods using fully supervised learning.

The use of atlas registration and graph cuts for prostate segmentation in magnetic resonance images

PURPOSE

An automatic method for 3D prostate segmentation in magnetic resonance (MR) images is presented for planning image-guided radiotherapy treatment of prostate cancer.

METHODS

A spatial prior based on intersubject atlas registration is combined with organ-specific intensity information in a graph cut segmentation framework. The segmentation is tested on 67 axial T2-weighted MR images in a leave-one-out cross validation experiment and compared with both manual reference segmentations and with multiatlas-based segmentations using majority voting atlas fusion. The impact of atlas selection is investigated in both the traditional atlas-based segmentation and the new graph cut method that combines atlas and intensity information in order to improve the segmentation accuracy. Best results were achieved using the method that combines intensity information, shape information, and atlas selection in the graph cut framework.

RESULTS

A mean Dice similarity coefficient (DSC) of 0.88 and a mean surface distance (MSD) of 1.45 mm with respect to the manual delineation were achieved.

CONCLUSIONS

This approaches the interobserver DSC of 0.90 and interobserver MSD 0f 1.15 mm and is comparable to other studies performing prostate segmentation in MR.

Prostate contours delineation using interactive directional active contours model and parametric shape prior model

Prostate contours delineation on Magnetic Resonance (MR) images is a challenging and important task in medical imaging with applications of guiding biopsy, surgery and therapy. While a fully automated method is highly desired for this application, it can be a very difficult task due to the structure and surrounding tissues of the prostate gland. Traditional active contours-based delineation algorithms are typically quite successful for piecewise constant images. Nevertheless, when MR images have diffuse edges or multiple similar objects (e.g. bladder close to prostate) within close proximity, such approaches have proven to be unsuccessful. In order to mitigate these problems, we proposed a new framework for bi-stage contours delineation algorithm based on directional active contours (DAC) incorporating prior knowledge of the prostate shape. We first explicitly addressed the prostate contour delineation problem based on fast globally DAC that incorporates both statistical and parametric shape prior model. In doing so, we were able to exploit the global aspects of contour delineation problem by incorporating a user feedback in contours delineation process where it is shown that only a small amount of user input can sometimes resolve ambiguous scenarios raised by DAC. In addition, once the prostate contours have been delineated, a cost functional is designed to incorporate both user feedback interaction and the parametric shape prior model. Using data from publicly available prostate MR datasets, which includes several challenging clinical datasets, we highlighted the effectiveness and the capability of the proposed algorithm. Besides, the algorithm has been compared with several state-of-the-art methods.

Automatic Substitute Computed Tomography Generation and Contouring for Magnetic Resonance Imaging (MRI)-Alone External Beam Radiation Therapy From Standard MRI Sequences

PURPOSE

To validate automatic substitute computed tomography CT (sCT) scans generated from standard T2-weighted (T2w) magnetic resonance (MR) pelvic scans for MR-Sim prostate treatment planning.

PATIENTS AND METHODS

A Siemens Skyra 3T MR imaging (MRI) scanner with laser bridge, flat couch, and pelvic coil mounts was used to scan 39 patients scheduled for external beam radiation therapy for localized prostate cancer. For sCT generation a whole-pelvis MRI scan (1.6 mm 3-dimensional isotropic T2w SPACE [Sampling Perfection with Application optimized Contrasts using different flip angle Evolution] sequence) was acquired. Three additional small field of view scans were acquired: T2w, T2*w, and T1w flip angle 80° for gold fiducials. Patients received a routine planning CT scan. Manual contouring of the prostate, rectum, bladder, and bones was performed independently on the CT and MR scans. Three experienced observers contoured each organ on MRI, allowing interobserver quantification. To generate a training database, each patient CT scan was coregistered to their whole-pelvis T2w using symmetric rigid registration and structure-guided deformable registration. A new multi-atlas local weighted voting method was used to generate automatic contours and sCT results.

RESULTS

The mean error in Hounsfield units between the sCT and corresponding patient CT (within the body contour) was 0.6 ± 14.7 (mean ± 1 SD), with a mean absolute error of 40.5 ± 8.2 Hounsfield units. Automatic contouring results were very close to the expert interobserver level (Dice similarity coefficient): prostate 0.80 ± 0.08, bladder 0.86 ± 0.12, rectum 0.84 ± 0.06, bones 0.91 ± 0.03, and body 1.00 ± 0.003. The change in monitor units between the sCT-based plans relative to the gold standard CT plan for the same dose prescription was found to be 0.3% ± 0.8%. The 3-dimensional γ pass rate was 1.00 ± 0.00 (2 mm/2%).

CONCLUSIONS

The MR-Sim setup and automatic sCT generation methods using standard MR sequences generates realistic contours and electron densities for prostate cancer radiation therapy dose planning and digitally reconstructed radiograph generation.

Robust inverse-consistent affine CT-MR registration in MRI-assisted and MRI-alone prostate radiation therapy

BACKGROUND

CT-MR registration is a critical component of many radiation oncology protocols. In prostate external beam radiation therapy, it allows the propagation of MR-derived contours to reference CT images at the planning stage, and it enables dose mapping during dosimetry studies. The use of carefully registered CT-MR atlases allows the estimation of patient specific electron density maps from MRI scans, enabling MRI-alone radiation therapy planning and treatment adaptation. In all cases, the precision and accuracy achieved by registration influences the quality of the entire process.

PROBLEM

Most current registration algorithms do not robustly generalize and lack inverse-consistency, increasing the risk of human error and acting as a source of bias in studies where information is propagated in a particular direction, e.g. CT to MR or vice versa. In MRI-based treatment planning where both CT and MR scans serve as spatial references, inverse-consistency is critical, if under-acknowledged.

PURPOSE

A robust, inverse-consistent, rigid/affine registration algorithm that is well suited to CT-MR alignment in prostate radiation therapy is presented.

METHOD

The presented method is based on a robust block-matching optimization process that utilises a half-way space definition to maintain inverse-consistency. Inverse-consistency substantially reduces the influence of the order of input images, simplifying analysis, and increasing robustness. An open source implementation is available online at http://aehrc.github.io/Mirorr/.

RESULTS

Experimental results on a challenging 35 CT-MR pelvis dataset demonstrate that the proposed method is more accurate than other popular registration packages and is at least as accurate as the state of the art, while being more robust and having an order of magnitude higher inverse-consistency than competing approaches.

CONCLUSION

The presented results demonstrate that the proposed registration algorithm is readily applicable to prostate radiation therapy planning.

Computer aided-diagnosis of prostate cancer on multiparametric MRI: a technical review of current research

Prostate cancer (PCa) is the most commonly diagnosed cancer among men in the United States. In this paper, we survey computer aided-diagnosis (CADx) systems that use multiparametric magnetic resonance imaging (MP-MRI) for detection and diagnosis of prostate cancer. We review and list mainstream techniques that are commonly utilized in image segmentation, registration, feature extraction, and classification. The performances of 15 state-of-the-art prostate CADx systems are compared through the area under their receiver operating characteristic curves (AUC). Challenges and potential directions to further the research of prostate CADx are discussed in this paper. Further improvements should be investigated to make prostate CADx systems useful in clinical practice.

Automated segmentation and analysis of normal and osteoarthritic knee menisci from magnetic resonance images--data from the Osteoarthritis Initiative

OBJECTIVE

To validate an automatic scheme for the segmentation and quantitative analysis of the medial meniscus (MM) and lateral meniscus (LM) in magnetic resonance (MR) images of the knee.

METHOD

We analysed sagittal water-excited double-echo steady-state MR images of the knee from a subset of the Osteoarthritis Initiative (OAI) cohort. The MM and LM were automatically segmented in the MR images based on a deformable model approach. Quantitative parameters including volume, subluxation and tibial-coverage were automatically calculated for comparison (Wilcoxon tests) between knees with variable radiographic osteoarthritis (rOA), medial and lateral joint space narrowing (mJSN, lJSN) and pain. Automatic segmentations and estimated parameters were evaluated for accuracy using manual delineations of the menisci in 88 pathological knee MR examinations at baseline and 12 months time-points.

RESULTS

The median (95% confidence-interval (CI)) Dice similarity index (DSI) (2 ∗|Auto ∩ Manual|/(|Auto|+|Manual|)∗ 100) between manual and automated segmentations for the MM and LM volumes were 78.3% (75.0-78.7), 83.9% (82.1-83.9) at baseline and 75.3% (72.8-76.9), 83.0% (81.6-83.5) at 12 months. Pearson coefficients between automatic and manual segmentation parameters ranged from r = 0.70 to r = 0.92. MM in rOA/mJSN knees had significantly greater subluxation and smaller tibial-coverage than no-rOA/no-mJSN knees. LM in rOA knees had significantly greater volumes and tibial-coverage than no-rOA knees.

CONCLUSION

Our automated method successfully segmented the menisci in normal and osteoarthritic knee MR images and detected meaningful morphological differences with respect to rOA and joint space narrowing (JSN). Our approach will facilitate analyses of the menisci in prospective MR cohorts such as the OAI for investigations into pathophysiological changes occurring in early osteoarthritis (OA) development.

Prostate segmentation based on variant scale patch and local independent projection

Accurate segmentation of the prostate in computed tomography (CT) images is important in image-guided radiotherapy; however, difficulties remain associated with this task. In this study, an automatic framework is designed for prostate segmentation in CT images. We propose a novel image feature extraction method, namely, variant scale patch, which can provide rich image information in a low dimensional feature space. We assume that the samples from different classes lie on different nonlinear submanifolds and design a new segmentation criterion called local independent projection (LIP). In our method, a dictionary containing training samples is constructed. To utilize the latest image information, we use an online updated strategy to construct this dictionary. In the proposed LIP, locality is emphasized rather than sparsity; local anchor embedding is performed to determine the dictionary coefficients. Several morphological operations are performed to improve the achieved results. The proposed method has been evaluated based on 330 3-D images of 24 patients. Results show that the proposed method is robust and effective in segmenting prostate in CT images.

Fast globally optimal segmentation of 3D prostate MRI with axial symmetry prior

We propose a novel global optimization approach to segmenting a given 3D prostate T2w magnetic resonance (MR) image, which enforces the inherent axial symmetry of the prostate shape and simultaneously performs a sequence of 2D axial slice-wise segmentations with a global 3D coherence prior. We show that the proposed challenging combinatorial optimization problem can be solved globally and exactly by means of convex relaxation. With this regard, we introduce a novel coupled continuous max-flow model, which is dual to the studied convex relaxed optimization formulation and leads to an efficient multiplier augmented algorithm based on the modern convex optimization theory. Moreover, the new continuous max-flow based algorithm was implemented on GPUs to achieve a substantial improvement in computation. Experimental results using public and in-house datasets demonstrate great advantages of the proposed method in terms of both accuracy and efficiency.

Prostate segmentation: an efficient convex optimization approach with axial symmetry using 3-D TRUS and MR images

We propose a novel global optimization-based approach to segmentation of 3-D prostate transrectal ultrasound (TRUS) and T2 weighted magnetic resonance (MR) images, enforcing inherent axial symmetry of prostate shapes to simultaneously adjust a series of 2-D slice-wise segmentations in a "global" 3-D sense. We show that the introduced challenging combinatorial optimization problem can be solved globally and exactly by means of convex relaxation. In this regard, we propose a novel coherent continuous max-flow model (CCMFM), which derives a new and efficient duality-based algorithm, leading to a GPU-based implementation to achieve high computational speeds. Experiments with 25 3-D TRUS images and 30 3-D T2w MR images from our dataset, and 50 3-D T2w MR images from a public dataset, demonstrate that the proposed approach can segment a 3-D prostate TRUS/MR image within 5-6 s including 4-5 s for initialization, yielding a mean Dice similarity coefficient of 93.2%±2.0% for 3-D TRUS images and 88.5%±3.5% for 3-D MR images. The proposed method also yields relatively low intra- and inter-observer variability introduced by user manual initialization, suggesting a high reproducibility, independent of observers.

Evaluation of prostate segmentation algorithms for MRI: the PROMISE12 challenge

Prostate MRI image segmentation has been an area of intense research due to the increased use of MRI as a modality for the clinical workup of prostate cancer. Segmentation is useful for various tasks, e.g. to accurately localize prostate boundaries for radiotherapy or to initialize multi-modal registration algorithms. In the past, it has been difficult for research groups to evaluate prostate segmentation algorithms on multi-center, multi-vendor and multi-protocol data. Especially because we are dealing with MR images, image appearance, resolution and the presence of artifacts are affected by differences in scanners and/or protocols, which in turn can have a large influence on algorithm accuracy. The Prostate MR Image Segmentation (PROMISE12) challenge was setup to allow a fair and meaningful comparison of segmentation methods on the basis of performance and robustness. In this work we will discuss the initial results of the online PROMISE12 challenge, and the results obtained in the live challenge workshop hosted by the MICCAI2012 conference. In the challenge, 100 prostate MR cases from 4 different centers were included, with differences in scanner manufacturer, field strength and protocol. A total of 11 teams from academic research groups and industry participated. Algorithms showed a wide variety in methods and implementation, including active appearance models, atlas registration and level sets. Evaluation was performed using boundary and volume based metrics which were combined into a single score relating the metrics to human expert performance. The winners of the challenge where the algorithms by teams Imorphics and ScrAutoProstate, with scores of 85.72 and 84.29 overall. Both algorithms where significantly better than all other algorithms in the challenge (p<0.05) and had an efficient implementation with a run time of 8min and 3s per case respectively. Overall, active appearance model based approaches seemed to outperform other approaches like multi-atlas registration, both on accuracy and computation time. Although average algorithm performance was good to excellent and the Imorphics algorithm outperformed the second observer on average, we showed that algorithm combination might lead to further improvement, indicating that optimal performance for prostate segmentation is not yet obtained. All results are available online at http://promise12.grand-challenge.org/.

The use of an active appearance model for automated prostate segmentation in magnetic resonance

BACKGROUND

The prostate gland is delineated as the clinical target volume (CTV) in treatment planning of prostate cancer. Therefore, an accurate delineation is a prerequisite for efficient treatment. Accurate automated prostate segmentation methods facilitate the delineation of the CTV without inter-observer variation. The purpose of this study is to present an automated three-dimensional (3D) segmentation of the prostate using an active appearance model.

MATERIAL AND METHODS

Axial T2-weighted magnetic resonance (MR) scans were used to build the active appearance model. The model was based on a principal component analysis of shape and texture features with a level-set representation of the prostate shape instead of the selection of landmarks in the traditional active appearance model. To achieve a better fit of the model to the target image, prior knowledge to predict how to correct the model and pose parameters was incorporated. The segmentation was performed as an iterative algorithm to minimize the squared difference between the target and the model image.

RESULTS

The model was trained using manual delineations from 30 patients and was validated using leave-one-out cross validation where the automated segmentations were compared with the manual reference delineations. The mean and median dice similarity coefficient was 0.84 and 0.86, respectively.

CONCLUSION

This study demonstrated the feasibility for an automated prostate segmentation using an active appearance with results comparable to other studies.

Automatic segmentation of male pelvic anatomy on computed tomography images: a comparison with multiple observers in the context of a multicentre clinical trial

Background

This study investigates the variation in segmentation of several pelvic anatomical structures on computed tomography (CT) between multiple observers and a commercial automatic segmentation method, in the context of quality assurance and evaluation during a multicentre clinical trial.

Methods

CT scans of two prostate cancer patients (‘benchmarking cases’), one high risk (HR) and one intermediate risk (IR), were sent to multiple radiotherapy centres for segmentation of prostate, rectum and bladder structures according to the TROG 03.04 “RADAR” trial protocol definitions. The same structures were automatically segmented using iPlan software for the same two patients, allowing structures defined by automatic segmentation to be quantitatively compared with those defined by multiple observers. A sample of twenty trial patient datasets were also used to automatically generate anatomical structures for quantitative comparison with structures defined by individual observers for the same datasets.

Results

There was considerable agreement amongst all observers and automatic segmentation of the benchmarking cases for bladder (mean spatial variations < 0.4 cm across the majority of image slices). Although there was some variation in interpretation of the superior-inferior (cranio-caudal) extent of rectum, human-observer contours were typically within a mean 0.6 cm of automatically-defined contours. Prostate structures were more consistent for the HR case than the IR case with all human observers segmenting a prostate with considerably more volume (mean +113.3%) than that automatically segmented. Similar results were seen across the twenty sample datasets, with disagreement between iPlan and observers dominant at the prostatic apex and superior part of the rectum, which is consistent with observations made during quality assurance reviews during the trial.

Conclusions

This study has demonstrated quantitative analysis for comparison of multi-observer segmentation studies. For automatic segmentation algorithms based on image-registration as in iPlan, it is apparent that agreement between observer and automatic segmentation will be a function of patient-specific image characteristics, particularly for anatomy with poor contrast definition. For this reason, it is suggested that automatic registration based on transformation of a single reference dataset adds a significant systematic bias to the resulting volumes and their use in the context of a multicentre trial should be carefully considered.

Sparse patch-based label propagation for accurate prostate localization in CT images

In this paper, we propose a new prostate computed tomography (CT) segmentation method for image guided radiation therapy. The main contributions of our method lie in the following aspects. 1) Instead of using voxel intensity information alone, patch-based representation in the discriminative feature space with logistic sparse LASSO is used as anatomical signature to deal with low contrast problem in prostate CT images. 2) Based on the proposed patch-based signature, a new multi-atlases label fusion method formulated under sparse representation framework is designed to segment prostate in the new treatment images, with guidance from the previous segmented images of the same patient. This method estimates the prostate likelihood of each voxel in the new treatment image from its nearby candidate voxels in the previous segmented images, based on the nonlocal mean principle and sparsity constraint. 3) A hierarchical labeling strategy is further designed to perform label fusion, where voxels with high confidence are first labeled for providing useful context information in the same image for aiding the labeling of the remaining voxels. 4) An online update mechanism is finally adopted to progressively collect more patient-specific information from newly segmented treatment images of the same patient, for adaptive and more accurate segmentation. The proposed method has been extensively evaluated on a prostate CT image database consisting of 24 patients where each patient has more than 10 treatment images, and further compared with several state-of-the-art prostate CT segmentation algorithms using various evaluation metrics. Experimental results demonstrate that the proposed method consistently achieves higher segmentation accuracy than any other methods under comparison.

Automatic prostate MR image segmentation with sparse label propagation and domain-specific manifold regularization

Automatic prostate segmentation in MR images plays an important role in prostate cancer diagnosis. However, there are two main challenges: (1) Large inter-subject prostate shape variations; (2) Inhomogeneous prostate appearance. To address these challenges, we propose a new hierarchical prostate MR segmentation method, with the main contributions lying in the following aspects: First, the most salient features are learnt from atlases based on a subclass discriminant analysis (SDA) method, which aims to find a discriminant feature subspace by simultaneously maximizing the inter-class distance and minimizing the intra-class variations. The projected features, instead of only voxel-wise intensity, will be served as anatomical signature of each voxel. Second, based on the projected features, a new multi-atlases sparse label fusion framework is proposed to estimate the prostate likelihood of each voxel in the target image from the coarse level. Third, a domain-specific semi-supervised manifold regularization method is proposed to incorporate the most reliable patient-specific information identified by the prostate likelihood map to refine the segmentation result from the fine level. Our method is evaluated on a T2 weighted prostate MR image dataset consisting of 66 patients and compared with two state-of-the-art segmentation methods. Experimental results show that our method consistently achieves the highest segmentation accuracies than other methods under comparison.