A level set based framework for quantitative evaluation of breast tissue density from MRI data

A publicly available deep learning model and dataset for segmentation of breast, fibroglandular tissue, and vessels in breast MRI

Breast density, or the amount of fibroglandular tissue (FGT) relative to the overall breast volume, increases the risk of developing breast cancer. Although previous studies have utilized deep learning to assess breast density, the limited public availability of data and quantitative tools hinders the development of better assessment tools. Our objective was to (1) create and share a large dataset of pixel-wise annotations according to well-defined criteria, and (2) develop, evaluate, and share an automated segmentation method for breast, FGT, and blood vessels using convolutional neural networks. We used the Duke Breast Cancer MRI dataset to randomly select 100 MRI studies and manually annotated the breast, FGT, and blood vessels for each study. Model performance was evaluated using the Dice similarity coefficient (DSC). The model achieved DSC values of 0.92 for breast, 0.86 for FGT, and 0.65 for blood vessels on the test set. The correlation between our model's predicted breast density and the manually generated masks was 0.95. The correlation between the predicted breast density and qualitative radiologist assessment was 0.75. Our automated models can accurately segment breast, FGT, and blood vessels using pre-contrast breast MRI data. The data and the models were made publicly available.

SHIP-MR and Radiology: 12 Years of Whole-Body Magnetic Resonance Imaging in a Single Center

The Study of Health in Pomerania (SHIP), a population-based study from a rural state in northeastern Germany with a relatively poor life expectancy, supplemented its comprehensive examination program in 2008 with whole-body MR imaging at 1.5 T (SHIP-MR). We reviewed more than 100 publications that used the SHIP-MR data and analyzed which sequences already produced fruitful scientific outputs and which manuscripts have been referenced frequently. Upon reviewing the publications about imaging sequences, those that used T1-weighted structured imaging of the brain and a gradient-echo sequence for R2* mapping obtained the highest scientific output; regarding specific body parts examined, most scientific publications focused on MR sequences involving the brain and the (upper) abdomen. We conclude that population-based MR imaging in cohort studies should define more precise goals when allocating imaging time. In addition, quality control measures might include recording the number and impact of published work, preferably on a bi-annual basis and starting 2 years after initiation of the study. Structured teaching courses may enhance the desired output in areas that appear underrepresented.

Segmentation of whole breast and fibroglandular tissue using nnU-Net in dynamic contrast enhanced MR images

PURPOSE

Segmentation of the whole breast and fibroglandular tissue (FGT) is important for quantitatively analyzing the breast cancer risk in the dynamic contrast-enhanced magnetic resonance (DCE-MR) images. The purpose of this study is to improve the accuracy and efficiency of the segmentation of the whole breast and FGT in 3-D fat-suppressed DCE-MR images with a versatile deep learning (DL) framework.

METHODS

We randomly collected 100 breast DCE-MR scans from Shanghai Cancer Hospital of Fudan University. The MR scans in the dataset were different in both the spatial resolution and the MR scanners employed. Furthermore, four breast density categories were assessed by radiologists based on Breast Imaging Reporting and Data System (BI-RADS) of American College of Radiology. The dataset was separated into the training and the testing sets, while keeping a balanced distribution of scans with different imaging parameters and density categories. The nnU-Net has been recently proposed to automatically adapt preprocessing strategies and network architectures for a given medical image dataset, thus showing a great potential in the systematic adaptation of DL methods to different datasets. In this study, we applied the nnU-Net to segment the whole breast and FGT in 3-D fat-suppressed DCE-MR images. Five-fold cross validation was employed to train and validate the segmentation method.

RESULTS

The segmentation performance was evaluated with the volume and surface agreement metrics between the DL-based automatic and the manually delineated masks, as quantified with the following measures: the average Dice volume overlap (0.968 ± 0.017 and 0.877 ± 0.081), the average surface distances (0.201 ± 0.080 mm and 0.310 ± 0.043 mm), and the Pearson correlation coefficient of masks (0.995 and 0.972) between the automatic and the manually delineated masks, as calculated for the whole breast and the FGT segmentation, respectively. The correlation coefficient between the breast densities obtained with the DL-based segmentation and the manual delineation was 0.981. There was a positive bias of 0.8% (DL-based relative to manual) in breast density measurement with the Bland-Altman plot. The execution time of the DL-based segmentation was approximately 20 s for the whole breast segmentation and 15 s for the FGT segmentation.

CONCLUSIONS

Our DL-based segmentation framework using nnU-Net could robustly achieve high accuracy and efficiency across variable MR imaging settings without extra pre- or post-processing procedures. It would be useful for developing DCE-MR-based CAD systems to quantify breast cancer risk and to be integrated into the clinical workflow.

Fully automatic quantification of fibroglandular tissue and background parenchymal enhancement with accurate implementation for axial and sagittal breast MRI protocols

PURPOSE

To propose and evaluate a fully automated technique for quantification of fibroglandular tissue (FGT) and background parenchymal enhancement (BPE) in breast MRI.

METHODS

We propose a fully automated method, where after preprocessing, FGT is segmented in T1-weighted, nonfat-saturated MRI. Incorporating an anatomy-driven prior probability for FGT and robust texture descriptors against intensity variations, our method effectively addresses major image processing challenges, including wide variations in breast anatomy and FGT appearance among individuals. Our framework then propagates this segmentation to dynamic contrast-enhanced (DCE)-MRI to quantify BPE within the segmented FGT regions. Axial and sagittal image data from 40 cancer-unaffected women were used to evaluate our proposed method vs a manually annotated reference standard.

RESULTS

High spatial correspondence was observed between the automatic and manual FGT segmentation (mean Dice similarity coefficient 81.14%). The FGT and BPE quantifications (denoted FGT% and BPE%) indicated high correlation (Pearson's r = 0.99 for both) between automatic and manual segmentations. Furthermore, the differences between the FGT% and BPE% quantified using automatic and manual segmentations were low (mean differences: -0.66 ± 2.91% for FGT% and -0.17 ± 1.03% for BPE%). When correlated with qualitative clinical BI-RADS ratings, the correlation coefficient for FGT% was still high (Spearman's ρ = 0.92), whereas that for BPE was lower (ρ = 0.65). Our proposed approach also performed significantly better than a previously validated method for sagittal breast MRI.

CONCLUSIONS

Our method demonstrated accurate fully automated quantification of FGT and BPE in both sagittal and axial breast MRI. Our results also suggested the complexity of BPE assessment, demonstrating relatively low correlation between segmentation and clinical rating.

A deep learning framework for efficient analysis of breast volume and fibroglandular tissue using MR data with strong artifacts

PURPOSE

The main purpose of this work is to develop, apply, and evaluate an efficient approach for breast density estimation in magnetic resonance imaging data, which contain strong artifacts including intensity inhomogeneities.

METHODS

We present a pipeline for breast density estimation, which consists of intensity inhomogeneity correction, breast volume segmentation, nipple extraction, and fibroglandular tissue segmentation. For the segmentation steps, a well-known deep learning architecture is employed.

RESULTS

The average Dice coefficient for the breast parenchyma is [Formula: see text], which outperforms the classical state-of-the-art approach by a margin of [Formula: see text].

CONCLUSION

The proposed solution is accurate and highly efficient and has potential to be applied for big epidemiological data with thousands of participants.

An investigation of the effect of fat suppression and dimensionality on the accuracy of breast MRI segmentation using U-nets

PURPOSE

Accurate segmentation of the breast is required for breast density estimation and the assessment of background parenchymal enhancement, both of which have been shown to be related to breast cancer risk. The MRI breast segmentation task is challenging, and recent work has demonstrated that convolutional neural networks perform well for this task. In this study, we have investigated the performance of several two-dimensional (2D) U-Net and three-dimensional (3D) U-Net configurations using both fat-suppressed and nonfat-suppressed images. We have also assessed the effect of changing the number and quality of the ground truth segmentations.

MATERIALS AND METHODS

We designed eight studies to investigate the effect of input types and the dimensionality of the U-Net operations for the breast MRI segmentation. Our training data contained 70 whole breast volumes of T1-weighted sequences without fat suppression (WOFS) and with fat suppression (FS). For each subject, we registered the WOFS and FS volumes together before manually segmenting the breast to generate ground truth. We compared four different input types to the U-nets: WOFS, FS, MIXED (WOFS and FS images treated as separate samples), and MULTI (WOFS and FS images combined into a single multichannel image). We trained 2D U-Nets and 3D U-Nets with these data, which resulted in our eight studies (2D-WOFS, 3D-WOFS, 2D-FS, 3D-FS, 2D-MIXED, 3D-MIXED, 2D-MULTI, and 3D-MULT). For each of these studies, we performed a systematic grid search to tune the hyperparameters of the U-Nets. A separate validation set with 15 whole breast volumes was used for hyperparameter tuning. We performed Kruskal-Walis test on the results of our hyperparameter tuning and did not find a statistically significant difference in the ten top models of each study. For this reason, we chose the best model as the model with the highest mean dice similarity coefficient (DSC) value on the validation set. The reported test results are the results of the top model of each study on our test set which contained 19 whole breast volumes annotated by three readers fused with the STAPLE algorithm. We also investigated the effect of the quality of the training annotations and the number of training samples for this task.

RESULTS

The study with the highest average DSC result was 3D-MULTI with 0.96 ± 0.02. The second highest average is 2D WOFS (0.96 ± 0.03), and the third is 2D MULTI (0.96 ± 0.03). We performed the Kruskal-Wallis one-way ANOVA test with Dunn's multiple comparison tests using Bonferroni P-value correction on the results of the selected model of each study and found that 3D-MULTI, 2D-MULTI, 3D-WOFS, 2D-WOFS, 2D-FS, and 3D-FS were not statistically different in their distributions, which indicates that comparable results could be obtained in fat-suppressed and nonfat-suppressed volumes and that there is no significant difference between the 3D and 2D approach. Our results also suggested that the networks trained on single sequence images or multiple sequence images organized in multichannel images perform better than the models trained on a mixture of volumes from different sequences. Our investigation of the size of the training set revealed that training a U-Net in this domain only requires a modest amount of training data and results obtained with 49 and 70 training datasets were not significantly different.

CONCLUSIONS

To summarize, we investigated the use of 2D U-Nets and 3D U-Nets for breast volume segmentation in T1 fat-suppressed and without fat-suppressed volumes. Although our highest score was obtained in the 3D MULTI study, when we took advantage of information in both fat-suppressed and nonfat-suppressed volumes and their 3D structure, all of the methods we explored gave accurate segmentations with an average DSC on >94% demonstrating that the U-Net is a robust segmentation method for breast MRI volumes.

Automatic and fast segmentation of breast region-of-interest (ROI) and density in MRIs

Accurate segmentation of the breast region of interest (BROI) and breast density (BD) is a significant challenge during the analysis of breast MR images. Most of the existing methods for breast segmentation are semi-automatic and limited in their ability to achieve accurate results. This is because of difficulties in removing landmarks from noisy magnetic resonance images (MRI) due to similar intensity levels and the close connection to BROI. This study proposes an innovative, fully automatic and fast segmentation approach to identify and remove landmarks such as the heart and pectoral muscles. The BROI segmentation is carried out with a framework consisting of three major steps. Firstly, we use adaptive wiener filtering and k-means clustering to minimize the influence of noises, preserve edges and remove unwanted artefacts. The second step systematically excludes the heart area by utilizing active contour based level sets where initial contour points are determined by the maximum entropy thresholding and convolution method. Finally, a pectoral muscle is removed by using morphological operations and local adaptive thresholding on MR images. Prior to the elimination of the pectoral muscle, the MR image is sub divided into three sections: left, right, and central based on the geometrical information. Subsequently, a BD segmentation is achieved with 4 level fuzzy c-means (FCM) thresholding on the denoised BROI segmentation. The proposed method is validated using the 1350 breast images from 15 female subjects. The pixel-based quantitative analysis showed excellent segmentation results when compared with manually drawn BROI and BD. Furthermore, the presented results in terms of evaluation matrices: Acc, Sp, AUC, MR, P, Se and DSC demonstrate the high quality of segmentations using the proposed method. The average computational time for the segmentation of BROI and BD is 1 minute and 50 seconds.

Automatic outer and inner breast tissue segmentation using multi-parametric MRI images of breast tumor patients

The objectives of the study were to develop a framework for automatic outer and inner breast tissue segmentation using multi-parametric MRI images of the breast tumor patients; and to perform breast density and tumor tissue analysis. MRI of the breast was performed on 30 patients at 3T-MRI. T₁, T₂ and PD-weighted(W) images, with and without fat saturation(WWFS), and dynamic-contrast-enhanced(DCE)-MRI data were acquired. The proposed automatic segmentation approach was performed in two steps. In step-1, outer segmentation of breast tissue from rest of body parts was performed on structural images (T₂-W/T₁-W/PD-W without fat saturation images) using automatic landmarks detection technique based on operations like profile screening, Otsu thresholding, morphological operations and empirical observation. In step-2, inner segmentation of breast tissue into fibro-glandular(FG), fatty and tumor tissue was performed. For validation of breast tissue segmentation, manual segmentation was carried out by two radiologists and similarity coefficients(Dice and Jaccard) were computed for outer as well as inner tissues. FG density and tumor volume were also computed and analyzed. The proposed outer and inner segmentation approach worked well for all the subjects and was validated by two radiologists. The average Dice and Jaccard coefficients value for outer segmentation using T₂-W images, obtained by two radiologists, were 0.977 and 0.951 respectively. These coefficient values for FG tissue were 0.915 and 0.875 respectively whereas for tumor tissue, values were 0.968 and 0.95 respectively. The volume of segmented tumor ranged over 2.1 cm3–7.08 cm³. The proposed approach provided automatic outer and inner breast tissue segmentation, which enables automatic calculations of breast tissue density and tumor volume. This is a complete framework for outer and inner breast segmentation method for all structural images.

Using deep learning to segment breast and fibroglandular tissue in MRI volumes

PURPOSE

Automated segmentation of breast and fibroglandular tissue (FGT) is required for various computer-aided applications of breast MRI. Traditional image analysis and computer vision techniques, such atlas, template matching, or, edge and surface detection, have been applied to solve this task. However, applicability of these methods is usually limited by the characteristics of the images used in the study datasets, while breast MRI varies with respect to the different MRI protocols used, in addition to the variability in breast shapes. All this variability, in addition to various MRI artifacts, makes it a challenging task to develop a robust breast and FGT segmentation method using traditional approaches. Therefore, in this study, we investigated the use of a deep-learning approach known as "U-net."

MATERIALS AND METHODS

We used a dataset of 66 breast MRI's randomly selected from our scientific archive, which includes five different MRI acquisition protocols and breasts from four breast density categories in a balanced distribution. To prepare reference segmentations, we manually segmented breast and FGT for all images using an in-house developed workstation. We experimented with the application of U-net in two different ways for breast and FGT segmentation. In the first method, following the same pipeline used in traditional approaches, we trained two consecutive (2C) U-nets: first for segmenting the breast in the whole MRI volume and the second for segmenting FGT inside the segmented breast. In the second method, we used a single 3-class (3C) U-net, which performs both tasks simultaneously by segmenting the volume into three regions: nonbreast, fat inside the breast, and FGT inside the breast. For comparison, we applied two existing and published methods to our dataset: an atlas-based method and a sheetness-based method. We used Dice Similarity Coefficient (DSC) to measure the performances of the automated methods, with respect to the manual segmentations. Additionally, we computed Pearson's correlation between the breast density values computed based on manual and automated segmentations.

RESULTS

The average DSC values for breast segmentation were 0.933, 0.944, 0.863, and 0.848 obtained from 3C U-net, 2C U-nets, atlas-based method, and sheetness-based method, respectively. The average DSC values for FGT segmentation obtained from 3C U-net, 2C U-nets, and atlas-based methods were 0.850, 0.811, and 0.671, respectively. The correlation between breast density values based on 3C U-net and manual segmentations was 0.974. This value was significantly higher than 0.957 as obtained from 2C U-nets (P < 0.0001, Steiger's Z-test with Bonferoni correction) and 0.938 as obtained from atlas-based method (P = 0.0016).

CONCLUSIONS

In conclusion, we applied a deep-learning method, U-net, for segmenting breast and FGT in MRI in a dataset that includes a variety of MRI protocols and breast densities. Our results showed that U-net-based methods significantly outperformed the existing algorithms and resulted in significantly more accurate breast density computation.

3D Regression Heat Map Analysis of Population Study Data

Epidemiological studies comprise heterogeneous data about a subject group to define disease-specific risk factors. These data contain information (features) about a subject's lifestyle, medical status as well as medical image data. Statistical regression analysis is used to evaluate these features and to identify feature combinations indicating a disease (the target feature). We propose an analysis approach of epidemiological data sets by incorporating all features in an exhaustive regression-based analysis. This approach combines all independent features w.r.t. a target feature. It provides a visualization that reveals insights into the data by highlighting relationships. The 3D Regression Heat Map, a novel 3D visual encoding, acts as an overview of the whole data set. It shows all combinations of two to three independent features with a specific target disease. Slicing through the 3D Regression Heat Map allows for the detailed analysis of the underlying relationships. Expert knowledge about disease-specific hypotheses can be included into the analysis by adjusting the regression model formulas. Furthermore, the influences of features can be assessed using a difference view comparing different calculation results. We applied our 3D Regression Heat Map method to a hepatic steatosis data set to reproduce results from a data mining-driven analysis. A qualitative analysis was conducted on a breast density data set. We were able to derive new hypotheses about relations between breast density and breast lesions with breast cancer. With the 3D Regression Heat Map, we present a visual overview of epidemiological data that allows for the first time an interactive regression-based analysis of large feature sets with respect to a disease.

An efficient level set method for simultaneous intensity inhomogeneity correction and segmentation of MR images

Intensity inhomogeneity (bias field) is a common artefact in magnetic resonance (MR) images, which hinders successful automatic segmentation. In this work, a novel algorithm for simultaneous segmentation and bias field correction is presented. The proposed energy functional allows for explicit regularization of the bias field term, making the model more flexible, which is crucial in presence of strong inhomogeneities. An efficient minimization procedure, attempting to find the global minimum, is applied to the energy functional. The algorithm is evaluated qualitatively and quantitatively using a synthetic example and real MR images of different organs. Comparisons with several state-of-the-art methods demonstrate the superior performance of the proposed technique. Desirable results are obtained even for images with strong and complicated inhomogeneity fields and sparse tissue structures.