Finding corresponding regions of interest in mediolateral oblique and craniocaudal mammographic views

Uncertainty Estimation for Dual View X-ray Mammographic Image Registration Using Deep Ensembles

Techniques are developed for generating uncertainty estimates for convolutional neural network (CNN)-based methods for registering the locations of lesions between the craniocaudal (CC) and mediolateral oblique (MLO) mammographic X-ray image views. Multi-view lesion correspondence is an important task that clinicians perform for characterizing lesions during routine mammographic exams. Automated registration tools can aid in this task, yet if the tools also provide confidence estimates, they can be of greater value to clinicians, especially in cases involving dense tissue where lesions may be difficult to see. A set of deep ensemble-based techniques, which leverage a negative log-likelihood (NLL)-based cost function, are implemented for estimating uncertainties. The ensemble architectures involve significant modifications to an existing CNN dual-view lesion registration algorithm. Three architectural designs are evaluated, and different ensemble sizes are compared using various performance metrics. The techniques are tested on synthetic X-ray data, real 2D X-ray data, and slices from real 3D X-ray data. The ensembles generate covariance-based uncertainty ellipses that are correlated with registration accuracy, such that the ellipse sizes can give a clinician an indication of confidence in the mapping between the CC and MLO views. The results also show that the ellipse sizes can aid in improving computer-aided detection (CAD) results by matching CC/MLO lesion detects and reducing false alarms from both views, adding to clinical utility. The uncertainty estimation techniques show promise as a means for aiding clinicians in confidently establishing multi-view lesion correspondence, thereby improving diagnostic capability.

Prediction of Short-Term Breast Cancer Risk with Fusion of CC- and MLO-Based Risk Models in Four-View Mammograms

This study performed and assessed a novel program to improve the accuracy of short-term breast cancer risk prediction by using information from craniocaudal (CC) and mediolateral-oblique (MLO) views of two breasts. An age-matched dataset of 556 patients with at least two sequential full-field digital mammography examinations was applied. In the second examination, 278 cases were diagnosed and pathologically verified as cancer, and 278 were negative, while all cases in the first examination were negative (not recalled). Two generalized linear-model-based risk prediction models were established with global- and local-based bilateral asymmetry features for CC and MLO views first. Then, a new fusion risk model was developed by fusing prediction results of the CC- and MLO-based risk models with an adaptive alpha-integration-based fusion method. The AUC of the fusion risk model was 0.72 ± 0.02, which was significantly higher than the AUC of CC- or MLO-based risk model (P < 0.05). The maximum odds ratio for CC- and MLO-based risk models were 8.09 and 5.25, respectively, and increased to 11.99 for the fusion risk model. For subgroups of patients aged 37-49 years, 50-65 years, and 66-87 years, the AUCs of 0.73, 0.71, and 0.75 for the fusion risk model were higher than AUC for CC- and MLO-based risk models. For the BIRADS 2 and 3 subgroups, the AUC values were 0.72 and 0.71 respectively for the fusion risk model which were higher than the AUC for the CC- and MLO-based risk models. This study demonstrated that the fusion risk model we established could effectively derive and integrate supplementary and useful information extracted from both CC and MLO view images and adaptively fuse them to increase the predictive power of the short-term breast cancer risk assessment model.

STATISTICAL MODELING OF GLANDULARITY FROM MAMMOGRAPHY IMAGES

This study presents a methodology for estimation of breast glandularity, which is an important factor to assess radiological risk in mammography patients. The investigation took place in an institution located at department of Antioquia-Colombia, where 200 patients participated. The models were obtained using partial least squares regression, where Dance's model was used as reference; parameters of mammography images, equipment and patient were used as predicting variables (kV, mAs, patient's weight, breast area and mean gray value of breast images). Coefficients of correlation equal to 89 and 88 were obtained for training and validation respectively in mediolateral oblique (MLO) and 84 and 89 for craniocaudal (CC). These models were used to estimate the mean glandular dose for all patients and later to obtain the institutional reference levels, 0.87 and 0.96 mGy for CC and MLO, respectively, following the recommendations of the ICRP publication No. 135. This study suggests that glandularity could be estimated with few parameters from equipment and patient.

Compare and contrast: Detecting mammographic soft-tissue lesions with C2-Net

Detecting breast soft-tissue lesions including masses, structural distortions and asymmetries is of great importance due to the high risk leading to breast cancer. Most existing deep learning based approaches detect lesions with only unilateral images. However, multi-view mammogram images provide highly related and complementary information which helps to make the clinical analysis more comprehensive and reliable. In this paper, we propose a multi-view network for breast soft-tissue lesion detection called C²-Net (Compare and Contrast, C²) that fuses information across different views. The proposed model contains the following three modules. The spatial context enhancing (SCE) module compares ipsilateral views and extracts complementary features to model lesion inherent 3D structure. The multi-scale kernel pooling (MKP) module contrasts contralateral views with added misalignment tolerance. Finally, the logic guided fusion (LGF) module fuses multi-view features by enhancing logic modeling capacity. Experimental results on both the public DDSM dataset and the in-house multi-center dataset demonstrate that the proposed method has achieved state-of-the-art performance.

A Radiomic feature-based Nipple Detection Algorithm on Digital Mammography

PURPOSE

In the diagnosis and detection of breast lesions, the nipple is an important anatomical landmark which can be used for the registration on multiview mammograms. In this study, we propose a new detection algorithm for nipples on digital mammography (DM) by applying pixel classification based on geometric and radiomic features extracted from breast boundary regions.

METHODS

The imaging characteristics of nipples are closely related to the visibility on mammograms. To locate the nipple on mammogram, a searching area is first determined based on the breast boundary and chest wall orientation. Two different approaches are developed for obvious and subtle nipples, respectively. For obvious nipples, top hat transformation is employed to detect the nipple region, whose geometric center is regarded as the nipple position. For subtle nipples, the curved searching area near the breast boundary is mapped onto a Cartesian plane through a revised rubber band straightening transformation. On the straightened searching area, the geometric and radiomic features are calculated along the normal direction of the breast boundary, and a random forest classifier is trained for subtle nipple localization.

RESULTS

Seven hundred and twenty-one DMs were collected for the evaluation of the proposed algorithm. The locations of nipples are manually identified by an experienced radiologist as the reference standard. The average Euclidean distance between the computed nipple position and the reference standard was 2.69 mm (obvious) and 7.81 mm (subtle), respectively. A total of 97.61% of the obvious nipples (613/628) and 88.17% of the subtle nipples (82/93) were detected within a 10-mm radius centered from the reference standard.

CONCLUSIONS

The evaluation results show that the proposed method is effective for nipple detection on DM, especially for subtle nipple detection.

Automatic detection of the nipple in screen-film and full-field digital mammograms using a novel Hessian-based method

Automatic detection of the nipple in mammograms is an important step in computerized systems that combine multiview information for accurate detection and diagnosis of breast cancer. Locating the nipple is a difficult task owing to variations in image quality, presence of noise, and distortion and displacement of the breast tissue due to compression. In this work, we propose a novel Hessian-based method to locate automatically the nipple in screen-film and full-field digital mammograms (FFDMs). The method includes detection of a plausible nipple/retroareolar area in a mammogram using geometrical constraints, analysis of the gradient vector field by mean and Gaussian curvature measurements, and local shape-based conditions. The proposed procedure was tested on 566 mammographic images consisting of 372 randomly selected scanned films from two public databases (mini-MIAS and DDSM), and 194 digital mammograms acquired with a GE Senographe 2000D FFDM system. A radiologist independently marked the centers of the nipples for evaluation of the results. The average error obtained was 6.7 mm (22 pixels) with reference to the center of the nipple as identified by the radiologist. Only two out of the 566 detected nipples (0.35 %) had an error larger than 50 mm. The method was also directly compared with two other techniques for the detection of the nipple. The results indicate that the proposed method outperforms other algorithms presented in the literature and can be used to identify accurately the nipple on various types of mammographic images.

Breast lesion co-localisation between X-ray and MR images using finite element modelling

This paper presents a novel X-ray and MR image registration technique based on individual-specific biomechanical finite element (FE) models of the breasts. Information from 3D magnetic resonance (MR) images was registered to X-ray mammographic images using non-linear FE models subject to contact mechanics constraints to simulate the large compressive deformations between the two imaging modalities. A physics-based perspective ray-casting algorithm was used to generate 2D pseudo-X-ray projections of the FE-warped 3D MR images. Unknown input parameters to the FE models, such as the location and orientation of the compression plates, were optimised to provide the best match between the pseudo and clinical X-ray images. The methods were validated using images taken before and during compression of a breast-shaped phantom, for which 12 inclusions were tracked between imaging modalities. These methods were then applied to X-ray and MR images from six breast cancer patients. Error measures (such as centroid and surface distances) of segmented tumours in simulated and actual X-ray mammograms were used to assess the accuracy of the methods. Sensitivity analysis of the lesion co-localisation accuracy to rotation about the anterior-posterior axis was then performed. For 10 of the 12 X-ray mammograms, lesion localisation accuracies of 14 mm and less were achieved. This analysis on the rotation about the anterior-posterior axis indicated that, in cases where the lesion lies in the plane parallel to the mammographic compression plates, that cuts through the nipple, such rotations have relatively minor effects.This has important implications for clinical applicability of this multi-modality lesion registration technique, which will aid in the diagnosis and treatment of breast cancer.

Automated medical image segmentation techniques

Accurate segmentation of medical images is a key step in contouring during radiotherapy planning. Computed topography (CT) and Magnetic resonance (MR) imaging are the most widely used radiographic techniques in diagnosis, clinical studies and treatment planning. This review provides details of automated segmentation methods, specifically discussed in the context of CT and MR images. The motive is to discuss the problems encountered in segmentation of CT and MR images, and the relative merits and limitations of methods currently available for segmentation of medical images.

Optimizing Case-based detection performance in a multiview CAD system for mammography

When reading mammograms, radiologists combine information from multiple views to detect abnormalities. Most computer-aided detection (CAD) systems, however, use primitive methods for inclusion of multiview context or analyze each view independently. In previous research it was found that in mammography lesion-based detection performance of CAD systems can be improved when correspondences between MLO and CC views are taken into account. However, detection at case level detection did not improve. In this paper, we propose a new learning method for multiview CAD systems, which is aimed at optimizing case-based detection performance. The method builds on a single-view lesion detection system and a correspondence classifier. The latter provides class probabilities for the various types of region pairs and correspondence features. The correspondence classifier output is used to bias the selection of training patterns for a multiview CAD system. In this way training can be forced to focus on optimization of case-based detection performance. The method is applied to the problem of detecting malignant masses and architectural distortions. Experiments involve 454 mammograms consisting of four views with a malignant region visible in at least one of the views. To evaluate performance, five-fold cross validation and FROC analysis was performed. Bootstrapping was used for statistical analysis. A significant increase of case-based detection performance was found when the proposed method was used. Mean sensitivity increased by 4.7% in the range of 0.01-0.5 false positives per image.

CT colonography computer-aided polyp detection: Effect on radiologist observers of polyp identification by CAD on both the supine and prone scans

RATIONALE AND OBJECTIVES

To determine whether the display of computer-aided detection (CAD) marks on individual polyps on both the supine and prone scans leads to improved polyp detection by radiologists compared to the display of CAD marks on individual polyps on either the supine or the prone scan, but not both.

MATERIALS AND METHODS

The acquisition of patient data for this study was approved by the Institutional Review Board and was Health Insurance Portability and Accountability Act-compliant. Subsequently, the use of the data was declared exempt from further institutional review board review. Four radiologists interpreted 33 computed tomography colonography cases, 21 of which had one adenoma 6-9 mm in size, with the assistance of a CAD system in the first reader mode (ie, the radiologists reviewed only the CAD marks). The radiologists were shown each case twice, with different sets of CAD marks for each of the two readings. In one reading, a true-positive CAD mark for the same polyp was displayed on both the supine and prone scans (a double-mark reading). In the other reading, a true-positive CAD mark was displayed either on the supine or prone scan, but not both (a single-mark reading). True-positive marks were randomized between readings and there was at least a 1-month delay between readings to minimize recall bias. Sensitivity and specificity were determined and receiver operating characteristic (ROC) and multiple-reader multiple-case analyses were performed.

RESULTS

The average per polyp sensitivities were 60% (38%-81%) versus 71% (52%-91%) (P = .03) for single-mark and double-mark readings, respectively. The areas (95% confidence intervals) under the ROC curves were 0.76 (0.62-0.88) and 0.79 (0.58-0.96), respectively (P = NS). Specificities were similar for the single-mark compared with the double-mark readings.

CONCLUSION

The display of CAD marks on a polyp on both the supine and prone scans led to more frequent detection of polyps by radiologists without adversely affecting specificity for detecting 6-9 mm adenomas.

Matching breast masses depicted on different views a comparison of three methods

RATIONALE AND OBJECTIVES

Computerized determination of optimal search areas on mammograms for matching breast mass regions depicted on two ipsilateral views remains a challenge for developing multiview-based computer-aided detection (CAD) schemes. The purpose of this study was to compare three methods aimed at matching CAD-cued mass regions depicted on two views and the associated impact on CAD performance.

MATERIALS AND METHODS

The three search methods used (1) an annular (fan-shaped) band, (2) a straight strip perpendicular to the estimated centerline, and (3) a mixed search area bound on the chest wall side by a straight line and an annular arc on the nipple side, respectively. An image database of 200 examinations with positive results depicting the masses on two views and 200 examinations with negative results was used for testing. Two performance assessment experiments were conducted. The first investigated the maximum matching sensitivity as a function of the search area size, and the second assessed the change in CAD performance using these three search methods.

RESULTS

To include all 200 paired mass regions within the search areas, maximum widths were 28 and 68 mm for the use of the straight strip and the annular band search methods, respectively. When applying a single-image-based CAD scheme to this image database, 172 masses (86% sensitivity) and 523 false-positive (FP) regions (0.33 per image) were detected and cued. Among the positive findings, 92 were cued by the CAD system on both views, and 80 were cued on only one view. In an attempt to match as many of the 172 CAD-cued masses (true-positive [TP] regions) on two views by incrementally reducing the CAD threshold inside the different search areas, the CAD scheme generated 158 TP-TP paired matches with 14 TP-FP paired matches, 142 TP-TP paired matches with 30 TP-FP paired matches, and 146 TP-TP paired matches with 26 TP-FP paired matches, using the methods involving the straight strip, the annular band, and the mixed search areas, respectively. Using the straight strip search method, the CAD also eliminated 25% of FP regions initially cued by the single-image-based CAD scheme and generated the lowest case-based FP detection rate, namely, 15% less than that generated by the annular band method.

CONCLUSIONS

This study showed that among these three search methods, the straight strip method required a smaller search area and achieved the highest level of CAD performance.

Robust initial detection of landmarks in film-screen mammograms using multiple FFDM atlases

Automated analysis of mammograms requires robust methods for pectoralis segmentation and nipple detection. Locating the nipple is especially important in multiview computer aided detection systems, in which findings are matched across images using the nipple-to-finding distance. Segmenting the pectoralis is a key preprocessing step to avoid false positives when detecting masses due to the similarity of the texture of mammographic parenchyma and the pectoral muscle. A multiatlas algorithm capable of providing very robust initial estimates of the nipple position and pectoral region in digitized mammograms is presented here. Ten full-field digital mammograms, which are easily annotated attributed to their excellent contrast, are robustly registered to the target digitized film-screen mammogram. The annotations are then propagated and fused into a final nipple position and pectoralis segmentation. Compared to other nipple detection methods in the literature, the system proposed here has the advantages that it is more robust and can provide a reliable estimate when the nipple is located outside the image. Our results show that the change in the correlation between nipple-to-finding distances in craniocaudal and mediolateral oblique views is not significant when the detected nipple positions replace the manual annotations. Moreover, the pectoralis segmentation is acceptable and can be used as initialization for a more complex algorithm to optimize the outline locally. A novel aspect of the method is that it is also capable of detecting and segmenting the pectoralis in craniocaudal views.

Improved mammographic CAD performance using multi-view information: a Bayesian network framework

Mammographic reading by radiologists requires the comparison of at least two breast projections (views) for the detection and the diagnosis of breast abnormalities. Despite their reported potential to support radiologists, most mammographic computer-aided detection (CAD) systems have a major limitation: as opposed to the radiologist's practice, computerized systems analyze each view independently. To tackle this problem, in this paper, we propose a Bayesian network framework for multi-view mammographic analysis, with main focus on breast cancer detection at a patient level. We use causal-independence models and context modeling over the whole breast represented as links between the regions detected by a single-view CAD system in the two breast projections. The proposed approach is implemented and tested with screening mammograms for 1063 cases of whom 385 had breast cancer. The single-view CAD system is used as a benchmark method for comparison. The results show that our multi-view modeling leads to significantly better performance in discriminating between normal and cancerous patients. We also demonstrate the potential of our multi-view system for selecting the most suspicious cases.

An ellipse-fitting based method for efficient registration of breast masses on two mammographic views

When reading mammograms, radiologists routinely search for and compare suspicious breast lesions identified on two corresponding craniocaudal (CC) and mediolateral oblique (MLO) views. Automatically identifying and matching the same true-positive breast lesions depicted on two views is an important step for developing successful multiview based computer-aided detection (CAD) schemes. The authors developed a method to automatically register breast areas and detect matching strips of interest used to identify the matched mass regions depicted on CC and MLO views. The method uses an ellipse based model to fit the breast boundary contour (skin line) and set a local Cartesian coordinate system for each view. One intersection point between the major/minor axis and the fitted ellipse perimeter passed through breast boundary is selected as the origin and the majoraxis and the minoraxis of the ellipse are used as the two axis of the Cartesian coordinate system. When a mass is identified on one view, the scheme computes its position in the local coordinate system. Then, the distance is mapped onto the local coordinate of the other view. At the end of the mapped distance a registered centerline of the matching strip is created. The authors established an image database that includes 200 test examinations each depicting one verified mass visible on the two views. They tested whether the registered centerline identified on another view can be used to locate the matched mass region. The experiments show that the average distance between the mass region centers and the registered centerlines was +/- 8.3 mm and in 91% of testing cases the registered centerline actually passes through the matched mass regions. A matching strip width of 47 mm was required to achieve 100% sensitivity for the test database. The results demonstrate the feasibility of the proposed method to automatically identify masses depicted on CC and MLO views, which may improve future development of multiview based CAD schemes.

Bilateral analysis based false positive reduction for computer-aided mass detection

We have developed a false positive (FP) reduction method based on analysis of bilateral mammograms for computerized mass detection systems. The mass candidates on each view were first detected by our unilateral computer-aided detection (CAD) system. For each detected object, a regional registration technique was used to define a region of interest (ROI) that is "symmetrical" to the object location on the contralateral mammogram. Texture features derived from the spatial gray level dependence matrices and morphological features were extracted from the ROI containing the detected object on a mammogram and its corresponding ROI on the contralateral mammogram. Bilateral features were then generated from corresponding pairs of unilateral features for each object. Two linear discriminant analysis (LDA) classifiers were trained from the unilateral and the bilateral feature spaces, respectively. Finally, the scores from the unilateral LDA classifier and the bilateral LDA asymmetry classifier were fused with a third LDA whose output score was used to distinguish true mass from FPs. A data set of 341 cases of bilateral two-view mammograms was used in this study, of which 276 cases with 552 bilateral pairs contained 110 malignant and 166 benign biopsy-proven masses and 65 cases with 130 bilateral pairs were normal. The mass data set was divided into two subsets for twofold cross-validation training and testing. The normal data set was used for estimation of FP rates. It was found that our bilateral CAD system achieved a case-based sensitivity of 70%, 80%, and 85% at average FP rates of 0.35, 0.75, and 0.95 FPs/image, respectively, on the test data sets with malignant masses. In comparison to the average FP rates for the unilateral CAD system of 0.58, 1.33, and 1.63, respectively, at the corresponding sensitivities, the FP rates were reduced by 40%, 44%, and 42% with the bilateral symmetry information. The improvement was statistically significance (p < 0.05) as estimated by JAFROC analysis.

Information-theoretic CAD system in mammography: Entropy-based indexing for computational efficiency and robust performance

We have previously presented a knowledge-based computer-assisted detection (KB-CADe) system for the detection of mammographic masses. The system is designed to compare a query mammographic region with mammographic templates of known ground truth. The templates are stored in an adaptive knowledge database. Image similarity is assessed with information theoretic measures (e.g., mutual information) derived directly from the image histograms. A previous study suggested that the diagnostic performance of the system steadily improves as the knowledge database is initially enriched with more templates. However, as the database increases in size, an exhaustive comparison of the query case with each stored template becomes computationally burdensome. Furthermore, blind storing of new templates may result in redundancies that do not necessarily improve diagnostic performance. To address these concerns we investigated an entropy-based indexing scheme for improving the speed of analysis and for satisfying database storage restrictions without compromising the overall diagnostic performance of our KB-CADe system. The indexing scheme was evaluated on two different datasets as (i) a search mechanism to sort through the knowledge database, and (ii) a selection mechanism to build a smaller, concise knowledge database that is easier to maintain but still effective. There were two important findings in the study. First, entropy-based indexing is an effective strategy to identify fast a subset of templates that are most relevant to a given query. Only this subset could be analyzed in more detail using mutual information for optimized decision making regarding the query. Second, a selective entropy-based deposit strategy may be preferable where only high entropy cases are maintained in the knowledge database. Overall, the proposed entropy-based indexing scheme was shown to reduce the computational cost of our KB-CADe system by 55% to 80% while maintaining the system's diagnostic performance.

Combining two mammographic projections in a computer aided mass detection method

A method is presented to improve computer aided detection (CAD) results for masses in mammograms by fusing information obtained from two views of the same breast. It is based on a previously developed approach to link potentially suspicious regions in mediolateral oblique (MLO) and craniocaudal (CC) views. Using correspondence between regions, we extended our CAD scheme by building a cascaded multiple-classifier system, in which the last stage computes suspiciousness of an initially detected region conditional on the existence and similarity of a linked candidate region in the other view. We compared the two-view detection system with the single-view detection method using free-response receiver operating characteristic (FROC) analysis and cross validation. The dataset used in the evaluation consisted of 948 four-view mammograms, including 412 cancer cases with a mass, architectural distortion, or asymmetry. A statistically significant improvement was found in the lesion based detection performance. At a false positive (FP) rate of 0.1 FP/image, the lesion sensitivity improved from 56% to 61%. Case based sensitivity did not improve.