Breast MRI and X-ray mammography registration using gradient values

Estimation of the biomechanical mammographic deformation of the breast using machine learning models

BACKGROUND

A typical problem in the registration of MRI and X-ray mammography is the nonlinear deformation applied to the breast during mammography. We have developed a method for virtual deformation of the breast using a biomechanical model automatically constructed from MRI. The virtual deformation is applied in two steps: unloaded state estimation and compression simulation. The finite element method is used to solve the deformation process. However, the extensive computational cost prevents its usage in clinical routine.

METHODS

We propose three machine learning models to overcome this problem: an extremely randomized tree (first model), extreme gradient boosting (second model), and deep learning-based bidirectional long short-term memory with an attention layer (third model) to predict the deformation of a biomechanical model. We evaluated our methods with 516 breasts with realistic compression ratios up to 76%.

FINDINGS

We first applied one-fold validation, in which the second and third models performed better than the first model. We then applied ten-fold validation. For the unloaded state estimation, the median RMSE for the second and third models is 0.8 mm and 1.2 mm, respectively. For the compression, the median RMSE is 3.4 mm for both models. We evaluated correlations between model accuracy and characteristics of the clinical datasets such as compression ratio, breast volume, and tissue types.

INTERPRETATION

Using the proposed models, we achieved accurate results comparable to the finite element model, with a speedup of factor 240 using the extreme gradient boosting model. These proposed models can replace the finite element model simulation, enabling clinically relevant real-time application.

Incorporating heterogeneity and anisotropy for surgical applications in breast deformation modeling

BACKGROUND

Simulating soft-tissue breast deformations is of interest for many applications including image fusion, longitudinal registration, and image-guided surgery. For the surgical use case, positional changes cause breast deformations that compromise the use of preoperative imaging to inform tumor excision. Even when acquiring imaging in the supine position, which better reflects surgical presentation, deformations still occur due to arm motion and orientation changes. A biomechanical modeling approach to simulate supine breast deformations for surgical applications must be both accurate and compatible with the clinical workflow.

METHODS

A supine MR breast imaging dataset from n = 11 healthy volunteers was used to simulate surgical deformations by acquiring images in arm-down and arm-up positions. Three linear-elastic modeling approaches with varying levels of complexity were used to predict deformations caused by this arm motion: a homogeneous isotropic model, a heterogeneous isotropic model, and a heterogeneous anisotropic model using a transverse-isotropic constitutive model.

FINDINGS

The average target registration errors for subsurface anatomical features were 5.4 ± 1.5 mm for the homogeneous isotropic model, 5.3 ± 1.5 mm for the heterogeneous isotropic model, and 4.7 ± 1.4 mm for the heterogeneous anisotropic model. A statistically significant improvement in target registration error was observed between the heterogeneous anisotropic model and both the homogeneous and the heterogeneous isotropic models (P < 0.01).

INTERPRETATION

While a model that fully incorporates all constitutive complexities of anatomical structure likely achieves the best accuracy, a computationally tractable heterogeneous anisotropic model provided significant improvement and may be applicable for image-guided breast surgeries.

Resolving hidden pixels beyond the resolution limit of projection imaging by square aperture

Projection imaging has been employed widely in many areas, such as x-ray radiography, due to its penetration power and ballistic geometry of their paths. However, its resolution limit remains a major challenge, caused by the conflict of source intensity and source size associated with image blurriness. A simple yet robust scheme has been proposed here to solve the problem. An unconventional square aperture, rather than the usual circular beam, is constructed, which allows for the straightforward deciphering of a blurred spot, to unravel hundreds originally hidden pixels. With numerical verification and experimental demonstration, our proposal is expected to benefit multiple disciplines, not limited to x-ray imaging.

Development and validation of a four-dimensional registration technique for DCE breast MRI

BACKGROUND

Patient motion can degrade image quality of dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) due to subtraction artifacts. By objectively and subjectively assessing the impact of principal component analysis (PCA)-based registration on pretreatment DCE-MRIs of breast cancer patients, we aim to validate four-dimensional registration for DCE breast MRI.

RESULTS

After applying a four-dimensional, PCA-based registration algorithm to 154 pretreatment DCE-MRIs of histopathologically well-described breast cancer patients, we quantitatively determined image quality in unregistered and registered images. For subjective assessment, we ranked motion severity in a clinical reading setting according to four motion categories (0: no motion, 1: mild motion, 2: moderate motion, 3: severe motion with nondiagnostic image quality). The median of images with either moderate or severe motion (median category 2, IQR 0) was reassigned to motion category 1 (IQR 0) after registration. Motion category and motion reduction by registration were correlated (Spearman's rho: 0.83, p < 0.001). For objective assessment, we performed perfusion model fitting using the extended Tofts model and calculated its volume transfer coefficient K^trans as surrogate parameter for motion artifacts. Mean K^trans decreased from 0.103 (± 0.077) before registration to 0.097 (± 0.070) after registration (p < 0.001). Uncertainty in perfusion quantification was reduced by 7.4% after registration (± 15.5, p < 0.001).

CONCLUSIONS

Four-dimensional, PCA-based image registration improves image quality of breast DCE-MRI by correcting for motion artifacts in subtraction images and reduces uncertainty in quantitative perfusion modeling. The improvement is most pronounced when moderate-to-severe motion artifacts are present.

Towards 2D/3D Registration of the Preoperative MRI to Intraoperative Fluoroscopic Images for Visualization of Bone Defects

Magnetic Resonance Imaging (MRI) is a medical imaging modality that allows for the evaluation of soft-tissue diseases and the assessment of bone quality. Preoperative MRI volumes are used by surgeons to identify defected bones, perform the segmentation of lesions, and generate surgical plans before the surgery. Nevertheless, conventional intraoperative imaging modalities such as fluoroscopy are less sensitive in detecting potential lesions. In this work, we propose a 2D/3D registration pipeline that aims to register preoperative MRI with intraoperative 2D fluoroscopic images. To showcase the feasibility of our approach, we use the core decompression procedure as a surgical example to perform 2D/3D femur registration. The proposed registration pipeline is evaluated using digitally reconstructed radiographs (DRRs) to simulate the intraoperative fluoroscopic images. The resulting transformation from the registration is later used to create overlays of preoperative MRI annotations and planning data to provide intraoperative visual guidance to surgeons. Our results suggest that the proposed registration pipeline is capable of achieving reasonable transformation between MRI and digitally reconstructed fluoroscopic images for intraoperative visualization applications.

Measurement of the hyperelastic properties of 72 normal homogeneous and heterogeneous ex vivo breast tissue samples

The mechanical properties of normal soft tissues, including breast tissue, have been of interest to the biomedical research community as there are many clinical and industrial applications that can benefit from quantitative information characterizing such properties. For instance, computer assisted surgery planning, elastography for breast cancer diagnosis, and bra design can all involve biomechanical modeling of the breast to predict its deformation or stress distribution. It is known that most biological soft tissues, including breast tissue, exhibit nonlinear mechanical response over large strains. As such, it is necessary to model such tissues as hyperelastic. In this work, we used indentation testing to estimate the hyperelastic parameters of 4 models (3rd order Ogden, 5-term polynomial, Veronda-Westman and Yeoh) estimated from 72 healthy ex vivo breast tissue samples covering adipose, fibroglandular, and mixed tissue. All estimated parameter sets were confirmed to represent stable material using Drucker's stability criterion. We observed that all three tissue types were statistically similar solidifying the use of homogenous breast modelling over large strain simulation.

A novel 2D/3D hierarchical registration framework via principal-directional Fourier transform operator

An effective registration framework between preoperative 3D computed tomography and intraoperative 2D x-ray images is crucial in image-guided therapy. In this paper, a novel 2D/3D hierarchical registration framework via principal-directional Fourier transform operator (HRF-PDFTO) is proposed. First, a PDFTO was established to obtain the in-plane translation and rotation invariance. Then, an initial free template-matching approach based on PDFTO was utilized to avoid initial value assignment and expand the capture range of registration. Finally, the hierarchical registration framework, HRF-PDFTO, was proposed to reduce the dimensions of the registration search space from n ⁶ to n ². The experimental results demonstrated that the proposed HRF-PDFTO has good performance with an accuracy of 0.72 mm, and a single registration time of 16 s, which improves the registration efficiency by ten times. Consequently, the HRF-PDFTO can meet the accuracy and efficiency requirements of 2D/3D registration in related clinical applications.

Brain MRI analysis using a deep learning based evolutionary approach

Convolutional neural network (CNN) models have recently demonstrated impressive performance in medical image analysis. However, there is no clear understanding of why they perform so well, or what they have learned. In this paper, a three-dimensional convolutional neural network (3D-CNN) is employed to classify brain MRI scans into two predefined groups. In addition, a genetic algorithm based brain masking (GABM) method is proposed as a visualization technique that provides new insights into the function of the 3D-CNN. The proposed GABM method consists of two main steps. In the first step, a set of brain MRI scans is used to train the 3D-CNN. In the second step, a genetic algorithm (GA) is applied to discover knowledgeable brain regions in the MRI scans. The knowledgeable regions are those areas of the brain which the 3D-CNN has mostly used to extract important and discriminative features from them. For applying GA on the brain MRI scans, a new chromosome encoding approach is proposed. The proposed framework has been evaluated using ADNI (including 140 subjects for Alzheimer's disease classification) and ABIDE (including 1000 subjects for Autism classification) brain MRI datasets. Experimental results show a 5-fold classification accuracy of 0.85 for the ADNI dataset and 0.70 for the ABIDE dataset. The proposed GABM method has extracted 6 to 65 knowledgeable brain regions in ADNI dataset (and 15 to 75 knowledgeable brain regions in ABIDE dataset). These regions are interpreted as the segments of the brain which are mostly used by the 3D-CNN to extract features for brain disease classification. Experimental results show that besides the model interpretability, the proposed GABM method has increased final performance of the classification model in some cases with respect to model parameters.

Logarithmic Fuzzy Entropy Function for Similarity Measurement in Multimodal Medical Images Registration

Multimodal medical images are useful for observing tissue structure clearly in clinical practice. To integrate multimodal information, multimodal registration is significant. The entropy-based registration applies a structure descriptor set to replace the original multimodal image and compute similarity to express the correlation of images. The accuracy and converging rate of the registration depend on this set. We propose a new method, logarithmic fuzzy entropy function, to compute the descriptor set. It is obvious that the proposed method can increase the upper bound value from log(r) to log(r) + ∆(r) so that a more representative structural descriptor set is formed. The experiment results show that our method has faster converging rate and wider quantified range in multimodal medical images registration.