Estimation of the biomechanical mammographic deformation of the breast using machine learning models

BACKGROUND

A typical problem in the registration of MRI and X-ray mammography is the nonlinear deformation applied to the breast during mammography. We have developed a method for virtual deformation of the breast using a biomechanical model automatically constructed from MRI. The virtual deformation is applied in two steps: unloaded state estimation and compression simulation. The finite element method is used to solve the deformation process. However, the extensive computational cost prevents its usage in clinical routine.

METHODS

We propose three machine learning models to overcome this problem: an extremely randomized tree (first model), extreme gradient boosting (second model), and deep learning-based bidirectional long short-term memory with an attention layer (third model) to predict the deformation of a biomechanical model. We evaluated our methods with 516 breasts with realistic compression ratios up to 76%.

FINDINGS

We first applied one-fold validation, in which the second and third models performed better than the first model. We then applied ten-fold validation. For the unloaded state estimation, the median RMSE for the second and third models is 0.8 mm and 1.2 mm, respectively. For the compression, the median RMSE is 3.4 mm for both models. We evaluated correlations between model accuracy and characteristics of the clinical datasets such as compression ratio, breast volume, and tissue types.

INTERPRETATION

Using the proposed models, we achieved accurate results comparable to the finite element model, with a speedup of factor 240 using the extreme gradient boosting model. These proposed models can replace the finite element model simulation, enabling clinically relevant real-time application.

A review of bioengineering techniques applied to breast tissue: Mechanical properties, tissue engineering and finite element analysis

Female breast cancer was the most prevalent cancer worldwide in 2020, according to the Global Cancer Observatory. As a prophylactic measure or as a treatment, mastectomy and lumpectomy are often performed at women. Following these surgeries, women normally do a breast reconstruction to minimize the impact on their physical appearance and, hence, on their mental health, associated with self-image issues. Nowadays, breast reconstruction is based on autologous tissues or implants, which both have disadvantages, such as volume loss over time or capsular contracture, respectively. Tissue engineering and regenerative medicine can bring better solutions and overcome these current limitations. Even though more knowledge needs to be acquired, the combination of biomaterial scaffolds and autologous cells appears to be a promising approach for breast reconstruction. With the growth and improvement of additive manufacturing, three dimensional (3D) printing has been demonstrating a lot of potential to produce complex scaffolds with high resolution. Natural and synthetic materials have been studied in this context and seeded mainly with adipose derived stem cells (ADSCs) since they have a high capability of differentiation. The scaffold must mimic the environment of the extracellular matrix (ECM) of the native tissue, being a structural support for cells to adhere, proliferate and migrate. Hydrogels (e.g., gelatin, alginate, collagen, and fibrin) have been a biomaterial widely studied for this purpose since their matrix resembles the natural ECM of the native tissues. A powerful tool that can be used in parallel with experimental techniques is finite element (FE) modeling, which can aid the measurement of mechanical properties of either breast tissues or scaffolds. FE models may help in the simulation of the whole breast or scaffold under different conditions, predicting what might happen in real life. Therefore, this review gives an overall summary concerning the human breast, specifically its mechanical properties using experimental and FE analysis, and the tissue engineering approaches to regenerate this particular tissue, along with FE models.

Development and validation of a four-dimensional registration technique for DCE breast MRI

BACKGROUND

Patient motion can degrade image quality of dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) due to subtraction artifacts. By objectively and subjectively assessing the impact of principal component analysis (PCA)-based registration on pretreatment DCE-MRIs of breast cancer patients, we aim to validate four-dimensional registration for DCE breast MRI.

RESULTS

After applying a four-dimensional, PCA-based registration algorithm to 154 pretreatment DCE-MRIs of histopathologically well-described breast cancer patients, we quantitatively determined image quality in unregistered and registered images. For subjective assessment, we ranked motion severity in a clinical reading setting according to four motion categories (0: no motion, 1: mild motion, 2: moderate motion, 3: severe motion with nondiagnostic image quality). The median of images with either moderate or severe motion (median category 2, IQR 0) was reassigned to motion category 1 (IQR 0) after registration. Motion category and motion reduction by registration were correlated (Spearman's rho: 0.83, p < 0.001). For objective assessment, we performed perfusion model fitting using the extended Tofts model and calculated its volume transfer coefficient K^trans as surrogate parameter for motion artifacts. Mean K^trans decreased from 0.103 (± 0.077) before registration to 0.097 (± 0.070) after registration (p < 0.001). Uncertainty in perfusion quantification was reduced by 7.4% after registration (± 15.5, p < 0.001).

CONCLUSIONS

Four-dimensional, PCA-based image registration improves image quality of breast DCE-MRI by correcting for motion artifacts in subtraction images and reduces uncertainty in quantitative perfusion modeling. The improvement is most pronounced when moderate-to-severe motion artifacts are present.

Deformable mapping technique to correlate lesions in digital breast tomosynthesis and automated breast ultrasound images

PURPOSE

To develop a deformable mapping technique to match corresponding lesions between digital breast tomosynthesis (DBT) and automated breast ultrasound (ABUS) images.

METHODS

External fiducial markers were attached to the surface of two CIRS multi-modality compressible breast phantoms (A and B) containing multiple simulated lesions. Both phantoms were imaged with DBT (upright positioning with cranial-caudal compression) and ABUS (supine positioning with anterior-to-chest wall compression). The lesions and markers were manually segmented by three different readers. Reader segmentation similarity and reader reproducibility were assessed using Dice similarity coefficients (DSC) and distances between centers of mass (d_COM ). For deformable mapping between the modalities each reader's segmented dataset was processed with an automated deformable mapping algorithm as follows: First, Morfeus, a finite element (FE) based multi-organ deformable image registration platform, converted segmentations into triangular surface meshes. Second, Altair HyperMesh, a FE pre-processor, created base FE models for the ABUS and DBT data sets. All deformation is performed on the DBT image data; the ABUS image sets remain fixed throughout the process. Deformation was performed on the external skin contour (DBT image set) to match the external skin contour on the ABUS set, and the locations of the external markers were used to morph the skin contours to be within a user-defined distance. Third, the base DBT-FE model was deformed with the FE analysis solver, Optistruct. Deformed DBT lesions were correlated with matching lesions in the base ABUS FE model. Performance (lesion correlation) was assessed with d_COM for all corresponding lesions and lesion overlap. Analysis was performed to determine the minimum number of external fiducial markers needed to create the desired correlation and the improvement of correlation with the use of external markers.

RESULTS

Average DSC for reader similarity ranged from 0.88 to 0.91 (ABUS) and 0.57 to 0.83 (DBT). Corresponding d_COM ranged from 0.20 to 0.36 mm (ABUS) and 0.11 to 1.16 mm (DBT). Lesion correlation is maximized when all corresponding markers are within a maximum distance of 5 mm. For deformable mapping of phantom A, without the use of external markers, only two of six correlated lesions showed overlap with an average lesion d_COM of 6.8 ± 2.8 mm. With use of three external fiducial markers, five of six lesions overlapped and average d_COM improved to 4.9 ± 2.4 mm. For deformable mapping of Phantom B without external markers analysis, four lesions were correlated of seven with overlap between only one of seven lesions, and an average lesion d_COM of 9.7 ± 3.5 mm. With three external markers, all seven possible lesions were correlated with overlap between four of seven lesions. The average d_COM was 8.5 ± 4.0 mm.

CONCLUSION

This work demonstrates the potential for a deformable mapping technique to relate corresponding lesions in DBT and ABUS images by showing improved lesion correspondence and reduced lesion registration errors with the use of external fiducial markers. The technique should improve radiologists' characterization of breast lesions which can reduce patient callbacks, misdiagnoses and unnecessary biopsies.

Multimodal Breast Parenchymal Patterns Correlation Using a Patient-Specific Biomechanical Model

In this paper, we aim to produce a realistic 2-D projection of the breast parenchymal distribution from a 3-D breast magnetic resonance image (MRI). To evaluate the accuracy of our simulation, we compare our results with the local breast density (i.e., density map) obtained from the complementary full-field digital mammogram. To achieve this goal, we have developed a fully automatic framework, which registers MRI volumes to X-ray mammograms using a subject-specific biomechanical model of the breast. The optimization step modifies the position, orientation, and elastic parameters of the breast model to perform the alignment between the images. When the model reaches an optimal solution, the MRI glandular tissue is projected and compared with the one obtained from the corresponding mammograms. To reduce the loss of information during the ray-casting, we introduce a new approach that avoids resampling the MRI volume. In the results, we focus our efforts on evaluating the agreement of the distributions of glandular tissue, the degree of structural similarity, and the correlation between the real and synthetic density maps. Our approach obtained a high-structural agreement regardless the glandularity of the breast, whilst the similarity of the glandular tissue distributions and correlation between both images increase in denser breasts. Furthermore, the synthetic images show continuity with respect to large structures in the density maps.

Towards an in-plane methodology to track breast lesions using mammograms and patient-specific finite-element simulations

In breast cancer screening or diagnosis, it is usual to combine different images in order to locate a lesion as accurately as possible. These images are generated using a single or several imaging techniques. As x-ray-based mammography is widely used, a breast lesion is located in the same plane of the image (mammogram), but tracking it across mammograms corresponding to different views is a challenging task for medical physicians. Accordingly, simulation tools and methodologies that use patient-specific numerical models can facilitate the task of fusing information from different images. Additionally, these tools need to be as straightforward as possible to facilitate their translation to the clinical area. This paper presents a patient-specific, finite-element-based and semi-automated simulation methodology to track breast lesions across mammograms. A realistic three-dimensional computer model of a patient's breast was generated from magnetic resonance imaging to simulate mammographic compressions in cranio-caudal (CC, head-to-toe) and medio-lateral oblique (MLO, shoulder-to-opposite hip) directions. For each compression being simulated, a virtual mammogram was obtained and posteriorly superimposed to the corresponding real mammogram, by sharing the nipple as a common feature. Two-dimensional rigid-body transformations were applied, and the error distance measured between the centroids of the tumors previously located on each image was 3.84 mm and 2.41 mm for CC and MLO compression, respectively. Considering that the scope of this work is to conceive a methodology translatable to clinical practice, the results indicate that it could be helpful in supporting the tracking of breast lesions.

A finite element-based machine learning approach for modeling the mechanical behavior of the breast tissues under compression in real-time

This work presents a data-driven method to simulate, in real-time, the biomechanical behavior of the breast tissues in some image-guided interventions such as biopsies or radiotherapy dose delivery as well as to speed up multimodal registration algorithms. Ten real breasts were used for this work. Their deformation due to the displacement of two compression plates was simulated off-line using the finite element (FE) method. Three machine learning models were trained with the data from those simulations. Then, they were used to predict in real-time the deformation of the breast tissues during the compression. The models were a decision tree and two tree-based ensemble methods (extremely randomized trees and random forest). Two different experimental setups were designed to validate and study the performance of these models under different conditions. The mean 3D Euclidean distance between nodes predicted by the models and those extracted from the FE simulations was calculated to assess the performance of the models in the validation set. The experiments proved that extremely randomized trees performed better than the other two models. The mean error committed by the three models in the prediction of the nodal displacements was under 2 mm, a threshold usually set for clinical applications. The time needed for breast compression prediction is sufficiently short to allow its use in real-time (<0.2 s).

A review of biomechanically informed breast image registration

Breast radiology encompasses the full range of imaging modalities from routine imaging via x-ray mammography, magnetic resonance imaging and ultrasound (both two- and three-dimensional), to more recent technologies such as digital breast tomosynthesis, and dedicated breast imaging systems for positron emission mammography and ultrasound tomography. In addition new and experimental modalities, such as Photoacoustics, Near Infrared Spectroscopy and Electrical Impedance Tomography etc, are emerging. The breast is a highly deformable structure however, and this greatly complicates visual comparison of imaging modalities for the purposes of breast screening, cancer diagnosis (including image guided biopsy), tumour staging, treatment monitoring, surgical planning and simulation of the effects of surgery and wound healing etc. Due primarily to the challenges posed by these gross, non-rigid deformations, development of automated methods which enable registration, and hence fusion, of information within and across breast imaging modalities, and between the images and the physical space of the breast during interventions, remains an active research field which has yet to translate suitable methods into clinical practice. This review describes current research in the field of breast biomechanical modelling and identifies relevant publications where the resulting models have been incorporated into breast image registration and simulation algorithms. Despite these developments there remain a number of issues that limit clinical application of biomechanical modelling. These include the accuracy of constitutive modelling, implementation of representative boundary conditions, failure to meet clinically acceptable levels of computational cost, challenges associated with automating patient-specific model generation (i.e. robust image segmentation and mesh generation) and the complexity of applying biomechanical modelling methods in routine clinical practice.