Using surface markers for MRI guided breast conserving surgery: a feasibility survey

A review on organ deformation modeling approaches for reliable surgical navigation using augmented reality

Augmented Reality (AR) holds the potential to revolutionize surgical procedures by allowing surgeons to visualize critical structures within the patient's body. This is achieved through superimposing preoperative organ models onto the actual anatomy. Challenges arise from dynamic deformations of organs during surgery, making preoperative models inadequate for faithfully representing intraoperative anatomy. To enable reliable navigation in augmented surgery, modeling of intraoperative deformation to obtain an accurate alignment of the preoperative organ model with the intraoperative anatomy is indispensable. Despite the existence of various methods proposed to model intraoperative organ deformation, there are still few literature reviews that systematically categorize and summarize these approaches. This review aims to fill this gap by providing a comprehensive and technical-oriented overview of modeling methods for intraoperative organ deformation in augmented reality in surgery. Through a systematic search and screening process, 112 closely relevant papers were included in this review. By presenting the current status of organ deformation modeling methods and their clinical applications, this review seeks to enhance the understanding of organ deformation modeling in AR-guided surgery, and discuss the potential topics for future advancements.

Soft Tissue Monitoring of the Surgical Field: Detection and Tracking of Breast Surface Deformations

OBJECTIVE

Deformable object tracking is common in the computer vision field, with applications typically focusing on nonrigid shape detection and usually not requiring specific three-dimensional point localization. In surgical guidance however, accurate navigation is intrinsically linked to precise correspondence of tissue structure. This work presents a contactless, automated fiducial acquisition method using stereo video of the operating field to provide reliable three-dimensional fiducial localization for an image guidance framework in breast conserving surgery.

METHODS

On n = 8 breasts from healthy volunteers, the breast surface was measured throughout the full range of arm motion in a supine mock-surgical position. Using hand-drawn inked fiducials, adaptive thresholding, and KAZE feature matching, precise three-dimensional fiducial locations were detected and tracked through tool interference, partial and complete marker occlusions, significant displacements and nonrigid shape distortions.

RESULTS

Compared to digitization with a conventional optically tracked stylus, fiducials were automatically localized with 1.6 ± 0.5 mm accuracy and the two measurement methods did not significantly differ. The algorithm provided an average false discovery rate <0.1% with all cases' rates below 0.2%. On average, 85.6 ± 5.9% of visible fiducials were automatically detected and tracked, and 99.1 ± 1.1% of frames provided only true positive fiducial measurements, which indicates the algorithm achieves a data stream that can be used for reliable on-line registration.

CONCLUSIONS

Tracking is robust to occlusions, displacements, and most shape distortions.

SIGNIFICANCE

This work-flow friendly data collection method provides highly accurate and precise three-dimensional surface data to drive an image guidance system for breast conserving surgery.

Computational Imaging to Compensate for Soft-Tissue Deformations in Image-Guided Breast Conserving Surgery

OBJECTIVE

During breast conserving surgery (BCS), magnetic resonance (MR) images aligned to accurately display intraoperative lesion locations can offer improved understanding of tumor extent and position relative to breast anatomy. Unfortunately, even under consistent supine conditions, soft tissue deformation compromises image-to-physical alignment and results in positional errors.

METHODS

A finite element inverse modeling technique has been developed to nonrigidly register preoperative supine MR imaging data to the surgical scene for improved localization accuracy during surgery. Registration is driven using sparse data compatible with acquisition during BCS, including corresponding surface fiducials, sparse chest wall contours, and the intra-fiducial skin surface. Deformation predictions were evaluated at surface fiducial locations and subsurface tissue features that were expertly identified and tracked. Among n = 7 different human subjects, an average of 22 ± 3 distributed subsurface targets were analyzed in each breast volume.

RESULTS

The average target registration error (TRE) decreased significantly when comparing rigid registration to this nonrigid approach (10.4 ± 2.3 mm vs 6.3 ± 1.4 mm TRE, respectively). When including a single subsurface feature as additional input data, the TRE significantly improved further (4.2 ± 1.0 mm TRE), and in a region of interest within 15 mm of a mock biopsy clip TRE was 3.9 ± 0.9 mm.

CONCLUSION

These results demonstrate accurate breast deformation estimates based on sparse-data-driven model predictions.

SIGNIFICANCE

The data suggest that a computational imaging approach can account for image-to-surgery shape changes to enhance surgical guidance during BCS.

Breast image registration for surgery: Insights on material mechanics modeling

Breast conserving surgery (BCS) is a common procedure for early-stage breast cancer patients. Supine preoperative magnetic resonance (MR) breast imaging for visualizing tumor location and extent, while not standard for procedural guidance, more closely represents the surgical presentation compared to conventional diagnostic pendant positioning. Optimal utilization for surgical guidance, however, requires a fast and accurate image-to-physical registration from preoperative imaging to intraoperative surgical presentation. In this study, three registration methods were investigated on healthy volunteers' breasts (n=11) with the arm-down position simulating preoperative imaging and arm-up position simulating intraoperative data. The registration methods included: (1) point-based rigid registration using synthetic fiducials, (2) non-rigid biomechanical model-based registration using sparse data, and (3) a data-dense 3D diffeomorphic image-based registration from the Advanced Normalization Tools (ANTs) repository. The average target registration errors (TRE) were 10.4 ± 2.3, 6.4 ± 1.5, and 2.8 ± 1.3 mm (mean ± standard deviation) and the average fiducial registration errors (FRE) were 7.8 ± 1.7, 2.5 ± 1.1, and 3.1 ± 1.1 mm (mean ± standard deviation) for the point-based rigid, nonrigid biomechanical, and ANTs registrations, respectively. Additionally, common mechanics-based deformation metrics (volume change and anisotropy) were calculated from the ANTs deformation field. The average metrics revealed anisotropic tissue behavior and a statistical difference in volume change between glandular and adipose tissue, suggesting that nonrigid modeling methods may be improved by incorporating material heterogeneity and anisotropy. Overall, registration accuracy significantly improved with increasingly flexible registration methods, which may inform future development of image guidance systems for lumpectomy procedures.

Tumor deformation correction for an image guidance system in breast conserving surgery

Breast cancer is the most common cancer in women, and surgical resection is standard of care for the majority of breast cancer patients. Unfortunately, current reoperation rates are 10-29%. Uncertainty in lesion localization is one of the main factors contributing to these high reoperation rates. This work uses the linearized iterative boundary reconstruction approach to model patient breast deformation due to abduction of the ipsilateral arm. A preoperative supine magnetic resonance (MR) image was obtained with the patient's arms down near the torso. A mock intraoperative breast shape was measured from a supine MR image obtained with the patient's arm up near the head. Sparse data was subsampled from the full volumetric image to represent realistic intraoperative data collection: surface fiducial points, the intra-fiducial skin surface, and the chest wall as measured with 7 tracked ultrasound images. The deformed preoperative arm-down data was compared to the ground truth arm-up data. From rigid registration to model correction the tumor centroid distance improves from 7.3 mm to 3.3 mm, average surface fiducial error across 9 synthetic fiducials and the nipple improves from 7.4 ± 2.2 to 1.3 ± 0.7, and average subsurface error across 14 corresponding features improves from 6.2 ± 1.4 mm to 3.5 ± 1.1 mm. Using preoperative supine MR imaging and sparse data in the deformed position, this modeling framework can correct for breast shape changes between imaging and surgery to more accurately predict intraoperative position of the tumor as well as 10 surface fiducials and 14 subsurface features.

Impact of deformation on a supine-positioned image-guided breast surgery approach

PURPOSE

To reduce reoperation rates for image-guided breast-conserving surgery, the enhanced sensitivity of magnetic resonance (MR) supine imaging may be leveraged. However, accurate tissue correspondence between images and their physical counterpart in the surgical presentation is challenging due to breast deformations (e.g., from patient/arm position changes, and operating room table rotation differences). In this study, standard rigid registration methods are employed and tissue deformation is characterized.

METHODS

On n = 10 healthy breasts, surface displacements were measured by comparing intraoperative fiducial locations as the arm was moved from conventional MR scanning positions (arm-down and arm-up) to the laterally extended surgical configuration. Supine MR images in the arm-down and arm-up positions were registered to mock intraoperative presentations.

RESULTS

Breast displacements from a supine MR imaging configuration to a mock surgical presentation were 28.9 ± 9.2 mm with shifts occurring primarily in the inferior/superior direction. With respect to supine MR to surgical alignment, the average fiducial, target, and maximum target registration errors were 9.0 ± 1.7 mm, 9.3 ± 1.7 mm, and 20.0 ± 7.6 mm, respectively. Even when maintaining similar arm positions in the MR image and mock surgery, the respective averages were 6.0 ± 1.0 mm, 6.5 ± 1.1 mm, and 12.5 ± 2.8 mm.

CONCLUSION

From supine MR positioning to surgical presentation, the breast undergoes large displacements (9.9-70.1 mm). The data also suggest that significant nonrigid deformations (9.3 ± 1.7 mm with 20.0 mm average maximum) exist that need to be considered in image guidance and modeling applications.

Recent Progress for the Techniques of MRI-Guided Breast Interventions and their applications on Surgical Strategy

With a high sensitivity of breast lesions, MRI can detect suspicious lesions which are occult in traditional breast examination equipment. However, the lower and variable specificity of MRI makes the MRI-guided intervention, including biopsies and localizations, necessary before surgery, especially for patients who need the treatment of breast-conserving surgery (BCS). MRI techniques and patient preparation should be first carefully considered before the intervention to avoid lengthening the procedure time and compromising targeting accuracy. Doctors and radiologists need to reconfirm the target of the lesion and be very familiar with the process approach and equipment techniques involving the computer-aided diagnosis (CAD) tools and the biopsy system and follow a correct way. The basic steps of MRI-guided biopsy and localization are nearly the same regardless of the vendor or platform, and this article systematically introduces detailed methods and techniques of MRI-guided intervention. The two interventions both face different challenging situations during procedures with solutions given in the article. Post-operative statistics show that the complications of MRI-guided intervention are infrequent and mild, and MRI-guided biopsy provides the pathological information for the subsequent surgical decisions and MRI-guided localization fully prepared for follow-up surgical biopsy. New techniques for MRI-guided intervention are also elaborated in the article, which leads to future development. In a word, MRI-guided intervention is a safe, accurate, and effective technique with a low complication rate and successful MRI-guided intervention is truly teamwork with efforts from patients to surgeons, radiologists, MRI technologists, and nurses.

Real-time and multimodality image-guided intelligent HIFU therapy for uterine fibroid

Rationale: High-intensity focused ultrasound (HIFU) therapy represents a noninvasive surgical approach to treat uterine fibroids. The operation of HIFU therapy relies on the information provided by medical images. In current HIFU therapy, all operations such as positioning of the lesion in magnetic resonance (MR) and ultrasound (US) images are manually performed by specifically trained doctors. Manual processing is an important limitation of the efficiency of HIFU therapy. In this paper, we aim to provide an automatic and accurate image guidance system, intelligent diagnosis, and treatment strategy for HIFU therapy by combining multimodality information.

Methods: In intelligent HIFU therapy, medical information and treatment strategy are automatically processed and generated by a real-time image guidance system. The system comprises a novel multistage deep convolutional neural network for preoperative diagnosis and a nonrigid US lesion tracking procedure for HIFU intraoperative image-assisted treatment. In the process of intelligent therapy, the treatment area is determined from the autogenerated lesion area. Based on the autodetected treatment area, the HIFU foci are distributed automatically according to the treatment strategy. Moreover, an image-based unexpected movement warning and other physiological monitoring are used during the intelligent treatment procedure for safety assurance.

Results: In the experiment, we integrated the intelligent treatment system on a commercial HIFU treatment device, and eight clinical experiments were performed. In the clinical validation, eight randomly selected clinical cases were used to verify the feasibility of the system. The results of the quantitative experiment indicated that our intelligent system met the HIFU clinical tracking accuracy and speed requirements. Moreover, the results of simulated repeated experiments confirmed that the autodistributed HIFU focus reached the level of intermediate clinical doctors. Operations performed by junior- or middle-level operators with the assistance of the proposed system can reach the level of operation performed by senior doctors. Various experiments prove that our proposed intelligent HIFU therapy process is feasible for treating common uterine fibroid cases.

Conclusion: We propose an intelligent HIFU therapy for uterine fibroid which integrates multiple medical information processing procedures. The experiment results demonstrated that the proposed procedures and methods can achieve monitored and automatic HIFU diagnosis and treatment. This research provides a possibility for intelligent and automatic noninvasive therapy for uterine fibroid.

Feasibility of mapping breast cancer with supine breast MRI in patients scheduled for oncoplastic surgery

OBJECTIVES

To prospectively determine the feasibility of preoperative supine breast MRI in breast cancer patients scheduled for oncoplastic breast-conserving surgery.

METHODS

In addition to a diagnostic prone breast MRI, a supplementary supine MRI was performed with the patient in the surgical position including skin markers. Tumours' locations were ink-marked on the skin according to findings obtained from supine MRI. Changes in tumours' largest diameter and locations between prone and supine MRI were measured and compared to histology. Nipple-to-tumour and tumour-to-chest wall distances were also measured. Tumours and suspicious areas were surgically removed according to skin ink-markings. The differences between MRI measurements with reference to histopathology were evaluated with the paired-sample t test.

RESULTS

Fourteen consecutive patients, 15 breasts and 27 lesions were analysed. Compared to histology, prone MRI overestimated tumour size by 47.1% (p = 0.01) and supine MRI by 14.5% (p = 0.259). In supine MRI, lesions' mean diameters and areas were smaller compared to prone MRI (- 20.9%, p = 0.009 and - 38.3%, p = 0.016, respectively). This difference in diameter was more pronounced in non-mass lesions (- 31.2%, p = 0.031) compared to mass lesions (- 9.2%, p = 0.009). Tumours' mean distance from chest wall diminished by 69.4% (p < 0.001) and from nipple by 18.2% (p < 0.001). Free microscopic margins were achieved in first operation in all patients.

CONCLUSIONS

Supine MRI in the surgical position is feasible and useful in the precise localisation of prone MRI-detected lesions and provides a helpful tool to implement in surgery. Supine MRI more accurately determines tumours' size and location and might have an important role to diminish overestimations.

KEY POINTS

• Breath-hold supine breast MRI is feasible using commercially available coils and sequences. • Size and area of lesions on MRI were consistently smaller when measured from the supine position as compared to the prone position. • Supine breast MRI is useful in the precise preoperative localisation of prone MRI-detected lesions. •.

Development of a Surface Marker for Fractional Anisotropy Maps Using Wood in a Phantom Study

Purpose:

To improve imaging, a reliable setup method is critical for the accurate localization of lesions and surface markers. Because an anisotropic marker has not yet been validated for MRI, direct localization of surface markers is not yet feasible in fractional anisotropy (FA) maps. This study aimed to develop an anisotropic surface marker using wood for an FA map and to determine whether a wood marker is useful for various sequences.

Methods:

Wood infiltrated with water was used to develop an anisotropic surface marker. The wood marker was compared with phantoms composed of clinically available markers, including MR-SPOTS Packets (Beekley Medical, Bristol, CT, USA), Breath Care Oral Refreshing Capsules (Kobayashi Pharmaceutical Co., Ltd., Osaka, Japan), and baby oil (Johnson & Johnson, New Brunswick, NJ, USA). Magnetic resonance images were acquired using the Achieva 3T TX MRI System (Philips HealthCare, Best, Netherlands) equipped with a QD head coil including T₁- and T₂-weighted imaging, proton-density-weighted imaging, T2* -weighted imaging, T₁-weighted imaging spectral pre-saturation with inversion recovery, T₂-weighted imaging spectral attenuated inversion recovery, proton-density-weighted imaging spectral attenuated inversion recovery, diffusion weighted imaging, and diffusion tensor imaging. Apparent diffusion coefficient, FA values, and signal-to-noise ratio (SNR) were measured and recorded, and the coefficient of variation was calculated for two consecutive imaging scans. The wood was observed using a microscope.

Results:

Breath Care Oral Refreshing Capsules and baby oil were not observed in the FA map. The FA value of the MR-SPOTS Packets was 0.18. The FA value of the wood marker was 0.80. The coefficient of variation of the MR-SPOTS Packets and the wood marker were 0.0263 and 0.0013, respectively, in the FA map. Microscopic observation revealed a wood anisotropic structure.

Conclusion:

The wood maker enabled direct localization in the FA map. Hence, wood markers may be useful to radiologists and contribute to obtaining useful findings.

Symmetric Biomechanically Guided Prone-to-Supine Breast Image Registration

Prone-to-supine breast image registration has potential application in the fields of surgical and radiotherapy planning, image guided interventions, and multi-modal cancer diagnosis, staging, and therapy response prediction. However, breast image registration of three dimensional images acquired in different patient positions is a challenging problem, due to large deformations induced to the soft breast tissue caused by the change in gravity loading. We present a symmetric, biomechanical simulation based registration framework which aligns the images in a central, virtually unloaded configuration. The breast tissue is modelled as a neo-Hookean material and gravity is considered as the main source of deformation in the original images. In addition to gravity, our framework successively applies image derived forces directly into the unloading simulation in place of a subsequent image registration step. This results in a biomechanically constrained deformation. Using a finite difference scheme avoids an explicit meshing step and enables simulations to be performed directly in the image space. The explicit time integration scheme allows the motion at the interface between chest and breast to be constrained along the chest wall. The feasibility and accuracy of the approach presented here was assessed by measuring the target registration error (TRE) using a numerical phantom with known ground truth deformations, nine clinical prone MRI and supine CT image pairs, one clinical prone-supine CT image pair and four prone-supine MRI image pairs. The registration reduced the mean TRE for the numerical phantom experiment from initially 19.3 to 0.9 mm and the combined mean TRE for all fourteen clinical data sets from 69.7 to 5.6 mm.

Realization of a biomechanical model-assisted image guidance system for breast cancer surgery using supine MRI

PURPOSE

Unfortunately, the current re-excision rates for breast conserving surgeries due to positive margins average 20-40 %. The high re-excision rates arise from difficulty in localizing tumor boundaries intraoperatively and lack of real-time information on the presence of residual disease. The work presented here introduces the use of supine magnetic resonance (MR) images, digitization technology, and biomechanical models to investigate the capability of using an image guidance system to localize tumors intraoperatively.

METHODS

Preoperative supine MR images were used to create patient-specific biomechanical models of the breast tissue, chest wall, and tumor. In a mock intraoperative setup, a laser range scanner was used to digitize the breast surface and tracked ultrasound was used to digitize the chest wall and tumor. Rigid registration combined with a novel nonrigid registration routine was used to align the preoperative and intraoperative patient breast and tumor. The registration framework is driven by breast surface data (laser range scan of visible surface), ultrasound chest wall surface, and MR-visible fiducials. Tumor localizations by tracked ultrasound were only used to evaluate the fidelity of aligning preoperative MR tumor contours to physical patient space. The use of tracked ultrasound to digitize subsurface features to constrain our nonrigid registration approach and to assess the fidelity of our framework makes this work unique. Two patient subjects were analyzed as a preliminary investigation toward the realization of this supine image-guided approach.

RESULTS

An initial rigid registration was performed using adhesive MR-visible fiducial markers for two patients scheduled for a lumpectomy. For patient 1, the rigid registration resulted in a root-mean-square fiducial registration error (FRE) of 7.5 mm and the difference between the intraoperative tumor centroid as visualized with tracked ultrasound imaging and the registered preoperative MR counterpart was 6.5 mm. Nonrigid correction resulted in a decrease in FRE to 2.9 mm and tumor centroid difference to 5.5 mm. For patient 2, rigid registration resulted in a FRE of 8.8 mm and a 3D tumor centroid difference of 12.5 mm. Following nonrigid correction for patient 2, the FRE was reduced to 7.4 mm and the 3D tumor centroid difference was reduced to 5.3 mm.

CONCLUSION

Using our prototype image-guided surgery platform, we were able to align intraoperative data with preoperative patient-specific models with clinically relevant accuracy; i.e., tumor centroid localizations of approximately 5.3-5.5 mm.