Displacement estimation with co-registered ultrasound for image guided neurosurgery: a quantitative in vivo porcine study

Enabling Navigation and Augmented Reality in the Sitting Position in Posterior Fossa Surgery Using Intraoperative Ultrasound

Despite its broad use in cranial and spinal surgery, navigation support and microscope-based augmented reality (AR) have not yet found their way into posterior fossa surgery in the sitting position. While this position offers surgical benefits, navigation accuracy and thereof the use of navigation itself seems limited. Intraoperative ultrasound (iUS) can be applied at any time during surgery, delivering real-time images that can be used for accuracy verification and navigation updates. Within this study, its applicability in the sitting position was assessed. Data from 15 patients with lesions within the posterior fossa who underwent magnetic resonance imaging (MRI)-based navigation-supported surgery in the sitting position were retrospectively analyzed using the standard reference array and new rigid image-based MRI-iUS co-registration. The navigation accuracy was evaluated based on the spatial overlap of the outlined lesions and the distance between the corresponding landmarks in both data sets, respectively. Image-based co-registration significantly improved (p < 0.001) the spatial overlap of the outlined lesion (0.42 ± 0.30 vs. 0.65 ± 0.23) and significantly reduced (p < 0.001) the distance between the corresponding landmarks (8.69 ± 6.23 mm vs. 3.19 ± 2.73 mm), allowing for the sufficient use of navigation and AR support. Navigated iUS can therefore serve as an easy-to-use tool to enable navigation support for posterior fossa surgery in the sitting position.

Navigated Intraoperative 3D Ultrasound in Glioblastoma Surgery: Analysis of Imaging Features and Impact on Extent of Resection

Background

Neuronavigation is routinely used in glioblastoma surgery, but its accuracy decreases during the operative procedure due to brain shift, which can be addressed utilizing intraoperative imaging. Intraoperative ultrasound (iUS) is widely available, offers excellent live imaging, and can be fully integrated into modern navigational systems. Here, we analyze the imaging features of navigated i3D US and its impact on the extent of resection (EOR) in glioblastoma surgery.

Methods

Datasets of 31 glioblastoma resection procedures were evaluated. Patient registration was established using intraoperative computed tomography (iCT). Pre-operative MRI (pre-MRI) and pre-resectional ultrasound (pre-US) datasets were compared regarding segmented tumor volume, spatial overlap (Dice coefficient), the Euclidean distance of the geometric center of gravity (CoG), and the Hausdorff distance. Post-resectional ultrasound (post-US) and post-operative MRI (post-MRI) tumor volumes were analyzed and categorized into subtotal resection (STR) or gross total resection (GTR) cases.

Results

The mean patient age was 59.3 ± 11.9 years. There was no significant difference in pre-resectional segmented tumor volumes (pre-MRI: 24.2 ± 22.3 cm³; pre-US: 24.0 ± 21.8 cm³). The Dice coefficient was 0.71 ± 0.21, the Euclidean distance of the CoG was 3.9 ± 3.0 mm, and the Hausdorff distance was 12.2 ± 6.9 mm. A total of 18 cases were categorized as GTR, 10 cases were concordantly classified as STR on MRI and ultrasound, and 3 cases had to be excluded from post-resectional analysis. In four cases, i3D US triggered further resection.

Conclusion

Navigated i3D US is reliably adjunct in a multimodal navigational setup for glioblastoma resection. Tumor segmentations revealed similar results in i3D US and MRI, demonstrating the capability of i3D US to delineate tumor boundaries. Additionally, i3D US has a positive influence on the EOR, allows live imaging, and depicts brain shift.

Navigated 3D Ultrasound in Brain Metastasis Surgery: Analyzing the Differences in Object Appearances in Ultrasound and Magnetic Resonance Imaging

Background: Implementation of intraoperative 3D ultrasound (i3D US) into modern neuronavigational systems offers the possibility of live imaging and subsequent imaging updates. However, different modalities, image acquisition strategies, and timing of imaging influence object appearances. We analyzed the differences in object appearances in ultrasound (US) and magnetic resonance imaging (MRI) in 35 cases of brain metastasis, which were operated in a multimodal navigational setup after intraoperative computed tomography based (iCT) registration. Method: Registration accuracy was determined using the target registration error (TRE). Lesions segmented in preoperative magnetic resonance imaging (preMRI) and i3D US were compared focusing on object size, location, and similarity. Results: The mean and standard deviation (SD) of the TRE was 0.84 ± 0.36 mm. Objects were similar in size (mean ± SD in preMRI: 13.6 ± 16.0 cm3 vs. i3D US: 13.5 ± 16.0 cm3). The Dice coefficient was 0.68 ± 0.22 (mean ± SD), the Hausdorff distance 8.1 ± 2.9 mm (mean ± SD), and the Euclidean distance of the centers of gravity 3.7 ± 2.5 mm (mean ± SD). Conclusion: i3D US clearly delineates tumor boundaries and allows live updating of imaging for compensation of brain shift, which can already be identified to a significant amount before dural opening.

Intraoperative Imaging Modalities and Compensation for Brain Shift in Tumor Resection Surgery

Intraoperative brain shift during neurosurgical procedures is a well-known phenomenon caused by gravity, tissue manipulation, tumor size, loss of cerebrospinal fluid (CSF), and use of medication. For the use of image-guided systems, this phenomenon greatly affects the accuracy of the guidance. During the last several decades, researchers have investigated how to overcome this problem. The purpose of this paper is to present a review of publications concerning different aspects of intraoperative brain shift especially in a tumor resection surgery such as intraoperative imaging systems, quantification, measurement, modeling, and registration techniques. Clinical experience of using intraoperative imaging modalities, details about registration, and modeling methods in connection with brain shift in tumor resection surgery are the focuses of this review. In total, 126 papers regarding this topic are analyzed in a comprehensive summary and are categorized according to fourteen criteria. The result of the categorization is presented in an interactive web tool. The consequences from the categorization and trends in the future are discussed at the end of this work.

Near real-time robust non-rigid registration of volumetric ultrasound images for neurosurgery

Ultrasound images are acquired before and after the resection of brain tumors to help the surgeon to localize the tumor and its extent and to minimize the amount of residual tumor after the resection. Because the brain undergoes large deformation between these two acquisitions, deformable image-based registration of these data sets is of substantial clinical importance. In this work, we present an algorithm for non-rigid registration of ultrasound images (RESOUND) that models the deformation with free-form cubic B-splines. We formulate a regularized cost function that uses normalized cross-correlation as the similarity metric. To optimize the cost function, we calculate its analytic derivative and use the stochastic gradient descent technique to achieve near real-time performance. We further propose a robust technique to minimize the effect of non-corresponding regions such as the resected tumor and possible hemorrhage in the post-resection image. Using manually labeled corresponding landmarks in the pre- and post-resection ultrasound volumes, we illustrate that our registration algorithm reduces the mean target registration error from an initial value of 3.7 to 1.5 mm. We also compare RESOUND with the previous work of Mercier et al. (2013) and illustrate that it has three important advantages: (i) it is fully automatic and does not require a manual segmentation of the tumor, (ii) it produces smaller registration errors and (iii) it is about 30 times faster. The clinical data set is available online on the BITE database website.

Gadolinium- and 5-aminolevulinic acid-induced protoporphyrin IX levels in human gliomas: an ex vivo quantitative study to correlate protoporphyrin IX levels and blood-brain barrier breakdown

In recent years, 5-aminolevulinic acid (ALA)-induced protoporphyrin IX (PpIX) fluorescence guidance has been used as a surgical adjunct to improve the extent of resection of gliomas. Exogenous administration of ALA before surgery leads to the accumulation of red fluorescent PpIX in tumor tissue that the surgeon can visualize and thereby discriminate between normal and tumor tissue. Selective accumulation of PpIX has been linked to numerous factors, of which blood-brain barrier breakdown has been suggested to be a key factor. To test the hypothesis that PpIX concentration positively correlates with gadolinium (Gd) concentrations, we performed ex vivo measurements of PpIX and of Gd using inductively coupled plasma mass spectrometry, the latter as a quantitative biomarker of blood-brain barrier breakdown; this was corroborated with immunohistochemistry of microvascular density in surgical biopsies of patients undergoing fluorescence-guided surgery for glioma. We found positive correlations between PpIX concentration and Gd concentration (r = 0.58, p < 0.0001) and between PpIX concentration and microvascular density (r = 0.55, p < 0.0001), suggesting a significant, yet limited, association between blood-brain barrier breakdown and ALA-induced PpIX fluorescence. To our knowledge, this is the first time that Gd measurements by inductively coupled plasma mass spectrometry have been used in human gliomas.

Preliminary study of a novel method for conveying corrected image volumes in surgical navigation

BACKGROUND

Commercial image-guided surgery systems rely on the fundamental assumption that preoperative medical images represent the physical state of the patient in the operating room. The guidance display typically consists of a three-dimensional (3D) model derived from medical images and three orthogonal views of the imaging data. A challenging question in image-guided surgery is: what happens when the images used in the guidance display no longer correspond to the current geometric state of the anatomy and guidance information is still desirable?

METHODS

We modify the conventional display with two techniques for incorporating a displacement field from a finite-element model into the guidance display and present a preliminary study of the effect of our method on performance with a simple surgical task. The topic of this paper is methods for conveying the computational model solution, not the model itself. To address the integration of the computational model solution into the display, a novel method of applying the deformation to the tool tip was developed, which quickly corrects for deformation but also maintains the pristine nature of the preoperative images. We compare the proposed technique to an existing method that applies the deformation field to the image volume.

RESULTS

A pilot study compared mean performance with our method of applying the deformation to the tool tip and the conventional technique. These methods were statistically similar with respect to accuracy of localization (p < 0.05) and amount of time (p < 0.05) required for localization of the target.

CONCLUSIONS

These results suggest that our new technique can be used in place of the computationally expensive task of deforming the image volume, without affecting the time or accuracy of the surgical task. Most notably, our work addresses the problem of incorporating deformation correction into the guidance display and offers a first step toward understanding its effect on surgical performance.

3D Rigid Registration of Intraoperative Ultrasound and Preoperative MR Brain Images Based on Hyperechogenic Structures

The registration of intraoperative ultrasound (US) images with preoperative magnetic resonance (MR) images is a challenging problem due to the difference of information contained in each image modality. To overcome this difficulty, we introduce a new probabilistic function based on the matching of cerebral hyperechogenic structures. In brain imaging, these structures are the liquid interfaces such as the cerebral falx and the sulci, and the lesions when the corresponding tissue is hyperechogenic. The registration procedure is achieved by maximizing the joint probability for a voxel to be included in hyperechogenic structures in both modalities. Experiments were carried out on real datasets acquired during neurosurgical procedures. The proposed validation framework is based on (i) visual assessment, (ii) manual expert estimations , and (iii) a robustness study. Results show that the proposed method (i) is visually efficient, (ii) produces no statistically different registration accuracy compared to manual-based expert registration, and (iii) converges robustly. Finally, the computation time required by our method is compatible with intraoperative use.

Serial FEM/XFEM-Based Update of Preoperative Brain Images Using Intraoperative MRI

Current neuronavigation systems cannot adapt to changing intraoperative conditions over time. To overcome this limitation, we present an experimental end-to-end system capable of updating 3D preoperative images in the presence of brain shift and successive resections. The heart of our system is a nonrigid registration technique using a biomechanical model, driven by the deformations of key surfaces tracked in successive intraoperative images. The biomechanical model is deformed using FEM or XFEM, depending on the type of deformation under consideration, namely, brain shift or resection. We describe the operation of our system on two patient cases, each comprising five intraoperative MR images, and we demonstrate that our approach significantly improves the alignment of nonrigidly registered images.

δ-aminolevulinic acid-induced protoporphyrin IX concentration correlates with histopathologic markers of malignancy in human gliomas: the need for quantitative fluorescence-guided resection to identify regions of increasing malignancy

Extent of resection is a major goal and prognostic factor in the treatment of gliomas. In this study we evaluate whether quantitative ex vivo tissue measurements of δ-aminolevulinic acid-induced protoporphyrin IX (PpIX) identify regions of increasing malignancy in low- and high-grade gliomas beyond the capabilities of current fluorescence imaging in patients undergoing fluorescence-guided resection (FGR). Surgical specimens were collected from 133 biopsies in 23 patients and processed for ex vivo neuropathological analysis: PpIX fluorimetry to measure PpIX concentrations (C(PpIX)) and Ki-67 immunohistochemistry to assess tissue proliferation. Samples displaying visible levels of fluorescence showed significantly higher levels of C(PpIX) and tissue proliferation. C(PpIX) was strongly correlated with histopathological score (nonparametric) and tissue proliferation (parametric), such that increasing levels of C(PpIX) were identified with regions of increasing malignancy. Furthermore, a large percentage of tumor-positive biopsy sites (∼40%) that were not visibly fluorescent under the operating microscope had levels of C(PpIX) greater than 0.1 µg/mL, which indicates that significant PpIX accumulation exists below the detection threshold of current fluorescence imaging. Although PpIX fluorescence is recognized as a visual biomarker for neurosurgical resection guidance, these data show that it is quantitatively related at the microscopic level to increasing malignancy in both low- and high-grade gliomas. This work suggests a need for improved PpIX fluorescence detection technologies to achieve better sensitivity and quantification of PpIX in tissue during surgery.

3D XFEM-based modeling of retraction for preoperative image update

Outcomes for neurosurgery patients can be improved by enhancing intraoperative navigation and guidance. Current navigation systems do not accurately account for intraoperative brain deformation. So far, most studies of brain deformation have focused on brain shift, whereas this paper focuses on the brain deformation due to retraction. The heart of our system is a 3D nonrigid registration technique using a biomechanical model driven by the deformations of key surfaces tracked between two intraoperative images. The key surfaces, e.g., the whole-brain region boundary and the lips of the retraction cut, thus deform due to the combination of gravity and retractor deployment. The tissue discontinuity due to retraction is handled via the eXtended Finite Element Method (XFEM), which has the appealing feature of being able to handle arbitrarily shaped discontinuity without any remeshing. Our approach is shown to significantly improve the alignment of intraoperative MRI.

A non rigid registration and deformation algorithm for ultrasound & MR images to guide prostate cancer therapies

Multimodality image registration is a critical issue in image-guided cancer ablation techniques. Focal therapies of prostate cancer are usually monitored using ultrasound imaging, while the dose planning is performed on MRI. In this study, a new multimodality images registration and deformation method, based on the Thin Plate Splines -Rigid Point Matching (TPS-RPM) algorithm, is introduced. The Method combines non-rigid mapping and interpolation to deform the images. Preliminary results obtained on phantom and clinical images showed that the registration is accurate and robust against landmarks initialization.

Model-updated image-guided liver surgery: preliminary results using surface characterization

The current protocol for image guidance in open abdominal liver tumor removal surgeries involves a rigid registration between the patient's operating room space and the pre-operative diagnostic image-space. Systematic studies have shown that the liver can deform up to 2 cm during surgeries in a non-rigid fashion thereby compromising the accuracy of these surgical navigation systems. Compensating for intra-operative deformations using mathematical models has shown promising results. In this work, we follow up the initial rigid registration with a computational approach that is geared towards minimizing the residual closest point distances between the un-deformed pre-operative surface and the rigidly registered intra-operative surface. We also use a surface Laplacian equation based filter that generates a realistic deformation field. Preliminary validation of the proposed computational framework was performed using phantom experiments and clinical trials. The proposed framework improved the rigid registration errors for the phantom experiments on average by 43%, and 74% using partial and full surface data, respectively. With respect to clinical data, it improved the closest point residual error associated with rigid registration by 54% on average for the clinical cases. These results are highly encouraging and suggest that computational models can be used to increase the accuracy of image-guided open abdominal liver tumor removal surgeries.

A fast and efficient method to compensate for brain shift for tumor resection therapies measured between preoperative and postoperative tomograms

In this paper, an efficient paradigm is presented to correct for brain shift during tumor resection therapies. For this study, high resolution preoperative (pre-op) and postoperative (post-op) MR images were acquired for eight in vivo patients, and surface/subsurface shift was identified by manual identification of homologous points between the pre-op and immediate post-op tomograms. Cortical surface deformation data were then used to drive an inverse problem framework. The manually identified subsurface deformations served as a comparison toward validation. The proposed framework recaptured 85% of the mean subsurface shift. This translated to a subsurface shift error of 0.4 +/- 0.4 mm for a measured shift of 3.1 +/- 0.6 mm. The patient's pre-op tomograms were also deformed volumetrically using displacements predicted by the model. Results presented allow a preliminary evaluation of correction both quantitatively and visually. While intraoperative (intra-op) MR imaging data would be optimal, the extent of shift measured from pre- to post-op MR was comparable to clinical conditions. This study demonstrates the accuracy of the proposed framework in predicting full-volume displacements from sparse shift measurements. It also shows that the proposed framework can be extended and used to update pre-op images on a time scale that is compatible with surgery.

Data assimilation using a gradient descent method for estimation of intraoperative brain deformation

Biomechanical models that simulate brain deformation are gaining attention as alternatives for brain shift compensation. One approach, known as the "forced-displacement method", constrains the model to exactly match the measured data through boundary condition (BC) assignment. Although it improves model estimates and is computationally attractive, the method generates fictitious forces and may be ill-advised due to measurement uncertainty. Previously, we have shown that by assimilating intraoperatively acquired brain displacements in an inversion scheme, the Representer algorithm (REP) is able to maintain stress-free BCs and improve model estimates by 33% over those without data guidance in a controlled environment. However, REP is computationally efficient only when a few data points are used for model guidance because its costs scale linearly in the number of data points assimilated, thereby limiting its utility (and accuracy) in clinical settings. In this paper, we present a steepest gradient descent algorithm (SGD) whose computational complexity scales nearly invariantly with the number of measurements assimilated by iteratively adjusting the forcing conditions to minimize the difference between measured and model-estimated displacements (model-data misfit). Solutions of full linear systems of equations are achieved with a parallelized direct solver on a shared-memory, eight-processor Linux cluster. We summarize the error contributions from the entire process of model-updated image registration compensation and we show that SGD is able to attain model estimates comparable to or better than those obtained with REP, capturing about 74-82% of tumor displacement, but with a computational effort that is significantly less (a factor of 4-fold or more reduction relative to REP) and nearly invariant to the amount of sparse data involved when the number of points assimilated is large. Based on five patient cases, an average computational cost of approximately 2 min for estimating whole-brain deformation has been achieved with SGD using 100 sparse data points, suggesting the new algorithm is sufficiently fast with adequate accuracy for routine use in the operating room (OR).

Cortical and subcortical brain shift during stereotactic procedures

Object

The success of stereotactic surgery depends upon accuracy. Tissue deformation, or brain shift, can result in clinically significant errors. The authors measured cortical and subcortical brain shift during stereotactic surgery and assessed several variables that may affect it.

Methods

Preoperative and postoperative magnetic resonance imaging volumes were fused and 3D vectors of deviation were calculated for the anterior commissure (AC), posterior commissure (PC), and frontal cortex. Potential preoperative (age, diagnosis, and ventricular volume), intraoperative (stereotactic target, penetration of ventricles, and duration of surgery), and postoperative (volume of pneumocephalus) variables were analyzed and correlated with cortical (frontal cortex) and subcortical (AC, PC) deviations.

Results

Of 66 cases, nine showed a shift of the AC by more than 1.5 mm, and five by more than 2.0 mm. The largest AC shift was 5.67 mm. Deviation in the x, y, and z dimensions for each case was determined, and most of the cortical and subcortical shift occurred in the posterior direction. The mean 3D vector deviations for frontal cortex, AC, and PC were 3.5 ± 2.0, 1.0 ± 0.8, and 0.7 ± 0.5 mm, respectively. The mean change in AC–PC length was −0.2 ± −0.9 mm (range −4.28 to 1.66 mm). The volume of postoperative pneumocephalus, assumed to represent cerebrospinal fluid (CSF) loss, was significantly correlated with shift of the frontal cortex (r = 0.640, 64 degrees of freedom, p < 0.001) and even more strongly with shift of the AC (r = 0.754, p < 0.001). No other factors were significantly correlated with AC shift. Interestingly, penetration of the ventricles during electrode insertion, whether unilateral or bilateral, did not affect volume of pneumocephalus.

Conclusions

Cortical and subcortical brain shift occurs during stereotactic surgery as a direct function of the volume of pneumocephalus, which probably reflects the volume of CSF that is lost. Clinically significant shifts appear to be uncommon, but stereotactic surgeons should be vigilant in preventing CSF loss.

Is the image guidance of ultrasonography beneficial for neurosurgical routine?

BACKGROUND

Intraoperative US has been widely used in neurosurgical procedures. However, images are often difficult to read. In the present study, we evaluate whether the image guidance of ultrasonography is helpful for the interpretation of US scans.

METHODS

Twenty-nine patients with tumor were operated on with the aid of intraoperative US from January to June 2005. Image-guided sonography was used in 13 cases and nonnavigated US technology in the remaining cases. We compared the 2 technologies retrospectively.

RESULTS

Although image quality was good in most cases, orientation remained difficult in 8 of the 16 patients where conventional sonography was used. With the aid of image fusion for navigated sonography, the orientation was judged superior to nonnavigated US.

CONCLUSION

In our experience, integration of the US into the navigation system facilitates anatomical understanding. Thus, we feel that this technology is beneficial for neurosurgical routine.

Application of soft tissue modelling to image-guided surgery

The deformation of soft tissue compromises the accuracy of image-guided surgery based on preoperative images, and restricts its applicability to surgery on or near bony structures. One way to overcome these limitations is to combine biomechanical models with sparse intraoperative data, in order to realistically warp the preoperative image to match the surgical situation. We detail the process of biomechanical modelling in the context of image-guided surgery. We focus in particular on the finite element method, which is shown to be a promising approach, and review the constitutive relationships which have been suggested for representing tissue during surgery. Appropriate intraoperative measurements are required to constrain the deformation, and we discuss the potential of the modalities which have been applied to this task. This technology is on the verge of transition into clinical practice, where it promises to increase the guidance accuracy and facilitate less invasive interventions. We describe here how soft tissue modelling techniques have been applied to image-guided surgery applications.

Stereopsis-guided brain shift compensation

Brain deformation models have proven to be a powerful tool in compensating for soft tissue deformation during image-guided neurosurgery. The accuracy of these models can be improved by incorporating intraoperative measurements of brain motion. We have designed and implemented a passive intraoperative stereo vision system capable of estimating the three-dimensional shape of the surgical scene in near real-time. This intraoperative shape is compared with the cortical surface in the co-registered preoperative magnetic resonance (MR) volume for the estimation of the cortical motion resulting from the open cranial surgery. The estimated cortical motion is then used to guide a full brain model, which updates a preoperative MR volume. We have found that the stereo vision system is accurate to within approximately 1 mm. Based on data from two representative clinical cases, we show that stereopsis guidance improves the accuracy of brain shift compensation both at and below the cortical surface.