Modeling of retraction and resection for intraoperative updating of images

Adaptive Physics-Based Non-Rigid Registration for Immersive Image-Guided Neuronavigation Systems

Objective: In image-guided neurosurgery, co-registered preoperative anatomical, functional, and diffusion tensor imaging can be used to facilitate a safe resection of brain tumors in eloquent areas of the brain. However, the brain deforms during surgery, particularly in the presence of tumor resection. Non-Rigid Registration (NRR) of the preoperative image data can be used to create a registered image that captures the deformation in the intraoperative image while maintaining the quality of the preoperative image. Using clinical data, this paper reports the results of a comparison of the accuracy and performance among several non-rigid registration methods for handling brain deformation. A new adaptive method that automatically removes mesh elements in the area of the resected tumor, thereby handling deformation in the presence of resection is presented. To improve the user experience, we also present a new way of using mixed reality with ultrasound, MRI, and CT.

Materials and methods: This study focuses on 30 glioma surgeries performed at two different hospitals, many of which involved the resection of significant tumor volumes. An Adaptive Physics-Based Non-Rigid Registration method (A-PBNRR) registers preoperative and intraoperative MRI for each patient. The results are compared with three other readily available registration methods: a rigid registration implemented in 3D Slicer v4.4.0; a B-Spline non-rigid registration implemented in 3D Slicer v4.4.0; and PBNRR implemented in ITKv4.7.0, upon which A-PBNRR was based. Three measures were employed to facilitate a comprehensive evaluation of the registration accuracy: (i) visual assessment, (ii) a Hausdorff Distance-based metric, and (iii) a landmark-based approach using anatomical points identified by a neurosurgeon.

Results: The A-PBNRR using multi-tissue mesh adaptation improved the accuracy of deformable registration by more than five times compared to rigid and traditional physics based non-rigid registration, and four times compared to B-Spline interpolation methods which are part of ITK and 3D Slicer. Performance analysis showed that A-PBNRR could be applied, on average, in <2 min, achieving desirable speed for use in a clinical setting.

Conclusions: The A-PBNRR method performed significantly better than other readily available registration methods at modeling deformation in the presence of resection. Both the registration accuracy and performance proved sufficient to be of clinical value in the operating room. A-PBNRR, coupled with the mixed reality system, presents a powerful and affordable solution compared to current neuronavigation systems.

Accounting for intraoperative brain shift ascribable to cavity collapse during intracranial tumor resection

Purpose: For many patients with intracranial tumors, accurate surgical resection is a mainstay of their treatment paradigm. During surgical resection, image guidance is used to aid in localization and resection. Intraoperative brain shift can invalidate these guidance systems. One cause of intraoperative brain shift is cavity collapse due to tumor resection, which will be referred to as "debulking." We developed an imaging-driven finite element model of debulking to create a comprehensive simulation data set to reflect possible intraoperative changes. The objective was to create a method to account for brain shift due to debulking for applications in image-guided neurosurgery. We hypothesized that accounting for tumor debulking in a deformation atlas data framework would improve brain shift predictions, which would enhance image-based surgical guidance. Approach: This was evaluated in a six-patient intracranial tumor resection intraoperative data set. The brain shift deformation atlas data framework consisted of n = 756 simulated deformations to account for effects due to gravity-induced and hyperosmotic drug-induced brain shift, which reflects previous developments. An additional complement of n = 84 deformations involving simulated tumor growth followed by debulking was created to capture observed intraoperative effects not previously included. Results: In five of six patient cases evaluated, inclusion of debulking mechanics improved brain shift correction by capturing global mass effects resulting from the resected tumor. Conclusions: These findings suggest imaging-driven brain shift models used to create a deformation simulation data framework of observed intraoperative events can be used to assist in more accurate image-guided surgical navigation in the brain.

Automatic segmentation of brain tumor resections in intraoperative ultrasound images using U-Net

To compensate for the intraoperative brain tissue deformation, computer-assisted intervention methods have been used to register preoperative magnetic resonance images with intraoperative images. In order to model the deformation due to tissue resection, the resection cavity needs to be segmented in intraoperative images. We present an automatic method to segment the resection cavity in intraoperative ultrasound (iUS) images. We trained and evaluated two-dimensional (2-D) and three-dimensional (3-D) U-Net networks on two datasets of 37 and 13 cases that contain images acquired from different ultrasound systems. The best overall performing method was the 3-D network, which resulted in a 0.72 mean and 0.88 median Dice score over the whole dataset. The 2-D network also had good results with less computation time, with a median Dice score over 0.8. We also evaluated the sensitivity of network performance to training and testing with images from different ultrasound systems and image field of view. In this application, we found specialized networks to be more accurate for processing similar images than a general network trained with all the data. Overall, promising results were obtained for both datasets using specialized networks. This motivates further studies with additional clinical data, to enable training and validation of a clinically viable deep-learning model for automated delineation of the tumor resection cavity in iUS images.

On the accuracy of optically tracked transducers for image-guided transcranial ultrasound

PURPOSE

Transcranial focused ultrasound (FUS) is increasingly being explored to modulate neuronal activity. To target neuromodulation, researchers often localize the FUS beam onto the brain region(s) of interest using spatially tracked tools overlaid on pre-acquired images. Here, we quantify the accuracy of optically tracked image-guided FUS with magnetic resonance imaging (MRI) thermometry, evaluate sources of error and demonstrate feasibility of these procedures to target the macaque somatosensory region.

METHODS

We developed an optically tracked FUS system capable of projecting the transducer focus onto a pre-acquired MRI volume. To measure the target registration error (TRE), we aimed the transducer focus at a desired target in a phantom under image guidance, heated the target while imaging with MR thermometry and then calculated the TRE as the difference between the targeted and heated locations. Multiple targets were measured using either an unbiased or bias-corrected calibration. We then targeted the macaque S1 brain region, where displacement induced by the acoustic radiation force was measured using MR acoustic radiation force imaging (MR-ARFI).

RESULTS

All calibration methods enabled registration with TRE on the order of 3 mm. Unbiased calibration resulted in an average TRE of 3.26 mm (min-max: 2.80-4.53 mm), which was not significantly changed by prospective bias correction (TRE of 3.05 mm; 2.06-3.81 mm, p = 0.55). Restricting motion between the transducer and target and increasing the distance between tracked markers reduced the TRE to 2.43 mm (min-max: 0.79-3.88 mm). MR-ARFI images showed qualitatively similar shape and extent as projected beam profiles in a living non-human primate.

CONCLUSIONS

Our study describes methods for image guidance of FUS neuromodulation and quantifies errors associated with this method in a large animal. The workflow is efficient enough for in vivo use, and we demonstrate transcranial MR-ARFI in vivo in macaques for the first time.

In vivo modeling of interstitial pressure in a porcine model: approximation of poroelastic properties and effects of enhanced anatomical structure modeling

The purpose of this investigation is to test whether a poroelastic model with enhanced structure can capture in vivo interstitial pressure dynamics in a brain undergoing mock surgical loads. Using interstitial pressure data from a porcine study, we use an inverse model to reconstruct material properties in an effort to capture these in vivo brain tissue dynamics. Four distinct models for the reconstruction of parameters are investigated (full anatomical condition description, condition without dural septa description, condition without ventricle boundary description, and the conventional fully saturated model). These models are systematic in their development to isolate the influence of three model characteristics: the dural septa, the treatment of the ventricles, and the treatment of the brain as a saturated media. This study demonstrates that to capture appropriate pressure compartmentalization, interstitial pressure gradients, pressure transient effects, and deformations within the brain, the proposed boundary conditions and structural enhancement coupled with a heterogeneous description invoking partial saturation are needed in a biphasic poroelastic model. These findings suggest that with enhanced anatomical modeling and appropriate model assumptions, poroelastic models can be used to capture quite complex brain deformations and interstitial pressure dynamics.

Stereovision-based integrated system for point cloud reconstruction and simulated brain shift validation

Intraoperative soft tissue deformation, referred to as brain shift, compromises the application of current image-guided surgery navigation systems in neurosurgery. A computational model driven by sparse data has been proposed as a cost-effective method to compensate for cortical surface and volumetric displacements. We present a mock environment developed to acquire stereoimages from a tracked operating microscope and to reconstruct three-dimensional point clouds from these images. A reconstruction error of 1 mm is estimated by using a phantom with a known geometry and independently measured deformation extent. The microscope is tracked via an attached tracking rigid body that facilitates the recording of the position of the microscope via a commercial optical tracking system as it moves during the procedure. Point clouds, reconstructed under different microscope positions, are registered into the same space to compute the feature displacements. Using our mock craniotomy device, realistic cortical deformations are generated. When comparing our tracked microscope stereo-pair measure of mock vessel displacements to that of the measurement determined by the independent optically tracked stylus marking, the displacement error was [Formula: see text] on average. These results demonstrate the practicality of using tracked stereoscopic microscope as an alternative to laser range scanners to collect sufficient intraoperative information for brain shift correction.

Retrospective study comparing model-based deformation correction to intraoperative magnetic resonance imaging for image-guided neurosurgery

Brain shift during tumor resection compromises the spatial validity of registered preoperative imaging data that is critical to image-guided procedures. One current clinical solution to mitigate the effects is to reimage using intraoperative magnetic resonance (iMR) imaging. Although iMR has demonstrated benefits in accounting for preoperative-to-intraoperative tissue changes, its cost and encumbrance have limited its widespread adoption. While iMR will likely continue to be employed for challenging cases, a cost-effective model-based brain shift compensation strategy is desirable as a complementary technology for standard resections. We performed a retrospective study of [Formula: see text] tumor resection cases, comparing iMR measurements with intraoperative brain shift compensation predicted by our model-based strategy, driven by sparse intraoperative cortical surface data. For quantitative assessment, homologous subsurface targets near the tumors were selected on preoperative MR and iMR images. Once rigidly registered, intraoperative shift measurements were determined and subsequently compared to model-predicted counterparts as estimated by the brain shift correction framework. When considering moderate and high shift ([Formula: see text], [Formula: see text] measurements per case), the alignment error due to brain shift reduced from [Formula: see text] to [Formula: see text], representing [Formula: see text] correction. These first steps toward validation are promising for model-based strategies.

In Vivo Investigation of the Effectiveness of a Hyper-viscoelastic Model in Simulating Brain Retraction

Intraoperative brain retraction leads to a misalignment between the intraoperative positions of the brain structures and their previous positions, as determined from preoperative images. In vitro swine brain sample uniaxial tests showed that the mechanical response of brain tissue to compression and extension could be described by the hyper-viscoelasticity theory. The brain retraction caused by the mechanical process is a combination of brain tissue compression and extension. In this paper, we first constructed a hyper-viscoelastic framework based on the extended finite element method (XFEM) to simulate intraoperative brain retraction. To explore its effectiveness, we then applied this framework to an in vivo brain retraction simulation. The simulation strictly followed the clinical scenario, in which seven swine were subjected to brain retraction. Our experimental results showed that the hyper-viscoelastic XFEM framework is capable of simulating intraoperative brain retraction and improving the navigation accuracy of an image-guided neurosurgery system (IGNS).

Computational Modeling for Enhancing Soft Tissue Image Guided Surgery: An Application in Neurosurgery

With the recent advances in computing, the opportunities to translate computational models to more integrated roles in patient treatment are expanding at an exciting rate. One area of considerable development has been directed towards correcting soft tissue deformation within image guided neurosurgery applications. This review captures the efforts that have been undertaken towards enhancing neuronavigation by the integration of soft tissue biomechanical models, imaging and sensing technologies, and algorithmic developments. In addition, the review speaks to the evolving role of modeling frameworks within surgery and concludes with some future directions beyond neurosurgical applications.

A nonrigid registration method for correcting brain deformation induced by tumor resection

PURPOSE

This paper presents a nonrigid registration method to align preoperative MRI with intraoperative MRI to compensate for brain deformation during tumor resection. This method extends traditional point-based nonrigid registration in two aspects: (1) allow the input data to be incomplete and (2) simulate the underlying deformation with a heterogeneous biomechanical model.

METHODS

The method formulates the registration as a three-variable (point correspondence, deformation field, and resection region) functional minimization problem, in which point correspondence is represented by a fuzzy assign matrix; Deformation field is represented by a piecewise linear function regularized by the strain energy of a heterogeneous biomechanical model; and resection region is represented by a maximal simply connected tetrahedral mesh. A nested expectation and maximization framework is developed to simultaneously resolve these three variables.

RESULTS

To evaluate this method, the authors conducted experiments on both synthetic data and clinical MRI data. The synthetic experiment confirmed their hypothesis that the removal of additional elements from the biomechanical model can improve the accuracy of the registration. The clinical MRI experiments on 25 patients showed that the proposed method outperforms the ITK implementation of a physics-based nonrigid registration method. The proposed method improves the accuracy by 2.88 mm on average when the error is measured by a robust Hausdorff distance metric on Canny edge points, and improves the accuracy by 1.56 mm on average when the error is measured by six anatomical points.

CONCLUSIONS

The proposed method can effectively correct brain deformation induced by tumor resection.

Evaluation of conoscopic holography for estimating tumor resection cavities in model-based image-guided neurosurgery

Surgical navigation relies on accurately mapping the intraoperative state of the patient to models derived from preoperative images. In image-guided neurosurgery, soft tissue deformations are common and have been shown to compromise the accuracy of guidance systems. In lieu of whole-brain intraoperative imaging, some advocate the use of intraoperatively acquired sparse data from laser-range scans, ultrasound imaging, or stereo reconstruction coupled with a computational model to drive subsurface deformations. Some authors have reported on compensating for brain sag, swelling, retraction, and the application of pharmaceuticals such as mannitol with these models. To date, strategies for modeling tissue resection have been limited. In this paper, we report our experiences with a novel digitization approach, called a conoprobe, to document tissue resection cavities and assess the impact of resection on model-based guidance systems. Specifically, the conoprobe was used to digitize the interior of the resection cavity during eight brain tumor resection surgeries and then compared against model prediction results of tumor locations. We should note that no effort was made to incorporate resection into the model but rather the objective was to determine if measurement was possible to study the impact on modeling tissue resection. In addition, the digitized resection cavity was compared with early postoperative MRI scans to determine whether these scans can further inform tissue resection. The results demonstrate benefit in model correction despite not having resection explicitly modeled. However, results also indicate the challenge that resection provides for model-correction approaches. With respect to the digitization technology, it is clear that the conoprobe provides important real-time data regarding resection and adds another dimension to our noncontact instrumentation framework for soft-tissue deformation compensation in guidance systems.

Near Real-Time Computer Assisted Surgery for Brain Shift Correction Using Biomechanical Models

Conventional image-guided neurosurgery relies on preoperative images to provide surgical navigational information and visualization. However, these images are no longer accurate once the skull has been opened and brain shift occurs. To account for changes in the shape of the brain caused by mechanical (e.g., gravity-induced deformations) and physiological effects (e.g., hyperosmotic drug-induced shrinking, or edema-induced swelling), updated images of the brain must be provided to the neuronavigation system in a timely manner for practical use in the operating room. In this paper, a novel preoperative and intraoperative computational processing pipeline for near real-time brain shift correction in the operating room was developed to automate and simplify the processing steps. Preoperatively, a computer model of the patient's brain with a subsequent atlas of potential deformations due to surgery is generated from diagnostic image volumes. In the case of interim gross changes between diagnosis, and surgery when reimaging is necessary, our preoperative pipeline can be generated within one day of surgery. Intraoperatively, sparse data measuring the cortical brain surface is collected using an optically tracked portable laser range scanner. These data are then used to guide an inverse modeling framework whereby full volumetric brain deformations are reconstructed from precomputed atlas solutions to rapidly match intraoperative cortical surface shift measurements. Once complete, the volumetric displacement field is used to update, i.e., deform, preoperative brain images to their intraoperative shifted state. In this paper, five surgical cases were analyzed with respect to the computational pipeline and workflow timing. With respect to postcortical surface data acquisition, the approximate execution time was 4.5 min. The total update process which included positioning the scanner, data acquisition, inverse model processing, and image deforming was ~11-13 min. In addition, easily implemented hardware, software, and workflow processes were identified for improved performance in the near future.

Toward a real time multi-tissue Adaptive Physics-Based Non-Rigid Registration framework for brain tumor resection

This paper presents an adaptive non-rigid registration method for aligning pre-operative MRI with intra-operative MRI (iMRI) to compensate for brain deformation during brain tumor resection. This method extends a successful existing Physics-Based Non-Rigid Registration (PBNRR) technique implemented in ITKv4.5. The new method relies on a parallel adaptive heterogeneous biomechanical Finite Element (FE) model for tissue/tumor removal depicted in the iMRI. In contrast the existing PBNRR in ITK relies on homogeneous static FE model designed for brain shift only (i.e., it is not designed to handle brain tumor resection). As a result, the new method (1) accurately captures the intra-operative deformations associated with the tissue removal due to tumor resection and (2) reduces the end-to-end execution time to within the time constraints imposed by the neurosurgical procedure. The evaluation of the new method is based on 14 clinical cases with: (i) brain shift only (seven cases), (ii) partial tumor resection (two cases), and (iii) complete tumor resection (five cases). The new adaptive method can reduce the alignment error up to seven and five times compared to a rigid and ITK's PBNRR registration methods, respectively. On average, the alignment error of the new method is reduced by 9.23 and 5.63 mm compared to the alignment error from the rigid and PBNRR method implemented in ITK. Moreover, the total execution time for all the case studies is about 1 min or less in a Linux Dell workstation with 12 Intel Xeon 3.47 GHz CPU cores and 96 GB of RAM.

More accurate neuronavigation data provided by biomechanical modeling instead of rigid registration

It is possible to improve neuronavigation during image-guided surgery by warping the high-quality preoperative brain images so that they correspond with the current intraoperative configuration of the brain. In this paper, the accuracy of registration results obtained using comprehensive biomechanical models is compared with the accuracy of rigid registration, the technology currently available to patients. This comparison allows investigation into whether biomechanical modeling provides good-quality image data for neuronavigation for a larger proportion of patients than rigid registration. Preoperative images for 33 neurosurgery cases were warped onto their respective intraoperative configurations using both the biomechanics-based method and rigid registration. The Hausdorff distance-based evaluation process, which measures the difference between images, was used to quantify the performance of both registration methods. A statistical test for difference in proportions was conducted to evaluate the null hypothesis that the proportion of patients for whom improved neuronavigation can be achieved is the same for rigid and biomechanics-based registration. The null hypothesis was confidently rejected (p < 10(-4)). Even the modified hypothesis that fewer than 25% of patients would benefit from the use of biomechanics-based registration was rejected at a significance level of 5% (p = 0.02). The biomechanics-based method proved particularly effective in cases demonstrating large craniotomy-induced brain deformations. The outcome of this analysis suggests that nonlinear biomechanics-based methods are beneficial to a large proportion of patients and can be considered for use in the operating theater as a possible means of improving neuronavigation and surgical outcomes.

A framework for correcting brain retraction based on an eXtended Finite Element Method using a laser range scanner

BACKGROUND

Brain retraction causes great distortion that limits the accuracy of an image-guided neurosurgery system that uses preoperative images. Therefore, brain retraction correction is an important intraoperative clinical application.

METHODS

We used a linear elastic biomechanical model, which deforms based on the eXtended Finite Element Method (XFEM) within a framework for brain retraction correction. In particular, a laser range scanner was introduced to obtain a surface point cloud of the exposed surgical field including retractors inserted into the brain. A brain retraction surface tracking algorithm converted these point clouds into boundary conditions applied to XFEM modeling that drive brain deformation. To test the framework, we performed a brain phantom experiment involving the retraction of tissue. Pairs of the modified Hausdorff distance between Canny edges extracted from model-updated images, pre-retraction, and post-retraction CT images were compared to evaluate the morphological alignment of our framework. Furthermore, the measured displacements of beads embedded in the brain phantom and the predicted ones were compared to evaluate numerical performance.

RESULTS

The modified Hausdorff distance of 19 pairs of images decreased from 1.10 to 0.76 mm. The forecast error of 23 stainless steel beads in the phantom was between 0 and 1.73 mm (mean 1.19 mm). The correction accuracy varied between 52.8 and 100 % (mean 81.4 %).

CONCLUSIONS

The results demonstrate that the brain retraction compensation can be incorporated intraoperatively into the model-updating process in image-guided neurosurgery systems.

Extra-dimensional Demons: a method for incorporating missing tissue in deformable image registration

PURPOSE

A deformable registration method capable of accounting for missing tissue (e.g., excision) is reported for application in cone-beam CT (CBCT)-guided surgical procedures. Excisions are identified by a segmentation step performed simultaneous to the registration process. Tissue excision is explicitly modeled by increasing the dimensionality of the deformation field to allow motion beyond the dimensionality of the image. The accuracy of the model is tested in phantom, simulations, and cadaver models.

METHODS

A variant of the Demons deformable registration algorithm is modified to include excision segmentation and modeling. Segmentation is performed iteratively during the registration process, with initial implementation using a threshold-based approach to identify voxels corresponding to "tissue" in the moving image and "air" in the fixed image. With each iteration of the Demons process, every voxel is assigned a probability of excision. Excisions are modeled explicitly during registration by increasing the dimensionality of the deformation field so that both deformations and excisions can be accounted for by in- and out-of-volume deformations, respectively. The out-of-volume (i.e., fourth) component of the deformation field at each voxel carries a magnitude proportional to the excision probability computed in the excision segmentation step. The registration accuracy of the proposed "extra-dimensional" Demons (XDD) and conventional Demons methods was tested in the presence of missing tissue in phantom models, simulations investigating the effect of excision size on registration accuracy, and cadaver studies emulating realistic deformations and tissue excisions imparted in CBCT-guided endoscopic skull base surgery.

RESULTS

Phantom experiments showed the normalized mutual information (NMI) in regions local to the excision to improve from 1.10 for the conventional Demons approach to 1.16 for XDD, and qualitative examination of the resulting images revealed major differences: the conventional Demons approach imparted unrealistic distortions in areas around tissue excision, whereas XDD provided accurate "ejection" of voxels within the excision site and maintained the registration accuracy throughout the rest of the image. Registration accuracy in areas far from the excision site (e.g., > ∼5 mm) was identical for the two approaches. Quantitation of the effect was consistent in analysis of NMI, normalized cross-correlation (NCC), target registration error (TRE), and accuracy of voxels ejected from the volume (true-positive and false-positive analysis). The registration accuracy for conventional Demons was found to degrade steeply as a function of excision size, whereas XDD was robust in this regard. Cadaver studies involving realistic excision of the clivus, vidian canal, and ethmoid sinuses demonstrated similar results, with unrealistic distortion of anatomy imparted by conventional Demons and accurate ejection and deformation for XDD.

CONCLUSIONS

Adaptation of the Demons deformable registration process to include segmentation (i.e., identification of excised tissue) and an extra dimension in the deformation field provided a means to accurately accommodate missing tissue between image acquisitions. The extra-dimensional approach yielded accurate "ejection" of voxels local to the excision site while preserving the registration accuracy (typically subvoxel) of the conventional Demons approach throughout the rest of the image. The ability to accommodate missing tissue volumes is important to application of CBCT for surgical guidance (e.g., skull base drillout) and may have application in other areas of CBCT guidance.

Preliminary study of a novel method for conveying corrected image volumes in surgical navigation

BACKGROUND

Commercial image-guided surgery systems rely on the fundamental assumption that preoperative medical images represent the physical state of the patient in the operating room. The guidance display typically consists of a three-dimensional (3D) model derived from medical images and three orthogonal views of the imaging data. A challenging question in image-guided surgery is: what happens when the images used in the guidance display no longer correspond to the current geometric state of the anatomy and guidance information is still desirable?

METHODS

We modify the conventional display with two techniques for incorporating a displacement field from a finite-element model into the guidance display and present a preliminary study of the effect of our method on performance with a simple surgical task. The topic of this paper is methods for conveying the computational model solution, not the model itself. To address the integration of the computational model solution into the display, a novel method of applying the deformation to the tool tip was developed, which quickly corrects for deformation but also maintains the pristine nature of the preoperative images. We compare the proposed technique to an existing method that applies the deformation field to the image volume.

RESULTS

A pilot study compared mean performance with our method of applying the deformation to the tool tip and the conventional technique. These methods were statistically similar with respect to accuracy of localization (p < 0.05) and amount of time (p < 0.05) required for localization of the target.

CONCLUSIONS

These results suggest that our new technique can be used in place of the computationally expensive task of deforming the image volume, without affecting the time or accuracy of the surgical task. Most notably, our work addresses the problem of incorporating deformation correction into the guidance display and offers a first step toward understanding its effect on surgical performance.

Serial FEM/XFEM-Based Update of Preoperative Brain Images Using Intraoperative MRI

Current neuronavigation systems cannot adapt to changing intraoperative conditions over time. To overcome this limitation, we present an experimental end-to-end system capable of updating 3D preoperative images in the presence of brain shift and successive resections. The heart of our system is a nonrigid registration technique using a biomechanical model, driven by the deformations of key surfaces tracked in successive intraoperative images. The biomechanical model is deformed using FEM or XFEM, depending on the type of deformation under consideration, namely, brain shift or resection. We describe the operation of our system on two patient cases, each comprising five intraoperative MR images, and we demonstrate that our approach significantly improves the alignment of nonrigidly registered images.

δ-aminolevulinic acid-induced protoporphyrin IX concentration correlates with histopathologic markers of malignancy in human gliomas: the need for quantitative fluorescence-guided resection to identify regions of increasing malignancy

Extent of resection is a major goal and prognostic factor in the treatment of gliomas. In this study we evaluate whether quantitative ex vivo tissue measurements of δ-aminolevulinic acid-induced protoporphyrin IX (PpIX) identify regions of increasing malignancy in low- and high-grade gliomas beyond the capabilities of current fluorescence imaging in patients undergoing fluorescence-guided resection (FGR). Surgical specimens were collected from 133 biopsies in 23 patients and processed for ex vivo neuropathological analysis: PpIX fluorimetry to measure PpIX concentrations (C(PpIX)) and Ki-67 immunohistochemistry to assess tissue proliferation. Samples displaying visible levels of fluorescence showed significantly higher levels of C(PpIX) and tissue proliferation. C(PpIX) was strongly correlated with histopathological score (nonparametric) and tissue proliferation (parametric), such that increasing levels of C(PpIX) were identified with regions of increasing malignancy. Furthermore, a large percentage of tumor-positive biopsy sites (∼40%) that were not visibly fluorescent under the operating microscope had levels of C(PpIX) greater than 0.1 µg/mL, which indicates that significant PpIX accumulation exists below the detection threshold of current fluorescence imaging. Although PpIX fluorescence is recognized as a visual biomarker for neurosurgical resection guidance, these data show that it is quantitatively related at the microscopic level to increasing malignancy in both low- and high-grade gliomas. This work suggests a need for improved PpIX fluorescence detection technologies to achieve better sensitivity and quantification of PpIX in tissue during surgery.

3D XFEM-based modeling of retraction for preoperative image update

Outcomes for neurosurgery patients can be improved by enhancing intraoperative navigation and guidance. Current navigation systems do not accurately account for intraoperative brain deformation. So far, most studies of brain deformation have focused on brain shift, whereas this paper focuses on the brain deformation due to retraction. The heart of our system is a 3D nonrigid registration technique using a biomechanical model driven by the deformations of key surfaces tracked between two intraoperative images. The key surfaces, e.g., the whole-brain region boundary and the lips of the retraction cut, thus deform due to the combination of gravity and retractor deployment. The tissue discontinuity due to retraction is handled via the eXtended Finite Element Method (XFEM), which has the appealing feature of being able to handle arbitrarily shaped discontinuity without any remeshing. Our approach is shown to significantly improve the alignment of intraoperative MRI.

Multimodal image registration for preoperative planning and image-guided neurosurgical procedures

Image registration is the process of transforming images acquired at different time points, or with different imaging modalities, into the same coordinate system. It is an essential part of any neurosurgical planning and navigation system because it facilitates combining images with important complementary, structural, and functional information to improve the information based on which a surgeon makes critical decisions. Brigham and Women's Hospital (BWH) has been one of the pioneers in developing intraoperative registration methods for aligning preoperative and intraoperative images of the brain. This article presents an overview of intraoperative registration and highlights some recent developments at BWH.

Virtual reality augmentation in skull base surgery

OBJECTIVE

Skull base anatomy is complex and subject to individual variation. Understanding the complexity of surgical anatomy is faster and easier with virtual models created from primary imaging data of the patient. This study was designed to investigate the usefulness of virtual reality in image guidance for skull base procedures.

DESIGN

Primary volumetric image data from 110 patients was acquired using magnetic resonance, computed tomography (CT), and CT angiography. Pathologies included lesions in the anterior, middle, and posterior skull base. The data were transferred to an infrared-based image-guidance system for creation of a virtual operating field (VOF) with translucent surface modulation and optional "fly-through" video mode. During surgery, the target registration error for anatomical landmarks was assessed and the VOF was compared with the patient's anatomy in the operative field.

RESULTS

Complex structures like the course of the sigmoid sinus, the carotid artery, and the outline of the paranasal sinuses were well visualized in the VOF and were recognized by the surgeon instantly. Perception was greatly facilitated as compared with routine mental reconstruction of triaxial images. Accurate assessment of the depth of field and very small objects was not possible in VOF images.

CONCLUSION

Supported by sound anatomical knowledge, creation of a virtual operating field for a surgical approach in an individual patient offers a déjà vu experience that can enhance the capabilities of a surgical team in skull base approaches. In addition, application of this technique in image-guided procedures assists in targeting or avoiding hidden anatomical structures.

2D XFEM-based modeling of retraction and successive resections for preoperative image update

This paper considers an approach to improving outcomes for neurosurgery patients by enhancing intraoperative navigation and guidance. Currently, intraoperative navigation systems do not accurately account for brain shift or tissue resection. We describe how preoperative images can be incrementally updated to take into account any type of brain tissue deformation that may occur during surgery, and thus to improve the accuracy of image-guided navigation systems. For this purpose, we have developed a non-rigid image registration technique using a biomechanical model, which deforms based on the Finite Element Method (FEM). While the FEM has been used successfully for dealing with deformations such as brain shift, it has difficulty with tissue discontinuities. Here, we describe a novel application of the eXtended Finite Element Method (XFEM) in the field of image-guided surgery in order to model brain deformations that imply tissue discontinuities. In particular, this paper presents a detailed account of the use of XFEM for dealing with retraction and successive resections, and demonstrates the feasibility of the approach by considering 2D examples based on intraoperative MR images. To evaluate our results, we compute the modified Hausdorff distance between Canny edges extracted from images before and after registration. We show that this distance decreases after registration, and thus demonstrate that our approach improves alignment of intraoperative images.

Patient-specific non-linear finite element modelling for predicting soft organ deformation in real-time: application to non-rigid neuroimage registration

Long computation times of non-linear (i.e. accounting for geometric and material non-linearity) biomechanical models have been regarded as one of the key factors preventing application of such models in predicting organ deformation for image-guided surgery. This contribution presents real-time patient-specific computation of the deformation field within the brain for six cases of brain shift induced by craniotomy (i.e. surgical opening of the skull) using specialised non-linear finite element procedures implemented on a graphics processing unit (GPU). In contrast to commercial finite element codes that rely on an updated Lagrangian formulation and implicit integration in time domain for steady state solutions, our procedures utilise the total Lagrangian formulation with explicit time stepping and dynamic relaxation. We used patient-specific finite element meshes consisting of hexahedral and non-locking tetrahedral elements, together with realistic material properties for the brain tissue and appropriate contact conditions at the boundaries. The loading was defined by prescribing deformations on the brain surface under the craniotomy. Application of the computed deformation fields to register (i.e. align) the preoperative and intraoperative images indicated that the models very accurately predict the intraoperative deformations within the brain. For each case, computing the brain deformation field took less than 4 s using an NVIDIA Tesla C870 GPU, which is two orders of magnitude reduction in computation time in comparison to our previous study in which the brain deformation was predicted using a commercial finite element solver executed on a personal computer.

A fast and efficient method to compensate for brain shift for tumor resection therapies measured between preoperative and postoperative tomograms

In this paper, an efficient paradigm is presented to correct for brain shift during tumor resection therapies. For this study, high resolution preoperative (pre-op) and postoperative (post-op) MR images were acquired for eight in vivo patients, and surface/subsurface shift was identified by manual identification of homologous points between the pre-op and immediate post-op tomograms. Cortical surface deformation data were then used to drive an inverse problem framework. The manually identified subsurface deformations served as a comparison toward validation. The proposed framework recaptured 85% of the mean subsurface shift. This translated to a subsurface shift error of 0.4 +/- 0.4 mm for a measured shift of 3.1 +/- 0.6 mm. The patient's pre-op tomograms were also deformed volumetrically using displacements predicted by the model. Results presented allow a preliminary evaluation of correction both quantitatively and visually. While intraoperative (intra-op) MR imaging data would be optimal, the extent of shift measured from pre- to post-op MR was comparable to clinical conditions. This study demonstrates the accuracy of the proposed framework in predicting full-volume displacements from sparse shift measurements. It also shows that the proposed framework can be extended and used to update pre-op images on a time scale that is compatible with surgery.

An atlas-based method to compensate for brain shift: preliminary results

Compensating for intraoperative brain shift using computational models has shown promising results. Since computational time is an important factor during neurosurgery, a priori knowledge of the possible sources of deformation can increase the accuracy of model-updated image-guided systems. In this paper, a strategy to compensate for distributed loading conditions in the brain such as brain sag, volume changes due to drug reactions, and brain swelling due to edema is presented. An atlas of model deformations based on these complex loading conditions is computed preoperatively and used with a constrained linear inverse model to predict the intraoperative distributed brain shift. This relatively simple inverse finite-element approach is investigated within the context of a series of phantom experiments, two in vivo cases, and a simulation study. Preliminary results indicate that the approach recaptured on average 93% of surface shift for the simulation, phantom, and in vivo experiments. With respect to subsurface shift, comparisons were only made with simulation and phantom experiments and demonstrated an ability to recapture 85% of the shift. This translates to a remaining surface and subsurface shift error of 0.7+/-0.3 mm, and 1.0+/-0.4 mm, respectively, for deformations on the order of 1cm.

Non-rigid alignment of pre-operative MRI, fMRI, and DT-MRI with intra-operative MRI for enhanced visualization and navigation in image-guided neurosurgery

OBJECTIVE

The usefulness of neurosurgical navigation with current visualizations is seriously compromised by brain shift, which inevitably occurs during the course of the operation, significantly degrading the precise alignment between the pre-operative MR data and the intra-operative shape of the brain. Our objectives were (i) to evaluate the feasibility of non-rigid registration that compensates for the brain deformations within the time constraints imposed by neurosurgery, and (ii) to create augmented reality visualizations of critical structural and functional brain regions during neurosurgery using pre-operatively acquired fMRI and DT-MRI.

MATERIALS AND METHODS

Eleven consecutive patients with supratentorial gliomas were included in our study. All underwent surgery at our intra-operative MR imaging-guided therapy facility and have tumors in eloquent brain areas (e.g. precentral gyrus and cortico-spinal tract). Functional MRI and DT-MRI, together with MPRAGE and T2w structural MRI were acquired at 3 T prior to surgery. SPGR and T2w images were acquired with a 0.5 T magnet during each procedure. Quantitative assessment of the alignment accuracy was carried out and compared with current state-of-the-art systems based only on rigid registration.

RESULTS

Alignment between pre-operative and intra-operative datasets was successfully carried out during surgery for all patients. Overall, the mean residual displacement remaining after non-rigid registration was 1.82 mm. There is a statistically significant improvement in alignment accuracy utilizing our non-rigid registration in comparison to the currently used technology (p<0.001).

CONCLUSIONS

We were able to achieve intra-operative rigid and non-rigid registration of (1) pre-operative structural MRI with intra-operative T1w MRI; (2) pre-operative fMRI with intra-operative T1w MRI, and (3) pre-operative DT-MRI with intra-operative T1w MRI. The registration algorithms as implemented were sufficiently robust and rapid to meet the hard real-time constraints of intra-operative surgical decision making. The validation experiments demonstrate that we can accurately compensate for the deformation of the brain and thus can construct an augmented reality visualization to aid the surgeon.

Incorporation of a laser range scanner into image-guided liver surgery: surface acquisition, registration, and tracking

As image guided surgical procedures become increasingly diverse, there will be more scenarios where point-based fiducials cannot be accurately localized for registration and rigid body assumptions no longer hold. As a result, procedures will rely more frequently on anatomical surfaces for the basis of image alignment and will require intraoperative geometric data to measure and compensate for tissue deformation in the organ. In this paper we outline methods for which a laser range scanner may be used to accomplish these tasks intraoperatively. A laser range scanner based on the optical principle of triangulation acquires a dense set of three-dimensional point data in a very rapid, noncontact fashion. Phantom studies were performed to test the ability to link range scan data with traditional modes of image-guided surgery data through localization, registration, and tracking in physical space. The experiments demonstrate that the scanner is capable of localizing point-based fiducials to within 0.2 mm and capable of achieving point and surface based registrations with target registration error of less than 2.0 mm. Tracking points in physical space with the range scanning system yields an error of 1.4 +/- 0.8 mm. Surface deformation studies were performed with the range scanner in order to determine if this device was capable of acquiring enough information for compensation algorithms. In the surface deformation studies, the range scanner was able to detect changes in surface shape due to deformation comparable to those detected by tomographic image studies. Use of the range scanner has been approved for clinical trials, and an initial intraoperative range scan experiment is presented. In all of these studies, the primary source of error in range scan data is deterministically related to the position and orientation of the surface within the scanner's field of view. However, this systematic error can be corrected, allowing the range scanner to provide a rapid, robust method of acquiring anatomical surfaces intraoperatively.