Comparison of physics-based deformable registration methods for image-guided neurosurgery

This paper compares three finite element-based methods used in a physics-based non-rigid registration approach and reports on the progress made over the last 15 years. Large brain shifts caused by brain tumor removal affect registration accuracy by creating point and element outliers. A combination of approximation- and geometry-based point and element outlier rejection improves the rigid registration error by 2.5 mm and meets the real-time constraints (4 min). In addition, the paper raises several questions and presents two open problems for the robust estimation and improvement of registration error in the presence of outliers due to sparse, noisy, and incomplete data. It concludes with preliminary results on leveraging Quantum Computing, a promising new technology for computationally intensive problems like Feature Detection and Block Matching in addition to finite element solver; all three account for 75% of computing time in deformable registration.

Real-Time Dynamic Data Driven Deformable Registration for Image-Guided Neurosurgery: Computational Aspects

Current neurosurgical procedures utilize medical images of various modalities to enable the precise location of tumors and critical brain structures to plan accurate brain tumor resection. The difficulty of using preoperative images during the surgery is caused by the intra-operative deformation of the brain tissue (brain shift), which introduces discrepancies concerning the preoperative configuration. Intra-operative imaging allows tracking such deformations but cannot fully substitute for the quality of the pre-operative data. Dynamic Data Driven Deformable Non-Rigid Registration (D4NRR) is a complex and time-consuming image processing operation that allows the dynamic adjustment of the pre-operative image data to account for intra-operative brain shift during the surgery. This paper summarizes the computational aspects of a specific adaptive numerical approximation method and its variations for registering brain MRIs. It outlines its evolution over the last 15 years and identifies new directions for the computational aspects of the technique.

A review of existing and potential computer user interfaces for modern radiology

Abstract

The digitalization of modern imaging has led radiologists to become very familiar with computers and their user interfaces (UI). New options for display and command offer expanded possibilities, but the mouse and keyboard remain the most commonly utilized, for usability reasons. In this work, we review and discuss different UI and their possible application in radiology. We consider two-dimensional and three-dimensional imaging displays in the context of interventional radiology, and discuss interest in touchscreens, kinetic sensors, eye detection, and augmented or virtual reality. We show that UI design specifically for radiologists is key for future use and adoption of such new interfaces. Next-generation UI must fulfil professional needs, while considering contextual constraints.

Teaching Points

• The mouse and keyboard remain the most utilized user interfaces for radiologists.

• Touchscreen, holographic, kinetic sensors and eye tracking offer new possibilities for interaction.

• 3D and 2D imaging require specific user interfaces.

• Holographic display and augmented reality provide a third dimension to volume imaging.

• Good usability is essential for adoption of new user interfaces by radiologists.

An ITK implementation of a physics-based non-rigid registration method for brain deformation in image-guided neurosurgery

As part of the ITK v4 project efforts, we have developed ITK filters for physics-based non-rigid registration (PBNRR), which satisfies the following requirements: account for tissue properties in the registration, improve accuracy compared to rigid registration, and reduce execution time using GPU and multi-core accelerators. The implementation has three main components: (1) Feature Point Selection, (2) Block Matching (mapped to both multi-core and GPU processors), and (3) a Robust Finite Element Solver. The use of multi-core and GPU accelerators in ITK v4 provides substantial performance improvements. For example, for the non-rigid registration of brain MRIs, the performance of the block matching filter on average is about 10 times faster when 12 hyperthreaded multi-cores are used and about 83 times faster when the NVIDIA Tesla GPU is used in Dell Workstation.

Spatial decision forests for MS lesion segmentation in multi-channel MR images

A new algorithm is presented for the automatic segmentation of Multiple Sclerosis (MS) lesions in 3D MR images. It builds on the discriminative random decision forest framework to provide a voxel-wise probabilistic classification of the volume. Our method uses multi-channel MIR intensities (T1, T2, Flair), spatial prior and long-range comparisons with 3D regions to discriminate lesions. A symmetry feature is introduced accounting for the fact that some MS lesions tend to develop in an asymmetric way. Quantitative evaluation of the data is carried out on publicly available labeled cases from the MS Lesion Segmentation Challenge 2008 dataset and demonstrates improved results over the state of the art.

Image guided personalization of reaction-diffusion type tumor growth models using modified anisotropic eikonal equations

Reaction-diffusion based tumor growth models have been widely used in the literature for modeling the growth of brain gliomas. Lately, recent models have started integrating medical images in their formulation. Including different tissue types, geometry of the brain and the directions of white matter fiber tracts improved the spatial accuracy of reaction-diffusion models. The adaptation of the general model to the specific patient cases on the other hand has not been studied thoroughly yet. In this paper, we address this adaptation. We propose a parameter estimation method for reaction-diffusion tumor growth models using time series of medical images. This method estimates the patient specific parameters of the model using the images of the patient taken at successive time instances. The proposed method formulates the evolution of the tumor delineation visible in the images based on the reaction-diffusion dynamics; therefore, it remains consistent with the information available. We perform thorough analysis of the method using synthetic tumors and show important couplings between parameters of the reaction-diffusion model. We show that several parameters can be uniquely identified in the case of fixing one parameter, namely the proliferation rate of tumor cells. Moreover, regardless of the value the proliferation rate is fixed to, the speed of growth of the tumor can be estimated in terms of the model parameters with accuracy. We also show that using the model-based speed, we can simulate the evolution of the tumor for the specific patient case. Finally, we apply our method to two real cases and show promising preliminary results.

Extrapolating glioma invasion margin in brain magnetic resonance images: suggesting new irradiation margins

Radiotherapy for brain glioma treatment relies on magnetic resonance (MR) and computed tomography (CT) images. These images provide information on the spatial extent of the tumor, but can only visualize parts of the tumor where cancerous cells are dense enough, masking the low density infiltration. In radiotherapy, a 2 m constant margin around the tumor is taken to account for this uncertainty. This approach however, does not consider the growth dynamics of gliomas, particularly the differential motility of tumor cells in the white and in the gray matter. In this article, we propose a novel method for estimating the full extent of the tumor infiltration starting from its visible mass in the patients' MR images. This estimation problem is a time independent problem where we do not have information about the temporal evolution of the pathology nor its initial conditions. Based on the reaction-diffusion models widely used in the literature, we derive a method to solve this extrapolation problem. Later, we use this formulation to tailor new tumor specific variable irradiation margins. We perform geometrical comparisons between the conventional constant and the proposed variable margins through determining the amount of targeted tumor cells and healthy tissue in the case of synthetic tumors. Results of these experiments suggest that the variable margin could be more effective at targeting cancerous cells and preserving healthy tissue.

DT-REFinD: diffusion tensor registration with exact finite-strain differential

In this paper, we propose the DT-REFinD algorithm for the diffeomorphic nonlinear registration of diffusion tensor images. Unlike scalar images, deforming tensor images requires choosing both a reorientation strategy and an interpolation scheme. Current diffusion tensor registration algorithms that use full tensor information face difficulties in computing the differential of the tensor reorientation strategy and consequently, these methods often approximate the gradient of the objective function. In the case of the finite-strain (FS) reorientation strategy, we borrow results from the pose estimation literature in computer vision to derive an analytical gradient of the registration objective function. By utilizing the closed-form gradient and the velocity field representation of one parameter subgroups of diffeomorphisms, the resulting registration algorithm is diffeomorphic and fast. We contrast the algorithm with a traditional FS alternative that ignores the reorientation in the gradient computation. We show that the exact gradient leads to significantly better registration at the cost of computation time. Independently of the choice of Euclidean or Log-Euclidean interpolation and sum of squared differences dissimilarity measure, the exact gradient achieves better alignment over an entire spectrum of deformation penalties. Alignment quality is assessed with a battery of metrics including tensor overlap, fractional anisotropy, inverse consistency and closeness to synthetic warps. The improvements persist even when a different reorientation scheme, preservation of principal directions, is used to apply the final deformations.

Compensation of geometric distortion effects on intraoperative magnetic resonance imaging for enhanced visualization in image-guided neurosurgery

OBJECTIVE

Preoperative magnetic resonance imaging (MRI), functional MRI, diffusion tensor MRI, magnetic resonance spectroscopy, and positron-emission tomographic scans may be aligned to intraoperative MRI to enhance visualization and navigation during image-guided neurosurgery. However, several effects (both machine- and patient-induced distortions) lead to significant geometric distortion of intraoperative MRI. Therefore, a precise alignment of these image modalities requires correction of the geometric distortion. We propose and evaluate a novel method to compensate for the geometric distortion of intraoperative 0.5-T MRI in image-guided neurosurgery.

METHODS

In this initial pilot study, 11 neurosurgical procedures were prospectively enrolled. The scheme used to correct the geometric distortion is based on a nonrigid registration algorithm introduced by our group. This registration scheme uses image features to establish correspondence between images. It estimates a smooth geometric distortion compensation field by regularizing the displacements estimated at the correspondences. A patient-specific linear elastic material model is used to achieve the regularization. The geometry of intraoperative images (0.5 T) is changed so that the images match the preoperative MRI scans (3 T).

RESULTS

We compared the alignment between preoperative and intraoperative imaging using 1) only rigid registration without correction of the geometric distortion, and 2) rigid registration and compensation for the geometric distortion. We evaluated the success of the geometric distortion correction algorithm by measuring the Hausdorff distance between boundaries in the 3-T and 0.5-T MRIs after rigid registration alone and with the addition of geometric distortion correction of the 0.5-T MRI. Overall, the mean magnitude of the geometric distortion measured on the intraoperative images is 10.3 mm with a minimum of 2.91 mm and a maximum of 21.5 mm. The measured accuracy of the geometric distortion compensation algorithm is 1.93 mm. There is a statistically significant difference between the accuracy of the alignment of preoperative and intraoperative images, both with and without the correction of geometric distortion (P < 0.001).

CONCLUSION

The major contributions of this study are 1) identification of geometric distortion of intraoperative images relative to preoperative images, 2) measurement of the geometric distortion, 3) application of nonrigid registration to compensate for geometric distortion during neurosurgery, 4) measurement of residual distortion after geometric distortion correction, and 5) phantom study to quantify geometric distortion.

Biocomputing: numerical simulation of glioblastoma growth using diffusion tensor imaging

Glioblastoma multiforma (GBM) is one of the most aggressive tumors of the central nervous system. It can be represented by two components: a proliferative component with a mass effect on brain structures and an invasive component. GBM has a distinct pattern of spread showing a preferential growth in the white fiber direction for the invasive component. By using the architecture of white matter fibers, we propose a new model to simulate the growth of GBM. This architecture is estimated by diffusion tensor imaging in order to determine the preferred direction for the diffusion component. It is then coupled with a mechanical component. To set up our growth model, we make a brain atlas including brain structures with a distinct response to tumor aggressiveness, white fiber diffusion tensor information and elasticity. In this atlas, we introduce a virtual GBM with a mechanical component coupled with a diffusion component. These two components are complementary, and can be tuned independently. Then, we tune the parameter set of our model with an MRI patient. We have compared simulated growth (initialized with the MRI patient) with observed growth six months later. The average and the odd ratio of image difference between observed and simulated images are computed. Displacements of reference points are compared to those simulated by the model. The results of our simulation have shown a good correlation with tumor growth, as observed on an MRI patient. Different tumor aggressiveness can also be simulated by tuning additional parameters. This work has demonstrated that modeling the complex behavior of brain tumors is feasible and will account for further validation of this new conceptual approach.

A recursive anisotropic fast marching approach to reaction diffusion equation: application to tumor growth modeling

Bridging the gap between clinical applications and mathematical models is one of the new challenges of medical image analysis. In this paper, we propose an efficient and accurate algorithm to solve anisotropic Eikonal equations, in order to link biological models using reaction-diffusion equations to clinical observations, such as medical images. The example application we use to demonstrate our methodology is tumor growth modeling. We simulate the motion of the tumor front visible in images and give preliminary results by solving the derived anisotropic Eikonal equation with the recursive fast marching algorithm.

Dynamic Model of Communicating Hydrocephalus for Surgery Simulation

We propose a dynamic model of cerebrospinal fluid and intracranial pressure regulation. In this model, we investigate the coupling of biological parameters with a 3-D model, to improve the behavior of the brain in surgical simulators. The model was assessed by comparing the simulated ventricular enlargement with a patient case study of communicating hydrocephalus. In our model, cerebro-spinal fluid production-resorption system is coupled with a 3-D representation of the brain parenchyma. We introduce a new bi-phasic model of the brain (brain tissue and extracellular fluid) allowing for fluid exchange between the brain extracellular space and the venous system. The time evolution of ventricular pressure has been recorded on a symptomatic patient after closing the ventricular shunt. A finite element model has been built based on a computed tomography scan of this patient, and quantitative comparisons between experimental measures and simulated data are proposed.

Non-rigid alignment of pre-operative MRI, fMRI, and DT-MRI with intra-operative MRI for enhanced visualization and navigation in image-guided neurosurgery

OBJECTIVE

The usefulness of neurosurgical navigation with current visualizations is seriously compromised by brain shift, which inevitably occurs during the course of the operation, significantly degrading the precise alignment between the pre-operative MR data and the intra-operative shape of the brain. Our objectives were (i) to evaluate the feasibility of non-rigid registration that compensates for the brain deformations within the time constraints imposed by neurosurgery, and (ii) to create augmented reality visualizations of critical structural and functional brain regions during neurosurgery using pre-operatively acquired fMRI and DT-MRI.

MATERIALS AND METHODS

Eleven consecutive patients with supratentorial gliomas were included in our study. All underwent surgery at our intra-operative MR imaging-guided therapy facility and have tumors in eloquent brain areas (e.g. precentral gyrus and cortico-spinal tract). Functional MRI and DT-MRI, together with MPRAGE and T2w structural MRI were acquired at 3 T prior to surgery. SPGR and T2w images were acquired with a 0.5 T magnet during each procedure. Quantitative assessment of the alignment accuracy was carried out and compared with current state-of-the-art systems based only on rigid registration.

RESULTS

Alignment between pre-operative and intra-operative datasets was successfully carried out during surgery for all patients. Overall, the mean residual displacement remaining after non-rigid registration was 1.82 mm. There is a statistically significant improvement in alignment accuracy utilizing our non-rigid registration in comparison to the currently used technology (p<0.001).

CONCLUSIONS

We were able to achieve intra-operative rigid and non-rigid registration of (1) pre-operative structural MRI with intra-operative T1w MRI; (2) pre-operative fMRI with intra-operative T1w MRI, and (3) pre-operative DT-MRI with intra-operative T1w MRI. The registration algorithms as implemented were sufficiently robust and rapid to meet the hard real-time constraints of intra-operative surgical decision making. The validation experiments demonstrate that we can accurately compensate for the deformation of the brain and thus can construct an augmented reality visualization to aid the surgeon.

Hybrid formulation of the model-based non-rigid registration problem to improve accuracy and robustness

We present a new algorithm to register 3D pre-operative Magnetic Resonance (MR) images with intra-operative MR images of the brain. This algorithm relies on a robust estimation of the deformation from a sparse set of measured displacements. We propose a new framework to compute iteratively the displacement field starting from an approximation formulation (minimizing the sum of a regularization term and a data error term) and converging toward an interpolation formulation (least square minimization of the data error term). The robustness of the algorithm is achieved through the introduction of an outliers rejection step in this gradual registration process. We ensure the validity of the deformation by the use of a biomechanical model of the brain specific to the patient, discretized with the finite element method. The algorithm has been tested on six cases of brain tumor resection, presenting a brain shift up to 13 mm.

Extrapolating Tumor Invasion Margins for Physiologically Determined Radiotherapy Regions

In radiotherapy, the constant margin taken around the visible tumor is a very coarse approximation of the invasion margin of cancerous cells. In this article, a new formulation to estimate the invasion margin of a tumor by extrapolating low tumor densities in magnetic resonance images (MRIs) is proposed. The current imaging techniques are able to show parts of the tumor where cancerous cells are dense enough. However, tissue parts containing small number of tumor cells are not enhanced in images. We propose a way to estimate these parts using the tumor mass visible in the image. Our formulation is based on the Fisher-Kolmogorov Equation that is been widely used to model the growth of brain tumors. As a proof of concept, we show some promising preliminary results, which demonstrate the feasibility of the approach.

Robust nonrigid registration to capture brain shift from intraoperative MRI

We present a new algorithm to register 3-D preoperative magnetic resonance (MR) images to intraoperative MR images of the brain which have undergone brain shift. This algorithm relies on a robust estimation of the deformation from a sparse noisy set of measured displacements. We propose a new framework to compute the displacement field in an iterative process, allowing the solution to gradually move from an approximation formulation (minimizing the sum of a regularization term and a data error term) to an interpolation formulation (least square minimization of the data error term). An outlier rejection step is introduced in this gradual registration process using a weighted least trimmed squares approach, aiming at improving the robustness of the algorithm. We use a patient-specific model discretized with the finite element method in order to ensure a realistic mechanical behavior of the brain tissue. To meet the clinical time constraint, we parallelized the slowest step of the algorithm so that we can perform a full 3-D image registration in 35 s (including the image update time) on a heterogeneous cluster of 15 personal computers. The algorithm has been tested on six cases of brain tumor resection, presenting a brain shift of up to 14 mm. The results show a good ability to recover large displacements, and a limited decrease of accuracy near the tumor resection cavity.

Realistic simulation of the 3-D growth of brain tumors in MR images coupling diffusion with biomechanical deformation

We propose a new model to simulate the three-dimensional (3-D) growth of glioblastomas multiforma (GBMs), the most aggressive glial tumors. The GBM speed of growth depends on the invaded tissue: faster in white than in gray matter, it is stopped by the dura or the ventricles. These different structures are introduced into the model using an atlas matching technique. The atlas includes both the segmentations of anatomical structures and diffusion information in white matter fibers. We use the finite element method (FEM) to simulate the invasion of the GBM in the brain parenchyma and its mechanical interaction with the invaded structures (mass effect). Depending on the considered tissue, the former effect is modeled with a reaction-diffusion or a Gompertz equation, while the latter is based on a linear elastic brain constitutive equation. In addition, we propose a new coupling equation taking into account the mechanical influence of the tumor cells on the invaded tissues. The tumor growth simulation is assessed by comparing the in-silico GBM growth with the real growth observed on two magnetic resonance images (MRIs) of a patient acquired with 6 mo difference. Results show the feasibility of this new conceptual approach and justifies its further evaluation.