Automatic Non-linear MRI-Ultrasound Registration for the Correction of Intra-operative Brain Deformations. In: Niessen WJ, Viergever MA, editors. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2001. Berlin: Springer Berlin Heidelberg; 2001. pp. 913-22. [DOI: 10.1007/3-540-45468-3_109]

Co-Sparse Analysis Model Based Image Registration to Compensate Brain Shift by Using Intra-Operative Ultrasound Imaging

Notwithstanding the widespread use of image guided neurosurgery systems in recent years, the accuracy of these systems is strongly limited by the intra-operative deformation of the brain tissue, the so-called brain shift. Intra-operative ultrasound (iUS) imaging as an effective solution to compensate complex brain shift phenomena update patients coordinate during surgery by registration of the intra-operative ultrasound and the pre-operative MRI data that is a challenging problem.In this work a non-rigid multimodal image registration technique based on co-sparse analysis model is proposed. This model captures the interdependency of two image modalities; MRI as an intensity image and iUS as a depth image. Based on this model, the transformation between the two modalities is minimized by using a bimodal pair of analysis operators which are learned by optimizing a joint co-sparsity function using a conjugate gradient.Experimental validation of our algorithm confirms that our registration approach outperforms several of other state-of-the-art registration methods quantitatively. The evaluation was performed using seven patient dataset with the mean registration error of only 1.83 mm. Our intensity-based co-sparse analysis model has improved the accuracy of non-rigid multimodal medical image registration by 15.37% compared to the curvelet based residual complexity as a powerful registration method, in a computational time compatible with clinical use.

Fast and robust multimodal image registration using a local derivative pattern

PURPOSE

Deformable multimodal image registration, which can benefit radiotherapy and image guided surgery by providing complementary information, remains a challenging task in the medical image analysis field due to the difficulty of defining a proper similarity measure. This article presents a novel, robust and fast binary descriptor, the discriminative local derivative pattern (dLDP), which is able to encode images of different modalities into similar image representations.

METHODS

dLDP calculates a binary string for each voxel according to the pattern of intensity derivatives in its neighborhood. The descriptor similarity is evaluated using the Hamming distance, which can be efficiently computed, instead of conventional L1 or L2 norms. For the first time, we validated the effectiveness and feasibility of the local derivative pattern for multimodal deformable image registration with several multi-modal registration applications.

RESULTS

dLDP was compared with three state-of-the-art methods in artificial image and clinical settings. In the experiments of deformable registration between different magnetic resonance imaging (MRI) modalities from BrainWeb, between computed tomography and MRI images from patient data, and between MRI and ultrasound images from BITE database, we show our method outperforms localized mutual information and entropy images in terms of both accuracy and time efficiency. We have further validated dLDP for the deformable registration of preoperative MRI and three-dimensional intraoperative ultrasound images. Our results indicate that dLDP reduces the average mean target registration error from 4.12 mm to 2.30 mm. This accuracy is statistically equivalent to the accuracy of the state-of-the-art methods in the study; however, in terms of computational complexity, our method significantly outperforms other methods and is even comparable to the sum of the absolute difference.

CONCLUSIONS

The results reveal that dLDP can achieve superior performance regarding both accuracy and time efficiency in general multimodal image registration. In addition, dLDP also indicates the potential for clinical ultrasound guided intervention.

Curvelet based residual complexity objective function for non-rigid registration of pre-operative MRI with intra-operative ultrasound images

Intra-operative ultrasound as an imaging based method has been recognized as an effective solution to compensate non rigid brain shift problem in recent years. Measuring brain shift requires registration of the pre-operative MRI images with the intra-operative ultrasound images which is a challenging task. In this study a novel hybrid method based on the matching echogenic structures such as sulci and tumor boundary in MRI with ultrasound images is proposed. The matching echogenic structures are achieved by optimizing the Residual Complexity (RC) in the curvelet domain. At the first step, the probabilistic map of the MR image is achieved and the residual image as the difference between this probabilistic map and intra-operative ultrasound is obtained. Then curvelet transform as a sparse function is used to minimize the complexity of residual image. The proposed method is a compromise between feature-based and intensity-based approaches. Evaluation was performed using 14 patients data set and the mean of registration error reached to 1.87 mm. This hybrid method based on RC improves accuracy of nonrigid multimodal image registration by 12.5% in a computational time compatible with clinical use.

A hybrid method for non-rigid registration of intra-operative ultrasound images with pre-operative MR images

In recent years intra-operative ultrasound images have been used for many procedures in neurosurgery. The registration of intra-operative ultrasound images with preoperative magnetic resonance images is still a challenging problem. In this study a new hybrid method based on residual complexity is proposed for this problem. A new two stages method based on the matching echogenic structures such as sulci is achieved by optimizing the residual complexity (RC) value with quantized coefficients between the ultrasound image and the probabilistic map of MR image. The proposed method is a compromise between feature-based and intensity-based approaches. The evaluation is performed on both a brain phantom and patient data set. The results of the phantom data set confirmed that the proposed method outperforms the accuracy of conventional RC by 39%. Also the mean of fiducial registration errors reached to 1.45, 1.94 mm when the method was applied on phantom and clinical data set, respectively. This hybrid method based on RC enables non-rigid multimodal image registration in a computational time compatible with clinical use as well as being accurate.

Registration of 3D fetal neurosonography and MRI

We propose a method for registration of 3D fetal brain ultrasound with a reconstructed magnetic resonance fetal brain volume. This method, for the first time, allows the alignment of models of the fetal brain built from magnetic resonance images with 3D fetal brain ultrasound, opening possibilities to develop new, prior information based image analysis methods for 3D fetal neurosonography. The reconstructed magnetic resonance volume is first segmented using a probabilistic atlas and a pseudo ultrasound image volume is simulated from the segmentation. This pseudo ultrasound image is then affinely aligned with clinical ultrasound fetal brain volumes using a robust block-matching approach that can deal with intensity artefacts and missing features in the ultrasound images. A qualitative and quantitative evaluation demonstrates good performance of the method for our application, in comparison with other tested approaches. The intensity average of 27 ultrasound images co-aligned with the pseudo ultrasound template shows good correlation with anatomy of the fetal brain as seen in the reconstructed magnetic resonance image.

Fast rigid registration of pre-operative magnetic resonance images to intra-operative ultrasound for neurosurgery based on high confidence gradient orientations

PURPOSE

We present a novel approach for the registration of pre-operative magnetic resonance images to intra-operative ultrasound images for the context of image-guided neurosurgery.

METHOD

Our technique relies on the maximization of gradient orientation alignment in a reduced set of high confidence locations of interest and allows for fast, accurate, and robust registration. Performance is compared with multiple state-of-the-art techniques including conventional intensity-based multi-modal registration strategies, as well as other context-specific approaches. All methods were evaluated on fourteen clinical neurosurgical cases with brain tumors, including low-grade and high-grade gliomas, from the publicly available MNI BITE dataset. Registration accuracy of each method is evaluated as the mean distance between homologous landmarks identified by two or three experts. We provide an analysis of the landmarks used and expose some of the limitations in validation brought forward by expert disagreement and uncertainty in identifying corresponding points.

RESULTS

The proposed approach yields a mean error of 2.57 mm across all cases (the smallest among all evaluated techniques). Additionally, it is the only evaluated technique that resolves all cases with a mean distance of less than 1 mm larger than the theoretical minimal mean distance when using a rigid transformation.

CONCLUSION

Finally, our proposed method provides reduced processing times with an average registration time of 0.76 s in a GPU-based implementation, thereby facilitating its integration into the operating room.

Towards realtime multimodal fusion for image-guided interventions using self-similarities

Image-guided interventions often rely on deformable multimodal registration to align pre-treatment and intra-operative scans. There are a number of requirements for automated image registration for this task, such as a robust similarity metric for scans of different modalities with different noise distributions and contrast, an efficient optimisation of the cost function to enable fast registration for this time-sensitive application, and an insensitive choice of registration parameters to avoid delays in practical clinical use. In this work, we build upon the concept of structural image representation for multi-modal similarity. Discriminative descriptors are densely extracted for the multi-modal scans based on the "self-similarity context". An efficient quantised representation is derived that enables very fast computation of point-wise distances between descriptors. A symmetric multi-scale discrete optimisation with diffusion reguIarisation is used to find smooth transformations. The method is evaluated for the registration of 3D ultrasound and MRI brain scans for neurosurgery and demonstrates a significantly reduced registration error (on average 2.1 mm) compared to commonly used similarity metrics and computation times of less than 30 seconds per 3D registration.

New prototype neuronavigation system based on preoperative imaging and intraoperative freehand ultrasound: system description and validation

PURPOSE

The aim of this report is to present IBIS (Interactive Brain Imaging System) NeuroNav, a new prototype neuronavigation system that has been developed in our research laboratory over the past decade that uses tracked intraoperative ultrasound to address surgical navigation issues related to brain shift. The unique feature of the system is its ability, when needed, to improve the initial patient-to-preoperative image alignment based on the intraoperative ultrasound data. Parts of IBIS Neuronav source code are now publicly available on-line.

METHODS

Four aspects of the system are characterized in this paper: the ultrasound probe calibration, the temporal calibration, the patient-to-image registration and the MRI-ultrasound registration. In order to characterize its real clinical precision and accuracy, the system was tested in a series of adult brain tumor cases.

RESULTS

Three metrics were computed to evaluate the precision and accuracy of the ultrasound calibration. 1) Reproducibility: 1.77 mm and 1.65 mm for the bottom corners of the ultrasound image, 2) point reconstruction precision 0.62-0.90 mm: and 3) point reconstruction accuracy: 0.49-0.74 mm. The temporal calibration error was estimated to be 0.82 ms. The mean fiducial registration error (FRE) of the homologous-point-based patient-to-MRI registration for our clinical data is 4.9 ± 1.1 mm. After the skin landmark-based registration, the mean misalignment between the ultrasound and MR images in the tumor region is 6.1 ± 3.4 mm.

CONCLUSIONS

The components and functionality of a new prototype system are described and its precision and accuracy evaluated. It was found to have an accuracy similar to other comparable systems in the literature.

Deformable ultrasound registration without reconstruction

Ultrasound (US) imaging is often proposed as an interoperative imaging modality. This use nearly always requires that the collected data be registered to preoperative data of another modality. Existing intensity-based registration approaches all begin by reconstructing a 3D US volume from the collected 2D slices. We propose to directly register the set of 2D slices to the preoperative images. We argue this has a number of advantages, including the omission of the potentially complex reconstruction step, greater adaptability of the similarity measures, and easier parallelization. We describe a system for performing this task and present results on phantom data that show that our slice based method consistently outperforms a reconstruction based method in both speed and accuracy.

Validation of vessel-based registration for correction of brain shift

The displacement and deformation of brain tissue is a major source of error in image-guided neurosurgery systems. We have designed and implemented a method to detect and correct brain shift using pre-operative MR images and intraoperative Doppler ultrasound data and present its validation with both real and simulated data. The algorithm uses segmented vessels from both modalities, and estimates the deformation using a modified version of the iterative closest point (ICP) algorithm. We use the least trimmed squares (LTS) to reduce the number of outliers in the point matching procedure. These points are used to drive a thin-plate spline transform to achieve non-linear registration. Validation was completed in two parts. First, the technique was tested and validated using realistic simulations where the results were compared to the known deformation. The registration technique recovered 75% of the deformation in the region of interest accounting for deformations as large as 20 mm. Second, we performed a PVA-cryogel phantom study where both MR and ultrasound images of the phantom were obtained for three different deformations. The registration results based on MR data were used as a gold standard to evaluate the performance of the ultrasound based registration. On average, deformations of 7.5 mm magnitude were corrected to within 1.6 mm for the ultrasound based registration and 1.07 mm for the MR based registration.

Strategies for brain shift evaluation

For the analysis of the brain shift phenomenon different strategies were applied. In 32 glioma cases pre- and intraoperative MR datasets were acquired in order to evaluate the maximum displacement of the brain surface and the deep tumor margin. After rigid registration using the software of the neuronavigation system, a direct comparison was made with 2D- and 3D visualizations. As a result, a great variability of the brain shift was observed ranging up to 24 mm for cortical displacement and exceeding 3 mm for the deep tumor margin in 66% of all cases. Following intraoperative imaging the neuronavigation system was updated in eight cases providing reliable guidance. For a more comprehensive analysis a voxel-based nonlinear registration was applied. Aiming at improved speed of alignment we performed all interpolation operations with 3D texture mapping based on OpenGL functions supported in graphics hardware. Further acceleration was achieved with an adaptive refinement of the underlying control point grid focusing on the main deformation areas. For a quick overview the registered datasets were evaluated with different 3D visualization approaches. Finally, the results were compared to the initial measurements contributing to a better understanding of the brain shift phenomenon. Overall, the experiments clearly demonstrate that deformations of the brain surface and deeper brain structures are uncorrelated.

Registration of real-time 3-D ultrasound images of the heart for novel 3-D stress echocardiography

Stress echocardiography is a routinely used clinical procedure to diagnose cardiac dysfunction by comparing wall motion information in prestress and poststress ultrasound images. Incomplete data, complicated imaging protocols and misaligned prestress and poststress views, however, are known limitations of conventional stress echocardiography. We discuss how the first two limitations are overcome via the use of real-time three-dimensional (3-D) ultrasound imaging, an emerging modality, and have called the new procedure "3-D stress echocardiography." We also show that the problem of misaligned views can be solved by registration of prestress and poststress 3-D image sequences. Such images are misaligned because of variations in placing the ultrasound transducer and stress-induced anatomical changes. We have developed a technique to temporally align 3-D images of the two sequences first and then to spatially register them to rectify probe placement error while preserving the stress-induced changes. The 3-D spatial registration is mutual information-based. Image registration used in conjunction with 3-D stress echocardiography can potentially improve the diagnostic accuracy of stress testing.