Validation of vessel-based registration for correction of brain shift

D2BGAN: Dual Discriminator Bayesian Generative Adversarial Network for Deformable MR-Ultrasound Registration Applied to Brain Shift Compensation

During neurosurgical procedures, the neuro-navigation system's accuracy is affected by the brain shift phenomenon. One popular strategy is to compensate for brain shift using intraoperative ultrasound (iUS) registration with pre-operative magnetic resonance (MR) scans. This requires a satisfactory multimodal image registration method, which is challenging due to the low image quality of ultrasound and the unpredictable nature of brain deformation during surgery. In this paper, we propose an automatic unsupervised end-to-end MR-iUS registration approach named the Dual Discriminator Bayesian Generative Adversarial Network (D2BGAN). The proposed network consists of two discriminators and a generator optimized by a Bayesian loss function to improve the functionality of the generator, and we add a mutual information loss function to the discriminator for similarity measurements. Extensive validation was performed on the RESECT and BITE datasets, where the mean target registration error (mTRE) of MR-iUS registration using D2BGAN was determined to be 0.75 ± 0.3 mm. The D2BGAN illustrated a clear advantage by achieving an 85% improvement in the mTRE over the initial error. Moreover, the results confirmed that the proposed Bayesian loss function, rather than the typical loss function, improved the accuracy of MR-iUS registration by 23%. The improvement in registration accuracy was further enhanced by the preservation of the intensity and anatomical information of the input images.

Image-to-Patient Registration in Computer-Assisted Surgery of Head and Neck: State-of-the-Art, Perspectives, and Challenges

Today, image-guided systems play a significant role in improving the outcome of diagnostic and therapeutic interventions. They provide crucial anatomical information during the procedure to decrease the size and the extent of the approach, to reduce intraoperative complications, and to increase accuracy, repeatability, and safety. Image-to-patient registration is the first step in image-guided procedures. It establishes a correspondence between the patient's preoperative imaging and the intraoperative data. When it comes to the head-and-neck region, the presence of many sensitive structures such as the central nervous system or the neurosensory organs requires a millimetric precision. This review allows evaluating the characteristics and the performances of different registration methods in the head-and-neck region used in the operation room from the perspectives of accuracy, invasiveness, and processing times. Our work led to the conclusion that invasive marker-based methods are still considered as the gold standard of image-to-patient registration. The surface-based methods are recommended for faster procedures and applied on the surface tissues especially around the eyes. In the near future, computer vision technology is expected to enhance these systems by reducing human errors and cognitive load in the operating room.

Photoacoustic-MR Image Registration Based on a Co-Sparse Analysis Model to Compensate for Brain Shift

Brain shift is an important obstacle to the application of image guidance during neurosurgical interventions. There has been a growing interest in intra-operative imaging to update the image-guided surgery systems. However, due to the innate limitations of the current imaging modalities, accurate brain shift compensation continues to be a challenging task. In this study, the application of intra-operative photoacoustic imaging and registration of the intra-operative photoacoustic with pre-operative MR images are proposed to compensate for brain deformation. Finding a satisfactory registration method is challenging due to the unpredictable nature of brain deformation. In this study, the co-sparse analysis model is proposed for photoacoustic-MR image registration, which can capture the interdependency of the two modalities. The proposed algorithm works based on the minimization of mapping transform via a pair of analysis operators that are learned by the alternating direction method of multipliers. The method was evaluated using an experimental phantom and ex vivo data obtained from a mouse brain. The results of the phantom data show about 63% improvement in target registration error in comparison with the commonly used normalized mutual information method. The results proved that intra-operative photoacoustic images could become a promising tool when the brain shift invalidates pre-operative MRI.

A functional ultrasound brain GPS for automatic vascular-based neuronavigation

Recent advances in ultrasound imaging triggered by transmission of ultrafast plane waves have rendered functional ultrasound (fUS) imaging a valuable neuroimaging modality capable of mapping cerebral vascular networks, but also for the indirect capture of neuronal activity with high sensitivity thanks to the neurovascular coupling. However, the expansion of fUS imaging is still limited by the difficulty to identify cerebral structures during experiments based solely on the Doppler images and the shape of the vessels. In order to tackle this challenge, this study introduces the vascular brain positioning system (BPS), a GPS of the brain. The BPS is a whole-brain neuronavigation system based on the on-the-fly automatic alignment of ultrafast ultrasensitive transcranial Power Doppler volumic images to common templates such as the Allen Mouse Brain Common Coordinates Framework. This method relies on the online registration of the complex cerebral vascular fingerprint of the studied animal to a pre-aligned reference vascular atlas, thus allowing rapid matching and identification of brain structures. We quantified the accuracy of the automatic registration using super-resolution vascular images obtained at the microscopic scale using Ultrasound Localization Microscopy and found a positioning error of 44 µm and 96 µm for intra-animal and inter-animal vascular registration, respectively. The proposed BPS approach outperforms the manual vascular landmark recognition performed by expert neuroscientists (inter-annotator errors of 215 µm and 259 µm). Using the online BPS approach coupled with the Allen Atlas, we demonstrated the capability of the system to position itself automatically over chosen anatomical structures and to obtain corresponding functional activation maps even in complex oblique planes. Finally, we show that the system can be used to acquire and estimate functional connectivity matrices automatically. The proposed functional ultrasound on-the-fly neuronavigation approach allows automatic brain navigation and could become a key asset to ensure standardized experiments and protocols for non-expert and expert researchers.

The role of diffusion tractography in refining glial tumor resection

Primary brain tumors are notoriously hard to resect surgically. Due to their infiltrative nature, finding the optimal resection boundary without damaging healthy tissue can be challenging. One potential tool to help make this decision is diffusion-weighted magnetic resonance imaging (dMRI) tractography. dMRI exploits the diffusion of water molecule along axons to generate a 3D modelization of the white matter bundles in the brain. This feature is particularly useful to visualize how a tumor affects its surrounding white matter and plan a surgical path. This paper reviews the different ways in which dMRI can be used to improve brain tumor resection, its benefits and also its limitations. We expose surgical tools that can be paired with dMRI to improve its impact on surgical outcome, such as loading the 3D tractography in the neuronavigation system and direct electrical stimulation to validate the position of the white matter bundles of interest. We also review articles validating dMRI findings using other anatomical investigation techniques, such as postmortem dissections, manganese-enhanced MRI, electrophysiological stimulations, and phantom studies with known ground truth. We will be discussing the areas of the brain where dMRI performs well and where the future challenges are. We will conclude this review with suggestions and take home messages for neurosurgeons, tractographers, and vendors for advancing the field and on how to benefit from tractography's use in clinical practice.

Co-Sparse Analysis Model Based Image Registration to Compensate Brain Shift by Using Intra-Operative Ultrasound Imaging

Notwithstanding the widespread use of image guided neurosurgery systems in recent years, the accuracy of these systems is strongly limited by the intra-operative deformation of the brain tissue, the so-called brain shift. Intra-operative ultrasound (iUS) imaging as an effective solution to compensate complex brain shift phenomena update patients coordinate during surgery by registration of the intra-operative ultrasound and the pre-operative MRI data that is a challenging problem.In this work a non-rigid multimodal image registration technique based on co-sparse analysis model is proposed. This model captures the interdependency of two image modalities; MRI as an intensity image and iUS as a depth image. Based on this model, the transformation between the two modalities is minimized by using a bimodal pair of analysis operators which are learned by optimizing a joint co-sparsity function using a conjugate gradient.Experimental validation of our algorithm confirms that our registration approach outperforms several of other state-of-the-art registration methods quantitatively. The evaluation was performed using seven patient dataset with the mean registration error of only 1.83 mm. Our intensity-based co-sparse analysis model has improved the accuracy of non-rigid multimodal medical image registration by 15.37% compared to the curvelet based residual complexity as a powerful registration method, in a computational time compatible with clinical use.

Advanced Ultrasound Imaging in Glioma Surgery: Beyond Gray-Scale B-mode

Introduction: Glioma surgery is aimed at obtaining maximal safe tumor resection while preserving or improving patient's neurological status. For this reason, there is growing interest for intra-operative imaging in neuro-oncological surgery. Intra-operative ultrasound (ioUS) provides the surgeon with real-time, anatomical and functional information. Despite this, in neurosurgery ioUS mainly relies only on gray-scale brightness mode (B-mode). Many other ultrasound imaging modalities, such as Fusion Imaging with pre-operative acquired magnetic resonance imaging (MRI), Doppler modes, Contrast Enhanced Ultrasound (CEUS), and elastosonography have been developed and have been extensively used in other organs. Although these modalities offer valuable real-time intra-operative information, so far their usage during neurosurgical procedures is still limited.

Purpose: To present an US-based multimodal approach for image-guidance in glioma surgery, highlighting the different features of advanced US modalities: fusion imaging with pre-operative acquired MRI for Virtual Navigation, B-mode, Doppler (power-, color-, spectral-), CEUS, and elastosonography.

Methods: We describe, in a step-by-step fashion, the applications of the most relevant advanced US modalities during different stages of surgery and their implications for surgical decision-making. Each US modality is illustrated from a technical standpoint and its application during glioma surgery is discussed.

Results: B-mode offers dynamic morphological information, which can be further implemented with fusion imaging to improve image understanding and orientation. Doppler imaging permits to evaluate anatomy and function of the vascular tree. CEUS allows to perform a real-time angiosonography, providing valuable information in regards of parenchyma and tumor vascularization and perfusion. This facilitates tumor detection and surgical strategy, also allowing to characterize tumor grade and to identify residual tumor. Elastosonography is a promising tool able to better define tumor margins, parenchymal infiltration, tumor consistency and permitting differentiation of high grade and low grade lesions.

Conclusions: Multimodal ioUS represents a valuable tool for glioma surgery being highly informative, rapid, repeatable, and real-time. It is able to differentiate low grade from high grade tumors and to provide the surgeon with relevant information for surgical decision-making. ioUS could be integrated with other intra-operative imaging and functional approaches in a synergistic manner to offer the best image guidance for each patient.

Brain shift in neuronavigation of brain tumors: A review

PURPOSE

Neuronavigation based on preoperative imaging data is a ubiquitous tool for image guidance in neurosurgery. However, it is rendered unreliable when brain shift invalidates the patient-to-image registration. Many investigators have tried to explain, quantify, and compensate for this phenomenon to allow extended use of neuronavigation systems for the duration of surgery. The purpose of this paper is to present an overview of the work that has been done investigating brain shift.

METHODS

A review of the literature dealing with the explanation, quantification and compensation of brain shift is presented. The review is based on a systematic search using relevant keywords and phrases in PubMed. The review is organized based on a developed taxonomy that classifies brain shift as occurring due to physical, surgical or biological factors.

RESULTS

This paper gives an overview of the work investigating, quantifying, and compensating for brain shift in neuronavigation while describing the successes, setbacks, and additional needs in the field. An analysis of the literature demonstrates a high variability in the methods used to quantify brain shift as well as a wide range in the measured magnitude of the brain shift, depending on the specifics of the intervention. The analysis indicates the need for additional research to be done in quantifying independent effects of brain shift in order for some of the state of the art compensation methods to become useful.

CONCLUSION

This review allows for a thorough understanding of the work investigating brain shift and introduces the needs for future avenues of investigation of the phenomenon.

Registration of Real-Time 3-D Ultrasound to Tomographic Images of the Abdominal Aorta

The purpose of this study was to develop an image-based method for registration of real-time 3-D ultrasound to computed tomography (CT) of the abdominal aorta, targeting future use in ultrasound-guided endovascular intervention. We proposed a method in which a surface model of the aortic wall was segmented from CT, and the approximate initial location of this model relative to the ultrasound volume was manually indicated. The model was iteratively transformed to automatically optimize correspondence to the ultrasound data. Feasibility was studied using data from a silicon phantom and in vivo data from a volunteer with previously acquired CT. Through visual evaluation, the ultrasound and CT data were seen to correspond well after registration. Both aortic lumen and branching arteries were well aligned. The processing was done offline, and the registration took approximately 0.2 s per ultrasound volume. The results encourage further patient studies to investigate accuracy, robustness and clinical value of the approach.

Curvelet based residual complexity objective function for non-rigid registration of pre-operative MRI with intra-operative ultrasound images

Intra-operative ultrasound as an imaging based method has been recognized as an effective solution to compensate non rigid brain shift problem in recent years. Measuring brain shift requires registration of the pre-operative MRI images with the intra-operative ultrasound images which is a challenging task. In this study a novel hybrid method based on the matching echogenic structures such as sulci and tumor boundary in MRI with ultrasound images is proposed. The matching echogenic structures are achieved by optimizing the Residual Complexity (RC) in the curvelet domain. At the first step, the probabilistic map of the MR image is achieved and the residual image as the difference between this probabilistic map and intra-operative ultrasound is obtained. Then curvelet transform as a sparse function is used to minimize the complexity of residual image. The proposed method is a compromise between feature-based and intensity-based approaches. Evaluation was performed using 14 patients data set and the mean of registration error reached to 1.87 mm. This hybrid method based on RC improves accuracy of nonrigid multimodal image registration by 12.5% in a computational time compatible with clinical use.

Marker-less tracking of brain surface deformations by non-rigid registration integrating surface and vessel/sulci features

PURPOSE

To compensate for brain shift in image-guided neurosurgery, we propose a new non-rigid registration method that integrates surface and vessel/sulci feature to noninvasively track the brain surface.

METHOD

Textured brain surfaces were acquired using phase-shift three-dimensional (3D) shape measurement, which offers 2D image pixels and their corresponding 3D points directly. Measured brain surfaces were noninvasively tracked using the proposed method by minimizing a new energy function, which is a weighted combination of 3D point corresponding estimation and surface deformation constraints. Initially, the measured surfaces were divided into featured and non-featured parts using a Frangi filter. The corresponding feature/non-feature points between intraoperative brain surfaces were estimated using the closest point algorithm. Subsequently, smoothness and rigidity constraints were introduced in the energy function for a smooth surface deformation and local surface detail conservation, respectively. Our 3D shape measurement accuracy was evaluated using 20 spheres for bias and precision errors. In addition, the proposed method was evaluated based on root mean square error (RMSE) and target registration error (TRE) with five porcine brains for which deformations were produced by gravity and pushing with different displacements in both the vertical and horizontal directions.

RESULTS

The minimum and maximum bias errors were 0.32 and 0.61 mm, respectively. The minimum and maximum precision errors were 0.025 and 0.30 mm, respectively. Quantitative validation with porcine brains showed that the average RMSE and TRE were 0.1 and 0.9 mm, respectively.

CONCLUSION

The proposed method appeared to be advantageous in integrating vessels/sulci feature, robust to changes in deformation magnitude and integrated feature numbers, and feasible in compensating for brain shift deformation in surgeries.

Superficial vessel reconstruction with a multiview camera system

We aim at reconstructing superficial vessels of the brain. Ultimately, they will serve to guide the deformation methods to compensate for the brain shift. A pipeline for three-dimensional (3-D) vessel reconstruction using three mono-complementary metal-oxide semiconductor cameras has been developed. Vessel centerlines are manually selected in the images. Using the properties of the Hessian matrix, the centerline points are assigned direction information. For correspondence matching, a combination of methods was used. The process starts with epipolar and spatial coherence constraints (geometrical constraints), followed by relaxation labeling and an iterative filtering where the 3-D points are compared to surfaces obtained using the thin-plate spline with decreasing relaxation parameter. Finally, the points are shifted to their local centroid position. Evaluation in virtual, phantom, and experimental images, including intraoperative data from patient experiments, shows that, with appropriate camera positions, the error estimates (root-mean square error and mean error) are [Formula: see text].

CustusX: an open-source research platform for image-guided therapy

Purpose

CustusX is an image-guided therapy (IGT) research platform dedicated to intraoperative navigation and ultrasound imaging. In this paper, we present CustusX as a robust, accurate, and extensible platform with full access to data and algorithms and show examples of application in technological and clinical IGT research.

Methods

CustusX has been developed continuously for more than 15 years based on requirements from clinical and technological researchers within the framework of a well-defined software quality process. The platform was designed as a layered architecture with plugins based on the CTK/OSGi framework, a superbuild that manages dependencies and features supporting the IGT workflow. We describe the use of the system in several different clinical settings and characterize major aspects of the system such as accuracy, frame rate, and latency.

Results

The validation experiments show a navigation system accuracy of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$<$$\end{document}<1.1 mm, a frame rate of 20 fps, and latency of 285 ms for a typical setup. The current platform is extensible, user-friendly and has a streamlined architecture and quality process. CustusX has successfully been used for IGT research in neurosurgery, laparoscopic surgery, vascular surgery, and bronchoscopy.

Conclusions

CustusX is now a mature research platform for intraoperative navigation and ultrasound imaging and is ready for use by the IGT research community. CustusX is open-source and freely available at http://www.custusx.org.

A hybrid method for non-rigid registration of intra-operative ultrasound images with pre-operative MR images

In recent years intra-operative ultrasound images have been used for many procedures in neurosurgery. The registration of intra-operative ultrasound images with preoperative magnetic resonance images is still a challenging problem. In this study a new hybrid method based on residual complexity is proposed for this problem. A new two stages method based on the matching echogenic structures such as sulci is achieved by optimizing the residual complexity (RC) value with quantized coefficients between the ultrasound image and the probabilistic map of MR image. The proposed method is a compromise between feature-based and intensity-based approaches. The evaluation is performed on both a brain phantom and patient data set. The results of the phantom data set confirmed that the proposed method outperforms the accuracy of conventional RC by 39%. Also the mean of fiducial registration errors reached to 1.45, 1.94 mm when the method was applied on phantom and clinical data set, respectively. This hybrid method based on RC enables non-rigid multimodal image registration in a computational time compatible with clinical use as well as being accurate.

Volumetric Measurements of Brain Shift Using Intraoperative Cone-Beam Computed Tomography: Preliminary Study

BACKGROUND

Cerebrospinal fluid leakage and ventricular compression during open surgery may lead to brain deformation called brain shift. Brain shift may affect intraoperative navigation that is based on image-based preoperative planning. Tools to correct or predict these anatomic modifications can be important to maintain precision during open guided neurosurgery.

OBJECTIVE

To obtain a reliable intraoperative volumetric deformation vector field describing brain shift during intracranial neurosurgical procedures.

METHODS

We acquired preoperative and intraoperative cone-beam computed tomography enhanced with intravenous injection of iodine contrast. These data sets were preprocessed and elastically registered to obtain the volumetric brain shift deformation vector fields.

RESULTS

We obtained the brain shift deformation vector field in 9 cases. The deformation fields proved to be highly nonlinear, particularly around the ventricles. Interpatient variability was considerable, with a maximum deformation ranging from 8.1 to 26.6 mm and a standard deviation ranging from 0.9 to 4.9 mm.

CONCLUSION

Contrast-enhanced cone-beam computed tomography provides a feasible technique for intraoperatively determining brain shift deformation vector fields. This technique can be used perioperatively to adjust preoperative planning and coregistration during neurosurgical procedures.

Automatic deformable MR-ultrasound registration for image-guided neurosurgery

In this work, we present a novel algorithm for registration of 3-D volumetric ultrasound (US) and MR using Robust PaTch-based cOrrelation Ratio (RaPTOR). RaPTOR computes local correlation ratio (CR) values on small patches and adds the CR values to form a global cost function. It is therefore invariant to large amounts of spatial intensity inhomogeneity. We also propose a novel outlier suppression technique based on the orientations of the RaPTOR gradients. Our deformation is modeled with free-form cubic B-splines. We analytically derive the derivatives of RaPTOR with respect to the transformation, i.e., the displacement of the B-spline nodes, and optimize RaPTOR using a stochastic gradient descent approach. RaPTOR is validated on MR and tracked US images of neurosurgery. Deformable registration of the US and MR images acquired, respectively, preoperation and postresection is of significant clinical significance, but challenging due to, among others, the large amount of missing correspondences between the two images. This work is also novel in that it performs automatic registration of this challenging dataset. To validate the results, we manually locate corresponding anatomical landmarks in the US and MR images of tumor resection in brain surgery. Compared to rigid registration based on the tracking system alone, RaPTOR reduces the mean initial mTRE over 13 patients from 5.9 to 2.9 mm, and the maximum initial TRE from 17.0 to 5.9 mm. Each volumetric registration using RaPTOR takes about 30 sec on a single CPU core. An important challenge in the field of medical image analysis is the shortage of publicly available dataset, which can both facilitate the advancement of new algorithms to clinical settings and provide a benchmark for comparison. To address this problem, we will make our manually located landmarks available online.

Joint intensity-and-point based registration of free-hand B-mode ultrasound and MRI of the carotid artery

PURPOSE

To introduce a semiautomatic algorithm to perform the registration of free-hand B-Mode ultrasound (US) and magnetic resonance imaging (MRI) of the carotid artery.

METHODS

The authors' approach combines geometrical features and intensity information. The only user interaction consists of placing three seed points in US and MRI. First, the lumen centerlines are used as landmarks for point based registration. Subsequently, in a joint optimization the distance between centerlines and the dissimilarity of the image intensities is minimized. Evaluation is performed in left and right carotids from six healthy volunteers and five patients with atherosclerosis. For the validation, the authors measure the Dice similarity coefficient (DSC) and the mean surface distance (MSD) between carotid lumen segmentations in US and MRI after registration. The effect of several design parameters on the registration accuracy is investigated by an exhaustive search on a training set of five volunteers and three patients. The optimum configuration is validated on the remaining images of one volunteer and two patients.

RESULTS

On the training set, the authors achieve an average DSC of 0.74 and a MSD of 0.66 mm on volunteer data. For the patient data, the authors obtain a DSC of 0.77 and a MSD of 0.69 mm. In the independent set composed of patient and volunteer data, the DSC is 0.69 and the MSD is 0.87 mm. The experiments with different design parameters show that nonrigid registration outperforms rigid registration, and that the combination of intensity and point information is superior to approaches that use intensity or points only.

CONCLUSIONS

The proposed method achieves an accurate registration of US and MRI, and may thus enable multimodal analysis of the carotid plaque.

Aligning 3D time-of-flight MRA datasets for quantitative longitudinal studies: evaluation of rigid registration techniques

OBJECTIVE

3D Time-of-flight (TOF) magnetic resonance angiography is commonly used for vascular analyses. A quantification of longitudinal morphological changes usually requires the registration of TOF image sequences acquired at different time points. The aim of this study was to evaluate the precision of different 3D rigid registration setups such that an optimal quantification of morphological changes can be achieved.

METHODS

Eight different rigid registration techniques were implemented and evaluated in this study using the target registration error (TRE) calculated based on 554 landmarks defined in twenty TOF datasets. The registration techniques differed in integration of brain and vessel segmentation masks and usage of a multi-resolution framework. Furthermore, the benefit of a prior volume-of-interest definition for registration accuracy was evaluated.

RESULTS

The results revealed that the highest registration accuracies can be achieved using a multi-resolution framework and a cerebrovascular segmentation as mask. Numerically, a mean TRE of 1.1mm was calculated. If applicable, a prior definition of a volume-of-interest allows a reduction of the TRE to only 0.6mm.

CONCLUSION

TOF datasets should be registered using vessel segmentations as mask, multi-resolution framework and previous volume-of-interest definition if possible to obtain the highest registration precision. This is especially the case for longitudinal datasets that are separated by several months while the registration technique seems less important for datasets that are only separated by a few days.

Intra-operative correction of brain-shift

BACKGROUND

Brain-shift is a major source of error in neuronavigation systems based on pre-operative images. In this paper, we present intra-operative correction of brain-shift using 3D ultrasound.

METHODS

The method is based on image registration of vessels extracted from pre-operative MRA and intra-operative power Doppler-based ultrasound and is fully integrated in the neuronavigation software.

RESULTS

We have performed correction of brain-shift in the operating room during surgery and provided the surgeon with updated information. Here, we present data from seven clinical cases with qualitative and quantitative error measures.

CONCLUSION

The registration algorithm is fast enough to provide the surgeon with updated information within minutes and accounts for large portions of the experienced shift. Correction of brain-shift can make pre-operative data like fMRI and DTI reliable for a longer period of time and increase the usefulness of the MR data as a supplement to intra-operative 3D ultrasound in terms of overview and interpretation.

Liver deformation in an animal model due to pneumoperitoneum assessed by a vessel-based deformable registration

PURPOSE

Surgical navigation based on preoperative images partly overcomes some of the drawbacks of minimally invasive interventions - reduction of free sight, lack of dexterity and tactile feedback. The usefulness of preoperative images is limited in laparoscopic liver surgery, as the liver shifts due to respiration, induction of pneumoperitoneum and surgical manipulation. In this study, we evaluated the shift and deformation in an animal liver caused by respiration and pneumopertioneum using intraoperative cone beam CT.

MATERIAL AND METHODS

3D cone beam CT scans were acquired with arterial contrast. The centerlines of the segmented vessels were extracted from the images taken at different respiration and pressure settings. A non-rigid registration method was used to measure the shift and deformation. The mean Euclidean distance between the annotated landmarks was used for evaluation.

RESULTS

A shift and deformation of 44.6 mm on average was introduced due to the combined effect of respiration and pneumoperitoneum. On average 91% of the deformations caused by the respiration and pneumoperitoneum were recovered.

CONCLUSION

The results can contribute to the use of intraoperative imaging to correct for anatomic shift so that preoperative data can be used with greater confidence and accuracy during guidance of laparoscopic liver procedures.

Deformable medical image registration: a survey

Deformable image registration is a fundamental task in medical image processing. Among its most important applications, one may cite: 1) multi-modality fusion, where information acquired by different imaging devices or protocols is fused to facilitate diagnosis and treatment planning; 2) longitudinal studies, where temporal structural or anatomical changes are investigated; and 3) population modeling and statistical atlases used to study normal anatomical variability. In this paper, we attempt to give an overview of deformable registration methods, putting emphasis on the most recent advances in the domain. Additional emphasis has been given to techniques applied to medical images. In order to study image registration methods in depth, their main components are identified and studied independently. The most recent techniques are presented in a systematic fashion. The contribution of this paper is to provide an extensive account of registration techniques in a systematic manner.

Two-stage point-based registration method between ultrasound and CT imaging of the liver based on ICP and unscented Kalman filter: a phantom study

PURPOSE

In recent years, image-guided liver surgery based on intraoperative ultrasound (US) imaging has become common. Using an efficient point-based registration method to improve both accuracy and computational time for the registration of predeformation computer tomography, liver images with postdeformation US images are important during surgical procedure. Although iterative closest point (ICP) algorithm is widely used in surface-based registration, its performance strongly depends on the presence of noise and initial alignment. A registration technique based on unscented Kalman filter (UKF), which has been proposed recently, can used to overcome the noise and outliers on an incremental basis; however, the technique is associated with computational complexity.

METHODS

To overcome the limitations of ICP and UKF algorithms, we proposed an incremental two-stage registration method based on the combination of ICP and UKF algorithms to update the registration process with the acquired new points from US images. The registration is based on both the vessels and surface information of the liver.

RESULTS

The two-stage method was examined using numerical simulations and phantom data sets. The results of the phantom data set confirmed that the two-stage method outperforms the accuracy of ICP by 23% and reduces the running time of UKF by 60%.

CONCLUSION

The convergence rate, computational speed, and accuracy of the UKF algorithm can be improved using the two-stage method.

Vessel bifurcation localization based on intraoperative three-dimensional ultrasound and catheter path for image-guided catheter intervention of oral cancers

We present a method to localize intraoperative target vessel bifurcations under bones for ultrasound (US) image-guided catheter interventions. A catheter path is recorded to acquire skeletons for the target vessel bifurcations that cannot be imaged by intraoperative US. The catheter path is combined with the centerlines of the three-dimensional (3D) US image to construct a preliminary skeleton. Based on the preliminary skeleton, the orientations of target vessels are determined by registration with the preoperative image and the bifurcations were localized by computing the vessel length. An accurate intraoperative vessel skeleton is obtained for correcting the preoperative image to compensate for vessel deformation. A reality check of the proposed method was performed in a phantom experiment. Reasonable results were obtained. The in vivo experiment verified the clinical workflow of the proposed method in an in vivo environment. The accuracy of the centerline length of the vessel for localizing the target artery bifurcation was 2.4mm. These results suggest that the proposed method can allow the catheter tip to stop at the target artery bifurcations and enter into the target arteries. This method can be applied for virtual reality-enhanced image-guided catheter intervention of oral cancers.

Analysis of carotid artery deformation in different head and neck positions for maxillofacial catheter navigation in advanced oral cancer treatment

Background

To improve the accuracy of catheter navigation, it is important to develop a method to predict shifts of carotid artery (CA) bifurcations caused by intraoperative deformation. An important factor affecting the accuracy of electromagnetic maxillofacial catheter navigation systems is CA deformations. We aimed to assess CA deformation in different head and neck positions.

Methods

Using two sets of computed tomography angiography (CTA) images of six patients, displacements of the skull (maxillofacial segments), C1–C4 cervical vertebrae, mandible (mandibular segment), and CA along with its branches were analyzed. Segmented rigid bones around CA were considered the main causes of CA deformation. After superimposition of maxillofacial segments, C1–C4 and mandible segments were superimposed separately for displacement measurements. Five bifurcation points (vA–vE) were assessed after extracting the CA centerline. A new standardized coordinate system, regardless of patient-specific scanning positions, was employed. It was created using the principal axes of inertia of the maxillofacial bone segments of patients. Position and orientation parameters were transferred to this coordinate system. CA deformation in different head and neck positions was assessed.

Results

Absolute shifts in the center of gravity in the bone models for different segments were C1, 1.02 ± 0.9; C2, 2.18 ± 1.81; C3, 4.25 ± 3.85; C4, 5.90 ± 5.14; and mandible, 1.75 ± 2.76 mm. Shifts of CA bifurcations were vA, 5.52 ± 4.12; vB, 4.02 ± 3.27; vC, 4.39 ± 2.42; vD, 4.48 ± 1.88; and vE, 2.47 ± 1.32. Displacements, position changes, and orientation changes of C1–C4 segments as well as the displacements of all CA bifurcation points were similar in individual patients.

Conclusions

CA deformation was objectively proven as an important factor contributing to errors in maxillofacial navigation. Our study results suggest that small movements of the bones around CA can result in small CA deformations. Although patients’ faces were not fixed properly during CT scanning, C1–C4 and vA–vE displacements were similar in individual patients. We proposed a novel method for accumulation of the displacement data, and this study indicated the importance of surrounding bone displacements in predicting CA bifurcation.

Validation of a hybrid Doppler ultrasound vessel-based registration algorithm for neurosurgery

PURPOSE

We describe and validate a novel hybrid nonlinear vessel registration algorithm for intra-operative updating of preoperative magnetic resonance (MR) images using Doppler ultrasound (US) images acquired on the dura for the correction of brain-shift and registration inaccuracies. We also introduce an US vessel appearance simulator that generates vessel images similar in appearance to that acquired with US from MR angiography data.

METHODS

Our registration uses the minimum amount of preprocessing to extract vessels from the raw volumetric images. This prevents the removal of important registration information and minimizes the introduction of artifacts that may affect robustness, while reducing the amount of extraneous information in the image to be processed, thus improving the convergence speed of the algorithm. We then completed 3 rounds of validation for our vessel registration method for robustness and accuracy using (i) a large number of synthetic trials generated with our US vessel simulator, (ii) US images acquired from a real physical phantom made from polyvinyl alcohol cryogel, and (iii) real clinical data gathered intra-operatively from 3 patients.

RESULTS

Resulting target registration errors (TRE) of less than 2.5 mm are achieved in more than 90 % of the synthetic trials when the initial TREs are less than 20 mm. TREs of less than 2 mm were achieved when the technique was applied to the physical phantom, and TREs of less than 3 mm were achieved on clinical data.

CONCLUSIONS

These test trials show that the proposed algorithm is not only accurate but also highly robust to noise and missing vessel segments when working with US images acquired in a wide range of real-world conditions.

3D Rigid Registration of Intraoperative Ultrasound and Preoperative MR Brain Images Based on Hyperechogenic Structures

The registration of intraoperative ultrasound (US) images with preoperative magnetic resonance (MR) images is a challenging problem due to the difference of information contained in each image modality. To overcome this difficulty, we introduce a new probabilistic function based on the matching of cerebral hyperechogenic structures. In brain imaging, these structures are the liquid interfaces such as the cerebral falx and the sulci, and the lesions when the corresponding tissue is hyperechogenic. The registration procedure is achieved by maximizing the joint probability for a voxel to be included in hyperechogenic structures in both modalities. Experiments were carried out on real datasets acquired during neurosurgical procedures. The proposed validation framework is based on (i) visual assessment, (ii) manual expert estimations , and (iii) a robustness study. Results show that the proposed method (i) is visually efficient, (ii) produces no statistically different registration accuracy compared to manual-based expert registration, and (iii) converges robustly. Finally, the computation time required by our method is compatible with intraoperative use.

On the performance of improved ICP algorithms for registration of intra-ultrasound with pre-MR images; a phantom study

Ultrasound imaging as a simple and being real time has been found very applicable for intra-operative updates of pre-operative MRI data in image guided neurosurgery system. The main challenge here is the presence of speckle noise which influences the accuracy of registration of US-MR images, intra-operatively. In this paper the performance of two improved versions of the well known Iterative Closest Point (ICP) algorithms to deal with noise and outliers are considered and compared with conventional ICP method. To perform this study in a condition close to real clinical setting, a PVA-C brain phantom is made. As the results show improved versions of ICP are found more robust and precise than ICP algorithms in the presence of noise and outliers. Then the effect of various de-noising methods including diffusion filters on the accuracy of point-based registration is evaluated. The role of a proper diffusion filter for de-noising of US images has also improved the performance of the ICP algorithm and its variants about 35% and 20%, respectively.

An efficient point based registration of intra-operative ultrasound images with MR images for computation of brain shift; a phantom study

Intra-operative brain deformation (brain shift) limits the accuracy of image-guided neuro-surgery systems. Ultrasound imaging as a simple, fast and being real time has become an alternative to MR imaging which is an expensive system for brain shift calculation. The main challenges due to speckle noise and artifacts in US images, is to perform an accurate and fast registration of Us images with pre-operative MR images. In this paper an efficient point based registration method based on the alignment of probability density functions called Coherent Point Drift (CPD) is implemented and compared to the conventional ICP method. To perform this, a brain phantom that allows simulating the brain deformation is made. As the results of our phantom study confirm the CPD method clearly outperforms the ICP algorithm for brain shift calculation. Also the result proves that using intra-operative US has led to recover almost 80% of displacement in the region of interest.

Image-guided interventions: technology review and clinical applications

Image-guided interventions are medical procedures that use computer-based systems to provide virtual image overlays to help the physician precisely visualize and target the surgical site. This field has been greatly expanded by the advances in medical imaging and computing power over the past 20 years. This review begins with a historical overview and then describes the component technologies of tracking, registration, visualization, and software. Clinical applications in neurosurgery, orthopedics, and the cardiac and thoracoabdominal areas are discussed, together with a description of an evolving technology named Natural Orifice Transluminal Endoscopic Surgery (NOTES). As the trend toward minimally invasive procedures continues, image-guided interventions will play an important role in enabling new procedures, while improving the accuracy and success of existing approaches. Despite this promise, the role of image-guided systems must be validated by clinical trials facilitated by partnerships between scientists and physicians if this field is to reach its full potential.

Rapid dynamic image registration of the beating heart for diagnosis and surgical navigation

Dynamic cardiac magnetic resonance imaging (MR) and computed tomography (CT) provide cardiologists and cardiac surgeons with high-quality 4-D images for diagnosis and therapy, yet the effective use of these high-quality anatomical models remains a challenge. Ultrasound (US) is a flexible imaging tool, but the US images produced are often difficult to interpret unless they are placed within their proper 3-D anatomical context. The ability to correlate real-time 3-D US volumes (RT3D US) with dynamic MR/CT images would offer a significant contribution to improve the quality of cardiac procedures. In this paper, we present a rapid two-step method for registering RT3D US to high-quality dynamic 3-D MR/CT images of the beating heart. This technique overcomes some major limitations of image registration (such as the correct registration result not necessarily occurring at the maximum of the mutual information (MI) metric) using the MI metric. We demonstrate the effectiveness of our method in a dynamic heart phantom (DHP) study and a human subject study. The achieved mean target registration error of CT+US images in the phantom study is 2.59 mm. Validation using human MR/US volumes shows a target registration error of 1.76 mm. We anticipate that this technique will substantially improve the quality of cardiac diagnosis and therapies.

Echocardiography to magnetic resonance image registration for use in image-guided cardiac catheterization procedures

We present a robust method to register three-dimensional echocardiography (echo) images to magnetic resonance images (MRI) based on anatomical features, which is designed to be used in the registration pipeline for overlaying MRI-derived roadmaps onto two-dimensional live x-ray images during cardiac catheterization procedures. The features used in image registration are the endocardial surface of the left ventricle and the centre line of the descending aorta. The MR-derived left ventricle surface is generated using a fully automated algorithm, and the echo-derived left ventricle surface is produced using a semi-automatic segmentation method provided by the QLab software (Philips Healthcare) that it is routinely used in clinical practice. We test our method on data from six volunteers and four patients. We validated registration accuracy using two methods: the first calculated a root mean square distance error using expert identified anatomical landmarks, and the second method used catheters as landmarks in two clinical electrophysiology procedures. Results show a mean error of 4.1 mm, which is acceptable for our clinical application, and no failed registrations were observed. In addition, our algorithm works on clinical data, is fast and only requires a small amount of manual input, and so it is applicable for use during cardiac catheterization procedures.

Infrared thermal imaging: a review of the literature and case report

Intraoperative Thermal Imaging (ITI) is a novel neuroimaging technique that can potentially locate the margins of primary and metastatic brain tumors. As a result, the additional real-time anatomical and pathophysiological information may significantly contribute to an improved extent of tumor resection. Our objectives in this article are i) to briefly discuss the current status of intraoperative imaging modalities including ITI and ii) to present a case report that evaluates the usefulness of ITI in detection of brain tumor and its margins. In this case report, ITI was used in a patient with a metastatic intracortical melanoma. The thermal profile of the tumor and surrounding normal cerebral cortex were mapped with a ThermaCAM P60 (TCP60) infrared camera by FLIR Systems. The data obtained by TCP60, intra-operatively, revealed a clear demarcation of tumor with significant temperature differences, up to 3.3 degrees C, between the tumor core (36.4 degrees C) and the surrounding normal tissue (33.1 degrees C). Ultrasound and pre-resection MR and CT confirmed the position and size of the metastasis. The volume of the tumor was preoperatively calculated using the CyberKnife software and postoperative volumetric measurement of the tumor residual was calculated by the Gamma Knife software. Our result, along with previously published results of others, suggests that thermal imaging could be used to provide a rapid, non-invasive, and real-time intra-operative imaging.