Slice-to-volume medical image registration: A survey

Deformable multimodal registration for navigation in beating-heart cardiac surgery

PURPOSE

Minimally invasive beating-heart surgery is currently performed using endoscopes and without navigation. Registration of intraoperative ultrasound to a preoperative cardiac CT scan is a valuable step toward image-guided navigation.

METHODS

The registration was achieved by first extracting a representative point set from each ultrasound image in the sequence using a deformable registration. A template shape representing the cardiac chambers was deformed through a hierarchy of affine transformations to match each ultrasound image using a generalized expectation maximization algorithm. These extracted point sets were matched to the CT by exhaustively searching over a large number of precomputed slices of 3D geometry. The result is a similarity transformation mapping the intraoperative ultrasound to preoperative CT.

RESULTS

Complete data sets were acquired for four patients. Transesophageal echocardiography ultrasound sequences were deformably registered to a model of oriented points with a mean error of 2.3 mm. Ultrasound and CT scans were registered to a mean of 3 mm, which is comparable to the error of 2.8 mm expected by merging ultrasound registration with uncertainty of cardiac CT.

CONCLUSION

The proposed algorithm registered 3D CT with dynamic 2D intraoperative imaging. The algorithm aligned the images in both space and time, needing neither dynamic CT imaging nor intraoperative electrocardiograms. The accuracy was sufficient for navigation in thoracoscopically guided beating-heart surgery.

An overview of deep learning in medical imaging focusing on MRI

What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI. Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of deep learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging.

Fully automated dual-resolution serial optical coherence tomography aimed at diffusion MRI validation in whole mouse brains

An automated dual-resolution serial optical coherence tomography (2R-SOCT) scanner is developed. The serial histology system combines a low-resolution ( 25    μ m / voxel ) 3 × OCT with a high-resolution ( 1.5    μ m / voxel ) 40 × OCT to acquire whole mouse brains at low resolution and to target specific regions of interest (ROIs) at high resolution. The 40 × ROIs positions are selected either manually by the microscope operator or using an automated ROI positioning selection algorithm. Additionally, a multimodal and multiresolution registration pipeline is developed in order to align the 2R-SOCT data onto diffusion MRI (dMRI) data acquired in the same ex vivo mouse brains prior to automated histology. Using this imaging system, 3 whole mouse brains are imaged, and 250 high-resolution 40 × three-dimensional ROIs are acquired. The capability of this system to perform multimodal imaging studies is demonstrated by labeling the ROIs using a mouse brain atlas and by categorizing the ROIs based on their associated dMRI measures. This reveals a good correspondence of the tissue microstructure imaged by the high-resolution OCT with various dMRI measures such as fractional anisotropy, number of fiber orientations, apparent fiber density, orientation dispersion, and intracellular volume fraction.

3-D Reconstruction in Canonical Co-Ordinate Space From Arbitrarily Oriented 2-D Images

Limited capture range, and the requirement to provide high quality initialization for optimization-based 2-D/3-D image registration methods, can significantly degrade the performance of 3-D image reconstruction and motion compensation pipelines. Challenging clinical imaging scenarios, which contain significant subject motion, such as fetal in-utero imaging, complicate the 3-D image and volume reconstruction process. In this paper, we present a learning-based image registration method capable of predicting 3-D rigid transformations of arbitrarily oriented 2-D image slices, with respect to a learned canonical atlas co-ordinate system. Only image slice intensity information is used to perform registration and canonical alignment, no spatial transform initialization is required. To find image transformations, we utilize a convolutional neural network architecture to learn the regression function capable of mapping 2-D image slices to a 3-D canonical atlas space. We extensively evaluate the effectiveness of our approach quantitatively on simulated magnetic resonance imaging (MRI), fetal brain imagery with synthetic motion and further demonstrate qualitative results on real fetal MRI data where our method is integrated into a full reconstruction and motion compensation pipeline. Our learning based registration achieves an average spatial prediction error of 7 mm on simulated data and produces qualitatively improved reconstructions for heavily moving fetuses with gestational ages of approximately 20 weeks. Our model provides a general and computationally efficient solution to the 2-D/3-D registration initialization problem and is suitable for real-time scenarios.

Fast and accurate Slicewise OutLIer Detection (SOLID) with informed model estimation for diffusion MRI data

The accurate characterization of the diffusion process in tissue using diffusion MRI is greatly challenged by the presence of artefacts. Subject motion causes not only spatial misalignments between diffusion weighted images, but often also slicewise signal intensity errors. Voxelwise robust model estimation is commonly used to exclude intensity errors as outliers. Slicewise outliers, however, become distributed over multiple adjacent slices after image registration and transformation. This challenges outlier detection with voxelwise procedures due to partial volume effects. Detecting the outlier slices before any transformations are applied to diffusion weighted images is therefore required. In this work, we present i) an automated tool coined SOLID for slicewise outlier detection prior to geometrical image transformation, and ii) a framework to naturally interpret data uncertainty information from SOLID and include it as such in model estimators. SOLID uses a straightforward intensity metric, is independent of the choice of the diffusion MRI model, and can handle datasets with a few or irregularly distributed gradient directions. The SOLID-informed estimation framework prevents the need to completely reject diffusion weighted images or individual voxel measurements by downweighting measurements with their degree of uncertainty, thereby supporting convergence and well-conditioning of iterative estimation algorithms. In comprehensive simulation experiments, SOLID detects outliers with a high sensitivity and specificity, and can achieve higher or at least similar sensitivity and specificity compared to other tools that are based on more complex and time-consuming procedures for the scenarios investigated. SOLID was further validated on data from 54 neonatal subjects which were visually inspected for outlier slices with the interactive tool developed as part of this study, showing its potential to quickly highlight problematic volumes and slices in large population studies. The informed model estimation framework was evaluated both in simulations and in vivo human data.

Nonrigid Image Registration Using Spatially Region-Weighted Correlation Ratio and GPU-Acceleration

OBJECTIVE

Nonrigid image registration with high accuracy and efficiency remains a challenging task for medical image analysis. In this paper, we present the spatially region-weighted correlation ratio (SRWCR) as a novel similarity measure to improve the registration performance.

METHODS

SRWCR is rigorously deduced from a three-dimension joint probability density function combining the intensity channels with an extra spatial information channel. SRWCR estimates the optimal functional dependence between the intensities for each spatial bin, in which the spatial distribution modeled by a cubic B-spline function is used to differentiate the contribution of voxels. We also analytically derive the gradient of SRWCR with respect to the transformation parameters and optimize it using a quasi-Newton approach. Furthermore, we propose a GPU-based parallel mechanism to accelerate the computation of SRWCR and its derivatives.

RESULTS

The experiments on synthetic images, public four-dimensional thoracic computed tomography (CT) dataset, retinal optical coherence tomography data, and clinical CT and positron emission tomography images confirm that SRWCR significantly outperforms some state-of-the-art techniques such as spatially encoded mutual information and Robust PaTch-based cOrrelation Ration.

CONCLUSION

This study demonstrates the advantages of SRWCR in tackling the practical difficulties due to distinct intensity changes, serious speckle noise, or different imaging modalities.

SIGNIFICANCE

The proposed registration framework might be more reliable to correct the nonrigid deformations and more potential for clinical applications.

Integrated fMRI Preprocessing Framework Using Extended Kalman Filter for Estimation of Slice-Wise Motion

Functional MRI acquisition is sensitive to subjects' motion that cannot be fully constrained. Therefore, signal corrections have to be applied a posteriori in order to mitigate the complex interactions between changing tissue localization and magnetic fields, gradients and readouts. To circumvent current preprocessing strategies limitations, we developed an integrated method that correct motion and spatial low-frequency intensity fluctuations at the level of each slice in order to better fit the acquisition processes. The registration of single or multiple simultaneously acquired slices is achieved online by an Iterated Extended Kalman Filter, favoring the robust estimation of continuous motion, while an intensity bias field is non-parametrically fitted. The proposed extraction of gray-matter BOLD activity from the acquisition space to an anatomical group template space, taking into account distortions, better preserves fine-scale patterns of activity. Importantly, the proposed unified framework generalizes to high-resolution multi-slice techniques. When tested on simulated and real data the latter shows a reduction of motion explained variance and signal variability when compared to the conventional preprocessing approach. These improvements provide more stable patterns of activity, facilitating investigation of cerebral information representation in healthy and/or clinical populations where motion is known to impact fine-scale data.

Automatic deformable registration of histological slides to μCT volume data

Localizing a histological section in the three-dimensional dataset of a different imaging modality is a challenging 2D-3D registration problem. In the literature, several approaches have been proposed to solve this problem; however, they cannot be considered as fully automatic. Recently, we developed an automatic algorithm that could successfully find the position of a histological section in a micro computed tomography (μCT) volume. For the majority of the datasets, the result of localization corresponded to the manual results. However, for some datasets, the matching μCT slice was off the ground-truth position. Furthermore, elastic distortions, due to histological preparation, could not be accounted for in this framework. In the current study, we introduce two optimization frameworks based on normalized mutual information, which enabled us to accurately register histology slides to volume data. The rigid approach allocated 81 % of histological sections with a median position error of 8.4 μm in jaw bone datasets, and the deformable approach improved registration by 33 μm with respect to the median distance error for four histological slides in the cerebellum dataset.

Volumetric reconstruction from printed films: Enabling 30 year longitudinal analysis in MR neuroimaging

Hospitals often hold historical MR image data printed on films without being able to make it accessible to modern image processing techniques. Having the possibility to recover geometrically consistent, volumetric images from scans acquired decades ago will enable more comprehensive, longitudinal studies to understand disease progressions. In this paper, we propose a consistent framework to reconstruct a volumetric representation from printed films holding thick single-slice brain MR image acquisitions dating back to the 1980's. We introduce a flexible framework based on semi-automatic slice extraction, followed by automated slice-to-volume registration with inter-slice transformation regularisation and slice intensity correction. Our algorithm is robust against numerous detrimental effects being present in archaic films. A subsequent, isotropic total variation deconvolution technique revitalises the visual appearance of the obtained volumes. We assess the accuracy and perform the validation of our reconstruction framework on a uniquely long-term MRI dataset where a ground-truth is available. This method will be used to facilitate a robust longitudinal analysis spanning 30 years of MRI scans.

Motion-Robust Spatially Constrained Parameter Estimation in Renal Diffusion-Weighted MRI by 3D Motion Tracking and Correction of Sequential Slices

In this work, we introduce a novel motion-robust spatially constrained parameter estimation (MOSCOPE) technique for kidney diffusion-weighted MRI. The proposed motion compensation technique does not require a navigator, trigger, or breath-hold but only uses the intrinsic features of the acquired data to track and compensate for motion to reconstruct precise models of the renal diffusion signal. We have developed a technique for physiological motion tracking based on robust state estimation and sequential registration of diffusion sensitized slices acquired within 200ms. This allows a sampling rate of 5Hz for state estimation in motion tracking that is sufficiently faster than both respiratory and cardiac motion rates in children and adults, which range between 0.8 to 0.2Hz, and 2.5 to 1Hz, respectively. We then apply the estimated motion parameters to data from each slice and use motion-compensated data for 1) robust intra-voxel incoherent motion (IVIM) model estimation in the kidney using a spatially constrained model fitting approach, and 2) robust weighted least squares estimation of the diffusion tensor model. Experimental results, including precision of IVIM model parameters using bootstrap-sampling and in-vivo whole kidney tractography, showed significant improvement in precision and accuracy of these models using the proposed method compared to models based on the original data and volumetric registration.