Hierarchical estimation of a dense deformation field for 3-D robust registration

Medical image registration via neural fields

Image registration is an essential step in many medical image analysis tasks. Traditional methods for image registration are primarily optimization-driven, finding the optimal deformations that maximize the similarity between two images. Recent learning-based methods, trained to directly predict transformations between two images, run much faster, but suffer from performance deficiencies due to domain shift. Here we present a new neural network based image registration framework, called NIR (Neural Image Registration), which is based on optimization but utilizes deep neural networks to model deformations between image pairs. NIR represents the transformation between two images with a continuous function implemented via neural fields, receiving a 3D coordinate as input and outputting the corresponding deformation vector. NIR provides two ways of generating deformation field: directly output a displacement vector field for general deformable registration, or output a velocity vector field and integrate the velocity field to derive the deformation field for diffeomorphic image registration. The optimal registration is discovered by updating the parameters of the neural field via stochastic mini-batch gradient descent. We describe several design choices that facilitate model optimization, including coordinate encoding, sinusoidal activation, coordinate sampling, and intensity sampling. NIR is evaluated on two 3D MR brain scan datasets, demonstrating highly competitive performance in terms of both registration accuracy and regularity. Compared to traditional optimization-based methods, our approach achieves better results in shorter computation times. In addition, our methods exhibit performance on a cross-dataset registration task, compared to the pre-trained learning-based methods.

Anatomy-aware and acquisition-agnostic joint registration with SynthMorph

Affine image registration is a cornerstone of medical-image analysis. While classical algorithms can achieve excellent accuracy, they solve a time-consuming optimization for every image pair. Deep-learning (DL) methods learn a function that maps an image pair to an output transform. Evaluating the function is fast, but capturing large transforms can be challenging, and networks tend to struggle if a test-image characteristic shifts from the training domain, such as the resolution. Most affine methods are agnostic to the anatomy the user wishes to align, meaning the registration will be inaccurate if algorithms consider all structures in the image. We address these shortcomings with SynthMorph, a fast, symmetric, diffeomorphic, and easy-to-use DL tool for joint affine-deformable registration of any brain image without preprocessing. First, we leverage a strategy that trains networks with widely varying images synthesized from label maps, yielding robust performance across acquisition specifics unseen at training. Second, we optimize the spatial overlap of select anatomical labels. This enables networks to distinguish anatomy of interest from irrelevant structures, removing the need for preprocessing that excludes content which would impinge on anatomy-specific registration. Third, we combine the affine model with a deformable hypernetwork that lets users choose the optimal deformation-field regularity for their specific data, at registration time, in a fraction of the time required by classical methods. This framework is applicable to learning anatomy-aware, acquisition-agnostic registration of any anatomy with any architecture, as long as label maps are available for training. We analyze how competing architectures learn affine transforms and compare state-of-the-art registration tools across an extremely diverse set of neuroimaging data, aiming to truly capture the behavior of methods in the real world. SynthMorph demonstrates high accuracy and is available at https://w3id.org/synthmorph, as a single complete end-to-end solution for registration of brain magnetic resonance imaging (MRI) data.

Collocation for Diffeomorphic Deformations in Medical Image Registration

Diffeomorphic deformation is a popular choice in medical image registration. A fundamental property of diffeomorphisms is invertibility, implying that once the relation between two points A to B is found, then the relation B to A is given per definition. Consistency is a measure of a numerical algorithm's ability to mimic this invertibility, and achieving consistency has proven to be a challenge for many state-of-the-art algorithms. We present CDD (Collocation for Diffeomorphic Deformations), a numerical solution to diffeomorphic image registration, which solves for the Stationary Velocity Field (SVF) using an implicit A-stable collocation method. CDD guarantees the preservation of the diffeomorphic properties at all discrete points and is thereby consistent to machine precision. We compared CDD's collocation method with the following standard methods: Scaling and Squaring, Forward Euler, and Runge-Kutta 4, and found that CDD is up to 9 orders of magnitude more consistent. Finally, we evaluated CDD on a number of standard bench-mark data sets and compared the results with current state-of-the-art methods: SPM-DARTEL, Diffeomorphic Demons and SyN. We found that CDD outperforms state-of-the-art methods in consistency and delivers comparable or superior registration precision.

Graphics Processing Unit-Accelerated Nonrigid Registration of MR Images to CT Images During CT-Guided Percutaneous Liver Tumor Ablations

RATIONALE AND OBJECTIVES

Accuracy and speed are essential for the intraprocedural nonrigid magnetic resonance (MR) to computed tomography (CT) image registration in the assessment of tumor margins during CT-guided liver tumor ablations. Although both accuracy and speed can be improved by limiting the registration to a region of interest (ROI), manual contouring of the ROI prolongs the registration process substantially. To achieve accurate and fast registration without the use of an ROI, we combined a nonrigid registration technique on the basis of volume subdivision with hardware acceleration using a graphics processing unit (GPU). We compared the registration accuracy and processing time of GPU-accelerated volume subdivision-based nonrigid registration technique to the conventional nonrigid B-spline registration technique.

MATERIALS AND METHODS

Fourteen image data sets of preprocedural MR and intraprocedural CT images for percutaneous CT-guided liver tumor ablations were obtained. Each set of images was registered using the GPU-accelerated volume subdivision technique and the B-spline technique. Manual contouring of ROI was used only for the B-spline technique. Registration accuracies (Dice similarity coefficient [DSC] and 95% Hausdorff distance [HD]) and total processing time including contouring of ROIs and computation were compared using a paired Student t test.

RESULTS

Accuracies of the GPU-accelerated registrations and B-spline registrations, respectively, were 88.3 ± 3.7% versus 89.3 ± 4.9% (P = .41) for DSC and 13.1 ± 5.2 versus 11.4 ± 6.3 mm (P = .15) for HD. Total processing time of the GPU-accelerated registration and B-spline registration techniques was 88 ± 14 versus 557 ± 116 seconds (P < .000000002), respectively; there was no significant difference in computation time despite the difference in the complexity of the algorithms (P = .71).

CONCLUSIONS

The GPU-accelerated volume subdivision technique was as accurate as the B-spline technique and required significantly less processing time. The GPU-accelerated volume subdivision technique may enable the implementation of nonrigid registration into routine clinical practice.

Deformable medical image registration: a survey

Deformable image registration is a fundamental task in medical image processing. Among its most important applications, one may cite: 1) multi-modality fusion, where information acquired by different imaging devices or protocols is fused to facilitate diagnosis and treatment planning; 2) longitudinal studies, where temporal structural or anatomical changes are investigated; and 3) population modeling and statistical atlases used to study normal anatomical variability. In this paper, we attempt to give an overview of deformable registration methods, putting emphasis on the most recent advances in the domain. Additional emphasis has been given to techniques applied to medical images. In order to study image registration methods in depth, their main components are identified and studied independently. The most recent techniques are presented in a systematic fashion. The contribution of this paper is to provide an extensive account of registration techniques in a systematic manner.

Medical image registration: a review

This paper presents a review of automated image registration methodologies that have been used in the medical field. The aim of this paper is to be an introduction to the field, provide knowledge on the work that has been developed and to be a suitable reference for those who are looking for registration methods for a specific application. The registration methodologies under review are classified into intensity or feature based. The main steps of these methodologies, the common geometric transformations, the similarity measures and accuracy assessment techniques are introduced and described.

Symmetric multi-scale image registration

It is known that the transformations estimated in most of the non-rigid image registration methods are not invertible. This means that the transformation is not one-to-one and also it defines the deformation only in one direction making the registration methods become inconsistent. The symmetric image registration method is a solution to constrain the estimated transformation to become more consistent. This technique promotes the quality of registration significantly. In this paper we present symmetric non-rigid image registration method using multi-scale approach, defining the registration progressively in four different image scales which used result of the previous level as the initial condition for the next level. Our proposed algorithm uses a symmetric cost function which includes forward and backward transformations. Regularization term is used for both forward and reverse deformation fields to control the spatial behavior of the transformation. The results show improving the image similarity and consistency compared to asymmetric methods.

Highly accurate inverse consistent registration: a robust approach

The registration of images is a task that is at the core of many applications in computer vision. In computational neuroimaging where the automated segmentation of brain structures is frequently used to quantify change, a highly accurate registration is necessary for motion correction of images taken in the same session, or across time in longitudinal studies where changes in the images can be expected. This paper, inspired by Nestares and Heeger (2000), presents a method based on robust statistics to register images in the presence of differences, such as jaw movement, differential MR distortions and true anatomical change. The approach we present guarantees inverse consistency (symmetry), can deal with different intensity scales and automatically estimates a sensitivity parameter to detect outlier regions in the images. The resulting registrations are highly accurate due to their ability to ignore outlier regions and show superior robustness with respect to noise, to intensity scaling and outliers when compared to state-of-the-art registration tools such as FLIRT (in FSL) or the coregistration tool in SPM.

Comparison of piece-wise linear, linear, and nonlinear atlas-to-patient warping techniques: analysis of the labeling of subcortical nuclei for functional neurosurgical applications

Digital atlases are commonly used in pre-operative planning in functional neurosurgical procedures performed to minimize the symptoms of Parkinson's disease. These atlases can be customized to fit an individual patient's anatomy through atlas-to-patient warping procedures. Once fitted to pre-operative magnetic resonance imaging (MRI) data, the customized atlas can be used to plan and navigate surgical procedures. Linear, piece-wise linear and nonlinear registration methods have been used to customize different digital atlases with varying accuracies. Our goal was to evaluate eight different registration methods for atlas-to-patient customization of a new digital atlas of the basal ganglia and thalamus to demonstrate the value of nonlinear registration for automated atlas-based subcortical target identification in functional neurosurgery. In this work, we evaluate the accuracy of two automated linear techniques, two piece-wise linear techniques (requiring the identification of manually placed anatomical landmarks), and four different automated nonlinear atlas-to-patient warping techniques (where two of the four nonlinear techniques are variants of the ANIMAL algorithm). Since a gold standard of the subcortical anatomy is not available, manual segmentations of the striatum, globus pallidus, and thalamus are used to derive a silver standard for evaluation. Four different metrics, including the kappa statistic, the mean distance between the surfaces, the maximum distance between surfaces, and the total structure volume are used to compare the warping techniques. The results show that nonlinear techniques perform statistically better than linear and piece-wise linear techniques. In addition, the results demonstrate statistically significant differences between the nonlinear techniques, with the ANIMAL algorithm yielding better results.

Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration

All fields of neuroscience that employ brain imaging need to communicate their results with reference to anatomical regions. In particular, comparative morphometry and group analysis of functional and physiological data require coregistration of brains to establish correspondences across brain structures. It is well established that linear registration of one brain to another is inadequate for aligning brain structures, so numerous algorithms have emerged to nonlinearly register brains to one another. This study is the largest evaluation of nonlinear deformation algorithms applied to brain image registration ever conducted. Fourteen algorithms from laboratories around the world are evaluated using 8 different error measures. More than 45,000 registrations between 80 manually labeled brains were performed by algorithms including: AIR, ANIMAL, ART, Diffeomorphic Demons, FNIRT, IRTK, JRD-fluid, ROMEO, SICLE, SyN, and four different SPM5 algorithms ("SPM2-type" and regular Normalization, Unified Segmentation, and the DARTEL Toolbox). All of these registrations were preceded by linear registration between the same image pairs using FLIRT. One of the most significant findings of this study is that the relative performances of the registration methods under comparison appear to be little affected by the choice of subject population, labeling protocol, and type of overlap measure. This is important because it suggests that the findings are generalizable to new subject populations that are labeled or evaluated using different labeling protocols. Furthermore, we ranked the 14 methods according to three completely independent analyses (permutation tests, one-way ANOVA tests, and indifference-zone ranking) and derived three almost identical top rankings of the methods. ART, SyN, IRTK, and SPM's DARTEL Toolbox gave the best results according to overlap and distance measures, with ART and SyN delivering the most consistently high accuracy across subjects and label sets. Updates will be published on the http://www.mindboggle.info/papers/ website.

Nonrigid Registration of Brain Tumor Resection MR Images Based on Joint Saliency Map and Keypoint Clustering

This paper proposes a novel global-to-local nonrigid brain MR image registration to compensate for the brain shift and the unmatchable outliers caused by the tumor resection. The mutual information between the corresponding salient structures, which are enhanced by the joint saliency map (JSM), is maximized to achieve a global rigid registration of the two images. Being detected and clustered at the paired contiguous matching areas in the globally registered images, the paired pools of DoG keypoints in combination with the JSM provide a useful cluster-to-cluster correspondence to guide the local control-point correspondence detection and the outlier keypoint rejection. Lastly, a quasi-inverse consistent deformation is smoothly approximated to locally register brain images through the mapping the clustered control points by compact support radial basis functions. The 2D implementation of the method can model the brain shift in brain tumor resection MR images, though the theory holds for the 3D case.

A geometry-driven optical flow warping for spatial normalization of cortical surfaces

Spatial normalization is frequently used to map data to a standard coordinate system by removing intersubject morphological differences, thereby allowing for group analysis to be carried out. The work presented in this paper is motivated by the need for an automated cortical surface normalization technique that will automatically identify homologous cortical landmarks and map them to the same coordinates on a standard manifold. The geometry of a cortical surface is analyzed using two shape measures that distinguish the sulcal and gyral regions in a multiscale framework. A multichannel optical flow warping procedure aligns these shape measures between a reference brain and a subject brain, creating the desired normalization. The partial differential equation that carries out the warping is implemented in a Euclidean framework in order to facilitate a multiresolution strategy, thereby permitting large deformations between the two surfaces. The technique is demonstrated by aligning 33 normal cortical surfaces and showing both improved structural alignment in manually labeled sulci and improved functional alignment in positron emission tomography data mapped to the surfaces. A quantitative comparison between our proposed surface-based spatial normalization method and a leading volumetric spatial normalization method is included to show that the surface-based spatial normalization performs better in matching homologous cortical anatomies.

Geometric distortion correction for echo planar images using nonrigid registration with spatially varying scale

One method used to correct geometric and intensity distortions in echo planar images is to register them to undistorted images via nonrigid deformations. However, some areas in the echo planar images are more distorted than others, thus suggesting the use of deformations whose characteristics are adapted spatially. In this article, we incorporate into our previously developed registration algorithm a spatially varying scale mechanism, which adapts the local scale properties of the transformation by means of a scale map. To compute the scale map, a technique is proposed that relies on an estimate of the expected deformation field. This estimate is generated using knowledge of the physical processes that induce distortions in echo planar images. We evaluate the method of spatially varying scale on both simulated and real data. We find that, in comparison with our earlier method using fixed scale, our new method finds deformation fields that are smoother and finds them faster without sacrificing accuracy.

A review of geometric transformations for nonrigid body registration

This paper provides a comprehensive and quantitative review of spatial transformations models for nonrigid image registration. It explains the theoretical foundation of the models and classifies them according to this basis. This results in two categories, physically based models described by partial differential equations of continuum mechanics (e.g., linear elasticity and fluid flow) and basis function expansions derived from interpolation and approximation theory (e.g., radial basis functions, B-splines and wavelets). Recent work on constraining the transformation so that it preserves the topology or is diffeomorphic is also described. The final section reviews some recent evaluation studies. The paper concludes by explaining under what conditions a particular transformation model is appropriate.

A duality based algorithm for TV-L1-optical-flow image registration

Nonlinear image registration is a challenging task in the field of medical image analysis. In many applications discontinuities may be present in the displacement field, and intensity variations may occur. In this work we therefore utilize an energy functional which is based on Total Variation regularization and a robust data term. We propose a novel, fast and stable numerical scheme to find the minimizer of this energy. Our approach combines a fixed-point procedure derived from duality principles combined with a fast thresholding step. We show experimental results on synthetic and clinical CT lung data sets at different breathing states as well as registration results on inter-subject brain MRIs.

Non-Rigid Multi-Modal Image Registration Using Cross-Cumulative Residual Entropy

In this paper we present a new approach for the non-rigid registration of multi-modality images. Our approach is based on an information theoretic measure called the cumulative residual entropy (CRE), which is a measure of entropy defined using cumulative distributions. Cross-CRE between two images to be registered is defined and maximized over the space of smooth and unknown non-rigid transformations. For efficient and robust computation of the non-rigid deformations, a tri-cubic B-spline based representation of the deformation function is used. The key strengths of combining CCRE with the tri-cubic B-spline representation in addressing the non-rigid registration problem are that, not only do we achieve the robustness due to the nature of the CCRE measure, we also achieve computational efficiency in estimating the non-rigid registration. The salient features of our algorithm are: (i) it accommodates images to be registered of varying contrast+brightness, (ii) faster convergence speed compared to other information theory-based measures used for non-rigid registration in literature, (iii) analytic computation of the gradient of CCRE with respect to the non-rigid registration parameters to achieve efficient and accurate registration, (iv) it is well suited for situations where the source and the target images have field of views with large non-overlapping regions. We demonstrate these strengths via experiments on synthesized and real image data.

Brain functional localization: a survey of image registration techniques

Functional localization is a concept which involves the application of a sequence of geometrical and statistical image processing operations in order to define the location of brain activity or to produce functional/parametric maps with respect to the brain structure or anatomy. Considering that functional brain images do not normally convey detailed structural information and, thus, do not present an anatomically specific localization of functional activity, various image registration techniques are introduced in the literature for the purpose of mapping functional activity into an anatomical image or a brain atlas. The problems addressed by these techniques differ depending on the application and the type of analysis, i.e., single-subject versus group analysis. Functional to anatomical brain image registration is the core part of functional localization in most applications and is accompanied by intersubject and subject-to-atlas registration for group analysis studies. Cortical surface registration and automatic brain labeling are some of the other tools towards establishing a fully automatic functional localization procedure. While several previous survey papers have reviewed and classified general-purpose medical image registration techniques, this paper provides an overview of brain functional localization along with a survey and classification of the image registration techniques related to this problem.

Quantitative comparison of AIR, SPM, and the fully deformable model for atlas-based segmentation of functional and structural MR images

Typical packages used for coregistration in functional image analyses include automated image registration (AIR) and statistical parametric mapping (SPM). However, both methods have limited-dimension deformation models. A fully deformable model, which combines the piecewise linear registration for coarse alignment with demons algorithm for voxel-level refinement, allows a higher degree of spatial deformation. This leads to a more accurate colocalization of the functional signal from different subjects and therefore can produce a more reliable group average signal. We quantitatively compared the performance of the three different registration approaches through a series of experiments and we found that the fully deformable model consistently produces a more accurate structural segmentation and a more reliable functional signal colocalization than does AIR or SPM.

Multigrid tomographic inversion with variable resolution data and image spaces

A multigrid inversion approach that uses variable resolutions of both the data space and the image space is proposed. Since the computational complexity of inverse problems typically increases with a larger number of unknown image pixels and a larger number of measurements, the proposed algorithm further reduces the computation relative to conventional multigrid approaches, which change only the image space resolution at coarse scales. The advantage is particularly important for data-rich applications, where data resolutions may differ for different scales. Applications of the approach to Bayesian reconstruction algorithms in transmission and emission tomography with a generalized Gaussian Markov random field image prior are presented, both with a Poisson noise model and with a quadratic data term. Simulation results indicate that the proposed multigrid approach results in significant improvement in convergence speed compared to the fixed-grid iterative coordinate descent method and a multigrid method with fixed-data resolution.

Medical image registration with partial data

We have developed a general-purpose registration algorithm for medical images and volumes. The transformation between images is modeled as locally affine but globally smooth, and explicitly accounts for local and global variations in image intensities. An explicit model of missing data is also incorporated, allowing us to simultaneously segment and register images with partial or missing data. The algorithm is built upon a differential multiscale framework and incorporates the expectation maximization algorithm. We show that this approach is highly effective in registering a range of synthetic and clinical medical images.

Diffeomorphic registration using B-splines

In this paper we propose a diffeomorphic non-rigid registration algorithm based on free-form deformations (FFDs) which are modelled by B-splines. In contrast to existing non-rigid registration methods based on FFDs the proposed diffeomorphic non-rigid registration algorithm based on free-form deformations (FFDs) which are modelled by B-splines. To construct a diffeomorphic transformation we compose a sequence of free-form deformations while ensuring that individual FFDs are one-to-one transformations. We have evaluated the algorithm on 20 normal brain MR images which have been manually segmented into 67 anatomical structures. Using the agreement between manual segmentation and segmentation propagation as a measure of registration quality we have compared the algorithm to an existing FFD registration algorithm and a modified FFD registration algorithm which penalises non-diffeomorphic transformations. The results show that the proposed algorithm generates diffeomorphic transformations while providing similar levels of performance as the existing FFD registration algorithm in terms of registration accuracy.

Assessment of Left Ventricular Function in Cardiac MSCT Imaging by a 4D Hierarchical Surface-Volume Matching Process

Multislice computed tomography (MSCT) scanners offer new perspectives for cardiac kinetics evaluation with 4D dynamic sequences of high contrast and spatiotemporal resolutions. A new method is proposed for cardiac motion extraction in multislice CT. Based on a 4D hierarchical surface-volume matching process, it provides the detection of the heart left cavities along the acquired sequence and the estimation of their 3D surface velocity fields. A Markov random field model is defined to find, according to topological descriptors, the best correspondences between a 3D mesh describing the left endocardium at one time and the 3D acquired volume at the following time. The global optimization of the correspondences is realized with a multiresolution process. Results obtained on simulated and real data show the capabilities to extract clinically relevant global and local motion parameters and highlight new perspectives in cardiac computed tomography imaging.

Robust nonrigid registration to capture brain shift from intraoperative MRI

We present a new algorithm to register 3-D preoperative magnetic resonance (MR) images to intraoperative MR images of the brain which have undergone brain shift. This algorithm relies on a robust estimation of the deformation from a sparse noisy set of measured displacements. We propose a new framework to compute the displacement field in an iterative process, allowing the solution to gradually move from an approximation formulation (minimizing the sum of a regularization term and a data error term) to an interpolation formulation (least square minimization of the data error term). An outlier rejection step is introduced in this gradual registration process using a weighted least trimmed squares approach, aiming at improving the robustness of the algorithm. We use a patient-specific model discretized with the finite element method in order to ensure a realistic mechanical behavior of the brain tissue. To meet the clinical time constraint, we parallelized the slowest step of the algorithm so that we can perform a full 3-D image registration in 35 s (including the image update time) on a heterogeneous cluster of 15 personal computers. The algorithm has been tested on six cases of brain tumor resection, presenting a brain shift of up to 14 mm. The results show a good ability to recover large displacements, and a limited decrease of accuracy near the tumor resection cavity.

A non-rigid registration approach for quantifying myocardial contraction in tagged MRI using generalized information measures

We address the problem of quantitatively assessing myocardial function from tagged MRI sequences. We develop a two-step method comprising (i) a motion estimation step using a novel variational non-rigid registration technique based on generalized information measures, and (ii) a measurement step, yielding local and segmental deformation parameters over the whole myocardium. Experiments on healthy and pathological data demonstrate that this method delivers, within a reasonable computation time and in a fully unsupervised way, reliable measurements for normal subjects and quantitative pathology-specific information. Beyond cardiac MRI, this work redefines the foundations of variational non-rigid registration for information-theoretic similarity criteria with potential interest in multimodal medical imaging.

3-D deformable image registration: a topology preservation scheme based on hierarchical deformation models and interval analysis optimization

This paper deals with topology preservation in three-dimensional (3-D) deformable image registration. This work is a nontrivial extension of, which addresses the case of two-dimensional (2-D) topology preserving mappings. In both cases, the deformation map is modeled as a hierarchical displacement field, decomposed on a multiresolution B-spline basis. Topology preservation is enforced by controlling the Jacobian of the transformation. Finding the optimal displacement parameters amounts to solving a constrained optimization problem: The residual energy between the target image and the deformed source image is minimized under constraints on the Jacobian. Unlike the 2-D case, in which simple linear constraints are derived, the 3-D B-spline-based deformable mapping yields a difficult (until now, unsolved) optimization problem. In this paper, we tackle the problem by resorting to interval analysis optimization techniques. Care is taken to keep the computational burden as low as possible. Results on multipatient 3-D MRI registration illustrate the ability of the method to preserve topology on the continuous image domain.

Effect of segmentation errors on 3D-to-2D registration of implant models in X-ray images

In many biomedical applications, it is desirable to estimate the three-dimensional (3D) position and orientation (pose) of a metallic rigid object (such as a knee or hip implant) from its projection in a two-dimensional (2D) X-ray image. If the geometry of the object is known, as well as the details of the image formation process, then the pose of the object with respect to the sensor can be determined. A common method for 3D-to-2D registration is to first segment the silhouette contour from the X-ray image; that is, identify all points in the image that belong to the 2D silhouette and not to the background. This segmentation step is then followed by a search for the 3D pose that will best match the observed contour with a predicted contour. Although the silhouette of a metallic object is often clearly visible in an X-ray image, adjacent tissue and occlusions can make the exact location of the silhouette contour difficult to determine in places. Occlusion can occur when another object (such as another implant component) partially blocks the view of the object of interest. In this paper, we argue that common methods for segmentation can produce errors in the location of the 2D contour, and hence errors in the resulting 3D estimate of the pose. We show, on a typical fluoroscopy image of a knee implant component, that interactive and automatic methods for segmentation result in segmented contours that vary significantly. We show how the variability in the 2D contours (quantified by two different metrics) corresponds to variability in the 3D poses. Finally, we illustrate how traditional segmentation methods can fail completely in the (not uncommon) cases of images with occlusion.

A general framework for nonlinear multigrid inversion

A variety of new imaging modalities, such as optical diffusion tomography, require the inversion of a forward problem that is modeled by the solution to a three-dimensional partial differential equation. For these applications, image reconstruction is particularly difficult because the forward problem is both nonlinear and computationally expensive to evaluate. In this paper, we propose a general framework for nonlinear multigrid inversion that is applicable to a wide variety of inverse problems. The multigrid inversion algorithm results from the application of recursive multigrid techniques to the solution of optimization problems arising from inverse problems. The method works by dynamically adjusting the cost functionals at different scales so that they are consistent with, and ultimately reduce, the finest scale cost functional. In this way, the multigrid inversion algorithm efficiently computes the solution to the desired fine-scale inversion problem. Importantly, the new algorithm can greatly reduce computation because both the forward and inverse problems are more coarsely discretized at lower resolutions. An application of our method to Bayesian optical diffusion tomography with a generalized Gaussian Markov random-field image prior model shows the potential for very large computational savings. Numerical data also indicates robust convergence with a range of initialization conditions for this nonconvex optimization problem.

Determining correspondence in 3-D MR brain images using attribute vectors as morphological signatures of voxels

Finding point correspondence in anatomical images is a key step in shape analysis and deformable registration. This paper proposes an automatic correspondence detection algorithm for intramodality MR brain images of different subjects using wavelet-based attribute vectors (WAVs) defined on every image voxel. The attribute vector (AV) is extracted from the wavelet subimages and reflects the image structure in a large neighborhood around the respective voxel in a multiscale fashion. It plays the role of a morphological signature for each voxel, and our goal is, therefore, to make it distinctive of the respective voxel. Correspondence is then determined from similarities of AVs. By incorporating the prior knowledge of the spatial relationship among voxels, the ability of the proposed algorithm to find anatomical correspondence is further improved. Experiments with MR images of human brains show that the algorithm performs similarly to experts, even for complex cortical structures.

Object-based morphometry of the cerebral cortex

Most of the approaches dedicated to automatic morphometry rely on a point-by-point strategy based on warping each brain toward a reference coordinate system. In this paper, we describe an alternative object-based strategy dedicated to the cortex. This strategy relies on an artificial neuroanatomist performing automatic recognition of the main cortical sulci and parcellation of the cortical surface into gyral patches. A set of shape descriptors, which can be compared across subjects, is then attached to the sulcus and gyrus related objects segmented by this process. The framework is used to perform a study of 142 brains of the International Consortium for Brain Mapping (ICBM) database. This study reveals some correlates of handedness on the size of the sulci located in motor areas, which was not detected previously using standard voxel based morphometry.

A hierarchical parametric algorithm for deformable multimodal image registration

Image fusion is of utmost importance for many applications in image analysis. Particularly in medical imaging, images of different modalities are necessary because they provide complementary information that must be merged for an optimal use. The fusion of these images, which can be achieved through a registration process, makes it possible to superimpose all available information on the same frame. In many cases, a rigid transformation is sufficient to align correctly the images. However, there are cases where a non-rigid transformation is needed: geometrical distortions present in one image, non-rigid motion, etc. The purpose of this paper is to propose a generic method to account for these deformations in case of multimodal images. We have applied the algorithm in the particular context of 3D medical images and present results on simulated and real data.

Fast free-form deformable registration via calculus of variations

In this paper, we present a fully automatic, fast and accurate deformable registration technique. This technique deals with free-form deformation. It minimizes an energy functional that combines both similarity and smoothness measures. By using calculus of variations, the minimization problem was represented as a set of nonlinear elliptic partial differential equations (PDEs). A Gauss-Seidel finite difference scheme is used to iteratively solve the PDE. The registration is refined by a multi-resolution approach. The whole process is fully automatic. It takes less than 3 min to register two three-dimensional (3D) image sets of size 256 x 256 x 61 using a single 933 MHz personal computer. Extensive experiments are presented. These experiments include simulations, phantom studies and clinical image studies. Experimental results show that our model and algorithm are suited for registration of temporal images of a deformable body. The registration of inspiration and expiration phases of the lung images shows that the method is able to deal with large deformations. When applied to the daily CT images of a prostate patient, the results show that registration based on iterative refinement of displacement field is appropriate to describe the local deformations in the prostate and the rectum. Similarity measures improved significantly after the registration. The target application of this paper is for radiotherapy treatment planning and evaluation that incorporates internal organ deformation throughout the course of radiation therapy. The registration method could also be equally applied in diagnostic radiology.

A framework to study the cortical folding patterns

This paper describes a decade-long research program focused on the variability of the cortical folding patterns. The program has developed a framework of using artificial neuroanatomists that are trained to identify sulci from a database. The framework relies on a renormalization of the brain warping problem, which consists in matching the cortices at the scale of the folds. Another component of the program is the search for the alphabet of the folding patterns, namely, a list of indivisible elementary sulci. The search relies on the study of the cortical folding process using antenatal imaging and on backward simulations of morphogenesis aimed at revealing traces of the embryologic dimples in the mature cortical surface. The importance of sulcal-based morphometry is illustrated by a simple study of the correlates of handedness on asymmetry indices. The study shows for instance that the central sulcus is larger in the dominant hemisphere.

Geodesic estimation for large deformation anatomical shape averaging and interpolation

The goal of this research is to promote variational methods for anatomical averaging that operate within the space of the underlying image registration problem. This approach is effective when using the large deformation viscous framework, where linear averaging is not valid, or in the elastic case. The theory behind this novel atlas building algorithm is similar to the traditional pairwise registration problem, but with single image forces replaced by average forces. These group forces drive an average transport ordinary differential equation allowing one to estimate the geodesic that moves an image toward the mean shape configuration. This model gives large deformation atlases that are optimal with respect to the shape manifold as defined by the data and the image registration assumptions. We use the techniques in the large deformation context here, but they also pertain to small deformation atlas construction. Furthermore, a natural, inherently inverse consistent image registration is gained for free, as is a tool for constant arc length geodesic shape interpolation. The geodesic atlas creation algorithm is quantitatively compared to the Euclidean anatomical average to elucidate the need for optimized atlases. The procedures generate improved average representations of highly variable anatomy from distinct populations.

A robust method for registration of three-dimensional knee implant models to two-dimensional fluoroscopy images

A method was developed for registering three-dimensional knee implant models to single plane X-ray fluoroscopy images. We use a direct image-to-image similarity measure, taking advantage of the speed of modern computer graphics workstations to quickly render simulated (predicted) images. As a result, the method does not require an accurate segmentation of the implant silhouette in the image (which can be prone to errors). A robust optimization algorithm (simulated annealing) is used that can escape local minima and find the global minimum (true solution). Although we focus on the analysis of total knee arthroplasty (TKA) in this paper, the method can be (and has been) applied to other implanted joints, including, but not limited to, hips, ankles, and temporomandibular joints. Convergence tests on an in vivo image show that the registration method can reliably find poses that are very close to the optimal (i.e., within 0.4 degrees and 0.1 mm), even from starting poses with large initial errors. However, the precision of translation measurement in the Z (out-of-plane) direction is not as good. We also show that the method is robust with respect to image noise and occlusions. However, a small amount of user supervision and intervention is necessary to detect cases when the optimization algorithm falls into a local minimum. Intervention is required less than 5% of the time when the initial starting pose is reasonably close to the correct answer, but up to 50% of the time when the initial starting pose is far away. Finally, extensive evaluations were performed on cadaver images to determine accuracy of relative pose measurement. Comparing against data derived from an optical sensor as a "gold standard," the overall root-mean-square error of the registration method was approximately 1.5 degrees and 0.65 mm (although Z translation error was higher). However, uncertainty in the optical sensor data may account for a large part of the observed error.

The adaptive bases algorithm for intensity-based nonrigid image registration

Nonrigid registration of medical images is important for a number of applications such as the creation of population averages, atlas-based segmentation, or geometric correction of functional magnetic resonance imaging (fMRI) images to name a few. In recent years, a number of methods have been proposed to solve this problem, one class of which involves maximizing a mutual information (MI)-based objective function over a regular grid of splines. This approach has produced good results but its computational complexity is proportional to the compliance of the transformation required to register the smallest structures in the image. Here, we propose a method that permits the spatial adaptation of the transformation's compliance. This spatial adaptation allows us to reduce the number of degrees of freedom in the overall transformation, thus speeding up the process and improving its convergence properties. To develop this method, we introduce several novelties: 1) we rely on radially symmetric basis functions rather than B-splines traditionally used to model the deformation field; 2) we propose a metric to identify regions that are poorly registered and over which the transformation needs to be improved; 3) we partition the global registration problem into several smaller ones; and 4) we introduce a new constraint scheme that allows us to produce transformations that are topologically correct. We compare the approach we propose to more traditional ones and show that our new algorithm compares favorably to those in current use.

Retrospective evaluation of intersubject brain registration

Although numerous methods to register brains of different individuals have been proposed, no work has been done, as far as we know, to evaluate and objectively compare the performances of different nonrigid (or elastic) registration methods on the same database of subjects. In this paper, we propose an evaluation framework, based on global and local measures of the relevance of the registration. We have chosen to focus more particularly on the matching of cortical areas, since intersubject registration methods are dedicated to anatomical and functional normalization, and also because other groups have shown the relevance of such registration methods for deep brain structures. Experiments were conducted using 6 methods on a database of 18 subjects. The global measures used show that the quality of the registration is directly related to the transformation's degrees of freedom. More surprisingly, local measures based on the matching of cortical sulci did not show significant differences between rigid and non rigid methods.

Coupling dense and landmark-based approaches for nonrigid registration

In this paper, we investigate the introduction of cortical constraints for non rigid intersubject brain registration. We extract sulcal patterns with the active ribbon method, presented by Le Goualher et al. (1997). An energy based registration method (Hellier et al., 2001), which will be called photometric registration method in this paper, makes it possible to incorporate the matching of cortical sulci. The local sparse similarity and the photometric similarity are, thus, expressed in a unified framework. We show the benefits of cortical constraints on a database of 18 subjects, with global and local assessment of the registration. This new registration scheme has also been evaluated on functional magnetoencephalography data. We show that the anatomically constrained registration leads to a substantial reduction of the intersubject functional variability.