Kernel Bundle Diffeomorphic Image Registration Using Stationary Velocity Fields and Wendland Basis Functions

Learning Deformable Image Registration From Optimization: Perspective, Modules, Bilevel Training and Beyond

Conventional deformable registration methods aim at solving an optimization model carefully designed on image pairs and their computational costs are exceptionally high. In contrast, recent deep learning-based approaches can provide fast deformation estimation. These heuristic network architectures are fully data-driven and thus lack explicit geometric constraints which are indispensable to generate plausible deformations, e.g., topology-preserving. Moreover, these learning-based approaches typically pose hyper-parameter learning as a black-box problem and require considerable computational and human effort to perform many training runs. To tackle the aforementioned problems, we propose a new learning-based framework to optimize a diffeomorphic model via multi-scale propagation. Specifically, we introduce a generic optimization model to formulate diffeomorphic registration and develop a series of learnable architectures to obtain propagative updating in the coarse-to-fine feature space. Further, we propose a new bilevel self-tuned training strategy, allowing efficient search of task-specific hyper-parameters. This training strategy increases the flexibility to various types of data while reduces computational and human burdens. We conduct two groups of image registration experiments on 3D volume datasets including image-to-atlas registration on brain MRI data and image-to-image registration on liver CT data. Extensive results demonstrate the state-of-the-art performance of the proposed method with diffeomorphic guarantee and extreme efficiency. We also apply our framework to challenging multi-modal image registration, and investigate how our registration to support the down-streaming tasks for medical image analysis including multi-modal fusion and image segmentation.

Deformation trajectory prediction using a neural network trained on finite element data-application to library of CTVs creation for cervical cancer

Purpose. We propose a neural network for fast prediction of realistic, time-parametrized deformations between pairs of input segmentations. The proposed method was used to generate a library of planning CTVs for cervical cancer radiotherapy.Methods.A 3D convolutional neural network (CNN) was introduced to predict a stationary velocity field given the distance maps of the cervix CTV in empty and full bladder anatomy. Diffeomorphic deformation trajectories between the two states were obtained by time integration. Intermediate deformation states were used to populate a library of cervix CTVs. The network was trained on cervix CTV deformations of 20 patients generated by finite element modeling (FEM). Validation was performed on FEM data of 9 healthy volunteers. Additionally, for these subjects, CTV deformations were observed in a series of repeat MR scans as the bladder filled from empty to full. Predicted and FEM libraries were compared, and benchmarked against the observed deformations. Finally, for an independent test set of 20 patients the predicted libraries were evaluated clinically, and compared to the current method.Results.The median Dice score over the validation subjects between the predicted and FEM libraries was >0.95 throughout the deformation, with a median 90 percentile surface distance of <3 mm. The ability to cover observed CTVs was similar for both the FEM-based and the proposed method, with residual offsets being about twice as large as the difference between the two methods. Clinical evaluation showed improved library properties over the method currently used in clinic.Conclusions.We proposed a CNN trained on FEM deformations, which predicts the deformation trajectory between two input states of the cervix CTV in one forward pass. We applied this to CTV library prediction for cervical cancer. The network is able to mimic FEM deformations well, while being much faster and simpler in use.

Uncertainty-Aware Annotation Protocol to Evaluate Deformable Registration Algorithms

Landmark correspondences are a widely used type of gold standard in image registration. However, the manual placement of corresponding points is subject to high inter-user variability in the chosen annotated locations and in the interpretation of visual ambiguities. In this paper, we introduce a principled strategy for the construction of a gold standard in deformable registration. Our framework: (i) iteratively suggests the most informative location to annotate next, taking into account its redundancy with previous annotations; (ii) extends traditional pointwise annotations by accounting for the spatial uncertainty of each annotation, which can either be directly specified by the user, or aggregated from pointwise annotations from multiple experts; and (iii) naturally provides a new strategy for the evaluation of deformable registration algorithms. Our approach is validated on four different registration tasks. The experimental results show the efficacy of suggesting annotations according to their informativeness, and an improved capacity to assess the quality of the outputs of registration algorithms. In addition, our approach yields, from sparse annotations only, a dense visualization of the errors made by a registration method. The source code of our approach supporting both 2D and 3D data is publicly available at https://github.com/LoicPeter/evaluation-deformable-registration.

Deformable Image Registration Using Functions of Bounded Deformation

Deformable image registration is a widely used technique in the field of computer vision and medical image processing. Basically, the task of deformable image registration is to find the displacement field between the moving image and the fixed image. Many variational models are proposed for deformable image registration, under the assumption that the displacement field is continuous and smooth. However, displacement fields may be discontinuous, especially for medical images with intensity inhomogeneity, pathological tissues, or heavy noises. In the mathematical theory of elastoplasticity, when the displacement fields are possibly discontinuous, a suitable framework for describing the displacement fields is the space of functions of bounded deformation (BD). Inspired by this, we propose a novel deformable registration model, called the BD model, which allows discontinuities of displacement fields in images. The BD model is formulated in a variational framework by supposing the displacement field to be a function of BD. The existence of solutions of this model is proven. Numerical experiments on 2D images show that the BD model outperforms the classical demons model, the log-domain diffeomorphic demons model, and the state-of-the-art vectorial total variation model. Numerical experiments on two public 3D databases show that the target registration error of the BD model is competitive compared with more than ten other models.

Multiscale Frame-Based Kernels for Large Deformation Diffeomorphic Metric Mapping

We present a set of multiscale frame-based kernels that can be used to construct diffeomorphic transformation in the large deformation diffeomorphic metric mapping (LDDMM) framework. We construct multiscale kernels via compact wavelet frames that are equipped with the hierarchical multiresolution analysis. We show that these kernels under certain conditions can form reproducing kernel Hilbert spaces of smooth velocity fields and hence can be used to generate multiscale diffeomorphic transformation for LDDMM. As a proof of concept, we incorporate these kernels in the LDDMM framework. We show the improvement of whole brain mapping accuracy using the LDDMM with frame-based kernels in comparison to that obtained using the LDDMM with Gaussian kernels. Moreover, we evaluate whole brain mapping accuracy of the LDDMM with frame-based kernels against that obtained from the 14 brain mapping methods given by Klein et al.. Our results suggest that the LDDMM with frame-based kernels has the potential to outperform the 14 brain mapping methods for whole brain mapping.

A LAGRANGIAN GAUSS-NEWTON-KRYLOV SOLVER FOR MASS- AND INTENSITY-PRESERVING DIFFEOMORPHIC IMAGE REGISTRATION

We present an efficient solver for diffeomorphic image registration problems in the framework of Large Deformations Diffeomorphic Metric Mappings (LDDMM). We use an optimal control formulation, in which the velocity field of a hyperbolic PDE needs to be found such that the distance between the final state of the system (the transformed/transported template image) and the observation (the reference image) is minimized. Our solver supports both stationary and non-stationary (i.e., transient or time-dependent) velocity fields. As transformation models, we consider both the transport equation (assuming intensities are preserved during the deformation) and the continuity equation (assuming mass-preservation). We consider the reduced form of the optimal control problem and solve the resulting unconstrained optimization problem using a discretize-then-optimize approach. A key contribution is the elimination of the PDE constraint using a Lagrangian hyperbolic PDE solver. Lagrangian methods rely on the concept of characteristic curves. We approximate these curves using a fourth-order Runge-Kutta method. We also present an efficient algorithm for computing the derivatives of the final state of the system with respect to the velocity field. This allows us to use fast Gauss-Newton based methods. We present quickly converging iterative linear solvers using spectral preconditioners that render the overall optimization efficient and scalable. Our method is embedded into the image registration framework FAIR and, thus, supports the most commonly used similarity measures and regularization functionals. We demonstrate the potential of our new approach using several synthetic and real world test problems with up to 14.7 million degrees of freedom.