Three-Dimensional Multi-Modality Registration for Orthopaedics and Cardiovascular Settings: State-of-the-Art and Clinical Applications

The multimodal and multidomain registration of medical images have gained increasing recognition in clinical practice as a powerful tool for fusing and leveraging useful information from different imaging techniques and in different medical fields such as cardiology and orthopedics. Image registration could be a challenging process, and it strongly depends on the correct tuning of registration parameters. In this paper, the robustness and accuracy of a landmarks-based approach have been presented for five cardiac multimodal image datasets. The study is based on 3D Slicer software and it is focused on the registration of a computed tomography (CT) and 3D ultrasound time-series of post-operative mitral valve repair. The accuracy of the method, as a function of the number of landmarks used, was performed by analysing root mean square error (RMSE) and fiducial registration error (FRE) metrics. The validation of the number of landmarks resulted in an optimal number of 10 landmarks. The mean RMSE and FRE values were 5.26 ± 3.17 and 2.98 ± 1.68 mm, respectively, showing comparable performances with respect to the literature. The developed registration process was also tested on a CT orthopaedic dataset to assess the possibility of reconstructing the damaged jaw portion for a pre-operative planning setting. Overall, the proposed work shows how 3D Slicer and registration by landmarks can provide a useful environment for multimodal/unimodal registration.

Developmental Mouse Brain Common Coordinate Framework

3D standard reference brains serve as key resources to understand the spatial organization of the brain and promote interoperability across different studies. However, unlike the adult mouse brain, the lack of standard 3D reference atlases for developing mouse brains has hindered advancement of our understanding of brain development. Here, we present a multimodal 3D developmental common coordinate framework (DevCCF) spanning mouse embryonic day (E) 11.5, E13.5, E15.5, E18.5, and postnatal day (P) 4, P14, and P56 with anatomical segmentations defined by a developmental ontology. At each age, the DevCCF features undistorted morphologically averaged atlas templates created from Magnetic Resonance Imaging and co-registered high-resolution templates from light sheet fluorescence microscopy. Expert-curated 3D anatomical segmentations at each age adhere to an updated prosomeric model and can be explored via an interactive 3D web-visualizer. As a use case, we employed the DevCCF to unveil the emergence of GABAergic neurons in embryonic brains. Moreover, we integrated the Allen CCFv3 into the P56 template with stereotaxic coordinates and mapped spatial transcriptome cell-type data with the developmental ontology. In summary, the DevCCF is an openly accessible resource that can be used for large-scale data integration to gain a comprehensive understanding of brain development.

CT to MR registration of complex deformations in the knee joint through dual quaternion interpolation of rigid transforms

Purpose.To develop a method that enables computed tomography (CT) to magnetic resonance (MR) image registration of complex deformations typically encountered in rotating joints such as the knee joint.Methods.We propose a workflow, denoted quaternion interpolated registration (QIR), consisting of three steps, which makes use of prior knowledge of tissue properties to initialise deformable registration. In the first step, the rigid skeletal components were individually registered. Next, the deformation of soft tissue was estimated using a dual quaternion-based interpolation method. In the final step, the registration was fine-tuned with a rigidity-constrained deformable registration step. The method was applied to paired, unregistered CT and MR images of the knee of 92 patients. It was compared to registration using B-Splines (BS) and B-Splines with a rigidity penalty (BSRP). Registration accuracy was evaluated using mutual information, and by calculating Dice similarity coefficient (DSC), mean absolute surface distance (MASD) and 95th percentile Hausdorff distance (HD95) on bone, and DSC on water and fat dominated tissue. To evaluate the rigidity of bone in the registration, the Jacobian determinant (JD) was calculated.Results.QIR achieved improved results with 0.93, 0.76 mm and 1.88 mm on the DSC, MASD and HD95 metrics on bone, compared to 0.87, 1.40 mm and 4.99 mm for method and 0.87, 1.40 mm and 3.56 mm for the BSRP method. The average DSC of water and fat was 0.77 and 0.86 for the QIR, 0.75 and 0.84 for BS and 0.74 and 0.84 for BSRP. Comparison of the median JD and median interquartile (IQR) ranges of the JD indicated that the QIR (1.00 median, 0.03 IQR) resulted in higher rigidity in the rigid skeletal tissues compared to the BS (0.98 median, 0.19 IQR) and BSRP (1.00 median, 0.05 IQR) methods.Conclusion.This study showed that QIR could improve the outcome of complex registration problems, encountered in joints involving rigid and non-rigid bodies such as occur in the knee, as compared to a conventional registration approach.

Multimodality-based knee joint modelling method with bone and cartilage structures for total knee arthroplasty

OBJECTIVE

We propose a robust and accurate knee joint modelling method with bone and cartilage structures to enable accurate surgical guidance for knee surgery.

METHODS

A multimodality registration strategy is proposed to fuse magnetic resonance (MR) and computed tomography (CT) images of the femur and tibia separately to remove spatial inconsistency caused by knee bending in CT/MR scans. Automatic segmentation of the femur, tibia and cartilages is carried out with region of interest clustering and intensity analysis based on the multimodal fusion of images.

RESULTS

Experimental results show that the registration error is 1.13 ± 0.30 mm. The Dice similarity coefficient values of the proposed segmentation method of the femur, tibia, femoral and tibial cartilages are 0.969, 0.966, 0.910 and 0.872, respectively.

CONCLUSIONS

This study demonstrates the feasibility and effectiveness of multimodality-based registration and segmentation methods for knee joint modelling. The proposed method can provide users with 3D anatomical models of the femur, tibia, and cartilages with few human inputs.

Pose estimation of a markerless fiber bundle for endoscopic optical biopsy

Purpose: We present a markerless vision-based method for on-the-fly three-dimensional (3D) pose estimation of a fiberscope instrument to target pathologic areas in the endoscopic view during exploration. Approach: A 2.5-mm-diameter fiberscope is inserted through the endoscope's operating channel and connected to an additional camera to perform complementary observation of a targeted area such as a multimodal magnifier. The 3D pose of the fiberscope is estimated frame-by-frame by maximizing the similarity between its silhouette (automatically detected in the endoscopic view using a deep learning neural network) and a cylindrical shape bound to a kinematic model reduced to three degrees-of-freedom. An alignment of the cylinder axis, based on Plücker coordinates from the straight edges detected in the image, makes convergence faster and more reliable. Results: The performance on simulations has been validated with a virtual trajectory mimicking endoscopic exploration and on real images of a chessboard pattern acquired with different endoscopic configurations. The experiments demonstrated a good accuracy and robustness of the proposed algorithm with errors of 0.33 ± 0.68    mm in distance position and 0.32 ± 0.11   deg in axis orientation for the 3D pose estimation, which reveals its superiority over previous approaches. This allows multimodal image registration with sufficient accuracy of < 3    pixels . Conclusion: Our pose estimation pipeline was executed on simulations and patterns; the results demonstrate the robustness of our method and the potential of fiber-optical instrument image-based tracking for pose estimation and multimodal registration. It can be fully implemented in software and therefore easily integrated into a routine clinical environment.

Fully automated dual-resolution serial optical coherence tomography aimed at diffusion MRI validation in whole mouse brains

An automated dual-resolution serial optical coherence tomography (2R-SOCT) scanner is developed. The serial histology system combines a low-resolution ( 25    μ m / voxel ) 3 × OCT with a high-resolution ( 1.5    μ m / voxel ) 40 × OCT to acquire whole mouse brains at low resolution and to target specific regions of interest (ROIs) at high resolution. The 40 × ROIs positions are selected either manually by the microscope operator or using an automated ROI positioning selection algorithm. Additionally, a multimodal and multiresolution registration pipeline is developed in order to align the 2R-SOCT data onto diffusion MRI (dMRI) data acquired in the same ex vivo mouse brains prior to automated histology. Using this imaging system, 3 whole mouse brains are imaged, and 250 high-resolution 40 × three-dimensional ROIs are acquired. The capability of this system to perform multimodal imaging studies is demonstrated by labeling the ROIs using a mouse brain atlas and by categorizing the ROIs based on their associated dMRI measures. This reveals a good correspondence of the tissue microstructure imaged by the high-resolution OCT with various dMRI measures such as fractional anisotropy, number of fiber orientations, apparent fiber density, orientation dispersion, and intracellular volume fraction.

Multimodal and time-lapse skin registration

BACKGROUND/PURPOSE

Computational skin analysis is revolutionizing modern dermatology. Patterns extracted from image sequences enable algorithmic evaluation. Stacking multiple images to analyze pattern variation implicitly assumes that the images are aligned per-pixel. However, breathing and involuntary motion of the patient causes significant misalignment. Alignment algorithms designed for multimodal and time-lapse skin images can solve this problem. Sequences from multi-modal imaging capture unique appearance features in each modality. Time-lapse image sequences capture skin appearance change over time.

METHODS

Multimodal skin images have been acquired under five different modalities: three in reflectance (visible, parallel-polarized, and cross-polarized) and two in fluorescence mode (UVA and blue light excitation). For time-lapse imagery, 39 images of acne lesions over a 3-month period have been collected. The method detects micro-level features like pores, wrinkles, and other skin texture markings in the acquired images. Images are automatically registered to subpixel accuracy.

RESULTS

The proposed registration approach precisely aligns multimodal and time-lapse images. Subsurface recovery from multimodal images has misregistration artefacts that can be eliminated using this approach. Registered time-lapse imaging captures the evolution of appearance of skin regions with time.

CONCLUSION

Misalignment in skin imaging has significant impact on any quantitative or qualitative image evaluation. Micro-level features can be used to obtain highly accurate registration. Multimodal images can be organized with maximal overlap for successful registration. The resulting point-to-point alignment improves the quality of skin image analysis.

Multimodal deformable registration of traumatic brain injury MR volumes via the Bhattacharyya distance

An important problem of neuroimaging data analysis for traumatic brain injury (TBI) is the task of coregistering MR volumes acquired using distinct sequences in the presence of widely variable pixel movements which are due to the presence and evolution of pathology. We are motivated by this problem to design a numerically stable registration algorithm which handles large deformations. To this end, we propose a new measure of probability distributions based on the Bhattacharyya distance, which is more stable than the widely used mutual information due to better behavior of the square root function than the logarithm at zero. Robustness is illustrated on two TBI patient datasets, each containing 12 MR modalities. We implement our method on graphics processing units (GPU) so as to meet the clinical requirement of time-efficient processing of TBI data. We find that 6 sare required to register a pair of volumes with matrix sizes of 256 × 256 × 60 on the GPU. In addition to exceptional time efficiency via its GPU implementation, this methodology provides a clinically informative method for the mapping and evaluation of anatomical changes in TBI.

Joint CT/CBCT deformable registration and CBCT enhancement for cancer radiotherapy

This paper details an algorithm to simultaneously perform registration of computed tomography (CT) and cone-beam computed (CBCT) images, and image enhancement of CBCT. The algorithm employs a viscous fluid model which naturally incorporates two components: a similarity measure for registration and an intensity correction term for image enhancement. Incorporating an intensity correction term improves the registration results. Furthermore, applying the image enhancement term to CBCT imagery leads to an intensity corrected CBCT with better image quality. To achieve minimal processing time, the algorithm is implemented on a graphic processing unit (GPU) platform. The advantage of the simultaneous optimization strategy is quantitatively validated and discussed using a synthetic example. The effectiveness of the proposed algorithm is then illustrated using six patient datasets, three head-and-neck datasets and three prostate datasets.