Evaluating Similarity Measures for Brain Image Registration

MICDIR: Multi-scale inverse-consistent deformable image registration using UNetMSS with self-constructing graph latent

Image registration is the process of bringing different images into a common coordinate system - a technique widely used in various applications of computer vision, such as remote sensing, image retrieval, and, most commonly, medical imaging. Deep learning based techniques have been applied successfully to tackle various complex medical image processing problems, including medical image registration. Over the years, several image registration techniques have been proposed using deep learning. Deformable image registration techniques such as Voxelmorph have been successful in capturing finer changes and providing smoother deformations. However, Voxelmorph, as well as ICNet and FIRE, do not explicitly encode global dependencies (i.e. the overall anatomical view of the supplied image) and, therefore, cannot track large deformations. In order to tackle the aforementioned problems, this paper extends the Voxelmorph approach in three different ways. To improve the performance in case of small as well as large deformations, supervision of the model at different resolutions has been integrated using a multi-scale UNet. To support the network to learn and encode the minute structural co-relations of the given image-pairs, a self-constructing graph network (SCGNet) has been used as the latent of the multi-scale UNet - which can improve the learning process of the model and help the model to generalise better. And finally, to make the deformations inverse-consistent, cycle consistency loss has been employed. On the task of registration of brain MRIs, the proposed method achieved significant improvements over ANTs and VoxelMorph, obtaining a Dice score of 0.8013 ± 0.0243 for intramodal and 0.6211 ± 0.0309 for intermodal, while VoxelMorph achieved 0.7747 ± 0.0260 and 0.6071 ± 0.0510, respectively.

Image Registration in Longitudinal Bone Assessment Using Computed Tomography

PURPOSE OF REVIEW

Rigid image registration is an important image processing tool for the assessment of musculoskeletal chronic disease. In this paper, we critically review applications of rigid image registration in terms of similarity measurement methods over the past three years (2019-2022) in the context of monitoring longitudinal changes to bone microstructure and mechanical properties using computed tomography. This review identifies critical assumptions and trade-offs underlying different similarity measurement methods used in image registration and demonstrates the effect of using different similarity measures on registration outcomes.

RECENT FINDINGS

Image registration has been used in recent studies for: correcting positional shifts between longitudinal scans to quantify changes to bone microstructural and mechanical properties over time, developing registration-based workflows for longitudinal assessment of bone properties in pre-clinical and clinical studies, and developing and validating registration techniques for longitudinal studies. In evaluating the recent literature, it was found that the assumptions at the root of different similarity measures used in rigid image registration are not always confirmed and reported. Each similarity measurement has its advantages and disadvantages, as well as underlying assumptions. Breaking these assumptions can lead to poor and inaccurate registration results. Thus, care must be taken with regards to the choice of similarity measurement and interpretation of results. We propose that understanding and verifying the assumptions of similarity measurements will enable more accurate and efficient quantitative assessments of structural changes over time.

Multi-Modal Rigid Image Registration and Segmentation Using Multi-Stage Forward Path Regenerative Genetic Algorithm

Medical image diagnosis and delineation of lesions in the human brain require information to combine from different imaging sensors. Image registration is considered to be an essential pre-processing technique of aligning images of different modalities. The brain is a naturally bilateral symmetrical organ, where the left half lobe resembles the right half lobe around the symmetrical axis. The identified symmetry axis in one MRI image can identify symmetry axes in multi-modal registered MRI images instantly. MRI sensors may induce different levels of noise and Intensity Non-Uniformity (INU) in images. These image degradations may cause difficulty in finding true transformation parameters for an optimization technique. We will be investigating the new variant of evolution strategy of genetic algorithm as an optimization technique that performs well even for the high level of noise and INU, compared to Nesterov, Limited-memory Broyden–Fletcher–Goldfarb–Shanno algorithm (LBFGS), Simulated Annealing (SA), and Single-Stage Genetic Algorithm (SSGA). The proposed new multi-modal image registration technique based on a genetic algorithm with increasing precision levels and decreasing search spaces in successive stages is called the Multi-Stage Forward Path Regenerative Genetic Algorithm (MFRGA). Our proposed algorithm is better in terms of overall registration error as compared to the standard genetic algorithm. MFRGA results in a mean registration error of 0.492 in case of the same level of noise (1–9)% and INU (0–40)% in both reference and template image, and 0.317 in case of a noise-free template and reference with noise levels (1–9)% and INU (0–40)%. Accurate registration results in good segmentation, and we apply registration transformations to segment normal brain structures for evaluating registration accuracy. The brain segmentation via registration with our proposed algorithm is better even in cases of high levels of noise and INU as compared to GA and LBFGS. The mean dice similarity coefficient of brain structures CSF, GM, and WM is 0.701, 0.792, and 0.913, respectively.

Symmetry-based representation for registration of multimodal images

We propose a new two-dimensional structural representation method for registration of multimodal images by using the local structural symmetry of images, which is similar at different modalities. The symmetry is measured in various orientations and the best is mapped and used for the representation image. The optimum performance is obtained when using only two different orientations, which is called binary dominant symmetry representation (BDSR). This representation is highly robust to noise and intensity non-uniformity. We also propose a new objective function based on L2 distance with low sensitivity to the overlapping region. Then, five different meta-heuristic algorithms are comparatively applied. Two of them have been used for the first time on image registration. BDSR remarkably outperforms the previous successful representations, such as entropy images, self-similarity context, and modality-independent local binary pattern, as well as mutual information-based registration, in terms of success rate, runtime, convergence error, and representation construction.

An overview of image registration for aligning mass spectrometry imaging with clinically relevant imaging modalities

•

Mass spectrometry imaging (MSI) is a powerful molecular imaging technique.

•

Integration with other imaging modalities is essential in clinical MSI.

•

Image integration is performed by image registration techniques.

•

Technical potential of image registration in MSI has not been fully exploited.

•

Roadmap proposed to improve registration accuracy.

Mass spectrometry imaging (MSI) is used in many aspects of clinical research, including pharmacokinetics, toxicology, personalised medicine, and surgical decision-making. Maximising its potential requires the spatial integration of MSI images with imaging data from existing clinical imaging modalities, such as histology and MRI. To ensure that the information is properly integrated, all contributing images must be accurately aligned. This process is called image registration and is the focus of this review. In light of the ever-increasing spatial resolution of MSI instrumentation and a diversification of multi-modal MSI studies (e.g., spatial omics, 3D-MSI), the accuracy, versatility, and precision of image registration must increase accordingly. We review the application of image registration to align MSI data with different clinically relevant ex vivo and in vivo imaging techniques. Based on this, we identify steps in the current image registration processes where there is potential for improvement. Finally, we propose a roadmap for community efforts to address these challenges in order to increase registration quality and help MSI to fully exploit its multi-modal potential.

End-to-end multimodal image registration via reinforcement learning

Multimodal image registration is a vital initial step in several medical image applications for providing complementary information from different data modalities. Since images with different modalities do not exhibit the same characteristics, finding their accurate correspondences remains a challenge. For convolutional multimodal registration methods, two components are quite significant: descriptive image feature as well as the suited similarity metric. However, these two components are often custom-designed and are infeasible to the high diversity of tissue appearance across modalities. In this paper, we translate image registration into a decision-making problem, where registration is achieved via an artificial agent trained by asynchronous reinforcement learning. More specifically, convolutional long-short-term-memory is incorporated after stacked convolutional layers in this method to extract spatial-temporal image features and learn the similarity metric implicitly. A customized reward function driven by landmark error is advocated to guide the agent to the correct registration direction. A Monte Carlo rollout strategy is also leveraged to perform as a look-ahead inference in the testing stage, to increase registration accuracy further. Experiments on paired CT and MR images of patients diagnosed as nasopharyngeal carcinoma demonstrate that our method achieves state-of-the-art performance in medical image registration.

Computed Tomography/Magnetic Resonance Imaging (CT/MRI) Image Registration and Fusion Assessment for Accurate Glioblastoma Radiotherapy Treatment Planning

Background: In this study, computed tomography/magnetic resonance imaging (CT/MRI) image registration and fusion in the 3D conformal radiotherapy treatment planning of Glioblastoma brain tumor was investigated. Good CT/MRI image registration and fusion made a great impact on dose calculation and treatment planning accuracy. Indeed, the uncertainly associated with the registration and fusion methods must be well verified and communicated. Unfortunately, there is no standard procedure or mathematical formalism to perform this verification due to noise, distortion, and complicated anatomical situations. Objectives: This study aimed at assessing the effective contribution of MRI in Glioma radiotherapy treatment by improving the localization of target volumes and organs at risk (OARs). It is also a question to provide clinicians with some suitable metrics to evaluate the CT/MRI image registration and fusion results. Methods: Quantitative image registration and fusion evaluation were used in this study to compare Eclipse TPS tools and Elastix CT/MRI image registration fusion. Thus, Dice score coefficient (DSC), Jaccard similarity coefficient (JSC), and Hausdorff distance (HD) were found to be suitable metrics for the evaluation and comparison of the image registration and fusion methods of Eclipse TPS and Elastix. Results: The programmed tumor’s volumes (PTV) delineated on CT slices were approximately 1.38 times smaller than those delineated on CT/MRI fused images. Large differences were observed for the edema and the brainstem. It was also found that MRI considerably optimized the dose to be delivered to the optic nerve and brainstem. Conclusions: Image registration and fusion is a fundamental step for suitable and efficient Glioma treatment planning in 3D conformal radiotherapy that ensure accurate dose delivery and unnecessary OAR irradiation. MRI can provide accurate localization of targeted volumes leading to better irradiation control of Glioma tumor.

Evaluation of a novel elastic registration algorithm for spinal imaging data: A pilot clinical study

BACKGROUND

Rigid image coregistration is an established technique that allows spatial aligning. However, rigid fusion is prone to deformation of the imaged anatomies. In this work, a novel fully automated elastic image registration method is evaluated.

METHODS

Cervical CT and MRI data of 10 patients were evaluated. The MRI was acquired with the patient in neutral, flexed, and rotated head position. Vertebrawise rigid fusions were performed to transfer bony landmarks for each vertebra from the CT to the MRI space serving as a reference.

RESULTS

Elastic fusion of 3D MRI data showed the highest image registration accuracy (target registration error of 3.26 mm with 95% confidence). Further, an elastic fusion of 2D axial MRI data (<4.75 mm with 95% c.) was more reliable than for 2D sagittal sequences (<6.02 mm with 95% c.).

CONCLUSIONS

The novel method enables elastic MRI-to-CT image coregistration for cervical indications with changes of the head position.

Evaluation of Medical Image Registration Techniques Based on Nature and Domain of the Transformation

A lot of research has been done during the past 20 years in the area of medical image registration for obtaining detailed, important, and complementary information from two or more images and aligning them into a single, more informative image. Nature of the transformation and domain of the transformation are two important medical image registration techniques that deal with characters of objects (motions) in images. This article presents a detailed survey of the registration techniques that belong to both categories with detailed elaboration on their features, issues, and challenges. An investigation estimating similarity and dissimilarity measures and performance evaluation is the main objective of this work. This article also provides reference knowledge in a compact form for researchers and clinicians looking for the proper registration technique for a particular application.

Spatial Mutual Information as Similarity Measure for 3-D Brain Image Registration

Information theoretic-based similarity measures, in particular mutual information, are widely used for intermodal/intersubject 3-D brain image registration. However, conventional mutual information does not consider spatial dependency between adjacent voxels in images, thus reducing its efficacy as a similarity measure in image registration. This paper first presents a review of the existing attempts to incorporate spatial dependency into the computation of mutual information (MI). Then, a recently introduced spatially dependent similarity measure, named spatial MI, is extended to 3-D brain image registration. This extension also eliminates its artifact for translational misregistration. Finally, the effectiveness of the proposed 3-D spatial MI as a similarity measure is compared with three existing MI measures by applying controlled levels of noise degradation to 3-D simulated brain images.

The benefits of skull stripping in the normalization of clinical fMRI data

Establishing a reliable correspondence between lesioned brains and a template is challenging using current normalization techniques. The optimum procedure has not been conclusively established, and a critical dichotomy is whether to use input data sets which contain skull signal, or whether skull signal should be removed. Here we provide a first investigation into whether clinical fMRI benefits from skull stripping, based on data from a presurgical language localization task. Brain activation changes related to deskulled/not-deskulled input data are determined in the context of very recently developed (New Segment, Unified Segmentation) and standard normalization approaches. Analysis of structural and functional data demonstrates that skull stripping improves language localization in MNI space — particularly when used in combination with the New Segment normalization technique.

•

First investigation of the possible effects of skull-stripping with clinical fMRI data.

•

Comparison of standard and most recent normalization approaches.

•

Skull stripping improves language localization in MNI space.