Nonrigid image registration using conditional mutual information

MURF: Mutually Reinforcing Multi-Modal Image Registration and Fusion

Existing image fusion methods are typically limited to aligned source images and have to "tolerate" parallaxes when images are unaligned. Simultaneously, the large variances between different modalities pose a significant challenge for multi-modal image registration. This study proposes a novel method called MURF, where for the first time, image registration and fusion are mutually reinforced rather than being treated as separate issues. MURF leverages three modules: shared information extraction module (SIEM), multi-scale coarse registration module (MCRM), and fine registration and fusion module (F2M). The registration is carried out in a coarse-to-fine manner. During coarse registration, SIEM first transforms multi-modal images into mono-modal shared information to eliminate the modal variances. Then, MCRM progressively corrects the global rigid parallaxes. Subsequently, fine registration to repair local non-rigid offsets and image fusion are uniformly implemented in F2M. The fused image provides feedback to improve registration accuracy, and the improved registration result further improves the fusion result. For image fusion, rather than solely preserving the original source information in existing methods, we attempt to incorporate texture enhancement into image fusion. We test on four types of multi-modal data (RGB-IR, RGB-NIR, PET-MRI, and CT-MRI). Extensive registration and fusion results validate the superiority and universality of MURF.

Scenario-feature identification from online reviews based on BERT

Scenario endows a product with meanings. It has become the key to win the competition to design a product according to specific usage scene. Traditional scenario identification and product feature association methods have disadvantages such as subjectivity, high cost, coarse granularity, and limited scenario can be identified. In this regard, we propose a BERT-based scenario-feature identification model to effectively extract the information about users' experience and usage scene from online reviews. First, the scenario-feature identification framework is proposed to depict the whole identification process. Then, the BERT-based scene-sentence recognition model is constructed. The Skip-gram and word vector similarity methods are used to construct the scene and feature lexicon. Finally, the triad is constructed through the analysis of scene-feature co-occurrence matrix, which realizes the association of scenario and product features. This proposed model is of great practical value for product developers to better understand customer's requirements in specific scenarios. The experiments of scenario-feature identification from the reviews of Pacific Auto verifies the effectiveness of this method.

Multimodal registration of ultrasound and MR images using weighted self-similarity structure vector

PROPOSE

Multimodal registration of 2D Ultrasound (US) and 3D Magnetic Resonance (MR) for fusion navigation can improve the intraoperative detection accuracy of lesion. However, multimodal registration remains a challenge because of the poor US image quality. In the study, a weighted self-similarity structure vector (WSSV) is proposed to registrate multimodal images.

METHOD

The self-similarity structure vector utilizes the normalized distance of symmetrically located patches in the neighborhood to describe the local structure information. The texture weights are extracted using the local standard deviation to reduce the speckle interference in the US images. The multimodal similarity metric is constructed by combining a self-similarity structure vector with a texture weight map.

RESULTS

Experiments were performed on US and MR images of the liver from 88 groups of data including 8 patients and 80 simulated samples. The average target registration error was reduced from 14.91 ± 3.86 mm to 4.95 ± 2.23 mm using the WSSV-based method.

CONCLUSIONS

The experimental results show that the WSSV-based registration method could robustly align the US and MR images of the liver. With further acceleration, the registration framework can be potentially applied in time-sensitive clinical settings, such as US-MR image registration in image-guided surgery.

Measurement of bone damage caused by quasi-static compressive loading-unloading to explore dental implants stability: Simultaneous use of in-vitro tests, μ-CT images, and digital volume correlation

Primary stability of dental implants is the initial mechanical engagement of the implant with its adjacent bone. Implantation and the subsequent loading may cause mechanical damage in the peripheral bone, which ultimately reduces the stability of the implant. This study aimed at evaluating primary stability of dental implants through applying stepwise compressive displacement-controlled, loading-unloading cycles to obtain overall stiffness and dissipated energy of the bone-implant structure; and quantifying induced plastic strains in surrounding bone using digital volume correlation (DVC) method, through comparing μCT images in different loading steps. To this end, dental implants were inserted into the cylindrical trabecular bones, then the bone-implant structure was undergone step-wise loading-unloading cycles, and μCT images were taken in some particular steps, then comparison was made between undeformed and deformed configurations using DVC to quantify plastic strain within the trabecular bone. Comparing stiffness reduction and dissipated energy values in different loading steps, obtained from the force-displacement curve in each loading step, revealed that the maximum displacement of 0.16 mm can be deemed as a safe threshold above which damages in peri-implant bone started to increase considerably (p < 0.05). In addition, it was found here that peri-implant bone strain linearly increased with decreasing bone-implant stiffness (p < 0.05). Moreover, strain concentration in peri-implant bone region showed that the plastic strain in trabecular bone spread up to a distance of about 2.5 mm away from the implant surface. Research of this kind can be used to optimize the design of dental implants, with the ultimate goal of improving their stability, also to validate in-silico models, e.g., micro-finite element models, which can help gain a deeper understanding of bone-implant construct behavior.

Registration of Multisensor Images through a Conditional Generative Adversarial Network and a Correlation-Type Similarity Measure

The automatic registration of multisensor remote sensing images is a highly challenging task due to the inherently different physical, statistical, and textural characteristics of the input data. Information-theoretic measures are often used to favor comparing local intensity distributions in the images. In this paper, a novel method based on the combination of a deep learning architecture and a correlation-type area-based functional is proposed for the registration of a multisensor pair of images, including an optical image and a synthetic aperture radar (SAR) image. The method makes use of a conditional generative adversarial network (cGAN) in order to address image-to-image translation across the optical and SAR data sources. Then, once the optical and SAR data are brought to a common domain, an area-based ℓ2 similarity measure is used together with the COBYLA constrained maximization algorithm for registration purposes. While correlation-type functionals are usually ineffective in the application to multisensor registration, exploiting the image-to-image translation capabilities of cGAN architectures allows moving the complexity of the comparison to the domain adaptation step, thus enabling the use of a simple ℓ2 similarity measure, favoring high computational efficiency, and opening the possibility to process a large amount of data at runtime. Experiments with multispectral and panchromatic optical data combined with SAR images suggest the effectiveness of this strategy and the capability of the proposed method to achieve more accurate registration as compared to state-of-the-art approaches.

Application of Multimodal Fusion Technology in Image Analysis of Pretreatment Examination of Patients with Spinal Injury

As one of the most common imaging screening techniques for spinal injuries, MRI is of great significance for the pretreatment examination of patients with spinal injuries. With rapid iterative update of imaging technology, imaging techniques such as diffusion weighted magnetic resonance imaging (DWI), dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI), and magnetic resonance spectroscopy are frequently used in the clinical diagnosis of spinal injuries. Multimodal medical image fusion technology can obtain richer lesion information by combining medical images in multiple modalities. Aiming at the two modalities of DCE-MRI and DWI images under MRI images of spinal injuries, by fusing the image data under the two modalities, more abundant lesion information can be obtained to diagnose spinal injuries. The research content includes the following: (1) A registration study based on DCE-MRI and DWI image data. To improve registration accuracy, a registration method is used, and VGG-16 network structure is selected as the basic registration network structure. An iterative VGG-16 network framework is proposed to realize the registration of DWI and DCE-MRI images. The experimental results show that the iterative VGG-16 network structure is more suitable for the registration of DWI and DCE-MRI image data. (2) Based on the fusion research of DCE-MRI and DWI image data. For the registered DCE-MRI and DWI images, this paper uses a fusion method combining feature level and decision level to classify spine images. The simple classifier decision tree, SVM, and KNN were used to predict the damage diagnosis classification of DCE-MRI and DWI images, respectively. By comparing and analyzing the classification results of the experiments, the performance of multimodal image fusion in the auxiliary diagnosis of spinal injuries was evaluated.

Long-range medical image registration through generalized mutual information (GMI): towards a fully automatic volumetric alignment

Objective. Mutual information (MI) is consolidated as a robust similarity metric often used for medical image registration. Although MI provides a robust registration, it usually fails when the transform needed to register an image is too large due to MI local minima traps. This paper proposes and evaluates Generalized MI (GMI), using Tsallis entropy, to improve affine registration. Approach. We assessed the GMI metric output space using separable affine transforms to seek a better gradient space. The range used was 150 mm for translations, 360° for rotations, [0.5, 2] for scaling, and [−1, 1] for skewness. The data were evaluated using 3D visualization of gradient and contour curves. A simulated gradient descent algorithm was also used to calculate the registration capability. The improvements detected were then tested through Monte Carlo simulation of actual registrations with brain T1 and T2 MRI from the HCP dataset. Main results. Results show significantly prolonged registration ranges, without local minima in the metric space, with a registration capability of 100% for translations, 88.2% for rotations, 100% for scaling and 100% for skewness. Tsallis entropy obtained 99.75% success in the Monte Carlo simulation of 2000 translation registrations with 1113 double randomized subjects T1 and T2 brain MRI against 56.5% success for the Shannon entropy. Significance. Tsallis entropy can improve brain MRI MI affine registration with long-range translation registration, lower-cost interpolation, and faster registrations through a better gradient space.

User Behaviors and User-Generated Content in Chinese Online Health Communities: Comparative Study

BACKGROUND

Online health communities (OHCs) have increasingly gained traction with patients, caregivers, and supporters globally. Chinese OHCs are no exception. However, user-generated content (UGC) and the associated user behaviors in Chinese OHCs are largely underexplored and rarely analyzed systematically, forfeiting valuable opportunities for optimizing treatment design and care delivery with insights gained from OHCs.

OBJECTIVE

This study aimed to reveal both the shared and distinct characteristics of 2 popular OHCs in China by systematically and comprehensively analyzing their UGC and the associated user behaviors.

METHODS

We concentrated on studying the lung cancer forum (LCF) and breast cancer forum (BCF) on Mijian, and the diabetes consultation forum (DCF) on Sweet Home, because of the importance of the 3 diseases among Chinese patients and their prevalence on Chinese OHCs in general. Our analysis explored the key user activities, small-world effect, and scale-free characteristics of each social network. We examined the UGC of these forums comprehensively and adopted the weighted knowledge network technique to discover salient topics and latent relations among these topics on each forum. Finally, we discussed the public health implications of our analysis findings.

RESULTS

Our analysis showed that the number of reads per thread on each forum followed gamma distribution (H_L=0, H_B=0, and H_D=0); the number of replies on each forum followed exponential distribution (adjusted R_L²=0.946, adjusted R_B²=0.958, and adjusted R_D²=0.971); and the number of threads a user is involved with (adjusted R_L²=0.978, adjusted R_B²=0.964, and adjusted R_D²=0.970), the number of followers of a user (adjusted R_L²=0.989, adjusted R_B²=0.962, and adjusted R_D²=0.990), and a user's degrees (adjusted R_L²=0.997, adjusted R_B²=0.994, and adjusted R_D²=0.968) all followed power-law distribution. The study further revealed that users are generally more active during weekdays, as commonly witnessed in all 3 forums. In particular, the LCF and DCF exhibited high temporal similarity (ρ=0.927; P<.001) in terms of the relative thread posting frequencies during each hour of the day. Besides, the study showed that all 3 forums exhibited the small-world effect (mean σ_L=517.15, mean σ_B=275.23, and mean σ_D=525.18) and scale-free characteristics, while the global clustering coefficients were lower than those of counterpart international OHCs. The study also discovered several hot topics commonly shared among the 3 disease forums, such as disease treatment, disease examination, and diagnosis. In particular, the study found that after the outbreak of COVID-19, users on the LCF and BCF were much more likely to bring up COVID-19-related issues while discussing their medical issues.

CONCLUSIONS

UGC and related online user behaviors in Chinese OHCs can be leveraged as important sources of information to gain insights regarding individual and population health conditions. Effective and timely mining and utilization of such content can continuously provide valuable firsthand clues for enhancing the situational awareness of health providers and policymakers.

Real-time multimodal image registration with partial intraoperative point-set data

We present Free Point Transformer (FPT) - a deep neural network architecture for non-rigid point-set registration. Consisting of two modules, a global feature extraction module and a point transformation module, FPT does not assume explicit constraints based on point vicinity, thereby overcoming a common requirement of previous learning-based point-set registration methods. FPT is designed to accept unordered and unstructured point-sets with a variable number of points and uses a "model-free" approach without heuristic constraints. Training FPT is flexible and involves minimizing an intuitive unsupervised loss function, but supervised, semi-supervised, and partially- or weakly-supervised training are also supported. This flexibility makes FPT amenable to multimodal image registration problems where the ground-truth deformations are difficult or impossible to measure. In this paper, we demonstrate the application of FPT to non-rigid registration of prostate magnetic resonance (MR) imaging and sparsely-sampled transrectal ultrasound (TRUS) images. The registration errors were 4.71 mm and 4.81 mm for complete TRUS imaging and sparsely-sampled TRUS imaging, respectively. The results indicate superior accuracy to the alternative rigid and non-rigid registration algorithms tested and substantially lower computation time. The rapid inference possible with FPT makes it particularly suitable for applications where real-time registration is beneficial.

miRID: Multi-Modal Image Registration Using Modality-Independent and Rotation-Invariant Descriptor

Axiomatically, symmetry is a fundamental property of mathematical functions defining similarity measures, where similarity measures are important tools in many areas of computer science, including machine learning and image processing. In this paper, we investigate a new technique to measure the similarity between two images, a fixed image and a moving image, in multi-modal image registration (MIR). MIR in medical image processing is essential and useful in diagnosis and therapy guidance, but still a very challenging task due to the lack of robustness against the rotational variance in the image transformation process. Our investigation leads to a novel, local self-similarity descriptor, called the modality-independent and rotation-invariant descriptor (miRID). By relying on the mean of the intensity values, an miRID is simply computable and can effectively handle the complicated intensity relationship between multi-modal images. Moreover, it can also overcome the problem of rotational variance by sorting the numerical values, each of which is the absolute difference between each pixel’s intensity and the mean of all pixel intensities within a patch of the image. The experimental result shows that our method outperforms others in both multi-modal rigid and non-rigid image registrations.

Biomechanically constrained non-rigid MR-TRUS prostate registration using deep learning based 3D point cloud matching

A non-rigid MR-TRUS image registration framework is proposed for prostate interventions. The registration framework consists of a convolutional neural networks (CNN) for MR prostate segmentation, a CNN for TRUS prostate segmentation and a point-cloud based network for rapid 3D point cloud matching. Volumetric prostate point clouds were generated from the segmented prostate masks using tetrahedron meshing. The point cloud matching network was trained using deformation field that was generated by finite element analysis. Therefore, the network implicitly models the underlying biomechanical constraint when performing point cloud matching. A total of 50 patients' datasets were used for the network training and testing. Alignment of prostate shapes after registration was evaluated using three metrics including Dice similarity coefficient (DSC), mean surface distance (MSD) and Hausdorff distance (HD). Internal point-to-point registration accuracy was assessed using target registration error (TRE). Jacobian determinant and strain tensors of the predicted deformation field were calculated to analyze the physical fidelity of the deformation field. On average, the mean and standard deviation were 0.94±0.02, 0.90±0.23 mm, 2.96±1.00 mm and 1.57±0.77 mm for DSC, MSD, HD and TRE, respectively. Robustness of our method to point cloud noise was evaluated by adding different levels of noise to the query point clouds. Our results demonstrated that the proposed method could rapidly perform MR-TRUS image registration with good registration accuracy and robustness.

Image Matching from Handcrafted to Deep Features: A Survey

As a fundamental and critical task in various visual applications, image matching can identify then correspond the same or similar structure/content from two or more images. Over the past decades, growing amount and diversity of methods have been proposed for image matching, particularly with the development of deep learning techniques over the recent years. However, it may leave several open questions about which method would be a suitable choice for specific applications with respect to different scenarios and task requirements and how to design better image matching methods with superior performance in accuracy, robustness and efficiency. This encourages us to conduct a comprehensive and systematic review and analysis for those classical and latest techniques. Following the feature-based image matching pipeline, we first introduce feature detection, description, and matching techniques from handcrafted methods to trainable ones and provide an analysis of the development of these methods in theory and practice. Secondly, we briefly introduce several typical image matching-based applications for a comprehensive understanding of the significance of image matching. In addition, we also provide a comprehensive and objective comparison of these classical and latest techniques through extensive experiments on representative datasets. Finally, we conclude with the current status of image matching technologies and deliver insightful discussions and prospects for future works. This survey can serve as a reference for (but not limited to) researchers and engineers in image matching and related fields.

Multimodal affine registration for ICGA and MCSL fundus images of high myopia

The registration between indocyanine green angiography (ICGA) and multi-color scanning laser (MCSL) imaging fundus images is vital for the joint linear lesion segmentation in ICGA and MCSL and the evaluation whether MCSL can replace ICGA as a non-invasive diagnosis for linear lesion. To our best knowledge, there are no studies focusing on the image registration between these two modalities. In this paper, we propose a framework based on convolutional neural networks for the multimodal affine registration between ICGA and MCSL images, which contains two parts: coarse registration stage and fine registration stage. In the coarse registration stage, the optic disc is segmented and its centroid is used as a matching point to perform coarse registration. The fine registration stage regresses affine parameters directly using jointly supervised and weakly-supervised loss function. Experimental results show the effectiveness of the proposed method, which lays a sound foundation for further evaluation of non-invasive diagnosis of linear lesion based on MCSL.

To Align Multimodal Lumbar Spine Images via Bending Energy Constrained Normalized Mutual Information

To align multimodal images is important for information fusion, clinical diagnosis, treatment planning, and delivery, while few methods have been dedicated to matching computerized tomography (CT) and magnetic resonance (MR) images of lumbar spine. This study proposes a coarse-to-fine registration framework to address this issue. Firstly, a pair of CT-MR images are rigidly aligned for global positioning. Then, a bending energy term is penalized into the normalized mutual information for the local deformation of soft tissues. In the end, the framework is validated on 40 pairs of CT-MR images from our in-house collection and 15 image pairs from the SpineWeb database. Experimental results show high overlapping ratio (in-house collection, vertebrae 0.97 ± 0.02, blood vessel 0.88 ± 0.07; SpineWeb, vertebrae 0.95 ± 0.03, blood vessel 0.93 ± 0.10) and low target registration error (in-house collection, ≤2.00 ± 0.62 mm; SpineWeb, ≤2.37 ± 0.76 mm) are achieved. The proposed framework concerns both the incompressibility of bone structures and the nonrigid deformation of soft tissues. It enables accurate CT-MR registration of lumbar spine images and facilitates image fusion, spine disease diagnosis, and interventional treatment delivery.

Real-time and multimodality image-guided intelligent HIFU therapy for uterine fibroid

Rationale: High-intensity focused ultrasound (HIFU) therapy represents a noninvasive surgical approach to treat uterine fibroids. The operation of HIFU therapy relies on the information provided by medical images. In current HIFU therapy, all operations such as positioning of the lesion in magnetic resonance (MR) and ultrasound (US) images are manually performed by specifically trained doctors. Manual processing is an important limitation of the efficiency of HIFU therapy. In this paper, we aim to provide an automatic and accurate image guidance system, intelligent diagnosis, and treatment strategy for HIFU therapy by combining multimodality information.

Methods: In intelligent HIFU therapy, medical information and treatment strategy are automatically processed and generated by a real-time image guidance system. The system comprises a novel multistage deep convolutional neural network for preoperative diagnosis and a nonrigid US lesion tracking procedure for HIFU intraoperative image-assisted treatment. In the process of intelligent therapy, the treatment area is determined from the autogenerated lesion area. Based on the autodetected treatment area, the HIFU foci are distributed automatically according to the treatment strategy. Moreover, an image-based unexpected movement warning and other physiological monitoring are used during the intelligent treatment procedure for safety assurance.

Results: In the experiment, we integrated the intelligent treatment system on a commercial HIFU treatment device, and eight clinical experiments were performed. In the clinical validation, eight randomly selected clinical cases were used to verify the feasibility of the system. The results of the quantitative experiment indicated that our intelligent system met the HIFU clinical tracking accuracy and speed requirements. Moreover, the results of simulated repeated experiments confirmed that the autodistributed HIFU focus reached the level of intermediate clinical doctors. Operations performed by junior- or middle-level operators with the assistance of the proposed system can reach the level of operation performed by senior doctors. Various experiments prove that our proposed intelligent HIFU therapy process is feasible for treating common uterine fibroid cases.

Conclusion: We propose an intelligent HIFU therapy for uterine fibroid which integrates multiple medical information processing procedures. The experiment results demonstrated that the proposed procedures and methods can achieve monitored and automatic HIFU diagnosis and treatment. This research provides a possibility for intelligent and automatic noninvasive therapy for uterine fibroid.

Multiphoton intravital microscopy in small animals: motion artefact challenges and technical solutions

Since its invention 29 years ago, two‐photon laser‐scanning microscopy has evolved from a promising imaging technique, to an established widely available imaging modality used throughout the biomedical research community. The establishment of two‐photon microscopy as the preferred method for imaging fluorescently labelled cells and structures in living animals can be attributed to the biophysical mechanism by which the generation of fluorescence is accomplished. The use of powerful lasers capable of delivering infrared light pulses within femtosecond intervals, facilitates the nonlinear excitation of fluorescent molecules only at the focal plane and determines by objective lens position. This offers numerous benefits for studies of biological samples at high spatial and temporal resolutions with limited photo‐damage and superior tissue penetration. Indeed, these attributes have established two‐photon microscopy as the ideal method for live‐animal imaging in several areas of biology and have led to a whole new field of study dedicated to imaging biological phenomena in intact tissues and living organisms. However, despite its appealing features, two‐photon intravital microscopy is inherently limited by tissue motion from heartbeat, respiratory cycles, peristalsis, muscle/vascular tone and physiological functions that change tissue geometry. Because these movements impede temporal and spatial resolution, they must be properly addressed to harness the full potential of two‐photon intravital microscopy and enable accurate data analysis and interpretation. In addition, the sources and features of these motion artefacts are varied, sometimes unpredictable and unique to specific organs and multiple complex strategies have previously been devised to address them. This review will discuss these motion artefacts requirement and technical solutions for their correction and after intravital two‐photon microscopy.

Logarithmic Fuzzy Entropy Function for Similarity Measurement in Multimodal Medical Images Registration

Multimodal medical images are useful for observing tissue structure clearly in clinical practice. To integrate multimodal information, multimodal registration is significant. The entropy-based registration applies a structure descriptor set to replace the original multimodal image and compute similarity to express the correlation of images. The accuracy and converging rate of the registration depend on this set. We propose a new method, logarithmic fuzzy entropy function, to compute the descriptor set. It is obvious that the proposed method can increase the upper bound value from log(r) to log(r) + ∆(r) so that a more representative structural descriptor set is formed. The experiment results show that our method has faster converging rate and wider quantified range in multimodal medical images registration.

Morphological active contour without edge-based model for real-time and non-rigid uterine fibroid tracking in HIFU treatment

High-intensity focused ultrasound (HIFU) therapy represents an image-guided and non-invasive surgical approach to treat uterine fibroid. During the HIFU operation, it is challenging to obtain the real-time and accurate lesion contour automatically in ultrasound (US) video. The current intraoperative image processing is completed manually or semi-automatic. In this Letter, the authors propose a morphological active contour without an edge-based model to obtain accurate real-time and non-rigid US lesion contour. Firstly, a targeted image pre-processing procedure is applied to reduce the influence of inadequate image quality. Then, an improved morphological contour detection method with a customised morphological kernel is harnessed to solve the low signal-to-noise ratio of HIFU US images and obtain an accurate non-rigid lesion contour. A more reasonable lesion tracking procedure is proposed to improve tracking accuracy especially in the case of large displacement and incomplete lesion area. The entire framework is accelerated by the GPU to achieve a high frame rate. Finally, a non-rigid, real-time and accurate lesion contouring for intraoperative US video is provided to the doctor. The proposed procedure could reach a speed of more than 30 frames per second in general computer and a Dice similarity coefficient of 90.67% and Intersection over Union of 90.14%.

Locally Adaptive Total p-Variation Regularization for Non-Rigid Image Registration With Sliding Motion

Due to the complicated thoracic movements which contain both sliding motion occurring at lung surfaces and smooth motion within individual organs, respiratory estimation is still an intrinsically challenging task. In this paper, we propose a novel regularization term called locally adaptive total p-variation (LaTpV) and embed it into a parametric registration framework to accurately recover lung motion. LaTpV originates from a modified L_p-norm constraint (1 < p < 2), where a prior distribution of p modeled by the Dirac-shaped function is constructed to specifically assign different values to voxels. LaTpV adaptively balances the smoothness and discontinuity of the displacement field to encourage an expected sliding interface. Additionally, we also analytically deduce the gradient of the cost function with respect to transformation parameters. To validate the performance of LaTpV, we not only test it on two mono-modal databases including synthetic images and pulmonary computed tomography (CT) images, but also on a more difficult thoracic CT and positron emission tomography (PET) dataset for the first time. For all experiments, both the quantitative and qualitative results indicate that LaTpV significantly surpasses some existing regularizers such as bending energy and parametric total variation. The proposed LaTpV based registration scheme might be more superior for sliding motion correction and more potential for clinical applications such as the diagnosis of pleural mesothelioma and the adjustment of radiotherapy plans.

High-Resolution Optical Remote Sensing Image Registration via Reweighted Random Walk based Hyper-Graph Matching

High-resolution optical remote sensing image registration is still a challenging task due to non-linearity in the intensity differences and geometric distortion. In this paper, an efficient method utilizing a hyper-graph matching algorithm is proposed, which can simultaneously use the high-order structure information and radiometric information, to obtain thousands of feature point pairs for accurate image registration. The method mainly consists of the following steps: firstly, initial matching by Uniform Robust Scale-Invariant Feature Transform (UR-SIFT) is carried out in the highest pyramid image level to derive the approximate geometric relationship between the images; secondly, two-stage point matching is performed to find the matches, that is, a rotation and scale invariant area-based matching method is used to derive matching candidates for each feature point and an efficient hyper-graph matching algorithm is applied to find the best match for each feature point; thirdly, a local quadratic polynomial constraint framework is used to eliminate match outliers; finally, the above process is iterated until finishing the matching in the original image. Then, the obtained correspondences are used to perform the image registration. The effectiveness of the proposed method is tested with six pairs of high-resolution optical images, covering different landscape types—such as mountain area, urban, suburb, and flat land—and registration accuracy of sub-pixel level is obtained. The experiments show that the proposed method outperforms the conventional matching algorithms such as SURF, AKAZE, ORB, BRISK, and FAST in terms of total number of correct matches and matching precision.

Fully experiment-based evaluation of few digital volume correlation techniques

Digital Volume Correlation (DVC) is a powerful set of techniques used to compute the local shifts of 3D images obtained, for instance, in tomographic experiments. It is utilized to analyze the geometric changes of the investigated object as well as to correct the corresponding image misalignments for further analysis. It can therefore be used to evaluate the local density changes of the same regions of the inspected specimens, which might be shifted between measurements. In recent years, various approaches and corresponding pieces of software were introduced. Accuracies for the computed shift vectors of up to about 1‰ of a single voxel size have been reported. These results, however, were based either on synthetic datasets or on an unrealistic setup. In this work, we propose two simple methods to evaluate the accuracy of DVC-techniques using more realistic input data and apply them to several DVC programs. We test these methods on three materials (tuff, sandstone, and concrete) that show different contrast and structural features.

Non-Rigid Multi-Modal 3D Medical Image Registration Based on Foveated Modality Independent Neighborhood Descriptor

The non-rigid multi-modal three-dimensional (3D) medical image registration is highly challenging due to the difficulty in the construction of similarity measure and the solution of non-rigid transformation parameters. A novel structural representation based registration method is proposed to address these problems. Firstly, an improved modality independent neighborhood descriptor (MIND) that is based on the foveated nonlocal self-similarity is designed for the effective structural representations of 3D medical images to transform multi-modal image registration into mono-modal one. The sum of absolute differences between structural representations is computed as the similarity measure. Subsequently, the foveated MIND based spatial constraint is introduced into the Markov random field (MRF) optimization to reduce the number of transformation parameters and restrict the calculation of the energy function in the image region involving non-rigid deformation. Finally, the accurate and efficient 3D medical image registration is realized by minimizing the similarity measure based MRF energy function. Extensive experiments on 3D positron emission tomography (PET), computed tomography (CT), T1, T2, and (proton density) PD weighted magnetic resonance (MR) images with synthetic deformation demonstrate that the proposed method has higher computational efficiency and registration accuracy in terms of target registration error (TRE) than the registration methods that are based on the hybrid L-BFGS-B and cat swarm optimization (HLCSO), the sum of squared differences on entropy images, the MIND, and the self-similarity context (SSC) descriptor, except that it provides slightly bigger TRE than the HLCSO for CT-PET image registration. Experiments on real MR and ultrasound images with unknown deformation have also be done to demonstrate the practicality and superiority of the proposed method.

Nonrigid registration of medical image based on adaptive local structure tensor and normalized mutual information

Nonrigid registration of medical images is especially critical in clinical treatment. Mutual information is a popular similarity measure for medical image registration; however, only the intensity statistical characteristics of the global consistency of image are considered in MI, and the spatial information is ignored. In this paper, a novel intensity-based similarity measure combining normalized mutual information with spatial information for nonrigid medical image registration is proposed. The different parameters of Gaussian filtering are defined according to the regional variance, the adaptive Gaussian filtering is introduced into the local structure tensor. Then, the obtained adaptive local structure tensor is used to extract the spatial information and define the weighting function. Finally, normalized mutual information is distributed to each pixel, and the discrete normalized mutual information is multiplied with a weighting term to obtain a new measure. The novel measure fully considers the spatial information of the image neighborhood, gives the location of the strong spatial information a larger weight, and the registration of the strong gradient regions has a priority over the small gradient regions. The simulated brain image with single-modality and multimodality are used for registration validation experiments. The results show that the new similarity measure improves the registration accuracy and robustness compared with the classical registration algorithm, reduces the risk of falling into local extremes during the registration process.

Image synthesis-based multi-modal image registration framework by using deep fully convolutional networks

Multi-modal image registration has significant meanings in clinical diagnosis, treatment planning, and image-guided surgery. Since different modalities exhibit different characteristics, finding a fast and accurate correspondence between images of different modalities is still a challenge. In this paper, we propose an image synthesis-based multi-modal registration framework. Image synthesis is performed by a ten-layer fully convolutional network (FCN). The network is composed of 10 convolutional layers combined with batch normalization (BN) and rectified linear unit (ReLU), which can be trained to learn an end-to-end mapping from one modality to the other. After the cross-modality image synthesis, multi-modal registration can be transformed into mono-modal registration. The mono-modal registration can be solved by methods with lower computational complexity, such as sum of squared differences (SSD). We tested our method in T1-weighted vs T2-weighted, T1-weighted vs PD, and T2-weighted vs PD image registrations with BrainWeb phantom data and IXI real patients' data. The result shows that our framework can achieve higher registration accuracy than the state-of-the-art multi-modal image registration methods, such as local mutual information (LMI) and α-mutual information (α-MI). The average registration errors of our method in experiment with IXI real patients' data were 1.19, 2.23, and 1.57 compared to 1.53, 2.60, and 2.36 of LMI and 1.34, 2.39, and 1.76 of α-MI in T2-weighted vs PD, T1-weighted vs PD, and T1-weighted vs T2-weighted image registration, respectively. In this paper, we propose an image synthesis-based multi-modal image registration framework. A deep FCN model is developed to perform image synthesis for this framework, which can capture the complex nonlinear relationship between different modalities and discover complex structural representations automatically by a large number of trainable mapping and parameters and perform accurate image synthesis. The framework combined with the deep FCN model and mono-modal registration methods (SSD) can achieve fast and robust results in multi-modal medical image registration. Graphical abstract The workflow of proposed multi-modal image registration framework.

A vessel segmentation method for serialized cerebralvascular DSA images based on spatial feature point set of rotating coordinate system

Cerebrovascular pathology is one of the main fatal diseases which seriously affect the human's health. Extracting the accurate image of cerebral vascular tissue is the key of clinical diagnosis. However, the motion artifacts in DSA images seriously affected the quality of vascular subtraction image. In this paper, an automatic and accurate segmentation method is presented to extract the vascular region in the live image of brain. Firstly, a coarse registration for the live image and the mask image is implemented. And then, the SIFT algorithm is utilized to detect geometrical feature points in the serialized subtraction images. After that, a spatial model of rotating coordinate system and a calculative strategy of contextual information are designed to eliminate the error feature points. Finally, based on a dynamic threshold method, the blood vessel image can be obtained by region growing. The context information in the adjacent subtraction images is fully used. The experimental result shows that the segmented cerebral vascular image is satisfactory. This method can provide accurate vessel image data for the clinical operation based on DSA interventional therapy.

Nonrigid Image Registration Using Spatially Region-Weighted Correlation Ratio and GPU-Acceleration

OBJECTIVE

Nonrigid image registration with high accuracy and efficiency remains a challenging task for medical image analysis. In this paper, we present the spatially region-weighted correlation ratio (SRWCR) as a novel similarity measure to improve the registration performance.

METHODS

SRWCR is rigorously deduced from a three-dimension joint probability density function combining the intensity channels with an extra spatial information channel. SRWCR estimates the optimal functional dependence between the intensities for each spatial bin, in which the spatial distribution modeled by a cubic B-spline function is used to differentiate the contribution of voxels. We also analytically derive the gradient of SRWCR with respect to the transformation parameters and optimize it using a quasi-Newton approach. Furthermore, we propose a GPU-based parallel mechanism to accelerate the computation of SRWCR and its derivatives.

RESULTS

The experiments on synthetic images, public four-dimensional thoracic computed tomography (CT) dataset, retinal optical coherence tomography data, and clinical CT and positron emission tomography images confirm that SRWCR significantly outperforms some state-of-the-art techniques such as spatially encoded mutual information and Robust PaTch-based cOrrelation Ration.

CONCLUSION

This study demonstrates the advantages of SRWCR in tackling the practical difficulties due to distinct intensity changes, serious speckle noise, or different imaging modalities.

SIGNIFICANCE

The proposed registration framework might be more reliable to correct the nonrigid deformations and more potential for clinical applications.

PCANet-Based Structural Representation for Nonrigid Multimodal Medical Image Registration

Nonrigid multimodal image registration remains a challenging task in medical image processing and analysis. The structural representation (SR)-based registration methods have attracted much attention recently. However, the existing SR methods cannot provide satisfactory registration accuracy due to the utilization of hand-designed features for structural representation. To address this problem, the structural representation method based on the improved version of the simple deep learning network named PCANet is proposed for medical image registration. In the proposed method, PCANet is firstly trained on numerous medical images to learn convolution kernels for this network. Then, a pair of input medical images to be registered is processed by the learned PCANet. The features extracted by various layers in the PCANet are fused to produce multilevel features. The structural representation images are constructed for two input images based on nonlinear transformation of these multilevel features. The Euclidean distance between structural representation images is calculated and used as the similarity metrics. The objective function defined by the similarity metrics is optimized by L-BFGS method to obtain parameters of the free-form deformation (FFD) model. Extensive experiments on simulated and real multimodal image datasets show that compared with the state-of-the-art registration methods, such as modality-independent neighborhood descriptor (MIND), normalized mutual information (NMI), Weber local descriptor (WLD), and the sum of squared differences on entropy images (ESSD), the proposed method provides better registration performance in terms of target registration error (TRE) and subjective human vision.

Region-adaptive Deformable Registration of CT/MRI Pelvic Images via Learning-based Image Synthesis

Registration of pelvic CT and MRI is highly desired as it can facilitate effective fusion of two modalities for prostate cancer radiation therapy, i.e., using CT for dose planning and MRI for accurate organ delineation. However, due to the large inter-modality appearance gaps and the high shape/appearance variations of pelvic organs, the pelvic CT/MRI registration is highly challenging. In this paper, we propose a region-adaptive deformable registration method for multi-modal pelvic image registration. Specifically, to handle the large appearance gaps, we first perform both CT-to-MRI and MRI-to-CT image synthesis by multi-target regression forest (MT-RF). Then, to use the complementary anatomical information in the two modalities for steering the registration, we select key points automatically from both modalities and use them together for guiding correspondence detection in the region-adaptive fashion. That is, we mainly use CT to establish correspondences for bone regions, and use MRI to establish correspondences for soft tissue regions. The number of key points is increased gradually during the registration, to hierarchically guide the symmetric estimation of the deformation fields. Experiments for both intra-subject and inter-subject deformable registration show improved performances compared with state-of-the-art multi-modal registration methods, which demonstrate the potentials of our method to be applied for the routine prostate cancer radiation therapy.

Image Registration Based on Low Rank Matrix: Rank-Regularized SSD

Similarity measure is a main core of image registration algorithms. Spatially varying intensity distortion is an important challenge, which affects the performance of similarity measures. Correlation among the pixels is the main characteristic of this distortion. Similarity measures such as sum-of-squared-differences (SSD) and mutual information ignore this correlation; hence, perfect registration cannot be achieved in the presence of this distortion. In this paper, we model this correlation with the aid of the low rank matrix theory. Based on this model, we compensate this distortion analytically and introduce rank-regularized SSD (RRSSD). This new similarity measure is a modified SSD based on singular values of difference image in mono-modal imaging. In fact, image registration and distortion correction are performed simultaneously in the proposed model. Based on our experiments, the RRSSD similarity measure achieves clinically acceptable registration results, and outperforms other state-of-the-art similarity measures, such as the well-known method of residual complexity.

A Review on Medical Image Registration as an Optimization Problem

Objective:

In the course of clinical treatment, several medical media are required by a phy-sician in order to provide accurate and complete information about a patient. Medical image registra-tion techniques can provide a richer diagnosis and treatment information to doctors and to provide a comprehensive reference source for the researchers involved in image registration as an optimization problem.

Methods:

The essence of image registration is associating two or more different images spatial asso-ciation, and getting the translation of their spatial relationship. For medical image registration, its pro-cess is not absolute. Its core purpose is finding the conversion relationship between different images.

Result:

The major step of image registration includes the change of geometrical dimensions, and change of the image of the combination, image similarity measure, iterative optimization and interpo-lation process.

Conclusion:

The contribution of this review is sort of related image registration research methods, can provide a brief reference for researchers about image registration.

Fast and robust multimodal image registration using a local derivative pattern

PURPOSE

Deformable multimodal image registration, which can benefit radiotherapy and image guided surgery by providing complementary information, remains a challenging task in the medical image analysis field due to the difficulty of defining a proper similarity measure. This article presents a novel, robust and fast binary descriptor, the discriminative local derivative pattern (dLDP), which is able to encode images of different modalities into similar image representations.

METHODS

dLDP calculates a binary string for each voxel according to the pattern of intensity derivatives in its neighborhood. The descriptor similarity is evaluated using the Hamming distance, which can be efficiently computed, instead of conventional L1 or L2 norms. For the first time, we validated the effectiveness and feasibility of the local derivative pattern for multimodal deformable image registration with several multi-modal registration applications.

RESULTS

dLDP was compared with three state-of-the-art methods in artificial image and clinical settings. In the experiments of deformable registration between different magnetic resonance imaging (MRI) modalities from BrainWeb, between computed tomography and MRI images from patient data, and between MRI and ultrasound images from BITE database, we show our method outperforms localized mutual information and entropy images in terms of both accuracy and time efficiency. We have further validated dLDP for the deformable registration of preoperative MRI and three-dimensional intraoperative ultrasound images. Our results indicate that dLDP reduces the average mean target registration error from 4.12 mm to 2.30 mm. This accuracy is statistically equivalent to the accuracy of the state-of-the-art methods in the study; however, in terms of computational complexity, our method significantly outperforms other methods and is even comparable to the sum of the absolute difference.

CONCLUSIONS

The results reveal that dLDP can achieve superior performance regarding both accuracy and time efficiency in general multimodal image registration. In addition, dLDP also indicates the potential for clinical ultrasound guided intervention.

Cross contrast multi-channel image registration using image synthesis for MR brain images

Multi-modal deformable registration is important for many medical image analysis tasks such as atlas alignment, image fusion, and distortion correction. Whereas a conventional method would register images with different modalities using modality independent features or information theoretic metrics such as mutual information, this paper presents a new framework that addresses the problem using a two-channel registration algorithm capable of using mono-modal similarity measures such as sum of squared differences or cross-correlation. To make it possible to use these same-modality measures, image synthesis is used to create proxy images for the opposite modality as well as intensity-normalized images from each of the two available images. The new deformable registration framework was evaluated by performing intra-subject deformation recovery, intra-subject boundary alignment, and inter-subject label transfer experiments using multi-contrast magnetic resonance brain imaging data. Three different multi-channel registration algorithms were evaluated, revealing that the framework is robust to the multi-channel deformable registration algorithm that is used. With a single exception, all results demonstrated improvements when compared against single channel registrations using the same algorithm with mutual information.

A Neonatal Bimodal MR-CT Head Template

Neonatal MR templates are appropriate for brain structural analysis and spatial normalization. However, they do not provide the essential accurate details of cranial bones and fontanels-sutures. Distinctly, CT images provide the best contrast for bone definition and fontanels-sutures. In this paper, we present, for the first time, an approach to create a fully registered bimodal MR-CT head template for neonates with a gestational age of 39 to 42 weeks. Such a template is essential for structural and functional brain studies, which require precise geometry of the head including cranial bones and fontanels-sutures. Due to the special characteristics of the problem (which requires inter-subject inter-modality registration), a two-step intensity-based registration method is proposed to globally and locally align CT images with an available MR template. By applying groupwise registration, the new neonatal CT template is then created in full alignment with the MR template to build a bimodal MR-CT template. The mutual information value between the CT and the MR template is 1.17 which shows their perfect correspondence in the bimodal template. Moreover, the average mutual information value between normalized images and the CT template proposed in this study is 1.24±0.07. Comparing this value with the one reported in a previously published approach (0.63±0.07) demonstrates the better generalization properties of the new created template and the superiority of the proposed method for the creation of CT template in the standard space provided by MR neonatal head template. The neonatal bimodal MR-CT head template is freely downloadable from https://www.u-picardie.fr/labo/GRAMFC.

Non-rigid MR-TRUS image registration for image-guided prostate biopsy using correlation ratio-based mutual information

BACKGROUND

To improve the accuracy of ultrasound-guided biopsy of the prostate, the non-rigid registration of magnetic resonance (MR) images onto transrectal ultrasound (TRUS) images has gained increasing attention. Mutual information (MI) is a widely used similarity criterion in MR-TRUS image registration. However, the use of MI has been challenged because of intensity distortion, noise and down-sampling. Hence, we need to improve the MI measure to get better registration effect.

METHODS

We present a novel two-dimensional non-rigid MR-TRUS registration algorithm that uses correlation ratio-based mutual information (CRMI) as the similarity criterion. CRMI includes a functional mapping of intensity values on the basis of a generalized version of intensity class correspondence. We also analytically acquire the derivative of CRMI with respect to deformation parameters. Furthermore, we propose an improved stochastic gradient descent (ISGD) optimization method based on the Metropolis acceptance criteria to improve the global optimization ability and decrease the registration time.

RESULTS

The performance of the proposed method is tested on synthetic images and 12 pairs of clinical prostate TRUS and MR images. By comparing label map registration frame (LMRF) and conditional mutual information (CMI), the proposed algorithm has a significant improvement in the average values of Hausdorff distance and target registration error. Although the average Dice Similarity coefficient is not significantly better than CMI, it still has a crucial increase over LMRF. The average computation time consumed by the proposed method is similar to LMRF, which is 16 times less than CMI.

CONCLUSION

With more accurate matching performance and lower sensitivity to noise and down-sampling, the proposed algorithm of minimizing CRMI by ISGD is more robust and has the potential for use in aligning TRUS and MR images for needle biopsy.

MIND Demons: Symmetric Diffeomorphic Deformable Registration of MR and CT for Image-Guided Spine Surgery

Intraoperative localization of target anatomy and critical structures defined in preoperative MR/CT images can be achieved through the use of multimodality deformable registration. We propose a symmetric diffeomorphic deformable registration algorithm incorporating a modality-independent neighborhood descriptor (MIND) and a robust Huber metric for MR-to-CT registration. The method, called MIND Demons, finds a deformation field between two images by optimizing an energy functional that incorporates both the forward and inverse deformations, smoothness on the integrated velocity fields, a modality-insensitive similarity function suitable to multimodality images, and smoothness on the diffeomorphisms themselves. Direct optimization without relying on the exponential map and stationary velocity field approximation used in conventional diffeomorphic Demons is carried out using a Gauss-Newton method for fast convergence. Registration performance and sensitivity to registration parameters were analyzed in simulation, phantom experiments, and clinical studies emulating application in image-guided spine surgery, and results were compared to mutual information (MI) free-form deformation (FFD), local MI (LMI) FFD, normalized MI (NMI) Demons, and MIND with a diffusion-based registration method (MIND-elastic). The method yielded sub-voxel invertibility (0.008 mm) and nonzero-positive Jacobian determinants. It also showed improved registration accuracy in comparison to the reference methods, with mean target registration error (TRE) of 1.7 mm compared to 11.3, 3.1, 5.6, and 2.4 mm for MI FFD, LMI FFD, NMI Demons, and MIND-elastic methods, respectively. Validation in clinical studies demonstrated realistic deformations with sub-voxel TRE in cases of cervical, thoracic, and lumbar spine.

A robust image registration method based on total variation regularization under complex illumination changes

BACKGROUND AND OBJECTIVE

Image registration is one of the fundamental and essential tasks for medical imaging and remote sensing applications. One of the most common challenges in this area is the presence of complex spatially varying intensity distortion in the images. The widely used similarity metrics, such as MI (Mutual Information), CC (Correlation Coefficient), SSD (Sum of Square Difference), SAD (Sum of Absolute Difference) and CR (Correlation Ratio), are not robust against this kind of distortion because stationarity assumption and the pixel-wise independence cannot be obeyed and captured by these metrics.

METHODS

In this paper, we propose a new intensity-based method for simultaneous image registration and intensity correction. We assume that the registered moving image can be reconstructed by the reference image through a linear function that consists of multiplicative and additive coefficients. We also assume that the illumination changes in the images are spatially smooth in each region, so we use weighted Total Variation as a regularization term to estimate the aforesaid multiplicative and additive coefficients. Using weighted Total Variation leads to reduce the smoothness-effect on the coefficients across the edges and causes low level segmentation on the coefficients. For minimizing the reconstruction error, as a dissimilarity term, we use l1norm which is more robust against illumination change and non-Gaussian noises than the l2 norm. Primal-Dual method is used for solving the optimization problem.

RESULTS

The proposed method is applied to simulated as well as real-world data consisting of clinically 4-D Computed Tomography, retina, Digital Subtraction Angiography (DSA), and iris image pairs. Then, the comparisons are made to MI, CC, SSD, SAD and RC qualitatively and sometimes quantitatively.

CONCLUSIONS

The experiment results are demonstrating that the proposed method produces more accurate registration results than conventional methods.

Deformable Registration of Biomedical Images Using 2D Hidden Markov Models

Robust registration of unimodal and multimodal images is a key task in biomedical image analysis, and is often utilized as an initial step on which subsequent analysis techniques critically depend. We propose a novel probabilistic framework, based on a variant of the 2D hidden Markov model, namely, the turbo hidden Markov model, to capture the deformation between pairs of images. The hidden Markov model is tailored to capture spatial transformations across images via state transitions, and modality-specific data costs via emission probabilities. The method is derived for the unimodal setting (where simpler matching metrics may be used) as well as the multimodal setting, where different modalities may provide very different representations for a given class of objects, necessitating the use of advanced similarity measures. We utilize a rich model with hundreds of model parameters to describe the deformation relationships across such modalities. We also introduce a local edge-adaptive constraint to allow for varying degrees of smoothness between object boundaries and homogeneous regions. The parameters of the described method are estimated in a principled manner from training data via maximum likelihood learning, and the deformation is subsequently estimated using an efficient dynamic programming algorithm. Experimental results demonstrate the improved performance of the proposed approach over the state-of-the-art deformable registration techniques, on both unimodal and multimodal biomedical data sets.

Deformable image registration between pathological images and MR image via an optical macro image

Computed tomography (CT) and magnetic resonance (MR) imaging have been widely used for visualizing the inside of the human body. However, in many cases, pathological diagnosis is conducted through a biopsy or resection of an organ to evaluate the condition of tissues as definitive diagnosis. To provide more advanced information onto CT or MR image, it is necessary to reveal the relationship between tissue information and image signals. We propose a registration scheme for a set of PT images of divided specimens and a 3D-MR image by reference to an optical macro image (OM image) captured by an optical camera. We conducted a fundamental study using a resected human brain after the death of a brain cancer patient. We constructed two kinds of registration processes using the OM image as the base for both registrations to make conversion parameters between the PT and MR images. The aligned PT images had shapes similar to the OM image. On the other hand, the extracted cross-sectional MR image was similar to the OM image. From these resultant conversion parameters, the corresponding region on the PT image could be searched and displayed when an arbitrary pixel on the MR image was selected. The relationship between the PT and MR images of the whole brain can be analyzed using the proposed method. We confirmed that same regions between the PT and MR images could be searched and displayed using resultant information obtained by the proposed method. In terms of the accuracy of proposed method, the TREs were 0.56±0.39mm and 0.87±0.42mm. We can analyze the relationship between tissue information and MR signals using the proposed method.

Sorted self-similarity for multi-modal image registration

In medical image analysis, registration of multimodal images has been challenging due to the complex intensity relationship between images. Classical multi-modal registration approaches evaluate the degree of the alignment by measuring the statistical dependency of the intensity values between images to be aligned. Employing statistical similarity measures, such as mutual information, is not promising in those cases with complex and spatially dependent intensity relations. A new similarity measure is proposed based on the assessing the similarity of pixels within an image, based on the idea that similar structures in an image are more probable to undergo similar intensity transformations. The most significant pixel similarity values are considered to transmit the most significant self-similarity information. The proposed method is employed in a framework to register different modalities of real brain scans and the performance of the method is compared to the conventional multi-modal registration approach. Quantitative evaluation of the method demonstrates the better registration accuracy in both rigid and non-rigid deformations.