Brain functional localization: a survey of image registration techniques

Fine-grained, nonlinear registration of live cell movies reveals spatiotemporal organization of diffuse molecular processes

We present an application of nonlinear image registration to align in microscopy time lapse sequences for every frame the cell outline and interior with the outline and interior of the same cell in a reference frame. The registration relies on a subcellular fiducial marker, a cell motion mask, and a topological regularization that enforces diffeomorphism on the registration without significant loss of granularity. This allows spatiotemporal analysis of extremely noisy and diffuse molecular processes across the entire cell. We validate the registration method for different fiducial markers by measuring the intensity differences between predicted and original time lapse sequences of Actin cytoskeleton images and by uncovering zones of spatially organized GEF- and GTPase signaling dynamics visualized by FRET-based activity biosensors in MDA-MB-231 cells. We then demonstrate applications of the registration method in conjunction with stochastic time-series analysis. We describe distinct zones of locally coherent dynamics of the cytoplasmic protein Profilin in U2OS cells. Further analysis of the Profilin dynamics revealed strong relationships with Actin cytoskeleton reorganization during cell symmetry-breaking and polarization. This study thus provides a framework for extracting information to explore functional interactions between cell morphodynamics, protein distributions, and signaling in cells undergoing continuous shape changes. Matlab code implementing the proposed registration method is available at https://github.com/DanuserLab/Mask-Regularized-Diffeomorphic-Cell-Registration.

An Iterative Centroid Approach for Diffeomorphic Online Atlasing

Online atlasing, i.e., incrementing an atlas with new images as they are acquired, is key when performing studies on very large, or still being gathered, databases. Regular approaches to atlasing however do not focus on this aspect and impose a complete reconstruction of the atlas when adding images. We propose instead a diffeomorphic online atlasing method that allows gradual updates to an atlas. In this iterative centroid approach, we integrate new subjects in the atlas in an iterative manner, gradually moving the centroid of the images towards its final position. This leads to a computationally cheap approach since it only necessitates one additional registration per new subject added. We validate our approach on several experiments with three main goals: 1- to evaluate atlas image quality of the obtained atlases with sharpness and overlap measures, 2- to assess the deviation in terms of transformations with respect to a conventional atlasing method and 3- to compare its computational time with regular approaches of the literature. We demonstrate that the transformations divergence with respect to a state-of-the-art atlas construction method is small and reaches a plateau, that the two construction methods have the same ability to map subject homologous regions onto a common space and produce images of equivalent quality. The computational time of our approach is also drastically reduced for regular updates. Finally, we also present a direct extension of our method to update spatio-temporal atlases, especially useful for developmental studies.

Study and analysis of different segmentation methods for brain tumor MRI application

Medical Resonance Imaging (MRI) is one of the preferred imaging methods for brain tumor diagnosis and getting detailed information on tumor type, location, size, identification, and detection. Segmentation divides an image into multiple segments and describes the separation of the suspicious region from pre-processed MRI images to make the simpler image that is more meaningful and easier to examine. There are many segmentation methods, embedded with detection devices, and the response of each method is different. The study article focuses on comparing the performance of several image segmentation algorithms for brain tumor diagnosis, such as Otsu's, watershed, level set, K-means, HAAR Discrete Wavelet Transform (DWT), and Convolutional Neural Network (CNN). All of the techniques are simulated in MATLAB using online images from the Brain Tumor Image Segmentation Benchmark (BRATS) dataset-2018. The performance of these methods is analyzed based on response time and measures such as recall, precision, F-measures, and accuracy. The measured accuracy of Otsu's, watershed, level set, K-means, DWT, and CNN methods is 71.42%, 78.26%, 80.45%, 84.34%, 86.95%, and 91.39 respectively. The response time of CNN is 2.519 s in the MATLAB simulation environment for the designed algorithm. The novelty of the work is that CNN has been proven the best algorithm in comparison to all other methods for brain tumor image segmentation. The simulated and estimated parameters provide the direction to researchers to choose the specific algorithm for embedded hardware solutions and develop the optimal machine-learning models, as the industries are looking for the optimal solutions of CNN and deep learning-based hardware models for the brain tumor.

Virtual liver needle biopsy from reconstructed three-dimensional histopathological images: Quantification of sampling error

INTRODUCTION

Prevalently considered as the "gold-standard" for diagnosis of hepatic fibrosis and cirrhosis, the clinical liver needle biopsy is known to be subject to inadequate sampling and a high mis-sampling rate. However, quantifying such sampling bias has been difficult as generating a large number of needle biopsies from the same living patient is practically infeasible. We construct a three-dimension (3D) virtual liver tissue volume by spatially registered high resolution Whole Slide Images (WSIs) of serial liver tissue sections with a novel dynamic registration method. We further develop a Virtual Needle Biopsy Sampling (VNBS) method that mimics the needle biopsy sampling process. We apply the VNBS method to the reconstructed digital liver volume at different tissue locations and angles. Additionally, we quantify Collagen Proportionate Area (CPA) in all resulting virtual needle biopsies in 2D and 3D.

RESULTS

The staging score of the center 2D longitudinal image plane from each 3D biopsy is used as the biopsy staging score, and the highest staging score of all sampled needle biopsies is the diagnostic staging score. The Mean Absolute Difference (MAD) in reference to the Scheuer and Ishak diagnostic staging scores are 0.22 and 1.00, respectively. The absolute Scheuer staging score difference in 22.22% of sampled biopsies is 1. By the Ishak staging method, 55.56% and 22.22% of sampled biopsies present score difference 1 and 2, respectively. There are 4 (Scheuer) and 6 (Ishak) out of 18 3D virtual needle biopsies with intra-needle variations. Additionally, we find a positive correlation between CPA and fibrosis stages by Scheuer but not Ishak method. Overall, CPA measures suffer large intra- and inter- needle variations.

CONCLUSIONS

The developed virtual liver needle biopsy sampling pipeline provides a computational avenue for investigating needle biopsy sampling bias with 3D virtual tissue volumes. This method can be applied to other tissue-based disease diagnoses where the needle biopsy sampling bias substantially affects the diagnostic results.

Medical Image Registration Algorithm Based on Bounded Generalized Gaussian Mixture Model

In this paper, a method for medical image registration based on the bounded generalized Gaussian mixture model is proposed. The bounded generalized Gaussian mixture model is used to approach the joint intensity of source medical images. The mixture model is formulated based on a maximum likelihood framework, and is solved by an expectation-maximization algorithm. The registration performance of the proposed approach on different medical images is verified through extensive computer simulations. Empirical findings confirm that the proposed approach is significantly better than other conventional ones.

Multistage Forward Path Regenerative Genetic Algorithm for Brain Magnetic Resonant Imaging Registration

In image registration, the search space used to compute the optimal transformation between the images depends on the group of pixels in the vicinity. Favorable results can be achieved by significantly increasing the number of neighboring pixels in the search space; however, this strategy increases the computational load, thus making it challenging to realize the most desirable solution in a reasonable amount of time. To address the mentioned problem, the genetic algorithm is used to find the optimum solution and the solution lies in finding the best chromosomes. In rigid image registration problem, chromosomes contain a set of three parameters, x-translation, y-translation, and rotation. The genetic algorithm iteratively improves chromosomes from generation to generation and selects the best one having the best fittest value. Chromosomes with high fitness value are the ones with an optimal solution where the template image best aligns reference image. Fitness function in the genetic algorithm for image registration problem uses similarity measure index measure to find the amount of similarity between two images. The best fittest value is the one with a high similarity measure that shows the best-aligned template and reference image. Here we used the structural similarity index measure in fitness function that helps in evaluating the best chromosome, even for the compressed images with low quality, intensity nonuniformity (INU), and noise degradation. Building on the genetic algorithm, we propose a novel approach called multistage forward path regenerative genetic algorithm (MFRGA), abbreviated as MFRGA, with reducing search space at each stage. Compared with the single stage of genetic algorithm, our approach proved to be more reliable and accurate in terms of finding true rigid image transformation for alignment. At each increasing stage of MFRGA, results are computed with decreasing search space and increasing precision levels. Moreover, to prove the robustness of our algorithm, we utilized compressed images of brain magnetic resonant imaging that vary in compression qualities ranging from 10 to 100. Furthermore, we added noise levels of 1%, 3%, 5%, 7%, and 9% with an INU of 20% and 40%, respectively, provided by the online BrainWeb simulator. We achieved the monomodal rigid image registration that proves to be successful using MFRGA, even when the noise is critical, the compression quality is the least, and the intensity is nonuniform.

Automated brain structures segmentation from PET/CT images based on landmark-constrained dual-modality atlas registration

Automated brain structures segmentation in positron emission tomography (PET) images has been widely investigated to help brain disease diagnosis and follow-up. To relieve the burden of a manual definition of volume of interest (VOI), automated atlas-based VOI definition algorithms were developed, but these algorithms mostly adopted a global optimization strategy which may not be particularly accurate for local small structures (especially the deep brain structures). This paper presents a PET/CT-based brain VOI segmentation algorithm combining anatomical atlas, local landmarks, and dual-modality information. The method incorporates local deep brain landmarks detected by the Deep Q-Network (DQN) to constrain the atlas registration process. Dual-modality PET/CT image information is also combined to improve the registration accuracy of the extracerebral contour. We compare our algorithm with the representative brain atlas registration methods based on 86 clinical PET/CT images. The proposed algorithm obtained accurate delineation of brain VOIs with an average Dice similarity score of 0.79, an average surface distance of 0.97 mm (sub-pixel level), and a volume recovery coefficient close to 1. The main advantage of our method is that it optimizes both global-scale brain matching and local-scale small structure alignment around the key landmarks, it is fully automated and produces high-quality parcellation of the brain structures from brain PET/CT images.

Simulation Study of a Frame-Based Motion Correction Algorithm for Positron Emission Imaging

Positron emission tomography (PET) is a functional non-invasive imaging modality that uses radioactive substances (radiotracers) to measure changes in metabolic processes. Advances in scanner technology and data acquisition in the last decade have led to the development of more sophisticated PET devices with good spatial resolution (1–3 mm of full width at half maximum (FWHM)). However, there are involuntary motions produced by the patient inside the scanner that lead to image degradation and potentially to a misdiagnosis. The adverse effect of the motion in the reconstructed image increases as the spatial resolution of the current scanners continues improving. In order to correct this effect, motion correction techniques are becoming increasingly popular and further studied. This work presents a simulation study of an image motion correction using a frame-based algorithm. The method is able to cut the acquired data from the scanner in frames, taking into account the size of the object of study. This approach allows working with low statistical information without losing image quality. The frames are later registered using spatio-temporal registration developed in a multi-level way. To validate these results, several performance tests are applied to a set of simulated moving phantoms. The results obtained show that the method minimizes the intra-frame motion, improves the signal intensity over the background in comparison with other literature methods, produces excellent values of similarity with the ground-truth (static) image and is able to find a limit in the patient-injected dose when some prior knowledge of the lesion is present.

Brain Extraction Using Active Contour Neighborhood-Based Graph Cuts Model

The extraction of brain tissue from brain MRI images is an important pre-procedure for the neuroimaging analyses. The brain is bilaterally symmetric both in coronal plane and transverse plane, but is usually asymmetric in sagittal plane. To address the over-smoothness, boundary leakage, local convergence and asymmetry problems in many popular methods, we developed a brain extraction method using an active contour neighborhood-based graph cuts model. The method defined a new asymmetric assignment of edge weights in graph cuts for brain MRI images. The new graph cuts model was performed iteratively in the neighborhood of brain boundary named the active contour neighborhood (ACN), and was effective to eliminate boundary leakage and avoid local convergence. The method was compared with other popular methods on the Internet Brain Segmentation Repository (IBSR) and OASIS data sets. In testing cross IBSR data set (18 scans with 1.5 mm thickness), IBSR data set (20 scans with 3.1 mm thickness) and OASIS data set (77 scans with 1 mm thickness), the mean Dice similarity coefficients obtained by the proposed method were 0.957 ± 0.013, 0.960 ± 0.009 and 0.936 ± 0.018 respectively. The result obtained by the proposed method is very similar with manual segmentation and achieved the best mean Dice similarity coefficient on IBSR data. Our experiments indicate that the proposed method can provide competitively accurate results and may obtain brain tissues with sharp brain boundary from brain MRI images.

Decoding dynamic affective responses to naturalistic videos with shared neural patterns

This study explored the feasibility of using shared neural patterns from brief affective episodes (viewing affective pictures) to decode extended, dynamic affective sequences in a naturalistic experience (watching movie-trailers). Twenty-eight participants viewed pictures from the International Affective Picture System (IAPS) and, in a separate session, watched various movie-trailers. We first located voxels at bilateral occipital cortex (LOC) responsive to affective picture categories by GLM analysis, then performed between-subject hyperalignment on the LOC voxels based on their responses during movie-trailer watching. After hyperalignment, we trained between-subject machine learning classifiers on the affective pictures, and used the classifiers to decode affective states of an out-of-sample participant both during picture viewing and during movie-trailer watching. Within participants, neural classifiers identified valence and arousal categories of pictures, and tracked self-reported valence and arousal during video watching. In aggregate, neural classifiers produced valence and arousal time series that tracked the dynamic ratings of the movie-trailers obtained from a separate sample. Our findings provide further support for the possibility of using pre-trained neural representations to decode dynamic affective responses during a naturalistic experience.

Surface-constrained volumetric registration for the early developing brain

The T1-weighted and T2-weighted MRI contrasts of the infant brain evolve drastically during the first year of life. This poses significant challenges to inter- and intra-subject registration, which is key to subsequent statistical analyses. Existing registration methods that do not consider temporal contrast changes are ineffective for infant brain MRI data. To address this problem, we present in this paper a method for deformable registration of infant brain MRI. The key advantage of our method is threefold: (i) To deal with appearance changes, registration is performed based on segmented tissue maps instead of image intensity. Segmentation is performed by using an infant-centric algorithm previously developed by our group. (ii) Registration is carried out with respect to both cortical surfaces and volumetric tissue maps, thus allowing precise alignment of both cortical and subcortical structures. (iii) A dynamic elasticity model is utilized to allow large non-linear deformation. Experimental results in comparison with well-established registration methods indicate that our method yields superior accuracy in both cortical and subcortical alignment.

Linear Registration of Brain MRI Using Knowledge-Based Multiple Intermediator Libraries

Linear registration is often the crucial first step for various types of image analysis. Although this is mathematically simple, failure is not uncommon. When investigating the brain by magnetic resonance imaging (MRI), the brain is the target organ for registration but the existence of other tissues, in addition to a variety of fields of view, different brain locations, orientations and anatomical features, poses some serious fundamental challenges. Consequently, a number of different algorithms have been put forward to minimize potential errors. In the present study, we tested a knowledge-based approach that can be combined with any form of registration algorithm. This approach consisted of a library of intermediate images (mediators) with known transformation to the target image. Test images were first registered to all mediators and the best mediator was selected to ensure optimum registration to the target. In order to select the best mediator, we evaluated two similarity criteria: the sum of squared differences and mutual information. This approach was applied to 48 mediators and 96 test images. In order to reduce one of the main drawbacks of the approach, increased computation time, we reduced the size of the library by clustering. Our results indicated clear improvement in registration accuracy.

A Novel Coarse-to-Fine Scheme for Remote Sensing Image Registration Based on SIFT and Phase Correlation

Automatic image registration has been wildly used in remote sensing applications. However, the feature-based registration method is sometimes inaccurate and unstable for images with large scale difference, grayscale and texture differences. In this manuscript, a coarse-to-fine registration scheme is proposed, which combines the advantage of feature-based registration and phase correlation-based registration. The scheme consists of four steps. First, feature-based registration method is adopted for coarse registration. A geometrical outlier removal method is applied to improve the accuracy of coarse registration, which uses geometric similarities of inliers. Then, the sensed image is modified through the coarse registration result under affine deformation model. After that, the modified sensed image is registered to the reference image by extended phase correlation. Lastly, the final registration results are calculated by the fusion of the coarse registration and the fine registration. High universality of feature-based registration and high accuracy of extended phase correlation-based registration are both preserved in the proposed method. Experimental results of several different remote sensing images, which come from several published image registration papers, demonstrate the high robustness and accuracy of the proposed method. The evaluation contains root mean square error (RMSE), Laplace mean square error (LMSE) and red–green image registration results.

Artificially enriching the training dataset of statistical shape models via constrained cage-based deformation

The construction of a powerful statistical shape model (SSM) requires a rich training dataset that includes the large variety of complex anatomical topologies. The lack of real data causes most SSMs unable to generalize possible unseen instances. Artificial enrichment of training data is one of the methods proposed to address this issue. In this paper, we introduce a novel technique called constrained cage-based deformation (CCBD), which has the ability to produce unlimited artificial data that promises to enrich variability within the training dataset. The proposed method is a two-step algorithm: in the first step, it moves a few handles together, and in the second step transfers the displacements of these handles to the base mesh vertices to generate a real new instance. The evaluation of statistical characteristics of the CCBD confirms that our proposed technique outperforms notable data-generating methods quantitatively, in terms of the generalization ability, and with respect to specificity.

On the Usage of Brain Atlases in Neuroimaging Research

Brain atlases play a key role in modern neuroimaging analysis of brain structure and function. We review available atlas databases for humans and animals and illustrate common state-of-the-art workflows in neuroimaging research based on image registration. Advances in noninvasive imaging methods, 3D ex vivo microscopy, and image processing are summarized which will eventually close the current resolution gap between brain atlases based on conventional 2D histology and those based on 3D in vivo imaging.

Peeking into the Black Box of Coregistration in Clinical fMRI: Which Registration Methods Are Used and How Well Do They Perform?

BACKGROUND AND PURPOSE

Interpretation of fMRI depends on accurate functional-to-structural alignment. This study explores registration methods used by FDA-approved software for clinical fMRI and aims to answer the following question: What is the degree of misalignment when registration is not performed, and how well do current registration methods perform?

MATERIALS AND METHODS

This retrospective study of presurgical fMRI for brain tumors compares nonregistered images and 5 registration cost functions: Hellinger, mutual information, normalized mutual information, correlation ratio, and local Pearson correlation. To adjudicate the accuracy of coregistration, we edge-enhanced echo-planar maps and rated them for alignment with structural anatomy. Lesion-to-activation distances were measured to evaluate the effects of different cost functions.

RESULTS

Transformation parameters were congruent among Hellinger, mutual information, normalized mutual information, and the correlation ratio but divergent from the local Pearson correlation. Edge-enhanced images validated the local Pearson correlation as the most accurate. Hellinger worsened misalignment in 59% of cases, primarily exaggerating the inferior translation; no cases were worsened by the local Pearson correlation. Three hundred twenty lesion-to-activation distances from 25 patients were analyzed among nonregistered images, Hellinger, and the local Pearson correlation. ANOVA analysis revealed significant differences in the coronal (P < .001) and sagittal (P = .04) planes. If registration is not performed, 8% of cases may have a >3-mm discrepancy and up to a 5.6-mm lesion-to-activation distance difference. If a poor registration method is used, 23% of cases may have a >3-mm discrepancy and up to a 6.9-mm difference.

CONCLUSIONS

The local Pearson correlation is a special-purpose cost function specifically designed for T2*-T1 coregistration and should be more widely incorporated into software tools as a better method for coregistration in clinical fMRI.

Normalized Subtraction of Serial Brain Magnetic Resonance Images and Fludeoxyglucose-Positron Emission Tomography Images for Tumor Treatment Monitoring: Case Report and Method Description

A 60-year-old Caucasian male with a long history of cigarette smoking was diagnosed with epidermal growth factor receptor-mutation negative lung adenocarcinoma. The single cerebral metastasis in the right frontal lobe was treated with stereotactic radiosurgery and systemic chemotherapies. Normalized subtraction (NS) method was used to evaluate the serial brain magnetic resonance (MR) and fludeoxyglucose-positron emission tomography (FDG-PET) findings retrospectively, and the potential benefit of concurrent NS of serial MR images (MRIs) and PET images was demonstrated. MIM 4.1 (MIM Software Inc., Cleveland, OH) was used to co-register MRI with PET data and to perform NS on the serial MRI and PET data. MIM 4.1 provides fully automated alignment of imaging data by maximization of mutual information. Cortical regions distant from the brain lesion were used to adjust for the intensity differences between scans, so the voxel differences in normal brain regions were near zero in the NS images. A difference of 15% or greater in voxel densities was used for both MRI and PET, above or below which a change in MR signal intensity and FDG avidity was considered significant. The use of NS, in this case, allowed for an enhanced correlation of morphologic and functional information, which may have added value in the early treatment monitoring of brain tumors and help distinguish recurrent tumor from postradiation changes.

Survey of Non-Rigid Registration Tools in Medicine

We catalogue available software solutions for non-rigid image registration to support scientists in selecting suitable tools for specific medical registration purposes. Registration tools were identified using non-systematic search in Pubmed, Web of Science, IEEE Xplore® Digital Library, Google Scholar, and through references in identified sources (n = 22). Exclusions are due to unavailability or inappropriateness. The remaining (n = 18) tools were classified by (i) access and technology, (ii) interfaces and application, (iii) living community, (iv) supported file formats, and (v) types of registration methodologies emphasizing the similarity measures implemented. Out of the 18 tools, (i) 12 are open source, 8 are released under a permissive free license, which imposes the least restrictions on the use and further development of the tool, 8 provide graphical processing unit (GPU) support; (ii) 7 are built on software platforms, 5 were developed for brain image registration; (iii) 6 are under active development but only 3 have had their last update in 2015 or 2016; (iv) 16 support the Analyze format, while 7 file formats can be read with only one of the tools; and (v) 6 provide multiple registration methods and 6 provide landmark-based registration methods. Based on open source, licensing, GPU support, active community, several file formats, algorithms, and similarity measures, the tools Elastics and Plastimatch are chosen for the platform ITK and without platform requirements, respectively. Researchers in medical image analysis already have a large choice of registration tools freely available. However, the most recently published algorithms may not be included in the tools, yet.

A Statistically Representative Atlas for Mapping Neuronal Circuits in the Drosophila Adult Brain

Imaging the expression patterns of reporter constructs is a powerful tool to dissect the neuronal circuits of perception and behavior in the adult brain of Drosophila, one of the major models for studying brain functions. To date, several Drosophila brain templates and digital atlases have been built to automatically analyze and compare collections of expression pattern images. However, there has been no systematic comparison of performances between alternative atlasing strategies and registration algorithms. Here, we objectively evaluated the performance of different strategies for building adult Drosophila brain templates and atlases. In addition, we used state-of-the-art registration algorithms to generate a new group-wise inter-sex atlas. Our results highlight the benefit of statistical atlases over individual ones and show that the newly proposed inter-sex atlas outperformed existing solutions for automated registration and annotation of expression patterns. Over 3,000 images from the Janelia Farm FlyLight collection were registered using the proposed strategy. These registered expression patterns can be searched and compared with a new version of the BrainBaseWeb system and BrainGazer software. We illustrate the validity of our methodology and brain atlas with registration-based predictions of expression patterns in a subset of clock neurons. The described registration framework should benefit to brain studies in Drosophila and other insect species.

Heterogeneous Optimization Framework: Reproducible Preprocessing of Multi-Spectral Clinical MRI for Neuro-Oncology Imaging Research

Neuroimaging research often relies on clinically acquired magnetic resonance imaging (MRI) datasets that can originate from multiple institutions. Such datasets are characterized by high heterogeneity of modalities and variability of sequence parameters. This heterogeneity complicates the automation of image processing tasks such as spatial co-registration and physiological or functional image analysis. Given this heterogeneity, conventional processing workflows developed for research purposes are not optimal for clinical data. In this work, we describe an approach called Heterogeneous Optimization Framework (HOF) for developing image analysis pipelines that can handle the high degree of clinical data non-uniformity. HOF provides a set of guidelines for configuration, algorithm development, deployment, interpretation of results and quality control for such pipelines. At each step, we illustrate the HOF approach using the implementation of an automated pipeline for Multimodal Glioma Analysis (MGA) as an example. The MGA pipeline computes tissue diffusion characteristics of diffusion tensor imaging (DTI) acquisitions, hemodynamic characteristics using a perfusion model of susceptibility contrast (DSC) MRI, and spatial cross-modal co-registration of available anatomical, physiological and derived patient images. Developing MGA within HOF enabled the processing of neuro-oncology MR imaging studies to be fully automated. MGA has been successfully used to analyze over 160 clinical tumor studies to date within several research projects. Introduction of the MGA pipeline improved image processing throughput and, most importantly, effectively produced co-registered datasets that were suitable for advanced analysis despite high heterogeneity in acquisition protocols.

Functional density and edge maps: Characterizing functional architecture in individuals and improving cross-subject registration

Population-level inferences and individual-level analyses are two important aspects in functional magnetic resonance imaging (fMRI) studies. Extracting reliable and informative features from fMRI data that capture biologically meaningful inter-subject variation is critical for aligning and comparing functional networks across subjects, and connecting the properties of functional brain organization with variations in behavior, cognition and genetics. In this study, we derive two new measures, which we term functional density map and edge map, and demonstrate their usefulness in characterizing the function of individual brains. Specifically, using data from the Human Connectome Project (HCP), we show that (1) both functional maps capture intrinsic properties of the functional connectivity pattern in individuals while exhibiting large variation across subjects; (2) functional maps derived from either resting-state or task-evoked fMRI can be used to accurately identify subjects from a population; and (3) cross-subject alignment using these functional maps considerably reduces functional variation and improves functional correspondence across subjects over state-of-the-art multimodal registration algorithms. Our results suggest that the proposed functional density and edge maps are promising features in characterizing the functional architecture in individuals and provide an alternative way to explore the functional variation across subjects.

A structural similarity-inspired performance assessment model for multisensor image registration algorithms

In order to assess the performance of multisensor image registration algorithms that are used in the multirobot information fusion, we propose a model based on structural similarity whose name is vision registration assessment model. First of all, this article introduces a new image concept named superimposed image for testing subjective and objective assessment methods. Therefore, we assess the superimposed image but not the registered image, which is different from previous image registration assessment methods that usually use reference and sensed images. Then, we calculate eight assessment indicators from different aspects for superimposed images. After that, vision registration assessment model fuses the eight indicators using canonical correlation analysis, which is used for evaluating the quality of an image registration results in different aspects. Finally, three kinds of images which include optical images, infrared images, and SAR images are used to test vision registration assessment model. After evaluating three state-of-the-art image registration methods, experiments indict that the proposed structural similarity-motivated model achieved almost same evaluation results with that of the human object with the consistency rate of 98.3%, which shows that vision registration assessment model is efficient and robust for evaluating multisensor image registration algorithms. Moreover, vision registration assessment model is independent of the emotional factors and outside environment, which is different from the human.

Minimum action principle and shape dynamics

In this paper, we propose a new method for computing a distance between two shapes embedded in three-dimensional space. Instead of comparing directly the geometric properties of the two shapes, we measure the cost of deforming one of the two shapes into the other. The deformation is computed as the geodesic between the two shapes in the space of shapes. The geodesic is found as a minimizer of the Onsager-Machlup action, based on an elastic energy for shapes that we define. Its length is set to be the integral of the action along that path; it defines an intrinsic quasi-metric on the space of shapes. We illustrate applications of our method to geometric morphometrics using three datasets representing bones and teeth of primates. Experiments on these datasets show that the variational quasi-metric we have introduced performs remarkably well both in shape recognition and in identifying evolutionary patterns, with success rates similar to, and in some cases better than, those obtained by expert observers.

FZUImageReg: A toolbox for medical image registration and dose fusion in cervical cancer radiotherapy

The combination external-beam radiotherapy and high-dose-rate brachytherapy is a standard form of treatment for patients with locally advanced uterine cervical cancer. Personalized radiotherapy in cervical cancer requires efficient and accurate dose planning and assessment across these types of treatment. To achieve radiation dose assessment, accurate mapping of the dose distribution from HDR-BT onto EBRT is extremely important. However, few systems can achieve robust dose fusion and determine the accumulated dose distribution during the entire course of treatment. We have therefore developed a toolbox (FZUImageReg), which is a user-friendly dose fusion system based on hybrid image registration for radiation dose assessment in cervical cancer radiotherapy. The main part of the software consists of a collection of medical image registration algorithms and a modular design with a user-friendly interface, which allows users to quickly configure, test, monitor, and compare different registration methods for a specific application. Owing to the large deformation, the direct application of conventional state-of-the-art image registration methods is not sufficient for the accurate alignment of EBRT and HDR-BT images. To solve this problem, a multi-phase non-rigid registration method using local landmark-based free-form deformation is proposed for locally large deformation between EBRT and HDR-BT images, followed by intensity-based free-form deformation. With the transformation, the software also provides a dose mapping function according to the deformation field. The total dose distribution during the entire course of treatment can then be presented. Experimental results clearly show that the proposed system can achieve accurate registration between EBRT and HDR-BT images and provide radiation dose warping and fusion results for dose assessment in cervical cancer radiotherapy in terms of high accuracy and efficiency.

Progressive disease in glioblastoma: Benefits and limitations of semi-automated volumetry

Purpose

Unambiguous evaluation of glioblastoma (GB) progression is crucial, both for clinical trials as well as day by day routine management of GB patients. 3D-volumetry in the follow-up of GB provides quantitative data on tumor extent and growth, and therefore has the potential to facilitate objective disease assessment. The present study investigated the utility of absolute changes in volume (delta) or regional, segmentation-based subtractions for detecting disease progression in longitudinal MRI follow-ups.

Methods

165 high resolution 3-Tesla MRIs of 30 GB patients (23m, mean age 60.2y) were retrospectively included in this single center study. Contrast enhancement (CV) and tumor-related signal alterations in FLAIR images (FV) were semi-automatically segmented. Delta volume (dCV, dFV) and regional subtractions (sCV, sFV) were calculated. Disease progression was classified for every follow-up according to histopathologic results, decisions of the local multidisciplinary CNS tumor board and a consensus rating of the neuro-radiologic report.

Results

A generalized logistic mixed model for disease progression (yes / no) with dCV, dFV, sCV and sFV as input variables revealed that only dCV was significantly associated with prediction of disease progression (P = .005). Delta volume had a better accuracy than regional, segmentation-based subtractions (79% versus 72%) and a higher area under the curve by trend in ROC curves (.83 versus .75).

Conclusion

Absolute volume changes of the contrast enhancing tumor part were the most accurate volumetric determinant to detect progressive disease in assessment of GB and outweighed FLAIR changes as well as regional, segmentation-based image subtractions. This parameter might be useful in upcoming objective response criteria for glioblastoma.

Joint alignment of multispectral images via semidefinite programming

In this paper, we introduce a novel feature-point-matching based framework for achieving an optimized joint-alignment of sequential images from multispectral imaging (MSI). It solves a low-rank and semidefinite matrix that stores all pairwise-image feature-mappings by minimizing the total amount of point-to-point matching cost via a convex optimization of a semidefinite programming formulation. This unique strategy takes a complete consideration of the information aggregated by all point-matching costs and enables the entire set of pairwise-image feature-mappings to be solved simultaneously and near-optimally. Our framework is capable of running in an automatic or interactive fashion, offering an effective tool for eliminating spatial misalignments introduced into sequential MSI images during the imaging process. Our experimental results obtained from a database of 28 sequences of MSI images of human eye demonstrate the superior performances of our approach to the state-of-the-art techniques. Our framework is potentially invaluable in a large variety of practical applications of MSI images.

Hierarchical Chunking of Sequential Memory on Neuromorphic Architecture with Reduced Synaptic Plasticity

Chunking refers to a phenomenon whereby individuals group items together when performing a memory task to improve the performance of sequential memory. In this work, we build a bio-plausible hierarchical chunking of sequential memory (HCSM) model to explain why such improvement happens. We address this issue by linking hierarchical chunking with synaptic plasticity and neuromorphic engineering. We uncover that a chunking mechanism reduces the requirements of synaptic plasticity since it allows applying synapses with narrow dynamic range and low precision to perform a memory task. We validate a hardware version of the model through simulation, based on measured memristor behavior with narrow dynamic range in neuromorphic circuits, which reveals how chunking works and what role it plays in encoding sequential memory. Our work deepens the understanding of sequential memory and enables incorporating it for the investigation of the brain-inspired computing on neuromorphic architecture.

Efficient Variational Approach to Multimodal Registration of Anatomical and Functional Intra-Patient Tumorous Brain Data

This paper addresses the functional localization of intra-patient images of the brain. Functional images of the brain (fMRI and PET) provide information about brain function and metabolism whereas anatomical images (MRI and CT) supply the localization of structures with high spatial resolution. The goal is to find the geometric correspondence between functional and anatomical images in order to complement and fuse the information provided by each imaging modality. The proposed approach is based on a variational formulation of the image registration problem in the frequency domain. It has been implemented as a C/C[Formula: see text] library which is invoked from a GUI. This interface is routinely used in the clinical setting by physicians for research purposes (Inscanner, Alicante, Spain), and may be used as well for diagnosis and surgical planning. The registration of anatomic and functional intra-patient images of the brain makes it possible to obtain a geometric correspondence which allows for the localization of the functional processes that occur in the brain. Through 18 clinical experiments, it has been demonstrated how the proposed approach outperforms popular state-of-the-art registration methods in terms of efficiency, information theory-based measures (such as mutual information) and actual registration error (distance in space of corresponding landmarks).

Groupwise registration of sequential images from multispectral imaging (MSI) of the retina and choroid

Multispectral Imaging (MSI) produces a sequence of discrete spectral slices that penetrate different light-absorbing species or chromophores and is a noninvasive technology useful for the early detection of various retinal, optic nerve and choroidal diseases. However, eye movement during the image acquisition process may introduce spatial misalignment between MSI images. This potentially causes trouble in the manual/automatic interpretation of MSI, but still remains an unresolved problem to this date. To deal with this MSI misalignment problem, we present a method on the groupwise registration of sequential images from MSI of the retina and choroid. The advantage of our algorithm is at least threefold: 1) simultaneous estimation of landmark correspondences and a parametric motion model via quadratic programming, 2) enforcement of temporal smoothness on the estimated motion, and 3) inclusion of a robust matching cost function. As validated in our experiments with a database of 22 MSI sequences, our algorithm outperforms two state-of-the-art registration techniques proposed originally in other domains. Our algorithm is potentially invaluable in ophthalmologists' clinical practice regarding various eye diseases.

Direct estimation of patient attributes from anatomical MRI based on multi-atlas voting

MRI brain atlases are widely used for automated image segmentation, and in particular, recent developments in multi-atlas techniques have shown highly accurate segmentation results. In this study, we extended the role of the atlas library from mere anatomical reference to a comprehensive knowledge database with various patient attributes, such as demographic, functional, and diagnostic information. In addition to using the selected (heavily-weighted) atlases to achieve high segmentation accuracy, we tested whether the non-anatomical attributes of the selected atlases could be used to estimate patient attributes. This can be considered a context-based image retrieval (CBIR) approach, embedded in the multi-atlas framework. We first developed an image similarity measurement to weigh the atlases on a structure-by-structure basis, and then, the attributes of the multiple atlases were weighted to estimate the patient attributes. We tested this concept first by estimating age in a normal population; we then performed functional and diagnostic estimations in Alzheimer's disease patients. The accuracy of the estimated patient attributes was measured against the actual clinical data, and the performance was compared to conventional volumetric analysis. The proposed CBIR framework by multi-atlas voting would be the first step toward a knowledge-based support system for quantitative radiological image reading and diagnosis.

•

Patient attributes are estimated directly by retrieving information from multi-atlas.

•

Non-imaging attributes of the atlases are weighted to estimate patient attributes.

•

The method achieved high accuracy in estimating age in the normal population.

•

The method can estimate functional and diagnostic attributes in dementia patients.

•

The estimation accuracy was higher than volumetric analysis in subcortical areas.

A Multi-Atlas Labeling Approach for Identifying Subject-Specific Functional Regions of Interest

The functional region of interest (fROI) approach has increasingly become a favored methodology in functional magnetic resonance imaging (fMRI) because it can circumvent inter-subject anatomical and functional variability, and thus increase the sensitivity and functional resolution of fMRI analyses. The standard fROI method requires human experts to meticulously examine and identify subject-specific fROIs within activation clusters. This process is time-consuming and heavily dependent on experts’ knowledge. Several algorithmic approaches have been proposed for identifying subject-specific fROIs; however, these approaches cannot easily incorporate prior knowledge of inter-subject variability. In the present study, we improved the multi-atlas labeling approach for defining subject-specific fROIs. In particular, we used a classifier-based atlas-encoding scheme and an atlas selection procedure to account for the large spatial variability across subjects. Using a functional atlas database for face recognition, we showed that with these two features, our approach efficiently circumvented inter-subject anatomical and functional variability and thus improved labeling accuracy. Moreover, in comparison with a single-atlas approach, our multi-atlas labeling approach showed better performance in identifying subject-specific fROIs.

Spatial Variance in Resting fMRI Networks of Schizophrenia Patients: An Independent Vector Analysis

Spatial variability in resting functional MRI (fMRI) brain networks has not been well studied in schizophrenia, a disease known for both neurodevelopmental and widespread anatomic changes. Motivated by abundant evidence of neuroanatomical variability from previous studies of schizophrenia, we draw upon a relatively new approach called independent vector analysis (IVA) to assess this variability in resting fMRI networks. IVA is a blind-source separation algorithm, which segregates fMRI data into temporally coherent but spatially independent networks and has been shown to be especially good at capturing spatial variability among subjects in the extracted networks. We introduce several new ways to quantify differences in variability of IVA-derived networks between schizophrenia patients (SZs = 82) and healthy controls (HCs = 89). Voxelwise amplitude analyses showed significant group differences in the spatial maps of auditory cortex, the basal ganglia, the sensorimotor network, and visual cortex. Tests for differences (HC-SZ) in the spatial variability maps suggest, that at rest, SZs exhibit more activity within externally focused sensory and integrative network and less activity in the default mode network thought to be related to internal reflection. Additionally, tests for difference of variance between groups further emphasize that SZs exhibit greater network variability. These results, consistent with our prediction of increased spatial variability within SZs, enhance our understanding of the disease and suggest that it is not just the amplitude of connectivity that is different in schizophrenia, but also the consistency in spatial connectivity patterns across subjects.

Landmark-free geometric methods in biological shape analysis

In this paper, we propose a new approach for computing a distance between two shapes embedded in three-dimensional space. We take as input a pair of triangulated genus zero surfaces that are topologically equivalent to spheres with no holes or handles, and construct a discrete conformal map f between the surfaces. The conformal map is chosen to minimize a symmetric deformation energy Esd(f) which we introduce. This measures the distance of f from an isometry, i.e. a non-distorting correspondence. We show that the energy of the minimizing map gives a well-behaved metric on the space of genus zero surfaces. In contrast to most methods in this field, our approach does not rely on any assignment of landmarks on the two surfaces. We illustrate applications of our approach to geometric morphometrics using three datasets representing the bones and teeth of primates. Experiments on these datasets show that our approach performs remarkably well both in shape recognition and in identifying evolutionary patterns, with success rates similar to, and in some cases better than, those obtained by expert observers.

Rat brain normalization templates for robust regional analysis of [11C]ABP688 positron emission tomography/computed tomography

A methodology to generate rat brain templates for spatial normalization of positron emission tomographic (PET)/computed tomographic (CT) images is described and applied to generate three different templates for imaging of [11C]ABP688, a PET ligand binding to the metabotropic glutamate 5 receptor. The templates are based on functional (PET), structural (CT), and combined PET and CT information, respectively. The templates are created from a test-retest study under normal conditions and are used to assess the different templates by using them in the analysis pipeline of a test-retest and a blocking experiment. The resulting average nondisplaceable binding potentials (BPND) show significant (analysis of variance, p < .05) and substantial (up to 23%) differences between the different approaches in several brain regions. The highest BPND values in receptor-rich regions are obtained using the PET-based approach. This approach also had the smallest variability in all tested regions (standard error of measurement of 9% versus 14% [PET/CT] and 20% [CT]). All approaches showed similar relative changes in BPND values with increased blocking. Taken together, these results suggest that the use of the tracer-specific PET-based template outperforms the other approaches with the performance of the combined PET/CT template between those of the PET and the tracer-independent CT template.

Moment to moment variability in functional brain networks during cognitive activity in EEG data

Functional brain networks (FBNs) are gaining increasing attention in computational neuroscience due to their ability to reveal dynamic interdependencies between brain regions. The dynamics of such networks during cognitive activity between stimulus and response using multi-channel electroencephalogram (EEG), recorded from 16 healthy human participants are explored in this research. Successive EEG segments of 500[Formula: see text]ms duration starting from the onset of cognitive stimulation have been used to analyze and understand the cognitive dynamics. The approach employs a combination of signal processing techniques, nonlinear statistical measures and graph-theoretical analysis. The efficacy of this approach in detecting and tracking cognitive load induced changes in EEG data is clearly demonstrated using graph metrics. It is revealed that most cognitive activity occurs within approximately 500[Formula: see text]ms of the stimulus presentation in addition to temporal variability in the FBNs. It is shown that mutual information (MI), a nonlinear measure, produces good correlations between the EEG channels thus enabling the construction of FBNs which are sensitive to cognitive load induced changes in EEG. Analyses of the dynamics of FBNs and the visualization approach reveal hard to detect subtle changes in cognitive function and hence may lead to a better understanding of cognitive processing in the brain. The techniques exploited have the potential to detect human cognitive dysfunction (impairments).

Neurosurgery planning in rodents using a magnetic resonance imaging assisted framework to target experimentally defined networks

BACKGROUND AND OBJECTIVE

Meaningful targeting of brain structures is required in a number of experimental designs in neuroscience. Current technological developments as high density electrode arrays for parallel electrophysiological recordings and optogenetic tools that allow fine control of activity in specific cell populations provide powerful tools to investigate brain physio-pathology. However, to extract the maximum yield from these fine developments, increased precision, reproducibility and cost-efficiency in experimental procedures is also required.

METHODS

We introduce here a framework based on magnetic resonance imaging (MRI) and digitized brain atlases to produce customizable 3D-environments for brain navigation. It allows the use of individualized anatomical and/or functional information from multiple MRI modalities to assist experimental neurosurgery planning and in vivo tissue processing.

RESULTS

As a proof of concept we show three examples of experimental designs facilitated by the presented framework, with extraordinary applicability in neuroscience.

CONCLUSIONS

The obtained results illustrate its feasibility for identifying and selecting functionally and/or anatomically connected neuronal population in vivo and directing electrode implantations to targeted nodes in the intricate system of brain networks.

Functional Magnetic Resonance Imaging Methods

Since its inception in 1992, Functional Magnetic Resonance Imaging (fMRI) has become an indispensible tool for studying cognition in both the healthy and dysfunctional brain. FMRI monitors changes in the oxygenation of brain tissue resulting from altered metabolism consequent to a task-based evoked neural response or from spontaneous fluctuations in neural activity in the absence of conscious mentation (the "resting state"). Task-based studies have revealed neural correlates of a large number of important cognitive processes, while fMRI studies performed in the resting state have demonstrated brain-wide networks that result from brain regions with synchronized, apparently spontaneous activity. In this article, we review the methods used to acquire and analyze fMRI signals.

Localization of Metal Electrodes in the Intact Rat Brain Using Registration of 3D Microcomputed Tomography Images to a Magnetic Resonance Histology Atlas

Simultaneous neural recordings taken from multiple areas of the rodent brain are garnering growing interest due to the insight they can provide about spatially distributed neural circuitry. The promise of such recordings has inspired great progress in methods for surgically implanting large numbers of metal electrodes into intact rodent brains. However, methods for localizing the precise location of these electrodes have remained severely lacking. Traditional histological techniques that require slicing and staining of physical brain tissue are cumbersome, and become increasingly impractical as the number of implanted electrodes increases. Here we solve these problems by describing a method that registers 3-D computerized tomography (CT) images of intact rat brains implanted with metal electrode bundles to a Magnetic Resonance Imaging Histology (MRH) Atlas. Our method allows accurate visualization of each electrode bundle's trajectory and location without removing the electrodes from the brain or surgically implanting external markers. In addition, unlike physical brain slices, once the 3D images of the electrode bundles and the MRH atlas are registered, it is possible to verify electrode placements from many angles by "re-slicing" the images along different planes of view. Further, our method can be fully automated and easily scaled to applications with large numbers of specimens. Our digital imaging approach to efficiently localizing metal electrodes offers a substantial addition to currently available methods, which, in turn, may help accelerate the rate at which insights are gleaned from rodent network neuroscience.

Low-Dimensional Non-Rigid Image Registration Using Statistical Deformation Models From Semi-Supervised Training Data

Accurate and robust image registration is a fundamental task in medical image analysis applications, and requires non-rigid transformations with a large number of degrees of freedom. Statistical deformation models (SDMs) attempt to learn the distribution of non-rigid deformations, and can be used both to reduce the transformation dimensionality and to constrain the registration process. However, high-dimensional SDMs are difficult to train given orders of magnitude fewer training samples. In this paper, we utilize both a small set of annotated imaging data and a large set of unlabeled data to effectively learn an SDM of non-rigid transformations in a semi-supervised training (SST) framework. We demonstrate results applying this framework towards inter-subject registration of skull-stripped, magnetic resonance (MR) brain images. Our approach makes use of 39 labeled MR datasets to create a set of supervised registrations, which we augment with a set of over 1200 unsupervised registrations using unlabeled MRIs. Through leave-one-out cross validation, we show that SST of a non-rigid SDM results in a robust registration algorithm with significantly improved accuracy compared to standard, intensity-based registration, and does so with a 99% reduction in transformation dimensionality.

Test-retest reliability of freesurfer measurements within and between sites: Effects of visual approval process

In the last decade, many studies have used automated processes to analyze magnetic resonance imaging (MRI) data such as cortical thickness, which is one indicator of neuronal health. Due to the convenience of image processing software (e.g., FreeSurfer), standard practice is to rely on automated results without performing visual inspection of intermediate processing. In this work, structural MRIs of 40 healthy controls who were scanned twice were used to determine the test-retest reliability of FreeSurfer-derived cortical measures in four groups of subjects-those 25 that passed visual inspection (approved), those 15 that failed visual inspection (disapproved), a combined group, and a subset of 10 subjects (Travel) whose test and retest scans occurred at different sites. Test-retest correlation (TRC), intraclass correlation coefficient (ICC), and percent difference (PD) were used to measure the reliability in the Destrieux and Desikan-Killiany (DK) atlases. In the approved subjects, reliability of cortical thickness/surface area/volume (DK atlas only) were: TRC (0.82/0.88/0.88), ICC (0.81/0.87/0.88), PD (0.86/1.19/1.39), which represent a significant improvement over these measures when disapproved subjects are included. Travel subjects' results show that cortical thickness reliability is more sensitive to site differences than the cortical surface area and volume. To determine the effect of visual inspection on sample size required for studies of MRI-derived cortical thickness, the number of subjects required to show group differences was calculated. Significant differences observed across imaging sites, between visually approved/disapproved subjects, and across regions with different sizes suggest that these measures should be used with caution.

Computing global minimizers to a constrained B-spline image registration problem from optimal l1 perturbations to block match data

PURPOSE

Block matching is a well-known strategy for estimating corresponding voxel locations between a pair of images according to an image similarity metric. Though robust to issues such as image noise and large magnitude voxel displacements, the estimated point matches are not guaranteed to be spatially accurate. However, the underlying optimization problem solved by the block matching procedure is similar in structure to the class of optimization problem associated with B-spline based registration methods. By exploiting this relationship, the authors derive a numerical method for computing a global minimizer to a constrained B-spline registration problem that incorporates the robustness of block matching with the global smoothness properties inherent to B-spline parameterization.

METHODS

The method reformulates the traditional B-spline registration problem as a basis pursuit problem describing the minimall1-perturbation to block match pairs required to produce a B-spline fitting error within a given tolerance. The sparsity pattern of the optimal perturbation then defines a voxel point cloud subset on which the B-spline fit is a global minimizer to a constrained variant of the B-spline registration problem. As opposed to traditional B-spline algorithms, the optimization step involving the actual image data is addressed by block matching.

RESULTS

The performance of the method is measured in terms of spatial accuracy using ten inhale/exhale thoracic CT image pairs (available for download atwww.dir-lab.com) obtained from the COPDgene dataset and corresponding sets of expert-determined landmark point pairs. The results of the validation procedure demonstrate that the method can achieve a high spatial accuracy on a significantly complex image set.

CONCLUSIONS

The proposed methodology is demonstrated to achieve a high spatial accuracy and is generalizable in that in can employ any displacement field parameterization described as a least squares fit to block match generated estimates. Thus, the framework allows for a wide range of image similarity block match metric and physical modeling combinations.