Feature-based morphometry: discovering group-related anatomical patterns

A review on Alzheimer's disease classification from normal controls and mild cognitive impairment using structural MR images

Alzheimer's disease (AD) is an irreversible neurodegenerative brain disorder that degrades the memory and cognitive ability in elderly people. The main reason for memory loss and reduction in cognitive ability is the structural changes in the brain that occur due to neuronal loss. These structural changes are most conspicuous in the hippocampus, cortex, and grey matter and can be assessed by using neuroimaging techniques viz. Positron Emission Tomography (PET), structural Magnetic Resonance Imaging (MRI) and functional MRI (fMRI), etc. Out of these neuroimaging techniques, structural MRI has evolved as the best technique as it indicates the best soft tissue contrast and high spatial resolution which is important for AD detection. Currently, the focus of researchers is on predicting the conversion of Mild Cognitive Impairment (MCI) into AD. MCI represents the transition state between expected cognitive changes with normal aging and Alzheimer's disease. Not every MCI patient progresses into Alzheimer's disease. MCI can develop into stable MCI (sMCI, patients are called non-converters) or into progressive MCI (pMCI, patients are diagnosed as MCI converters). This paper discusses the prognosis of MCI to AD conversion and presents a review of structural MRI-based studies for AD detection. AD detection framework includes feature extraction, feature selection, and classification process. This paper reviews the studies for AD detection based on different feature extraction methods and machine learning algorithms for classification. The performance of various feature extraction methods has been compared and it has been observed that the wavelet transform-based feature extraction method would give promising results for AD classification. The present study indicates that researchers are successful in classifying AD from Normal Controls (N_rmC) but, it still requires a lot of work to be done for MCI/ N_rmC and MCI/AD, which would help in detecting AD at its early stage.

Digital Analysis of Smart Registration Methods for Magnetic Resonance Images in Public Healthcare

Brain development and atrophy accompany people's life. Brain development diseases, such as autism and Alzheimer's disease, affect a large part of the population. Analyzing brain development is very important in public healthcare, and image registration is essential in medical brain image analysis. Many previous studies investigate registration accuracy by the “ground truth” dataset, marker-based similarity calculation, and expert check to find the best registration algorithms. But the evaluation of image registration technology only at the accuracy level is not comprehensive. Here, we compare the performance of three publicly available registration techniques in brain magnetic resonance imaging (MRI) analysis based on some key features widely used in previous MRI studies for classification and detection tasks. According to the analysis results, SPM12 has a stable speed and success rate, and it always works as a guiding tool for newcomers to medical image analysis. It can preserve maximum contrast information, which will facilitate studies such as tumor diagnosis. FSL is a mature and widely applicable toolkit for users, with a relatively stable success rate and good performance. It has complete functions and its function-based integrated toolbox can meet the requirements of different researchers. AFNI is a flexible and complex tool that is more suitable for professional researchers. It retains most details in medical image analysis, which makes it useful in fine-grained analysis such as volume estimation. Our study provides a new idea for comparing registration tools, where tool selection strategy mainly depends on the research task in which the selected tool can leverage its unique advantages.

Efficient Pairwise Neuroimage Analysis Using the Soft Jaccard Index and 3D Keypoint Sets

We propose a novel pairwise distance measure between image keypoint sets, for the purpose of large-scale medical image indexing. Our measure generalizes the Jaccard index to account for soft set equivalence (SSE) between keypoint elements, via an adaptive kernel framework modeling uncertainty in keypoint appearance and geometry. A new kernel is proposed to quantify the variability of keypoint geometry in location and scale. Our distance measure may be estimated between O (N ²) image pairs in [Formula: see text] operations via keypoint indexing. Experiments report the first results for the task of predicting family relationships from medical images, using 1010 T1-weighted MRI brain volumes of 434 families including monozygotic and dizygotic twins, siblings and half-siblings sharing 100%-25% of their polymorphic genes. Soft set equivalence and the keypoint geometry kernel improve upon standard hard set equivalence (HSE) and appearance kernels alone in predicting family relationships. Monozygotic twin identification is near 100%, and three subjects with uncertain genotyping are automatically paired with their self-reported families, the first reported practical application of image-based family identification. Our distance measure can also be used to predict group categories, sex is predicted with an AUC = 0.97. Software is provided for efficient fine-grained curation of large, generic image datasets.

Automated detection, selection and classification of hippocampal landmark points for the diagnosis of Alzheimer's disease

BACKGROUND AND OBJECTIVE

Alzheimer's disease (AD) is a neurodegenerative, progressive, and irreversible disease that accounts for up to 80% of all dementia cases. AD predominantly affects older adults, and its clinical diagnosis is a challenging evaluation process, with imprecision rates between 12 and 23%. Structural magnetic resonance (MR) imaging has been widely used in studies related to AD because this technique provides images with excellent anatomical details and information about structural changes induced by the disease in the brain. Current studies are focused on detecting AD in its initial stage, i.e., mild cognitive impairment (MCI), since treatments for preventing or delaying the onset of symptoms is more effective when administered at the early stages of the disease. This study proposes a new technique to perform MR image classification in AD diagnosis using discriminative hippocampal point landmarks among the cognitively normal (CN), MCI, and AD populations.

METHODS

Our approach, based on a two-level classification, first detects and selects discriminative landmark points from two diagnosis populations based on their matching distance compared to a probabilistic atlas of 3-D labeled landmark points. The points are classified using attributes computed in a spherical support region around each point using information from brain probability image tissues of gray matter, white matter, and cerebrospinal fluid as sources of information. Next, at the second level, the images are classified based on a quantitative evaluation obtained from the first-level classifier outputs.

RESULTS

For the CN×MCI experiment, we achieved an AUC of 0.83, an accuracy of 75.58%, with 72.9% of sensitivity and 77.81% of specificity. For the MCI×AD experiment, we achieved an AUC value of 0.73, an accuracy of 69.8%, a sensitivity of 74.09% and specificity of 64.57%. Finally, for the CN×AD, we achieved an AUC of 0.95, an accuracy of 89.24%, with 85.58% of sensitivity and 92.71% of specificity.

CONCLUSIONS

The obtained classification results are similar to (or even higher than) other studies that classify AD compared to CN individuals and comparable to those classified patients with MCI.

Automatic recognition of specific local cortical folding patterns

The study of local cortical folding patterns showed links with psychiatric illnesses as well as cognitive functions. Despite the tools now available to visualize cortical folds in 3D, manually classifying local sulcal patterns is a time-consuming and tedious task. In fact, 3D visualization of folds helps experts to identify different sulcal patterns but fold variability is so high that the distinction between these patterns sometimes requires the definition of complex criteria, making manual classification difficult and not reliable. However, the assessment of the impact of these patterns on the functional organization of the cortex could benefit from the study of large databases, especially when studying rare patterns. In this paper, several algorithms for the automatic classification of fold patterns are proposed to allow morphological studies to be extended and confirmed on such large databases. Three methods are proposed, the first based on a Support Vector Machine (SVM) classifier, the second on the Scoring by Non-local Image Patch Estimator (SNIPE) approach and the third based on a 3D Convolution Neural Network (CNN). These methods are generic enough to be applicable to a wide range of folding patterns. They are tested on two types of patterns for which there is currently no method to automatically identify them: the Anterior Cingulate Cortex (ACC) patterns and the Power Button Sign (PBS). The two ACC patterns are almost equally present whereas PBS is a particularly rare pattern in the general population. The three models proposed achieve balanced accuracies of approximately 80% for ACC patterns classification and 60% for PBS classification. The CNN-based model is more interesting for the classification of ACC patterns thanks to its rapid execution. However, SVM and SNIPE-based models are more effective in managing unbalanced problems such as PBS recognition.

Alzheimer's diagnosis using deep learning in segmenting and classifying 3D brain MR images

BACKGROUND AND OBJECTIVES

Dementia is one of the brain diseases with serious symptoms such as memory loss, and thinking problems. According to the World Alzheimer Report 2016, in the world, there are 47 million people having dementia and it can be 131 million by 2050. There is no standard method to diagnose dementia, and consequently unable to access the treatment effectively. Hence, the computational diagnosis of the disease from brain Magnetic Resonance Image (MRI) scans plays an important role in supporting the early diagnosis. Alzheimer's Disease (AD), a common type of Dementia, includes problems related to disorientation, mood swings, not managing self-care, and behavioral issues. In this article, we present a new computational method to diagnosis Alzheimer's disease from 3D brain MR images.

METHODS

An efficient approach to diagnosis Alzheimer's disease from brain MRI scans is proposed comprising two phases: I) segmentation and II) classification, both based on deep learning. After the brain tissues are segmented by a model that combines Gaussian Mixture Model (GMM) and Convolutional Neural Network (CNN), a new model combining Extreme Gradient Boosting (XGBoost) and Support Vector Machine (SVM) is used to classify Alzheimer's disease based on the segmented tissues.

RESULTS

We present two evaluations for segmentation and classification. For comparison, the new method was evaluated using the AD-86 and AD-126 datasets leading to Dice 0.96 for segmentation in both datasets and accuracies 0.88, and 0.80 for classification, respectively.

CONCLUSION

Deep learning gives prominent results for segmentation and feature extraction in medical image processing. The combination of XGboost and SVM improves the results obtained.

The Development of a Practical Artificial Intelligence Tool for Diagnosing and Evaluating Autism Spectrum Disorder: Multicenter Study

Background

Autism spectrum disorder (ASD) is a complex neurodevelopmental disorder with an unknown etiology. Early diagnosis and intervention are key to improving outcomes for patients with ASD. Structural magnetic resonance imaging (sMRI) has been widely used in clinics to facilitate the diagnosis of brain diseases such as brain tumors. However, sMRI is less frequently used to investigate neurological and psychiatric disorders, such as ASD, owing to the subtle, if any, anatomical changes of the brain.

Objective

This study aimed to investigate the possibility of identifying structural patterns in the brain of patients with ASD as potential biomarkers in the diagnosis and evaluation of ASD in clinics.

Methods

We developed a novel 2-level histogram-based morphometry (HBM) classification framework in which an algorithm based on a 3D version of the histogram of oriented gradients (HOG) was used to extract features from sMRI data. We applied this framework to distinguish patients with ASD from healthy controls using 4 datasets from the second edition of the Autism Brain Imaging Data Exchange, including the ETH Zürich (ETH), NYU Langone Medical Center: Sample 1, Oregon Health and Science University, and Stanford University (SU) sites. We used a stratified 10-fold cross-validation method to evaluate the model performance, and we applied the Naive Bayes approach to identify the predictive ASD-related brain regions based on classification contributions of each HOG feature.

Results

On the basis of the 3D HOG feature extraction method, our proposed HBM framework achieved an area under the curve (AUC) of >0.75 in each dataset, with the highest AUC of 0.849 in the ETH site. We compared the 3D HOG algorithm with the original 2D HOG algorithm, which showed an accuracy improvement of >4% in each dataset, with the highest improvement of 14% (6/42) in the SU site. A comparison of the 3D HOG algorithm with the scale-invariant feature transform algorithm showed an AUC improvement of >18% in each dataset. Furthermore, we identified ASD-related brain regions based on the sMRI images. Some of these regions (eg, frontal gyrus, temporal gyrus, cingulate gyrus, postcentral gyrus, precuneus, caudate, and hippocampus) are known to be implicated in ASD in prior neuroimaging literature. We also identified less well-known regions that may play unrecognized roles in ASD and be worth further investigation.

Conclusions

Our research suggested that it is possible to identify neuroimaging biomarkers that can distinguish patients with ASD from healthy controls based on the more cost-effective sMRI images of the brain. We also demonstrated the potential of applying data-driven artificial intelligence technology in the clinical setting of neurological and psychiatric disorders, which usually harbor subtle anatomical changes in the brain that are often invisible to the human eye.

Keypoint Transfer for Fast Whole-Body Segmentation

We introduce an approach for image segmentation based on sparse correspondences between keypoints in testing and training images. Keypoints represent automatically identified distinctive image locations, where each keypoint correspondence suggests a transformation between images. We use these correspondences to transfer the label maps of entire organs from the training images to the test image. The keypoint transfer algorithm includes three steps: 1) keypoint matching; 2) voting-based keypoint labeling; and 3) keypoint-based probabilistic transfer of organ segmentations. We report segmentation results for abdominal organs in whole-body CT and MRI, as well as in contrast-enhanced CT and MRI. Our method offers a speed-up of about three orders of magnitude in comparison with common multi-atlas segmentation while achieving an accuracy that compares favorably. Moreover, keypoint transfer does not require the registration to an atlas or a training phase. Finally, the method allows for the segmentation of scans with a highly variable field-of-view.

Neuroimage signature from salient keypoints is highly specific to individuals and shared by close relatives

Neuroimaging studies typically adopt a common feature space for all data, which may obscure aspects of neuroanatomy only observable in subsets of a population, e.g. cortical folding patterns unique to individuals or shared by close relatives. Here, we propose to model individual variability using a distinctive keypoint signature: a set of unique, localized patterns, detected automatically in each image by a generic saliency operator. The similarity of an image pair is then quantified by the proportion of keypoints they share using a novel Jaccard-like measure of set overlap. Experiments demonstrate the keypoint method to be highly efficient and accurate, using a set of 7536 T1-weighted MRIs pooled from four public neuroimaging repositories, including twins, non-twin siblings, and 3334 unique subjects. All same-subject image pairs are identified by a similarity threshold despite confounds including aging and neurodegenerative disease progression. Outliers reveal previously unknown data labeling inconsistencies, demonstrating the usefulness of the keypoint signature as a computational tool for curating large neuroimage datasets.

An empirical assessment of appearance descriptors applied to MRI for automated diagnosis of TLE and MCI

OBJECTIVE

Differential diagnosis of mild cognitive impairment MCI and temporal lobe epilepsy TLE is a debated issue, specifically because these conditions may coincide in the elderly population. We evaluate automated differential diagnosis based on characteristics derived from structural brain MRI of different brain regions.

METHODS

In 22 healthy controls, 19 patients with MCI, and 17 patients with TLE we used scale invariant feature transform (SIFT), local binary patterns (LBP), and wavelet-based features and investigate their predictive performance for MCI and TLE.

RESULTS

The classification based on SIFT features resulted in an accuracy of 81% of MCI vs. TLE and reasonable generalizability. Local binary patterns yielded satisfactory diagnostic performance with up to 94.74% sensitivity and 88.24% specificity in the right Thalamus for the distinction of MCI vs. TLE, but with limited generalizable. Wavelet features yielded similar results as LPB with 94.74% sensitivity and 82.35% specificity but generalize better.

SIGNIFICANCE

Features beyond volume analysis are a valid approach when applied to specific regions of the brain. Most significant information could be extracted from the thalamus, frontal gyri, and temporal regions, among others. These results suggest that analysis of changes of the central nervous system should not be limited to the most typical regions of interest such as the hippocampus and parahippocampal areas. Region-independent approaches can add considerable information for diagnosis. We emphasize the need to characterize generalizability in future studies, as our results demonstrate that not doing so can lead to overestimation of classification results.

LIMITATIONS

The data used within this study allows for separation of MCI and TLE subjects using a simple age threshold. While we present a strong indication that the presented method is age-invariant and therefore agnostic to this situation, new data would be needed for a rigorous empirical assessment of this findings.

Multi-modal brain fingerprinting: A manifold approximation based framework

This work presents an efficient framework, based on manifold approximation, for generating brain fingerprints from multi-modal data. The proposed framework represents images as bags of local features which are used to build a subject proximity graph. Compact fingerprints are obtained by projecting this graph in a low-dimensional manifold using spectral embedding. Experiments using the T1/T2-weighted MRI, diffusion MRI, and resting-state fMRI data of 945 Human Connectome Project subjects demonstrate the benefit of combining multiple modalities, with multi-modal fingerprints more discriminative than those generated from individual modalities. Results also highlight the link between fingerprint similarity and genetic proximity, monozygotic twins having more similar fingerprints than dizygotic or non-twin siblings. This link is also reflected in the differences of feature correspondences between twin/sibling pairs, occurring in major brain structures and across hemispheres. The robustness of the proposed framework to factors like image alignment and scan resolution, as well as the reproducibility of results on retest scans, suggest the potential of multi-modal brain fingerprinting for characterizing individuals in a large cohort analysis.

A hybrid computational approach for efficient Alzheimer's disease classification based on heterogeneous data

There is currently a lack of an efficient, objective and systemic approach towards the classification of Alzheimer's disease (AD), due to its complex etiology and pathogenesis. As AD is inherently dynamic, it is also not clear how the relationships among AD indicators vary over time. To address these issues, we propose a hybrid computational approach for AD classification and evaluate it on the heterogeneous longitudinal AIBL dataset. Specifically, using clinical dementia rating as an index of AD severity, the most important indicators (mini-mental state examination, logical memory recall, grey matter and cerebrospinal volumes from MRI and active voxels from PiB-PET brain scans, ApoE, and age) can be automatically identified from parallel data mining algorithms. In this work, Bayesian network modelling across different time points is used to identify and visualize time-varying relationships among the significant features, and importantly, in an efficient way using only coarse-grained data. Crucially, our approach suggests key data features and their appropriate combinations that are relevant for AD severity classification with high accuracy. Overall, our study provides insights into AD developments and demonstrates the potential of our approach in supporting efficient AD diagnosis.

Characterization of brain anatomical patterns by comparing region intensity distributions: Applications to the description of Alzheimer's disease

PURPOSE

This work presents an automatic characterization of the Alzheimer's disease describing the illness as a multidirectional departure from a baseline defining the control state, being these directions determined by a distance between functional-equivalent anatomical regions.

METHODS

After a brain parcellation, a region is described by its histogram of gray levels, and the Earth mover's distance establishes how close or far these regions are. The medoid of the control group is set as the reference and any brain is characterized by its set of distances to this medoid.

EVALUATION

This hypothesis was assessed by separating groups of patients with mild Alzheimer's disease and mild cognitive impairment from control subjects, using a subset of the Open Access Series of Imaging Studies (OASIS) database. An additional experiment evaluated the method generalization and consisted in training with the OASIS data and testing with the Minimal Interval Resonance Imaging in Alzheimer's disease (MIRIAD) database.

RESULTS

Classification between controls and patients with AD resulted in an equal error rate of 0.1 (90% of sensitivity and specificity at the same time). The automatic ranking of regions resulting is in strong agreement with those regions described as important in clinical practice. Classification with different databases results in a sensitivity of 85% and a specificity of 91%.

CONCLUSIONS

This method automatically finds out a multidimensional expression of the AD, which is directly related to the anatomical changes in specific areas such as the hippocampus, the amygdala, the planum temporale, and thalamus.

Robust automated constellation-based landmark detection in human brain imaging

A robust fully automated algorithm for identifying an arbitrary number of landmark points in the human brain is described and validated. The proposed method combines statistical shape models with trained brain morphometric measures to estimate midbrain landmark positions reliably and accurately. Gross morphometric constraints provided by automatically identified eye centers and the center of the head mass are shown to provide robust initialization in the presence of large rotations in the initial head orientation. Detection of primary midbrain landmarks are used as the foundation from which extended detection of an arbitrary set of secondary landmarks in different brain regions by applying a linear model estimation and principle component analysis. This estimation model sequentially uses the knowledge of each additional detected landmark as an improved foundation for improved prediction of the next landmark location. The accuracy and robustness of the presented method was evaluated by comparing the automatically generated results to two manual raters on 30 identified landmark points extracted from each of 30 T1-weighted magnetic resonance images. For the landmarks with unambiguous anatomical definitions, the average discrepancy between the algorithm results and each human observer differed by less than 1 mm from the average inter-observer variability when the algorithm was evaluated on imaging data collected from the same site as the model building data. Similar results were obtained when the same model was applied to a set of heterogeneous image volumes from seven different collection sites representing 3 scanner manufacturers. This method is reliable for general application in large-scale multi-site studies that consist of a variety of imaging data with different orientations, spacings, origins, and field strengths.

Classification of brain disease in magnetic resonance images using two-stage local feature fusion

BACKGROUND

Many classification methods have been proposed based on magnetic resonance images. Most methods rely on measures such as volume, the cerebral cortical thickness and grey matter density. These measures are susceptible to the performance of registration and limited in representation of anatomical structure. This paper proposes a two-stage local feature fusion method, in which deformable registration is not desired and anatomical information is represented from moderate scale.

METHODS

Keypoints are firstly extracted from scale-space to represent anatomical structure. Then, two kinds of local features are calculated around the keypoints, one for correspondence and the other for representation. Scores are assigned for keypoints to quantify their effect in classification. The sum of scores for all effective keypoints is used to determine which group the test subject belongs to.

RESULTS

We apply this method to magnetic resonance images of Alzheimer's disease and Parkinson's disease. The advantage of local feature in correspondence and representation contributes to the final classification. With the help of local feature (Scale Invariant Feature Transform, SIFT) in correspondence, the performance becomes better. Local feature (Histogram of Oriented Gradient, HOG) extracted from 16×16 cell block obtains better results compared with 4×4 and 8×8 cell block.

DISCUSSION

This paper presents a method which combines the effect of SIFT descriptor in correspondence and the representation ability of HOG descriptor in anatomical structure. This method has the potential in distinguishing patients with brain disease from controls.

Classification of brain disease from magnetic resonance images based on multi-level brain partitions

In this paper, we present a classification method based on the multi-level brain partitions. Bag-of-visual-words model is used. Firstly, the representative SIFT features are extracted from brain template as the basic visual words. Secondly, individual MR images are described using the basic visual words and support vector machine classifiers are trained for different brain partitions respectively. Thirdly, the final classification is derived from the combination of multiple classifiers. We apply this method to MR images of Alzheimer's disease and Parkinson's disease. The results demonstrate that the multi-level partitions favors the classification accuracy of brain disease from MR images.

Classification of amyloid status using machine learning with histograms of oriented 3D gradients

Brain amyloid burden may be quantitatively assessed from positron emission tomography imaging using standardised uptake value ratios. Using these ratios as an adjunct to visual image assessment has been shown to improve inter-reader reliability, however, the amyloid positivity threshold is dependent on the tracer and specific image regions used to calculate the uptake ratio. To address this problem, we propose a machine learning approach to amyloid status classification, which is independent of tracer and does not require a specific set of regions of interest. Our method extracts feature vectors from amyloid images, which are based on histograms of oriented three-dimensional gradients. We optimised our method on 133 ¹⁸F-florbetapir brain volumes, and applied it to a separate test set of 131 volumes. Using the same parameter settings, we then applied our method to 209 ¹¹C-PiB images and 128 ¹⁸F-florbetaben images. We compared our method to classification results achieved using two other methods: standardised uptake value ratios and a machine learning method based on voxel intensities. Our method resulted in the largest mean distances between the subjects and the classification boundary, suggesting that it is less likely to make low-confidence classification decisions. Moreover, our method obtained the highest classification accuracy for all three tracers, and consistently achieved above 96% accuracy.

•

A machine learning approach to brain amyloid status classification is proposed.

•

The method is independent of PET tracer and requires little recalibration.

•

Classification accuracy was higher than SUVR for three amyloid tracers.

Dictionary Pruning with Visual Word Significance for Medical Image Retrieval

Content-based medical image retrieval (CBMIR) is an active research area for disease diagnosis and treatment but it can be problematic given the small visual variations between anatomical structures. We propose a retrieval method based on a bag-of-visual-words (BoVW) to identify discriminative characteristics between different medical images with Pruned Dictionary based on Latent Semantic Topic description. We refer to this as the PD-LST retrieval. Our method has two main components. First, we calculate a topic-word significance value for each visual word given a certain latent topic to evaluate how the word is connected to this latent topic. The latent topics are learnt, based on the relationship between the images and words, and are employed to bridge the gap between low-level visual features and high-level semantics. These latent topics describe the images and words semantically and can thus facilitate more meaningful comparisons between the words. Second, we compute an overall-word significance value to evaluate the significance of a visual word within the entire dictionary. We designed an iterative ranking method to measure overall-word significance by considering the relationship between all latent topics and words. The words with higher values are considered meaningful with more significant discriminative power in differentiating medical images. We evaluated our method on two public medical imaging datasets and it showed improved retrieval accuracy and efficiency.

Pairwise Latent Semantic Association for Similarity Computation in Medical Imaging

Retrieving medical images that present similar diseases is an active research area for diagnostics and therapy. However, it can be problematic given the visual variations between anatomical structures. In this paper, we propose a new feature extraction method for similarity computation in medical imaging. Instead of the low-level visual appearance, we design a CCA-PairLDA feature representation method to capture the similarity between images with high-level semantics. First, we extract the PairLDA topics to represent an image as a mixture of latent semantic topics in an image pair context. Second, we generate a CCA-correlation model to represent the semantic association between an image pair for similarity computation. While PairLDA adjusts the latent topics for all image pairs, CCA-correlation helps to associate an individual image pair. In this way, the semantic descriptions of an image pair are closely correlated, and naturally correspond to similarity computation between images. We evaluated our method on two public medical imaging datasets for image retrieval and showed improved performance.

Learning-based prediction of gestational age from ultrasound images of the fetal brain

We propose an automated framework for predicting gestational age (GA) and neurodevelopmental maturation of a fetus based on 3D ultrasound (US) brain image appearance. Our method capitalizes on age-related sonographic image patterns in conjunction with clinical measurements to develop, for the first time, a predictive age model which improves on the GA-prediction potential of US images. The framework benefits from a manifold surface representation of the fetal head which delineates the inner skull boundary and serves as a common coordinate system based on cranial position. This allows for fast and efficient sampling of anatomically-corresponding brain regions to achieve like-for-like structural comparison of different developmental stages. We develop bespoke features which capture neurosonographic patterns in 3D images, and using a regression forest classifier, we characterize structural brain development both spatially and temporally to capture the natural variation existing in a healthy population (N=447) over an age range of active brain maturation (18-34weeks). On a routine clinical dataset (N=187) our age prediction results strongly correlate with true GA (r=0.98,accurate within±6.10days), confirming the link between maturational progression and neurosonographic activity observable across gestation. Our model also outperforms current clinical methods by ±4.57 days in the third trimester-a period complicated by biological variations in the fetal population. Through feature selection, the model successfully identified the most age-discriminating anatomies over this age range as being the Sylvian fissure, cingulate, and callosal sulci.

A Feature-Based Approach to Big Data Analysis of Medical Images

This paper proposes an inference method well-suited to large sets of medical images. The method is based upon a framework where distinctive 3D scale-invariant features are indexed efficiently to identify approximate nearest-neighbor (NN) feature matches-in O (log N) computational complexity in the number of images N. It thus scales well to large data sets, in contrast to methods based on pair-wise image registration or feature matching requiring O(N) complexity. Our theoretical contribution is a density estimator based on a generative model that generalizes kernel density estimation and K-nearest neighbor (KNN) methods.. The estimator can be used for on-the-fly queries, without requiring explicit parametric models or an off-line training phase. The method is validated on a large multi-site data set of 95,000,000 features extracted from 19,000 lung CT scans. Subject-level classification identifies all images of the same subjects across the entire data set despite deformation due to breathing state, including unintentional duplicate scans. State-of-the-art performance is achieved in predicting chronic pulmonary obstructive disorder (COPD) severity across the 5-category GOLD clinical rating, with an accuracy of 89% if both exact and one-off predictions are considered correct.

Keypoint Transfer Segmentation

We present an image segmentation method that transfers label maps of entire organs from the training images to the novel image to be segmented. The transfer is based on sparse correspondences between keypoints that represent automatically identified distinctive image locations. Our segmentation algorithm consists of three steps: (i) keypoint matching, (ii) voting-based keypoint labeling, and (iii) keypoint-based probabilistic transfer of organ label maps. We introduce generative models for the inference of keypoint labels and for image segmentation, where keypoint matches are treated as a latent random variable and are marginalized out as part of the algorithm. We report segmentation results for abdominal organs in whole-body CT and in contrast-enhanced CT images. The accuracy of our method compares favorably to common multi-atlas segmentation while offering a speed-up of about three orders of magnitude. Furthermore, keypoint transfer requires no training phase or registration to an atlas. The algorithm's robustness enables the segmentation of scans with highly variable field-of-view.

Comparative evaluation of registration algorithms in different brain databases with varying difficulty: results and insights

Evaluating various algorithms for the inter-subject registration of brain magnetic resonance images (MRI) is a necessary topic receiving growing attention. Existing studies evaluated image registration algorithms in specific tasks or using specific databases (e.g., only for skull-stripped images, only for single-site images, etc.). Consequently, the choice of registration algorithms seems task- and usage/parameter-dependent. Nevertheless, recent large-scale, often multi-institutional imaging-related studies create the need and raise the question whether some registration algorithms can 1) generally apply to various tasks/databases posing various challenges; 2) perform consistently well, and while doing so, 3) require minimal or ideally no parameter tuning. In seeking answers to this question, we evaluated 12 general-purpose registration algorithms, for their generality, accuracy and robustness. We fixed their parameters at values suggested by algorithm developers as reported in the literature. We tested them in 7 databases/tasks, which present one or more of 4 commonly-encountered challenges: 1) inter-subject anatomical variability in skull-stripped images; 2) intensity homogeneity, noise and large structural differences in raw images; 3) imaging protocol and field-of-view (FOV) differences in multi-site data; and 4) missing correspondences in pathology-bearing images. Totally 7,562 registrations were performed. Registration accuracies were measured by (multi-)expert-annotated landmarks or regions of interest (ROIs). To ensure reproducibility, we used public software tools, public databases (whenever possible), and we fully disclose the parameter settings. We show evaluation results, and discuss the performances in light of algorithms' similarity metrics, transformation models and optimization strategies. We also discuss future directions for the algorithm development and evaluations.

Classification and localization of early-stage Alzheimer's disease in magnetic resonance images using a patch-based classifier ensemble

INTRODUCTION

Classification methods have been proposed to detect Alzheimer’s disease (AD) using magnetic resonance images. Most rely on features such as the shape/volume of brain structures that need to be defined a priori. In this work, we propose a method that does not require either the segmentation of specific brain regions or the nonlinear alignment to a template. Besides classification, we also analyze which brain regions are discriminative between a group of normal controls and a group of AD patients.

METHODS

We perform 3D texture analysis using Local Binary Patterns computed at local image patches in the whole brain, combined in a classifier ensemble.We evaluate our method in a publicly available database including very mild-to-mild AD subjects and healthy elderly controls.

RESULTS

For the subject cohort including only mild AD subjects, the best results are obtained using a combination of large (30×30×30 and 40×40×40 voxels) patches. A spatial analysis on the best performing patches shows that these are located in the medial-temporal lobe and in the periventricular regions. When very mild AD subjects are included in the dataset, the small (10×10×10 voxels) patches perform best, with the most discriminative ones being located near the left hippocampus.

CONCLUSION

We show that our method is able not only to perform accurate classification, but also to localize dis-criminative brain regions, which are in accordance with the medical literature. This is achieved without the need to segment-specific brain structures and without performing nonlinear registration to a template, indicating that the method may be suitable for a clinical implementation that can help to diagnose AD at an earlier stage.

Extracting salient brain patterns for imaging-based classification of neurodegenerative diseases

Neurodegenerative diseases comprise a wide variety of mental symptoms whose evolution is not directly related to the visual analysis made by radiologists, who can hardly quantify systematic differences. Moreover, automatic brain morphometric analyses, that do perform this quantification, contribute very little to the comprehension of the disease, i.e., many of these methods classify but they do not produce useful anatomo-functional correlations. This paper presents a new fully automatic image analysis method that reveals discriminative brain patterns associated to the presence of neurodegenerative diseases, mining systematic differences and therefore grading objectively any neurological disorder. This is accomplished by a fusion strategy that mixes together bottom-up and top-down information flows. Bottom-up information comes from a multiscale analysis of different image features, while the top-down stage includes learning and fusion strategies formulated as a max-margin multiple-kernel optimization problem. The capacity of finding discriminative anatomic patterns was evaluated using the Alzheimer's disease (AD) as the use case. The classification performance was assessed under different configurations of the proposed approach in two public brain magnetic resonance datasets (OASIS-MIRIAD) with patients diagnosed with AD, showing an improvement varying from 6.2% to 13% in the equal error rate measure, with respect to what has been reported by the feature-based morphometry strategy. In terms of the anatomical analysis, discriminant regions found by the proposed approach highly correlates to what has been reported in clinical studies of AD.

Predicting fetal neurodevelopmental age from ultrasound images

We propose an automated framework for predicting age and neurodevelopmental maturation of a fetus based on 3D ultrasound (US) brain image appearance. A topology-preserving manifold representation of the fetal skull enabled design of bespoke scale-invariant image features. Our regression forest model used these features to learn a mapping from age-related sonographic image patterns to fetal age and development. The Sylvian Fissure was identified as a critical region for accurate age estimation, and restricting the search space to this anatomy improved prediction accuracy on a set of 130 healthy fetuses (error ± 3.8 days; r = 0.98 performing the best current clinical method. Our framework remained robust when applied to a routine clinical population.

Combined analysis of sMRI and fMRI imaging data provides accurate disease markers for hearing impairment

In this research, we developed a robust two-layer classifier that can accurately classify normal hearing (NH) from hearing impaired (HI) infants with congenital sensori-neural hearing loss (SNHL) based on their Magnetic Resonance (MR) images. Unlike traditional methods that examine the intensity of each single voxel, we extracted high-level features to characterize the structural MR images (sMRI) and functional MR images (fMRI). The Scale Invariant Feature Transform (SIFT) algorithm was employed to detect and describe the local features in sMRI. For fMRI, we constructed contrast maps and detected the most activated/de-activated regions in each individual. Based on those salient regions occurring across individuals, the bag-of-words strategy was introduced to vectorize the contrast maps. We then used a two-layer model to integrate these two types of features together. With the leave-one-out cross-validation approach, this integrated model achieved an AUC score of 0.90. Additionally, our algorithm highlighted several important brain regions that differentiated between NH and HI children. Some of these regions, e.g. planum temporale and angular gyrus, were well known auditory and visual language association regions. Others, e.g. the anterior cingulate cortex (ACC), were not necessarily expected to play a role in differentiating HI from NH children and provided a new understanding of brain function and of the disorder itself. These important brain regions provided clues about neuroimaging markers that may be relevant to the future use of functional neuroimaging to guide predictions about speech and language outcomes in HI infants who receive a cochlear implant. This type of prognostic information could be extremely useful and is currently not available to clinicians by any other means.

•

We probe brain structural and functional changes in hearing impaired (HI) infants.

•

We build a robust two-layer classifier that integrates sMRI and fMRI data.

•

This integrated model accurately separates HI from normal infants (AUC 0.9).

•

Our method detects important brain regions different between HI and normal infants.

•

Our method can include diverse types of data and be applied to other diseases.

Detecting brain structural changes as biomarker from magnetic resonance images using a local feature based SVM approach

Detecting brain structural changes from magnetic resonance (MR) images can facilitate early diagnosis and treatment of neurological and psychiatric diseases. Many existing methods require an accurate deformation registration, which is difficult to achieve and therefore prevents them from obtaining high accuracy. We develop a novel local feature based support vector machine (SVM) approach to detect brain structural changes as potential biomarkers. This approach does not require deformation registration and thus is less influenced by artifacts such as image distortion. We represent the anatomical structures based on scale invariant feature transform (SIFT). Likelihood scores calculated using feature-based morphometry is used as the criterion to categorize image features into three classes (healthy, patient and noise). Regional SVMs are trained to classify the three types of image features in different brain regions. Only healthy and patient features are used to predict the disease status of new brain images. An ensemble classifier is built from the regional SVMs to obtain better prediction accuracy. We apply this approach to 3D MR images of Alzheimer's disease, Parkinson's disease and bipolar disorder. The classification accuracy ranges between 70% and 87%. The highly predictive disease-related regions, which represent significant anatomical differences between the healthy and diseased, are shown in heat maps. The common and disease-specific brain regions are identified by comparing the highly predictive regions in each disease. All of the top-ranked regions are supported by literature. Thus, this approach will be a promising tool for assisting automatic diagnosis and advancing mechanism studies of neurological and psychiatric diseases.

Quantitative analysis of structural neuroimaging of mesial temporal lobe epilepsy

Mesial temporal lobe epilepsy (MTLE) is the most common of the surgically remediable drug-resistant epilepsies. MRI is the primary diagnostic tool to detect anatomical abnormalities and, when combined with EEG, can more accurately identify an epileptogenic lesion, which is often hippocampal sclerosis in cases of MTLE. As structural imaging technology has advanced the surgical treatment of MTLE and other lesional epilepsies, so too have the analysis techniques that are used to measure different structural attributes of the brain. These techniques, which are reviewed here and have been used chiefly in basic research of epilepsy and in studies of MTLE, have identified different types and the extent of anatomical abnormalities that can extend beyond the affected hippocampus. These results suggest that structural imaging and sophisticated imaging analysis could provide important information to identify networks capable of generating spontaneous seizures and ultimately help guide surgical therapy that improves postsurgical seizure-freedom outcomes.

Efficient and robust model-to-image alignment using 3D scale-invariant features

This paper presents feature-based alignment (FBA), a general method for efficient and robust model-to-image alignment. Volumetric images, e.g. CT scans of the human body, are modeled probabilistically as a collage of 3D scale-invariant image features within a normalized reference space. Features are incorporated as a latent random variable and marginalized out in computing a maximum a posteriori alignment solution. The model is learned from features extracted in pre-aligned training images, then fit to features extracted from a new image to identify a globally optimal locally linear alignment solution. Novel techniques are presented for determining local feature orientation and efficiently encoding feature intensity in 3D. Experiments involving difficult magnetic resonance (MR) images of the human brain demonstrate FBA achieves alignment accuracy similar to widely-used registration methods, while requiring a fraction of the memory and computation resources and offering a more robust, globally optimal solution. Experiments on CT human body scans demonstrate FBA as an effective system for automatic human body alignment where other alignment methods break down.

Speed of facial affect intensity recognition as an endophenotype of first-episode psychosis and associated limbic-cortical grey matter systems

BACKGROUND

Psychotic disorders are highly heritable such that the unaffected relatives of patients may manifest characteristics, or endophenotypes, that are more closely related to risk genes than the overt clinical condition. Facial affect processing is dependent on a distributed cortico-limbic network that is disrupted in psychosis. This study assessed facial affect processing and related brain structure as a candidate endophenotype of first-episode psychosis (FEP).

METHOD

Three samples comprising 30 FEP patients, 30 of their first-degree relatives and 31 unrelated healthy controls underwent assessment of facial affect processing and structural magnetic resonance imaging (sMRI) data. Multivariate analysis (partial least squares, PLS) was used to identify a grey matter (GM) system in which anatomical variation was associated with variation in facial affect processing speed.

RESULTS

The groups did not differ in their accuracy of facial affect intensity rating but differed significantly in speed of response, with controls responding faster than relatives, who responded faster than patients. Within the control group, variation in speed of affect processing was significantly associated with variation of GM density in amygdala, lateral temporal cortex, frontal cortex and cerebellum. However, this association between cortico-limbic GM density and speed of facial affect processing was absent in patients and their relatives.

CONCLUSIONS

Speed of facial affect processing presents as a candidate endophenotype of FEP. The normal association between speed of facial affect processing and cortico-limbic GM variation was disrupted in FEP patients and their relatives.

Automatic brain caudate nuclei segmentation and classification in diagnostic of Attention-Deficit/Hyperactivity Disorder

We present a fully automatic diagnostic imaging test for Attention-Deficit/Hyperactivity Disorder diagnosis assistance based on previously found evidences of caudate nucleus volumetric abnormalities. The proposed method consists of different steps: a new automatic method for external and internal segmentation of caudate based on Machine Learning methodologies; the definition of a set of new volume relation features, 3D Dissociated Dipoles, used for caudate representation and classification. We separately validate the contributions using real data from a pediatric population and show precise internal caudate segmentation and discrimination power of the diagnostic test, showing significant performance improvements in comparison to other state-of-the-art methods.

Multi-centre diagnostic classification of individual structural neuroimaging scans from patients with major depressive disorder

Quantitative abnormalities of brain structure in patients with major depressive disorder have been reported at a group level for decades. However, these structural differences appear subtle in comparison with conventional radiologically defined abnormalities, with considerable inter-subject variability. Consequently, it has not been possible to readily identify scans from patients with major depressive disorder at an individual level. Recently, machine learning techniques such as relevance vector machines and support vector machines have been applied to predictive classification of individual scans with variable success. Here we describe a novel hybrid method, which combines machine learning with feature selection and characterization, with the latter aimed at maximizing the accuracy of machine learning prediction. The method was tested using a multi-centre dataset of T(1)-weighted 'structural' scans. A total of 62 patients with major depressive disorder and matched controls were recruited from referred secondary care clinical populations in Aberdeen and Edinburgh, UK. The generalization ability and predictive accuracy of the classifiers was tested using data left out of the training process. High prediction accuracy was achieved (~90%). While feature selection was important for maximizing high predictive accuracy with machine learning, feature characterization contributed only a modest improvement to relevance vector machine-based prediction (~5%). Notably, while the only information provided for training the classifiers was T(1)-weighted scans plus a categorical label (major depressive disorder versus controls), both relevance vector machine and support vector machine 'weighting factors' (used for making predictions) correlated strongly with subjective ratings of illness severity. These results indicate that machine learning techniques have the potential to inform clinical practice and research, as they can make accurate predictions about brain scan data from individual subjects. Furthermore, machine learning weighting factors may reflect an objective biomarker of major depressive disorder illness severity, based on abnormalities of brain structure.

Blob-like feature extraction and matching for brain MR images

The cerebral cortex of the human brain is highly folded. It is useful for neuroscientists and clinical researchers to identify and/or quantify cortical folding patterns across individuals. The top (gyri) and bottom (sulci) of these folds resemble the "blob-like" features used in computer vision. In this article, we evaluate different blob detectors and descriptors on brain MR images, and introduce our own, the "brain blob detector and descriptor (BBDD)." For the first time blob detectors are considered as spatial filters under the scale-space framework and their impulse responses are manipulated for detecting the structures in our interest. The BBDD detector is tailored to the scale and structure of blob-like features that coincide with cortical folds, and its descriptors performed well at discriminating these features in our evaluation.

Morphometry based on effective and accurate correspondences of localized patterns (MEACOLP)

Local features in volumetric images have been used to identify correspondences of localized anatomical structures for brain morphometry. However, the correspondences are often sparse thus ineffective in reflecting the underlying structures, making it unreliable to evaluate specific morphological differences. This paper presents a morphometry method (MEACOLP) based on correspondences with improved effectiveness and accuracy. A novel two-level scale-invariant feature transform is used to enhance the detection repeatability of local features and to recall the correspondences that might be missed in previous studies. Template patterns whose correspondences could be commonly identified in each group are constructed to serve as the basis for morphometric analysis. A matching algorithm is developed to reduce the identification errors by comparing neighboring local features and rejecting unreliable matches. The two-sample t-test is finally adopted to analyze specific properties of the template patterns. Experiments are performed on the public OASIS database to clinically analyze brain images of Alzheimer's disease (AD) and normal controls (NC). MEACOLP automatically identifies known morphological differences between AD and NC brains, and characterizes the differences well as the scaling and translation of underlying structures. Most of the significant differences are identified in only a single hemisphere, indicating that AD-related structures are characterized by strong anatomical asymmetry. In addition, classification trials to differentiate AD subjects from NC confirm that the morphological differences are reliably related to the groups of interest.

Hippocampal atrophy in Alzheimer’s disease

SUMMARY New research criteria for Alzheimer’s disease (AD) and mild cognitive impairment (MCI) emphasize the use of imaging biomarkers in clinical diagnosis of these disorders. The volume loss of medial temporal lobe structures, especially hippocampal atrophy, is the best validated marker of AD. Manual tracing on MRI is the present gold standard for evaluating hippocampal volume; however, it is laborious and tracer-dependent. We categorized the most recent full- or semi-automated methods by the nature of the output of the method: size and shape of subcortical structures, cortical thickness, atrophy-rate and voxel- and region-based characteristics. The features of each method are introduced. The findings in structural MRI studies, especially in those studies utilizing the most recent methods, and the accuracies of those new methods in differentiating AD from healthy controls and stable MCI from progressive MCI are reviewed.

A feature-based developmental model of the infant brain in structural MRI

In this paper, anatomical development is modeled as a collection of distinctive image patterns localized in space and time. A Bayesian posterior probability is defined over a random variable of subject age, conditioned on data in the form of scale-invariant image features. The model is automatically learned from a large set of images exhibiting significant variation, used to discover anatomical structure related to age and development, and fit to new images to predict age. The model is applied to a set of 230 infant structural MRIs of 92 subjects acquired at multiple sites over an age range of 8-590 days. Experiments demonstrate that the model can be used to identify age-related anatomical structure, and to predict the age of new subjects with an average error of 72 days.

ODVBA: optimally-discriminative voxel-based analysis

Gaussian smoothing of images prior to applying voxel-based statistics is an important step in voxel-based analysis and statistical parametric mapping (VBA-SPM) and is used to account for registration errors, to Gaussianize the data and to integrate imaging signals from a region around each voxel. However, it has also become a limitation of VBA-SPM based methods, since it is often chosen empirically and lacks spatial adaptivity to the shape and spatial extent of the region of interest, such as a region of atrophy or functional activity. In this paper, we propose a new framework, named optimally-discriminative voxel-based analysis (ODVBA), for determining the optimal spatially adaptive smoothing of images, followed by applying voxel-based group analysis. In ODVBA, nonnegative discriminative projection is applied regionally to get the direction that best discriminates between two groups, e.g., patients and controls; this direction is equivalent to local filtering by an optimal kernel whose coefficients define the optimally discriminative direction. By considering all the neighborhoods that contain a given voxel, we then compose this information to produce the statistic for each voxel. Finally, permutation tests are used to obtain a statistical parametric map of group differences. ODVBA has been evaluated using simulated data in which the ground truth is known and with data from an Alzheimer's disease (AD) study. The experimental results have shown that the proposed ODVBA can precisely describe the shape and location of structural abnormality.