A Riemannian Framework for Tensor Computing

Log-Cholesky filtering of diffusion tensor fields: Impact on noise reduction

Diffusion tensor imaging (DTI) is a powerful neuroimaging technique that provides valuable insights into the microstructure and connectivity of the brain. By measuring the diffusion of water molecules along neuronal fibers, DTI allows the visualization and study of intricate networks of neural pathways. DTI is a noise-sensitive method, where a low signal-to-noise ratio (SNR) results in significant errors in the estimated tensor field. Tensor field regularization is an effective solution for noise reduction. Diffusion tensors are represented by symmetric positive-definite (SPD) matrices. The space of SPD matrices may be viewed as a Riemannian manifold after defining a suitable metric on its tangent bundle. The Log-Cholesky metric is a recently developed concept with advantages over previously defined Riemannian metrics, such as the affine-invariant and Log-Euclidean metrics. The utility of the Log-Cholesky metric for tensor field regularization and noise reduction has not been investigated in detail. This manuscript provides a quantitative investigation of the impact of Log-Cholesky filtering on noise reduction in DTI. It also provides sufficient details of the linear algebra and abstract differential geometry concepts necessary to implement this technique as a simple and effective solution to filtering diffusion tensor fields.

A self-supervised deep Riemannian representation to classify parkinsonian fixational patterns

Parkinson's disease (PD) is the second most prevalent neurodegenerative disorder, and it remains incurable. Currently there is no definitive biomarker for detecting PD, measuring its severity, or monitoring of treatments. Recently, oculomotor fixation abnormalities have emerged as a sensitive biomarker to discriminate Parkinsonian patterns from a control population, even at early stages. For oculomotor analysis, current experimental setups use invasive and restrictive capture protocols that limit the transfer in clinical routine. Alternatively, computational approaches to support the PD diagnosis are strictly based on supervised strategies, depending of large labeled data, and introducing an inherent expert-bias. This work proposes a self-supervised architecture based on Riemannian deep representation to learn oculomotor fixation patterns from compact descriptors. Firstly, deep convolutional features are recovered from oculomotor fixation video slices, and then encoded in compact symmetric positive matrices (SPD) to summarize second-order relationships. Each SPD input matrix is projected onto a Riemannian encoder until obtain a SPD embedding. Then, a Riemannian decoder reconstructs SPD matrices while preserving the geometrical manifold structure. The proposed architecture successfully recovers geometric patterns in the embeddings without any label diagnosis supervision, and demonstrates the capability to be discriminative regarding PD patterns. In a retrospective study involving 13 healthy adults and 13 patients diagnosed with PD, the proposed Riemannian representation achieved an average accuracy of 95.6% and an AUC of 99% during a binary classification task using a Support Vector Machine.

ℛSCZ: A Riemannian schizophrenia diagnosis framework based on the multiplexity of EEG-based dynamic functional connectivity patterns

Abnormal electrophysiological (EEG) activity has been largely reported in schizophrenia (SCZ). In the last decade, research has focused to the automatic diagnosis of SCZ via the investigation of an EEG aberrant activity and connectivity linked to this mental disorder. These studies followed various preprocessing steps of EEG activity focusing on frequency-dependent functional connectivity brain network (FCBN) construction disregarding the topological dependency among edges. FCBN belongs to a family of symmetric positive definite (SPD) matrices forming the Riemannian manifold. Due to its unique geometric properties, the whole analysis of FCBN can be performed on the Riemannian geometry of the SPD space. The advantage of the analysis of FCBN on the SPD space is that it takes into account all the pairwise interdependencies as a whole. However, only a few studies have adopted a FCBN analysis on the SPD manifold, while no study exists on the analysis of dynamic FCBN (dFCBN) tailored to SCZ. In the present study, I analyzed two open EEG-SCZ datasets under a Riemannian geometry of SPD matrices for the dFCBN analysis proposing also a multiplexity index that quantifies the associations of multi-frequency brainwave patterns. I adopted a machine learning procedure employing a leave-one-subject-out cross-validation (LOSO-CV) using snapshots of dFCBN from (N-1) subjects to train a battery of classifiers. Each classifier operated in the inter-subject dFCBN distances of sample covariance matrices (SCMs) following a rhythm-dependent decision and a multiplex-dependent one. The proposed ℛSCZ decoder supported both the Riemannian geometry of SPD and the multiplexity index DC reaching an absolute accuracy (100 %) in both datasets in the virtual default mode network (DMN) source space.

Bayesian estimation of covariate assisted principal regression for brain functional connectivity

This paper presents a Bayesian reformulation of covariate-assisted principal regression for covariance matrix outcomes to identify low-dimensional components in the covariance associated with covariates. By introducing a geometric approach to the covariance matrices and leveraging Euclidean geometry, we estimate dimension reduction parameters and model covariance heterogeneity based on covariates. This method enables joint estimation and uncertainty quantification of relevant model parameters associated with heteroscedasticity. We demonstrate our approach through simulation studies and apply it to analyze associations between covariates and brain functional connectivity using data from the Human Connectome Project.

Log-Euclidean metric based covariance propagation on SPD manifold for continuous-discrete extended Kalman filtering

For nonlinear systems with continuous dynamic and discrete measurements, a Log-Euclidean metric (LEM) based novel scheme is proposed to refine the covariance integration steps of continuous-discrete Extended Kalman filter (CDEKF). In CDEKF, the covariance differential equation is usually integrated with regular Euclidean matrix operations, which actually ignores the Riemannian structure of underlying space and poses a limit on the further improvement of estimation accuracy. To overcome this drawback, this work proposes to define the covariance variable on the manifold of symmetric positive definite (SPD) matrices and propagate it using the Log-Euclidean metric. To embed the LEM based novel propagation scheme, the manifold integration of the covariance for LEMCDEKF is proposed together with the details of efficient realization, which can integrate the covariance on SPD manifold and avoid the drawback of Euclidean scheme. Numerical simulations certify the new method's superior accuracy than conventional methods.

Transitional chelal digit patterns in saprophagous astigmatan mites

Changes in the functional shape of astigmatan mite moveable digit profiles are examined to test if Tyrophagus putrescentiae (Acaridae) is a trophic intermediate between a typical micro-saprophagous carpoglyphid (Carpoglyphus lactis) and a common macro-saprophagous glycyphagid (Glycyphagus domesticus). Digit tip elongation in these mites is decoupled from the basic physics of optimising moveable digit inertia. Investment in the basal ramus/coronoid process compared to that for the moveable digit mastication length varies with feeding style. A differentiated ascending ramus is indicated in C. lactis and in T. putrescentiae for different trophic reasons. Culturing affects relative investments in C. lactis. A markedly different style of feeding is inferred for the carpoglyphid. The micro-saprophagous acarid does not have an intermediate pattern of trophic functional form between the other two species. Mastication surface shape complexity confirms the acarid to be heterodontous. T. putrescentiae is a particularly variably formed species trophically. A plausible evolutionary path for the gradation of forms is illustrated. Digit form and strengthening to resist bending under occlusive loads is explored in detail. Extensions to the analytical approach are suggested to confirm the decoupling of moveable digit pattern from cheliceral and chelal adaptations. Caution is expressed when interpreting ordinations of multidimensional data in mites.

EEG functional connectivity as a Riemannian mediator: An application to malnutrition and cognition

Mediation analysis assesses whether an exposure directly produces changes in cognitive behavior or is influenced by intermediate "mediators". Electroencephalographic (EEG) spectral measurements have been previously used as effective mediators representing diverse aspects of brain function. However, it has been necessary to collapse EEG measures onto a single scalar using standard mediation methods. In this article, we overcome this limitation and examine EEG frequency-resolved functional connectivity measures as a mediator using the full EEG cross-spectral tensor (CST). Since CST samples do not exist in Euclidean space but in the Riemannian manifold of positive-definite tensors, we transform the problem, allowing for the use of classic multivariate statistics. Toward this end, we map the data from the original manifold space to the Euclidean tangent space, eliminating redundant information to conform to a "compressed CST." The resulting object is a matrix with rows corresponding to frequencies and columns to cross spectra between channels. We have developed a novel matrix mediation approach that leverages a nuclear norm regularization to determine the matrix-valued regression parameters. Furthermore, we introduced a global test for the overall CST mediation and a test to determine specific channels and frequencies driving the mediation. We validated the method through simulations and applied it to our well-studied 50+-year Barbados Nutrition Study dataset by comparing EEGs collected in school-age children (5-11 years) who were malnourished in the first year of life with those of healthy classmate controls. We hypothesized that the CST mediates the effect of malnutrition on cognitive performance. We can now explicitly pinpoint the frequencies (delta, theta, alpha, and beta bands) and regions (frontal, central, and occipital) in which functional connectivity was altered in previously malnourished children, an improvement to prior studies. Understanding the specific networks impacted by a history of postnatal malnutrition could pave the way for developing more targeted and personalized therapeutic interventions. Our methods offer a versatile framework applicable to mediation studies encompassing matrix and Hermitian 3D tensor mediators alongside scalar exposures and outcomes, facilitating comprehensive analyses across diverse research domains.

Modeling Tree-like Heterophily on Symmetric Matrix Manifolds

Tree-like structures, characterized by hierarchical relationships and power-law distributions, are prevalent in a multitude of real-world networks, ranging from social networks to citation networks and protein-protein interaction networks. Recently, there has been significant interest in utilizing hyperbolic space to model these structures, owing to its capability to represent them with diminished distortions compared to flat Euclidean space. However, real-world networks often display a blend of flat, tree-like, and circular substructures, resulting in heterophily. To address this diversity of substructures, this study aims to investigate the reconstruction of graph neural networks on the symmetric manifold, which offers a comprehensive geometric space for more effective modeling of tree-like heterophily. To achieve this objective, we propose a graph convolutional neural network operating on the symmetric positive-definite matrix manifold, leveraging Riemannian metrics to facilitate the scheme of information propagation. Extensive experiments conducted on semi-supervised node classification tasks validate the superiority of the proposed approach, demonstrating that it outperforms comparative models based on Euclidean and hyperbolic geometries.

Machine learning for hand pose classification from phasic and tonic EMG signals during bimanual activities in virtual reality

Myoelectric prostheses have recently shown significant promise for restoring hand function in individuals with upper limb loss or deficiencies, driven by advances in machine learning and increasingly accessible bioelectrical signal acquisition devices. Here, we first introduce and validate a novel experimental paradigm using a virtual reality headset equipped with hand-tracking capabilities to facilitate the recordings of synchronized EMG signals and hand pose estimation. Using both the phasic and tonic EMG components of data acquired through the proposed paradigm, we compare hand gesture classification pipelines based on standard signal processing features, convolutional neural networks, and covariance matrices with Riemannian geometry computed from raw or xDAWN-filtered EMG signals. We demonstrate the performance of the latter for gesture classification using EMG signals. We further hypothesize that introducing physiological knowledge in machine learning models will enhance their performances, leading to better myoelectric prosthesis control. We demonstrate the potential of this approach by using the neurophysiological integration of the "move command" to better separate the phasic and tonic components of the EMG signals, significantly improving the performance of sustained posture recognition. These results pave the way for the development of new cutting-edge machine learning techniques, likely refined by neurophysiology, that will further improve the decoding of real-time natural gestures and, ultimately, the control of myoelectric prostheses.

Parent-child couples display shared neural fingerprints while listening to stories

Neural fingerprinting is a method to identify individuals from a group of people. Here, we established a new connectome-based identification model and used diffusion maps to show that biological parent-child couples share functional connectivity patterns while listening to stories. These shared fingerprints enabled the identification of children and their biological parents from a group of parents and children. Functional patterns were evident in both cognitive and sensory brain networks. Defining "typical" shared biological parent-child brain patterns may enable predicting or even preventing impaired parent-child connections that develop due to genetic or environmental causes. Finally, we argue that the proposed framework opens new opportunities to link similarities in connectivity patterns to behavioral, psychological, and medical phenomena among other populations. To our knowledge, this is the first study to reveal the neural fingerprint that represents distinct biological parent-child couples.

Modeling Complex EEG Data Distribution on the Riemannian Manifold Toward Outlier Detection and Multimodal Classification

OBJECTIVE

The usage of Riemannian geometry for Brain-computer interfaces (BCIs) has gained momentum in recent years. Most of the machine learning techniques proposed for Riemannian BCIs consider the data distribution on a manifold to be unimodal. However, the distribution is likely to be multimodal rather than unimodal since high-data variability is a crucial limitation of electroencephalography (EEG). In this paper, we propose a novel data modeling method for considering complex data distributions on a Riemannian manifold of EEG covariance matrices, aiming to improve BCI reliability.

METHODS

Our method, Riemannian spectral clustering (RiSC), represents EEG covariance matrix distribution on a manifold using a graph with proposed similarity measurement based on geodesic distances, then clusters the graph nodes through spectral clustering. This allows flexibility to model both a unimodal and a multimodal distribution on a manifold. RiSC can be used as a basis to design an outlier detector named outlier detection Riemannian spectral clustering (odenRiSC) and a multimodal classifier named multimodal classifier Riemannian spectral clustering (mcRiSC). All required parameters of odenRiSC/mcRiSC are selected in data-driven manner. Moreover, there is no need to pre-set a threshold for outlier detection and the number of modes for multimodal classification.

RESULTS

The experimental evaluation revealed odenRiSC can detect EEG outliers more accurately than existing methods and mcRiSC outperformed the standard unimodal classifier, especially on high-variability datasets.

CONCLUSION

odenRiSC/mcRiSC are anticipated to contribute to making real-life BCIs outside labs and neuroergonomics applications more robust.

SIGNIFICANCE

RiSC can work as a robust EEG outlier detector and multimodal classifier.

Riemannian frameworks for the harmonization of resting-state functional MRI scans

Magnetic Resonance Imaging provides unprecedented images of the brain. Unfortunately, scanners and acquisition protocols can significantly impact MRI scans. The development of statistical methods able to reduce this variability without altering the relevant information in the scans, often coined harmonization methods, has been the topic of an increasing research effort supported by the recent growth of publicly available neuroimaging data sets and new possibilities for combining them to achieve greater statistical power. In this work, we focus on the challenges specifically raised by the harmonization of resting-state functional MRI scans. We propose to harmonize resting-state fMRI scans by reducing the impact of covariates such as scanner differences and scanning protocols on their associated functional connectomes and then propagating the changes back to the rs-fMRI time series. We use Riemannian geometric frameworks to preserve the mathematical properties of functional connectomes during their harmonization, and we demonstrate how state-of-the-art harmonization methods can be embedded within these frameworks to reduce covariates effects while preserving the relevant clinical information associated with aging or brain disorders. During our experiments, a large set of synthetic data was generated and processed to compare eighty variants of the proposed approach. The framework achieving the best harmonization was then applied to three low-dimensional data sets made of 712 sets of fMRI time series provided by the ABIDE consortium and two high-dimensional data sets obtained by processing 1527 rs-fMRI scans provided by the Human Connectome Project, the Framingham Heart Study and the Genetics of Brain Structure and Function study. These experiments established that our new framework could successfully harmonize low-dimensional connectomes and voxelwise functional time series and confirmed the need for preserving connectomes properties during their harmonization.

Neonatal White Matter Damage Analysis Using DTI Super-Resolution and Multi-Modality Image Registration

Punctate White Matter Damage (PWMD) is a common neonatal brain disease, which can easily cause neurological disorder and strongly affect life quality in terms of neuromotor and cognitive performance. Especially, at the neonatal stage, the best cure time can be easily missed because PWMD is not conducive to the diagnosis based on current existing methods. The lesion of PWMD is relatively straightforward on T1-weighted Magnetic Resonance Imaging (T1 MRI), showing semi-oval, cluster or linear high signals. Diffusion Tensor Magnetic Resonance Image (DT-MRI, referred to as DTI) is a noninvasive technique that can be used to study brain microstructures in vivo, and provide information on movement and cognition-related nerve fiber tracts. Therefore, a new method was proposed to use T1 MRI combined with DTI for better neonatal PWMD analysis based on DTI super-resolution and multi-modality image registration. First, after preprocessing, neonatal DTI super-resolution was performed with the three times B-spline interpolation algorithm based on the Log-Euclidean space to improve DTIs' resolution to fit the T1 MRIs and facilitate nerve fiber tractography. Second, the symmetric diffeomorphic registration algorithm and inverse b0 image were selected for multi-modality image registration of DTI and T1 MRI. Finally, the 3D lesion models were combined with fiber tractography results to analyze and predict the degree of PWMD lesions affecting fiber tracts. Extensive experiments demonstrated the effectiveness and super performance of our proposed method. This streamlined technique can play an essential auxiliary role in diagnosing and treating neonatal PWMD.

Signature methods for brain-computer interfaces

Brain-computer interfaces (BCIs) allow direct communication between one's central nervous system and a computer without any muscle movement hence by-passing the peripheral nervous system. They can restore disabled people's ability to interact with their environment, e.g. communication and wheelchair control. However, to this day their performance is still hindered by the non-stationarity of electroencephalography (EEG) signals, as well as their susceptibility to noise from the users' environment and from their own physiological activity. Moreover, a non-negligible amount of users struggle to use BCI systems based on motor imagery. In this paper, a new method based on the path signature is introduced to tackle this problem by using features which are different from the usual power-based ones. The path signature is a series of iterated integrals computed from a multidimensional path. It is invariant under translation and time reparametrization, which makes it a robust feature for multichannel EEG time series. The performance can be further boosted by combining the path signature with the gold standard Riemannian classifier in the BCI field exploiting the geometric structure of symmetric positive definite (SPD) matrices. The results obtained on publicly available datasets show that the signature method is more robust to inter-user variability than classical ones, especially on noisy and low-quality data. Hence, this study paves the way towards the use of mathematical tools that until now have been neglected, in order to tackle the EEG-based BCI variability issue. It also sheds light on the lead-lag relationship captured by path signature which seems relevant to assess the underlying neural mechanisms.

Tensor-CSPNet: A Novel Geometric Deep Learning Framework for Motor Imagery Classification

Deep learning (DL) has been widely investigated in a vast majority of applications in electroencephalography (EEG)-based brain-computer interfaces (BCIs), especially for motor imagery (MI) classification in the past five years. The mainstream DL methodology for the MI-EEG classification exploits the temporospatial patterns of EEG signals using convolutional neural networks (CNNs), which have been particularly successful in visual images. However, since the statistical characteristics of visual images depart radically from EEG signals, a natural question arises whether an alternative network architecture exists apart from CNNs. To address this question, we propose a novel geometric DL (GDL) framework called Tensor-CSPNet, which characterizes spatial covariance matrices derived from EEG signals on symmetric positive definite (SPD) manifolds and fully captures the temporospatiofrequency patterns using existing deep neural networks on SPD manifolds, integrating with experiences from many successful MI-EEG classifiers to optimize the framework. In the experiments, Tensor-CSPNet attains or slightly outperforms the current state-of-the-art performance on the cross-validation and holdout scenarios in two commonly used MI-EEG datasets. Moreover, the visualization and interpretability analyses also exhibit the validity of Tensor-CSPNet for the MI-EEG classification. To conclude, in this study, we provide a feasible answer to the question by generalizing the DL methodologies on SPD manifolds, which indicates the start of a specific GDL methodology for the MI-EEG classification.

Improving EEG-based decoding of the locus of auditory attention through domain adaptation. J Neural Eng 2023;20:066022. [PMID: 37988748 DOI: 10.1088/1741-2552/ad0e7b] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Objective.This paper presents a novel domain adaptation (DA) framework to enhance the accuracy of electroencephalography (EEG)-based auditory attention classification, specifically for classifying the direction (left or right) of attended speech. The framework aims to improve the performances for subjects with initially low classification accuracy, overcoming challenges posed by instrumental and human factors. Limited dataset size, variations in EEG data quality due to factors such as noise, electrode misplacement or subjects, and the need for generalization across different trials, conditions and subjects necessitate the use of DA methods. By leveraging DA methods, the framework can learn from one EEG dataset and adapt to another, potentially resulting in more reliable and robust classification models.Approach.This paper focuses on investigating a DA method, based on parallel transport, for addressing the auditory attention classification problem. The EEG data utilized in this study originates from an experiment where subjects were instructed to selectively attend to one of the two spatially separated voices presented simultaneously.Main results.Significant improvement in classification accuracy was observed when poor data from one subject was transported to the domain of good data from different subjects, as compared to the baseline. The mean classification accuracy for subjects with poor data increased from 45.84% to 67.92%. Specifically, the highest achieved classification accuracy from one subject reached 83.33%, a substantial increase from the baseline accuracy of 43.33%.Significance.The findings of our study demonstrate the improved classification performances achieved through the implementation of DA methods. This brings us a step closer to leveraging EEG in neuro-steered hearing devices.

Capturing functional connectomics using Riemannian partial least squares

For neurological disorders and diseases, functional and anatomical connectomes of the human brain can be used to better inform targeted interventions and treatment strategies. Functional magnetic resonance imaging (fMRI) is a non-invasive neuroimaging technique that captures spatio-temporal brain function through change in blood-oxygen-level-dependent (BOLD) signals over time. FMRI can be used to study the functional connectome through the functional connectivity matrix; that is, Pearson's correlation matrix between time series from the regions of interest of an fMRI image. One approach to analysing functional connectivity is using partial least squares (PLS), a multivariate regression technique designed for high-dimensional predictor data. However, analysing functional connectivity with PLS ignores a key property of the functional connectivity matrix; namely, these matrices are positive definite. To account for this, we introduce a generalisation of PLS to Riemannian manifolds, called R-PLS, and apply it to symmetric positive definite matrices with the affine invariant geometry. We apply R-PLS to two functional imaging datasets: COBRE, which investigates functional differences between schizophrenic patients and healthy controls, and; ABIDE, which compares people with autism spectrum disorder and neurotypical controls. Using the variable importance in the projection statistic on the results of R-PLS, we identify key functional connections in each dataset that are well represented in the literature. Given the generality of R-PLS, this method has the potential to investigate new functional connectomes in the brain, and with future application to structural data can open up further avenues of research in multi-modal imaging analysis.

Automatic tractography and segmentation using finsler geometry based on higher-order tensor fields

BACKGROUND AND OBJECTIVE

We focus on three-dimensional higher-order tensorial (HOT) images using Finsler geometry. In biomedical image analysis, these images are widely used, and they are based on the diffusion profiles inside the voxels. The diffusion information is stored in the so-called diffusion tensor D. Our objective is to present new methods revealing the architecture of neural fibers in presence of crossings and high curvatures. After tracking the fibers, we achieve direct 3D image segmentation to analyse the brain's white matter structures.

METHODS

To deal with the construction of the underlying fibers, the inverse of the second-order diffusion tensor D, understood as the metric tensor D-1, is commonly used in DTI modality. For crossing and highly curved fibers, higher order tensors are more relevant, but it is challenging to find an analogue of such an inverse in the HOT case. We employ an innovative approach to metrics based on higher order tensors to track the fibers properly. We propose to feed the tracked fibers as the internal initial contours in an efficient version of 3D segmentation.

RESULTS

We propose a brand-new approach to the inversion of a diffusion HOT, and an effective way of fiber tracking in the Finsler setting, based on innovative classification of the individual voxels. Thus, we can handle complex structures with high curvatures and crossings, even in the presence of noise. Based on our novel tractography approach, we also introduce a new segmentation method. We feed the detected fibers as the initial position of the contour surfaces to segment the image using a relevant active contour method (i.e., initiating the segmentation from inside the structures).

CONCLUSIONS

This is a pilot work, enhancing methods for fiber tracking and segmentation. The implemented algorithms were successfully tested on both synthetic and real data. The new features make our algorithms robust and fast, and they allow distinguishing individual objects in complex structures, even under noise.

Tangent functional connectomes uncover more unique phenotypic traits

Functional connectomes (FCs) containing pairwise estimations of functional couplings between pairs of brain regions are commonly represented by correlation matrices. As symmetric positive definite matrices, FCs can be transformed via tangent space projections, resulting into tangent-FCs. Tangent-FCs have led to more accurate models predicting brain conditions or aging. Motivated by the fact that tangent-FCs seem to be better biomarkers than FCs, we hypothesized that tangent-FCs have also a higher fingerprint. We explored the effects of six factors: fMRI condition, scan length, parcellation granularity, reference matrix, main-diagonal regularization, and distance metric. Our results showed that identification rates are systematically higher when using tangent-FCs across the "fingerprint gradient" (here including test-retest, monozygotic and dizygotic twins). Highest identification rates were achieved when minimally (0.01) regularizing FCs while performing tangent space projection using Riemann reference matrix and using correlation distance to compare the resulting tangent-FCs. Such configuration was validated in a second dataset (resting-state).

Score-Based Data Generation for EEG Spatial Covariance Matrices: Towards Boosting BCI Performance

The efficacy of Electroencephalogram (EEG) classifiers can be augmented by increasing the quantity of available data. In the case of geometric deep learning classifiers, the input consists of spatial covariance matrices derived from EEGs. In order to synthesize these spatial covariance matrices and facilitate future improvements of geometric deep learning classifiers, we propose a generative modeling technique based on state-of-the-art score-based models. The quality of generated samples is evaluated through visual and quantitative assessments using a left/right-hand-movement motor imagery dataset. The exceptional pixel-level resolution of these generative samples highlights the formidable capacity of score-based generative modeling. Additionally, the center (Fréchet mean) of the generated samples aligns with neurophysiological evidence that event-related desynchronization and synchronization occur on electrodes C3 and C4 within the Mu and Beta frequency bands during motor imagery processing. The quantitative evaluation revealed that 84.3% of the generated samples could be accurately predicted by a pre-trained classifier and an improvement of up to 8.7% in the average accuracy over ten runs for a specific test subject in a holdout experiment.

Learning to Optimize on Riemannian Manifolds

Many learning tasks are modeled as optimization problems with nonlinear constraints, such as principal component analysis and fitting a Gaussian mixture model. A popular way to solve such problems is resorting to Riemannian optimization algorithms, which yet heavily rely on both human involvement and expert knowledge about Riemannian manifolds. In this paper, we propose a Riemannian meta-optimization method to automatically learn a Riemannian optimizer. We parameterize the Riemannian optimizer by a novel recurrent network and utilize Riemannian operations to ensure that our method is faithful to the geometry of manifolds. The proposed method explores the distribution of the underlying data by minimizing the objective of updated parameters, and hence is capable of learning task-specific optimizations. We introduce a Riemannian implicit differentiation training scheme to achieve efficient training in terms of numerical stability and computational cost. Unlike conventional meta-optimization training schemes that need to differentiate through the whole optimization trajectory, our training scheme is only related to the final two optimization steps. In this way, our training scheme avoids the exploding gradient problem, and significantly reduces the computational load and memory footprint. We discuss experimental results across various constrained problems, including principal component analysis on Grassmann manifolds, face recognition, person re-identification, and texture image classification on Stiefel manifolds, clustering and similarity learning on symmetric positive definite manifolds, and few-shot learning on hyperbolic manifolds.

Discovering functional connectivity features characterizing multiple sclerosis phenotypes using explainable artificial intelligence

Multiple sclerosis (MS) is a neurological condition characterized by severe structural brain damage and by functional reorganization of the main brain networks that try to limit the clinical consequences of structural burden. Resting-state (RS) functional connectivity (FC) abnormalities found in this condition were shown to be variable across different MS phases, according to the severity of clinical manifestations. The article describes a system exploiting machine learning on RS FC matrices to discriminate different MS phenotypes and to identify relevant functional connections for MS stage characterization. To this end, the system exploits some mathematical properties of covariance-based RS FC representation, which can be described by a Riemannian manifold. The classification performance of the proposed framework was significantly above the chance level for all MS phenotypes. Moreover, the proposed system was successful in identifying relevant RS FC alterations contributing to an accurate phenotype classification.

Few-shot remote sensing image scene classification based on multiscale covariance metric network (MCMNet)

Few-shot learning (FSL) is a paradigm that simulates the fast learning ability of human beings, which can learn the feature differences between two groups of small-scale samples with common label space, and the label space of the training set and the test set is not repeated. By this way, it can quickly identify the categories of the unseen image in the test set. This method is widely used in image scene recognition, and it is expected to overcome difficulties of scarce annotated samples in remote sensing (RS). However, among most existing FSL methods, images were embed into Euclidean space, and the similarity between features at the last layer of deep network were measured by Euclidean distance. It is difficult to measure the inter-class similarity and intra-class difference of RS images. In this paper, we propose a multi-scale covariance network (MCMNet) for the application of remote sensing scene classification (RSSC). Taking Conv64F as the backbone, we mapped the features of the 1, 2, and 4 layers of the network to the manifold space by constructing a regional covariance matrix to form a covariance network with different scales. For each layer of features, we introduce the center in manifold space as a prototype for different categories of features. We simultaneously measure the similarity of three prototypes on the manifold space with different scales to form three loss functions and optimize the whole network by episodic training strategy. We conducted comparative experiments on three public datasets. The results show that the classification accuracy (CA) of our proposed method is from 1.35 % to 2.36% higher than that of the most excellent method, which demonstrates that the performance of MCMNet outperforms other methods.

U-SPDNet: An SPD manifold learning-based neural network for visual classification

With the development of neural networking techniques, several architectures for symmetric positive definite (SPD) matrix learning have recently been put forward in the computer vision and pattern recognition (CV&PR) community for mining fine-grained geometric features. However, the degradation of structural information during multi-stage feature transformation limits their capacity. To cope with this issue, this paper develops a U-shaped neural network on the SPD manifolds (U-SPDNet) for visual classification. The designed U-SPDNet contains two subsystems, one of which is a shrinking path (encoder) making up of a prevailing SPD manifold neural network (SPDNet (Huang and Van Gool, 2017)) for capturing compact representations from the input data. Another is a constructed symmetric expanding path (decoder) to upsample the encoded features, trained by a reconstruction error term. With this design, the degradation problem will be gradually alleviated during training. To enhance the representational capacity of U-SPDNet, we also append skip connections from encoder to decoder, realized by manifold-valued geometric operations, namely Riemannian barycenter and Riemannian optimization. On the MDSD, Virus, FPHA, and UAV-Human datasets, the accuracy achieved by our method is respectively 6.92%, 8.67%, 1.57%, and 1.08% higher than SPDNet, certifying its effectiveness.

Machine Learning Full NMR Chemical Shift Tensors of Silicon Oxides with Equivariant Graph Neural Networks

The nuclear magnetic resonance (NMR) chemical shift tensor is a highly sensitive probe of the electronic structure of an atom and furthermore its local structure. Recently, machine learning has been applied to NMR in the prediction of isotropic chemical shifts from a structure. Current machine learning models, however, often ignore the full chemical shift tensor for the easier-to-predict isotropic chemical shift, effectively ignoring a multitude of structural information available in the NMR chemical shift tensor. Here we use an equivariant graph neural network (GNN) to predict full ²⁹Si chemical shift tensors in silicate materials. The equivariant GNN model predicts full tensors to a mean absolute error of 1.05 ppm and is able to accurately determine the magnitude, anisotropy, and tensor orientation in a diverse set of silicon oxide local structures. When compared with other models, the equivariant GNN model outperforms the state-of-the-art machine learning models by 53%. The equivariant GNN model also outperforms historic analytical models by 57% for isotropic chemical shift and 91% for anisotropy. The software is available as a simple-to-use open-source repository, allowing similar models to be created and trained with ease.

On the Eigenvalues of Global Covariance Pooling for Fine-Grained Visual Recognition

The Fine-Grained Visual Categorization (FGVC) is challenging because the subtle inter-class variations are difficult to be captured. One notable research line uses the Global Covariance Pooling (GCP) layer to learn powerful representations with second-order statistics, which can effectively model inter-class differences. In our previous conference paper, we show that truncating small eigenvalues of the GCP covariance can attain smoother gradient and improve the performance on large-scale benchmarks. However, on fine-grained datasets, truncating the small eigenvalues would make the model fail to converge. This observation contradicts the common assumption that the small eigenvalues merely correspond to the noisy and unimportant information. Consequently, ignoring them should have little influence on the performance. To diagnose this peculiar behavior, we propose two attribution methods whose visualizations demonstrate that the seemingly unimportant small eigenvalues are crucial as they are in charge of extracting the discriminative class-specific features. Inspired by this observation, we propose a network branch dedicated to magnifying the importance of small eigenvalues. Without introducing any additional parameters, this branch simply amplifies the small eigenvalues and achieves state-of-the-art performances of GCP methods on three fine-grained benchmarks. Furthermore, the performance is also competitive against other FGVC approaches on larger datasets. Code is available at https://github.com/KingJamesSong/DifferentiableSVD.

Detection of autism spectrum disorder using graph representation learning algorithms and deep neural network, based on fMRI signals

Introduction

Can we apply graph representation learning algorithms to identify autism spectrum disorder (ASD) patients within a large brain imaging dataset? ASD is mainly identified by brain functional connectivity patterns. Attempts to unveil the common neural patterns emerged in ASD are the essence of ASD classification. We claim that graph representation learning methods can appropriately extract the connectivity patterns of the brain, in such a way that the method can be generalized to every recording condition, and phenotypical information of subjects. These methods can capture the whole structure of the brain, both local and global properties.

Methods

The investigation is done for the worldwide brain imaging multi-site database known as ABIDE I and II (Autism Brain Imaging Data Exchange). Among different graph representation techniques, we used AWE, Node2vec, Struct2vec, multi node2vec, and Graph2Img. The best approach was Graph2Img, in which after extracting the feature vectors representative of the brain nodes, the PCA algorithm is applied to the matrix of feature vectors. The classifier adapted to the features embedded in graphs is an LeNet deep neural network.

Results and discussion

Although we could not outperform the previous accuracy of 10-fold cross-validation in the identification of ASD versus control patients in this dataset, for leave-one-site-out cross-validation, we could obtain better results (our accuracy: 80%). The result is that graph embedding methods can prepare the connectivity matrix more suitable for applying to a deep network.

Revealing the spatiotemporal requirements for accurate subject identification with resting-state functional connectivity: a simultaneous fNIRS-fMRI study

SIGNIFICANCE

Brain fingerprinting refers to identifying participants based on their functional patterns. Despite its success with functional magnetic resonance imaging (fMRI), brain fingerprinting with functional near-infrared spectroscopy (fNIRS) still lacks adequate validation.

AIM

We investigated how fNIRS-specific acquisition features (limited spatial information and nonneural contributions) influence resting-state functional connectivity (rsFC) patterns at the intra-subject level and, therefore, brain fingerprinting.

APPROACH

We performed multiple simultaneous fNIRS and fMRI measurements in 29 healthy participants at rest. Data were preprocessed following the best practices, including the removal of motion artifacts and global physiology. The rsFC maps were extracted with the Pearson correlation coefficient. Brain fingerprinting was tested with pairwise metrics and a simple linear classifier.

RESULTS

Our results show that average classification accuracy with fNIRS ranges from 75% to 98%, depending on the number of runs and brain regions used for classification. Under the right conditions, brain fingerprinting with fNIRS is close to the 99.9% accuracy found with fMRI. Overall, the classification accuracy is more impacted by the number of runs and the spatial coverage than the choice of the classification algorithm.

CONCLUSIONS

This work provides evidence that brain fingerprinting with fNIRS is robust and reliable for extracting unique individual features at the intra-subject level once relevant spatiotemporal constraints are correctly employed.

Revisiting Riemannian geometry-based EEG decoding through approximate joint diagonalization

Objective.The wider adoption of Riemannian geometry in electroencephalography (EEG) processing is hindered by two factors: (a) it involves the manipulation of complex mathematical formulations and, (b) it leads to computationally demanding tasks. The main scope of this work is to simplify particular notions of Riemannian geometry and provide an efficient and comprehensible scheme for neuroscientific explorations.Approach.To overcome the aforementioned shortcomings, we exploit the concept of approximate joint diagonalization in order to reconstruct the spatial covariance matrices assuming the existence of (and identifying) a common eigenspace in which the application of Riemannian geometry is significantly simplified.Main results.The employed reconstruction process abides to physiologically plausible assumptions, reduces the computational complexity in Riemannian geometry schemes and bridges the gap between rigorous mathematical procedures and computational neuroscience. Our approach is both formally established and experimentally validated by employing real and synthetic EEG data.Significance.The implications of the introduced reconstruction process are highlighted by reformulating and re-introducing two signal processing methodologies, namely the 'Symmetric Positive Definite (SPD) Matrix Quantization' and the 'Coding over SPD Atoms'. The presented approach paves the way for robust and efficient neuroscientific explorations that exploit Riemannian geometry schemes.

Interpretation for Individual Brain Age Prediction Based on Gray Matter Volume

The relationship between age and the central nervous system (CNS) in humans has been a classical issue that has aroused extensive attention. Especially for individuals, it is of far greater importance to clarify the mechanisms between CNS and age. The primary goal of existing methods is to use MR images to derive high-accuracy predictions for age or degenerative diseases. However, the associated mechanisms between the images and the age have rarely been investigated. In this paper, we address the correlation between gray matter volume (GMV) and age, both in terms of gray matter themselves and their interaction network, using interpretable machine learning models for individuals. Our goal is not only to predict age accurately but more importantly, to explore the relationship between GMV and age. In addition to targeting each individual, we also investigate the dynamic properties of gray matter and their interaction network with individual age. The results show that the mean absolute error (MAE) of age prediction is 7.95 years. More notably, specific locations of gray matter and their interactions play different roles in age, and these roles change dynamically with age. The proposed method is a data-driven approach, which provides a new way to study aging mechanisms and even to diagnose degenerative brain diseases.

Improved Variance Reduction Methods for Riemannian Non-Convex Optimization

Variance reduction is popular in accelerating gradient descent and stochastic gradient descent for optimization problems defined on both euclidean space and Riemannian manifold. This paper further improves on existing variance reduction methods for non-convex Riemannian optimization, including R-SVRG and R-SRG/R-SPIDER by providing a unified framework for batch size adaptation. Such framework is more general than the existing works by considering retraction and vector transport and mini-batch stochastic gradients. We show that the adaptive-batch variance reduction methods require lower gradient complexities for both general non-convex and gradient dominated functions, under both finite-sum and online optimization settings. Moreover, under the new framework, we complete the analysis of R-SVRG and R-SRG, which is currently missing in the literature. We prove convergence of R-SVRG with much simpler analysis, which leads to curvature-free complexity bounds. We also show improved results for R-SRG under double-loop convergence, which match the optimal complexities as the R-SPIDER. In addition, we prove the first online complexity results for R-SVRG and R-SRG. Lastly, we discuss the potential of adapting batch size for non-smooth, constrained and second-order Riemannian optimizers. Extensive experiments on a variety of applications support the analysis and claims in the paper.

Supervised and Semisupervised Manifold Embedded Knowledge Transfer in Motor Imagery-Based BCI

A long calibration procedure limits the use in practice for a motor imagery (MI)-based brain-computer interface (BCI) system. To tackle this problem, we consider supervised and semisupervised transfer learning. However, it is a challenge for them to cope with high intersession/subject variability in the MI electroencephalographic (EEG) signals. Based on the framework of unsupervised manifold embedded knowledge transfer (MEKT), we propose a supervised MEKT algorithm (sMEKT) and a semisupervised MEKT algorithm (ssMEKT), respectively. sMEKT only has limited labelled samples from a target subject and abundant labelled samples from multiple source subjects. Compared to sMEKT, ssMEKT adds comparably more unlabelled samples from the target subject. After performing Riemannian alignment (RA) and tangent space mapping (TSM), both sMEKT and ssMEKT execute domain adaptation to shorten the differences among subjects. During domain adaptation, to make use of the available samples, two algorithms preserve the source domain discriminability, and ssMEKT preserves the geometric structure embedded in the labelled and unlabelled target domains. Moreover, to obtain a subject-specific classifier, sMEKT minimizes the joint probability distribution shift between the labelled target and source domains, whereas ssMEKT performs the joint probability distribution shift minimization between the unlabelled target domain and all labelled domains. Experimental results on two publicly available MI datasets demonstrate that our algorithms outperform the six competing algorithms, where the sizes of labelled and unlabelled target domains are variable. Especially for the target subjects with 10 labelled samples and 270/190 unlabelled samples, ssMEKT shows 5.27% and 2.69% increase in average accuracy on the two abovementioned datasets compared to the previous best semisupervised transfer learning algorithm (RA-regularized common spatial patterns-weighted adaptation regularization, RA-RCSP-wAR), respectively. Therefore, our algorithms can effectively reduce the need of labelled samples for the target subject, which is of importance for the MI-based BCI application.

RNeuMark: A Riemannian EEG Analysis Framework for Neuromarketing

Neuromarketing exploits neuroimaging techniques so as to reinforce the predictive power of conventional marketing tools, like questionnaires and focus groups. Electroencephalography (EEG) is the most commonly encountered neuroimaging technique due to its non-invasiveness, low-cost, and its very recent embedding in wearable devices. The transcription of brainwave patterns to consumer attitude is supported by various signal descriptors, while the quest for profitable novel ways is still an open research question. Here, we suggest the use of sample covariance matrices as alternative descriptors, that encapsulate the coordinated neural activity from distinct brain areas, and the adoption of Riemannian geometry for their handling. We first establish the suitability of Riemannian approach for neuromarketing-related problems and then suggest a relevant decoding scheme for predicting consumers' choices (e.g., willing to buy or not a specific product). Since the decision-making process involves the concurrent interaction of various cognitive processes and consequently of distinct brain rhythms, the proposed decoder takes the form of an ensemble classifier that builds upon a multi-view perspective, with each view dedicated to a specific frequency band. Adopting a standard machine learning procedure, and using a set of trials (training data) in conjunction with the associated behavior labels ("buy"/ "not buy"), we train a battery of classifiers accordingly. Each classifier is designed to operate in the space recovered from the inter-trial distances of SCMs and to cast a rhythm-depended decision that is eventually combined with the predictions of the rest ones. The demonstration and evaluation of the proposed approach are performed in 2 neuromarketing-related datasets of different nature. The first is employed to showcase the potential of the suggested descriptor, while the second to showcase the decoder's superiority against popular alternatives in the field.

Learning Log-Determinant Divergences for Positive Definite Matrices

Representations in the form of Symmetric Positive Definite (SPD) matrices have been popularized in a variety of visual learning applications due to their demonstrated ability to capture rich second-order statistics of visual data. There exist several similarity measures for comparing SPD matrices with documented benefits. However, selecting an appropriate measure for a given problem remains a challenge and in most cases, is the result of a trial-and-error process. In this paper, we propose to learn similarity measures in a data-driven manner. To this end, we capitalize on the αβ-log-det divergence, which is a meta-divergence parametrized by scalars α and β, subsuming a wide family of popular information divergences on SPD matrices for distinct and discrete values of these parameters. Our key idea is to cast these parameters in a continuum and learn them from data. We systematically extend this idea to learn vector-valued parameters, thereby increasing the expressiveness of the underlying non-linear measure. We conjoin the divergence learning problem with several standard tasks in machine learning, including supervised discriminative dictionary learning and unsupervised SPD matrix clustering. We present Riemannian gradient descent schemes for optimizing our formulations efficiently, and show the usefulness of our method on eight standard computer vision tasks.

Class-distinctiveness-based frequency band selection on the Riemannian manifold for oscillatory activity-based BCIs: preliminary results

Considering user-specific settings is known to enhance Brain-Computer Interface (BCI) performances. In particular, the optimal frequency band for oscillatory activity classification is highly user-dependent and many frequency band selection methods have been developed in the past two decades. However, it is not well studied whether those conventional methods can be efficiently applied to the Riemannian BCIs, a recent family of BCI systems that utilize the non-Euclidean nature of the data unlike conventional BCI pipelines. In this paper, we proposed a novel frequency band selection method working on the Riemannian manifold. The frequency band is selected considering the class distinctiveness as quantified based on the inter-class distance and the intra-class variance on the manifold. An advantage of this method is that the frequency bandwidth can be adjusted for each individual without intensive optimization steps. In a comparative experiment using a public dataset of motor imagery-based BCI, our method showed a substantial improvement in average accuracy over both a fixed broad frequency band and a popular conventional frequency band selection method. In particular, our method substantially improved performances for subjects with initially low accuracies. This preliminary result suggests the importance of developing new user-specific setting algorithms considering the manifold properties, rather than directly applying methods developed prior to the rise of the Riemannian BCIs.

A novel channel selection scheme for olfactory EEG signal classification on Riemannian manifolds

OBJECTIVE

The classification of olfactory-induced electroencephalogram (olfactory EEG) signals has potential applications in disease diagnosis, emotion regulation, multimedia, and so on. To achieve high-precision classification, numerous EEG channels are usually used, but this also brings problems such as information redundancy, overfitting and high computational load. Consequently, channel selection is necessary to find and use the most effective channels.

APPROACH

In this study, we proposed a multi-strategy fusion binary harmony search (MFBHS) algorithm and combined it with the Riemannian geometry (RG) classification framework to select the optimal channel sets for olfactory EEG signal classification. MFBHS was designed by simultaneously integrating three strategies into the binary harmony search (BHS) algorithm, including an opposition-based learning strategy (OBL) for generating high-quality initial population, an adaptive parameter strategy (APS) for improving search capability, and a bitwise operation strategy (BOS) for maintaining population diversity. It performed channel selection directly on the covariance matrix of EEG signals, and used the number of selected channels and the classification accuracy computed by a Riemannian classifier to evaluate the newly generated subset of channels.

MAIN RESULTS

With five different classification protocols designed based on two public olfactory EEG datasets, the performance of MFBHS was evaluated and compared with some state-of-the-art algorithms. Experimental results reveal that our method can minimize the number of channels while achieving high classification accuracy compatible with using all the channels. In addition, cross-subject generalization tests of MFBHS channel selection show that subject-independent channels obtained through training can be directly used on untrained subjects without greatly compromising classification accuracy.

SIGNIFICANCE

The proposed MFBHS algorithm is a practical technique for effective use of EEG channels in olfactory recognition.

Learning Non-Euclidean Representations With SPD Manifold for Myoelectric Pattern Recognition

How to learn informative representations from Electromyography (EMG) signals is of vital importance for myoelectric control systems. Traditionally, hand-crafted features are extracted from individual EMG channels and combined together for pattern recognition. The spatial topological information between different channels can also be informative, which is seldom considered. This paper presents a radically novel approach to extract spatial structural information within diverse EMG channels based on the symmetric positive definite (SPD) manifold. The object is to learn non-Euclidean representations inside EMG signals for myoelectric pattern recognition. The performance is compared with two classical feature sets using accuracy and F1-score. The algorithm is tested on eleven gestures collected from ten subjects, and the best accuracy reaches 84.85%±5.15% with an improvement of 4.04%~20.25%, which outperforms the contrast method, and reaches a significant improvement with the Wilcoxon signed-rank test. Eleven gestures from three public databases involving Ninapro DB2, DB4, and DB5 are also evaluated, and better performance is observed. Furthermore, the computational cost is less than the contrast method, making it more suitable for low-cost systems. It shows the effectiveness of the presented approach and contributes a new way for myoelectric pattern recognition.

Transfer learning for motor imagery based brain-computer interfaces: A tutorial

A brain-computer interface (BCI) enables a user to communicate directly with an external device, e.g., a computer, using brain signals. It can be used to research, map, assist, augment, or repair human cognitive or sensory-motor functions. A closed-loop BCI system performs signal acquisition, temporal filtering, spatial filtering, feature engineering and classification, before sending out the control signal to an external device. Transfer learning (TL) has been widely used in motor imagery (MI) based BCIs to reduce the calibration effort for a new subject, greatly increasing their utility. This tutorial describes how TL can be considered in as many components of a BCI system as possible, and introduces a complete TL pipeline for MI-based BCIs. Examples on two MI datasets demonstrated the advantages of considering TL in multiple components of MI-based BCIs. Especially, integrating data alignment and sophisticated TL approaches can significantly improve the classification performance, and hence greatly reduces the calibration effort.