1
|
Ivanov N, Chau T. Corrigendum: Riemannian geometry-based metrics to measure and reinforce user performance changes during brain-computer interface user training. Front Comput Neurosci 2023; 17:1286681. [PMID: 38045092 PMCID: PMC10691671 DOI: 10.3389/fncom.2023.1286681] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Accepted: 11/06/2023] [Indexed: 12/05/2023] Open
Abstract
[This corrects the article DOI: 10.3389/fncom.2023.1108889.].
Collapse
Affiliation(s)
- Nicolas Ivanov
- PRISM Lab, Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada
| | - Tom Chau
- PRISM Lab, Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
2
|
Zhang M, Huang J, Ni S. Recognition of motor intentions from EEGs of the same upper limb by signal traceability and Riemannian geometry features. Front Neurosci 2023; 17:1270785. [PMID: 38027473 PMCID: PMC10643198 DOI: 10.3389/fnins.2023.1270785] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Accepted: 10/03/2023] [Indexed: 12/01/2023] Open
Abstract
Introduction The electroencephalographic (EEG) based on the motor imagery task is derived from the physiological electrical signal caused by the autonomous activity of the brain. Its weak potential difference changes make it easy to be overwhelmed by noise, and the EEG acquisition method has a natural limitation of low spatial resolution. These have brought significant obstacles to high-precision recognition, especially the recognition of the motion intention of the same upper limb. Methods This research proposes a method that combines signal traceability and Riemannian geometric features to identify six motor intentions of the same upper limb, including grasping/holding of the palm, flexion/extension of the elbow, and abduction/adduction of the shoulder. First, the EEG data of electrodes irrelevant to the task were screened out by low-resolution brain electromagnetic tomography. Subsequently, tangential spatial features are extracted by the Riemannian geometry framework in the covariance matrix estimated from the reconstructed EEG signals. The learned Riemannian geometric features are used for pattern recognition by a support vector machine with a linear kernel function. Results The average accuracy of the six classifications on the data set of 15 participants is 22.47%, the accuracy is 19.34% without signal traceability, the accuracy is 18.07% when the features are the filter bank common spatial pattern (FBCSP), and the accuracy is 16.7% without signal traceability and characterized by FBCSP. Discussion The results show that the proposed method can significantly improve the accuracy of intent recognition. In addressing the issue of temporal variability in EEG data for active Brain-Machine Interfaces, our method achieved an average standard deviation of 2.98 through model transfer on different days' data.
Collapse
Affiliation(s)
- Meng Zhang
- School of Mechanical and Power Engineering, Nanjing Tech University Nanjing, Nanjing, China
- Research Institute, NeuralEcho Technology Co., Ltd., Beijing, China
| | - Jinfeng Huang
- Faculty of Human Sciences, University of Tsukuba, Ibaraki, Japan
- Research Institute, NeuralEcho Technology Co., Ltd., Beijing, China
| | - Shoudong Ni
- School of Mechanical and Power Engineering, Nanjing Tech University Nanjing, Nanjing, China
| |
Collapse
|
3
|
Tang S, Liang Y, Li Z. Mind wandering state detection during video-based learning via EEG. Front Hum Neurosci 2023; 17:1182319. [PMID: 37323927 PMCID: PMC10267732 DOI: 10.3389/fnhum.2023.1182319] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Accepted: 05/16/2023] [Indexed: 06/17/2023] Open
Abstract
The aim of this study is to explore the potential of technology for detecting mind wandering, particularly during video-based distance learning, with the ultimate benefit of improving learning outcomes. To overcome the challenges of previous mind wandering research in ecological validity, sample balance, and dataset size, this study utilized practical electroencephalography (EEG) recording hardware and designed a paradigm consisting of viewing short-duration video lectures under a focused learning condition and a future planning condition. Participants estimated statistics of their attentional state at the end of each video, and we combined this rating scale feedback with self-caught key press responses during video watching to obtain binary labels for classifier training. EEG was recorded using an 8-channel system, and spatial covariance features processed by Riemannian geometry were employed. The results demonstrate that a radial basis function kernel support vector machine classifier, using Riemannian-processed covariance features from delta, theta, alpha, and beta bands, can detect mind wandering with a mean area under the receiver operating characteristic curve (AUC) of 0.876 for within-participant classification and AUC of 0.703 for cross-lecture classification. Furthermore, our results suggest that a short duration of training data is sufficient to train a classifier for online decoding, as cross-lecture classification remained at an average AUC of 0.689 when using 70% of the training set (about 9 min). The findings highlight the potential for practical EEG hardware in detecting mind wandering with high accuracy, which has potential application to improving learning outcomes during video-based distance learning.
Collapse
Affiliation(s)
- Shaohua Tang
- School of Systems Science, Beijing Normal University, Beijing, China
- International Academic Center of Complex Systems, Beijing Normal University, Zhuhai, China
- Center for Cognition and Neuroergonomics, State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Zhuhai, China
| | - Yutong Liang
- Center for Cognition and Neuroergonomics, State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Zhuhai, China
| | - Zheng Li
- Center for Cognition and Neuroergonomics, State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Zhuhai, China
| |
Collapse
|
4
|
Parker MC, Jeynes C. Relating a System's Hamiltonian to Its Entropy Production Using a Complex Time Approach. Entropy (Basel) 2023; 25:e25040629. [PMID: 37190417 PMCID: PMC10137557 DOI: 10.3390/e25040629] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/21/2023] [Revised: 03/30/2023] [Accepted: 04/03/2023] [Indexed: 05/17/2023]
Abstract
We exploit the properties of complex time to obtain an analytical relationship based on considerations of causality between the two Noether-conserved quantities of a system: its Hamiltonian and its entropy production. In natural units, when complexified, the one is simply the Wick-rotated complex conjugate of the other. A Hilbert transform relation is constructed in the formalism of quantitative geometrical thermodynamics, which enables system irreversibility to be handled analytically within a framework that unifies both the microscopic and macroscopic scales, and which also unifies the treatment of both reversibility and irreversibility as complementary parts of a single physical description. In particular, the thermodynamics of two unitary entities are considered: the alpha particle, which is absolutely stable (that is, trivially reversible with zero entropy production), and a black hole whose unconditional irreversibility is characterized by a non-zero entropy production, for which we show an alternate derivation, confirming our previous one. The thermodynamics of a canonical decaying harmonic oscillator are also considered. In this treatment, the complexification of time also enables a meaningful physical interpretation of both "imaginary time" and "imaginary energy".
Collapse
Affiliation(s)
- Michael C Parker
- School of Computer Sciences & Electronic Engineering, University of Essex, Colchester CO4 3SQ, UK
| | - Chris Jeynes
- Ion Beam Centre, University of Surrey, Guildford GU2 7XH, UK
| |
Collapse
|
5
|
Delisle-Rodriguez D, Silva L, Bastos-Filho T. EEG changes during passive movements improve the motor imagery feature extraction in BCIs-based sensory feedback calibration. J Neural Eng 2023; 20. [PMID: 36716494 DOI: 10.1088/1741-2552/acb73b] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2022] [Accepted: 01/30/2023] [Indexed: 01/31/2023]
Abstract
Objective.This work proposes a method for two calibration schemes based on sensory feedback to extract reliable motor imagery (MI) features, and provide classification outputs more correlated to the user's intention.Method.After filtering the raw electroencephalogram (EEG), a two-step method for spatial feature extraction by using the Riemannian covariance matrices (RCM) method and common spatial patterns is proposed here. It uses EEG data from trials providing feedback, in an intermediate step composed of bothkth nearest neighbors and probability analyses, to find periods of time in which the user probably performed well the MI task without feedback. These periods are then used to extract features with better separability, and train a classifier for MI recognition. For evaluation, an in-house dataset with eight healthy volunteers and two post-stroke patients that performed lower-limb MI, and consequently received passive movements as feedback was used. Other popular public EEG datasets (such as BCI Competition IV dataset IIb, among others) from healthy subjects that executed upper-and lower-limbs MI tasks under continuous visual sensory feedback were further used.Results.The proposed system based on the Riemannian geometry method in two-steps (RCM-RCM) outperformed significantly baseline methods, reaching average accuracy up to 82.29%. These findings show that EEG data on periods providing passive movement can be used to contribute greatly during MI feature extraction.Significance.Unconscious brain responses elicited over the sensorimotor areas may be avoided or greatly reduced by applying our approach in MI-based brain-computer interfaces (BCIs). Therefore, BCI's outputs more correlated to the user's intention can be obtained.
Collapse
Affiliation(s)
- Denis Delisle-Rodriguez
- Edmond and Lily Safra International Institute of Neurosciences, Santos Dumont Institute, 59288-899 Macaiba, Brazil
| | - Leticia Silva
- Postgraduate Program in Electrical Engineering, Federal University of Espirito Santo, 29075-910 Vitoria, Brazil
| | - Teodiano Bastos-Filho
- Postgraduate Program in Electrical Engineering, Federal University of Espirito Santo, 29075-910 Vitoria, Brazil
| |
Collapse
|
6
|
Ivanov N, Chau T. Riemannian geometry-based metrics to measure and reinforce user performance changes during brain-computer interface user training. Front Comput Neurosci 2023; 17:1108889. [PMID: 36860616 PMCID: PMC9968793 DOI: 10.3389/fncom.2023.1108889] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2022] [Accepted: 01/25/2023] [Indexed: 02/15/2023] Open
Abstract
Despite growing interest and research into brain-computer interfaces (BCI), their usage remains limited outside of research laboratories. One reason for this is BCI inefficiency, the phenomenon where a significant number of potential users are unable to produce machine-discernible brain signal patterns to control the devices. To reduce the prevalence of BCI inefficiency, some have advocated for novel user-training protocols that enable users to more effectively modulate their neural activity. Important considerations for the design of these protocols are the assessment measures that are used for evaluating user performance and for providing feedback that guides skill acquisition. Herein, we present three trial-wise adaptations (running, sliding window and weighted average) of Riemannian geometry-based user-performance metrics (classDistinct reflecting the degree of class separability and classStability reflecting the level of within-class consistency) to enable feedback to the user following each individual trial. We evaluated these metrics, along with conventional classifier feedback, using simulated and previously recorded sensorimotor rhythm-BCI data to assess their correlation with and discrimination of broader trends in user performance. Analysis revealed that the sliding window and weighted average variants of our proposed trial-wise Riemannian geometry-based metrics more accurately reflected performance changes during BCI sessions compared to conventional classifier output. The results indicate the metrics are a viable method for evaluating and tracking user performance changes during BCI-user training and, therefore, further investigation into how these metrics may be presented to users during training is warranted.
Collapse
Affiliation(s)
- Nicolas Ivanov
- PRISM Lab, Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada
| | - Tom Chau
- PRISM Lab, Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
7
|
Duan X, Xie S, Lv Y, Xie X, Obermayer K, Yan H. A transfer learning-based feedback training motivates the performance of SMR-BCI. J Neural Eng 2023; 20. [PMID: 36577144 DOI: 10.1088/1741-2552/acaee7] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2022] [Accepted: 12/28/2022] [Indexed: 12/29/2022]
Abstract
Objective. Feedback training is a practical approach to brain-computer interface (BCI) end-users learning to modulate their sensorimotor rhythms (SMRs). BCI self-regulation learning has been shown to be influenced by subjective psychological factors, such as motivation. However, few studies have taken into account the users' self-motivation as additional guidance for the cognitive process involved in BCI learning. In this study we tested a transfer learning (TL) feedback method designed to increase self-motivation by providing information about past performance.Approach. Electroencephalography (EEG) signals from the previous runs were affine transformed and displayed as points on the screen, along with the newly recorded EEG signals in the current run, giving the subjects a context for self-motivation. Subjects were asked to separate the feedback points for the current run under the display of the separability of prior training. We conducted a between-subject feedback training experiment, in which 24 healthy SMR-BCI naive subjects were trained to imagine left- and right-hand movements. The participants were provided with either TL feedback or typical cursor-bar (CB) feedback (control condition), for three sessions on separate days.Main results. The behavioral results showed an increased challenge and stable mastery confidence, suggesting that subjects' motivation grew as the feedback training went on. The EEG results showed favorable overall training effects with TL feedback in terms of the class distinctiveness and EEG discriminancy. Performance was 28.5% higher in the third session than in the first. About 41.7% of the subjects were 'learners' including not only low-performance subjects, but also good-performance subjects who might be affected by the ceiling effect. Subjects were able to control BCI with TL feedback with a higher performance of 60.5% during the last session compared to CB feedback.Significance. The present study demonstrated that the proposed TL feedback method boosted psychological engagement through the self-motivated context, and further allowed subjects to modulate SMR effectively. The proposed TL feedback method also provided an alternative to typical CB feedback.
Collapse
Affiliation(s)
- Xu Duan
- School of Electronics and Information, Northwestern Polytechnical University, Dongxiang Road 1, Xi'an 710129, People's Republic of China.,Key Laboratory for Artificial Intelligence and Cognitive Neuroscience of Language, Xi'an International Studies University, Wenyuan South Road, Xi'an 710128, People's Republic of China
| | - Songyun Xie
- School of Electronics and Information, Northwestern Polytechnical University, Dongxiang Road 1, Xi'an 710129, People's Republic of China
| | - Yanxia Lv
- School of Electronics and Information, Northwestern Polytechnical University, Dongxiang Road 1, Xi'an 710129, People's Republic of China
| | - Xinzhou Xie
- School of Electronics and Information, Northwestern Polytechnical University, Dongxiang Road 1, Xi'an 710129, People's Republic of China
| | - Klaus Obermayer
- Faculty of Electrical Engineering and Computer Science, Technical University Berlin, Marchstraße 23, Berlin 10587, Germany
| | - Hao Yan
- Key Laboratory for Artificial Intelligence and Cognitive Neuroscience of Language, Xi'an International Studies University, Wenyuan South Road, Xi'an 710128, People's Republic of China
| |
Collapse
|
8
|
Benfenati A, Marta A. A singular Riemannian geometry approach to deep neural networks II. Reconstruction of 1-D equivalence classes. Neural Netw 2023; 158:344-358. [PMID: 36512986 DOI: 10.1016/j.neunet.2022.11.026] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2021] [Revised: 09/22/2022] [Accepted: 11/17/2022] [Indexed: 11/24/2022]
Abstract
We proposed in a previous work a geometric framework to study a deep neural network, seen as sequence of maps between manifolds, employing singular Riemannian geometry. In this paper, we present an application of this framework, proposing a way to build the class of equivalence of an input point: such class is defined as the set of the points on the input manifold mapped to the same output by the neural network. In other words, we build the preimage of a point in the output manifold in the input space. In particular. We focus for simplicity on the case of neural networks maps from n-dimensional real spaces to (n-1)-dimensional real spaces, we propose an algorithm allowing to build the set of points lying on the same class of equivalence. This approach leads to two main applications: the generation of new synthetic data and it may provides some insights on how a classifier can be confused by small perturbation on the input data (e.g. a penguin image classified as an image containing a chihuahua). In addition, for neural networks from 2D to 1D real spaces, we also discuss how to find the preimages of closed intervals of the real line. We also present some numerical experiments with several neural networks trained to perform non-linear regression tasks, including the case of a binary classifier.
Collapse
Affiliation(s)
- Alessandro Benfenati
- Environmental Science and Policy Department, Università di Milano, Via Celoria 2, Milano, 20133, Italy; Gruppo Nazionale Calcolo Scientifico, INDAM, Italy.
| | - Alessio Marta
- Dipartimento di Scienze del Sistema Nervoso e del Comportamento, Università di Pavia, Via Agostino Bassi, 21, Pavia, 27100, Italy; Gruppo Nazionale per la Fisica Matematica, INDAM, Italy; Istituto Nazionale di Fisica Nucleare, sezione di Milano, INFN, Italy.
| |
Collapse
|
9
|
Kalaganis FP, Laskaris NA, Oikonomou VP, Nikopolopoulos S, Kompatsiaris I. Revisiting Riemannian geometry-based EEG decoding through approximate joint diagonalization. J Neural Eng 2022; 19. [PMID: 36541502 DOI: 10.1088/1741-2552/aca4fc] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2022] [Accepted: 11/22/2022] [Indexed: 11/23/2022]
Abstract
Objective.The wider adoption of Riemannian geometry in electroencephalography (EEG) processing is hindered by two factors: (a) it involves the manipulation of complex mathematical formulations and, (b) it leads to computationally demanding tasks. The main scope of this work is to simplify particular notions of Riemannian geometry and provide an efficient and comprehensible scheme for neuroscientific explorations.Approach.To overcome the aforementioned shortcomings, we exploit the concept of approximate joint diagonalization in order to reconstruct the spatial covariance matrices assuming the existence of (and identifying) a common eigenspace in which the application of Riemannian geometry is significantly simplified.Main results.The employed reconstruction process abides to physiologically plausible assumptions, reduces the computational complexity in Riemannian geometry schemes and bridges the gap between rigorous mathematical procedures and computational neuroscience. Our approach is both formally established and experimentally validated by employing real and synthetic EEG data.Significance.The implications of the introduced reconstruction process are highlighted by reformulating and re-introducing two signal processing methodologies, namely the 'Symmetric Positive Definite (SPD) Matrix Quantization' and the 'Coding over SPD Atoms'. The presented approach paves the way for robust and efficient neuroscientific explorations that exploit Riemannian geometry schemes.
Collapse
Affiliation(s)
- Fotis P Kalaganis
- Centre for Research and Technology Hellas, Information Technologies Institute, Multimedia Knowledge and Social Media Analytics Laboratory, Thermi-Thessaloniki 57001, Greece
| | - Nikos A Laskaris
- Aristotle University of Thessaloniki, Department of Informatics, AIIA lab, Thessaloniki 54124, Greece
| | - Vangelis P Oikonomou
- Centre for Research and Technology Hellas, Information Technologies Institute, Multimedia Knowledge and Social Media Analytics Laboratory, Thermi-Thessaloniki 57001, Greece
| | - Spiros Nikopolopoulos
- Centre for Research and Technology Hellas, Information Technologies Institute, Multimedia Knowledge and Social Media Analytics Laboratory, Thermi-Thessaloniki 57001, Greece
| | - Ioannis Kompatsiaris
- Centre for Research and Technology Hellas, Information Technologies Institute, Multimedia Knowledge and Social Media Analytics Laboratory, Thermi-Thessaloniki 57001, Greece
| |
Collapse
|
10
|
Tong J, Wei X, Dong E, Sun Z, Du S, Duan F. Hybrid mental tasks based human computer interface via integration of pronunciation and motor imagery. J Neural Eng 2022; 19. [PMID: 36228578 DOI: 10.1088/1741-2552/ac9a01] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2022] [Accepted: 10/13/2022] [Indexed: 12/24/2022]
Abstract
Objective.Among the existing active brain-computer interfaces (BCI), the motor imagination (MI) is widely used. To operate the MI BCI effectively, subjects need to carry out trainings on corresponding imagining tasks. Here, we studied how to reduce the discomfort and fatigue of active BCI imaginary tasks and the inability to concentrate on them while improving the accuracy.Approach.This paper proposes a hybrid BCI composed of MI and pronunciation imagination (PI). The electroencephalogram signals of ten subjects are recognized by the adaptive Riemannian distance classification and the improved frequency selective filter-bank Common Spatial Pattern recognition.Main results.The results show that under the new paradigm with the combination of MI and PI, the recognition accuracy is higher than the MI alone. The highest recognition rate of the proposed hybrid system can reach more than 90%. Furthermore, through the subjects' scoring results of the operation difficulty, it is concluded that the designed hybrid paradigm is more operable than the traditional BCI paradigm.Significance.The separable tasks in the active BCI are limited and the accuracy needs to be improved. The new hybrid paradigm proposed by us improves the accuracy and operability of the active BCI system, providing a new possibility for the research direction of the active BCI.
Collapse
Affiliation(s)
- Jigang Tong
- Tianjin Key Laboratory of Control Theory and Applications in Complicated Systems, TianjinUniversity of Technology, Tianjin 300384, People's Republic of China
| | - Xiaoying Wei
- Tianjin Key Laboratory of Control Theory and Applications in Complicated Systems, TianjinUniversity of Technology, Tianjin 300384, People's Republic of China
| | - Enzeng Dong
- Tianjin Key Laboratory of Control Theory and Applications in Complicated Systems, TianjinUniversity of Technology, Tianjin 300384, People's Republic of China
| | - Zhe Sun
- Computational Engineering Applications Unit, Head Office for Information Systems and Cybersecurity, RIKEN, Saitama, Japan
| | - Shengzhi Du
- Department of Electrical Engineering, Tshwane University of Technology, Pretoria 0001, South Africa
| | - Feng Duan
- College of Artificial Intelligence, Nankai University, Tianjin, People's Republic of China
| |
Collapse
|
11
|
Anctil-Robitaille B, Théberge A, Jodoin PM, Descoteaux M, Desrosiers C, Lombaert H. Manifold-aware synthesis of high-resolution diffusion from structural imaging. Front Neuroimaging 2022; 1:930496. [PMID: 37555146 PMCID: PMC10406190 DOI: 10.3389/fnimg.2022.930496] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/28/2022] [Accepted: 08/16/2022] [Indexed: 08/10/2023]
Abstract
The physical and clinical constraints surrounding diffusion-weighted imaging (DWI) often limit the spatial resolution of the produced images to voxels up to eight times larger than those of T1w images. The detailed information contained in accessible high-resolution T1w images could help in the synthesis of diffusion images with a greater level of detail. However, the non-Euclidean nature of diffusion imaging hinders current deep generative models from synthesizing physically plausible images. In this work, we propose the first Riemannian network architecture for the direct generation of diffusion tensors (DT) and diffusion orientation distribution functions (dODFs) from high-resolution T1w images. Our integration of the log-Euclidean Metric into a learning objective guarantees, unlike standard Euclidean networks, the mathematically-valid synthesis of diffusion. Furthermore, our approach improves the fractional anisotropy mean squared error (FA MSE) between the synthesized diffusion and the ground-truth by more than 23% and the cosine similarity between principal directions by almost 5% when compared to our baselines. We validate our generated diffusion by comparing the resulting tractograms to our expected real data. We observe similar fiber bundles with streamlines having <3% difference in length, <1% difference in volume, and a visually close shape. While our method is able to generate diffusion images from structural inputs in a high-resolution space within 15 s, we acknowledge and discuss the limits of diffusion inference solely relying on T1w images. Our results nonetheless suggest a relationship between the high-level geometry of the brain and its overall white matter architecture that remains to be explored.
Collapse
Affiliation(s)
- Benoit Anctil-Robitaille
- The Shape Lab, Department of Computer and Software Engineering, ETS Montreal, Montreal, QC, Canada
| | - Antoine Théberge
- Sherbrooke Connectivity Imaging Laboratory (SCIL), Department of Computer Science, Sherbrooke University, Sherbrooke, QC, Canada
| | - Pierre-Marc Jodoin
- Sherbrooke Connectivity Imaging Laboratory (SCIL), Department of Computer Science, Sherbrooke University, Sherbrooke, QC, Canada
| | - Maxime Descoteaux
- Sherbrooke Connectivity Imaging Laboratory (SCIL), Department of Computer Science, Sherbrooke University, Sherbrooke, QC, Canada
| | - Christian Desrosiers
- The Shape Lab, Department of Computer and Software Engineering, ETS Montreal, Montreal, QC, Canada
| | - Hervé Lombaert
- The Shape Lab, Department of Computer and Software Engineering, ETS Montreal, Montreal, QC, Canada
| |
Collapse
|
12
|
Engemann DA, Mellot A, Höchenberger R, Banville H, Sabbagh D, Gemein L, Ball T, Gramfort A. A reusable benchmark of brain-age prediction from M/EEG resting-state signals. Neuroimage 2022; 262:119521. [PMID: 35905809 DOI: 10.1016/j.neuroimage.2022.119521] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2021] [Revised: 07/04/2022] [Accepted: 07/25/2022] [Indexed: 01/02/2023] Open
Abstract
Population-level modeling can define quantitative measures of individual aging by applying machine learning to large volumes of brain images. These measures of brain age, obtained from the general population, helped characterize disease severity in neurological populations, improving estimates of diagnosis or prognosis. Magnetoencephalography (MEG) and Electroencephalography (EEG) have the potential to further generalize this approach towards prevention and public health by enabling assessments of brain health at large scales in socioeconomically diverse environments. However, more research is needed to define methods that can handle the complexity and diversity of M/EEG signals across diverse real-world contexts. To catalyse this effort, here we propose reusable benchmarks of competing machine learning approaches for brain age modeling. We benchmarked popular classical machine learning pipelines and deep learning architectures previously used for pathology decoding or brain age estimation in 4 international M/EEG cohorts from diverse countries and cultural contexts, including recordings from more than 2500 participants. Our benchmarks were built on top of the M/EEG adaptations of the BIDS standard, providing tools that can be applied with minimal modification on any M/EEG dataset provided in the BIDS format. Our results suggest that, regardless of whether classical machine learning or deep learning was used, the highest performance was reached by pipelines and architectures involving spatially aware representations of the M/EEG signals, leading to R^2 scores between 0.60-0.71. Hand-crafted features paired with random forest regression provided robust benchmarks even in situations in which other approaches failed. Taken together, this set of benchmarks, accompanied by open-source software and high-level Python scripts, can serve as a starting point and quantitative reference for future efforts at developing M/EEG-based measures of brain aging. The generality of the approach renders this benchmark reusable for other related objectives such as modeling specific cognitive variables or clinical endpoints.
Collapse
Affiliation(s)
- Denis A Engemann
- Roche Pharma Research and Early Development, Neuroscience and Rare Diseases, Roche Innovation Center Basel, F. Hoffmann-La Roche Ltd., Basel, Switzerland; Université Paris-Saclay, Inria, CEA, Palaiseau, France; Max Planck Institute for Human Cognitive and Brain Sciences, Department of Neurology, D-04103, Leipzig, Germany.
| | | | | | - Hubert Banville
- Université Paris-Saclay, Inria, CEA, Palaiseau, France; Inserm, UMRS-942, Paris Diderot University, Paris, France
| | - David Sabbagh
- Université Paris-Saclay, Inria, CEA, Palaiseau, France; Neuromedical AI Lab, Department of Neurosurgery, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Engelbergerstr. 21, 79106, Freiburg, Germany
| | - Lukas Gemein
- Neurorobotics Lab, Computer Science Department - University of Freiburg, Faculty of Engineering, University of Freiburg, Georges-Köhler-Allee 80, 79110, Freiburg, Germany; BrainLinks-BrainTools Cluster of Excellence, University of Freiburg, Freiburg, Germany
| | - Tonio Ball
- Neurorobotics Lab, Computer Science Department - University of Freiburg, Faculty of Engineering, University of Freiburg, Georges-Köhler-Allee 80, 79110, Freiburg, Germany; InteraXon Inc., Toronto, Canada
| | | |
Collapse
|
13
|
Zhang X, Meng QH, Zeng M. A novel channel selection scheme for olfactory EEG signal classification on Riemannian manifolds. J Neural Eng 2022; 19. [PMID: 35732136 DOI: 10.1088/1741-2552/ac7b4a] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2022] [Accepted: 06/22/2022] [Indexed: 11/12/2022]
Abstract
OBJECTIVE The classification of olfactory-induced electroencephalogram (olfactory EEG) signals has potential applications in disease diagnosis, emotion regulation, multimedia, and so on. To achieve high-precision classification, numerous EEG channels are usually used, but this also brings problems such as information redundancy, overfitting and high computational load. Consequently, channel selection is necessary to find and use the most effective channels. APPROACH In this study, we proposed a multi-strategy fusion binary harmony search (MFBHS) algorithm and combined it with the Riemannian geometry (RG) classification framework to select the optimal channel sets for olfactory EEG signal classification. MFBHS was designed by simultaneously integrating three strategies into the binary harmony search (BHS) algorithm, including an opposition-based learning strategy (OBL) for generating high-quality initial population, an adaptive parameter strategy (APS) for improving search capability, and a bitwise operation strategy (BOS) for maintaining population diversity. It performed channel selection directly on the covariance matrix of EEG signals, and used the number of selected channels and the classification accuracy computed by a Riemannian classifier to evaluate the newly generated subset of channels. MAIN RESULTS With five different classification protocols designed based on two public olfactory EEG datasets, the performance of MFBHS was evaluated and compared with some state-of-the-art algorithms. Experimental results reveal that our method can minimize the number of channels while achieving high classification accuracy compatible with using all the channels. In addition, cross-subject generalization tests of MFBHS channel selection show that subject-independent channels obtained through training can be directly used on untrained subjects without greatly compromising classification accuracy. SIGNIFICANCE The proposed MFBHS algorithm is a practical technique for effective use of EEG channels in olfactory recognition.
Collapse
Affiliation(s)
- Xiaonei Zhang
- Tianjin University, School of Electrical and Information Engineering, Tianjin, 300072, CHINA
| | - Qing-Hao Meng
- Tianjin University, School of Electrical and Information Engineering, Tianjin, 300072, CHINA
| | - Ming Zeng
- Tianjin University, School of Electrical and Information Engineering, Tianjin, 300072, CHINA
| |
Collapse
|
14
|
Dan T, Huang Z, Cai H, Lyday RG, Laurienti PJ, Wu G. Uncovering shape signatures of resting-state functional connectivity by geometric deep learning on Riemannian manifold. Hum Brain Mapp 2022; 43:3970-3986. [PMID: 35538672 PMCID: PMC9374896 DOI: 10.1002/hbm.25897] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2021] [Revised: 04/13/2022] [Accepted: 04/20/2022] [Indexed: 11/21/2022] Open
Abstract
Functional neural activities manifest geometric patterns, as evidenced by the evolving network topology of functional connectivities (FC) even in the resting state. In this work, we propose a novel manifold‐based geometric neural network for functional brain networks (called “Geo‐Net4Net” for short) to learn the intrinsic low‐dimensional feature representations of resting‐state brain networks on the Riemannian manifold. This tool allows us to answer the scientific question of how the spontaneous fluctuation of FC supports behavior and cognition. We deploy a set of positive maps and rectified linear unit (ReLU) layers to uncover the intrinsic low‐dimensional feature representations of functional brain networks on the Riemannian manifold taking advantage of the symmetric positive‐definite (SPD) form of the correlation matrices. Due to the lack of well‐defined ground truth in the resting state, existing learning‐based methods are limited to unsupervised methodologies. To go beyond this boundary, we propose to self‐supervise the feature representation learning of resting‐state functional networks by leveraging the task‐based counterparts occurring before and after the underlying resting state. With this extra heuristic, our Geo‐Net4Net allows us to establish a more reasonable understanding of resting‐state FCs by capturing the geometric patterns (aka. spectral/shape signature) associated with resting states on the Riemannian manifold. We have conducted extensive experiments on both simulated data and task‐based functional resonance magnetic imaging (fMRI) data from the Human Connectome Project (HCP) database, where our Geo‐Net4Net not only achieves more accurate change detection results than other state‐of‐the‐art counterpart methods but also yields ubiquitous geometric patterns that manifest putative insights into brain function.
Collapse
Affiliation(s)
- Tingting Dan
- School of Computer Science and Engineering, South China University of Technology, Guangzhou, China
| | - Zhuobin Huang
- School of Computer Science and Engineering, South China University of Technology, Guangzhou, China
| | - Hongmin Cai
- School of Computer Science and Engineering, South China University of Technology, Guangzhou, China
| | - Robert G Lyday
- Department of Radiology, Wake Forest School of Medicine, Winston Salem, North Carolina, USA
| | - Paul J Laurienti
- Department of Radiology, Wake Forest School of Medicine, Winston Salem, North Carolina, USA
| | - Guorong Wu
- Department of Psychiatry, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA.,Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA.,Department of Statistics and Operations Research, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA.,Carolina Institute for Developmental Disabilities (CIDD), University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA.,UNC NeuroScience Center, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| |
Collapse
|
15
|
Li M, Wang Y, Lopez-Naranjo C, Hu S, Reyes RCG, Paz-Linares D, Areces-Gonzalez A, Hamid AIA, Evans AC, Savostyanov AN, Calzada-Reyes A, Villringer A, Tobon-Quintero CA, Garcia-Agustin D, Yao D, Dong L, Aubert-Vazquez E, Reza F, Razzaq FA, Omar H, Abdullah JM, Galler JR, Ochoa-Gomez JF, Prichep LS, Galan-Garcia L, Morales-Chacon L, Valdes-Sosa MJ, Tröndle M, Zulkifly MFM, Abdul Rahman MRB, Milakhina NS, Langer N, Rudych P, Koenig T, Virues-Alba TA, Lei X, Bringas-Vega ML, Bosch-Bayard JF, Valdes-Sosa PA. Harmonized-Multinational qEEG norms (HarMNqEEG). Neuroimage 2022; 256:119190. [PMID: 35398285 DOI: 10.1016/j.neuroimage.2022.119190] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2022] [Revised: 03/23/2022] [Accepted: 04/05/2022] [Indexed: 12/14/2022] Open
Abstract
This paper extends frequency domain quantitative electroencephalography (qEEG) methods pursuing higher sensitivity to detect Brain Developmental Disorders. Prior qEEG work lacked integration of cross-spectral information omitting important functional connectivity descriptors. Lack of geographical diversity precluded accounting for site-specific variance, increasing qEEG nuisance variance. We ameliorate these weaknesses. (i) Create lifespan Riemannian multinational qEEG norms for cross-spectral tensors. These norms result from the HarMNqEEG project fostered by the Global Brain Consortium. We calculate the norms with data from 9 countries, 12 devices, and 14 studies, including 1564 subjects. Instead of raw data, only anonymized metadata and EEG cross-spectral tensors were shared. After visual and automatic quality control, developmental equations for the mean and standard deviation of qEEG traditional and Riemannian DPs were calculated using additive mixed-effects models. We demonstrate qEEG "batch effects" and provide methods to calculate harmonized z-scores. (ii) We also show that harmonized Riemannian norms produce z-scores with increased diagnostic accuracy predicting brain dysfunction produced by malnutrition in the first year of life and detecting COVID induced brain dysfunction. (iii) We offer open code and data to calculate different individual z-scores from the HarMNqEEG dataset. These results contribute to developing bias-free, low-cost neuroimaging technologies applicable in various health settings.
Collapse
Affiliation(s)
- Min Li
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Ying Wang
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Carlos Lopez-Naranjo
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Shiang Hu
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China; Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, Key Laboratory of Intelligent Computing & Signal Processing of Ministry of Education, School of Computer Science and Technology, Anhui University, Hefei 230601, China
| | | | - Deirel Paz-Linares
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China; Cuban Center for Neurocience, La Habana, Cuba
| | - Ariosky Areces-Gonzalez
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China; University of Pinar del Río "Hermanos Saiz Montes de Oca", Pinar del Río, Cuba
| | - Aini Ismafairus Abd Hamid
- Brain and Behaviour Cluster, School of Medical Sciences, Universiti Sains Malaysia, Health Campus, Kota Bharu, Kelantan 16150, Malaysia; Hospital Universiti Sains Malaysia, Universiti Sains Malaysia, Health Campus, Kota Bharu, Kelantan 16150, Malaysia; McGill Centre for Integrative Neuroscience, Ludmer Centre for Neuroinformatics and Mental Health, Montreal Neurological Institute, Canada
| | - Alan C Evans
- McGill Centre for Integrative Neuroscience, Ludmer Centre for Neuroinformatics and Mental Health, Montreal Neurological Institute, Canada
| | - Alexander N Savostyanov
- Humanitarian Institute, Novosibirsk State University, Novosibirsk 630090, Russia; Laboratory of Psychophysiology of Individual Differences, Federal State Budgetary Scientific Institution Scientific Research Institute of Neurosciences and Medicine, Novosibirsk 630117, Russia; Laboratory of Psychological Genetics at the Institute of Cytology and Genetics Siberian Branch of the Russian Academy of Sciences, Novosibirsk 630090, Russia
| | | | - Arno Villringer
- Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany; Department of Cognitive Neurology, University Hospital Leipzig, Leipzig, Germany; Center for Stroke Research, Charité-Universitätsmedizin Berlin, Berlin, Germany
| | - Carlos A Tobon-Quintero
- Grupo Neuropsicología y Conducta - GRUNECO, Faculty of Medicine, Universidad de Antioquia, Colombia; Research Department, Institución Prestadora de Servicios de Salud IPS Universitaria, Colombia
| | - Daysi Garcia-Agustin
- Cuban Center for Neurocience, La Habana, Cuba; The Cuban center aging longevity and health, Havana Cuba
| | - Dezhong Yao
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China; Research Unit of NeuroInformation, Chinese Academy of Medical Sciences, Chengdu 2019RU035, China; School of Electrical Engineering, Zhengzhou University, Zhengzhou 450001, China
| | - Li Dong
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China; Research Unit of NeuroInformation, Chinese Academy of Medical Sciences, Chengdu 2019RU035, China; Sichuan Institute for Brain Science and Brain-Inspired Intelligence, Chengdu 611731, China
| | | | - Faruque Reza
- Brain and Behaviour Cluster, School of Medical Sciences, Universiti Sains Malaysia, Health Campus, Kota Bharu, Kelantan 16150, Malaysia; Hospital Universiti Sains Malaysia, Universiti Sains Malaysia, Health Campus, Kota Bharu, Kelantan 16150, Malaysia; McGill Centre for Integrative Neuroscience, Ludmer Centre for Neuroinformatics and Mental Health, Montreal Neurological Institute, Canada
| | - Fuleah Abdul Razzaq
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Hazim Omar
- Brain and Behaviour Cluster, School of Medical Sciences, Universiti Sains Malaysia, Health Campus, Kota Bharu, Kelantan 16150, Malaysia; Hospital Universiti Sains Malaysia, Universiti Sains Malaysia, Health Campus, Kota Bharu, Kelantan 16150, Malaysia; McGill Centre for Integrative Neuroscience, Ludmer Centre for Neuroinformatics and Mental Health, Montreal Neurological Institute, Canada
| | - Jafri Malin Abdullah
- Department of Neurosciences, School of Medical Sciences, Universiti Sains Malaysia, Universiti Sains Malaysia Health Campus, Kota Bharu, Kelantan 16150, Malaysia; Hospital Universiti Sains Malaysia, Universiti Sains Malaysia, Health Campus, Kota Bharu, Kelantan 16150, Malaysia; McGill Centre for Integrative Neuroscience, Ludmer Centre for Neuroinformatics and Mental Health, Montreal Neurological Institute, Canada
| | - Janina R Galler
- Division of Pediatric Gastroenterology and Nutrition, Massachusetts General Hospital for Children, Boston, MA, United States Massachusetts General Hospital for Children, Boston, MA, United States
| | - John F Ochoa-Gomez
- Grupo Neuropsicología y Conducta - GRUNECO, Faculty of Medicine, Universidad de Antioquia, Colombia; Grupo de Neurociencias de Antioquia, Universidad de Antioquia, School of Medicine. Medellín, Colombia
| | - Leslie S Prichep
- Research & Development, BrainScope Company, Inc. Bethesda, MD, United States; Department of Psychiatry (Ret.), Brain Research Laboratories, NYU School of Medicine, New York, NY, United States
| | | | - Lilia Morales-Chacon
- Department of Clinical Neurophysiology, International Center for Neurological Restoration, Playa, Havana 11300, Cuba
| | | | - Marius Tröndle
- Department of Methods of Plasticity Research, Institute of Psychology, University of Zurich, Zurich, Switzerland; University Research Priority Program "Dynamic of Healthy Aging", University of Zurich, Zurich, Switzerland; Neuroscience Center Zurich (ZNZ), Zurich, Switzerland
| | - Mohd Faizal Mohd Zulkifly
- Brain and Behaviour Cluster, School of Medical Sciences, Universiti Sains Malaysia, Health Campus, Kota Bharu, Kelantan 16150, Malaysia; Hospital Universiti Sains Malaysia, Universiti Sains Malaysia, Health Campus, Kota Bharu, Kelantan 16150, Malaysia; McGill Centre for Integrative Neuroscience, Ludmer Centre for Neuroinformatics and Mental Health, Montreal Neurological Institute, Canada
| | - Muhammad Riddha Bin Abdul Rahman
- Brain and Behaviour Cluster, School of Medical Sciences, Universiti Sains Malaysia, Health Campus, Kota Bharu, Kelantan 16150, Malaysia; Hospital Universiti Sains Malaysia, Universiti Sains Malaysia, Health Campus, Kota Bharu, Kelantan 16150, Malaysia; School of Medical Imaging, Faculty of Health Sciences, Universiti Sultan Zainal Abidin, Kuala Nerus 21300, Malaysia
| | - Natalya S Milakhina
- Laboratory of Psychophysiology of Individual Differences, Federal State Budgetary Scientific Institution Scientific Research Institute of Neurosciences and Medicine, Novosibirsk 630117, Russia; Laboratory of Psychological Genetics at the Institute of Cytology and Genetics Siberian Branch of the Russian Academy of Sciences, Novosibirsk 630090, Russia
| | - Nicolas Langer
- Department of Methods of Plasticity Research, Institute of Psychology, University of Zurich, Zurich, Switzerland; University Research Priority Program "Dynamic of Healthy Aging", University of Zurich, Zurich, Switzerland; Neuroscience Center Zurich (ZNZ), Zurich, Switzerland
| | - Pavel Rudych
- Laboratory of Psychophysiology of Individual Differences, Federal State Budgetary Scientific Institution Scientific Research Institute of Neurosciences and Medicine, Novosibirsk 630117, Russia; Department of Information Technologies Novosibirsk State University, Novosibirsk 630090, Russia; Federal Research Center for Information and Computational Technologies, Biomedical Data Processing Lab, Novosibirsk 630090, Russia
| | - Thomas Koenig
- University Hospital of Psychiatry and Psychotherapy, University of Bern, Bern, Switzerland
| | | | - Xu Lei
- Sleep and NeuroImaging Center, Faculty of Psychology, Southwest University, Chongqing 400715, China
| | - Maria L Bringas-Vega
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China; Cuban Center for Neurocience, La Habana, Cuba.
| | - Jorge F Bosch-Bayard
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China; Cuban Center for Neurocience, La Habana, Cuba; McGill Centre for Integrative Neuroscience, Ludmer Centre for Neuroinformatics and Mental Health, Montreal Neurological Institute, Canada.
| | - Pedro Antonio Valdes-Sosa
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China; Cuban Center for Neurocience, La Habana, Cuba.
| |
Collapse
|
16
|
Roy RN, Hinss MF, Darmet L, Ladouce S, Jahanpour ES, Somon B, Xu X, Drougard N, Dehais F, Lotte F. Retrospective on the First Passive Brain-Computer Interface Competition on Cross-Session Workload Estimation. Front Neuroergon 2022; 3:838342. [PMID: 38235453 PMCID: PMC10790860 DOI: 10.3389/fnrgo.2022.838342] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/17/2021] [Accepted: 02/16/2022] [Indexed: 01/19/2024]
Abstract
As is the case in several research domains, data sharing is still scarce in the field of Brain-Computer Interfaces (BCI), and particularly in that of passive BCIs-i.e., systems that enable implicit interaction or task adaptation based on a user's mental state(s) estimated from brain measures. Moreover, research in this field is currently hindered by a major challenge, which is tackling brain signal variability such as cross-session variability. Hence, with a view to develop good research practices in this field and to enable the whole community to join forces in working on cross-session estimation, we created the first passive brain-computer interface competition on cross-session workload estimation. This competition was part of the 3rd International Neuroergonomics conference. The data were electroencephalographic recordings acquired from 15 volunteers (6 females; average 25 y.o.) who performed 3 sessions-separated by 7 days-of the Multi-Attribute Task Battery-II (MATB-II) with 3 levels of difficulty per session (pseudo-randomized order). The data -training and testing sets-were made publicly available on Zenodo along with Matlab and Python toy code (https://doi.org/10.5281/zenodo.5055046). To this day, the database was downloaded more than 900 times (unique downloads of all version on the 10th of December 2021: 911). Eleven teams from 3 continents (31 participants) submitted their work. The best achieving processing pipelines included a Riemannian geometry-based method. Although better than the adjusted chance level (38% with an α at 0.05 for a 3-class classification problem), the results still remained under 60% of accuracy. These results clearly underline the real challenge that is cross-session estimation. Moreover, they confirmed once more the robustness and effectiveness of Riemannian methods for BCI. On the contrary, chance level results were obtained by one third of the methods-4 teams- based on Deep Learning. These methods have not demonstrated superior results in this contest compared to traditional methods, which may be due to severe overfitting. Yet this competition is the first step toward a joint effort to tackle BCI variability and to promote good research practices including reproducibility.
Collapse
Affiliation(s)
- Raphaëlle N. Roy
- ISAE-SUPAERO, Université de Toulouse, Toulouse, France
- Artificial and Natural Intelligence Toulouse Institute ANITI, Toulouse, France
| | | | | | - Simon Ladouce
- ISAE-SUPAERO, Université de Toulouse, Toulouse, France
| | | | - Bertille Somon
- ISAE-SUPAERO, Université de Toulouse, Toulouse, France
- Artificial and Natural Intelligence Toulouse Institute ANITI, Toulouse, France
| | - Xiaoqi Xu
- ISAE-SUPAERO, Université de Toulouse, Toulouse, France
- Artificial and Natural Intelligence Toulouse Institute ANITI, Toulouse, France
| | - Nicolas Drougard
- ISAE-SUPAERO, Université de Toulouse, Toulouse, France
- Artificial and Natural Intelligence Toulouse Institute ANITI, Toulouse, France
| | - Frédéric Dehais
- ISAE-SUPAERO, Université de Toulouse, Toulouse, France
- Artificial and Natural Intelligence Toulouse Institute ANITI, Toulouse, France
| | - Fabien Lotte
- Inria Bordeaux Sud-Ouest, Talence, France
- LaBRI (CNRS, Univ. Bordeaux, INP), Bordeaux, France
| |
Collapse
|
17
|
Wriessnegger SC, Raggam P, Kostoglou K, Müller-Putz GR. Mental State Detection Using Riemannian Geometry on Electroencephalogram Brain Signals. Front Hum Neurosci 2021; 15:746081. [PMID: 34899215 PMCID: PMC8663761 DOI: 10.3389/fnhum.2021.746081] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2021] [Accepted: 10/12/2021] [Indexed: 11/16/2022] Open
Abstract
The goal of this study was to implement a Riemannian geometry (RG)-based algorithm to detect high mental workload (MWL) and mental fatigue (MF) using task-induced electroencephalogram (EEG) signals. In order to elicit high MWL and MF, the participants performed a cognitively demanding task in the form of the letter n-back task. We analyzed the time-varying characteristics of the EEG band power (BP) features in the theta and alpha frequency band at different task conditions and cortical areas by employing a RG-based framework. MWL and MF were considered as too high, when the Riemannian distances of the task-run EEG reached or surpassed the threshold of the baseline EEG. The results of this study showed a BP increase in the theta and alpha frequency bands with increasing experiment duration, indicating elevated MWL and MF that impedes/hinders the task performance of the participants. High MWL and MF was detected in 8 out of 20 participants. The Riemannian distances also showed a steady increase toward the threshold with increasing experiment duration, with the most detections occurring toward the end of the experiment. To support our findings, subjective ratings (questionnaires concerning fatigue and workload levels) and behavioral measures (performance accuracies and response times) were also considered.
Collapse
Affiliation(s)
- Selina C Wriessnegger
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria.,BioTechMed-Graz, Graz, Austria
| | - Philipp Raggam
- Research Group Neuroinformatics, Faculty of Computer Science, University of Vienna, Vienna, Austria.,Department of Neurology and Stroke, Hertie Institute for Clinical Brain Research, University of Tübingen, Tübingen, Germany
| | - Kyriaki Kostoglou
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria
| | - Gernot R Müller-Putz
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria.,BioTechMed-Graz, Graz, Austria
| |
Collapse
|
18
|
Tang S, Liu C, Zhang Q, Gu H, Li X, Li Z. Mental workload classification based on ignored auditory probes and spatial covariance. J Neural Eng 2021; 18. [PMID: 34280906 DOI: 10.1088/1741-2552/ac15e5] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2021] [Accepted: 07/19/2021] [Indexed: 11/12/2022]
Abstract
Objective.Estimation of mental workload (MWL) levels by electroencephalography (EEG)-based mental state monitoring systems has been widely explored. Using event-related potentials (ERPs), elicited by ignored auditory probes, minimizes intrusiveness and has shown high performance for estimating MWL level when tested in laboratory situations. However, when facing real-world applications, the characteristics of ERP waveforms, like latency and amplitude, are often affected by noise, which leads to a decrease in classification performance. One approach to mitigating this is using spatial covariance, which is less sensitive to latency and amplitude distortion. In this study, we used ignored auditory probes in a single-stimulus paradigm and tested Riemannian processed covariance-based features for MWL level estimation in a realistic flight-control task.Approach.We recorded EEG data with an eight-channel system from participants while they performed a simulated drone-control task and manipulated MWL levels (high and low) by task difficulty. We compared support vector machine classification performance based on frequency band power features versus features generated via the Riemannian log map operator from spatial covariance matrices. We also compared accuracy of using data segmented as auditory ERPs versus non-ERPs, for which data windows did not overlap with the ERPs.Main results.Classification accuracy of both types of features showed no significant difference between ERPs and non-ERPs. When we ignore auditory stimuli to perform continuous decoding, covariance-based features in the gamma band had area under the receiver-operating-characteristic curve (AUC) of 0.883, which was significantly higher than band power features (AUC = 0.749).Significance.This study demonstrates that Riemannian-processed covariance features are viable for MWL classification under a realistic experimental scenario.
Collapse
Affiliation(s)
- Shaohua Tang
- Center for Cognition and Neuroergonomics, State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University at Zhuhai, Zhuhai, People's Republic of China
| | - Chuancai Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, People's Republic of China
| | - Qiankun Zhang
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, People's Republic of China
| | - Heng Gu
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, People's Republic of China
| | - Xiaoli Li
- Center for Cognition and Neuroergonomics, State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University at Zhuhai, Zhuhai, People's Republic of China.,State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, People's Republic of China
| | - Zheng Li
- Center for Cognition and Neuroergonomics, State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University at Zhuhai, Zhuhai, People's Republic of China.,State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, People's Republic of China
| |
Collapse
|
19
|
Shariat A, Zarei A, Karvigh SA, Asl BM. Automatic detection of epileptic seizures using Riemannian geometry from scalp EEG recordings. Med Biol Eng Comput 2021; 59:1431-1445. [PMID: 34128177 DOI: 10.1007/s11517-021-02385-z] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2020] [Accepted: 05/15/2021] [Indexed: 11/30/2022]
Abstract
This paper proposes a new framework for epileptic seizure detection using non-invasive scalp electroencephalogram (sEEG) signals. The major innovation of the current study is using the Riemannian geometry for transforming the covariance matrices estimated from the EEG channels into a feature vector. The spatial covariance matrices are considered as features in order to extract the spatial information of the sEEG signals without applying any spatial filtering. Since these matrices are symmetric and positive definite (SPD), they belong to a special manifold called the Riemannian manifold. Furthermore, a kernel based on Riemannian geometry is proposed. This kernel maps the SPD matrices onto the Riemannian tangent space. The SPD matrices, obtained from all channels of the segmented sEEG signals, have high dimensions and extra information. For these reasons, the sequential forward feature selection method is applied to select the best features and reduce the computational burden in the classification step. The selected features are fed into a support vector machine (SVM) with an RBF kernel to classify the feature vectors into seizure and non-seizure classes. The performance of the proposed method is evaluated using two long-term scalp EEG (CHB-MIT benchmark and private) databases. Experimental results on all 23 subjects of the CHB-MIT database reveal an accuracy of 99.87%, a sensitivity of 99.91%, and a specificity of 99.82%. In addition, the introduced algorithm is tested on the private sEEG signals recorded from 20 patients, having 1380 seizures. The proposed approach achieves an accuracy, a sensitivity, and a specificity of 98.14%, 98.16%, and 98.12%, respectively. The experimental results on both sEEG databases demonstrate the effectiveness of the proposed method for automated epileptic seizure detection, especially for the private database which has noisier signals in comparison to the CHB-MIT database. Graphical Abstract Block diagram of the proposed epileptic seizure detection algorithm.
Collapse
Affiliation(s)
- Atefeh Shariat
- Department of Biomedical Engineering, Tarbiat Modares University, Tehran, Iran
| | - Asghar Zarei
- Department of Biomedical Engineering, Tarbiat Modares University, Tehran, Iran
| | - Sanaz Ahmadi Karvigh
- Department of Neurology, Sina Hospital, Tehran University of Medical Sciences, Tehran, Iran
| | | |
Collapse
|
20
|
Duan X, Xie S, Xie X, Obermayer K, Cui Y, Wang Z. An Online Data Visualization Feedback Protocol for Motor Imagery-Based BCI Training. Front Hum Neurosci 2021; 15:625983. [PMID: 34163337 PMCID: PMC8215169 DOI: 10.3389/fnhum.2021.625983] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2020] [Accepted: 04/29/2021] [Indexed: 11/13/2022] Open
Abstract
Brain-computer interface (BCI) has developed rapidly over the past two decades, mainly due to advancements in machine learning. Subjects must learn to modulate their brain activities to ensure a successful BCI. Feedback training is a practical approach to this learning process; however, the commonly used classifier-dependent approaches have inherent limitations such as the need for calibration and a lack of continuous feedback over long periods of time. This paper proposes an online data visualization feedback protocol that intuitively reflects the EEG distribution in Riemannian geometry in real time. Rather than learning a hyperplane, the Riemannian geometry formulation allows iterative learning of prototypical covariance matrices that are translated into visualized feedback through diffusion map process. Ten subjects were recruited for MI-BCI (motor imagery-BCI) training experiments. The subjects learned to modulate their sensorimotor rhythm to centralize the points within one category and to separate points belonging to different categories. The results show favorable overall training effects in terms of the class distinctiveness and EEG feature discriminancy over a 3-day training with 30% learners. A steadily increased class distinctiveness in the last three sessions suggests that the advanced training protocol is effective. The optimal frequency band was consistent during the 3-day training, and the difference between subjects with good or low MI-BCI performance could be clearly observed. We believe that the proposed feedback protocol has promising application prospect.
Collapse
Affiliation(s)
- Xu Duan
- School of Electronics and Information, Northwestern Polytechnical University, Xi'an, China
| | - Songyun Xie
- School of Electronics and Information, Northwestern Polytechnical University, Xi'an, China
| | - Xinzhou Xie
- School of Electronics and Information, Northwestern Polytechnical University, Xi'an, China
| | - Klaus Obermayer
- Faculty of Electrical Engineering and Computer Science, Technical University Berlin, Berlin, Germany
| | - Yujie Cui
- School of Electronics and Information, Northwestern Polytechnical University, Xi'an, China
| | - Zhenzhen Wang
- School of Electronics and Information, Northwestern Polytechnical University, Xi'an, China
| |
Collapse
|
21
|
Neilson PD, Neilson MD, Bye RT. A Riemannian Geometry Theory of Synergy Selection for Visually-Guided Movement. Vision (Basel) 2021; 5:26. [PMID: 34070234 PMCID: PMC8163178 DOI: 10.3390/vision5020026] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2021] [Revised: 05/06/2021] [Accepted: 05/08/2021] [Indexed: 11/16/2022] Open
Abstract
Bringing together a Riemannian geometry account of visual space with a complementary account of human movement synergies we present a neurally-feasible computational formulation of visuomotor task performance. This cohesive geometric theory addresses inherent nonlinear complications underlying the match between a visual goal and an optimal action to achieve that goal: (i) the warped geometry of visual space causes the position, size, outline, curvature, velocity and acceleration of images to change with changes in the place and orientation of the head, (ii) the relationship between head place and body posture is ill-defined, and (iii) mass-inertia loads on muscles vary with body configuration and affect the planning of minimum-effort movement. We describe a partitioned visuospatial memory consisting of the warped posture-and-place-encoded images of the environment, including images of visible body parts. We depict synergies as low-dimensional submanifolds embedded in the warped posture-and-place manifold of the body. A task-appropriate synergy corresponds to a submanifold containing those postures and places that match the posture-and-place-encoded visual images that encompass the required visual goal. We set out a reinforcement learning process that tunes an error-reducing association memory network to minimize any mismatch, thereby coupling visual goals with compatible movement synergies. A simulation of a two-degrees-of-freedom arm illustrates that, despite warping of both visual space and posture space, there exists a smooth one-to-one and onto invertible mapping between vision and proprioception.
Collapse
Affiliation(s)
- Peter D. Neilson
- School of Electrical Engineering and Telecommunications, University of New South Wales, Sydney, NSW 2052, Australia
| | - Megan D. Neilson
- Independent Researcher, late School of Electrical Engineering and Telecommunications, University of New South Wales, Sydney, NSW 2052, Australia;
| | - Robin T. Bye
- Cyber-Physical Systems Laboratory, Department of ICT and Natural Sciences, NTNU—Norwegian University of Science and Technology, Postboks 1517, NO-6009 Ålesund, Norway;
| |
Collapse
|
22
|
Abstract
The firstgeneration of brain-computer interfaces (BCI) classifies multi-channel electroencephalographic (EEG) signals, enhanced by optimized spatial filters.The second generation directly classifies covariance matrices estimated on EEG signals, based on straightforward algorithms such as the minimum-distance-to-Riemannian-mean (MDRM). Classification results vary greatly depending on the chosen Riemannian distance or divergence, whose definitions and reference implementations are spread across a wide mathematical literature. This paper reviews all the Riemannian distances and divergences to process covariance matrices, with an implementation compatible with BCI constraints. The impact of using different metrics is assessed on a steady-state visually evoked potentials (SSVEP) dataset, evaluating centers of classes and classification accuracy. Riemannian approaches embed crucial properties to process EEG data. The Riemannian centers of classes outperform Euclidean ones both in offline and online setups. Some Riemannian distances and divergences have better performances in terms of classification accuracy, while others have appealing computational efficiency.
Collapse
|
23
|
Larzabal C, Auboiroux V, Karakas S, Charvet G, Benabid AL, Chabardès S, Costecalde T, Bonnet S. The Riemannian Spatial Pattern method: mapping and clustering movement imagery using Riemannian geometry. J Neural Eng 2021; 18. [PMID: 33770779 DOI: 10.1088/1741-2552/abf291] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2020] [Accepted: 03/26/2021] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Over the last decade, Riemannian geometry has shown promising results for motor imagery classification. However, extracting the underlying spatial features is not as straightforward as for applying Common Spatial Pattern (CSP) filtering prior to classification. In this article, we propose a simple way to extract the spatial patterns obtained from Riemannian classification: the Riemannian Spatial Pattern (RSP) method, which is based on the backward channel selection procedure. APPROACH The RSP method was compared to the CSP approach on ECoG data obtained from a quadriplegic patient while performing imagined movements of arm articulations and fingers. MAIN RESULTS Similar results were found between the RSP and CSP methods for mapping each motor imagery task with activations following the classical somatotopic organization. Clustering obtained by pairwise comparisons of imagined motor movements however, revealed higher differentiation for the RSP method compared to the CSP approach. Importantly, the RSP approach could provide a precise comparison of the imagined finger flexions which added supplementary information to the mapping results. SIGNIFICANCE Our new RSP method illustrates the interest of the Riemannian framework in the spatial domain and as such offers new avenues for the neuroimaging community. This study is part of an ongoing clinical trial registered with ClinicalTrials.gov, NCT02550522.
Collapse
Affiliation(s)
| | - Vincent Auboiroux
- CEA de Grenoble, Clinatec, Grenoble, Auvergne-Rhône-Alpes, 38054, FRANCE
| | - Serpil Karakas
- CEA de Grenoble, Clinatec, Grenoble, Auvergne-Rhône-Alpes, 38054, FRANCE
| | - Guillaume Charvet
- CEA de Grenoble, Clinatec, Grenoble, Auvergne-Rhône-Alpes, 38054, FRANCE
| | - Alim-Louis Benabid
- CEA de Grenoble, Clinatec, Grenoble, Auvergne-Rhône-Alpes, 38054, FRANCE
| | | | - Thomas Costecalde
- CEA de Grenoble, Clinatec, Grenoble, Auvergne-Rhône-Alpes, 38054, FRANCE
| | - Stephane Bonnet
- CEA de Grenoble, DTBS, Grenoble, Auvergne-Rhône-Alpes, 38000, FRANCE
| |
Collapse
|
24
|
Goodbrake C, Goriely A, Yavari A. The mathematical foundations of anelasticity: existence of smooth global intermediate configurations. Proc Math Phys Eng Sci 2021; 477:20200462. [PMID: 33642925 DOI: 10.1098/rspa.2020.0462] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2020] [Accepted: 12/01/2020] [Indexed: 11/12/2022] Open
Abstract
A central tool of nonlinear anelasticity is the multiplicative decomposition of the deformation tensor that assumes that the deformation gradient can be decomposed as a product of an elastic and an anelastic tensor. It is usually justified by the existence of an intermediate configuration. Yet, this configuration cannot exist in Euclidean space, in general, and the mathematical basis for this assumption is on unsatisfactory ground. Here, we derive a sufficient condition for the existence of global intermediate configurations, starting from a multiplicative decomposition of the deformation gradient. We show that these global configurations are unique up to isometry. We examine the result of isometrically embedding these configurations in higher-dimensional Euclidean space, and construct multiplicative decompositions of the deformation gradient reflecting these embeddings. As an example, for a family of radially symmetric deformations, we construct isometric embeddings of the resulting intermediate configurations, and compute the residual stress fields explicitly.
Collapse
Affiliation(s)
| | - Alain Goriely
- Mathematical Institute, University of Oxford, Oxford OX2 6GG, UK
| | - Arash Yavari
- School of Civil and Environmental Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA.,The George W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA
| |
Collapse
|
25
|
Simar C, Cebolla AM, Chartier G, Petieau M, Bontempi G, Berthoz A, Cheron G. Hyperscanning EEG and Classification Based on Riemannian Geometry for Festive and Violent Mental State Discrimination. Front Neurosci 2020; 14:588357. [PMID: 33424535 PMCID: PMC7793677 DOI: 10.3389/fnins.2020.588357] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2020] [Accepted: 11/04/2020] [Indexed: 12/14/2022] Open
Abstract
Interactions between two brains constitute the essence of social communication. Daily movements are commonly executed during social interactions and are determined by different mental states that may express different positive or negative behavioral intent. In this context, the effective recognition of festive or violent intent before the action execution remains crucial for survival. Here, we hypothesize that the EEG signals contain the distinctive features characterizing movement intent already expressed before movement execution and that such distinctive information can be identified by state-of-the-art classification algorithms based on Riemannian geometry. We demonstrated for the first time that a classifier based on covariance matrices and Riemannian geometry can effectively discriminate between neutral, festive, and violent mental states only on the basis of non-invasive EEG signals in both the actor and observer participants. These results pave the way for new electrophysiological discrimination of mental states based on non-invasive EEG recordings and cutting-edge machine learning techniques.
Collapse
Affiliation(s)
- Cédric Simar
- Machine Learning Group (MLG), Computer Science Department, Université libre de Bruxelles (ULB), Brussels, Belgium
| | - Ana-Maria Cebolla
- Laboratory of Neurophysiology and Movement Biomechanics, ULB Neuroscience Institute, Université libre de Bruxelles, Brussels, Belgium
| | - Gaëlle Chartier
- Centre Interdisciplinaire de Biologie, Collège de France-CNRS, Paris, France.,Department of Health, Medicine and Human Biology, Université Paris 13, Bobigny, France
| | - Mathieu Petieau
- Laboratory of Neurophysiology and Movement Biomechanics, ULB Neuroscience Institute, Université libre de Bruxelles, Brussels, Belgium
| | - Gianluca Bontempi
- Machine Learning Group (MLG), Computer Science Department, Université libre de Bruxelles (ULB), Brussels, Belgium
| | - Alain Berthoz
- Centre Interdisciplinaire de Biologie, Collège de France-CNRS, Paris, France
| | - Guy Cheron
- Laboratory of Neurophysiology and Movement Biomechanics, ULB Neuroscience Institute, Université libre de Bruxelles, Brussels, Belgium.,Laboratory of Electrophysiology, Université de Mons-Hainaut, Mons, Belgium
| |
Collapse
|
26
|
Kim YW, Kim S, Shim M, Jin MJ, Jeon H, Lee SH, Im CH. Riemannian classifier enhances the accuracy of machine-learning-based diagnosis of PTSD using resting EEG. Prog Neuropsychopharmacol Biol Psychiatry 2020; 102:109960. [PMID: 32376342 DOI: 10.1016/j.pnpbp.2020.109960] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/22/2019] [Revised: 04/19/2020] [Accepted: 04/30/2020] [Indexed: 12/14/2022]
Abstract
Recently, objective and automated methods for the diagnosis of post-traumatic stress disorder (PTSD) have attracted increasing attention. However, previous studies on machine-learning-based diagnosis of PTSD with resting-state electroencephalogram (EEG) have reported poor accuracies of as low as 60%. Here, a Riemannian geometry-based classifier, the Fisher geodesic minimum distance to the mean (FgMDM), was employed for PTSD classification for the first time. Eyes-closed resting-state EEG data of 39 healthy individuals and 42 PTSD patients were used for the analysis. EEG source activities in 148 cortical regions were parcellated based on the Destrieux atlas, and their covariances were evaluated for each individual. Thirty epochs of preprocessed EEG were employed to calculate source activities. In addition, the FgMDM approach was applied to each EEG source covariance to construct the classifier. For a comparison, linear discriminant analysis (LDA), support vector machine (SVM), and random forest (RF) classifiers employing source band powers and network features as feature candidates were also tested. The FgMDM classifier showed an average classification accuracy of 75.240.80%. In contrast, the maximum accuracies of LDA, SVM, and RF classifiers were 66.54 ± 2.99%, 61.11 ± 2.98%, and 60.99 ± 2.19%, respectively. Our study demonstrated that the diagnostic accuracy of PTSD with resting-state EEG could be significantly improved by employing the FgMDM framework, which is a type of Riemannian geometry-based classifier.
Collapse
Affiliation(s)
- Yong-Wook Kim
- Department of Biomedical Engineering, Hanyang University, Seoul, Republic of Korea; Clinical Emotion and Cognition Research Laboratory, Inje University, Goyang, Republic of Korea
| | - Sungkean Kim
- Department of Biomedical Engineering, Hanyang University, Seoul, Republic of Korea; Clinical Emotion and Cognition Research Laboratory, Inje University, Goyang, Republic of Korea
| | - Miseon Shim
- Department of Psychiatry, University of Missouri-Kansas City, Kansas City, MO, USA
| | - Min Jin Jin
- Clinical Emotion and Cognition Research Laboratory, Inje University, Goyang, Republic of Korea; Department of psychology, Chung-Ang University, Seoul, Republic of Korea
| | - Hyeonjin Jeon
- Clinical Emotion and Cognition Research Laboratory, Inje University, Goyang, Republic of Korea
| | - Seung-Hwan Lee
- Clinical Emotion and Cognition Research Laboratory, Inje University, Goyang, Republic of Korea; Department of Psychiatry, Inje University, Ilsan-Paik Hospital, Goyang, Republic of Korea.
| | - Chang-Hwan Im
- Department of Biomedical Engineering, Hanyang University, Seoul, Republic of Korea.
| |
Collapse
|
27
|
Gemein LAW, Schirrmeister RT, Chrabąszcz P, Wilson D, Boedecker J, Schulze-Bonhage A, Hutter F, Ball T. Machine-learning-based diagnostics of EEG pathology. Neuroimage 2020; 220:117021. [PMID: 32534126 DOI: 10.1016/j.neuroimage.2020.117021] [Citation(s) in RCA: 60] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2019] [Revised: 03/16/2020] [Accepted: 05/26/2020] [Indexed: 02/07/2023] Open
Abstract
Machine learning (ML) methods have the potential to automate clinical EEG analysis. They can be categorized into feature-based (with handcrafted features), and end-to-end approaches (with learned features). Previous studies on EEG pathology decoding have typically analyzed a limited number of features, decoders, or both. For a I) more elaborate feature-based EEG analysis, and II) in-depth comparisons of both approaches, here we first develop a comprehensive feature-based framework, and then compare this framework to state-of-the-art end-to-end methods. To this aim, we apply the proposed feature-based framework and deep neural networks including an EEG-optimized temporal convolutional network (TCN) to the task of pathological versus non-pathological EEG classification. For a robust comparison, we chose the Temple University Hospital (TUH) Abnormal EEG Corpus (v2.0.0), which contains approximately 3000 EEG recordings. The results demonstrate that the proposed feature-based decoding framework can achieve accuracies on the same level as state-of-the-art deep neural networks. We find accuracies across both approaches in an astonishingly narrow range from 81 to 86%. Moreover, visualizations and analyses indicated that both approaches used similar aspects of the data, e.g., delta and theta band power at temporal electrode locations. We argue that the accuracies of current binary EEG pathology decoders could saturate near 90% due to the imperfect inter-rater agreement of the clinical labels, and that such decoders are already clinically useful, such as in areas where clinical EEG experts are rare. We make the proposed feature-based framework available open source and thus offer a new tool for EEG machine learning research.
Collapse
Affiliation(s)
- Lukas A W Gemein
- Neuromedical AI Lab, Department of Neurosurgery, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Engelbergerstr. 21, 79106, Freiburg, Germany; Machine Learning Lab, Computer Science Department - University of Freiburg, Faculty of Engineering, University of Freiburg, Georges-Köhler-Allee 74, 79110, Freiburg, Germany; Neurorobotics Lab, Computer Science Department - University of Freiburg, Faculty of Engineering, University of Freiburg, Georges-Köhler-Allee 80, 79110, Freiburg, Germany.
| | - Robin T Schirrmeister
- Neuromedical AI Lab, Department of Neurosurgery, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Engelbergerstr. 21, 79106, Freiburg, Germany; Machine Learning Lab, Computer Science Department - University of Freiburg, Faculty of Engineering, University of Freiburg, Georges-Köhler-Allee 74, 79110, Freiburg, Germany
| | - Patryk Chrabąszcz
- Neuromedical AI Lab, Department of Neurosurgery, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Engelbergerstr. 21, 79106, Freiburg, Germany; Machine Learning Lab, Computer Science Department - University of Freiburg, Faculty of Engineering, University of Freiburg, Georges-Köhler-Allee 74, 79110, Freiburg, Germany
| | - Daniel Wilson
- Neuromedical AI Lab, Department of Neurosurgery, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Engelbergerstr. 21, 79106, Freiburg, Germany
| | - Joschka Boedecker
- Neurorobotics Lab, Computer Science Department - University of Freiburg, Faculty of Engineering, University of Freiburg, Georges-Köhler-Allee 80, 79110, Freiburg, Germany
| | - Andreas Schulze-Bonhage
- Freiburg Epilepsy Center, Department of Neurosurgery, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Breisacher Str. 64, 79106, Freiburg, Germany
| | - Frank Hutter
- Machine Learning Lab, Computer Science Department - University of Freiburg, Faculty of Engineering, University of Freiburg, Georges-Köhler-Allee 74, 79110, Freiburg, Germany
| | - Tonio Ball
- Neuromedical AI Lab, Department of Neurosurgery, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Engelbergerstr. 21, 79106, Freiburg, Germany; Freiburg Epilepsy Center, Department of Neurosurgery, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Breisacher Str. 64, 79106, Freiburg, Germany
| |
Collapse
|
28
|
Sabbagh D, Ablin P, Varoquaux G, Gramfort A, Engemann DA. Predictive regression modeling with MEG/EEG: from source power to signals and cognitive states. Neuroimage 2020; 222:116893. [PMID: 32439535 DOI: 10.1016/j.neuroimage.2020.116893] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2019] [Revised: 04/17/2020] [Accepted: 04/27/2020] [Indexed: 01/22/2023] Open
Abstract
Predicting biomedical outcomes from Magnetoencephalography and Electroencephalography (M/EEG) is central to applications like decoding, brain-computer-interfaces (BCI) or biomarker development and is facilitated by supervised machine learning. Yet, most of the literature is concerned with classification of outcomes defined at the event-level. Here, we focus on predicting continuous outcomes from M/EEG signal defined at the subject-level, and analyze about 600 MEG recordings from Cam-CAN dataset and about 1000 EEG recordings from TUH dataset. Considering different generative mechanisms for M/EEG signals and the biomedical outcome, we propose statistically-consistent predictive models that avoid source-reconstruction based on the covariance as representation. Our mathematical analysis and ground-truth simulations demonstrated that consistent function approximation can be obtained with supervised spatial filtering or by embedding with Riemannian geometry. Additional simulations revealed that Riemannian methods were more robust to model violations, in particular geometric distortions induced by individual anatomy. To estimate the relative contribution of brain dynamics and anatomy to prediction performance, we propose a novel model inspection procedure based on biophysical forward modeling. Applied to prediction of outcomes at the subject-level, the analysis revealed that the Riemannian model better exploited anatomical information while sensitivity to brain dynamics was similar across methods. We then probed the robustness of the models across different data cleaning options. Environmental denoising was globally important but Riemannian models were strikingly robust and continued performing well even without preprocessing. Our results suggest each method has its niche: supervised spatial filtering is practical for event-level prediction while the Riemannian model may enable simple end-to-end learning.
Collapse
Affiliation(s)
- David Sabbagh
- Université Paris-Saclay, Inria, CEA, Palaiseau, France; Inserm, UMRS-942, Paris Diderot University, Paris, France; Department of Anaesthesiology and Critical Care, Lariboisière Hospital, Assistance Publique Hôpitaux de Paris, Paris, France.
| | - Pierre Ablin
- Université Paris-Saclay, Inria, CEA, Palaiseau, France
| | | | | | - Denis A Engemann
- Université Paris-Saclay, Inria, CEA, Palaiseau, France; Max Planck Institute for Human Cognitive and Brain Sciences, Department of Neurology, D-04103, Leipzig, Germany.
| |
Collapse
|
29
|
Abstract
A major goal of neuroimaging studies is to develop predictive models to analyze the relationship between whole brain functional connectivity patterns and behavioural traits. However, there is no single widely-accepted standard pipeline for analyzing functional connectivity. The common procedure for designing functional connectivity based predictive models entails three main steps: parcellating the brain, estimating the interaction between defined parcels, and lastly, using these integrated associations between brain parcels as features fed to a classifier for predicting non-imaging variables e.g., behavioural traits, demographics, emotional measures, etc. There are also additional considerations when using correlation-based measures of functional connectivity, resulting in three supplementary steps: utilising Riemannian geometry tangent space parameterization to preserve the geometry of functional connectivity; penalizing the connectivity estimates with shrinkage approaches to handle challenges related to short time-series (and noisy) data; and removing confounding variables from brain-behaviour data. These six steps are contingent on each-other, and to optimise a general framework one should ideally examine these various methods simultaneously. In this paper, we investigated strengths and short-comings, both independently and jointly, of the following measures: parcellation techniques of four kinds (categorized further depending upon number of parcels), five measures of functional connectivity, the decision of staying in the ambient space of connectivity matrices or in tangent space, the choice of applying shrinkage estimators, six alternative techniques for handling confounds and finally four novel classifiers/predictors. For performance evaluation, we have selected two of the largest datasets, UK Biobank and the Human Connectome Project resting state fMRI data, and have run more than 9000 different pipeline variants on a total of ∼14000 individuals to determine the optimum pipeline. For independent performance validation, we have run some best-performing pipeline variants on ABIDE and ACPI datasets (∼1000 subjects) to evaluate the generalisability of proposed network modelling methods.
Collapse
Affiliation(s)
- Usama Pervaiz
- Oxford Centre for Functional MRI of the Brain (FMRIB), Wellcome Centre for Integrative Neuroimaging, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, OX3 9DU, United Kingdom.
| | - Diego Vidaurre
- Oxford Centre for Human Brain Activity (OHBA), Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford, OX3 7JX, United Kingdom; Department of Clinical Medicine, Aarhus University, Denmark
| | - Mark W Woolrich
- Oxford Centre for Human Brain Activity (OHBA), Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford, OX3 7JX, United Kingdom
| | - Stephen M Smith
- Oxford Centre for Functional MRI of the Brain (FMRIB), Wellcome Centre for Integrative Neuroimaging, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, OX3 9DU, United Kingdom
| |
Collapse
|
30
|
Abstract
Electroencephalography (EEG)-based brain–computer interface (BCI) systems infer brain signals recorded via EEG without using common neuromuscular pathways. User brain response to BCI error is a contributor to non-stationarity of the EEG signal and poses challenges in developing reliable active BCI control. Many passive BCI implementations, on the other hand, have the detection of error-related brain activity as their primary goal. Therefore, reliable detection of this signal is crucial in both active and passive BCIs. In this work, we propose CREST: a novel covariance-based method that uses Riemannian and Euclidean geometry and combines spatial and temporal aspects of the feedback-related brain activity in response to BCI error. We evaluate our proposed method with two datasets: an active BCI for 1-D cursor control using motor imagery and a passive BCI for 2-D cursor control. We show significant improvement across participants in both datasets compared to existing methods.
Collapse
Affiliation(s)
- M Mousavi
- Department of Electrical and Computer Engineering, University of California, San Diego, CA, USA
| | - V R de Sa
- Department of Cognitive Science, University of California, San Diego, CA, USA.,Halıcıoğlu Data Science Institute, University of California, San Diego, CA, USA
| |
Collapse
|
31
|
Guttmann-Flury E, Sheng X, Zhang D, Zhu X. A new algorithm for blink correction adaptive to inter- and intra-subject variability. Comput Biol Med 2019; 114:103442. [PMID: 31550554 DOI: 10.1016/j.compbiomed.2019.103442] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2019] [Revised: 09/04/2019] [Accepted: 09/07/2019] [Indexed: 11/30/2022]
Abstract
Electroencephalographic (EEG) signals are constantly superimposed with biological artifacts. In particular, spontaneous blinks represent a recurrent event that cannot be easily avoided. The main goal of this paper is to present a new algorithm for blink correction (ABC) that is adaptive to inter- and intra-subject variability. The whole process of designing a Brain-Computer Interface (BCI)-based EEG experiment is highlighted. From sample size determination to classification, a mixture of the standardized low-resolution electromagnetic tomography (sLORETA) for source localization and time restriction, followed by Riemannian geometry classifiers is featured. Comparison between ABC and the commonly-used Independent Component Analysis (ICA) for blinks removal shows a net amelioration with ABC. With the same pipeline using uncorrected data as a reference, ABC improves classification by 5.38% in average, whereas ICA deteriorates by -2.67%. Furthermore, while ABC accurately reconstructs blink-free data from simulated data, ICA yields a potential difference up to 200% from the original blink-free signal and an increased variance of 30.42%. Finally, ABC's major advantages are ease of visualization and understanding, low computation load favoring simple real-time implementation, and lack of spatial filtering, which allows for more flexibility during the classification step.
Collapse
Affiliation(s)
- E Guttmann-Flury
- State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, 800 Dongchuan Road, Minhang District, Shanghai, 200240, PR China.
| | - X Sheng
- State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, 800 Dongchuan Road, Minhang District, Shanghai, 200240, PR China
| | - D Zhang
- State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, 800 Dongchuan Road, Minhang District, Shanghai, 200240, PR China
| | - X Zhu
- State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, 800 Dongchuan Road, Minhang District, Shanghai, 200240, PR China.
| | -
- State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, 800 Dongchuan Road, Minhang District, Shanghai, 200240, PR China
| |
Collapse
|
32
|
Majidov I, Whangbo T. Efficient Classification of Motor Imagery Electroencephalography Signals Using Deep Learning Methods. Sensors (Basel) 2019; 19:s19071736. [PMID: 30978978 PMCID: PMC6479542 DOI: 10.3390/s19071736] [Citation(s) in RCA: 45] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/24/2019] [Revised: 03/29/2019] [Accepted: 04/08/2019] [Indexed: 11/20/2022]
Abstract
Single-trial motor imagery classification is a crucial aspect of brain–computer applications. Therefore, it is necessary to extract and discriminate signal features involving motor imagery movements. Riemannian geometry-based feature extraction methods are effective when designing these types of motor-imagery-based brain–computer interface applications. In the field of information theory, Riemannian geometry is mainly used with covariance matrices. Accordingly, investigations showed that if the method is used after the execution of the filterbank approach, the covariance matrix preserves the frequency and spatial information of the signal. Deep-learning methods are superior when the data availability is abundant and while there is a large number of features. The purpose of this study is to a) show how to use a single deep-learning-based classifier in conjunction with BCI (brain–computer interface) applications with the CSP (common spatial features) and the Riemannian geometry feature extraction methods in BCI applications and to b) describe one of the wrapper feature-selection algorithms, referred to as the particle swarm optimization, in combination with a decision tree algorithm. In this work, the CSP method was used for a multiclass case by using only one classifier. Additionally, a combination of power spectrum density features with covariance matrices mapped onto the tangent space of a Riemannian manifold was used. Furthermore, the particle swarm optimization method was implied to ease the training by penalizing bad features, and the moving windows method was used for augmentation. After empirical study, the convolutional neural network was adopted to classify the pre-processed data. Our proposed method improved the classification accuracy for several subjects that comprised the well-known BCI competition IV 2a dataset.
Collapse
Affiliation(s)
- Ikhtiyor Majidov
- Department of Computer Science Gachon University, Sujeong-Gu, Seongnam-Si, Gyeonggi-Do 13109, Korea.
| | - Taegkeun Whangbo
- Department of Computer Science Gachon University, Sujeong-Gu, Seongnam-Si, Gyeonggi-Do 13109, Korea.
| |
Collapse
|
33
|
Saifutdinova E, Congedo M, Dudysova D, Lhotska L, Koprivova J, Gerla V. An Unsupervised Multichannel Artifact Detection Method for Sleep EEG Based on Riemannian Geometry. Sensors (Basel) 2019; 19:E602. [PMID: 30709001 DOI: 10.3390/s19030602] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/15/2018] [Revised: 01/11/2019] [Accepted: 01/29/2019] [Indexed: 11/22/2022]
Abstract
In biomedical signal processing, we often face the problem of artifacts that distort the original signals. This concerns also sleep recordings, such as EEG. Artifacts may severely affect or even make impossible visual inspection, as well as automatic processing. Many proposed methods concentrate on certain artifact types. Therefore, artifact-free data are often obtained after sequential application of different methods. Moreover, single-channel approaches must be applied to all channels alternately. The aim of this study is to develop a multichannel artifact detection method for multichannel sleep EEG capable of rejecting different artifact types at once. The inspiration for the study is gained from recent advances in the field of Riemannian geometry. The method we propose is tested on real datasets. The performance of the proposed method is measured by comparing detection results with the expert labeling as a reference and evaluated against a simpler method based on Riemannian geometry that has previously been proposed, as well as against the state-of-the-art method FASTER. The obtained results prove the effectiveness of the proposed method.
Collapse
|
34
|
Han CH, Kim YW, Kim DY, Kim SH, Nenadic Z, Im CH. Electroencephalography-based endogenous brain-computer interface for online communication with a completely locked-in patient. J Neuroeng Rehabil 2019; 16:18. [PMID: 30700310 PMCID: PMC6354345 DOI: 10.1186/s12984-019-0493-0] [Citation(s) in RCA: 35] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2018] [Accepted: 01/23/2019] [Indexed: 01/29/2023] Open
Abstract
Background Brain–computer interfaces (BCIs) have demonstrated the potential to provide paralyzed individuals with new means of communication, but an electroencephalography (EEG)-based endogenous BCI has never been successfully used for communication with a patient in a completely locked-in state (CLIS). Methods In this study, we investigated the possibility of using an EEG-based endogenous BCI paradigm for online binary communication by a patient in CLIS. A female patient in CLIS participated in this study. She had not communicated even with her family for more than one year with complete loss of motor function. Offline and online experiments were conducted to validate the feasibility of the proposed BCI system. In the offline experiment, we determined the best combination of mental tasks and the optimal classification strategy leading to the best performance. In the online experiment, we investigated whether our BCI system could be potentially used for real-time communication with the patient. Results An online classification accuracy of 87.5% was achieved when Riemannian geometry-based classification was applied to real-time EEG data recorded while the patient was performing one of two mental-imagery tasks for 5 s. Conclusions Our results suggest that an EEG-based endogenous BCI has the potential to be used for online communication with a patient in CLIS.
Collapse
Affiliation(s)
- Chang-Hee Han
- Department of Biomedical Engineering, Hanyang University, Seoul, 04763, South Korea
| | - Yong-Wook Kim
- Department of Biomedical Engineering, Hanyang University, Seoul, 04763, South Korea
| | - Do Yeon Kim
- Department of Biomedical Engineering, Hanyang University, Seoul, 04763, South Korea
| | - Seung Hyun Kim
- Department of Neurology, College of Medicine, Hanyang University, Seoul, 04763, South Korea
| | - Zoran Nenadic
- Department of Biomedical Engineering, University of California, Irvine, CA, 92697, USA
| | - Chang-Hwan Im
- Department of Biomedical Engineering, Hanyang University, Seoul, 04763, South Korea.
| |
Collapse
|
35
|
Abstract
The authors define a class of functions on Riemannian manifolds, which are called geodesic semilocal E-preinvex functions, as a generalization of geodesic semilocal E-convex and geodesic semi E-preinvex functions, and some of its properties are established. Furthermore, a nonlinear fractional multiobjective programming is considered, where the functions involved are geodesic E-η-semidifferentiability, sufficient optimality conditions are obtained. A dual is formulated and duality results are proved by using concepts of geodesic semilocal E-preinvex functions, geodesic pseudo-semilocal E-preinvex functions, and geodesic quasi-semilocal E-preinvex functions.
Collapse
Affiliation(s)
- Adem Kılıçman
- Department of Mathematics and Institute for Mathematical Research, University Putra Malaysia, Serdang, Malaysia
| | - Wedad Saleh
- Department of Mathematics, Taibah University, Al-Medina, Saudi Arabia
| |
Collapse
|
36
|
Neilson PD, Neilson MD, Bye RT. A Riemannian Geometry Theory of Three-Dimensional Binocular Visual Perception. Vision (Basel) 2018; 2:E43. [PMID: 31735906 PMCID: PMC6836267 DOI: 10.3390/vision2040043] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2018] [Revised: 11/02/2018] [Accepted: 11/27/2018] [Indexed: 11/17/2022] Open
Abstract
We present a Riemannian geometry theory to examine the systematically warped geometry of perceived visual space attributable to the size-distance relationship of retinal images associated with the optics of the human eye. Starting with the notion of a vector field of retinal image features over cortical hypercolumns endowed with a metric compatible with that size-distance relationship, we use Riemannian geometry to construct a place-encoded theory of spatial representation within the human visual system. The theory draws on the concepts of geodesic spray fields, covariant derivatives, geodesics, Christoffel symbols, curvature tensors, vector bundles and fibre bundles to produce a neurally-feasible geometric theory of visuospatial memory. The characteristics of perceived 3D visual space are examined by means of a series of simulations around the egocentre. Perceptions of size and shape are elucidated by the geometry as are the removal of occlusions and the generation of 3D images of objects. Predictions of the theory are compared with experimental observations in the literature. We hold that the variety of reported geometries is accounted for by cognitive perturbations of the invariant physically-determined geometry derived here. When combined with previous description of the Riemannian geometry of human movement this work promises to account for the non-linear dynamical invertible visual-proprioceptive maps and selection of task-compatible movement synergies required for the planning and execution of visuomotor tasks.
Collapse
Affiliation(s)
- Peter D. Neilson
- School of Electrical Engineering and Telecommunications, University of New South Wales, Sydney NSW 2052, Australia
| | - Megan D. Neilson
- Independent Researcher, late School of Electrical Engineering and Telecommunications, University of New South Wales, Sydney NSW 2052, Australia
| | - Robin T. Bye
- Cyber-Physical Systems Laboratory, Department of ICT and Natural Sciences, NTNU—Norwegian University of Science and Technology, NO-6025 Ålesund, Norway
| |
Collapse
|
37
|
Bakker C, Halappanavar M, Visweswara Sathanur A. Dynamic graphs, community detection, and Riemannian geometry. Appl Netw Sci 2018; 3:3. [PMID: 30839776 PMCID: PMC6214282 DOI: 10.1007/s41109-018-0059-2] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/31/2017] [Accepted: 02/26/2018] [Indexed: 06/09/2023]
Abstract
A community is a subset of a wider network where the members of that subset are more strongly connected to each other than they are to the rest of the network. In this paper, we consider the problem of identifying and tracking communities in graphs that change over time - dynamic community detection - and present a framework based on Riemannian geometry to aid in this task. Our framework currently supports several important operations such as interpolating between and averaging over graph snapshots. We compare these Riemannian methods with entry-wise linear interpolation and find that the Riemannian methods are generally better suited to dynamic community detection. Next steps with the Riemannian framework include producing a Riemannian least-squares regression method for working with noisy data and developing support methods, such as spectral sparsification, to improve the scalability of our current methods.
Collapse
Affiliation(s)
- Craig Bakker
- Pacific Northwest National Laboratory, 902 Battelle Boulevard, Richland, 99352 WA United States
| | - Mahantesh Halappanavar
- Pacific Northwest National Laboratory, 902 Battelle Boulevard, Richland, 99352 WA United States
| | | |
Collapse
|
38
|
Rexiti M, Felice D, Mancini S. The Volume of Two-Qubit States by Information Geometry. Entropy (Basel) 2018; 20:E146. [PMID: 33265237 DOI: 10.3390/e20020146] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/22/2017] [Revised: 02/20/2018] [Accepted: 02/22/2018] [Indexed: 11/17/2022]
Abstract
Using the information geometry approach, we determine the volume of the set of two-qubit states with maximally disordered subsystems. Particular attention is devoted to the behavior of the volume of sub-manifolds of separable and entangled states with fixed purity. We show that the usage of the classical Fisher metric on phase space probability representation of quantum states gives the same qualitative results with respect to different versions of the quantum Fisher metric.
Collapse
|
39
|
Holbrook A, Lan S, Vandenberg-Rodes A, Shahbaba B. Geodesic Lagrangian Monte Carlo over the space of positive definite matrices: with application to Bayesian spectral density estimation. J STAT COMPUT SIM 2017; 88:982-1002. [PMID: 31105358 PMCID: PMC6521973 DOI: 10.1080/00949655.2017.1416470] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Abstract
We present geodesic Lagrangian Monte Carlo, an extension of Hamiltonian Monte Carlo for sampling from posterior distributions defined on general Riemannian manifolds. We apply this new algorithm to Bayesian inference on symmetric or Hermitian positive definite matrices. To do so, we exploit the Riemannian structure induced by Cartan's canonical metric. The geodesics that correspond to this metric are available in closed-form and-within the context of Lagrangian Monte Carlo-provide a principled way to travel around the space of positive definite matrices. Our method improves Bayesian inference on such matrices by allowing for a broad range of priors, so we are not limited to conjugate priors only. In the context of spectral density estimation, we use the (non-conjugate) complex reference prior as an example modeling option made available by the algorithm. Results based on simulated and real-world multivariate time series are presented in this context, and future directions are outlined.
Collapse
Affiliation(s)
- Andrew Holbrook
- University of California, Irvine, Department of Statistics, Irvine, CA, USA
| | - Shiwei Lan
- California Institute of Technology, Department of Computing and Mathematical Sciences, Pasadena, CA, USA
| | | | - Babak Shahbaba
- University of California, Irvine, Department of Statistics, Irvine, CA, USA
| |
Collapse
|
40
|
Ng B, Varoquaux G, Poline JB, Thirion B, Greicius MD, Poston KL. Distinct alterations in Parkinson's medication-state and disease-state connectivity. Neuroimage Clin 2017; 16:575-585. [PMID: 28971008 PMCID: PMC5608603 DOI: 10.1016/j.nicl.2017.09.004] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/27/2017] [Revised: 08/30/2017] [Accepted: 09/05/2017] [Indexed: 01/05/2023]
Abstract
Altered brain connectivity has been described in people with Parkinson's disease and in response to dopaminergic medications. However, it is unclear whether dopaminergic medications primarily 'normalize' disease related connectivity changes or if they induce unique alterations in brain connectivity. Further, it is unclear how these disease- and medication-associated changes in brain connectivity relate differently to specific motor manifestations of disease, such as bradykinesia/rigidity and tremor. In this study, we applied a novel covariance projection approach in combination with a bootstrapped permutation test to resting state functional MRI data from 57 Parkinson's disease and 20 healthy control participants to determine the Parkinson's medication-state and disease-state connectivity changes associated with different motor manifestations of disease. First, we identified brain connections that best classified Parkinson's disease ON versus OFF dopamine and Parkinson's disease versus healthy controls, achieving 96.9 ± 5.9% and 72.7 ± 12.4% classification accuracy, respectively. Second, we investigated the connections that significantly contribute to the classifications. We found that the connections greater in Parkinson's disease OFF compared to ON dopamine are primarily between motor (cerebellum and putamen) and posterior cortical regions, such as the posterior cingulate cortex. By contrast, connections that are greater in ON compared to OFF dopamine are between the right and left medial prefrontal cortex. We also identified the connections that are greater in healthy control compared to Parkinson's disease and found the most significant connections are associated with primary motor regions, such as the striatum and the supplementary motor area. Notably, these are different connections than those identified in Parkinson's disease OFF compared to ON. Third, we determined which of the Parkinson's medication-state and disease-state connections are associated with the severity of different motor symptoms. We found two connections correlate with both bradykinesia/rigidity severity and tremor severity, whereas four connections correlate with only bradykinesia/rigidity severity, and five connections correlate with only tremor severity. Connections that correlate with only tremor severity are anchored by the cerebellum and the supplemental motor area, but only those connections that include the supplemental motor area predict dopaminergic improvement in tremor. Our results suggest that dopaminergic medications do not simply 'normalize' abnormal brain connectivity associated with Parkinson's disease, but rather dopamine drives distinct connectivity changes, only some of which are associated with improved motor symptoms. In addition, the dissociation between of connections related to severity of bradykinesia/rigidity versus tremor highlights the distinct abnormalities in brain circuitry underlying these specific motor symptoms.
Collapse
Affiliation(s)
- Bernard Ng
- Mostafavi Lab, Department of Statistics, University of British Columbia, Vancouver, BC, Canada.,Parietal team, INRIA Saclay, Gif-sur-Yvette, France.,Functional Imaging in Neuropsychiatric Disorders (FIND) Lab, Department of Neurology & Neurological Sciences, Stanford University School of Medicine, Stanford, CA, United States
| | | | | | | | - Michael D Greicius
- Functional Imaging in Neuropsychiatric Disorders (FIND) Lab, Department of Neurology & Neurological Sciences, Stanford University School of Medicine, Stanford, CA, United States
| | - Kathleen L Poston
- Department of Neurology & Neurological Sciences, Stanford University School of Medicine, Stanford, CA, United States.,Department of Neurosurgery, Stanford University School of Medicine, Stanford, CA, United States
| |
Collapse
|
41
|
Abstract
A new model for macroscopic root growth based on a dynamical Riemannian geometry is presented. Assuming that the thickness of the root is much less than its length, the model is restricted to growth in one dimension (1D). We treat 1D tissues as continuous, deformable, growing geometries for sizes larger than 1 mm. The dynamics of the growing root are described by a set of coupled tensor equations for the metric of the tissue and velocity field of material transport in non-Euclidean space. These coupled equations represent a novel feedback mechanism between growth and geometry. We compare 1D numerical simulations of these tissue growth equations to two measures of root growth. First, sectional growth along the simulated root shows an elongation zone common to many species of plant roots. Second, the relative elemental growth rate calculated in silico exhibits spatio-temporal dynamics recently characterized in high-resolution root growth studies but which thus far lack a biological hypothesis to explain them. In our model, these dynamics are a direct consequence of considering growth as both a geometric reaction-diffusion process and expansion due to a distributed source of new materials.
Collapse
|
42
|
Alimardani F, Boostani R, Blankertz B. Weighted spatial based geometric scheme as an efficient algorithm for analyzing single-trial EEGS to improve cue-based BCI classification. Neural Netw 2017; 92:69-76. [PMID: 28385624 DOI: 10.1016/j.neunet.2017.02.014] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2016] [Revised: 02/09/2017] [Accepted: 02/09/2017] [Indexed: 11/27/2022]
Abstract
There is a growing interest in analyzing the geometrical behavior of electroencephalogram (EEG) covariance matrix in the context of brain computer interface (BCI). The bottleneck of the current Riemannian framework is the bias of the mean vector of EEG signals to the noisy trials, which deteriorates the covariance matrix in the manifold space. This study presents a spatial weighting scheme to reduce the effect of noisy trials on the mean vector. To assess the proposed method, dataset IIa from BCI competition IV, containing the EEG trials of 9 subjects performing four mental tasks, was utilized. The performance of the proposed method is compared to the classical Riemannian method along with Common Spatial Pattern (CSP) on the dataset. The results show that when considering just two imagery classes, the proposed method performs on par with CSP method, whereas in the multi class scenario, the proposed algorithm outperforms the CSP approach on seven out of nine subjects. Incidentally, the proposed method obtains better accuracy for the majority of subjects compared to the classical Riemannian method.
Collapse
Affiliation(s)
- Fatemeh Alimardani
- Department of Computer Science and Engineering, Shiraz University, Shiraz, Iran; Institute for Advanced Studies in Basic Sciences, GavaZang, Zanjan, Iran.
| | - Reza Boostani
- Department of Computer Science and Engineering, Shiraz University, Shiraz, Iran.
| | | |
Collapse
|
43
|
Affiliation(s)
- Guang Cheng
- Department of Computer & Information Science & Engineering, University of Florida
| | | |
Collapse
|