1
|
Gallos IK, Lehmberg D, Dietrich F, Siettos C. Data-driven modelling of brain activity using neural networks, diffusion maps, and the Koopman operator. CHAOS (WOODBURY, N.Y.) 2024; 34:013151. [PMID: 38285718 DOI: 10.1063/5.0157881] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/12/2023] [Accepted: 12/22/2023] [Indexed: 01/31/2024]
Abstract
We propose a machine-learning approach to construct reduced-order models (ROMs) to predict the long-term out-of-sample dynamics of brain activity (and in general, high-dimensional time series), focusing mainly on task-dependent high-dimensional fMRI time series. Our approach is a three stage one. First, we exploit manifold learning and, in particular, diffusion maps (DMs) to discover a set of variables that parametrize the latent space on which the emergent high-dimensional fMRI time series evolve. Then, we construct ROMs on the embedded manifold via two techniques: Feedforward Neural Networks (FNNs) and the Koopman operator. Finally, for predicting the out-of-sample long-term dynamics of brain activity in the ambient fMRI space, we solve the pre-image problem, i.e., the construction of a map from the low-dimensional manifold to the original high-dimensional (ambient) space by coupling DMs with Geometric Harmonics (GH) when using FNNs and the Koopman modes per se. For our illustrations, we have assessed the performance of the two proposed schemes using two benchmark fMRI time series: (i) a simplistic five-dimensional model of stochastic discrete-time equations used just for a "transparent" illustration of the approach, thus knowing a priori what one expects to get, and (ii) a real fMRI dataset with recordings during a visuomotor task. We show that the proposed Koopman operator approach provides, for any practical purposes, equivalent results to the FNN-GH approach, thus bypassing the need to train a non-linear map and to use GH to extrapolate predictions in the ambient space; one can use instead the low-frequency truncation of the DMs function space of L2-integrable functions to predict the entire list of coordinate functions in the ambient space and to solve the pre-image problem.
Collapse
Affiliation(s)
- Ioannis K Gallos
- Institute of Communication and Computer Systems, National Technical University of Athens, Zografos Campus, 15780 Athens, Greece
| | - Daniel Lehmberg
- School of Computation, Information and Technology, Technical University of Munich, Munich 80333, Germany
| | - Felix Dietrich
- School of Computation, Information and Technology, Technical University of Munich, Munich 80333, Germany
| | - Constantinos Siettos
- Dipartimento di Matematica e Applicazioni "Renato Caccioppoli," Universitá degli Studi di Napoli Federico II, Naples 80125, Italy
| |
Collapse
|
2
|
Liu Y, Zhao R, Xiong X, Ren X. A Bibliometric Analysis of Consumer Neuroscience towards Sustainable Consumption. Behav Sci (Basel) 2023; 13:bs13040298. [PMID: 37102812 PMCID: PMC10136158 DOI: 10.3390/bs13040298] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2023] [Revised: 03/13/2023] [Accepted: 03/15/2023] [Indexed: 04/03/2023] Open
Abstract
Consumer neuroscience is a new paradigm for studying consumer behavior, focusing on neuroscientific tools to explore the underlying neural processes and behavioral implications of consumption. Based on the bibliometric analysis tools, this paper provides a review of progress in research on consumer neuroscience during 2000–2021. In this paper, we identify research hotspots and frontiers in the field through a statistical analysis of bibliometric indicators, including the number of publications, countries, institutions, and keywords. Aiming at facilitating carbon neutrality via sustainable consumption, this paper discusses the prospects of applying neuroscience to sustainable consumption. The results show 364 publications in the field during 2000–2021, showing a rapid upward trend, indicating that consumer neuroscience research is gaining ground. The majority of these consumer neuroscience studies chose to use electroencephalogram tools, accounting for 63.8% of the total publications; the cutting-edge research mainly involved event-related potential (ERP) studies of various marketing stimuli interventions, functional magnetic resonance imaging (fMRI)-based studies of consumer decision-making and emotion-specific brain regions, and machine-learning-based studies of consumer decision-making optimization models.
Collapse
|
3
|
Vijayan A, Diwakar S. A cerebellum inspired spiking neural network as a multi-model for pattern classification and robotic trajectory prediction. Front Neurosci 2022; 16:909146. [DOI: 10.3389/fnins.2022.909146] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Accepted: 11/02/2022] [Indexed: 11/29/2022] Open
Abstract
Spiking neural networks were introduced to understand spatiotemporal information processing in neurons and have found their application in pattern encoding, data discrimination, and classification. Bioinspired network architectures are considered for event-driven tasks, and scientists have looked at different theories based on the architecture and functioning. Motor tasks, for example, have networks inspired by cerebellar architecture where the granular layer recodes sparse representations of the mossy fiber (MF) inputs and has more roles in motor learning. Using abstractions from cerebellar connections and learning rules of deep learning network (DLN), patterns were discriminated within datasets, and the same algorithm was used for trajectory optimization. In the current work, a cerebellum-inspired spiking neural network with dynamics of cerebellar neurons and learning mechanisms attributed to the granular layer, Purkinje cell (PC) layer, and cerebellar nuclei interconnected by excitatory and inhibitory synapses was implemented. The model’s pattern discrimination capability was tested for two tasks on standard machine learning (ML) datasets and on following a trajectory of a low-cost sensor-free robotic articulator. Tuned for supervised learning, the pattern classification capability of the cerebellum-inspired network algorithm has produced more generalized models than data-specific precision models on smaller training datasets. The model showed an accuracy of 72%, which was comparable to standard ML algorithms, such as MLP (78%), Dl4jMlpClassifier (64%), RBFNetwork (71.4%), and libSVM-linear (85.7%). The cerebellar model increased the network’s capability and decreased storage, augmenting faster computations. Additionally, the network model could also implicitly reconstruct the trajectory of a 6-degree of freedom (DOF) robotic arm with a low error rate by reconstructing the kinematic parameters. The variability between the actual and predicted trajectory points was noted to be ± 3 cm (while moving to a position in a cuboid space of 25 × 30 × 40 cm). Although a few known learning rules were implemented among known types of plasticity in the cerebellum, the network model showed a generalized processing capability for a range of signals, modulating the data through the interconnected neural populations. In addition to potential use on sensor-free or feed-forward based controllers for robotic arms and as a generalized pattern classification algorithm, this model adds implications to motor learning theory.
Collapse
|
4
|
|
5
|
Design of MRI structured spiking neural networks and learning algorithms for personalized modelling, analysis, and prediction of EEG signals. Sci Rep 2021; 11:12064. [PMID: 34103545 PMCID: PMC8187669 DOI: 10.1038/s41598-021-90029-5] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2020] [Accepted: 04/09/2021] [Indexed: 12/04/2022] Open
Abstract
This paper proposes a novel method and algorithms for the design of MRI structured personalized 3D spiking neural network models (MRI-SNN) for a better analysis, modeling, and prediction of EEG signals. It proposes a novel gradient-descent learning algorithm integrated with a spike-time-dependent-plasticity algorithm. The models capture informative personal patterns of interaction between EEG channels, contrary to single EEG signal modeling methods or to spike-based approaches which do not use personal MRI data to pre-structure a model. The proposed models can not only learn and model accurately measured EEG data, but they can also predict signals at 3D model locations that correspond to non-monitored brain areas, e.g. other EEG channels, from where data has not been collected. This is the first study in this respect. As an illustration of the method, personalized MRI-SNN models are created and tested on EEG data from two subjects. The models result in better prediction accuracy and a better understanding of the personalized EEG signals than traditional methods due to the MRI and EEG information integration. The models are interpretable and facilitate a better understanding of related brain processes. This approach can be applied for personalized modeling, analysis, and prediction of EEG signals across brain studies such as the study and prediction of epilepsy, peri-perceptual brain activities, brain-computer interfaces, and others.
Collapse
|
6
|
Durai M, Doborjeh Z, Sanders PJ, Vajsakovic D, Wendt A, Searchfield GD. Behavioral Outcomes and Neural Network Modeling of a Novel, Putative, Recategorization Sound Therapy. Brain Sci 2021; 11:554. [PMID: 33925762 PMCID: PMC8146945 DOI: 10.3390/brainsci11050554] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Revised: 04/23/2021] [Accepted: 04/24/2021] [Indexed: 02/07/2023] Open
Abstract
The mechanisms underlying sound's effect on tinnitus perception are unclear. Tinnitus activity appears to conflict with perceptual expectations of "real" sound, resulting in it being a salient signal. Attention diverted towards tinnitus during the later stages of object processing potentially disrupts high-order auditory streaming, and its uncertain nature results in negative psychological responses. This study investigated the benefits and neurophysiological basis of passive perceptual training and informational counseling to recategorize phantom perception as a more real auditory object. Specifically, it examined underlying psychoacoustic correlates of tinnitus and the neural activities associated with tinnitus auditory streaming and how malleable these are to change with targeted intervention. Eighteen participants (8 females, 10 males, mean age = 61.6 years) completed the study. The study consisted of 2 parts: (1) An acute exposure over 30 min to a sound that matched the person's tinnitus (Tinnitus Avatar) that was cross-faded to a selected nature sound (Cicadas, Fan, Water Sound/Rain, Birds, Water and Bird). (2) A chronic exposure for 3 months to the same "morphed" sound. A brain-inspired spiking neural network (SNN) architecture was used to model and compare differences between electroencephalography (EEG) patterns recorded prior to morphing sound presentation, during, after (3-month), and post-follow-up. Results showed that the tinnitus avatar generated was a good match to an individual's tinnitus as rated on likeness scales and was not rated as unpleasant. The five environmental sounds selected for this study were also rated as being appropriate matches to individuals' tinnitus and largely pleasant to listen to. There was a significant reduction in the Tinnitus Functional Index score and subscales of intrusiveness of the tinnitus signal and ability to concentrate with the tinnitus trial end compared to baseline. There was a significant decrease in how strong the tinnitus signal was rated as well as ratings of how easy it was to ignore the tinnitus signal on severity rating scales. Qualitative analysis found that the environmental sound interacted with the tinnitus in a positive way, but participants did not experience change in severity, however, characteristics of tinnitus, including pitch and uniformity of sound, were reported to change. The results indicate the feasibility of the computational SNN method and preliminary evidence that the sound exposure may change activation of neural tinnitus networks and greater bilateral hemispheric involvement as the sound morphs over time into natural environmental sound; particularly relating to attention and discriminatory judgments (dorsal attention network, precentral gyrus, ventral anterior network). This is the first study that attempts to recategorize tinnitus using passive auditory training to a sound that morphs from resembling the person's tinnitus to a natural sound. These findings will be used to design future-controlled trials to elucidate whether the approach used differs in effect and mechanism from conventional Broadband Noise (BBN) sound therapy.
Collapse
Affiliation(s)
- Mithila Durai
- Section of Audiology, School of Population Health, The University of Auckland, Auckland 1023, New Zealand; (M.D.); (Z.D.); (P.J.S.); (D.V.)
- Eisdell Moore Centre, Auckland 1023, New Zealand
- Centre for Brain Research, The University of Auckland, Auckland 1023, New Zealand
| | - Zohreh Doborjeh
- Section of Audiology, School of Population Health, The University of Auckland, Auckland 1023, New Zealand; (M.D.); (Z.D.); (P.J.S.); (D.V.)
- Eisdell Moore Centre, Auckland 1023, New Zealand
- Centre for Brain Research, The University of Auckland, Auckland 1023, New Zealand
| | - Philip J. Sanders
- Section of Audiology, School of Population Health, The University of Auckland, Auckland 1023, New Zealand; (M.D.); (Z.D.); (P.J.S.); (D.V.)
- Eisdell Moore Centre, Auckland 1023, New Zealand
- Centre for Brain Research, The University of Auckland, Auckland 1023, New Zealand
| | - Dunja Vajsakovic
- Section of Audiology, School of Population Health, The University of Auckland, Auckland 1023, New Zealand; (M.D.); (Z.D.); (P.J.S.); (D.V.)
- Eisdell Moore Centre, Auckland 1023, New Zealand
- Centre for Brain Research, The University of Auckland, Auckland 1023, New Zealand
| | - Anne Wendt
- Knowledge Engineering & Discovery Research Institute, Auckland University of Technology, Auckland 1010, New Zealand;
| | - Grant D. Searchfield
- Section of Audiology, School of Population Health, The University of Auckland, Auckland 1023, New Zealand; (M.D.); (Z.D.); (P.J.S.); (D.V.)
- Eisdell Moore Centre, Auckland 1023, New Zealand
- Centre for Brain Research, The University of Auckland, Auckland 1023, New Zealand
- Brain Research New Zealand—Rangahau Roro Aotearoa, The University of Auckland, Auckland 1142, New Zealand
| |
Collapse
|
7
|
Gong S, Xing K, Cichocki A, Li J. Deep Learning in EEG: Advance of the Last Ten-Year Critical Period. IEEE Trans Cogn Dev Syst 2021. [DOI: 10.1109/tcds.2021.3079712] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
8
|
Bazzani A, Ravaioli S, Trieste L, Faraguna U, Turchetti G. Is EEG Suitable for Marketing Research? A Systematic Review. Front Neurosci 2020; 14:594566. [PMID: 33408608 PMCID: PMC7779633 DOI: 10.3389/fnins.2020.594566] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2020] [Accepted: 11/26/2020] [Indexed: 11/30/2022] Open
Abstract
Background: In the past decade, marketing studies have greatly benefited from the adoption of neuroscience techniques to explore conscious and unconscious drivers of consumer behavior. Electroencephalography (EEG) is one of the most frequently applied neuroscientific techniques for marketing studies, thanks to its low cost and high temporal resolution. Objective: We present an overview of EEG applications in consumer neuroscience. The aim of this review is to facilitate future research and to highlight reliable approaches for deriving research and managerial implications. Method: We conducted a systematic review by querying five databases for the titles of articles published up to June 2020 with the terms [EEG] AND [neuromarketing] OR [consumer neuroscience]. Results: We screened 264 abstracts and analyzed 113 articles, classified based on research topics (e.g., product characteristics, pricing, advertising attention and memorization, rational, and emotional messages) and characteristics of the experimental design (tasks, stimuli, participants, additional techniques). Conclusions: This review highlights the main applications of EEG to consumer neuroscience research and suggests several ways EEG technique can complement traditional experimental paradigms. Further research areas, including consumer profiling and social consumer neuroscience, have not been sufficiently explored yet and would benefit from EEG techniques to address unanswered questions.
Collapse
Affiliation(s)
- Andrea Bazzani
- Institute of Management, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Silvio Ravaioli
- Department of Economics, Columbia University, New York, NY, United States
| | - Leopoldo Trieste
- Institute of Management, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Ugo Faraguna
- Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, Pisa, Italy
- Department of Developmental Neuroscience, Istituto di Ricovero e Cura a Carattere Scientifico (IRCCS) Fondazione Stella Maris, Pisa, Italy
| | | |
Collapse
|
9
|
Doborjeh Z, Doborjeh M, Crook-Rumsey M, Taylor T, Wang GY, Moreau D, Krägeloh C, Wrapson W, Siegert RJ, Kasabov N, Searchfield G, Sumich A. Interpretability of Spatiotemporal Dynamics of the Brain Processes Followed by Mindfulness Intervention in a Brain-Inspired Spiking Neural Network Architecture. SENSORS (BASEL, SWITZERLAND) 2020; 20:E7354. [PMID: 33371459 PMCID: PMC7767448 DOI: 10.3390/s20247354] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/24/2020] [Revised: 12/16/2020] [Accepted: 12/17/2020] [Indexed: 01/05/2023]
Abstract
Mindfulness training is associated with improvements in psychological wellbeing and cognition, yet the specific underlying neurophysiological mechanisms underpinning these changes are uncertain. This study uses a novel brain-inspired artificial neural network to investigate the effect of mindfulness training on electroencephalographic function. Participants completed a 4-tone auditory oddball task (that included targets and physically similar distractors) at three assessment time points. In Group A (n = 10), these tasks were given immediately prior to 6-week mindfulness training, immediately after training and at a 3-week follow-up; in Group B (n = 10), these were during an intervention waitlist period (3 weeks prior to training), pre-mindfulness training and post-mindfulness training. Using a spiking neural network (SNN) model, we evaluated concurrent neural patterns generated across space and time from features of electroencephalographic data capturing the neural dynamics associated with the event-related potential (ERP). This technique capitalises on the temporal dynamics of the shifts in polarity throughout the ERP and spatially across electrodes. Findings support anteriorisation of connection weights in response to distractors relative to target stimuli. Right frontal connection weights to distractors were associated with trait mindfulness (positively) and depression (inversely). Moreover, mindfulness training was associated with an increase in connection weights to targets (bilateral frontal, left frontocentral, and temporal regions only) and distractors. SNN models were superior to other machine learning methods in the classification of brain states as a function of mindfulness training. Findings suggest SNN models can provide useful information that differentiates brain states based on distinct task demands and stimuli, as well as changes in brain states as a function of psychological intervention.
Collapse
Affiliation(s)
- Zohreh Doborjeh
- Faculty of Medical and Health Sciences, School of Population Health, Section of Audiology, The University of Auckland, Auckland 1142, New Zealand;
- Eisdell Moore Centre, The University of Auckland, Auckland 1142, New Zealand
- Centre for Brain Research, The University of Auckland, Auckland 1142, New Zealand;
| | - Maryam Doborjeh
- Information Technology and Software Engineering Department, Auckland University of Technology, Auckland 1010, New Zealand;
| | - Mark Crook-Rumsey
- School of Psychology, Nottingham Trent University, Nottingham NG25 0QF, UK; (M.C.-R.); (A.S.)
| | - Tamasin Taylor
- Faculty of Medical and Health Sciences, The University of Auckland, Auckland 1142, New Zealand;
| | - Grace Y. Wang
- Department of Psychology and Neuroscience, Auckland University of Technology, Auckland 0627, New Zealand; (G.Y.W.); (C.K.); (R.J.S.)
| | - David Moreau
- Centre for Brain Research, The University of Auckland, Auckland 1142, New Zealand;
- School of Psychology, The University of Auckland, Auckland 1142, New Zealand
| | - Christian Krägeloh
- Department of Psychology and Neuroscience, Auckland University of Technology, Auckland 0627, New Zealand; (G.Y.W.); (C.K.); (R.J.S.)
| | - Wendy Wrapson
- School of Public Health and Interdisciplinary Studies, Auckland University of Technology, Auckland 0627, New Zealand;
| | - Richard J. Siegert
- Department of Psychology and Neuroscience, Auckland University of Technology, Auckland 0627, New Zealand; (G.Y.W.); (C.K.); (R.J.S.)
| | - Nikola Kasabov
- Intelligent Systems Research Centre, Ulster University, Londonderry BT48 7JL, UK
- School of Engineering, Computing and Mathematical Sciences, Auckland University of Technology, Auckland 1010, New Zealand
| | - Grant Searchfield
- Faculty of Medical and Health Sciences, School of Population Health, Section of Audiology, The University of Auckland, Auckland 1142, New Zealand;
- Eisdell Moore Centre, The University of Auckland, Auckland 1142, New Zealand
- Centre for Brain Research, The University of Auckland, Auckland 1142, New Zealand;
| | - Alexander Sumich
- School of Psychology, Nottingham Trent University, Nottingham NG25 0QF, UK; (M.C.-R.); (A.S.)
| |
Collapse
|
10
|
Kelsen BA, Sumich A, Kasabov N, Liang SHY, Wang GY. What has social neuroscience learned from hyperscanning studies of spoken communication? A systematic review. Neurosci Biobehav Rev 2020; 132:1249-1262. [PMID: 33022298 DOI: 10.1016/j.neubiorev.2020.09.008] [Citation(s) in RCA: 39] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2020] [Revised: 09/03/2020] [Accepted: 09/08/2020] [Indexed: 12/12/2022]
Abstract
A growing body of literature examining the neurocognitive processes of interpersonal linguistic interaction indicates the emergence of neural alignment as participants engage in oral communication. However, questions have arisen whether the study results can be interpreted beyond observations of cortical functionality and extended to the mutual understanding between communicators. This review presents evidence from electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) hyperscanning studies of interbrain synchrony (IBS) in which participants communicated via spoken language. The studies are classified into: knowledge sharing; turn-taking speech co-ordination; cooperation, problem-solving and creativity; and naturalistic discussion paradigms according to the type of interaction specified in each study. Alignment predominantly occurred in the frontal and temporo-parietal areas, which may reflect activation of the mirror and mentalizing systems. We argue that the literature presents a significant contribution to advancing our understanding of IBS and mutual understanding between communicators. We end with suggestions for future research, including analytical approaches and experimental conditions and hypothesize that brain-inspired neural networks are promising techniques for better understanding of IBS through hyperscanning.
Collapse
Affiliation(s)
- Brent A Kelsen
- Department of Psychology and Neuroscience, Auckland University of Technology, Auckland, New Zealand; Language Center, National Taipei University, New Taipei City, Taiwan
| | - Alexander Sumich
- Division of Psychology, Nottingham Trent University, Nottingham, United Kingdom
| | - Nikola Kasabov
- Knowledge Engineering and Discovery Research Institute (KEDRI), Auckland University of Technology, Auckland, New Zealand
| | - Sophie H Y Liang
- Department of Medicine, Chang Gung University, Taoyuan, Taiwan; Department of Child & Adolescent Psychiatry, Chang Gung Memorial Hospital at Taoyuan, Taoyuan, Taiwan
| | - Grace Y Wang
- Department of Psychology and Neuroscience, Auckland University of Technology, Auckland, New Zealand.
| |
Collapse
|
11
|
Tan C, Šarlija M, Kasabov N. Spiking Neural Networks: Background, Recent Development and the NeuCube Architecture. Neural Process Lett 2020. [DOI: 10.1007/s11063-020-10322-8] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
12
|
A Concealed Information Test System Based on Functional Brain Connectivity and Signal Entropy of Audio–Visual ERP. IEEE Trans Cogn Dev Syst 2020. [DOI: 10.1109/tcds.2020.2991359] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
13
|
Rim B, Sung NJ, Min S, Hong M. Deep Learning in Physiological Signal Data: A Survey. SENSORS (BASEL, SWITZERLAND) 2020; 20:E969. [PMID: 32054042 PMCID: PMC7071412 DOI: 10.3390/s20040969] [Citation(s) in RCA: 66] [Impact Index Per Article: 16.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/07/2020] [Revised: 01/31/2020] [Accepted: 02/09/2020] [Indexed: 12/11/2022]
Abstract
Deep Learning (DL), a successful promising approach for discriminative and generative tasks, has recently proved its high potential in 2D medical imaging analysis; however, physiological data in the form of 1D signals have yet to be beneficially exploited from this novel approach to fulfil the desired medical tasks. Therefore, in this paper we survey the latest scientific research on deep learning in physiological signal data such as electromyogram (EMG), electrocardiogram (ECG), electroencephalogram (EEG), and electrooculogram (EOG). We found 147 papers published between January 2018 and October 2019 inclusive from various journals and publishers. The objective of this paper is to conduct a detailed study to comprehend, categorize, and compare the key parameters of the deep-learning approaches that have been used in physiological signal analysis for various medical applications. The key parameters of deep-learning approach that we review are the input data type, deep-learning task, deep-learning model, training architecture, and dataset sources. Those are the main key parameters that affect system performance. We taxonomize the research works using deep-learning method in physiological signal analysis based on: (1) physiological signal data perspective, such as data modality and medical application; and (2) deep-learning concept perspective such as training architecture and dataset sources.
Collapse
Affiliation(s)
- Beanbonyka Rim
- Department of Computer Science, Soonchunhyang University, Asan 31538, Korea
| | - Nak-Jun Sung
- Department of Computer Science, Soonchunhyang University, Asan 31538, Korea
| | - Sedong Min
- Department of Medical IT Engineering, Soonchunhyang University, Asan 31538, Korea
| | - Min Hong
- Department of Computer Software Engineering, Soonchunhyang University, Asan 31538, Korea
| |
Collapse
|
14
|
Kumarasinghe K, Kasabov N, Taylor D. Deep learning and deep knowledge representation in Spiking Neural Networks for Brain-Computer Interfaces. Neural Netw 2019; 121:169-185. [PMID: 31568895 DOI: 10.1016/j.neunet.2019.08.029] [Citation(s) in RCA: 22] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2019] [Revised: 08/26/2019] [Accepted: 08/26/2019] [Indexed: 01/21/2023]
Abstract
OBJECTIVE This paper argues that Brain-Inspired Spiking Neural Network (BI-SNN) architectures can learn and reveal deep in time-space functional and structural patterns from spatio-temporal data. These patterns can be represented as deep knowledge, in a partial case in the form of deep spatio-temporal rules. This is a promising direction for building new types of Brain-Computer Interfaces called Brain-Inspired Brain-Computer Interfaces (BI-BCI). A theoretical framework and its experimental validation on deep knowledge extraction and representation using SNN are presented. RESULTS The proposed methodology was applied in a case study to extract deep knowledge of the functional and structural organisation of the brain's neural network during the execution of a Grasp and Lift task. The BI-BCI successfully extracted the neural trajectories that represent the dorsal and ventral visual information processing streams as well as its connection to the motor cortex in the brain. Deep spatiotemporal rules on functional and structural interaction of distinct brain areas were then used for event prediction in BI-BCI. SIGNIFICANCE The computational framework can be used for unveiling the topological patterns of the brain and such knowledge can be effectively used to enhance the state-of-the-art in BCI.
Collapse
Affiliation(s)
- Kaushalya Kumarasinghe
- Knowledge Engineering and Discovery Research Institute, Auckland University of Technology, Auckland, New Zealand; Health and Rehabilitation Research Institute, Auckland University of Technology, Auckland, New Zealand.
| | - Nikola Kasabov
- Knowledge Engineering and Discovery Research Institute, Auckland University of Technology, Auckland, New Zealand.
| | - Denise Taylor
- Health and Rehabilitation Research Institute, Auckland University of Technology, Auckland, New Zealand.
| |
Collapse
|
15
|
Behrenbeck J, Tayeb Z, Bhiri C, Richter C, Rhodes O, Kasabov N, Espinosa-Ramos JI, Furber S, Cheng G, Conradt J. Classification and regression of spatio-temporal signals using NeuCube and its realization on SpiNNaker neuromorphic hardware. J Neural Eng 2018; 16:026014. [PMID: 30577030 DOI: 10.1088/1741-2552/aafabc] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
OBJECTIVE The objective of this work is to use the capability of spiking neural networks to capture the spatio-temporal information encoded in time-series signals and decode them without the use of hand-crafted features and vector-based learning and the realization of the spiking model on low-power neuromorphic hardware. APPROACH The NeuCube spiking model was used to classify different grasp movements directly from raw surface electromyography signals (sEMG), the estimations of the applied finger forces as well as the classification of two motor imagery movements from raw electroencephalography (EEG). In a parallel investigation, the designed spiking decoder was implemented on SpiNNaker neuromorphic hardware, which allows low-energy real-time processing. MAIN RESULTS Experimental results reveal a better classification accuracy using the NeuCube model compared to traditional machine learning methods. For sEMG classification, we reached a training accuracy of 85% and a test accuracy of 84.8%, as well as less than 19% of relative root mean square error (rRMSE) when estimating finger forces from six subjects. For the EEG classification, a mean accuracy of 75% was obtained when tested on raw EEG data from nine subjects from the existing 2b dataset from 'BCI competition IV'. SIGNIFICANCE This work provides a proof of concept for a successful implementation of the NeuCube spiking model on the SpiNNaker neuromorphic platform for raw sEMG and EEG decoding, which could chart a route ahead for a new generation of portable closed-loop and low-power neuroprostheses.
Collapse
Affiliation(s)
- Jan Behrenbeck
- Department of Mechanical Engineering, Technical University of Munich, Munich, Germany
| | | | | | | | | | | | | | | | | | | |
Collapse
|
16
|
Tavanaei A, Ghodrati M, Kheradpisheh SR, Masquelier T, Maida A. Deep learning in spiking neural networks. Neural Netw 2018; 111:47-63. [PMID: 30682710 DOI: 10.1016/j.neunet.2018.12.002] [Citation(s) in RCA: 205] [Impact Index Per Article: 34.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2018] [Revised: 12/02/2018] [Accepted: 12/03/2018] [Indexed: 12/14/2022]
Abstract
In recent years, deep learning has revolutionized the field of machine learning, for computer vision in particular. In this approach, a deep (multilayer) artificial neural network (ANN) is trained, most often in a supervised manner using backpropagation. Vast amounts of labeled training examples are required, but the resulting classification accuracy is truly impressive, sometimes outperforming humans. Neurons in an ANN are characterized by a single, static, continuous-valued activation. Yet biological neurons use discrete spikes to compute and transmit information, and the spike times, in addition to the spike rates, matter. Spiking neural networks (SNNs) are thus more biologically realistic than ANNs, and are arguably the only viable option if one wants to understand how the brain computes at the neuronal description level. The spikes of biological neurons are sparse in time and space, and event-driven. Combined with bio-plausible local learning rules, this makes it easier to build low-power, neuromorphic hardware for SNNs. However, training deep SNNs remains a challenge. Spiking neurons' transfer function is usually non-differentiable, which prevents using backpropagation. Here we review recent supervised and unsupervised methods to train deep SNNs, and compare them in terms of accuracy and computational cost. The emerging picture is that SNNs still lag behind ANNs in terms of accuracy, but the gap is decreasing, and can even vanish on some tasks, while SNNs typically require many fewer operations and are the better candidates to process spatio-temporal data.
Collapse
Affiliation(s)
- Amirhossein Tavanaei
- School of Computing and Informatics, University of Louisiana at Lafayette, Lafayette, LA 70504, USA.
| | - Masoud Ghodrati
- Department of Physiology, Monash University, Clayton, VIC, Australia
| | - Saeed Reza Kheradpisheh
- Department of Computer Science, Faculty of Mathematical Sciences and Computer, Kharazmi University, Tehran, Iran
| | | | - Anthony Maida
- School of Computing and Informatics, University of Louisiana at Lafayette, Lafayette, LA 70504, USA
| |
Collapse
|
17
|
Wongupparaj P, Sumich A, Wickens M, Kumari V, Morris RG. Individual differences in working memory and general intelligence indexed by P200 and P300: A latent variable model. Biol Psychol 2018; 139:96-105. [PMID: 30392828 DOI: 10.1016/j.biopsycho.2018.10.009] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2017] [Revised: 10/12/2018] [Accepted: 10/16/2018] [Indexed: 02/02/2023]
Abstract
A robust relationship between working memory (WM) and general intelligence (g) has been well established. Nevertheless, explanations for this relationship in terms of underlying neurocognitive processes are still inadequate. This study addresses this issue using an individual differences approach in which Central Executive System (CES) and Short-Term Storage (STS) components of WM are measured comprehensively and examined for their relationship with g via event-related potentials components (P200 and P300) as mediators. Participants (n = 115) completed tests of the WM, CES and STS, as well as g. P200 and P300 components were recorded during 3-back WM task performance. Structural equation modelling showed significant negative associations between the P200 latency for target stimuli and CES shifting processes, and between the P300 amplitude for target stimuli and CES inhibition and updating processes. The relationship between CES processes and g was mediated in a localized fashion by the P300 amplitude. These findings further support the notion that the CES has a multidimensional structure and, importantly, reveal that the inhibition and updating functions of the CES are crucial in explaining the relationship between WM and g. Negative relations between ERP indices (P200 latency and P300 amplitude for target stimuli) and g support a neural efficiency hypothesis related to high intelligence.
Collapse
Affiliation(s)
- Peera Wongupparaj
- Department of Psychology, Institute of Psychiatry, Psychology and Neuroscience, King's College London, UK; Cognitive Science and Innovation Research Unit (CSIRU), College of Research Methodology and Cognitive Science, Burapha University, Thailand. Peera.wongupparaj.1.@kcl.ac.uk
| | - Alexander Sumich
- Department of Psychology, Institute of Psychiatry, Psychology and Neuroscience, King's College London, UK; Division of Psychology, School of Social Sciences, Nottingham Trent University, Nottingham, UK.
| | - Megan Wickens
- Department of Psychological Science, Albion College, Michigan, USA.
| | - Veena Kumari
- Department of Psychology, Institute of Psychiatry, Psychology and Neuroscience, King's College London, UK; Centre for Cognitive Neuroscience, College of Health and Life Sciences, Brunel University London, Uxbridge, UK.
| | - Robin G Morris
- Department of Psychology, Institute of Psychiatry, Psychology and Neuroscience, King's College London, UK.
| |
Collapse
|