1
|
Pham MD, D’Angiulli A, Dehnavi MM, Chhabra R. From Brain Models to Robotic Embodied Cognition: How Does Biological Plausibility Inform Neuromorphic Systems? Brain Sci 2023; 13:1316. [PMID: 37759917 PMCID: PMC10526461 DOI: 10.3390/brainsci13091316] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2023] [Revised: 09/05/2023] [Accepted: 09/07/2023] [Indexed: 09/29/2023] Open
Abstract
We examine the challenging "marriage" between computational efficiency and biological plausibility-A crucial node in the domain of spiking neural networks at the intersection of neuroscience, artificial intelligence, and robotics. Through a transdisciplinary review, we retrace the historical and most recent constraining influences that these parallel fields have exerted on descriptive analysis of the brain, construction of predictive brain models, and ultimately, the embodiment of neural networks in an enacted robotic agent. We study models of Spiking Neural Networks (SNN) as the central means enabling autonomous and intelligent behaviors in biological systems. We then provide a critical comparison of the available hardware and software to emulate SNNs for investigating biological entities and their application on artificial systems. Neuromorphics is identified as a promising tool to embody SNNs in real physical systems and different neuromorphic chips are compared. The concepts required for describing SNNs are dissected and contextualized in the new no man's land between cognitive neuroscience and artificial intelligence. Although there are recent reviews on the application of neuromorphic computing in various modules of the guidance, navigation, and control of robotic systems, the focus of this paper is more on closing the cognition loop in SNN-embodied robotics. We argue that biologically viable spiking neuronal models used for electroencephalogram signals are excellent candidates for furthering our knowledge of the explainability of SNNs. We complete our survey by reviewing different robotic modules that can benefit from neuromorphic hardware, e.g., perception (with a focus on vision), localization, and cognition. We conclude that the tradeoff between symbolic computational power and biological plausibility of hardware can be best addressed by neuromorphics, whose presence in neurorobotics provides an accountable empirical testbench for investigating synthetic and natural embodied cognition. We argue this is where both theoretical and empirical future work should converge in multidisciplinary efforts involving neuroscience, artificial intelligence, and robotics.
Collapse
Affiliation(s)
- Martin Do Pham
- Department of Computer Science, University of Toronto, Toronto, ON M5S 1A1, Canada; (M.D.P.); (M.M.D.)
| | - Amedeo D’Angiulli
- Department of Neuroscience, Carleton University, Ottawa, ON K1S 5B6, Canada;
| | - Maryam Mehri Dehnavi
- Department of Computer Science, University of Toronto, Toronto, ON M5S 1A1, Canada; (M.D.P.); (M.M.D.)
| | - Robin Chhabra
- Department of Mechanical and Aerospace Engineering, Carleton University, Ottawa, ON K1S 5B6, Canada
| |
Collapse
|
2
|
Hwang S, Hwang Y, Kim D, Lee J, Choe HK, Lee J, Kang H, Kung J. ReplaceNet: real-time replacement of a biological neural circuit with a hardware-assisted spiking neural network. Front Neurosci 2023; 17:1161592. [PMID: 37638314 PMCID: PMC10448768 DOI: 10.3389/fnins.2023.1161592] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Accepted: 07/28/2023] [Indexed: 08/29/2023] Open
Abstract
Recent developments in artificial neural networks and their learning algorithms have enabled new research directions in computer vision, language modeling, and neuroscience. Among various neural network algorithms, spiking neural networks (SNNs) are well-suited for understanding the behavior of biological neural circuits. In this work, we propose to guide the training of a sparse SNN in order to replace a sub-region of a cultured hippocampal network with limited hardware resources. To verify our approach with a realistic experimental setup, we record spikes of cultured hippocampal neurons with a microelectrode array (in vitro). The main focus of this work is to dynamically cut unimportant synapses during SNN training on the fly so that the model can be realized on resource-constrained hardware, e.g., implantable devices. To do so, we adopt a simple STDP learning rule to easily select important synapses that impact the quality of spike timing learning. By combining the STDP rule with online supervised learning, we can precisely predict the spike pattern of the cultured network in real-time. The reduction in the model complexity, i.e., the reduced number of connections, significantly reduces the required hardware resources, which is crucial in developing an implantable chip for the treatment of neurological disorders. In addition to the new learning algorithm, we prototype a sparse SNN hardware on a small FPGA with pipelined execution and parallel computing to verify the possibility of real-time replacement. As a result, we can replace a sub-region of the biological neural circuit within 22 μs using 2.5 × fewer hardware resources, i.e., by allowing 80% sparsity in the SNN model, compared to the fully-connected SNN model. With energy-efficient algorithms and hardware, this work presents an essential step toward real-time neuroprosthetic computation.
Collapse
Affiliation(s)
- Sangwoo Hwang
- Department of Electrical Engineering and Computer Science, DGIST, Daegu, Republic of Korea
| | - Yujin Hwang
- Department of Electrical Engineering and Computer Science, DGIST, Daegu, Republic of Korea
| | - Duhee Kim
- Department of Electrical Engineering and Computer Science, DGIST, Daegu, Republic of Korea
| | - Junhee Lee
- Department of Electrical Engineering and Computer Science, DGIST, Daegu, Republic of Korea
| | - Han Kyoung Choe
- Department of Brain Sciences, DGIST, Daegu, Republic of Korea
| | - Junghyup Lee
- Department of Electrical Engineering and Computer Science, DGIST, Daegu, Republic of Korea
| | - Hongki Kang
- Department of Electrical Engineering and Computer Science, DGIST, Daegu, Republic of Korea
| | - Jaeha Kung
- School of Electrical Engineering, Korea University, Seoul, Republic of Korea
| |
Collapse
|
3
|
Rahman MA, Brown DJ, Mahmud M, Harris M, Shopland N, Heym N, Sumich A, Turabee ZB, Standen B, Downes D, Xing Y, Thomas C, Haddick S, Premkumar P, Nastase S, Burton A, Lewis J. Enhancing biofeedback-driven self-guided virtual reality exposure therapy through arousal detection from multimodal data using machine learning. Brain Inform 2023; 10:14. [PMID: 37341863 DOI: 10.1186/s40708-023-00193-9] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2022] [Accepted: 05/15/2023] [Indexed: 06/22/2023] Open
Abstract
Virtual reality exposure therapy (VRET) is a novel intervention technique that allows individuals to experience anxiety-evoking stimuli in a safe environment, recognise specific triggers and gradually increase their exposure to perceived threats. Public-speaking anxiety (PSA) is a prevalent form of social anxiety, characterised by stressful arousal and anxiety generated when presenting to an audience. In self-guided VRET, participants can gradually increase their tolerance to exposure and reduce anxiety-induced arousal and PSA over time. However, creating such a VR environment and determining physiological indices of anxiety-induced arousal or distress is an open challenge. Environment modelling, character creation and animation, psychological state determination and the use of machine learning (ML) models for anxiety or stress detection are equally important, and multi-disciplinary expertise is required. In this work, we have explored a series of ML models with publicly available data sets (using electroencephalogram and heart rate variability) to predict arousal states. If we can detect anxiety-induced arousal, we can trigger calming activities to allow individuals to cope with and overcome distress. Here, we discuss the means of effective selection of ML models and parameters in arousal detection. We propose a pipeline to overcome the model selection problem with different parameter settings in the context of virtual reality exposure therapy. This pipeline can be extended to other domains of interest where arousal detection is crucial. Finally, we have implemented a biofeedback framework for VRET where we successfully provided feedback as a form of heart rate and brain laterality index from our acquired multimodal data for psychological intervention to overcome anxiety.
Collapse
Affiliation(s)
- Muhammad Arifur Rahman
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK
| | - David J Brown
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK
| | - Mufti Mahmud
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK.
- Medical Technologies Innovation Facility, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK.
- Computing and Informatics Research Centre, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK.
| | - Matthew Harris
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK
| | - Nicholas Shopland
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK
| | - Nadja Heym
- School of Social Sciences, Nottingham Trent University, Shakespeare St, Nottingham, NG1 4FQ, UK
| | - Alexander Sumich
- School of Social Sciences, Nottingham Trent University, Shakespeare St, Nottingham, NG1 4FQ, UK
| | - Zakia Batool Turabee
- School of Social Sciences, Nottingham Trent University, Shakespeare St, Nottingham, NG1 4FQ, UK
| | - Bradley Standen
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK
| | - David Downes
- Nottingham School of Art & Design, Nottingham Trent University, Shakespeare St, Nottingham, NG1 4FQ, UK
| | - Yangang Xing
- School of ADBE, Nottingham Trent University, Shakespeare St, Nottingham, NG1 4FQ, UK
| | - Carolyn Thomas
- Nottingham School of Art & Design, Nottingham Trent University, Shakespeare St, Nottingham, NG1 4FQ, UK
| | - Sean Haddick
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK
| | - Preethi Premkumar
- Division of Psychology, London South Bank University, London, SE1 0AA, UK
| | | | - Andrew Burton
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK
| | - James Lewis
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK
| |
Collapse
|
4
|
Gu YQ, Zhu Y. Underlying mechanisms of mindfulness meditation: Genomics, circuits, and networks. World J Psychiatry 2022; 12:1141-1149. [PMID: 36186506 PMCID: PMC9521538 DOI: 10.5498/wjp.v12.i9.1141] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/30/2022] [Revised: 04/29/2022] [Accepted: 08/18/2022] [Indexed: 02/05/2023] Open
Abstract
Understanding neuropsychological mechanisms of mindfulness meditation (MM) has been a hot topic in recent years. This review was conducted with the goal of synthesizing empirical relationships via the genomics, circuits and networks between MM and mental disorders. We describe progress made in assessing the effects of MM on gene expression in immune cells, with particular focus on stress-related inflammatory markers and associated biological pathways. We then focus on key brain circuits associated with mindfulness practices and effects on symptoms of mental disorders, and expand our discussion to identify three key brain networks associated with mindfulness practices including default mode network, central executive network, and salience network. More research efforts need to be devoted into identifying underlying neuropsychological mechanisms of MM on how it alleviates the symptoms of mental disorders.
Collapse
Affiliation(s)
- Ying-Qi Gu
- Department of Psychology, Zhejiang Sci-Tech University, Hangzhou 310018, Zhejiang Province, China
| | - Yi Zhu
- School of Psychology, Hainan Medical University, Haikou 571199, Hainan Province, China
- Department of Psychology, The First Affiliated Hospital of Hainan Medical University, Haikou 570102, Hainan Province, China
| |
Collapse
|
5
|
Personalized Spiking Neural Network Models of Clinical and Environmental Factors to Predict Stroke. Cognit Comput 2022. [DOI: 10.1007/s12559-021-09975-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
Abstract
AbstractThe high incidence of stroke occurrence necessitates the understanding of its causes and possible ways for early prediction and prevention. In this respect, statistical methods offer the “big picture,” but they have a weak predictive ability at an individual level. This research proposes a new personalized modeling method based on computational spiking neural networks (SNN) for the identification of causal associations between clinical and environmental time series data that can be used to predict individual stroke events. The method is tested on 804 stroke patients. Given a clinical data set of patients who experienced a stroke in the past and the corresponding environmental time-series data for a selected time-window before the stroke event, the method identifies the clusters of individuals with a high risk for stroke under similar conditions. The methodology involves a pipeline of processes when creating a personalized model for an individual $$x$$
x
: (1) selecting a group of individuals $$Gx$$
Gx
with similar personal records to $$x$$
x
; (2) training a personalized SNN $$x$$
x
model of several days of environmental data related to the $$Gx$$
Gx
group to predict the risk of stroke for $$x$$
x
at least one day earlier; (3) model interpretability through 3D visualization; (4) discovery of personalized predictive markers. The results are twofold, first proposing a new computational methodology and second presenting new findings. It is found that certain environmental factors, such as SO2, PM10, CO, and PM2.5, increase the risk of stroke if an individual $$x$$
x
belongs to a certain cluster of people, characterized by a combination of family history of stroke and diabetes, overweight, vascular/heart disease, age, and other. For the used population data, the proposed method can predict accurately individual risk of stroke before the day of the stroke. The paper presents a new methodology for personalized machine learning methods to define subgroups of the population with a high risk of stroke and to predict early individual risk of the stroke event. This makes the proposed cognitive computation method useful to reduce morbidity and mortality in society. The method is broadly applicable for predicting individual risk of other diseases and mental health conditions.
Collapse
|
6
|
Zou Z, Alimohamadi H, Zakeri A, Imani F, Kim Y, Najafi MH, Imani M. Memory-inspired spiking hyperdimensional network for robust online learning. Sci Rep 2022; 12:7641. [PMID: 35538126 PMCID: PMC9090930 DOI: 10.1038/s41598-022-11073-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2021] [Accepted: 04/08/2022] [Indexed: 11/09/2022] Open
Abstract
Recently, brain-inspired computing models have shown great potential to outperform today's deep learning solutions in terms of robustness and energy efficiency. Particularly, Spiking Neural Networks (SNNs) and HyperDimensional Computing (HDC) have shown promising results in enabling efficient and robust cognitive learning. Despite the success, these two brain-inspired models have different strengths. While SNN mimics the physical properties of the human brain, HDC models the brain on a more abstract and functional level. Their design philosophies demonstrate complementary patterns that motivate their combination. With the help of the classical psychological model on memory, we propose SpikeHD, the first framework that fundamentally combines Spiking neural network and hyperdimensional computing. SpikeHD generates a scalable and strong cognitive learning system that better mimics brain functionality. SpikeHD exploits spiking neural networks to extract low-level features by preserving the spatial and temporal correlation of raw event-based spike data. Then, it utilizes HDC to operate over SNN output by mapping the signal into high-dimensional space, learning the abstract information, and classifying the data. Our extensive evaluation on a set of benchmark classification problems shows that SpikeHD provides the following benefit compared to SNN architecture: (1) significantly enhance learning capability by exploiting two-stage information processing, (2) enables substantial robustness to noise and failure, and (3) reduces the network size and required parameters to learn complex information.
Collapse
Affiliation(s)
- Zhuowen Zou
- University of California San Diego, La Jolla, CA, 92093, USA
- University of California Irvine, Irvine, CA, 92697, USA
| | | | - Ali Zakeri
- University of California Irvine, Irvine, CA, 92697, USA
| | - Farhad Imani
- University of Connecticut, Storrs, CT, 06269, USA
| | - Yeseong Kim
- Daegu Gyeongbuk Institute of Science and Technology, Daegu, South Korea
| | | | - Mohsen Imani
- University of California Irvine, Irvine, CA, 92697, USA.
| |
Collapse
|
7
|
Singanamalla SKR, Lin CT. Spike-Representation of EEG Signals for Performance Enhancement of Brain-Computer Interfaces. Front Neurosci 2022; 16:792318. [PMID: 35444515 PMCID: PMC9014221 DOI: 10.3389/fnins.2022.792318] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2021] [Accepted: 01/12/2022] [Indexed: 11/13/2022] Open
Abstract
Brain-computer interfaces (BCI) relying on electroencephalography (EEG) based neuroimaging mode has shown prospects for real-world usage due to its portability and optional selectivity of fewer channels for compactness. However, noise and artifacts often limit the capacity of BCI systems especially for event-related potentials such as P300 and error-related negativity (ERN), whose biomarkers are present in short time segments at the time-series level. Contrary to EEG, invasive recording is less prone to noise but requires a tedious surgical procedure. But EEG signal is the result of aggregation of neuronal spiking information underneath the scalp surface and transforming the relevant BCI task's EEG signal to spike representation could potentially help improve the BCI performance. In this study, we designed an approach using a spiking neural network (SNN) which is trained using surrogate-gradient descent to generate task-related multi-channel EEG template signals of all classes. The trained model is in turn leveraged to obtain the latent spike representation for each EEG sample. Comparing the classification performance of EEG signal and its spike-representation, the proposed approach enhanced the performance of ERN dataset from 79.22 to 82.27% with naive bayes and for P300 dataset, the accuracy was improved from 67.73 to 69.87% using xGboost. In addition, principal component analysis and correlation metrics were evaluated on both EEG signals and their spike-representation to identify the reason for such improvement.
Collapse
Affiliation(s)
- Sai Kalyan Ranga Singanamalla
- Computational Intelligence and Brain Computer Interface Lab, School of Computer Science, University of Technology Sydney, Sydney, NSW, Australia
| | - Chin-Teng Lin
- Computational Intelligence and Brain Computer Interface Lab, School of Computer Science, University of Technology Sydney, Sydney, NSW, Australia
- Australian Artificial Intelligence Institute, University of Technology Sydney, Sydney, NSW, Australia
- *Correspondence: Chin-Teng Lin
| |
Collapse
|
8
|
Towards Machine Learning Driven Self-guided Virtual Reality Exposure Therapy Based on Arousal State Detection from Multimodal Data. Brain Inform 2022. [DOI: 10.1007/978-3-031-15037-1_17] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022] Open
|
9
|
Spiking Neural Networks for Computational Intelligence: An Overview. BIG DATA AND COGNITIVE COMPUTING 2021. [DOI: 10.3390/bdcc5040067] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Deep neural networks with rate-based neurons have exhibited tremendous progress in the last decade. However, the same level of progress has not been observed in research on spiking neural networks (SNN), despite their capability to handle temporal data, energy-efficiency and low latency. This could be because the benchmarking techniques for SNNs are based on the methods used for evaluating deep neural networks, which do not provide a clear evaluation of the capabilities of SNNs. Particularly, the benchmarking of SNN approaches with regards to energy efficiency and latency requires realization in suitable hardware, which imposes additional temporal and resource constraints upon ongoing projects. This review aims to provide an overview of the current real-world applications of SNNs and identifies steps to accelerate research involving SNNs in the future.
Collapse
|
10
|
Gu Y, Zhu Y, Brown KW. Mindfulness and Attention Deficit Hyperactivity Disorder: A Neuropsychological Perspective. J Nerv Ment Dis 2021; 209:796-801. [PMID: 34292276 DOI: 10.1097/nmd.0000000000001388] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Abstract
ABSTRACT Understanding the underlying mechanisms of mindfulness has been a hot topic in recent years, not only in clinical fields but also in neuroscience. Most neuroimaging findings demonstrate that critical brain regions involved in mindfulness are responsible for cognitive functions and mental states. However, the brain is a complex system operating via multiple circuits and networks, rather than isolated brain regions solely responsible for specific functions. Mindfulness-based treatments for attention deficit hyperactivity disorder (ADHD) have emerged as promising adjunctive or alternative intervention approaches. We focus on four key brain circuits associated with mindfulness practices and effects on symptoms of ADHD and its cognitive dysfunction, including executive attention circuit, sustained attention circuit, impulsivity circuit, and hyperactivity circuit. We also expand our discussion to identify three key brain networks associated with mindfulness practices, including central executive network, default mode network, and salience network. We conclude by suggesting that more research efforts need to be devoted into identifying putative neuropsychological mechanisms of mindfulness on how it alleviates ADHD symptoms.
Collapse
Affiliation(s)
- Yingqi Gu
- Department of Psychology, Zhejiang Sci-Tech University, Hangzhou
| | | | - Kirk Warren Brown
- Department of Psychology, Virginia Commonwealth University, Richmond, Virginia
| |
Collapse
|
11
|
Calvetti D, Johnson B, Pascarella A, Pitolli F, Somersalo E, Vantaggi B. Mining the Mind: Linear Discriminant Analysis of MEG Source Reconstruction Time Series Supports Dynamic Changes in Deep Brain Regions During Meditation Sessions. Brain Topogr 2021; 34:840-862. [PMID: 34652578 PMCID: PMC8556220 DOI: 10.1007/s10548-021-00874-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2021] [Accepted: 09/27/2021] [Indexed: 10/25/2022]
Abstract
Meditation practices have been claimed to have a positive effect on the regulation of mood and emotions for quite some time by practitioners, and in recent times there has been a sustained effort to provide a more precise description of the influence of meditation on the human brain. Longitudinal studies have reported morphological changes in cortical thickness and volume in selected brain regions due to meditation practice, which is interpreted as an evidence its effectiveness beyond the subjective self reporting. Using magnetoencephalography (MEG) or electroencephalography to quantify the changes in brain activity during meditation practice represents a challenge, as no clear hypothesis about the spatial or temporal pattern of such changes is available to date. In this article we consider MEG data collected during meditation sessions of experienced Buddhist monks practicing focused attention (Samatha) and open monitoring (Vipassana) meditation, contrasted by resting state with eyes closed. The MEG data are first mapped to time series of brain activity averaged over brain regions corresponding to a standard Destrieux brain atlas. Next, by bootstrapping and spectral analysis, the data are mapped to matrices representing random samples of power spectral densities in [Formula: see text], [Formula: see text], [Formula: see text], and [Formula: see text] frequency bands. We use linear discriminant analysis to demonstrate that the samples corresponding to different meditative or resting states contain enough fingerprints of the brain state to allow a separation between different states, and we identify the brain regions that appear to contribute to the separation. Our findings suggest that the cingulate cortex, insular cortex and some of the internal structures, most notably the accumbens, the caudate and the putamen nuclei, the thalamus and the amygdalae stand out as separating regions, which seems to correlate well with earlier findings based on longitudinal studies.
Collapse
Affiliation(s)
- Daniela Calvetti
- Department of Mathematics, Applied Mathematics and Statistics, Case Western Reserve University, 10900 Euclid Avenue, Cleveland, OH, 44106, USA
| | - Brian Johnson
- Department of Mathematics, Applied Mathematics and Statistics, Case Western Reserve University, 10900 Euclid Avenue, Cleveland, OH, 44106, USA
| | - Annalisa Pascarella
- Istituto per le Applicazioni del Calcolo "Mauro Picone" - CNR, Via dei Taurini 19, 00185, Rome, Italy
| | - Francesca Pitolli
- Department of Basic and Applied Sciences for Engineering, University of Rome "La Sapienza", Via Scarpa 16, 00161, Rome, Italy.
| | - Erkki Somersalo
- Department of Mathematics, Applied Mathematics and Statistics, Case Western Reserve University, 10900 Euclid Avenue, Cleveland, OH, 44106, USA
| | - Barbara Vantaggi
- Department MEMOTEF, University of Rome "La Sapienza", Via del Castro Laurenziano 9, 00161, Rome, Italy
| |
Collapse
|
12
|
Wang GY, Taylor T, Sumich A, Krägeloh C, Lee CQ, Siegert RJ. Cognitive Effect Following a Blended (Face to Face and Videoconference-Delivered) Format Mindfulness Training. Front Psychol 2021; 12:701459. [PMID: 34393937 PMCID: PMC8360837 DOI: 10.3389/fpsyg.2021.701459] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2021] [Accepted: 07/08/2021] [Indexed: 11/23/2022] Open
Abstract
While evidence supports the feasibility of online mindfulness training (MT), the effect of this approach on cognition remains unclear. The present study investigated changes in cognition following a newly developed 6-week videoconference-delivered MT program on cognitive function in two groups. The first group (n = 17) had two baseline assessments prior to MT [3 weeks after group two (n = 15)] to allow for evaluation of practice and learning effects. Four participants from each group were excluded from the final analysis due to missing data. Following MT, there was an improvement in switching of attention, working memory, executive function, and social cognition, but some of these effects were not easily accounted for by learning or practice effects. No significant changes were found on tasks measuring sustained attention, cognitive flexibility and inhibition, information processing, and sensory-motor function. Our findings suggest that domain-specific cognition might be enhanced by a brief videoconference-delivered MT, and larger, controlled studies to delineate the effects of online MT on subdomains of cognition are needed.
Collapse
Affiliation(s)
- Grace Y Wang
- Department of Psychology and Neuroscience, School of Clinical Sciences, Auckland University of Technology, Auckland, New Zealand
| | - Tamasin Taylor
- Faculty of Medical and Health Sciences, University of Auckland, Auckland, New Zealand
| | - Alexander Sumich
- Department of Psychology and Neuroscience, School of Clinical Sciences, Auckland University of Technology, Auckland, New Zealand.,Division of Psychology, School of Social Sciences, Nottingham Trent University, Nottingham, United Kingdom
| | - Chris Krägeloh
- Department of Psychology and Neuroscience, School of Clinical Sciences, Auckland University of Technology, Auckland, New Zealand
| | - Carol Qinglian Lee
- Department of Psychology and Neuroscience, School of Clinical Sciences, Auckland University of Technology, Auckland, New Zealand
| | - Richard J Siegert
- Department of Psychology and Neuroscience, School of Clinical Sciences, Auckland University of Technology, Auckland, New Zealand
| |
Collapse
|
13
|
Doborjeh M, Doborjeh Z, Kasabov N, Barati M, Wang GY. Deep Learning of Explainable EEG Patterns as Dynamic Spatiotemporal Clusters and Rules in a Brain-Inspired Spiking Neural Network. SENSORS 2021; 21:s21144900. [PMID: 34300640 PMCID: PMC8309947 DOI: 10.3390/s21144900] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/21/2021] [Revised: 07/14/2021] [Accepted: 07/14/2021] [Indexed: 11/18/2022]
Abstract
The paper proposes a new method for deep learning and knowledge discovery in a brain-inspired Spiking Neural Networks (SNN) architecture that enhances the model’s explainability while learning from streaming spatiotemporal brain data (STBD) in an incremental and on-line mode of operation. This led to the extraction of spatiotemporal rules from SNN models that explain why a certain decision (output prediction) was made by the model. During the learning process, the SNN created dynamic neural clusters, captured as polygons, which evolved in time and continuously changed their size and shape. The dynamic patterns of the clusters were quantitatively analyzed to identify the important STBD features that correspond to the most activated brain regions. We studied the trend of dynamically created clusters and their spike-driven events that occur together in specific space and time. The research contributes to: (1) enhanced interpretability of SNN learning behavior through dynamic neural clustering; (2) feature selection and enhanced accuracy of classification; (3) spatiotemporal rules to support model explainability; and (4) a better understanding of the dynamics in STBD in terms of feature interaction. The clustering method was applied to a case study of Electroencephalogram (EEG) data, recorded from a healthy control group (n = 21) and opiate use (n = 18) subjects while they were performing a cognitive task. The SNN models of EEG demonstrated different trends of dynamic clusters across the groups. This suggested to select a group of marker EEG features and resulted in an improved accuracy of EEG classification to 92%, when compared with all-feature classification. During learning of EEG data, the areas of neurons in the SNN model that form adjacent clusters (corresponding to neighboring EEG channels) were detected as fuzzy boundaries that explain overlapping activity of brain regions for each group of subjects.
Collapse
Affiliation(s)
- Maryam Doborjeh
- School of Engineering, Computer and Mathematical Sciences, Auckland University of Technology, Auckland 1010, New Zealand; (N.K.); (M.B.)
- Correspondence:
| | - Zohreh Doborjeh
- Department of Audiology, Faculty of Medical and Health Sciences, School of Population Health, The University of Auckland, Auckland 1023, New Zealand;
| | - Nikola Kasabov
- School of Engineering, Computer and Mathematical Sciences, Auckland University of Technology, Auckland 1010, New Zealand; (N.K.); (M.B.)
- George Moore Chair of Data Analytics, School of Computing, Engineering and Intelligent Systems, Ulster University, Derry/Londonderry BT48 7JL, UK
| | - Molood Barati
- School of Engineering, Computer and Mathematical Sciences, Auckland University of Technology, Auckland 1010, New Zealand; (N.K.); (M.B.)
| | - Grace Y. Wang
- Department of Psychology and Neuroscience, Auckland University of Technology, Auckland 0627, New Zealand;
| |
Collapse
|
14
|
Fang SL, Han CY, Liu WH, Li X, Wang XL, Huang XD, Wan J, Fan SQ, Zhang GH, Geng L. Multilevel resistive random access memory achieved by MoO 3/Hf/MoO 3stack and its application in tunable high-pass filter. NANOTECHNOLOGY 2021; 32:385203. [PMID: 34116525 DOI: 10.1088/1361-6528/ac0ac4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/07/2021] [Accepted: 06/11/2021] [Indexed: 06/12/2023]
Abstract
In this work, the multilevel resistive random access memories (RRAMs) have been achieved by using the structure of Pt/MoO3/Hf/MoO3/Pt with four stable resistance states. The devices show good retention property of each state (>104s) and large memory window (>104). The simulation and experimental study reveal that the resistive switching mechanism is ascribed to combination of the conductive filament in the stack of MoO3/Hf next to the top electrode and redox reaction at the interface of Hf/MoO3next to bottom electrode. The fitting results of current-voltage characteristics under low sweep voltage indicate that the conduction of HRSs is dominated by the Poole-Frenkel emission and that of LRS is governed by the Ohmic conduction. Based on the RRAM, the tunable high-pass filter (HPF) with configurable filtering characteristics has been realized. The gain-frequency characteristics of the programmable HPF show that the filter has high resolution and wide programming range, demonstrating the viability of the multilevel RRAMs for future spiking neural network and shrinking the programmable filters with low power consumption.
Collapse
Affiliation(s)
- Sheng Li Fang
- School of Microelectronics, Faculty of Electronics and Information Engineering, Xi'an Jiaotong University, Xi'an 710049, People's Republic of China
| | - Chuan Yu Han
- School of Microelectronics, Faculty of Electronics and Information Engineering, Xi'an Jiaotong University, Xi'an 710049, People's Republic of China
| | - Wei Hua Liu
- School of Microelectronics, Faculty of Electronics and Information Engineering, Xi'an Jiaotong University, Xi'an 710049, People's Republic of China
| | - Xin Li
- School of Microelectronics, Faculty of Electronics and Information Engineering, Xi'an Jiaotong University, Xi'an 710049, People's Republic of China
| | - Xiao Li Wang
- School of Microelectronics, Faculty of Electronics and Information Engineering, Xi'an Jiaotong University, Xi'an 710049, People's Republic of China
| | - Xiao Dong Huang
- Key Laboratory of MEMS of the Ministry of Education, School of Electronic Science and Engineering, Southeast University, Nanjing 211189, People's Republic of China
| | - Jun Wan
- College of Metrology & Measurement Engineering, China Jiliang University, Hangzhou 310018, People's Republic of China
- Advanced Materials Technology & Engineering, Inc., Wuxi 214000, People's Republic of China
| | - Shi Quan Fan
- School of Microelectronics, Faculty of Electronics and Information Engineering, Xi'an Jiaotong University, Xi'an 710049, People's Republic of China
| | - Guo He Zhang
- School of Microelectronics, Faculty of Electronics and Information Engineering, Xi'an Jiaotong University, Xi'an 710049, People's Republic of China
- The Key Lab of Micro-Nano Electronics and System Integration of Xi'an City, Xi'an 710049, People's Republic of China
| | - Li Geng
- School of Microelectronics, Faculty of Electronics and Information Engineering, Xi'an Jiaotong University, Xi'an 710049, People's Republic of China
| |
Collapse
|
15
|
Design of MRI structured spiking neural networks and learning algorithms for personalized modelling, analysis, and prediction of EEG signals. Sci Rep 2021; 11:12064. [PMID: 34103545 PMCID: PMC8187669 DOI: 10.1038/s41598-021-90029-5] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2020] [Accepted: 04/09/2021] [Indexed: 12/04/2022] Open
Abstract
This paper proposes a novel method and algorithms for the design of MRI structured personalized 3D spiking neural network models (MRI-SNN) for a better analysis, modeling, and prediction of EEG signals. It proposes a novel gradient-descent learning algorithm integrated with a spike-time-dependent-plasticity algorithm. The models capture informative personal patterns of interaction between EEG channels, contrary to single EEG signal modeling methods or to spike-based approaches which do not use personal MRI data to pre-structure a model. The proposed models can not only learn and model accurately measured EEG data, but they can also predict signals at 3D model locations that correspond to non-monitored brain areas, e.g. other EEG channels, from where data has not been collected. This is the first study in this respect. As an illustration of the method, personalized MRI-SNN models are created and tested on EEG data from two subjects. The models result in better prediction accuracy and a better understanding of the personalized EEG signals than traditional methods due to the MRI and EEG information integration. The models are interpretable and facilitate a better understanding of related brain processes. This approach can be applied for personalized modeling, analysis, and prediction of EEG signals across brain studies such as the study and prediction of epilepsy, peri-perceptual brain activities, brain-computer interfaces, and others.
Collapse
|
16
|
Durai M, Doborjeh Z, Sanders PJ, Vajsakovic D, Wendt A, Searchfield GD. Behavioral Outcomes and Neural Network Modeling of a Novel, Putative, Recategorization Sound Therapy. Brain Sci 2021; 11:554. [PMID: 33925762 PMCID: PMC8146945 DOI: 10.3390/brainsci11050554] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Revised: 04/23/2021] [Accepted: 04/24/2021] [Indexed: 02/07/2023] Open
Abstract
The mechanisms underlying sound's effect on tinnitus perception are unclear. Tinnitus activity appears to conflict with perceptual expectations of "real" sound, resulting in it being a salient signal. Attention diverted towards tinnitus during the later stages of object processing potentially disrupts high-order auditory streaming, and its uncertain nature results in negative psychological responses. This study investigated the benefits and neurophysiological basis of passive perceptual training and informational counseling to recategorize phantom perception as a more real auditory object. Specifically, it examined underlying psychoacoustic correlates of tinnitus and the neural activities associated with tinnitus auditory streaming and how malleable these are to change with targeted intervention. Eighteen participants (8 females, 10 males, mean age = 61.6 years) completed the study. The study consisted of 2 parts: (1) An acute exposure over 30 min to a sound that matched the person's tinnitus (Tinnitus Avatar) that was cross-faded to a selected nature sound (Cicadas, Fan, Water Sound/Rain, Birds, Water and Bird). (2) A chronic exposure for 3 months to the same "morphed" sound. A brain-inspired spiking neural network (SNN) architecture was used to model and compare differences between electroencephalography (EEG) patterns recorded prior to morphing sound presentation, during, after (3-month), and post-follow-up. Results showed that the tinnitus avatar generated was a good match to an individual's tinnitus as rated on likeness scales and was not rated as unpleasant. The five environmental sounds selected for this study were also rated as being appropriate matches to individuals' tinnitus and largely pleasant to listen to. There was a significant reduction in the Tinnitus Functional Index score and subscales of intrusiveness of the tinnitus signal and ability to concentrate with the tinnitus trial end compared to baseline. There was a significant decrease in how strong the tinnitus signal was rated as well as ratings of how easy it was to ignore the tinnitus signal on severity rating scales. Qualitative analysis found that the environmental sound interacted with the tinnitus in a positive way, but participants did not experience change in severity, however, characteristics of tinnitus, including pitch and uniformity of sound, were reported to change. The results indicate the feasibility of the computational SNN method and preliminary evidence that the sound exposure may change activation of neural tinnitus networks and greater bilateral hemispheric involvement as the sound morphs over time into natural environmental sound; particularly relating to attention and discriminatory judgments (dorsal attention network, precentral gyrus, ventral anterior network). This is the first study that attempts to recategorize tinnitus using passive auditory training to a sound that morphs from resembling the person's tinnitus to a natural sound. These findings will be used to design future-controlled trials to elucidate whether the approach used differs in effect and mechanism from conventional Broadband Noise (BBN) sound therapy.
Collapse
Affiliation(s)
- Mithila Durai
- Section of Audiology, School of Population Health, The University of Auckland, Auckland 1023, New Zealand; (M.D.); (Z.D.); (P.J.S.); (D.V.)
- Eisdell Moore Centre, Auckland 1023, New Zealand
- Centre for Brain Research, The University of Auckland, Auckland 1023, New Zealand
| | - Zohreh Doborjeh
- Section of Audiology, School of Population Health, The University of Auckland, Auckland 1023, New Zealand; (M.D.); (Z.D.); (P.J.S.); (D.V.)
- Eisdell Moore Centre, Auckland 1023, New Zealand
- Centre for Brain Research, The University of Auckland, Auckland 1023, New Zealand
| | - Philip J. Sanders
- Section of Audiology, School of Population Health, The University of Auckland, Auckland 1023, New Zealand; (M.D.); (Z.D.); (P.J.S.); (D.V.)
- Eisdell Moore Centre, Auckland 1023, New Zealand
- Centre for Brain Research, The University of Auckland, Auckland 1023, New Zealand
| | - Dunja Vajsakovic
- Section of Audiology, School of Population Health, The University of Auckland, Auckland 1023, New Zealand; (M.D.); (Z.D.); (P.J.S.); (D.V.)
- Eisdell Moore Centre, Auckland 1023, New Zealand
- Centre for Brain Research, The University of Auckland, Auckland 1023, New Zealand
| | - Anne Wendt
- Knowledge Engineering & Discovery Research Institute, Auckland University of Technology, Auckland 1010, New Zealand;
| | - Grant D. Searchfield
- Section of Audiology, School of Population Health, The University of Auckland, Auckland 1023, New Zealand; (M.D.); (Z.D.); (P.J.S.); (D.V.)
- Eisdell Moore Centre, Auckland 1023, New Zealand
- Centre for Brain Research, The University of Auckland, Auckland 1023, New Zealand
- Brain Research New Zealand—Rangahau Roro Aotearoa, The University of Auckland, Auckland 1142, New Zealand
| |
Collapse
|
17
|
NeuroSense: Short-term emotion recognition and understanding based on spiking neural network modelling of spatio-temporal EEG patterns. Neurocomputing 2021. [DOI: 10.1016/j.neucom.2020.12.098] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
|
18
|
Yamazaki T, Igarashi J, Yamaura H. Human-scale Brain Simulation via Supercomputer: A Case Study on the Cerebellum. Neuroscience 2021; 462:235-246. [PMID: 33482329 DOI: 10.1016/j.neuroscience.2021.01.014] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2020] [Revised: 12/30/2020] [Accepted: 01/06/2021] [Indexed: 01/03/2023]
Abstract
Performance of supercomputers has been steadily and exponentially increasing for the past 20 years, and is expected to increase further. This unprecedented computational power enables us to build and simulate large-scale neural network models composed of tens of billions of neurons and tens of trillions of synapses with detailed anatomical connections and realistic physiological parameters. Such "human-scale" brain simulation could be considered a milestone in computational neuroscience and even in general neuroscience. Towards this milestone, it is mandatory to introduce modern high-performance computing technology into neuroscience research. In this article, we provide an introductory landscape about large-scale brain simulation on supercomputers from the viewpoints of computational neuroscience and modern high-performance computing technology for specialists in experimental as well as computational neurosciences. This introduction to modeling and simulation methods is followed by a review of various representative large-scale simulation studies conducted to date. Then, we direct our attention to the cerebellum, with a review of more simulation studies specific to that region. Furthermore, we present recent simulation results of a human-scale cerebellar network model composed of 86 billion neurons on the Japanese flagship supercomputer K (now retired). Finally, we discuss the necessity and importance of human-scale brain simulation, and suggest future directions of such large-scale brain simulation research.
Collapse
Affiliation(s)
- Tadashi Yamazaki
- Graduate School of Informatics and Engineering, The University of Electro-Communications, Japan.
| | | | - Hiroshi Yamaura
- Graduate School of Informatics and Engineering, The University of Electro-Communications, Japan
| |
Collapse
|
19
|
Doborjeh Z, Doborjeh M, Crook-Rumsey M, Taylor T, Wang GY, Moreau D, Krägeloh C, Wrapson W, Siegert RJ, Kasabov N, Searchfield G, Sumich A. Interpretability of Spatiotemporal Dynamics of the Brain Processes Followed by Mindfulness Intervention in a Brain-Inspired Spiking Neural Network Architecture. SENSORS (BASEL, SWITZERLAND) 2020; 20:E7354. [PMID: 33371459 PMCID: PMC7767448 DOI: 10.3390/s20247354] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/24/2020] [Revised: 12/16/2020] [Accepted: 12/17/2020] [Indexed: 01/05/2023]
Abstract
Mindfulness training is associated with improvements in psychological wellbeing and cognition, yet the specific underlying neurophysiological mechanisms underpinning these changes are uncertain. This study uses a novel brain-inspired artificial neural network to investigate the effect of mindfulness training on electroencephalographic function. Participants completed a 4-tone auditory oddball task (that included targets and physically similar distractors) at three assessment time points. In Group A (n = 10), these tasks were given immediately prior to 6-week mindfulness training, immediately after training and at a 3-week follow-up; in Group B (n = 10), these were during an intervention waitlist period (3 weeks prior to training), pre-mindfulness training and post-mindfulness training. Using a spiking neural network (SNN) model, we evaluated concurrent neural patterns generated across space and time from features of electroencephalographic data capturing the neural dynamics associated with the event-related potential (ERP). This technique capitalises on the temporal dynamics of the shifts in polarity throughout the ERP and spatially across electrodes. Findings support anteriorisation of connection weights in response to distractors relative to target stimuli. Right frontal connection weights to distractors were associated with trait mindfulness (positively) and depression (inversely). Moreover, mindfulness training was associated with an increase in connection weights to targets (bilateral frontal, left frontocentral, and temporal regions only) and distractors. SNN models were superior to other machine learning methods in the classification of brain states as a function of mindfulness training. Findings suggest SNN models can provide useful information that differentiates brain states based on distinct task demands and stimuli, as well as changes in brain states as a function of psychological intervention.
Collapse
Affiliation(s)
- Zohreh Doborjeh
- Faculty of Medical and Health Sciences, School of Population Health, Section of Audiology, The University of Auckland, Auckland 1142, New Zealand;
- Eisdell Moore Centre, The University of Auckland, Auckland 1142, New Zealand
- Centre for Brain Research, The University of Auckland, Auckland 1142, New Zealand;
| | - Maryam Doborjeh
- Information Technology and Software Engineering Department, Auckland University of Technology, Auckland 1010, New Zealand;
| | - Mark Crook-Rumsey
- School of Psychology, Nottingham Trent University, Nottingham NG25 0QF, UK; (M.C.-R.); (A.S.)
| | - Tamasin Taylor
- Faculty of Medical and Health Sciences, The University of Auckland, Auckland 1142, New Zealand;
| | - Grace Y. Wang
- Department of Psychology and Neuroscience, Auckland University of Technology, Auckland 0627, New Zealand; (G.Y.W.); (C.K.); (R.J.S.)
| | - David Moreau
- Centre for Brain Research, The University of Auckland, Auckland 1142, New Zealand;
- School of Psychology, The University of Auckland, Auckland 1142, New Zealand
| | - Christian Krägeloh
- Department of Psychology and Neuroscience, Auckland University of Technology, Auckland 0627, New Zealand; (G.Y.W.); (C.K.); (R.J.S.)
| | - Wendy Wrapson
- School of Public Health and Interdisciplinary Studies, Auckland University of Technology, Auckland 0627, New Zealand;
| | - Richard J. Siegert
- Department of Psychology and Neuroscience, Auckland University of Technology, Auckland 0627, New Zealand; (G.Y.W.); (C.K.); (R.J.S.)
| | - Nikola Kasabov
- Intelligent Systems Research Centre, Ulster University, Londonderry BT48 7JL, UK
- School of Engineering, Computing and Mathematical Sciences, Auckland University of Technology, Auckland 1010, New Zealand
| | - Grant Searchfield
- Faculty of Medical and Health Sciences, School of Population Health, Section of Audiology, The University of Auckland, Auckland 1142, New Zealand;
- Eisdell Moore Centre, The University of Auckland, Auckland 1142, New Zealand
- Centre for Brain Research, The University of Auckland, Auckland 1142, New Zealand;
| | - Alexander Sumich
- School of Psychology, Nottingham Trent University, Nottingham NG25 0QF, UK; (M.C.-R.); (A.S.)
| |
Collapse
|
20
|
Al-Nafjan A, Alharthi K, Kurdi H. Lightweight Building of an Electroencephalogram-Based Emotion Detection System. Brain Sci 2020; 10:brainsci10110781. [PMID: 33114646 PMCID: PMC7693518 DOI: 10.3390/brainsci10110781] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2020] [Revised: 10/23/2020] [Accepted: 10/23/2020] [Indexed: 11/24/2022] Open
Abstract
Brain–computer interface (BCI) technology provides a direct interface between the brain and an external device. BCIs have facilitated the monitoring of conscious brain electrical activity via electroencephalogram (EEG) signals and the detection of human emotion. Recently, great progress has been made in the development of novel paradigms for EEG-based emotion detection. These studies have also attempted to apply BCI research findings in varied contexts. Interestingly, advances in BCI technologies have increased the interest of scientists because such technologies’ practical applications in human–machine relationships seem promising. This emphasizes the need for a building process for an EEG-based emotion detection system that is lightweight, in terms of a smaller EEG dataset size and no involvement of feature extraction methods. In this study, we investigated the feasibility of using a spiking neural network to build an emotion detection system from a smaller version of the DEAP dataset with no involvement of feature extraction methods while maintaining decent accuracy. The results showed that by using a NeuCube-based spiking neural network, we could detect the valence emotion level using only 60 EEG samples with 84.62% accuracy, which is a comparable accuracy to that of previous studies.
Collapse
Affiliation(s)
- Abeer Al-Nafjan
- Computer Science Department, Imam Muhammad ibn Saud Islamic University, Riyadh 11432, Saudi Arabia;
| | - Khulud Alharthi
- Computer Science Department, King Saud University, Riyadh 11543, Saudi Arabia;
- Computer Science Department, Taif University, Taif 26571, Saudi Arabia
| | - Heba Kurdi
- Computer Science Department, King Saud University, Riyadh 11543, Saudi Arabia;
- Mechanical Engineering Department, Massachusetts Institute of Technology (MIT), Cambridge, MA 02139, USA
- Correspondence: or ; Tel.: +966-11-805-9637
| |
Collapse
|
21
|
Kelsen BA, Sumich A, Kasabov N, Liang SHY, Wang GY. What has social neuroscience learned from hyperscanning studies of spoken communication? A systematic review. Neurosci Biobehav Rev 2020; 132:1249-1262. [PMID: 33022298 DOI: 10.1016/j.neubiorev.2020.09.008] [Citation(s) in RCA: 39] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2020] [Revised: 09/03/2020] [Accepted: 09/08/2020] [Indexed: 12/12/2022]
Abstract
A growing body of literature examining the neurocognitive processes of interpersonal linguistic interaction indicates the emergence of neural alignment as participants engage in oral communication. However, questions have arisen whether the study results can be interpreted beyond observations of cortical functionality and extended to the mutual understanding between communicators. This review presents evidence from electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) hyperscanning studies of interbrain synchrony (IBS) in which participants communicated via spoken language. The studies are classified into: knowledge sharing; turn-taking speech co-ordination; cooperation, problem-solving and creativity; and naturalistic discussion paradigms according to the type of interaction specified in each study. Alignment predominantly occurred in the frontal and temporo-parietal areas, which may reflect activation of the mirror and mentalizing systems. We argue that the literature presents a significant contribution to advancing our understanding of IBS and mutual understanding between communicators. We end with suggestions for future research, including analytical approaches and experimental conditions and hypothesize that brain-inspired neural networks are promising techniques for better understanding of IBS through hyperscanning.
Collapse
Affiliation(s)
- Brent A Kelsen
- Department of Psychology and Neuroscience, Auckland University of Technology, Auckland, New Zealand; Language Center, National Taipei University, New Taipei City, Taiwan
| | - Alexander Sumich
- Division of Psychology, Nottingham Trent University, Nottingham, United Kingdom
| | - Nikola Kasabov
- Knowledge Engineering and Discovery Research Institute (KEDRI), Auckland University of Technology, Auckland, New Zealand
| | - Sophie H Y Liang
- Department of Medicine, Chang Gung University, Taoyuan, Taiwan; Department of Child & Adolescent Psychiatry, Chang Gung Memorial Hospital at Taoyuan, Taoyuan, Taiwan
| | - Grace Y Wang
- Department of Psychology and Neuroscience, Auckland University of Technology, Auckland, New Zealand.
| |
Collapse
|
22
|
Durai M, Sanders P, Doborjeh Z, Doborjeh M, Wendt A, Kasabov N, Searchfield GD. Prediction of tinnitus masking benefit within a case series using a spiking neural network model. PROGRESS IN BRAIN RESEARCH 2020; 260:129-165. [PMID: 33637215 DOI: 10.1016/bs.pbr.2020.08.003] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
Masking has been widely used as a tinnitus therapy, with large individual differences in its effectiveness. The basis of this variation is unknown. We examined individual tinnitus and psychological responses to three masking types, energetic masking (bilateral broadband static or rain noise [BBN]), informational masking (BBN with a notch at tinnitus pitch and 3-dimensional cues) and a masker combining both effects (BBN with spatial cues). Eleven participants with chronic tinnitus were followed for 12 months, each person used each masking approach for 3 months with a 1 month washout-baseline. The Tinnitus Functional Index (TFI), Tinnitus Rating Scales, Positive and Negative Affect Scale and Depression Anxiety Stress Scales, were measured every month of treatment. Electroencephalography (EEG) and psychoacoustic assessment was undertaken at baseline and following 3 months of each masking sound. The computational modeling of EEG data was based on the framework of brain-inspired Spiking Neural Network (SNN) architecture called NeuCube, designed for this study for mapping, learning, visualizing and classifying of brain activity patterns. EEG was related to clinically significant change in the TFI using the SNN model. The SNN framework was able to predict sound therapy responders (93% accuracy) from non-responders (100% accuracy) using baseline EEG recordings. The combination of energetic and informational masking was an effective treatment sound in more individuals than the other sounds used. Although the findings are promising, they are preliminary and require confirmation in independent and larger samples.
Collapse
Affiliation(s)
- Mithila Durai
- Section of Audiology, The University of Auckland, Auckland, New Zealand; Eisdell Moore Centre, The University of Auckland, Auckland, New Zealand; Centre for Brain Research, The University of Auckland, Auckland, New Zealand; Brain Research New Zealand-Rangahau Roro Aotearoa, Auckland, New Zealand
| | - Philip Sanders
- Section of Audiology, The University of Auckland, Auckland, New Zealand; Eisdell Moore Centre, The University of Auckland, Auckland, New Zealand; Centre for Brain Research, The University of Auckland, Auckland, New Zealand; Brain Research New Zealand-Rangahau Roro Aotearoa, Auckland, New Zealand
| | - Zohreh Doborjeh
- Section of Audiology, The University of Auckland, Auckland, New Zealand; Eisdell Moore Centre, The University of Auckland, Auckland, New Zealand; Centre for Brain Research, The University of Auckland, Auckland, New Zealand; Brain Research New Zealand-Rangahau Roro Aotearoa, Auckland, New Zealand
| | - Maryam Doborjeh
- Information Technology and Software Engineering department, Auckland University of Technology, Auckland, New Zealand
| | - Anne Wendt
- School of Engineering, Computing and Mathematical Sciences, Auckland University of Technology, Auckland, New Zealand
| | - Nikola Kasabov
- School of Engineering, Computing and Mathematical Sciences, Auckland University of Technology, Auckland, New Zealand; Intelligent Systems Research Centre, Ulster University, Newtownabbey, United Kingdom
| | - Grant D Searchfield
- Section of Audiology, The University of Auckland, Auckland, New Zealand; Eisdell Moore Centre, The University of Auckland, Auckland, New Zealand; Centre for Brain Research, The University of Auckland, Auckland, New Zealand; Brain Research New Zealand-Rangahau Roro Aotearoa, Auckland, New Zealand.
| |
Collapse
|