1
|
Hamzah HA, Abdalla KK. EEG-based emotion recognition systems; comprehensive study. Heliyon 2024; 10:e31485. [PMID: 38818173 PMCID: PMC11137547 DOI: 10.1016/j.heliyon.2024.e31485] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2024] [Accepted: 05/16/2024] [Indexed: 06/01/2024] Open
Abstract
Emotion recognition technology through EEG signal analysis is currently a fundamental concept in artificial intelligence. This recognition has major practical implications in emotional health care, human-computer interaction, and so on. This paper provides a comprehensive study of different methods for extracting electroencephalography (EEG) features for emotion recognition from four different perspectives, including time domain features, frequency domain features, time-frequency features, and nonlinear features. We summarize the current pattern recognition methods adopted in most related works, and with the rapid development of deep learning (DL) attracting the attention of researchers in this field, we pay more attention to deep learning-based studies and analyse the characteristics, advantages, disadvantages, and applicable scenarios. Finally, the current challenges and future development directions in this field were summarized. This paper can help novice researchers in this field gain a systematic understanding of the current status of emotion recognition research based on EEG signals and provide ideas for subsequent related research.
Collapse
Affiliation(s)
- Hussein Ali Hamzah
- Electrical Engineering Department, College of Engineering, University of Babylon, Iraq
| | - Kasim K. Abdalla
- Electrical Engineering Department, College of Engineering, University of Babylon, Iraq
| |
Collapse
|
2
|
Lim RY, Lew WCL, Ang KK. Review of EEG Affective Recognition with a Neuroscience Perspective. Brain Sci 2024; 14:364. [PMID: 38672015 PMCID: PMC11048077 DOI: 10.3390/brainsci14040364] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2024] [Revised: 04/02/2024] [Accepted: 04/06/2024] [Indexed: 04/28/2024] Open
Abstract
Emotions are a series of subconscious, fleeting, and sometimes elusive manifestations of the human innate system. They play crucial roles in everyday life-influencing the way we evaluate ourselves, our surroundings, and how we interact with our world. To date, there has been an abundance of research on the domains of neuroscience and affective computing, with experimental evidence and neural network models, respectively, to elucidate the neural circuitry involved in and neural correlates for emotion recognition. Recent advances in affective computing neural network models often relate closely to evidence and perspectives gathered from neuroscience to explain the models. Specifically, there has been growing interest in the area of EEG-based emotion recognition to adopt models based on the neural underpinnings of the processing, generation, and subsequent collection of EEG data. In this respect, our review focuses on providing neuroscientific evidence and perspectives to discuss how emotions potentially come forth as the product of neural activities occurring at the level of subcortical structures within the brain's emotional circuitry and the association with current affective computing models in recognizing emotions. Furthermore, we discuss whether such biologically inspired modeling is the solution to advance the field in EEG-based emotion recognition and beyond.
Collapse
Affiliation(s)
- Rosary Yuting Lim
- Institute for Infocomm Research, Agency for Science, Technology and Research, A*STAR, 1 Fusionopolis Way, #21-01 Connexis, Singapore 138632, Singapore; (R.Y.L.); (W.-C.L.L.)
| | - Wai-Cheong Lincoln Lew
- Institute for Infocomm Research, Agency for Science, Technology and Research, A*STAR, 1 Fusionopolis Way, #21-01 Connexis, Singapore 138632, Singapore; (R.Y.L.); (W.-C.L.L.)
- School of Computer Science and Engineering, Nanyang Technological University, 50 Nanyang Ave., 32 Block N4 02a, Singapore 639798, Singapore
| | - Kai Keng Ang
- Institute for Infocomm Research, Agency for Science, Technology and Research, A*STAR, 1 Fusionopolis Way, #21-01 Connexis, Singapore 138632, Singapore; (R.Y.L.); (W.-C.L.L.)
- School of Computer Science and Engineering, Nanyang Technological University, 50 Nanyang Ave., 32 Block N4 02a, Singapore 639798, Singapore
| |
Collapse
|
3
|
Gao Y, Zhang C, Huang J, Meng M. EEG multi-domain feature transfer based on sparse regularized Tucker decomposition. Cogn Neurodyn 2024; 18:185-197. [PMID: 38406207 PMCID: PMC10881956 DOI: 10.1007/s11571-023-09936-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2022] [Revised: 12/21/2022] [Accepted: 01/24/2023] [Indexed: 02/19/2023] Open
Abstract
Tensor analysis of electroencephalogram (EEG) can extract the activity information and the potential interaction between different brain regions. However, EEG data varies between subjects, and the existing tensor decomposition algorithms cannot guarantee that the features across subjects are distributed in the same domain, which leads to the non-objectivity of the classification result and analysis, In addition, traditional Tucker decomposition is prone to the explosion of feature dimensions. To solve these problems, combined with the idea of feature transfer, a novel EEG tensor transfer algorithm, Tensor Subspace Learning based on Sparse Regularized Tucker Decomposition (TSL-SRT), is proposed in this paper. In TSL-SRT, new EEG samples are considered as the target domain and original samples as the source domain. The target features can be obtained by projecting the target tensor to the source feature space to ensure that all features are in the same domain. Furthermore, to solve the problem of dimension explosion caused by TSL-SRT, a redundant EEG features screening algorithm is adopted to eliminate the redundant features, and achieves 77.8%, 73.2% and 75.3% accuracy on three BCI datasets. By visualizing the spatial basic matrix of the feature space, it can be seen that TSL-SRT is effective in extracting the features of active brain regions in the BCI task and it can extract the multi-domain features of different subjects in the same domain simultaneously, which provides a new method for the tensor analysis of EEG.
Collapse
Affiliation(s)
- Yunyuan Gao
- College of Automation, Hangzhou Dianzi University, Hangzhou, 310018 Zhejiang People’s Republic of China
- Zhejiang Key Laboratory of Brain Computer Collaborative Intelligence, Hangzhou Dianzi University, Hangzhou, 310018 Zhejiang People’s Republic of China
| | - Congrui Zhang
- College of Automation, Hangzhou Dianzi University, Hangzhou, 310018 Zhejiang People’s Republic of China
| | - Jincheng Huang
- HDU-ITMO Joint Institute, Hangzhou Dianzi University, Hangzhou, 310018 Zhejiang People’s Republic of China
| | - Ming Meng
- College of Automation, Hangzhou Dianzi University, Hangzhou, 310018 Zhejiang People’s Republic of China
- Zhejiang Key Laboratory of Brain Computer Collaborative Intelligence, Hangzhou Dianzi University, Hangzhou, 310018 Zhejiang People’s Republic of China
| |
Collapse
|
4
|
Sen O, Sheehan AM, Raman PR, Khara KS, Khalifa A, Chatterjee B. Machine-Learning Methods for Speech and Handwriting Detection Using Neural Signals: A Review. SENSORS (BASEL, SWITZERLAND) 2023; 23:5575. [PMID: 37420741 DOI: 10.3390/s23125575] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/20/2023] [Revised: 06/09/2023] [Accepted: 06/12/2023] [Indexed: 07/09/2023]
Abstract
Brain-Computer Interfaces (BCIs) have become increasingly popular in recent years due to their potential applications in diverse fields, ranging from the medical sector (people with motor and/or communication disabilities), cognitive training, gaming, and Augmented Reality/Virtual Reality (AR/VR), among other areas. BCI which can decode and recognize neural signals involved in speech and handwriting has the potential to greatly assist individuals with severe motor impairments in their communication and interaction needs. Innovative and cutting-edge advancements in this field have the potential to develop a highly accessible and interactive communication platform for these people. The purpose of this review paper is to analyze the existing research on handwriting and speech recognition from neural signals. So that the new researchers who are interested in this field can gain thorough knowledge in this research area. The current research on neural signal-based recognition of handwriting and speech has been categorized into two main types: invasive and non-invasive studies. We have examined the latest papers on converting speech-activity-based neural signals and handwriting-activity-based neural signals into text data. The methods of extracting data from the brain have also been discussed in this review. Additionally, this review includes a brief summary of the datasets, preprocessing techniques, and methods used in these studies, which were published between 2014 and 2022. This review aims to provide a comprehensive summary of the methodologies used in the current literature on neural signal-based recognition of handwriting and speech. In essence, this article is intended to serve as a valuable resource for future researchers who wish to investigate neural signal-based machine-learning methods in their work.
Collapse
Affiliation(s)
- Ovishake Sen
- Department of ECE, University of Florida, Gainesville, FL 32611, USA
| | - Anna M Sheehan
- Department of ECE, University of Florida, Gainesville, FL 32611, USA
| | - Pranay R Raman
- Department of ECE, University of Florida, Gainesville, FL 32611, USA
| | - Kabir S Khara
- Department of ECE, University of Florida, Gainesville, FL 32611, USA
| | - Adam Khalifa
- Department of ECE, University of Florida, Gainesville, FL 32611, USA
| | | |
Collapse
|
5
|
Wading corvus optimization based text generation using deep CNN and BiLSTM classifiers. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103969] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
|
6
|
Lopez-Bernal D, Balderas D, Ponce P, Molina A. A State-of-the-Art Review of EEG-Based Imagined Speech Decoding. Front Hum Neurosci 2022; 16:867281. [PMID: 35558735 PMCID: PMC9086783 DOI: 10.3389/fnhum.2022.867281] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2022] [Accepted: 03/24/2022] [Indexed: 11/13/2022] Open
Abstract
Currently, the most used method to measure brain activity under a non-invasive procedure is the electroencephalogram (EEG). This is because of its high temporal resolution, ease of use, and safety. These signals can be used under a Brain Computer Interface (BCI) framework, which can be implemented to provide a new communication channel to people that are unable to speak due to motor disabilities or other neurological diseases. Nevertheless, EEG-based BCI systems have presented challenges to be implemented in real life situations for imagined speech recognition due to the difficulty to interpret EEG signals because of their low signal-to-noise ratio (SNR). As consequence, in order to help the researcher make a wise decision when approaching this problem, we offer a review article that sums the main findings of the most relevant studies on this subject since 2009. This review focuses mainly on the pre-processing, feature extraction, and classification techniques used by several authors, as well as the target vocabulary. Furthermore, we propose ideas that may be useful for future work in order to achieve a practical application of EEG-based BCI systems toward imagined speech decoding.
Collapse
Affiliation(s)
- Diego Lopez-Bernal
- Tecnologico de Monterrey, National Department of Research, Mexico City, Mexico
| | | | | | | |
Collapse
|
7
|
Aquila-eagle-based Deep Convolutional neural network for speech recognition using EEG signals. INTERNATIONAL JOURNAL OF SWARM INTELLIGENCE RESEARCH 2022. [DOI: 10.4018/ijsir.302608] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
The conventional BCI system experiences several issues such as background noise interference, lower precision rate and high cost. Hence, a novel speech recognition model which is based on the optimized Deep-CNN is proposed in this research article so as to restrain the issues related to the conventional speech recognition method. The significance of the research relies on the proposed method algorithm known as Aquila-eagle optimization, which effectively tunes the parameters of Deep-CNN. The most significant features are extracted in the feature selection process, which enhance the precision of the speech recognition model. Further unwanted noises in the EEG signals are constructively removed in the pre-processing stage to boost the accuracy of the Deep-CNN classifier.From the experimental outcomes it is demonstrated that the proposed Aquila-eagle-based DeepCNN outperformed other state-of-the-art techniques in terms of accuracy, precision, and recall with the values of 93.11%, 90.89%, and 93.11%, respectively.
Collapse
|
8
|
Abstract
The Semantic Web emerged as an extension to the traditional Web, adding meaning (semantics) to a distributed Web of structured and linked information. At its core, the concept of ontology provides the means to semantically describe and structure information, and expose it to software and human agents in a machine and human-readable form. For software agents to be realized, it is crucial to develop powerful artificial intelligence and machine-learning techniques, able to extract knowledge from information sources, and represent it in the underlying ontology. This survey aims to provide insight into key aspects of ontology-based knowledge extraction from various sources such as text, databases, and human expertise, realized in the realm of feature selection. First, common classification and feature selection algorithms are presented. Then, selected approaches, which utilize ontologies to represent features and perform feature selection and classification, are described. The selective and representative approaches span diverse application domains, such as document classification, opinion mining, manufacturing, recommendation systems, urban management, information security systems, and demonstrate the feasibility and applicability of such methods. This survey, in addition to the criteria-based presentation of related works, contributes a number of open issues and challenges related to this still active research topic.
Collapse
|