1
|
Çelebi M, Öztürk S, Kaplan K. An emotion recognition method based on EWT-3D-CNN-BiLSTM-GRU-AT model. Comput Biol Med 2024; 169:107954. [PMID: 38183705 DOI: 10.1016/j.compbiomed.2024.107954] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2023] [Revised: 12/28/2023] [Accepted: 01/01/2024] [Indexed: 01/08/2024]
Abstract
This has become a significant study area in recent years because of its use in brain-machine interaction (BMI). The robustness problem of emotion classification is one of the most basic approaches for improving the quality of emotion recognition systems. One of the two main branches of these approaches deals with the problem by extracting the features using manual engineering and the other is the famous artificial intelligence approach, which infers features of EEG data. This study proposes a novel method that considers the characteristic behavior of EEG recordings and based on the artificial intelligence method. The EEG signal is a noisy signal with a non-stationary and non-linear form. Using the Empirical Wavelet Transform (EWT) signal decomposition method, the signal's frequency components are obtained. Then, frequency-based features, linear and non-linear features are extracted. The resulting frequency-based, linear, and nonlinear features are mapped to the 2-D axis according to the positions of the EEG electrodes. By merging this 2-D images, 3-D images are constructed. In this way, the multichannel brain frequency of EEG recordings, spatial and temporal relationship are combined. Lastly, 3-D deep learning framework was constructed, which was combined with convolutional neural network (CNN), bidirectional long-short term memory (BiLSTM) and gated recurrent unit (GRU) with self-attention (AT). This model is named EWT-3D-CNN-BiLSTM-GRU-AT. As a result, we have created framework comprising handcrafted features generated and cascaded from state-of-the-art deep learning models. The framework is evaluated on the DEAP recordings based on the person-independent approach. The experimental findings demonstrate that the developed model can achieve classification accuracies of 90.57 % and 90.59 % for valence and arousal axes, respectively, for the DEAP database. Compared with existing cutting-edge emotion classification models, the proposed framework exhibits superior results for classifying human emotions.
Collapse
Affiliation(s)
- Muharrem Çelebi
- Electronics and Communication Engineering, Kocaeli University, Kocaeli, 41001, Turkey.
| | - Sıtkı Öztürk
- Electronics and Communication Engineering, Kocaeli University, Kocaeli, 41001, Turkey.
| | - Kaplan Kaplan
- Software Engineering, Kocaeli University, Kocaeli, 41001, Turkey.
| |
Collapse
|
2
|
Chen Z, Yang R, Huang M, Li F, Lu G, Wang Z. EEGProgress: A fast and lightweight progressive convolution architecture for EEG classification. Comput Biol Med 2024; 169:107901. [PMID: 38159400 DOI: 10.1016/j.compbiomed.2023.107901] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Revised: 12/11/2023] [Accepted: 12/23/2023] [Indexed: 01/03/2024]
Abstract
Because of the intricate topological structure and connection of the human brain, extracting deep spatial features from electroencephalograph (EEG) signals is a challenging and time-consuming task. The extraction of topological spatial information plays a crucial role in EEG classification, and the architecture of the spatial convolution greatly affects the performance and complexity of convolutional neural network (CNN) based EEG classification models. In this study, a progressive convolution CNN architecture named EEGProgress is proposed, aiming to efficiently extract the topological spatial information of EEG signals from multi-scale levels (electrode, brain region, hemisphere, global) with superior speed. To achieve this, the raw EEG data is permuted using the empirical topological permutation rule, integrating the EEG data with numerous topological properties. Subsequently, the spatial features are extracted by a progressive feature extractor including prior, electrode, region, and hemisphere convolution blocks, progressively extracting the deep spatial features with reduced parameters and speed. Finally, the comparison and ablation experiments under both cross-subject and within-subject scenarios are conducted on a public dataset to verify the performance of the proposed EEGProgress and the effectiveness of the topological permutation. The results demonstrate the superior feature extraction ability of the proposed EEGProgress, with an average increase of 4.02% compared to other CNN-based EEG classification models under both cross-subject and within-subject scenarios. Furthermore, with the obtained average testing time, FLOPs, and parameters, the proposed EEGProgress outperforms other comparison models in terms of model complexity.
Collapse
Affiliation(s)
- Zhige Chen
- School of Advanced Technology, Xi'an Jiaotong-Liverpool University, Suzhou 215123, China; School of Electrical Engineering, Electronics and Computer Science, University of Liverpool, Liverpool L69 3BX, United Kingdom
| | - Rui Yang
- School of Advanced Technology, Xi'an Jiaotong-Liverpool University, Suzhou 215123, China
| | - Mengjie Huang
- Design School, Xi'an Jiaotong-Liverpool University, Suzhou 215123, China.
| | - Fumin Li
- School of Advanced Technology, Xi'an Jiaotong-Liverpool University, Suzhou 215123, China; School of Electrical Engineering, Electronics and Computer Science, University of Liverpool, Liverpool L69 3BX, United Kingdom
| | - Guoping Lu
- School of Electrical Engineering, Nantong University, Nantong 226019, China
| | - Zidong Wang
- Department of Computer Science, Brunel University London, Uxbridge, Middlesex UB8 3PH, United Kingdom
| |
Collapse
|
3
|
Liu Z, Li H, Li W, Zhang F, Ouyang W, Wang S, Zhi A, Pan X. Development of an Expert-Level Right Ventricular Abnormality Detection Algorithm Based on Deep Learning. Interdiscip Sci 2023; 15:653-662. [PMID: 37470945 DOI: 10.1007/s12539-023-00581-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Revised: 07/06/2023] [Accepted: 07/10/2023] [Indexed: 07/21/2023]
Abstract
PURPOSE Studies relating to the right ventricle (RV) are inadequate, and specific diagnostic algorithms still need to be improved. This essay is designed to make exploration and verification on an algorithm of deep learning based on imaging and clinical data to detect RV abnormalities. METHODS The Automated Cardiac Diagnosis Challenge dataset includes 20 subjects with RV abnormalities (an RV cavity volume which is higher than 110 mL/m2 or RV ejection fraction which is lower than 40%) and 20 normal subjects who suffered from both cardiac MRI. The subjects were separated into training and validation sets in a ratio of 7:3 and were modeled by utilizing a nerve net of deep-learning and six machine-learning algorithms. Eight MRI specialists from multiple centers independently determined whether each subject in the validation group had RV abnormalities. Model performance was evaluated based on the AUC, accuracy, recall, sensitivity and specificity. Furthermore, a preliminary assessment of patient disease risk was performed based on clinical information using a nomogram. RESULTS The deep-learning neural network outperformed the other six machine-learning algorithms, with an AUC value of 1 (95% confidence interval: 1-1) on both training group and validation group. This algorithm surpassed most human experts (87.5%). In addition, the nomogram model could evaluate a population with a disease risk of 0.2-0.8. CONCLUSIONS A deep-learning algorithm could effectively identify patients with RV abnormalities. This AI algorithm developed specifically for right ventricular abnormalities will improve the detection of right ventricular abnormalities at all levels of care units and facilitate the timely diagnosis and treatment of related diseases. In addition, this study is the first to validate the algorithm's ability to classify RV abnormalities by comparing it with human experts.
Collapse
Affiliation(s)
- Zeye Liu
- Department of Structural Heart Disease, National Center for Cardiovascular Disease, China and Fuwai Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100037, China
- National Health Commission Key Laboratory of Cardiovascular Regeneration Medicine, Beijing, 100037, China
- Key Laboratory of Innovative Cardiovascular Devices, Chinese Academy of Medical Sciences, Beijing, 100037, China
- National Clinical Research Center for Cardiovascular Diseases, Fuwai Hospital, Chinese Academy of Medical Sciences, Beijing, 100037, China
| | - Hang Li
- Department of Structural Heart Disease, National Center for Cardiovascular Disease, China and Fuwai Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100037, China
- National Health Commission Key Laboratory of Cardiovascular Regeneration Medicine, Beijing, 100037, China
- Key Laboratory of Innovative Cardiovascular Devices, Chinese Academy of Medical Sciences, Beijing, 100037, China
- National Clinical Research Center for Cardiovascular Diseases, Fuwai Hospital, Chinese Academy of Medical Sciences, Beijing, 100037, China
| | - Wenchao Li
- Pediatric Cardiac Surgery, Henan Provincial People's Hospital, Huazhong Fuwai Hospital, Zhengzhou University People's Hospital, Zhengzhou, 450000, China
| | - Fengwen Zhang
- Department of Structural Heart Disease, National Center for Cardiovascular Disease, China and Fuwai Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100037, China
- National Health Commission Key Laboratory of Cardiovascular Regeneration Medicine, Beijing, 100037, China
- Key Laboratory of Innovative Cardiovascular Devices, Chinese Academy of Medical Sciences, Beijing, 100037, China
- National Clinical Research Center for Cardiovascular Diseases, Fuwai Hospital, Chinese Academy of Medical Sciences, Beijing, 100037, China
| | - Wenbin Ouyang
- Department of Structural Heart Disease, National Center for Cardiovascular Disease, China and Fuwai Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100037, China
- National Health Commission Key Laboratory of Cardiovascular Regeneration Medicine, Beijing, 100037, China
- Key Laboratory of Innovative Cardiovascular Devices, Chinese Academy of Medical Sciences, Beijing, 100037, China
- National Clinical Research Center for Cardiovascular Diseases, Fuwai Hospital, Chinese Academy of Medical Sciences, Beijing, 100037, China
| | - Shouzheng Wang
- Department of Structural Heart Disease, National Center for Cardiovascular Disease, China and Fuwai Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100037, China
- National Health Commission Key Laboratory of Cardiovascular Regeneration Medicine, Beijing, 100037, China
- Key Laboratory of Innovative Cardiovascular Devices, Chinese Academy of Medical Sciences, Beijing, 100037, China
- National Clinical Research Center for Cardiovascular Diseases, Fuwai Hospital, Chinese Academy of Medical Sciences, Beijing, 100037, China
| | - Aihua Zhi
- Department of Medical Imaging, Fuwai Yunnan Cardiovascular Hospital, Kunming, 650000, China
| | - Xiangbin Pan
- Department of Structural Heart Disease, National Center for Cardiovascular Disease, China and Fuwai Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100037, China.
- National Health Commission Key Laboratory of Cardiovascular Regeneration Medicine, Beijing, 100037, China.
- Key Laboratory of Innovative Cardiovascular Devices, Chinese Academy of Medical Sciences, Beijing, 100037, China.
- National Clinical Research Center for Cardiovascular Diseases, Fuwai Hospital, Chinese Academy of Medical Sciences, Beijing, 100037, China.
| |
Collapse
|
4
|
Zong J, Xiong X, Zhou J, Ji Y, Zhou D, Zhang Q. FCAN-XGBoost: A Novel Hybrid Model for EEG Emotion Recognition. SENSORS (BASEL, SWITZERLAND) 2023; 23:5680. [PMID: 37420845 DOI: 10.3390/s23125680] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/11/2023] [Revised: 06/03/2023] [Accepted: 06/15/2023] [Indexed: 07/09/2023]
Abstract
In recent years, artificial intelligence (AI) technology has promoted the development of electroencephalogram (EEG) emotion recognition. However, existing methods often overlook the computational cost of EEG emotion recognition, and there is still room for improvement in the accuracy of EEG emotion recognition. In this study, we propose a novel EEG emotion recognition algorithm called FCAN-XGBoost, which is a fusion of two algorithms, FCAN and XGBoost. The FCAN module is a feature attention network (FANet) that we have proposed for the first time, which processes the differential entropy (DE) and power spectral density (PSD) features extracted from the four frequency bands of the EEG signal and performs feature fusion and deep feature extraction. Finally, the deep features are fed into the eXtreme Gradient Boosting (XGBoost) algorithm to classify the four emotions. We evaluated the proposed method on the DEAP and DREAMER datasets and achieved a four-category emotion recognition accuracy of 95.26% and 94.05%, respectively. Additionally, our proposed method reduces the computational cost of EEG emotion recognition by at least 75.45% for computation time and 67.51% for memory occupation. The performance of FCAN-XGBoost outperforms the state-of-the-art four-category model and reduces computational costs without losing classification performance compared with other models.
Collapse
Affiliation(s)
- Jing Zong
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China
| | - Xin Xiong
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China
| | - Jianhua Zhou
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China
| | - Ying Ji
- Graduate School, Kunming Medical University, Kunming 650500, China
| | - Diao Zhou
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China
| | - Qi Zhang
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China
| |
Collapse
|
5
|
Hou HR, Han RX, Zhang XN, Meng QH. Pleasantness Recognition Induced by Different Odor Concentrations Using Olfactory Electroencephalogram Signals. SENSORS (BASEL, SWITZERLAND) 2022; 22:8808. [PMID: 36433405 PMCID: PMC9695492 DOI: 10.3390/s22228808] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/11/2022] [Revised: 11/08/2022] [Accepted: 11/11/2022] [Indexed: 06/16/2023]
Abstract
Olfactory-induced emotion plays an important role in communication, decision-making, multimedia, and disorder treatment. Using electroencephalogram (EEG) technology, this paper focuses on (1) exploring the possibility of recognizing pleasantness induced by different concentrations of odors, (2) finding the EEG rhythm wave that is most suitable for the recognition of different odor concentrations, (3) analyzing recognition accuracies with concentration changes, and (4) selecting a suitable classifier for this classification task. To explore these issues, first, emotions induced by five different concentrations of rose or rotten odors are divided into five kinds of pleasantness by averaging subjective evaluation scores. Then, the power spectral density features of EEG signals and support vector machine (SVM) are used for classification tasks. Classification results on the EEG signals collected from 13 participants show that for pleasantness recognition induced by pleasant or disgusting odor concentrations, considerable average classification accuracies of 93.5% or 92.2% are obtained, respectively. The results indicate that (1) using EEG technology, pleasantness recognition induced by different odor concentrations is possible; (2) gamma frequency band outperformed other EEG rhythm-based frequency bands in terms of classification accuracy, and as the maximum frequency of the EEG spectrum increases, the pleasantness classification accuracy gradually increases; (3) for both rose and rotten odors, the highest concentration obtains the best classification accuracy, followed by the lowest concentration.
Collapse
Affiliation(s)
- Hui-Rang Hou
- Tianjin Key Laboratory of Process Measurement and Control, Institute of Robotics and Autonomous Systems, School of Electrical and Information Engineering, Tianjin University, Tianjin 300350, China
| | - Rui-Xue Han
- Tianjin Navigation Instruments Research Institute, Tianjin 300131, China
| | - Xiao-Nei Zhang
- Tianjin Key Laboratory of Process Measurement and Control, Institute of Robotics and Autonomous Systems, School of Electrical and Information Engineering, Tianjin University, Tianjin 300350, China
| | - Qing-Hao Meng
- Tianjin Key Laboratory of Process Measurement and Control, Institute of Robotics and Autonomous Systems, School of Electrical and Information Engineering, Tianjin University, Tianjin 300350, China
| |
Collapse
|