1
|
Nia AF, Tang V, Talou GM, Billinghurst M. Synthesizing affective neurophysiological signals using generative models: A review paper. J Neurosci Methods 2024; 406:110129. [PMID: 38614286 DOI: 10.1016/j.jneumeth.2024.110129] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2023] [Revised: 01/04/2024] [Accepted: 04/03/2024] [Indexed: 04/15/2024]
Abstract
The integration of emotional intelligence in machines is an important step in advancing human-computer interaction. This demands the development of reliable end-to-end emotion recognition systems. However, the scarcity of public affective datasets presents a challenge. In this literature review, we emphasize the use of generative models to address this issue in neurophysiological signals, particularly Electroencephalogram (EEG) and Functional Near-Infrared Spectroscopy (fNIRS). We provide a comprehensive analysis of different generative models used in the field, examining their input formulation, deployment strategies, and methodologies for evaluating the quality of synthesized data. This review serves as a comprehensive overview, offering insights into the advantages, challenges, and promising future directions in the application of generative models in emotion recognition systems. Through this review, we aim to facilitate the progression of neurophysiological data augmentation, thereby supporting the development of more efficient and reliable emotion recognition systems.
Collapse
Affiliation(s)
- Alireza F Nia
- Auckland Bioengineering Institute, 70 Symonds Street, Auckland, 1010, New Zealand.
| | - Vanessa Tang
- Auckland Bioengineering Institute, 70 Symonds Street, Auckland, 1010, New Zealand
| | - Gonzalo Maso Talou
- Auckland Bioengineering Institute, 70 Symonds Street, Auckland, 1010, New Zealand
| | - Mark Billinghurst
- Auckland Bioengineering Institute, 70 Symonds Street, Auckland, 1010, New Zealand
| |
Collapse
|
2
|
Kotikam G, Selvaraj L. Golden eagle based improved Att-BiLSTM model for big data classification with hybrid feature extraction and feature selection techniques. NETWORK (BRISTOL, ENGLAND) 2024; 35:154-189. [PMID: 38155542 DOI: 10.1080/0954898x.2023.2293895] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Accepted: 12/06/2023] [Indexed: 12/30/2023]
Abstract
The remarkable development in technology has led to the increase of massive big data. Machine learning processes provide a way for investigators to examine and particularly classify big data. Besides, several machine learning models rely on powerful feature extraction and feature selection techniques for their success. In this paper, a big data classification approach is developed using an optimized deep learning classifier integrated with hybrid feature extraction and feature selection approaches. The proposed technique uses local linear embedding-based kernel principal component analysis and perturbation theory, respectively, to extract more representative data and select the appropriate features from the big data environment. In addition, the feature selection task is fine-tuned by using perturbation theory through heuristic search based on their output accuracy. This feature selection heuristic search method is analysed with five recent heuristic optimization algorithms for deciding the final feature subset. Finally, the data are categorized through an attention-based bidirectional long short-term memory classifier that is optimized with a golden eagle-inspired algorithm. The performance of the proposed model is experimentally verified on publicly accessible datasets. From the experimental outcomes, it is demonstrated that the proposed framework is capable of classifying large datasets with more than 90% accuracy.
Collapse
Affiliation(s)
- Gnanendra Kotikam
- Research Scholar, Department of Information and Communication Engineering, Anna University, Chennai, India
| | - Lokesh Selvaraj
- Department of Computer Science and Engineering, PSG Institute of Technology and Applied Research, Coimbatore, India
| |
Collapse
|
3
|
Wu M, Lin S, Xiao C, Xiao X, Xu S, Yu S. The emotion prediction of college students with attention LSTM during the COVID19 epidemic. Sci Rep 2023; 13:22825. [PMID: 38129509 PMCID: PMC10739690 DOI: 10.1038/s41598-023-50322-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2023] [Accepted: 12/18/2023] [Indexed: 12/23/2023] Open
Abstract
During the COVID19 pandemic, there is a pronounced collective mental health issue among college students. Forecasting the trend of emotional changes in on-campus students is crucial to effectively address this issue. This study proposes an Attention-LSTM neural network model that performs deep learning on key input sequence information, so as to predict the distribution of emotional states in college students. By testing 60 consecutive days of emotional data, the model successfully predicts students' emotional distribution, triggers and resolution strategies, with an accuracy rate of no less than 99%. Compared with models such as ARIMA, SARIMA and VAR, this model shows significant advantages in accuracy, operational efficiency, and data collection requirements. The integration of deep learning technology with student management in this study offers a novel approach to address emotional issues among students under exceptional circumstances.
Collapse
Affiliation(s)
- Mengwei Wu
- College of Mechanical and Intelligent Manufacturing, Fujian Chuanzheng Communications College, Fuzhou, 350007, China
| | - Shaodan Lin
- College of Mechanical and Intelligent Manufacturing, Fujian Chuanzheng Communications College, Fuzhou, 350007, China.
| | - Chenhan Xiao
- College of Mechanical and Intelligent Manufacturing, Fujian Chuanzheng Communications College, Fuzhou, 350007, China
| | - Xiulin Xiao
- College of Mechanical and Intelligent Manufacturing, Fujian Chuanzheng Communications College, Fuzhou, 350007, China
| | - Siwei Xu
- College of Mechanical and Intelligent Manufacturing, Fujian Chuanzheng Communications College, Fuzhou, 350007, China
| | - Shuhan Yu
- College of Mechanical and Intelligent Manufacturing, Fujian Chuanzheng Communications College, Fuzhou, 350007, China
| |
Collapse
|
4
|
Li M, Zeng X, Wu F, Chu Y, Wei W, Fan M, Pang C, Hu X. Build a bridge between ECG and EEG signals for atrial fibrillation diagnosis using AI methods. Comput Biol Med 2023; 166:107429. [PMID: 37734354 DOI: 10.1016/j.compbiomed.2023.107429] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2023] [Revised: 07/07/2023] [Accepted: 08/28/2023] [Indexed: 09/23/2023]
Abstract
Atrial fibrillation (AF) is a very common type of cardiac arrhythmia. The main characteristic of AF is an abnormally rapid and disordered atrial rhythm causing an atrial dysfunction, which can be visualized on an electrocardiograph (ECG) and distinguished by irregular fluctuations. Despite continuous and considerable efforts to analyze the pathophysiology of AF, it is challenging to determine the underlying pathogenesis of the disease in individual patients. This study aims to build a bridge between ECG and electroencephalogram (EEG) signals to probe the strong influence between human brain activity and AF by AI methods. We first found that the one-second data fragment shows the most excellent performance in our time window configuration. Secondly, in our proposed measurement, most cortical potentials were partly associated with AF. Thirdly, we found that only a few channels of data were sufficient for analysis. Finally, our experiment shows δ wave has the best performance compared to other wave bands. By AI methods, the paper contributes to concluding that δ wave band of EEG is the most associated brain wave type with AF. By EEG signals from typical regions, the central region, parietal and Occipital might be the most associated encephalic regions with AF. The clinical trial registration number for our study is ChiCTR2300068625.
Collapse
Affiliation(s)
- Moqing Li
- Academy for Engineering and Technology, Fudan University, No. 220, Handan Rd, Yangpu District, Shanghai, 200433, China.
| | - Xinhua Zeng
- Academy for Engineering and Technology, Fudan University, No. 220, Handan Rd, Yangpu District, Shanghai, 200433, China.
| | - Feng Wu
- Department of Cardiology, Yueyang Hospital of Integrated Traditional Chinese and Western Medicine, Shanghai University of Traditional Chinese Medicine, No. 110, Ganhe Rd, Hongkou District, Shanghai, 200437, China.
| | - Yang Chu
- Department of Cardiology, Yueyang Hospital of Integrated Traditional Chinese and Western Medicine, Shanghai University of Traditional Chinese Medicine, No. 110, Ganhe Rd, Hongkou District, Shanghai, 200437, China.
| | - Weiguo Wei
- Department of Cardiology, Yueyang Hospital of Integrated Traditional Chinese and Western Medicine, Shanghai University of Traditional Chinese Medicine, No. 110, Ganhe Rd, Hongkou District, Shanghai, 200437, China.
| | - Min Fan
- Department of Cardiology, Yueyang Hospital of Integrated Traditional Chinese and Western Medicine, Shanghai University of Traditional Chinese Medicine, No. 110, Ganhe Rd, Hongkou District, Shanghai, 200437, China.
| | - Chengxin Pang
- School of Electronics and Information Engineering, Shanghai University of Electric Power, No. 1851, Hucheng Ring Rd, Pudong New Area, Shanghai, 201306, China.
| | - Xing Hu
- Optical Electrical and Computer Engineering, University of Shanghai for Science and Technology, No. 516, Jungong Rd, Yangpu District, Shanghai, 200093, China.
| |
Collapse
|
5
|
Multilocation and Multiscale Learning Framework with Skip Connection for Fault Diagnosis of Bearing under Complex Working Conditions. SENSORS 2021; 21:s21093226. [PMID: 34066598 PMCID: PMC8124480 DOI: 10.3390/s21093226] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/20/2021] [Revised: 04/29/2021] [Accepted: 05/02/2021] [Indexed: 12/03/2022]
Abstract
Considering various fault states under severe working conditions, the comprehensive feature extraction from the raw vibration signal is still a challenge for the diagnosis task of rolling bearing. To deal with strong coupling and high nonlinearity of the vibration signal, this article proposes a novel multilocation and multikernel scale learning framework based on deep convolution encoder (DCE) and bidirectional long short-term memory network (BiLSTM). The procedure of the proposed method using a cascade structure is developed in three stages. In the first stage, each parallel branch of the multifeature learning combines the skip connection and the DCE, and uses different size kernels. The multifeature learning network can automatically extract and fuse global and local features from different network depths and time scales of the raw vibration signal. In the second stage, the BiLSTM as the feature protection network is designed to employ the internal calculating data of the forward propagation and backward propagation at the same network propagation node. The feature protection network is used for further mining sensitive and complementary features. In the third stage of bearing diagnosis, the classifier identifies the fault types. Consequently, the proposed network scheme can perform well in generalization capability. The performance of the proposed method is verified on the two kinds of bearing datasets. The diagnostic results demonstrate that the proposed method can diagnose multiple fault types more accurately. Also, the method performs better in load and speed adaptation compared with other intelligent fault classification methods.
Collapse
|
6
|
Alzahab NA, Apollonio L, Di Iorio A, Alshalak M, Iarlori S, Ferracuti F, Monteriù A, Porcaro C. Hybrid Deep Learning (hDL)-Based Brain-Computer Interface (BCI) Systems: A Systematic Review. Brain Sci 2021; 11:75. [PMID: 33429938 PMCID: PMC7827826 DOI: 10.3390/brainsci11010075] [Citation(s) in RCA: 32] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2020] [Revised: 12/12/2020] [Accepted: 01/04/2021] [Indexed: 01/15/2023] Open
Abstract
BACKGROUND Brain-Computer Interface (BCI) is becoming more reliable, thanks to the advantages of Artificial Intelligence (AI). Recently, hybrid Deep Learning (hDL), which combines different DL algorithms, has gained momentum over the past five years. In this work, we proposed a review on hDL-based BCI starting from the seminal studies in 2015. OBJECTIVES We have reviewed 47 papers that apply hDL to the BCI system published between 2015 and 2020 extracting trends and highlighting relevant aspects to the topic. METHODS We have queried four scientific search engines (Google Scholar, PubMed, IEEE Xplore and Elsevier Science Direct) and different data items were extracted from each paper such as the database used, kind of application, online/offline training, tasks used for the BCI, pre-processing methodology adopted, type of normalization used, which kind of features were extracted, type of DL architecture used, number of layers implemented and which optimization approach were used as well. All these items were then investigated one by one to uncover trends. RESULTS Our investigation reveals that Electroencephalography (EEG) has been the most used technique. Interestingly, despite the lower Signal-to-Noise Ratio (SNR) of the EEG data that makes pre-processing of that data mandatory, we have found that the pre-processing has only been used in 21.28% of the cases by showing that hDL seems to be able to overcome this intrinsic drawback of the EEG data. Temporal-features seem to be the most effective with 93.94% accuracy, while spatial-temporal features are the most used with 33.33% of the cases investigated. The most used architecture has been Convolutional Neural Network-Recurrent Neural Network CNN-RNN with 47% of the cases. Moreover, half of the studies have used a low number of layers to achieve a good compromise between the complexity of the network and computational efficiency. SIGNIFICANCE To give useful information to the scientific community, we make our summary table of hDL-based BCI papers available and invite the community to published work to contribute to it directly. We have indicated a list of open challenges, emphasizing the need to use neuroimaging techniques other than EEG, such as functional Near-Infrared Spectroscopy (fNIRS), deeper investigate the advantages and disadvantages of using pre-processing and the relationship with the accuracy obtained. To implement new combinations of architectures, such as RNN-based and Deep Belief Network DBN-based, it is necessary to better explore the frequency and temporal-frequency features of the data at hand.
Collapse
Affiliation(s)
- Nibras Abo Alzahab
- Department of Information Engineering, Università Politecnica delle Marche, Via Brecce Bianche 12, 60131 Ancona, Italy; (N.A.A.); (L.A.); (A.D.I.); (M.A.); (S.I.); (F.F.); (A.M.)
| | - Luca Apollonio
- Department of Information Engineering, Università Politecnica delle Marche, Via Brecce Bianche 12, 60131 Ancona, Italy; (N.A.A.); (L.A.); (A.D.I.); (M.A.); (S.I.); (F.F.); (A.M.)
| | - Angelo Di Iorio
- Department of Information Engineering, Università Politecnica delle Marche, Via Brecce Bianche 12, 60131 Ancona, Italy; (N.A.A.); (L.A.); (A.D.I.); (M.A.); (S.I.); (F.F.); (A.M.)
| | - Muaaz Alshalak
- Department of Information Engineering, Università Politecnica delle Marche, Via Brecce Bianche 12, 60131 Ancona, Italy; (N.A.A.); (L.A.); (A.D.I.); (M.A.); (S.I.); (F.F.); (A.M.)
| | - Sabrina Iarlori
- Department of Information Engineering, Università Politecnica delle Marche, Via Brecce Bianche 12, 60131 Ancona, Italy; (N.A.A.); (L.A.); (A.D.I.); (M.A.); (S.I.); (F.F.); (A.M.)
| | - Francesco Ferracuti
- Department of Information Engineering, Università Politecnica delle Marche, Via Brecce Bianche 12, 60131 Ancona, Italy; (N.A.A.); (L.A.); (A.D.I.); (M.A.); (S.I.); (F.F.); (A.M.)
| | - Andrea Monteriù
- Department of Information Engineering, Università Politecnica delle Marche, Via Brecce Bianche 12, 60131 Ancona, Italy; (N.A.A.); (L.A.); (A.D.I.); (M.A.); (S.I.); (F.F.); (A.M.)
| | - Camillo Porcaro
- Department of Information Engineering, Università Politecnica delle Marche, Via Brecce Bianche 12, 60131 Ancona, Italy; (N.A.A.); (L.A.); (A.D.I.); (M.A.); (S.I.); (F.F.); (A.M.)
- Institute of Cognitive Sciences and Technologies (ISTC)—National Research Council (CNR), 00185 Rome, Italy
- S. Anna Institute and Research in Advanced Neurorehabilitation (RAN), 88900 Crotone, Italy
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham B15 2TT, UK
- Research Center for Motor Control and Neuroplasticity, KU Leuven, 3000 Leuven, Belgium
| |
Collapse
|