1
|
Pang B, Peng Y, Gao J, Kong W. Semi-supervised bipartite graph construction with active EEG sample selection for emotion recognition. Med Biol Eng Comput 2024; 62:2805-2824. [PMID: 38700614 DOI: 10.1007/s11517-024-03094-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2023] [Accepted: 04/10/2024] [Indexed: 08/18/2024]
Abstract
Electroencephalogram (EEG) signals are derived from the central nervous system and inherently difficult to camouflage, leading to the recent popularity of EEG-based emotion recognition. However, due to the non-stationary nature of EEG, inter-subject variabilities become obstacles for recognition models to well adapt to different subjects. In this paper, we propose a novel approach called semi-supervised bipartite graph construction with active EEG sample selection (SBGASS) for cross-subject emotion recognition, which offers two significant advantages. Firstly, SBGASS adaptively learns a bipartite graph to characterize the underlying relationships between labeled and unlabeled EEG samples, effectively implementing the semantic connection for samples from different subjects. Secondly, we employ active sample selection technique in this paper to reduce the impact of negative samples (outliers or noise in the data) on bipartite graph construction. Drawing from the experimental results with the SEED-IV data set, we have gained the following three insights. (1) SBGASS actively rejects negative labeled samples, which helps mitigate the impact of negative samples when constructing the optimal bipartite graph and improves the model performance. (2) Through the learned optimal bipartite graph in SBGASS, the transferability of labeled EEG samples is quantitatively analyzed, which exhibits a decreasing tendency as the distance between each labeled sample and the corresponding class centroid increases. (3) Besides the improved recognition accuracy, the spatial-frequency patterns in emotion recognition are investigated by the acquired projection matrix.
Collapse
Affiliation(s)
- Bowen Pang
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, 310018, Zhejiang, China
| | - Yong Peng
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, 310018, Zhejiang, China.
- Zhejiang Key Laboratory of Brain-Machine Collaborative Intelligence, Hangzhou, 310018, Zhejiang, China.
| | - Jian Gao
- Department of Rehabilitation, Hangzhou Mingzhou Brain Rehabilitation Hospital, Hangzhou, 311215, Zhejiang, China
| | - Wanzeng Kong
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, 310018, Zhejiang, China
- Zhejiang Key Laboratory of Brain-Machine Collaborative Intelligence, Hangzhou, 310018, Zhejiang, China
| |
Collapse
|
2
|
Mobbs A, Kahn M, Williams G, Mentiplay BF, Pua YH, Clark RA. Machine learning for automating subjective clinical assessment of gait impairment in people with acquired brain injury - a comparison of an image extraction and classification system to expert scoring. J Neuroeng Rehabil 2024; 21:124. [PMID: 39039594 PMCID: PMC11264460 DOI: 10.1186/s12984-024-01406-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2024] [Accepted: 06/14/2024] [Indexed: 07/24/2024] Open
Abstract
BACKGROUND Walking impairment is a common disability post acquired brain injury (ABI), with visually evident arm movement abnormality identified as negatively impacting a multitude of psychological factors. The International Classification of Functioning, Disability and Health (ICF) qualifiers scale has been used to subjectively assess arm movement abnormality, showing strong intra-rater and test-retest reliability, however, only moderate inter-rater reliability. This impacts clinical utility, limiting its use as a measurement tool. To both automate the analysis and overcome these errors, the primary aim of this study was to evaluate the ability of a novel two-level machine learning model to assess arm movement abnormality during walking in people with ABI. METHODS Frontal plane gait videos were used to train four networks with 50%, 75%, 90%, and 100% of participants (ABI: n = 42, healthy controls: n = 34) to automatically identify anatomical landmarks using DeepLabCut™ and calculate two-dimensional kinematic joint angles. Assessment scores from three experienced neurorehabilitation clinicians were used with these joint angles to train random forest networks with nested cross-validation to predict assessor scores for all videos. Agreement between unseen participant (i.e. test group participants that were not used to train the model) predictions and each individual assessor's scores were compared using quadratic weighted kappa. One sample t-tests (to determine over/underprediction against clinician ratings) and one-way ANOVA (to determine differences between networks) were applied to the four networks. RESULTS The machine learning predictions have similar agreement to experienced human assessors, with no statistically significant (p < 0.05) difference for any match contingency. There was no statistically significant difference between the predictions from the four networks (F = 0.119; p = 0.949). The four networks did however under-predict scores with small effect sizes (p range = 0.007 to 0.040; Cohen's d range = 0.156 to 0.217). CONCLUSIONS This study demonstrated that machine learning can perform similarly to experienced clinicians when subjectively assessing arm movement abnormality in people with ABI. The relatively small sample size may have resulted in under-prediction of some scores, albeit with small effect sizes. Studies with larger sample sizes that objectively and automatically assess dynamic movement in both local and telerehabilitation assessments, for example using smartphones and edge-based machine learning, to reduce measurement error and healthcare access inequality are needed.
Collapse
Affiliation(s)
- Ashleigh Mobbs
- School of Health, University of the Sunshine Coast, Sippy Downs, QLD, Australia
| | - Michelle Kahn
- Department of Physiotherapy, Epworth Healthcare, Richmond, VIC, Australia
| | - Gavin Williams
- Department of Physiotherapy, Epworth Healthcare, Richmond, VIC, Australia
- School of Health Sciences, University of Melbourne, Parkville, VIC, Australia
| | - Benjamin F Mentiplay
- School of Allied Health, Human Services and Sport, La Trobe University, Bundoora, VIC, Australia
| | - Yong-Hao Pua
- Department of Physiotherapy, Singapore General Hospital, Singapore, Singapore
- Duke-National University of Singapore Medical School, Singapore, Singapore
| | - Ross A Clark
- School of Health, University of the Sunshine Coast, Sippy Downs, QLD, Australia.
| |
Collapse
|
3
|
Trabassi D, Castiglia SF, Bini F, Marinozzi F, Ajoudani A, Lorenzini M, Chini G, Varrecchia T, Ranavolo A, De Icco R, Casali C, Serrao M. Optimizing Rare Disease Gait Classification through Data Balancing and Generative AI: Insights from Hereditary Cerebellar Ataxia. SENSORS (BASEL, SWITZERLAND) 2024; 24:3613. [PMID: 38894404 PMCID: PMC11175240 DOI: 10.3390/s24113613] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/09/2024] [Revised: 05/28/2024] [Accepted: 05/31/2024] [Indexed: 06/21/2024]
Abstract
The interpretability of gait analysis studies in people with rare diseases, such as those with primary hereditary cerebellar ataxia (pwCA), is frequently limited by the small sample sizes and unbalanced datasets. The purpose of this study was to assess the effectiveness of data balancing and generative artificial intelligence (AI) algorithms in generating synthetic data reflecting the actual gait abnormalities of pwCA. Gait data of 30 pwCA (age: 51.6 ± 12.2 years; 13 females, 17 males) and 100 healthy subjects (age: 57.1 ± 10.4; 60 females, 40 males) were collected at the lumbar level with an inertial measurement unit. Subsampling, oversampling, synthetic minority oversampling, generative adversarial networks, and conditional tabular generative adversarial networks (ctGAN) were applied to generate datasets to be input to a random forest classifier. Consistency and explainability metrics were also calculated to assess the coherence of the generated dataset with known gait abnormalities of pwCA. ctGAN significantly improved the classification performance compared with the original dataset and traditional data augmentation methods. ctGAN are effective methods for balancing tabular datasets from populations with rare diseases, owing to their ability to improve diagnostic models with consistent explainability.
Collapse
Affiliation(s)
- Dante Trabassi
- Department of Medical and Surgical Sciences and Biotechnologies, “Sapienza” University of Rome, 04100 Latina, Italy; (D.T.); (C.C.); (M.S.)
| | - Stefano Filippo Castiglia
- Department of Medical and Surgical Sciences and Biotechnologies, “Sapienza” University of Rome, 04100 Latina, Italy; (D.T.); (C.C.); (M.S.)
- Department of Brain and Behavioral Sciences, University of Pavia, 27100 Pavia, Italy;
| | - Fabiano Bini
- Department of Mechanical and Aerospace Engineering, Sapienza University of Rome, 00184 Rome, Italy; (F.B.); (F.M.)
| | - Franco Marinozzi
- Department of Mechanical and Aerospace Engineering, Sapienza University of Rome, 00184 Rome, Italy; (F.B.); (F.M.)
| | - Arash Ajoudani
- Department of Advanced Robotics, Italian Institute of Technology, 16163 Genoa, Italy; (A.A.); (M.L.)
| | - Marta Lorenzini
- Department of Advanced Robotics, Italian Institute of Technology, 16163 Genoa, Italy; (A.A.); (M.L.)
| | - Giorgia Chini
- Department of Occupational and Environmental Medicine, Epidemiology and Hygiene, INAIL, Monte Porzio Catone, 00078 Rome, Italy; (G.C.); (T.V.); (A.R.)
| | - Tiwana Varrecchia
- Department of Occupational and Environmental Medicine, Epidemiology and Hygiene, INAIL, Monte Porzio Catone, 00078 Rome, Italy; (G.C.); (T.V.); (A.R.)
| | - Alberto Ranavolo
- Department of Occupational and Environmental Medicine, Epidemiology and Hygiene, INAIL, Monte Porzio Catone, 00078 Rome, Italy; (G.C.); (T.V.); (A.R.)
| | - Roberto De Icco
- Department of Brain and Behavioral Sciences, University of Pavia, 27100 Pavia, Italy;
- Headache Science & Neurorehabilitation Unit, IRCCS Mondino Foundation, 27100 Pavia, Italy
| | - Carlo Casali
- Department of Medical and Surgical Sciences and Biotechnologies, “Sapienza” University of Rome, 04100 Latina, Italy; (D.T.); (C.C.); (M.S.)
| | - Mariano Serrao
- Department of Medical and Surgical Sciences and Biotechnologies, “Sapienza” University of Rome, 04100 Latina, Italy; (D.T.); (C.C.); (M.S.)
- Movement Analysis Laboratory, Policlinico Italia, 00162 Rome, Italy
| |
Collapse
|
4
|
Lu H, Xu S, Zhao S, Hu X, Ma R, Hu B. EPIC: Emotion Perception by Spatio-Temporal Interaction Context of Gait. IEEE J Biomed Health Inform 2024; 28:2592-2601. [PMID: 37018306 DOI: 10.1109/jbhi.2022.3233597] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/03/2023]
Abstract
Recently, psychophysiological computing has received considerable attention. Due to easy acquisition at a distance and less conscious initiation, gait-based emotion recognition is considered as a valuable research branch in the field of psychophysiological computing. However, most existing methods rarely explore the spatio-temporal context of gait, which limits the ability to capture the higher-order relationship between emotion and gait. In this paper, we utilize a range of research, including psychophysiological computing and artificial intelligence, to propose an integrated emotion perception framework called EPIC, which can find novel joint topology and generate thousands of synthetic gaits by spatio-temporal interaction context. First, we analyze the joint coupling among non-adjacent joints by calculating Phase Lag Index (PLI), which can discover the latent connection among body joints. Second, to synthesize more sophisticated and accurate gait sequences, we explore the effect of spatio-temporal constraints, and propose a new loss function that utilizes the Dynamic Time Warping (DTW) algorithm and pseudo-velocity curve to constrain the output of Gated Recurrent Units (GRU). Finally, Spatial Temporal Graph Convolution Networks (ST-GCN) is used to classify emotions using the generation and the real data. Experimental results demonstrate our approach achieves the accuracy of 89.66%, and outperforms the state-of-the-art methods on Emotion-Gait dataset.
Collapse
|
5
|
Boolani A, Gruber AH, Torad AA, Stamatis A. Identifying Current Feelings of Mild and Moderate to High Depression in Young, Healthy Individuals Using Gait and Balance: An Exploratory Study. SENSORS (BASEL, SWITZERLAND) 2023; 23:6624. [PMID: 37514917 PMCID: PMC10384769 DOI: 10.3390/s23146624] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/23/2023] [Revised: 06/27/2023] [Accepted: 07/20/2023] [Indexed: 07/30/2023]
Abstract
Depressive mood states in healthy populations are prevalent but often under-reported. Biases exist in self-reporting of depression in otherwise healthy individuals. Gait and balance control can serve as objective markers for identifying those individuals, particularly in real-world settings. We utilized inertial measurement units (IMU) to measure gait and balance control. An exploratory, cross-sectional design was used to compare individuals who reported feeling depressed at the moment (n = 49) with those who did not (n = 84). The Quality Assessment Tool for Observational Cohort and Cross-sectional Studies was employed to ensure internal validity. We recruited 133 participants aged between 18-36 years from the university community. Various instruments were used to evaluate participants' present depressive symptoms, sleep, gait, and balance. Gait and balance variables were used to detect depression, and participants were categorized into three groups: not depressed, mild depression, and moderate-high depression. Participant characteristics were analyzed using ANOVA and Kruskal-Wallis tests, and no significant differences were found in age, height, weight, BMI, and prior night's sleep between the three groups. Classification models were utilized for depression detection. The most accurate model incorporated both gait and balance variables, yielding an accuracy rate of 84.91% for identifying individuals with moderate-high depression compared to non-depressed individuals.
Collapse
Affiliation(s)
- Ali Boolani
- Honors Department, Clarkson University, Potsdam, NY 13699, USA
| | - Allison H Gruber
- Department of Kinesiology, Indiana University, Bloomington, IN 47405, USA
| | - Ahmed Ali Torad
- Faculty of Physical Therapy, Kafrelsheik University, Kafr El Sheik 33516, Egypt
| | - Andreas Stamatis
- Department of Health and Sport Sciences, University of Louisville, Louisville, KY 40292, USA
| |
Collapse
|
6
|
Zia Ur Rehman M, Ahmed F, Alsuhibany SA, Jamal SS, Zulfiqar Ali M, Ahmad J. Classification of Skin Cancer Lesions Using Explainable Deep Learning. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22186915. [PMID: 36146271 PMCID: PMC9505745 DOI: 10.3390/s22186915] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/05/2022] [Revised: 09/05/2022] [Accepted: 09/07/2022] [Indexed: 05/14/2023]
Abstract
Skin cancer is among the most prevalent and life-threatening forms of cancer that occur worldwide. Traditional methods of skin cancer detection need an in-depth physical examination by a medical professional, which is time-consuming in some cases. Recently, computer-aided medical diagnostic systems have gained popularity due to their effectiveness and efficiency. These systems can assist dermatologists in the early detection of skin cancer, which can be lifesaving. In this paper, the pre-trained MobileNetV2 and DenseNet201 deep learning models are modified by adding additional convolution layers to effectively detect skin cancer. Specifically, for both models, the modification includes stacking three convolutional layers at the end of both the models. A thorough comparison proves that the modified models show their superiority over the original pre-trained MobileNetV2 and DenseNet201 models. The proposed method can detect both benign and malignant classes. The results indicate that the proposed Modified DenseNet201 model achieves 95.50% accuracy and state-of-the-art performance when compared with other techniques present in the literature. In addition, the sensitivity and specificity of the Modified DenseNet201 model are 93.96% and 97.03%, respectively.
Collapse
Affiliation(s)
| | - Fawad Ahmed
- Department of Cyber Security, Pakistan Navy Engineering College, National University of Sciences & Technology, Karachi 75350, Pakistan
| | - Suliman A. Alsuhibany
- Department of Computer Science, College of Computer, Qassim University, Buraydah 51452, Saudi Arabia
- Correspondence:
| | - Sajjad Shaukat Jamal
- Department of Mathematics, College of Science, King Khalid University, Abha 61413, Saudi Arabia
| | | | - Jawad Ahmad
- School of Computing, Edinburgh Napier University, Edinburgh EH10 5DT, UK
| |
Collapse
|