1
|
Zhao X, Li J, Hua C. FP-GCN: Frequency Pyramid Graph Convolutional Network for Enhancing Pathological Gait Classification. SENSORS (BASEL, SWITZERLAND) 2024; 24:3352. [PMID: 38894144 PMCID: PMC11174585 DOI: 10.3390/s24113352] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/03/2024] [Revised: 05/21/2024] [Accepted: 05/22/2024] [Indexed: 06/21/2024]
Abstract
Gait, a manifestation of one's walking pattern, intricately reflects the harmonious interplay of various bodily systems, offering valuable insights into an individual's health status. However, the current study has shortcomings in the extraction of temporal and spatial dependencies in joint motion, resulting in inefficiencies in pathological gait classification. In this paper, we propose a Frequency Pyramid Graph Convolutional Network (FP-GCN), advocating to complement temporal analysis and further enhance spatial feature extraction. specifically, a spectral decomposition component is adopted to extract gait data with different time frames, which can enhance the detection of rhythmic patterns and velocity variations in human gait and allow a detailed analysis of the temporal features. Furthermore, a novel pyramidal feature extraction approach is developed to analyze the inter-sensor dependencies, which can integrate features from different pathways, enhancing both temporal and spatial feature extraction. Our experimentation on diverse datasets demonstrates the effectiveness of our approach. Notably, FP-GCN achieves an impressive accuracy of 98.78% on public datasets and 96.54% on proprietary data, surpassing existing methodologies and underscoring its potential for advancing pathological gait classification. In summary, our innovative FP-GCN contributes to advancing feature extraction and pathological gait recognition, which may offer potential advancements in healthcare provisions, especially in regions with limited access to medical resources and in home-care environments. This work lays the foundation for further exploration and underscores the importance of remote health monitoring, diagnosis, and personalized interventions.
Collapse
Affiliation(s)
- Xiaoheng Zhao
- Institute of Intelligent Robots and Pattern Recognition, School of Cyber Science and Engineering, Liaoning University, Shenyang 110036, China
| | - Jia Li
- Department of Endocrinology and Metabolism, The Fourth Affiliated Hospital of China Medical University, Shenyang 110032, China
| | - Chunsheng Hua
- Institute of Intelligent Robots and Pattern Recognition, School of Cyber Science and Engineering, Liaoning University, Shenyang 110036, China
| |
Collapse
|
2
|
Li C, Mellbin Y, Krogager J, Polikovsky S, Holmberg M, Ghorbani N, Black MJ, Kjellström H, Zuffi S, Hernlund E. The Poses for Equine Research Dataset (PFERD). Sci Data 2024; 11:497. [PMID: 38750064 PMCID: PMC11096353 DOI: 10.1038/s41597-024-03312-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2023] [Accepted: 04/25/2024] [Indexed: 05/18/2024] Open
Abstract
Studies of quadruped animal motion help us to identify diseases, understand behavior and unravel the mechanics behind gaits in animals. The horse is likely the best-studied animal in this aspect, but data capture is challenging and time-consuming. Computer vision techniques improve animal motion extraction, but the development relies on reference datasets, which are scarce, not open-access and often provide data from only a few anatomical landmarks. Addressing this data gap, we introduce PFERD, a video and 3D marker motion dataset from horses using a full-body set-up of densely placed over 100 skin-attached markers and synchronized videos from ten camera angles. Five horses of diverse conformations provide data for various motions from basic poses (eg. walking, trotting) to advanced motions (eg. rearing, kicking). We further express the 3D motions with current techniques and a 3D parameterized model, the hSMAL model, establishing a baseline for 3D horse markerless motion capture. PFERD enables advanced biomechanical studies and provides a resource of ground truth data for the methodological development of markerless motion capture.
Collapse
Affiliation(s)
- Ci Li
- KTH Royal Institute of Technology, Stockholm, Sweden
| | - Ylva Mellbin
- Swedish University of Agricultural Sciences, Uppsala, Sweden
| | | | | | | | - Nima Ghorbani
- Sporttotal.tv, Immersive Technologies, Cologne, Germany
| | - Michael J Black
- Max Planck Institute for Intelligent Systems, Tübingen, Germany
| | - Hedvig Kjellström
- KTH Royal Institute of Technology, Stockholm, Sweden
- Swedish University of Agricultural Sciences, Uppsala, Sweden
| | - Silvia Zuffi
- CNR Institute for Applied Mathematics and Information Technologies, Milan, Italy
| | - Elin Hernlund
- Swedish University of Agricultural Sciences, Uppsala, Sweden.
| |
Collapse
|
3
|
Ali MM, Medhat Hassan M, Zaki M. Human Pose Estimation for Clinical Analysis of Gait Pathologies. Bioinform Biol Insights 2024; 18:11779322241231108. [PMID: 38757143 PMCID: PMC11097739 DOI: 10.1177/11779322241231108] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Accepted: 01/19/2024] [Indexed: 05/18/2024] Open
Abstract
Gait analysis serves as a critical diagnostic tool for identifying neurologic and musculoskeletal damage. Traditional manual analysis of motion data, however, is labor-intensive and heavily reliant on the expertise and judgment of the therapist. This study introduces a binary classification method for the quantitative assessment of gait impairments, specifically focusing on Duchenne muscular dystrophy (DMD), a prevalent and fatal neuromuscular genetic disorder. The research compares spatiotemporal and sagittal kinematic gait features derived from 2D and 3D human pose estimation trajectories against concurrently recorded 3D motion capture (MoCap) data from healthy children. The proposed model leverages a novel benchmark dataset, collected from YouTube and publicly available datasets of their typically developed peers, to extract time-distance variables (e.g. speed, step length, stride time, and cadence) and sagittal joint angles of the lower extremity (e.g. hip, knee, and knee flexion angles). Machine learning and deep learning techniques are employed to discern patterns that can identify children exhibiting DMD gait disturbances. While the current model is capable of distinguishing between healthy subjects and those with DMD, it does not specifically differentiate between DMD patients and patients with other gait impairments. Experimental results validate the efficacy of our cost-effective method, which relies on recorded RGB video, in detecting gait abnormalities, achieving a prediction accuracy of 96.2% for Support Vector Machine (SVM) and 97% for the deep network.
Collapse
Affiliation(s)
- Manal Mostafa Ali
- Department of Computer and System Engineering, Al-Azhar University, Cairo, Egypt
| | - Maha Medhat Hassan
- Department of Computer and System Engineering, Al-Azhar University, Cairo, Egypt
| | - M Zaki
- Department of Computer and System Engineering, Al-Azhar University, Cairo, Egypt
| |
Collapse
|
4
|
Passias A, Tsakalos KA, Kansizoglou I, Kanavaki AM, Gkrekidis A, Menychtas D, Aggelousis N, Michalopoulou M, Gasteratos A, Sirakoulis GC. A Biologically Inspired Movement Recognition System with Spiking Neural Networks for Ambient Assisted Living Applications. Biomimetics (Basel) 2024; 9:296. [PMID: 38786506 PMCID: PMC11117771 DOI: 10.3390/biomimetics9050296] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2023] [Revised: 02/20/2024] [Accepted: 03/12/2024] [Indexed: 05/25/2024] Open
Abstract
This study presents a novel solution for ambient assisted living (AAL) applications that utilizes spiking neural networks (SNNs) and reconfigurable neuromorphic processors. As demographic shifts result in an increased need for eldercare, due to a large elderly population that favors independence, there is a pressing need for efficient solutions. Traditional deep neural networks (DNNs) are typically energy-intensive and computationally demanding. In contrast, this study turns to SNNs, which are more energy-efficient and mimic biological neural processes, offering a viable alternative to DNNs. We propose asynchronous cellular automaton-based neurons (ACANs), which stand out for their hardware-efficient design and ability to reproduce complex neural behaviors. By utilizing the remote supervised method (ReSuMe), this study improves spike train learning efficiency in SNNs. We apply this to movement recognition in an elderly population, using motion capture data. Our results highlight a high classification accuracy of 83.4%, demonstrating the approach's efficacy in precise movement activity classification. This method's significant advantage lies in its potential for real-time, energy-efficient processing in AAL environments. Our findings not only demonstrate SNNs' superiority over conventional DNNs in computational efficiency but also pave the way for practical neuromorphic computing applications in eldercare.
Collapse
Affiliation(s)
- Athanasios Passias
- Department of Electrical and Computer Engineering, Democritus University of Thrace, 67100 Xanthi, Greece; (A.P.); (K.-A.T.)
| | - Karolos-Alexandros Tsakalos
- Department of Electrical and Computer Engineering, Democritus University of Thrace, 67100 Xanthi, Greece; (A.P.); (K.-A.T.)
| | - Ioannis Kansizoglou
- Department of Production and Management Engineering, Democritus University of Thrace, 67100 Xanthi, Greece; (I.K.); (A.G.)
| | - Archontissa Maria Kanavaki
- School of Physical Education and Sport Science, Democritus University of Thrace, 69100 Komotini, Greece; (A.M.K.); (A.G.); (D.M.); (N.A.); (M.M.)
| | - Athanasios Gkrekidis
- School of Physical Education and Sport Science, Democritus University of Thrace, 69100 Komotini, Greece; (A.M.K.); (A.G.); (D.M.); (N.A.); (M.M.)
| | - Dimitrios Menychtas
- School of Physical Education and Sport Science, Democritus University of Thrace, 69100 Komotini, Greece; (A.M.K.); (A.G.); (D.M.); (N.A.); (M.M.)
| | - Nikolaos Aggelousis
- School of Physical Education and Sport Science, Democritus University of Thrace, 69100 Komotini, Greece; (A.M.K.); (A.G.); (D.M.); (N.A.); (M.M.)
| | - Maria Michalopoulou
- School of Physical Education and Sport Science, Democritus University of Thrace, 69100 Komotini, Greece; (A.M.K.); (A.G.); (D.M.); (N.A.); (M.M.)
| | - Antonios Gasteratos
- Department of Production and Management Engineering, Democritus University of Thrace, 67100 Xanthi, Greece; (I.K.); (A.G.)
| | - Georgios Ch. Sirakoulis
- Department of Electrical and Computer Engineering, Democritus University of Thrace, 67100 Xanthi, Greece; (A.P.); (K.-A.T.)
| |
Collapse
|
5
|
Lan Z, Lempereur M, Gueret G, Houx L, Cacioppo M, Pons C, Mensah J, Rémy-Néris O, Aïssa-El-Bey A, Rousseau F, Brochard S. Towards a diagnostic tool for neurological gait disorders in childhood combining 3D gait kinematics and deep learning. Comput Biol Med 2024; 171:108095. [PMID: 38350399 DOI: 10.1016/j.compbiomed.2024.108095] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2023] [Revised: 01/31/2024] [Accepted: 02/01/2024] [Indexed: 02/15/2024]
Abstract
Gait abnormalities are frequent in children and can be caused by different pathologies, such as cerebral palsy, neuromuscular disease, toe walker syndrome, etc. Analysis of the "gait pattern" (i.e., the way the person walks) using 3D analysis provides highly relevant clinical information. This information is used to guide therapeutic choices; however, it is underused in diagnostic processes, probably because of the lack of standardization of data collection methods. Therefore, 3D gait analysis is currently used as an assessment rather than a diagnostic tool. In this work, we aimed to determine if deep learning could be combined with 3D gait analysis data to diagnose gait disorders in children. We tested the diagnostic accuracy of deep learning methods combined with 3D gait analysis data from 371 children (148 with unilateral cerebral palsy, 60 with neuromuscular disease, 19 toe walkers, 60 with bilateral cerebral palsy, 25 stroke, and 59 typically developing children), with a total of 6400 gait cycles. We evaluated the accuracy, sensitivity, specificity, F1 score, Area Under the Curve (AUC) score, and confusion matrix of the predictions by ResNet, LSTM, and InceptionTime deep learning architectures for time series data. The deep learning-based models had good to excellent diagnostic accuracy (ranging from 0.77 to 0.99) for discrimination between healthy and pathological gait, discrimination between different etiologies of pathological gait (binary and multi-classification); and determining stroke onset time. LSTM performed best overall. This study revealed that the gait pattern contains specific, pathology-related information. These results open the way for an extension of 3D gait analysis from evaluation to diagnosis. Furthermore, the method we propose is a data-driven diagnostic model that can be trained and used without human intervention or expert knowledge. Furthermore, the method could be used to distinguish gait-related pathologies and their onset times beyond those studied in this research.
Collapse
Affiliation(s)
- Zhengyang Lan
- Laboratoire de Traitement de l'Information Médicale INSERM U1101, Brest, France; IMT Atlantique, LaTIM U1101 INSERM, Brest, France
| | - Mathieu Lempereur
- Laboratoire de Traitement de l'Information Médicale INSERM U1101, Brest, France; Université de Bretagne Occidentale, Brest, France; CHU de Brest, Hôpital Morvan, service de médecine physique et de réadaptation, Brest, France.
| | - Gwenael Gueret
- CHU de Brest, Hôpital Morvan, service de médecine physique et de réadaptation, Brest, France
| | - Laetitia Houx
- Laboratoire de Traitement de l'Information Médicale INSERM U1101, Brest, France; Université de Bretagne Occidentale, Brest, France; CHU de Brest, Hôpital Morvan, service de médecine physique et de réadaptation, Brest, France; Fondation Ildys, Brest, France
| | - Marine Cacioppo
- Laboratoire de Traitement de l'Information Médicale INSERM U1101, Brest, France; Université de Bretagne Occidentale, Brest, France; CHU de Brest, Hôpital Morvan, service de médecine physique et de réadaptation, Brest, France
| | - Christelle Pons
- Laboratoire de Traitement de l'Information Médicale INSERM U1101, Brest, France; Université de Bretagne Occidentale, Brest, France; CHU de Brest, Hôpital Morvan, service de médecine physique et de réadaptation, Brest, France; Fondation Ildys, Brest, France
| | - Johanne Mensah
- Laboratoire de Traitement de l'Information Médicale INSERM U1101, Brest, France; Université de Bretagne Occidentale, Brest, France; CHU de Brest, Hôpital Morvan, service de médecine physique et de réadaptation, Brest, France; Fondation Ildys, Brest, France
| | - Olivier Rémy-Néris
- Laboratoire de Traitement de l'Information Médicale INSERM U1101, Brest, France; Université de Bretagne Occidentale, Brest, France; CHU de Brest, Hôpital Morvan, service de médecine physique et de réadaptation, Brest, France
| | | | - François Rousseau
- Laboratoire de Traitement de l'Information Médicale INSERM U1101, Brest, France; IMT Atlantique, LaTIM U1101 INSERM, Brest, France
| | - Sylvain Brochard
- Laboratoire de Traitement de l'Information Médicale INSERM U1101, Brest, France; Université de Bretagne Occidentale, Brest, France; CHU de Brest, Hôpital Morvan, service de médecine physique et de réadaptation, Brest, France; Fondation Ildys, Brest, France
| |
Collapse
|
6
|
Tian H, Li H, Jiang W, Ma X, Li X, Wu H, Li Y. Cross-Spatiotemporal Graph Convolution Networks for Skeleton-Based Parkinsonian Gait MDS-UPDRS Score Estimation. IEEE Trans Neural Syst Rehabil Eng 2024; 32:412-421. [PMID: 38198272 DOI: 10.1109/tnsre.2024.3352004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2024]
Abstract
Gait impairment in Parkinson's Disease (PD) is quantitatively assessed using the Movement Disorder Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS), a well-established clinical tool. Objective and efficient PD gait assessment is crucial for developing interventions to slow or halt its advancement. Skeleton-based PD gait MDS-UPDRS score estimation has attracted increasing interest in improving diagnostic efficiency and objectivity. However, previous works ignore the important cross-spacetime dependencies between joints in PD gait. Moreover, existing PD gait skeleton datasets are very small, which is a big issue in deep learning-based gait studies. In this work, we collect a sizable PD gait skeleton dataset by multi-view Azure Kinect sensors. The collected dataset contains 102 PD patients and 30 healthy older adults. In addition, gait data from 16 young adults (aged 24-50 years) are collected to further examine the effect of age on PD gait assessment. For skeleton-based automatic PD gait analysis, we propose a novel cross-spatiotemporal graph convolution network (CST-GCN) to learn complex features of gait patterns. Specifically, a gait graph labeling strategy is designed to assemble and group cross-spacetime neighbors of the root node according to the spatiotemporal semantics of the gait skeleton. Based on this strategy, the CST-GCN module explicitly models the cross-spacetime dependencies among joints. Finally, a dual-path model is presented to realize the modeling and fusion of spatial, temporal, and cross-spacetime gait features. Extensive experiments validate the effectiveness of our method on the collected dataset.
Collapse
|
7
|
Topham LK, Khan W, Al-Jumeily D, Waraich A, Hussain AJ. A diverse and multi-modal gait dataset of indoor and outdoor walks acquired using multiple cameras and sensors. Sci Data 2023; 10:320. [PMID: 37237014 DOI: 10.1038/s41597-023-02161-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Accepted: 04/18/2023] [Indexed: 05/28/2023] Open
Abstract
Gait datasets are often limited by a lack of diversity in terms of the participants, appearance, viewing angle, environments, annotations, and availability. We present a primary gait dataset comprising 1,560 annotated casual walks from 64 participants, in both indoor and outdoor real-world environments. We used two digital cameras and a wearable digital goniometer to capture visual as well as motion signal gait-data respectively. Traditional methods of gait identification are often affected by the viewing angle and appearance of the participant therefore, this dataset mainly considers the diversity in various aspects (e.g., participants' attributes, background variations, and view angles). The dataset is captured from 8 viewing angles in 45° increments along-with alternative appearances for each participant, for example, via a change of clothing. The dataset provides 3,120 videos, containing approximately 748,800 image frames with detailed annotations including approximately 56,160,000 bodily keypoint annotations, identifying 75 keypoints per video frame, and approximately 1,026,480 motion data points captured from a digital goniometer for three limb segments (thigh, upper arm, and head).
Collapse
Affiliation(s)
| | - Wasiq Khan
- Liverpool John Moores University, Liverpool, UK.
| | | | | | | |
Collapse
|
8
|
Walking Speed Classification from Marker-Free Video Images in Two-Dimension Using Optimum Data and a Deep Learning Method. Bioengineering (Basel) 2022; 9:bioengineering9110715. [DOI: 10.3390/bioengineering9110715] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2022] [Revised: 11/09/2022] [Accepted: 11/16/2022] [Indexed: 11/22/2022] Open
Abstract
Walking speed is considered a reliable assessment tool for any movement-related functional activities of an individual (i.e., patients and healthy controls) by caregivers and clinicians. Traditional video surveillance gait monitoring in clinics and aged care homes may employ modern artificial intelligence techniques to utilize walking speed as a screening indicator of various physical outcomes or accidents in individuals. Specifically, ratio-based body measurements of walking individuals are extracted from marker-free and two-dimensional video images to create a walk pattern suitable for walking speed classification using deep learning based artificial intelligence techniques. However, the development of successful and highly predictive deep learning architecture depends on the optimal use of extracted data because redundant data may overburden the deep learning architecture and hinder the classification performance. The aim of this study was to investigate the optimal combination of ratio-based body measurements needed for presenting potential information to define and predict a walk pattern in terms of speed with high classification accuracy using a deep learning-based walking speed classification model. To this end, the performance of different combinations of five ratio-based body measurements was evaluated through a correlation analysis and a deep learning-based walking speed classification test. The results show that a combination of three ratio-based body measurements can potentially define and predict a walk pattern in terms of speed with classification accuracies greater than 92% using a bidirectional long short-term memory deep learning method.
Collapse
|
9
|
Chen B, Chen C, Hu J, Sayeed Z, Qi J, Darwiche HF, Little BE, Lou S, Darwish M, Foote C, Palacio-Lascano C. Computer Vision and Machine Learning-Based Gait Pattern Recognition for Flat Fall Prediction. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22207960. [PMID: 36298311 PMCID: PMC9612353 DOI: 10.3390/s22207960] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Revised: 10/12/2022] [Accepted: 10/14/2022] [Indexed: 05/27/2023]
Abstract
BACKGROUND Gait recognition has been applied in the prediction of the probability of elderly flat ground fall, functional evaluation during rehabilitation, and the training of patients with lower extremity motor dysfunction. Gait distinguishing between seemingly similar kinematic patterns associated with different pathological entities is a challenge for the clinician. How to realize automatic identification and judgment of abnormal gait is a significant challenge in clinical practice. The long-term goal of our study is to develop a gait recognition computer vision system using artificial intelligence (AI) and machine learning (ML) computing. This study aims to find an optimal ML algorithm using computer vision techniques and measure variables from lower limbs to classify gait patterns in healthy people. The purpose of this study is to determine the feasibility of computer vision and machine learning (ML) computing in discriminating different gait patterns associated with flat-ground falls. METHODS We used the Kinect® Motion system to capture the spatiotemporal gait data from seven healthy subjects in three walking trials, including normal gait, pelvic-obliquity-gait, and knee-hyperextension-gait walking. Four different classification methods including convolutional neural network (CNN), support vector machine (SVM), K-nearest neighbors (KNN), and long short-term memory (LSTM) neural networks were used to automatically classify three gait patterns. Overall, 750 sets of data were collected, and the dataset was divided into 80% for algorithm training and 20% for evaluation. RESULTS The SVM and KNN had a higher accuracy than CNN and LSTM. The SVM (94.9 ± 3.36%) had the highest accuracy in the classification of gait patterns, followed by KNN (94.0 ± 4.22%). The accuracy of CNN was 87.6 ± 7.50% and that of LSTM 83.6 ± 5.35%. CONCLUSIONS This study revealed that the proposed AI machine learning (ML) techniques can be used to design gait biometric systems and machine vision for gait pattern recognition. Potentially, this method can be used to remotely evaluate elderly patients and help clinicians make decisions regarding disposition, follow-up, and treatment.
Collapse
Affiliation(s)
- Biao Chen
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Chaoyang Chen
- Orthopaedic Surgery and Sports Medicine, Detroit Medical Center, Detroit, MI 48201, USA
| | - Jie Hu
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Zain Sayeed
- Orthopaedic Surgery and Sports Medicine, Detroit Medical Center, Detroit, MI 48201, USA
| | - Jin Qi
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Hussein F. Darwiche
- Orthopaedic Surgery and Sports Medicine, Detroit Medical Center, Detroit, MI 48201, USA
| | - Bryan E. Little
- Orthopaedic Surgery and Sports Medicine, Detroit Medical Center, Detroit, MI 48201, USA
| | - Shenna Lou
- South Texas Health System—McAllen Department of Trauma, McAllen, TX 78503, USA
| | - Muhammad Darwish
- South Texas Health System—McAllen Department of Trauma, McAllen, TX 78503, USA
| | - Christopher Foote
- South Texas Health System—McAllen Department of Trauma, McAllen, TX 78503, USA
| | | |
Collapse
|
10
|
GaitRec-Net: A Deep Neural Network for Gait Disorder Detection Using Ground Reaction Force. PPAR Res 2022; 2022:9355015. [PMID: 36046063 PMCID: PMC9424014 DOI: 10.1155/2022/9355015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Revised: 07/10/2022] [Accepted: 07/25/2022] [Indexed: 11/18/2022] Open
Abstract
Walking (gait) irregularities and abnormalities are predictors and symptoms of disorder and disability. In the past, elaborate video (camera-based) systems, pressure mats, or a mix of the two has been used in clinical settings to monitor and evaluate gait. This article presents an artificial intelligence-based comprehensive investigation of ground reaction force (GRF) pattern to classify the healthy control and gait disorders using the large-scale ground reaction force. The used dataset comprised GRF measurements from different patients. The article includes machine learning- and deep learning-based models to classify healthy and gait disorder patients using ground reaction force. A deep learning-based architecture GaitRec-Net is proposed for this classification. The classification results were evaluated using various metrics, and each experiment was analysed using a fivefold cross-validation approach. Compared to machine learning classifiers, the proposed deep learning model is found better for feature extraction resulting in high accuracy of classification. As a result, the proposed framework presents a promising step in the direction of automatic categorization of abnormal gait pattern.
Collapse
|
11
|
Kim J, Seo H, Naseem MT, Lee CS. Pathological-Gait Recognition Using Spatiotemporal Graph Convolutional Networks and Attention Model. SENSORS (BASEL, SWITZERLAND) 2022; 22:4863. [PMID: 35808358 PMCID: PMC9269520 DOI: 10.3390/s22134863] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/23/2022] [Revised: 06/19/2022] [Accepted: 06/22/2022] [Indexed: 06/15/2023]
Abstract
Walking is an exercise that uses muscles and joints of the human body and is essential for understanding body condition. Analyzing body movements through gait has been studied and applied in human identification, sports science, and medicine. This study investigated a spatiotemporal graph convolutional network model (ST-GCN), using attention techniques applied to pathological-gait classification from the collected skeletal information. The focus of this study was twofold. The first objective was extracting spatiotemporal features from skeletal information presented by joint connections and applying these features to graph convolutional neural networks. The second objective was developing an attention mechanism for spatiotemporal graph convolutional neural networks, to focus on important joints in the current gait. This model establishes a pathological-gait-classification system for diagnosing sarcopenia. Experiments on three datasets, namely NTU RGB+D, pathological gait of GIST, and multimodal-gait symmetry (MMGS), validate that the proposed model outperforms existing models in gait classification.
Collapse
Affiliation(s)
- Jungi Kim
- Department of Automotive Lighting Convergence Engineering, Yeungnam University, Gyeongsan 38541, Korea;
| | - Haneol Seo
- Research Institute of Human Ecology, Yeungnam University, Gyeongsan 38541, Korea; (H.S.); (M.T.N.)
| | - Muhammad Tahir Naseem
- Research Institute of Human Ecology, Yeungnam University, Gyeongsan 38541, Korea; (H.S.); (M.T.N.)
| | - Chan-Su Lee
- Department of Electronic Engineering, Yeungnam University, Gyeongsan 38541, Korea
| |
Collapse
|
12
|
Tian H, Ma X, Wu H, Li Y. Skeleton-based abnormal gait recognition with spatio-temporal attention enhanced gait-structural graph convolutional networks. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2021.12.004] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
|
13
|
Application of Deep Learning Architectures for Satellite Image Time Series Prediction: A Review. REMOTE SENSING 2021. [DOI: 10.3390/rs13234822] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Satellite image time series (SITS) is a sequence of satellite images that record a given area at several consecutive times. The aim of such sequences is to use not only spatial information but also the temporal dimension of the data, which is used for multiple real-world applications, such as classification, segmentation, anomaly detection, and prediction. Several traditional machine learning algorithms have been developed and successfully applied to time series for predictions. However, these methods have limitations in some situations, thus deep learning (DL) techniques have been introduced to achieve the best performance. Reviews of machine learning and DL methods for time series prediction problems have been conducted in previous studies. However, to the best of our knowledge, none of these surveys have addressed the specific case of works using DL techniques and satellite images as datasets for predictions. Therefore, this paper concentrates on the DL applications for SITS prediction, giving an overview of the main elements used to design and evaluate the predictive models, namely the architectures, data, optimization functions, and evaluation metrics. The reviewed DL-based models are divided into three categories, namely recurrent neural network-based models, hybrid models, and feed-forward-based models (convolutional neural networks and multi-layer perceptron). The main characteristics of satellite images and the major existing applications in the field of SITS prediction are also presented in this article. These applications include weather forecasting, precipitation nowcasting, spatio-temporal analysis, and missing data reconstruction. Finally, current limitations and proposed workable solutions related to the use of DL for SITS prediction are also highlighted.
Collapse
|
14
|
Remote Gait Type Classification System Using Markerless 2D Video. Diagnostics (Basel) 2021; 11:diagnostics11101824. [PMID: 34679521 PMCID: PMC8534997 DOI: 10.3390/diagnostics11101824] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2021] [Revised: 09/22/2021] [Accepted: 09/28/2021] [Indexed: 11/17/2022] Open
Abstract
Several pathologies can alter the way people walk, i.e., their gait. Gait analysis can be used to detect such alterations and, therefore, help diagnose certain pathologies or assess people's health and recovery. Simple vision-based systems have a considerable potential in this area, as they allow the capture of gait in unconstrained environments, such as at home or in a clinic, while the required computations can be done remotely. State-of-the-art vision-based systems for gait analysis use deep learning strategies, thus requiring a large amount of data for training. However, to the best of our knowledge, the largest publicly available pathological gait dataset contains only 10 subjects, simulating five types of gait. This paper presents a new dataset, GAIT-IT, captured from 21 subjects simulating five types of gait, at two severity levels. The dataset is recorded in a professional studio, making the sequences free of background camouflage, variations in illumination and other visual artifacts. The dataset is used to train a novel automatic gait analysis system. Compared to the state-of-the-art, the proposed system achieves a drastic reduction in the number of trainable parameters, memory requirements and execution times, while the classification accuracy is on par with the state-of-the-art. Recognizing the importance of remote healthcare, the proposed automatic gait analysis system is integrated with a prototype web application. This prototype is presently hosted in a private network, and after further tests and development it will allow people to upload a video of them walking and execute a web service that classifies their gait. The web application has a user-friendly interface usable by healthcare professionals or by laypersons. The application also makes an association between the identified type of gait and potential gait pathologies that exhibit the identified characteristics.
Collapse
|
15
|
Teufl W, Taetz B, Miezal M, Dindorf C, Fröhlich M, Trinler U, Hogan A, Bleser G. Automated detection and explainability of pathological gait patterns using a one-class support vector machine trained on inertial measurement unit based gait data. Clin Biomech (Bristol, Avon) 2021; 89:105452. [PMID: 34481198 DOI: 10.1016/j.clinbiomech.2021.105452] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/10/2020] [Revised: 08/12/2021] [Accepted: 08/13/2021] [Indexed: 02/07/2023]
Abstract
BACKGROUND Machine learning approaches for the classification of pathological gait based on kinematic data, e.g. derived from inertial sensors, are commonly used in terms of a multi-class classification problem. However, there is a lack of research regarding one-class classifiers that are independent of certain pathologies. Therefore, it was the aim of this work to design a one-class classifier based on healthy norm-data that provides not only a prediction probability but rather an explanation of the classification decision, increasing the acceptance of this machine learning approach. METHODS The inertial sensor based gait kinematics of 25 healthy subjects was employed to train a one-class support vector machine. 25 healthy subjects, 20 patients after total hip arthroplasty and one transfemoral amputee served to validate the classifier. Prediction probabilities and feature importance scores were estimated for each subject. FINDINGS The support vector machine predicted 100% of the patients as outliers from the healthy group. Three healthy subjects were predicted as outliers. The feature importance calculation revealed the hip in the sagittal plane as most relevant feature concerning the group after total hip arthroplasty. For the misclassified healthy subject with the lowest probability score the knee flexion and the pelvis obliquity were identified. INTERPRETATION The support vector machine seems a useful tool to identify outliers from a healthy norm-group. The feature importance examination proved to provide valuable information on the musculoskeletal status of the subjects. In this combination, the present approach could be employed in various disciplines to identify abnormal gait and suggest subsequent training.
Collapse
Affiliation(s)
- Wolfgang Teufl
- University of Salzburg, Department of Sport Science, Schlossallee 49, 5400 Hallein, Austria.
| | - Bertram Taetz
- Technische Universität Kaiserslautern, Department of Computer Science, Gottlieb-Daimler-Straße 48, 67663 Kaiserslautern, Germany.
| | - Markus Miezal
- Technische Universität Kaiserslautern, Department of Computer Science, Gottlieb-Daimler-Straße 48, 67663 Kaiserslautern, Germany.
| | - Carlo Dindorf
- Technische Universität Kaiserslautern, Department of Sport Science, Erwin-Schrödinger-Straße 57, 67663 Kaiserslautern, Germany.
| | - Michael Fröhlich
- Technische Universität Kaiserslautern, Department of Sport Science, Erwin-Schrödinger-Straße 57, 67663 Kaiserslautern, Germany.
| | - Ursula Trinler
- BG Klinik Ludwigshafen, Ludwig-Guttmann-Straße 13, 67071 Ludwigshafen, Germany.
| | - Aidan Hogan
- BG Klinik Ludwigshafen, Ludwig-Guttmann-Straße 13, 67071 Ludwigshafen, Germany.
| | - Gabriele Bleser
- Technische Universität Kaiserslautern, Department of Computer Science, Gottlieb-Daimler-Straße 48, 67663 Kaiserslautern, Germany.
| |
Collapse
|
16
|
Albuquerque P, Verlekar TT, Correia PL, Soares LD. A Spatiotemporal Deep Learning Approach for Automatic Pathological Gait Classification. SENSORS 2021; 21:s21186202. [PMID: 34577408 PMCID: PMC8473368 DOI: 10.3390/s21186202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/28/2021] [Revised: 09/08/2021] [Accepted: 09/13/2021] [Indexed: 12/03/2022]
Abstract
Human motion analysis provides useful information for the diagnosis and recovery assessment of people suffering from pathologies, such as those affecting the way of walking, i.e., gait. With recent developments in deep learning, state-of-the-art performance can now be achieved using a single 2D-RGB-camera-based gait analysis system, offering an objective assessment of gait-related pathologies. Such systems provide a valuable complement/alternative to the current standard practice of subjective assessment. Most 2D-RGB-camera-based gait analysis approaches rely on compact gait representations, such as the gait energy image, which summarize the characteristics of a walking sequence into one single image. However, such compact representations do not fully capture the temporal information and dependencies between successive gait movements. This limitation is addressed by proposing a spatiotemporal deep learning approach that uses a selection of key frames to represent a gait cycle. Convolutional and recurrent deep neural networks were combined, processing each gait cycle as a collection of silhouette key frames, allowing the system to learn temporal patterns among the spatial features extracted at individual time instants. Trained with gait sequences from the GAIT-IT dataset, the proposed system is able to improve gait pathology classification accuracy, outperforming state-of-the-art solutions and achieving improved generalization on cross-dataset tests.
Collapse
Affiliation(s)
- Pedro Albuquerque
- Instituto de Telecomunicações, Instituto Superior Técnico, Universidade de Lisboa, Av. Rovisco Pais 1, 1049-001 Lisboa, Portugal; (P.A.); (P.L.C.)
| | - Tanmay Tulsidas Verlekar
- Department of CSIS and APPCAIR, BITS Pilani, K K Birla, Goa Campus, Goa 403726, India
- Correspondence:
| | - Paulo Lobato Correia
- Instituto de Telecomunicações, Instituto Superior Técnico, Universidade de Lisboa, Av. Rovisco Pais 1, 1049-001 Lisboa, Portugal; (P.A.); (P.L.C.)
| | - Luís Ducla Soares
- Instituto de Telecomunicações, Instituto Universitário de Lisboa (ISCTE-IUL), Av. das Forças Armadas, 1649-026 Lisboa, Portugal;
| |
Collapse
|
17
|
Gait Phase Estimation by Using LSTM in IMU-Based Gait Analysis-Proof of Concept. SENSORS 2021; 21:s21175749. [PMID: 34502640 PMCID: PMC8433817 DOI: 10.3390/s21175749] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/11/2021] [Revised: 08/10/2021] [Accepted: 08/23/2021] [Indexed: 12/22/2022]
Abstract
Gait phase detection in IMU-based gait analysis has some limitations due to walking style variations and physical impairments of individuals. Therefore, available algorithms may not work properly when the gait data is noisy, or the person rarely reaches a steady state of walking. The aim of this work was to employ Artificial Intelligence (AI), specifically a long short-term memory (LSTM) algorithm, to overcome these weaknesses. Three supervised LSTM-based models were designed to estimate the expected gait phases, including foot-off (FO), mid-swing (MidS) and foot-contact (FC). For collecting gait data two tri-axial inertial sensors were located above each ankle. The angular velocity magnitude, rotation matrix magnitude and free acceleration magnitude were captured for data labeling and turning detection and to strengthen the model, respectively. To do so, a train dataset based on a novel movement protocol was acquired. A validation dataset similar to a train dataset was generated as well. Five test datasets from already existing data were also created to independently evaluate the models. After testing the models on validation and test datasets, all three models demonstrated promising performance in estimating desired gait phases. The proposed approach proves the possibility of employing AI-based algorithms to predict labeled gait phases from a time series of gait data.
Collapse
|
18
|
Vilas-Boas MDC, Rocha AP, Cardoso MN, Fernandes JM, Coelho T, Cunha JPS. Supporting the Assessment of Hereditary Transthyretin Amyloidosis Patients Based On 3-D Gait Analysis and Machine Learning. IEEE Trans Neural Syst Rehabil Eng 2021; 29:1350-1362. [PMID: 34252029 DOI: 10.1109/tnsre.2021.3096433] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Hereditary Transthyretin Amyloidosis (vATTR-V30M) is a rare and highly incapacitating sensorimotor neuropathy caused by an inherited mutation (Val30Met), which typically affects gait, among other symptoms. In this context, we investigated the possibility of using machine learning (ML) techniques to build a model(s) that can be used to support the detection of the Val30Met mutation (possibility of developing the disease), as well as symptom onset detection for the disease, given the gait characteristics of a person. These characteristics correspond to 24 gait parameters computed from 3-D body data, provided by a Kinect v2 camera, acquired from a person while walking towards the camera. To build the model(s), different ML algorithms were explored: k-nearest neighbors, decision tree, random forest, support vector machines (SVM), and multilayer perceptron. For a dataset corresponding to 66 subjects (25 healthy controls, 14 asymptomatic mutation carriers, and 27 patients) and several gait cycles per subject, we were able to obtain a model that distinguishes between controls and vATTR-V30M mutation carriers (with or without symptoms) with a mean accuracy of 92% (SVM). We also obtained a model that distinguishes between asymptomatic and symptomatic carriers with a mean accuracy of 98% (SVM). These results are very relevant, since this is the first study that proposes a ML approach to support vATTR-V30M patient assessment based on gait, being a promising foundation for the development of a computer-aided diagnosis tool to help clinicians in the identification and follow-up of this disease. Furthermore, the proposed method may also be used for other neuropathies.
Collapse
|
19
|
Cha B, Lee KH, Ryu J. Deep-Learning-Based Emergency Stop Prediction for Robotic Lower-Limb Rehabilitation Training Systems. IEEE Trans Neural Syst Rehabil Eng 2021; 29:1120-1128. [PMID: 34106857 DOI: 10.1109/tnsre.2021.3087725] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Robotic lower-limb rehabilitation training is a better alternative for the physical training efforts of a therapist due to advantages, such as intensive repetitive motions, economical therapy, and quantitative assessment of the level of motor recovery through the measurement of force and movement patterns. However, in actual robotic rehabilitation training, emergency stops occur frequently to prevent injury to patients. However, frequent stopping is a waste of time and resources of both therapists and patients. Therefore, early detection of emergency stops in real-time is essential to take appropriate actions. In this paper, we propose a novel deep-learning-based technique for detecting emergency stops as early as possible. First, a bidirectional long short-term memory prediction model was trained using only the normal joint data collected from a real robotic training system. Next, a real-time threshold-based algorithm was developed with cumulative error. The experimental results revealed a precision of 0.94, recall of 0.93, and F1 score of 0.93. Additionally, it was observed that the prediction model was robust for variations in measurement noise.
Collapse
|
20
|
Sikandar T, Rabbi MF, Ghazali KH, Altwijri O, Alqahtani M, Almijalli M, Altayyar S, Ahamed NU. Using a Deep Learning Method and Data from Two-Dimensional (2D) Marker-Less Video-Based Images for Walking Speed Classification. SENSORS 2021; 21:s21082836. [PMID: 33920617 PMCID: PMC8072769 DOI: 10.3390/s21082836] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/26/2021] [Revised: 04/10/2021] [Accepted: 04/13/2021] [Indexed: 01/09/2023]
Abstract
Human body measurement data related to walking can characterize functional movement and thereby become an important tool for health assessment. Single-camera-captured two-dimensional (2D) image sequences of marker-less walking individuals might be a simple approach for estimating human body measurement data which could be used in walking speed-related health assessment. Conventional body measurement data of 2D images are dependent on body-worn garments (used as segmental markers) and are susceptible to changes in the distance between the participant and camera in indoor and outdoor settings. In this study, we propose five ratio-based body measurement data that can be extracted from 2D images and can be used to classify three walking speeds (i.e., slow, normal, and fast) using a deep learning-based bidirectional long short-term memory classification model. The results showed that average classification accuracies of 88.08% and 79.18% could be achieved in indoor and outdoor environments, respectively. Additionally, the proposed ratio-based body measurement data are independent of body-worn garments and not susceptible to changes in the distance between the walking individual and camera. As a simple but efficient technique, the proposed walking speed classification has great potential to be employed in clinics and aged care homes.
Collapse
Affiliation(s)
- Tasriva Sikandar
- Faculty of Electrical and Electronics Engineering, Universiti Malaysia Pahang, Pekan 26600, Malaysia; (T.S.); (K.H.G.)
| | - Mohammad F. Rabbi
- School of Allied Health Sciences, Griffith University, Gold Coast, QLD 4222, Australia;
| | - Kamarul H. Ghazali
- Faculty of Electrical and Electronics Engineering, Universiti Malaysia Pahang, Pekan 26600, Malaysia; (T.S.); (K.H.G.)
| | - Omar Altwijri
- Biomedical Technology Department, College of Applied Medical Sciences, King Saud University, Riyadh 11451, Saudi Arabia; (O.A.); (M.A.); (M.A.); (S.A.)
| | - Mahdi Alqahtani
- Biomedical Technology Department, College of Applied Medical Sciences, King Saud University, Riyadh 11451, Saudi Arabia; (O.A.); (M.A.); (M.A.); (S.A.)
| | - Mohammed Almijalli
- Biomedical Technology Department, College of Applied Medical Sciences, King Saud University, Riyadh 11451, Saudi Arabia; (O.A.); (M.A.); (M.A.); (S.A.)
| | - Saleh Altayyar
- Biomedical Technology Department, College of Applied Medical Sciences, King Saud University, Riyadh 11451, Saudi Arabia; (O.A.); (M.A.); (M.A.); (S.A.)
| | - Nizam U. Ahamed
- Neuromuscular Research Laboratory/Warrior Human Performance Research Center, Department of Sports Medicine and Nutrition, University of Pittsburgh, Pittsburgh, PA 15203, USA
- Correspondence:
| |
Collapse
|
21
|
Sanchez-Casanova J, Liu-Jimenez J, Tirado-Martin P, Sanchez-Reillo R. Unsupervised and scalable low train pathology detection system based on neural networks. Heliyon 2021; 7:e06270. [PMID: 33659760 PMCID: PMC7895758 DOI: 10.1016/j.heliyon.2021.e06270] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2020] [Revised: 12/14/2020] [Accepted: 02/09/2021] [Indexed: 11/23/2022] Open
Abstract
Currently, there exist different technologies applied in the world of medicine dedicated to the detection of health problems such as cancer, heart diseases, etc. However, these technologies are not applied to the detection of lower body pathologies. In this article, a Neural Network (NN)-based system capable of classifying pathologies of the lower train by the way of walking in a non-controlled scenario, with the ability to add new users without retraining the system is presented. All the signals are filtered and processed in order to extract the Gait Cycles (GCs), and those cycles are used as input for the NN. To optimize the network a random search optimization process has been performed. To test the system a database with 51 users and 3 visits per user has been collected. After some improvements, the algorithm can correctly classify the 92% of the cases with 60% of training data. This algorithm is a first approach of creating a system to make a first stage pathology detection without the requirement to move to a specific place.
Collapse
Affiliation(s)
| | - Judith Liu-Jimenez
- University Group for ID technologies (GUTI), University Carlos III of Madrid, Spain
| | - Paloma Tirado-Martin
- University Group for ID technologies (GUTI), University Carlos III of Madrid, Spain
| | - Raul Sanchez-Reillo
- University Group for ID technologies (GUTI), University Carlos III of Madrid, Spain
| |
Collapse
|
22
|
Zhou Z, Liang B, Huang G, Liu B, Nong J, Xie L. Individualized Gait Generation for Rehabilitation Robots Based on Recurrent Neural Networks. IEEE Trans Neural Syst Rehabil Eng 2020; 29:273-281. [PMID: 33332274 DOI: 10.1109/tnsre.2020.3045425] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Individualized reference gait patterns for lower limb rehabilitation robots can greatly improve the effectiveness of rehabilitation. However, previous methods can only generate customized gait patterns at several fixed discrete walking speeds and generating gaits at continuously varying speeds and stride lengths remains unsolved. This work proposes an individualized gait pattern generation method based on a recurrent neural network (RNN), which is proficient in series modeling. We collected the largest gait data set of this kind, which consists of 4,425 gait patterns from 137 subjects. Using this data set, we trained an RNN to create a function mapping from body parameters and gait parameters to a gait pattern. The experimental results indicate that our model is able to generate gait patterns at continuously varying walking speeds and stride lengths while also reducing the errors in the ankle, knee, and hip measurements by 12.83%, 20.95%, and 28.25%, respectively, compared to previous state-of-the-art method.
Collapse
|
23
|
Appropriate Feature Set and Window Parameters Selection for Efficient Motion Intent Characterization towards Intelligently Smart EMG-PR System. Symmetry (Basel) 2020. [DOI: 10.3390/sym12101710] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
The constantly rising number of limb stroke survivors and amputees has motivated the development of intelligent prosthetic/rehabilitation devices for their arm function restoration. The device often integrates a pattern recognition (PR) algorithm that decodes amputees’ limb movement intent from electromyogram (EMG) signals, characterized by neural information and symmetric distribution. However, the control performance of the prostheses mostly rely on the interrelations among multiple dynamic factors of feature set, windowing parameters, and signal conditioning that have rarely been jointly investigated to date. This study systematically investigated the interaction effects of these dynamic factors on the performance of EMG-PR system towards constructing optimal parameters for accurately robust movement intent decoding in the context of prosthetic control. In this regard, the interaction effects of various features across window lengths (50 ms~300 ms), increments (50 ms~125 ms), robustness to external interferences and sensor channels (2 ch~6 ch), were examined using EMG signals obtained from twelve subjects through a symmetrical movement elicitation protocol. Compared to single features, multiple features consistently achieved minimum decoding error below 10% across optimal windowing parameters of 250 ms/100 ms. Also, the multiple features showed high robustness to additive noise with obvious trade-offs between accuracy and computation time. Consequently, our findings may provide proper insight for appropriate parameter selection in the context of robust PR-based control strategy for intelligent rehabilitation device.
Collapse
|
24
|
Chakraborty S, Nandy A. Automatic Diagnosis of Cerebral Palsy Gait Using Computational Intelligence Techniques: A Low-Cost Multi-Sensor Approach. IEEE Trans Neural Syst Rehabil Eng 2020; 28:2488-2496. [PMID: 33001807 DOI: 10.1109/tnsre.2020.3028203] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Automatic diagnosing of Cerebral Palsy (CP) gait is crucial in quantitative evaluation of a therapeutic intervention. Existing systems for such gait assessment are expensive and require user intervention. This study proposes a low-cost gait assessment system equipped with multiple Kinect sensors. Forty subjects (20 CP patients and 20 normal) were recruited for the experiment. To remove outlier frames from the combined gait signal of multiple sensors a data driven algorithm was proposed. Different supervised classifiers along with extreme learning machine were investigated to diagnose CP gait. In addition, a feature level analysis was also performed. Several spatio-temporal features (i.e. step length, stride length, stride time, etc.) were extracted. The strength of walking ratio, a speed invariant feature, to detect CP gait was thoroughly analyzed. The proposed system outperformed state-of-the-art with ≈98% of accuracy (sensitivity: 100%, and specificity: 96.87%). Results indicate a substantial improvement in abnormality detection performance after outlier removal. Based on ReliefF feature ranking algorithm, walking ratio ranked the best among other classical gait features. Performance of all classifiers increased substantially using walking ratio as a feature. Extreme learning machine demonstrated a competing performance in all cases. The higher classification accuracy of this low-cost system using only a single feature makes it attractive for CP gait detection.
Collapse
|
25
|
Zhao A, Li J, Ahmed M. SpiderNet: A spiderweb graph neural network for multi-view gait recognition. Knowl Based Syst 2020. [DOI: 10.1016/j.knosys.2020.106273] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
26
|
Sardari F, Paiement A, Hannuna S, Mirmehdi M. VI-Net-View-Invariant Quality of Human Movement Assessment. SENSORS 2020; 20:s20185258. [PMID: 32942561 PMCID: PMC7570706 DOI: 10.3390/s20185258] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/11/2020] [Revised: 09/05/2020] [Accepted: 09/09/2020] [Indexed: 12/30/2022]
Abstract
We propose a view-invariant method towards the assessment of the quality of human movements which does not rely on skeleton data. Our end-to-end convolutional neural network consists of two stages, where at first a view-invariant trajectory descriptor for each body joint is generated from RGB images, and then the collection of trajectories for all joints are processed by an adapted, pre-trained 2D convolutional neural network (CNN) (e.g., VGG-19 or ResNeXt-50) to learn the relationship amongst the different body parts and deliver a score for the movement quality. We release the only publicly-available, multi-view, non-skeleton, non-mocap, rehabilitation movement dataset (QMAR), and provide results for both cross-subject and cross-view scenarios on this dataset. We show that VI-Net achieves average rank correlation of 0.66 on cross-subject and 0.65 on unseen views when trained on only two views. We also evaluate the proposed method on the single-view rehabilitation dataset KIMORE and obtain 0.66 rank correlation against a baseline of 0.62.
Collapse
Affiliation(s)
- Faegheh Sardari
- Department of Computer Science, University of Bristol, Bristol BS8 1UB, UK; (S.H.); (M.M.)
- Correspondence: ; Tel.:+44-(0)117-954 5139
| | - Adeline Paiement
- Université de Toulon, Aix Marseille Univ, CNRS, LIS, Marseille, France;
| | - Sion Hannuna
- Department of Computer Science, University of Bristol, Bristol BS8 1UB, UK; (S.H.); (M.M.)
| | - Majid Mirmehdi
- Department of Computer Science, University of Bristol, Bristol BS8 1UB, UK; (S.H.); (M.M.)
| |
Collapse
|
27
|
Zhen T, Yan L, Kong JL. An Acceleration Based Fusion of Multiple Spatiotemporal Networks for Gait Phase Detection. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2020; 17:E5633. [PMID: 32764244 PMCID: PMC7460503 DOI: 10.3390/ijerph17165633] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/26/2020] [Revised: 07/24/2020] [Accepted: 07/27/2020] [Indexed: 11/17/2022]
Abstract
Human-gait-phase-recognition is an important technology in the field of exoskeleton robot control and medical rehabilitation. Inertial sensors with accelerometers and gyroscopes are easy to wear, inexpensive and have great potential for analyzing gait dynamics. However, current deep-learning methods extract spatial and temporal features in isolation-while ignoring the inherent correlation in high-dimensional spaces-which limits the accuracy of a single model. This paper proposes an effective hybrid deep-learning framework based on the fusion of multiple spatiotemporal networks (FMS-Net), which is used to detect asynchronous phases from IMU signals. More specifically, it first uses a gait-information acquisition system to collect IMU sensor data fixed on the lower leg. Through data preprocessing, the framework constructs a spatial feature extractor with CNN module and a temporal feature extractor, combined with LSTM module. Finally, a skip-connection structure and the two-layer fully connected layer fusion module are used to achieve the final gait recognition. Experimental results show that this method has better identification accuracy than other comparative methods with the macro-F1 reaching 96.7%.
Collapse
Affiliation(s)
- Tao Zhen
- College of Engineering, Beijing Forestry University, Beijing 100083, China;
| | - Lei Yan
- College of Engineering, Beijing Forestry University, Beijing 100083, China;
| | - Jian-lei Kong
- Artificial Intelligence Academy, Beijing Technology and Business University, Beijing 100048, China
- National Key Laboratory of Environmental Protection Food Chain Pollution Prevention, Beijing 100048, China
| |
Collapse
|