1
|
Khodadadzadeh M, Sloan AT, Jones NA, Coyle D, Kelso JAS. Artificial intelligence detects awareness of functional relation with the environment in 3 month old babies. Sci Rep 2024; 14:15580. [PMID: 38971875 PMCID: PMC11227524 DOI: 10.1038/s41598-024-66312-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2023] [Accepted: 07/01/2024] [Indexed: 07/08/2024] Open
Abstract
A recent experiment probed how purposeful action emerges in early life by manipulating infants' functional connection to an object in the environment (i.e., tethering an infant's foot to a colorful mobile). Vicon motion capture data from multiple infant joints were used here to create Histograms of Joint Displacements (HJDs) to generate pose-based descriptors for 3D infant spatial trajectories. Using HJDs as inputs, machine and deep learning systems were tasked with classifying the experimental state from which snippets of movement data were sampled. The architectures tested included k-Nearest Neighbour (kNN), Linear Discriminant Analysis (LDA), Fully connected network (FCNet), 1D-Convolutional Neural Network (1D-Conv), 1D-Capsule Network (1D-CapsNet), 2D-Conv and 2D-CapsNet. Sliding window scenarios were used for temporal analysis to search for topological changes in infant movement related to functional context. kNN and LDA achieved higher classification accuracy with single joint features, while deep learning approaches, particularly 2D-CapsNet, achieved higher accuracy on full-body features. For each AI architecture tested, measures of foot activity displayed the most distinct and coherent pattern alterations across different experimental stages (reflected in the highest classification accuracy rate), indicating that interaction with the world impacts the infant behaviour most at the site of organism~world connection.
Collapse
Affiliation(s)
- Massoud Khodadadzadeh
- School of Computer Science and Technology, University of Bedfordshire, Luton, LU1 3JU, UK.
- The Bath Institute for the Augmented Human, University of Bath, Bath, BA2 7AY, UK.
- Intelligent Systems Research Centre, Ulster University, Derry, Londonderry, BT48 7JL, UK.
| | - Aliza T Sloan
- Human Brain and Behaviour Laboratory, Center for Complex Systems and Brain Sciences, Florida Atlantic University, Boca Raton, FL, 33431, US
| | - Nancy Aaron Jones
- Human Brain and Behaviour Laboratory, Center for Complex Systems and Brain Sciences, Florida Atlantic University, Boca Raton, FL, 33431, US
| | - Damien Coyle
- The Bath Institute for the Augmented Human, University of Bath, Bath, BA2 7AY, UK
- Intelligent Systems Research Centre, Ulster University, Derry, Londonderry, BT48 7JL, UK
| | - J A Scott Kelso
- Human Brain and Behaviour Laboratory, Center for Complex Systems and Brain Sciences, Florida Atlantic University, Boca Raton, FL, 33431, US
- Intelligent Systems Research Centre, Ulster University, Derry, Londonderry, BT48 7JL, UK
| |
Collapse
|
2
|
Jamsrandorj A, Jung D, Kumar KS, Arshad MZ, Lim H, Kim J, Mun KR. View-independent gait events detection using CNN-transformer hybrid network. J Biomed Inform 2023; 147:104524. [PMID: 37838288 DOI: 10.1016/j.jbi.2023.104524] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2023] [Revised: 09/11/2023] [Accepted: 10/10/2023] [Indexed: 10/16/2023]
Abstract
Accurate gait detection is crucial in utilizing the ample health information embedded in it. Vision-based approaches for gait detection have emerged as an alternative to the exacting sensor-based approaches, but their application has been rather limited due to complicated feature engineering processes and heavy reliance on lateral views. Thus, this study aimed to find a simple vision-based approach that is view-independent and accurate. A total of 22 participants performed six different actions representing standard and peculiar gaits, and the videos acquired from these actions were used as the input of the deep learning networks. Four networks, including a 2D convolutional neural network and an attention-based deep learning network, were trained with standard gaits, and their detection performance for both standard and peculiar gaits was assessed using measures including F1-scores. While all networks achieved remarkable detection performance, the CNN-Transformer network achieved the best performance for both standard and peculiar gaits. Little deviation by the speed of actions or view angles was found. The study is expected to contribute to the wider application of vision-based approaches in gait detection and gait-based health monitoring both at home and in clinical settings.
Collapse
Affiliation(s)
- Ankhzaya Jamsrandorj
- Department of Human Computer Interface & Robotics Engineering, KIST School, University of Science & Technology, Seoul, Republic of Korea; Center for Artificial Intelligence, Korea Institute of Science and Technology, Seoul, Republic of Korea
| | - Dawoon Jung
- Center for Artificial Intelligence, Korea Institute of Science and Technology, Seoul, Republic of Korea
| | - Konki Sravan Kumar
- Center for Artificial Intelligence, Korea Institute of Science and Technology, Seoul, Republic of Korea
| | - Muhammad Zeeshan Arshad
- Center for Artificial Intelligence, Korea Institute of Science and Technology, Seoul, Republic of Korea
| | - Hwasup Lim
- Center for Artificial Intelligence, Korea Institute of Science and Technology, Seoul, Republic of Korea
| | - Jinwook Kim
- Center for Artificial Intelligence, Korea Institute of Science and Technology, Seoul, Republic of Korea.
| | - Kyung-Ryoul Mun
- Center for Artificial Intelligence, Korea Institute of Science and Technology, Seoul, Republic of Korea; KHU-KIST Department of Converging Science and Technology, Kyung Hee University, Seoul, Republic of Korea.
| |
Collapse
|
3
|
Romijnders R, Salis F, Hansen C, Küderle A, Paraschiv-Ionescu A, Cereatti A, Alcock L, Aminian K, Becker C, Bertuletti S, Bonci T, Brown P, Buckley E, Cantu A, Carsin AE, Caruso M, Caulfield B, Chiari L, D'Ascanio I, Del Din S, Eskofier B, Fernstad SJ, Fröhlich MS, Garcia Aymerich J, Gazit E, Hausdorff JM, Hiden H, Hume E, Keogh A, Kirk C, Kluge F, Koch S, Mazzà C, Megaritis D, Micó-Amigo E, Müller A, Palmerini L, Rochester L, Schwickert L, Scott K, Sharrack B, Singleton D, Soltani A, Ullrich M, Vereijken B, Vogiatzis I, Yarnall A, Schmidt G, Maetzler W. Ecological validity of a deep learning algorithm to detect gait events from real-life walking bouts in mobility-limiting diseases. Front Neurol 2023; 14:1247532. [PMID: 37909030 PMCID: PMC10615212 DOI: 10.3389/fneur.2023.1247532] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Accepted: 09/18/2023] [Indexed: 11/02/2023] Open
Abstract
Introduction The clinical assessment of mobility, and walking specifically, is still mainly based on functional tests that lack ecological validity. Thanks to inertial measurement units (IMUs), gait analysis is shifting to unsupervised monitoring in naturalistic and unconstrained settings. However, the extraction of clinically relevant gait parameters from IMU data often depends on heuristics-based algorithms that rely on empirically determined thresholds. These were mainly validated on small cohorts in supervised settings. Methods Here, a deep learning (DL) algorithm was developed and validated for gait event detection in a heterogeneous population of different mobility-limiting disease cohorts and a cohort of healthy adults. Participants wore pressure insoles and IMUs on both feet for 2.5 h in their habitual environment. The raw accelerometer and gyroscope data from both feet were used as input to a deep convolutional neural network, while reference timings for gait events were based on the combined IMU and pressure insoles data. Results and discussion The results showed a high-detection performance for initial contacts (ICs) (recall: 98%, precision: 96%) and final contacts (FCs) (recall: 99%, precision: 94%) and a maximum median time error of -0.02 s for ICs and 0.03 s for FCs. Subsequently derived temporal gait parameters were in good agreement with a pressure insoles-based reference with a maximum mean difference of 0.07, -0.07, and <0.01 s for stance, swing, and stride time, respectively. Thus, the DL algorithm is considered successful in detecting gait events in ecologically valid environments across different mobility-limiting diseases.
Collapse
Affiliation(s)
- Robbin Romijnders
- Digital Signal Processing and System Theory, Electrical and Information Engineering, Faculty of Engineering, Kiel University, Kiel, Germany
- Arbeitsgruppe Neurogeriatrie, Department of Neurology, Universitätsklinikum Schleswig-Holstein, Kiel, Germany
| | - Francesca Salis
- Department of Biomedical Sciences, University of Sassari, Sassari, Italy
| | - Clint Hansen
- Arbeitsgruppe Neurogeriatrie, Department of Neurology, Universitätsklinikum Schleswig-Holstein, Kiel, Germany
| | - Arne Küderle
- Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Anisoara Paraschiv-Ionescu
- Laboratory of Movement Analysis and Measurement, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| | - Andrea Cereatti
- Department of Electronics and Telecommunications, Polytechnic of Turin, Turin, Italy
| | - Lisa Alcock
- Faculty of Medical Sciences, Newcastle University, Newcastle upon Tyne, United Kingdom
| | - Kamiar Aminian
- Laboratory of Movement Analysis and Measurement, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| | - Clemens Becker
- Gesellschaft für Medizinische Forschung, Robert-Bosch Foundation GmbH, Stuttgart, Germany
| | - Stefano Bertuletti
- Department of Biomedical Sciences, University of Sassari, Sassari, Italy
| | - Tecla Bonci
- INSIGNEO Institute for In Silico Medicine, The University of Sheffield, Sheffield, United Kingdom
- Department of Mechanical Engineering, The University of Sheffield, Sheffield, United Kingdom
| | - Philip Brown
- Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, United Kingdom
| | - Ellen Buckley
- INSIGNEO Institute for In Silico Medicine, The University of Sheffield, Sheffield, United Kingdom
- Department of Mechanical Engineering, The University of Sheffield, Sheffield, United Kingdom
| | - Alma Cantu
- School of Computing, Newcastle University, Newcastle upon Tyne, United Kingdom
| | - Anne-Elie Carsin
- Barcelona Institute for Global Health (ISGlobal), Barcelona, Spain
- Faculty of Health and Life Sciences, Universitat Pompeu Fabra, Barcelona, Spain
- CIBER Epidemiología y Salud Pública, Madrid, Spain
| | - Marco Caruso
- Department of Electronics and Telecommunications, Polytechnic of Turin, Turin, Italy
| | - Brian Caulfield
- Insight Centre for Data Analytics, University College Dublin, Dublin, Ireland
- School of Public Health, Physiotherapy and Sports Science, University College Dublin, Dublin, Ireland
| | - Lorenzo Chiari
- Department of Electrical, Electronic and Information Engineering “Guglielmo Marconi”, University of Bologna, Bologna, Italy
- Health Sciences and Technologies—Interdepartmental Center for Industrial Research (CIRISDV), University of Bologna, Bologna, Italy
| | - Ilaria D'Ascanio
- Department of Electrical, Electronic and Information Engineering “Guglielmo Marconi”, University of Bologna, Bologna, Italy
| | - Silvia Del Din
- Faculty of Medical Sciences, Newcastle University, Newcastle upon Tyne, United Kingdom
- Translational and Clinical Research Institute, Newcastle University, Newcastle upon Tyne, United Kingdom
| | - Björn Eskofier
- Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | | | | | - Judith Garcia Aymerich
- Barcelona Institute for Global Health (ISGlobal), Barcelona, Spain
- Faculty of Health and Life Sciences, Universitat Pompeu Fabra, Barcelona, Spain
- CIBER Epidemiología y Salud Pública, Madrid, Spain
| | - Eran Gazit
- Center for the Study of Movement, Cognition and Mobility, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel
| | - Jeffrey M. Hausdorff
- Center for the Study of Movement, Cognition and Mobility, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel
- Department of Physical Therapy, Sackler Faculty of Medicine & Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel
| | - Hugo Hiden
- School of Computing, Newcastle University, Newcastle upon Tyne, United Kingdom
| | - Emily Hume
- Department of Sport, Exercise and Rehabilitation, Northumbria University, Newcastle upon Tyne, United Kingdom
| | - Alison Keogh
- Insight Centre for Data Analytics, University College Dublin, Dublin, Ireland
- School of Public Health, Physiotherapy and Sports Science, University College Dublin, Dublin, Ireland
| | - Cameron Kirk
- Faculty of Medical Sciences, Newcastle University, Newcastle upon Tyne, United Kingdom
| | - Felix Kluge
- Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
- Novartis Institute of Biomedical Research, Novartis Pharma AG, Basel, Switzerland
| | - Sarah Koch
- Barcelona Institute for Global Health (ISGlobal), Barcelona, Spain
- Faculty of Health and Life Sciences, Universitat Pompeu Fabra, Barcelona, Spain
- CIBER Epidemiología y Salud Pública, Madrid, Spain
| | - Claudia Mazzà
- INSIGNEO Institute for In Silico Medicine, The University of Sheffield, Sheffield, United Kingdom
- Department of Mechanical Engineering, The University of Sheffield, Sheffield, United Kingdom
| | - Dimitrios Megaritis
- Department of Sport, Exercise and Rehabilitation, Northumbria University, Newcastle upon Tyne, United Kingdom
| | - Encarna Micó-Amigo
- Faculty of Medical Sciences, Newcastle University, Newcastle upon Tyne, United Kingdom
| | - Arne Müller
- Novartis Institute of Biomedical Research, Novartis Pharma AG, Basel, Switzerland
| | - Luca Palmerini
- Department of Electrical, Electronic and Information Engineering “Guglielmo Marconi”, University of Bologna, Bologna, Italy
- Health Sciences and Technologies—Interdepartmental Center for Industrial Research (CIRISDV), University of Bologna, Bologna, Italy
| | - Lynn Rochester
- Faculty of Medical Sciences, Newcastle University, Newcastle upon Tyne, United Kingdom
- Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, United Kingdom
| | - Lars Schwickert
- Gesellschaft für Medizinische Forschung, Robert-Bosch Foundation GmbH, Stuttgart, Germany
| | - Kirsty Scott
- INSIGNEO Institute for In Silico Medicine, The University of Sheffield, Sheffield, United Kingdom
- Department of Mechanical Engineering, The University of Sheffield, Sheffield, United Kingdom
| | - Basil Sharrack
- Department of Neuroscience and Sheffield NIHR Translational Neuroscience BRC, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, United Kingdom
| | - David Singleton
- Insight Centre for Data Analytics, University College Dublin, Dublin, Ireland
- School of Public Health, Physiotherapy and Sports Science, University College Dublin, Dublin, Ireland
| | - Abolfazl Soltani
- Laboratory of Movement Analysis and Measurement, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
- Digital Health Department, CSEM SA, Neuchâtel, Switzerland
| | - Martin Ullrich
- Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Beatrix Vereijken
- Department of Neuromedicine and Movement Science, Norwegian University of Science and Technology, Trondheim, Norway
| | - Ioannis Vogiatzis
- Department of Sport, Exercise and Rehabilitation, Northumbria University, Newcastle upon Tyne, United Kingdom
| | - Alison Yarnall
- Faculty of Medical Sciences, Newcastle University, Newcastle upon Tyne, United Kingdom
- Department of Mechanical Engineering, The University of Sheffield, Sheffield, United Kingdom
| | - Gerhard Schmidt
- Digital Signal Processing and System Theory, Electrical and Information Engineering, Faculty of Engineering, Kiel University, Kiel, Germany
| | - Walter Maetzler
- Arbeitsgruppe Neurogeriatrie, Department of Neurology, Universitätsklinikum Schleswig-Holstein, Kiel, Germany
| |
Collapse
|
4
|
Dumphart B, Slijepcevic D, Zeppelzauer M, Kranzl A, Unglaube F, Baca A, Horsak B. Robust deep learning-based gait event detection across various pathologies. PLoS One 2023; 18:e0288555. [PMID: 37566568 PMCID: PMC10420363 DOI: 10.1371/journal.pone.0288555] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2022] [Accepted: 06/29/2023] [Indexed: 08/13/2023] Open
Abstract
The correct estimation of gait events is essential for the interpretation and calculation of 3D gait analysis (3DGA) data. Depending on the severity of the underlying pathology and the availability of force plates, gait events can be set either manually by trained clinicians or detected by automated event detection algorithms. The downside of manually estimated events is the tedious and time-intensive work which leads to subjective assessments. For automated event detection algorithms, the drawback is, that there is no standardized method available. Algorithms show varying robustness and accuracy on different pathologies and are often dependent on setup or pathology-specific thresholds. In this paper, we aim at closing this gap by introducing a novel deep learning-based gait event detection algorithm called IntellEvent, which shows to be accurate and robust across multiple pathologies. For this study, we utilized a retrospective clinical 3DGA dataset of 1211 patients with four different pathologies (malrotation deformities of the lower limbs, club foot, infantile cerebral palsy (ICP), and ICP with only drop foot characteristics) and 61 healthy controls. We propose a recurrent neural network architecture based on long-short term memory (LSTM) and trained it with 3D position and velocity information to predict initial contact (IC) and foot off (FO) events. We compared IntellEvent to a state-of-the-art heuristic approach and a machine learning method called DeepEvent. IntellEvent outperforms both methods and detects IC events on average within 5.4 ms and FO events within 11.3 ms with a detection rate of ≥ 99% and ≥ 95%, respectively. Our investigation on generalizability across laboratories suggests that models trained on data from a different laboratory need to be applied with care due to setup variations or differences in capturing frequencies.
Collapse
Affiliation(s)
- Bernhard Dumphart
- Center for Digital Health & Social Innovation, St. Pölten University of Applied Sciences, St. Pölten, Austria
- Institute of Health Sciences, St. Pölten University of Applied Sciences, St. Pölten, Austria
- Doctoral School of Pharmaceutical, Nutritional and Sport Sciences, University of Vienna, Vienna, Austria
| | - Djordje Slijepcevic
- Institute of Creative\Media/Technologies, St. Pölten University of Applied Sciences, St. Pölten, Austria
| | - Matthias Zeppelzauer
- Institute of Creative\Media/Technologies, St. Pölten University of Applied Sciences, St. Pölten, Austria
| | - Andreas Kranzl
- Laboratory of Gait and Movement Analysis, Orthopaedic Hospital Vienna-Speising, Vienna, Austria
| | - Fabian Unglaube
- Laboratory of Gait and Movement Analysis, Orthopaedic Hospital Vienna-Speising, Vienna, Austria
| | - Arnold Baca
- Centre for Sport Science and University Sports, University of Vienna, Vienna, Austria
| | - Brian Horsak
- Center for Digital Health & Social Innovation, St. Pölten University of Applied Sciences, St. Pölten, Austria
- Institute of Health Sciences, St. Pölten University of Applied Sciences, St. Pölten, Austria
| |
Collapse
|
5
|
Abbasi H, Mollet SR, Williams SA, Lim L, Battin MR, Besier TF, McMorland AJC. Deep-Learning Markerless Tracking of Infant General Movements using Standard Video Recordings. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38083202 DOI: 10.1109/embc40787.2023.10340116] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Monitoring spontaneous General Movements (GM) of infants 6-20 weeks post-term age is a reliable tool to assess the quality of neurodevelopment in early infancy. Abnormal or absent GMs are reliable prognostic indicators of whether an infant is at risk of developing neurological impairments and disorders such as cerebral palsy (CP). Therapeutic interventions are most effective at improving neuromuscular outcomes if administered in early infancy. Current clinical protocols require trained assessors to rate videos of infant movements, a time-intensive task. This work proposes a simple, inexpensive, and broadly applicable markerless pose-estimation approach for automatic infant movement tracking using conventional video recordings from handheld devices (e.g., tablets and mobile phones). We leverage the enhanced capabilities of deep-learning technology in image processing to identify 12 anatomical locations (3 per limb) in each video frame, tracking a baby's natural movement throughout the recordings. We validate the capability of resnet152 and a mobile-net-v2-1 to identify body-parts in unseen frames from a full-term male infant, using a novel automatic unsupervised approach that fuses likelihood outputs of a Kalman filter and the deep-nets. Both deep-net models were found to perform very well in the identification of anatomical locations in the unseen data with high average Percentage of Correct Keypoints (aPCK) performances of >99.65% across all locations.Clinical relevance-Results of this research confirm the feasibility of a low-cost and publicly accessible technology to automatically track infants' GMs and diagnose those at higher risk of developing neurological conditions early, when clinical interventions are most effective.
Collapse
|
6
|
Gehlhar R, Tucker M, Young AJ, Ames AD. A Review of Current State-of-the-Art Control Methods for Lower-Limb Powered Prostheses. ANNUAL REVIEWS IN CONTROL 2023; 55:142-164. [PMID: 37635763 PMCID: PMC10449377 DOI: 10.1016/j.arcontrol.2023.03.003] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/29/2023]
Abstract
Lower-limb prostheses aim to restore ambulatory function for individuals with lower-limb amputations. While the design of lower-limb prostheses is important, this paper focuses on the complementary challenge - the control of lower-limb prostheses. Specifically, we focus on powered prostheses, a subset of lower-limb prostheses, which utilize actuators to inject mechanical power into the walking gait of a human user. In this paper, we present a review of existing control strategies for lower-limb powered prostheses, including the control objectives, sensing capabilities, and control methodologies. We separate the various control methods into three main tiers of prosthesis control: high-level control for task and gait phase estimation, mid-level control for desired torque computation (both with and without the use of reference trajectories), and low-level control for enforcing the computed torque commands on the prosthesis. In particular, we focus on the high- and mid-level control approaches in this review. Additionally, we outline existing methods for customizing the prosthetic behavior for individual human users. Finally, we conclude with a discussion on future research directions for powered lower-limb prostheses based on the potential of current control methods and open problems in the field.
Collapse
Affiliation(s)
- Rachel Gehlhar
- Department of Mechanical and Civil Engineering, California Institute of Technology, 1200 E. California Blvd., Pasadena, 91125, CA, USA
| | - Maegan Tucker
- Department of Mechanical and Civil Engineering, California Institute of Technology, 1200 E. California Blvd., Pasadena, 91125, CA, USA
| | - Aaron J Young
- Woodruff School of Mechanical Engineering, Georgia Institute of Technology, North Avenue, Atlanta, 30332, GA, USA
- Institute for Robotics and Intelligent Machines, Georgia Institute of Technology, North Avenue, Atlanta, 30332, GA, USA
| | - Aaron D Ames
- Department of Mechanical and Civil Engineering, California Institute of Technology, 1200 E. California Blvd., Pasadena, 91125, CA, USA
- Department of Computing and Mathematical Sciences, California Institute of Technology, 1200 E. California Blvd., Pasadena, 91125, CA, USA
| |
Collapse
|
7
|
Slemenšek J, Fister I, Geršak J, Bratina B, van Midden VM, Pirtošek Z, Šafarič R. Human Gait Activity Recognition Machine Learning Methods. SENSORS (BASEL, SWITZERLAND) 2023; 23:745. [PMID: 36679546 PMCID: PMC9865094 DOI: 10.3390/s23020745] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Revised: 12/23/2022] [Accepted: 01/03/2023] [Indexed: 06/17/2023]
Abstract
Human gait activity recognition is an emerging field of motion analysis that can be applied in various application domains. One of the most attractive applications includes monitoring of gait disorder patients, tracking their disease progression and the modification/evaluation of drugs. This paper proposes a robust, wearable gait motion data acquisition system that allows either the classification of recorded gait data into desirable activities or the identification of common risk factors, thus enhancing the subject's quality of life. Gait motion information was acquired using accelerometers and gyroscopes mounted on the lower limbs, where the sensors were exposed to inertial forces during gait. Additionally, leg muscle activity was measured using strain gauge sensors. As a matter of fact, we wanted to identify different gait activities within each gait recording by utilizing Machine Learning algorithms. In line with this, various Machine Learning methods were tested and compared to establish the best-performing algorithm for the classification of the recorded gait information. The combination of attention-based convolutional and recurrent neural networks algorithms outperformed the other tested algorithms and was individually tested further on the datasets of five subjects and delivered the following averaged results of classification: 98.9% accuracy, 96.8% precision, 97.8% sensitivity, 99.1% specificity and 97.3% F1-score. Moreover, the algorithm's robustness was also verified with the successful detection of freezing gait episodes in a Parkinson's disease patient. The results of this study indicate a feasible gait event classification method capable of complete algorithm personalization.
Collapse
Affiliation(s)
- Jan Slemenšek
- Faculty of Mechanical Engineering, University of Maribor, 2000 Maribor, Slovenia
| | - Iztok Fister
- Faculty of Electrical Engineering and Computer Science, University of Maribor, 2000 Maribor, Slovenia
| | - Jelka Geršak
- Faculty of Mechanical Engineering, University of Maribor, 2000 Maribor, Slovenia
| | - Božidar Bratina
- Faculty of Electrical Engineering and Computer Science, University of Maribor, 2000 Maribor, Slovenia
| | | | - Zvezdan Pirtošek
- Department of Neurology, University Clinical Centre, 1000 Ljubljana, Slovenia
| | - Riko Šafarič
- Faculty of Electrical Engineering and Computer Science, University of Maribor, 2000 Maribor, Slovenia
| |
Collapse
|
8
|
Wu J, Maurenbrecher H, Schaer A, Becsek B, Awai Easthope C, Chatzipirpiridis G, Ergeneman O, Pané S, Nelson BJ. Human gait-labeling uncertainty and a hybrid model for gait segmentation. Front Neurosci 2022; 16:976594. [PMID: 36570841 PMCID: PMC9773262 DOI: 10.3389/fnins.2022.976594] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2022] [Accepted: 11/18/2022] [Indexed: 12/13/2022] Open
Abstract
Motion capture systems are widely accepted as ground-truth for gait analysis and are used for the validation of other gait analysis systems. To date, their reliability and limitations in manual labeling of gait events have not been studied. Objectives Evaluate manual labeling uncertainty and introduce a hybrid stride detection and gait-event estimation model for autonomous, long-term, and remote monitoring. Methods Estimate inter-labeler inconsistencies by computing the limits-of-agreement. Develop a hybrid model based on dynamic time warping and convolutional neural network to identify valid strides and eliminate non-stride data in inertial (walking) data collected by a wearable device. Finally, detect gait events within a valid stride region. Results The limits of inter-labeler agreement for key gait events heel off, toe off, heel strike, and flat foot are 72, 16, 24, and 80 ms, respectively; The hybrid model's classification accuracy for stride and non-stride are 95.16 and 84.48%, respectively; The mean absolute error for detected heel off, toe off, heel strike, and flat foot are 24, 5, 9, and 13 ms, respectively, when compared to the average human labels. Conclusions The results show the inherent labeling uncertainty and the limits of human gait labeling of motion capture data; The proposed hybrid-model's performance is comparable to that of human labelers, and it is a valid model to reliably detect strides and estimate the gait events in human gait data. Significance This work establishes the foundation for fully automated human gait analysis systems with performances comparable to human-labelers.
Collapse
Affiliation(s)
- Jiaen Wu
- Multi-Scale Robotics Lab, ETH Zurich, Zurich, Switzerland,Magnes AG, Zurich, Switzerland,*Correspondence: Jiaen Wu
| | | | | | | | - Chris Awai Easthope
- Cereneo Foundation, Center for Interdisciplinary Research (CEFIR), Vitznau, Switzerland
| | | | | | - Salvador Pané
- Multi-Scale Robotics Lab, ETH Zurich, Zurich, Switzerland
| | | |
Collapse
|
9
|
Meyer F, Lund-Hansen M, Seeberg TM, Kocbach J, Sandbakk Ø, Austeng A. Inner-Cycle Phases Can Be Estimated from a Single Inertial Sensor by Long Short-Term Memory Neural Network in Roller-Ski Skating. SENSORS (BASEL, SWITZERLAND) 2022; 22:9267. [PMID: 36501969 PMCID: PMC9739028 DOI: 10.3390/s22239267] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Revised: 11/04/2022] [Accepted: 11/11/2022] [Indexed: 06/17/2023]
Abstract
OBJECTIVE The aim of this study was to provide a new machine learning method to determine temporal events and inner-cycle parameters (e.g., cycle, pole and ski contact and swing time) in cross-country roller-ski skating on the field, using a single inertial measurement unit (IMU). METHODS The developed method is based on long short-term memory neural networks to detect the initial and final contact of the poles and skis with the ground during the cyclic movements. Eleven athletes skied four laps of 2.5 km at a low and high intensity using skis with two different rolling coefficients. They were equipped with IMUs attached to the upper back, lower back and to the sternum. Data from force insoles and force poles were used as the reference system. RESULTS The IMU placed on the upper back provided the best results, as the LSTM network was able to determine the temporal events with a mean error ranging from -1 to 11 ms and had a standard deviation (SD) of the error between 64 and 70 ms. The corresponding inner-cycle parameters were calculated with a mean error ranging from -11 to 12 ms and an SD between 66 and 74 ms. The method detected 95% of the events for the poles and 87% of the events for the skis. CONCLUSION The proposed LSTM method provides a promising tool for assessing temporal events and inner-cycle phases in roller-ski skating, showing the potential of using a single IMU to estimate different spatiotemporal parameters of human locomotion.
Collapse
Affiliation(s)
- Frédéric Meyer
- Department of Informatics, University of Oslo, 0373 Oslo, Norway
| | - Magne Lund-Hansen
- Department of Physical Performance, Norwegian School of Sport Science, 0806 Oslo, Norway
| | - Trine M. Seeberg
- SINTEF Digital, Forskningsveien 1, 0373 Oslo, Norway
- Centre for Elite Sports Research, Department of Neuromedicine and Movement Science, Norwegian University of Science and Technology, 7491 Trondheim, Norway
| | - Jan Kocbach
- Centre for Elite Sports Research, Department of Neuromedicine and Movement Science, Norwegian University of Science and Technology, 7491 Trondheim, Norway
| | - Øyvind Sandbakk
- Centre for Elite Sports Research, Department of Neuromedicine and Movement Science, Norwegian University of Science and Technology, 7491 Trondheim, Norway
| | - Andreas Austeng
- Department of Informatics, University of Oslo, 0373 Oslo, Norway
| |
Collapse
|
10
|
Kim YK, Visscher RMS, Viehweger E, Singh NB, Taylor WR, Vogl F. A deep-learning approach for automatically detecting gait-events based on foot-marker kinematics in children with cerebral palsy-Which markers work best for which gait patterns? PLoS One 2022; 17:e0275878. [PMID: 36227847 PMCID: PMC9562216 DOI: 10.1371/journal.pone.0275878] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2022] [Accepted: 09/25/2022] [Indexed: 11/06/2022] Open
Abstract
Neuromotor pathologies often cause motor deficits and deviations from typical locomotion, reducing the quality of life. Clinical gait analysis is used to effectively classify these motor deficits to gain deeper insights into resulting walking behaviours. To allow the ensemble averaging of spatio-temporal metrics across individuals during walking, gait events, such as initial contact (IC) or toe-off (TO), are extracted through either manual annotation based on video data, or through force thresholds using force plates. This study developed a deep-learning long short-term memory (LSTM) approach to detect IC and TO automatically based on foot-marker kinematics of 363 cerebral palsy subjects (age: 11.8 ± 3.2). These foot-marker kinematics, including 3D positions and velocities of the markers located on the hallux (HLX), calcaneus (HEE), distal second metatarsal (TOE), and proximal fifth metatarsal (PMT5), were extracted retrospectively from standard barefoot gait analysis sessions. Different input combinations of these four foot-markers were evaluated across three gait subgroups (IC with the heel, midfoot, or forefoot). For the overall group, our approach detected 89.7% of ICs within 16ms of the true event with a 18.5% false alarm rate. For TOs, only 71.6% of events were detected with a 33.8% false alarm rate. While the TOE|HEE marker combination performed well across all subgroups for IC detection, optimal performance for TO detection required different input markers per subgroup with performance differences of 5-10%. Thus, deep-learning LSTM based detection of IC events using the TOE|HEE markers offers an automated alternative to avoid operator-dependent and laborious manual annotation, as well as the limited step coverage and inability to measure assisted walking for force plate-based detection of IC events.
Collapse
Affiliation(s)
- Yong Kuk Kim
- Laboratory for Movement Biomechanics, Institute for Biomechanics, ETH Zürich, Zürich, Switzerland
- * E-mail:
| | - Rosa M. S. Visscher
- Laboratory for Movement Biomechanics, Institute for Biomechanics, ETH Zürich, Zürich, Switzerland
| | - Elke Viehweger
- Laboratory for Movement Analysis, Department of Orthopedics, University Children’s Hospital Basel, Basel, Switzerland
| | - Navrag B. Singh
- Laboratory for Movement Biomechanics, Institute for Biomechanics, ETH Zürich, Zürich, Switzerland
| | - William R. Taylor
- Laboratory for Movement Biomechanics, Institute for Biomechanics, ETH Zürich, Zürich, Switzerland
| | - Florian Vogl
- Laboratory for Movement Biomechanics, Institute for Biomechanics, ETH Zürich, Zürich, Switzerland
| |
Collapse
|
11
|
Fonseca M, Dumas R, Armand S. Automatic gait event detection in pathologic gait using an auto-selection approach among concurrent methods. Gait Posture 2022; 96:271-274. [PMID: 35716485 DOI: 10.1016/j.gaitpost.2022.06.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/20/2021] [Revised: 02/01/2022] [Accepted: 06/02/2022] [Indexed: 02/02/2023]
Abstract
BACKGROUND Accurate gait event detection is crucial to analyze pathological gait data. Existing methods relying on marker trajectories were reported to be sensitive to different gait patterns, which is an inherent characteristic of pathologic gait. RESEARCH QUESTION We propose a new approach based on auto-selection among different methods, original and taken from the literature. METHODS The auto-selection approach evaluates the accuracy of the implemented methods for both foot-strike and foot-off on all available events detected by the force platforms, independently, and automatically selects the most accurate one to be used on the whole gait session. Pathological gait data from 272 patients with cerebral palsy and idiopathic toe walking were used retrospectively to evaluate the accuracy of this approach. Three methods previously reported in literature together with original methods developed based on auto-correlation were implemented and constituted our auto-selection approach. The accuracy and precision were compared to a recently reported method based on deep events as it is the method that showed the best performance in literature. RESULTS Results showed that the proposed approach outperformed all implemented methods used alone, with an accuracy of - 2.0 ms and - 0.9 ms for foot strike and foot-off, respectively. Additionally, more than 99% and 93% of events detected were detected within 20 ms and 10 ms of accuracy, respectively. SIGNIFICANCE The proposed methodology has demonstrated to improve the accuracy and precision of gait event detection in gait analysis.
Collapse
Affiliation(s)
- Mickael Fonseca
- Geneva University Hospitals and University of Geneva, Rue Gabrielle Perret Gentil 4, 1205 Geneva, Switzerland; Université Gustave Eiffel, 25 avenue François Mitterand, Case 24, 77454 Marne-la-Vallée cedex 2, France.
| | - Raphaël Dumas
- Université Gustave Eiffel, 25 avenue François Mitterand, Case 24, 77454 Marne-la-Vallée cedex 2, France.
| | - Stéphane Armand
- Geneva University Hospitals and University of Geneva, Rue Gabrielle Perret Gentil 4, 1205 Geneva, Switzerland.
| |
Collapse
|
12
|
Bonci T, Salis F, Scott K, Alcock L, Becker C, Bertuletti S, Buckley E, Caruso M, Cereatti A, Del Din S, Gazit E, Hansen C, Hausdorff JM, Maetzler W, Palmerini L, Rochester L, Schwickert L, Sharrack B, Vogiatzis I, Mazzà C. An Algorithm for Accurate Marker-Based Gait Event Detection in Healthy and Pathological Populations During Complex Motor Tasks. Front Bioeng Biotechnol 2022; 10:868928. [PMID: 35721859 PMCID: PMC9201978 DOI: 10.3389/fbioe.2022.868928] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2022] [Accepted: 04/20/2022] [Indexed: 11/13/2022] Open
Abstract
There is growing interest in the quantification of gait as part of complex motor tasks. This requires gait events (GEs) to be detected under conditions different from straight walking. This study aimed to propose and validate a new marker-based GE detection method, which is also suitable for curvilinear walking and step negotiation. The method was first tested against existing algorithms using data from healthy young adults (YA, n = 20) and then assessed in data from 10 individuals from the following five cohorts: older adults, chronic obstructive pulmonary disease, multiple sclerosis, Parkinson’s disease, and proximal femur fracture. The propagation of the errors associated with GE detection on the calculation of stride length, duration, speed, and stance/swing durations was investigated. All participants performed a variety of motor tasks including curvilinear walking and step negotiation, while reference GEs were identified using a validated methodology exploiting pressure insole signals. Sensitivity, positive predictive values (PPV), F1-score, bias, precision, and accuracy were calculated. Absolute agreement [intraclass correlation coefficient (ICC2,1)] between marker-based and pressure insole stride parameters was also tested. In the YA cohort, the proposed method outperformed the existing ones, with sensitivity, PPV, and F1 scores ≥ 99% for both GEs and conditions, with a virtually null bias (<10 ms). Overall, temporal inaccuracies minimally impacted stride duration, length, and speed (median absolute errors ≤1%). Similar algorithm performances were obtained for all the other five cohorts in GE detection and propagation to the stride parameters, where an excellent absolute agreement with the pressure insoles was also found (ICC2,1=0.817− 0.999). In conclusion, the proposed method accurately detects GE from marker data under different walking conditions and for a variety of gait impairments.
Collapse
Affiliation(s)
- Tecla Bonci
- Department of Mechanical Engineering, Insigno Institute for In Silico Medicine, The University of Sheffield, Sheffield, United Kingdom
- *Correspondence: Tecla Bonci,
| | - Francesca Salis
- Department of Biomedical Sciences, University of Sassari, Sassari, Italy
| | - Kirsty Scott
- Department of Mechanical Engineering, Insigno Institute for In Silico Medicine, The University of Sheffield, Sheffield, United Kingdom
| | - Lisa Alcock
- Translational and Clinical Research Institute, Faculty of Medical Sciences, Newcastle University, Newcastle Upon Tyne, United Kingdom
| | - Clemens Becker
- Department for Geriatric Rehabilitation, Robert-Bosch-Hospital, Stuttgart, Germany
| | - Stefano Bertuletti
- Department of Biomedical Sciences, University of Sassari, Sassari, Italy
| | - Ellen Buckley
- Department of Mechanical Engineering, Insigno Institute for In Silico Medicine, The University of Sheffield, Sheffield, United Kingdom
| | - Marco Caruso
- Department of Electronics and Telecommunications, Politecnico Di Torino, Torino, Italy
| | - Andrea Cereatti
- Department of Electronics and Telecommunications, Politecnico Di Torino, Torino, Italy
| | - Silvia Del Din
- Translational and Clinical Research Institute, Faculty of Medical Sciences, Newcastle University, Newcastle Upon Tyne, United Kingdom
| | - Eran Gazit
- Centre for the Study of Movement, Cognition and Mobility, Tel Aviv Sourasky Medical Centre, Tel Aviv, Israel
| | - Clint Hansen
- Department of Neurology, University Hospital Schleswig-Holstein, Campus Kiel, Kiel University, Kiel, Germany
| | - Jeffrey M. Hausdorff
- Centre for the Study of Movement, Cognition and Mobility, Tel Aviv Sourasky Medical Centre, Tel Aviv, Israel
- Department of Physical Therapy, Sackler Faculty of Medicine, Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel
- Department of Orthopaedic Surgery, Rush Alzheimer’s Disease Center, Rush University Medical Center, Chicago, IL, United States
| | - Walter Maetzler
- Department of Neurology, University Hospital Schleswig-Holstein, Campus Kiel, Kiel University, Kiel, Germany
| | - Luca Palmerini
- Department of Electrical, Electronic, and Information Engineering “Guglielmo Marconi”, University of Bologna, Bologna, Italy
- Health Sciences and Technologies–Interdepartmental Center for Industrial Research (CIRI-SDV), University of Bologna, Bologna, Italy
| | - Lynn Rochester
- Translational and Clinical Research Institute, Faculty of Medical Sciences, Newcastle University, Newcastle Upon Tyne, United Kingdom
- The Newcastle Upon Tyne Hospitals NHS Foundation Trust, Newcastle Upon Tyne, United Kingdom
| | - Lars Schwickert
- Department for Geriatric Rehabilitation, Robert-Bosch-Hospital, Stuttgart, Germany
| | - Basil Sharrack
- Department of Neuroscience, Sheffield NIHR Translational Neuroscience BRC, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, United Kingdom
| | - Ioannis Vogiatzis
- Department of Sport, Exercise and Rehabilitation, Northumbria University, Newcastle Upon Tyne, United Kingdom
| | - Claudia Mazzà
- Department of Mechanical Engineering, Insigno Institute for In Silico Medicine, The University of Sheffield, Sheffield, United Kingdom
| |
Collapse
|
13
|
Filtjens B, Ginis P, Nieuwboer A, Slaets P, Vanrumste B. Automated freezing of gait assessment with marker-based motion capture and multi-stage spatial-temporal graph convolutional neural networks. J Neuroeng Rehabil 2022; 19:48. [PMID: 35597950 PMCID: PMC9124420 DOI: 10.1186/s12984-022-01025-3] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2021] [Accepted: 05/10/2022] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Freezing of gait (FOG) is a common and debilitating gait impairment in Parkinson's disease. Further insight into this phenomenon is hampered by the difficulty to objectively assess FOG. To meet this clinical need, this paper proposes an automated motion-capture-based FOG assessment method driven by a novel deep neural network. METHODS Automated FOG assessment can be formulated as an action segmentation problem, where temporal models are tasked to recognize and temporally localize the FOG segments in untrimmed motion capture trials. This paper takes a closer look at the performance of state-of-the-art action segmentation models when tasked to automatically assess FOG. Furthermore, a novel deep neural network architecture is proposed that aims to better capture the spatial and temporal dependencies than the state-of-the-art baselines. The proposed network, termed multi-stage spatial-temporal graph convolutional network (MS-GCN), combines the spatial-temporal graph convolutional network (ST-GCN) and the multi-stage temporal convolutional network (MS-TCN). The ST-GCN captures the hierarchical spatial-temporal motion among the joints inherent to motion capture, while the multi-stage component reduces over-segmentation errors by refining the predictions over multiple stages. The proposed model was validated on a dataset of fourteen freezers, fourteen non-freezers, and fourteen healthy control subjects. RESULTS The experiments indicate that the proposed model outperforms four state-of-the-art baselines. Moreover, FOG outcomes derived from MS-GCN predictions had an excellent (r = 0.93 [0.87, 0.97]) and moderately strong (r = 0.75 [0.55, 0.87]) linear relationship with FOG outcomes derived from manual annotations. CONCLUSIONS The proposed MS-GCN may provide an automated and objective alternative to labor-intensive clinician-based FOG assessment. Future work is now possible that aims to assess the generalization of MS-GCN to a larger and more varied verification cohort.
Collapse
Affiliation(s)
- Benjamin Filtjens
- Department of Electrical Engineering (ESAT), eMedia Research Lab/STADIUS, KU Leuven, Andreas Vesaliusstraat 13, 3000, Leuven, Belgium. .,Department of Mechanical Engineering, Intelligent Mobile Platforms Research Group, KU Leuven, Andreas Vesaliusstraat 13, 3000, Leuven, Belgium.
| | - Pieter Ginis
- Department of Rehabilitation Sciences, Research Group for Neurorehabilitation (eNRGy), KU Leuven, Tervuursevest 101, 3001, Heverlee, Belgium
| | - Alice Nieuwboer
- Department of Rehabilitation Sciences, Research Group for Neurorehabilitation (eNRGy), KU Leuven, Tervuursevest 101, 3001, Heverlee, Belgium
| | - Peter Slaets
- Department of Mechanical Engineering, Intelligent Mobile Platforms Research Group, KU Leuven, Andreas Vesaliusstraat 13, 3000, Leuven, Belgium
| | - Bart Vanrumste
- Department of Electrical Engineering (ESAT), eMedia Research Lab/STADIUS, KU Leuven, Andreas Vesaliusstraat 13, 3000, Leuven, Belgium
| |
Collapse
|
14
|
A Deep Learning Approach for Gait Event Detection from a Single Shank-Worn IMU: Validation in Healthy and Neurological Cohorts. SENSORS 2022; 22:s22103859. [PMID: 35632266 PMCID: PMC9143761 DOI: 10.3390/s22103859] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/22/2022] [Revised: 05/12/2022] [Accepted: 05/17/2022] [Indexed: 12/17/2022]
Abstract
Many algorithms use 3D accelerometer and/or gyroscope data from inertial measurement unit (IMU) sensors to detect gait events (i.e., initial and final foot contact). However, these algorithms often require knowledge about sensor orientation and use empirically derived thresholds. As alignment cannot always be controlled for in ambulatory assessments, methods are needed that require little knowledge on sensor location and orientation, e.g., a convolutional neural network-based deep learning model. Therefore, 157 participants from healthy and neurologically diseased cohorts walked 5 m distances at slow, preferred, and fast walking speed, while data were collected from IMUs on the left and right ankle and shank. Gait events were detected and stride parameters were extracted using a deep learning model and an optoelectronic motion capture (OMC) system for reference. The deep learning model consisted of convolutional layers using dilated convolutions, followed by two independent fully connected layers to predict whether a time step corresponded to the event of initial contact (IC) or final contact (FC), respectively. Results showed a high detection rate for both initial and final contacts across sensor locations (recall ≥92%, precision ≥97%). Time agreement was excellent as witnessed from the median time error (0.005 s) and corresponding inter-quartile range (0.020 s). The extracted stride-specific parameters were in good agreement with parameters derived from the OMC system (maximum mean difference 0.003 s and corresponding maximum limits of agreement (−0.049 s, 0.051 s) for a 95% confidence level). Thus, the deep learning approach was considered a valid approach for detecting gait events and extracting stride-specific parameters with little knowledge on exact IMU location and orientation in conditions with and without walking pathologies due to neurological diseases.
Collapse
|
15
|
Foot-to-Ground Phases Detection: A Comparison of Data Representation Formatting Methods with Respect to Adaption of Deep Learning Architectures. COMPUTERS 2022. [DOI: 10.3390/computers11050058] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
Abstract
Identifying the foot stance and foot swing phases, also known as foot-to-ground (FTG) detection, is a branch of Human Activity Recognition (HAR). Our study aims to detect two main phases of the gait (i.e., foot-off and foot-contact) corresponding to the moments when each foot is in contact with the ground or not. This will allow the medical professionals to characterize and identify the different phases of the human gait and their respective patterns. This detection process is paramount for extracting gait features (e.g., step width, stride width, gait speed, cadence, etc.) used by medical experts to highlight gait anomalies, stance issues, or any other walking irregularities. It will be used to assist health practitioners with patient monitoring, in addition to developing a full pipeline for FTG detection that would help compute gait indicators. In this paper, a comparison of different training configurations, including model architectures, data formatting, and pre-processing, was conducted to select the parameters leading to the highest detection accuracy. This binary classification provides a label for each timestamp informing whether the foot is in contact with the ground or not. Models such as CNN, LSTM, and ConvLSTM were the best fits for this study. Yet, we did not exclude DNNs and Machine Learning models, such as Random Forest and XGBoost from our work in order to have a wide range of possible comparisons. As a result of our experiments, which included 27 senior participants who had a stroke in the past wearing IMU sensors on their ankles, the ConvLSTM model achieved a high accuracy of 97.01% for raw windowed data with a size of 3 frames per window, and each window was formatted to have two superimposed channels (accelerometer and gyroscope channels). The model was trained to have the best detection without any knowledge of the participants’ personal information including age, gender, health condition, the type of activity, or the used foot. In other words, the model’s input data only originated from IMU sensors. Overall, in terms of FTG detection, the combination of the ConvLSTM model and the data representation had an important impact in outperforming other start-of-the-art configurations; in addition, the compromise between the model’s complexity and its accuracy is a major asset for deploying this model and developing real-time solutions.
Collapse
|
16
|
Jamsrandorj A, Nguyen MD, Park M, Kumar KS, Mun KR, Kim J. Vision-Based Gait Events Detection Using Deep Convolutional Neural Networks. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:1936-1941. [PMID: 34891666 DOI: 10.1109/embc46164.2021.9630431] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Accurate gait events detection from the video would be a challenging problem. However, most vision-based methods for gait event detection highly rely on gait features that are estimated using gait silhouettes and human pose information for accurate gait data acquisition. This paper presented an accurate, multi-view approach with deep convolutional neural networks for efficient and practical gait event detection without requiring additional gait feature engineering. Especially, we aimed to detect gait events from frontal views as well as lateral views. We conducted the experiments with four different deep CNN models on our own dataset that includes three different walking actions from 11 healthy participants. Models took 9 subsequence frames stacking together as inputs, while outputs of models were probability vectors of gait events: toe-off and heel-strike for each frame. The deep CNN models trained only with video frames enabled to detect gait events with 93% or higher accuracy while the user is walking straight and walking around on both frontal and lateral views.
Collapse
|
17
|
Gait Phase Estimation by Using LSTM in IMU-Based Gait Analysis-Proof of Concept. SENSORS 2021; 21:s21175749. [PMID: 34502640 PMCID: PMC8433817 DOI: 10.3390/s21175749] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/11/2021] [Revised: 08/10/2021] [Accepted: 08/23/2021] [Indexed: 12/22/2022]
Abstract
Gait phase detection in IMU-based gait analysis has some limitations due to walking style variations and physical impairments of individuals. Therefore, available algorithms may not work properly when the gait data is noisy, or the person rarely reaches a steady state of walking. The aim of this work was to employ Artificial Intelligence (AI), specifically a long short-term memory (LSTM) algorithm, to overcome these weaknesses. Three supervised LSTM-based models were designed to estimate the expected gait phases, including foot-off (FO), mid-swing (MidS) and foot-contact (FC). For collecting gait data two tri-axial inertial sensors were located above each ankle. The angular velocity magnitude, rotation matrix magnitude and free acceleration magnitude were captured for data labeling and turning detection and to strengthen the model, respectively. To do so, a train dataset based on a novel movement protocol was acquired. A validation dataset similar to a train dataset was generated as well. Five test datasets from already existing data were also created to independently evaluate the models. After testing the models on validation and test datasets, all three models demonstrated promising performance in estimating desired gait phases. The proposed approach proves the possibility of employing AI-based algorithms to predict labeled gait phases from a time series of gait data.
Collapse
|
18
|
Rivadulla A, Chen X, Weir G, Cazzola D, Trewartha G, Hamill J, Preatoni E. Development and validation of FootNet; a new kinematic algorithm to improve foot-strike and toe-off detection in treadmill running. PLoS One 2021; 16:e0248608. [PMID: 34370747 PMCID: PMC8351929 DOI: 10.1371/journal.pone.0248608] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2021] [Accepted: 07/05/2021] [Indexed: 11/20/2022] Open
Abstract
The accurate detection of foot-strike and toe-off is often critical in the assessment of running biomechanics. The gold standard method for step event detection requires force data which are not always available. Although kinematics-based algorithms can also be used, their accuracy and generalisability are limited, often requiring corrections for speed or foot-strike pattern. The purpose of this study was to develop FootNet, a novel kinematics and deep learning-based algorithm for the detection of step events in treadmill running. Five treadmill running datasets were gathered and processed to obtain segment and joint kinematics, and to identify the contact phase within each gait cycle using force data. The proposed algorithm is based on a long short-term memory recurrent neural network and takes the distal tibia anteroposterior velocity, ankle dorsiflexion/plantar flexion angle and the anteroposterior and vertical velocities of the foot centre of mass as input features to predict the contact phase within a given gait cycle. The chosen model architecture underwent 5-fold cross-validation and the final model was tested in a subset of participants from each dataset (30%). Non-parametric Bland-Altman analyses (bias and [95% limits of agreement]) and root mean squared error (RMSE) were used to compare FootNet against the force data step event detection method. The association between detection errors and running speed, foot-strike angle and incline were also investigated. FootNet outperformed previously published algorithms (foot-strike bias = 0 [–10, 7] ms, RMSE = 5 ms; toe-off bias = 0 [–10, 10] ms, RMSE = 6 ms; and contact time bias = 0 [–15, 15] ms, RMSE = 8 ms) and proved robust to different running speeds, foot-strike angles and inclines. We have made FootNet’s source code publicly available for step event detection in treadmill running when force data are not available.
Collapse
Affiliation(s)
- Adrian Rivadulla
- Department for Health, University of Bath, Bath, United Kingdom
- * E-mail:
| | - Xi Chen
- Department of Computer Science, University of Bath, Bath, United Kingdom
| | - Gillian Weir
- Department of Kinesiology, University of Massachusetts Amherst, Amherst, MA, United States of America
| | - Dario Cazzola
- Department for Health, University of Bath, Bath, United Kingdom
| | - Grant Trewartha
- Department for Health, University of Bath, Bath, United Kingdom
| | - Joseph Hamill
- Department of Kinesiology, University of Massachusetts Amherst, Amherst, MA, United States of America
| | - Ezio Preatoni
- Department for Health, University of Bath, Bath, United Kingdom
| |
Collapse
|
19
|
Huang L, Zheng J, Hu H. Online Gait Phase Detection in Complex Environment Based on Distance and Multi-Sensors Information Fusion Using Inertial Measurement Units. Int J Soc Robot 2021. [DOI: 10.1007/s12369-021-00794-6] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
20
|
Morbidoni C, Cucchiarelli A, Agostini V, Knaflitz M, Fioretti S, Di Nardo F. Machine-Learning-Based Prediction of Gait Events From EMG in Cerebral Palsy Children. IEEE Trans Neural Syst Rehabil Eng 2021; 29:819-830. [PMID: 33909568 DOI: 10.1109/tnsre.2021.3076366] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Machine-learning techniques are suitably employed for gait-event prediction from only surface electromyographic (sEMG) signals in control subjects during walking. Nevertheless, a reference approach is not available in cerebral-palsy hemiplegic children, likely due to the large variability of foot-floor contacts. This study is designed to investigate a machine-learning-based approach, specifically developed to binary classify gait events and to predict heel-strike (HS) and toe-off (TO) timing from sEMG signals in hemiplegic-child walking. To this objective, sEMG signals are acquired from five hemiplegic-leg muscles in nearly 2500 strides from 20 hemiplegic children, acknowledged as Winters' group 1 and 2. sEMG signals, segmented in overlapping windows of 600 samples (pace = 5 samples), are used to train a multi-layer perceptron model. Intra-subject and inter-subject experimental settings are tested. The best-performing intra-subject approach is able to provide in the hemiplegic population a mean classification accuracy (±SD) of 0.97±0.01 and a suitable prediction of HS and TO events, in terms of average mean absolute error (MAE, 14.8±3.2 ms for HS and 17.6±4.2 ms for TO) and F1-score (0.95±0.03 for HS and 0.92±0.07 for TO). These results outperform previous sEMG-based attempts in cerebral-palsy populations and are comparable with outcomes achieved by reference approaches in control populations. In conclusion, the findings of the study prove the feasibility of neural networks in predicting the two main gait events using surface EMG signals, also in condition of high variability of the signal to predict as in hemiplegic cerebral palsy.
Collapse
|
21
|
Vu HTT, Dong D, Cao HL, Verstraten T, Lefeber D, Vanderborght B, Geeroms J. A Review of Gait Phase Detection Algorithms for Lower Limb Prostheses. SENSORS (BASEL, SWITZERLAND) 2020; 20:E3972. [PMID: 32708924 PMCID: PMC7411778 DOI: 10.3390/s20143972] [Citation(s) in RCA: 42] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/11/2020] [Revised: 07/08/2020] [Accepted: 07/15/2020] [Indexed: 01/01/2023]
Abstract
Fast and accurate gait phase detection is essential to achieve effective powered lower-limb prostheses and exoskeletons. As the versatility but also the complexity of these robotic devices increases, the research on how to make gait detection algorithms more performant and their sensing devices smaller and more wearable gains interest. A functional gait detection algorithm will improve the precision, stability, and safety of prostheses, and other rehabilitation devices. In the past years the state-of-the-art has advanced significantly in terms of sensors, signal processing, and gait detection algorithms. In this review, we investigate studies and developments in the field of gait event detection methods, more precisely applied to prosthetic devices. We compared advantages and limitations between all the proposed methods and extracted the relevant questions and recommendations about gait detection methods for future developments.
Collapse
Affiliation(s)
- Huong Thi Thu Vu
- Robotics & Multibody Mechanics Research Group (R & MM), Vrije Universiteit Brussel and Flanders Make, 1050 Brussels, Belgium; (D.D.); (H.-L.C.); (T.V.); (D.L.); (B.V.); (J.G.)
- Faculty of Electronics Engineering Technology, Hanoi University of Industry, Hanoi 100000, Vietnam
| | - Dianbiao Dong
- Robotics & Multibody Mechanics Research Group (R & MM), Vrije Universiteit Brussel and Flanders Make, 1050 Brussels, Belgium; (D.D.); (H.-L.C.); (T.V.); (D.L.); (B.V.); (J.G.)
- School of Mechanical Engineering, Northwestern Polytechnical University, Xi’an 710072, China
| | - Hoang-Long Cao
- Robotics & Multibody Mechanics Research Group (R & MM), Vrije Universiteit Brussel and Flanders Make, 1050 Brussels, Belgium; (D.D.); (H.-L.C.); (T.V.); (D.L.); (B.V.); (J.G.)
- College of Engineering Technology, Can Tho University, Can Tho 90000, Vietnam
| | - Tom Verstraten
- Robotics & Multibody Mechanics Research Group (R & MM), Vrije Universiteit Brussel and Flanders Make, 1050 Brussels, Belgium; (D.D.); (H.-L.C.); (T.V.); (D.L.); (B.V.); (J.G.)
| | - Dirk Lefeber
- Robotics & Multibody Mechanics Research Group (R & MM), Vrije Universiteit Brussel and Flanders Make, 1050 Brussels, Belgium; (D.D.); (H.-L.C.); (T.V.); (D.L.); (B.V.); (J.G.)
| | - Bram Vanderborght
- Robotics & Multibody Mechanics Research Group (R & MM), Vrije Universiteit Brussel and Flanders Make, 1050 Brussels, Belgium; (D.D.); (H.-L.C.); (T.V.); (D.L.); (B.V.); (J.G.)
| | - Joost Geeroms
- Robotics & Multibody Mechanics Research Group (R & MM), Vrije Universiteit Brussel and Flanders Make, 1050 Brussels, Belgium; (D.D.); (H.-L.C.); (T.V.); (D.L.); (B.V.); (J.G.)
| |
Collapse
|
22
|
Dot T, Quijoux F, Oudre L, Vienne-Jumeau A, Moreau A, Vidal PP, Ricard D. Non-Linear Template-Based Approach for the Study of Locomotion. SENSORS 2020; 20:s20071939. [PMID: 32235667 PMCID: PMC7180476 DOI: 10.3390/s20071939] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/24/2020] [Revised: 03/17/2020] [Accepted: 03/26/2020] [Indexed: 12/25/2022]
Abstract
The automatic detection of gait events (i.e., Initial Contact (IC) and Final Contact (FC)) is crucial for the characterisation of gait from Inertial Measurements Units. In this article, we present a method for detecting steps (i.e., IC and FC) from signals of gait sequences of individuals recorded with a gyrometer. The proposed approach combines the use of a dictionary of templates and a Dynamic Time Warping (DTW) measure of fit to retrieve these templates into input signals. Several strategies for choosing and learning the adequate templates from annotated data are also described. The method is tested on thirteen healthy subjects and compared to gold standard. Depending of the template choice, the proposed algorithm achieves average errors from 0.01 to 0.03 s for the detection of IC, FC and step duration. Results demonstrate that the use of DTW allows achieving these performances with only one single template. DTW is a convenient tool to perform pattern recognition on gait gyrometer signals. This study paves the way for new step detection methods: it shows that using one single template associated with non-linear deformations may be sufficient to model the gait of healthy subjects.
Collapse
Affiliation(s)
- Tristan Dot
- Université Paris-Saclay, ENS Paris-Saclay, CNRS, Centre Borelli, F-94235 Cachan, France
- Université de Paris, CNRS, Centre Borelli, F-75005 Paris, France
| | - Flavien Quijoux
- Université Paris-Saclay, ENS Paris-Saclay, CNRS, Centre Borelli, F-94235 Cachan, France
- Université de Paris, CNRS, Centre Borelli, F-75005 Paris, France
- ORPEA Group, F-92813 Puteaux, France
| | - Laurent Oudre
- Université Sorbonne Paris Nord, L2TI, UR 3043, F-93430 Villetaneuse, France
- Correspondence: ; Tel.: +33-1-49-40-40-63
| | - Aliénor Vienne-Jumeau
- Université Paris-Saclay, ENS Paris-Saclay, CNRS, Centre Borelli, F-94235 Cachan, France
- Université de Paris, CNRS, Centre Borelli, F-75005 Paris, France
| | - Albane Moreau
- Université Paris-Saclay, ENS Paris-Saclay, CNRS, Centre Borelli, F-94235 Cachan, France
- Université de Paris, CNRS, Centre Borelli, F-75005 Paris, France
- Service de Neurologie, Service de Santé des Armées, Hôpital d’Instruction des Armées Percy, F-92190 Clamart, France
| | - Pierre-Paul Vidal
- Université Paris-Saclay, ENS Paris-Saclay, CNRS, Centre Borelli, F-94235 Cachan, France
- Université de Paris, CNRS, Centre Borelli, F-75005 Paris, France
- Hangzhou Dianzi University, Hangzhou C-310005, China
| | - Damien Ricard
- Université Paris-Saclay, ENS Paris-Saclay, CNRS, Centre Borelli, F-94235 Cachan, France
- Université de Paris, CNRS, Centre Borelli, F-75005 Paris, France
- Service de Neurologie, Service de Santé des Armées, Hôpital d’Instruction des Armées Percy, F-92190 Clamart, France
- Ecole du Val-de-Grâce, Ecole de Santé des Armées, F-75005 Paris, France
| |
Collapse
|