1
|
Vun DSY, Bowers R, McGarry A. Vision-based motion capture for the gait analysis of neurodegenerative diseases: A review. Gait Posture 2024; 112:95-107. [PMID: 38754258 DOI: 10.1016/j.gaitpost.2024.04.029] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/01/2023] [Revised: 04/25/2024] [Accepted: 04/26/2024] [Indexed: 05/18/2024]
Abstract
BACKGROUND Developments in vision-based systems and human pose estimation algorithms have the potential to detect, monitor and intervene early on neurodegenerative diseases through gait analysis. However, the gap between the technology available and actual clinical practice is evident as most clinicians still rely on subjective observational gait analysis or objective marker-based analysis that is time-consuming. RESEARCH QUESTION This paper aims to examine the main developments of vision-based motion capture and how such advances may be integrated into clinical practice. METHODS The literature review was conducted in six online databases using Boolean search terms. A commercial system search was also included. A predetermined methodological criterion was then used to assess the quality of the selected articles. RESULTS A total of seventeen studies were evaluated, with thirteen studies focusing on gait classification systems and four studies on gait measurement systems. Of the gait classification systems, nine studies utilized artificial intelligence-assisted techniques, while four studies employed statistical techniques. The results revealed high correlations of gait features identified by classifier models with existing clinical rating scales. These systems demonstrated generally high classification accuracies and were effective in diagnosing disease severity levels. Gait measurement systems that extract spatiotemporal and kinematic joint information from video data generally found accurate measurements of gait parameters with low mean absolute errors, high intra- and inter-rater reliability. SIGNIFICANCE Low cost, portable vision-based systems can provide proof of concept for the quantification of gait, expansion of gait assessment tools, remote gait analysis of neurodegenerative diseases and a point of care system for orthotic evaluation. However, certain challenges, including small sample sizes, occlusion risks, and selection bias in training models, need to be addressed. Nevertheless, these systems can serve as complementary tools, equipping clinicians with essential gait information to objectively assess disease severity and tailor personalized treatment for enhanced patient care.
Collapse
Affiliation(s)
- David Sing Yee Vun
- National Centre for Prosthetics and Orthotics, Department of Biomedical Engineering, University of Strathclyde, Glasgow, UK
| | - Robert Bowers
- National Centre for Prosthetics and Orthotics, Department of Biomedical Engineering, University of Strathclyde, Glasgow, UK
| | - Anthony McGarry
- National Centre for Prosthetics and Orthotics, Department of Biomedical Engineering, University of Strathclyde, Glasgow, UK.
| |
Collapse
|
2
|
Liang A. Assessing gait dysfunction severity in Parkinson's Disease using 2-Stream Spatial-Temporal Neural Network. J Biomed Inform 2024:104679. [PMID: 38925280 DOI: 10.1016/j.jbi.2024.104679] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2024] [Revised: 06/18/2024] [Accepted: 06/19/2024] [Indexed: 06/28/2024]
Abstract
Parkinson's Disease (PD), a neurodegenerative disorder, significantly impacts the quality of life for millions of people worldwide. PD primarily impacts dopaminergic neurons in the brain's substantia nigra, resulting in dopamine deficiency and gait impairments such as bradykinesia and rigidity. Currently, several well-established tools, such as the Movement Disorder Society-Unified Parkinson's Disease Rating Scale (MDS-UPDRS) and Hoehn and Yahr (H&Y) Scale, are used for evaluating gait dysfunction in PD. While insightful, these methods are subjective, time-consuming, and often ineffective in early-stage diagnosis. Other methods using specialized sensors and equipment to measure movement disorders are cumbersome and expensive, limiting their accessibility. This study introduces a hierarchical approach to evaluating gait dysfunction in PD through videos. The novel 2-Stream Spatial-Temporal Neural Network (2S-STNN) leverages the spatial-temporal features from the skeleton and silhouette streams for PD classification. This approach achieves an accuracy rate of 89% and outperforms other state-of-the-art models. The study also employs saliency values to highlight critical body regions that significantly influence model decisions and are severely affected by the disease. For a more detailed analysis, the study investigates 21 specific gait attributes for a more nuanced quantification of gait disorders. Parameters such as walking pace, step length, and neck forward angle are found to be strongly correlated with PD gait severity categories. This approach offers a comprehensive and convenient solution for PD management in clinical settings, enabling patients to receive a more precise evaluation and monitoring of their gait impairments.
Collapse
Affiliation(s)
- Andrew Liang
- The Harker School, 500 Saratoga Ave, San Jose, 95129, USA.
| |
Collapse
|
3
|
Bonato P, Feipel V, Corniani G, Arin-Bal G, Leardini A. Position paper on how technology for human motion analysis and relevant clinical applications have evolved over the past decades: Striking a balance between accuracy and convenience. Gait Posture 2024; 113:191-203. [PMID: 38917666 DOI: 10.1016/j.gaitpost.2024.06.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/24/2024] [Revised: 05/30/2024] [Accepted: 06/10/2024] [Indexed: 06/27/2024]
Abstract
BACKGROUND Over the past decades, tremendous technological advances have emerged in human motion analysis (HMA). RESEARCH QUESTION How has technology for analysing human motion evolved over the past decades, and what clinical applications has it enabled? METHODS The literature on HMA has been extensively reviewed, focusing on three main approaches: Fully-Instrumented Gait Analysis (FGA), Wearable Sensor Analysis (WSA), and Deep-Learning Video Analysis (DVA), considering both technical and clinical aspects. RESULTS FGA techniques relying on data collected using stereophotogrammetric systems, force plates, and electromyographic sensors have been dramatically improved providing highly accurate estimates of the biomechanics of motion. WSA techniques have been developed with the advances in data collection at home and in community settings. DVA techniques have emerged through artificial intelligence, which has marked the last decade. Some authors have considered WSA and DVA techniques as alternatives to "traditional" HMA techniques. They have suggested that WSA and DVA techniques are destined to replace FGA. SIGNIFICANCE We argue that FGA, WSA, and DVA complement each other and hence should be accounted as "synergistic" in the context of modern HMA and its clinical applications. We point out that DVA techniques are especially attractive as screening techniques, WSA methods enable data collection in the home and community for extensive periods of time, and FGA does maintain superior accuracy and should be the preferred technique when a complete and highly accurate biomechanical data is required. Accordingly, we envision that future clinical applications of HMA would favour screening patients using DVA in the outpatient setting. If deemed clinically appropriate, then WSA would be used to collect data in the home and community to derive relevant information. If accurate kinetic data is needed, then patients should be referred to specialized centres where an FGA system is available, together with medical imaging and thorough clinical assessments.
Collapse
Affiliation(s)
- Paolo Bonato
- Dept of PM&R, Harvard Medical School, Spaulding Rehabilitation Hospital, Boston, USA
| | - Véronique Feipel
- Laboratory of Functional Anatomy, Faculty of Motor Sciences, Laboratory of Anatomy, Biomechanics and Organogenesis, Faculty of Medicine, Université Libre de Bruxelles, Brussels, Belgium
| | - Giulia Corniani
- Dept of PM&R, Harvard Medical School, Spaulding Rehabilitation Hospital, Boston, USA
| | - Gamze Arin-Bal
- Faculty of Physical Therapy and Rehabilitation, Hacettepe University, Ankara, Turkey; Movement Analysis Laboratory, IRCCS Istituto Ortopedico Rizzoli, Bologna, Italy.
| | - Alberto Leardini
- Movement Analysis Laboratory, IRCCS Istituto Ortopedico Rizzoli, Bologna, Italy
| |
Collapse
|
4
|
Tian H, Li H, Jiang W, Ma X, Li X, Wu H, Li Y. Cross-Spatiotemporal Graph Convolution Networks for Skeleton-Based Parkinsonian Gait MDS-UPDRS Score Estimation. IEEE Trans Neural Syst Rehabil Eng 2024; 32:412-421. [PMID: 38198272 DOI: 10.1109/tnsre.2024.3352004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2024]
Abstract
Gait impairment in Parkinson's Disease (PD) is quantitatively assessed using the Movement Disorder Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS), a well-established clinical tool. Objective and efficient PD gait assessment is crucial for developing interventions to slow or halt its advancement. Skeleton-based PD gait MDS-UPDRS score estimation has attracted increasing interest in improving diagnostic efficiency and objectivity. However, previous works ignore the important cross-spacetime dependencies between joints in PD gait. Moreover, existing PD gait skeleton datasets are very small, which is a big issue in deep learning-based gait studies. In this work, we collect a sizable PD gait skeleton dataset by multi-view Azure Kinect sensors. The collected dataset contains 102 PD patients and 30 healthy older adults. In addition, gait data from 16 young adults (aged 24-50 years) are collected to further examine the effect of age on PD gait assessment. For skeleton-based automatic PD gait analysis, we propose a novel cross-spatiotemporal graph convolution network (CST-GCN) to learn complex features of gait patterns. Specifically, a gait graph labeling strategy is designed to assemble and group cross-spacetime neighbors of the root node according to the spatiotemporal semantics of the gait skeleton. Based on this strategy, the CST-GCN module explicitly models the cross-spacetime dependencies among joints. Finally, a dual-path model is presented to realize the modeling and fusion of spatial, temporal, and cross-spacetime gait features. Extensive experiments validate the effectiveness of our method on the collected dataset.
Collapse
|
5
|
Jamsrandorj A, Jung D, Kumar KS, Arshad MZ, Lim H, Kim J, Mun KR. View-independent gait events detection using CNN-transformer hybrid network. J Biomed Inform 2023; 147:104524. [PMID: 37838288 DOI: 10.1016/j.jbi.2023.104524] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2023] [Revised: 09/11/2023] [Accepted: 10/10/2023] [Indexed: 10/16/2023]
Abstract
Accurate gait detection is crucial in utilizing the ample health information embedded in it. Vision-based approaches for gait detection have emerged as an alternative to the exacting sensor-based approaches, but their application has been rather limited due to complicated feature engineering processes and heavy reliance on lateral views. Thus, this study aimed to find a simple vision-based approach that is view-independent and accurate. A total of 22 participants performed six different actions representing standard and peculiar gaits, and the videos acquired from these actions were used as the input of the deep learning networks. Four networks, including a 2D convolutional neural network and an attention-based deep learning network, were trained with standard gaits, and their detection performance for both standard and peculiar gaits was assessed using measures including F1-scores. While all networks achieved remarkable detection performance, the CNN-Transformer network achieved the best performance for both standard and peculiar gaits. Little deviation by the speed of actions or view angles was found. The study is expected to contribute to the wider application of vision-based approaches in gait detection and gait-based health monitoring both at home and in clinical settings.
Collapse
Affiliation(s)
- Ankhzaya Jamsrandorj
- Department of Human Computer Interface & Robotics Engineering, KIST School, University of Science & Technology, Seoul, Republic of Korea; Center for Artificial Intelligence, Korea Institute of Science and Technology, Seoul, Republic of Korea
| | - Dawoon Jung
- Center for Artificial Intelligence, Korea Institute of Science and Technology, Seoul, Republic of Korea
| | - Konki Sravan Kumar
- Center for Artificial Intelligence, Korea Institute of Science and Technology, Seoul, Republic of Korea
| | - Muhammad Zeeshan Arshad
- Center for Artificial Intelligence, Korea Institute of Science and Technology, Seoul, Republic of Korea
| | - Hwasup Lim
- Center for Artificial Intelligence, Korea Institute of Science and Technology, Seoul, Republic of Korea
| | - Jinwook Kim
- Center for Artificial Intelligence, Korea Institute of Science and Technology, Seoul, Republic of Korea.
| | - Kyung-Ryoul Mun
- Center for Artificial Intelligence, Korea Institute of Science and Technology, Seoul, Republic of Korea; KHU-KIST Department of Converging Science and Technology, Kyung Hee University, Seoul, Republic of Korea.
| |
Collapse
|
6
|
Gupta R, Kumari S, Senapati A, Ambasta RK, Kumar P. New era of artificial intelligence and machine learning-based detection, diagnosis, and therapeutics in Parkinson's disease. Ageing Res Rev 2023; 90:102013. [PMID: 37429545 DOI: 10.1016/j.arr.2023.102013] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2023] [Revised: 06/26/2023] [Accepted: 07/06/2023] [Indexed: 07/12/2023]
Abstract
Parkinson's disease (PD) is characterized by the loss of neuronal cells, which leads to synaptic dysfunction and cognitive defects. Despite the advancements in treatment strategies, the management of PD is still a challenging event. Early prediction and diagnosis of PD are of utmost importance for effective management of PD. In addition, the classification of patients with PD as compared to normal healthy individuals also imposes drawbacks in the early diagnosis of PD. To address these challenges, artificial intelligence (AI) and machine learning (ML) models have been implicated in the diagnosis, prediction, and treatment of PD. Recent times have also demonstrated the implication of AI and ML models in the classification of PD based on neuroimaging methods, speech recording, gait abnormalities, and others. Herein, we have briefly discussed the role of AI and ML in the diagnosis, treatment, and identification of novel biomarkers in the progression of PD. We have also highlighted the role of AI and ML in PD management through altered lipidomics and gut-brain axis. We briefly explain the role of early PD detection through AI and ML algorithms based on speech recordings, handwriting patterns, gait abnormalities, and neuroimaging techniques. Further, the review discuss the potential role of the metaverse, the Internet of Things, and electronic health records in the effective management of PD to improve the quality of life. Lastly, we also focused on the implementation of AI and ML-algorithms in neurosurgical process and drug discovery.
Collapse
Affiliation(s)
- Rohan Gupta
- Molecular Neuroscience and Functional Genomics Laboratory, Department of Biotechnology, Delhi Technological, University, USA.
| | - Smita Kumari
- Molecular Neuroscience and Functional Genomics Laboratory, Department of Biotechnology, Delhi Technological, University, USA
| | | | - Rashmi K Ambasta
- Molecular Neuroscience and Functional Genomics Laboratory, Department of Biotechnology, Delhi Technological, University, USA
| | - Pravir Kumar
- Molecular Neuroscience and Functional Genomics Laboratory, Department of Biotechnology, Delhi Technological, University, USA.
| |
Collapse
|
7
|
Korivand S, Jalili N, Gong J. Experiment protocols for brain-body imaging of locomotion: A systematic review. Front Neurosci 2023; 17:1051500. [PMID: 36937690 PMCID: PMC10014824 DOI: 10.3389/fnins.2023.1051500] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2022] [Accepted: 02/06/2023] [Indexed: 03/05/2023] Open
Abstract
Introduction Human locomotion is affected by several factors, such as growth and aging, health conditions, and physical activity levels for maintaining overall health and well-being. Notably, impaired locomotion is a prevalent cause of disability, significantly impacting the quality of life of individuals. The uniqueness and high prevalence of human locomotion have led to a surge of research to develop experimental protocols for studying the brain substrates, muscle responses, and motion signatures associated with locomotion. However, from a technical perspective, reproducing locomotion experiments has been challenging due to the lack of standardized protocols and benchmarking tools, which impairs the evaluation of research quality and the validation of previous findings. Methods This paper addresses the challenges by conducting a systematic review of existing neuroimaging studies on human locomotion, focusing on the settings of experimental protocols, such as locomotion intensity, duration, distance, adopted brain imaging technologies, and corresponding brain activation patterns. Also, this study provides practical recommendations for future experiment protocols. Results The findings indicate that EEG is the preferred neuroimaging sensor for detecting brain activity patterns, compared to fMRI, fNIRS, and PET. Walking is the most studied human locomotion task, likely due to its fundamental nature and status as a reference task. In contrast, running has received little attention in research. Additionally, cycling on an ergometer at a speed of 60 rpm using fNIRS has provided some research basis. Dual-task walking tasks are typically used to observe changes in cognitive function. Moreover, research on locomotion has primarily focused on healthy individuals, as this is the scenario most closely resembling free-living activity in real-world environments. Discussion Finally, the paper outlines the standards and recommendations for setting up future experiment protocols based on the review findings. It discusses the impact of neurological and musculoskeletal factors, as well as the cognitive and locomotive demands, on the experiment design. It also considers the limitations imposed by the sensing techniques used, including the acceptable level of motion artifacts in brain-body imaging experiments and the effects of spatial and temporal resolutions on brain sensor performance. Additionally, various experiment protocol constraints that need to be addressed and analyzed are explained.
Collapse
Affiliation(s)
- Soroush Korivand
- Department of Mechanical Engineering, The University of Alabama, Tuscaloosa, AL, United States
- Department of Computer Science, The University of Alabama, Tuscaloosa, AL, United States
| | - Nader Jalili
- Department of Mechanical Engineering, The University of Alabama, Tuscaloosa, AL, United States
| | - Jiaqi Gong
- Department of Computer Science, The University of Alabama, Tuscaloosa, AL, United States
- *Correspondence: Jiaqi Gong
| |
Collapse
|
8
|
Guo Y, Yang J, Liu Y, Chen X, Yang GZ. Detection and assessment of Parkinson's disease based on gait analysis: A survey. Front Aging Neurosci 2022; 14:916971. [PMID: 35992585 PMCID: PMC9382193 DOI: 10.3389/fnagi.2022.916971] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2022] [Accepted: 07/08/2022] [Indexed: 11/13/2022] Open
Abstract
Neurological disorders represent one of the leading causes of disability and mortality in the world. Parkinson's Disease (PD), for example, affecting millions of people worldwide is often manifested as impaired posture and gait. These impairments have been used as a clinical sign for the early detection of PD, as well as an objective index for pervasive monitoring of the PD patients in daily life. This review presents the evidence that demonstrates the relationship between human gait and PD, and illustrates the role of different gait analysis systems based on vision or wearable sensors. It also provides a comprehensive overview of the available automatic recognition systems for the detection and management of PD. The intervening measures for improving gait performance are summarized, in which the smart devices for gait intervention are emphasized. Finally, this review highlights some of the new opportunities in detecting, monitoring, and treating of PD based on gait, which could facilitate the development of objective gait-based biomarkers for personalized support and treatment of PD.
Collapse
Affiliation(s)
- Yao Guo
- Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China
| | - Jianxin Yang
- Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China
| | - Yuxuan Liu
- Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China
| | - Xun Chen
- Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China
| | - Guang-Zhong Yang
- Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China
| |
Collapse
|