Combining features on vertical ground reaction force signal analysis for multiclass diagnosing neurodegenerative diseases

Neurodegenerative diseases (NDDs), which are caused by the degeneration of neurons and their functions, affect a significant part of the world's population. Although gait disorders are one of the critical and common markers to determine the presence of NDDs, diagnosing which NDD the patients have among a group of NDDs using gait data is still a significant challenge to be addressed. In this study, we addressed the multi-class classification of NDDs and aim to diagnose Parkinson's disease (PD), Amyotrophic lateral sclerosis disease (AD), and Huntington's disease (HD) from a group containing NDDs and healthy control subjects. We also examined the impact of disease-specific identified features derived from VGRF signals. Detrended Fluctuation Analysis (DFA), Dynamic Time Warping (DTW) and Autocorrelation (AC) were used for feature extraction on Vertical Ground Reaction Force (VGRF) signals. To compare the performance of the features, we employed Support Vector Machines, K-Nearest Neighbors, and Neural Networks as classifiers. In three-class problem addressing the classification of AD, PD and HD 93.3% accuracy rate was achieved, while in the four classes case, in which NDDs and HC groups were considered together, 93.5% accuracy rate was yielded. Considering the disease-specific impact of features, it is revealed that while DFA based features diagnose patients with AD with the highest accuracy, DTW has been shown to be more successful in diagnosing PD. AC based features provided the highest accuracy in diagnosing HD. Although gait disorder is common for NDDs, each disease may have its own distinctive gait rhythms; therefore, it is important to identify disease-specific patterns and parameters for the diagnosis of each disease. To increase the diagnostic accuracy, it is necessary to use a combination of features, which were effective for each disease diagnosis. Determining a limited number of disease-specific features would provide NDD diagnostic systems suitable to be deployed in edge-computing environments.

Evaluating the effects of safety incentives on worker safety behavior control through image-based activity classification

Introduction

Construction worker safety remains a major concern even as task automation increases. Although safety incentives have been introduced to encourage safety compliance, it is still difficult to accurately measure the effectiveness of these measures. A simple count of accident rates and lower numbers do not necessarily mean that workers are properly complying with safety regulations. To address this problem, this study proposes an image-based approach to monitor moment-by-moment worker safety behavior and evaluate the effects of different safety incentive scenarios.

Methods

By capturing workers' safety behaviors using a model integrated with OpenPose and spatiotemporal graph convolutional network, this study evaluated the effects of safety-incentive scenarios on workers' compliance with rules while on the job. The safety incentive scenarios in this study were designed as 1) varying the type (i.e., providing rewards and penalties) of incentives and 2) varying the frequency of feedback about ones' own compliance status during tasks. The effects of the scenarios were compared to the average compliance rates of three safety regulations (i.e., personal protective equipment self-monitoring hazard avoidance, and arranging the safety hook) for each scenario.

Results

The results show that 1) rewarding a good-compliance is more effective when there is no feedback on compliance status, and 2) penalizing non-compliance is more effective when there are three feedbacks during the tasks.

Discussion

This study provides a more accurate assessment of safety incentives and their effectiveness by focusing on safe behaviors to promote safety compliance among construction workers.

Markerless Motion Capture to Quantify Functional Performance in Neurodegeneration: Systematic Review

BACKGROUND

Markerless motion capture (MMC) uses video cameras or depth sensors for full body tracking and presents a promising approach for objectively and unobtrusively monitoring functional performance within community settings, to aid clinical decision-making in neurodegenerative diseases such as dementia.

OBJECTIVE

The primary objective of this systematic review was to investigate the application of MMC using full-body tracking, to quantify functional performance in people with dementia, mild cognitive impairment, and Parkinson disease.

METHODS

A systematic search of the Embase, MEDLINE, CINAHL, and Scopus databases was conducted between November 2022 and February 2023, which yielded a total of 1595 results. The inclusion criteria were MMC and full-body tracking. A total of 157 studies were included for full-text screening, out of which 26 eligible studies that met the selection criteria were included in the review. .

RESULTS

Primarily, the selected studies focused on gait analysis (n=24), while other functional tasks, such as sit to stand (n=5) and stepping in place (n=1), were also explored. However, activities of daily living were not evaluated in any of the included studies. MMC models varied across the studies, encompassing depth cameras (n=18) versus standard video cameras (n=5) or mobile phone cameras (n=2) with postprocessing using deep learning models. However, only 6 studies conducted rigorous comparisons with established gold-standard motion capture models.

CONCLUSIONS

Despite its potential as an effective tool for analyzing movement and posture in individuals with dementia, mild cognitive impairment, and Parkinson disease, further research is required to establish the clinical usefulness of MMC in quantifying mobility and functional performance in the real world.

Causality-Enhanced Multiple Instance Learning With Graph Convolutional Networks for Parkinsonian Freezing-of-Gait Assessment

Freezing of gait (FoG) is a common disabling symptom of Parkinson's disease (PD). It is clinically characterized by sudden and transient walking interruptions for specific human body parts, and it presents the localization in time and space. Due to the difficulty in extracting global fine-grained features from lengthy videos, developing an automated five-point FoG scoring system is quite challenging. Therefore, we propose a novel video-based automated five-classification FoG assessment method with a causality-enhanced multiple-instance-learning graph convolutional network (GCN). This method involves developing a temporal segmentation GCN to segment each video into three motion stages for stage-level feature modeling, followed by a multiple-instance-learning framework to divide each stage into short clips for instance-level feature extraction. Subsequently, an uncertainty-driven multiple-instance-learning GCN is developed to capture spatial and temporal fine-grained features through GCN scheme and uncertainty learning, respectively, for acquiring global representations. Finally, a causality-enhanced graph generation strategy is proposed to exploit causal inference for mining and enhancing human structures causally related to clinical assessment, thereby extracting spatial causal features. Extensive experimental results demonstrate the excellent performance of the proposed method on five-classification FoG assessment with an accuracy of 62.72% and an acceptable accuracy of 91.32%, which is confirmed by independent testing. Additionally, it enables temporal and spatial localization of FoG events to a certain extent, facilitating reasonable clinical interpretations. In conclusion, our method provides a valuable tool for automated FoG assessment in PD, and the proposed causality-related component exhibits promising potential for extension to other general and medical fine-grained action recognition tasks.

Clinical gait analysis using video-based pose estimation: Multiple perspectives, clinical populations, and measuring change

Gait dysfunction is common in many clinical populations and often has a profound and deleterious impact on independence and quality of life. Gait analysis is a foundational component of rehabilitation because it is critical to identify and understand the specific deficits that should be targeted prior to the initiation of treatment. Unfortunately, current state-of-the-art approaches to gait analysis (e.g., marker-based motion capture systems, instrumented gait mats) are largely inaccessible due to prohibitive costs of time, money, and effort required to perform the assessments. Here, we demonstrate the ability to perform quantitative gait analyses in multiple clinical populations using only simple videos recorded using low-cost devices (tablets). We report four primary advances: 1) a novel, versatile workflow that leverages an open-source human pose estimation algorithm (OpenPose) to perform gait analyses using videos recorded from multiple different perspectives (e.g., frontal, sagittal), 2) validation of this workflow in three different populations of participants (adults without gait impairment, persons post-stroke, and persons with Parkinson's disease) via comparison to ground-truth three-dimensional motion capture, 3) demonstration of the ability to capture clinically relevant, condition-specific gait parameters, and 4) tracking of within-participant changes in gait, as is required to measure progress in rehabilitation and recovery. Importantly, our workflow has been made freely available and does not require prior gait analysis expertise. The ability to perform quantitative gait analyses in nearly any setting using only low-cost devices and computer vision offers significant potential for dramatic improvement in the accessibility of clinical gait analysis across different patient populations.

Parkinson's severity diagnosis explainable model based on 3D multi-head attention residual network

The severity evaluation of Parkinson's disease (PD) is of great significance for the treatment of PD. However, existing methods either have limitations based on prior knowledge or are invasive methods. To propose a more generalized severity evaluation model, this paper proposes an explainable 3D multi-head attention residual convolution network. First, we introduce the 3D attention-based convolution layer to extract video features. Second, features will be fed into LSTM and residual backbone networks, which can be used to capture the contextual information of the video. Finally, we design a feature compression module to condense the learned contextual features. We develop some interpretable experiments to better explain this black-box model so that it can be better generalized. Experiments show that our model can achieve state-of-the-art diagnosis performance. The proposed lightweight but effective model is expected to serve as a suitable end-to-end deep learning baseline in future research on PD video-based severity evaluation and has the potential for large-scale application in PD telemedicine. The source code is available at https://github.com/JackAILab/MARNet.

Cross-Spatiotemporal Graph Convolution Networks for Skeleton-Based Parkinsonian Gait MDS-UPDRS Score Estimation

Gait impairment in Parkinson's Disease (PD) is quantitatively assessed using the Movement Disorder Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS), a well-established clinical tool. Objective and efficient PD gait assessment is crucial for developing interventions to slow or halt its advancement. Skeleton-based PD gait MDS-UPDRS score estimation has attracted increasing interest in improving diagnostic efficiency and objectivity. However, previous works ignore the important cross-spacetime dependencies between joints in PD gait. Moreover, existing PD gait skeleton datasets are very small, which is a big issue in deep learning-based gait studies. In this work, we collect a sizable PD gait skeleton dataset by multi-view Azure Kinect sensors. The collected dataset contains 102 PD patients and 30 healthy older adults. In addition, gait data from 16 young adults (aged 24-50 years) are collected to further examine the effect of age on PD gait assessment. For skeleton-based automatic PD gait analysis, we propose a novel cross-spatiotemporal graph convolution network (CST-GCN) to learn complex features of gait patterns. Specifically, a gait graph labeling strategy is designed to assemble and group cross-spacetime neighbors of the root node according to the spatiotemporal semantics of the gait skeleton. Based on this strategy, the CST-GCN module explicitly models the cross-spacetime dependencies among joints. Finally, a dual-path model is presented to realize the modeling and fusion of spatial, temporal, and cross-spacetime gait features. Extensive experiments validate the effectiveness of our method on the collected dataset.

Clinical reliability and validity of a video-based markerless gait evaluation method

Objective

To explore the reliability and validity of gait parameters obtained from gait assessment system software employing a human posture estimation algorithm based on markerless videos of children walking in clinical practice.

Methods

Eighteen typical developmental (TD) children and ten children with developmental dysplasia of the hip (DDH) were recruited to walk along a designated sidewalk at a comfortable walking speed. A 3-dimensional gait analysis (3D GA) and a 2-dimensional markerless (2D ML) gait evaluation system were used to extract the gait kinematics parameters twice at an interval of 2 h.

Results

The two measurements of the children's kinematic gait parameters revealed no significant differences (P > 0.05). Intra-class correlation coefficients (ICC) were generally high (ICC >0.7), showing moderate to good relative reliability. The standard error of measurement (SEM) values of all gait parameters measured by the two walks were 1.26°-2.91°. The system software had good to excellent validity compared to the 3D GA, with ICC values between 0.835 and 0.957 and SEM values of 0.87°-1.71° for the gait parameters measured by both methods. The Bland-Altman plot analysis indicated no significant systematic errors.

Conclusions

The feasibility of the markerless gait assessment method using the human posture estimation-based algorithm may provide reliable and valid gait analysis results for practical clinical applications.

Evaluating the ability of a predictive vision-based machine learning model to measure changes in gait in response to medication and DBS within individuals with Parkinson's disease

INTRODUCTION

Gait impairments in Parkinson's disease (PD) are treated with dopaminergic medication or deep-brain stimulation (DBS), although the magnitude of the response is variable between individuals. Computer vision-based approaches have previously been evaluated for measuring the severity of parkinsonian gait in videos, but have not been evaluated for their ability to identify changes within individuals in response to treatment. This pilot study examines whether a vision-based model, trained on videos of parkinsonism, is able to detect improvement in parkinsonian gait in people with PD in response to medication and DBS use.

METHODS

A spatial-temporal graph convolutional model was trained to predict MDS-UPDRS-gait scores in 362 videos from 14 older adults with drug-induced parkinsonism. This model was then used to predict MDS-UPDRS-gait scores on a different dataset of 42 paired videos from 13 individuals with PD, recorded while ON and OFF medication and DBS treatment during the same clinical visit. Statistical methods were used to assess whether the model was responsive to changes in gait in the ON and OFF states.

RESULTS

The MDS-UPDRS-gait scores predicted by the model were lower on average (representing improved gait; p = 0.017, Cohen's d = 0.495) during the ON medication and DBS treatment conditions. The magnitude of the differences between ON and OFF state was significantly correlated between model predictions and clinician annotations (p = 0.004). The predicted scores were significantly correlated with the clinician scores (Kendall's tau-b = 0.301, p = 0.010), but were distributed in a smaller range as compared to the clinician scores.

CONCLUSION

A vision-based model trained on parkinsonian gait did not accurately predict MDS-UPDRS-gait scores in a different PD cohort, but detected weak, but statistically significant proportional changes in response to medication and DBS use. Large, clinically validated datasets of videos captured in many different settings and treatment conditions are required to develop accurate vision-based models of parkinsonian gait.

Automatic labeling of Parkinson's Disease gait videos with weak supervision

Motor dysfunction in Parkinson's Disease (PD) patients is typically assessed by clinicians employing the Movement Disorder Society's Unified Parkinson's Disease Rating Scale (MDS-UPDRS). Such comprehensive clinical assessments are time-consuming, expensive, semi-subjective, and may potentially result in conflicting labels across different raters. To address this problem, we propose an automatic, objective, and weakly-supervised method for labeling PD patients' gait videos. The proposed method accepts videos of patients and classifies their gait scores as normal (Gait score in MDS-UPDRS = 0) or PD (MDS-UPDRS ≥ 1). Unlike previous work, the proposed method does not require a priori MDS-UPDRS ratings for training, utilizing only domain-specific knowledge obtained from neurologists. We propose several labeling functions that classify patients' gait and use a generative model to learn the accuracy of each labeling function in a self-supervised manner. Since results depended upon the estimated values of the patients' 3D poses, and existing pre-trained 3D pose estimators did not yield accurate results, we propose a weakly-supervised 3D human pose estimation method for fine-tuning pre-trained models in a clinical setting. Using leave-one-out evaluations, the proposed method obtains an accuracy of 89% on a dataset of 29 PD subjects - a significant improvement compared to previous work by 7%-10% depending upon the dataset. The method obtained state-of-the-art results on the Human3.6M dataset. Our results suggest that the use of labeling functions may provide a robust means to interpret and classify patient-oriented videos involving motor tasks.

Multi-view emotional expressions dataset using 2D pose estimation

Human body expressions convey emotional shifts and intentions of action and, in some cases, are even more effective than other emotion models. Despite many datasets of body expressions incorporating motion capture available, there is a lack of more widely distributed datasets regarding naturalized body expressions based on the 2D video. In this paper, therefore, we report the multi-view emotional expressions dataset (MEED) using 2D pose estimation. Twenty-two actors presented six emotional (anger, disgust, fear, happiness, sadness, surprise) and neutral body movements from three viewpoints (left, front, right). A total of 4102 videos were captured. The MEED consists of the corresponding pose estimation results (i.e., 397,809 PNG files and 397,809 JSON files). The size of MEED exceeds 150 GB. We believe this dataset will benefit the research in various fields, including affective computing, human-computer interaction, social neuroscience, and psychiatry.

Ambient Monitoring of Gait and Machine Learning Models for Dynamic and Short-Term Falls Risk Assessment in People With Dementia

Falls are a leading cause of morbidity and mortality in older adults with dementia residing in long-term care. Having access to a frequently updated and accurate estimate of the likelihood of a fall over a short time frame for each resident will enable care staff to provide targeted interventions to prevent falls and resulting injuries. To this end, machine learning models to estimate and frequently update the risk of a fall within the next 4 weeks were trained on longitudinal data from 54 older adult participants with dementia. Data from each participant included baseline clinical assessments of gait, mobility, and fall risk at the time of admission, daily medication intake in three medication categories, and frequent assessments of gait performed via a computer vision-based ambient monitoring system. Systematic ablations investigated the effects of various hyperparameters and feature sets and experimentally identified differential contributions from baseline clinical assessments, ambient gait analysis, and daily medication intake. In leave-one-subject-out cross-validation, the best performing model predicts the likelihood of a fall over the next 4 weeks with a sensitivity and specificity of 72.8 and 73.2, respectively, and achieved an area under the receiver operating characteristic curve (AUROC) of 76.2. By contrast, the best model excluding ambient gait features achieved an AUROC of 56.2 with a sensitivity and specificity of 51.9 and 54.0, respectively. Future research will focus on externally validating these findings to prepare for the implementation of this technology to reduce fall and fall-related injuries in long-term care.

WM-STGCN: A Novel Spatiotemporal Modeling Method for Parkinsonian Gait Recognition

Parkinson's disease (PD) is a neurodegenerative disorder that causes gait abnormalities. Early and accurate recognition of PD gait is crucial for effective treatment. Recently, deep learning techniques have shown promising results in PD gait analysis. However, most existing methods focus on severity estimation and frozen gait detection, while the recognition of Parkinsonian gait and normal gait from the forward video has not been reported. In this paper, we propose a novel spatiotemporal modeling method for PD gait recognition, named WM-STGCN, which utilizes a Weighted adjacency matrix with virtual connection and Multi-scale temporal convolution in a Spatiotemporal Graph Convolution Network. The weighted matrix enables different intensities to be assigned to different spatial features, including virtual connections, while the multi-scale temporal convolution helps to effectively capture the temporal features at different scales. Moreover, we employ various approaches to augment skeleton data. Experimental results show that our proposed method achieved the best accuracy of 87.1% and an F1 score of 92.85%, outperforming Long short-term memory (LSTM), K-nearest neighbors (KNN), Decision tree, AdaBoost, and ST-GCN models. Our proposed WM-STGCN provides an effective spatiotemporal modeling method for PD gait recognition that outperforms existing methods. It has the potential for clinical application in PD diagnosis and treatment.

Multi-speed transformer network for neurodegenerative disease assessment and activity recognition

BACKGROUND AND OBJECTIVE

Neurodegenerative diseases are the most frequent age-related diseases. This type of disease, if not discovered in the initial stage, will compromise the quality of life of the affected subject. Thus, a timely diagnosis is of paramount importance. One of the most used tasks from neurologists to detect and determine the severity of the disease is analysing human gait. This work presents the dataset named "Beside Gait" containing timeseries of coordinates of extracted body joints of people with neurodegenerative diseases in various stages of the disease as well as control subjects. In addition, the novel Multi-Speed transformer technique will be presented and benchmarked against several other techniques making use of deep learning and Shallow Learning. The objective is to recognize subjects affected by some form of neurodegenerative disease in early stage using a computer vision technique making use of deep learning that can be integrated into a smartphone app for offline inference with the aim of promptly initiate investigations and treatment to improve the patient's quality of life.

METHODS

The recorded videos were processed, and the skeleton of the person in the video was extracted using pose estimation. The raw time-series coordinates of the joints extracted by the pose estimation algorithm were tested against novel deep neural network architectures and Shallow Learning techniques. In this work, the proposed Multi-Speed Transformer is benchmarked against other deep neural networks such as Temporal Convolutional Neural Networks, Transformers, as well as Shallow Learning techniques making use of feature extraction and different classifiers such as Random Forests, K Nearest Neighbours, Ada Boost, Linear and RBF SVM. The proposed Multi-Speed Transformer architecture has been developed to learn short and long-term patterns to model the various pathological gaits.

RESULTS

The Multi-Speed Transformer outperformed all other existing models reaching an accuracy of 96.9%, a sensitivity of 96.9%, a precision of 97.7%, and a specificity of 97.1% in binary classification. The accuracy in multi-class classification for detecting the presence of the disease in various stages is 71.6%, the sensitivity is 67.7%, and the specificity is 71.8%. In addition, tests have also been conducted against two other different activity recognition datasets, namely SHREC and JHMDB, in the exact same conditions. Multi-Speed Transformer has demonstrated to beat always all other tested techniques as well as the techniques reviewed in the state-of-the-art with respectively of accuracy 91.8% and 74%. Having those datasets more than two classes, specificity was not computed.

CONCLUSIONS

The Multi-Speed Transformer is a valuable technique for neurodegenerative disease assessment through computer vision. In addition, the novel dataset "Beside Gait" here presented is an important starting point for future research work on automatic recognition of neurodegenerative diseases using gait analysis.

Artificial intelligence technologies and compassion in healthcare: A systematic scoping review

Background

Advances in artificial intelligence (AI) technologies, together with the availability of big data in society, creates uncertainties about how these developments will affect healthcare systems worldwide. Compassion is essential for high-quality healthcare and research shows how prosocial caring behaviors benefit human health and societies. However, the possible association between AI technologies and compassion is under conceptualized and underexplored.

Objectives

The aim of this scoping review is to provide a comprehensive depth and a balanced perspective of the emerging topic of AI technologies and compassion, to inform future research and practice. The review questions were: How is compassion discussed in relation to AI technologies in healthcare? How are AI technologies being used to enhance compassion in healthcare? What are the gaps in current knowledge and unexplored potential? What are the key areas where AI technologies could support compassion in healthcare?

Materials and methods

A systematic scoping review following five steps of Joanna Briggs Institute methodology. Presentation of the scoping review conforms with PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews). Eligibility criteria were defined according to 3 concept constructs (AI technologies, compassion, healthcare) developed from the literature and informed by medical subject headings (MeSH) and key words for the electronic searches. Sources of evidence were Web of Science and PubMed databases, articles published in English language 2011-2022. Articles were screened by title/abstract using inclusion/exclusion criteria. Data extracted (author, date of publication, type of article, aim/context of healthcare, key relevant findings, country) was charted using data tables. Thematic analysis used an inductive-deductive approach to generate code categories from the review questions and the data. A multidisciplinary team assessed themes for resonance and relevance to research and practice.

Results

Searches identified 3,124 articles. A total of 197 were included after screening. The number of articles has increased over 10 years (2011, n = 1 to 2021, n = 47 and from Jan-Aug 2022 n = 35 articles). Overarching themes related to the review questions were: (1) Developments and debates (7 themes) Concerns about AI ethics, healthcare jobs, and loss of empathy; Human-centered design of AI technologies for healthcare; Optimistic speculation AI technologies will address care gaps; Interrogation of what it means to be human and to care; Recognition of future potential for patient monitoring, virtual proximity, and access to healthcare; Calls for curricula development and healthcare professional education; Implementation of AI applications to enhance health and wellbeing of the healthcare workforce. (2) How AI technologies enhance compassion (10 themes) Empathetic awareness; Empathetic response and relational behavior; Communication skills; Health coaching; Therapeutic interventions; Moral development learning; Clinical knowledge and clinical assessment; Healthcare quality assessment; Therapeutic bond and therapeutic alliance; Providing health information and advice. (3) Gaps in knowledge (4 themes) Educational effectiveness of AI-assisted learning; Patient diversity and AI technologies; Implementation of AI technologies in education and practice settings; Safety and clinical effectiveness of AI technologies. (4) Key areas for development (3 themes) Enriching education, learning and clinical practice; Extending healing spaces; Enhancing healing relationships.

Conclusion

There is an association between AI technologies and compassion in healthcare and interest in this association has grown internationally over the last decade. In a range of healthcare contexts, AI technologies are being used to enhance empathetic awareness; empathetic response and relational behavior; communication skills; health coaching; therapeutic interventions; moral development learning; clinical knowledge and clinical assessment; healthcare quality assessment; therapeutic bond and therapeutic alliance; and to provide health information and advice. The findings inform a reconceptualization of compassion as a human-AI system of intelligent caring comprising six elements: (1) Awareness of suffering (e.g., pain, distress, risk, disadvantage); (2) Understanding the suffering (significance, context, rights, responsibilities etc.); (3) Connecting with the suffering (e.g., verbal, physical, signs and symbols); (4) Making a judgment about the suffering (the need to act); (5) Responding with an intention to alleviate the suffering; (6) Attention to the effect and outcomes of the response. These elements can operate at an individual (human or machine) and collective systems level (healthcare organizations or systems) as a cyclical system to alleviate different types of suffering. New and novel approaches to human-AI intelligent caring could enrich education, learning, and clinical practice; extend healing spaces; and enhance healing relationships.

Implications

In a complex adaptive system such as healthcare, human-AI intelligent caring will need to be implemented, not as an ideology, but through strategic choices, incentives, regulation, professional education, and training, as well as through joined up thinking about human-AI intelligent caring. Research funders can encourage research and development into the topic of AI technologies and compassion as a system of human-AI intelligent caring. Educators, technologists, and health professionals can inform themselves about the system of human-AI intelligent caring.

Enabling Gait Analysis in the Telemedicine Practice through Portable and Accurate 3D Human Pose Estimation

Human pose estimation (HPE) through deep learning-based software applications is a trend topic for markerless motion analysis. Thanks to the accuracy of the state-of-the-art technology, HPE could enable gait analysis in the telemedicine practice. On the other hand, delivering such a service at a distance requires the system to satisfy multiple and different constraints like accuracy, portability, real-time, and privacy compliance at the same time. Existing solutions either guarantee accuracy and real-time (e.g., the widespread OpenPose software on well-equipped computing platforms) or portability and data privacy (e.g., light convolutional neural networks on mobile phones). We propose a portable and low-cost platform that implements real-time and accurate 3D HPE through an embedded software on a low-power off-the-shelf computing device that guarantees privacy by default and by design. We present an extended evaluation of both accuracy and performance of the proposed solution conducted with a marker-based motion capture system (i.e., Vicon) as ground truth. The results show that the platform achieves real-time performance and high-accuracy with a deviation below the error tolerance when compared to the marker-based motion capture system (e.g., less than an error of 5^∘ on the estimated knee flexion difference on the entire gait cycle and correlation 0.91<ρ<0.99). We provide a proof-of-concept study, showing that such portable technology, considering the limited discrepancies with respect to the marker-based motion capture system and its working tolerance, could be used for gait analysis at a distance without leading to different clinical interpretation.

Detection and assessment of Parkinson's disease based on gait analysis: A survey

Neurological disorders represent one of the leading causes of disability and mortality in the world. Parkinson's Disease (PD), for example, affecting millions of people worldwide is often manifested as impaired posture and gait. These impairments have been used as a clinical sign for the early detection of PD, as well as an objective index for pervasive monitoring of the PD patients in daily life. This review presents the evidence that demonstrates the relationship between human gait and PD, and illustrates the role of different gait analysis systems based on vision or wearable sensors. It also provides a comprehensive overview of the available automatic recognition systems for the detection and management of PD. The intervening measures for improving gait performance are summarized, in which the smart devices for gait intervention are emphasized. Finally, this review highlights some of the new opportunities in detecting, monitoring, and treating of PD based on gait, which could facilitate the development of objective gait-based biomarkers for personalized support and treatment of PD.

2D Gait Skeleton Data Normalization for Quantitative Assessment of Movement Disorders from Freehand Single Camera Video Recordings

Overlapping phenotypic features between Early Onset Ataxia (EOA) and Developmental Coordination Disorder (DCD) can complicate the clinical distinction of these disorders. Clinical rating scales are a common way to quantify movement disorders but in children these scales also rely on the observer's assessment and interpretation. Despite the introduction of inertial measurement units for objective and more precise evaluation, special hardware is still required, restricting their widespread application. Gait video recordings of movement disorder patients are frequently captured in routine clinical settings, but there is presently no suitable quantitative analysis method for these recordings. Owing to advancements in computer vision technology, deep learning pose estimation techniques may soon be ready for convenient and low-cost clinical usage. This study presents a framework based on 2D video recording in the coronal plane and pose estimation for the quantitative assessment of gait in movement disorders. To allow the calculation of distance-based features, seven different methods to normalize 2D skeleton keypoint data derived from pose estimation using deep neural networks applied to freehand video recording of gait were evaluated. In our experiments, 15 children (five EOA, five DCD and five healthy controls) were asked to walk naturally while being videotaped by a single camera in 1280 × 720 resolution at 25 frames per second. The high likelihood of the prediction of keypoint locations (mean = 0.889, standard deviation = 0.02) demonstrates the potential for distance-based features derived from routine video recordings to assist in the clinical evaluation of movement in EOA and DCD. By comparison of mean absolute angle error and mean variance of distance, the normalization methods using the Euclidean (2D) distance of left shoulder and right hip, or the average distance from left shoulder to right hip and from right shoulder to left hip were found to better perform for deriving distance-based features and further quantitative assessment of movement disorders.

Anomalous gait feature classification from 3-D motion capture data

The gait kinematics of an individual is affected by various factors, including age, anthropometry, gender, and disease. Detecting anomalous gait features aids in the diagnosis and treatment of gait-related diseases. The objective of this study was to develop a machine learning method for automatically classifying five anomalous gait features, i.e., toe-out, genu varum, pes planus, hindfoot valgus, and forward head posture features, from three-dimensional data on gait kinematics. Gait data and gait feature labels of 488 subjects were acquired. The orientations of the human body segments during a gait cycle were mapped to a low-dimensional latent gait vector using a variational autoencoder. A two-layer neural network was trained to classify five gait features using logistic regression and calculate an anomalous gait feature vector (AGFV). The proposed network showed balanced accuracies of 82.8% for a toe-out, 85.9% for hindfoot valgus, 80.2% for pes planus, 73.2% for genu varum, and 92.9% for forward head posture when the AGFV was rounded to the nearest zero or 1. Multiple anomalous gait features were detectable using the proposed method, which has a practical advantage over current gait indices, including the gait deviation index with a single value. The overall results confirmed the feasibility of using the proposed method for screening subjects with anomalous gait features using three-dimensional motion capture data.