Vision-based Estimation of MDS-UPDRS Gait Scores for Assessing Parkinson's Disease Motor Severity

A validation study demonstrating portable motion capture cameras accurately characterize gait metrics when compared to a pressure-sensitive walkway

Digital quantification of gait can be used to measure aging- and disease-related decline in mobility. Gait performance also predicts prognosis, disease progression, and response to therapies. Most gait analysis systems require large amounts of space, resources, and expertise to implement and are not widely accessible. Thus, there is a need for a portable system that accurately characterizes gait. Here, depth video from two portable cameras accurately reconstructed gait metrics comparable to those reported by a pressure-sensitive walkway. 392 research participants walked across a four-meter pressure-sensitive walkway while depth video was recorded. Gait speed, cadence, and step and stride durations and lengths strongly correlated (r > 0.9) between modalities, with root-mean-squared-errors (RMSE) of 0.04 m/s, 2.3 steps/min, 0.03 s, and 0.05-0.08 m for speed, cadence, step/stride duration, and step/stride length, respectively. Step, stance, and double support durations (gait cycle percentage) significantly correlated (r > 0.6) between modalities, with 5% RMSE for step and stance and 10% RMSE for double support. In an exploratory analysis, gait speed from both modalities significantly related to healthy, mild, moderate, or severe categorizations of Charleson Comorbidity Indices (ANOVA, Tukey's HSD, p < 0.0125). These findings demonstrate the viability of using depth video to expand access to quantitative gait assessments.

Assisted technology in Parkinson's disease gait: what's up?

BACKGROUND

 Gait disturbances are prevalent and debilitating symptoms, diminishing mobility and quality of life for Parkinson's disease (PD) individuals. While traditional treatments offer partial relief, there is a growing interest in alternative interventions to address this challenge. Recently, a remarkable surge in assisted technology (AT) development was witnessed to aid individuals with PD.

OBJECTIVE

 To explore the burgeoning landscape of AT interventions tailored to alleviate PD-related gait impairments and describe current research related to such aim.

METHODS

 In this review, we searched on PubMed for papers published in English (2018-2023). Additionally, the abstract of each study was read to ensure inclusion. Four researchers searched independently, including studies according to our inclusion and exclusion criteria.

RESULTS

 We included studies that met all inclusion criteria. We identified key trends in assistive technology of gait parameters analysis in PD. These encompass wearable sensors, gait analysis, real-time feedback and cueing techniques, virtual reality, and robotics.

CONCLUSION

 This review provides a resource for guiding future research, informing clinical decisions, and fostering collaboration among researchers, clinicians, and policymakers. By delineating this rapidly evolving field's contours, it aims to inspire further innovation, ultimately improving the lives of PD patients through more effective and personalized interventions.

Detecting the symptoms of Parkinson's disease with non-standard video

BACKGROUND

Neurodegenerative diseases, such as Parkinson's disease (PD), necessitate frequent clinical visits and monitoring to identify changes in motor symptoms and provide appropriate care. By applying machine learning techniques to video data, automated video analysis has emerged as a promising approach to track and analyze motor symptoms, which could facilitate more timely intervention. However, existing solutions often rely on specialized equipment and recording procedures, which limits their usability in unstructured settings like the home. In this study, we developed a method to detect PD symptoms from unstructured videos of clinical assessments, without the need for specialized equipment or recording procedures.

METHODS

Twenty-eight individuals with Parkinson's disease completed a video-recorded motor examination that included the finger-to-nose and hand pronation-supination tasks. Clinical staff provided ground truth scores for the level of Parkinsonian symptoms present. For each video, we used a pre-existing model called PIXIE to measure the location of several joints on the person's body and quantify how they were moving. Features derived from the joint angles and trajectories, designed to be robust to recording angle, were then used to train two types of machine-learning classifiers (random forests and support vector machines) to detect the presence of PD symptoms.

RESULTS

The support vector machine trained on the finger-to-nose task had an F1 score of 0.93 while the random forest trained on the same task yielded an F1 score of 0.85. The support vector machine and random forest trained on the hand pronation-supination task had F1 scores of 0.20 and 0.33, respectively.

CONCLUSION

These results demonstrate the feasibility of developing video analysis tools to track motor symptoms across variable perspectives. These tools do not work equally well for all tasks, however. This technology has the potential to overcome barriers to access for many individuals with degenerative neurological diseases like PD, providing them with a more convenient and timely method to monitor symptom progression, without requiring a structured video recording procedure. Ultimately, more frequent and objective home assessments of motor function could enable more precise telehealth optimization of interventions to improve clinical outcomes inside and outside of the clinic.

Cross-Spatiotemporal Graph Convolution Networks for Skeleton-Based Parkinsonian Gait MDS-UPDRS Score Estimation

Gait impairment in Parkinson's Disease (PD) is quantitatively assessed using the Movement Disorder Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS), a well-established clinical tool. Objective and efficient PD gait assessment is crucial for developing interventions to slow or halt its advancement. Skeleton-based PD gait MDS-UPDRS score estimation has attracted increasing interest in improving diagnostic efficiency and objectivity. However, previous works ignore the important cross-spacetime dependencies between joints in PD gait. Moreover, existing PD gait skeleton datasets are very small, which is a big issue in deep learning-based gait studies. In this work, we collect a sizable PD gait skeleton dataset by multi-view Azure Kinect sensors. The collected dataset contains 102 PD patients and 30 healthy older adults. In addition, gait data from 16 young adults (aged 24-50 years) are collected to further examine the effect of age on PD gait assessment. For skeleton-based automatic PD gait analysis, we propose a novel cross-spatiotemporal graph convolution network (CST-GCN) to learn complex features of gait patterns. Specifically, a gait graph labeling strategy is designed to assemble and group cross-spacetime neighbors of the root node according to the spatiotemporal semantics of the gait skeleton. Based on this strategy, the CST-GCN module explicitly models the cross-spacetime dependencies among joints. Finally, a dual-path model is presented to realize the modeling and fusion of spatial, temporal, and cross-spacetime gait features. Extensive experiments validate the effectiveness of our method on the collected dataset.

miR-221 and Parkinson's disease: A biomarker with therapeutic potential

Parkinson's disease (PD) is a neurodegenerative disorder characterized by the loss of dopaminergic neurons in the substantia nigra, leading to various motor and non-motor symptoms. Several cellular and molecular mechanisms such as alpha-synuclein (α-syn) accumulation, mitochondrial dysfunction, oxidative stress and neuroinflammation are involved in the pathogenesis of this disease. MicroRNAs (miRNAs) play important roles in post-transcriptional gene regulation. They are typically about 21-25 nucleotides in length and are involved in the regulation of gene expression by binding to the messenger RNA (mRNA) molecules. miRNAs like miR-221 play important roles in various biological processes, including development, cell proliferation, differentiation and apoptosis. miR-221 promotes neuronal survival against oxidative stress and neurite outgrowth and neuronal differentiation. Additionally, the role of miR-221 in PD has been investigated in several studies. According to the results of these studies, (1) miR-221 protects PC12 cells against oxidative stress induced by 6-hydroxydopamine; (2) miR-221 prevents Bax/caspase-3 signalling activation by stopping Bim; (3) miR-221 has moderate predictive power for PD; (4) miR-221 directly targets PTEN, and PTEN over-expression eliminates the protective action of miR-221 on p-AKT expression in PC12 cells; and (5) miRNA-221 controls cell viability and apoptosis by manipulating the Akt signalling pathway in PD. This review study suggested that miR-221 has the potential to be used as a clinical biomarker for PD diagnosis and stage assignment.

Evaluating the ability of a predictive vision-based machine learning model to measure changes in gait in response to medication and DBS within individuals with Parkinson's disease

INTRODUCTION

Gait impairments in Parkinson's disease (PD) are treated with dopaminergic medication or deep-brain stimulation (DBS), although the magnitude of the response is variable between individuals. Computer vision-based approaches have previously been evaluated for measuring the severity of parkinsonian gait in videos, but have not been evaluated for their ability to identify changes within individuals in response to treatment. This pilot study examines whether a vision-based model, trained on videos of parkinsonism, is able to detect improvement in parkinsonian gait in people with PD in response to medication and DBS use.

METHODS

A spatial-temporal graph convolutional model was trained to predict MDS-UPDRS-gait scores in 362 videos from 14 older adults with drug-induced parkinsonism. This model was then used to predict MDS-UPDRS-gait scores on a different dataset of 42 paired videos from 13 individuals with PD, recorded while ON and OFF medication and DBS treatment during the same clinical visit. Statistical methods were used to assess whether the model was responsive to changes in gait in the ON and OFF states.

RESULTS

The MDS-UPDRS-gait scores predicted by the model were lower on average (representing improved gait; p = 0.017, Cohen's d = 0.495) during the ON medication and DBS treatment conditions. The magnitude of the differences between ON and OFF state was significantly correlated between model predictions and clinician annotations (p = 0.004). The predicted scores were significantly correlated with the clinician scores (Kendall's tau-b = 0.301, p = 0.010), but were distributed in a smaller range as compared to the clinician scores.

CONCLUSION

A vision-based model trained on parkinsonian gait did not accurately predict MDS-UPDRS-gait scores in a different PD cohort, but detected weak, but statistically significant proportional changes in response to medication and DBS use. Large, clinically validated datasets of videos captured in many different settings and treatment conditions are required to develop accurate vision-based models of parkinsonian gait.

Clinically Informed Automated Assessment of Finger Tapping Videos in Parkinson's Disease

The utilization of Artificial Intelligence (AI) for assessing motor performance in Parkinson's Disease (PD) offers substantial potential, particularly if the results can be integrated into clinical decision-making processes. However, the precise quantification of PD symptoms remains a persistent challenge. The current standard Unified Parkinson's Disease Rating Scale (UPDRS) and its variations serve as the primary clinical tools for evaluating motor symptoms in PD, but are time-intensive and prone to inter-rater variability. Recent work has applied data-driven machine learning techniques to analyze videos of PD patients performing motor tasks, such as finger tapping, a UPDRS task to assess bradykinesia. However, these methods often use abstract features that are not closely related to clinical experience. In this paper, we introduce a customized machine learning approach for the automated scoring of UPDRS bradykinesia using single-view RGB videos of finger tapping, based on the extraction of detailed features that rigorously conform to the established UPDRS guidelines. We applied the method to 75 videos from 50 PD patients collected in both a laboratory and a realistic clinic environment. The classification performance agreed well with expert assessors, and the features selected by the Decision Tree aligned with clinical knowledge. Our proposed framework was designed to remain relevant amid ongoing patient recruitment and technological progress. The proposed approach incorporates features that closely resonate with clinical reasoning and shows promise for clinical implementation in the foreseeable future.

The Parkinson's disease waiting room of the future: measurements, not magazines

Utilizing technology to precisely quantify Parkinson's disease motor symptoms has evolved over the past 50 years from single point in time assessments using traditional biomechanical approaches to continuous monitoring of performance with wearables. Despite advances in the precision, usability, availability and affordability of technology, the "gold standard" for assessing Parkinson's motor symptoms continues to be a subjective clinical assessment as none of these technologies have been fully integrated into routine clinical care of Parkinson's disease patients. To facilitate the integration of technology into routine clinical care, the Develop with Clinical Intent (DCI) model was created. The DCI model takes a unique approach to the development and integration of technology into clinical practice by focusing on the clinical problem to be solved by technology rather than focusing on the technology and then contemplating how it could be integrated into clinical care. The DCI model was successfully used to develop the Parkinson's disease Waiting Room of the Future (WROTF) within the Center for Neurological Restoration at the Cleveland Clinic. Within the WROTF, Parkinson's disease patients complete the self-directed PD-Optimize application on an iPad. The PD-Optimize platform contains cognitive and motor assessments to quantify PD symptoms that are difficult and time-consuming to evaluate clinically. PD-Optimize is completed by the patient prior to their medical appointment and the results are immediately integrated into the electronic health record for discussion with the movement disorder neurologist. Insights from the clinical use of PD-Optimize has spurred the development of a virtual reality technology to evaluate instrumental activities of daily living in PD patients. This new technology will undergo rigorous assessment and validation as dictated by the DCI model. The DCI model is intended to serve as a health enablement roadmap to formalize and accelerate the process of bringing the advantages of cutting-edge technology to those who could benefit the most: the patient.

An Explainable Geometric-Weighted Graph Attention Network for Identifying Functional Networks Associated with Gait Impairment

One of the hallmark symptoms of Parkinson's Disease (PD) is the progressive loss of postural reflexes, which eventually leads to gait difficulties and balance problems. Identifying disruptions in brain function associated with gait impairment could be crucial in better understanding PD motor progression, thus advancing the development of more effective and personalized therapeutics. In this work, we present an explainable, geometric, weighted-graph attention neural network (xGW-GAT) to identify functional networks predictive of the progression of gait difficulties in individuals with PD. xGW-GAT predicts the multi-class gait impairment on the MDS-Unified PD Rating Scale (MDS-UPDRS). Our computational- and data-efficient model represents functional connectomes as symmetric positive definite (SPD) matrices on a Riemannian manifold to explicitly encode pairwise interactions of entire connectomes, based on which we learn an attention mask yielding individual- and group-level explain-ability. Applied to our resting-state functional MRI (rs-fMRI) dataset of individuals with PD, xGW-GAT identifies functional connectivity patterns associated with gait impairment in PD and offers interpretable explanations of functional subnetworks associated with motor impairment. Our model successfully outperforms several existing methods while simultaneously revealing clinically-relevant connectivity patterns. The source code is available at https://github.com/favour-nerrise/xGW-GAT.

Computer vision quantification of whole-body Parkinsonian bradykinesia using a large multi-site population

Parkinson's disease (PD) is a common neurological disorder, with bradykinesia being one of its cardinal features. Objective quantification of bradykinesia using computer vision has the potential to standardise decision-making, for patient treatment and clinical trials, while facilitating remote assessment. We utilised a dataset of part-3 MDS-UPDRS motor assessments, collected at four independent clinical and one research sites on two continents, to build computer-vision-based models capable of inferring the correct severity rating robustly and consistently across all identifiable subgroups of patients. These results contrast with previous work limited by small sample sizes and small numbers of sites. Our bradykinesia estimation corresponded well with clinician ratings (interclass correlation 0.74). This agreement was consistent across four clinical sites. This result demonstrates how such technology can be successfully deployed into existing clinical workflows, with consumer-grade smartphone or tablet devices, adding minimal equipment cost and time.

Remote scoring models of rigidity and postural stability of Parkinson's disease based on indirect motions and a low-cost RGB algorithm

Background and objectives

The Movement Disorder Society's Unified Parkinson's Disease Rating Scale Part III (MDS-UPDRS III) is mostly common used for assessing the motor symptoms of Parkinson's disease (PD). In remote circumstances, vision-based techniques have many strengths over wearable sensors. However, rigidity (item 3.3) and postural stability (item 3.12) in the MDS-UPDRS III cannot be assessed remotely since participants need to be touched by a trained examiner during testing. We developed the four scoring models of rigidity of the neck, rigidity of the lower extremities, rigidity of the upper extremities, and postural stability based on features extracted from other available and touchless motions.

Methods

The red, green, and blue (RGB) computer vision algorithm and machine learning were combined with other available motions from the MDS-UPDRS III evaluation. A total of 104 patients with PD were split into a train set (89 individuals) and a test set (15 individuals). The light gradient boosting machine (LightGBM) multiclassification model was trained. Weighted kappa (k), absolute accuracy (ACC ± 0), and Spearman's correlation coefficient (rho) were used to evaluate the performance of model.

Results

For model of rigidity of the upper extremities, k = 0.58 (moderate), ACC ± 0 = 0.73, and rho = 0.64 (moderate). For model of rigidity of the lower extremities, k = 0.66 (substantial), ACC ± 0 = 0.70, and rho = 0.76 (strong). For model of rigidity of the neck, k = 0.60 (moderate), ACC ± 0 = 0.73, and rho = 0.60 (moderate). For model of postural stability, k = 0.66 (substantial), ACC ± 0 = 0.73, and rho = 0.68 (moderate).

Conclusion

Our study can be meaningful for remote assessments, especially when people have to maintain social distance, e.g., in situations such as the coronavirus disease-2019 (COVID-19) pandemic.

ErgoExplorer: Interactive Ergonomic Risk Assessment from Video Collections

Ergonomic risk assessment is now, due to an increased awareness, carried out more often than in the past. The conventional risk assessment evaluation, based on expert-assisted observation of the workplaces and manually filling in score tables, is still predominant. Data analysis is usually done with a focus on critical moments, although without the support of contextual information and changes over time. In this paper we introduce ErgoExplorer, a system for the interactive visual analysis of risk assessment data. In contrast to the current practice, we focus on data that span across multiple actions and multiple workers while keeping all contextual information. Data is automatically extracted from video streams. Based on carefully investigated analysis tasks, we introduce new views and their corresponding interactions. These views also incorporate domain-specific score tables to guarantee an easy adoption by domain experts. All views are integrated into ErgoExplorer, which relies on coordinated multiple views to facilitate analysis through interaction. ErgoExplorer makes it possible for the first time to examine complex relationships between risk assessments of individual body parts over long sessions that span multiple operations. The newly introduced approach supports analysis and exploration at several levels of detail, ranging from a general overview, down to inspecting individual frames in the video stream, if necessary. We illustrate the usefulness of the newly proposed approach applying it to several datasets.

GaitForeMer: Self-Supervised Pre-Training of Transformers via Human Motion Forecasting for Few-Shot Gait Impairment Severity Estimation

Parkinson's disease (PD) is a neurological disorder that has a variety of observable motor-related symptoms such as slow movement, tremor, muscular rigidity, and impaired posture. PD is typically diagnosed by evaluating the severity of motor impairments according to scoring systems such as the Movement Disorder Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS). Automated severity prediction using video recordings of individuals provides a promising route for non-intrusive monitoring of motor impairments. However, the limited size of PD gait data hinders model ability and clinical potential. Because of this clinical data scarcity and inspired by the recent advances in self-supervised large-scale language models like GPT-3, we use human motion forecasting as an effective self-supervised pre-training task for the estimation of motor impairment severity. We introduce GaitForeMer, Gait Forecasting and impairment estimation transforMer, which is first pre-trained on public datasets to forecast gait movements and then applied to clinical data to predict MDS-UPDRS gait impairment severity. Our method outperforms previous approaches that rely solely on clinical data by a large margin, achieving an F1 score of 0.76, precision of 0.79, and recall of 0.75. Using GaitForeMer, we show how public human movement data repositories can assist clinical use cases through learning universal motion representations. The code is available at https://github.com/markendo/GaitForeMer.

Video-Based Automated Assessment of Movement Parameters Consistent with MDS-UPDRS III in Parkinson's Disease

BACKGROUND

Among motor symptoms of Parkinson's disease (PD), including rigidity and resting tremor, bradykinesia is a mandatory feature to define the parkinsonian syndrome. MDS-UPDRS III is the worldwide reference scale to evaluate the parkinsonian motor impairment, especially bradykinesia. However, MDS-UPDRS III is an agent-based score making reproducible measurements and follow-up challenging.

OBJECTIVE

Using a deep learning approach, we developed a tool to compute an objective score of bradykinesia based on the guidelines of the gold-standard MDS-UPDRS III.

METHODS

We adapted and applied two deep learning algorithms to detect a two-dimensional (2D) skeleton of the hand composed of 21 predefined points, and transposed it into a three-dimensional (3D) skeleton for a large database of videos of parkinsonian patients performing MDS-UPDRS III protocols acquired in the Movement Disorder unit of Avicenne University Hospital.

RESULTS

We developed a 2D and 3D automated analysis tool to study the evolution of several key parameters during the protocol repetitions of the MDS-UPDRS III. Scores from 2D automated analysis showed a significant correlation with gold-standard ratings of MDS-UPDRS III, measured with coefficients of determination for the tapping (0.609) and hand movements (0.701) protocols using decision tree algorithms. The individual correlations of the different parameters measured with MDS-UPDRS III scores carry meaningful information and are consistent with MDS-UPDRS III guidelines.

CONCLUSION

We developed a deep learning-based tool to precisely analyze movement parameters allowing to reliably score bradykinesia for parkinsonian patients in a MDS-UPDRS manner.

Applications of Pose Estimation in Human Health and Performance across the Lifespan

The emergence of pose estimation algorithms represents a potential paradigm shift in the study and assessment of human movement. Human pose estimation algorithms leverage advances in computer vision to track human movement automatically from simple videos recorded using common household devices with relatively low-cost cameras (e.g., smartphones, tablets, laptop computers). In our view, these technologies offer clear and exciting potential to make measurement of human movement substantially more accessible; for example, a clinician could perform a quantitative motor assessment directly in a patient's home, a researcher without access to expensive motion capture equipment could analyze movement kinematics using a smartphone video, and a coach could evaluate player performance with video recordings directly from the field. In this review, we combine expertise and perspectives from physical therapy, speech-language pathology, movement science, and engineering to provide insight into applications of pose estimation in human health and performance. We focus specifically on applications in areas of human development, performance optimization, injury prevention, and motor assessment of persons with neurologic damage or disease. We review relevant literature, share interdisciplinary viewpoints on future applications of these technologies to improve human health and performance, and discuss perceived limitations.

A Clinically Interpretable Computer-Vision Based Method for Quantifying Gait in Parkinson's Disease

Gait is a core motor function and is impaired in numerous neurological diseases, including Parkinson's disease (PD). Treatment changes in PD are frequently driven by gait assessments in the clinic, commonly rated as part of the Movement Disorder Society (MDS) Unified PD Rating Scale (UPDRS) assessment (item 3.10). We proposed and evaluated a novel approach for estimating severity of gait impairment in Parkinson's disease using a computer vision-based methodology. The system we developed can be used to obtain an estimate for a rating to catch potential errors, or to gain an initial rating in the absence of a trained clinician-for example, during remote home assessments. Videos (n=729) were collected as part of routine MDS-UPDRS gait assessments of Parkinson's patients, and a deep learning library was used to extract body key-point coordinates for each frame. Data were recorded at five clinical sites using commercially available mobile phones or tablets, and had an associated severity rating from a trained clinician. Six features were calculated from time-series signals of the extracted key-points. These features characterized key aspects of the movement including speed (step frequency, estimated using a novel Gamma-Poisson Bayesian model), arm swing, postural control and smoothness (or roughness) of movement. An ordinal random forest classification model (with one class for each of the possible ratings) was trained and evaluated using 10-fold cross validation. Step frequency point estimates from the Bayesian model were highly correlated with manually labelled step frequencies of 606 video clips showing patients walking towards or away from the camera (Pearson's r=0.80, p<0.001). Our classifier achieved a balanced accuracy of 50% (chance = 25%). Estimated UPDRS ratings were within one of the clinicians' ratings in 95% of cases. There was a significant correlation between clinician labels and model estimates (Spearman's ρ=0.52, p<0.001). We show how the interpretability of the feature values could be used by clinicians to support their decision-making and provide insight into the model's objective UPDRS rating estimation. The severity of gait impairment in Parkinson's disease can be estimated using a single patient video, recorded using a consumer mobile device and within standard clinical settings; i.e., videos were recorded in various hospital hallways and offices rather than gait laboratories. This approach can support clinicians during routine assessments by providing an objective rating (or second opinion), and has the potential to be used for remote home assessments, which would allow for more frequent monitoring.

Anomalous gait feature classification from 3-D motion capture data

The gait kinematics of an individual is affected by various factors, including age, anthropometry, gender, and disease. Detecting anomalous gait features aids in the diagnosis and treatment of gait-related diseases. The objective of this study was to develop a machine learning method for automatically classifying five anomalous gait features, i.e., toe-out, genu varum, pes planus, hindfoot valgus, and forward head posture features, from three-dimensional data on gait kinematics. Gait data and gait feature labels of 488 subjects were acquired. The orientations of the human body segments during a gait cycle were mapped to a low-dimensional latent gait vector using a variational autoencoder. A two-layer neural network was trained to classify five gait features using logistic regression and calculate an anomalous gait feature vector (AGFV). The proposed network showed balanced accuracies of 82.8% for a toe-out, 85.9% for hindfoot valgus, 80.2% for pes planus, 73.2% for genu varum, and 92.9% for forward head posture when the AGFV was rounded to the nearest zero or 1. Multiple anomalous gait features were detectable using the proposed method, which has a practical advantage over current gait indices, including the gait deviation index with a single value. The overall results confirmed the feasibility of using the proposed method for screening subjects with anomalous gait features using three-dimensional motion capture data.