Automatic Classification and Severity Estimation of Ataxia From Finger Tapping Videos

Automatic two-dimensional & three-dimensional video analysis with deep learning for movement disorders: A systematic review

The advent of computer vision technology and increased usage of video cameras in clinical settings have facilitated advancements in movement disorder analysis. This review investigated these advancements in terms of providing practical, low-cost solutions for the diagnosis and analysis of movement disorders, such as Parkinson's disease, ataxia, dyskinesia, and Tourette syndrome. Traditional diagnostic methods for movement disorders are typically reliant on the subjective assessment of motor symptoms, which poses inherent challenges. Furthermore, early symptoms are often overlooked, and overlapping symptoms across diseases can complicate early diagnosis. Consequently, deep learning has been used for the objective video-based analysis of movement disorders. This study systematically reviewed the latest advancements in automatic two-dimensional & three-dimensional video analysis using deep learning for movement disorders. We comprehensively analyzed the literature published until September 2023 by searching the Web of Science, PubMed, Scopus, and Embase databases. We identified 68 relevant studies and extracted information on their objectives, datasets, modalities, and methodologies. The study aimed to identify, catalogue, and present the most significant advancements, offering a consolidated knowledge base on the role of video analysis and deep learning in movement disorder analysis. First, the objectives, including specific PD symptom quantification, ataxia assessment, cerebral palsy assessment, gait disorder analysis, tremor assessment, tic detection (in the context of Tourette syndrome), dystonia assessment, and abnormal movement recognition were discussed. Thereafter, the datasets used in the study were examined. Subsequently, video modalities and deep learning methodologies related to the topic were investigated. Finally, the challenges and opportunities in terms of datasets, interpretability, evaluation methods, and home/remote monitoring were discussed.

Validation and application of computer vision algorithms for video-based tremor analysis

Tremor is one of the most common neurological symptoms. Its clinical and neurobiological complexity necessitates novel approaches for granular phenotyping. Instrumented neurophysiological analyses have proven useful, but are highly resource-intensive and lack broad accessibility. In contrast, bedside scores are simple to administer, but lack the granularity to capture subtle but relevant tremor features. We utilise the open-source computer vision pose tracking algorithm Mediapipe to track hands in clinical video recordings and use the resulting time series to compute canonical tremor features. This approach is compared to marker-based 3D motion capture, wrist-worn accelerometry, clinical scoring and a second, specifically trained tremor-specific algorithm in two independent clinical cohorts. These cohorts consisted of 66 patients diagnosed with essential tremor, assessed in different task conditions and states of deep brain stimulation therapy. We find that Mediapipe-derived tremor metrics exhibit high convergent clinical validity to scores (Spearman's ρ = 0.55-0.86, p≤ .01) as well as an accuracy of up to 2.60 mm (95% CI [-3.13, 8.23]) and ≤0.21 Hz (95% CI [-0.05, 0.46]) for tremor amplitude and frequency measurements, matching gold-standard equipment. Mediapipe, but not the disease-specific algorithm, was capable of analysing videos involving complex configurational changes of the hands. Moreover, it enabled the extraction of tremor features with diagnostic and prognostic relevance, a dimension which conventional tremor scores were unable to provide. Collectively, this demonstrates that current computer vision algorithms can be transformed into an accurate and highly accessible tool for video-based tremor analysis, yielding comparable results to gold standard tremor recordings.

Rates of change of pons and middle cerebellar peduncle diameters are diagnostic of multiple system atrophy of the cerebellar type

Definitive diagnosis of multiple system atrophy of the cerebellar type (MSA-C) is challenging. We hypothesized that rates of change of pons and middle cerebellar peduncle diameters on MRI would be unique to MSA-C and serve as diagnostic biomarkers. We defined the normative data for anterior-posterior pons and transverse middle cerebellar peduncle diameters on brain MRI in healthy controls, performed diameter-volume correlations and measured intra- and inter-rater reliability. We studied an Exploratory cohort (2002-2014) of 88 MSA-C and 78 other cerebellar ataxia patients, and a Validation cohort (2015-2021) of 49 MSA-C, 13 multiple system atrophy of the parkinsonian type (MSA-P), 99 other cerebellar ataxia patients and 314 non-ataxia patients. We measured anterior-posterior pons and middle cerebellar peduncle diameters on baseline and subsequent MRIs, and correlated results with Brief Ataxia Rating Scale scores. We assessed midbrain:pons and middle cerebellar peduncle:pons ratios over time. The normative anterior-posterior pons diameter was 23.6 ± 1.6 mm, and middle cerebellar peduncle diameter 16.4 ± 1.4 mm. Pons diameter correlated with volume, r = 0.94, P < 0.0001. The anterior-posterior pons and middle cerebellar peduncle measures were smaller at first scan in MSA-C compared to all other ataxias; anterior-posterior pons diameter: Exploratory, 19.3 ± 2.6 mm versus 20.7 ± 2.6 mm, Validation, 19.9 ± 2.1 mm versus 21.1 ± 2.1 mm; middle cerebellar peduncle transverse diameter, Exploratory, 12.0 ± 2.6 mm versus 14.3 ±2.1 mm, Validation, 13.6 ± 2.1 mm versus 15.1 ± 1.8 mm, all P < 0.001. The anterior-posterior pons and middle cerebellar peduncle rates of change were faster in MSA-C than in all other ataxias; anterior-posterior pons diameter rates of change: Exploratory, -0.87 ± 0.04 mm/year versus -0.09 ± 0.02 mm/year, Validation, -0.89 ± 0.48 mm/year versus -0.10 ± 0.21 mm/year; middle cerebellar peduncle transverse diameter rates of change: Exploratory, -0.84 ± 0.05 mm/year versus -0.08 ± 0.02 mm/year, Validation, -0.94 ± 0.64 mm/year versus -0.11 ± 0.27 mm/year, all values P < 0.0001. Anterior-posterior pons and middle cerebellar peduncle diameters were indistinguishable between Possible, Probable and Definite MSA-C. The rate of anterior-posterior pons atrophy was linear, correlating with ataxia severity. Using a lower threshold anterior-posterior pons diameter decrease of -0.4 mm/year to balance sensitivity and specificity, area under the curve analysis discriminating MSA-C from other ataxias was 0.94, yielding sensitivity 0.92 and specificity 0.87. For the middle cerebellar peduncle, with threshold decline -0.5 mm/year, area under the curve was 0.90 yielding sensitivity 0.85 and specificity 0.79. The midbrain:pons ratio increased progressively in MSA-C, whereas the middle cerebellar peduncle:pons ratio was almost unchanged. Anterior-posterior pons and middle cerebellar peduncle diameters were smaller in MSA-C than in MSA-P, P < 0.001. We conclude from this 20-year longitudinal clinical and imaging study that anterior-posterior pons and middle cerebellar peduncle diameters are phenotypic imaging biomarkers of MSA-C. In the correct clinical context, an anterior-posterior pons and transverse middle cerebellar peduncle diameter decline of ∼0.8 mm/year is sufficient for and diagnostic of MSA-C.

Ataxia Rating Scales Reflect Patient Experience: an Examination of the Relationship Between Clinician Assessments of Cerebellar Ataxia and Patient-Reported Outcomes

Ataxia rating scales are observer administered clinical outcome assessments (COAs) of the cerebellar motor syndrome. It is not known whether these COAs mirror patient experience of their disease. Here we test the hypothesis that ataxia COAs are related to and reflect patient reported symptoms and impact of illness. A concept library of symptoms and activities impacted by ataxia was created by reviewing (a) concept elicitation data from surveys completed by 147 ataxia patients and 80 family members and (b) cognitive debrief data from focus groups of 17 ataxia patients used to develop the Patient Reported Outcome Measure of Ataxia. These findings were mapped across the items on 4 clinical measures of ataxia (SARA, BARS, ICARS and FARS). Symptoms reported most commonly related to balance, gait or walking, speech, tremor and involuntary movements, and vision impairment. Symptoms reported less frequently related to hand coordination, loss of muscle control, dizziness and vertigo, muscle discomfort or pain, swallowing, and incontinence. There was a mosaic mapping of items in the observer-derived ataxia COAs with the subjective reports by ataxia patients/families of the relevance of these items to their daily lives. Most COA item mapped onto multiple real-life manifestations; and most of the real-life impact of disease mapped onto multiple COA items. The 4 common ataxia COAs reflect patient reported symptoms and impact of illness. These results validate the relevance of the COAs to patients' lives and underscore the inadvisability of singling out any one COA item to represent the totality of the patient experience.

Using AI to measure Parkinson's disease severity at home

We present an artificial intelligence (AI) system to remotely assess the motor performance of individuals with Parkinson's disease (PD). In our study, 250 global participants performed a standardized motor task involving finger-tapping in front of a webcam. To establish the severity of Parkinsonian symptoms based on the finger-tapping task, three expert neurologists independently rated the recorded videos on a scale of 0-4, following the Movement Disorder Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS). The inter-rater reliability was excellent, with an intra-class correlation coefficient (ICC) of 0.88. We developed computer algorithms to obtain objective measurements that align with the MDS-UPDRS guideline and are strongly correlated with the neurologists' ratings. Our machine learning model trained on these measures outperformed two MDS-UPDRS certified raters, with a mean absolute error (MAE) of 0.58 points compared to the raters' average MAE of 0.83 points. However, the model performed slightly worse than the expert neurologists (0.53 MAE). The methodology can be replicated for similar motor tasks, providing the possibility of evaluating individuals with PD and other movement disorders remotely, objectively, and in areas with limited access to neurological care.

An Explainable Spatial-Temporal Graphical Convolutional Network to Score Freezing of Gait in Parkinsonian Patients

Freezing of gait (FOG) is a poorly understood heterogeneous gait disorder seen in patients with parkinsonism which contributes to significant morbidity and social isolation. FOG is currently measured with scales that are typically performed by movement disorders specialists (i.e., MDS-UPDRS), or through patient completed questionnaires (N-FOG-Q) both of which are inadequate in addressing the heterogeneous nature of the disorder and are unsuitable for use in clinical trials The purpose of this study was to devise a method to measure FOG objectively, hence improving our ability to identify it and accurately evaluate new therapies. A major innovation of our study is that it is the first study of its kind that uses the largest sample size (>30 h, N = 57) in order to apply explainable, multi-task deep learning models for quantifying FOG over the course of the medication cycle and at varying levels of parkinsonism severity. We trained interpretable deep learning models with multi-task learning to simultaneously score FOG (cross-validated F1 score 97.6%), identify medication state (OFF vs. ON levodopa; cross-validated F1 score 96.8%), and measure total PD severity (MDS-UPDRS-III score prediction error ≤ 2.7 points) using kinematic data of a well-characterized sample of N = 57 patients during levodopa challenge tests. The proposed model was able to explain how kinematic movements are associated with each FOG severity level that were highly consistent with the features, in which movement disorders specialists are trained to identify as characteristics of freezing. Overall, we demonstrate that deep learning models' capability to capture complex movement patterns in kinematic data can automatically and objectively score FOG with high accuracy. These models have the potential to discover novel kinematic biomarkers for FOG that can be used for hypothesis generation and potentially as clinical trial outcome measures.

Objective Assessment of Upper-Extremity Motor Functions in Spinocerebellar Ataxia Using Wearable Sensors

The study presents a novel approach to objectively assessing the upper-extremity motor symptoms in spinocerebellar ataxia (SCA) using data collected via a wearable sensor worn on the patient's wrist during upper-extremity tasks associated with the Assessment and Rating of Ataxia (SARA). First, we developed an algorithm for detecting/extracting the cycles of the finger-to-nose test (FNT). We extracted multiple features from the detected cycles and identified features and parameters correlated with the SARA scores. Additionally, we developed models to predict the severity of symptoms based on the FNT. The proposed technique was validated on a dataset comprising the seventeen (n = 17) participants' assessments. The cycle detection technique showed an accuracy of 97.6% in a Bland-Altman analysis and a 94% accuracy (F1-score of 0.93) in predicting the severity of the FNT. Furthermore, the dependency of the upper-extremity tests was investigated through statistical analysis, and the results confirm dependency and potential redundancies in the upper-extremity SARA assessments. Our findings pave the way to enhance the utility of objective measures of SCA assessments. The proposed wearable-based platform has the potential to eliminate subjectivity and inter-rater variabilities in assessing ataxia.

Automatic extraction of upper-limb kinematic activity using deep learning-based markerless tracking during deep brain stimulation implantation for Parkinson's disease: A proof of concept study

Optimal placement of deep brain stimulation (DBS) therapy for treating movement disorders routinely relies on intraoperative motor testing for target determination. However, in current practice, motor testing relies on subjective interpretation and correlation of motor and neural information. Recent advances in computer vision could improve assessment accuracy. We describe our application of deep learning-based computer vision to conduct markerless tracking for measuring motor behaviors of patients undergoing DBS surgery for the treatment of Parkinson's disease. Video recordings were acquired during intraoperative kinematic testing (N = 5 patients), as part of standard of care for accurate implantation of the DBS electrode. Kinematic data were extracted from videos post-hoc using the Python-based computer vision suite DeepLabCut. Both manual and automated (80.00% accuracy) approaches were used to extract kinematic episodes from threshold derived kinematic fluctuations. Active motor epochs were compressed by modeling upper limb deflections with a parabolic fit. A semi-supervised classification model, support vector machine (SVM), trained on the parameters defined by the parabolic fit reliably predicted movement type. Across all cases, tracking was well calibrated (i.e., reprojection pixel errors 0.016-0.041; accuracies >95%). SVM predicted classification demonstrated high accuracy (85.70%) including for two common upper limb movements, arm chain pulls (92.30%) and hand clenches (76.20%), with accuracy validated using a leave-one-out process for each patient. These results demonstrate successful capture and categorization of motor behaviors critical for assessing the optimal brain target for DBS surgery. Conventional motor testing procedures have proven informative and contributory to targeting but have largely remained subjective and inaccessible to non-Western and rural DBS centers with limited resources. This approach could automate the process and improve accuracy for neuro-motor mapping, to improve surgical targeting, optimize DBS therapy, provide accessible avenues for neuro-motor mapping and DBS implantation, and advance our understanding of the function of different brain areas.