Validation and application of computer vision algorithms for video-based tremor analysis

Tremor is one of the most common neurological symptoms. Its clinical and neurobiological complexity necessitates novel approaches for granular phenotyping. Instrumented neurophysiological analyses have proven useful, but are highly resource-intensive and lack broad accessibility. In contrast, bedside scores are simple to administer, but lack the granularity to capture subtle but relevant tremor features. We utilise the open-source computer vision pose tracking algorithm Mediapipe to track hands in clinical video recordings and use the resulting time series to compute canonical tremor features. This approach is compared to marker-based 3D motion capture, wrist-worn accelerometry, clinical scoring and a second, specifically trained tremor-specific algorithm in two independent clinical cohorts. These cohorts consisted of 66 patients diagnosed with essential tremor, assessed in different task conditions and states of deep brain stimulation therapy. We find that Mediapipe-derived tremor metrics exhibit high convergent clinical validity to scores (Spearman's ρ = 0.55-0.86, p≤ .01) as well as an accuracy of up to 2.60 mm (95% CI [-3.13, 8.23]) and ≤0.21 Hz (95% CI [-0.05, 0.46]) for tremor amplitude and frequency measurements, matching gold-standard equipment. Mediapipe, but not the disease-specific algorithm, was capable of analysing videos involving complex configurational changes of the hands. Moreover, it enabled the extraction of tremor features with diagnostic and prognostic relevance, a dimension which conventional tremor scores were unable to provide. Collectively, this demonstrates that current computer vision algorithms can be transformed into an accurate and highly accessible tool for video-based tremor analysis, yielding comparable results to gold standard tremor recordings.

Visualisation of Parkinsonian, essential and physiological tremor planes in 3Dspace

Based on the fact that tremors display some distinct 3D spatial characteristics, we decided to visualise tremor planes in 3D space. We obtained 3-axial linear accelerometer signals of hand tremors from 58 patients with Parkinson´s disease (PD), 37 with isolated resting tremor (iRT), 75 with essential tremor (ET), and 44 healthy volunteers with physiological tremor (Ph). For each group analysis was done with subsequent spatial 3D regression of the input data i.e. along the x, y and z axes; the projected vector lengths in the individual (vertical transversal XY, vertical longitudinal XZ and horizontal YZ) reference frame planes and their angles. Most meaningful and statistically significant differences were found in the analyses of the 3D vector lengths. The tremor of the PD and the iRT group was oriented mainly in the horizontal YZ plane. The tremors of the patients with ET and Ph were oriented approximately in the midway between the all three referential planes with less tilt toward the vertical longitudinal XZ plane.