Sub-millimetre accurate human hand kinematics: from surface to skeleton

Evaluation of a multi-sensor Leap Motion setup for biomechanical motion capture of the hand

The Leap Motion controller (LMC) offers a low-cost means of markerless hand tracking, however, its utility is limited by a small field of view and reliance on appropriate sensor positioning. A recent update from Leap Motion has enabled the use of a multiple LMC device on a single computer, allowing the tracking of hands from multiple orientations, potentially overcoming the aforementioned limitations. This study describes a method of implementing a multi-LMC setup and evaluates its effect on the validity and reliability of the derived kinematics. This study implemented a Kabsch algorithm and Kalman filter to re-orientate and fuse the trajectories captured by three LMC at different orientations. Reliability was assessed by comparing between-day differences in maximum joint angles (ΔMJA) and a calculated coefficient of multiple correlations (CMC). Validity was assessed by comparing the LMC to the gold standard, a Vicon markered motion capture (MMC) system, and calculating the ΔMJA and applying the linear fit method. The proposed method was evaluated by comparing the reliability and validity of the single-LMC setups to the multi-LMC setup. A multi-LMC setup proved successful in improving the reliability and validity of kinematic data, most notably where reliability and validity were poor and variation was high between the single-LMC setups. Findings suggest that through implementing the proposed method, limitations associated with single-LMC setups, notably its reliance on optimal sensor positioning, can be overcome.

3D Tracking of Human Motion Using Visual Skeletonization and Stereoscopic Vision

The design of markerless systems to reconstruct human motion in a timely, unobtrusive and externally valid manner is still an open challenge. Artificial intelligence algorithms based on automatic landmarks identification on video images opened to a new approach, potentially e-viable with low-cost hardware. OpenPose is a library that t using a two-branch convolutional neural network allows for the recognition of skeletons in the scene. Although OpenPose-based solutions are spreading, their metrological performances relative to video setup are still largely unexplored. This paper aimed at validating a two-cameras OpenPose-based markerless system for gait analysis, considering its accuracy relative to three factors: cameras' relative distance, gait direction and video resolution. Two volunteers performed a walking test within a gait analysis laboratory. A marker-based optical motion capture system was taken as a reference. Procedures involved: calibration of the stereoscopic system; acquisition of video recordings, simultaneously with the reference marker-based system; video processing within OpenPose to extract the subject's skeleton; videos synchronization; triangulation of the skeletons in the two videos to obtain the 3D coordinates of the joints. Two set of parameters were considered for the accuracy assessment: errors in trajectory reconstruction and error in selected gait space-temporal parameters (step length, swing and stance time). The lowest error in trajectories (~20 mm) was obtained with cameras 1.8 m apart, highest resolution and straight gait, and the highest (~60 mm) with the 1.0 m, low resolution and diagonal gait configuration. The OpenPose-based system tended to underestimate step length of about 1.5 cm, while no systematic biases were found for swing/stance time. Step length significantly changed according to gait direction (p = 0.008), camera distance (p = 0.020), and resolution (p < 0.001). Among stance and swing times, the lowest errors (0.02 and 0.05 s for stance and swing, respectively) were obtained with the 1 m, highest resolution and straight gait configuration. These findings confirm the feasibility of tracking kinematics and gait parameters of a single subject in a 3D space using two low-cost webcams and the OpenPose engine. In particular, the maximization of cameras distance and video resolution enabled to achieve the highest metrological performances.

Quantifying Soft Tissue Artefacts and Imaging Variability in Motion Capture of the Fingers

This study assessed the accuracy of marker-based kinematic analysis of the fingers, considering soft tissue artefacts (STA) and marker imaging uncertainty. We collected CT images of the hand from healthy volunteers with fingers in full extension, mid- and full-flexion, including motion capture markers. Bones and markers were segmented and meshed. The bone meshes for each volunteer's scans were aligned using the proximal phalanx to study the proximal interphalangeal joint (PIP), and using the middle phalanx to study the distal interphalangeal joint (DIP). The angle changes between positions were extracted. The HAWK protocol was used to calculate PIP and DIP joint flexion angles in each position based on the marker centroids. Finally the marker locations were 'corrected' relative to the underlying bones, and the flexion angles recalculated. Static and dynamic marker imaging uncertainty was evaluated using a wand. A strong positive correlation was observed between marker- and CT-based joint angle changes with 0.980 and 0.892 regression slopes for PIP and DIP, respectively, and Root Mean Squared Errors below 4°. Notably for the PIP joint, correlation was worsened by STA correction. The 95% imaging uncertainty interval was < ± 1° for joints, and < ± 0.25 mm for segment lengths. In summary, the HAWK marker set's accuracy was characterised for finger joint flexion angle changes in a small group of healthy individuals and static poses, and was found to benefit from skin movements during flexion.

Recognition of American Sign Language Gestures in a Virtual Reality Using Leap Motion

We perform gesture recognition in a Virtual Reality (VR) environment using dataproduced by the Leap Motion device. Leap Motion generates a virtual three-dimensional (3D) handmodel by recognizing and tracking user‘s hands. From this model, the Leap Motion applicationprogramming interface (API) provides hand and finger locations in the 3D space. We present asystem that is capable of learning gestures by using the data from the Leap Motion device and theHidden Markov classification (HMC) algorithm. We have achieved the gesture recognitionaccuracy (mean ± SD) is 86.1 ± 8.2% and gesture typing speed is 3.09 ± 0.53 words per minute(WPM), when recognizing the gestures of the American Sign Language (ASL).