HRDepthNet: Depth Image-Based Marker-Less Tracking of Body Joints

Development of a Gait Analysis Application for Assessing Upper and Lower Limb Movements to Detect Pathological Gait

Pathological gait in patients with Hakim's disease (HD, synonymous with idiopathic normal-pressure hydrocephalus; iNPH), Parkinson's disease (PD), and cervical myelopathy (CM) has been subjectively evaluated in this study. We quantified the characteristics of upper and lower limb movements in patients with pathological gait. We analyzed 1491 measurements of 1 m diameter circular walking from 122, 12, and 93 patients with HD, PD, and CM, respectively, and 200 healthy volunteers using the Three-Dimensional Pose Tracker for Gait Test. Upper and lower limb movements of 2D coordinates projected onto body axis sections were derived from estimated 3D relative coordinates. The hip and knee joint angle ranges on the sagittal plane were significantly smaller in the following order: healthy > CM > PD > HD, whereas the shoulder and elbow joint angle ranges were significantly smaller, as follows: healthy > CM > HD > PD. The outward shift of the leg on the axial plane was significantly greater, as follows: healthy < CM < PD < HD, whereas the outward shift of the upper limb followed the order of healthy > CM > HD > PD. The strongest correlation between the upper and lower limb movements was identified in the angle ranges of the hip and elbow joints on the sagittal plane. The lower and upper limb movements during circular walking were correlated. Patients with HD and PD exhibited reduced back-and-forth swings of the upper and lower limbs.

Development of Smartphone Application for Markerless Three-Dimensional Motion Capture Based on Deep Learning Model

To quantitatively assess pathological gait, we developed a novel smartphone application for full-body human motion tracking in real time from markerless video-based images using a smartphone monocular camera and deep learning. As training data for deep learning, the original three-dimensional (3D) dataset comprising more than 1 million captured images from the 3D motion of 90 humanoid characters and the two-dimensional dataset of COCO 2017 were prepared. The 3D heatmap offset data consisting of 28 × 28 × 28 blocks with three red-green-blue colors at the 24 key points of the entire body motion were learned using the convolutional neural network, modified ResNet34. At each key point, the hottest spot deviating from the center of the cell was learned using the tanh function. Our new iOS application could detect the relative tri-axial coordinates of the 24 whole-body key points centered on the navel in real time without any markers for motion capture. By using the relative coordinates, the 3D angles of the neck, lumbar, bilateral hip, knee, and ankle joints were estimated. Any human motion could be quantitatively and easily assessed using a new smartphone application named Three-Dimensional Pose Tracker for Gait Test (TDPT-GT) without any body markers or multipoint cameras.

Automatic Markerless Motion Detector Method against Traditional Digitisation for 3-Dimensional Movement Kinematic Analysis of Ball Kicking in Soccer Field Context

Kicking is a fundamental skill in soccer that often contributes to match outcomes. Lower limb movement features (e.g., joint position and velocity) are determinants of kick performance. However, obtaining kicking kinematics under field conditions generally requires time-consuming manual tracking. The current study aimed to compare a contemporary markerless automatic motion estimation algorithm (OpenPose) with manual digitisation (DVIDEOW software) in obtaining on-field kicking kinematic parameters. An experimental dataset of under-17 players from all outfield positions was used. Kick attempts were performed in an official pitch against a goalkeeper. Four digital video cameras were used to record full-body motion during support and ball contact phases of each kick. Three-dimensional positions of hip, knee, ankle, toe and foot centre-of-mass (CMfoot) generally showed no significant differences when computed by automatic as compared to manual tracking (whole kicking movement cycle), while only z-coordinates of knee and calcaneus markers at specific points differed between methods. The resulting time-series matrices of positions (r2 = 0.94) and velocity signals (r2 = 0.68) were largely associated (all p < 0.01). The mean absolute error of OpenPose motion tracking was 3.49 cm for determining positions (ranging from 2.78 cm (CMfoot) to 4.13 cm (dominant hip)) and 1.29 m/s for calculating joint velocity (0.95 m/s (knee) to 1.50 m/s (non-dominant hip)) as compared to reference measures by manual digitisation. Angular range-of-motion showed significant correlations between methods for the ankle (r = 0.59, p < 0.01, large) and knee joint displacements (r = 0.84, p < 0.001, very large) but not in the hip (r = 0.04, p = 0.85, unclear). Markerless motion tracking (OpenPose) can help to successfully obtain some lower limb position, velocity, and joint angular outputs during kicks performed in a naturally occurring environment.