Development of Smartphone Application for Markerless Three-Dimensional Motion Capture Based on Deep Learning Model

Classification of Fashion Models' Walking Styles Using Publicly Available Data, Pose Detection Technology, and Multivariate Analysis: From Past to Current Trendy Walking Styles

Understanding past and current trends is crucial in the fashion industry to forecast future market demands. This study quantifies and reports the characteristics of the trendy walking styles of fashion models during real-world runway performances using three cutting-edge technologies: (a) publicly available video resources, (b) human pose detection technology, and (c) multivariate human-movement analysis techniques. The skeletal coordinates of the whole body during one gait cycle, extracted from publicly available video resources of 69 fashion models, underwent principal component analysis to reduce the dimensionality of the data. Then, hierarchical cluster analysis was used to classify the data. The results revealed that (1) the gaits of the fashion models analyzed in this study could be classified into five clusters, (2) there were significant differences in the median years in which the shows were held between the clusters, and (3) reconstructed stick-figure animations representing the walking styles of each cluster indicate that an exaggerated leg-crossing gait has become less common over recent years. Accordingly, we concluded that the level of leg crossing while walking is one of the major changes in trendy walking styles, from the past to the present, directed by the world's leading brands.

Exercise quantification from single camera view markerless 3D pose estimation

Sports physiotherapists and coaches are tasked with evaluating the movement quality of athletes across the spectrum of ability and experience. However, the accuracy of visual observation is low and existing technology outside of expensive lab-based solutions has limited adoption, leading to an unmet need for an efficient and accurate means to measure static and dynamic joint angles during movement, converted to movement metrics useable by practitioners. This paper proposes a set of pose landmarks for computing frequently used joint angles as metrics of interest to sports physiotherapists and coaches in assessing common strength-building human exercise movements. It then proposes a set of rules for computing these metrics for a range of common exercises (single and double drop jumps and counter-movement jumps, deadlifts and various squats) from anatomical key-points detected using video, and evaluates the accuracy of these using a published 3D human pose model trained with ground truth data derived from VICON motion capture of common rehabilitation exercises. Results show a set of mathematically defined metrics which are derived from the chosen pose landmarks, and which are sufficient to compute the metrics for each of the exercises under consideration. Comparison to ground truth data showed that root mean square angle errors were within 10° for all exercises for the following metrics: shin angle, knee varus/valgus and left/right flexion, hip flexion and pelvic tilt, trunk angle, spinal flexion lower/upper/mid and rib flare. Larger errors (though still all within 15°) were observed for shoulder flexion and ASIS asymmetry in some exercises, notably front squats and drop-jumps. In conclusion, the contribution of this paper is that a set of sufficient key-points and associated metrics for exercise assessment from 3D human pose have been uniquely defined. Further, we found generally very good accuracy of the Strided Transformer 3D pose model in predicting these metrics for the chosen set of exercises from a single mobile device camera, when trained on a suitable set of functional exercises recorded using a VICON motion capture system. Future assessment of generalization is needed.

Effective evaluation of HGcnMLP method for markerless 3D pose estimation of musculoskeletal diseases patients based on smartphone monocular video

Markerless pose estimation based on computer vision provides a simpler and cheaper alternative to human motion capture, with great potential for clinical diagnosis and remote rehabilitation assessment. Currently, the markerless 3D pose estimation is mainly based on multi-view technology, while the more promising single-view technology has defects such as low accuracy and reliability, which seriously limits clinical application. This study proposes a high-resolution graph convolutional multilayer perception (HGcnMLP) human 3D pose estimation framework for smartphone monocular videos and estimates 15 healthy adults and 12 patients with musculoskeletal disorders (sarcopenia and osteoarthritis) gait spatiotemporal, knee angle, and center-of-mass (COM) velocity parameters, etc., and compared with the VICON gold standard system. The results show that most of the calculated parameters have excellent reliability (VICON, ICC (2, k): 0.853-0.982; Phone, ICC (2, k): 0.839-0.975) and validity (Pearson r: 0.808-0.978, p< 0.05). In addition, the proposed system can better evaluate human gait balance ability, and the K-means++ clustering algorithm can successfully distinguish patients into different recovery level groups. This study verifies the potential of a single smartphone video for 3D human pose estimation for rehabilitation auxiliary diagnosis and balance level recognition, and is an effective attempt at the clinical application of emerging computer vision technology. In the future, it is hoped that the corresponding smartphone program will be developed to provide a low-cost, effective, and simple new tool for remote monitoring and rehabilitation assessment of patients.

Fluctuations in Upper and Lower Body Movement during Walking in Normal Pressure Hydrocephalus and Parkinson's Disease Assessed by Motion Capture with a Smartphone Application, TDPT-GT

We aimed to capture the fluctuations in the dynamics of body positions and find the characteristics of them in patients with idiopathic normal pressure hydrocephalus (iNPH) and Parkinson's disease (PD). With the motion-capture application (TDPT-GT) generating 30 Hz coordinates at 27 points on the body, walking in a circle 1 m in diameter was recorded for 23 of iNPH, 23 of PD, and 92 controls. For 128 frames of calculated distances from the navel to the other points, after the Fourier transforms, the slopes (the representatives of fractality) were obtained from the graph plotting the power spectral density against the frequency in log-log coordinates. Differences in the average slopes were tested by one-way ANOVA and multiple comparisons between every two groups. A decrease in the absolute slope value indicates a departure from the 1/f noise characteristic observed in healthy variations. Significant differences in the patient groups and controls were found in all body positions, where patients always showed smaller absolute values. Our system could measure the whole body's movement and temporal variations during walking. The impaired fluctuations of body movement in the upper and lower body may contribute to gait and balance disorders in patients.

Artificial Intelligence Distinguishes Pathological Gait: The Analysis of Markerless Motion Capture Gait Data Acquired by an iOS Application (TDPT-GT)

Distinguishing pathological gait is challenging in neurology because of the difficulty of capturing total body movement and its analysis. We aimed to obtain a convenient recording with an iPhone and establish an algorithm based on deep learning. From May 2021 to November 2022 at Yamagata University Hospital, Shiga University, and Takahata Town, patients with idiopathic normal pressure hydrocephalus (n = 48), Parkinson's disease (n = 21), and other neuromuscular diseases (n = 45) comprised the pathological gait group (n = 114), and the control group consisted of 160 healthy volunteers. iPhone application TDPT-GT captured the subjects walking in a circular path of about 1 meter in diameter, a markerless motion capture system, with an iPhone camera, which generated the three-axis 30 frames per second (fps) relative coordinates of 27 body points. A light gradient boosting machine (Light GBM) with stratified k-fold cross-validation (k = 5) was applied for gait collection for about 1 min per person. The median ability model tested 200 frames of each person's data for its distinction capability, which resulted in the area under a curve of 0.719. The pathological gait captured by the iPhone could be distinguished by artificial intelligence.

Emerging biological insights enabled by high-resolution 3D motion data: promises, perspectives and pitfalls

Deconstructing motion to better understand it is a key prerequisite in the field of comparative biomechanics. Since Marey and Muybridge's work, technical constraints have been the largest limitation to motion capture and analysis, which, in turn, limited what kinds of questions biologists could ask or answer. Throughout the history of our field, conceptual leaps and significant technical advances have generally worked hand in hand. Recently, high-resolution, three-dimensional (3D) motion data have become easier to acquire, providing new opportunities for comparative biomechanics. We describe how adding a third dimension of information has fuelled major paradigm shifts, not only leading to a reinterpretation of long-standing scientific questions but also allowing new questions to be asked. In this paper, we highlight recent work published in Journal of Experimental Biology and influenced by these studies, demonstrating the biological breakthroughs made with 3D data. Although amazing opportunities emerge from these technical and conceptual advances, high-resolution data often come with a price. Here, we discuss challenges of 3D data, including low-throughput methodology, costly equipment, low sample sizes, and complex analyses and presentation. Therefore, we propose guidelines for how and when to pursue 3D high-resolution data. We also suggest research areas that are poised for major new biological advances through emerging 3D data collection.

Quantitative Gait Feature Assessment on Two-Dimensional Body Axis Projection Planes Converted from Three-Dimensional Coordinates Estimated with a Deep Learning Smartphone App

To assess pathological gaits quantitatively, three-dimensional coordinates estimated with a deep learning model were converted into body axis plane projections. First, 15 healthy volunteers performed four gait patterns; that is, normal, shuffling, short-stepped, and wide-based gaits, with the Three-Dimensional Pose Tracker for Gait Test (TDPT-GT) application. Second, gaits of 47 patients with idiopathic normal pressure hydrocephalus (iNPH) and 92 healthy elderly individuals in the Takahata cohort were assessed with the TDPT-GT. Two-dimensional relative coordinates were calculated from the three-dimensional coordinates by projecting the sagittal, coronal, and axial planes. Indices of the two-dimensional relative coordinates associated with a pathological gait were comprehensively explored. The candidate indices for the shuffling gait were the angle range of the hip joint < 30° and relative vertical amplitude of the heel < 0.1 on the sagittal projection plane. For the short-stepped gait, the angle range of the knee joint < 45° on the sagittal projection plane was a candidate index. The candidate index for the wide-based gait was the leg outward shift > 0.1 on the axial projection plane. In conclusion, the two-dimensional coordinates on the body axis projection planes calculated from the 3D relative coordinates estimated by the TDPT-GT application enabled the quantification of pathological gait features.