Can Markerless Pose Estimation Algorithms Estimate 3D Mass Centre Positions and Velocities during Linear Sprinting Activities?

MocapMe: DeepLabCut-Enhanced Neural Network for Enhanced Markerless Stability in Sit-to-Stand Motion Capture

This study examined the efficacy of an optimized DeepLabCut (DLC) model in motion capture, with a particular focus on the sit-to-stand (STS) movement, which is crucial for assessing the functional capacity in elderly and postoperative patients. This research uniquely compared the performance of this optimized DLC model, which was trained using 'filtered' estimates from the widely used OpenPose (OP) model, thereby emphasizing computational effectiveness, motion-tracking precision, and enhanced stability in data capture. Utilizing a combination of smartphone-captured videos and specifically curated datasets, our methodological approach included data preparation, keypoint annotation, and extensive model training, with an emphasis on the flow of the optimized model. The findings demonstrate the superiority of the optimized DLC model in various aspects. It exhibited not only higher computational efficiency, with reduced processing times, but also greater precision and consistency in motion tracking thanks to the stability brought about by the meticulous selection of the OP data. This precision is vital for developing accurate biomechanical models for clinical interventions. Moreover, this study revealed that the optimized DLC maintained higher average confidence levels across datasets, indicating more reliable and accurate detection capabilities compared with standalone OP. The clinical relevance of these findings is profound. The optimized DLC model's efficiency and enhanced point estimation stability make it an invaluable tool in rehabilitation monitoring and patient assessments, potentially streamlining clinical workflows. This study suggests future research directions, including integrating the optimized DLC model with virtual reality environments for enhanced patient engagement and leveraging its improved data quality for predictive analytics in healthcare. Overall, the optimized DLC model emerged as a transformative tool for biomechanical analysis and physical rehabilitation, promising to enhance the quality of patient care and healthcare delivery efficiency.

Agreement between a markerless and a marker-based motion capture systems for balance related quantities

Balance studies usually focus on quantities describing the global body motion. Assessing such quantities using classical marker-based approach can be tedious and modify the participant's behaviour. The recent development of markerless motion capture methods could bypass the issues related to the use of markers. This work compared dynamic balance related quantities obtained with markers and videos. Sixteen young healthy participants performed four different motor tasks: walking at self-selected speed, balance loss, walking on a narrow beam and countermovement jumps. Their movements were recorded simultaneously by marker-based and markerless motion capture systems. Videos were processed using a commercial markerless pose estimation software, Theia3D. The centre of mass position (CoM) was computed, and the associated extrapolated centre of mass position (XCoM) and whole-body angular momentum (WBAM) were derived. Bland-Altman analysis was performed and root mean square difference (RMSD) and coefficient of correlation were computed to compare the results obtained with marker-based and markerless methods. Bias remained of the magnitude of a few mm for CoM and XCoM positions, and RMSD of CoM and XCoM was around 1 cm. RMSD of the WBAM was less than 10 % of the total amplitude in any direction, and bias was less than 1 %. Results suggest that outcomes of balance studies will be similar whether marker-based or markerless motion capture system are used. Nevertheless, one should be careful when assessing dynamic movements such as jumping, as they displayed the biggest differences (both bias and RMSD), although it is unclear whether these differences are due to errors in markerless or marker-based motion capture system.

Feasibility of OpenPose markerless motion analysis in a real athletics competition

This study tested the performance of OpenPose on footage collected by two cameras at 200 Hz from a real-life competitive setting by comparing it with manually analyzed data in SIMI motion. The same take-off recording from the men's Long Jump finals at the 2017 World Athletics Championships was used for both approaches (markerless and manual) to reconstruct the 3D coordinates from each of the camera's 2D coordinates. Joint angle and Centre of Mass (COM) variables during the final step and take-off phase of the jump were determined. Coefficients of Multiple Determinations (CMD) for joint angle waveforms showed large variation between athletes with the knee angle values typically being higher (take-off leg: 0.727 ± 0.242; swing leg: 0.729 ± 0.190) than those for hip (take-off leg: 0.388 ± 0.193; swing leg: 0.370 ± 0.227) and ankle angle (take-off leg: 0.247 ± 0.172; swing leg: 0.155 ± 0.228). COM data also showed considerable variation between athletes and parameters, with position (0.600 ± 0.322) and projection angle (0.658 ± 0.273) waveforms generally showing better agreement than COM velocity (0.217 ± 0.241). Agreement for discrete data was generally poor with high random error for joint kinematics and COM parameters at take-off and an average ICC across variables of 0.17. The poor agreement statistics and a range of unrealistic values returned by the pose estimation underline that OpenPose is not suitable for in-competition performance analysis in events such as the long jump, something that manual analysis still achieves with high levels of accuracy and reliability.

Validity of AI-Based Gait Analysis for Simultaneous Measurement of Bilateral Lower Limb Kinematics Using a Single Video Camera

Accuracy validation of gait analysis using pose estimation with artificial intelligence (AI) remains inadequate, particularly in objective assessments of absolute error and similarity of waveform patterns. This study aimed to clarify objective measures for absolute error and waveform pattern similarity in gait analysis using pose estimation AI (OpenPose). Additionally, we investigated the feasibility of simultaneous measuring both lower limbs using a single camera from one side. We compared motion analysis data from pose estimation AI using video footage that was synchronized with a three-dimensional motion analysis device. The comparisons involved mean absolute error (MAE) and the coefficient of multiple correlation (CMC) to compare the waveform pattern similarity. The MAE ranged from 2.3 to 3.1° on the camera side and from 3.1 to 4.1° on the opposite side, with slightly higher accuracy on the camera side. Moreover, the CMC ranged from 0.936 to 0.994 on the camera side and from 0.890 to 0.988 on the opposite side, indicating a "very good to excellent" waveform similarity. Gait analysis using a single camera revealed that the precision on both sides was sufficiently robust for clinical evaluation, while measurement accuracy was slightly superior on the camera side.

Towards Single Camera Human 3D-Kinematics

Markerless estimation of 3D Kinematics has the great potential to clinically diagnose and monitor movement disorders without referrals to expensive motion capture labs; however, current approaches are limited by performing multiple de-coupled steps to estimate the kinematics of a person from videos. Most current techniques work in a multi-step approach by first detecting the pose of the body and then fitting a musculoskeletal model to the data for accurate kinematic estimation. Errors in training data of the pose detection algorithms, model scaling, as well the requirement of multiple cameras limit the use of these techniques in a clinical setting. Our goal is to pave the way toward fast, easily applicable and accurate 3D kinematic estimation. To this end, we propose a novel approach for direct 3D human kinematic estimation D3KE from videos using deep neural networks. Our experiments demonstrate that the proposed end-to-end training is robust and outperforms 2D and 3D markerless motion capture based kinematic estimation pipelines in terms of joint angles error by a large margin (35% from 5.44 to 3.54 degrees). We show that D3KE is superior to the multi-step approach and can run at video framerate speeds. This technology shows the potential for clinical analysis from mobile devices in the future.

Pose2Sim: An End-to-End Workflow for 3D Markerless Sports Kinematics—Part 2: Accuracy

Two-dimensional deep-learning pose estimation algorithms can suffer from biases in joint pose localizations, which are reflected in triangulated coordinates, and then in 3D joint angle estimation. Pose2Sim, our robust markerless kinematics workflow, comes with a physically consistent OpenSim skeletal model, meant to mitigate these errors. Its accuracy was concurrently validated against a reference marker-based method. Lower-limb joint angles were estimated over three tasks (walking, running, and cycling) performed multiple times by one participant. When averaged over all joint angles, the coefficient of multiple correlation (CMC) remained above 0.9 in the sagittal plane, except for the hip in running, which suffered from a systematic 15° offset (CMC = 0.65), and for the ankle in cycling, which was partially occluded (CMC = 0.75). When averaged over all joint angles and all degrees of freedom, mean errors were 3.0°, 4.1°, and 4.0°, in walking, running, and cycling, respectively; and range of motion errors were 2.7°, 2.3°, and 4.3°, respectively. Given the magnitude of error traditionally reported in joint angles computed from a marker-based optoelectronic system, Pose2Sim is deemed accurate enough for the analysis of lower-body kinematics in walking, cycling, and running.

Applications and limitations of current markerless motion capture methods for clinical gait biomechanics

BACKGROUND

Markerless motion capture has the potential to perform movement analysis with reduced data collection and processing time compared to marker-based methods. This technology is now starting to be applied for clinical and rehabilitation applications and therefore it is crucial that users of these systems understand both their potential and limitations. This literature review aims to provide a comprehensive overview of the current state of markerless motion capture for both single camera and multi-camera systems. Additionally, this review explores how practical applications of markerless technology are being used in clinical and rehabilitation settings, and examines the future challenges and directions markerless research must explore to facilitate full integration of this technology within clinical biomechanics.

METHODOLOGY

A scoping review is needed to examine this emerging broad body of literature and determine where gaps in knowledge exist, this is key to developing motion capture methods that are cost effective and practically relevant to clinicians, coaches and researchers around the world. Literature searches were performed to examine studies that report accuracy of markerless motion capture methods, explore current practical applications of markerless motion capture methods in clinical biomechanics and identify gaps in our knowledge that are relevant to future developments in this area.

RESULTS

Markerless methods increase motion capture data versatility, enabling datasets to be re-analyzed using updated pose estimation algorithms and may even provide clinicians with the capability to collect data while patients are wearing normal clothing. While markerless temporospatial measures generally appear to be equivalent to marker-based motion capture, joint center locations and joint angles are not yet sufficiently accurate for clinical applications. Pose estimation algorithms are approaching similar error rates of marker-based motion capture, however, without comparison to a gold standard, such as bi-planar videoradiography, the true accuracy of markerless systems remains unknown.

CONCLUSIONS

Current open-source pose estimation algorithms were never designed for biomechanical applications, therefore, datasets on which they have been trained are inconsistently and inaccurately labelled. Improvements to labelling of open-source training data, as well as assessment of markerless accuracy against gold standard methods will be vital next steps in the development of this technology.

Automatic Markerless Motion Detector Method against Traditional Digitisation for 3-Dimensional Movement Kinematic Analysis of Ball Kicking in Soccer Field Context

Kicking is a fundamental skill in soccer that often contributes to match outcomes. Lower limb movement features (e.g., joint position and velocity) are determinants of kick performance. However, obtaining kicking kinematics under field conditions generally requires time-consuming manual tracking. The current study aimed to compare a contemporary markerless automatic motion estimation algorithm (OpenPose) with manual digitisation (DVIDEOW software) in obtaining on-field kicking kinematic parameters. An experimental dataset of under-17 players from all outfield positions was used. Kick attempts were performed in an official pitch against a goalkeeper. Four digital video cameras were used to record full-body motion during support and ball contact phases of each kick. Three-dimensional positions of hip, knee, ankle, toe and foot centre-of-mass (CM_foot) generally showed no significant differences when computed by automatic as compared to manual tracking (whole kicking movement cycle), while only z-coordinates of knee and calcaneus markers at specific points differed between methods. The resulting time-series matrices of positions (r² = 0.94) and velocity signals (r² = 0.68) were largely associated (all p < 0.01). The mean absolute error of OpenPose motion tracking was 3.49 cm for determining positions (ranging from 2.78 cm (CM_foot) to 4.13 cm (dominant hip)) and 1.29 m/s for calculating joint velocity (0.95 m/s (knee) to 1.50 m/s (non-dominant hip)) as compared to reference measures by manual digitisation. Angular range-of-motion showed significant correlations between methods for the ankle (r = 0.59, p < 0.01, large) and knee joint displacements (r = 0.84, p < 0.001, very large) but not in the hip (r = 0.04, p = 0.85, unclear). Markerless motion tracking (OpenPose) can help to successfully obtain some lower limb position, velocity, and joint angular outputs during kicks performed in a naturally occurring environment.

Single Camera-Based Remote Physical Therapy: Verification on a Large Video Dataset

In recent years, several systems have been developed to capture human motion in real-time using common RGB cameras. This approach has great potential to become widespread among the general public as it allows the remote evaluation of exercise at no additional cost. The concept of using these systems in rehabilitation in the home environment has been discussed, but no work has addressed the practical problem of detecting basic body parts under different sensing conditions on a large scale. In this study, we evaluate the ability of the OpenPose pose estimation algorithm to perform keypoint detection of anatomical landmarks under different conditions. We infer the quality of detection based on the keypoint confidence values reported by the OpenPose. We used more than two thousand unique exercises for the evaluation. We focus on the influence of the camera view and the influence of the position of the trainees, which are essential in terms of the use for home exercise. Our results show that the position of the trainee has the greatest effect, in the following increasing order of suitability across all camera views: lying position, position on the knees, sitting position, and standing position. On the other hand, the effect of the camera view was only marginal, showing that the side view is having slightly worse results. The results might also indicate that the quality of detection of lower body joints is lower across all conditions than the quality of detection of upper body joints. In this practical overview, we present the possibilities and limitations of current camera-based systems in telerehabilitation.

The accuracy of several pose estimation methods for 3D joint centre localisation

Human movement researchers are often restricted to laboratory environments and data capture techniques that are time and/or resource intensive. Markerless pose estimation algorithms show great potential to facilitate large scale movement studies 'in the wild', i.e., outside of the constraints imposed by marker-based motion capture. However, the accuracy of such algorithms has not yet been fully evaluated. We computed 3D joint centre locations using several pre-trained deep-learning based pose estimation methods (OpenPose, AlphaPose, DeepLabCut) and compared to marker-based motion capture. Participants performed walking, running and jumping activities while marker-based motion capture data and multi-camera high speed images (200 Hz) were captured. The pose estimation algorithms were applied to 2D image data and 3D joint centre locations were reconstructed. Pose estimation derived joint centres demonstrated systematic differences at the hip and knee (~ 30-50 mm), most likely due to mislabeling of ground truth data in the training datasets. Where systematic differences were lower, e.g., the ankle, differences of 1-15 mm were observed depending on the activity. Markerless motion capture represents a highly promising emerging technology that could free movement scientists from laboratory environments but 3D joint centre locations are not yet consistently comparable to marker-based motion capture.

A Systematic Review of the Application of Camera-Based Human Pose Estimation in the Field of Sport and Physical Exercise

Human Pose Estimation (HPE) has received considerable attention during the past years, improving its performance thanks to the use of Deep Learning, and introducing new interesting uses, such as its application in Sport and Physical Exercise (SPE). The aim of this systematic review is to analyze the literature related to the application of HPE in SPE, the available data, methods, performance, opportunities, and challenges. One reviewer applied different inclusion and exclusion criteria, as well as quality metrics, to perform the paper filtering through the paper databases. The Association for Computing Machinery Digital Library, Web of Science, and dblp included more than 500 related papers after the initial filtering, finally resulting in 20. In addition, research was carried out regarding the publicly available data related to this topic. It can be concluded that even if related public data can be found, much more data is needed to be able to obtain good performance in different contexts. In relation with the methods of the authors, the use of general purpose systems as base, such as Openpose, combined with other methods and adaptations to the specific use case can be found. Finally, the limitations, opportunities, and challenges are presented.