A Systematic Review of the Application of Camera-Based Human Pose Estimation in the Field of Sport and Physical Exercise

Strategic assessment model of smart stadiums based on genetic algorithms and literature visualization analysis: A case study from Chengdu, China

This paper leverages Citespace and VOSviewer software to perform a comprehensive bibliometric analysis on a corpus of 384 references related to smart sports venues, spanning from 1998 to 2022. The analysis encompasses various facets, including author network analysis, institutional network analysis, temporal mapping, keyword clustering, and co-citation network analysis. Moreover, this paper constructs a smart stadiums strategic assessment model (SSSAM) to compensate for confusion and aimlessness by genetic algorithms (GA). Our findings indicate an exponential growth in publications on smart sports venues year over year. Arizona State University emerges as the institution with the highest number of collaborative publications, Energy and Buildings becomes the publication with the most documents. While, Wang X stands out as the scholar with the most substantial contribution to the field. In scrutinizing the betweenness centrality indicators, a paradigm shift in research hotspots becomes evident-from intelligent software to the domains of the Internet of Things (IoT), intelligent services, and artificial intelligence (AI). The SSSAM model based on artificial neural networks (ANN) and GA algorithms also reached similar conclusions through a case study of the International University Sports Federation (FISU), building Information Modeling (BIM), cloud computing and artificial intelligence Internet of Things (AIoT) are expected to develop in the future. Three key themes developed over time. Finally, a comprehensive knowledge system with common references and future hot spots is proposed.

AI in evaluating ambulation of stroke patients: severity classification with video and functional ambulation category scale

BACKGROUND

The evaluation of gait function and severity classification of stroke patients are important to determine the rehabilitation goal and the level of exercise. Physicians often qualitatively evaluate patients' walking ability through visual gait analysis using naked eye, video images, or standardized assessment tools. Gait evaluation through observation relies on the doctor's empirical judgment, potentially introducing subjective opinions. Therefore, conducting research to establish a basis for more objective judgment is crucial.

OBJECTIVE

To verify a deep learning model that classifies gait image data of stroke patients according to Functional Ambulation Category (FAC) scale.

METHODS

Gait vision data from 203 stroke patients and 182 healthy individuals recruited from six medical institutions were collected to train a deep learning model for classifying gait severity in stroke patients. The recorded videos were processed using OpenPose. The dataset was randomly split into 80% for training and 20% for testing.

RESULTS

The deep learning model attained a training accuracy of 0.981 and test accuracy of 0.903. Area Under the Curve(AUC) values of 0.93, 0.95, and 0.96 for discriminating among the mild, moderate, and severe stroke groups, respectively.

CONCLUSION

This confirms the potential of utilizing human posture estimation based on vision data not only to develop gait parameter models but also to develop models to classify severity according to the FAC criteria used by physicians. To develop an AI-based severity classification model, a large amount and variety of data is necessary and data collected in non-standardized real environments, not in laboratories, can also be used meaningfully.

Climbing Technique Evaluation by Means of Skeleton Video Stream Analysis

Due to the growing interest in climbing, increasing importance has been given to research in the field of non-invasive, camera-based motion analysis. While existing work uses invasive technologies such as wearables or modified walls and holds, or focuses on competitive sports, we for the first time present a system that uses video analysis to automatically recognize six movement errors that are typical for novices with limited climbing experience. Climbing a complete route consists of three repetitive climbing phases. Therefore, a characteristic joint arrangement may be detected as an error in a specific climbing phase, while this exact arrangement may not considered to be an error in another climbing phase. That is why we introduced a finite state machine to determine the current phase and to check for errors that commonly occur in the current phase. The transition between the phases depends on which joints are being used. To capture joint movements, we use a fourth-generation iPad Pro with LiDAR to record climbing sequences in which we convert the climber's 2-D skeleton provided by the Vision framework from Apple into 3-D joints using the LiDAR depth information. Thereupon, we introduced a method that derives whether a joint moves or not, determining the current phase. Finally, the 3-D joints are analyzed with respect to defined characteristic joint arrangements to identify possible motion errors. To present the feedback to the climber, we imitate a virtual mentor by realizing an application on the iPad that creates an analysis immediately after the climber has finished the route by pointing out the detected errors and by giving suggestions for improvement. Quantitative tests with three experienced climbers that were able to climb reference routes without any errors and intentionally with errors resulted in precision-recall curves evaluating the error detection performance. The results demonstrate that while the number of false positives is still in an acceptable range, the number of detected errors is sufficient to provide climbing novices with adequate suggestions for improvement. Moreover, our study reveals limitations that mainly originate from incorrect joint localizations caused by the LiDAR sensor range. With human pose estimation becoming increasingly reliable and with the advance of sensor capabilities, these limitations will have a decreasing impact on our system performance.

Extracting spatial knowledge from track and field broadcasts for monocular 3D human pose estimation

Collecting large datasets for investigations into human locomotion is an expensive and labor-intensive process. Methods for 3D human pose estimation in the wild are becoming increasingly accurate and could soon be sufficient to assist with the collection of datasets for analysis into running kinematics from TV broadcast data. In the domain of biomechanical research, small differences in 3D angles play an important role. More precisely, the error margins of the data collection process need to be smaller than the expected variation between athletes. In this work, we propose a method to infer the global geometry of track and field stadium recordings using lane demarcations. By projecting estimated 3D skeletons back into the image using this global geometry, we show that current state-of-the-art 3D human pose estimation methods are not (yet) accurate enough to be used in kinematics research.

Assessing physical abilities of sarcopenia patients using gait analysis and smart insole for development of digital biomarker

The aim of this study is to compare variable importance across multiple measurement tools, and to use smart insole and artificial intelligence (AI) gait analysis to create variables that can evaluate the physical abilities of sarcopenia patients. By analyzing and comparing sarcopenia patients with non sarcopenia patients, this study aims to develop predictive and classification models for sarcopenia and discover digital biomarkers. The researchers used smart insole equipment to collect plantar pressure data from 83 patients, and a smart phone to collect video data for pose estimation. A Mann-Whitney U was conducted to compare the sarcopenia group of 23 patients and the control group of 60 patients. Smart insole and pose estimation were used to compare the physical abilities of sarcopenia patients with a control group. Analysis of joint point variables showed significant differences in 12 out of 15 variables, but not in knee mean, ankle range, and hip range. These findings suggest that digital biomarkers can be used to differentiate sarcopenia patients from the normal population with improved accuracy. This study compared musculoskeletal disorder patients to sarcopenia patients using smart insole and pose estimation. Multiple measurement methods are important for accurate sarcopenia diagnosis and digital technology has potential for improving diagnosis and treatment.

Estimating Ground Reaction Forces from Two-Dimensional Pose Data: A Biomechanics-Based Comparison of AlphaPose, BlazePose, and OpenPose

The adoption of computer vision pose estimation approaches, used to identify keypoint locations which are intended to reflect the necessary anatomical landmarks relied upon by biomechanists for musculoskeletal modelling, has gained increasing traction in recent years. This uptake has been further accelerated by keypoint use as inputs into machine learning models used to estimate biomechanical parameters such as ground reaction forces (GRFs) in the absence of instrumentation required for direct measurement. This study first aimed to investigate the keypoint detection rate of three open-source pose estimation models (AlphaPose, BlazePose, and OpenPose) across varying movements, camera views, and trial lengths. Second, this study aimed to assess the suitability and interchangeability of keypoints detected by each pose estimation model when used as inputs into machine learning models for the estimation of GRFs. The keypoint detection rate of BlazePose was distinctly lower than that of AlphaPose and OpenPose. All pose estimation models achieved a high keypoint detection rate at the centre of an image frame and a lower detection rate in the true sagittal plane camera field of view, compared with slightly anteriorly or posteriorly located quasi-sagittal plane camera views. The three-dimensional ground reaction force, instantaneous loading rate, and peak force for running could be estimated using the keypoints of all three pose estimation models. However, only AlphaPose and OpenPose keypoints could be used interchangeably with a machine learning model trained to estimate GRFs based on AlphaPose keypoints resulting in a high estimation accuracy when OpenPose keypoints were used as inputs and vice versa. The findings of this study highlight the need for further evaluation of computer vision-based pose estimation models for application in biomechanical human modelling, and the limitations of machine learning-based GRF estimation models that rely on 2D keypoints. This is of particular relevance given that machine learning models informing athlete monitoring guidelines are being developed for application related to athlete well-being.

xLength: Predicting Expected Ski Jump Length Shortly after Take-Off Using Deep Learning

With tracking systems becoming more widespread in sports research and regular training and competitions, more data are available for sports analytics and performance prediction. We analyzed 2523 ski jumps from 205 athletes on five venues. For every jump, the dataset includes the 3D trajectory, 3D velocity, skis' orientation, and metadata such as wind, starting gate, and ski jumping hill data. Using this dataset, we aimed to predict the expected jump length (xLength) inspired by the expected goals metric in soccer (xG). We evaluate the performance of a fully connected neural network, a convolutional neural network (CNN), a long short-term memory (LSTM), and a ResNet architecture to estimate the xLength. For the prediction of the jump length one second after take-off, we achieve a mean absolute error (MAE) of 5.3 m for the generalization to new athletes and an MAE of 5.9 m for the generalization to new ski jumping hills using ResNet architectures. Additionally, we investigated the influence of the input time after the take-off on the predictions' accuracy. As expected, the MAE becomes smaller with longer inputs. Due to the real-time transmission of the sensor's data, xLength can be updated during the flight phase and used in live TV broadcasting. xLength could also be used as an analysis tool for experts to quantify the quality of the take-off and flight phases.

Unified End-to-End YOLOv5-HR-TCM Framework for Automatic 2D/3D Human Pose Estimation for Real-Time Applications

Three-dimensional human pose estimation is widely applied in sports, robotics, and healthcare. In the past five years, the number of CNN-based studies for 3D human pose estimation has been numerous and has yielded impressive results. However, studies often focus only on improving the accuracy of the estimation results. In this paper, we propose a fast, unified end-to-end model for estimating 3D human pose, called YOLOv5-HR-TCM (YOLOv5-HRet-Temporal Convolution Model). Our proposed model is based on the 2D to 3D lifting approach for 3D human pose estimation while taking care of each step in the estimation process, such as person detection, 2D human pose estimation, and 3D human pose estimation. The proposed model is a combination of best practices at each stage. Our proposed model is evaluated on the Human 3.6M dataset and compared with other methods at each step. The method achieves high accuracy, not sacrificing processing speed. The estimated time of the whole process is 3.146 FPS on a low-end computer. In particular, we propose a sports scoring application based on the deviation angle between the estimated 3D human posture and the standard (reference) origin. The average deviation angle evaluated on the Human 3.6M dataset (Protocol #1-Pro #1) is 8.2 degrees.

Analysis of Competition and Training Videos of Speed Climbing Athletes Using Feature and Human Body Keypoint Detection Algorithms

Compared to 25 years ago, the climbing sport itself has changed dramatically. From a rock climbing modification to a separation in three independent disciplines, the requirements to athletes and trainers increased rapidly. To ensure continuous improvement of the sport itself, the usage of measurement and sensor technology is unavoidable. Especially in the field of the discipline speed climbing, which will be performed as a single discipline at the Olympic Games 2024 in Paris, the current state of the art of movement analysis only consists of video analysis and the benefit of the experience of trainers. Therefore, this paper presents a novel method, which supports trainers and athletes and enables analysis of motion sequences and techniques. Prerecorded video footage is combined with existing feature and human body keypoint detection algorithms and standardized boundary conditions. Therefore, several image processing steps are necessary to convert the recorded movement of different speed climbing athletes to significant parameters for detailed analysis. By studying climbing trials of professional athletes and the used techniques in different sections of the speed climbing wall, the aim among others is to get comparable results and detect mistakes. As a conclusion, the presented method enables powerful analysis of speed climbing training and competition and serves with the aid of a user-friendly designed interface as a support for trainers and athletes for the evaluation of motion sequences.

MEMe: A Mutually Enhanced Modeling Method for Efficient and Effective Human Pose Estimation

In this paper, a mutually enhanced modeling method (MEMe) is presented for human pose estimation, which focuses on enhancing lightweight model performance, but with low complexity. To obtain higher accuracy, a traditional model scale is largely expanded with heavy deployment difficulties. However, for a more lightweight model, there is a large performance gap compared to the former; thus, an urgent need for a way to fill it. Therefore, we propose a MEMe to reconstruct a lightweight baseline model, EffBase transferred intuitively from EfficientDet, into the efficient and effective pose (EEffPose) net, which contains three mutually enhanced modules: the Enhanced EffNet (EEffNet) backbone, the total fusion neck (TFNeck), and the final attention head (FAHead). Extensive experiments on COCO and MPII benchmarks show that our MEMe-based models reach state-of-the-art performances, with limited parameters. Specifically, in the same conditions, our EEffPose-P0 with 256 × 192 can use only 8.98 M parameters to achieve 75.4 AP on the COCO val set, which outperforms HRNet-W48, but with only 14% of its parameters.

Single Camera-Based Remote Physical Therapy: Verification on a Large Video Dataset

In recent years, several systems have been developed to capture human motion in real-time using common RGB cameras. This approach has great potential to become widespread among the general public as it allows the remote evaluation of exercise at no additional cost. The concept of using these systems in rehabilitation in the home environment has been discussed, but no work has addressed the practical problem of detecting basic body parts under different sensing conditions on a large scale. In this study, we evaluate the ability of the OpenPose pose estimation algorithm to perform keypoint detection of anatomical landmarks under different conditions. We infer the quality of detection based on the keypoint confidence values reported by the OpenPose. We used more than two thousand unique exercises for the evaluation. We focus on the influence of the camera view and the influence of the position of the trainees, which are essential in terms of the use for home exercise. Our results show that the position of the trainee has the greatest effect, in the following increasing order of suitability across all camera views: lying position, position on the knees, sitting position, and standing position. On the other hand, the effect of the camera view was only marginal, showing that the side view is having slightly worse results. The results might also indicate that the quality of detection of lower body joints is lower across all conditions than the quality of detection of upper body joints. In this practical overview, we present the possibilities and limitations of current camera-based systems in telerehabilitation.