Towards Automating Personal Exercise Assessment and Guidance with Affordable Mobile Technology

Physical activity (PA) offers many benefits for human health. However, beginners often feel discouraged when introduced to basic exercise routines. Due to lack of experience and personal guidance, they might abandon efforts or experience musculoskeletal injuries. Additionally, due to phenomena such as pandemics and limited access to supervised exercise spaces, especially for the elderly, the need to develop personalized systems has become apparent. In this work, we develop a monitored physical exercise system that offers real-time guidance and recommendations during exercise, designed to assist users in their home environment. For this purpose, we used posture estimation interfaces that recognize body movement using a computer or smartphone camera. The chosen pose estimation model was BlazePose. Machine learning and signal processing techniques were used to identify the exercise currently being performed. The performances of three machine learning classifiers were evaluated for the exercise recognition task, achieving test-set accuracy between 94.76% and 100%. The research methodology included kinematic analysis (KA) of five selected exercises and statistical studies on performance and range of motion (ROM), which enabled the identification of deviations from the expected exercise execution to support guidance. To this end, data was collected from 57 volunteers, contributing to a comprehensive understanding of exercise performance. By leveraging the capabilities of the BlazePose model, an interactive tool for patients is proposed that could support rehabilitation programs remotely.

Ergonomic investigations on novel dynamic postural estimator using blaze pose and transfer learning

The aim is to develop a computer-based assessment model for novel dynamic postural evaluation using RULA. The present study proposed a camera-based, three-dimensional (3D) dynamic human pose estimation model using 'BlazePose' with a data set of 50,000 action-level-based images. The model was investigated using the Deep Neural Network (DNN) and Transfer Learning (TL) approach. The model has been trained to evaluate the posture with high accuracy, precision, and recall for each output prediction class. The model can quickly analyse the ergonomics of dynamic posture online and offline with a promising accuracy of 94.12%. A novel dynamic postural estimator using blaze pose and transfer learning is proposed and assessed for accuracy. The model is subjected to a constant muscle loading factor and foot support score that could evaluate one person with good image clarity at a time.Practitioner summary: A detailed investigation of dynamic work postures is largely missing in the literature. Experimental analysis has been performed using transfer learning, BlazePose, and RULA action levels. An overall accuracy of 94.12% is achieved for dynamic postural assessment.

Designing and Developing a Vision-Based System to Investigate the Emotional Effects of News on Short Sleep at Noon: An Experimental Case Study

Background: Sleep is a critical factor in maintaining good health, and its impact on various diseases has been recognized by scientists. Understanding sleep patterns and quality is crucial for investigating sleep-related disorders and their potential links to health conditions. The development of non-intrusive and contactless methods for analyzing sleep data is essential for accurate diagnosis and treatment. Methods: A novel system called the sleep visual analyzer (VSleep) was designed to analyze sleep movements and generate reports based on changes in body position angles. The system utilized camera data without requiring any physical contact with the body. A Python graphical user interface (GUI) section was developed to analyze body movements during sleep and present the data in an Excel format. To evaluate the effectiveness of the VSleep system, a case study was conducted. The participants' movements during daytime naps were recorded. The study also examined the impact of different types of news (positive, neutral, and negative) on sleep patterns. Results: The system successfully detected and recorded various angles formed by participants' bodies, providing detailed information about their sleep patterns. The results revealed distinct effects based on the news category, highlighting the potential impact of external factors on sleep quality and behaviors. Conclusions: The sleep visual analyzer (VSleep) demonstrated its efficacy in analyzing sleep-related data without the need for accessories. The VSleep system holds great potential for diagnosing and investigating sleep-related disorders. The proposed system is affordable, easy to use, portable, and a mobile application can be developed to perform the experiment and prepare the results.

A novel approach for yoga pose estimation based on in-depth analysis of human body joint detection accuracy

Virtual motion and pose from images and video can be estimated by detecting body joints and their interconnection. The human body has diverse and complicated poses in yoga, making its classification challenging. This study estimates yoga poses from the images using a neural network. Five different yoga poses, viz. downdog, tree, plank, warrior2, and goddess in the form of RGB images are used as the target inputs. The BlazePose model was used to localize the body joints of the yoga poses. It detected a maximum of 33 body joints, referred to as keypoints, covering almost all the body parts. Keypoints achieved from the model are considered as predicted joint locations. True keypoints, as the ground truth body joint for individual yoga poses, are identified manually using the open source image annotation tool named Makesense AI. A detailed analysis of the body joint detection accuracy is proposed in the form of percentage of corrected keypoints (PCK) and percentage of detected joints (PDJ) for individual body parts and individual body joints, respectively. An algorithm is designed to measure PCK and PDJ in which the distance between the predicted joint location and true joint location is calculated. The experiment evaluation suggests that the adopted model obtained 93.9% PCK for the goddess pose. The maximum PCK achieved for the goddess pose, i.e., 93.9%, PDJ evaluation was carried out in the staggering mode where maximum PDJ is obtained as 90% to 100% for almost all the body joints.

Internet-of-Things-Enabled Markerless Running Gait Assessment from a Single Smartphone Camera

Running gait assessment is essential for the development of technical optimization strategies as well as to inform injury prevention and rehabilitation. Currently, running gait assessment relies on (i) visual assessment, exhibiting subjectivity and limited reliability, or (ii) use of instrumented approaches, which often carry high costs and can be intrusive due to the attachment of equipment to the body. Here, the use of an IoT-enabled markerless computer vision smartphone application based upon Google’s pose estimation model BlazePose was evaluated for running gait assessment for use in low-resource settings. That human pose estimation architecture was used to extract contact time, swing time, step time, knee flexion angle, and foot strike location from a large cohort of runners. The gold-standard Vicon 3D motion capture system was used as a reference. The proposed approach performs robustly, demonstrating good (ICC(2,1) > 0.75) to excellent (ICC(2,1) > 0.90) agreement in all running gait outcomes. Additionally, temporal outcomes exhibit low mean error (0.01−0.014 s) in left foot outcomes. However, there are some discrepancies in right foot outcomes, due to occlusion. This study demonstrates that the proposed low-cost and markerless system provides accurate running gait assessment outcomes. The approach may help routine running gait assessment in low-resource environments.

Fall Detection for Shipboard Seafarers Based on Optimized BlazePose and LSTM

Aiming to avoid personal injury caused by the failure of timely medical assistance following a fall by seafarer members working on ships, research on the detection of seafarer's falls and timely warnings to safety officers can reduce the loss and severe consequences of falls to seafarers. To improve the detection accuracy and real-time performance of the seafarer fall detection algorithm, a seafarer fall detection algorithm based on BlazePose-LSTM is proposed. This algorithm can automatically extract the human body key point information from the video image obtained by the vision sensor, analyze its internal data correlation characteristics, and realize the process from RGB camera image processing to seafarer fall detection. This fall detection algorithm extracts the human body key point information through the optimized BlazePose human body key point information extraction network. In this section, a new method for human bounding-box acquisition is proposed. In this study, a head detector based on the Vitruvian theory was used to replace the pre-trained SSD body detector in the BlazePose preheating module. Simultaneously, an offset vector is proposed to update the bounding box obtained. This method can reduce the frequency of repeated use of the head detection module. The algorithm then uses the long short-term memory neural network to detect seafarer falls. After extracting fall and related behavior data from the URFall public data set and FDD public data set to enrich the self-made data set, the experimental results show that the algorithm can achieve 100% accuracy and 98.5% specificity for the seafarer's falling behavior, indicating that the algorithm has reasonable practicability and strong generalization ability. The detection frame rate can reach 29 fps on a CPU, which can meet the effect of real-time detection. The proposed method can be deployed on common vision sensors.