The Potential of Computer Vision-Based Marker-Less Human Motion Analysis for Rehabilitation

Breaking the ice through an effective translationality in neurorehabilitation: are we heading to the right direction?

INTRODUCTION

Translational medicine has been facing a persistent crisis for decades, and the field of neurorehabilitation is no exception. The challenges and delays that prevent patients, caregivers, and clinicians from promptly benefiting from advancements in bioengineering and new technological discoveries are well-documented.

AREAS-COVERED

This perspective presents some ideas to underline the consolidated problems and highlight new obstacles to overcome in the context of translational neurorehabilitation, also considering the increasingly stringent laws for medical devices that are emerging throughout the world.

EXPERT OPINION

The objective of the entire medical-scientific community must be to ensure that patients and their loved ones receive the best care available with the most advanced systems. Bioengineers, healthcare policy makers, certifiers and clinicians must always take translational aspects into consideration and find solutions to mitigate possible problems and delays. The goal of the entire medical and scientific community should be to ensure that patients and their families receive the highest quality care through the most advanced systems. To achieve this, bioengineers, healthcare policymakers, certifiers, and clinicians must consistently address translational challenges and work collaboratively to find solutions that minimize potential problems and delays.

Proposal and validation of a new approach in tele-rehabilitation with 3D human posture estimation: a randomized controlled trial in older individuals with sarcopenia

OBJECTIVE

Through a randomized controlled trial on older adults with sarcopenia, this study compared the training effects of an AI-based remote training group using deep learning-based 3D human pose estimation technology with those of a face-to-face traditional training group and a general remote training group.

METHODS

Seventy five older adults with sarcopenia aged 60-75 from community organizations in Changchun city were randomly divided into a face-to-face traditional training group (TRHG), a general remote training group (GTHG), and an AI-based remote training group (AITHG). All groups underwent a 3-month program consisting of 24-form Taichi exercises, with a frequency of 3 sessions per week and each session lasting 40 min. The participants underwent Appendicular Skeletal Muscle Mass Index (ASMI), grip strength, 6-meter walking pace, Timed Up and Go test (TUGT), and quality of life score (QoL) tests before the experiment, during the mid-term, and after the experiment. This study used SPSS26.0 software to perform one-way ANOVA and repeated measures ANOVA tests to compare the differences among the three groups. A significance level of p < 0.05 was defined as having significant difference, while p < 0.01 was defined as having a highly significant difference.

RESULTS

(1) The comparison between the mid-term and pre-term indicators showed that TRHG experienced significant improvements in ASMI, 6-meter walking pace, and QoL (p < 0.01), and a significant improvement in TUGT timing test (p < 0.05); GTHG experienced extremely significant improvements in 6-meter walking pace and QoL (p < 0.01); AITHG experienced extremely significant improvements in ASMI, 6-meter walking pace, and QoL (p < 0.01), and a significant improvement in TUGT timing test (p < 0.05). (2) The comparison between the post-term and pre-term indicators showed that TRHG experienced extremely significant improvements in TUGT timing test (p < 0.01); GTHG experienced significant improvements in ASMI and TUGT timing test (p < 0.05); and AITHG experienced extremely significant improvements in TUGT timing test (p < 0.01). (3) During the mid-term, there was no significant difference among the groups in all tests (p > 0.05). The same was in post-term tests (p > 0.05).

CONCLUSION

Compared to the pre-experiment, there was no significant difference at the post- experiment in the recovery effects on the muscle quality, physical activity ability, and life quality of patients with sarcopenia between the AI-based remote training group and the face-to-face traditional training group. 3D pose estimation is equally as effective as traditional rehabilitation methods in enhancing muscle quality, functionality and life quality in older adults with sarcopenia.

TRIAL REGISTRATION

The trial was registered in ClinicalTrials.gov (NCT05767710).

Empowering sickle cell disease care: the rise of TechnoRehabLab in Sub-Saharan Africa for enhanced patient's perspectives

Sickle-cell Disease (SCD) is a major public health problem in Africa, and there are significant obstacles to its comprehensive management, particularly in terms of access to appropriate healthcare. This calls for inventive approaches to improve patients' prospects. Among the major challenges to be met are the primary and secondary prevention of certain serious complications associated with the disease, such as neurocognitive, motor and respiratory functional disorders. This perspective argues for the rapid creation of specific, cost-effective, technology-supported rehabilitation centres to advance SCD care, identify patients at high risk of stroke and implement tailored rehabilitation strategies. The TechnoRehabLab in Lubumbashi illustrates this shift in thinking by using cutting-edge technologies such as virtual reality (VR), serious games and mobile health to create a comprehensive and easily accessible rehabilitation framework. Diagnostic tools used to perform functional assessment can be used to identify cognitive, balance and walking deficits respectively. Transcranial Doppler enables early detection of sickle cell cerebral vasculopathy, making it possible to provide early and appropriate treatment. VR technology and serious games enable effective rehabilitation and cognitive stimulation, which is particularly advantageous for remote or community-based rehabilitation. In the context of African countries where there is a glaring disparity in access to digital resources, the TechnoRehabLab serves as a tangible example, demonstrating the flexibility and accessibility of technology-assisted rehabilitation. This perspective is an urgent call to governments, non-governmental organisations and the international community to allocate resources to the replication and expansion of similar facilities across Africa.

Evaluation of the Rennes Universal Measurement Method (RUMM), an artificial intelligence application for hand joint angle assessment

Although goniometric measurement is considered the gold standard for the measurement of digital range of motion, visual estimation is often employed due to its simplicity despite being inconsistent with recommended guidelines. We evaluated the Rennes Universal Measurement Method, an innovative tool employing artificial intelligence to concurrently analyse hand joint angles based on a single photograph. We found a strong correlation between the goniometric method and the photograph-based approach (Spearman correlation coefficient 0.7). The mean standard error of measurement was -1° (SD 17°). Regarding reproducibility with different photographic angles, an excellent intraclass correlation coefficient of 0.9 was noted. The tool had a processing time of less than 0.1 s per hand, while traditional goniometric methods took 20-30 s per finger. Combining simplicity, high reproducibility and good inter-rater reliability, this is a potentially useful tool that can be used to monitor patient progress in place of traditional goniometry.

A systematic literature review of computer vision-based biomechanical models for physical workload estimation

Ergonomic risks, driven by strenuous physical demands in complex work settings, are prevalent across industries. Addressing these challenges through detailed assessment and effective interventions enhances safety and employee well-being. Proper and timely measurement of physical workloads is the initial step towards holistic ergonomic control. This study comprehensively explores existing computer vision-based biomechanical analysis methods for workload assessment, assessing their performance against traditional techniques, and categorising them for easier use. Recent strides in artificial intelligence have revolutionised workload assessment, especially in realistic work settings where conventional methods fall short. However, understanding the accuracy, characteristics, and practicality of computer vision-based methods versus traditional approaches remains limited. To bridge this knowledge gap, a literature review along with a meta-analysis was completed in this study to illuminate model accuracy, advantages, and challenges, offering valuable insights for refined technology implementation in diverse work environments.

Effective evaluation of HGcnMLP method for markerless 3D pose estimation of musculoskeletal diseases patients based on smartphone monocular video

Markerless pose estimation based on computer vision provides a simpler and cheaper alternative to human motion capture, with great potential for clinical diagnosis and remote rehabilitation assessment. Currently, the markerless 3D pose estimation is mainly based on multi-view technology, while the more promising single-view technology has defects such as low accuracy and reliability, which seriously limits clinical application. This study proposes a high-resolution graph convolutional multilayer perception (HGcnMLP) human 3D pose estimation framework for smartphone monocular videos and estimates 15 healthy adults and 12 patients with musculoskeletal disorders (sarcopenia and osteoarthritis) gait spatiotemporal, knee angle, and center-of-mass (COM) velocity parameters, etc., and compared with the VICON gold standard system. The results show that most of the calculated parameters have excellent reliability (VICON, ICC (2, k): 0.853-0.982; Phone, ICC (2, k): 0.839-0.975) and validity (Pearson r: 0.808-0.978, p< 0.05). In addition, the proposed system can better evaluate human gait balance ability, and the K-means++ clustering algorithm can successfully distinguish patients into different recovery level groups. This study verifies the potential of a single smartphone video for 3D human pose estimation for rehabilitation auxiliary diagnosis and balance level recognition, and is an effective attempt at the clinical application of emerging computer vision technology. In the future, it is hoped that the corresponding smartphone program will be developed to provide a low-cost, effective, and simple new tool for remote monitoring and rehabilitation assessment of patients.

Feasibility of 3D Body Tracking from Monocular 2D Video Feeds in Musculoskeletal Telerehabilitation

Musculoskeletal conditions affect millions of people globally; however, conventional treatments pose challenges concerning price, accessibility, and convenience. Many telerehabilitation solutions offer an engaging alternative but rely on complex hardware for body tracking. This work explores the feasibility of a model for 3D Human Pose Estimation (HPE) from monocular 2D videos (MediaPipe Pose) in a physiotherapy context, by comparing its performance to ground truth measurements. MediaPipe Pose was investigated in eight exercises typically performed in musculoskeletal physiotherapy sessions, where the Range of Motion (ROM) of the human joints was the evaluated parameter. This model showed the best performance for shoulder abduction, shoulder press, elbow flexion, and squat exercises. Results have shown a MAPE ranging between 14.9% and 25.0%, Pearson's coefficient ranging between 0.963 and 0.996, and cosine similarity ranging between 0.987 and 0.999. Some exercises (e.g., seated knee extension and shoulder flexion) posed challenges due to unusual poses, occlusions, and depth ambiguities, possibly related to a lack of training data. This study demonstrates the potential of HPE from monocular 2D videos, as a markerless, affordable, and accessible solution for musculoskeletal telerehabilitation approaches. Future work should focus on exploring variations of the 3D HPE models trained on physiotherapy-related datasets, such as the Fit3D dataset, and post-preprocessing techniques to enhance the model's performance.

Effectiveness of a digital rehabilitation program based on computer vision and augmented reality for isolated meniscus injury: protocol for a prospective randomized controlled trial

BACKGROUND

The lack of access to physical therapists in developing countries and rural areas poses a significant challenge in supervising postsurgical rehabilitation, potentially impeding desirable outcomes following surgical interventions. For this reason, this study aims to evaluate the feasibility, safety, and effectiveness of utilizing a digital rehabilitation program based on computer vision and augmented reality in comparison with traditional care for patients who will undergo isolated meniscus repair, since to date, there is no literature on this topic.

METHODS

This study intends to enroll two groups of participants, each to be provided with informed consent before undergoing randomization into either the experimental or control group. The experimental group will undergo a digital rehabilitation program utilizing computer vision and augmented reality (AR) technology following their surgical procedure, while the control group will receive conventional care, involving in-clinic physical therapy sessions weekly. Both groups will adhere to a standardized rehabilitation protocol over a six-month duration. Follow-up assessments will be conducted at various intervals, including preoperatively, and at 2 weeks, 6 weeks, 12 weeks, and 24 weeks postoperatively. Imaging assessments and return-to-play evaluations will be conducted during the final follow-up. Clinical functionality will be assessed based on improvements in International Knee Documentation Committee (IKDC) and Visual Analog Scale (VAS) scores.

REGISTRATION NUMBER

ChiCTR2300070582.

A deep learning system to monitor and assess rehabilitation exercises in home-based remote and unsupervised conditions

In the domain of physical rehabilitation, the progress in machine learning and the availability of cost-effective motion capture technologies have paved the way for innovative systems capable of capturing human movements, automatically analyzing recorded data, and evaluating movement quality. This study introduces a novel, economically viable system designed for monitoring and assessing rehabilitation exercises. The system enables real-time evaluation of exercises, providing precise insights into deviations from correct execution. The evaluation comprises two significant components: range of motion (ROM) classification and compensatory pattern recognition. To develop and validate the effectiveness of the system, a unique dataset of 6 resistance training exercises was acquired. The proposed system demonstrated impressive capabilities in motion monitoring and evaluation. Notably, we achieved promising results, with mean accuracies of 89% for evaluating ROM-class and 98% for classifying compensatory patterns. By complementing conventional rehabilitation assessments conducted by skilled clinicians, this cutting-edge system has the potential to significantly improve rehabilitation practices. Additionally, its integration in home-based rehabilitation programs can greatly enhance patient outcomes and increase access to high-quality care.

Recognizing hand use and hand role at home after stroke from egocentric video

Hand function is a central determinant of independence after stroke. Measuring hand use in the home environment is necessary to evaluate the impact of new interventions, and calls for novel wearable technologies. Egocentric video can capture hand-object interactions in context, as well as show how more-affected hands are used during bilateral tasks (for stabilization or manipulation). Automated methods are required to extract this information. The objective of this study was to use artificial intelligence-based computer vision to classify hand use and hand role from egocentric videos recorded at home after stroke. Twenty-one stroke survivors participated in the study. A random forest classifier, a SlowFast neural network, and the Hand Object Detector neural network were applied to identify hand use and hand role at home. Leave-One-Subject-Out-Cross-Validation (LOSOCV) was used to evaluate the performance of the three models. Between-group differences of the models were calculated based on the Mathews correlation coefficient (MCC). For hand use detection, the Hand Object Detector had significantly higher performance than the other models. The macro average MCCs using this model in the LOSOCV were 0.50 ± 0.23 for the more-affected hands and 0.58 ± 0.18 for the less-affected hands. Hand role classification had macro average MCCs in the LOSOCV that were close to zero for all models. Using egocentric video to capture the hand use of stroke survivors at home is technically feasible. Pose estimation to track finger movements may be beneficial to classifying hand roles in the future.

Comparison of a single-view image-based system to a multi-camera marker-based system for human static pose estimation

The purpose of this study was to compare human static pose estimation data measured with a single-view image-based system and a multi-camera marker-based system. Thirty participants (20 male/10 female, mean ± standard deviation 29.1 ± 10.0 years old, 1.75 ± 0.10 m tall, 79.1 ± 18.0 kg) performed six repetitions each of static holds of arm-raises and squats, in a different orientation for each repetition. These trials were captured simultaneously with a 120-Hz 12-camera marker-based system and a variable-frequency single-view image-based system. Data for each trial were time-synchronized between the two systems using a near-infrared LED-light that was visible to both systems. Discrete measurements of bilateral shoulder angles during arm-raises and bilateral knee angles during squats were compared between the systems using Bland-Altman plots and descriptive statistics. Pearson correlation coefficients were calculated, comparing the participant trial mean values across systems. Finally, a two-way ANOVA was used to examine whether participant orientation in the capture volume significantly affected either system. Biases for discrete measurements ranged in magnitude from 1.3 to 1.9°, and standard deviations of the differences between systems ranged from 2.4 to 4.7°. Pearson correlation coefficients were all above 0.97, and the ANOVA was unable to find a statistically significant orientation effect for either system. Thus, the marker-based and image-based systems produced similar measurements of static shoulder and knee angles. Future work should examine more complex measurements using volumetric scan-based models and also investigate the ability of single-view image-based systems to measure dynamic movements.

Wearable rehabilitation wristband for distal radius fractures

Background

Distal radius fractures are a common type of fracture. For patients treated with closed reduction with splinting, a period of rehabilitation is still required after the removal of the splint. However, there is a general lack of attention and low compliance to rehabilitation training during this period, so it is necessary to build a rehabilitation training monitoring system to improve the efficiency of patients' rehabilitation.

Methods

A wearable rehabilitation training wristband was proposed, which could be used in the patient's daily rehabilitation training scenario and could recognize four common wrist rehabilitation actions in real-time by using three thin film pressure sensors to detect the pressure change curve at three points on the wrist. An algorithmic framework for classifying rehabilitation training actions was proposed. In our framework, an action pre-detection strategy was designed to exclude false detections caused by switching initial gestures during rehabilitation training and wait for the arrival of the complete signal. To classify the action signals into four categories, firstly an autoencoder was used to downscale the original signal. Six SVMs were then used for evaluation and voting, and the final action with the highest number of votes would be used as the prediction result.

Results

Experimental results showed that the proposed algorithmic framework achieved an average recognition accuracy of 89.62%, an average recognition recall of 88.93%, and an f1 score of 89.27% on the four rehabilitation training actions.

Conclusion

The developed device has the advantages of being small size and easy to wear, which can quickly and accurately identify and classify four common rehabilitation training actions. It can easily be combined with peripheral devices and technologies (e.g., cell phones, computers, Internet) to build different rehabilitation training scenarios, making it worthwhile to use and promote in clinical settings.

Rehabilitation for Musculoskeletal Disorders: The Emergence of Serious Games and the Promise of Personalized Versions Using Artificial Intelligence

According to the World Health Organization (WHO), musculoskeletal conditions are among the most common health problems, affecting approximately 1 [...].

Types of anomalies in two-dimensional video-based gait analysis in uncontrolled environments

Two-dimensional video-based pose estimation is a technique that can be used to estimate human skeletal coordinates from video data alone. It is also being applied to gait analysis and in particularly, due to its simplicity of measurement, it has the potential to be applied to gait analysis of large populations. However, it is considered difficult to completely homogenize the environment and settings during the measurement of large populations. Therefore, it is necessary to appropriately deal with technical errors that are not related to the biological factors of interest. In this study, by analyzing a large cohort database, we have identified four major types of anomalies that occur during gait analysis using OpenPose in uncontrolled environments: anatomical, biomechanical, and physical anomalies and errors due to estimation. We have also developed a workflow for identifying and correcting these anomalies and confirmed that this workflow is reproducible through simulation experiments. Our results will help obtain a comprehensive understanding of the anomalies to be addressed during pre-processing for 2D video-based gait analysis of large populations.

Validity and feasibility of remote measurement systems for functional movement and posture assessments in people with axial spondylarthritis

INTRODUCTION

This study aimed to estimate the criterion validity of functional movement and posture measurement using remote technology systems in people with and without Axial spondylarthritis (axSpA).

METHODS

Validity and agreement of the remote-technology measurement of functional movement and posture were tested cross-sectionally and compared to a standard clinical measurement by a physiotherapist. The feasibility of remote implementation was tested in a home environment. There were two cohorts of participants: people with axSpA and people without longstanding back pain. In addition, a cost-consequence analysis was performed.

RESULTS

Sixty-two participants (31 with axSPA, 53% female, age = 45(SD14), BMI = 26.6(SD4.6) completed the study. In the axSpA group, cervical rotation, lumbar flexion, lumbar side flexion, shoulder flexion, hip abduction, tragus-to-wall and thoracic kyphosis showed a significant moderate to strong correlation; in the non-back pain group, the same measures showed significant correlation ranging from weak to strong.

CONCLUSIONS

Although not valid for clinical use in its current form, the remote technologies demonstrated moderate to strong correlation and agreement in most functional and postural tests measured in people with AxSA. Testing the CV-aided system in a home environment suggests it is a safe and feasible method. Yet, validity testing in this environment still needs to be performed.

Posture Estimation Using Surface Electromyography during Wheelchair Hand-Rim Operations

This study examined competitive wheelchairs that facilitate sports participation. They can be moved straight ahead using only one arm. Our designed and developed competitive wheel-chairs have a dual hand-rim system. Their two hand-rims, attached to a drive wheel on one side, can be operated simultaneously for straight-ahead movement. Specifically, based on integrated electromyography (iEMG) data calculated from surface electromyography (sEMG), we examined the wheelchair loading characteristics, posture estimation, and effects on body posture during one-arm propulsion movement. The first experiment yielded insights into arm and shoulder-joint muscle activation from iEMG results obtained for two-hand propulsion and dual hand-rim system propulsion. Results suggest that muscle activation of one arm can produce equal propulsive force to that produced by two arms. The second experiment estimated the movement posture from iEMG during one-arm wheelchair propulsion. The external oblique abdominis is particularly important for one-arm wheelchair propulsion. The iEMG posture estimation validity was verified based on changes in the user body axis and seat pressure distribution. In conclusion, as confirmed by iEMG, which is useful to estimate posture during movement, one-arm wheelchair use requires different muscle activation sites and posture than when using two arms.