Vision-Based Pose Estimation for Robot-Mediated Hand Telerehabilitation

Investigating the efficacy of AI-enhanced telerehabilitation in sarcopenic older individuals

OBJECTIVE

This study explores the effectiveness of 3D pose estimation technology in Yi Jin Jing (a traditional Chinese exercise) interventions for sarcopenic older individuals.

DESIGN

A randomized controlled trial involving 93 participants (mean age: 71.64 ± 7.09 years; 41 males and 52 females) divided into three groups: a face-to-face offline traditional training group (OFFG), a general remote online training group (ONG), and an AI-based online remote training group (AIONG).

METHODS

Participants in each group underwent their respective training programs. The effectiveness of the interventions was measured using Appendicular Skeletal Muscle Mass Index, Grip Strength, 6-meter Walking Speed, Timed-Up-and-Go Test, and Quality of Life assessments.

RESULTS

Significant improvements were observed across all groups in ASMI, Grip Strength, 6-meter Walking Speed, TUGT, and QoL. However, there were no statistically significant differences between the groups in terms of the magnitude of these improvements. AIONG showed outcomes comparable to OFFG and ONG methods.

CONCLUSIONS

AI-based telerehabilitation with 3D pose estimation is a viable and effective alternative for remote exercise interventions. It offers precise guidance and enhances the quality of rehabilitation training, demonstrating outcomes comparable to traditional and general online methods.

Patient-Therapist Cooperative Hand Telerehabilitation through a Novel Framework Involving the Virtual Glove System

Telerehabilitation is important for post-stroke or post-surgery rehabilitation because the tasks it uses are reproducible. When combined with assistive technologies, such as robots, virtual reality, tracking systems, or a combination of them, it can also allow the recording of a patient's progression and rehabilitation monitoring, along with an objective evaluation. In this paper, we present the structure, from actors and functionalities to software and hardware views, of a novel framework that allows cooperation between patients and therapists. The system uses a computer-vision-based system named virtual glove for real-time hand tracking (40 fps), which is translated into a light and precise system. The novelty of this work lies in the fact that it gives the therapist quantitative, not only qualitative, information about the hand's mobility, for every hand joint separately, while at the same time providing control of the result of the rehabilitation by also quantitatively monitoring the progress of the hand mobility. Finally, it also offers a strategy for patient-therapist interaction and therapist-therapist data sharing.

A Review of Hand Function Rehabilitation Systems Based on Hand Motion Recognition Devices and Artificial Intelligence

The incidence of stroke and the burden on health care and society are expected to increase significantly in the coming years, due to the increasing aging of the population. Various sensory, motor, cognitive and psychological disorders may remain in the patient after survival from a stroke. In hemiplegic patients with movement disorders, the impairment of upper limb function, especially hand function, dramatically limits the ability of patients to perform activities of daily living (ADL). Therefore, one of the essential goals of post-stroke rehabilitation is to restore hand function. The recovery of motor function is achieved chiefly through compensatory strategies, such as hand rehabilitation robots, which have been available since the end of the last century. This paper reviews the current research status of hand function rehabilitation devices based on various types of hand motion recognition technologies and analyzes their advantages and disadvantages, reviews the application of artificial intelligence in hand rehabilitation robots, and summarizes the current research limitations and discusses future research directions.

Single Camera-Based Remote Physical Therapy: Verification on a Large Video Dataset

In recent years, several systems have been developed to capture human motion in real-time using common RGB cameras. This approach has great potential to become widespread among the general public as it allows the remote evaluation of exercise at no additional cost. The concept of using these systems in rehabilitation in the home environment has been discussed, but no work has addressed the practical problem of detecting basic body parts under different sensing conditions on a large scale. In this study, we evaluate the ability of the OpenPose pose estimation algorithm to perform keypoint detection of anatomical landmarks under different conditions. We infer the quality of detection based on the keypoint confidence values reported by the OpenPose. We used more than two thousand unique exercises for the evaluation. We focus on the influence of the camera view and the influence of the position of the trainees, which are essential in terms of the use for home exercise. Our results show that the position of the trainee has the greatest effect, in the following increasing order of suitability across all camera views: lying position, position on the knees, sitting position, and standing position. On the other hand, the effect of the camera view was only marginal, showing that the side view is having slightly worse results. The results might also indicate that the quality of detection of lower body joints is lower across all conditions than the quality of detection of upper body joints. In this practical overview, we present the possibilities and limitations of current camera-based systems in telerehabilitation.

Applications of Pose Estimation in Human Health and Performance across the Lifespan

The emergence of pose estimation algorithms represents a potential paradigm shift in the study and assessment of human movement. Human pose estimation algorithms leverage advances in computer vision to track human movement automatically from simple videos recorded using common household devices with relatively low-cost cameras (e.g., smartphones, tablets, laptop computers). In our view, these technologies offer clear and exciting potential to make measurement of human movement substantially more accessible; for example, a clinician could perform a quantitative motor assessment directly in a patient's home, a researcher without access to expensive motion capture equipment could analyze movement kinematics using a smartphone video, and a coach could evaluate player performance with video recordings directly from the field. In this review, we combine expertise and perspectives from physical therapy, speech-language pathology, movement science, and engineering to provide insight into applications of pose estimation in human health and performance. We focus specifically on applications in areas of human development, performance optimization, injury prevention, and motor assessment of persons with neurologic damage or disease. We review relevant literature, share interdisciplinary viewpoints on future applications of these technologies to improve human health and performance, and discuss perceived limitations.

Using Biometric Technology for Telehealth and Telerehabilitation

This article discusses the use of physical and biometric sensors in telerehabilitation. It also discusses synchronous tele-physical assessment using haptics and augmented reality and asynchronous physical assessment using remote pose estimation. The article additionally focuses on computational models that have the potential to monitor and evaluate changes in kinematic and kinetic properties during telerehabilitation using biometric sensors such as electromyography and other wearable and noncontact sensors based on force and speed. And finally, the article discusses how virtual reality environments can be facilitated in telerehabilitation.

Disturbance-Estimated Adaptive Backstepping Sliding Mode Control of a Pneumatic Muscles-Driven Ankle Rehabilitation Robot

A rehabilitation robot plays an important role in relieving the therapists' burden and helping patients with ankle injuries to perform more accurate and effective rehabilitation training. However, a majority of current ankle rehabilitation robots are rigid and have drawbacks in terms of complex structure, poor flexibility and lack of safety. Taking advantages of pneumatic muscles' good flexibility and light weight, we developed a novel two degrees of freedom (2-DOF) parallel compliant ankle rehabilitation robot actuated by pneumatic muscles (PMs). To solve the PM's nonlinear characteristics during operation and to tackle the human-robot uncertainties in rehabilitation, an adaptive backstepping sliding mode control (ABS-SMC) method is proposed in this paper. The human-robot external disturbance can be estimated by an observer, who is then used to adjust the robot output to accommodate external changes. The system stability is guaranteed by the Lyapunov stability theorem. Experimental results on the compliant ankle rehabilitation robot show that the proposed ABS-SMC is able to estimate the external disturbance online and adjust the control output in real time during operation, resulting in a higher trajectory tracking accuracy and better response performance especially in dynamic conditions.

Design and Construction of a Bilateral Haptic System for the Remote Assessment of the Stiffness and Range of Motion of the Hand

The use of haptic devices in the rehabilitation of impaired limbs has become rather popular, given the proven effectiveness in promoting recovery. In a standard framework, such devices are used in rehabilitation centers, where patients interact with virtual tasks, presented on a screen. To track their sessions, kinematic/dynamic parameters or performance scores are recorded. However, as Internet access is now available at almost every home and in order to reduce the hospitalization time of the patient, the idea of doing rehabilitation at home is gaining wide consent. Medical care programs can be synchronized with the home rehabilitation device; patient data can be sent to the central server that could redirect to the therapist laptop (tele-healthcare). The controversial issue is that the recorded data do not actually represent the clinical conditions of the patients according to the medical assessment scales, forcing them to frequently undergo clinical tests at the hospital. To respond to this demand, we propose the use of a bilateral master/slave haptic system that could allow the clinician, who interacts with the master, to assess remotely and in real time the clinical conditions of the patient that uses the home rehabilitation device as the slave. In this paper, we describe a proof of concept to highlight the main issues of such an application, limited to one degree of freedom, and to the measure of the stiffness and range of motion of the hand.

Are We Ready to Build a System for Assisting Blind People in Tactile Exploration of Bas-Reliefs?

Nowadays, the creation of methodologies and tools for facilitating the 3D reproduction of artworks and, contextually, to make their exploration possible and more meaningful for blind users is becoming increasingly relevant in society. Accordingly, the creation of integrated systems including both tactile media (e.g., bas-reliefs) and interfaces capable of providing the users with an experience cognitively comparable to the one originally envisioned by the artist, may be considered the next step for enhancing artworks exploration. In light of this, the present work provides a description of a first-attempt system designed to aid blind people (BP) in the tactile exploration of bas-reliefs. In detail, consistent hardware layout, comprising a hand-tracking system based on Kinect^® sensor and an audio device, together with a number of methodologies, algorithms and information related to physical design are proposed. Moreover, according to experimental test on the developed system related to the device position, some design alternatives are suggested so as to discuss pros and cons.