Upper limb posture estimation in robotic and virtual reality-based rehabilitation

Robotic systems for upper-limb rehabilitation in multiple sclerosis: a SWOT analysis and the synergies with virtual and augmented environments

The robotics discipline is exploring precise and versatile solutions for upper-limb rehabilitation in Multiple Sclerosis (MS). People with MS can greatly benefit from robotic systems to help combat the complexities of this disease, which can impair the ability to perform activities of daily living (ADLs). In order to present the potential and the limitations of smart mechatronic devices in the mentioned clinical domain, this review is structured to propose a concise SWOT (Strengths, Weaknesses, Opportunities, and Threats) Analysis of robotic rehabilitation in MS. Through the SWOT Analysis, a method mostly adopted in business management, this paper addresses both internal and external factors that can promote or hinder the adoption of upper-limb rehabilitation robots in MS. Subsequently, it discusses how the synergy with another category of interaction technologies - the systems underlying virtual and augmented environments - may empower Strengths, overcome Weaknesses, expand Opportunities, and handle Threats in rehabilitation robotics for MS. The impactful adaptability of these digital settings (extensively used in rehabilitation for MS, even to approach ADL-like tasks in safe simulated contexts) is the main reason for presenting this approach to face the critical issues of the aforementioned SWOT Analysis. This methodological proposal aims at paving the way for devising further synergistic strategies based on the integration of medical robotic devices with other promising technologies to help upper-limb functional recovery in MS.

Beyond Human Detection: A Benchmark for Detecting Common Human Posture

Human detection is the task of locating all instances of human beings present in an image, which has a wide range of applications across various fields, including search and rescue, surveillance, and autonomous driving. The rapid advancement of computer vision and deep learning technologies has brought significant improvements in human detection. However, for more advanced applications like healthcare, human-computer interaction, and scene understanding, it is crucial to obtain information beyond just the localization of humans. These applications require a deeper understanding of human behavior and state to enable effective and safe interactions with humans and the environment. This study presents a comprehensive benchmark, the Common Human Postures (CHP) dataset, aimed at promoting a more informative and more encouraging task beyond mere human detection. The benchmark dataset comprises a diverse collection of images, featuring individuals in different environments, clothing, and occlusions, performing a wide range of postures and activities. The benchmark aims to enhance research in this challenging task by designing novel and precise methods specifically for it. The CHP dataset consists of 5250 human images collected from different scenes, annotated with bounding boxes for seven common human poses. Using this well-annotated dataset, we have developed two baseline detectors, namely CHP-YOLOF and CHP-YOLOX, building upon two identity-preserved human posture detectors: IPH-YOLOF and IPH-YOLOX. We evaluate the performance of these baseline detectors through extensive experiments. The results demonstrate that these baseline detectors effectively detect human postures on the CHP dataset. By releasing the CHP dataset, we aim to facilitate further research on human pose estimation and to attract more researchers to focus on this challenging task.

Identification of inverse kinematic parameters in redundant systems: Towards quantification of inter-joint coordination in the human upper extremity

Understanding and quantifying inter-joint coordination is valuable in several domains such as neurorehabilitation, robot-assisted therapy, robotic prosthetic arms, and control of supernumerary arms. Inter-joint coordination is often understood as a consistent spatiotemporal relation among kinematically redundant joints performing functional and goal-oriented movements. However, most approaches in the literature to investigate inter-joint coordination are limited to analysis of the end-point trajectory or correlation analysis of the joint rotations without considering the underlying task; e.g., creating a desirable hand movement toward a goal as in reaching motions. This work goes beyond this limitation by taking a model-based approach to quantifying inter-joint coordination. More specifically, we use the weighted pseudo-inverse of the Jacobian matrix and its associated null-space to explain the human kinematics in reaching tasks. We propose a novel algorithm to estimate such Inverse Kinematics weights from observed kinematic data. These estimated weights serve as a quantification for spatial inter-joint coordination; i.e., how costly a redundant joint is in its contribution to creating an end-effector velocity. We apply our estimation algorithm to datasets obtained from two different experiments. In the first experiment, the estimated Inverse Kinematics weights pinpoint how individuals change their Inverse Kinematics strategy when exposed to the viscous field wearing an exoskeleton. The second experiment shows how the resulting Inverse Kinematics weights can quantify a robotic prosthetic arm's contribution (or the level of assistance).

The Effects of Sit-to-Stand Training Combined with Real-Time Visual Feedback on Strength, Balance, Gait Ability, and Quality of Life in Patients with Stroke: A Randomized Controlled Trial

This study aimed to investigate the effects of lower limbs muscles' strength, balance, walking, and quality of life through sit-to-stand training combined with real-time visual feedback (RVF-STS group) in patients with stroke and to compare the effects of classic sit-to-stand training (C-STS group). Thirty patients with stroke were randomly divided into two groups. The RVF-STS group received sit-to-stand training combined with real-time visual feedback using a Wii Balance Board (n = 15), and the C-STS group received classic sit-to-stand training (n = 15). All participants received training for 20 min once a day, 5 days a week for 6 weeks, and both groups underwent general physical therapy for 30 min before training. Before and after the training, the muscle strength of the hip flexor, abductor, and knee extensor were measured, and the Wii Balance Board was used to perform the center of pressure test and Berg Balance Scale to evaluate static and dynamic balance. Additionally, the 10 m walking test and the Timed Up and Go test were performed to evaluate gait function. The Stroke-Specific Quality of Life was used to measure the quality of life. The results showed that the lower extremity muscle strength, balance ability, walking ability, and quality of life of the RVF-STS group significantly improved in comparison of the pre- and post-differences (p < 0.05), and it also showed significant differences between groups (p < 0.05). This study showed that sit-to-stand training combined with real-time visual feedback was effective at improving the muscle strength of the lower extremities, balance, gait, and quality of life in patients with stroke. Therefore, repeating sit-to-stand training combined with real-time visual feedback could be used as an effective treatment method for patients with stroke.

A self-aligning end-effector robot for individual joint training of the human arm

Aim

Intense training of arm movements using robotic devices can help reduce impairments in stroke. Recent evidence indicates that independent training of individual joints of the arm with robots can be as effective as coordinated multi-joint arm training. This makes a case for designing and developing robots made for training individual joints, which can be simpler and more compact than the ones for coordinate multi-joint arm training. The design of such a robot is the aim of the work presented in this paper.

Methods

An end-effector robot kinematic design was developed and the optimal robot link lengths were estimated using an optimization procedure. A simple algorithm for automatically detecting human limb parameters is proposed and its performance was evaluated through a simulation study.

Results

A six-degrees-of-freedom end-effector robot with three actuated degrees-of-freedom and three non-actuated self-aligning degrees-of-freedom for safe assisted training of the individual joints (shoulder or elbow) of the human arm was conceived. The proposed robot has relaxed constraints on the relative positioning of the human limb with respect to the robot. The optimized link lengths chosen for the robot allow it to cover about 80% of the human limb's workspace, and possess good overall manipulability. The simple estimation procedure was demonstrated to estimate human limb parameters with low bias and variance.

Discussion

The proposed robot with three actuated and three non-actuated degrees-of-freedom has a compact structure suitable for both the left and right arms without any change to its structure. The proposed automatic estimation procedure allows the robot to safely apply forces and impose movements to the human limb, without the need for any manual measurements. Such compact robots have the highest potential for clinical translation.

Serious Gaming Technology in Upper Extremity Rehabilitation: Scoping Review

Background

Serious gaming has increasingly gained attention as a potential new component in clinical practice. Specifically, its use in the rehabilitation of motor dysfunctions has been intensively researched during the past three decades.

Objective

The aim of this scoping review was to evaluate the current role of serious games in upper extremity rehabilitation, and to identify common methods and practice as well as technology patterns. This objective was approached via the exploration of published research efforts over time.

Methods

The literature search, using the PubMed and Scopus databases, included articles published from 1999 to 2019. The eligibility criteria were (i) any form of game-based arm rehabilitation; (ii) published in a peer-reviewed journal or conference; (iii) introduce a game in an electronic format; (iv) published in English; and (v) not a review, meta-analysis, or conference abstract. The search strategy identified 169 relevant articles.

Results

The results indicated an increasing research trend in the domain of serious gaming deployment in upper extremity rehabilitation. Furthermore, differences regarding the number of publications and the game approach were noted between studies that used commercial devices in their rehabilitation systems and those that proposed a custom-made robotic arm, glove, or other devices for the connection and interaction with the game platform. A particularly relevant observation concerns the evaluation of the introduced systems. Although one-third of the studies evaluated their implementations with patients, in most cases, there is the need for a larger number of participants and better testing of the rehabilitation scheme efficiency over time. Most of the studies that included some form of assessment for the introduced rehabilitation game mentioned user experience as one of the factors considered for evaluation of the system. Besides user experience assessment, the most common evaluation method involving patients was the use of standard medical tests. Finally, a few studies attempted to extract game features to introduce quantitative measurements for the evaluation of patient improvement.

Conclusions

This paper presents an overview of a significant research topic and highlights the current state of the field. Despite extensive attempts for the development of gamified rehabilitation systems, there is no definite answer as to whether a serious game is a favorable means for upper extremity functionality improvement; however, this certainly constitutes a supplementary means for motivation. The development of a unified performance quantification framework and more extensive experiments could generate richer evidence and contribute toward this direction.

Speed-Interactive Pedaling Training Using Smartphone Virtual Reality Application for Stroke Patients: Single-Blinded, Randomized Clinical Trial

This study aimed to investigate the effects of speed-interactive pedaling training (SIPT) using a smartphone virtual reality application to improve lower limb motor function, trunk sitting balance, and gait in stroke patients. Forty-two patients who had previously experienced a stroke and could sit independently participated in the study. The subjects were assigned to the SIPT group (n = 21) and the control group (n = 21). The SIPT group had cycle training with SIPT for 40 min a day, five days a week, in a six-week period, in addition to conventional therapy. The control group had cycle training without SIPT and conventional therapy. The Fugl–Meyer Assessment, postural sway, modified functional reach test, trunk impairment scale, and spatiotemporal parameters of gait were used to assess the changes in lower extremity function, the static balance of sitting, the dynamic balance of sitting, and gait ability after the intervention. The Fugl–Meyer Assessment, postural sway, modified functional reach test, trunk impairment scale, and gait ability in the SIPT group were significantly better compared to that of the control group (p < 0.05). Based on this result, we propose that SIPT, which improves function, balance, and gait, could be used as an effective training method to improve patients’ functional activities in the clinical setting. The results of this study suggest that SIPT could be used as an effective training method to restore a patient’s function by improving trunk balance and motor function.

Evaluating the effect and mechanism of upper limb motor function recovery induced by immersive virtual-reality-based rehabilitation for subacute stroke subjects: study protocol for a randomized controlled trial

BACKGROUND

There is compelling evidence of beneficial effects of non-immersive virtual reality (VR)-based intervention in the rehabilitation of patients with stroke, whereby patients experience both the real world and the virtual environment. However, to date, research on immersive VR-based rehabilitation is minimal. This study aims to design a randomized controlled trial to assess the effectiveness of immersive VR-based upper extremity rehabilitation in patients with subacute stroke and explore the underlying brain mechanisms of immersive VR-based rehabilitation.

METHODS

Subjects (n = 60) with subacute stroke (defined as more than 1 week and less than 12 weeks after stroke onset) will be recruited to participate in a single-blinded, randomized controlled trial. Subjects will be randomized 1:1 to either (1) an experimental intervention group, or (2) a conventional group (control). Over a 3-week time period immediately following baseline assessments and randomization, subjects in the experimental group will receive both immersive VR and conventional rehabilitation, while those in the control group will receive conventional rehabilitation only. During the rehabilitation period and over the following 12 weeks, upper extremity function, cognitive function, mental status, and daily living activity performance will be evaluated in the form of questionnaires. To trace brain reorganization in which upper extremity functions previously performed by ischemic-related brain areas are assumed by other brain areas, subjects will have brain scans immediately following enrollment but before randomization, immediately following the conclusion of rehabilitation, and 12 weeks after rehabilitation has concluded.

DISCUSSION

Effectiveness is assessed by evaluating motor improvement using the arm motor section of the Fugl-Meyer assessment. The study utilizes a cutting-edge brain neuroimaging approach to longitudinally trace the effectiveness of both VR-based and conventional training on stroke rehabilitation, which will hopefully describe the effects of the brain mechanisms of the intervention on recovery from stroke. Findings from the trial will greatly contribute to evidence on the use of immersive-VR-based training for stroke rehabilitation.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT03086889 . Registered on March 22, 2017.

A survey of human shoulder functional kinematic representations

In this survey, we review the field of human shoulder functional kinematic representations. The central question of this review is to evaluate whether the current approaches in shoulder kinematics can meet the high-reliability computational challenge. This challenge is posed by applications such as robot-assisted rehabilitation. Currently, the role of kinematic representations in such applications has been mostly overlooked. Therefore, we have systematically searched and summarised the existing literature on shoulder kinematics. The shoulder is an important functional joint, and its large range of motion (ROM) poses several mathematical and practical challenges. Frequently, in kinematic analysis, the role of the shoulder articulation is approximated to a ball-and-socket joint. Following the high-reliability computational challenge, our review challenges this inappropriate use of reductionism. Therefore, we propose that this challenge could be met by kinematic representations, that are redundant, that use an active interpretation and that emphasise on functional understanding.

A Subject-Specific Kinematic Model to Predict Human Motion in Exoskeleton-Assisted Gait

The relative motion between human and exoskeleton is a crucial factor that has remarkable consequences on the efficiency, reliability and safety of human-robot interaction. Unfortunately, its quantitative assessment has been largely overlooked in the literature. Here, we present a methodology that allows predicting the motion of the human joints from the knowledge of the angular motion of the exoskeleton frame. Our method combines a subject-specific skeletal model with a kinematic model of a lower limb exoskeleton (H2, Technaid), imposing specific kinematic constraints between them. To calibrate the model and validate its ability to predict the relative motion in a subject-specific way, we performed experiments on seven healthy subjects during treadmill walking tasks. We demonstrate a prediction accuracy lower than 3.5° globally, and around 1.5° at the hip level, which represent an improvement up to 66% compared to the traditional approach assuming no relative motion between the user and the exoskeleton.

Optical Enhancement of Exoskeleton-Based Estimation of Glenohumeral Angles

In Robot-Assisted Rehabilitation (RAR) the accurate estimation of the patient limb joint angles is critical for assessing therapy efficacy. In RAR, the use of classic motion capture systems (MOCAPs) (e.g., optical and electromagnetic) to estimate the Glenohumeral (GH) joint angles is hindered by the exoskeleton body, which causes occlusions and magnetic disturbances. Moreover, the exoskeleton posture does not accurately reflect limb posture, as their kinematic models differ. To address the said limitations in posture estimation, we propose installing the cameras of an optical marker-based MOCAP in the rehabilitation exoskeleton. Then, the GH joint angles are estimated by combining the estimated marker poses and exoskeleton Forward Kinematics. Such hybrid system prevents problems related to marker occlusions, reduced camera detection volume, and imprecise joint angle estimation due to the kinematic mismatch of the patient and exoskeleton models. This paper presents the formulation, simulation, and accuracy quantification of the proposed method with simulated human movements. In addition, a sensitivity analysis of the method accuracy to marker position estimation errors, due to system calibration errors and marker drifts, has been carried out. The results show that, even with significant errors in the marker position estimation, method accuracy is adequate for RAR.

Inverse Kinematics for Upper Limb Compound Movement Estimation in Exoskeleton-Assisted Rehabilitation

Robot-Assisted Rehabilitation (RAR) is relevant for treating patients affected by nervous system injuries (e.g., stroke and spinal cord injury). The accurate estimation of the joint angles of the patient limbs in RAR is critical to assess the patient improvement. The economical prevalent method to estimate the patient posture in Exoskeleton-based RAR is to approximate the limb joint angles with the ones of the Exoskeleton. This approximation is rough since their kinematic structures differ. Motion capture systems (MOCAPs) can improve the estimations, at the expenses of a considerable overload of the therapy setup. Alternatively, the Extended Inverse Kinematics Posture Estimation (EIKPE) computational method models the limb and Exoskeleton as differing parallel kinematic chains. EIKPE has been tested with single DOF movements of the wrist and elbow joints. This paper presents the assessment of EIKPE with elbow-shoulder compound movements (i.e., object prehension). Ground-truth for estimation assessment is obtained from an optical MOCAP (not intended for the treatment stage). The assessment shows EIKPE rendering a good numerical approximation of the actual posture during the compound movement execution, especially for the shoulder joint angles. This work opens the horizon for clinical studies with patient groups, Exoskeleton models, and movements types.

A Virtual Reality-Cycling Training System for Lower Limb Balance Improvement

Stroke survivors might lose their walking and balancing abilities, but many studies pointed out that cycling is an effective means for lower limb rehabilitation. However, during cycle training, the unaffected limb tends to compensate for the affected one, which resulted in suboptimal rehabilitation. To address this issue, we present a Virtual Reality-Cycling Training System (VRCTS), which senses the cycling force and speed in real-time, analyzes the acquired data to produce feedback to patients with a controllable VR car in a VR rehabilitation program, and thus specifically trains the affected side. The aim of the study was to verify the functionality of the VRCTS and to verify the results from the ten stroke patients participants and to compare the result of Asymmetry Ratio Index (ARI) between the experimental group and the control group, after their training, by using the bilateral pedal force and force plate to determine any training effect. The results showed that after the VRCTS training in bilateral pedal force it had improved by 0.22 (p = 0.046) and in force plate the stand balance has also improved by 0.29 (p = 0.031); thus both methods show the significant difference.

Assessment of movement quality in robot- assisted upper limb rehabilitation after stroke: a review

Studies of stroke patients undergoing robot-assisted rehabilitation have revealed various kinematic parameters describing movement quality of the upper limb. However, due to the different level of stroke impairment and different assessment criteria and interventions, the evaluation of the effectiveness of rehabilitation program is undermined. This paper presents a systematic review of kinematic assessments of movement quality of the upper limb and identifies the suitable parameters describing impairments in stroke patients. A total of 41 different clinical and pilot studies on different phases of stroke recovery utilizing kinematic parameters are evaluated. Kinematic parameters describing movement accuracy are mostly reported for chronic patients with statistically significant outcomes and correlate strongly with clinical assessments. Meanwhile, parameters describing feed-forward sensorimotor control are the most frequently reported in studies on sub-acute patients with significant outcomes albeit without correlation to any clinical assessments. However, lack of measures in coordinated movement and proximal component of upper limb enunciate the difficulties to distinguish the exploitation of joint redundancies exhibited by stroke patients in completing the movement. A further study on overall measures of coordinated movement is recommended.