Can We Achieve Intuitive Prosthetic Elbow Control Based on Healthy Upper Limb Motor Strategies?

Intuitive movement-based prosthesis control enables arm amputees to reach naturally in virtual reality

Impressive progress is being made in bionic limbs design and control. Yet, controlling the numerous joints of a prosthetic arm necessary to place the hand at a correct position and orientation to grasp objects remains challenging. Here, we designed an intuitive, movement-based prosthesis control that leverages natural arm coordination to predict distal joints missing in people with transhumeral limb loss based on proximal residual limb motion and knowledge of the movement goal. This control was validated on 29 participants, including seven with above-elbow limb loss, who picked and placed bottles in a wide range of locations in virtual reality, with median success rates over 99% and movement times identical to those of natural movements. This control also enabled 15 participants, including three with limb differences, to reach and grasp real objects with a robotic arm operated according to the same principle. Remarkably, this was achieved without any prior training, indicating that this control is intuitive and instantaneously usable. It could be used for phantom limb pain management in virtual reality, or to augment the reaching capabilities of invasive neural interfaces usually more focused on hand and grasp control.

Kinematic analysis of impairments and compensatory motor behavior during prosthetic grasping in below-elbow amputees

After a major upper limb amputation, the use of myoelectric prosthesis as assistive devices is possible. However, these prostheses remain quite difficult to control for grasping and manipulation of daily life objects. The aim of the present observational case study is to document the kinematics of grasping in a group of 10 below-elbow amputated patients fitted with a myoelectric prosthesis in order to describe and better understand their compensatory strategies. They performed a grasping to lift task toward 3 objects (a mug, a cylinder and a cone) placed at two distances within the reaching area in front of the patients. The kinematics of the trunk and upper-limb on the non-amputated and prosthetic sides were recorded with 3 electromagnetic Polhemus sensors placed on the hand, the forearm (or the corresponding site on the prosthesis) and the ipsilateral acromion. The 3D position of the elbow joint and the shoulder and elbow angles were calculated thanks to a preliminary calibration of the sensor position. We examined first the effect of side, distance and objects with non-parametric statistics. Prosthetic grasping was characterized by severe temporo-spatial impairments consistent with previous clinical or kinematic observations. The grasping phase was prolonged and the reaching and grasping components uncoupled. The 3D hand displacement was symmetrical in average, but with some differences according to the objects. Compensatory strategies involved the trunk and the proximal part of the upper-limb, as shown by a greater 3D displacement of the elbow for close target and a greater forward displacement of the acromion, particularly for far targets. The hand orientation at the time of grasping showed marked side differences with a more frontal azimuth, and a more "thumb-up" roll. The variation of hand orientation with the object on the prosthetic side, suggested that the lack of finger and wrist mobility imposed some adaptation of hand pose relative to the object. The detailed kinematic analysis allows more insight into the mechanisms of the compensatory strategies that could be due to both increased distal or proximal kinematic constraints. A better knowledge of those compensatory strategies is important for the prevention of musculoskeletal disorders and the development of innovative prosthetics.

Contributions of the Cybathlon championship to the literature on functional electrical stimulation cycling among individuals with spinal cord injury: A bibliometric review

BACKGROUND

Due to its clinically proven safety and health benefits, functional electrical stimulation (FES) cycling has become a popular exercise modality for individuals with spinal cord injury (SCI). Since its inception in 2013, the Cybathlon championship has been a platform for publicizing the potential of FES cycling in rehabilitation and exercise for individuals with SCI. This study aimed to evaluate the contribution of the Cybathlon championship to the literature on FES cycling for individuals with SCI 3 years pre and post the staging of the Cybathlon championship in 2016.

METHODS

Web of Science, Scopus, ScienceDirect, IEEE Xplore, and Google Scholar databases were searched for relevant studies published between January 2013 and July 2019. The quality of the included studies was objectively evaluated using the Downs and Black checklist.

RESULTS

A total of 129 articles on FES cycling were retained for analysis. A total of 51 articles related to Cybathlon were reviewed, and 14 articles were ultimately evaluated for the quality. In 2017, the year following the Cybathlon championship, Web of Science cited 23 published studies on the championship, which was almost 5-fold more than that in 2016 (n = 5). Training was most often reported as a topic of interest in these studies, which mostly (76.7%) highlighted the training parameters of interest to participating teams in their effort to maximize their FES cycling performance during the Cybathlon championship.

CONCLUSION

The present study indicates that the Cybathlon championship in 2016 contributed to the number of literature published in 2017 on FES cycling for individuals with SCI. This finding may contribute to the lessons that can be learned from participation in the Cybathlon and potentially provide additional insights into research in the field of race-based FES cycling.

Wearable Assistive Robotics: A Perspective on Current Challenges and Future Trends

Wearable assistive robotics is an emerging technology with the potential to assist humans with sensorimotor impairments to perform daily activities. This assistance enables individuals to be physically and socially active, perform activities independently, and recover quality of life. These benefits to society have motivated the study of several robotic approaches, developing systems ranging from rigid to soft robots with single and multimodal sensing, heuristics and machine learning methods, and from manual to autonomous control for assistance of the upper and lower limbs. This type of wearable robotic technology, being in direct contact and interaction with the body, needs to comply with a variety of requirements to make the system and assistance efficient, safe and usable on a daily basis by the individual. This paper presents a brief review of the progress achieved in recent years, the current challenges and trends for the design and deployment of wearable assistive robotics including the clinical and user need, material and sensing technology, machine learning methods for perception and control, adaptability and acceptability, datasets and standards, and translation from lab to the real world.

Intention Prediction and Human Health Condition Detection in Reaching Tasks with Machine Learning Techniques

Detecting human motion and predicting human intentions by analyzing body signals are challenging but fundamental steps for the implementation of applications presenting human–robot interaction in different contexts, such as robotic rehabilitation in clinical environments, or collaborative robots in industrial fields. Machine learning techniques (MLT) can face the limit of small data amounts, typical of this kind of applications. This paper studies the illustrative case of the reaching movement in 10 healthy subjects and 21 post-stroke patients, comparing the performance of linear discriminant analysis (LDA) and random forest (RF) in: (i) predicting the subject’s intention of moving towards a specific direction among a set of possible choices, (ii) detecting if the subject is moving according to a healthy or pathological pattern, and in the case of discriminating the damage location (left or right hemisphere). Data were captured with wearable electromagnetic sensors, and a sub-section of the acquired signals was required for the analyses. The possibility of detecting with which arm (left or right hand) the motion was performed, and the sensitivity of the MLT to variations in the length of the signal sub-section were also evaluated. LDA and RF prediction accuracies were compared: Accuracy improves when only healthy subjects or longer signals portions are considered up to 11% and at least 10%, respectively. RF reveals better estimation performance both as intention predictor (on average 59.91% versus the 62.19% of LDA), and health condition detector (over 90% in all the tests).

A novel framework for designing a multi-DoF prosthetic wrist control using machine learning

Prosthetic arms can significantly increase the upper limb function of individuals with upper limb loss, however despite the development of various multi-DoF prosthetic arms the rate of prosthesis abandonment is still high. One of the major challenges is to design a multi-DoF controller that has high precision, robustness, and intuitiveness for daily use. The present study demonstrates a novel framework for developing a controller leveraging machine learning algorithms and movement synergies to implement natural control of a 2-DoF prosthetic wrist for activities of daily living (ADL). The data was collected during ADL tasks of ten individuals with a wrist brace emulating the absence of wrist function. Using this data, the neural network classifies the movement and then random forest regression computes the desired velocity of the prosthetic wrist. The models were trained/tested with ADLs where their robustness was tested using cross-validation and holdout data sets. The proposed framework demonstrated high accuracy (F-1 score of 99% for the classifier and Pearson's correlation of 0.98 for the regression). Additionally, the interpretable nature of random forest regression was used to verify the targeted movement synergies. The present work provides a novel and effective framework to develop an intuitive control for multi-DoF prosthetic devices.

Gesture Recognition Using Surface Electromyography and Deep Learning for Prostheses Hand: State-of-the-Art, Challenges, and Future

Amputation of the upper limb brings heavy burden to amputees, reduces their quality of life, and limits their performance in activities of daily life. The realization of natural control for prosthetic hands is crucial to improving the quality of life of amputees. Surface electromyography (sEMG) signal is one of the most widely used biological signals for the prediction of upper limb motor intention, which is an essential element of the control systems of prosthetic hands. The conversion of sEMG signals into effective control signals often requires a lot of computational power and complex process. Existing commercial prosthetic hands can only provide natural control for very few active degrees of freedom. Deep learning (DL) has performed surprisingly well in the development of intelligent systems in recent years. The significant improvement of hardware equipment and the continuous emergence of large data sets of sEMG have also boosted the DL research in sEMG signal processing. DL can effectively improve the accuracy of sEMG pattern recognition and reduce the influence of interference factors. This paper analyzes the applicability and efficiency of DL in sEMG-based gesture recognition and reviews the key techniques of DL-based sEMG pattern recognition for the prosthetic hand, including signal acquisition, signal preprocessing, feature extraction, classification of patterns, post-processing, and performance evaluation. Finally, the current challenges and future prospects in clinical application of these techniques are outlined and discussed.

Hand Bionic Score: a clinical follow-up study of severe hand injuries and development of a recommendation score to supply bionic prosthesis

Background

Severe hand injuries significantly limit function and esthetics of the affected hand due to massive trauma in skeletal and soft tissues. Surgical reconstruction is often unsatisfactory, so bionic prostheses are a consideration. However, assessment of functional outcomes and quality of life after surgical reconstruction to guide clinical decisions immediately after injury and in the course of treatment remain difficult.

Methods

We conducted a prospective follow-up analysis of patients with severe hand injuries during 2016–2018. We retrospectively evaluated initial trauma severity and examined current functional status, quality of life, general function, and satisfaction in everyday situations of the hand. We also developed a novel Hand Bionic Score to guide clinical recommendation for selective amputation and bionic prosthesis supply.

Results

We examined 30 patients with a mean age of 53.8 years and mean initial severity of hand injury (iHISS) of 138.4. Measures indicated moderate quality of life limitations, moderate to severe limitation of overall hand function, and slight to moderate limitation of actual hand strength and function. Mean time to follow-up examination was 3.67 years. Using the measured outcomes, we developed a Hand Bionic Score that showed good ability to differentiate patients based on outcome markers. Appropriate cutoff scores for all measured outcome markers were used to determine Hand Bionic Score classifications to guide clinical recommendation for elective amputation and bionic prosthetic supply: < 10 points, bionic hand supply not recommended; 10–14, bionic supply should be considered; or > 14, bionic supply is recommended.

Conclusions

While iHISS can guide early clinical decisions following severe hand injury, our novel Hand Bionic Score provides orientation for clinical decision-making regarding elective amputation and bionic prosthesis supply later during the course of treatment. The score not only considers hand function but also psychological outcomes and quality of life, which are important considerations for patients with severe hand injuries. However, future randomized multicenter studies are needed to validate Hand Bionic Score before further clinical application.

Level of evidence: Level III, risk/prognostic study.

Personalized Online Adaptation of Kinematic Synergies for Human-Prosthesis Interfaces

Synergies have been adopted in prosthetic limb applications to reduce the complexity of design, but typically involve a single synergy setting for a population and ignore individual preference or adaptation capacity. However, personalization of the synergy setting is necessary for the effective operation of the prosthetic device. Two major challenges hinder the personalization of synergies in human-prosthesis interfaces (HPIs). The first is related to the process of human motor adaptation and the second to the variation in motor learning dynamics of individuals. In this paper, a systematic personalization of kinematic synergies for HPIs using online measurements from each individual is proposed. The task of reaching using the upper limb is described by an objective function and the interface is parameterized by a kinematic synergy. Consequently, personalizing the interface for a given individual can be formulated as finding an optimal personalized parameter. A structure to model the observed motor behavior that allows for the personalized traits of motor preference and motor learning is proposed, and subsequently used in an online optimization scheme to identify the synergies for an individual. The knowledge of the common features contained in the model enables online adaptation of the HPI to happen concurrently to human motor adaptation without the need to retune the personalization algorithm for each individual. Human-in-the-loop experimental results with able-bodied subjects, performed in a virtual reality environment to emulate amputation and prosthesis use, show that the proposed personalization algorithm was effective in obtaining optimal synergies with a fast uniform convergence speed across a group of individuals.

Task-Space Synergies for Reaching Using Upper-Limb Prostheses

Synergistic prostheses enable the coordinated movement of the human-prosthetic arm, as required by activities of daily living. This is achieved by coupling the motion of the prosthesis to the human command, such as the residual limb movement in motion-based interfaces. Previous studies demonstrated that developing human-prosthetic synergies in joint-space must consider individual motor behaviour and the intended task to be performed, requiring personalisation and task calibration. In this work, an alternative synergy-based strategy, utilising a synergistic relationship expressed in task-space, is proposed. This task-space synergy has the potential to replace the need for personalisation and task calibration with a model-based approach requiring knowledge of the individual user's arm kinematics, the anticipated hand motion during the task and voluntary information from the prosthetic user. The proposed method is compared with surface electromyography-based and joint-space synergy-based prosthetic interfaces in a study of motor behaviour and task performance on able-bodied subjects using a VR-based transhumeral prosthesis. Experimental results showed that for a set of forward reaching tasks the proposed task-space synergy achieves comparable performance to joint-space synergies without the need to rely on time-consuming calibration processes or human motor learning. Case study results with an amputee subject motivate the further development of the proposed task-space synergy method.

Shoulder kinematics plus contextual target information enable control of multiple distal joints of a simulated prosthetic arm and hand

Background

Prosthetic restoration of reach and grasp function after a trans-humeral amputation requires control of multiple distal degrees of freedom in elbow, wrist and fingers. However, such a high level of amputation reduces the amount of available myoelectric and kinematic information from the residual limb.

Methods

To overcome these limits, we added contextual information about the target’s location and orientation such as can now be extracted from gaze tracking by computer vision tools. For the task of picking and placing a bottle in various positions and orientations in a 3D virtual scene, we trained artificial neural networks to predict postures of an intact subject’s elbow, forearm and wrist (4 degrees of freedom) either solely from shoulder kinematics or with additional knowledge of the movement goal. Subjects then performed the same tasks in the virtual scene with distal joints predicted from the context-aware network.

Results

Average movement times of 1.22s were only slightly longer than the naturally controlled movements (0.82 s). When using a kinematic-only network, movement times were much longer (2.31s) and compensatory movements from trunk and shoulder were much larger. Integrating contextual information also gave rise to motor synergies closer to natural joint coordination.

Conclusions

Although notable challenges remain before applying the proposed control scheme to a real-world prosthesis, our study shows that adding contextual information to command signals greatly improves prediction of distal joint angles for prosthetic control.

Kinematic Synergy of Multi-DoF Movement in Upper Limb and Its Application for Rehabilitation Exoskeleton Motion Planning

It is important for rehabilitation exoskeletons to move with a spatiotemporal motion patterns that well match the upper-limb joint kinematic characteristics. However, few efforts have been made to manipulate the motion control based on human kinematic synergies. This work analyzed the spatiotemporal kinematic synergies of right arm reaching movement and investigated their potential usage in upper limb assistive exoskeleton motion planning. Ten right-handed subjects were asked to reach 10 target button locations placed on a cardboard in front. The kinematic data of right arm were tracked by a motion capture system. Angular velocities over time for shoulder flexion/extension, shoulder abduction/adduction, shoulder internal/external rotation, and elbow flexion/extension were computed. Principal component analysis (PCA) was used to derive kinematic synergies from the reaching task for each subject. We found that the first four synergies can explain more than 94% of the variance. Moreover, the joint coordination patterns were dynamically regulated over time as the number of kinematic synergy (PC) increased. The synergies with different order played different roles in reaching movement. Our results indicated that the low-order synergies represented the overall trend of motion patterns, while the high-order synergies described the fine motions at specific moving phases. A 4-DoF upper limb assistive exoskeleton was modeled in SolidWorks to simulate assistive exoskeleton movement pattern based on kinematic synergy. An exoskeleton Denavit-Hartenberg (D-H) model was established to estimate the exoskeleton moving pattern in reaching tasks. The results further confirmed that kinematic synergies could be used for exoskeleton motion planning, and different principal components contributed to the motion trajectory and end-point accuracy to some extent. The findings of this study may provide novel but simplified strategies for the development of rehabilitation and assistive robotic systems approximating the motion pattern of natural upper-limb motor function.

Simulated robotic device malfunctions resembling malicious cyberattacks impact human perception of trust, satisfaction, and frustration

Robot assistants and wearable devices are highly useful; however, these artificial systems are susceptible to hackers. In this article, two sets of experiments were conducted. The first part of this study simulated a malicious attack on a prosthetic arm system to adversely affect the operation of the prosthetic system, while the perception of 10 human subjects was surveyed. These 10 able-bodied subjects controlled the prosthetic arm and hand with electromyogram signals, while an artificial sensation of touch was conveyed to their arms as they operated the system, which enabled them to feel what the prosthetic hand was grasping as they were asked to transport an object from one location to another. This haptic feedback was provided in both the normal and abnormal operational modes but was disabled in the extremely abnormal mode. The electromyogram control signals for the arm were reversed in both the abnormal and extremely abnormal modes. Results from the simulated malicious attack on a prosthetic arm system showed that the subjects found the haptic feedback helpful in both the normal and abnormal modes of operation. Both the abnormal and extremely abnormal modes of operation negatively impacted the self-reported levels of trust, satisfaction, and frustration with the prosthetic system as the subjects grasped and transported an object. While these metrics were negatively impacted by system malfunctions resembling a malicious attack on the control functionality, it was possible to rebuild them to their former higher levels after the functionality of the prosthetic system was restored. A parallel study in this article involved simulating a malicious attack on a robot assistant to unfavorably affect the delivery operation modes, while the perception of 20 human subjects was surveyed. Results showed that the simulated malfunctions unfavorably impacted the perception of trust, satisfaction, and frustration, but it was possible to restore these metrics in two different ways as the device functionality was restored.

Design and Assessment of Control Maps for Multi-Channel sEMG-Driven Prostheses and Supernumerary Limbs

Proportional and simultaneous control algorithms are considered as one of the most effective ways of mapping electromyographic signals to an artificial device. However, the applicability of these methods is limited by the high number of electromyographic features that they require to operate-typically twice as many the actuators to be controlled. Indeed, extracting many independent electromyographic signals is challenging for a number of reasons-ranging from technological to anatomical. On the contrary, the number of actively moving parts in classic prostheses or extra-limbs is often high. This paper faces this issue, by proposing and experimentally assessing a set of algorithms which are capable of proportionally and simultaneously control as many actuators as there are independent electromyographic signals available. Two sets of solutions are considered. The first uses as input electromyographic signals only, while the second adds postural measurements to the sources of information. At first, all the proposed algorithms are experimentally tested in terms of precision, efficiency, and usability on twelve able-bodied subjects, in a virtual environment. A state-of-the-art controller using twice the amount of electromyographic signals as input is adopted as benchmark. We then performed qualitative tests, where the maps are used to control a prototype of upper limb prosthesis. The device is composed of a robotic hand and a wrist implementing active prono-supination movement. Eight able-bodied subjects participated to this second round of testings. Finally, the proposed strategies were tested in exploratory experiments involving two subjects with limb loss. Results coming from the evaluations in virtual and realistic settings show encouraging results and suggest the effectiveness of the proposed approach.

Design and Evaluation of a Surface Electromyography-Controlled Steering Assistance Interface

Millions of drivers could experience shoulder muscle overload when rapidly rotating steering wheels and reduced steering ability at increased steering wheel angles. In order to address these issues for drivers with disability, surface electromyography (sEMG) sensors measuring biceps brachii muscle activity were incorporated into a steering assistance system for remote steering wheel rotation. The path-following accuracy of the sEMG interface with respect to a game steering wheel was evaluated through driving simulator trials. Human participants executed U-turns with differing radii of curvature. For a radius of curvature equal to the minimum vehicle turning radius of 3.6 m, the sEMG interface had significantly greater accuracy than the game steering wheel, with intertrial median lateral errors of 0.5 m and 1.2 m, respectively. For a U-turn with a radius of 7.2 m, the sEMG interface and game steering wheel were comparable in accuracy, with respective intertrial median lateral errors of 1.6 m and 1.4 m. The findings of this study could be utilized to realize accurate sEMG-controlled automobile steering for persons with disability.

Movement-Based Control for Upper-Limb Prosthetics: Is the Regression Technique the Key to a Robust and Accurate Control?

Due to the limitations of myoelectric control (such as dependence on muscular fatigue and on electrodes shift, difficulty in decoding complex patterns or in dealing with simultaneous movements), there is a renewal of interest in the movement-based control approaches for prosthetics. The latter use residual limb movements rather than muscular activity as command inputs, in order to develop more natural and intuitive control techniques. Among those, several research works rely on the interjoint coordinations that naturally exist in human upper limb movements. These relationships are modeled to control the distal joints (e.g., elbow) based on the motions of proximal ones (e.g., shoulder). The regression techniques, used to model the coordinations, are various [Artificial Neural Networks, Principal Components Analysis (PCA), etc.] and yet, analysis of their performance and impact on the prosthesis control is missing in the literature. Is there one technique really more efficient than the others to model interjoint coordinations? To answer this question, we conducted an experimental campaign to compare the performance of three common regression techniques in the control of the elbow joint on a transhumeral prosthesis. Ten non-disabled subjects performed a reaching task, while wearing an elbow prosthesis which was driven by several interjoint coordination models obtained through different regression techniques. The models of the shoulder-elbow kinematic relationship were built from the recordings of fifteen different non-disabled subjects that performed a similar reaching task with their healthy arm. Among Radial Basis Function Networks (RBFN), Locally Weighted Regression (LWR), and PCA, RBFN was found to be the most robust, based on the analysis of several criteria including the quality of generated movements but also the compensatory strategies exhibited by users. Yet, RBFN does not significantly outperform LWR and PCA. The regression technique seems not to be the most significant factor for improvement of interjoint coordinations-based control. By characterizing the impact of the modeling techniques through closed-loop experiments with human users instead of purely offline simulations, this work could also help in improving movement-based control approaches and in bringing them closer to a real use by patients.