Feedback control of arm movements using Neuro-Muscular Electrical Stimulation (NMES) combined with a lockable, passive exoskeleton for gravity compensation

Arm muscle synergies enhance hand posture prediction in combination with forearm muscle synergies

Objective.We analyze and interpret arm and forearm muscle activity in relation with the kinematics of hand pre-shaping during reaching and grasping from the perspective of human synergistic motor control.Approach.Ten subjects performed six tasks involving reaching, grasping and object manipulation. We recorded electromyographic (EMG) signals from arm and forearm muscles with a mix of bipolar electrodes and high-density grids of electrodes. Motion capture was concurrently recorded to estimate hand kinematics. Muscle synergies were extracted separately for arm and forearm muscles, and postural synergies were extracted from hand joint angles. We assessed whether activation coefficients of postural synergies positively correlate with and can be regressed from activation coefficients of muscle synergies. Each type of synergies was clustered across subjects.Main results.We found consistency of the identified synergies across subjects, and we functionally evaluated synergy clusters computed across subjects to identify synergies representative of all subjects. We found a positive correlation between pairs of activation coefficients of muscle and postural synergies with important functional implications. We demonstrated a significant positive contribution in the combination between arm and forearm muscle synergies in estimating hand postural synergies with respect to estimation based on muscle synergies of only one body segment, either arm or forearm (p< 0.01). We found that dimensionality reduction of multi-muscle EMG root mean square (RMS) signals did not significantly affect hand posture estimation, as demonstrated by comparable results with regression of hand angles from EMG RMS signals.Significance.We demonstrated that hand posture prediction improves by combining activity of arm and forearm muscles and we evaluate, for the first time, correlation and regression between activation coefficients of arm muscle and hand postural synergies. Our findings can be beneficial for myoelectric control of hand prosthesis and upper-limb exoskeletons, and for biomarker evaluation during neurorehabilitation.

The spinal cord facilitates cerebellar upper limb motor learning and control; inputs from neuromusculoskeletal simulation

Complex interactions between brain regions and the spinal cord (SC) govern body motion, which is ultimately driven by muscle activation. Motor planning or learning are mainly conducted at higher brain regions, whilst the SC acts as a brain-muscle gateway and as a motor control centre providing fast reflexes and muscle activity regulation. Thus, higher brain areas need to cope with the SC as an inherent and evolutionary older part of the body dynamics. Here, we address the question of how SC dynamics affects motor learning within the cerebellum; in particular, does the SC facilitate cerebellar motor learning or constitute a biological constraint? We provide an exploratory framework by integrating biologically plausible cerebellar and SC computational models in a musculoskeletal upper limb control loop. The cerebellar model, equipped with the main form of cerebellar plasticity, provides motor adaptation; whilst the SC model implements stretch reflex and reciprocal inhibition between antagonist muscles. The resulting spino-cerebellar model is tested performing a set of upper limb motor tasks, including external perturbation studies. A cerebellar model, lacking the implemented SC model and directly controlling the simulated muscles, was also tested in the same. The performances of the spino-cerebellar and cerebellar models were then compared, thus allowing directly addressing the SC influence on cerebellar motor adaptation and learning, and on handling external motor perturbations. Performance was assessed in both joint and muscle space, and compared with kinematic and EMG recordings from healthy participants. The differences in cerebellar synaptic adaptation between both models were also studied. We conclude that the SC facilitates cerebellar motor learning; when the SC circuits are in the loop, faster convergence in motor learning is achieved with simpler cerebellar synaptic weight distributions. The SC is also found to improve robustness against external perturbations, by better reproducing and modulating muscle cocontraction patterns.

Recent advancements in multimodal human-robot interaction

Robotics have advanced significantly over the years, and human-robot interaction (HRI) is now playing an important role in delivering the best user experience, cutting down on laborious tasks, and raising public acceptance of robots. New HRI approaches are necessary to promote the evolution of robots, with a more natural and flexible interaction manner clearly the most crucial. As a newly emerging approach to HRI, multimodal HRI is a method for individuals to communicate with a robot using various modalities, including voice, image, text, eye movement, and touch, as well as bio-signals like EEG and ECG. It is a broad field closely related to cognitive science, ergonomics, multimedia technology, and virtual reality, with numerous applications springing up each year. However, little research has been done to summarize the current development and future trend of HRI. To this end, this paper systematically reviews the state of the art of multimodal HRI on its applications by summing up the latest research articles relevant to this field. Moreover, the research development in terms of the input signal and the output signal is also covered in this manuscript.

Hybrid FES-exoskeleton control: Using MPC to distribute actuation for elbow and wrist movements

Introduction

Individuals who have suffered a cervical spinal cord injury prioritize the recovery of upper limb function for completing activities of daily living. Hybrid FES-exoskeleton systems have the potential to assist this population by providing a portable, powered, and wearable device; however, realization of this combination of technologies has been challenging. In particular, it has been difficult to show generalizability across motions, and to define optimal distribution of actuation, given the complex nature of the combined dynamic system.

Methods

In this paper, we present a hybrid controller using a model predictive control (MPC) formulation that combines the actuation of both an exoskeleton and an FES system. The MPC cost function is designed to distribute actuation on a single degree of freedom to favor FES control effort, reducing exoskeleton power consumption, while ensuring smooth movements along different trajectories. Our controller was tested with nine able-bodied participants using FES surface stimulation paired with an upper limb powered exoskeleton. The hybrid controller was compared to an exoskeleton alone controller, and we measured trajectory error and torque while moving the participant through two elbow flexion/extension trajectories, and separately through two wrist flexion/extension trajectories.

Results

The MPC-based hybrid controller showed a reduction in sum of squared torques by an average of 48.7 and 57.9% on the elbow flexion/extension and wrist flexion/extension joints respectively, with only small differences in tracking accuracy compared to the exoskeleton alone.

Discussion

To realize practical implementation of hybrid FES-exoskeleton systems, the control strategy requires translation to multi-DOF movements, achieving more consistent improvement across participants, and balancing control to more fully leverage the muscles' capabilities.

Shared Control of Elbow Movements with Functional Electrical Stimulation and Exoskeleton Assistance

Individuals who suffer from paralysis as a result of a spinal cord injury list restoration of arm and hand function as a top priority. FES helps restore movement using the user's own muscles, but does not produce accurate and repeatable movements necessary for many functional tasks. Robots can assist users in achieving accurate and repeatable movements, but often require bulky hardware to generate the necessary torques. We propose sharing torque requirements between a robot and FES to reduce robot torque output compared to a robot acting alone, yet maintain high accuracy. Cooperative PD and model predictive control algorithms were designed to share the control between these two torque sources. Corresponding PD and MPC algorithms that do not use FES were also designed. The control algorithms were tested with 10 able-bodied subjects. Torque and position tracking accuracy were compared when the system was commanded to follow a functional elbow flexion/extension trajectory. The robot torque required to achieve these movements was reduced for the shared control cases compared to the algorithms acting without FES. We observed a reduction in position accuracy with the MPC shared controller compared to the PD shared controller, while the MPC shared controller resulted in greater reductions in torque requirements. Both of these shared algorithms showed improvements over existing options, and can be used on any given trajectory, allowing for better transferability to functional tasks.

Proof of Concept of an Assistive Robotic Arm Control Using Artificial Stereovision and Eye-Tracking

Assistive robotic arms have become popular to help users with upper limb disabilities achieve autonomy in their daily tasks, such as drinking and grasping objects in general. Usually, these robotic arms are controlled with an adapted joystick. Joysticks are user-friendly when it comes to a general approach to an object. However, they are not as intuitive when having to accurately approach an object, especially when obstacles are present. Alternatively, the combined use of artificial stereovision and eye-tracking seems to be a promising solution, as the user's vision is usually dissociated from their upper limb disability. Hence, the objective of this study was to develop a proof of concept for the control of an assistive robotic arm using a low-cost combination of stereovision and eye-tracking. Using the developed control system, a typically developed person was able to control the robotic arm successfully reaching and grasping an object for 92% of the trials without obstacles with an average time of 13.8 seconds. Then, another set of trials with one obstacle had a success rate of 91% with an average time of 17.3 seconds. Finally, the last set of trials with two obstacles had a success rate of 98% with an average time of 18.4 seconds. Furthermore, the cost of an eye-tracker and stereovision remains below 400$.

BRIDGE - Behavioural reaching interfaces during daily antigravity activities through upper limb exoskeleton: Preliminary results

People with neuromuscular diseases such as muscular dystrophy experience a distributed and evolutive weakness in the whole body. Recent technological developments have changed the daily life of disabled people strongly improving the perceived quality of life, mostly concentrating on powered wheelchairs, so to assure autonomous mobility and respiratory assistance, essential for survival. The key concept of the BRIDGE project is to contrast the everyday experience of losing functions by providing them of a system able to exploit the best their own residual capabilities in arm movements so to keep them functional and autonomous as much as possible. BRIDGE is composed by a light, wearable and powered five degrees of freedom upper limb exoskeleton under the direct control of the user through a joystick or gaze control. An inverse kinematic model allows to determine joints position so to track patient desired hand position. BRIDGE prototype has been successfully tested in simulation environment, and by a small group of healthy volunteers. Preliminary results show a good tracking performance of the implemented control scheme. The interaction procedure was easy to understand, and the interaction with the system was successful.

The combined action of a passive exoskeleton and an EMG-controlled neuroprosthesis for upper limb stroke rehabilitation: First results of the RETRAINER project

The combined use of Functional Electrical Stimulation (FES) and robotic technologies is advocated to improve rehabilitation outcomes after stroke. This work describes an arm rehabilitation system developed within the European project RETRAINER. The system consists of a passive 4-degrees-of-freedom exoskeleton equipped with springs to provide gravity compensation and electromagnetic brakes to hold target positions. FES is integrated in the system to provide additional support to the most impaired muscles. FES is triggered based on the volitional EMG signal of the same stimulated muscle; in order to encourage the active involvement of the patient the volitional EMG is also monitored throughout the task execution and based on it a happy or sad emoji is visualized at the end of each task. The control interface control of the system provides a GUI and multiple software tools to organize rehabilitation exercises and monitor rehabilitation progress. The functionality and the usability of the system was evaluated on four stroke patients. All patients were able to use the system and judged positively its wearability and the provided support. They were able to trigger the stimulation based on their residual muscle activity and provided different levels of active involvement in the exercise, in agreement with their level of impairment. A randomized controlled trial aimed at evaluating the effectiveness of the RETRAINER system to improve arm function after stroke is currently ongoing.

Training an Actor-Critic Reinforcement Learning Controller for Arm Movement Using Human-Generated Rewards

Functional Electrical Stimulation (FES) employs neuroprostheses to apply electrical current to the nerves and muscles of individuals paralyzed by spinal cord injury to restore voluntary movement. Neuroprosthesis controllers calculate stimulation patterns to produce desired actions. To date, no existing controller is able to efficiently adapt its control strategy to the wide range of possible physiological arm characteristics, reaching movements, and user preferences that vary over time. Reinforcement learning (RL) is a control strategy that can incorporate human reward signals as inputs to allow human users to shape controller behavior. In this paper, ten neurologically intact human participants assigned subjective numerical rewards to train RL controllers, evaluating animations of goal-oriented reaching tasks performed using a planar musculoskeletal human arm simulation. The RL controller learning achieved using human trainers was compared with learning accomplished using human-like rewards generated by an algorithm; metrics included success at reaching the specified target; time required to reach the target; and target overshoot. Both sets of controllers learned efficiently and with minimal differences, significantly outperforming standard controllers. Reward positivity and consistency were found to be unrelated to learning success. These results suggest that human rewards can be used effectively to train RL-based FES controllers.

Upper-limb stroke rehabilitation using electrode-array based functional electrical stimulation with sensing and control innovations

Functional electrical stimulation (FES) has shown effectiveness in restoring upper-limb movement post-stroke when applied to assist participants' voluntary intention during repeated, motivating tasks. Recent clinical trials have used advanced controllers that precisely adjust FES to assist functional reach and grasp tasks with FES applied to three muscle groups, showing significant reduction in impairment. The system reported in this paper advances the state-of-the-art by: (1) integrating an FES electrode array on the forearm to assist complex hand and wrist gestures; (2) utilising non-contact depth cameras to accurately record the arm, hand and wrist position in 3D; and (3) employing an interactive touch table to present motivating virtual reality (VR) tasks. The system also uses iterative learning control (ILC), a model-based control strategy which adjusts the applied FES based on the tracking error recorded on previous task attempts. Feasibility of the system has been evaluated in experimental trials with 2 unimpaired participants and clinical trials with 4 hemiparetic, chronic stroke participants. The stroke participants attended 17, 1 hour training sessions in which they performed functional tasks, such as button pressing using the touch table and closing a drawer. Stroke participant results show that the joint angle error norm reduced by an average of 50.3% over 6 attempts at each task when assisted by FES.

Sites of electrical stimulation used in neurology

Rehabilitation aims to decrease neurological impairments, in guiding plasticity. Electrical stimulation has been used for many years in rehabilitation treatment of neurological disabilities as a tool for neuromodulation inducing plasticity, although the mechanisms of its action are not well known. The applications vary, encompassing therapeutic and rehabilitative aims. The type and site of stimulation vary depending on the objectives. Some techniques are widely used in clinical practice; others are still in the research stage. They may be invasive, epidural or in direct contact with neurons; they may be noninvasive, applied transcutaneously or indirectly by current vectors. The indications vary: mobility, functionality, pain as well as pharyngeal, respiratory, and perineal function. This paper aims to summarize current data on electrical neuromodulation techniques used in neurorehabilitation, their effects and their mechanisms of action.