Integrating Upper-Limb Prostheses with the Human Body: Technology Advances, Readiness, and Roles in Human-Prosthesis Interaction

Significant advances in bionic prosthetics have occurred in the past two decades. The field's rapid expansion has yielded many exciting technologies that can enhance the physical, functional, and cognitive integration of a prosthetic limb with a human. We review advances in the engineering of prosthetic devices and their interfaces with the human nervous system, as well as various surgical techniques for altering human neuromusculoskeletal systems for seamless human-prosthesis integration. We discuss significant advancements in research and clinical translation, focusing on upper limbprosthetics since they heavily rely on user intent for daily operation, although many discussed technologies have been extended to lower limb prostheses as well. In addition, our review emphasizes the roles of advanced prosthetics technologies in complex interactions with humans and the technology readiness levels (TRLs) of individual research advances. Finally, we discuss current gaps and controversies in the field and point out future research directions, guided by TRLs.

Feature recognition in multiple CNNs using sEMG images from a prototype comfort test

OBJECTIVE

Deep learning-based CNN networks have recently been investigated to solve the problem of body posture recognition based on surface electromyographic signals (sEMG). Influenced by these studies, to develop a combined approach of sEMG and CNNs in the study of human-product interactions and the impact of body comfort, and to compare the advantages and disadvantages of various CNNs networks.

METHODS

In this study, sEMG measurements were carried out by building a prototype usability experiment, and the data were divided into four categories, with two types of datasets: training and testing. Four CNNs, LeNet-5, VGGNet-11, InceptionNet V4, and DenseNet, were used for the recognition of sEMG images.

RESULTS

DenseNet is another type of convolutional neural network with deep layers, which has a unique advantage over other algorithms. unique advantages over other algorithms. DenseNet has fewer layers and better accuracy than InceptionNet V4, but not only does it bypass enhanced feature reuse, but its network is easier to train and has some regularization effects, while also mitigating the problems of gradient disappearance and model degradation.

CONCLUSION

These findings could lead to a more appropriate CNN model and a useful tool for developing comfort judgments of surface EMG signals, furthering the development of products that come into contact with the human body without the need for routine retraining.

Progressive unsupervised control of myoelectric upper limbs

Objective.Unsupervised myocontrol methods aim to create control models for myoelectric prostheses while avoiding the complications of acquiring reliable, regular, and sufficient labeled training data. A limitation of current unsupervised methods is that they fix the number of controlled prosthetic functions a priori, thus requiring an initial assessment of the user's motor skills and neglecting the development of novel motor skills over time.Approach.We developed a progressive unsupervised myocontrol (PUM) paradigm in which the user and the control model coadaptively identify distinct muscle synergies, which are then used to control arbitrarily associated myocontrol functions, each corresponding to a hand or wrist movement. The interaction starts with learning a single function and the user may request additional functions after mastering the available ones, which aligns the evolution of their motor skills with an increment in system complexity. We conducted a multi-session user study to evaluate PUM and compare it against a state-of-the-art non-progressive unsupervised alternative. Two participants with congenital upper-limb differences tested PUM, while ten non-disabled control participants tested either PUM or the non-progressive baseline. All participants engaged in myoelectric control of a virtual hand and wrist.Main results.PUM enabled autonomous learning of three myocontrol functions for participants with limb differences, and of all four available functions for non-disabled subjects, using both existing or newly identified muscle synergies. Participants with limb differences achieved similar success rates to non-disabled ones on myocontrol tests, but faced greater difficulties in internalizing new motor skills and exhibited slightly inferior movement quality. The performance was comparable with either PUM or the non-progressive baseline for the group of non-disabled participants.Significance.The PUM paradigm enables users to autonomously learn to operate the myocontrol system, adapts to the users' varied preexisting motor skills, and supports the further development of those skills throughout practice.

The use of nonnormalized surface EMG and feature inputs for LSTM-based powered ankle prosthesis control algorithm development

Advancements in instrumentation support improved powered ankle prostheses hardware development. However, control algorithms have limitations regarding number and type of sensors utilized and achieving autonomous adaptation, which is key to a natural ambulation. Surface electromyogram (sEMG) sensors are promising. With a minimized number of sEMG inputs an economic control algorithm can be developed, whereas limiting the use of lower leg muscles will provide a practical algorithm for both ankle disarticulation and transtibial amputation. To determine appropriate sensor combinations, a systematic assessment of the predictive success of variations of multiple sEMG inputs in estimating ankle position and moment has to conducted. More importantly, tackling the use of nonnormalized sEMG data in such algorithm development to overcome processing complexities in real-time is essential, but lacking. We used healthy population level walking data to (1) develop sagittal ankle position and moment predicting algorithms using nonnormalized sEMG, and (2) rank all muscle combinations based on success to determine economic and practical algorithms. Eight lower extremity muscles were studied as sEMG inputs to a long-short-term memory (LSTM) neural network architecture: tibialis anterior (TA), soleus (SO), medial gastrocnemius (MG), peroneus longus (PL), rectus femoris (RF), vastus medialis (VM), biceps femoris (BF) and gluteus maximus (GMax). Five features extracted from nonnormalized sEMG amplitudes were used: integrated EMG (IEMG), mean absolute value (MAV), Willison amplitude (WAMP), root mean square (RMS) and waveform length (WL). Muscle and feature combination variations were ranked using Pearson's correlation coefficient (r > 0.90 indicates successful correlations), the root-mean-square error and one-dimensional statistical parametric mapping between the original data and LSTM response. The results showed that IEMG+WL yields the best feature combination performance. The best performing variation was MG + RF + VM (r_position = 0.9099 and r_moment = 0.9707) whereas, PL (r_position = 0.9001, r_moment = 0.9703) and GMax+VM (r_position = 0.9010, r_moment = 0.9718) were distinguished as the economic and practical variations, respectively. The study established for the first time the use of nonnormalized sEMG in control algorithm development for level walking.

Simultaneous assessment and training of an upper-limb amputee using incremental machine-learning-based myocontrol: a single-case experimental design

BACKGROUND

Machine-learning-based myocontrol of prosthetic devices suffers from a high rate of abandonment due to dissatisfaction with the training procedure and with the reliability of day-to-day control. Incremental myocontrol is a promising approach as it allows on-demand updating of the system, thus enforcing continuous interaction with the user. Nevertheless, a long-term study assessing the efficacy of incremental myocontrol is still missing, partially due to the lack of an adequate tool to do so. In this work we close this gap and report about a person with upper-limb absence who learned to control a dexterous hand prosthesis using incremental myocontrol through a novel functional assessment protocol called SATMC (Simultaneous Assessment and Training of Myoelectric Control).

METHODS

The participant was fitted with a custom-made prosthetic setup with a controller based on Ridge Regression with Random Fourier Features (RR-RFF), a non-linear, incremental machine learning method, used to build and progressively update the myocontrol system. During a 13-month user study, the participant performed increasingly complex daily-living tasks, requiring fine bimanual coordination and manipulation with a multi-fingered hand prosthesis, in a realistic laboratory setup. The SATMC was used both to compose the tasks and continually assess the participant's progress. Patient satisfaction was measured using Visual Analog Scales.

RESULTS

Over the course of the study, the participant progressively improved his performance both objectively, e.g., the time required to complete each task became shorter, and subjectively, meaning that his satisfaction improved. The SATMC actively supported the improvement of the participant by progressively increasing the difficulty of the tasks in a structured way. In combination with the incremental RR-RFF allowing for small adjustments when required, the participant was capable of reliably using four actions of the prosthetic hand to perform all required tasks at the end of the study.

CONCLUSIONS

Incremental myocontrol enabled an upper-limb amputee to reliably control a dexterous hand prosthesis while providing a subjectively satisfactory experience. The SATMC can be an effective tool to this aim.

Toward higher-performance bionic limbs for wider clinical use

Most prosthetic limbs can autonomously move with dexterity, yet they are not perceived by the user as belonging to their own body. Robotic limbs can convey information about the environment with higher precision than biological limbs, but their actual performance is substantially limited by current technologies for the interfacing of the robotic devices with the body and for transferring motor and sensory information bidirectionally between the prosthesis and the user. In this Perspective, we argue that direct skeletal attachment of bionic devices via osseointegration, the amplification of neural signals by targeted muscle innervation, improved prosthesis control via implanted muscle sensors and advanced algorithms, and the provision of sensory feedback by means of electrodes implanted in peripheral nerves, should all be leveraged towards the creation of a new generation of high-performance bionic limbs. These technologies have been clinically tested in humans, and alongside mechanical redesigns and adequate rehabilitation training should facilitate the wider clinical use of bionic limbs.

Full-hand electrotactile feedback using electronic skin and matrix electrodes for high-bandwidth human-machine interfacing

Tactile feedback is relevant in a broad range of human-machine interaction systems (e.g. teleoperation, virtual reality and prosthetics). The available tactile feedback interfaces comprise few sensing and stimulation units, which limits the amount of information conveyed to the user. The present study describes a novel technology that relies on distributed sensing and stimulation to convey comprehensive tactile feedback to the user of a robotic end effector. The system comprises six flexible sensing arrays (57 sensors) integrated on the fingers and palm of a robotic hand, embedded electronics (64 recording channels), a multichannel stimulator and seven flexible electrodes (64 stimulation pads) placed on the volar side of the subject's hand. The system was tested in seven subjects asked to recognize contact positions and identify contact sliding on the electronic skin, using distributed anode configuration (DAC) and single dedicated anode configuration. The experiments demonstrated that DAC resulted in substantially better performance. Using DAC, the system successfully translated the contact patterns into electrotactile profiles that the subjects could recognize with satisfactory accuracy ([Formula: see text] for static and [Formula: see text] for dynamic patterns). The proposed system is an important step towards the development of a high-density human-machine interfacing between the user and a robotic hand. This article is part of the theme issue 'Advanced neurotechnologies: translating innovation for health and well-being'.

Noncontact Electromagnetic Wireless Recognition for Prosthesis Based on Intelligent Metasurface

With the development of artificial intelligence and Internet of Things, hand gesture recognition techniques have attracted great attention owing to their excellent applications in developing human-machine interaction (HMI). Here, the authors propose a non-contact hand gesture recognition method based on intelligent metasurface. Owing to the advantage of dynamically controlling the electromagnetic (EM) focusing in the wavefront engineering, a transmissive programmable metasurface is presented to illuminate the forearm with more focusing spots and obtain comprehensive echo data, which can be processed under the machine learning technology to reach the non-contact gesture recognition with high accuracy. Compared with the traditional passive antennas, unique variations of echo coefficients resulted from near fields perturbed by finger and wrist agonist muscles can be aquired through the programmable metasurface by switching the positions of EM focusing. The authors realize the gesture recognition using support vector machine algorithm based on five individual focusing spots data and all-five-spot data. The influences of the focusing spots on the gesture recognition are analyzed through linear discriminant analysis algorithm and Fisher score. Experimental verifications prove that the proposed metasurface-based non-contact wireless design can realize the classification of hand gesture recognition with higher accuracy than traditional passive antennas, and give an HMI solution.

Simultaneous and proportional control of wrist and hand degrees of freedom with kinematic prediction models from high-density EMG

To improve intuitive control and reduce training time for active upper limb prostheses, we developed a myocontrol system for 3 degrees of freedom (DoFs) of the hand and wrist. In an offline study, we systematically investigated movement sets used to train this system, to identify the optimal compromise between training time and performance. High-density surface electromyography (HDsEMG) and optical marker motion capture were recorded concurrently from the lower arms of 8 subjects performing a series of wrist and hand movements activating DoFs individually, sequentially, and simultaneously. The root mean square (RMS) feature extracted from the EMG signal and kinematics obtained from motion capture were used to train regression and classification models to predict the kinematics of wrist movements and opening and closing of the hand, respectively. Results showed successful predictions of kinematics when training with the complete training set (r2 = 0.78 for wrist regression and recall = 0.85 for hand closing/opening classification). In further analysis, the training set was substantially reduced by removing the simultaneous movements. This led to a statistically significant, but relatively small reduction of the effectiveness of the wrist controller (r2 = 0.70, p<0.05), without changes for the hand controller (closing recall = 0.83). Reducing the training time and complexity needed to control a prosthesis with simultaneous wrist control as well as detection of intention to close the hand can lead to improved uptake of upper limb prosthetics.

Muscle Synergy-driven Motor Unit Clustering for Human-Machine Interfacing

Electromyographic signals (EMGs) can provide information on the overall activity of the innervating motor neuros in any given muscle but also globally reflect the underlying neuromechanics of human movement (e.g., muscle synergies). motor unit(MU) decomposition is a technique based on the deconvolution of high-density EMGs (HD-EMG) in order to derive the activities of the corresponding motor neurons. This powerful yet very sensitive tool has seen some traction in human-machine interfacing (HMI) for rehabilitation. Here, we propose combining the synergy-inspired channel clustering in order to isolate the most prominent regions of EMG activation in each targeted degree of freedom (DoF) and thus cater to decomposition's sensitivity demands. Our assumption is that this will lead to a higher number of extracted MUs and consequently better motion estimation in HMIs. Indeed, in four subjects, we have shown a 69% average increase in the number of MUs when decomposition was done using muscle-synergy channel clustering. Consequently, all three of our kinematic estimators benefited from an increased pool of units, with the linear regressor showing the greatest improvement once compared to, the artificial neural network and the gated recurrent unit, which had the overall best performance. Clinical Relevance- The results demonstrated in this work provide a new perspective on the online EMG-driven HMI systems that can be greatly beneficial in the rehabilitation of motor disorders.

Exploring Biomimetic Stiffness Modulation and Wearable Finger Haptics for Improving Myoelectric Control of Virtual Hand

The embodiment of virtual hand (VH) by the user is generally deemed to be important for virtual reality (VR) based hand rehabilitation applications, which may help to engage the user and promote motor skill relearning. In particular, it requires that the VH should produce task-dependent interaction behaviors from rigid to soft. While such a capability is inherent to humans via hand stiffness regulation and haptic interactions, yet it have not been successfully imitated by VH in existing studies. In this paper, we present a work which integrates biomimetic stiffness regulation and wearable finger force feedback in VR scenarios involving myoelectric control of VH. On one hand, the biomimetic stiffness modulation intuitively enables VH to imitate the stiffness profile of the user's hand in real time. On the other hand, the wearable finger force-feedback device elicits a natural and realistic sensation of external force on the fingertip, which provides the user a proper understanding of the environment for enhancing his/her stiffness regulation. The benefits of the proposed integrated system were evaluated with eight healthy subjects that performed two tasks with opposite stiffness requirements. The achieved performance is compared with reduced versions of the integrated system, where either biomimetic impedance control or wearable force feedback is excluded. The results suggest that the proposed integrated system enables the stiffness of VH to be adaptively regulated by the user through the perception of interaction torques and vision, resulting in task-dependent behaviors from rigid to soft for VH.

SeNic: An Open Source Dataset for sEMG-Based Gesture Recognition in Non-ideal Conditions

In order to reduce the gap between the laboratory environment and actual use in daily life of human-machine interaction based on surface electromyogram (sEMG) intent recognition, this paper presents a benchmark dataset of sEMG in non-ideal conditions (SeNic). The dataset mainly consists of 8-channel sEMG signals, and electrode shifts from an 3D-printed annular ruler. A total of 36 subjects participate in our data acquisition experiments of 7 gestures in non-ideal conditions, where non-ideal factors of 1) electrode shifts, 2) individual difference, 3) muscle fatigue, 4) inter-day difference, and 5) arm postures are elaborately involved. Signals of sEMG are validated first in temporal and frequency domains. Results of recognizing gestures in ideal conditions indicate the high quality of the dataset. Adverse impacts in non-ideal conditions are further revealed in the amplitudes of these data and recognition accuracies. To be concluded, SeNic is a benchmark dataset that introduces several non-ideal factors which often degrade the robustness of sEMG-based systems. It could be used as a freely available dataset and a common platform for researchers in the sEMG-based recognition community. The benchmark dataset SeNic are available online via the website3.

The Influence of Training with Visual Biofeedback on the Predictability of Myoelectric Control Usability

Studies have shown that closed-loop myoelectric control schemes can lead to changes in user performance and behavior compared to open-loop systems. When users are placed within the control loop, such as during real-time use, they must correct for errors made by the controller and learn what behavior is necessary to produce desired outcomes. Augmented feedback, consequently, has been used to incorporate the user throughout the training process and to facilitate learning. This work explores the effect of visual feedback presented during user training on both the performance and predictability of a myoelectric classification-based control system. Our results suggest that properly designed feedback mechanisms and training tasks can influence the quality of the training data and the ability to predict usability using linear combinations of metrics derived from feature space. Furthermore, our results confirm that the most common in-lab training protocol, screen guided training, may yield training data that are less representative of online use than training protocols that incorporate the user in the loop. These results suggest that training protocols should be designed that better parallel the testing environment to more effectively prepare both the algorithms and users for real-time control.

Real-Time Control of a Multi-Degree-of-Freedom Mirror Myoelectric Interface During Functional Task Training

Motor learning mediated by motor training has in the past been explored for rehabilitation. Myoelectric interfaces together with exoskeletons allow patients to receive real-time feedback about their muscle activity. However, the number of degrees of freedom that can be simultaneously controlled is limited, which hinders the training of functional tasks and the effectiveness of the rehabilitation therapy. The objective of this study was to develop a myoelectric interface that would allow multi-degree-of-freedom control of an exoskeleton involving arm, wrist and hand joints, with an eye toward rehabilitation. We tested the effectiveness of a myoelectric decoder trained with data from one upper limb and mirrored to control a multi-degree-of-freedom exoskeleton with the opposite upper limb (i.e., mirror myoelectric interface) in 10 healthy participants. We demonstrated successful simultaneous control of multiple upper-limb joints by all participants. We showed evidence that subjects learned the mirror myoelectric model within the span of a five-session experiment, as reflected by a significant decrease in the time to execute trials and in the number of failed trials. These results are the necessary precursor to evaluating if a decoder trained with EMG from the healthy limb could foster learning of natural EMG patterns and lead to motor rehabilitation in stroke patients.

Discrete action control for prosthetic digits

We aim to develop a paradigm for simultaneous and independent control of multiple degrees of freedom (DOFs) for upper-limb prostheses. To that end, we introduce action control, a novel method to operate prosthetic digits with surface electromyography (EMG) based on multi-output, multi-class classification. At each time step, the decoder classifies movement intent for each controllable DOF into one of three categories: open, close, or stall (i.e., no movement). We implemented a real-time myoelectric control system using this method and evaluated it by running experiments with one unilateral and two bilateral amputees. Participants controlled a six-DOF bar interface on a computer display, with each DOF corresponding to a motor function available in multi-articulated prostheses. We show that action control can significantly and systematically outperform the state-of-the-art method of position control via multi-output regression in both task- and non-task-related measures. Using the action control paradigm, improvements in median task performance over regression-based control ranged from 20.14% to 62.32% for individual participants. Analysis of a post-experimental survey revealed that all participants rated action higher than position control in a series of qualitative questions and expressed an overall preference for the former. Action control has the potential to improve the dexterity of upper-limb prostheses. In comparison with regression-based systems, it only requires discrete instead of real-valued ground truth labels, typically collected with motion tracking systems. This feature makes the system both practical in a clinical setting and also suitable for bilateral amputation. This work is the first demonstration of myoelectric digit control in bilateral upper-limb amputees. Further investigation and preclinical evaluation are required to assess the translational potential of the method.

Co-adaptive control of bionic limbs via unsupervised adaptation of muscle synergies

OBJECTIVE

In this work, we present a myoelectric interface that extracts natural motor synergies from multi-muscle signals and adapts in real-time with new user inputs. With this unsupervised adaptive myocontrol (UAM) system, optimal synergies for control are continuously co-adapted with changes in user motor control, or as a function of perturbed conditions via online non-negative matrix factorization guided by physiologically informed sparseness constraints in lieu of explicit data labelling.

METHODS

UAM was tested in a set of virtual target reaching tasks completed by able-bodied and amputee subjects. Tests were conducted under normative and electrode perturbed conditions to gauge control robustness with comparisons to non-adaptive and supervised adaptive myocontrol schemes. Furthermore, UAM was used to interface an amputee with a multi-functional powered hand prosthesis during standardized Clothespin Relocation Tests, also conducted in normative and perturbed conditions.

RESULTS

In virtual tests, UAM effectively mitigated performance degradation caused by electrode displacement, affording greater resilience over an existing supervised adaptive system for amputee subjects. Induced electrode shifts also had negligible effect on the real world control performance of UAM with consistent completion times (23.91±1.33 s) achieved across Clothespin Relocation Tests in the normative and electrode perturbed conditions.

CONCLUSION

UAM affords comparable robustness improvements to existing supervised adaptive myocontrol interfaces whilst providing additional practical advantages for clinical deployment.

SIGNIFICANCE

The proposed system uniquely incorporates neuromuscular control principles with unsupervised online learning methods and presents a working example of a freely co-adaptive bionic interface.

Muscle-Specific High-Density Electromyography Arrays for Hand Gesture Classification

OBJECTIVE

Dexterous hand motion is critical for object manipulation. Electrophysiological studies of the hand are key to understanding its underlying mechanisms. High-density electromyography (HD-EMG) provides spatio-temporal information about the underlying electrical activity of muscles, which can be used in neurophysiological research, rehabilitation and control applications. However, existing EMG electrodes platforms are not muscle-specific, which makes the assessment of intrinsic hand muscles difficult.

METHODS

Muscle-specific flexible HD-EMG electrode arrays were developed to capture intrinsic hand muscle myoelectric activity during manipulation tasks. The arrays consist of 60 individual electrodes targeting 10 intrinsic hand muscles. Myoelectric activity was displayed as spatio-temporal amplitude maps to visualize muscle activation. Time-domain and temporal-spatial HD-EMG features were extracted to train cubic support vector machine machine-learning classifiers to classify the intended user motion.

RESULTS

Experimental data was collected from 5 subjects performing a range of 10 common hand motions. Spatio-temporal EMG maps showed distinct activation areas correlated to the muscles recruited during each movement. The thenar muscle fiber conduction velocity (CV) was estimated to be at 4.70.3 m/s for all subjects. Hand motions were successfully classified and average accuracy for all subjects was directly related to spatial resolution based on the number of channels used as inputs; ranging from 744% when using only 5 channels and up to 922% when using 41 channels. Temporal-spatial features were shown to provide increased motion-specific accuracy when similar muscles were recruited for different gestures.

CONCLUSIONS

Muscle-specific electrodes were capable of accurately recording HD-EMG signals from intrinsic hand muscles and accurately predicting motion.

SIGNIFICANCE

The muscle-specific electrode arrays could improve electrophysiological research studies using EMG decomposition techniques to assess motor unit activity and in applications involving the analysis of dexterous hand motions.

Learning to teleoperate an upper-limb assistive humanoid robot for bimanual daily-living tasks

Objective.Bimanual humanoid platforms for home assistance are nowadays available, both as academic prototypes and commercially. Although they are usually thought of as daily helpers for non-disabled users, their ability to move around, together with their dexterity, makes them ideal assistive devices for upper-limb disabled persons, too. Indeed, teleoperating a bimanual robotic platform via muscle activation could revolutionize the way stroke survivors, amputees and patients with spinal injuries solve their daily home chores. Moreover, with respect to direct prosthetic control, teleoperation has the advantage of freeing the user from the burden of the prosthesis itself, overpassing several limitations regarding size, weight, or integration, and thus enables a much higher level of functionality.Approach.In this study, nine participants, two of whom suffer from severe upper-limb disabilities, teleoperated a humanoid assistive platform, performing complex bimanual tasks requiring high precision and bilateral arm/hand coordination, simulating home/office chores. A wearable body posture tracker was used for position control of the robotic torso and arms, while interactive machine learning applied to electromyography of the forearms helped the robot to build an increasingly accurate model of the participant's intent over time.Main results.All participants, irrespective of their disability, were uniformly able to perform the demanded tasks. Completion times, subjective evaluation scores, as well as energy- and time- efficiency show improvement over time on short and long term.Significance.This is the first time a hybrid setup, involving myoeletric and inertial measurements, is used by disabled people to teleoperate a bimanual humanoid robot. The proposed setup, taking advantage of interactive machine learning, is simple, non-invasive, and offers a new assistive solution for disabled people in their home environment. Additionnally, it has the potential of being used in several other applications in which fine humanoid robot control is required.

Evaluation of Synergy-Based Hand Gesture Recognition Method Against Force Variation for Robust Myoelectric Control

The non-stationary characteristics of surface electromyography (sEMG) and possible adverse variations in real-world conditions make it still an open challenge to realize robust myoelectric control (MEC) for multifunctional prostheses. Variable muscle contraction level is one of the handicaps that may degrade the performance of MEC. In this study, we proposed a force-invariant intent recognition method based on muscle synergy analysis (MSA) in the setting of three self-defined force levels (low, medium, and high). Specifically, a fast matrix factorization algorithm based on alternating non-negativity constrained least squares (NMF/ANLS) was chosen to extract task-specific synergies associated with each of six hand gestures in the training stage; while for the testing samples, we used the non-negative least square (NNLS) method to estimate neural commands for movement classification. The performance of proposed method was compared with conventional pattern recognition (PR) method consisting of LDA (linear discrimination analysis) classifier and representative features in three offline evaluation scenarios. Statistical tests on ten able-bodied subjects revealed no significant difference in intra-force-level (p = 0.353) and multi-force-level (p = 0.695) accuracy; But the synergy-based method performed significantly better than conventional PR-based method under inter-force-level conditions (p < 0.05). Similar results were observed for nine amputee subjects though there was a drop in the classification accuracy. This study was the first to concurrently demonstrate the robustness and predictive power of task-specific synergies under variant force levels and explore their potential for reliable intent recognition against force variation. Although the online performance is yet to be demonstrated, the proposed method is characterized by simple training procedure and acceptable computational efficiency, which would potentially provide an alternative approach for the development of clinically viable prostheses and rehabilitation robots driven by sEMG.

Design and Validation of a Sensor Fault-Tolerant Module for Real-Time High-Density EMG Pattern Recognition

With the advancements in electronics technology, high-density (HD) EMG sensing systems have become available and have been investigated for their feasibility and performance in neural-machine interface (NMI) applications. Comparing to the traditional single channel-based targeted muscle sensing method, HD EMG sensing performs a sampling of the electrical activity over a larger surface area and has the promise of 1) providing richer neural information from one temporal and two spatial dimensions and 2) ease of wear in real life without the need of anatomically targeted electrode placement. To use HD EMG in real-time NMI applications, challenges including high computational burden and unreliability of EMG recordings over time need to be addressed. This paper presented an HD EMG PR based NMI which seamlessly integrates HD EMG PR with a Sensor Fault-Tolerant Module (SFTM) which aimed to provide robust PR in real time. Experimental results showed that the SFTM was able to recover the PR accuracies by 6%-22% from disturbances including contact artifacts and loose contacts. A Python-based implementation of the proposed HD EMG SFTM was developed and was demonstrated to be computationally efficient for real-time performance. These results have demonstrated the feasibility of a robust real-time HD EMG PR-based NMI.

Towards optimizing electrode configurations for silent speech recognition based on high-density surface electromyography

Objective. Silent speech recognition (SSR) based on surface electromyography (sEMG) is an attractive non-acoustic modality of human-machine interfaces that convert the neuromuscular electrophysiological signals into computer-readable textual messages. The speaking process involves complex neuromuscular activities spanning a large area over the facial and neck muscles, thus the locations of the sEMG electrodes considerably affected the performance of the SSR system. However, most of the previous studies used only a quite limited number of electrodes that were placed empirically without prior quantitative analysis, resulting in uncertainty and unreliability of the SSR outcomes. Approach. In this study, the technique of high-density sEMG was proposed to provide a full representation of the articulatory muscle activities so that the optimal electrode configuration for SSR could be systemically explored. A total of 120 closely spaced electrodes were placed on the facial and neck muscles to collect the high-density sEMG signals for classifying ten digits (0–9) silently spoken in both English and Chinese. The sequential forward selection algorithm was adopted to explore the optimal electrodes configurations. Main Results. The results showed that the classification accuracy increased rapidly and became saturated quickly when the number of selected electrodes increased from 1 to 120. Using only ten optimal electrodes could achieve a classification accuracy of 86% for English and 94% for Chinese, whereas as many as 40 non-optimized electrodes were required to obtain comparable accuracies. Also, the optimally selected electrodes seemed to be mostly distributed on the neck instead of the facial region, and more electrodes were required for English recognition to achieve the same accuracy. Significance. The findings of this study can provide useful guidelines about electrode placement for developing a clinically feasible SSR system and implementing a promising approach of human-machine interface, especially for patients with speaking difficulties.

Shoulder kinematics plus contextual target information enable control of multiple distal joints of a simulated prosthetic arm and hand

Background

Prosthetic restoration of reach and grasp function after a trans-humeral amputation requires control of multiple distal degrees of freedom in elbow, wrist and fingers. However, such a high level of amputation reduces the amount of available myoelectric and kinematic information from the residual limb.

Methods

To overcome these limits, we added contextual information about the target’s location and orientation such as can now be extracted from gaze tracking by computer vision tools. For the task of picking and placing a bottle in various positions and orientations in a 3D virtual scene, we trained artificial neural networks to predict postures of an intact subject’s elbow, forearm and wrist (4 degrees of freedom) either solely from shoulder kinematics or with additional knowledge of the movement goal. Subjects then performed the same tasks in the virtual scene with distal joints predicted from the context-aware network.

Results

Average movement times of 1.22s were only slightly longer than the naturally controlled movements (0.82 s). When using a kinematic-only network, movement times were much longer (2.31s) and compensatory movements from trunk and shoulder were much larger. Integrating contextual information also gave rise to motor synergies closer to natural joint coordination.

Conclusions

Although notable challenges remain before applying the proposed control scheme to a real-world prosthesis, our study shows that adding contextual information to command signals greatly improves prediction of distal joint angles for prosthetic control.

Dimensionality analysis of forearm muscle activation for myoelectric control in transradial amputees

Establishing a natural communication interface between the user and the terminal device is one of the central challenges of hand neuroprosthetics research. Surface electromyography (EMG) is the most common source of neural signals for interpreting a user’s intent in these interfaces. However, how the capacity of EMG generation is affected by various clinical parameters remains largely unknown. In this study, we examined the EMG activity of forearm muscles recorded from 11 transradially amputated subjects who performed a wide range of movements. EMG recordings from 40 able-bodied subjects were also analyzed to provide comparative benchmarks. By using non-negative matrix factorization, we extracted the synergistic EMG patterns for each subject to estimate the dimensionality of muscle control, under the framework of motor synergies. We found that amputees exhibited less than four synergies (with substantial variability related to the length of remaining limb and age), whereas able-bodied subjects commonly demonstrate five or more synergies. The results of this study provide novel insight into the muscle synergy framework and the design of natural myoelectric control interfaces.

Development of a neural network based control algorithm for powered ankle prosthesis

Lower limb amputation is partial or complete removal of the limb due to disease, accident or trauma. Surface electromyograms (sEMG) of a large number of muscles and force sensors have been used to develop control algorithms for lower limb powered prostheses, but there are no commercial sEMG controlled prostheses available to date. Unlike ankle disarticulation, transtibial amputation yields less intact lower leg muscle mass. Therefore, minimizing the use of sEMG muscle sources utilized will make powered prosthesis controller economic, and limiting the use of specifically the lower leg muscles will make it flexible. Presently, we have used healthy population data to (1) test the feasibility of the neural network (NN) approach for developing a powered ankle prosthesis control algorithm that successfully predicts sagittal ankle angle and moment during walking using exclusively sEMG, and (2) rank all muscle combination variations according to their success to determine the economic and flexible NN's. sEMG amplitudes of five lower extremity muscles were used as inputs: the tibialis anterior (TA), medial gastrocnemius (MG), rectus femoris (RF), biceps femoris (BF) and gluteus maximus (GM). A time-delay feed-forward-multilayer-architecture NN algorithm was developed. Muscle combination variations were ranked using Pearson's correlation coefficient (r > 0.95 indicates successful correlations) and root-mean-square error between actual vs. estimated ankle position and moment. The trained NN TA + MG was successful (r_position = 0.952, r_moment = 0.997) whereas, TA + MG + BF (r_position = 0.981, r_moment = 0.996) and MG + BF + GM (r_position = 0.955, r_moment = 0.995) were distinguished as the economic and flexible variations, respectively. The algorithms developed should be trained and tested for data acquired from amputees in new studies.

On the Classification of Electromyography Signals to Control a Four Degree-Of-Freedom Prosthetic Device

This study investigates the applicability of Electromyography (EMG) signal classification algorithms with relatively low training time to control prosthetic devices. The perceived quality of control depends on many factors, such as the 1) accuracy of the algorithm, 2) the complexity of the control, and 3) the ability to compensate for the error. The high granularity of control in the time domain reduces the perceived effect of error but also limits the classification accuracy. This work aims to find the borderline for the accuracy of algorithms to be selected as a control strategy for hand prosthetic devices and thus shorten the gap between laboratory devices and commercially available devices. In particular, we compared five classification algorithms and selected one for real-time testing. The results from a test conducted on four subjects showed that the EMG-based control strategy has comparable performances with an IMU-based controller.

Robust Simultaneous and Proportional Myoelectric Control Scheme for Individuals with Transradial Amputations

Commercial myoelectric prostheses need to be accurate and clinically viable to be successful. This study proposed a simultaneous and proportional control scheme with frequency division technique (SPEC-FDT) to address limitations in current myoelectric prosthesis control, specifically to address non-stationaries such as contraction level variations and unintended activations. Twenty able-bodied participants (14 males and 6 females, age 23.4 ± 3.0) and four individuals with transradial amputations performed wrist movements (flexion/extension, rotations and combined movements) in two degrees-of freedom virtual tasks. The SPEC-FDT had a completion rate (CR)>90% for both control and clinical participants which was significantly higher than the conventional technique (CR=68%). Our results showed that SPEC-FDT is highly accurate for both able-bodied and clinical participants and provides a robust myoelectric control scheme allowing for increased prosthetic hand functions.

A Review of Sensory Feedback in Upper-Limb Prostheses From the Perspective of Human Motor Control

This manuscript reviews historical and recent studies that focus on supplementary sensory feedback for use in upper limb prostheses. It shows that the inability of many studies to speak to the issue of meaningful performance improvements in real-life scenarios is caused by the complexity of the interactions of supplementary sensory feedback with other types of feedback along with other portions of the motor control process. To do this, the present manuscript frames the question of supplementary feedback from the perspective of computational motor control, providing a brief review of the main advances in that field over the last 20 years. It then separates the studies on the closed-loop prosthesis control into distinct categories, which are defined by relating the impact of feedback to the relevant components of the motor control framework, and reviews the work that has been done over the last 50+ years in each of those categories. It ends with a discussion of the studies, along with suggestions for experimental construction and connections with other areas of research, such as machine learning.

A Myoelectric Control Interface for Upper-Limb Robotic Rehabilitation Following Spinal Cord Injury

Spinal cord injury (SCI) is a widespread, life-altering injury leading to impairment of sensorimotor function that, while once thought to be permanent, is now being treated with the hope of one day being able to restore function. Surface electromyography (EMG) presents an opportunity to examine and promote human engagement at the neuromuscular level, enabling new protocols for intervention that could be combined with robotic rehabilitation, particularly when robot motion or force sensing may be unusable due to the user's impairment. In this paper, a myoelectric control interface to an exoskeleton for the elbow and wrist was evaluated on a population of ten able-bodied participants and four individuals with cervical-level SCI. The ability of an EMG classifier to discern intended direction of motion in single-degree-of-freedom (DoF) and multi-DoF control modes was assessed for usability in a therapy-like setting. The classifier demonstrated high accuracy for able-bodied participants (averages over 99% for single-DoF and near 90% for multi-DoF), and performance in the SCI group was promising, warranting further study (averages ranging from 85% to 95% for single-DoF, and variable multi-DoF performance averaging around 60%). These results are encouraging for the future use of myoelectric interfaces in robotic rehabilitation for SCI.

Online Natural Myocontrol of Combined Hand and Wrist Actions Using Tactile Myography and the Biomechanics of Grasping

Objective: Despite numerous recent advances in the field of rehabilitation robotics, simultaneous, and proportional control of hand and/or wrist prostheses is still unsolved. In this work we concentrate on myocontrol of combined actions, for instance power grasping while rotating the wrist, by only using training data gathered from single actions. This is highly desirable since gathering data for all possible combined actions would be unfeasibly long and demanding for the amputee. Approach: We first investigated physiologically feasible limits for muscle activation during combined actions. Using these limits we involved 12 intact participants and one amputee in a Target Achievement Control test, showing that tactile myography, i.e., high-density force myography, solves the problem of combined actions to a remarkable extent using simple linear regression. Since real-time usage of many sensors can be computationally demanding, we compare this approach with another one using a reduced feature set. These reduced features are obtained using a fast, spatial first-order approximation of the sensor values. Main results: By using the training data of single actions only, i.e., power grasp or wrist movements, subjects achieved an average success rate of 70.0% in the target achievement test using ridge regression. When combining wrist actions, e.g., pronating and flexing the wrist simultaneously, similar results were obtained with an average of 68.1%. If a power grasp is added to the pool of actions, combined actions are much more difficult to achieve (36.1%). Significance: To the best of our knowledge, for the first time, the effectiveness of tactile myography on single and combined actions is evaluated in a target achievement test. The present study includes 3 DoFs control instead of the two generally used in the literature. Additionally, we define a set of physiologically plausible muscle activation limits valid for most experiments of this kind.

Estimation of User-Applied Isometric Force/Torque Using Upper Extremity Force Myography

Hand force estimation is critical for applications that involve physical human-machine interactions for force monitoring and machine control. Force Myography (FMG) is a potential technique to be used for estimating hand force/torque. The FMG signals reflect the volumetric changes in the arm muscles due to muscle contraction or expansion. This paper investigates the feasibility of employing force-sensing resistors (FSRs) worn on the arm to measure the FMG signals for isometric force/torque estimation. Nine participants were recruited in this study and were asked to exert isometric force along three perpendicular axes, torque about the same three axes, and force and torque simultaneously. During the tests, the isometric force and torque were measured using a 6-degree-of-freedom (DoF) (i.e., force in three axes and torque around the same axes) load cell for ground truth labels whereas the FMG signals were recorded using a total number of 60 FSRs, which were embedded into four bands worn on the different locations of the arm. A two-stage regression strategy was employed to enhance the performance of the FMG bands, where three regression algorithms including general regression neural network (GRNN), support vector regression (SVR), and random forest regression (RF) models were employed, respectively, in the first stage and GRNN was used in the second stage. Two cases were considered to explore the performance of the FMG bands in estimating: (1) 3-DoF force and 3-DoF torque at once and (2) 6-DoF force and torque. In addition, the impact of sensor placement and the spatial coverage of FMG measurements were studied. This preliminary investigation demonstrates promising potential of FMG to estimate multi-DoF isometric force/torque. Specifically, R² accuracies of 0.83 for the 3-DoF force, 0.84 for 3-DoF torque, and 0.77 for the combination of force and torque (6-DoF) regressions were obtained using the four bands on the arm in cross-trial evaluation.

Myoelectric Control for Upper Limb Prostheses

State-of-the-art high-end prostheses are electro-mechanically able to provide a great variety of movements. Nevertheless, in order to functionally replace a human limb, it is essential that each movement is properly controlled. This is the goal of prosthesis control, which has become a growing research field in the last decades, with the ultimate goal of reproducing biological limb control. Therefore, exploration and development of prosthesis control are crucial to improve many aspects of an amputee’s life. Nowadays, a large divergence between academia and industry has become evident in commercial systems. Although several studies propose more natural control systems with promising results, basic one degree of freedom (DoF), a control switching system is the most widely used option in industry because of simplicity, robustness and inertia. A few classification controlled prostheses have emerged in the last years but they are still a low percentage of the used ones. One of the factors that generate this situation is the lack of robustness of more advanced control algorithms in daily life activities outside of laboratory conditions. Because of this, research has shifted towards more functional prosthesis control. This work reviews the most recent literature in upper limb prosthetic control. It covers commonly used variants of possible biological inputs, its processing and translation to actual control, mostly focusing on electromyograms as well as the problems it will have to overcome in near future.

Detection of Participation and Training Task Difficulty Applied to the Multi-Sensor Systems of Rehabilitation Robots

In the process of rehabilitation training for stroke patients, the rehabilitation effect is positively affected by how much physical activity the patients take part in. Most of the signals used to measure the patients' participation are EMG signals or oxygen consumption, which increase the cost and the complexity of the robotic device. In this work, we design a multi-sensor system robot with torque and six-dimensional force sensors to gauge the patients' participation in training. By establishing the static equation of the mechanical leg, the man-machine interaction force of the patient can be accurately extracted. Using the impedance model, the auxiliary force training mode is established, and the difficulty of the target task is changed by adjusting the K value of auxiliary force. Participation models with three intensities were developed offline using support vector machines, for which the C and σ parameters are optimized by the hybrid quantum particle swarm optimization and support vector machines (Hybrid QPSO-SVM) algorithm. An experimental statistical analysis was conducted on ten volunteers' motion representation in different training tasks, which are divided into three stages: over-challenge, challenge, less challenge, by choosing characteristic quantities with significant differences among the various difficulty task stages, as a training set for the support vector machines (SVM). Experimental results from 12 volunteers, with tasks conducted on the lower limb rehabilitation robot LLR-II show that the rehabilitation robot can accurately predict patient participation and training task difficulty. The prediction accuracy reflects the superiority of the Hybrid QPSO-SVM algorithm.

A Modular Transradial Bypass Socket for Surface Myoelectric Prosthetic Control in Non-Amputees

Bypass sockets allow researchers to perform tests of prosthetic systems from the prosthetic user's perspective. We designed a modular upper-limb bypass socket with 3D-printed components that can be easily modified for use with a variety of terminal devices. Our bypass socket preserves access to forearm musculature and the hand, which are necessary for surface electromyography and to provide substituted sensory feedback. Our bypass socket allows a sufficient range of motion to complete tasks in the frontal working area, as measured on non-amputee participants. We examined the performance of non-amputee participants using the bypass socket on the original and modified Box and Block Tests. Participants moved 11.3 ± 2.7 and 11.7 ± 2.4 blocks in the original and modified Box and Block Tests (mean ± SD), respectively, within the range of reported scores using amputee participants. Range of motion for users wearing the bypass socket meets or exceeds most reported range of motion requirements for activities of daily living. The bypass socket was originally designed with a freely rotating wrist; we found that adding elastic resistance to user wrist rotation while wearing the bypass socket had no significant effect on motor decode performance. We have open-sourced the design files and an assembly manual for the bypass socket. We anticipate that the bypass socket will be a useful tool to evaluate and develop sensorized myoelectric prosthesis technology.

Effect of User Practice on Prosthetic Finger Control With an Intuitive Myoelectric Decoder

Machine learning-based myoelectric control is regarded as an intuitive paradigm, because of the mapping it creates between muscle co-activation patterns and prosthesis movements that aims to simulate the physiological pathways found in the human arm. Despite that, there has been evidence that closed-loop interaction with a classification-based interface results in user adaptation, which leads to performance improvement with experience. Recently, there has been a focus shift toward continuous prosthesis control, yet little is known about whether and how user adaptation affects myoelectric control performance in dexterous, intuitive tasks. We investigate the effect of short-term adaptation with independent finger position control by conducting real-time experiments with 10 able-bodied and two transradial amputee subjects. We demonstrate that despite using an intuitive decoder, experience leads to significant improvements in performance. We argue that this is due to the lack of an utterly natural control scheme, which is mainly caused by differences in the anatomy of human and artificial hands, movement intent decoding inaccuracies, and lack of proprioception. Finally, we extend previous work in classification-based and wrist continuous control by verifying that offline analyses cannot reliably predict real-time performance, thereby reiterating the importance of validating myoelectric control algorithms with real-time experiments.

Myoelectric Signal Classification of Targeted Muscles Using Dictionary Learning

Surface electromyography (sEMG) signals comprise electrophysiological information related to muscle activity. As this signal is easy to record, it is utilized to control several myoelectric prostheses devices. Several studies have been conducted to process sEMG signals more efficiently. However, research on optimal algorithms and electrode placements for the processing of sEMG signals is still inconclusive. In addition, very few studies have focused on minimizing the number of electrodes. In this study, we investigated the most effective method for myoelectric signal classification with a small number of electrodes. A total of 23 subjects participated in the study, and the sEMG data of 14 different hand movements of the subjects were acquired from targeted muscles and untargeted muscles. Furthermore, the study compared the classification accuracy of the sEMG data using discriminative feature-oriented dictionary learning (DFDL) and other conventional classifiers. DFDL demonstrated the highest classification accuracy among the classifiers, and its higher quality performance became more apparent as the number of channels decreased. The targeted method was superior to the untargeted method, particularly when classifying sEMG signals with DFDL. Therefore, it was concluded that the combination of the targeted method and the DFDL algorithm could classify myoelectric signals more effectively with a minimal number of channels.

Extraction of Multi-Labelled Movement Information from the Raw HD-sEMG Image with Time-Domain Depth

In contemporary muscle-computer interfaces for upper limb prosthetics there is often a trade-off between control robustness and range of executable movements. As a very low movement error rate is necessary in practical applications, this often results in a quite severe limitation of controllability; a problem growing ever more salient as the mechanical sophistication of multifunctional myoelectric prostheses continues to improve. A possible remedy for this could come from the use of multi-label machine learning methods, where complex movements can be expressed as the superposition of several simpler movements. Here, we investigate this claim by applying a multi-labeled classification scheme in the form of a deep convolutional neural network (CNN) to high density surface electromyography (HD-sEMG) recordings. We use 16 independent labels to model the movements of the hand and forearm state, representing its major degrees of freedom. By training the neural network on 16 × 8 sEMG image sequences 24 samples long with a sampling rate of 2048 Hz to detect these labels, we achieved a mean exact match rate of 78.7% and a mean Hamming loss of 2.9% across 14 healthy test subjects. With this, we demonstrate the feasibility of highly versatile and responsive sEMG control interfaces without loss of accuracy.

Spatial Reorganization of Myoelectric Activities in Extensor Digitorum for Sustained Finger Force Production

The study aims to explore the spatial distribution of multi-tendinous muscle modulated by central nervous system (CNS) during sustained contraction. Nine subjects were recruited to trace constant target forces with right index finger extension. Surface electromyography (sEMG) of extensor digitorum (ED) were recorded with a 32-channel electrode array. Nine successive topographic maps (TM) were obtained. Pixel wise analysis was utilized to extract subtracted topographic maps (STM), which exhibited inhomogeneous distribution. STMs were characterized into hot, warm, and cool regions corresponding to higher, moderate, and lower change ranges, respectively. The relative normalized area (normalized to the first phase) of these regions demonstrated different changing trends as rising, plateauing, and falling over time, respectively. Moreover, the duration of these trends were found to be affected by force level. The rising/falling periods were longer at lower force levels, while the plateau can be achieved from the initial phase for higher force output (45% maximal voluntary contraction). The results suggested muscle activity reorganization in ED plays a role to maintain sustained contraction. Furthermore, the decreased dynamical regulation ability to spatial reorganization may be prone to induce fatigue. This finding implied that spatial reorganization of muscle activity as a regulation mechanism contribute to maintain constant force production.

Adaptive Auto-Regressive Proportional Myoelectric Control

In proportional myographic control, one can control either position or velocity of movement. Here, we propose to use adaptive auto-regressive filters, so as to gradually adjust between the two. We implemented this in an adaptive system with closed-loop feedback, where both the user and the machine simultaneously attempt to follow a cursor on a 2-D arena. We tested this on 15 able-bodied and three limb-deficient participants using an eight-channel myoelectric armband. The human-machine pairs learn to perform smoother cursor movements with a larger range of motion when using the auto-regressive filters, as compared with our previous effortswithmoving-average filters. Importantly, the human-machine system converges to an approximate velocity control strategy resulting in faster and more accuratemovements with lessmuscle effort. The method is not specific tomyoelectriccontroland could be used equally well for motion control using high-dimensional signals from reinnervatedmuscles or direct brain recordings.

Classification and regression of spatio-temporal signals using NeuCube and its realization on SpiNNaker neuromorphic hardware

OBJECTIVE

The objective of this work is to use the capability of spiking neural networks to capture the spatio-temporal information encoded in time-series signals and decode them without the use of hand-crafted features and vector-based learning and the realization of the spiking model on low-power neuromorphic hardware.

APPROACH

The NeuCube spiking model was used to classify different grasp movements directly from raw surface electromyography signals (sEMG), the estimations of the applied finger forces as well as the classification of two motor imagery movements from raw electroencephalography (EEG). In a parallel investigation, the designed spiking decoder was implemented on SpiNNaker neuromorphic hardware, which allows low-energy real-time processing.

MAIN RESULTS

Experimental results reveal a better classification accuracy using the NeuCube model compared to traditional machine learning methods. For sEMG classification, we reached a training accuracy of 85% and a test accuracy of 84.8%, as well as less than 19% of relative root mean square error (rRMSE) when estimating finger forces from six subjects. For the EEG classification, a mean accuracy of 75% was obtained when tested on raw EEG data from nine subjects from the existing 2b dataset from 'BCI competition IV'.

SIGNIFICANCE

This work provides a proof of concept for a successful implementation of the NeuCube spiking model on the SpiNNaker neuromorphic platform for raw sEMG and EEG decoding, which could chart a route ahead for a new generation of portable closed-loop and low-power neuroprostheses.

Robotic interfaces for cognitive psychology and embodiment research: A research roadmap

Advanced human-machine interfaces render robotic devices applicable to study and enhance human cognition. This turns robots into formidable neuroscientific tools to study processes such as the adaptation between a human operator and the operated robotic device and how this adaptation modulates human embodiment and embodied cognition. We analyze bidirectional human-machine interface (bHMI) technologies for transparent information transfer between a human and a robot via efferent and afferent channels. Even if such interfaces have a tremendous positive impact on feedback loops and embodiment, advanced bHMIs face immense technological challenges. We critically discuss existing technical approaches, mainly focusing on haptics, and suggest extensions thereof, which include other aspects of touch. Moreover, we point out other potential constraints such as limited functionality, semi-autonomy, intent-detection, and feedback methods. From this, we develop a research roadmap to guide understanding and development of bidirectional human-machine interfaces that enable robotic experiments to empirically study the human mind and embodiment. We conclude the integration of dexterous control and multisensory feedback to be a promising roadmap towards future robotic interfaces, especially regarding applications in the cognitive sciences. This article is categorized under: Computer Science > Robotics Psychology > Motor Skill and Performance Neuroscience > Plasticity.

Design and effectiveness evaluation of mirror myoelectric interfaces: a novel method to restore movement in hemiplegic patients

The motor impairment occurring after a stroke is characterized by pathological muscle activation patterns or synergies. However, while robot-aided myoelectric interfaces have been proposed for stroke rehabilitation, they do not address this issue, which might result in inefficient interventions. Here, we present a novel paradigm that relies on the correction of the pathological muscle activity as a way to elicit rehabilitation, even in patients with complete paralysis. Previous studies demonstrated that there are no substantial inter-limb differences in the muscle synergy organization of healthy individuals. We propose building a subject-specific model of muscle activity from the healthy limb and mirroring it to use it as a learning tool for the patient to reproduce the same healthy myoelectric patterns on the paretic limb during functional task training. Here, we aim at understanding how this myoelectric model, which translates muscle activity into continuous movements of a 7-degree of freedom upper limb exoskeleton, could transfer between sessions, arms and tasks. The experiments with 8 healthy individuals and 2 chronic stroke patients proved the feasibility and effectiveness of such myoelectric interface. We anticipate the proposed method to become an efficient strategy for the correction of maladaptive muscle activity and the rehabilitation of stroke patients.

Causes of Performance Degradation in Non-invasive Electromyographic Pattern Recognition in Upper Limb Prostheses

Surface Electromyography (EMG)-based pattern recognition methods have been investigated over the past years as a means of controlling upper limb prostheses. Despite the very good reported performance of myoelectric controlled prosthetic hands in lab conditions, real-time performance in everyday life conditions is not as robust and reliable, explaining the limited clinical use of pattern recognition control. The main reason behind the instability of myoelectric pattern recognition control is that EMG signals are non-stationary in real-life environments and present a lot of variability over time and across subjects, hence affecting the system's performance. This can be the result of one or many combined changes, such as muscle fatigue, electrode displacement, difference in arm posture, user adaptation on the device over time and inter-subject singularity. In this paper an extensive literature review is performed to present the causes of the drift of EMG signals, ways of detecting them and possible techniques to counteract for their effects in the application of upper limb prostheses. The suggested techniques are organized in a table that can be used to recognize possible problems in the clinical application of EMG-based pattern recognition methods for upper limb prosthesis applications and state-of-the-art methods to deal with such problems.