Neural interfaces for control of upper limb prostheses: the state of the art and future possibilities

Towards Control of a Transhumeral Prosthesis with EEG Signals

Robotic prostheses are expected to allow amputees greater freedom and mobility. However, available options to control transhumeral prostheses are reduced with increasing amputation level. In addition, for electromyography-based control of prostheses, the residual muscles alone cannot generate sufficiently different signals for accurate distal arm function. Thus, controlling a multi-degree of freedom (DoF) transhumeral prosthesis is challenging with currently available techniques. In this paper, an electroencephalogram (EEG)-based hierarchical two-stage approach is proposed to achieve multi-DoF control of a transhumeral prosthesis. In the proposed method, the motion intention for arm reaching or hand lifting is identified using classifiers trained with motion-related EEG features. For this purpose, neural network and k-nearest neighbor classifiers are used. Then, elbow motion and hand endpoint motion is estimated using a different set of neural-network-based classifiers, which are trained with motion information recorded using healthy subjects. The predictions from the classifiers are compared with residual limb motion to generate a final prediction of motion intention. This can then be used to realize multi-DoF control of a prosthesis. The experimental results show the feasibility of the proposed method for multi-DoF control of a transhumeral prosthesis. This proof of concept study was performed with healthy subjects.

Motor-commands decoding using peripheral nerve signals: a review

During the last few decades, substantial scientific and technological efforts have been focused on the development of neuroprostheses. The major emphasis has been on techniques for connecting the human nervous system with a robotic prosthesis via natural-feeling interfaces. The peripheral nerves provide access to highly processed and segregated neural command signals from the brain that can in principle be used to determine user intent and control muscles. If these signals could be used, they might allow near-natural and intuitive control of prosthetic limbs with multiple degrees of freedom. This review summarizes the history of neuroprosthetic interfaces and their ability to record from and stimulate peripheral nerves. We also discuss the types of interfaces available and their applications, the kinds of peripheral nerve signals that are used, and the algorithms used to decode them. Finally, we explore the prospects for future development in this area.

Case-study of a user-driven prosthetic arm design: bionic hand versus customized body-powered technology in a highly demanding work environment

BACKGROUND

Prosthetic arm research predominantly focuses on "bionic" but not body-powered arms. However, any research orientation along user needs requires sufficiently precise workplace specifications and sufficiently hard testing. Forensic medicine is a demanding environment, also physically, also for non-disabled people, on several dimensions (e.g., distances, weights, size, temperature, time).

METHODS

As unilateral below elbow amputee user, the first author is in a unique position to provide direct comparison of a "bionic" myoelectric iLimb Revolution (Touch Bionics) and a customized body-powered arm which contains a number of new developments initiated or developed by the user: (1) quick lock steel wrist unit; (2) cable mount modification; (3) cast shape modeled shoulder anchor; (4) suspension with a soft double layer liner (Ohio Willowwood) and tube gauze (Molnlycke) combination. The iLimb is mounted on an epoxy socket; a lanyard fixed liner (Ohio Willowwood) contains magnetic electrodes (Liberating Technologies). An on the job usage of five years was supplemented with dedicated and focused intensive two-week use tests at work for both systems.

RESULTS

The side-by-side comparison showed that the customized body-powered arm provides reliable, comfortable, effective, powerful as well as subtle service with minimal maintenance; most notably, grip reliability, grip force regulation, grip performance, center of balance, component wear down, sweat/temperature independence and skin state are good whereas the iLimb system exhibited a number of relevant serious constraints.

CONCLUSIONS

Research and development of functional prostheses may want to focus on body-powered technology as it already performs on manually demanding and heavy jobs whereas eliminating myoelectric technology's constraints seems out of reach. Relevant testing could be developed to help expediting this. This is relevant as Swiss disability insurance specifically supports prostheses that enable actual work integration. Myoelectric and cosmetic arm improvement may benefit from a less forgiving focus on perfecting anthropomorphic appearance.

In vivo biocompatibility of porous and non-porous polypyrrole based trilayered actuators

Trilayered polypyrrole (PPy) actuators have high stress density, low modulus and have wide potential biological applications including use in artificial muscles and in limb prosthesis after limb amputation. This article examines the in vivo biocompatibility of actuators in muscle using rabbit models. The actuators were specially designed with pores to encourage tissue in growth; this study also assessed the effect of such pores on the stability of the actuators in vivo. Trilayered PPy actuators were either laser cut with 150 µm pores or left pore-less and implanted into rabbit muscle for 3 days, 2 weeks, 4 weeks and 8 weeks and retrieved subsequently for histological analysis. In a second set of experiments, the cut edges of pores in porous actuator strips were further sealed by PPy after laser cutting to further improve its stability in vivo. Porous actuators with and without PPy sealing of pore edges were implanted intramuscularly for 4 and 8 weeks and assessed with histology. Pore-less actuators incited a mild inflammatory response, becoming progressively walled off by a thin layer of fibrous tissue. Porous actuators showed increased PPy fragmentation and delamination with associated greater foreign body response compared to pore-less actuators. The PPy fragmentation was minimized when the pore edges were sealed off by PPy after laser cutting showing less PPy debris. Laser cutting of the actuators with pores destabilizes the PPy. This can be overcome by sealing the cut edges of the pores with PPy after laser. The findings in this article have implications in future design and manufacturing of PPy actuator for use in vivo.

Surface EMG signals in very late-stage of Duchenne muscular dystrophy: a case study

Background

Robotic arm supports aim at improving the quality of life for adults with Duchenne muscular dystrophy (DMD) by augmenting their residual functional abilities. A critical component of robotic arm supports is the control interface, as is it responsible for the human-machine interaction. Our previous studies showed the feasibility of using surface electromyography (sEMG) as a control interface to operate robotic arm supports in adults with DMD (22-24 years-old). However, in the biomedical engineering community there is an often raised skepticism on whether adults with DMD at the last stage of their disease have sEMG signals that can be measured and used for control.

Findings

In this study sEMG signals from Biceps and Triceps Brachii muscles were measured for the first time in a 37 year-old man with DMD (Brooke 6) that lost his arm function 15 years ago. The sEMG signals were measured during maximal and sub-maximal voluntary isometric contractions and evaluated in terms of signal-to-noise ratio and co-activation ratio. Beyond the profound deterioration of the muscles, we found that sEMG signals from both Biceps and Triceps muscles were measurable in this individual, although with a maximum signal amplitude 100 times lower compared to sEMG from healthy subjects. The participant was able to voluntarily modulate the required level of muscle activation during the sub-maximal voluntary isometric contractions. Despite the low sEMG amplitude and a considerable level of muscle co-activation, simulations of an elbow orthosis using the measured sEMG as driving signal indicated that the sEMG signals of the participant had the potential to provide control of elbow movements.

Conclusions

To the best of our knowledge this is the first time that sEMG signals from a man with DMD at the last-stage of the disease were measured, analyzed and reported. These findings offer promising perspectives to the use of sEMG as an intuitive and natural control interface for robotic arm supports in adults with DMD until the last stage of the disease.

Electronic supplementary material

The online version of this article (doi:10.1186/s12984-017-0292-4) contains supplementary material, which is available to authorized users.

A passive wrist with switchable stiffness for a body-powered hydraulically actuated hand prosthesis

State of art upper limb prostheses lack several degrees of freedom (DoF) and force the individuals to compensate for them by changing the motions of their arms and body. Such movements often yield to articulation injuries, nonetheless these could be prevented by adding DoFs, for instance, an articulated passive wrist. Available stiff or compliant wrists with passive flexion/extension and/or radial/ulnar deviation are sub-optimal solutions. Indeed, stiff wrists induce the individuals wearing them to perform exaggerated compensatory movements during the reaching phase while compliant wrists proved to be unpractical while manipulating heavy objects. Here we present a wrist capable of combining the benefits of stiff and compliant wrists. It is provided with two switchable levels of passive compliance that are automatically selected. The prototype was integrated in a body-powered hydraulic hand prosthesis and actuated using the same hydraulic circuit of the hand. Detailed analysis of the parameters that affect the compliance, the critical load and the performance of the prosthesis are presented.

Comparison between sEMG and force as control interfaces to support planar arm movements in adults with Duchenne: a feasibility study

BACKGROUND

Adults with Duchenne muscular dystrophy (DMD) can benefit from devices that actively support their arm function. A critical component of such devices is the control interface as it is responsible for the human-machine interaction. Our previous work indicated that surface electromyography (sEMG) and force-based control with active gravity and joint-stiffness compensation were feasible solutions for the support of elbow movements (one degree of freedom). In this paper, we extend the evaluation of sEMG- and force-based control interfaces to simultaneous and proportional control of planar arm movements (two degrees of freedom).

METHODS

Three men with DMD (18-23 years-old) with different levels of arm function (i.e. Brooke scores of 4, 5 and 6) performed a series of line-tracing tasks over a tabletop surface using an experimental active arm support. The arm movements were controlled using three control methods: sEMG-based control, force-based control with stiffness compensation (FSC), and force-based control with no compensation (FNC). The movement performance was evaluated in terms of percentage of task completion, tracing error, smoothness and speed.

RESULTS

For subject S1 (Brooke 4) FNC was the preferred method and performed better than FSC and sEMG. FNC was not usable for subject S2 (Brooke 5) and S3 (Brooke 6). Subject S2 presented significantly lower movement speed with sEMG than with FSC, yet he preferred sEMG since FSC was perceived to be too fatiguing. Subject S3 could not successfully use neither of the two force-based control methods, while with sEMG he could reach almost his entire workspace.

CONCLUSIONS

Movement performance and subjective preference of the three control methods differed with the level of arm function of the participants. Our results indicate that all three control methods have to be considered in real applications, as they present complementary advantages and disadvantages. The fact that the two weaker subjects (S2 and S3) experienced the force-based control interfaces as fatiguing suggests that sEMG-based control interfaces could be a better solution for adults with DMD. Yet force-based control interfaces can be a better alternative for those cases in which voluntary forces are higher than the stiffness forces of the arms.

A murine model of a novel surgical architecture for proprioceptive muscle feedback and its potential application to control of advanced limb prostheses

OBJECTIVE

Proprioceptive mechanisms play a critical role in both reflexive and volitional lower extremity control. Significant strides have been made in the development of bionic limbs that are capable of bi-directional communication with the peripheral nervous system, but none of these systems have been capable of providing physiologically-relevant muscle-based proprioceptive feedback through natural neural pathways. In this study, we present the agonist-antagonist myoneural interface (AMI), a surgical approach with the capacity to provide graded kinesthetic feedback from a prosthesis through mechanical activation of native mechanoreceptors within residual agonist-antagonist muscle pairs.

APPROACH

(1) Sonomicrometery and electroneurography measurement systems were validated using a servo-based muscle tensioning system. (2) A heuristic controller was implemented to modulate functional electrical stimulation of an agonist muscle, using sonomicrometric measurements of stretch from a mechanically-coupled antagonist muscle as feedback. (3) One AMI was surgically constructed in the hindlimb of each rat. (4) The gastrocnemius-soleus complex of the rat was cycled through a series of ramp-and-hold stretches in two different muscle architectures: native (physiologically-intact) and AMI (modified). Integrated electroneurography from the tibial nerve was compared across the two architectures.

MAIN RESULTS

Correlation between stretch and afferent signal demonstrated that the AMI is capable of provoking graded afferent signals in response to ramp-and-hold stretches, in a manner similar to the native muscle architecture. The response magnitude in the AMI was reduced when compared to the native architecture, likely due to lower stretch amplitudes. The closed-loop control system showed robustness at high stretch magnitudes, with some oscillation at low stretch magnitudes.

SIGNIFICANCE

These results indicate that the AMI has the potential to communicate meaningful kinesthetic feedback from a prosthetic limb by replicating the agonist-antagonist relationships that are fundamental to physiological proprioception.

Hybrid EEG-fNIRS-Based Eight-Command Decoding for BCI: Application to Quadcopter Control

In this paper, a hybrid electroencephalography–functional near-infrared spectroscopy (EEG–fNIRS) scheme to decode eight active brain commands from the frontal brain region for brain–computer interface is presented. A total of eight commands are decoded by fNIRS, as positioned on the prefrontal cortex, and by EEG, around the frontal, parietal, and visual cortices. Mental arithmetic, mental counting, mental rotation, and word formation tasks are decoded with fNIRS, in which the selected features for classification and command generation are the peak, minimum, and mean ΔHbO values within a 2-s moving window. In the case of EEG, two eyeblinks, three eyeblinks, and eye movement in the up/down and left/right directions are used for four-command generation. The features in this case are the number of peaks and the mean of the EEG signal during 1 s window. We tested the generated commands on a quadcopter in an open space. An average accuracy of 75.6% was achieved with fNIRS for four-command decoding and 86% with EEG for another four-command decoding. The testing results show the possibility of controlling a quadcopter online and in real-time using eight commands from the prefrontal and frontal cortices via the proposed hybrid EEG–fNIRS interface.

A motion-classification strategy based on sEMG-EEG signal combination for upper-limb amputees

Background

Most of the modern motorized prostheses are controlled with the surface electromyography (sEMG) recorded on the residual muscles of amputated limbs. However, the residual muscles are usually limited, especially after above-elbow amputations, which would not provide enough sEMG for the control of prostheses with multiple degrees of freedom. Signal fusion is a possible approach to solve the problem of insufficient control commands, where some non-EMG signals are combined with sEMG signals to provide sufficient information for motion intension decoding. In this study, a motion-classification method that combines sEMG and electroencephalography (EEG) signals were proposed and investigated, in order to improve the control performance of upper-limb prostheses.

Methods

Four transhumeral amputees without any form of neurological disease were recruited in the experiments. Five motion classes including hand-open, hand-close, wrist-pronation, wrist-supination, and no-movement were specified. During the motion performances, sEMG and EEG signals were simultaneously acquired from the skin surface and scalp of the amputees, respectively. The two types of signals were independently preprocessed and then combined as a parallel control input. Four time-domain features were extracted and fed into a classifier trained by the Linear Discriminant Analysis (LDA) algorithm for motion recognition. In addition, channel selections were performed by using the Sequential Forward Selection (SFS) algorithm to optimize the performance of the proposed method.

Results

The classification performance achieved by the fusion of sEMG and EEG signals was significantly better than that obtained by single signal source of either sEMG or EEG. An increment of more than 14% in classification accuracy was achieved when using a combination of 32-channel sEMG and 64-channel EEG. Furthermore, based on the SFS algorithm, two optimized electrode arrangements (10-channel sEMG + 10-channel EEG, 10-channel sEMG + 20-channel EEG) were obtained with classification accuracies of 84.2 and 87.0%, respectively, which were about 7.2 and 10% higher than the accuracy by using only 32-channel sEMG input.

Conclusions

This study demonstrated the feasibility of fusing sEMG and EEG signals towards improving motion classification accuracy for above-elbow amputees, which might enhance the control performances of multifunctional myoelectric prostheses in clinical application.

Trial registration

The study was approved by the ethics committee of Institutional Review Board of Shenzhen Institutes of Advanced Technology, and the reference number is SIAT-IRB-150515-H0077.

Clinical applications of penetrating neural interfaces and Utah Electrode Array technologies

This paper briefly describes some of the recent progress in the development of penetrating microelectrode arrays and highlights the use of two of these devices, Utah electrode arrays and Utah slanted electrode arrays, in two therapeutic interventions: recording volitional skeletal motor commands from the central nervous system, and recording motor commands and evoking somatosensory percepts in the peripheral nervous system (PNS). The paper also briefly explores other potential sites for microelectrode array interventions that could be profitably pursued and that could have important consequences in enhancing the quality of life of patients that has been compromised by disorders of the central and PNSs.

Independent Long Fingers are not Essential for a Grasping Hand

The human hand is a complex integrated system with motor and sensory components that provides individuals with high functionality and elegant behaviour. In direct connection with the brain, the hand is capable of performing countless actions ranging from fine digit manipulation to the handling of heavy objects. However the question of which movements mostly contribute to the manipulation skills of the hand, and thus should be included in prosthetic hands, is yet to be answered. Building from our previous work, and assuming that a hand with independent long fingers allowed performance comparable to a hand with coupled fingers, here we explored the actual contribution of independent fingers while performing activities of daily living using custom built orthoses. Our findings show that, when an opposable thumb is present, independent long fingers provide a measureable advantage in performing activities of daily living only when precision grasps are involved. In addition, the results suggest that the remarkable grasping skills of the human hand rely more on the independent abduction/adduction of the fingers than on their independent flexion/extension. These findings are of interest to the designers of artificial hands, including biomimetic prostheses and exoskeletons.

Development of a Regenerative Peripheral Nerve Interface for Control of a Neuroprosthetic Limb

Background. The purpose of this experiment was to develop a peripheral nerve interface using cultured myoblasts within a scaffold to provide a biologically stable interface while providing signal amplification for neuroprosthetic control and preventing neuroma formation. Methods. A Regenerative Peripheral Nerve Interface (RPNI) composed of a scaffold and cultured myoblasts was implanted on the end of a divided peroneal nerve in rats (n = 25). The scaffold material consisted of either silicone mesh, acellular muscle, or acellular muscle with chemically polymerized poly(3,4-ethylenedioxythiophene) conductive polymer. Average implantation time was 93 days. Electrophysiological tests were performed at endpoint to determine RPNI viability and ability to transduce neural signals. Tissue samples were examined using both light microscopy and immunohistochemistry. Results. All implanted RPNIs, regardless of scaffold type, remained viable and displayed robust vascularity. Electromyographic activity and stimulated compound muscle action potentials were successfully recorded from all RPNIs. Physiologic efferent motor action potentials were detected from RPNIs in response to sensory foot stimulation. Histology and transmission electron microscopy revealed mature muscle fibers, axonal regeneration without neuroma formation, neovascularization, and synaptogenesis. Desmin staining confirmed the preservation and maturation of myoblasts within the RPNIs. Conclusions. RPNI demonstrates significant myoblast maturation, innervation, and vascularization without neuroma formation.

Literature Review on Needs of Upper Limb Prosthesis Users

The loss of one hand can significantly affect the level of autonomy and the capability of performing daily living, working and social activities. The current prosthetic solutions contribute in a poor way to overcome these problems due to limitations in the interfaces adopted for controlling the prosthesis and to the lack of force or tactile feedback, thus limiting hand grasp capabilities. This paper presents a literature review on needs analysis of upper limb prosthesis users, and points out the main critical aspects of the current prosthetic solutions, in terms of users satisfaction and activities of daily living they would like to perform with the prosthetic device. The ultimate goal is to provide design inputs in the prosthetic field and, contemporary, increase user satisfaction rates and reduce device abandonment. A list of requirements for upper limb prostheses is proposed, grounded on the performed analysis on user needs. It wants to (i) provide guidelines for improving the level of acceptability and usefulness of the prosthesis, by accounting for hand functional and technical aspects; (ii) propose a control architecture of PNS-based prosthetic systems able to satisfy the analyzed user wishes; (iii) provide hints for improving the quality of the methods (e.g., questionnaires) adopted for understanding the user satisfaction with their prostheses.

Control of Prosthetic Hands via the Peripheral Nervous System

This paper intends to provide a critical review of the literature on the technological issues on control and sensorization of hand prostheses interfacing with the Peripheral Nervous System (i.e., PNS), and their experimental validation on amputees. The study opens with an in-depth analysis of control solutions and sensorization features of research and commercially available prosthetic hands. Pros and cons of adopted technologies, signal processing techniques and motion control solutions are investigated. Special emphasis is then dedicated to the recent studies on the restoration of tactile perception in amputees through neural interfaces. The paper finally proposes a number of suggestions for designing the prosthetic system able to re-establish a bidirectional communication with the PNS and foster the prosthesis natural control.

The Evolution of Neuroprosthetic Interfaces

The ideal neuroprosthetic interface permits high-quality neural recording and stimulation of the nervous system while reliably providing clinical benefits over chronic periods. Although current technologies have made notable strides in this direction, significant improvements must be made to better achieve these design goals and satisfy clinical needs. This article provides an overview of the state of neuroprosthetic interfaces, starting with the design and placement of these interfaces before exploring the stimulation and recording platforms yielded from contemporary research. Finally, we outline emerging research trends in an effort to explore the potential next generation of neuroprosthetic interfaces.

Multi-walled carbon nanotubes change morpho-functional and GABA characteristics of mouse cortical astrocytes

BACKGROUND

Multi-walled carbon nanotubes (MW-CNTs) have been extensively explored for their possible beneficial use in the nervous system. CNTs have shown to modulate neuronal growth and electrical properties, but its effect that varying length of MW-CNTs on primary astrocyte roles have not been clearly demonstrated yet.

RESULTS

We investigate here the effect of MW-CNTs on astrocytic morphology, cell-cell interaction and the distribution of intracellular GABA (gamma-amino butyric acid). Primary cultured cortical astrocytes on MW-CNT-coated glass coverslips grow rounder and make more cell-cell interactions, with many cell processes, compared to astrocytes on poly-D-lysine (PDL) coverslips. In addition, intracellular GABA spreads into the cell processes of astrocytes on MW-CNT coverslips. When this GABA spreads into cell processes from the cell body GABA can be released more easily and in larger quantities compared to astrocytes on PDL coverslips.

CONCLUSIONS

Our result confirm that MW-CNTs modulate astrocytic morphology, the distribution of astrocytic GABA, cell-cell interactions and the extension of cell processes. CNTs look to be a promising material for use neuroprosthetics such as brain-machine interface technologies.

In Vivo Electrochemical Analysis of a PEDOT/MWCNT Neural Electrode Coating

Neural electrodes hold tremendous potential for improving understanding of brain function and restoring lost neurological functions. Multi-walled carbon nanotube (MWCNT) and dexamethasone (Dex)-doped poly(3,4-ethylenedioxythiophene) (PEDOT) coatings have shown promise to improve chronic neural electrode performance. Here, we employ electrochemical techniques to characterize the coating in vivo. Coated and uncoated electrode arrays were implanted into rat visual cortex and subjected to daily cyclic voltammetry (CV) and electrochemical impedance spectroscopy (EIS) for 11 days. Coated electrodes experienced a significant decrease in 1 kHz impedance within the first two days of implantation followed by an increase between days 4 and 7. Equivalent circuit analysis showed that the impedance increase is the result of surface capacitance reduction, likely due to protein and cellular processes encapsulating the porous coating. Coating's charge storage capacity remained consistently higher than uncoated electrodes, demonstrating its in vivo electrochemical stability. To decouple the PEDOT/MWCNT material property changes from the tissue response, in vitro characterization was conducted by soaking the coated electrodes in PBS for 11 days. Some coated electrodes exhibited steady impedance while others exhibiting large increases associated with large decreases in charge storage capacity suggesting delamination in PBS. This was not observed in vivo, as scanning electron microscopy of explants verified the integrity of the coating with no sign of delamination or cracking. Despite the impedance increase, coated electrodes successfully recorded neural activity throughout the implantation period.

High accuracy decoding of user intentions using EEG to control a lower-body exoskeleton

Brain-Machine Interface (BMI) systems allow users to control external mechanical systems using their thoughts. Commonly used in literature are invasive techniques to acquire brain signals and decode user's attempted motions to drive these systems (e.g. a robotic manipulator). In this work we use a lower-body exoskeleton and measure the users brain activity using non-invasive electroencephalography (EEG). The main focus of this study is to decode a paraplegic subject's motion intentions and provide him with the ability of walking with a lower-body exoskeleton accordingly. We present our novel method of decoding with high offline evaluation accuracies (around 98%), our closed loop implementation structure with considerably short on-site training time (around 38 sec), and preliminary results from the real-time closed loop implementation (NeuroRex) with a paraplegic test subject.

A lower limb exoskeleton control system based on steady state visual evoked potentials

OBJECTIVE

We have developed an asynchronous brain-machine interface (BMI)-based lower limb exoskeleton control system based on steady-state visual evoked potentials (SSVEPs).

APPROACH

By decoding electroencephalography signals in real-time, users are able to walk forward, turn right, turn left, sit, and stand while wearing the exoskeleton. SSVEP stimulation is implemented with a visual stimulation unit, consisting of five light emitting diodes fixed to the exoskeleton. A canonical correlation analysis (CCA) method for the extraction of frequency information associated with the SSVEP was used in combination with k-nearest neighbors.

MAIN RESULTS

Overall, 11 healthy subjects participated in the experiment to evaluate performance. To achieve the best classification, CCA was first calibrated in an offline experiment. In the subsequent online experiment, our results exhibit accuracies of 91.3 ± 5.73%, a response time of 3.28 ± 1.82 s, an information transfer rate of 32.9 ± 9.13 bits/min, and a completion time of 1100 ± 154.92 s for the experimental parcour studied.

SIGNIFICANCE

The ability to achieve such high quality BMI control indicates that an SSVEP-based lower limb exoskeleton for gait assistance is becoming feasible.

Combined use of sEMG and accelerometer in hand motion classification considering forearm rotation

Hand motion classification using surface electromyography (sEMG) has been widely studied for its applications in upper-limb prosthesis and human-machine interface etc. Pattern-recognition based control methods have many advantages, and the reported classification accuracy can meet the requirements of practical applications. However, the pattern instability of sEMG in actual use limited their real implementations, and limb position variations may be one of the potential factors. In this paper, we give a pilot study of the reverse effect of forearm rotations on hand motion classification, and the results show that the forearm rotations can substantially degrade the classifier's performance: the average intra-position error is only 2.4%, but the average interposition classification error is as high as 44.0%. To solve this problem, we use an extra accelerometer to estimate the forearm rotation angles, and the best combination of sEMG data and accelerometer outputs can reduce the average classification error to 3.3%.

Global cortical activity predicts shape of hand during grasping

Recent studies show that the amplitude of cortical field potentials is modulated in the time domain by grasping kinematics. However, it is unknown if these low frequency modulations persist and contain enough information to decode grasp kinematics in macro-scale activity measured at the scalp via electroencephalography (EEG). Further, it is unclear as to whether joint angle velocities or movement synergies are the optimal kinematics spaces to decode. In this offline decoding study, we infer from human EEG, hand joint angular velocities as well as synergistic trajectories as subjects perform natural reach-to-grasp movements. Decoding accuracy, measured as the correlation coefficient (r) between the predicted and actual movement kinematics, was r = 0.49 ± 0.02 across 15 hand joints. Across the first three kinematic synergies, decoding accuracies were r = 0.59 ± 0.04, 0.47 ± 0.06, and 0.32 ± 0.05. The spatial-temporal pattern of EEG channel recruitment showed early involvement of contralateral frontal-central scalp areas followed by later activation of central electrodes over primary sensorimotor cortical areas. Information content in EEG about the grasp type peaked at 250 ms after movement onset. The high decoding accuracies in this study are significant not only as evidence for time-domain modulation in macro-scale brain activity, but for the field of brain-machine interfaces as well. Our decoding strategy, which harnesses the neural “symphony” as opposed to local members of the neural ensemble (as in intracranial approaches), may provide a means of extracting information about motor intent for grasping without the need for penetrating electrodes and suggests that it may be soon possible to develop non-invasive neural interfaces for the control of prosthetic limbs.

Development of a control system for artificially rehabilitated limbs: a review

Development of an advanced control system for prostheses (artificial limbs) is necessary to provide functionality, effectiveness, and preferably the feeling of a sound living limb. The development of the control system has introduced varieties of control strategies depending on the application. This paper reviews some control systems used for prosthetics, orthotics, and exoskeletons. The advantages and limitations of different control systems for particular applications have been discussed and presented in a comparative manner to help in deciding the appropriate method for pertinent application.

Neural stimulation and recording with bidirectional, soft carbon nanotube fiber microelectrodes

The development of microelectrodes capable of safely stimulating and recording neural activity is a critical step in the design of many prosthetic devices, brain-machine interfaces, and therapies for neurologic or nervous-system-mediated disorders. Metal electrodes are inadequate prospects for the miniaturization needed to attain neuronal-scale stimulation and recording because of their poor electrochemical properties, high stiffness, and propensity to fail due to bending fatigue. Here we demonstrate neural recording and stimulation using carbon nanotube (CNT) fiber electrodes. In vitro characterization shows that the tissue contact impedance of CNT fibers is remarkably lower than that of state-of-the-art metal electrodes, making them suitable for recording single-neuron activity without additional surface treatments. In vivo chronic studies in parkinsonian rodents show that CNT fiber microelectrodes stimulate neurons as effectively as metal electrodes with 10 times larger surface area, while eliciting a significantly reduced inflammatory response. The same CNT fiber microelectrodes can record neural activity for weeks, paving the way for the development of novel multifunctional and dynamic neural interfaces with long-term stability.

Effects of kinematic vibrotactile feedback on learning to control a virtual prosthetic arm

Background

After a limb is lost a prosthesis can restore function. For maximum utility, prosthetic limbs should accept movement commands and provide force and motion feedback, which can be conveyed with vibrotactile feedback (VIBF). While prior studies have shown that force-based VIBF benefits control, the merits of motion-based VIBF are unclear. Our goal was to clarify the effectiveness of position- and velocity-based VIBF for prosthetic arm control.

Methods

Healthy adults with normal limb function practiced a goal-directed task with a virtual myoelectric prosthetic arm. A linear resonant actuator on the wrist provided VIBF. Two groups with nine subjects each received amplitude modulated VIBF in addition to visual feedback while practicing the task. In one group, the VIBF was proportional to the virtual arm’s position, and in the other group, velocity. A control group of nine subjects received only visual feedback. Subjects practiced for 240 trials, followed by 180 trials with feedback manipulations for the VIBF groups. Performance was characterized by end-point error, movement time, and a composite skill measure that combined these quantities. A second experiment with a new group of five subjects assessed discrimination capabilities between different position- and velocity-based VIBF profiles.

Results

With practice all groups improved their skill in controlling the virtual prosthetic arm. Subjects who received additional position- and velocity-based VIBF learned at the same rate as the control group, who received only visual feedback (learning rate time constant: about 40 trials). When visual feedback was subsequently removed leaving only VIBF, performance was no better than with no feedback at all. When VIBF was removed leaving only visual feedback, about half of the participants performed better, instead of worse. The VIBF discrimination tests showed that subjects could detect virtual arm angular position and velocity differences of about 5 deg and 20 deg/s, respectively.

Conclusions

Kinematic VIBF did not increase the rate of skill acquisition or improve performance when controlling a virtual myoelectric prosthetic arm, whether provided in isolation or coupled with visual feedback. VIBF had a deleterious effect on performance for some individuals, who may have had difficulty integrating kinematic VIBF information into their control strategies.

Development of a prototype over-actuated biomimetic prosthetic hand

The loss of a hand can greatly affect quality of life. A prosthetic device that can mimic normal hand function is very important to physical and mental recuperation after hand amputation, but the currently available prosthetics do not fully meet the needs of the amputee community. Most prosthetic hands are not dexterous enough to grasp a variety of shaped objects, and those that are tend to be heavy, leading to discomfort while wearing the device. In order to attempt to better simulate human hand function, a dexterous hand was developed that uses an over-actuated mechanism to form grasp shape using intrinsic joint mounted motors in addition to a finger tendon to produce large flexion force for a tight grip. This novel actuation method allows the hand to use small actuators for grip shape formation, and the tendon to produce high grip strength. The hand was capable of producing fingertip flexion force suitable for most activities of daily living. In addition, it was able to produce a range of grasp shapes with natural, independent finger motion, and appearance similar to that of a human hand. The hand also had a mass distribution more similar to a natural forearm and hand compared to contemporary prosthetics due to the more proximal location of the heavier components of the system. This paper describes the design of the hand and controller, as well as the test results.

Regenerative Engineering and Bionic Limbs

Amputations of the upper extremity are severely debilitating, current treatments support very basic limb movement, and patients undergo extensive physiotherapy and psychological counselling. There is no prosthesis that allows the amputees near-normal function. With increasing number of amputees due to injuries sustained in accidents, natural calamities and international conflicts, there is a growing requirement for novel strategies and new discoveries. Advances have been made in technological, material and in prosthesis integration where researchers are now exploring artificial prosthesis that integrate with the residual tissues and function based on signal impulses received from the residual nerves. Efforts are focused on challenging experts in different disciplines to integrate ideas and technologies to allow for the regeneration of injured tissues, recording on tissue signals and feed-back to facilitate responsive movements and gradations of muscle force. A fully functional replacement and regenerative or integrated prosthesis will rely on interface of biological process with robotic systems to allow individual control of movement such as at the elbow, forearm, digits and thumb in the upper extremity. Regenerative engineering focused on the regeneration of complex tissue and organ systems will be realized by the cross-fertilization of advances over the past thirty years in the fields of tissue engineering, nanotechnology, stem cell science, and developmental biology. The convergence of toolboxes crated within each discipline will allow interdisciplinary teams from engineering, science, and medicine to realize new strategies, mergers of disparate technologies, such as biophysics, smart bionics, and the healing power of the mind. Tackling the clinical challenges, interfacing the biological process with bionic technologies, engineering biological control of the electronic systems, and feed-back will be the important goals in regenerative engineering over the next two decades.

Recent advances in bioelectric prostheses

Worldwide prevalence of amputation has created an increasing demand for improved upper and lower extremity prostheses. Current prosthetics are often uncomfortable and difficult to control and provide limited functional restoration. Moreover, the inability to normalize anthropomorphic biomechanics with a prosthesis increases one's risk of developing long-term health risks such as arthritis, skin breakdown, and pain. Recent advances in bionic prosthetic development hold great promise for rehabilitation and improving quality of life with limb loss. This brief review discusses the current state of advanced prostheses, the integration of robotics in the care of individuals with major limb amputation, and some innovative surgical techniques that are being explored for clinical feasibility.

Delaying discharge after the stimulus significantly decreases muscle activation thresholds with small impact on the selectivity: an in vivo study using TIME

The number of devices for electrical stimulation of nerve fibres implanted worldwide for medical applications is constantly increasing. Stimulation charge is one of the most important parameters of stimulation. High stimulation charge may cause tissue and electrode damage and also compromise the battery life of the electrical stimulators. Therefore, the objective of minimizing stimulation charge is an important issue. Delaying the second phase of biphasic stimulation waveform may decrease the charge required for fibre activation, but its impact on stimulation selectivity is not known. This information is particularly relevant when transverse intrafascicular multichannel electrode (TIME) is used, since it has been designed to provide for high selectivity. In this in vivo study, the rat sciatic nerve was electrically stimulated using monopolar and bipolar configurations with TIME. The results demonstrated that the inclusion of a 100-μs delay between the cathodic and the anodic phase of the stimulus allows to reduce charge requirements by around 30 %, while only slightly affecting stimulation selectivity. This study shows that adding a delay to the typical stimulation waveform significantly ([Formula: see text]) reduces the charge required for nerve fibres activation. Therefore, waveforms with the delayed discharge phase are more suitable for electrical stimulation of nerve fibres.

Non-invasive control interfaces for intention detection in active movement-assistive devices

Active movement-assistive devices aim to increase the quality of life for patients with neuromusculoskeletal disorders. This technology requires interaction between the user and the device through a control interface that detects the user’s movement intention. Researchers have explored a wide variety of invasive and non-invasive control interfaces. To summarize the wide spectrum of strategies, this paper presents a comprehensive review focused on non-invasive control interfaces used to operate active movement-assistive devices. A novel systematic classification method is proposed to categorize the control interfaces based on: (I) the source of the physiological signal, (II) the physiological phenomena responsible for generating the signal, and (III) the sensors used to measure the physiological signal. The proposed classification method can successfully categorize all the existing control interfaces providing a comprehensive overview of the state of the art. Each sensing modality is briefly described in the body of the paper following the same structure used in the classification method. Furthermore, we discuss several design considerations, challenges, and future directions of non-invasive control interfaces for active movement-assistive devices.

The role of muscle synergies in myoelectric control: trends and challenges for simultaneous multifunction control

Myoelectric control is filled with potential to significantly change human-robot interaction due to the ability to non-invasively measure human motion intent. However, current control schemes have struggled to achieve the robust performance that is necessary for use in commercial applications. As demands in myoelectric control trend toward simultaneous multifunctional control, multi-muscle coordinations, or synergies, play larger roles in the success of the control scheme. Detecting and refining patterns in muscle activations robust to the high variance and transient changes associated with surface electromyography is essential for efficient, user-friendly control. This article reviews the role of muscle synergies in myoelectric control schemes by dissecting each component of the scheme with respect to associated challenges for achieving robust simultaneous control of myoelectric interfaces. Electromyography recording details, signal feature extraction, pattern recognition and motor learning based control schemes are considered, and future directions are proposed as steps toward fulfilling the potential of myoelectric control in clinically and commercially viable applications.

First-in-man demonstration of a fully implanted myoelectric sensors system to control an advanced electromechanical prosthetic hand

BACKGROUND

Advanced motorized prosthetic devices are currently controlled by EMG signals generated by residual muscles and recorded by surface electrodes on the skin. These surface recordings are often inconsistent and unreliable, leading to high prosthetic abandonment rates for individuals with upper limb amputation. Surface electrodes are limited because of poor skin contact, socket rotation, residual limb sweating, and their ability to only record signals from superficial muscles, whose function frequently does not relate to the intended prosthetic function. More sophisticated prosthetic devices require a stable and reliable interface between the user and robotic hand to improve upper limb prosthetic function.

NEW METHOD

Implantable Myoelectric Sensors (IMES(®)) are small electrodes intended to detect and wirelessly transmit EMG signals to an electromechanical prosthetic hand via an electro-magnetic coil built into the prosthetic socket. This system is designed to simultaneously capture EMG signals from multiple residual limb muscles, allowing the natural control of multiple degrees of freedom simultaneously.

RESULTS

We report the status of the first FDA-approved clinical trial of the IMES(®) System. This study is currently in progress, limiting reporting to only preliminary results.

COMPARISON WITH EXISTING METHODS

Our first subject has reported the ability to accomplish a greater variety and complexity of tasks in his everyday life compared to what could be achieved with his previous myoelectric prosthesis.

CONCLUSION

The interim results of this study indicate the feasibility of utilizing IMES(®) technology to reliably sense and wirelessly transmit EMG signals from residual muscles to intuitively control a three degree-of-freedom prosthetic arm.

Microelectrode array recordings from the ventral roots in chronically implanted cats

The ventral spinal roots contain the axons of spinal motoneurons and provide the only location in the peripheral nervous system where recorded neural activity can be assured to be motor rather than sensory. This study demonstrates recordings of single unit activity from these ventral root axons using floating microelectrode arrays (FMAs). Ventral root recordings were characterized by examining single unit yield and signal-to-noise ratios (SNR) with 32-channel FMAs implanted chronically in the L6 and L7 spinal roots of nine cats. Single unit recordings were performed for implant periods of up to 12 weeks. Motor units were identified based on active discharge during locomotion and inactivity under anesthesia. Motor unit yield and SNR were calculated for each electrode, and results were grouped by electrode site size, which were varied systematically between 25 and 160 μm to determine effects on signal quality. The unit yields and SNR did not differ significantly across this wide range of electrode sizes. Both SNR and yield decayed over time, but electrodes were able to record spikes with SNR >2 up to 12 weeks post-implant. These results demonstrate that it is feasible to record single unit activity from multiple isolated motor units with penetrating microelectrode arrays implanted chronically in the ventral spinal roots. This approach could be useful for creating a spinal nerve interface for advanced neural prostheses, and results of this study will be used to improve design of microelectrodes for chronic neural recording in the ventral spinal roots.

Evaluation of EMG, force and joystick as control interfaces for active arm supports

BACKGROUND

The performance capabilities and limitations of control interfaces for the operation of active movement-assistive devices remain unclear. Selecting an optimal interface for an application requires a thorough understanding of the performance of multiple control interfaces.

METHODS

In this study the performance of EMG-, force- and joystick-based control interfaces were assessed in healthy volunteers with a screen-based one-dimensional position-tracking task. The participants had to track a target that was moving according to a multisine signal with a bandwidth of 3 Hz. The velocity of the cursor was proportional to the interface signal. The performance of the control interfaces were evaluated in terms of tracking error, gain margin crossover frequency, information transmission rate and effort.

RESULTS

None of the evaluated interfaces was superior in all four performance descriptors. The EMG-based interface was superior in tracking error and gain margin crossover frequency compared to the force- and the joystick-based interfaces. The force-based interface provided higher information transmission rate and lower effort than the EMG-based interface. The joystick-based interface did not present any significant difference with the force-based interface for any of the four performance descriptors. We found that significant differences in terms of tracking error and information transmission rate were present beyond 0.9 and 1.4 Hz respectively.

CONCLUSIONS

Despite the fact that the EMG-based interface is far from the natural way of interacting with the environment, while the force-based interface is closer, the EMG-based interface presented very similar and for some descriptors even a better performance than the force-based interface for frequencies below 1.4 Hz. The classical joystick presented a similar performance to the force-based interface and holds the advantage of being a well established interface for the control of many assistive devices. From these findings we concluded that all the control interfaces considered in this study can be regarded as a candidate interface for the control of an active arm support.

Innovations in prosthetic interfaces for the upper extremity

Advancements in modern robotic technology have led to the development of highly sophisticated upper extremity prosthetic limbs. High-fidelity volitional control of these devices is dependent on the critical interface between the patient and the mechanical prosthesis. Recent innovations in prosthetic interfaces have focused on several control strategies. Targeted muscle reinnervation is currently the most immediately applicable prosthetic control strategy and is particularly indicated in proximal upper extremity amputations. Investigation into various brain interfaces has allowed acquisition of neuroelectric signals directly or indirectly from the central nervous system for prosthetic control. Peripheral nerve interfaces permit signal transduction from both motor and sensory nerves with a higher degree of selectivity. This article reviews the current developments in each of these interface systems and discusses the potential of these approaches to facilitate motor control and sensory feedback in upper extremity neuroprosthetic devices.

Interfaces with the peripheral nerve for the control of neuroprostheses

Nervous system injuries lead to loss of control of sensory, motor, and autonomic functions of the affected areas of the body. Provided the high amount of people worldwide suffering from these injuries and the impact on their everyday life, numerous and different neuroprostheses and hybrid bionic systems have been developed to restore or partially mimic the lost functions. A key point for usable neuroprostheses is the electrode that interfaces the nervous system and translates not only motor orders into electrical outputs that activate the prosthesis but is also able to transform sensory information detected by the machine into signals that are transmitted to the central nervous system. Nerve electrodes have been classified with regard to their invasiveness in extraneural, intraneural, and regenerative. The more invasive is the implant the more selectivity of interfacing can be reached. However, boosting invasiveness and selectivity may also heighten nerve damage. This chapter provides a general overview of nerve electrodes as well as the state-of-the-art of their biomedical applications in neuroprosthetic systems.

Repeated training with augmentative vibrotactile feedback increases object manipulation performance

Most users of prosthetic hands must rely on visual feedback alone, which requires visual attention and cognitive resources. Providing haptic feedback of variables relevant to manipulation, such as contact force, may thus improve the usability of prosthetic hands for tasks of daily living. Vibrotactile stimulation was explored as a feedback modality in ten unimpaired participants across eight sessions in a two-week period. Participants used their right index finger to perform a virtual object manipulation task with both visual and augmentative vibrotactile feedback related to force. Through repeated training, participants were able to learn to use the vibrotactile feedback to significantly improve object manipulation. Removal of vibrotactile feedback in session 8 significantly reduced task performance. These results suggest that vibrotactile feedback paired with training may enhance the manipulation ability of prosthetic hand users without the need for more invasive strategies.

Adaptive affective response identification for hearing threshold detection

Emotional arousal, or affective patterns, can be probed using observable bioelectric signals, in particular using the fluctuations of electroencephalographic potentials from the human scalp. Hearing impairment related to increased threshold of audio tone detection may cause the loss of intelligibility of speech resulting in an innate automatic emotional response. An adaptive support vector machine can be trained to identify a subject's unique affective response based upon an audiogram hearing test. This paper presents the efficacy of our model, initial SVM classification data, and discusses potential application.

Manipulable objects facilitate cross-modal integration in peripersonal space

Previous studies have shown that tool use often modifies one's peripersonal space – i.e. the space directly surrounding our body. Given our profound experience with manipulable objects (e.g. a toothbrush, a comb or a teapot) in the present study we hypothesized that the observation of pictures representing manipulable objects would result in a remapping of peripersonal space as well. Subjects were required to report the location of vibrotactile stimuli delivered to the right hand, while ignoring visual distractors superimposed on pictures representing everyday objects. Pictures could represent objects that were of high manipulability (e.g. a cell phone), medium manipulability (e.g. a soap dispenser) and low manipulability (e.g. a computer screen). In the first experiment, when subjects attended to the action associated with the objects, a strong cross-modal congruency effect (CCE) was observed for pictures representing medium and high manipulability objects, reflected in faster reaction times if the vibrotactile stimulus and the visual distractor were in the same location, whereas no CCE was observed for low manipulability objects. This finding was replicated in a second experiment in which subjects attended to the visual properties of the objects. These findings suggest that the observation of manipulable objects facilitates cross-modal integration in peripersonal space.

Imitation of hand and tool actions is effector-independent

Following the theoretical notion that tools often extend one’s body, in the present study, we investigated whether imitation of hand or tool actions is modulated by effector-specific information. Subjects performed grasping actions toward an object with either a handheld tool or their right hand. Actions were initiated in response to pictures representing a grip at an object that could be congruent or incongruent with the required action (grip-type congruency). Importantly, actions could be cued by means of a tool cue, a hand cue, and a symbolic cue (effector-type congruency). For both hand and tool actions, an action congruency effect was observed, reflected in faster reaction times if the observed grip type was congruent with the required movement. However, neither hand actions nor tool actions were differentially affected by the effector represented in the picture (i.e., when performing a tool action, the action congruency effect was similar for tool cues and hand cues). This finding suggests that imitation of hand and tool actions is effector-independent and thereby supports generalist rather than specialist theories of imitation.

Design principles for noninvasive brain-machine interfaces

With the advent of sophisticated prosthetic limbs, the challenge is now to develop and demonstrate optimal closed-loop control of the these limbs using neural measurements from single/multiple unit activity (SUA/MUA), electrocorticography (ECoG), local field potentials (LFP), scalp electroencephalography (EEG) or even electromyography (EMG) after targeted muscle reinnervation (TMR) in subjects with upper limb disarticulation. In this paper we propose design principles for developing a noninvasive EEG-based brain-machine interface (BMI) for dexterous control of a high degree-of-freedom, biologically realistic limb.