Continuous detection and decoding of dexterous finger flexions with implantable myoelectric sensors

Neuromuscular implants: Interfacing with skeletal muscle for improved clinical translation of prosthetic limbs

After an amputation, advanced prosthetic limbs can be used to interface with the nervous system and restore motor function. Despite numerous breakthroughs in the field, many of the recent research advancements have not been widely integrated into clinical practice. This review highlights recent innovations in neuromuscular implants-specifically those that interface with skeletal muscle-which could improve the clinical translation of prosthetic technologies. Skeletal muscle provides a physiologic gateway to harness and amplify signals from the nervous system. Recent surgical advancements in muscle reinnervation surgeries leverage the "bio-amplification" capabilities of muscle, enabling more intuitive control over a greater number of degrees of freedom in prosthetic limbs than previously achieved. We anticipate that state-of-the-art implantable neuromuscular interfaces that integrate well with skeletal muscle and novel surgical interventions will provide a long-term solution for controlling advanced prostheses. Flexible electrodes are expected to play a crucial role in reducing foreign body responses and improving the longevity of the interface. Additionally, innovations in device miniaturization and ongoing exploration of shape memory polymers could simplify surgical procedures for implanting such interfaces. Once implanted, wireless strategies for powering and transferring data from the interface can eliminate bulky external wires, reduce infection risk, and enhance day-to-day usability. By outlining the current limitations of neuromuscular interfaces along with potential future directions, this review aims to guide continued research efforts and future collaborations between engineers and specialists in the field of neuromuscular and musculoskeletal medicine.

Toward higher-performance bionic limbs for wider clinical use

Most prosthetic limbs can autonomously move with dexterity, yet they are not perceived by the user as belonging to their own body. Robotic limbs can convey information about the environment with higher precision than biological limbs, but their actual performance is substantially limited by current technologies for the interfacing of the robotic devices with the body and for transferring motor and sensory information bidirectionally between the prosthesis and the user. In this Perspective, we argue that direct skeletal attachment of bionic devices via osseointegration, the amplification of neural signals by targeted muscle innervation, improved prosthesis control via implanted muscle sensors and advanced algorithms, and the provision of sensory feedback by means of electrodes implanted in peripheral nerves, should all be leveraged towards the creation of a new generation of high-performance bionic limbs. These technologies have been clinically tested in humans, and alongside mechanical redesigns and adequate rehabilitation training should facilitate the wider clinical use of bionic limbs.

The Role of Fascial Tissue Layer in Electric Signal Transmission from the Forearm Musculature to the Cutaneous Layer as a Possibility for Increased Signal Strength in Myoelectric Forearm Exoprosthesis Development

Myoelectric exoprostheses serve to aid in the everyday activities of patients with forearm or hand amputations. While electrical signals are known key factors controlling exoprosthesis, little is known about how we can improve their transmission strength from the forearm muscles as to obtain better sEMG. The purpose of this study is to evaluate the role of the forearm fascial layer in transmitting myoelectrical current. We examined the sEMG signals in three individual muscles, each from six healthy forearms (Group 1) and six amputation stumps (Group 2), along with their complete biometric characteristics. Following the tests, one patient underwent a circumferential osteoneuromuscular stump revision surgery (CONM) that also involved partial removal of fascia and subcutaneous fat in the amputation stump, with re-testing after complete healing. In group 1, we obtained a stronger sEMG signal than in Group 2. In the CONM case, after surgery, the patient’s data suggest that the removal of fascia, alongside the fibrotic and subcutaneous fat tissue, generates a stronger sEMG signal. Therefore, a reduction in the fascial layer, especially if accompanied by a reduction of the subcutaneous fat layer may prove significant for improving the strength of sEMG signals used in the control of modern exoprosthetics.

Flexible Electronics and Devices as Human-Machine Interfaces for Medical Robotics

Medical robots are invaluable players in non-pharmaceutical treatment of disabilities. Particularly, using prosthetic and rehabilitation devices with human-machine interfaces can greatly improve the quality of life for impaired patients. In recent years, flexible electronic interfaces and soft robotics have attracted tremendous attention in this field due to their high biocompatibility, functionality, conformability, and low-cost. Flexible human-machine interfaces on soft robotics will make a promising alternative to conventional rigid devices, which can potentially revolutionize the paradigm and future direction of medical robotics in terms of rehabilitation feedback and user experience. In this review, the fundamental components of the materials, structures, and mechanisms in flexible human-machine interfaces are summarized by recent and renowned applications in five primary areas: physical and chemical sensing, physiological recording, information processing and communication, soft robotic actuation, and feedback stimulation. This review further concludes by discussing the outlook and current challenges of these technologies as a human-machine interface in medical robotics.

A parallel classification strategy to simultaneous control elbow, wrist, and hand movements

Background

In the field of myoelectric control systems, pattern recognition (PR) algorithms have become always more interesting for predicting complex electromyography patterns involving movements with more than 2 Degrees of Freedom (DoFs). The majority of classification strategies, used for the prosthetic control, are based on single, hierarchical and parallel linear discriminant analysis (LDA) classifiers able to discriminate up to 19 wrist/hand gestures (in the 3-DoFs case), considering both combined and discrete motions. However, these strategies were introduced to simultaneously classify only 2 DoFs and their use is limited by the lack of online performance measures. This study introduces a novel classification strategy based on the Logistic Regression (LR) algorithm with regularization parameter to provide simultaneous classification of 3 DoFs motion classes.

Methods

The parallel PR-based strategy was tested on 15 healthy subjects, by using only six surface EMG sensors. Twenty-seven discrete and complex elbow, hand and wrist motions were classified by keeping the number of electromyographic (EMG) electrodes to a bare minimum and the classification error rate under 10 %. To this purpose, the parallel classification strategy was implemented by using three classifiers one for each DoF: the “Elbow classifier”, the “Wrist classifier”, and the “Hand classifier” provided the simultaneous control of the elbow, hand, and wrist joints, respectively.

Results

Both the offline and real-time performance metrics were evaluated and compared with the LDA parallel classification results. The real-time recognition results were statistically better with the LR classifier with respect to the LDA classifier, for all motion classes (elbow, hand and wrist).

Conclusions

In this paper, a novel parallel PR-based strategy was proposed for classifying up to 3 DoFs: three joint classifiers were employed simultaneously for classifying 27 motion classes related to the elbow, wrist, and hand and promising results were obtained.

Targeted Muscle Reinnervation for Prosthetic Control

Targeted muscle reinnervation (TMR) is a surgical procedure, whereby nerves without muscle targets after extremity amputation are transferred to residual stump muscles. Thereby, the control of prosthesis is improved by increasing the number of independent muscle signals. The authors describe indications for TMR to improve prosthetic control and present standard nerve transfer matrices suitable for transhumeral and glenohumeral amputees. In addition, the perioperative procedure is described, including preoperative testing, surgical approach, and postoperative rehabilitation. Based on recent neurophysiological insights and technological advances, they present an outlook into the future of prosthetic control combining TMR and implantable electromyographic technology.

Simultaneous Prediction of Wrist/Hand Motion via Wearable Ultrasound Sensing

The ability to predict wrist and hand motions simultaneously is essential for natural controls of hand protheses. In this paper, we propose a novel method that includes subclass discriminant analysis (SDA) and principal component analysis for the simultaneous prediction of wrist rotation (pronation/supination) and finger gestures using wearable ultrasound. We tested the method on eight finger gestures with concurrent wrist rotations. Results showed that SDA was able to achieve accurate classification of both finger gestures and wrist rotations under dynamic wrist rotations. When grouping the wrist rotations into three subclasses, about 99.2 ± 1.2% of finger gestures and 92.8 ± 1.4% of wrist rotations can be accurately classified. Moreover, we found that the first principal component (PC1) of the selected ultrasound features was linear to the wrist rotation angle regardless of finger gestures. We further used PC1 in an online tracking task for continuous wrist control and demonstrated that a wrist tracking precision ( R² ) of 0.954 ± 0.012 and a finger gesture classification accuracy of 96.5 ± 1.7% can be simultaneously achieved, with only two minutes of user training. Our proposed simultaneous wrist/hand control scheme is training-efficient and robust, paving the way for musculature-driven artificial hand control and rehabilitation treatment.

Experimental Testing of Bionic Peripheral Nerve and Muscle Interfaces: Animal Model Considerations

Introduction: Man-machine interfacing remains the main challenge for accurate and reliable control of bionic prostheses. Implantable electrodes in nerves and muscles may overcome some of the limitations by significantly increasing the interface's reliability and bandwidth. Before human application, experimental preclinical testing is essential to assess chronic in-vivo biocompatibility and functionality. Here, we analyze available animal models, their costs and ethical challenges in special regards to simulating a potentially life-long application in a short period of time and in non-biped animals.

Methods: We performed a literature analysis following the PRISMA guidelines including all animal models used to record neural or muscular activity via implantable electrodes, evaluating animal models, group size, duration, origin of publication as well as type of interface. Furthermore, behavioral, ethical, and economic considerations of these models were analyzed. Additionally, we discuss experience and surgical approaches with rat, sheep, and primate models and an approach for international standardized testing.

Results: Overall, 343 studies matched the search terms, dominantly originating from the US (55%) and Europe (34%), using mainly small animal models (rat: 40%). Electrode placement was dominantly neural (77%) compared to muscular (23%). Large animal models had a mean duration of 135 ± 87.2 days, with a mean of 5.3 ± 3.4 animals per trial. Small animal models had a mean duration of 85 ± 11.2 days, with a mean of 12.4 ± 1.7 animals.

Discussion: Only 37% animal models were by definition chronic tests (>3 months) and thus potentially provide information on long-term performance. Costs for large animals were up to 45 times higher than small animals. However, costs are relatively small compared to complication costs in human long-term applications. Overall, we believe a combination of small animals for preliminary primary electrode testing and large animals to investigate long-term biocompatibility, impedance, and tissue regeneration parameters provides sufficient data to ensure long-term human applications.

Proprioceptive Sonomyographic Control: A novel method for intuitive and proportional control of multiple degrees-of-freedom for individuals with upper extremity limb loss

Technological advances in multi-articulated prosthetic hands have outpaced the development of methods to intuitively control these devices. In fact, prosthetic users often cite "difficulty of use" as a key contributing factor for abandoning their prostheses. To overcome the limitations of the currently pervasive myoelectric control strategies, namely unintuitive proportional control of multiple degrees-of-freedom, we propose a novel approach: proprioceptive sonomyographiccontrol. Unlike myoelectric control strategies which measure electrical activation of muscles and use the extracted signals to determine the velocity of an end-effector; our sonomyography-based strategy measures mechanical muscle deformation directly with ultrasound and uses the extracted signals to proportionally control the position of an end-effector. Therefore, our sonomyography-based control is congruent with a prosthetic user’s innate proprioception of muscle deformation in the residual limb. In this work, we evaluated proprioceptive sonomyographic control with 5 prosthetic users and 5 able-bodied participants in a virtual target achievement and holding task for 5 different hand motions. We observed that with limited training, the performance of prosthetic users was comparable to that of able-bodied participants and thus conclude that proprioceptive sonomyographic control is a robust and intuitive prosthetic control strategy.

Simultaneous sEMG Classification of Hand/Wrist Gestures and Forces

Surface electromyography (sEMG) signals represent a promising approach for decoding the motor intention of amputees to control a multifunctional prosthetic hand in a non-invasive way. Several approaches based on proportional amplitude methods or simple thresholds on sEMG signals have been proposed to control a single degree of freedom at time, without the possibility of increasing the number of controllable multiple DoFs in a natural manner. Myoelectric control based on PR techniques have been introduced to add multiple DoFs by keeping low the number of electrodes and allowing the discrimination of different muscular patterns for each class of motion. However, the use of PR algorithms to simultaneously decode both gestures and forces has never been studied deeply. This paper introduces a hierarchical classification approach with the aim to assess the desired hand/wrist gestures, as well as the desired force levels to exert during grasping tasks. A Finite State Machine was introduced to manage and coordinate three classifiers based on the Non-Linear Logistic Regression algorithm. The classification architecture was evaluated across 31 healthy subjects. The “hand/wrist gestures classifier,” introduced for the discrimination of seven hand/wrist gestures, presented a mean classification accuracy of 98.78%, while the “Spherical and Tip force classifier,” created for the identification of three force levels, reached an average accuracy of 98.80 and 96.09%, respectively. These results were confirmed by Linear Discriminant Analysis (LDA) with time domain features extraction, considered as ground truth for the final validation of the performed analysis. A Wilcoxon Signed-Rank test was carried out for the statistical analysis of comparison between NLR and LDA and statistical significance was considered at p < 0.05. The comparative analysis reports not statistically significant differences in terms of F1Score performance between NLR and LDA. Thus, this study reveals that the use of non-linear classification algorithm, as NLR, is as much suitable as the benchmark LDA classifier for implementing an EMG pattern recognition system, able both to decode hand/wrist gestures and to associate different performed force levels to grasping actions.

An LDA-Based Approach for Real-Time Simultaneous Classification of Movements Using Surface Electromyography

Myoelectric-based decoding strategies offer significant advantages in the areas of human-machine interactions because they are intuitive and require less cognitive effort from the users. However, a general drawback in using machine learning techniques for classification is that the decoder is limited to predicting only one movement at any instant and hence restricted to performing the motion in a sequential manner, whereas human motor control strategy involves simultaneous actuation of multiple degrees of freedom (DOFs) and is considered to be a natural and efficient way of performing tasks. Simultaneous decoding in the context of myoelectric-based movement control is a challenge that is being addressed recently and is increasingly popular. In this paper, we propose a novel classification strategy capable of decoding both the individual and combined movements, by collecting data from only the individual motions. Additionally, we exploit low-dimensional representation of the myoelectric signals using a supervised decomposition algorithm called linear discriminant analysis, to simplify the complexity of control and reduce computational cost. The performance of the decoding algorithm is tested in an online context for the two DOFs task comprising the hand and wrist movements. Results indicate an overall classification accuracy of 88.02% for both the individual and combined motions.

Design and testing of a 96-channel neural interface module for the Networked Neuroprosthesis system

Background

The loss of motor functions resulting from spinal cord injury can have devastating implications on the quality of one’s life. Functional electrical stimulation has been used to help restore mobility, however, current functional electrical stimulation (FES) systems require residual movements to control stimulation patterns, which may be unintuitive and not useful for individuals with higher level cervical injuries. Brain machine interfaces (BMI) offer a promising approach for controlling such systems; however, they currently still require transcutaneous leads connecting indwelling electrodes to external recording devices. While several wireless BMI systems have been designed, high signal bandwidth requirements limit clinical translation. Case Western Reserve University has developed an implantable, modular FES system, the Networked Neuroprosthesis (NNP), to perform combinations of myoelectric recording and neural stimulation for controlling motor functions. However, currently the existing module capabilities are not sufficient for intracortical recordings.

Methods

Here we designed and tested a 1 × 4 cm, 96-channel neural recording module prototype to fit within the specifications to mate with the NNP. The neural recording module extracts power between 0.3–1 kHz, instead of transmitting the raw, high bandwidth neural data to decrease power requirements.

Results

The module consumed 33.6 mW while sampling 96 channels at approximately 2 kSps. We also investigated the relationship between average spiking band power and neural spike rate, which produced a maximum correlation of R = 0.8656 (Monkey N) and R = 0.8023 (Monkey W).

Conclusion

Our experimental results show that we can record and transmit 96 channels at 2ksps within the power restrictions of the NNP system and successfully communicate over the NNP network. We believe this device can be used as an extension to the NNP to produce a clinically viable, fully implantable, intracortically-controlled FES system and advance the field of bioelectronic medicine.

Motor unit drive: a neural interface for real-time upper limb prosthetic control

OBJECTIVE

Modern prosthetic limbs have made strident gains in recent years, incorporating terminal electromechanical devices that are capable of mimicking the human hand. However, access to these advanced control capabilities has been prevented by fundamental limitations of amplitude-based myoelectric neural interfaces, which have remained virtually unchanged for over four decades. Consequently, nearly 23% of adults and 32% of children with major traumatic or congenital upper-limb loss abandon regular use of their myoelectric prosthesis. To address this healthcare need, we have developed a noninvasive neural interface technology that maps natural motor unit increments of neural control and force into biomechanically informed signals for improved prosthetic control.

APPROACH

Our technology, referred to as motor unit drive (MU Drive), utilizes real-time machine learning algorithms for directly measuring motor unit firings from surface electromyographic signals recorded from residual muscles of an amputated or congenitally missing limb. The extracted firings are transformed into biomechanically informed signals based on the force generating properties of individual motor units to provide a control source that represents the intended movement.

MAIN RESULTS

We evaluated the characteristics of the MU Drive control signals and compared them to conventional amplitude-based myoelectric signals in healthy subjects as well as subjects with congenital or traumatic trans-radial limb-loss. Our analysis established a vital proof-of-concept: MU Drive provides a more responsive real-time signal with improved smoothness and more faithful replication of intended limb movement that overcomes the trade-off between performance and latency inherent to amplitude-based myoelectric methods.

SIGNIFICANCE

MU Drive is the first neural interface for prosthetic control that provides noninvasive real-time access to the natural motor control mechanisms of the human nervous system. This new neural interface holds promise for improving prosthetic function by achieving advanced control that better reflects the user intent. Beyond the immediate advantages in the field of prosthetics, MU Drive provides an innovative alternative for advancing the control of exoskeletons, assistive devices, and other robotic rehabilitation applications.

Comparing Surface and Intramuscular Electromyography for Simultaneous and Proportional Control Based on a Musculoskeletal Model: A Pilot Study

Simultaneous and proportional control (SPC) of neural-machine interfaces uses magnitudes of smoothed electromyograms (EMG) as control inputs. Though surface EMG (sEMG) electrodes are common for clinical neural-machine interfaces, intramuscular EMG (iEMG) electrodes may be indicated in some circumstances (e.g., for controlling many degrees of freedom). However, differences in signal characteristics between sEMG and iEMG may influence SPC performance. We conducted a pilot study to determine the effect of electrode type (sEMG and iEMG) on real-time task performance with SPC based on a novel 2-degree-of-freedom EMG-driven musculoskeletal model of the wrist and hand. Four able-bodied subjects and one transradial amputee performed a virtual posture matching task with either sEMG or iEMG. There was a trend of better task performance with sEMG than iEMG for both able-bodied and amputee subjects, though the difference was not statistically significant. Thus, while iEMG may permit targeted recording of EMG, its signal characteristics may not be as ideal for SPC as those of sEMG. The tradeoff between recording specificity and signal characteristics is an important consideration for development and clinical implementation of SPC for neural-machine interfaces.

Broadband Prosthetic Interfaces: Combining Nerve Transfers and Implantable Multichannel EMG Technology to Decode Spinal Motor Neuron Activity

Modern robotic hands/upper limbs may replace multiple degrees of freedom of extremity function. However, their intuitive use requires a high number of control signals, which current man-machine interfaces do not provide. Here, we discuss a broadband control interface that combines targeted muscle reinnervation, implantable multichannel electromyographic sensors, and advanced decoding to address the increasing capabilities of modern robotic limbs. With targeted muscle reinnervation, nerves that have lost their targets due to an amputation are surgically transferred to residual stump muscles to increase the number of intuitive prosthetic control signals. This surgery re-establishes a nerve-muscle connection that is used for sensing nerve activity with myoelectric interfaces. Moreover, the nerve transfer determines neurophysiological effects, such as muscular hyper-reinnervation and cortical reafferentation that can be exploited by the myoelectric interface. Modern implantable multichannel EMG sensors provide signals from which it is possible to disentangle the behavior of single motor neurons. Recent studies have shown that the neural drive to muscles can be decoded from these signals and thereby the user's intention can be reliably estimated. By combining these concepts in chronic implants and embedded electronics, we believe that it is in principle possible to establish a broadband man-machine interface, with specific applications in prosthesis control. This perspective illustrates this concept, based on combining advanced surgical techniques with recording hardware and processing algorithms. Here we describe the scientific evidence for this concept, current state of investigations, challenges, and alternative approaches to improve current prosthetic interfaces.

Control of Prosthetic Hands via the Peripheral Nervous System

This paper intends to provide a critical review of the literature on the technological issues on control and sensorization of hand prostheses interfacing with the Peripheral Nervous System (i.e., PNS), and their experimental validation on amputees. The study opens with an in-depth analysis of control solutions and sensorization features of research and commercially available prosthetic hands. Pros and cons of adopted technologies, signal processing techniques and motion control solutions are investigated. Special emphasis is then dedicated to the recent studies on the restoration of tactile perception in amputees through neural interfaces. The paper finally proposes a number of suggestions for designing the prosthetic system able to re-establish a bidirectional communication with the PNS and foster the prosthesis natural control.

Multichannel Electrotactile Feedback With Spatial and Mixed Coding for Closed-Loop Control of Grasping Force in Hand Prostheses

Providing somatosensory feedback to the user of a myoelectric prosthesis is an important goal since it can improve the utility as well as facilitate the embodiment of the assistive system. Most often, the grasping force was selected as the feedback variable and communicated through one or more individual single channel stimulation units (e.g., electrodes, vibration motors). In the present study, an integrated, compact, multichannel solution comprising an array electrode and a programmable stimulator was presented. Two coding schemes (15 levels), spatial and mixed (spatial and frequency) modulation, were tested in able-bodied subjects, psychometrically and in force control with routine grasping and force tracking using real and simulated prosthesis. The results demonstrated that mixed and spatial coding, although substantially different in psychometric tests, resulted in a similar performance during both force control tasks. Furthermore, the ideal, visual feedback was not better than the tactile feedback in routine grasping. To explain the observed results, a conceptual model was proposed emphasizing that the performance depends on multiple factors, including feedback uncertainty, nature of the task and the reliability of the feedforward control. The study outcomes, specific conclusions and the general model, are relevant for the design of closed-loop myoelectric prostheses utilizing tactile feedback.

Towards reducing the impacts of unwanted movements on identification of motion intentions

Surface electromyogram (sEMG) has been extensively used as a control signal in prosthesis devices. However, it is still a great challenge to make multifunctional myoelectric prostheses clinically available due to a number of critical issues associated with existing EMG based control strategy. One such issue would be the effect of unwanted movements (UMs) that are inadvertently done by users on the performance of movement classification in EMG pattern recognition based algorithms. Since UMs are not considered in training a classifier, they would decay the performance of a trained classifier in identifying the target movements (TMs), which would cause some undesired actions in control of multifunctional prostheses. In this study, the impact of UMs was systemically investigated in both able-bodied subjects and transradial amputees. Our results showed that the UMs would be unevenly classified into all classes of the TMs. To reduce the impact of the UMs on the performance of a classifier, a new training strategy that would categorize all possible UMs into a new movement class was proposed and a metric called Reject Ratio that is a measure of how many UMs is rejected by a trained classifier was adopted. The results showed that the average Reject Ratio across all the participants was greater than 91%, meanwhile the average classification accuracy of TMs was above 99% when UMs occurred. This suggests that the proposed training strategy could greatly reduce the impact of UMs on the performance of the trained classifier in identifying the TMs and may enhance the robustness of myoelectric control in clinical applications.

Sensor fusion and computer vision for context-aware control of a multi degree-of-freedom prosthesis

OBJECTIVE

Myoelectric activity volitionally generated by the user is often used for controlling hand prostheses in order to replicate the synergistic actions of muscles in healthy humans during grasping. Muscle synergies in healthy humans are based on the integration of visual perception, heuristics and proprioception. Here, we demonstrate how sensor fusion that combines artificial vision and proprioceptive information with the high-level processing characteristics of biological systems can be effectively used in transradial prosthesis control.

APPROACH

We developed a novel context- and user-aware prosthesis (CASP) controller integrating computer vision and inertial sensing with myoelectric activity in order to achieve semi-autonomous and reactive control of a prosthetic hand. The presented method semi-automatically provides simultaneous and proportional control of multiple degrees-of-freedom (DOFs), thus decreasing overall physical effort while retaining full user control. The system was compared against the major commercial state-of-the art myoelectric control system in ten able-bodied and one amputee subject. All subjects used transradial prosthesis with an active wrist to grasp objects typically associated with activities of daily living.

MAIN RESULTS

The CASP significantly outperformed the myoelectric interface when controlling all of the prosthesis DOF. However, when tested with less complex prosthetic system (smaller number of DOF), the CASP was slower but resulted with reaching motions that contained less compensatory movements. Another important finding is that the CASP system required minimal user adaptation and training.

SIGNIFICANCE

The CASP constitutes a substantial improvement for the control of multi-DOF prostheses. The application of the CASP will have a significant impact when translated to real-life scenarious, particularly with respect to improving the usability and acceptance of highly complex systems (e.g., full prosthetic arms) by amputees.

Monte Carlo point process estimation of electromyographic envelopes from motor cortical spikes for brain-machine interfaces

OBJECTIVE

Representation of movement in the motor cortex (M1) has been widely studied in brain-machine interfaces (BMIs). The electromyogram (EMG) has greater bandwidth than the conventional kinematic variables (such as position, velocity), and is functionally related to the discharge of cortical neurons. As the stochastic information of EMG is derived from the explicit spike time structure, point process (PP) methods will be a good solution for decoding EMG directly from neural spike trains. Previous studies usually assume linear or exponential tuning curves between neural firing and EMG, which may not be true.

APPROACH

In our analysis, we estimate the tuning curves in a data-driven way and find both the traditional functional-excitatory and functional-inhibitory neurons, which are widely found across a rat's motor cortex. To accurately decode EMG envelopes from M1 neural spike trains, the Monte Carlo point process (MCPP) method is implemented based on such nonlinear tuning properties.

MAIN RESULTS

Better reconstruction of EMG signals is shown on baseline and extreme high peaks, as our method can better preserve the nonlinearity of the neural tuning during decoding. The MCPP improves the prediction accuracy (the normalized mean squared error) 57% and 66% on average compared with the adaptive point process filter using linear and exponential tuning curves respectively, for all 112 data segments across six rats. Compared to a Wiener filter using spike rates with an optimal window size of 50 ms, MCPP decoding EMG from a point process improves the normalized mean square error (NMSE) by 59% on average.

SIGNIFICANCE

These results suggest that neural tuning is constantly changing during task execution and therefore, the use of spike timing methodologies and estimation of appropriate tuning curves needs to be undertaken for better EMG decoding in motor BMIs.

Comparison of surface and intramuscular EMG pattern recognition for simultaneous wrist/hand motion classification

The simultaneous control of multiple degrees of freedom (DOFs) is important for the intuitive, life-like control of artificial limbs. The objective of this study was to determine whether the use of intramuscular electromyogram (EMG) improved pattern classification of simultaneous wrist/hand movements compared to surface EMG. Two pattern classification methods were used in this analysis, and were trained to predict 1-DOF and 2-DOF movements involving wrist rotation, wrist flexion/extension, and hand open/close. The classification methods used were (1) a single pattern classifier discriminating between 1-DOF and 2-DOF motion classes, and (2) a parallel set of three classifiers to predict the activity of each of the 3 DOFs. We demonstrate that in this combined wrist/hand classification task, the use of intramuscular EMG significantly decreases classification error compared to surface EMG for the parallel configuration (p<0.01), but not for the single classifier. We also show that the use of intramuscular EMG mitigates the increase in errors produced when the parallel classifier method is trained without 2-DOF motion class data.

The role of muscle synergies in myoelectric control: trends and challenges for simultaneous multifunction control

Myoelectric control is filled with potential to significantly change human-robot interaction due to the ability to non-invasively measure human motion intent. However, current control schemes have struggled to achieve the robust performance that is necessary for use in commercial applications. As demands in myoelectric control trend toward simultaneous multifunctional control, multi-muscle coordinations, or synergies, play larger roles in the success of the control scheme. Detecting and refining patterns in muscle activations robust to the high variance and transient changes associated with surface electromyography is essential for efficient, user-friendly control. This article reviews the role of muscle synergies in myoelectric control schemes by dissecting each component of the scheme with respect to associated challenges for achieving robust simultaneous control of myoelectric interfaces. Electromyography recording details, signal feature extraction, pattern recognition and motor learning based control schemes are considered, and future directions are proposed as steps toward fulfilling the potential of myoelectric control in clinically and commercially viable applications.

Stereovision and augmented reality for closed-loop control of grasping in hand prostheses

OBJECTIVE

Technologically advanced assistive devices are nowadays available to restore grasping, but effective and effortless control integrating both feed-forward (commands) and feedback (sensory information) is still missing. The goal of this work was to develop a user friendly interface for the semi-automatic and closed-loop control of grasping and to test its feasibility.

APPROACH

We developed a controller based on stereovision to automatically select grasp type and size and augmented reality (AR) to provide artificial proprioceptive feedback. The system was experimentally tested in healthy subjects using a dexterous hand prosthesis to grasp a set of daily objects. The subjects wore AR glasses with an integrated stereo-camera pair, and triggered the system via a simple myoelectric interface.

MAIN RESULTS

The results demonstrated that the subjects got easily acquainted with the semi-autonomous control. The stereovision grasp decoder successfully estimated the grasp type and size in realistic, cluttered environments. When allowed (forced) to correct the automatic system decisions, the subjects successfully utilized the AR feedback and achieved close to ideal system performance.

SIGNIFICANCE

The new method implements a high level, low effort control of complex functions in addition to the low level closed-loop control. The latter is achieved by providing rich visual feedback, which is integrated into the real life environment. The proposed system is an effective interface applicable with small alterations for many advanced prosthetic and orthotic/therapeutic rehabilitation devices.

A real-time comparison between direct control, sequential pattern recognition control and simultaneous pattern recognition control using a Fitts' law style assessment procedure

Background

Pattern recognition (PR) based strategies for the control of myoelectric upper limb prostheses are generally evaluated through offline classification accuracy, which is an admittedly useful metric, but insufficient to discuss functional performance in real time. Existing functional tests are extensive to set up and most fail to provide a challenging, objective framework to assess the strategy performance in real time.

Methods

Nine able-bodied and two amputee subjects gave informed consent and participated in the local Institutional Review Board approved study. We designed a two-dimensional target acquisition task, based on the principles of Fitts’ law for human motor control. Subjects were prompted to steer a cursor from the screen center of into a series of subsequently appearing targets of different difficulties. Three cursor control systems were tested, corresponding to three electromyography-based prosthetic control strategies: 1) amplitude-based direct control (the clinical standard of care), 2) sequential PR control, and 3) simultaneous PR control, allowing for a concurrent activation of two degrees of freedom (DOF). We computed throughput (bits/second), path efficiency (%), reaction time (second), and overshoot (%)) and used general linear models to assess significant differences between the strategies for each metric.

Results

We validated the proposed methodology by achieving very high coefficients of determination for Fitts’ law. Both PR strategies significantly outperformed direct control in two-DOF targets and were more intuitive to operate. In one-DOF targets, the simultaneous approach was the least precise. The direct control was efficient in one-DOF targets but cumbersome to operate in two-DOF targets through a switch-depended sequential cursor control.

Conclusions

We designed a test, capable of comprehensively describing prosthetic control strategies in real time. When implemented on control subjects, the test was able to capture statistically significant differences (p < 0.05) in control strategies when considering throughputs, path efficiencies and reaction times. Of particular note, we found statistically significant (p < 0.01) improvements in throughputs and path efficiencies with simultaneous PR when compared to direct control or sequential PR. Amputees could readily achieve the task; however a limited number of subjects was tested and a statistical analysis was not performed with that population.

Epidural electrocorticography of phantom hand movement following long-term upper-limb amputation

INTRODUCTION

Prostheses for upper-limb amputees are currently controlled by either myoelectric or peripheral neural signals. Performance and dexterity of these devices is still limited, particularly when it comes to controlling hand function. Movement-related brain activity might serve as a complementary bio-signal for motor control of hand prosthesis.

METHODS

We introduced a methodology to implant a cortical interface without direct exposure of the brain surface in an upper-limb amputee. This bi-directional interface enabled us to explore the cortical physiology following long-term transhumeral amputation. In addition, we investigated neurofeedback of electrocorticographic brain activity related to the patient's motor imagery to open his missing hand, i.e., phantom hand movement, for real-time control of a virtual hand prosthesis.

RESULTS

Both event-related brain activity and cortical stimulation revealed mutually overlapping cortical representations of the phantom hand. Phantom hand movements could be robustly classified and the patient required only three training sessions to gain reliable control of the virtual hand prosthesis in an online closed-loop paradigm that discriminated between hand opening and rest.

CONCLUSION

Epidural implants may constitute a powerful and safe alternative communication pathway between the brain and external devices for upper-limb amputees, thereby facilitating the integrated use of different signal sources for more intuitive and specific control of multi-functional devices in clinical use.

Dexterous control of a prosthetic hand using fine-wire intramuscular electrodes in targeted extrinsic muscles

Restoring dexterous motor function equivalent to that of the human hand after amputation is one of the major goals in rehabilitation engineering. To achieve this requires the implementation of a effortless human-machine interface that bridges the artificial hand to the sources of volition. Attempts to tap into the neural signals and to use them as control inputs for neuroprostheses range in invasiveness and hierarchical location in the neuromuscular system. Nevertheless today, the primary clinically viable control technique is the electromyogram measured peripherally by surface electrodes. This approach is neither physiologically appropriate nor dexterous because arbitrary finger movements or hand postures cannot be obtained. Here we demonstrate the feasibility of achieving real-time, continuous and simultaneous control of a multi-digit prosthesis directly from forearm muscles signals using intramuscular electrodes on healthy subjects. Subjects contracted physiologically appropriate muscles to control four degrees of freedom of the fingers of a physical robotic hand independently. Subjects described the control as intuitive and showed the ability to drive the hand into 12 postures without explicit training. This is the first study in which peripheral neural correlates were processed in real-time and used to control multiple digits of a physical hand simultaneously in an intuitive and direct way.

A comparison of the real-time controllability of pattern recognition to conventional myoelectric control for discrete and simultaneous movements

Myoelectric control has been used for decades to control powered upper limb prostheses. Conventional, amplitude-based control has been employed to control a single prosthesis degree of freedom (DOF) such as closing and opening of the hand. Within the last decade, new and advanced arm and hand prostheses have been constructed that are capable of actuating numerous DOFs. Pattern recognition control has been proposed to control a greater number of DOFs than conventional control, but has traditionally been limited to sequentially controlling DOFs one at a time. However, able-bodied individuals use multiple DOFs simultaneously, and it may be beneficial to provide amputees the ability to perform simultaneous movements. In this study, four amputees who had undergone targeted motor reinnervation (TMR) surgery with previous training using myoelectric prostheses were configured to use three control strategies: 1) conventional amplitude-based myoelectric control, 2) sequential (one-DOF) pattern recognition control, 3) simultaneous pattern recognition control. Simultaneous pattern recognition was enabled by having amputees train each simultaneous movement as a separate motion class. For tasks that required control over just one DOF, sequential pattern recognition based control performed the best with the lowest average completion times, completion rates and length error. For tasks that required control over 2 DOFs, the simultaneous pattern recognition controller performed the best with the lowest average completion times, completion rates and length error compared to the other control strategies. In the two strategies in which users could employ simultaneous movements (conventional and simultaneous pattern recognition), amputees chose to use simultaneous movements 78% of the time with simultaneous pattern recognition and 64% of the time with conventional control for tasks that required two DOF motions to reach the target. These results suggest that when amputees are given the ability to control multiple DOFs simultaneously, they choose to perform tasks that utilize multiple DOFs with simultaneous movements. Additionally, they were able to perform these tasks with higher performance (faster speed, lower length error and higher completion rates) without losing substantial performance in 1 DOF tasks.

Real-time control of an interactive impulsive virtual prosthesis

An interactive virtual dynamic environment for testing control strategies for neural machine interfacing with artificial limbs offers several advantages. The virtual environment is low-cost, easily configured, and offers a wealth of data for post-hoc analysis compared with real physical prostheses and robots. For use with prosthetics and research involving amputee subjects it allows the valuable time with the subject to be spent in experiments rather than fixing hardware issues. The usefulness of the virtual environment increases as the realism of the environment increases. Most tasks performed with limbs require interactions with objects in the environment. To simulate these tasks the dynamics of frictional contact, in addition to inertial limb dynamics must be modeled. Here, subjects demonstrate real-time control of an eight degree-of-freedom virtual prosthesis while performing an interactive box-and-blocks task. With practice, four nonamputee subjects and one shoulder disarticulation subject were able to successfully transfer blocks in the virtual environment at an average rate of just under two blocks per minute. The virtual environment is configurable in terms of the virtual arm design, control strategy, and task.

Novel Method for Predicting Dexterous Individual Finger Movements by Imaging Muscle Activity Using a Wearable Ultrasonic System

Recently there have been major advances in the electro-mechanical design of upper extremity prosthetics. However, the development of control strategies for such prosthetics has lagged significantly behind. Conventional noninvasive myoelectric control strategies rely on the amplitude of electromyography (EMG) signals from flexor and extensor muscles in the forearm. Surface EMG has limited specificity for deep contiguous muscles because of cross talk and cannot reliably differentiate between individual digit and joint motions. We present a novel ultrasound imaging based control strategy for upper arm prosthetics that can overcome many of the limitations of myoelectric control. Real time ultrasound images of the forearm muscles were obtained using a wearable mechanically scanned single element ultrasound system, and analyzed to create maps of muscle activity based on changes in the ultrasound echogenicity of the muscle during contraction. Individual digit movements were associated with unique maps of activity. These maps were correlated with previously acquired training data to classify individual digit movements. Preliminary results using ten healthy volunteers demonstrated this approach could provide robust classification of individual finger movements with 98% accuracy (precision 96%-100% and recall 97%-100% for individual finger flexions). The change in ultrasound echogenicity was found to be proportional to the digit flexion speed (R(2)=0.9), and thus our proposed strategy provided a proportional signal that can be used for fine control. We anticipate that ultrasound imaging based control strategies could be a significant improvement over conventional myoelectric control of prosthetics.

Implantable multichannel wireless electromyography for prosthesis control

We have developed a prototype implantable device for recording multiple independent channels of EMG and sending those signals wirelessly to an external receiver. This design records multichannel EMG signals for providing simultaneous multi-axis control of prosthetic limbs. This proof-of-concept study demonstrates benchtop performance of the bioamplifier in dry and soaked in saline configurations, as well as system performance in a short-term in vivo study in six dogs. The amplifier was shown to have an input-referred noise of 2.2 µV(RMS), a common mode rejection ratio greater than 55 dB, and neighboring channel isolation averaging 66 dB. The prototype devices were constructed of an amplifier ASIC along with discrete components for wireless function. These devices were coated in silicone and implanted for at least one week in each dog. EMG recorded from each animal as it walked down a hallway had very low noise and swing/stance phases of gait were clearly shown. This study demonstrates this device design can be used to amplify and transmit muscle signals.

BioPatRec: A modular research platform for the control of artificial limbs based on pattern recognition algorithms

Background

Processing and pattern recognition of myoelectric signals have been at the core of prosthetic control research in the last decade. Although most studies agree on reporting the accuracy of predicting predefined movements, there is a significant amount of study-dependent variables that make high-resolution inter-study comparison practically impossible. As an effort to provide a common research platform for the development and evaluation of algorithms in prosthetic control, we introduce BioPatRec as open source software. BioPatRec allows a seamless implementation of a variety of algorithms in the fields of (1) Signal processing; (2) Feature selection and extraction; (3) Pattern recognition; and, (4) Real-time control. Furthermore, since the platform is highly modular and customizable, researchers from different fields can seamlessly benchmark their algorithms by applying them in prosthetic control, without necessarily knowing how to obtain and process bioelectric signals, or how to produce and evaluate physically meaningful outputs.

Results

BioPatRec is demonstrated in this study by the implementation of a relatively new pattern recognition algorithm, namely Regulatory Feedback Networks (RFN). RFN produced comparable results to those of more sophisticated classifiers such as Linear Discriminant Analysis and Multi-Layer Perceptron. BioPatRec is released with these 3 fundamentally different classifiers, as well as all the necessary routines for the myoelectric control of a virtual hand; from data acquisition to real-time evaluations. All the required instructions for use and development are provided in the online project hosting platform, which includes issue tracking and an extensive “wiki”. This transparent implementation aims to facilitate collaboration and speed up utilization. Moreover, BioPatRec provides a publicly available repository of myoelectric signals that allow algorithms benchmarking on common data sets. This is particularly useful for researchers lacking of data acquisition hardware, or with limited access to patients.

Conclusions

BioPatRec has been made openly and freely available with the hope to accelerate, through the community contributions, the development of better algorithms that can potentially improve the patient’s quality of life. It is currently used in 3 different continents and by researchers of different disciplines, thus proving to be a useful tool for development and collaboration.

Classification of simultaneous movements using surface EMG pattern recognition

Advanced upper limb prostheses capable of actuating multiple degrees of freedom (DOFs) are now commercially available. Pattern recognition algorithms that use surface electromyography (EMG) signals show great promise as multi-DOF controllers. Unfortunately, current pattern recognition systems are limited to activate only one DOF at a time. This study introduces a novel classifier based on Bayesian theory to provide classification of simultaneous movements. This approach and two other classification strategies for simultaneous movements were evaluated using nonamputee and amputee subjects classifying up to three DOFs, where any two DOFs could be classified simultaneously. Similar results were found for nonamputee and amputee subjects. The new approach, based on a set of conditional parallel classifiers was the most promising with errors significantly less (p < 0.05) than a single linear discriminant analysis (LDA) classifier or a parallel approach. For three-DOF classification, the conditional parallel approach had error rates of 6.6% on discrete and 10.9% on combined motions, while the single LDA had error rates of 9.4% on discrete and 14.1% on combined motions. The low error rates demonstrated suggest than pattern recognition techniques on surface EMG can be extended to identify simultaneous movements, which could provide more life-like motions for amputees compared to exclusively classifying sequential movements.

Optimal control-based bayesian detection of clinical and behavioral state transitions

Accurately detecting hidden clinical or behavioral states from sequential measurements is an emerging topic in neuroscience and medicine, which may dramatically impact neural prosthetics, brain-computer interface and drug delivery. For example, early detection of an epileptic seizure from sequential electroencephalographic (EEG) measurements would allow timely administration of anticonvulsant drugs or neurostimulation, thus reducing physical impairment and risks of overtreatment. We develop a Bayesian paradigm for state transition detection that combines optimal control and Markov processes. We define a hidden Markov model of the state evolution and develop a detection policy that minimizes a loss function of both probability of false positives and accuracy (i.e., lag between estimated and actual transition time). Our strategy automatically adapts to each newly acquired measurement based on the state evolution model and the relative loss for false positives and accuracy, thus resulting in a time varying threshold policy. The paradigm was used in two applications: 1) detection of movement onset (behavioral state) from subthalamic single unit recordings in Parkinson's disease patients performing a motor task; 2) early detection of an approaching seizure (clinical state) from multichannel intracranial EEG recordings in rodents treated with pentylenetetrazol chemoconvulsant. Our paradigm performs significantly better than chance and improves over widely used detection algorithms.

On the viability of implantable electrodes for the natural control of artificial limbs: review and discussion

The control of robotic prostheses based on pattern recognition algorithms is a widely studied subject that has shown promising results in acute experiments. The long-term implementation of this technology, however, has not yet been achieved due to practical issues that can be mainly attributed to the use of surface electrodes and their highly environmental dependency. This paper describes several implantable electrodes and discusses them as a solution for the natural control of artificial limbs. In this context "natural" is defined as producing control over limb movement analogous to that of an intact physiological system. This includes coordinated and simultaneous movements of different degrees of freedom. It also implies that the input signals must come from nerves or muscles that were originally meant to produce the intended movement and that feedback is perceived as originating in the missing limb without requiring burdensome levels of concentration. After scrutinizing different electrode designs and their clinical implementation, we concluded that the epimysial and cuff electrodes are currently promising candidates to achieving a long-term stable and natural control of robotic prosthetics, provided that communication from the electrodes to the outside of the body is guaranteed.

Decoding of grasping information from neural signals recorded using peripheral intrafascicular interfaces

Background

The restoration of complex hand functions by creating a novel bidirectional link between the nervous system and a dexterous hand prosthesis is currently pursued by several research groups. This connection must be fast, intuitive, with a high success rate and quite natural to allow an effective bidirectional flow of information between the user's nervous system and the smart artificial device. This goal can be achieved with several approaches and among them, the use of implantable interfaces connected with the peripheral nervous system, namely intrafascicular electrodes, is considered particularly interesting.

Methods

Thin-film longitudinal intra-fascicular electrodes were implanted in the median and ulnar nerves of an amputee's stump during a four-week trial. The possibility of decoding motor commands suitable to control a dexterous hand prosthesis was investigated for the first time in this research field by implementing a spike sorting and classification algorithm.

Results

The results showed that motor information (e.g., grip types and single finger movements) could be extracted with classification accuracy around 85% (for three classes plus rest) and that the user could improve his ability to govern motor commands over time as shown by the improved discrimination ability of our classification algorithm.

Conclusions

These results open up new and promising possibilities for the development of a neuro-controlled hand prosthesis.

Development of an implantable myoelectric sensor for advanced prosthesis control

Modern hand and wrist prostheses afford a high level of mechanical sophistication, but the ability to control them in an intuitive and repeatable manner lags. Commercially available systems using surface electromyographic (EMG) or myoelectric control can supply at best two degrees of freedom (DOF), most often sequentially controlled. This limitation is partially due to the nature of surface-recorded EMG, for which the signal contains components from multiple muscle sources. We report here on the development of an implantable myoelectric sensor using EMG sensors that can be chronically implanted into an amputee's residual muscles. Because sensing occurs at the source of muscle contraction, a single principal component of EMG is detected by each sensor, corresponding to intent to move a particular effector. This system can potentially provide independent signal sources for control of individual effectors within a limb prosthesis. The use of implanted devices supports inter-day signal repeatability. We report on efforts in preparation for human clinical trials, including animal testing, and a first-in-human proof of principle demonstration where the subject was able to intuitively and simultaneously control two DOF in a hand and wrist prosthesis.