Concurrent control of a brain-computer interface and natural overt movements

Control limitations in the null-space of the wrist muscle system

The redundancy present within the musculoskeletal system may offer a non-invasive source of signals for movement augmentation, where the set of muscle activations that do not produce force/torque (muscle-to-force null-space) could be controlled simultaneously to the natural limbs. Here, we investigated the viability of extracting movement augmentation control signals from the muscles of the wrist complex. Our study assessed (i) if controlled variation of the muscle activation patterns in the wrist joint's null-space is possible; and (ii) whether force and null-space cursor targets could be reached concurrently. During the null-space target reaching condition, participants used muscle-to-force null-space muscle activation to move their cursor towards a displayed target while minimising the exerted force as visualised through the cursor's size. Initial targets were positioned to require natural co-contraction in the null-space and if participants showed a consistent ability to reach for their current target, they would rotate 5∘ incrementally to generate muscle activation patterns further away from their natural co-contraction. In contrast, during the concurrent target reaching condition participants were required to match a target position and size, where their cursor position was instead controlled by their exerted flexion-extension and radial-ulnar deviation, while its size was changed by their natural co-contraction magnitude. The results collected from 10 participants suggest that while there was variation in each participant's co-contraction behaviour, most did not possess the ability to control this variation for muscle-to-force null-space virtual reaching. In contrast, participants did show a direction and target size dependent ability to vary isometric force and co-contraction activity concurrently. Our results indicate the limitations of using the muscle-to-force null-space activity of joints with a low level of redundancy as a possible command signal for movement augmentation.

Human motor augmentation with an extra robotic arm without functional interference

Extra robotic arms (XRAs) are gaining interest in neuroscience and robotics, offering potential tools for daily activities. However, this compelling opportunity poses new challenges for sensorimotor control strategies and human-machine interfaces (HMIs). A key unsolved challenge is allowing users to proficiently control XRAs without hindering their existing functions. To address this, we propose a pipeline to identify suitable HMIs given a defined task to accomplish with the XRA. Following such a scheme, we assessed a multimodal motor HMI based on gaze detection and diaphragmatic respiration in a purposely designed modular neurorobotic platform integrating virtual reality and a bilateral upper limb exoskeleton. Our results show that the proposed HMI does not interfere with speaking or visual exploration and that it can be used to control an extra virtual arm independently from the biological ones or in coordination with them. Participants showed significant improvements in performance with daily training and retention of learning, with no further improvements when artificial haptic feedback was provided. As a final proof of concept, naïve and experienced participants used a simplified version of the HMI to control a wearable XRA. Our analysis indicates how the presented HMI can be effectively used to control XRAs. The observation that experienced users achieved a success rate 22.2% higher than that of naïve users, combined with the result that naïve users showed average success rates of 74% when they first engaged with the system, endorses the viability of both the virtual reality-based testing and training and the proposed pipeline.

Nine decades of electrocorticography: A comparison between epidural and subdural recordings

In recent years, electrocorticography (ECoG) has arisen as a neural signal recording tool in the development of clinically viable neural interfaces. ECoG electrodes are generally placed below the dura mater (subdural) but can also be placed on top of the dura (epidural). In deciding which of these modalities best suits long-term implants, complications and signal quality are important considerations. Conceptually, epidural placement may present a lower risk of complications as the dura is left intact but also a lower signal quality due to the dura acting as a signal attenuator. The extent to which complications and signal quality are affected by the dura, however, has been a matter of debate. To improve our understanding of the effects of the dura on complications and signal quality, we conducted a literature review. We inventorized the effect of the dura on signal quality, decodability and longevity of acute and chronic ECoG recordings in humans and non-human primates. Also, we compared the incidence and nature of serious complications in studies that employed epidural and subdural ECoG. Overall, we found that, even though epidural recordings exhibit attenuated signal amplitude over subdural recordings, particularly for high-density grids, the decodability of epidural recorded signals does not seem to be markedly affected. Additionally, we found that the nature of serious complications was comparable between epidural and subdural recordings. These results indicate that both epidural and subdural ECoG may be suited for long-term neural signal recordings, at least for current generations of clinical and high-density ECoG grids.

Principles of human movement augmentation and the challenges in making it a reality

Augmenting the body with artificial limbs controlled concurrently to one's natural limbs has long appeared in science fiction, but recent technological and neuroscientific advances have begun to make this possible. By allowing individuals to achieve otherwise impossible actions, movement augmentation could revolutionize medical and industrial applications and profoundly change the way humans interact with the environment. Here, we construct a movement augmentation taxonomy through what is augmented and how it is achieved. With this framework, we analyze augmentation that extends the number of degrees-of-freedom, discuss critical features of effective augmentation such as physiological control signals, sensory feedback and learning as well as application scenarios, and propose a vision for the field.

Simultaneous control of natural and extra degrees of freedom by isometric force and electromyographic activity in the muscle-to-force null space

OBJECTIVE

Muscle activation patterns in the muscle-to-force null space, i.e., patterns that do not generate task-relevant forces, may provide an opportunity for motor augmentation by allowing to control additional end-effectors simultaneously to natural limbs. Here we tested the feasibility of muscular null space control for augmentation by assessing simultaneous control of natural and extra degrees of freedom.

APPROACH

We instructed eight participants to control translation and rotation of a virtual 3D end-effector by simultaneous generation of isometric force at the hand and null space activity extracted in real-time from the electromyographic signals recorded from 15 shoulder and arm muscles. First, we identified the null space components that each participant could control more naturally by voluntary co-contraction. Then, participants performed several blocks of a reaching and holding task. They displaced an ellipsoidal cursor to reach one of nine targets by generating force, and simultaneously rotated the cursor to match the target orientation by activating null space components. We developed an information-theoretic metric, an index of difficulty defined as the sum of a spatial and a temporal term, to assess individual null space control ability for both reaching and holding.

MAIN RESULTS

On average, participants could reach the targets in most trials already in the first block (72%) and they improved with practice (maximum 93%) but holding performance remained lower (maximum 43%). As there was a high inter-individual variability in performance, we performed a simulation with different spatial and temporal task conditions to estimate those for which each individual participants would have performed best.

SIGNIFICANCE

Muscular null space control is feasible and may be used to control additional virtual or robotics end-effectors. However, decoding of motor commands must be optimized according to individual null space control ability.

Towards human motor augmentation by voluntary decoupling beta activity in the neural drive to muscle and force production

Objective.Effective human motor augmentation should rely on biological signals that can be volitionally modulated without compromising natural motor control.Approach.We provided human subjects with real-time information on the power of two separate spectral bands of the spiking activity of motor neurons innervating the tibialis anterior muscle: the low-frequency band (<7 Hz), which is directly translated into natural force control, and the beta band (13-30 Hz), which is outside the dynamics of the neuromuscular system.Main Results.Subjects could gain control over the powers in these two bands to navigate a cursor towards specific targets in a 2D space (experiment 1) and to up- and down-modulate beta activity while keeping steady force contractions (experiment 2).Significance.Results indicate that beta projections to the spinal motor neuron pool can be voluntarily controlled partially decoupled from natural muscle contractions and, therefore, they could be valid control signals for implementing effective human motor augmentation platforms.

Augmented manipulation ability in humans with six-fingered hands

Neurotechnology attempts to develop supernumerary limbs, but can the human brain deal with the complexity to control an extra limb and yield advantages from it? Here, we analyzed the neuromechanics and manipulation abilities of two polydactyly subjects who each possess six fingers on their hands. Anatomical MRI of the supernumerary finger (SF) revealed that it is actuated by extra muscles and nerves, and fMRI identified a distinct cortical representation of the SF. In both subjects, the SF was able to move independently from the other fingers. Polydactyly subjects were able to coordinate the SF with their other fingers for more complex movements than five fingered subjects, and so carry out with only one hand tasks normally requiring two hands. These results demonstrate that a body with significantly more degrees-of-freedom can be controlled by the human nervous system without causing motor deficits or impairments and can instead provide superior manipulation abilities.