Progress towards restoring upper limb movement and sensation through intracortical brain-computer interfaces

Simultaneous cortical, subcortical, and brainstem mapping of sensory activation

Nonpainful tactile sensory stimuli are processed in the cortex, subcortex, and brainstem. Recent functional magnetic resonance imaging studies have highlighted the value of whole-brain, systems-level investigation for examining sensory processing. However, whole-brain functional magnetic resonance imaging studies are uncommon, in part due to challenges with signal to noise when studying the brainstem. Furthermore, differentiation of small sensory brainstem structures such as the cuneate and gracile nuclei necessitates high-resolution imaging. To address this gap in systems-level sensory investigation, we employed a whole-brain, multi-echo functional magnetic resonance imaging acquisition at 3T with multi-echo independent component analysis denoising and brainstem-specific modeling to enable detection of activation across the entire sensory system. In healthy participants, we examined patterns of activity in response to nonpainful brushing of the right hand, left hand, and right foot (n = 10 per location), and found the expected lateralization, with distinct cortical and subcortical responses for upper and lower limb stimulation. At the brainstem level, we differentiated the adjacent cuneate and gracile nuclei, corresponding to hand and foot stimulation respectively. Our findings demonstrate that simultaneous cortical, subcortical, and brainstem mapping at 3T could be a key tool to understand the sensory system in both healthy individuals and clinical cohorts with sensory deficits.

Simultaneous cortical, subcortical, and brainstem mapping of sensory activation

Non-painful tactile sensory stimuli are processed in the cortex, subcortex, and brainstem. Recent functional magnetic resonance imaging (fMRI) studies have highlighted the value of whole-brain, systems-level investigation for examining pain processing. However, whole-brain fMRI studies are uncommon, in part due to challenges with signal to noise when studying the brainstem. Furthermore, the differentiation of small sensory brainstem structures such as the cuneate and gracile nuclei necessitates high resolution imaging. To address this gap in systems-level sensory investigation, we employed a whole-brain, multi-echo fMRI acquisition at 3T with multi-echo independent component analysis (ME-ICA) denoising and brainstem-specific modeling to enable detection of activation across the entire sensory system. In healthy participants, we examined patterns of activity in response to non-painful brushing of the right hand, left hand, and right foot, and found the expected lateralization, with distinct cortical and subcortical responses for upper and lower limb stimulation. At the brainstem level, we were able to differentiate the small, adjacent cuneate and gracile nuclei, corresponding to hand and foot stimulation respectively. Our findings demonstrate that simultaneous cortical, subcortical, and brainstem mapping at 3T could be a key tool to understand the sensory system in both healthy individuals and clinical cohorts with sensory deficits.

Intracortical brain-computer interfaces for improved motor function: a systematic review

In this systematic review, we address the status of intracortical brain-computer interfaces (iBCIs) applied to the motor cortex to improve function in patients with impaired motor ability. This study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 Guidelines for Systematic Reviews. Risk Of Bias In Non-randomized Studies - of Interventions (ROBINS-I) and the Effective Public Health Practice Project (EPHPP) were used to assess bias and quality. Advances in iBCIs in the last two decades demonstrated the use of iBCI to activate limbs for functional tasks, achieve neural typing for communication, and other applications. However, the inconsistency of performance metrics employed by these studies suggests the need for standardization. Each study was a pilot clinical trial consisting of 1-4, majority male (64.28 %) participants, with most trials featuring participants treated for more than 12 months (55.55 %). The systems treated patients with various conditions: amyotrophic lateral sclerosis, stroke, spinocerebellar degeneration without cerebellar involvement, and spinal cord injury. All participants presented with tetraplegia at implantation and were implanted with microelectrode arrays via pneumatic insertion, with nearly all electrode locations solely at the precentral gyrus of the motor cortex (88.88 %). The development of iBCI devices using neural signals from the motor cortex to improve motor-impaired patients has enhanced the ability of these systems to return ability to their users. However, many milestones remain before these devices can prove their feasibility for recovery. This review summarizes the achievements and shortfalls of these systems and their respective trials.

Reactive Amine Functionalized Microelectrode Arrays Provide Short-Term Benefit but Long-Term Detriment to In Vivo Recording Performance

Intracortical microelectrode arrays (MEAs) are used for recording neural signals. However, indwelling devices result in chronic neuroinflammation, which leads to decreased recording performance through degradation of the device and surrounding tissue. Coating the MEAs with bioactive molecules is being explored to mitigate neuroinflammation. Such approaches often require an intermediate functionalization step such as (3-aminopropyl)triethoxysilane (APTES), which serves as a linker. However, the standalone effect of this intermediate step has not been previously characterized. Here, we investigated the effect of coating MEAs with APTES by comparing APTES-coated to uncoated controls in vivo and ex vivo. First, we measured water contact angles between silicon uncoated and APTES-coated substrates to verify the hydrophilic characteristics of the APTES coating. Next, we implanted MEAs in the motor cortex (M1) of Sprague-Dawley rats with uncoated or APTES-coated devices. We assessed changes in the electrochemical impedance and neural recording performance over a chronic implantation period of 16 weeks. Additionally, histology and bulk gene expression were analyzed to understand further the reactive tissue changes arising from the coating. Results showed that APTES increased the hydrophilicity of the devices and decreased electrochemical impedance at 1 kHz. APTES coatings proved detrimental to the recording performance, as shown by a constant decay up to 16 weeks postimplantation. Bulk gene analysis showed differential changes in gene expression between groups that were inconclusive with regard to the long-term effect on neuronal tissue. Together, these results suggest that APTES coatings are ultimately detrimental to chronic neural recordings. Furthermore, interpretations of studies using APTES as a functionalization step should consider the potential consequences if the final functionalization step is incomplete.

Neuronal representation of bimanual arm motor imagery in the motor cortex of a tetraplegia human, a pilot study

Introduction

How the human brain coordinates bimanual movements is not well-established.

Methods

Here, we recorded neural signals from a paralyzed individual's left motor cortex during both unimanual and bimanual motor imagery tasks and quantified the representational interaction between arms by analyzing the tuning parameters of each neuron.

Results

We found a similar proportion of neurons preferring each arm during unimanual movements, however, when switching to bimanual movements, the proportion of contralateral preference increased to 71.8%, indicating contralateral lateralization. We also observed a decorrelation process for each arm's representation across the unimanual and bimanual tasks. We further confined that these changes in bilateral relationships are mainly caused by the alteration of tuning parameters, such as the increased bilateral preferred direction (PD) shifts and the significant suppression in bilateral modulation depths (MDs), especially the ipsilateral side.

Discussion

These results contribute to the knowledge of bimanual coordination and thus the design of cutting-edge bimanual brain-computer interfaces.

Overcoming the Domain Gap in Neural Action Representations

Relating behavior to brain activity in animals is a fundamental goal in neuroscience, with practical applications in building robust brain-machine interfaces. However, the domain gap between individuals is a major issue that prevents the training of general models that work on unlabeled subjects. Since 3D pose data can now be reliably extracted from multi-view video sequences without manual intervention, we propose to use it to guide the encoding of neural action representations together with a set of neural and behavioral augmentations exploiting the properties of microscopy imaging. To test our method, we collect a large dataset that features flies and their neural activity. To reduce the domain gap, during training, we mix features of neural and behavioral data across flies that seem to be performing similar actions. To show our method can generalize further neural modalities and other downstream tasks, we test our method on a human neural Electrocorticography dataset, and another RGB video data of human activities from different viewpoints. We believe our work will enable more robust neural decoding algorithms to be used in future brain-machine interfaces.

Clinical neuroscience and neurotechnology: An amazing symbiosis

In the last decades, clinical neuroscience found a novel ally in neurotechnologies, devices able to record and stimulate electrical activity in the nervous system. These technologies improved the ability to diagnose and treat neural disorders. Neurotechnologies are concurrently enabling a deeper understanding of healthy and pathological dynamics of the nervous system through stimulation and recordings during brain implants. On the other hand, clinical neurosciences are not only driving neuroengineering toward the most relevant clinical issues, but are also shaping the neurotechnologies thanks to clinical advancements. For instance, understanding the etiology of a disease informs the location of a therapeutic stimulation, but also the way stimulation patterns should be designed to be more effective/naturalistic. Here, we describe cases of fruitful integration such as Deep Brain Stimulation and cortical interfaces to highlight how this symbiosis between clinical neuroscience and neurotechnology is closer to a novel integrated framework than to a simple interdisciplinary interaction.

Learning neural decoders without labels using multiple data streams

OBJECTIVE

Recent advances in neural decoding have accelerated the development of brain-computer interfaces aimed at assisting users with everyday tasks such as speaking, walking, and manipulating objects. However, current approaches for training neural decoders commonly require large quantities of labeled data, which can be laborious or infeasible to obtain in real-world settings. Alternatively, self-supervised models that share self-generated pseudo-labels between two data streams have shown exceptional performance on unlabeled audio and video data, but it remains unclear how well they extend to neural decoding.

APPROACH

We learn neural decoders without labels by leveraging multiple simultaneously recorded data streams, including neural, kinematic, and physiological signals. Specifically, we apply cross-modal, self-supervised deep clustering to train decoders that can classify movements from brain recordings. After training, we then isolate the decoders for each input data stream and compare the accuracy of decoders trained using cross-modal deep clustering against supervised and unimodal, self-supervised models.

MAIN RESULTS

We find that sharing pseudo-labels between two data streams during training substantially increases decoding performance compared to unimodal, self-supervised models, with accuracies approaching those of supervised decoders trained on labeled data. Next, we extend cross-modal decoder training to three or more modalities, achieving state-of-the-art neural decoding accuracy that matches or slightly exceeds the performance of supervised models. Significance: We demonstrate that cross-modal, self-supervised decoding can be applied to train neural decoders when few or no labels are available and extend the cross-modal framework to share information among three or more data streams, further improving self-supervised training.

Artificial Intelligence Enables Real-Time and Intuitive Control of Prostheses via Nerve Interface

OBJECTIVE

The next generation prosthetic hand that moves and feels like a real hand requires a robust neural interconnection between the human minds and machines.

METHODS

Here we present a neuroprosthetic system to demonstrate that principle by employing an artificial intelligence (AI) agent to translate the amputees movement intent through a peripheral nerve interface. The AI agent is designed based on the recurrent neural network (RNN) and could simultaneously decode six degree-of-freedom (DOF) from multichannel nerve data in real-time. The decoder's performance is characterized in motor decoding experiments with three human amputees.

RESULTS

First, we show the AI agent enables amputees to intuitively control a prosthetic hand with individual finger and wrist movements up to 97-98% accuracy. Second, we demonstrate the AI agent's real-time performance by measuring the reaction time and information throughput in a hand gesture matching task. Third, we investigate the AI agent's long-term uses and show the decoder's robust predictive performance over a 16-month implant duration. Conclusion & significance: Our study demonstrates the potential of AI-enabled nerve technology, underling the next generation of dexterous and intuitive prosthetic hands.

Rotational dynamics in motor cortex are consistent with a feedback controller

Recent studies have identified rotational dynamics in motor cortex (MC), which many assume arise from intrinsic connections in MC. However, behavioral and neurophysiological studies suggest that MC behaves like a feedback controller where continuous sensory feedback and interactions with other brain areas contribute substantially to MC processing. We investigated these apparently conflicting theories by building recurrent neural networks that controlled a model arm and received sensory feedback from the limb. Networks were trained to counteract perturbations to the limb and to reach toward spatial targets. Network activities and sensory feedback signals to the network exhibited rotational structure even when the recurrent connections were removed. Furthermore, neural recordings in monkeys performing similar tasks also exhibited rotational structure not only in MC but also in somatosensory cortex. Our results argue that rotational structure may also reflect dynamics throughout the voluntary motor system involved in online control of motor actions.

Finding Kinematics-Driven Latent Neural States From Neuronal Population Activity for Motor Decoding

While intracortical brain-machine interfaces (BMIs) demonstrate feasibility to restore mobility to people with paralysis, it is still challenging to maintain high-performance decoding in clinical BMIs. One of the main obstacles for high-performance BMI is the noise-prone nature of traditional decoding methods that connect neural response explicitly with physical quantity, such as velocity. In contrast, the recent development of latent neural state model enables a robust readout of large-scale neuronal population activity contents. However, these latent neural states do not necessarily contain kinematic information useful for decoding. Therefore, this study proposes a new approach to finding kinematics-dependent latent factors by extracting latent factors' kinematics-dependent components using linear regression. We estimated these components from the population activity through nonlinear mapping. The proposed kinematics-dependent latent factors generate neural trajectories that discriminate latent neural states before and after the motion onset. We compared the decoding performance of the proposed analysis model with the results from other popular models. They are factor analysis (FA), Gaussian process factor analysis (GPFA), latent factor analysis via dynamical systems (LFADS), preferential subspace identification (PSID), and neuronal population firing rates. The proposed analysis model results in higher decoding accuracy than do the others ( % improvement on average). Our approach may pave a new way to extract latent neural states specific to kinematic information from motor cortices, potentially improving decoding performance for online intracortical BMIs.

Encoding of limb state by single neurons in the cuneate nucleus of awake monkeys

The cuneate nucleus (CN) is among the first sites along the neuraxis where proprioceptive signals can be integrated, transformed, and modulated. The objective of the study was to characterize the proprioceptive representations in CN. To this end, we recorded from single CN neurons in three monkeys during active reaching and passive limb perturbation. We found that many neurons exhibited responses that were tuned approximately sinusoidally to limb movement direction, as has been found for other sensorimotor neurons. The distribution of their preferred directions (PDs) was highly nonuniform and resembled that of muscle spindles within individual muscles, suggesting that CN neurons typically receive inputs from only a single muscle. We also found that the responses of proprioceptive CN neurons tended to be modestly amplified during active reaching movements compared to passive limb perturbations, in contrast to cutaneous CN neurons whose responses were not systematically different in the active and passive conditions. Somatosensory signals thus seem to be subject to a "spotlighting" of relevant sensory information rather than uniform suppression as has been suggested previously.NEW & NOTEWORTHY The cuneate nucleus (CN) is the somatosensory gateway into the brain, and only recently has it been possible to record these signals from an awake animal. We recorded single CN neurons in monkeys. Proprioceptive CN neurons appear to receive input from very few muscles, and their sensitivity to movement changes reliably during reaching relative to passive arm perturbations. Sensitivity is generally increased, but not exclusively so, as though CN "spotlights" critical proprioceptive information during reaching.

Perception of microstimulation frequency in human somatosensory cortex

Microstimulation in the somatosensory cortex can evoke artificial tactile percepts and can be incorporated into bidirectional brain–computer interfaces (BCIs) to restore function after injury or disease. However, little is known about how stimulation parameters themselves affect perception. Here, we stimulated through microelectrode arrays implanted in the somatosensory cortex of two human participants with cervical spinal cord injury and varied the stimulus amplitude, frequency, and train duration. Increasing the amplitude and train duration increased the perceived intensity on all tested electrodes. Surprisingly, we found that increasing the frequency evoked more intense percepts on some electrodes but evoked less-intense percepts on other electrodes. These different frequency–intensity relationships were divided into three groups, which also evoked distinct percept qualities at different stimulus frequencies. Neighboring electrode sites were more likely to belong to the same group. These results support the idea that stimulation frequency directly controls tactile perception and that these different percepts may be related to the organization of somatosensory cortex, which will facilitate principled development of stimulation strategies for bidirectional BCIs.

Generalized neural decoders for transfer learning across participants and recording modalities

OBJECTIVE

Advances in neural decoding have enabled brain-computer interfaces to perform increasingly complex and clinically-relevant tasks. However, such decoders are often tailored to specific participants, days, and recording sites, limiting their practical long-term usage. Therefore, a fundamental challenge is to develop neural decoders that can robustly train on pooled, multi-participant data and generalize to new participants.

APPROACH

We introduce a new decoder, HTNet, which uses a convolutional neural network with two innovations: (1) a Hilbert transform that computes spectral power at data-driven frequencies and (2) a layer that projects electrode-level data onto predefined brain regions. The projection layer critically enables applications with intracranial electrocorticography (ECoG), where electrode locations are not standardized and vary widely across participants. We trained HTNet to decode arm movements using pooled ECoG data from 11 of 12 participants and tested performance on unseen ECoG or electroencephalography (EEG) participants; these pretrained models were also subsequently fine-tuned to each test participant.

MAIN RESULTS

HTNet outperformed state-of-the-art decoders when tested on unseen participants, even when a different recording modality was used. By fine-tuning these generalized HTNet decoders, we achieved performance approaching the best tailored decoders with as few as 50 ECoG or 20 EEG events. We were also able to interpret HTNet's trained weights and demonstrate its ability to extract physiologically-relevant features.

SIGNIFICANCE

By generalizing to new participants and recording modalities, robustly handling variations in electrode placement, and allowing participant-specific fine-tuning with minimal data, HTNet is applicable across a broader range of neural decoding applications compared to current state-of-the-art decoders.

Restoring Somatosensation: Advantages and Current Limitations of Targeting the Brainstem Dorsal Column Nuclei Complex

Current neural prostheses can restore limb movement to tetraplegic patients by translating brain signals coding movements to control a variety of actuators. Fast and accurate somatosensory feedback is essential for normal movement, particularly dexterous tasks, but is currently lacking in motor neural prostheses. Attempts to restore somatosensory feedback have largely focused on cortical stimulation which, thus far, have succeeded in eliciting minimal naturalistic sensations. Yet, a question that deserves more attention is whether the cortex is the best place to activate the central nervous system to restore somatosensation. Here, we propose that the brainstem dorsal column nuclei are an ideal alternative target to restore somatosensation. We review some of the recent literature investigating the dorsal column nuclei functional organization and neurophysiology and highlight some of the advantages and limitations of the dorsal column nuclei as a future neural prosthetic target. Recent evidence supports the dorsal column nuclei as a potential neural prosthetic target, but also identifies several gaps in our knowledge as well as potential limitations which need to be addressed before such a goal can become reality.

Advanced technologies for intuitive control and sensation of prosthetics

The Department of Defense, Department of Veterans Affairs and National Institutes of Health have invested significantly in advancing prosthetic technologies over the past 25 years, with the overall intent to improve the function, participation and quality of life of Service Members, Veterans, and all United States Citizens living with limb loss. These investments have contributed to substantial advancements in the control and sensory perception of prosthetic devices over the past decade. While control of motorized prosthetic devices through the use of electromyography has been widely available since the 1980s, this technology is not intuitive. Additionally, these systems do not provide stimulation for sensory perception. Recent research has made significant advancement not only in the intuitive use of electromyography for control but also in the ability to provide relevant meaningful perceptions through various stimulation approaches. While much of this previous work has traditionally focused on those with upper extremity amputation, new developments include advanced bidirectional neuroprostheses that are applicable to both the upper and lower limb amputation. The goal of this review is to examine the state-of-the-science in the areas of intuitive control and sensation of prosthetic devices and to discuss areas of exploration for the future. Current research and development efforts in external systems, implanted systems, surgical approaches, and regenerative approaches will be explored.

Workshops of the Seventh International Brain-Computer Interface Meeting: Not Getting Lost in Translation

The Seventh International Brain-Computer Interface (BCI) Meeting was held May 21-25th, 2018 at the Asilomar Conference Grounds, Pacific Grove, California, United States. The interactive nature of this conference was embodied by 25 workshops covering topics in BCI (also called brain-machine interface) research. Workshops covered foundational topics such as hardware development and signal analysis algorithms, new and imaginative topics such as BCI for virtual reality and multi-brain BCIs, and translational topics such as clinical applications and ethical assumptions of BCI development. BCI research is expanding in the diversity of applications and populations for whom those applications are being developed. BCI applications are moving toward clinical readiness as researchers struggle with the practical considerations to make sure that BCI translational efforts will be successful. This paper summarizes each workshop, providing an overview of the topic of discussion, references for additional information, and identifying future issues for research and development that resulted from the interactions and discussion at the workshop.

Creating a neuroprosthesis for active tactile exploration of textures

Intracortical microstimulation (ICMS) of the primary somatosensory cortex (S1) can produce percepts that mimic somatic sensation and, thus, has potential as an approach to sensorize prosthetic limbs. However, it is not known whether ICMS could recreate active texture exploration-the ability to infer information about object texture by using one's fingertips to scan a surface. Here, we show that ICMS of S1 can convey information about the spatial frequencies of invisible virtual gratings through a process of active tactile exploration. Two rhesus monkeys scanned pairs of visually identical screen objects with the fingertip of a hand avatar-controlled first via a joystick and later via a brain-machine interface-to find the object with denser virtual gratings. The gratings consisted of evenly spaced ridges that were signaled through individual ICMS pulses generated whenever the avatar's fingertip crossed a ridge. The monkeys learned to interpret these ICMS patterns, evoked by the interplay of their voluntary movements and the virtual textures of each object, to perform a sensory discrimination task. Discrimination accuracy followed Weber's law of just-noticeable differences (JND) across a range of grating densities; a finding that matches normal cutaneous sensation. Moreover, 1 monkey developed an active scanning strategy where avatar velocity was integrated with the ICMS pulses to interpret the texture information. We propose that this approach could equip upper-limb neuroprostheses with direct access to texture features acquired during active exploration of natural objects.