Hand Shape Representations in the Human Posterior Parietal Cortex

Convolutional neural networks reveal properties of reach-to-grasp encoding in posterior parietal cortex

Deep neural networks (DNNs) are widely adopted to decode motor states from both non-invasively and invasively recorded neural signals, e.g., for realizing brain-computer interfaces. However, the neurophysiological interpretation of how DNNs make the decision based on the input neural activity is limitedly addressed, especially when applied to invasively recorded data. This reduces decoder reliability and transparency, and prevents the exploitation of decoders to better comprehend motor neural encoding. Here, we adopted an explainable artificial intelligence approach - based on a convolutional neural network and an explanation technique - to reveal spatial and temporal neural properties of reach-to-grasping from single-neuron recordings of the posterior parietal area V6A. The network was able to accurately decode 5 different grip types, and the explanation technique automatically identified the cells and temporal samples that most influenced the network prediction. Grip encoding in V6A neurons already started at movement preparation, peaking during movement execution. A difference was found within V6A: dorsal V6A neurons progressively encoded more for increasingly advanced grips, while ventral V6A neurons for increasingly rudimentary grips, with both subareas following a linear trend between the amount of grip encoding and the level of grip skills. By revealing the elements of the neural activity most relevant for each grip with no a priori assumptions, our approach supports and advances current knowledge about reach-to-grasp encoding in V6A, and it may represent a general tool able to investigate neural correlates of motor or cognitive tasks (e.g., attention and memory tasks) from single-neuron recordings.

The influence of hand posture on tactile processing: Evidence from a 7T functional magnetic resonance imaging study

Although behavioral evidence has shown that postural changes influence the ability to localize or detect tactile stimuli, little is known regarding the brain areas that modulate these effects. This 7T functional magnetic resonance imaging (fMRI) study explores the effects of touch of the hand as a function of hand location (right or left side of the body) and hand configuration (open or closed). We predicted that changes in hand configuration would be represented in contralateral primary somatosensory cortex (S1) and the anterior intraparietal area (aIPS), whereas change in position of the hand would be associated with alterations in activation in the superior parietal lobule. Multivoxel pattern analysis and a region of interest approach partially supported our predictions. Decoding accuracy for hand location was above chance level in superior parietal lobule (SPL) and in the anterior intraparietal (aIPS) area; above chance classification of hand configuration was observed in SPL and S1. This evidence confirmed the role of the parietal cortex in postural effects on touch and the possible role of S1 in coding the body form representation of the hand.

Reconnecting the Hand and Arm to the Brain: Efficacy of Neural Interfaces for Sensorimotor Restoration After Tetraplegia

BACKGROUND AND OBJECTIVES

Paralysis after spinal cord injury involves damage to pathways that connect neurons in the brain to peripheral nerves in the limbs. Re-establishing this communication using neural interfaces has the potential to bridge the gap and restore upper extremity function to people with high tetraplegia. We report a novel approach for restoring upper extremity function using selective peripheral nerve stimulation controlled by intracortical microelectrode recordings from sensorimotor networks, along with restoration of tactile sensation of the hand using intracortical microstimulation.

METHODS

A 27-year-old right-handed man with AIS-B (motor-complete, sensory-incomplete) C3-C4 tetraplegia was enrolled into the clinical trial. Six 64-channel intracortical microelectrode arrays were implanted into left hemisphere regions involved in upper extremity function, including primary motor and sensory cortices, inferior frontal gyrus, and anterior intraparietal area. Nine 16-channel extraneural peripheral nerve electrodes were implanted to allow targeted stimulation of right median, ulnar (2), radial, axillary, musculocutaneous, suprascapular, lateral pectoral, and long thoracic nerves, to produce selective muscle contractions on demand. Proof-of-concept studies were performed to demonstrate feasibility of using a brain-machine interface to read from and write to the brain for restoring motor and sensory functions of the participant's own arm and hand.

RESULTS

Multiunit neural activity that correlated with intended motor action was successfully recorded from intracortical arrays. Microstimulation of electrodes in somatosensory cortex produced repeatable sensory percepts of individual fingers for restoration of touch sensation. Selective electrical activation of peripheral nerves produced antigravity muscle contractions, resulting in functional movements that the participant was able to command under brain control to perform virtual and actual arm and hand movements. The system was well tolerated with no operative complications.

CONCLUSION

The combination of implanted cortical electrodes and nerve cuff electrodes has the potential to create bidirectional restoration of motor and sensory functions of the arm and hand after neurological injury.

Spatio-temporal evolution of human neural activity during visually cued hand movements

Making hand movements in response to visual cues is common in daily life. It has been well known that this process activates multiple areas in the brain, but how these neural activations progress across space and time remains largely unknown. Taking advantage of intracranial electroencephalographic (iEEG) recordings using depth and subdural electrodes from 36 human subjects using the same task, we applied single-trial and cross-trial analyses to high-frequency iEEG activity. The results show that the neural activation was widely distributed across the human brain both within and on the surface of the brain, and focused specifically on certain areas in the parietal, frontal, and occipital lobes, where parietal lobes present significant left lateralization on the activation. We also demonstrate temporal differences across these brain regions. Finally, we evaluated the degree to which the timing of activity within these regions was related to sensory or motor function. The findings of this study promote the understanding of task-related neural processing of the human brain, and may provide important insights for translational applications.

Decoding and geometry of ten finger movements in human posterior parietal cortex and motor cortex

Objective. Enable neural control of individual prosthetic fingers for participants with upper-limb paralysis.Approach. Two tetraplegic participants were each implanted with a 96-channel array in the left posterior parietal cortex (PPC). One of the participants was additionally implanted with a 96-channel array near the hand knob of the left motor cortex (MC). Across tens of sessions, we recorded neural activity while the participants attempted to move individual fingers of the right hand. Offline, we classified attempted finger movements from neural firing rates using linear discriminant analysis with cross-validation. The participants then used the neural classifier online to control individual fingers of a brain-machine interface (BMI). Finally, we characterized the neural representational geometry during individual finger movements of both hands.Main Results. The two participants achieved 86% and 92% online accuracy during BMI control of the contralateral fingers (chance = 17%). Offline, a linear decoder achieved ten-finger decoding accuracies of 70% and 66% using respective PPC recordings and 75% using MC recordings (chance = 10%). In MC and in one PPC array, a factorized code linked corresponding finger movements of the contralateral and ipsilateral hands.Significance. This is the first study to decode both contralateral and ipsilateral finger movements from PPC. Online BMI control of contralateral fingers exceeded that of previous finger BMIs. PPC and MC signals can be used to control individual prosthetic fingers, which may contribute to a hand restoration strategy for people with tetraplegia.

Neural representations of economic decision variables in human posterior parietal cortex

Decision making has been intensively studied in the posterior parietal cortex in non-human primates on a single neuron level. In humans decision making has mainly been studied with psychophysical tools or with fMRI. Here, we investigated how single neurons from human posterior parietal cortex represent numeric values informing future decisions during a complex two-player game. The tetraplegic study participant was implanted with a Utah electrode array in the anterior intraparietal area (AIP). We played a simplified variant of Black Jack with the participant while neuronal data was recorded. During the game two players are presented with numbers which are added up. Each time a number is presented the player has to decide to proceed or to stop. Once the first player stops or the score reaches a limit the turn passes on to the second player who tries to beat the score of the first player. Whoever is closer to the limit (without overshooting) wins the game. We found that many AIP neurons selectively responded to the face value of the presented number. Other neurons tracked the cumulative score or were selectively active for the upcoming decision of the study participant. Interestingly, some cells also kept track of the opponent's score. Our findings show that parietal regions engaged in hand action control also represent numbers and their complex transformations. This is also the first demonstration of complex economic decisions being possible to track in single neuron activity in human AIP. Our findings show how tight are the links between parietal neural circuits underlying hand control, numerical cognition and complex decision-making.

Magnetoencephalogram-based brain-computer interface for hand-gesture decoding using deep learning

Advancements in deep learning algorithms over the past decade have led to extensive developments in brain-computer interfaces (BCI). A promising imaging modality for BCI is magnetoencephalography (MEG), which is a non-invasive functional imaging technique. The present study developed a MEG sensor-based BCI neural network to decode Rock-Paper-scissors gestures (MEG-RPSnet). Unique preprocessing pipelines in tandem with convolutional neural network deep-learning models accurately classified gestures. On a single-trial basis, we found an average of 85.56% classification accuracy in 12 subjects. Our MEG-RPSnet model outperformed two state-of-the-art neural network architectures for electroencephalogram-based BCI as well as a traditional machine learning method, and demonstrated equivalent and/or better performance than machine learning methods that have employed invasive, electrocorticography-based BCI using the same task. In addition, MEG-RPSnet classification performance using an intra-subject approach outperformed a model that used a cross-subject approach. Remarkably, we also found that when using only central-parietal-occipital regional sensors or occipitotemporal regional sensors, the deep learning model achieved classification performances that were similar to the whole-brain sensor model. The MEG-RSPnet model also distinguished neuronal features of individual hand gestures with very good accuracy. Altogether, these results show that noninvasive MEG-based BCI applications hold promise for future BCI developments in hand-gesture decoding.

Reconnecting the Hand and Arm to the Brain: Efficacy of Neural Interfaces for Sensorimotor Restoration after Tetraplegia

Background

Paralysis after spinal cord injury involves damage to pathways that connect neurons in the brain to peripheral nerves in the limbs. Re-establishing this communication using neural interfaces has the potential to bridge the gap and restore upper extremity function to people with high tetraplegia.

Objective

We report a novel approach for restoring upper extremity function using selective peripheral nerve stimulation controlled by intracortical microelectrode recordings from sensorimotor networks, along with restoration of tactile sensation of the hand using intracortical microstimulation.

Methods

A right-handed man with motor-complete C3-C4 tetraplegia was enrolled into the clinical trial. Six 64-channel intracortical microelectrode arrays were implanted into left hemisphere regions involved in upper extremity function, including primary motor and sensory cortices, inferior frontal gyrus, and anterior intraparietal area. Nine 16-channel extraneural peripheral nerve electrodes were implanted to allow targeted stimulation of right median, ulnar (2), radial, axillary, musculocutaneous, suprascapular, lateral pectoral, and long thoracic nerves, to produce selective muscle contractions on demand. Proof-of-concept studies were performed to demonstrate feasibility of a bidirectional brain-machine interface to restore function of the participant's own arm and hand.

Results

Multi-unit neural activity that correlated with intended motor action was successfully recorded from intracortical arrays. Microstimulation of electrodes in somatosensory cortex produced repeatable sensory percepts of individual fingers for restoration of touch sensation. Selective electrical activation of peripheral nerves produced antigravity muscle contractions. The system was well tolerated with no operative complications.

Conclusion

The combination of implanted cortical electrodes and nerve cuff electrodes has the potential to allow restoration of motor and sensory functions of the arm and hand after neurological injury.

Characteristics and stability of sensorimotor activity driven by isolated-muscle group activation in a human with tetraplegia

Understanding the cortical representations of movements and their stability can shed light on improved brain-machine interface (BMI) approaches to decode these representations without frequent recalibration. Here, we characterize the spatial organization (somatotopy) and stability of the bilateral sensorimotor map of forearm muscles in an incomplete-high spinal-cord injury study participant implanted bilaterally in the primary motor and sensory cortices with Utah microelectrode arrays (MEAs). We built representation maps by recording bilateral multiunit activity (MUA) and surface electromyography (EMG) as the participant executed voluntary contractions of the extensor carpi radialis (ECR), and attempted motions in the flexor carpi radialis (FCR), which was paralytic. To assess stability, we repeatedly mapped and compared left- and right-wrist-extensor-related activity throughout several sessions, comparing somatotopy of active electrodes, as well as neural signals both at the within-electrode (multiunit) and cross-electrode (network) levels. Wrist motions showed significant activation in motor and sensory cortical electrodes. Within electrodes, firing strength stability diminished as the time increased between consecutive measurements (hours within a session, or days across sessions), with higher stability observed in sensory cortex than in motor, and in the contralateral hemisphere than in the ipsilateral. However, we observed no differences at network level, and no evidence of decoding instabilities for wrist EMG, either across timespans of hours or days, or across recording area. While map stability differs between brain area and hemisphere at multiunit/electrode level, these differences are nullified at ensemble level.

The science and engineering behind sensitized brain-controlled bionic hands

Advances in our understanding of brain function, along with the development of neural interfaces that allow for the monitoring and activation of neurons, have paved the way for brain-machine interfaces (BMIs), which harness neural signals to reanimate the limbs via electrical activation of the muscles or to control extracorporeal devices, thereby bypassing the muscles and senses altogether. BMIs consist of reading out motor intent from the neuronal responses monitored in motor regions of the brain and executing intended movements with bionic limbs, reanimated limbs, or exoskeletons. BMIs also allow for the restoration of the sense of touch by electrically activating neurons in somatosensory regions of the brain, thereby evoking vivid tactile sensations and conveying feedback about object interactions. In this review, we discuss the neural mechanisms of motor control and somatosensation in able-bodied individuals and describe approaches to use neuronal responses as control signals for movement restoration and to activate residual sensory pathways to restore touch. Although the focus of the review is on intracortical approaches, we also describe alternative signal sources for control and noninvasive strategies for sensory restoration.

Decoding grasp and speech signals from the cortical grasp circuit in a tetraplegic human

The cortical grasp network encodes planning and execution of grasps and processes spoken and written aspects of language. High-level cortical areas within this network are attractive implant sites for brain-machine interfaces (BMIs). While a tetraplegic patient performed grasp motor imagery and vocalized speech, neural activity was recorded from the supramarginal gyrus (SMG), ventral premotor cortex (PMv), and somatosensory cortex (S1). In SMG and PMv, five imagined grasps were well represented by firing rates of neuronal populations during visual cue presentation. During motor imagery, these grasps were significantly decodable from all brain areas. During speech production, SMG encoded both spoken grasp types and the names of five colors. Whereas PMv neurons significantly modulated their activity during grasping, SMG's neural population broadly encoded features of both motor imagery and speech. Together, these results indicate that brain signals from high-level areas of the human cortex could be used for grasping and speech BMI applications.

Enhancing the decoding accuracy of EEG signals by the introduction of anchored-STFT and adversarial data augmentation method

Brain-computer interfaces (BCIs) enable communication between humans and machines by translating brain activity into control commands. Electroencephalography (EEG) signals are one of the most used brain signals in non-invasive BCI applications but are often contaminated with noise. Therefore, it is possible that meaningful patterns for classifying EEG signals are deeply hidden. State-of-the-art deep-learning algorithms are successful in learning hidden, meaningful patterns. However, the quality and the quantity of the presented inputs are pivotal. Here, we propose a feature extraction method called anchored Short Time Fourier Transform (anchored-STFT), which is an advanced version of STFT, as it minimizes the trade-off between temporal and spectral resolution presented by STFT. In addition, we propose a data augmentation method derived from l2-norm fast gradient sign method (FGSM), called gradient norm adversarial augmentation (GNAA). GNAA is not only an augmentation method but is also used to harness adversarial inputs in EEG data, which not only improves the classification accuracy but also enhances the robustness of the classifier. In addition, we also propose a CNN architecture, namely Skip-Net, for the classification of EEG signals. The proposed pipeline outperforms the current state-of-the-art methods and yields classification accuracies of 90.7% on BCI competition II dataset III and 89.5%, 81.8%, 76.0% and 85.4%, 69.1%, 80.9% on different data distributions of BCI Competition IV dataset 2b and 2a, respectively.

Effects of anodal stimulation and motor practice on limb-kinetic apraxia in Parkinson's disease

Limb-kinetic apraxia, the loss of the ability to make precise, independent but coordinated finger and hand movements affects quality of life in patients with Parkinson's disease. We aimed to examine the effects of anodal transcranial direct current stimulation of the left posterior parietal cortex and upper extremity motor practice on limb-kinetic apraxia in Parkinson's disease. This study was conducted in a randomized, double-blind, sham-controlled fashion. Patients confirmed to have Parkinson's disease were recruited. Twenty-eight participants completed the study and were randomized to two groups: anodal or sham stimulation. For participants assigned to active stimulation, anodal stimulation of the left posterior parietal cortex was performed using 2 mA current for 20 min. Patients received anodal or sham stimulation, followed by motor practice in both groups. The primary outcome measure was time-performing sequential buttoning and unbuttoning, and several secondary outcome measures were obtained. A statistically significant interaction between stimulation type and timepoint on time taken to perform buttoning and unbuttoning was found. Patients who received anodal stimulation were found to have a significant decrease in sequential buttoning and unbuttoning time immediately following stimulation and at 24 h in the medication-ON state, compared to the medication-OFF state (31% and 29% decrease, respectively). Anodal stimulation of the left posterior parietal cortex prior to motor practice appears to be effective for limb-kinetic apraxia in Parkinson's disease. Future long-term, multi-session studies looking at the long-term effects of anodal stimulation and motor practice on limb-kinetic apraxia in Parkinson's disease may be worthwhile.

Exploring Cognition with Brain-Machine Interfaces

Traditional brain-machine interfaces decode cortical motor commands to control external devices. These commands are the product of higher-level cognitive processes, occurring across a network of brain areas, that integrate sensory information, plan upcoming motor actions, and monitor ongoing movements. We review cognitive signals recently discovered in the human posterior parietal cortex during neuroprosthetic clinical trials. These signals are consistent with small regions of cortex having a diverse role in cognitive aspects of movement control and body monitoring, including sensorimotor integration, planning, trajectory representation, somatosensation, action semantics, learning, and decision making. These variables are encoded within the same population of cells using structured representations that bind related sensory and motor variables, an architecture termed partially mixed selectivity. Diverse cognitive signals provide complementary information to traditional motor commands to enable more natural and intuitive control of external devices.

Data-efficient Causal Decoding of Spiking Neural Activity using Weighted Voting

Brain-Computer Interface systems can contribute to a vast set of applications such as overcoming physical disabilities in people with neural injuries or hands-free control of devices in healthy individuals. However, having systems that can accurately interpret intention online remains a challenge in this field. Robust and data-efficient decoding-despite the dynamical nature of cortical activity and causality requirements for physical function-is among the most important challenges that limit the widespread use of these devices for real-world applications. Here, we present a causal, data-efficient neural decoding pipeline that predicts intention by first classifying recordings in short sliding windows. Next, it performs weighted voting over initial predictions up to the current point in time to report a refined final prediction. We demonstrate its utility by classifying spiking neural activity collected from the human posterior parietal cortex for a cue, delay, imaginary motor task. This pipeline provides higher classification accuracy than state-of-the-art time windowed spiking activity based causal methods, and is robust to the choice of hyper-parameters.

Long-term intracortical microelectrode array performance in a human: a 5 year retrospective analysis

Objective. Brain-computer interfaces (BCIs) that record neural activity using intracortical microelectrode arrays (MEAs) have shown promise for mitigating disability associated with neurological injuries and disorders. While the chronic performance and failure modes of MEAs have been well studied and systematically described in non-human primates, there is far less reported about long-term MEA performance in humans. Our group has collected one of the largest neural recording datasets from a Utah MEA in a human subject, spanning over 5 years (2014-2019). Here we present both long-term signal quality and BCI performance as well as highlight several acute signal disruption events observed during the clinical study.Approach. Long-term Utah array performance was evaluated by analyzing neural signal metric trends and decoding accuracy for tasks regularly performed across 448 clinical recording sessions. For acute signal disruptions, we identify or hypothesize the root cause of the disruption, show how the disruption manifests in the collected data, and discuss potential identification and mitigation strategies for the disruption.Main results. Neural signal quality metrics deteriorated rapidly within the first year, followed by a slower decline through the remainder of the study. Nevertheless, BCI performance remained high 5 years after implantation, which is encouraging for the translational potential of this technology as an assistive device. We also present examples of unanticipated signal disruptions during chronic MEA use, which are critical to detect as BCI technology progresses toward home usage.Significance. Our work fills a gap in knowledge around long-term MEA performance in humans, providing longevity and efficacy data points to help characterize the performance of implantable neural sensors in a human population. The trial was registered on ClinicalTrials.gov (Identifier NCT01997125) and conformed to institutional requirements for the conduct of human subjects research.

Intracortical Microelectrode Array Unit Yield under Chronic Conditions: A Comparative Evaluation

While microelectrode arrays (MEAs) offer the promise of elucidating functional neural circuitry and serve as the basis for a cortical neuroprosthesis, the challenge of designing and demonstrating chronically reliable technology remains. Numerous studies report “chronic” data but the actual time spans and performance measures corresponding to the experimental work vary. In this study, we reviewed the experimental durations that constitute chronic studies across a range of MEA types and animal species to gain an understanding of the widespread variability in reported study duration. For rodents, which are the most commonly used animal model in chronic studies, we examined active electrode yield (AEY) for different array types as a means to contextualize the study duration variance, as well as investigate and interpret the performance of custom devices in comparison to conventional MEAs. We observed wide-spread variance within species for the chronic implantation period and an AEY that decayed linearly in rodent models that implanted commercially-available devices. These observations provide a benchmark for comparing the performance of new technologies and highlight the need for consistency in chronic MEA studies. Additionally, to fully derive performance under chronic conditions, the duration of abiotic failure modes, biological processes induced by indwelling probes, and intended application of the device are key determinants.

Detection of human white matter activation and evaluation of its function in movement decoding using stereo-electroencephalography (SEEG)

Objective. White matter tissue takes up approximately 50% of the human brain volume and it is widely known as a messenger conducting information between areas of the central nervous system. However, the characteristics of white matter neural activity and whether white matter neural recordings can contribute to movement decoding are often ignored and still remain largely unknown. In this work, we make quantitative analyses to investigate these two important questions using invasive neural recordings.Approach. We recorded stereo-electroencephalography (SEEG) data from 32 human subjects during a visually-cued motor task, where SEEG recordings can tap into gray and white matter electrical activity simultaneously. Using the proximal tissue density method, we identified the location (i.e. gray or white matter) of each SEEG contact. Focusing on alpha oscillatory and high gamma activities, we compared the activation patterns between gray matter and white matter. Then, we evaluated the performance of such white matter activation in movement decoding.Main results. The results show that white matter also presents activation under the task, in a similar way with the gray matter but at a significantly lower amplitude. Additionally, this work also demonstrates that combing white matter neural activities together with that of gray matter significantly promotes the movement decoding accuracy than using gray matter signals only.Significance. Taking advantage of SEEG recordings from a large number of subjects, we reveal the response characteristics of white matter neural signals under the task and demonstrate its enhancing function in movement decoding. This study highlights the importance of taking white matter activities into consideration in further scientific research and translational applications.

The Neural Representation of Force across Grasp Types in Motor Cortex of Humans with Tetraplegia

Intracortical brain-computer interfaces (iBCIs) have the potential to restore hand grasping and object interaction to individuals with tetraplegia. Optimal grasping and object interaction require simultaneous production of both force and grasp outputs. However, since overlapping neural populations are modulated by both parameters, grasp type could affect how well forces are decoded from motor cortex in a closed-loop force iBCI. Therefore, this work quantified the neural representation and offline decoding performance of discrete hand grasps and force levels in two human participants with tetraplegia. Participants attempted to produce three discrete forces (light, medium, hard) using up to five hand grasp configurations. A two-way Welch ANOVA was implemented on multiunit neural features to assess their modulation to force and grasp Demixed principal component analysis (dPCA) was used to assess for population-level tuning to force and grasp and to predict these parameters from neural activity. Three major findings emerged from this work: (1) force information was neurally represented and could be decoded across multiple hand grasps (and, in one participant, across attempted elbow extension as well); (2) grasp type affected force representation within multiunit neural features and offline force classification accuracy; and (3) grasp was classified more accurately and had greater population-level representation than force. These findings suggest that force and grasp have both independent and interacting representations within cortex, and that incorporating force control into real-time iBCI systems is feasible across multiple hand grasps if the decoder also accounts for grasp type.

Auditory cues reveal intended movement information in middle frontal gyrus neuronal ensemble activity of a person with tetraplegia

Intracortical brain-computer interfaces (iBCIs) allow people with paralysis to directly control assistive devices using neural activity associated with the intent to move. Realizing the full potential of iBCIs critically depends on continued progress in understanding how different cortical areas contribute to movement control. Here we present the first comparison between neuronal ensemble recordings from the left middle frontal gyrus (MFG) and precentral gyrus (PCG) of a person with tetraplegia using an iBCI. As expected, PCG was more engaged in selecting and generating intended movements than in earlier perceptual stages of action planning. By contrast, MFG displayed movement-related information during the sensorimotor processing steps preceding the appearance of the action plan in PCG, but only when the actions were instructed using auditory cues. These results describe a previously unreported function for neurons in the human left MFG in auditory processing contributing to motor control.

Neural encoding of actual and imagined touch within human posterior parietal cortex

In the human posterior parietal cortex (PPC), single units encode high-dimensional information with partially mixed representations that enable small populations of neurons to encode many variables relevant to movement planning, execution, cognition, and perception. Here, we test whether a PPC neuronal population previously demonstrated to encode visual and motor information is similarly engaged in the somatosensory domain. We recorded neurons within the PPC of a human clinical trial participant during actual touch presentation and during a tactile imagery task. Neurons encoded actual touch at short latency with bilateral receptive fields, organized by body part, and covered all tested regions. The tactile imagery task evoked body part-specific responses that shared a neural substrate with actual touch. Our results are the first neuron-level evidence of touch encoding in human PPC and its cognitive engagement during a tactile imagery task, which may reflect semantic processing, attention, sensory anticipation, or imagined touch.

Classifying Intracortical Brain-Machine Interface Signal Disruptions Based on System Performance and Applicable Compensatory Strategies: A Review

Brain-machine interfaces (BMIs) record and translate neural activity into a control signal for assistive or other devices. Intracortical microelectrode arrays (MEAs) enable high degree-of-freedom BMI control for complex tasks by providing fine-resolution neural recording. However, chronically implanted MEAs are subject to a dynamic in vivo environment where transient or systematic disruptions can interfere with neural recording and degrade BMI performance. Typically, neural implant failure modes have been categorized as biological, material, or mechanical. While this categorization provides insight into a disruption's causal etiology, it is less helpful for understanding degree of impact on BMI function or possible strategies for compensation. Therefore, we propose a complementary classification framework for intracortical recording disruptions that is based on duration of impact on BMI performance and requirement for and responsiveness to interventions: (1) Transient disruptions interfere with recordings on the time scale of minutes to hours and can resolve spontaneously; (2) Reversible disruptions cause persistent interference in recordings but the root cause can be remedied by an appropriate intervention; (3) Irreversible compensable disruptions cause persistent or progressive decline in signal quality, but their effects on BMI performance can be mitigated algorithmically; and (4) Irreversible non-compensable disruptions cause permanent signal loss that is not amenable to remediation or compensation. This conceptualization of intracortical BMI disruption types is useful for highlighting specific areas for potential hardware improvements and also identifying opportunities for algorithmic interventions. We review recording disruptions that have been reported for MEAs and demonstrate how biological, material, and mechanical mechanisms of disruption can be further categorized according to their impact on signal characteristics. Then we discuss potential compensatory protocols for each of the proposed disruption classes. Specifically, transient disruptions may be minimized by using robust neural decoder features, data augmentation methods, adaptive machine learning models, and specialized signal referencing techniques. Statistical Process Control methods can identify reparable disruptions for rapid intervention. In-vivo diagnostics such as impedance spectroscopy can inform neural feature selection and decoding models to compensate for irreversible disruptions. Additional compensatory strategies for irreversible disruptions include information salvage techniques, data augmentation during decoder training, and adaptive decoding methods to down-weight damaged channels.

Brain Computer Interfaces in Rehabilitation Medicine

One innovation currently influencing physical medicine and rehabilitation is brain-computer interface (BCI) technology. BCI systems used for motor control record neural activity associated with thoughts, perceptions, and motor intent; decode brain signals into commands for output devices; and perform the user's intended action through an output device. BCI systems used for sensory augmentation transduce environmental stimuli into neural signals interpretable by the central nervous system. Both types of systems have potential for reducing disability by facilitating a user's interaction with the environment. Investigational BCI systems are being used in the rehabilitation setting both as neuroprostheses to replace lost function and as potential plasticity-enhancing therapy tools aimed at accelerating neurorecovery. Populations benefitting from motor and somatosensory BCI systems include those with spinal cord injury, motor neuron disease, limb amputation, and stroke. This article discusses the basic components of BCI for rehabilitation, including recording systems and locations, signal processing and translation algorithms, and external devices controlled through BCI commands. An overview of applications in motor and sensory restoration is provided, along with ethical questions and user perspectives regarding BCI technology.

SpikeDeeptector: a deep-learning based method for detection of neural spiking activity

OBJECTIVE

In electrophysiology, microelectrodes are the primary source for recording neural data (single unit activity). These microelectrodes can be implanted individually or in the form of arrays containing dozens to hundreds of channels. Recordings of some channels contain neural activity, which are often contaminated with noise. Another fraction of channels does not record any neural data, but only noise. By noise, we mean physiological activities unrelated to spiking, including technical artifacts and neural activities of neurons that are too far away from the electrode to be usefully processed. For further analysis, an automatic identification and continuous tracking of channels containing neural data is of great significance for many applications, e.g. automated selection of neural channels during online and offline spike sorting. Automated spike detection and sorting is also critical for online decoding in brain-computer interface (BCI) applications, in which only simple threshold crossing events are often considered for feature extraction. To our knowledge, there is no method that can universally and automatically identify channels containing neural data. In this study, we aim to identify and track channels containing neural data from implanted electrodes, automatically and more importantly universally. By universally, we mean across different recording technologies, different subjects and different brain areas.

APPROACH

We propose a novel algorithm based on a new way of feature vector extraction and a deep learning method, which we call SpikeDeeptector. SpikeDeeptector considers a batch of waveforms to construct a single feature vector and enables contextual learning. The feature vectors are then fed to a deep learning method, which learns contextualized, temporal and spatial patterns, and classifies them as channels containing neural spike data or only noise.

MAIN RESULTS

We trained the model of SpikeDeeptector on data recorded from a single tetraplegic patient with two Utah arrays implanted in different areas of the brain. The trained model was then evaluated on data collected from six epileptic patients implanted with depth electrodes, unseen data from the tetraplegic patient and data from another tetraplegic patient implanted with two Utah arrays. The cumulative evaluation accuracy was 97.20% on 1.56 million hand labeled test inputs.

SIGNIFICANCE

The results demonstrate that SpikeDeeptector generalizes not only to the new data, but also to different brain areas, subjects, and electrode types not used for training.

CLINICAL TRIAL REGISTRATION NUMBER

The clinical trial registration number for patients implanted with the Utah array is NCT01849822. For the epilepsy patients, approval from the local ethics committee at the Ruhr-University Bochum, Germany, was obtained prior to implantation.

Brain-Inspired Coding of Robot Body Schema Through Visuo-Motor Integration of Touched Events

Representing objects in space is difficult because sensorimotor events are anchored in different reference frames, which can be either eye-, arm-, or target-centered. In the brain, Gain-Field (GF) neurons in the parietal cortex are involved in computing the necessary spatial transformations for aligning the tactile, visual and proprioceptive signals. In reaching tasks, these GF neurons exploit a mechanism based on multiplicative interaction for binding simultaneously touched events from the hand with visual and proprioception information.By doing so, they can infer new reference frames to represent dynamically the location of the body parts in the visual space (i.e., the body schema) and nearby targets (i.e., its peripersonal space). In this line, we propose a neural model based on GF neurons for integrating tactile events with arm postures and visual locations for constructing hand- and target-centered receptive fields in the visual space. In robotic experiments using an artificial skin, we show how our neural architecture reproduces the behaviors of parietal neurons (1) for encoding dynamically the body schema of our robotic arm without any visual tags on it and (2) for estimating the relative orientation and distance of targets to it. We demonstrate how tactile information facilitates the integration of visual and proprioceptive signals in order to construct the body space.

Cognition in Sensorimotor Control: Interfacing With the Posterior Parietal Cortex

Millions of people worldwide are afflicted with paralysis from a disruption of neural pathways between the brain and the muscles. Because their cortical architecture is often preserved, these patients are able to plan movements despite an inability to execute them. In such people, brain machine interfaces have great potential to restore lost function through neuroprosthetic devices, circumventing dysfunctional corticospinal circuitry. These devices have typically derived control signals from the motor cortex (M1) which provides information highly correlated with desired movement trajectories. However, sensorimotor control simultaneously engages multiple cognitive processes such as intent, state estimation, decision making, and the integration of multisensory feedback. As such, cortical association regions upstream of M1 such as the posterior parietal cortex (PPC) that are involved in higher order behaviors such as planning and learning, rather than in encoding movement itself, may enable enhanced, cognitive control of neuroprosthetics, termed cognitive neural prosthetics (CNPs). We illustrate in this review, through a small sampling, the cognitive functions encoded in the PPC and discuss their neural representation in the context of their relevance to motor neuroprosthetics. We aim to highlight through examples a role for cortical signals from the PPC in developing CNPs, and to inspire future avenues for exploration in their research and development.

Reduced neural representation of arm/hand actions in the medial posterior parietal cortex

Several investigations at a single-cell level demonstrated that the medial posterior parietal area V6A is involved in encoding reaching and grasping actions in different visual conditions. Here, we looked for a “low-dimensional” representation of these encoding processes by studying macaque V6A neurons tested in three different tasks with a dimensionality reduction technique, the demixed principal component analysis (dPCA), which is very suitable for neuroprosthetics readout. We compared neural activity in reaching and grasping tasks by highlighting the portions of population variance involved in the encoding of visual information, target position, wrist orientation and grip type. The weight of visual information and task parameters in the encoding process was dependent on the task. We found that the distribution of variance captured by visual information in the three tasks did not differ significantly among the tasks, whereas the variance captured by target position and grip type parameters were significantly higher with respect to that captured by wrist orientation regardless of the number of conditions considered in each task. These results suggest a different use of relevant information according to the type of planned and executed action. This study shows a simplified picture of encoding that describes how V6A processes relevant information for action planning and execution.

Exploring representations of human grasping in neural, muscle and kinematic signals

Movement covariates, such as electromyographic or kinematic activity, have been proposed as candidates for the neural representation of hand control. However, it remains unclear how these movement covariates are reflected in electroencephalographic (EEG) activity during different stages of grasping movements. In this exploratory study, we simultaneously acquired EEG, kinematic and electromyographic recordings of human subjects performing 33 types of grasps, yielding the largest such dataset to date. We observed that EEG activity reflected different movement covariates in different stages of grasping. During the pre-shaping stage, centro-parietal EEG in the lower beta frequency band reflected the object's shape and size, whereas during the finalization and holding stages, contralateral parietal EEG in the mu frequency band reflected muscle activity. These findings contribute to the understanding of the temporal organization of neural grasping patterns, and could inform the design of noninvasive neuroprosthetics and brain-computer interfaces with more natural control.

Visual and Motor Recovery After "Cognitive Therapeutic Exercises" in Cortical Blindness: A Case Study

BACKGROUND AND PURPOSE

Spontaneous visual recovery is rare after cortical blindness. While visual rehabilitation may improve performance, no visual therapy has been widely adopted, as clinical outcomes are variable and rarely translate into improvements in activities of daily living (ADLs). We explored the potential value of a novel rehabilitation approach "cognitive therapeutic exercises" for cortical blindness.

CASE DESCRIPTION

The subject of this case study was 48-year-old woman with cortical blindness and tetraplegia after cardiac arrest. Prior to the intervention, she was dependent in ADLs and poorly distinguished shapes and colors after 19 months of standard visual and motor rehabilitation. Computed tomographic images soon after symptom onset demonstrated acute infarcts in both occipital cortices.

INTERVENTION

The subject underwent 8 months of intensive rehabilitation with "cognitive therapeutic exercises" consisting of discrimination exercises correlating sensory and visual information.

OUTCOMES

Visual fields increased; object recognition improved; it became possible to watch television; voluntary arm movements improved in accuracy and smoothness; walking improved; and ADL independence and self-reliance increased. Subtraction of neuroimaging acquired before and after rehabilitation showed that focal glucose metabolism increases bilaterally in the occipital poles.

DISCUSSION

This study demonstrates feasibility of "cognitive therapeutic exercises" in an individual with cortical blindness, who experienced impressive visual and sensorimotor recovery, with marked ADL improvement, more than 2 years after ischemic cortical damage.Video Abstract available for additional insights from the authors (see Video, Supplemental Digital Content 1, available at: http://links.lww.com/JNPT/A173).

Finger movements are mainly represented by a linear transformation of energy in band-specific ECoG signals

Electrocardiogram (ECoG) recordings are very attractive for Brain Machine Interface (BMI) applications due to their balance between good signal to noise ratio and minimal invasiveness. The design of ECoG signal decoders is an open research area to date which requires a better understanding of the nature of these signals and how information is encoded in them. In this study, a linear and a non-linear method, Linear Regression Model (LRM) and Artificial Neural Network (ANN) respectively, were used to decode finger movements from energy in band-specific ECoG signals. It is shown that the ANN only slightly outperformed the LRM, which suggests that finger movements are mainly represented by a linear transformation of energy in band-specific ECoG signals. In addition, comparing our results to similar Electroencephalogram (EEG) studies illustrated that the spatio-temporal summation of multiple neural signals is itself linearly correlated with movement, and is not an artifact introduced by the scalp or cranium. Furthermore, a new algorithm was employed to reduce the number of spectral features of the input signals required for either of the decoding methods.

Dexterous Control of Seven Functional Hand Movements Using Cortically-Controlled Transcutaneous Muscle Stimulation in a Person With Tetraplegia

Individuals with tetraplegia identify restoration of hand function as a critical, unmet need to regain their independence and improve quality of life. Brain-Computer Interface (BCI)-controlled Functional Electrical Stimulation (FES) technology addresses this need by reconnecting the brain with paralyzed limbs to restore function. In this study, we quantified performance of an intuitive, cortically-controlled, transcutaneous FES system on standardized object manipulation tasks from the Grasp and Release Test (GRT). We found that a tetraplegic individual could use the system to control up to seven functional hand movements, each with >95% individual accuracy. He was able to select one movement from the possible seven movements available to him and use it to appropriately manipulate all GRT objects in real-time using naturalistic grasps. With the use of the system, the participant not only improved his GRT performance over his baseline, demonstrating an increase in number of transfers for all objects except the Block, but also significantly improved transfer times for the heaviest objects (videocassette (VHS), Can). Analysis of underlying motor cortex neural representations associated with the hand grasp states revealed an overlap or non-separability in neural activation patterns for similarly shaped objects that affected BCI-FES performance. These results suggest that motor cortex neural representations for functional grips are likely more related to hand shape and force required to hold objects, rather than to the objects themselves. These results, demonstrating multiple, naturalistic functional hand movements with the BCI-FES, constitute a further step toward translating BCI-FES technologies from research devices to clinical neuroprosthetics.

Extinction as a deficit of the decision-making circuitry in the posterior parietal cortex

Extinction is a common neurologic deficit that often occurs as one of a constellation of symptoms seen with lesions of the posterior parietal cortex (PPC). Although extinction has typically been considered a deficit in the allocation of attention, new findings, particularly from nonhuman primate studies, point to one potential and important source of extinction as damage to decision-making circuits for actions within the PPC. This new understanding provides clues to potential therapies for extinction. Also the finding that the PPC is important for action decisions and action planning has led to new neuroprosthetic applications using PPC recordings as control signals to assist paralyzed patients.

Parietofrontal network upregulation after motor stroke

Objective

Motor recovery after stroke shows a high inter-subject variability. The brain's potential to form new connections determines individual levels of recovery of motor function. Most of our daily activities require visuomotor integration, which engages parietal areas. Compared to the frontal motor system, less is known about the parietal motor system's reconfiguration related to stroke recovery. Here, we tested if functional connectivity among parietal and frontal motor areas undergoes plastic changes after stroke and assessed the behavioral relevance for motor function after stroke.

Methods

We investigated stroke lesion-induced changes in functional connectivity by measuring high-density electroencephalography (EEG) and assessing task-related changes in coherence during a visually guided grip task with the paretic hand in 30 chronic stroke patients with variable motor deficits and 19 healthy control subjects. Quantitative changes in task-related coherence in sensorimotor rhythms were compared to the residual motor deficit.

Results

Parietofrontal coupling was significantly stronger in patients compared to controls. Whereas motor network coupling generally increased during the task in both groups, the task-related coherence between the parietal and primary motor cortex in the stroke lesioned hemisphere showed increased connectivity across a broad range of sensorimotor rhythms. Particularly the parietofrontal task-induced coupling pattern was significantly and positively related to residual impairment in the Nine-Hole Peg Test performance and grip force.

Interpretation

These results demonstrate that parietofrontal motor system integration during visually guided movements is stronger in the stroke-lesioned brain. The correlation with the residual motor deficit could either indicate an unspecific marker of motor network damage or it might indicate that upregulated parietofrontal connectivity has some impact on post-stroke motor function.

Task Context Overrules Object- and Category-Related Representational Content in the Human Parietal Cortex

The dorsal, parietal visual stream is activated when seeing objects, but the exact nature of parietal object representations is still under discussion. Here we test 2 specific hypotheses. First, parietal cortex is biased to host some representations more than others, with a different bias compared with ventral areas. A prime example would be object action representations. Second, parietal cortex forms a general multiple-demand network with frontal areas, showing similar task effects and representational content compared with frontal areas. To differentiate between these hypotheses, we implemented a human neuroimaging study with a stimulus set that dissociates associated object action from object category while manipulating task context to be either action- or category-related. Representations in parietal as well as prefrontal areas represented task-relevant object properties (action representations in the action task), with no sign of the irrelevant object property (category representations in the action task). In contrast, irrelevant object properties were represented in ventral areas. These findings emphasize that human parietal cortex does not preferentially represent particular object properties irrespective of task, but together with frontal areas is part of a multiple-demand and content-rich cortical network representing task-relevant object properties.

Single-Neuron Representation of Memory Strength and Recognition Confidence in Left Human Posterior Parietal Cortex

The human posterior parietal cortex (PPC) is thought to contribute to memory retrieval, but little is known about its specific role. We recorded single PPC neurons of two human tetraplegic subjects implanted with microelectrode arrays, who performed a recognition memory task. We found two groups of neurons that signaled memory-based choices. Memory-selective neurons preferred either novel or familiar stimuli, scaled their response as a function of confidence, and signaled subjective choices regardless of truth. Confidence-selective neurons signaled confidence regardless of stimulus familiarity. Memory-selective signals appeared 553 ms after stimulus onset, but before action onset. Neurons also encoded spoken numbers, but these number-tuned neurons did not carry recognition signals. Together, this functional separation reveals action-independent coding of declarative memory-based familiarity and confidence of choices in human PPC. These data suggest that, in addition to sensory-motor integration, a function of human PPC is to utilize memory signals to make choices.

Movement-Related Activity of Human Subthalamic Neurons during a Reach-to-Grasp Task

The aim of the study was to record movement-related single unit activity (SUA) in the human subthalamic nucleus (STN) during a standardized motor task of the upper limb. We performed microrecordings from the motor region of the human STN and registered kinematic data in 12 patients with Parkinson’s disease (PD) undergoing deep brain stimulation surgery (seven women, mean age 62.0 ± 4.7 years) while they intraoperatively performed visually cued reach-to-grasp movements using a grip device. SUA was analyzed offline in relation to different aspects of the movement (attention, start of the movement, movement velocity, button press) in terms of firing frequency, firing pattern, and oscillation. During the reach-to-grasp movement, 75/114 isolated subthalamic neurons exhibited movement-related activity changes. The largest proportion of single units showed modulation of firing frequency during several phases of the reach and grasp (polymodal neurons, 45/114), particularly an increase of firing rate during the reaching phase of the movement, which often correlated with movement velocity. The firing pattern (bursting, irregular, or tonic) remained unchanged during movement compared to rest. Oscillatory single unit firing activity (predominantly in the theta and beta frequency) decreased with movement onset, irrespective of oscillation frequency. This study shows for the first time specific, task-related, SUA changes during the reach-to-grasp movement in humans.

Partially Mixed Selectivity in Human Posterior Parietal Association Cortex

To clarify the organization of motor representations in posterior parietal cortex, we test how three motor variables (body side, body part, cognitive strategy) are coded in the human anterior intraparietal cortex. All tested movements were encoded, arguing against strict anatomical segregation of effectors. Single units coded for diverse conjunctions of variables, with different dimensions anatomically overlapping. Consistent with recent studies, neurons encoding body parts exhibited mixed selectivity. This mixed selectivity resulted in largely orthogonal coding of body parts, which "functionally segregate" the effector responses despite the high degree of anatomical overlap. Body side and strategy were not coded in a mixed manner as effector determined their organization. Mixed coding of some variables over others, what we term "partially mixed coding," argues that the type of functional encoding depends on the compared dimensions. This structure is advantageous for neuroprosthetics, allowing a single array to decode movements of a large extent of the body.