Cognitive neural prosthetics

Decoding spatial locations from primate lateral prefrontal cortex neural activity during virtual navigation

Objective. Decoding the intended trajectories from brain signals using a brain-computer interface system could be used to improve the mobility of patients with disabilities.Approach. Neuronal activity associated with spatial locations was examined while macaques performed a navigation task within a virtual environment.Main results.Here, we provide proof of principle that multi-unit spiking activity recorded from the lateral prefrontal cortex (LPFC) of non-human primates can be used to predict the location of a subject in a virtual maze during a navigation task. The spatial positions within the maze that require a choice or are associated with relevant task events can be better predicted than the locations where no relevant events occur. Importantly, within a task epoch of a single trial, multiple locations along the maze can be independently identified using a support vector machine model.Significance. Considering that the LPFC of macaques and humans share similar properties, our results suggest that this area could be a valuable implant location for an intracortical brain-computer interface system used for spatial navigation in patients with disabilities.

Combining Reflexes and External Sensory Information in a Neuromusculoskeletal Model to Control a Quadruped Robot

This article examines the importance of integrating locomotion and cognitive information for achieving dynamic locomotion from a viewpoint combining biology and ecological psychology. We present a mammalian neuromusculoskeletal model from external sensory information processing to muscle activation, which includes: 1) a visual-attention control mechanism for controlling attention to external inputs; 2) object recognition representing the primary motor cortex; 3) a motor control model that determines motor commands traveling down the corticospinal and reticulospinal tracts; 4) a central pattern generation model representing pattern generation in the spinal cord; and 5) a muscle reflex model representing the muscle model and its reflex mechanism. The proposed model is able to generate the locomotion of a quadruped robot in flat and natural terrain. The experiment also shows the importance of a postural reflex mechanism when experiencing a sudden obstacle. We show the reflex mechanism when a sudden obstacle is separately detected from both external (retina) and internal (touching afferent) sensory information. We present the biological rationale for supporting the proposed model. Finally, we discuss future contributions, trends, and the importance of the proposed research.

Gamma-band modulation in the human amygdala during reaching movements

OBJECTIVE

Motor brain-computer interface (BCI) represents a new frontier in neurological surgery that could provide significant benefits for patients living with motor deficits. Both the primary motor cortex and posterior parietal cortex have successfully been used as a neural source for human motor BCI, leading to interest in exploring other brain areas involved in motor control. The amygdala is one area that has been shown to have functional connectivity to the motor system; however, its role in movement execution is not well studied. Gamma oscillations (30-200 Hz) are known to be prokinetic in the human cortex, but their role is poorly understood in subcortical structures. Here, the authors use direct electrophysiological recordings and the classic "center-out" direct-reach experiment to study amygdaloid gamma-band modulation in 8 patients with medically refractory epilepsy.

METHODS

The study population consisted of 8 epilepsy patients (2 men; age range 21-62 years) who underwent implantation of micro-macro depth electrodes for seizure localization and EEG monitoring. Data from the macro contacts sampled at 2000 Hz were used for analysis. The classic center-out direct-reach experiment was used, which consists of an intertrial interval phase, a fixation phase, and a response phase. The authors assessed the statistical significance of neural modulation by inspecting for nonoverlapping areas in the 95% confidence intervals of spectral power for the response and fixation phases.

RESULTS

In 5 of the 8 patients, power spectral analysis showed a statistically significant increase in power within regions of the gamma band during the response phase compared with the fixation phase. In these 5 patients, the 95% bootstrapped confidence intervals of trial-averaged power in contiguous frequencies of the gamma band during the response phase were above, and did not overlap with, the confidence intervals of trial-averaged power during the fixation phase.

CONCLUSIONS

To the authors' knowledge, this is the first time that direct neural recordings have been used to show gamma-band modulation in the human amygdala during the execution of voluntary movement. This work indicates that gamma-band modulation in the amygdala could be a contributing source of neural signals for use in a motor BCI system.

Decoding of standard and non-standard visuomotor associations from parietal cortex

OBJECTIVE

Neural signals can be decoded and used to move neural prostheses with the purpose of restoring motor function in patients with mobility impairments. Such patients typically have intact eye movement control and visual function, suggesting that cortical visuospatial signals could be used to guide external devices. Neurons in parietal cortex mediate sensory-motor transformations, encode the spatial coordinates for reaching goals, hand position and movements, and other spatial variables. We studied how spatial information is represented at the population level, and the possibility to decode not only the position of visual targets and the plans to reach them, but also conditional, non-spatial motor responses.

APPROACH

The animals first fixated one of nine targets in 3D space and then, after the target changed color, either reached toward it, or performed a non-spatial motor response (lift hand from a button). Spiking activity of parietal neurons was recorded in monkeys during two tasks. We then decoded different task related parameters.

MAIN RESULTS

We first show that a maximum-likelihood estimation (MLE) algorithm trained separately in each task transformed neural activity into accurate metric predictions of target location. Furthermore, by combining MLE with a Naïve Bayes classifier, we decoded the monkey's motor intention (reach or hand lift) and the different phases of the tasks. These results show that, although V6A encodes the spatial location of a target during a delay period, the signals they carry are updated around the movement execution in an intention/motor specific way.

SIGNIFICANCE

These findings show the presence of multiple levels of information in parietal cortex that could be decoded and used in brain machine interfaces to control both goal-directed movements and more cognitive visuomotor associations.

Context-dependent relationship in high-resolution micro-ECoG studies during finger movements

OBJECTIVE

The activation of the sensorimotor cortex as measured by electrocorticographic (ECoG) signals has been correlated with contralateral hand movements in humans, as precisely as the level of individual digits. However, the relationship between individual and multiple synergistic finger movements and the neural signal as detected by ECoG has not been fully explored. The authors used intraoperative high-resolution micro-ECoG (µECoG) on the sensorimotor cortex to link neural signals to finger movements across several context-specific motor tasks.

METHODS

Three neurosurgical patients with cortical lesions over eloquent regions participated. During awake craniotomy, a sensorimotor cortex area of hand movement was localized by high-frequency responses measured by an 8 × 8 µECoG grid of 3-mm interelectrode spacing. Patients performed a flexion movement of the thumb or index finger, or a pinch movement of both, based on a visual cue. High-gamma (HG; 70-230 Hz) filtered µECoG was used to identify dominant electrodes associated with thumb and index movement. Hand movements were recorded by a dataglove simultaneously with µECoG recording.

RESULTS

In all 3 patients, the electrodes controlling thumb and index finger movements were identifiable approximately 3-6-mm apart by the HG-filtered µECoG signal. For HG power of cortical activation measured with µECoG, the thumb and index signals in the pinch movement were similar to those observed during thumb-only and index-only movement, respectively (all p > 0.05). Index finger movements, measured by the dataglove joint angles, were similar in both the index-only and pinch movements (p > 0.05). However, despite similar activation across the conditions, markedly decreased thumb movement was observed in pinch relative to independent thumb-only movement (all p < 0.05).

CONCLUSIONS

HG-filtered µECoG signals effectively identify dominant regions associated with thumb and index finger movement. For pinch, the µECoG signal comprises a combination of the signals from individual thumb and index movements. However, while the relationship between the index finger joint angle and HG-filtered signal remains consistent between conditions, there is not a fixed relationship for thumb movement. Although the HG-filtered µECoG signal is similar in both thumb-only and pinch conditions, the actual thumb movement is markedly smaller in the pinch condition than in the thumb-only condition. This implies a nonlinear relationship between the cortical signal and the motor output for some, but importantly not all, movement types. This analysis provides insight into the tuning of the motor cortex toward specific types of motor behaviors.

Mapping of primary somatosensory cortex of the hand area using a high-density electrocorticography grid for closed-loop brain computer interface

The ideal modality for generating sensation in sensorimotor brain computer interfaces (BCI) has not been determined. Here we report the feasibility of using a high-density "mini"-electrocorticography (mECoG) grid in a somatosensory BCI system. Thirteen subjects with intractable epilepsy underwent standard clinical implantation of subdural electrodes for the purpose of seizure localization. An additional high-density mECoG grid was placed (Adtech, 8 by 8, 1.2-mm exposed, 3-mm center-to-center spacing) over the hand area of primary somatosensory cortex. Following implantation, cortical mapping was performed with stimulation parameters of frequency: 50 Hz, pulse-width: 250 µs, pulse duration: 4 s, polarity: alternating, and current that ranged from 0.5 mA to 12 mA at the discretion of the epileptologist. Location of the evoked sensory percepts was recorded along with a description of the sensation. The hand was partitioned into 48 distinct boxes. A box was included if sensation was felt anywhere within the box. The percentage of the hand covered was 63.9% (± 34.4%) (mean ± s.d.). Mean redundancy, measured as electrode pairs stimulating the same box, was 1.9 (± 2.2) electrodes per box; and mean resolution, measured as boxes included per electrode pair stimulation, was 11.4 (± 13.7) boxes with 8.1 (± 10.7) boxes in the digits and 3.4 (± 6.0) boxes in the palm. Functional utility of the system was assessed by quantifying usable percepts. Under the strictest classification, "dermatomally exclusive" percepts, the mean was 2.8 usable percepts per grid. Allowing "perceptually unique" percepts at the same anatomical location, the mean was 5.5 usable percepts per grid. Compared to the small area of coverage and redundancy of a microelectrode system, or the poor resolution of a standard ECoG grid, a mECoG is likely the best modality for a somatosensory BCI system with good coverage of the hand and minimal redundancy.

Decoding Movement From Electrocorticographic Activity: A Review

Electrocorticography (ECoG) holds promise to provide efficient neuroprosthetic solutions for people suffering from neurological disabilities. This recording technique combines adequate temporal and spatial resolution with the lower risks of medical complications compared to the other invasive methods. ECoG is routinely used in clinical practice for preoperative cortical mapping in epileptic patients. During the last two decades, research utilizing ECoG has considerably grown, including the paradigms where behaviorally relevant information is extracted from ECoG activity with decoding algorithms of different complexity. Several research groups have advanced toward the development of assistive devices driven by brain-computer interfaces (BCIs) that decode motor commands from multichannel ECoG recordings. Here we review the evolution of this field and its recent tendencies, and discuss the potential areas for future development.

Engineering Artificial Somatosensation Through Cortical Stimulation in Humans

Sensory feedback is a critical aspect of motor control rehabilitation following paralysis or amputation. Current human studies have demonstrated the ability to deliver some of this sensory information via brain-machine interfaces, although further testing is needed to understand the stimulation parameters effect on sensation. Here, we report a systematic evaluation of somatosensory restoration in humans, using cortical stimulation with subdural mini-electrocorticography (mini-ECoG) grids. Nine epilepsy patients undergoing implantation of cortical electrodes for seizure localization were also implanted with a subdural 64-channel mini-ECoG grid over the hand area of the primary somatosensory cortex (S1). We mapped the somatotopic location and size of receptive fields evoked by stimulation of individual channels of the mini-ECoG grid. We determined the effects on perception by varying stimulus parameters of pulse width, current amplitude, and frequency. Finally, a target localization task was used to demonstrate the use of artificial sensation in a behavioral task. We found a replicable somatotopic representation of the hand on the mini-ECoG grid across most subjects during electrical stimulation. The stimulus-evoked sensations were usually of artificial quality, but in some cases were more natural and of a cutaneous or proprioceptive nature. Increases in pulse width, current strength and frequency generally produced similar quality sensations at the same somatotopic location, but with a perception of increased intensity. The subjects produced near perfect performance when using the evoked sensory information in target acquisition tasks. These findings indicate that electrical stimulation of somatosensory cortex through mini-ECoG grids has considerable potential for restoring useful sensation to patients with paralysis and amputation.

Promises and limitations of human intracranial electroencephalography

Intracranial electroencephalography (iEEG), also known as electrocorticography when using subdural grid electrodes or stereotactic EEG when using depth electrodes, is blossoming in various fields of human neuroscience. In this article, we highlight the potentials of iEEG in exploring functions of the human brain while also considering its limitations. The iEEG signal provides anatomically precise information about the selective engagement of neuronal populations at the millimeter scale and the temporal dynamics of their engagement at the millisecond scale. If several nodes of a given network are monitored simultaneously with implanted electrodes, the iEEG signals can also reveal information about functional interactions within and across networks during different stages of neural computation. As such, human iEEG can complement other methods of neuroscience beyond simply replicating what is already known, or can be known, from noninvasive lines of research in humans or from invasive recordings in nonhuman mammalian brains.

What can neuronal populations tell us about cognition?

Nowadays, it is possible to record the activity of hundreds of cells at the same time in behaving animals. However, these data are often treated and analyzed as if they consisted of many independently recorded neurons. How can neuronal populations be uniquely used to learn about cognition? We describe recent work that shows that populations of simultaneously recorded neurons are fundamental to understand the basis of decision-making, including processes such as ongoing deliberations and decision confidence, which generally fall outside the reach of single-cell analysis. Thus, neuronal population data allow addressing novel questions, but they also come with so far unsolved challenges.

Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation

Brain-machine interfaces (BMIs) combine methods, approaches, and concepts derived from neurophysiology, computer science, and engineering in an effort to establish real-time bidirectional links between living brains and artificial actuators. Although theoretical propositions and some proof of concept experiments on directly linking the brains with machines date back to the early 1960s, BMI research only took off in earnest at the end of the 1990s, when this approach became intimately linked to new neurophysiological methods for sampling large-scale brain activity. The classic goals of BMIs are 1) to unveil and utilize principles of operation and plastic properties of the distributed and dynamic circuits of the brain and 2) to create new therapies to restore mobility and sensations to severely disabled patients. Over the past decade, a wide range of BMI applications have emerged, which considerably expanded these original goals. BMI studies have shown neural control over the movements of robotic and virtual actuators that enact both upper and lower limb functions. Furthermore, BMIs have also incorporated ways to deliver sensory feedback, generated from external actuators, back to the brain. BMI research has been at the forefront of many neurophysiological discoveries, including the demonstration that, through continuous use, artificial tools can be assimilated by the primate brain's body schema. Work on BMIs has also led to the introduction of novel neurorehabilitation strategies. As a result of these efforts, long-term continuous BMI use has been recently implicated with the induction of partial neurological recovery in spinal cord injury patients.

Ethical considerations in providing an upper limb exoskeleton device for stroke patients

The health care system needs to face new and advanced medical technologies that can improve the patients' quality of life by replacing lost or decreased functions. In stroke patients, the disabilities that follow cerebral lesions may impair the mandatory daily activities of an independent life. These activities are dependent mostly on the patient's upper limb function so that they can carry out most of the common activities associated with a normal life. Therefore, an upper limb exoskeleton device for stroke patients can contribute a real improvement of quality of their life. The ethical problems that need to be considered are linked to the correct adjustment of the upper limb skills in order to satisfy the patient's expectations, but within physiological limits. The debate regarding the medical devices dedicated to neurorehabilitation is focused on their ability to be beneficial to the patient's life, keeping away damages, injustice, and risks.

Decoding Lower Limb Muscle Activity and Kinematics from Cortical Neural Spike Trains during Monkey Performing Stand and Squat Movements

Extensive literatures have shown approaches for decoding upper limb kinematics or muscle activity using multichannel cortical spike recordings toward brain machine interface (BMI) applications. However, similar topics regarding lower limb remain relatively scarce. We previously reported a system for training monkeys to perform visually guided stand and squat tasks. The current study, as a follow-up extension, investigates whether lower limb kinematics and muscle activity characterized by electromyography (EMG) signals during monkey performing stand/squat movements can be accurately decoded from neural spike trains in primary motor cortex (M1). Two monkeys were used in this study. Subdermal intramuscular EMG electrodes were implanted to 8 right leg/thigh muscles. With ample data collected from neurons from a large brain area, we performed a spike triggered average (SpTA) analysis and got a series of density contours which revealed the spatial distributions of different muscle-innervating neurons corresponding to each given muscle. Based on the guidance of these results, we identified the locations optimal for chronic electrode implantation and subsequently carried on chronic neural data recordings. A recursive Bayesian estimation framework was proposed for decoding EMG signals together with kinematics from M1 spike trains. Two specific algorithms were implemented: a standard Kalman filter and an unscented Kalman filter. For the latter one, an artificial neural network was incorporated to deal with the nonlinearity in neural tuning. High correlation coefficient and signal to noise ratio between the predicted and the actual data were achieved for both EMG signals and kinematics on both monkeys. Higher decoding accuracy and faster convergence rate could be achieved with the unscented Kalman filter. These results demonstrate that lower limb EMG signals and kinematics during monkey stand/squat can be accurately decoded from a group of M1 neurons with the proposed algorithms. Our findings provide new insights for extending current BMI design concepts and techniques on upper limbs to lower limb circumstances. Brain controlled exoskeleton, prostheses or neuromuscular electrical stimulators for lower limbs are expected to be developed, which enables the subject to manipulate complex biomechatronic devices with mind in more harmonized manner.

Interfacing to the brain's motor decisions

It has been long known that neural activity, recorded with electrophysiological methods, contains rich information about a subject's motor intentions, sensory experiences, allocation of attention, action planning, and even abstract thoughts. All these functions have been the subject of neurophysiological investigations, with the goal of understanding how neuronal activity represents behavioral parameters, sensory inputs, and cognitive functions. The field of brain-machine interfaces (BMIs) strives for a somewhat different goal: it endeavors to extract information from neural modulations to create a communication link between the brain and external devices. Although many remarkable successes have been already achieved in the BMI field, questions remain regarding the possibility of decoding high-order neural representations, such as decision making. Could BMIs be employed to decode the neural representations of decisions underlying goal-directed actions? In this review we lay out a framework that describes the computations underlying goal-directed actions as a multistep process performed by multiple cortical and subcortical areas. We then discuss how BMIs could connect to different decision-making steps and decode the neural processing ongoing before movements are initiated. Such decision-making BMIs could operate as a system with prediction that offers many advantages, such as shorter reaction time, better error processing, and improved unsupervised learning. To present the current state of the art, we review several recent BMIs incorporating decision-making components.

Recurrent Network Models of Sequence Generation and Memory

Sequential activation of neurons is a common feature of network activity during a variety of behaviors, including working memory and decision making. Previous network models for sequences and memory emphasized specialized architectures in which a principled mechanism is pre-wired into their connectivity. Here we demonstrate that, starting from random connectivity and modifying a small fraction of connections, a largely disordered recurrent network can produce sequences and implement working memory efficiently. We use this process, called Partial In-Network Training (PINning), to model and match cellular resolution imaging data from the posterior parietal cortex during a virtual memory-guided two-alternative forced-choice task. Analysis of the connectivity reveals that sequences propagate by the cooperation between recurrent synaptic interactions and external inputs, rather than through feedforward or asymmetric connections. Together our results suggest that neural sequences may emerge through learning from largely unstructured network architectures.

Grasping with the Press of a Button: Grasp-selective Responses in the Human Anterior Intraparietal Sulcus Depend on Nonarbitrary Causal Relationships between Hand Movements and End-effector Actions

Evidence implicates ventral parieto-premotor cortices in representing the goal of grasping independent of the movements or effectors involved [Umilta, M. A., Escola, L., Intskirveli, I., Grammont, F., Rochat, M., Caruana, F., et al. When pliers become fingers in the monkey motor system. Proceedings of the National Academy of Sciences, U.S.A., 105, 2209-2213, 2008; Tunik, E., Frey, S. H., & Grafton, S. T. Virtual lesions of the anterior intraparietal area disrupt goal-dependent on-line adjustments of grasp. Nature Neuroscience, 8, 505-511, 2005]. Modern technologies that enable arbitrary causal relationships between hand movements and tool actions provide a strong test of this hypothesis. We capitalized on this unique opportunity by recording activity with fMRI during tasks in which healthy adults performed goal-directed reach and grasp actions manually or by depressing buttons to initiate these same behaviors in a remotely located robotic arm (arbitrary causal relationship). As shown previously [Binkofski, F., Dohle, C., Posse, S., Stephan, K. M., Hefter, H., Seitz, R. J., et al. Human anterior intraparietal area subserves prehension: A combined lesion and functional MRI activation study. Neurology, 50, 1253-1259, 1998], we detected greater activity in the vicinity of the anterior intraparietal sulcus (aIPS) during manual grasp versus reach. In contrast to prior studies involving tools controlled by nonarbitrarily related hand movements [Gallivan, J. P., McLean, D. A., Valyear, K. F., & Culham, J. C. Decoding the neural mechanisms of human tool use. Elife, 2, e00425, 2013; Jacobs, S., Danielmeier, C., & Frey, S. H. Human anterior intraparietal and ventral premotor cortices support representations of grasping with the hand or a novel tool. Journal of Cognitive Neuroscience, 22, 2594-2608, 2010], however, responses within the aIPS and premotor cortex exhibited no evidence of selectivity for grasp when participants employed the robot. Instead, these regions showed comparable increases in activity during both the reach and grasp conditions. Despite equivalent sensorimotor demands, the right cerebellar hemisphere displayed greater activity when participants initiated the robot's actions versus when they pressed a button known to be nonfunctional and watched the very same actions undertaken autonomously. This supports the hypothesis that the cerebellum predicts the forthcoming sensory consequences of volitional actions [Blakemore, S. J., Frith, C. D., & Wolpert, D. M. The cerebellum is involved in predicting the sensory consequences of action. NeuroReport, 12, 1879-1884, 2001]. We conclude that grasp-selective responses in the human aIPS and premotor cortex depend on the existence of nonarbitrary causal relationships between hand movements and end-effector actions.

Brain-machine interface for eye movements

A number of studies in tetraplegic humans and healthy nonhuman primates (NHPs) have shown that neuronal activity from reach-related cortical areas can be used to predict reach intentions using brain-machine interfaces (BMIs) and therefore assist tetraplegic patients by controlling external devices (e.g., robotic limbs and computer cursors). However, to our knowledge, there have been no studies that have applied BMIs to eye movement areas to decode intended eye movements. In this study, we recorded the activity from populations of neurons from the lateral intraparietal area (LIP), a cortical node in the NHP saccade system. Eye movement plans were predicted in real time using Bayesian inference from small ensembles of LIP neurons without the animal making an eye movement. Learning, defined as an increase in the prediction accuracy, occurred at the level of neuronal ensembles, particularly for difficult predictions. Population learning had two components: an update of the parameters of the BMI based on its history and a change in the responses of individual neurons. These results provide strong evidence that the responses of neuronal ensembles can be shaped with respect to a cost function, here the prediction accuracy of the BMI. Furthermore, eye movement plans could be decoded without the animals emitting any actual eye movements and could be used to control the position of a cursor on a computer screen. These findings show that BMIs for eye movements are promising aids for assisting paralyzed patients.

Toward more versatile and intuitive cortical brain-machine interfaces

Brain-machine interfaces have great potential for the development of neuroprosthetic applications to assist patients suffering from brain injury or neurodegenerative disease. One type of brain-machine interface is a cortical motor prosthetic, which is used to assist paralyzed subjects. Motor prosthetics to date have typically used the motor cortex as a source of neural signals for controlling external devices. The review will focus on several new topics in the arena of cortical prosthetics. These include using: recordings from cortical areas outside motor cortex; local field potentials as a source of recorded signals; somatosensory feedback for more dexterous control of robotics; and new decoding methods that work in concert to form an ecology of decode algorithms. These new advances promise to greatly accelerate the applicability and ease of operation of motor prosthetics.

Synchronization patterns suggest different functional organization in parietal reach region and dorsal premotor cortex

The parietal reach region (PRR) and dorsal premotor cortex (PMd) form part of the fronto-parietal reach network. While neural selectivity profiles of single-cell activity in these areas can be remarkably similar, other data suggest that both areas serve different computational functions in visually guided reaching. Here we test the hypothesis that different neural functional organizations characterized by different neural synchronization patterns might be underlying the putatively different functional roles. We use cross-correlation analysis on single-unit activity (SUA) and multiunit activity (MUA) to determine the prevalence of synchronized neural ensembles within each area. First, we reliably find synchronization in PRR but not in PMd. Second, we demonstrate that synchronization in PRR is present in different cognitive states, including "idle" states prior to task-relevant instructions and without neural tuning. Third, we show that local field potentials (LFPs) in PRR but not PMd are characterized by an increased power and spike field coherence in the beta frequency range (12-30 Hz), further indicating stronger synchrony in PRR compared with PMd. Finally, we show that neurons with similar tuning properties tend to be correlated in their random spike rate fluctuations in PRR but not in PMd. Our data support the idea that PRR and PMd, despite striking similarity in single-cell tuning properties, are characterized by unequal local functional organization, which likely reflects different network architectures to support different functional roles within the fronto-parietal reach network.

Intentional signal in prefrontal cortex generalizes across different sensory modalities

Biofeedback-EEG training to learn the mental control of an external device (e.g., a cursor on the screen) has been an important paradigm to attempt to understand the involvements of various areas of the brain in the volitional control and the modulation of intentional thought processes. Often the areas to adapt and to monitor progress are selected a priori. Less explored, however, has been the notion of automatically emerging activation in a particular area or subregions within that area recruited above and beyond the rest of the brain. Likewise, the notion of evoking such a signal as an amodal, abstract one remaining robust across different sensory modalities could afford some exploration. Here we develop a simple binary control task in the context of brain-computer interface (BCI) and use a Bayesian sparse probit classification algorithm to automatically uncover brain regional activity that maximizes task performance. We trained and tested 19 participants using the visual modality for instructions and feedback. Across training blocks we quantified coupling of the frontoparietal nodes and selective involvement of visual and auditory regions as a function of the real-time sensory feedback. The testing phase under both forms of sensory feedback revealed automatic recruitment of the prefrontal cortex with a parcellation of higher strength levels in Brodmann's areas 9, 10, and 11 significantly above those in other brain areas. We propose that the prefrontal signal may be a neural correlate of externally driven intended direction and discuss our results in the context of various aspects involved in the cognitive control of our thoughts.

Virtual Electrode Recording Tool for EXtracellular potentials (VERTEX): comparing multi-electrode recordings from simulated and biological mammalian cortical tissue

Local field potentials (LFPs) sampled with extracellular electrodes are frequently used as a measure of population neuronal activity. However, relating such measurements to underlying neuronal behaviour and connectivity is non-trivial. To help study this link, we developed the Virtual Electrode Recording Tool for EXtracellular potentials (VERTEX). We first identified a reduced neuron model that retained the spatial and frequency filtering characteristics of extracellular potentials from neocortical neurons. We then developed VERTEX as an easy-to-use Matlab tool for simulating LFPs from large populations (>100,000 neurons). A VERTEX-based simulation successfully reproduced features of the LFPs from an in vitro multi-electrode array recording of macaque neocortical tissue. Our model, with virtual electrodes placed anywhere in 3D, allows direct comparisons with the in vitro recording setup. We envisage that VERTEX will stimulate experimentalists, clinicians, and computational neuroscientists to use models to understand the mechanisms underlying measured brain dynamics in health and disease.

Functional dynamics of primate cortico-striatal networks during volitional movements

The motor cortex and dorsal striatum (caudate nucleus and putamen) are key regions in motor processing but the interface between the cortex and striatum is not well understood. While dorsal striatum integrates information from multiple brain regions to shape motor learning and habit formation, the disruption of cortico-striatal circuits compromises the functionality of these circuits resulting in a multitude of neurologic disorders, including Parkinson's disease. To better understand the modulation of the cortico-striatal circuits we recorded simultaneously single neuron activity from four brain regions, primary motor, and sensory cortices, together with the rostral and caudal segments of the putamen in rhesus monkeys performing a visual motor task. Results show that spatial and temporal-task related firing relationships between these cortico-striatal circuit regions were modified by the independent administration of the two drugs (cocaine and baclofen). Spatial tuning and correlated firing of neurons from motor cortex and putamen were severely disrupted by cocaine and baclofen on correct trials, while the two drugs have dramatically decreased the functional connectivity of the motor cortical-striatal network. These findings provide insight into the modulation of cortical-striatal firing related to movement with implications for therapeutic approaches to Parkinson's disease and related disorders.

A real-time brain-machine interface combining motor target and trajectory intent using an optimal feedback control design

Real-time brain-machine interfaces (BMI) have focused on either estimating the continuous movement trajectory or target intent. However, natural movement often incorporates both. Additionally, BMIs can be modeled as a feedback control system in which the subject modulates the neural activity to move the prosthetic device towards a desired target while receiving real-time sensory feedback of the state of the movement. We develop a novel real-time BMI using an optimal feedback control design that jointly estimates the movement target and trajectory of monkeys in two stages. First, the target is decoded from neural spiking activity before movement initiation. Second, the trajectory is decoded by combining the decoded target with the peri-movement spiking activity using an optimal feedback control design. This design exploits a recursive Bayesian decoder that uses an optimal feedback control model of the sensorimotor system to take into account the intended target location and the sensory feedback in its trajectory estimation from spiking activity. The real-time BMI processes the spiking activity directly using point process modeling. We implement the BMI in experiments consisting of an instructed-delay center-out task in which monkeys are presented with a target location on the screen during a delay period and then have to move a cursor to it without touching the incorrect targets. We show that the two-stage BMI performs more accurately than either stage alone. Correct target prediction can compensate for inaccurate trajectory estimation and vice versa. The optimal feedback control design also results in trajectories that are smoother and have lower estimation error. The two-stage decoder also performs better than linear regression approaches in offline cross-validation analyses. Our results demonstrate the advantage of a BMI design that jointly estimates the target and trajectory of movement and more closely mimics the sensorimotor control system.

A primer on brain-machine interfaces, concepts, and technology: a key element in the future of functional neurorestoration

Conventionally, the practice of neurosurgery has been characterized by the removal of pathology, congenital or acquired. The emerging complement to the removal of pathology is surgery for the specific purpose of restoration of function. Advents in neuroscience, technology, and the understanding of neural circuitry are creating opportunities to intervene in disease processes in a reparative manner, thereby advancing toward the long-sought-after concept of neurorestoration. Approaching the issue of neurorestoration from a biomedical engineering perspective is the rapidly growing arena of implantable devices. Implantable devices are becoming more common in medicine and are making significant advancements to improve a patient's functional outcome. Devices such as deep brain stimulators, vagus nerve stimulators, and spinal cord stimulators are now becoming more commonplace in neurosurgery as we utilize our understanding of the nervous system to interpret neural activity and restore function. One of the most exciting prospects in neurosurgery is the technologically driven field of brain-machine interface, also known as brain-computer interface, or neuroprosthetics. The successful development of this technology will have far-reaching implications for patients suffering from a great number of diseases, including but not limited to spinal cord injury, paralysis, stroke, or loss of limb. This article provides an overview of the issues related to neurorestoration using implantable devices with a specific focus on brain-machine interface technology.

A new multi-site probe array with monolithically integrated parylene flexible cable for neural prostheses

This work presents a new multi-site probe array applied with parylene technology, used for neural prostheses to record high-level cognitive neural signals. Instead of inorganic materials (e.g. silicon dioxide, silicon nitride), the electrodes and conduction traces on probes are insulated by parylene, which is a polymer material with high electrical resistivity, mechanical flexibility, biocompatibility and easy deposition process. As a result, the probes exhibit better electrical and mechanical properties. The all dry process is demonstrated to fabricate these probe arrays with monolithically integrated parylene flexible cables using double-side-polished (DSP) wafers. With the parylene flexible cables, the probes can be easily assembled to a high density 3-D array for chronic implantation.

Decoding semantic information from human electrocorticographic (ECoG) signals

This study examined the feasibility of decoding semantic information from human cortical activity. Four human subjects undergoing presurgical brain mapping and seizure foci localization participated in this study. Electrocorticographic (ECoG) signals were recorded while the subjects performed simple language tasks involving semantic information processing, such as a picture naming task where subjects named pictures of objects belonging to different semantic categories. Robust high-gamma band (60-120 Hz) activation was observed at the left inferior frontal gyrus (LIFG) and the posterior portion of the superior temporal gyrus (pSTG) with a temporal sequence corresponding to speech production and perception. Furthermore, Gaussian Naïve Bayes and Support Vector Machine classifiers, two commonly used machine learning algorithms for pattern recognition, were able to predict the semantic category of an object using cortical activity captured by ECoG electrodes covering the frontal, temporal and parietal cortices. These findings have implications for both basic neuroscience research and development of semantic-based brain-computer interface systems (BCI) that can help individuals with severe motor or communication disorders to express their intention and thoughts.

Single-trial analysis of neuroimaging data: inferring neural networks underlying perceptual decision-making in the human brain

Advances in neural signal and image acquisition as well as in multivariate signal processing and machine learning are enabling a richer and more rigorous understanding of the neural basis of human decision-making. Decision-making is essentially characterized behaviorally by the variability of the decision across individual trials--e.g., error and response time distributions. To infer the neural processes that govern decision-making requires identifying neural correlates of such trial-to-trial behavioral variability. In this paper, we review efforts that utilize signal processing and machine learning to enable single-trial analysis of neural signals acquired while subjects perform simple decision-making tasks. Our focus is on neuroimaging data collected noninvasively via electroencephalograpy (EEG) and functional magnetic resonance imaging (fMRI). We review the specific framework for extracting decision-relevant neural components from the neuroimaging data, the goal being to analyze the trial-to-trial variability of the neural signal along these component directions and to relate them to elements of the decision-making process. We review results for perceptual decision-making and discrimination tasks, including paradigms in which EEG variability is used to inform an fMRI analysis. We discuss how single-trial analysis reveals aspects of the underlying decision-making networks that are unobservable using traditional trial-averaging methods.

Carbon nanotube composite coating of neural microelectrodes preferentially improves the multiunit signal-to-noise ratio

Extracellular metal microelectrodes are widely used to record single neuron activity in vivo. However, their signal-to-noise ratio (SNR) is often far from optimal due to their high impedance value. It has been recently reported that carbon nanotube (CNT) coatings may decrease microelectrode impedance, thus improving their performance. To tease out the different contributions to SNR of CNT-coated microelectrodes we carried out impedance and noise spectroscopy measurements of platinum/tungsten microelectrodes coated with a polypyrrole-CNT composite. Neuronal signals were recorded in vivo from rat cortex by employing tetrodes with two recording sites coated with polypyrrole-CNT and the remaining two left untreated. We found that polypyrrole-CNT coating significantly reduced the microelectrode impedance at all neuronal signal frequencies (from 1 to 10 000 Hz) and induced a significant improvement of the SNR, up to fourfold on average, in the 150-1500 Hz frequency range, largely corresponding to the multiunit frequency band. An equivalent circuit, previously proposed for porous conducting polymer coatings, reproduced the impedance spectra of our coated electrodes but could not explain the frequency dependence of SNR improvement following polypyrrole-CNT coating. This implies that neither the neural signal amplitude, as recorded by a CNT-coated metal microelectrode, nor noise can be fully described by the equivalent circuit model we used here and suggests that a more detailed approach may be needed to better understand the signal propagation at the electrode-solution interface. Finally, the presence of significant noise components that are neither thermal nor electronic makes it difficult to establish a direct relationship between the actual electrode noise and the impedance spectra.

Use of a Bayesian maximum-likelihood classifier to generate training data for brain-machine interfaces

Brain-machine interface decoding algorithms need to be predicated on assumptions that are easily met outside of an experimental setting to enable a practical clinical device. Given present technological limitations, there is a need for decoding algorithms which (a) are not dependent upon a large number of neurons for control, (b) are adaptable to alternative sources of neuronal input such as local field potentials (LFPs), and (c) require only marginal training data for daily calibrations. Moreover, practical algorithms must recognize when the user is not intending to generate a control output and eliminate poor training data. In this paper, we introduce and evaluate a Bayesian maximum-likelihood estimation strategy to address the issues of isolating quality training data and self-paced control. Six animal subjects demonstrate that a multiple state classification task, loosely based on the standard center-out task, can be accomplished with fewer than five engaged neurons while requiring less than ten trials for algorithm training. In addition, untrained animals quickly obtained accurate device control, utilizing LFPs as well as neurons in cingulate cortex, two non-traditional neural inputs.

Medicine could be constructing human bodies in the future

In the 21st century human life has been profoundly changed by developments in sanitation, medical interventions and public health measures. Practically every person born into a developed nation population has a chance to survive throughout entire reproductive life and well beyond. Human body has evolved in the past adaptations to hunting-gathering, and later, agricultural ways of life. In the new situation of practically non-existent premature mortality and technologically developed complex societies medical practice will devote less attention to "saving lives"--preventing premature deaths--and more to enhancing capacities of our biological organisms and providing for maintenance of the bodies beyond their biological limits established by evolution. The role of advances in nanotechnology, information technology, neuroscience and biotechnology is discussed in the context of mind and body enhancements.

The Neurochip-2: an autonomous head-fixed computer for recording and stimulating in freely behaving monkeys

The Neurochip-2 is a second generation, battery-powered device for neural recording and stimulating that is small enough to be carried in a chamber on a monkey's head. It has three recording channels, with user-adjustable gains, filters, and sampling rates, that can be optimized for recording single unit activity, local field potentials, electrocorticography, electromyography, arm acceleration, etc. Recorded data are stored on a removable, flash memory card. The Neurochip-2 also has three separate stimulation channels. Two "programmable-system-on-chips" (PSoCs) control the data acquisition and stimulus output. The PSoCs permit flexible real-time processing of the recorded data, such as digital filtering and time-amplitude window discrimination. The PSoCs can be programmed to deliver stimulation contingent on neural events or deliver preprogrammed stimuli. Access pins to the microcontroller are also available to connect external devices, such as accelerometers. The Neurochip-2 can record and stimulate autonomously for up to several days in freely behaving monkeys, enabling a wide range of novel neurophysiological and neuroengineering experiments.

Comparison of LFP-based and spike-based spectro-temporal receptive fields and cross-correlation in cat primary auditory cortex

Multi-electrode array recordings of spike and local field potential (LFP) activity were made from primary auditory cortex of 12 normal hearing, ketamine-anesthetized cats. We evaluated 259 spectro-temporal receptive fields (STRFs) and 492 frequency-tuning curves (FTCs) based on LFPs and spikes simultaneously recorded on the same electrode. We compared their characteristic frequency (CF) gradients and their cross-correlation distances. The CF gradient for spike-based FTCs was about twice that for 2–40 Hz-filtered LFP-based FTCs, indicating greatly reduced frequency selectivity for LFPs. We also present comparisons for LFPs band-pass filtered between 4–8 Hz, 8–16 Hz and 16–40 Hz, with spike-based STRFs, on the basis of their marginal frequency distributions. We find on average a significantly larger correlation between the spike based marginal frequency distributions and those based on the 16–40 Hz filtered LFP, compared to those based on the 4–8 Hz, 8–16 Hz and 2–40 Hz filtered LFP. This suggests greater frequency specificity for the 16–40 Hz LFPs compared to those of lower frequency content. For spontaneous LFP and spike activity we evaluated 1373 pair correlations for pairs with >200 spikes in 900 s per electrode. Peak correlation-coefficient space constants were similar for the 2–40 Hz filtered LFP (5.5 mm) and the 16–40 Hz LFP (7.4 mm), whereas for spike-pair correlations it was about half that, at 3.2 mm. Comparing spike-pairs with 2–40 Hz (and 16–40 Hz) LFP-pair correlations showed that about 16% (9%) of the variance in the spike-pair correlations could be explained from LFP-pair correlations recorded on the same electrodes within the same electrode array. This larger correlation distance combined with the reduced CF gradient and much broader frequency selectivity suggests that LFPs are not a substitute for spike activity in primary auditory cortex.

Comparison of force and power generation patterns and their predictions under different external dynamic environments

Use of neural activity to predict kinematic variables such as position, velocity and direction etc of movements has been implemented in real-time control of robotic systems and computer cursors. In everyday life, however, we generate variable amounts of force to manipulate objects of different inertial properties or to follow the same trajectory under different external dynamic environments like air or water. The resultant work during such movements, and its time derivative power, should depend on the dynamics of the movement. In order to give the users of a brain-machine interface (BMI) comprehensive control of a prosthetic limb under different dynamic conditions, it is imperative to consider the dynamics-related parameters like end-effector forces, joint torques or power. In this paper, we show distribution patterns of two such dynamics parameters - force and power - and their predictive efficiency under different dynamic environmental conditions. We intend to find the force-related parameter, which has optimal predictive efficiency across different dynamic environments that is generalization. Our ultimate goal is to materialize a force-based brain-machine interface (fBMI).

A symbiotic brain-machine interface through value-based decision making

Background

In the development of Brain Machine Interfaces (BMIs), there is a great need to enable users to interact with changing environments during the activities of daily life. It is expected that the number and scope of the learning tasks encountered during interaction with the environment as well as the pattern of brain activity will vary over time. These conditions, in addition to neural reorganization, pose a challenge to decoding neural commands for BMIs. We have developed a new BMI framework in which a computational agent symbiotically decoded users' intended actions by utilizing both motor commands and goal information directly from the brain through a continuous Perception-Action-Reward Cycle (PARC).

Methodology

The control architecture designed was based on Actor-Critic learning, which is a PARC-based reinforcement learning method. Our neurophysiology studies in rat models suggested that Nucleus Accumbens (NAcc) contained a rich representation of goal information in terms of predicting the probability of earning reward and it could be translated into an evaluative feedback for adaptation of the decoder with high precision. Simulated neural control experiments showed that the system was able to maintain high performance in decoding neural motor commands during novel tasks or in the presence of reorganization in the neural input. We then implanted a dual micro-wire array in the primary motor cortex (M1) and the NAcc of rat brain and implemented a full closed-loop system in which robot actions were decoded from the single unit activity in M1 based on an evaluative feedback that was estimated from NAcc.

Conclusions

Our results suggest that adapting the BMI decoder with an evaluative feedback that is directly extracted from the brain is a possible solution to the problem of operating BMIs in changing environments with dynamic neural signals. During closed-loop control, the agent was able to solve a reaching task by capturing the action and reward interdependency in the brain.

Cognitive neural prosthetics

The cognitive neural prosthetic (CNP) is a very versatile method for assisting paralyzed patients and patients with amputations. The CNP records the cognitive state of the subject, rather than signals strictly related to motor execution or sensation. We review a number of high-level cortical signals and their application for CNPs, including intention, motor imagery, decision making, forward estimation, executive function, attention, learning, and multi-effector movement planning. CNPs are defined by the cognitive function they extract, not the cortical region from which the signals are recorded. However, some cortical areas may be better than others for particular applications. Signals can also be extracted in parallel from multiple cortical areas using multiple implants, which in many circumstances can increase the range of applications of CNPs. The CNP approach relies on scientific understanding of the neural processes involved in cognition, and many of the decoding algorithms it uses also have parallels to underlying neural circuit functions.

Neural mind reading of multi-dimensional decisions by monkey mid-brain activity

Brain-machine interfaces (BMIs) have the potential to improve the quality of life for individuals with disabilities. We engaged in the development of neural mind-reading techniques for cognitive BMIs to provide a readout of decision processes. We trained 2 monkeys on go/no-go tasks, and monitored the activity of groups of neurons in their mid-brain superior colliculus (SC). We designed a virtual decision function (VDF) reflecting the continuous progress of binary decisions on a single-trial basis, and applied it to the ensemble activity of SC neurons. Post hoc analyses using the VDF predicted the cue location as well as the monkey's motor choice (go or no-go) soon after the presentation of the cue. These results suggest that our neural mind-reading techniques have the potential to provide rapid real-time control of communication support devices.

Evolution of brain-computer interfaces: going beyond classic motor physiology

The notion that a computer can decode brain signals to infer the intentions of a human and then enact those intentions directly through a machine is becoming a realistic technical possibility. These types of devices are known as brain-computer interfaces (BCIs). The evolution of these neuroprosthetic technologies could have significant implications for patients with motor disabilities by enhancing their ability to interact and communicate with their environment. The cortical physiology most investigated and used for device control has been brain signals from the primary motor cortex. To date, this classic motor physiology has been an effective substrate for demonstrating the potential efficacy of BCI-based control. However, emerging research now stands to further enhance our understanding of the cortical physiology underpinning human intent and provide further signals for more complex brain-derived control. In this review, the authors report the current status of BCIs and detail the emerging research trends that stand to augment clinical applications in the future.

A Robotic Neural Interface for Autonomous Positioning of Extracellular Recording Electrodes

In this paper we describe a set of algorithms and a novel miniature device that together can autonomously position electrodes in neural tissue to obtain high-quality extracellular recordings. This robotic system moves each electrode to detect the signals of individual neurons, optimize the signal quality of a target neuron, and then maintain this signal over time. Such neuronal signals provide the key inputs for emerging neuroprosthetic medical devices and serve as the foundation of basic neuroscientific and medical research. Experimental results from extensive use of the robotic electrodes in macaque parietal cortex are presented to validate the method and to quantify its effectiveness.

Flexible polymer substrate and tungsten microelectrode array for an implantable neural recording system

This paper describes the process flow and testing of a substrate for a fully implantable neural recording system. Tungsten microwires are hybrid-packaged on a micromachined flexible polymer substrate forming an intracortical microelectrode array for brain machine interfaces. The microelectrode array is characterized on the bench top and tested in vivo. The microelectrode noise floor is less than 2 microV and acute recording results show a signal to noise ratio of 9.9-17.3 dB. The technique of hybrid fabrication of the electrodes on a flexible substrate provides a general platform for the development of an implantable neural recording system.

What is the real shape of extracellular spikes?

We show that the standard filters used for on-line spike detection in most hardware acquisition systems introduce distortions in the recorded spike shapes. This is because on-line spike detection is done after band pass filtering the data with causal filters. As illustrated with three clusters of spike shapes from a real single cell recording in a human subject, causal filtering can create a spurious negative rebound and a smooth looking appearance of the spikes. We also show that these filtering distortions can make artifacts look similar to real spikes.

Brain-computer interface systems: progress and prospects

Brain-computer interface (BCI) systems support communication through direct measures of neural activity without muscle activity. BCIs may provide the best and sometimes the only communication option for users disabled by the most severe neuromuscular disorders and may eventually become useful to less severely disabled and/or healthy individuals across a wide range of applications. This review discusses the structure and functions of BCI systems, clarifies terminology and addresses practical applications. Progress and opportunities in the field are also identified and explicated.