Cortical adaptation to a chronic micro-electrocorticographic brain computer interface

Information Transmission through Biotic-Abiotic Interfaces to Restore or Enhance Human Function

Advancements in reliable information transfer across biotic-abiotic interfaces have enabled the restoration of lost human function. For example, communication between neuronal cells and electrical devices restores the ability to walk to a tetraplegic patient and vision to patients blinded by retinal disease. These impactful medical achievements are aided by tailored biotic-abiotic interfaces that maximize information transfer fidelity by considering the physical properties of the underlying biological and synthetic components. This Review develops a modular framework to define and describe the engineering of biotic and abiotic components as well as the design of interfaces to facilitate biotic-abiotic information transfer using light or electricity. Delineating the properties of the biotic, interface, and abiotic components that enable communication can serve as a guide for future research in this highly interdisciplinary field. Application of synthetic biology to engineer light-sensitive proteins has facilitated the control of neural signaling and the restoration of rudimentary vision after retinal blindness. Electrophysiological methodologies that use brain-computer interfaces and stimulating implants to bypass spinal column injuries have led to the rehabilitation of limb movement and walking ability. Cellular interfacing methodologies and on-chip learning capability have been made possible by organic transistors that mimic the information processing capacity of neurons. The collaboration of molecular biologists, material scientists, and electrical engineers in the emerging field of biotic-abiotic interfacing will lead to the development of prosthetics capable of responding to thought and experiencing touch sensation via direct integration into the human nervous system. Further interdisciplinary research will improve electrical and optical interfacing technologies for the restoration of vision, offering greater visual acuity and potentially color vision in the near future.

Nine decades of electrocorticography: A comparison between epidural and subdural recordings

In recent years, electrocorticography (ECoG) has arisen as a neural signal recording tool in the development of clinically viable neural interfaces. ECoG electrodes are generally placed below the dura mater (subdural) but can also be placed on top of the dura (epidural). In deciding which of these modalities best suits long-term implants, complications and signal quality are important considerations. Conceptually, epidural placement may present a lower risk of complications as the dura is left intact but also a lower signal quality due to the dura acting as a signal attenuator. The extent to which complications and signal quality are affected by the dura, however, has been a matter of debate. To improve our understanding of the effects of the dura on complications and signal quality, we conducted a literature review. We inventorized the effect of the dura on signal quality, decodability and longevity of acute and chronic ECoG recordings in humans and non-human primates. Also, we compared the incidence and nature of serious complications in studies that employed epidural and subdural ECoG. Overall, we found that, even though epidural recordings exhibit attenuated signal amplitude over subdural recordings, particularly for high-density grids, the decodability of epidural recorded signals does not seem to be markedly affected. Additionally, we found that the nature of serious complications was comparable between epidural and subdural recordings. These results indicate that both epidural and subdural ECoG may be suited for long-term neural signal recordings, at least for current generations of clinical and high-density ECoG grids.

Stability of a chronic implanted brain-computer interface in late-stage amyotrophic lateral sclerosis

OBJECTIVE

We investigated the long-term functional stability and home use of a fully implanted electrocorticography (ECoG)-based brain-computer interface (BCI) for communication by an individual with late-stage Amyotrophic Lateral Sclerosis (ALS).

METHODS

Data recorded from the cortical surface of the motor and prefrontal cortex with an implanted brain-computer interface device was evaluated for 36 months after implantation of the system in an individual with late-stage ALS. In addition, electrode impedance and BCI control accuracy were assessed. Key measures included frequency of use of the system for communication, user and system performance, and electrical signal characteristics.

RESULTS

User performance was high consistently over the three years. Power in the high frequency band, used for the control signal, declined slowly in the motor cortex, but control over the signal remained unaffected by time. Impedance increased until month 5, and then remained constant. Frequency of home use increased steadily, indicating adoption of the system by the user.

CONCLUSIONS

The implanted brain-computer interface proves to be robust in an individual with late-stage ALS, given stable performance and control signal for over 36 months.

SIGNIFICANCE

These findings are relevant for the future of implantable brain-computer interfaces along with other brain-sensing technologies, such as responsive neurostimulation.

The use of intracranial recordings to decode human language: Challenges and opportunities

Decoding speech from intracranial recordings serves two main purposes: understanding the neural correlates of speech processing and decoding speech features for targeting speech neuroprosthetic devices. Intracranial recordings have high spatial and temporal resolution, and thus offer a unique opportunity to investigate and decode the electrophysiological dynamics underlying speech processing. In this review article, we describe current approaches to decoding different features of speech perception and production - such as spectrotemporal, phonetic, phonotactic, semantic, and articulatory components - using intracranial recordings. A specific section is devoted to the decoding of imagined speech, and potential applications to speech prosthetic devices. We outline the challenges in decoding human language, as well as the opportunities in scientific and neuroengineering applications.

Configuration of electrical spinal cord stimulation through real-time processing of gait kinematics

Epidural electrical stimulation (EES) of the spinal cord and real-time processing of gait kinematics are powerful methods for the study of locomotion and the improvement of motor control after injury or in neurological disorders. Here, we describe equipment and surgical procedures that can be used to acquire chronic electromyographic (EMG) recordings from leg muscles and to implant targeted spinal cord stimulation systems that remain stable up to several months after implantation in rats and nonhuman primates. We also detail how to exploit these implants to configure electrical spinal cord stimulation policies that allow control over the degree of extension and flexion of each leg during locomotion. This protocol uses real-time processing of gait kinematics and locomotor performance, and can be configured within a few days. Once configured, stimulation bursts are delivered over specific spinal cord locations with precise timing that reproduces the natural spatiotemporal activation of motoneurons during locomotion. These protocols can also be easily adapted for the safe implantation of systems in the vicinity of the spinal cord and to conduct experiments involving real-time movement feedback and closed-loop controllers.

Brain-Machine Interfaces: Powerful Tools for Clinical Treatment and Neuroscientific Investigations

Brain-machine interfaces (BMIs) have exploded in popularity in the past decade. BMIs, also called brain-computer interfaces, provide a direct link between the brain and a computer, usually to control an external device. BMIs have a wide array of potential clinical applications, ranging from restoring communication to people unable to speak due to amyotrophic lateral sclerosis or a stroke, to restoring movement to people with paralysis from spinal cord injury or motor neuron disease, to restoring memory to people with cognitive impairment. Because BMIs are controlled directly by the activity of prespecified neurons or cortical areas, they also provide a powerful paradigm with which to investigate fundamental questions about brain physiology, including neuronal behavior, learning, and the role of oscillations. This article reviews the clinical and neuroscientific applications of BMIs, with a primary focus on motor BMIs.

Age-dependent differences in learning to control a robot arm using a body-machine interface

Body-machine interfaces, i.e. interfaces that rely on body movements to control external assistive devices, have been proposed as a safe and robust means of achieving movement and mobility; however, how children learn these novel interfaces is poorly understood. Here we characterized the learning of a body-machine interface in young unimpaired adults, two groups of typically developing children (9-year and 12-year olds), and one child with congenital limb deficiency. Participants had to control the end-effector of a robot arm in 2D using movements of the shoulder and torso. Results showed a striking effect of age - children had much greater difficulty in learning the task compared to adults, with a majority of the 9-year old group unable to even complete the task. The 12-year olds also showed poorer task performance compared to adults (as measured by longer movement times and greater path lengths), which were associated with less effective search strategies. The child with congenital limb deficiency showed superior task performance compared to age-matched children, but had qualitatively distinct coordination strategies from the adults. Taken together, these results imply that children have difficulty learning non-intuitive interfaces and that the design of body-machine interfaces should account for these differences in pediatric populations.

Progress in the Field of Micro-Electrocorticography

Since the 1940s electrocorticography (ECoG) devices and, more recently, in the last decade, micro-electrocorticography (µECoG) cortical electrode arrays were used for a wide set of experimental and clinical applications, such as epilepsy localization and brain⁻computer interface (BCI) technologies. Miniaturized implantable µECoG devices have the advantage of providing greater-density neural signal acquisition and stimulation capabilities in a minimally invasive fashion. An increased spatial resolution of the µECoG array will be useful for greater specificity diagnosis and treatment of neuronal diseases and the advancement of basic neuroscience and BCI research. In this review, recent achievements of ECoG and µECoG are discussed. The electrode configurations and varying material choices used to design µECoG arrays are discussed, including advantages and disadvantages of µECoG technology compared to electroencephalography (EEG), ECoG, and intracortical electrode arrays. Electrode materials that are the primary focus include platinum, iridium oxide, poly(3,4-ethylenedioxythiophene) (PEDOT), indium tin oxide (ITO), and graphene. We discuss the biological immune response to µECoG devices compared to other electrode array types, the role of µECoG in clinical pathology, and brain⁻computer interface technology. The information presented in this review will be helpful to understand the current status, organize available knowledge, and guide future clinical and research applications of µECoG technologies.

The Potential for a Speech Brain-Computer Interface Using Chronic Electrocorticography

A brain-computer interface (BCI) is a technology that uses neural features to restore or augment the capabilities of its user. A BCI for speech would enable communication in real time via neural correlates of attempted or imagined speech. Such a technology would potentially restore communication and improve quality of life for locked-in patients and other patients with severe communication disorders. There have been many recent developments in neural decoders, neural feature extraction, and brain recording modalities facilitating BCI for the control of prosthetics and in automatic speech recognition (ASR). Indeed, ASR and related fields have developed significantly over the past years, and many lend many insights into the requirements, goals, and strategies for speech BCI. Neural speech decoding is a comparatively new field but has shown much promise with recent studies demonstrating semantic, auditory, and articulatory decoding using electrocorticography (ECoG) and other neural recording modalities. Because the neural representations for speech and language are widely distributed over cortical regions spanning the frontal, parietal, and temporal lobes, the mesoscopic scale of population activity captured by ECoG surface electrode arrays may have distinct advantages for speech BCI, in contrast to the advantages of microelectrode arrays for upper-limb BCI. Nevertheless, there remain many challenges for the translation of speech BCIs to clinical populations. This review discusses and outlines the current state-of-the-art for speech BCI and explores what a speech BCI using chronic ECoG might entail.

Single reach plans in dorsal premotor cortex during a two-target task

In many situations, we are faced with multiple potential actions, but must wait for more information before knowing which to perform. Movement scientists have long asked whether in these delayed-response situations the brain plans both potential movements simultaneously, or if it simply chooses one and then switches later if necessary. To answer this question, we used simultaneously recorded activity from populations of neurons in macaque dorsal premotor cortex to track moment-by-moment deliberation between two potential reach targets. We found that the neural activity only ever indicated a single-reach plan (with some targets favored more than others), and that initial plans often predicted the monkeys’ behavior on both Free-Choice trials and incorrect Cued trials. Our results suggest that premotor cortex plans only one option at a time, and that decisions are strongly influenced by the initial response to the available set of movement options.

It is debated whether motor cortical activity reflects plans for multiple potential actions. Here, the authors report that in a delayed response task with two potential reach targets, population activity in the dorsal premotor cortex at any moment in time represents only one of the targets.

Decoding Inner Speech Using Electrocorticography: Progress and Challenges Toward a Speech Prosthesis

Certain brain disorders resulting from brainstem infarcts, traumatic brain injury, cerebral palsy, stroke, and amyotrophic lateral sclerosis, limit verbal communication despite the patient being fully aware. People that cannot communicate due to neurological disorders would benefit from a system that can infer internal speech directly from brain signals. In this review article, we describe the state of the art in decoding inner speech, ranging from early acoustic sound features, to higher order speech units. We focused on intracranial recordings, as this technique allows monitoring brain activity with high spatial, temporal, and spectral resolution, and therefore is a good candidate to investigate inner speech. Despite intense efforts, investigating how the human cortex encodes inner speech remains an elusive challenge, due to the lack of behavioral and observable measures. We emphasize various challenges commonly encountered when investigating inner speech decoding, and propose potential solutions in order to get closer to a natural speech assistive device.

Signal quality of simultaneously recorded endovascular, subdural and epidural signals are comparable

Recent work has demonstrated the feasibility of minimally-invasive implantation of electrodes into a cortical blood vessel. However, the effect of the dura and blood vessel on recording signal quality is not understood and may be a critical factor impacting implementation of a closed-loop endovascular neuromodulation system. The present work compares the performance and recording signal quality of a minimally-invasive endovascular neural interface with conventional subdural and epidural interfaces. We compared bandwidth, signal-to-noise ratio, and spatial resolution of recorded cortical signals using subdural, epidural and endovascular arrays four weeks after implantation in sheep. We show that the quality of the signals (bandwidth and signal-to-noise ratio) of the endovascular neural interface is not significantly different from conventional neural sensors. However, the spatial resolution depends on the array location and the frequency of recording. We also show that there is a direct correlation between the signal-noise-ratio and classification accuracy, and that decoding accuracy is comparable between electrode arrays. These results support the consideration for use of an endovascular neural interface in a clinical trial of a novel closed-loop neuromodulation technology.

A Cognition-Related Neural Oscillation Pattern, Generated in the Prelimbic Cortex, Can Control Operant Learning in Rats

The prelimbic (PrL) cortex constitutes one of the highest levels of cortical hierarchy dedicated to the execution of adaptive behaviors. We have identified a specific local field potential (LFP) pattern generated in the PrL cortex and associated with cognition-related behaviors. We used this pattern to trigger the activation of a visual display on a touch screen as part of an operant conditioning task. Rats learned to increase the presentation rate of the selected θ to β-γ (θ/β-γ) transition pattern across training sessions. The selected LFP pattern appeared to coincide with a significant decrease in the firing of PrL pyramidal neurons and did not seem to propagate to other cortical or subcortical areas. An indication of the PrL cortex's cognitive nature is that the experimental disruption of this θ/β-γ transition pattern prevented the proper performance of the acquired task without affecting the generation of other motor responses. The use of this LFP pattern to trigger an operant task evoked only minor changes in its electrophysiological properties. Thus, the PrL cortex has the capability of generating an oscillatory pattern for dealing with environmental constraints. In addition, the selected θ/β-γ transition pattern could be a useful tool to activate the presentation of external cues or to modify the current circumstances.SIGNIFICANCE STATEMENT Brain-machine interfaces represent a solution for physically impaired people to communicate with external devices. We have identified a specific local field potential pattern generated in the prelimbic cortex and associated with goal-directed behaviors. We used the pattern to trigger the activation of a visual display on a touch screen as part of an operant conditioning task. Rats learned to increase the presentation rate of the selected field potential pattern across training. The selected pattern was not modified when used to activate the touch screen. Electrical stimulation of the recording site prevented the proper performance of the task. Our findings show that the prelimbic cortex can generate oscillatory patterns that rats can use to control their environment for achieving specific goals.

Physiological properties of brain-machine interface input signals

Brain-machine interfaces (BMIs), also called brain-computer interfaces (BCIs), decode neural signals and use them to control some type of external device. Despite many experimental successes and terrific demonstrations in animals and humans, a high-performance, clinically viable device has not yet been developed for widespread usage. There are many factors that impact clinical viability and BMI performance. Arguably, the first of these is the selection of brain signals used to control BMIs. In this review, we summarize the physiological characteristics and performance-including movement-related information, longevity, and stability-of multiple types of input signals that have been used in invasive BMIs to date. These include intracortical spikes as well as field potentials obtained inside the cortex, at the surface of the cortex (electrocorticography), and at the surface of the dura mater (epidural signals). We also discuss the potential for future enhancements in input signal performance, both by improving hardware and by leveraging the knowledge of the physiological characteristics of these signals to improve decoding and stability.

Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation

Brain-machine interfaces (BMIs) combine methods, approaches, and concepts derived from neurophysiology, computer science, and engineering in an effort to establish real-time bidirectional links between living brains and artificial actuators. Although theoretical propositions and some proof of concept experiments on directly linking the brains with machines date back to the early 1960s, BMI research only took off in earnest at the end of the 1990s, when this approach became intimately linked to new neurophysiological methods for sampling large-scale brain activity. The classic goals of BMIs are 1) to unveil and utilize principles of operation and plastic properties of the distributed and dynamic circuits of the brain and 2) to create new therapies to restore mobility and sensations to severely disabled patients. Over the past decade, a wide range of BMI applications have emerged, which considerably expanded these original goals. BMI studies have shown neural control over the movements of robotic and virtual actuators that enact both upper and lower limb functions. Furthermore, BMIs have also incorporated ways to deliver sensory feedback, generated from external actuators, back to the brain. BMI research has been at the forefront of many neurophysiological discoveries, including the demonstration that, through continuous use, artificial tools can be assimilated by the primate brain's body schema. Work on BMIs has also led to the introduction of novel neurorehabilitation strategies. As a result of these efforts, long-term continuous BMI use has been recently implicated with the induction of partial neurological recovery in spinal cord injury patients.

Workshops of the Sixth International Brain-Computer Interface Meeting: brain-computer interfaces past, present, and future

The Sixth International Brain-Computer Interface (BCI) Meeting was held 30 May-3 June 2016 at the Asilomar Conference Grounds, Pacific Grove, California, USA. The conference included 28 workshops covering topics in BCI and brain-machine interface research. Topics included BCI for specific populations or applications, advancing BCI research through use of specific signals or technological advances, and translational and commercial issues to bring both implanted and non-invasive BCIs to market. BCI research is growing and expanding in the breadth of its applications, the depth of knowledge it can produce, and the practical benefit it can provide both for those with physical impairments and the general public. Here we provide summaries of each workshop, illustrating the breadth and depth of BCI research and highlighting important issues and calls for action to support future research and development.

Continuous decoding of human grasp kinematics using epidural and subdural signals

OBJECTIVE

Restoring or replacing function in paralyzed individuals will one day be achieved through the use of brain-machine interfaces. Regaining hand function is a major goal for paralyzed patients. Two competing prerequisites for the widespread adoption of any hand neuroprosthesis are accurate control over the fine details of movement, and minimized invasiveness. Here, we explore the interplay between these two goals by comparing our ability to decode hand movements with subdural and epidural field potentials (EFPs).

APPROACH

We measured the accuracy of decoding continuous hand and finger kinematics during naturalistic grasping motions in five human subjects. We recorded subdural surface potentials (electrocorticography; ECoG) as well as with EFPs, with both standard- and high-resolution electrode arrays.

MAIN RESULTS

In all five subjects, decoding of continuous kinematics significantly exceeded chance, using either EGoG or EFPs. ECoG decoding accuracy compared favorably with prior investigations of grasp kinematics (mean ± SD grasp aperture variance accounted for was 0.54 ± 0.05 across all subjects, 0.75 ± 0.09 for the best subject). In general, EFP decoding performed comparably to ECoG decoding. The 7-20 Hz and 70-115 Hz spectral bands contained the most information about grasp kinematics, with the 70-115 Hz band containing greater information about more subtle movements. Higher-resolution recording arrays provided clearly superior performance compared to standard-resolution arrays.

SIGNIFICANCE

To approach the fine motor control achieved by an intact brain-body system, it will be necessary to execute motor intent on a continuous basis with high accuracy. The current results demonstrate that this level of accuracy might be achievable not just with ECoG, but with EFPs as well. Epidural placement of electrodes is less invasive, and therefore may incur less risk of encephalitis or stroke than subdural placement of electrodes. Accurately decoding motor commands at the epidural level may be an important step towards a clinically viable brain-machine interface.

Spatial co-adaptation of cortical control columns in a micro-ECoG brain-computer interface

OBJECTIVE

Electrocorticography (ECoG) has been used for a range of applications including electrophysiological mapping, epilepsy monitoring, and more recently as a recording modality for brain-computer interfaces (BCIs). Studies that examine ECoG electrodes designed and implanted chronically solely for BCI applications remain limited. The present study explored how two key factors influence chronic, closed-loop ECoG BCI: (i) the effect of inter-electrode distance on BCI performance and (ii) the differences in neural adaptation and performance when fixed versus adaptive BCI decoding weights are used.

APPROACH

The amplitudes of epidural micro-ECoG signals between 75 and 105 Hz with 300 μm diameter electrodes were used for one-dimensional and two-dimensional BCI tasks. The effect of inter-electrode distance on BCI control was tested between 3 and 15 mm. Additionally, the performance and cortical modulation differences between constant, fixed decoding using a small subset of channels versus adaptive decoding weights using the entire array were explored.

MAIN RESULTS

Successful BCI control was possible with two electrodes separated by 9 and 15 mm. Performance decreased and the signals became more correlated when the electrodes were only 3 mm apart. BCI performance in a 2D BCI task improved significantly when using adaptive decoding weights (80%-90%) compared to using constant, fixed weights (50%-60%). Additionally, modulation increased for channels previously unavailable for BCI control under the fixed decoding scheme upon switching to the adaptive, all-channel scheme.

SIGNIFICANCE

Our results clearly show that neural activity under a BCI recording electrode (which we define as a 'cortical control column') readily adapts to generate an appropriate control signal. These results show that the practical minimal spatial resolution of these control columns with micro-ECoG BCI is likely on the order of 3 mm. Additionally, they show that the combination and interaction between neural adaptation and machine learning are critical to optimizing ECoG BCI performance.

Category-Selectivity in Human Visual Cortex Follows Cortical Topology: A Grouped icEEG Study

Neuroimaging studies suggest that category-selective regions in higher-order visual cortex are topologically organized around specific anatomical landmarks: the mid-fusiform sulcus (MFS) in the ventral temporal cortex (VTC) and lateral occipital sulcus (LOS) in the lateral occipital cortex (LOC). To derive precise structure-function maps from direct neural signals, we collected intracranial EEG (icEEG) recordings in a large human cohort (n = 26) undergoing implantation of subdural electrodes. A surface-based approach to grouped icEEG analysis was used to overcome challenges from sparse electrode coverage within subjects and variable cortical anatomy across subjects. The topology of category-selectivity in bilateral VTC and LOC was assessed for five classes of visual stimuli-faces, animate non-face (animals/body-parts), places, tools, and words-using correlational and linear mixed effects analyses. In the LOC, selectivity for living (faces and animate non-face) and non-living (places and tools) classes was arranged in a ventral-to-dorsal axis along the LOS. In the VTC, selectivity for living and non-living stimuli was arranged in a latero-medial axis along the MFS. Written word-selectivity was reliably localized to the intersection of the left MFS and the occipito-temporal sulcus. These findings provide direct electrophysiological evidence for topological information structuring of functional representations within higher-order visual cortex.

Decoding three-dimensional reaching movements using electrocorticographic signals in humans

OBJECTIVE

Electrocorticography (ECoG) signals have emerged as a potential control signal for brain-computer interface (BCI) applications due to balancing signal quality and implant invasiveness. While there have been numerous demonstrations in which ECoG signals were used to decode motor movements and to develop BCI systems, the extent of information that can be decoded has been uncertain. Therefore, we sought to determine if ECoG signals could be used to decode kinematics (speed, velocity, and position) of arm movements in 3D space.

APPROACH

To investigate this, we designed a 3D center-out reaching task that was performed by five epileptic patients undergoing temporary placement of ECoG arrays. We used the ECoG signals within a hierarchical partial-least squares (PLS) regression model to perform offline prediction of hand speed, velocity, and position.

MAIN RESULTS

The hierarchical PLS regression model enabled us to predict hand speed, velocity, and position during 3D reaching movements from held-out test sets with accuracies above chance in each patient with mean correlation coefficients between 0.31 and 0.80 for speed, 0.27 and 0.54 for velocity, and 0.22 and 0.57 for position. While beta band power changes were the most significant features within the model used to classify movement and rest, the local motor potential and high gamma band power changes, were the most important features in the prediction of kinematic parameters.

SIGNIFICANCE

We believe that this study represents the first demonstration that truly three-dimensional movements can be predicted from ECoG recordings in human patients. Furthermore, this prediction underscores the potential to develop BCI systems with multiple degrees of freedom in human patients using ECoG.

Individual finger control of a modular prosthetic limb using high-density electrocorticography in a human subject

OBJECTIVE

We used native sensorimotor representations of fingers in a brain-machine interface (BMI) to achieve immediate online control of individual prosthetic fingers.

APPROACH

Using high gamma responses recorded with a high-density electrocorticography (ECoG) array, we rapidly mapped the functional anatomy of cued finger movements. We used these cortical maps to select ECoG electrodes for a hierarchical linear discriminant analysis classification scheme to predict: (1) if any finger was moving, and, if so, (2) which digit was moving. To account for sensory feedback, we also mapped the spatiotemporal activation elicited by vibrotactile stimulation. Finally, we used this prediction framework to provide immediate online control over individual fingers of the Johns Hopkins University Applied Physics Laboratory modular prosthetic limb.

MAIN RESULTS

The balanced classification accuracy for detection of movements during the online control session was 92% (chance: 50%). At the onset of movement, finger classification was 76% (chance: 20%), and 88% (chance: 25%) if the pinky and ring finger movements were coupled. Balanced accuracy of fully flexing the cued finger was 64%, and 77% had we combined pinky and ring commands. Offline decoding yielded a peak finger decoding accuracy of 96.5% (chance: 20%) when using an optimized selection of electrodes. Offline analysis demonstrated significant finger-specific activations throughout sensorimotor cortex. Activations either prior to movement onset or during sensory feedback led to discriminable finger control.

SIGNIFICANCE

Our results demonstrate the ability of ECoG-based BMIs to leverage the native functional anatomy of sensorimotor cortical populations to immediately control individual finger movements in real time.

Brain-computer interfaces for dissecting cognitive processes underlying sensorimotor control

Sensorimotor control engages cognitive processes such as prediction, learning, and multisensory integration. Understanding the neural mechanisms underlying these cognitive processes with arm reaching is challenging because we currently record only a fraction of the relevant neurons, the arm has nonlinear dynamics, and multiple modalities of sensory feedback contribute to control. A brain-computer interface (BCI) is a well-defined sensorimotor loop with key simplifying advantages that address each of these challenges, while engaging similar cognitive processes. As a result, BCI is becoming recognized as a powerful tool for basic scientific studies of sensorimotor control. Here, we describe the benefits of BCI for basic scientific inquiries and review recent BCI studies that have uncovered new insights into the neural mechanisms underlying sensorimotor control.

Current Challenges Facing the Translation of Brain Computer Interfaces from Preclinical Trials to Use in Human Patients

Current research in brain computer interface (BCI) technology is advancing beyond preclinical studies, with trials beginning in human patients. To date, these trials have been carried out with several different types of recording interfaces. The success of these devices has varied widely, but different factors such as the level of invasiveness, timescale of recorded information, and ability to maintain stable functionality of the device over a long period of time all must be considered in addition to accuracy in decoding intent when assessing the most practical type of device moving forward. Here, we discuss various approaches to BCIs, distinguishing between devices focusing on control of operations extrinsic to the subject (e.g., prosthetic limbs, computer cursors) and those focusing on control of operations intrinsic to the brain (e.g., using stimulation or external feedback), including closed-loop or adaptive devices. In this discussion, we consider the current challenges facing the translation of various types of BCI technology to eventual human application.

Sub-mm functional decoupling of electrocortical signals through closed-loop BMI learning

Volitional control of neural activity lies at the heart of the Brain-Machine Interface (BMI) paradigm. In this work we investigated if subdural field potentials recorded by electrodes < 1mm apart can be decoupled through closed-loop BMI learning. To this end, we fabricated custom, flexible microelectrode arrays with 200 µm electrode pitch and increased the effective electrode area by electrodeposition of platinum black to reduce thermal noise. We have chronically implanted these arrays subdurally over primary motor cortex (M1) of 5 male Long-Evans Rats and monitored the electrochemical electrode impedance in vivo to assess the stability of these neural interfaces. We successfully trained the rodents to perform a one-dimensional center-out task using closed-loop brain control to adjust the pitch of an auditory cursor by differentially modulating high gamma (70-110 Hz) power on pairs of surface microelectrodes that were separated by less than 1 mm.

Strategies for optical control and simultaneous electrical readout of extended cortical circuits

BACKGROUND

To dissect the intricate workings of neural circuits, it is essential to gain precise control over subsets of neurons while retaining the ability to monitor larger-scale circuit dynamics. This requires the ability to both evoke and record neural activity simultaneously with high spatial and temporal resolution.

NEW METHOD

In this paper we present approaches that address this need by combining micro-electrocorticography (μECoG) with optogenetics in ways that avoid photovoltaic artifacts.

RESULTS

We demonstrate that variations of this approach are broadly applicable across three commonly studied mammalian species - mouse, rat, and macaque monkey - and that the recorded μECoG signal shows complex spectral and spatio-temporal patterns in response to optical stimulation.

COMPARISON WITH EXISTING METHODS

While optogenetics provides the ability to excite or inhibit neural subpopulations in a targeted fashion, large-scale recording of resulting neural activity remains challenging. Recent advances in optical physiology, such as genetically encoded Ca(2+) indicators, are promising but currently do not allow simultaneous recordings from extended cortical areas due to limitations in optical imaging hardware.

CONCLUSIONS

We demonstrate techniques for the large-scale simultaneous interrogation of cortical circuits in three commonly used mammalian species.

Cortical neuroprosthetics from a clinical perspective

Recent pilot clinical studies have demonstrated that subjects with severe disorders of movement and communication can exert direct neural control over assistive devices using invasive Brain-Machine Interface (BMI) technology, also referred to as 'cortical neuroprosthetics'. These important proof-of-principle studies have generated great interest among those with disability and clinicians who provide general medical, neurological and/or rehabilitative care. Taking into account the perspective of providers who may be unfamiliar with the field, we first review the clinical goals and fundamentals of invasive BMI technology, and then briefly summarize the vast body of basic science research demonstrating its feasibility. We emphasize recent translational progress in the target clinical populations and discuss translational challenges and future directions.

Brain computer interface learning for systems based on electrocorticography and intracortical microelectrode arrays

A brain-computer interface (BCI) system transforms neural activity into control signals for external devices in real time. A BCI user needs to learn to generate specific cortical activity patterns to control external devices effectively. We call this process BCI learning, and it often requires significant effort and time. Therefore, it is important to study this process and develop novel and efficient approaches to accelerate BCI learning. This article reviews major approaches that have been used for BCI learning, including computer-assisted learning, co-adaptive learning, operant conditioning, and sensory feedback. We focus on BCIs based on electrocorticography and intracortical microelectrode arrays for restoring motor function. This article also explores the possibility of brain modulation techniques in promoting BCI learning, such as electrical cortical stimulation, transcranial magnetic stimulation, and optogenetics. Furthermore, as proposed by recent BCI studies, we suggest that BCI learning is in many ways analogous to motor and cognitive skill learning, and therefore skill learning should be a useful metaphor to model BCI learning.

Closed-Loop Optogenetic Brain Interface

This paper presents a new approach for implementation of closed-loop brain-machine interface algorithms by combining optogenetic neural stimulation with electrocorticography and fluorescence microscopy. We used a new generation of microfabricated electrocorticography (micro-ECoG) devices in which electrode arrays are embedded within an optically transparent biocompatible substrate that provides optical access to the brain tissue during electrophysiology recording. An optical setup was designed capable of projecting arbitrary patterns of light for optogenetic stimulation and performing fluorescence microscopy through the implant. For realization of a closed-loop system using this platform, the feedback can be taken from electrophysiology data or fluorescence imaging. In the closed-loop systems discussed in this paper, the feedback signal was taken from the micro-ECoG. In these algorithms, the electrophysiology data are continuously transferred to a computer and compared with some predefined spatial-temporal patterns of neural activity. The computer which processes the data also readjusts the duration and distribution of optogenetic stimulating pulses to minimize the difference between the recorded activity and the predefined set points so that after a limited period of transient response the recorded activity follows the set points. Details of the system design and implementation of typical closed-loop paradigms are discussed in this paper.

Studying brain functions with mesoscopic measurements: Advances in electrocorticography for non-human primates

Our brain is organized in a modular structure. Information in different modalities is processed within distinct cortical areas. However, individual cortical areas cannot enable complex cognitive functions without interacting with other cortical areas. Electrocorticography (ECoG) has recently become an important tool for studying global network activity across cortical areas in animal models. With stable recordings of electrical field potentials from multiple cortical areas, ECoG provides an opportunity to systematically study large-scale cortical activity at a mesoscopic spatiotemporal resolution under various experimental conditions. Recent developments in thin, flexible ECoG electrodes permit recording field potentials from not only gyral but intrasulcal cortical surfaces. Our review here focuses on the recent advances of ECoG applications to non-human primates.

Which motor cortical region best predicts imagined movement?

In brain-computer interfacing (BCI), motor imagery is used to provide a gateway to an effector action or behavior. However, in contrast to the main functional role of the primary motor cortex (M1) in motor execution, the M1's involvement in motor imagery has been debated, while the roles of secondary motor areas such as the premotor cortex (PMC) and supplementary motor area (SMA) in motor imagery have been proposed. We examined which motor cortical region had the greatest predictive ability for imagined movement among the primary and secondary motor areas. For two modes of motor performance, executed movement and imagined movement, in 12 healthy subjects who performed two types of motor task, hand grasping and hand rotation, we used the multivariate Bayes method to compare predictive ability between the primary and secondary motor areas (M1, PMC, and SMA) contralateral to the moved hand. With the distributed representation of activation, executed movement was best predicted from the M1 while imagined movement from the SMA, among the three motor cortical regions, in both types of motor task. In addition, the most predictive information about the distinction between executed movement and imagined movement was contained in the M1. The greater predictive ability of the SMA for imagined movement suggests its functional role that could be applied to motor imagery-based BCI.

A multi-channel, flex-rigid ECoG microelectrode array for visual cortical interfacing

High-density electrocortical (ECoG) microelectrode arrays are promising signal-acquisition platforms for brain-computer interfaces envisioned, e.g., as high-performance communication solutions for paralyzed persons. We propose a multi-channel microelectrode array capable of recording ECoG field potentials with high spatial resolution. The proposed array is of a 150 mm2 total recording area; it has 124 circular electrodes (100, 300 and 500 µm in diameter) situated on the edges of concentric hexagons (min. 0.8 mm interdistance) and a skull-facing reference electrode (2.5 mm2 surface area). The array is processed as a free-standing device to enable monolithic integration of a rigid interposer, designed for soldering of fine-pitch SMD-connectors on a minimal assembly area. Electrochemical characterization revealed distinct impedance spectral bands for the 100, 300 and 500 µm-type electrodes, and for the array's own reference. Epidural recordings from the primary visual cortex (V1) of an awake Rhesus macaque showed natural electrophysiological signals and clear responses to standard visual stimulation. The ECoG electrodes of larger surface area recorded signals with greater spectral power in the gamma band, while the skull-facing reference electrode provided higher average gamma power spectral density (γPSD) than the common average referencing technique.

Optogenetic brain interfaces

The brain is a large network of interconnected neurons where each cell functions as a nonlinear processing element. Unraveling the mysteries of information processing in the complex networks of the brain requires versatile neurostimulation and imaging techniques. Optogenetics is a new stimulation method which allows the activity of neurons to be modulated by light. For this purpose, the cell-types of interest are genetically targeted to produce light-sensitive proteins. Once these proteins are expressed, neural activity can be controlled by exposing the cells to light of appropriate wavelengths. Optogenetics provides a unique combination of features, including multimodal control over neural function and genetic targeting of specific cell-types. Together, these versatile features combine to a powerful experimental approach, suitable for the study of the circuitry of psychiatric and neurological disorders. The advent of optogenetics was followed by extensive research aimed to produce new lines of light-sensitive proteins and to develop new technologies: for example, to control the distribution of light inside the brain tissue or to combine optogenetics with other modalities including electrophysiology, electrocorticography, nonlinear microscopy, and functional magnetic resonance imaging. In this paper, the authors review some of the recent advances in the field of optogenetics and related technologies and provide their vision for the future of the field.

Advanced Materials for Neural Surface Electrodes

Designing electrodes for neural interfacing applications requires deep consideration of a multitude of materials factors. These factors include, but are not limited to, the stiffness, biocompatibility, biostability, dielectric, and conductivity properties of the materials involved. The combination of materials properties chosen not only determines the ability of the device to perform its intended function, but also the extent to which the body reacts to the presence of the device after implantation. Advances in the field of materials science continue to yield new and improved materials with properties well-suited for neural applications. Although many of these materials have been well-established for non-biological applications, their use in medical devices is still relatively novel. The intention of this review is to outline new material advances for neural electrode arrays, in particular those that interface with the surface of the nervous tissue, as well as to propose future directions for neural surface electrode development.

Neural constraints on learning

Motor, sensory, and cognitive learning require networks of neurons to generate new activity patterns. Because some behaviors are easier to learn than others^1,2, we wondered if some neural activity patterns are easier to generate than others. We asked whether the existing network constrains the patterns that a subset of its neurons is capable of exhibiting, and if so, what principles define the constraint. We employed a closed-loop intracortical brain-computer interface (BCI) learning paradigm in which Rhesus monkeys controlled a computer cursor by modulating neural activity patterns in primary motor cortex. Using the BCI paradigm, we could specify and alter how neural activity mapped to cursor velocity. At the start of each session, we observed the characteristic activity patterns of the recorded neural population. These patterns comprise a low-dimensional space (termed the intrinsic manifold, or IM) within the high-dimensional neural firing rate space. They presumably reflect constraints imposed by the underlying neural circuitry. We found that the animals could readily learn to proficiently control the cursor using neural activity patterns that were within the IM. However, animals were less able to learn to proficiently control the cursor using activity patterns that were outside of the IM. This result suggests that the existing structure of a network can shape learning. On the timescale of hours, it appears to be difficult to learn to generate neural activity patterns that are not consistent with the existing network structure. These findings offer a network-level explanation for the observation that we are more readily able to learn new skills when they are related to the skills that we already possess^3,4.

Epidural electrocorticography for monitoring of arousal in locked-in state

Electroencephalography (EEG) often fails to assess both the level (i.e., arousal) and the content (i.e., awareness) of pathologically altered consciousness in patients without motor responsiveness. This might be related to a decline of awareness, to episodes of low arousal and disturbed sleep patterns, and/or to distorting and attenuating effects of the skull and intermediate tissue on the recorded brain signals. Novel approaches are required to overcome these limitations. We introduced epidural electrocorticography (ECoG) for monitoring of cortical physiology in a late-stage amytrophic lateral sclerosis patient in completely locked-in state (CLIS). Despite long-term application for a period of six months, no implant-related complications occurred. Recordings from the left frontal cortex were sufficient to identify three arousal states. Spectral analysis of the intrinsic oscillatory activity enabled us to extract state-dependent dominant frequencies at <4, ~7 and ~20 Hz, representing sleep-like periods, and phases of low and elevated arousal, respectively. In the absence of other biomarkers, ECoG proved to be a reliable tool for monitoring circadian rhythmicity, i.e., avoiding interference with the patient when he was sleeping and exploiting time windows of responsiveness. Moreover, the effects of interventions addressing the patient's arousal, e.g., amantadine medication, could be evaluated objectively on the basis of physiological markers, even in the absence of behavioral parameters. Epidural ECoG constitutes a feasible trade-off between surgical risk and quality of recorded brain signals to gain information on the patient's present level of arousal. This approach enables us to optimize the timing of interactions and medical interventions, all of which should take place when the patient is in a phase of high arousal. Furthermore, avoiding low-responsiveness periods will facilitate measures to implement alternative communication pathways involving brain-computer interfaces (BCI).

Brain–Computer Interface after Nervous System Injury

Brain–computer interface (BCI) has proven to be a useful tool for providing alternative communication and mobility to patients suffering from nervous system injury. BCI has been and will continue to be implemented into rehabilitation practices for more interactive and speedy neurological recovery. The most exciting BCI technology is evolving to provide therapeutic benefits by inducing cortical reorganization via neuronal plasticity. This article presents a state-of-the-art review of BCI technology used after nervous system injuries, specifically: amyotrophic lateral sclerosis, Parkinson’s disease, spinal cord injury, stroke, and disorders of consciousness. Also presented is transcending, innovative research involving new treatment of neurological disorders.

Neural coding for effective rehabilitation

Successful neurological rehabilitation depends on accurate diagnosis, effective treatment, and quantitative evaluation. Neural coding, a technology for interpretation of functional and structural information of the nervous system, has contributed to the advancements in neuroimaging, brain-machine interface (BMI), and design of training devices for rehabilitation purposes. In this review, we summarized the latest breakthroughs in neuroimaging from microscale to macroscale levels with potential diagnostic applications for rehabilitation. We also reviewed the achievements in electrocorticography (ECoG) coding with both animal models and human beings for BMI design, electromyography (EMG) interpretation for interaction with external robotic systems, and robot-assisted quantitative evaluation on the progress of rehabilitation programs. Future rehabilitation would be more home-based, automatic, and self-served by patients. Further investigations and breakthroughs are mainly needed in aspects of improving the computational efficiency in neuroimaging and multichannel ECoG by selection of localized neuroinformatics, validation of the effectiveness in BMI guided rehabilitation programs, and simplification of the system operation in training devices.

Extracting kinetic information from human motor cortical signals

Brain machine interfaces (BMIs) have the potential to provide intuitive control of neuroprostheses to restore grasp to patients with paralyzed or amputated upper limbs. For these neuroprostheses to function, the ability to accurately control grasp force is critical. Grasp force can be decoded from neuronal spikes in monkeys, and hand kinematics can be decoded using electrocorticogram (ECoG) signals recorded from the surface of the human motor cortex. We hypothesized that kinetic information about grasping could also be extracted from ECoG, and sought to decode continuously-graded grasp force. In this study, we decoded isometric pinch force with high accuracy from ECoG in 10 human subjects. The predicted signals explained from 22% to 88% (60 ± 6%, mean ± SE) of the variance in the actual force generated. We also decoded muscle activity in the finger flexors, with similar accuracy to force decoding. We found that high gamma band and time domain features of the ECoG signal were most informative about kinetics, similar to our previous findings with intracortical LFPs. In addition, we found that peak cortical representations of force applied by the index and little fingers were separated by only about 4mm. Thus, ECoG can be used to decode not only kinematics, but also kinetics of movement. This is an important step toward restoring intuitively-controlled grasp to impaired patients.

Comparing temporal aspects of visual, tactile, and microstimulation feedback for motor control

OBJECTIVES

Current brain-computer interfaces (BCIs) rely on visual feedback, requiring sustained visual attention to use the device. Improvements to BCIs may stem from the development of an effective way to provide quick feedback independent of vision. Tactile stimuli, either delivered on the skin surface, or directly to the brain via microstimulation in somatosensory cortex, could serve that purpose. We examined the effectiveness of vibrotactile stimuli and microstimulation as a means of non-visual feedback by using a fundamental element of feedback: the ability to react to a stimulus while already in motion.

APPROACH

Human and monkey subjects performed a center-out reach task which was, on occasion, interrupted with a stimulus cue that instructed a change in reach target.

MAIN RESULTS

Subjects generally responded faster to tactile cues than to visual cues. However, when we delivered cues via microstimuation in a monkey, its response was slower on average than for both tactile and visual cues.

SIGNIFICANCE

Tactile and microstimulation feedback can be used to rapidly adjust movements mid-flight. The relatively slow speed of microstimulation is surprising and warrants further investigation. Overall, these results highlight the importance of considering temporal aspects of feedback when designing alternative forms of feedback for BCIs.

Brain-computer interfaces: a powerful tool for scientific inquiry

Brain-computer interfaces (BCIs) are devices that record from the nervous system, provide input directly to the nervous system, or do both. Sensory BCIs such as cochlear implants have already had notable clinical success and motor BCIs have shown great promise for helping patients with severe motor deficits. Clinical and engineering outcomes aside, BCIs can also be tremendously powerful tools for scientific inquiry into the workings of the nervous system. They allow researchers to inject and record information at various stages of the system, permitting investigation of the brain in vivo and facilitating the reverse engineering of brain function. Most notably, BCIs are emerging as a novel experimental tool for investigating the tremendous adaptive capacity of the nervous system.

Subject-specific modulation of local field potential spectral power during brain-machine interface control in primates

OBJECTIVE

Intracortical brain-machine interfaces (BMIs) have predominantly utilized spike activity as the control signal. However, an increasing number of studies have shown the utility of local field potentials (LFPs) for decoding motor related signals. Currently, it is unclear how well different LFP frequencies can serve as features for continuous, closed-loop BMI control.

APPROACH

We demonstrate 2D continuous LFP-based BMI control using closed-loop decoder adaptation, which adapts decoder parameters to subject-specific LFP feature modulations during BMI control. We trained two macaque monkeys to control a 2D cursor in a center-out task by modulating LFP power in the 0-150 Hz range.

MAIN RESULTS

While both monkeys attained control, they used different strategies involving different frequency bands. One monkey primarily utilized the low-frequency spectrum (0-80 Hz), which was highly correlated between channels, and obtained proficient performance even with a single channel. In contrast, the other monkey relied more on higher frequencies (80-150 Hz), which were less correlated between channels, and had greater difficulty with control as the number of channels decreased. We then restricted the monkeys to use only various sub-ranges (0-40, 40-80, and 80-150 Hz) of the 0-150 Hz band. Interestingly, although both monkeys performed better with some sub-ranges than others, they were able to achieve BMI control with all sub-ranges after decoder adaptation, demonstrating broad flexibility in the frequencies that could potentially be used for LFP-based BMI control.

SIGNIFICANCE

Overall, our results demonstrate proficient, continuous BMI control using LFPs and provide insight into the subject-specific spectral patterns of LFP activity modulated during control.

Optogenetic micro-electrocorticography for modulating and localizing cerebral cortex activity

OBJECTIVE

Spatial localization of neural activity from within the brain with electrocorticography (ECoG) and electroencephalography remains a challenge in clinical and research settings, and while microfabricated ECoG (micro-ECoG) array technology continues to improve, complementary methods to simultaneously modulate cortical activity while recording are needed.

APPROACH

We developed a neural interface utilizing optogenetics, cranial windowing, and micro-ECoG arrays fabricated on a transparent polymer. This approach enabled us to directly modulate neural activity at known locations around micro-ECoG arrays in mice expressing Channelrhodopsin-2. We applied photostimuli varying in time, space and frequency to the cortical surface, and we targeted multiple depths within the cortex using an optical fiber while recording micro-ECoG signals.

MAIN RESULTS

Negative potentials of up to 1.5 mV were evoked by photostimuli applied to the entire cortical window, while focally applied photostimuli evoked spatially localized micro-ECoG potentials. Two simultaneously applied focal stimuli could be separated, depending on the distance between them. Photostimuli applied within the cortex with an optical fiber evoked more complex micro-ECoG potentials with multiple positive and negative peaks whose relative amplitudes depended on the depth of the fiber.

SIGNIFICANCE

Optogenetic ECoG has potential applications in the study of epilepsy, cortical dynamics, and neuroprostheses.