Wireless Cortical Brain-Machine Interface for Whole-Body Navigation in Primates

Towards hippocampal navigation for brain-computer interfaces

Automatic wheelchairs directly controlled by brain activity could provide autonomy to severely paralyzed individuals. Current approaches mostly rely on non-invasive measures of brain activity and translate individual commands into wheelchair movements. For example, an imagined movement of the right hand would steer the wheelchair to the right. No research has investigated decoding higher-order cognitive processes to accomplish wheelchair control. We envision an invasive neural prosthetic that could provide input for wheelchair control by decoding navigational intent from hippocampal signals. Navigation has been extensively investigated in hippocampal recordings, but not for the development of neural prostheses. Here we show that it is possible to train a decoder to classify virtual-movement speeds from hippocampal signals recorded during a virtual-navigation task. These results represent the first step toward exploring the feasibility of an invasive hippocampal BCI for wheelchair control.

Decoding spatial locations from primate lateral prefrontal cortex neural activity during virtual navigation

Objective. Decoding the intended trajectories from brain signals using a brain-computer interface system could be used to improve the mobility of patients with disabilities.Approach. Neuronal activity associated with spatial locations was examined while macaques performed a navigation task within a virtual environment.Main results.Here, we provide proof of principle that multi-unit spiking activity recorded from the lateral prefrontal cortex (LPFC) of non-human primates can be used to predict the location of a subject in a virtual maze during a navigation task. The spatial positions within the maze that require a choice or are associated with relevant task events can be better predicted than the locations where no relevant events occur. Importantly, within a task epoch of a single trial, multiple locations along the maze can be independently identified using a support vector machine model.Significance. Considering that the LPFC of macaques and humans share similar properties, our results suggest that this area could be a valuable implant location for an intracortical brain-computer interface system used for spatial navigation in patients with disabilities.

Home-Enclosure-Based Behavioral and Wireless Neural Recording Setup for Unrestrained Rhesus Macaques

Electrophysiological studies with behaving nonhuman primates often require the separation of animals from their social group as well as partial movement restraint to perform well-controlled experiments. When the research goal per se does not mandate constraining the animals' movements, there are often still experimental needs imposed by tethered data acquisition. Recent technological advances meanwhile allow wireless neurophysiological recordings at high band-width in limited-size enclosures. Here, we demonstrate wireless neural recordings at single unit resolution from unrestrained rhesus macaques while they performed self-paced, structured visuomotor tasks on our custom-built, stand-alone touchscreen system [eXperimental Behavioral Instrument (XBI)] in their home environment. We were able to successfully characterize neural tuning to task parameters, such as visuo-spatial selectivity during movement planning and execution, as expected from existing findings obtained via setup-based neurophysiology recordings. We conclude that when movement restraint and/or a highly controlled, insulated environment are not necessary for scientific reasons, cage-based wireless neural recordings are a viable option. We propose an approach that allows the animals to engage in a self-paced manner with our XBI device, both for fully automatized training and cognitive testing, as well as neural data acquisition in their familiar environment, maintaining auditory and sometimes visual contact with their conspecifics.

Development of Living “Bio-Robots” for Autonomous Actuations

The implementation of autonomous functions, such as autonomous actuation, self-healing, and learning functions, has been a potent strategy to realize adaptation abilities against changes in environments and sudden incidents. Organic materials, such as living cells and tissues, can be used as robot parts for the implementation of autonomous functions because they can modify biological functions and remodel tissue morphologies in response to the environment. A brain organoid is a cell aggregate formed by recapitulating the development processes of the fetal brain in vitro. Because the brain organoid reproduces complex 3D structures and various cells, it can be used as a living regulator of robots for implementing complex autonomous functions. In contrast, engineered muscle tissues constructed by culturing myoblasts with biomaterials can also be used as a living actuator for robots. Therefore, to implement autonomous functions for robots, we have proposed methods for connecting the brain organoid with engineered muscle tissue and for co-culturing complex in a culture vessel.

Cortical Control of Virtual Self-Motion Using Task-Specific Subspaces

Brain-machine interfaces (BMIs) for reaching have enjoyed continued performance improvements, yet there remains significant need for BMIs that control other movement classes. Recent scientific findings suggest that the intrinsic covariance structure of neural activity depends strongly on movement class, potentially necessitating different decode algorithms across classes. To address this possibility, we developed a self-motion BMI based on cortical activity as monkeys cycled a hand-held pedal to progress along a virtual track. Unlike during reaching, we found no high-variance dimensions that directly correlated with to-be-decoded variables. This was due to no neurons having consistent correlations between their responses and kinematic variables. Yet we could decode a single variable-self-motion-by nonlinearly leveraging structure that spanned multiple high-variance neural dimensions. Resulting online BMI-control success rates approached those during manual control. These findings make two broad points regarding how to build decode algorithms that harmonize with the empirical structure of neural activity in motor cortex. First, even when decoding from the same cortical region (e.g., arm-related motor cortex), different movement classes may need to employ very different strategies. Although correlations between neural activity and hand velocity are prominent during reaching tasks, they are not a fundamental property of motor cortex and cannot be counted on to be present in general. Second, although one generally desires a low-dimensional readout, it can be beneficial to leverage a multidimensional high-variance subspace. Fully embracing this approach requires highly nonlinear approaches tailored to the task at hand, but can produce near-native levels of performance.SIGNIFICANCE STATEMENT Many brain-machine interface decoders have been constructed for controlling movements normally performed with the arm. Yet it is unclear how these will function beyond the reach-like scenarios where they were developed. Existing decoders implicitly assume that neural covariance structure, and correlations with to-be-decoded kinematic variables, will be largely preserved across tasks. We find that the correlation between neural activity and hand kinematics, a feature typically exploited when decoding reach-like movements, is essentially absent during another task performed with the arm: cycling through a virtual environment. Nevertheless, the use of a different strategy, one focused on leveraging the highest-variance neural signals, supported high performance real-time brain-machine interface control.

A markerless platform for ambulatory systems neuroscience

Motor systems neuroscience seeks to understand how the brain controls movement. To minimize confounding variables, large-animal studies typically constrain body movement from areas not under observation, ensuring consistent, repeatable behaviors. Such studies have fueled decades of research, but they may be artificially limiting the richness of neural data observed, preventing generalization to more natural movements and settings. Neuroscience studies of unconstrained movement would capture a greater range of behavior and a more complete view of neuronal activity, but instrumenting an experimental rig suitable for large animals presents substantial engineering challenges. Here, we present a markerless, full-body motion tracking and synchronized wireless neural electrophysiology platform for large, ambulatory animals. Composed of four depth (RGB-D) cameras that provide a 360° view of a 4.5-square-meters enclosed area, this system is designed to record a diverse range of neuroethologically relevant behaviors. This platform also allows for the simultaneous acquisition of hundreds of wireless neural recording channels in multiple brain regions. As behavioral and neuronal data are generated at rates below 200 megabytes per second, a single desktop can facilitate hours of continuous recording. This setup is designed for systems neuroscience and neuroengineering research, where synchronized kinematic behavior and neural data are the foundation for investigation. By enabling the study of previously unexplored movement tasks, this system can generate insights into the functioning of the mammalian motor system and provide a platform to develop brain-machine interfaces for unconstrained applications.

Recording Strategies for High Channel Count, Densely Spaced Microelectrode Arrays

Neuroscience research into how complex brain functions are implemented at an extra-cellular level requires in vivo neural recording interfaces, including microelectrodes and read-out circuitry, with increased observability and spatial resolution. The trend in neural recording interfaces toward employing high-channel-count probes or 2D microelectrodes arrays with densely spaced recording sites for recording large neuronal populations makes it harder to save on resources. The low-noise, low-power requirement specifications of the analog front-end usually requires large silicon occupation, making the problem even more challenging. One common approach to alleviating this consumption area burden relies on time-division multiplexing techniques in which read-out electronics are shared, either partially or totally, between channels while preserving the spatial and temporal resolution of the recordings. In this approach, shared elements have to operate over a shorter time slot per channel and active area is thus traded off against larger operating frequencies and signal bandwidths. As a result, power consumption is only mildly affected, although other performance metrics such as in-band noise or crosstalk may be degraded, particularly if the whole read-out circuit is multiplexed at the analog front-end input. In this article, we review the different implementation alternatives reported for time-division multiplexing neural recording systems, analyze their advantages and drawbacks, and suggest strategies for improving performance.

Home Use of a Percutaneous Wireless Intracortical Brain-Computer Interface by Individuals With Tetraplegia

OBJECTIVE

Individuals with neurological disease or injury such as amyotrophic lateral sclerosis, spinal cord injury or stroke may become tetraplegic, unable to speak or even locked-in. For people with these conditions, current assistive technologies are often ineffective. Brain-computer interfaces are being developed to enhance independence and restore communication in the absence of physical movement. Over the past decade, individuals with tetraplegia have achieved rapid on-screen typing and point-and-click control of tablet apps using intracortical brain-computer interfaces (iBCIs) that decode intended arm and hand movements from neural signals recorded by implanted microelectrode arrays. However, cables used to convey neural signals from the brain tether participants to amplifiers and decoding computers and require expert oversight, severely limiting when and where iBCIs could be available for use. Here, we demonstrate the first human use of a wireless broadband iBCI.

METHODS

Based on a prototype system previously used in pre-clinical research, we replaced the external cables of a 192-electrode iBCI with wireless transmitters and achieved high-resolution recording and decoding of broadband field potentials and spiking activity from people with paralysis. Two participants in an ongoing pilot clinical trial completed on-screen item selection tasks to assess iBCI-enabled cursor control.

RESULTS

Communication bitrates were equivalent between cabled and wireless configurations. Participants also used the wireless iBCI to control a standard commercial tablet computer to browse the web and use several mobile applications. Within-day comparison of cabled and wireless interfaces evaluated bit error rate, packet loss, and the recovery of spike rates and spike waveforms from the recorded neural signals. In a representative use case, the wireless system recorded intracortical signals from two arrays in one participant continuously through a 24-hour period at home.

SIGNIFICANCE

Wireless multi-electrode recording of broadband neural signals over extended periods introduces a valuable tool for human neuroscience research and is an important step toward practical deployment of iBCI technology for independent use by individuals with paralysis. On-demand access to high-performance iBCI technology in the home promises to enhance independence and restore communication and mobility for individuals with severe motor impairment.

Multilevel analysis: Integrating multiple levels of neurobehavioral systems

Traditional disciplines have frequently dealt with complex phenomena from a given level of analysis, be that molecular, cellular, organ system, or organismic level. This can yield highly valuable information on biological and psychological processes. There is an explanatory value added, however, by an integrative multilevel approach, in which different levels of analysis and different levels of the neural organization are considered in the models and theories of psychological functions. This is the essence of the emerging discipline of social neuroscience, promoted by John Cacioppo and Gary Berntson, which seeks to inform the interactions between social psychological and biological processes.

Wireless recording from unrestrained monkeys reveals motor goal encoding beyond immediate reach in frontoparietal cortex

System neuroscience of motor cognition regarding the space beyond immediate reach mandates free, yet experimentally controlled movements. We present an experimental environment (Reach Cage) and a versatile visuo-haptic interaction system (MaCaQuE) for investigating goal-directed whole-body movements of unrestrained monkeys. Two rhesus monkeys conducted instructed walk-and-reach movements towards targets flexibly positioned in the cage. We tracked 3D multi-joint arm and head movements using markerless motion capture. Movements show small trial-to-trial variability despite being unrestrained. We wirelessly recorded 192 broad-band neural signals from three cortical sensorimotor areas simultaneously. Single unit activity is selective for different reach and walk-and-reach movements. Walk-and-reach targets could be decoded from premotor and parietal but not motor cortical activity during movement planning. The Reach Cage allows systems-level sensorimotor neuroscience studies with full-body movements in a configurable 3D spatial setting with unrestrained monkeys. We conclude that the primate frontoparietal network encodes reach goals beyond immediate reach during movement planning.

Studying the visual brain in its natural rhythm

How the brain fluidly orchestrates visual behavior is a central question in cognitive neuroscience. Researchers studying neural responses in humans and nonhuman primates have mapped out visual response profiles and cognitive modulation in a large number of brain areas, most often using pared down stimuli and highly controlled behavioral paradigms. The historical emphasis on reductionism has placed most studies at one pole of an inherent trade-off between strictly controlled experimental variables and open designs that monitor the brain during its natural modes of operation. This bias toward simplified experiments has strongly shaped the field of visual neuroscience, with little guarantee that the principles and concepts established within that framework will apply more generally. In recent years, a growing number of studies have begun to relax strict experimental control with the aim of understanding how the brain responds under more naturalistic conditions. In this article, we survey research that has explicitly embraced the complexity and rhythm of natural vision. We focus on those studies most pertinent to understanding high-level visual specializations in brains of humans and nonhuman primates. We conclude that representationalist concepts borne from conventional visual experiments fall short in their ability to capture the real-life visual operations undertaken by the brain. More naturalistic approaches, though fraught with experimental and analytic challenges, provide fertile ground for neuroscientists seeking new inroads to investigate how the brain supports core aspects of our daily visual experience.

Agency and Accountability: Ethical Considerations for Brain-Computer Interfaces

Brain-computer interfaces (BCIs) are systems in which a user's real-time brain activity is used to control an external device, such as a prosthetic limb. BCIs have great potential for restoring lost motor functions in a wide range of patients. However, this futuristic technology raises several ethical questions, especially concerning the degree of agency a BCI affords its user and the extent to which a BCI user ought to be accountable for actions undertaken via the device. This paper examines these and other ethical concerns found at each of the three major parts of the BCI system: the sensor that records neural activity, the decoder that converts raw data into usable signals, and the translator that uses these signals to control the movement of an external device.

Decoder calibration with ultra small current sample set for intracortical brain-machine interface

OBJECTIVE

Intracortical brain-machine interfaces (iBMIs) aim to restore efficient communication and movement ability for paralyzed patients. However, frequent recalibration is required for consistency and reliability, and every recalibration will require relatively large most current sample set. The aim in this study is to develop an effective decoder calibration method that can achieve good performance while minimizing recalibration time.

APPROACH

Two rhesus macaques implanted with intracortical microelectrode arrays were trained separately on movement and sensory paradigm. Neural signals were recorded to decode reaching positions or grasping postures. A novel principal component analysis-based domain adaptation (PDA) method was proposed to recalibrate the decoder with only ultra small current sample set by taking advantage of large historical data, and the decoding performance was compared with other three calibration methods for evaluation.

MAIN RESULTS

The PDA method closed the gap between historical and current data effectively, and made it possible to take advantage of large historical data for decoder recalibration in current data decoding. Using only ultra small current sample set (five trials of each category), the decoder calibrated using the PDA method could achieve much better and more robust performance in all sessions than using other three calibration methods in both monkeys.

SIGNIFICANCE

(1) By this study, transfer learning theory was brought into iBMIs decoder calibration for the first time. (2) Different from most transfer learning studies, the target data in this study were ultra small sample set and were transferred to the source data. (3) By taking advantage of historical data, the PDA method was demonstrated to be effective in reducing recalibration time for both movement paradigm and sensory paradigm, indicating a viable generalization. By reducing the demand for large current training data, this new method may facilitate the application of intracortical brain-machine interfaces in clinical practice.

Decoding Movements from Cortical Ensemble Activity Using a Long Short-Term Memory Recurrent Network

Although many real-time neural decoding algorithms have been proposed for brain-machine interface (BMI) applications over the years, an optimal, consensual approach remains elusive. Recent advances in deep learning algorithms provide new opportunities for improving the design of BMI decoders, including the use of recurrent artificial neural networks to decode neuronal ensemble activity in real time. Here, we developed a long-short term memory (LSTM) decoder for extracting movement kinematics from the activity of large ( N = 134–402) populations of neurons, sampled simultaneously from multiple cortical areas, in rhesus monkeys performing motor tasks. Recorded regions included primary motor, dorsal premotor, supplementary motor, and primary somatosensory cortical areas. The LSTM's capacity to retain information for extended periods of time enabled accurate decoding for tasks that required both movements and periods of immobility. Our LSTM algorithm significantly outperformed the state-of-the-art unscented Kalman filter when applied to three tasks: center-out arm reaching, bimanual reaching, and bipedal walking on a treadmill. Notably, LSTM units exhibited a variety of well-known physiological features of cortical neuronal activity, such as directional tuning and neuronal dynamics across task epochs. LSTM modeled several key physiological attributes of cortical circuits involved in motor tasks. These findings suggest that LSTM-based approaches could yield a better algorithm strategy for neuroprostheses that employ BMIs to restore movement in severely disabled patients.

Decoding of muscle activity from the sensorimotor cortex in freely behaving monkeys

Remarkable advances have recently been made in the development of Brain-Machine Interface (BMI) technologies for restoring or enhancing motor function. However, the application of these technologies may be limited to patients in static conditions, as these developments have been largely based on studies of animals (e.g., non-human primates) in constrained movement conditions. The ultimate goal of BMI technology is to enable individuals to move their bodies naturally or control external devices without physical constraints. Here, we demonstrate accurate decoding of muscle activity from electrocorticogram (ECoG) signals in unrestrained, freely behaving monkeys. We recorded ECoG signals from the sensorimotor cortex as well as electromyogram signals from multiple muscles in the upper arm while monkeys performed two types of movements with no physical restraints, as follows: forced forelimb movement (lever-pull task) and natural whole-body movement (free movement within the cage). As in previous reports using restrained monkeys, we confirmed that muscle activity during forced forelimb movement was accurately predicted from simultaneously recorded ECoG data. More importantly, we demonstrated that accurate prediction of muscle activity from ECoG data was possible in monkeys performing natural whole-body movement. We found that high-gamma activity in the primary motor cortex primarily contributed to the prediction of muscle activity during natural whole-body movement as well as forced forelimb movement. In contrast, the contribution of high-gamma activity in the premotor and primary somatosensory cortices was significantly larger during natural whole-body movement. Thus, activity in a larger area of the sensorimotor cortex was needed to predict muscle activity during natural whole-body movement. Furthermore, decoding models obtained from forced forelimb movement could not be generalized to natural whole-body movement, which suggests that decoders should be built individually and according to different behavior types. These results contribute to the future application of BMI systems in unrestrained individuals.

A novel neural prosthesis providing long-term electrocorticography recording and cortical stimulation for epilepsy and brain-computer interface

OBJECTIVE

Wireless technology is a novel tool for the transmission of cortical signals. Wireless electrocorticography (ECoG) aims to improve the safety and diagnostic gain of procedures requiring invasive localization of seizure foci and also to provide long-term recording of brain activity for brain-computer interfaces (BCIs). However, no wireless devices aimed at these clinical applications are currently available. The authors present the application of a fully implantable and externally rechargeable neural prosthesis providing wireless ECoG recording and direct cortical stimulation (DCS). Prolonged wireless ECoG monitoring was tested in nonhuman primates by using a custom-made device (the ECoG implantable wireless 16-electrode [ECOGIW-16E] device) containing a 16-contact subdural grid. This is a preliminary step toward large-scale, long-term wireless ECoG recording in humans.

METHODS

The authors implanted the ECOGIW-16E device over the left sensorimotor cortex of a nonhuman primate (Macaca fascicularis), recording ECoG signals over a time span of 6 months. Daily electrode impedances were measured, aiming to maintain the impedance values below a threshold of 100 KΩ. Brain mapping was obtained through wireless cortical stimulation at fixed intervals (1, 3, and 6 months). After 6 months, the device was removed. The authors analyzed cortical tissues by using conventional histological and immunohistological investigation to assess whether there was evidence of damage after the long-term implantation of the grid.

RESULTS

The implant was well tolerated; no neurological or behavioral consequences were reported in the monkey, which resumed his normal activities within a few hours of the procedure. The signal quality of wireless ECoG remained excellent over the 6-month observation period. Impedance values remained well below the threshold value; the average impedance per contact remains approximately 40 KΩ. Wireless cortical stimulation induced movements of the upper and lower limbs, and elicited fine movements of the digits as well. After the monkey was euthanized, the grid was found to be encapsulated by a newly formed dural sheet. The grid removal was performed easily, and no direct adhesions of the grid to the cortex were found. Conventional histological studies showed no cortical damage in the brain region covered by the grid, except for a single microscopic spot of cortical necrosis (not visible to the naked eye) in a region that had undergone repeated procedures of electrical stimulation. Immunohistological studies of the cortex underlying the grid showed a mild inflammatory process.

CONCLUSIONS

This preliminary experience in a nonhuman primate shows that a wireless neuroprosthesis, with related long-term ECoG recording (up to 6 months) and multiple DCSs, was tolerated without sequelae. The authors predict that epilepsy surgery could realize great benefit from this novel prosthesis, providing an extended time span for ECoG recording.

Reliability of motor and sensory neural decoding by threshold crossings for intracortical brain-machine interface

OBJECTIVE

For intracortical neurophysiological studies, spike sorting is an important procedure to isolate single units for analyzing specific functions. However, whether spike sorting is necessary or not for neural decoding applications is controversial. Several studies showed that using threshold crossings (TC) instead of spike sorting could also achieve a similar satisfactory performance. However, such studies were limited in similar behavioral tasks, and the neural signal source mainly focused on the motor-related cortical regions. It is not certain if this conclusion is applicable to other situations. Therefore, we compared the performance of TC and spike sorting in neural decoding with more comprehensive paradigms and parameters.

APPROACH

Two rhesus macaques implanted with Utah or floating microelectrode arrays (FMAs) in motor or sensory-related cortical regions were trained to perform a motor or a sensory task. Data from each monkey were preprocessed with three different schemes: TC, automatic sorting (AS), and manual sorting (MS). A support vector machine was used as the decoder, and the decoding accuracy was used for evaluating the performance of three preprocessing methods. Different neural signal sources, different decoders, and related parameters and decoding stability were further tested to systematically compare three preprocessing methods.

MAIN RESULTS

TC could achieve a similar (-4.5 RMS threshold) or better (-3.0 RMS threshold) decoding performance compared to the other two sorting methods in the motor or sensory tasks even if the neural signal sources or decoder-related parameters were changed. Moreover, TC was much more stable in neural decoding across sessions and robust to changes of threshold.

SIGNIFICANCE

Our results indicated that spike-firing patterns could be stably extracted through TC from multiple cortices in both motor and sensory neural decoding applications. Considering the stability of TC, it might be more suitable for neural decoding compared to sorting methods.

Implantable Neural Probes for Brain-Machine Interfaces - Current Developments and Future Prospects

A Brain-Machine interface (BMI) allows for direct communication between the brain and machines. Neural probes for recording neural signals are among the essential components of a BMI system. In this report, we review research regarding implantable neural probes and their applications to BMIs. We first discuss conventional neural probes such as the tetrode, Utah array, Michigan probe, and electroencephalography (ECoG), following which we cover advancements in next-generation neural probes. These next-generation probes are associated with improvements in electrical properties, mechanical durability, biocompatibility, and offer a high degree of freedom in practical settings. Specifically, we focus on three key topics: (1) novel implantable neural probes that decrease the level of invasiveness without sacrificing performance, (2) multi-modal neural probes that measure both electrical and optical signals, (3) and neural probes developed using advanced materials. Because safety and precision are critical for practical applications of BMI systems, future studies should aim to enhance these properties when developing next-generation neural probes.

Spike detection and spike sorting with a hidden Markov model improves offline decoding of motor cortical recordings

OBJECTIVE

Detection and sorting (classification) of action potentials from extracellular recordings are two important pre-processing steps for brain-computer interfaces (BCIs) and some neuroscientific studies. Traditional approaches perform these two steps serially, but using shapes of action potential waveforms during detection, i.e. combining the two steps, may lead to better performance, especially during high noise. We propose a hidden Markov model (HMM) based method for combined detecting and sorting of spikes, with the aim of improving the final decoding accuracy of BCIs.

APPROACH

The states of the HMM indicate whether there is a spike, what unit a spike belongs to, and the time course within a waveform. The HMM outputs probabilities of spike detection, and from this we can calculate expectations of spike counts in time bins, which can replace integer spike counts as input to BCI decoders. We evaluate the HMM method on simulated spiking data. We then examine the impact of using this method on decoding real neural data recorded from primary motor cortex of two Rhesus monkeys.

MAIN RESULTS

Our comparisons on simulated data to detection-then-sorting approaches and combined detection-and-sorting algorithms indicate that the HMM method performs more accurately at detection and sorting (0.93 versus 0.73 spike count correlation, 0.73 versus 0.49 adjusted mutual information). On real neural data, the HMM method led to higher adjusted mutual information between spike counts and kinematics (monkey K: 0.034 versus 0.027; monkey M: 0.033 versus 0.022) and better neuron encoding model predictions (K: 0.016 dB improvement; M: 0.056 dB improvement). Lastly, the HMM method facilitated higher offline decoding accuracy (Kalman filter, K: 8.5% mean squared error reduction, M: 18.6% reduction).

SIGNIFICANCE

The HMM spike detection and sorting method offers a new approach to spike pre-processing for BCIs and neuroscientific studies.

Silicon Valley new focus on brain computer interface: hype or hope for new applications?

In the last year there has been increasing interest and investment into developing devices to interact with the central nervous system, in particular developing a robust brain-computer interface (BCI). In this article, we review the most recent research advances and the current host of engineering and neurological challenges that must be overcome for clinical application. In particular, space limitations, isolation of targeted structures, replacement of probes following failure, delivery of nanomaterials and processing and understanding recorded data. Neural engineering has developed greatly over the past half-century, which has allowed for the development of better neural recording techniques and clinical translation of neural interfaces. Implementation of general purpose BCIs face a number of constraints arising from engineering, computational, ethical and neuroscientific factors that still have to be addressed. Electronics have become orders of magnitude smaller and computationally faster than neurons, however there is much work to be done in decoding the neural circuits. New interest and funding from the non-medical community may be a welcome catalyst for focused research and development; playing an important role in future advancements in the neuroscience community.

Place Cell-Like Activity in the Primary Sensorimotor and Premotor Cortex During Monkey Whole-Body Navigation

Primary motor (M1), primary somatosensory (S1) and dorsal premotor (PMd) cortical areas of rhesus monkeys previously have been associated only with sensorimotor control of limb movements. Here we show that a significant number of neurons in these areas also represent body position and orientation in space. Two rhesus monkeys (K and M) used a wheelchair controlled by a brain-machine interface (BMI) to navigate in a room. During this whole-body navigation, the discharge rates of M1, S1, and PMd neurons correlated with the two-dimensional (2D) room position and the direction of the wheelchair and the monkey head. This place cell-like activity was observed in both monkeys, with 44.6% and 33.3% of neurons encoding room position in monkeys K and M, respectively, and the overlapping populations of 41.0% and 16.0% neurons encoding head direction. These observations suggest that primary sensorimotor and premotor cortical areas in primates are likely involved in allocentrically representing body position in space during whole-body navigation, which is an unexpected finding given the classical hierarchical model of cortical processing that attributes functional specialization for spatial processing to the hippocampal formation.

Analysis of User Interaction with a Brain-Computer Interface Based on Steady-State Visually Evoked Potentials: Case Study of a Game

This paper presents a systematic analysis of a game controlled by a Brain-Computer Interface (BCI) based on Steady-State Visually Evoked Potentials (SSVEP). The objective is to understand BCI systems from the Human-Computer Interface (HCI) point of view, by observing how the users interact with the game and evaluating how the interface elements influence the system performance. The interactions of 30 volunteers with our computer game, named “Get Coins,” through a BCI based on SSVEP, have generated a database of brain signals and the corresponding responses to a questionnaire about various perceptual parameters, such as visual stimulation, acoustic feedback, background music, visual contrast, and visual fatigue. Each one of the volunteers played one match using the keyboard and four matches using the BCI, for comparison. In all matches using the BCI, the volunteers achieved the goals of the game. Eight of them achieved a perfect score in at least one of the four matches, showing the feasibility of the direct communication between the brain and the computer. Despite this successful experiment, adaptations and improvements should be implemented to make this innovative technology accessible to the end user.

Integration of Nanobots Into Neural Circuits As a Future Therapy for Treating Neurodegenerative Disorders

Recent neuroscientific research demonstrates that the human brain is becoming altered by technological devices. Improvements in biotechnologies and computer based technologies are now increasing the likelihood for the development of brain augmentation devices in the next 20 years. We have developed the idea of an “Endomyccorhizae like interface” (ELI) nanocognitive device as a new kind of future neuroprosthetic which aims to facilitate neuronal network properties in individuals with neurodegenerative disorders. The design of our ELI may overcome the problems of invasive neuroprosthetics, post-operative inflammation, and infection and neuroprosthetic degradation. The method in which our ELI is connected and integrated to neuronal networks is based on a mechanism similar to endomyccorhizae which is the oldest and most widespread form of plant symbiosis. We propose that the principle of Endomyccorhizae could be relevant for developing a crossing point between the ELI and neuronal networks. Similar to endomyccorhizae the ELI will be designed to form webs, each of which connects multiple neurons together. The ELI will function to sense action potentials and deliver it to the neurons it connects to. This is expected to compensate for neuronal loss in some neurodegenerative disorders, such as Alzheimer's disease and Parkinson's disease.

Interbrain cortical synchronization encodes multiple aspects of social interactions in monkey pairs

While it is well known that the primate brain evolved to cope with complex social contingencies, the neurophysiological manifestation of social interactions in primates is not well understood. Here, concurrent wireless neuronal ensemble recordings from pairs of monkeys were conducted to measure interbrain cortical synchronization (ICS) during a whole-body navigation task that involved continuous social interaction of two monkeys. One monkey, the passenger, was carried in a robotic wheelchair to a food dispenser, while a second monkey, the observer, remained stationary, watching the passenger. The two monkeys alternated the passenger and the observer roles. Concurrent neuronal ensemble recordings from the monkeys' motor cortex and the premotor dorsal area revealed episodic occurrence of ICS with probability that depended on the wheelchair kinematics, the passenger-observer distance, and the passenger-food distance - the social-interaction factors previously described in behavioral studies. These results suggest that ICS represents specific aspects of primate social interactions.

Organismal Engineering: Towards a Robotic Taxonomic Key for Devices Using Organic Materials

Can we create robots with the behavioral flexibility and robustness of animals? Engineers often use bio-inspiration to mimic animals. Recent advances in tissue engineering now allow the use of components from animals. By integrating organic and synthetic components, researchers are moving towards the development of engineered organisms whose structural framework, actuation, sensing, and control are partially or completely organic. This review discusses recent exciting work demonstrating how organic components can be used for all facets of robot development. Based on this analysis, we propose a Robotic Taxonomic Key to guide the field towards a unified lexicon for device description.

Preventing Neurodegenerative Memory Loss in Hopfield Neuronal Networks Using Cerebral Organoids or External Microelectronics

Developing technologies have made significant progress towards linking the brain with brain-machine interfaces (BMIs) which have the potential to aid damaged brains to perform their original motor and cognitive functions. We consider the viability of such devices for mitigating the deleterious effects of memory loss that is induced by neurodegenerative diseases and/or traumatic brain injury (TBI). Our computational study considers the widely used Hopfield network, an autoassociative memory model in which neurons converge to a stable state pattern after receiving an input resembling the given memory. In this study, we connect an auxiliary network of neurons, which models the BMI device, to the original Hopfield network and train it to converge to its own auxiliary memory patterns. Injuries to the original Hopfield memory network, induced through neurodegeneration, for instance, can then be analyzed with the goal of evaluating the ability of the BMI to aid in memory retrieval tasks. Dense connectivity between the auxiliary and Hopfield networks is shown to promote robustness of memory retrieval tasks for both optimal and nonoptimal memory sets. Our computations estimate damage levels and parameter ranges for which full or partial memory recovery is achievable, providing a starting point for novel therapeutic strategies.

Using insects to drive mobile robots - hybrid robots bridge the gap between biological and artificial systems

The use of mobile robots is an effective method of validating sensory-motor models of animals in a real environment. The well-identified insect sensory-motor systems have been the major targets for modeling. Furthermore, mobile robots implemented with such insect models attract engineers who aim to avail advantages from organisms. However, directly comparing the robots with real insects is still difficult, even if we successfully model the biological systems, because of the physical differences between them. We developed a hybrid robot to bridge the gap. This hybrid robot is an insect-controlled robot, in which a tethered male silkmoth (Bombyx mori) drives the robot in order to localize an odor source. This robot has the following three advantages: 1) from a biomimetic perspective, the robot enables us to evaluate the potential performance of future insect-mimetic robots; 2) from a biological perspective, the robot enables us to manipulate the closed-loop of an onboard insect for further understanding of its sensory-motor system; and 3) the robot enables comparison with insect models as a reference biological system. In this paper, we review the recent works regarding insect-controlled robots and discuss the significance for both engineering and biology.

Recording gaze trajectory of wheelchair users by a spherical camera

Wheelchairs are widely used in the facilities of rehabilitation. In this paper, we propose a method of recording the gaze trajectory of wheelchair users by using a spherical camera mounted on the wheelchairs. A spherical camera has a full field of view and can observe the entire surrounding scenes. First, the gaze point of a user sitting on a wheelchair is estimated from the corneal reflection image observed by a wearable eye camera. Then, the gaze point is mapped onto the full-view image captured by the spherical camera via feature matching. Since it is not guaranteed that the gaze point in an eye image is a distinctive feature point, the matching of a gaze point between these two images cannot be carried out directly. To cope with this problem, we use a coarse-to-fine approach, in which, first, distinctive feature points are used to estimate the relative orientation between the eye camera and the spherical camera, and then, the estimated relative orientation matrix is used to determine the location of gaze points. The effectiveness of the proposed method is shown by real-world experimental results.

A novel paraplegia model in awake behaving macaques

Lower limb paralysis from spinal cord injury (SCI) or neurological disease carries a poor prognosis for recovery and remains a large societal burden. Neurophysiological and neuroprosthetic research have the potential to improve quality of life for these patients; however, the lack of an ethical and sustainable nonhuman primate model for paraplegia hinders their advancement. Therefore, our multidisciplinary team developed a way to induce temporary paralysis in awake behaving macaques by creating a fully implantable lumbar epidural catheter-subcutaneous port system that enables easy and reliable targeted drug delivery for sensorimotor blockade. During treadmill walking, aliquots of 1.5% lidocaine with 1:200,000 epinephrine were percutaneously injected into the ports of three rhesus macaques while surface electromyography (EMG) recorded muscle activity from their quadriceps and gastrocnemii. Diminution of EMG amplitude, loss of voluntary leg movement, and inability to bear weight were achieved for 60-90 min in each animal, followed by a complete recovery of function. The monkeys remained alert and cooperative during the paralysis trials and continued to take food rewards, and the ports remained functional after several months. This technique will enable recording from the cortex and/or spinal cord in awake behaving nonhuman primates during the onset, maintenance, and resolution of paraplegia for the first time, thus opening the door to answering basic neurophysiological questions about the acute neurological response to spinal cord injury and recovery. It will also negate the need to permanently injure otherwise high-value research animals for certain experimental paradigms aimed at developing and testing neural interface decoding algorithms for patients with lower extremity dysfunction.NEW & NOTEWORTHY A novel implantable lumbar epidural catheter-subcutaneous port system enables targeted drug delivery and induction of temporary paraplegia in awake, behaving nonhuman primates. Three macaques displayed loss of voluntary leg movement for 60-90 min after injection of lidocaine with epinephrine, followed by a full recovery. This technique for the first time will enable ethical live recording from the proximal central nervous system during the acute onset, maintenance, and resolution of paraplegia.

Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation

Brain-machine interfaces (BMIs) combine methods, approaches, and concepts derived from neurophysiology, computer science, and engineering in an effort to establish real-time bidirectional links between living brains and artificial actuators. Although theoretical propositions and some proof of concept experiments on directly linking the brains with machines date back to the early 1960s, BMI research only took off in earnest at the end of the 1990s, when this approach became intimately linked to new neurophysiological methods for sampling large-scale brain activity. The classic goals of BMIs are 1) to unveil and utilize principles of operation and plastic properties of the distributed and dynamic circuits of the brain and 2) to create new therapies to restore mobility and sensations to severely disabled patients. Over the past decade, a wide range of BMI applications have emerged, which considerably expanded these original goals. BMI studies have shown neural control over the movements of robotic and virtual actuators that enact both upper and lower limb functions. Furthermore, BMIs have also incorporated ways to deliver sensory feedback, generated from external actuators, back to the brain. BMI research has been at the forefront of many neurophysiological discoveries, including the demonstration that, through continuous use, artificial tools can be assimilated by the primate brain's body schema. Work on BMIs has also led to the introduction of novel neurorehabilitation strategies. As a result of these efforts, long-term continuous BMI use has been recently implicated with the induction of partial neurological recovery in spinal cord injury patients.

Neurobionics and the brain-computer interface: current applications and future horizons

The brain-computer interface (BCI) is an exciting advance in neuroscience and engineering. In a motor BCI, electrical recordings from the motor cortex of paralysed humans are decoded by a computer and used to drive robotic arms or to restore movement in a paralysed hand by stimulating the muscles in the forearm. Simultaneously integrating a BCI with the sensory cortex will further enhance dexterity and fine control. BCIs are also being developed to: provide ambulation for paraplegic patients through controlling robotic exoskeletons; restore vision in people with acquired blindness; detect and control epileptic seizures; and improve control of movement disorders and memory enhancement. High-fidelity connectivity with small groups of neurons requires microelectrode placement in the cerebral cortex. Electrodes placed on the cortical surface are less invasive but produce inferior fidelity. Scalp surface recording using electroencephalography is much less precise. BCI technology is still in an early phase of development and awaits further technical improvements and larger multicentre clinical trials before wider clinical application and impact on the care of people with disabilities. There are also many ethical challenges to explore as this technology evolves.

System-Level Design of a 64-Channel Low Power Neural Spike Recording Sensor

This paper reports an integrated 64-channel neural spike recording sensor, together with all the circuitry to process and configure the channels, process the neural data, transmit via a wireless link the information and receive the required instructions. Neural signals are acquired, filtered, digitized and compressed in the channels. Additionally, each channel implements an auto-calibration algorithm which individually configures the transfer characteristics of the recording site. The system has two transmission modes; in one case the information captured by the channels is sent as uncompressed raw data; in the other, feature vectors extracted from the detected neural spikes are released. Data streams coming from the channels are serialized by the embedded digital processor. Experimental results, including in vivo measurements, show that the power consumption of the complete system is lower than 330 μW.

Enhancing Nervous System Recovery through Neurobiologics, Neural Interface Training, and Neurorehabilitation

After an initial period of recovery, human neurological injury has long been thought to be static. In order to improve quality of life for those suffering from stroke, spinal cord injury, or traumatic brain injury, researchers have been working to restore the nervous system and reduce neurological deficits through a number of mechanisms. For example, neurobiologists have been identifying and manipulating components of the intra- and extracellular milieu to alter the regenerative potential of neurons, neuro-engineers have been producing brain-machine and neural interfaces that circumvent lesions to restore functionality, and neurorehabilitation experts have been developing new ways to revitalize the nervous system even in chronic disease. While each of these areas holds promise, their individual paths to clinical relevance remain difficult. Nonetheless, these methods are now able to synergistically enhance recovery of native motor function to levels which were previously believed to be impossible. Furthermore, such recovery can even persist after training, and for the first time there is evidence of functional axonal regrowth and rewiring in the central nervous system of animal models. To attain this type of regeneration, rehabilitation paradigms that pair cortically-based intent with activation of affected circuits and positive neurofeedback appear to be required-a phenomenon which raises new and far reaching questions about the underlying relationship between conscious action and neural repair. For this reason, we argue that multi-modal therapy will be necessary to facilitate a truly robust recovery, and that the success of investigational microscopic techniques may depend on their integration into macroscopic frameworks that include task-based neurorehabilitation. We further identify critical components of future neural repair strategies and explore the most updated knowledge, progress, and challenges in the fields of cellular neuronal repair, neural interfacing, and neurorehabilitation, all with the goal of better understanding neurological injury and how to improve recovery.

Neural Probes for Chronic Applications

Developed over approximately half a century, neural probe technology is now a mature technology in terms of its fabrication technology and serves as a practical alternative to the traditional microwires for extracellular recording. Through extensive exploration of fabrication methods, structural shapes, materials, and stimulation functionalities, neural probes are now denser, more functional and reliable. Thus, applications of neural probes are not limited to extracellular recording, brain-machine interface, and deep brain stimulation, but also include a wide range of new applications such as brain mapping, restoration of neuronal functions, and investigation of brain disorders. However, the biggest limitation of the current neural probe technology is chronic reliability; neural probes that record with high fidelity in acute settings often fail to function reliably in chronic settings. While chronic viability is imperative for both clinical uses and animal experiments, achieving one is a major technological challenge due to the chronic foreign body response to the implant. Thus, this review aims to outline the factors that potentially affect chronic recording in chronological order of implantation, summarize the methods proposed to minimize each factor, and provide a performance comparison of the neural probes developed for chronic applications.