Brain-Machine Interfaces beyond Neuroprosthetics

Bioelectronics for electrical stimulation: materials, devices and biomedical applications

Bioelectronics is a hot research topic, yet an important tool, as it facilitates the creation of advanced medical devices that interact with biological systems to effectively diagnose, monitor and treat a broad spectrum of health conditions. Electrical stimulation (ES) is a pivotal technique in bioelectronics, offering a precise, non-pharmacological means to modulate and control biological processes across molecular, cellular, tissue, and organ levels. This method holds the potential to restore or enhance physiological functions compromised by diseases or injuries by integrating sophisticated electrical signals, device interfaces, and designs tailored to specific biological mechanisms. This review explains the mechanisms by which ES influences cellular behaviors, introduces the essential stimulation principles, discusses the performance requirements for optimal ES systems, and highlights the representative applications. From this review, we can realize the potential of ES based bioelectronics in therapy, regenerative medicine and rehabilitation engineering technologies, ranging from tissue engineering to neurological technologies, and the modulation of cardiovascular and cognitive functions. This review underscores the versatility of ES in various biomedical contexts and emphasizes the need to adapt to complex biological and clinical landscapes it addresses.

Event detection and classification from multimodal time series with application to neural data

The detection of events in time-series data is a common signal-processing problem. When the data can be modeled as a known template signal with an unknown delay in Gaussian noise, detection of the template signal can be done with a traditional matched filter. However, in many applications, the event of interest is represented in multimodal data consisting of both Gaussian and point-process time series. Neuroscience experiments, for example, can simultaneously record multimodal neural signals such as local field potentials (LFPs), which can be modeled as Gaussian, and neuronal spikes, which can be modeled as point processes. Currently, no method exists for event detection from such multimodal data, and as such our objective in this work is to develop a method to meet this need. Here we address this challenge by developing the multimodal event detector (MED) algorithm which simultaneously estimates event times and classes. To do this, we write a multimodal likelihood function for Gaussian and point-process observations and derive the associated maximum likelihood estimator of simultaneous event times and classes. We additionally introduce a cross-modal scaling parameter to account for model mismatch in real datasets. We validate this method in extensive simulations as well as in a neural spike-LFP dataset recorded during an eye-movement task, where the events of interest are eye movements with unknown times and directions. We show that the MED can successfully detect eye movement onset and classify eye movement direction. Further, the MED successfully combines information across data modalities, with multimodal performance exceeding unimodal performance. This method can facilitate applications such as the discovery of latent events in multimodal neural population activity and the development of brain-computer interfaces for naturalistic settings without constrained tasks or prior knowledge of event times.

Spike sorting in the presence of stimulation artifacts: a dynamical control systems approach

Objective. Bi-directional electronic neural interfaces, capable of both electrical recording and stimulation, communicate with the nervous system to permit precise calibration of electrical inputs by capturing the evoked neural responses. However, one significant challenge is that stimulation artifacts often mask the actual neural signals. To address this issue, we introduce a novel approach that employs dynamical control systems to detect and decipher electrically evoked neural activity despite the presence of electrical artifacts.Approach. Our proposed method leverages the unique spatiotemporal patterns of neural activity and electrical artifacts to distinguish and identify individual neural spikes. We designed distinctive dynamical models for both the stimulation artifact and each neuron observed during spontaneous neural activity. We can estimate which neurons were active by analyzing the recorded voltage responses across multiple electrodes post-stimulation. This technique also allows us to exclude signals from electrodes heavily affected by stimulation artifacts, such as the stimulating electrode itself, yet still accurately differentiate between evoked spikes and electrical artifacts.Main results. We applied our method to high-density multi-electrode recordings from the primate retina in anex vivosetup, using a grid of 512 electrodes. Through repeated electrical stimulations at varying amplitudes, we were able to construct activation curves for each neuron. The curves obtained with our method closely resembled those derived from manual spike sorting. Additionally, the stimulation thresholds we estimated strongly agreed with those determined through manual analysis, demonstrating high reliability (R2=0.951for human 1 andR2=0.944for human 2).Significance. Our method can effectively separate evoked neural spikes from stimulation artifacts by exploiting the distinct spatiotemporal propagation patterns captured by a dense, large-scale multi-electrode array. This technique holds promise for future applications in real-time closed-loop stimulation systems and for managing multi-channel stimulation strategies.

Artificial Neuron Devices

Efforts to design devices emulating complex cognitive abilities and response processes of biological systems have long been a coveted goal. Recent advancements in flexible electronics, mirroring human tissue's mechanical properties, hold significant promise. Artificial neuron devices, hinging on flexible artificial synapses, bioinspired sensors, and actuators, are meticulously engineered to mimic the biological systems. However, this field is in its infancy, requiring substantial groundwork to achieve autonomous systems with intelligent feedback, adaptability, and tangible problem-solving capabilities. This review provides a comprehensive overview of recent advancements in artificial neuron devices. It starts with fundamental principles of artificial synaptic devices and explores artificial sensory systems, integrating artificial synapses and bioinspired sensors to replicate all five human senses. A systematic presentation of artificial nervous systems follows, designed to emulate fundamental human nervous system functions. The review also discusses potential applications and outlines existing challenges, offering insights into future prospects. We aim for this review to illuminate the burgeoning field of artificial neuron devices, inspiring further innovation in this captivating area of research.

The neural basis of mental navigation in rats

A brain-machine interface demonstrates volitional control of hippocampal activity.

Invariant neural dynamics drive commands to control different movements

It has been proposed that the nervous system has the capacity to generate a wide variety of movements because it reuses some invariant code. Previous work has identified that dynamics of neural population activity are similar during different movements, where dynamics refer to how the instantaneous spatial pattern of population activity changes in time. Here, we test whether invariant dynamics of neural populations are actually used to issue the commands that direct movement. Using a brain-machine interface (BMI) that transforms rhesus macaques' motor-cortex activity into commands for a neuroprosthetic cursor, we discovered that the same command is issued with different neural-activity patterns in different movements. However, these different patterns were predictable, as we found that the transitions between activity patterns are governed by the same dynamics across movements. These invariant dynamics are low dimensional, and critically, they align with the BMI, so that they predict the specific component of neural activity that actually issues the next command. We introduce a model of optimal feedback control (OFC) that shows that invariant dynamics can help transform movement feedback into commands, reducing the input that the neural population needs to control movement. Altogether our results demonstrate that invariant dynamics drive commands to control a variety of movements and show how feedback can be integrated with invariant dynamics to issue generalizable commands.

Microenvironments Matter: Advances in Brain-on-Chip

To highlight the particular needs with respect to modeling the unique and complex organization of the human brain structure, we reviewed the state-of-the-art in devising brain models with engineered instructive microenvironments. To acquire a better perspective on the brain's working mechanisms, we first summarize the importance of regional stiffness gradients in brain tissue, varying per layer and the cellular diversities of the layers. Through this, one can acquire an understanding of the essential parameters in emulating the brain in vitro. In addition to the brain's organizational architecture, we addressed also how the mechanical properties have an impact on neuronal cell responses. In this respect, advanced in vitro platforms emerged and profoundly changed the methods of brain modeling efforts from the past, mainly focusing on animal or cell line research. The main challenges in imitating features of the brain in a dish are with regard to composition and functionality. In neurobiological research, there are now methods that aim to cope with such challenges by the self-assembly of human-derived pluripotent stem cells (hPSCs), i.e., brainoids. Alternatively, these brainoids can be used stand-alone or in conjunction with Brain-on-Chip (BoC) platform technology, 3D-printed gels, and other types of engineered guidance features. Currently, advanced in vitro methods have made a giant leap forward regarding cost-effectiveness, ease-of-use, and availability. We bring these recent developments together into one review. We believe our conclusions will give a novel perspective towards advancing instructive microenvironments for BoCs and the understanding of the brain's cellular functions either in modeling healthy or diseased states of the brain.

Tracking momentary fluctuations in human attention with a cognitive brain-machine interface

Selective attention produces systematic effects on neural states. It is unclear whether, conversely, momentary fluctuations in neural states have behavioral significance for attention. We investigated this question in the human brain with a cognitive brain-machine interface (cBMI) for tracking electrophysiological steady-state visually evoked potentials (SSVEPs) in real-time. Discrimination accuracy (d') was significantly higher when target stimuli were triggered at high, versus low, SSVEP power states. Target and distractor SSVEP power was uncorrelated across the hemifields, and target d' was unaffected by distractor SSVEP power states. Next, we trained participants on an auditory neurofeedback paradigm to generate biased, cross-hemispheric competitive interactions between target and distractor SSVEPs. The strongest behavioral effects emerged when competitive SSVEP dynamics unfolded at a timescale corresponding to the deployment of endogenous attention. In sum, SSVEP power dynamics provide a reliable readout of attentional state, a result with critical implications for tracking and training human attention.

Optimising the classification of feature-based attention in frequency-tagged electroencephalography data

Brain-computer interfaces (BCIs) are a rapidly expanding field of study and require accurate and reliable real-time decoding of patterns of neural activity. These protocols often exploit selective attention, a neural mechanism that prioritises the sensory processing of task-relevant stimulus features (feature-based attention) or task-relevant spatial locations (spatial attention). Within the visual modality, attentional modulation of neural responses to different inputs is well indexed by steady-state visual evoked potentials (SSVEPs). These signals are reliably present in single-trial electroencephalography (EEG) data, are largely resilient to common EEG artifacts, and allow separation of neural responses to numerous concurrently presented visual stimuli. To date, efforts to use single-trial SSVEPs to classify visual attention for BCI control have largely focused on spatial attention rather than feature-based attention. Here, we present a dataset that allows for the development and benchmarking of algorithms to classify feature-based attention using single-trial EEG data. The dataset includes EEG and behavioural responses from 30 healthy human participants who performed a feature-based motion discrimination task on frequency tagged visual stimuli.

Far-field electric potentials provide access to the output from the spinal cord from wrist-mounted sensors

OBJECTIVE

Neural interfaces need to become more unobtrusive and socially acceptable to appeal to general consumers outside rehabilitation settings.

APPROACH

We developed a non-invasive neural interface that provides access to spinal motor neuron activities from the wrist, which is the preferred location for a wearable. The interface decodes far-field potentials present at the tendon endings of the forearm muscles using blind source separation. First, we evaluated the reliability of the interface to detect motor neuron firings based on far-field potentials, and thereafter we used the decoded motor neuron activity for the prediction of finger contractions in offline and real-time conditions.

MAIN RESULTS

The results showed that motor neuron activity decoded from the far-field potentials at the wrist accurately predicted individual and combined finger commands and therefore allowed for highly accurate real-time task classification.

SIGNIFICANCE

These findings demonstrate the feasibility of a non-invasive, neural interface at the wrist for precise real-time control based on the output of the spinal cord.

Modeling multiscale causal interactions between spiking and field potential signals during behavior

Objective. Brain recordings exhibit dynamics at multiple spatiotemporal scales, which are measured with spike trains and larger-scale field potential signals. To study neural processes, it is important to identify and model causal interactions not only at a single scale of activity, but also across multiple scales, i.e. between spike trains and field potential signals. Standard causality measures are not directly applicable here because spike trains are binary-valued but field potentials are continuous-valued. It is thus important to develop computational tools to recover multiscale neural causality during behavior, assess their performance on neural datasets, and study whether modeling multiscale causalities can improve the prediction of neural signals beyond what is possible with single-scale causality. Approach. We design a multiscale model-based Granger-like causality method based on directed information and evaluate its success both in realistic biophysical spike-field simulations and in motor cortical datasets from two non-human primates (NHP) performing a motor behavior. To compute multiscale causality, we learn point-process generalized linear models that predict the spike events at a given time based on the history of both spike trains and field potential signals. We also learn linear Gaussian models that predict the field potential signals at a given time based on their own history as well as either the history of binary spike events or that of latent firing rates. Main results. We find that our method reveals the true multiscale causality network structure in biophysical simulations despite the presence of model mismatch. Further, models with the identified multiscale causalities in the NHP neural datasets lead to better prediction of both spike trains and field potential signals compared to just modeling single-scale causalities. Finally, we find that latent firing rates are better predictors of field potential signals compared with the binary spike events in the NHP datasets. Significance. This multiscale causality method can reveal the directed functional interactions across spatiotemporal scales of brain activity to inform basic science investigations and neurotechnologies.

Closed-loop Modulation of the Self-regulating Brain: A Review on Approaches, Emerging Paradigms, and Experimental Designs

Closed-loop approaches, setups, and experimental designs have been applied within the field of neuroscience to enhance the understanding of basic neurophysiology principles (closed-loop neuroscience; CLNS) and to develop improved procedures for modulating brain circuits and networks for clinical purposes (closed-loop neuromodulation; CLNM). The contents of this review are thus arranged into the following sections. First, we describe basic research findings that have been made using CLNS. Next, we provide an overview of the application, rationale, and therapeutic aspects of CLNM for clinical purposes. Finally, we summarize methodological concerns and critics in clinical practice of neurofeedback and novel applications of closed-loop perspective and techniques to improve and optimize its experiments. Moreover, we outline the theoretical explanations and experimental ideas to test animal models of neurofeedback and discuss technical issues and challenges associated with implementing closed-loop systems. We hope this review is helpful for both basic neuroscientists and clinical/ translationally-oriented scientists interested in applying closed-loop methods to improve mental health and well-being.

Graphene-Based Electrode Materials for Neural Activity Detection

The neural electrode technique is a powerful tool for monitoring and regulating neural activity, which has a wide range of applications in basic neuroscience and the treatment of neurological diseases. Constructing a high-performance electrode-nerve interface is required for the long-term stable detection of neural signals by electrodes. However, conventional neural electrodes are mainly fabricated from rigid materials that do not match the mechanical properties of soft neural tissues, thus limiting the high-quality recording of neuroelectric signals. Meanwhile, graphene-based nanomaterials can form stable electrode-nerve interfaces due to their high conductivity, excellent flexibility, and biocompatibility. In this literature review, we describe various graphene-based electrodes and their potential application in neural activity detection. We also discuss the biological safety of graphene neural electrodes, related challenges, and their prospects.

The Influence of Frequency Bands and Brain Region on ECoG-Based BMI Learning Performance

Numerous brain–machine interface (BMI) studies have shown that various frequency bands (alpha, beta, and gamma bands) can be utilized in BMI experiments and modulated as neural information for machine control after several BMI learning trial sessions. In addition to frequency range as a neural feature, various areas of the brain, such as the motor cortex or parietal cortex, have been selected as BMI target brain regions. However, although the selection of target frequency and brain region appears to be crucial in obtaining optimal BMI performance, the direct comparison of BMI learning performance as it relates to various brain regions and frequency bands has not been examined in detail. In this study, ECoG-based BMI learning performances were compared using alpha, beta, and gamma bands, respectively, in a single rodent model. Brain area dependence of learning performance was also evaluated in the frontal cortex, the motor cortex, and the parietal cortex. The findings indicated that BMI learning performance was best in the case of the gamma frequency band and worst in the alpha band (one-way ANOVA, F = 4.41, p < 0.05). In brain area dependence experiments, better BMI learning performance appears to be shown in the primary motor cortex (one-way ANOVA, F = 4.36, p < 0.05). In the frontal cortex, two out of four animals failed to learn the feeding tube control even after a maximum of 10 sessions. In conclusion, the findings reported in this study suggest that the selection of target frequency and brain region should be carefully considered when planning BMI protocols and for performing optimized BMI.

A neuromimetic realization of hippocampal CA1 for theta wave generation

Recent advances in neural engineering allowed the development of neuroprostheses which facilitate functionality in people with neurological problems. In this research, a real-time neuromorphic system is proposed to artificially reproduce the theta wave and firing patterns of different neuronal populations in the CA1, a sub-region of the hippocampus. The hippocampal theta oscillations (4-12 Hz) are an important electrophysiological rhythm that contributes in various cognitive functions, including navigation, memory, and novelty detection. The proposed CA1 neuromimetic circuit includes 100 linearized Pinsky-Rinzel neurons and 668 excitatory and inhibitory synapses on a field programmable gate array (FPGA). The implemented spiking neural network of the CA1 includes the main neuronal populations for the theta rhythm generation: excitatory pyramidal cells, PV+ basket cells, and Oriens Lacunosum-Moleculare (OLM) cells which are inhibitory interneurons. Moreover, the main inputs to the CA1 region from the entorhinal cortex via the perforant pathway, the CA3 via Schaffer collaterals, and the medial septum via fimbria-fornix are also implemented on the FPGA using a bursting leaky-integrate and fire (LIF) neuron model. The results of hardware realization show that the proposed CA1 neuromimetic circuit successfully reconstructs the theta oscillations and functionally illustrates the phase relations between firing responses of the different neuronal populations. It is also evaluated the impact of medial septum elimination on the firing patterns of the CA1 neuronal population and the theta wave's characteristics. This neuromorphic system can be considered as a potential platform that opens opportunities for neuroprosthetic applications in future works.

Nonlinear Decoding of Natural Images From Large-Scale Primate Retinal Ganglion Recordings

Decoding sensory stimuli from neural activity can provide insight into how the nervous system might interpret the physical environment, and facilitates the development of brain-machine interfaces. Nevertheless, the neural decoding problem remains a significant open challenge. Here, we present an efficient nonlinear decoding approach for inferring natural scene stimuli from the spiking activities of retinal ganglion cells (RGCs). Our approach uses neural networks to improve on existing decoders in both accuracy and scalability. Trained and validated on real retinal spike data from more than 1000 simultaneously recorded macaque RGC units, the decoder demonstrates the necessity of nonlinear computations for accurate decoding of the fine structures of visual stimuli. Specifically, high-pass spatial features of natural images can only be decoded using nonlinear techniques, while low-pass features can be extracted equally well by linear and nonlinear methods. Together, these results advance the state of the art in decoding natural stimuli from large populations of neurons.

Learning is shaped by abrupt changes in neural engagement

Internal states such as arousal, attention and motivation modulate brain-wide neural activity, but how these processes interact with learning is not well understood. During learning, the brain modifies its neural activity to improve behavior. How do internal states affect this process? Using a brain-computer interface learning paradigm in monkeys, we identified large, abrupt fluctuations in neural population activity in motor cortex indicative of arousal-like internal state changes, which we term 'neural engagement.' In a brain-computer interface, the causal relationship between neural activity and behavior is known, allowing us to understand how neural engagement impacted behavioral performance for different task goals. We observed stereotyped changes in neural engagement that occurred regardless of how they impacted performance. This allowed us to predict how quickly different task goals were learned. These results suggest that changes in internal states, even those seemingly unrelated to goal-seeking behavior, can systematically influence how behavior improves with learning.

Recent progress in translational engineered in vitro models of the central nervous system

The complexity of the human brain poses a substantial challenge for the development of models of the CNS. Current animal models lack many essential human characteristics (in addition to raising operational challenges and ethical concerns), and conventional in vitro models, in turn, are limited in their capacity to provide information regarding many functional and systemic responses. Indeed, these challenges may underlie the notoriously low success rates of CNS drug development efforts. During the past 5 years, there has been a leap in the complexity and functionality of in vitro systems of the CNS, which have the potential to overcome many of the limitations of traditional model systems. The availability of human-derived induced pluripotent stem cell technology has further increased the translational potential of these systems. Yet, the adoption of state-of-the-art in vitro platforms within the CNS research community is limited. This may be attributable to the high costs or the immaturity of the systems. Nevertheless, the costs of fabrication have decreased, and there are tremendous ongoing efforts to improve the quality of cell differentiation. Herein, we aim to raise awareness of the capabilities and accessibility of advanced in vitro CNS technologies. We provide an overview of some of the main recent developments (since 2015) in in vitro CNS models. In particular, we focus on engineered in vitro models based on cell culture systems combined with microfluidic platforms (e.g. 'organ-on-a-chip' systems). We delve into the fundamental principles underlying these systems and review several applications of these platforms for the study of the CNS in health and disease. Our discussion further addresses the challenges that hinder the implementation of advanced in vitro platforms in personalized medicine or in large-scale industrial settings, and outlines the existing differentiation protocols and industrial cell sources. We conclude by providing practical guidelines for laboratories that are considering adopting organ-on-a-chip technologies.

Sensory substitution reveals a manipulation bias

Sensory substitution is a promising therapeutic approach for replacing a missing or diseased sensory organ by translating inaccessible information into another sensory modality. However, many substitution systems are not well accepted by subjects. To explore the effect of sensory substitution on voluntary action repertoires and their associated affective valence, we study deaf songbirds to which we provide visual feedback as a substitute of auditory feedback. Surprisingly, deaf birds respond appetitively to song-contingent binary visual stimuli. They skillfully adapt their songs to increase the rate of visual stimuli, showing that auditory feedback is not required for making targeted changes to vocal repertoires. We find that visually instructed song learning is basal-ganglia dependent. Because hearing birds respond aversively to the same visual stimuli, sensory substitution reveals a preference for actions that elicit sensory feedback over actions that do not, suggesting that substitution systems should be designed to exploit the drive to manipulate.

Adenosine A2A receptor blockade improves neuroprosthetic learning by volitional control of population calcium signal in M1 cortical neurons

Volitional control is at the core of brain-machine interfaces (BMI) adaptation and neuroprosthetic-driven learning to restore motor function for disabled patients, but neuroplasticity changes and neuromodulation underlying volitional control of neuroprosthetic learning are largely unexplored. To better study volitional control at annotated neural population, we have developed an operant neuroprosthetic task with closed-loop feedback system by volitional conditioning of population calcium signal in the M1 cortex using fiber photometry recording. Importantly, volitional conditioning of the population calcium signal in M1 neurons did not improve within-session adaptation, but specifically enhanced across-session neuroprosthetic skill learning with reduced time-to-target and the time to complete 50 successful trials. With brain-behavior causality of the neuroprosthetic paradigm, we revealed that proficiency of neuroprosthetic learning by volitional conditioning of calcium signal was associated with the stable representational (plasticity) mapping in M1 neurons with the reduced calcium peak. Furthermore, pharmacological blockade of adenosine A_2A receptors facilitated volitional conditioning of neuroprosthetic learning and converted an ineffective volitional conditioning protocol to be the effective for neuroprosthetic learning. These findings may help to harness neuroplasticity for better volitional control of neuroprosthetic training and suggest a novel pharmacological strategy to improve neuroprosthetic learning in BMI adaptation by targeting striatal A_2A receptors.

Neuronal Activity Distributed in Multiple Cortical Areas during Voluntary Control of the Native Arm or a Brain-Computer Interface

Voluntary control of visually-guided upper extremity movements involves neuronal activity in multiple areas of the cerebral cortex. Studies of brain-computer interfaces (BCIs) that use spike recordings for input, however, have focused largely on activity in the region from which those neurons that directly control the BCI, which we call BCI units, are recorded. We hypothesized that just as voluntary control of the arm and hand involves activity in multiple cortical areas, so does voluntary control of a BCI. In two subjects (Macaca mulatta) performing a center-out task both with a hand-held joystick and with a BCI directly controlled by four primary motor cortex (M1) BCI units, we recorded the activity of other, non-BCI units in M1, dorsal premotor cortex (PMd) and ventral premotor cortex (PMv), primary somatosensory cortex (S1), dorsal posterior parietal cortex (dPPC), and the anterior intraparietal area (AIP). In most of these areas, non-BCI units were active in similar percentages and at similar modulation depths during both joystick and BCI trials. Both BCI and non-BCI units showed changes in preferred direction (PD). Additionally, the prevalence of effective connectivity between BCI and non-BCI units was similar during both tasks. The subject with better BCI performance showed increased percentages of modulated non-BCI units with increased modulation depth and increased effective connectivity during BCI as compared with joystick trials; such increases were not found in the subject with poorer BCI performance. During voluntary, closed-loop control, non-BCI units in a given cortical area may function similarly whether the effector is the native upper extremity or a BCI-controlled device.

Learning in brain-computer interface control evidenced by joint decomposition of brain and behavior

OBJECTIVE

Motor imagery-based brain-computer interfaces (BCIs) use an individual's ability to volitionally modulate localized brain activity, often as a therapy for motor dysfunction or to probe causal relations between brain activity and behavior. However, many individuals cannot learn to successfully modulate their brain activity, greatly limiting the efficacy of BCI for therapy and for basic scientific inquiry. Formal experiments designed to probe the nature of BCI learning have offered initial evidence that coherent activity across spatially distributed and functionally diverse cognitive systems is a hallmark of individuals who can successfully learn to control the BCI. However, little is known about how these distributed networks interact through time to support learning.

APPROACH

Here, we address this gap in knowledge by constructing and applying a multimodal network approach to decipher brain-behavior relations in motor imagery-based brain-computer interface learning using magnetoencephalography. Specifically, we employ a minimally constrained matrix decomposition method - non-negative matrix factorization - to simultaneously identify regularized, covarying subgraphs of functional connectivity, to assess their similarity to task performance, and to detect their time-varying expression.

MAIN RESULTS

We find that learning is marked by diffuse brain-behavior relations: good learners displayed many subgraphs whose temporal expression tracked performance. Individuals also displayed marked variation in the spatial properties of subgraphs such as the connectivity between the frontal lobe and the rest of the brain, and in the temporal properties of subgraphs such as the stage of learning at which they reached maximum expression. From these observations, we posit a conceptual model in which certain subgraphs support learning by modulating brain activity in sensors near regions important for sustaining attention. To test this model, we use tools that stipulate regional dynamics on a networked system (network control theory), and find that good learners display a single subgraph whose temporal expression tracked performance and whose architecture supports easy modulation of sensors located near brain regions important for attention.

SIGNIFICANCE

The nature of our contribution to the neuroscience of BCI learning is therefore both computational and theoretical; we first use a minimally-constrained, individual specific method of identifying mesoscale structure in dynamic brain activity to show how global connectivity and interactions between distributed networks supports BCI learning, and then we use a formal network model of control to lend theoretical support to the hypothesis that these identified subgraphs are well suited to modulate attention.

The combination of brain-computer interfaces and artificial intelligence: applications and challenges

Brain-computer interfaces (BCIs) have shown great prospects as real-time bidirectional links between living brains and actuators. Artificial intelligence (AI), which can advance the analysis and decoding of neural activity, has turbocharged the field of BCIs. Over the past decade, a wide range of BCI applications with AI assistance have emerged. These "smart" BCIs including motor and sensory BCIs have shown notable clinical success, improved the quality of paralyzed patients' lives, expanded the athletic ability of common people and accelerated the evolution of robots and neurophysiological discoveries. However, despite technological improvements, challenges remain with regard to the long training periods, real-time feedback, and monitoring of BCIs. In this article, the authors review the current state of AI as applied to BCIs and describe advances in BCI applications, their challenges and where they could be headed in the future.

Bacomics: a comprehensive cross area originating in the studies of various brain-apparatus conversations

The brain is the most important organ of the human body, and the conversations between the brain and an apparatus can not only reveal a normally functioning or a dysfunctional brain but also can modulate the brain. Here, the apparatus may be a nonbiological instrument, such as a computer, and the consequent brain-computer interface is now a very popular research area with various applications. The apparatus may also be a biological organ or system, such as the gut and muscle, and their efficient conversations with the brain are vital for a healthy life. Are there any common bases that bind these different scenarios? Here, we propose a new comprehensive cross area: Bacomics, which comes from brain-apparatus conversations (BAC) + omics. We take Bacomics to cover at least three situations: (1) The brain is normal, but the conversation channel is disabled, as in amyotrophic lateral sclerosis. The task is to reconstruct or open up new channels to reactivate the brain function. (2) The brain is in disorder, such as in Parkinson's disease, and the work is to utilize existing or open up new channels to intervene, repair and modulate the brain by medications or stimulation. (3) Both the brain and channels are in order, and the goal is to enhance coordinated development between the brain and apparatus. In this paper, we elaborate the connotation of BAC into three aspects according to the information flow: the issue of output to the outside (BAC-1), the issue of input to the brain (BAC-2) and the issue of unity of brain and apparatus (BAC-3). More importantly, there are no less than five principles that may be taken as the cornerstones of Bacomics, such as feedforward and feedback control, brain plasticity, harmony, the unity of opposites and systems principles. Clearly, Bacomics integrates these seemingly disparate domains, but more importantly, opens a much wider door for the research and development of the brain, and the principles further provide the general framework in which to realize or optimize these various conversations.

Extracellular voltage thresholds for maximizing information extraction in primate auditory cortex: implications for a brain computer interface

OBJECTIVE

Research by Oby et al (2016) demonstrated that the optimal threshold for extracting information from visual and motor cortices may differ from the optimal threshold for identifying single neurons via spike sorting methods. The optimal threshold for extracting information from auditory cortex has yet to be identified, nor has the optimal temporal scale for representing auditory cortical activity. Here, we describe a procedure to jointly optimize the extracellular threshold and bin size with respect to the decoding accuracy achieved by a linear classifier for a diverse set of auditory stimuli.

APPROACH

We used linear multichannel arrays to record extracellular neural activity from the auditory cortex of awake squirrel monkeys passively listening to both simple and complex sounds. We executed a grid search of the coordinate space defined by the voltage threshold (in units of standard deviation) and the bin size (in units of milliseconds), and computed decoding accuracy at each point.

MAIN RESULTS

The optimal threshold for information extraction was consistently near two standard deviations below the voltage trace mean, which falls significantly below the range of three to five standard deviations typically used as inputs to spike sorting algorithms in basic research and in brain-computer interface (BCI) applications. The optimal binwidth was minimized at the optimal voltage threshold, particularly for acoustic stimuli dominated by temporally dynamic features, indicating that permissive thresholding permits readout of cortical responses with temporal precision on the order of a few milliseconds.

SIGNIFICANCE

The improvements in decoding accuracy we observed for optimal readout parameters suggest that standard thresholding methods substantially underestimate the information present in auditory cortical spiking patterns. The fact that optimal thresholds were relatively low indicates that local populations of cortical neurons exhibit high temporal coherence that could be leveraged in service of future auditory BCI applications.

Learning active sensing strategies using a sensory brain-machine interface

Diverse organisms, from insects to humans, actively seek out sensory information that best informs goal-directed actions. Efficient active sensing requires congruity between sensor properties and motor strategies, as typically honed through evolution. However, it has been difficult to study whether active sensing strategies are also modified with experience. Here, we used a sensory brain-machine interface paradigm, permitting both free behavior and experimental manipulation of sensory feedback, to study learning of active sensing strategies. Rats performed a searching task in a water maze in which the only task-relevant sensory feedback was provided by intracortical microstimulation (ICMS) encoding egocentric bearing to the hidden goal location. The rats learned to use the artificial goal direction sense to find the platform with the same proficiency as natural vision. Manipulation of the acuity of the ICMS feedback revealed distinct search strategy adaptations. Using an optimization model, the different strategies were found to minimize the effort required to extract the most salient task-relevant information. The results demonstrate that animals can adjust motor strategies to match novel sensor properties for efficient goal-directed behavior.

The Energy Homeostasis Principle: Neuronal Energy Regulation Drives Local Network Dynamics Generating Behavior

A major goal of neuroscience is understanding how neurons arrange themselves into neural networks that result in behavior. Most theoretical and experimental efforts have focused on a top-down approach which seeks to identify neuronal correlates of behaviors. This has been accomplished by effectively mapping specific behaviors to distinct neural patterns, or by creating computational models that produce a desired behavioral outcome. Nonetheless, these approaches have only implicitly considered the fact that neural tissue, like any other physical system, is subjected to several restrictions and boundaries of operations. Here, we proposed a new, bottom-up conceptual paradigm: The Energy Homeostasis Principle, where the balance between energy income, expenditure, and availability are the key parameters in determining the dynamics of neuronal phenomena found from molecular to behavioral levels. Neurons display high energy consumption relative to other cells, with metabolic consumption of the brain representing 20% of the whole-body oxygen uptake, contrasting with this organ representing only 2% of the body weight. Also, neurons have specialized surrounding tissue providing the necessary energy which, in the case of the brain, is provided by astrocytes. Moreover, and unlike other cell types with high energy demands such as muscle cells, neurons have strict aerobic metabolism. These facts indicate that neurons are highly sensitive to energy limitations, with Gibb's free energy dictating the direction of all cellular metabolic processes. From this activity, the largest energy, by far, is expended by action potentials and post-synaptic potentials; therefore, plasticity can be reinterpreted in terms of their energy context. Consequently, neurons, through their synapses, impose energy demands over post-synaptic neurons in a close loop-manner, modulating the dynamics of local circuits. Subsequently, the energy dynamics end up impacting the homeostatic mechanisms of neuronal networks. Furthermore, local energy management also emerges as a neural population property, where most of the energy expenses are triggered by sensory or other modulatory inputs. Local energy management in neurons may be sufficient to explain the emergence of behavior, enabling the assessment of which properties arise in neural circuits and how. Essentially, the proposal of the Energy Homeostasis Principle is also readily testable for simple neuronal networks.

Decoding of muscle activity from the sensorimotor cortex in freely behaving monkeys

Remarkable advances have recently been made in the development of Brain-Machine Interface (BMI) technologies for restoring or enhancing motor function. However, the application of these technologies may be limited to patients in static conditions, as these developments have been largely based on studies of animals (e.g., non-human primates) in constrained movement conditions. The ultimate goal of BMI technology is to enable individuals to move their bodies naturally or control external devices without physical constraints. Here, we demonstrate accurate decoding of muscle activity from electrocorticogram (ECoG) signals in unrestrained, freely behaving monkeys. We recorded ECoG signals from the sensorimotor cortex as well as electromyogram signals from multiple muscles in the upper arm while monkeys performed two types of movements with no physical restraints, as follows: forced forelimb movement (lever-pull task) and natural whole-body movement (free movement within the cage). As in previous reports using restrained monkeys, we confirmed that muscle activity during forced forelimb movement was accurately predicted from simultaneously recorded ECoG data. More importantly, we demonstrated that accurate prediction of muscle activity from ECoG data was possible in monkeys performing natural whole-body movement. We found that high-gamma activity in the primary motor cortex primarily contributed to the prediction of muscle activity during natural whole-body movement as well as forced forelimb movement. In contrast, the contribution of high-gamma activity in the premotor and primary somatosensory cortices was significantly larger during natural whole-body movement. Thus, activity in a larger area of the sensorimotor cortex was needed to predict muscle activity during natural whole-body movement. Furthermore, decoding models obtained from forced forelimb movement could not be generalized to natural whole-body movement, which suggests that decoders should be built individually and according to different behavior types. These results contribute to the future application of BMI systems in unrestrained individuals.

Brain-Machine Interfaces: Powerful Tools for Clinical Treatment and Neuroscientific Investigations

Brain-machine interfaces (BMIs) have exploded in popularity in the past decade. BMIs, also called brain-computer interfaces, provide a direct link between the brain and a computer, usually to control an external device. BMIs have a wide array of potential clinical applications, ranging from restoring communication to people unable to speak due to amyotrophic lateral sclerosis or a stroke, to restoring movement to people with paralysis from spinal cord injury or motor neuron disease, to restoring memory to people with cognitive impairment. Because BMIs are controlled directly by the activity of prespecified neurons or cortical areas, they also provide a powerful paradigm with which to investigate fundamental questions about brain physiology, including neuronal behavior, learning, and the role of oscillations. This article reviews the clinical and neuroscientific applications of BMIs, with a primary focus on motor BMIs.

Plausibility of a Neural Network Classifier-Based Neuroprosthesis for Depression Detection via Laughter Records

The present work explores the diagnostic performance for depression of neural network classifiers analyzing the sound structures of laughter as registered from clinical patients and healthy controls. The main methodological novelty of this work is that simple sound variables of laughter are used as inputs, instead of electrophysiological signals or local field potentials (LFPs) or spoken language utterances, which are the usual protocols up-to-date. In the present study, involving 934 laughs from 30 patients and 20 controls, four different neural networks models were tested for sensitivity analysis, and were additionally trained for depression detection. Some elementary sound variables were extracted from the records: timing, fundamental frequency mean, first three formants, average power, and the Shannon-Wiener entropy. In the results obtained, two of the neural networks show a diagnostic discrimination capability of 93.02 and 91.15% respectively, while the third and fourth ones have an 87.96 and 82.40% percentage of success. Remarkably, entropy turns out to be a fundamental variable to distinguish between patients and controls, and this is a significant factor which becomes essential to understand the deep neurocognitive relationships between laughter and depression. In biomedical terms, our neural network classifier-based neuroprosthesis opens up the possibility of applying the same methodology to other mental-health and neuropsychiatric pathologies. Indeed, exploring the application of laughter in the early detection and prognosis of Alzheimer and Parkinson would represent an enticing possibility, both from the biomedical and the computational points of view.

Quantitative simulation of extracellular single unit recording from the surface of cortex

OBJECTIVE

Neural recording is important for a wide variety of clinical applications. Until recently, recording from the surface of the brain, even when using micro-electrocorticography (μECoG) arrays, was not thought to enable recording from individual neurons. Recent results suggest that when the surface electrode contact size is sufficiently small, it may be possible to record single neurons from the brain's surface. In this study, we use computational techniques to investigate the ability of surface electrodes to record the activity of single neurons.

APPROACH

The computational model included the rat head, μECoG electrode, two existing multi-compartmental neuron models, and a novel multi-compartmental neuron model derived from patch clamp experiments in layer 1 of the cortex.

MAIN RESULTS

Using these models, we reproduced single neuron recordings from μECoG arrays, and elucidated their possible source. The model resembles the experimental data when spikes originate from layer 1 neurons that are less than 60 μm from the cortical surface. We further used the model to explore the design space for surface electrodes. Although this model does not include biological or thermal noise, the results indicate the electrode contact area should be 100 μm² or smaller to maintain a detectable waveform amplitude. Furthermore, the model shows the width of lateral insulation could be reduced, which may reduce scar formation, while retaining 95% of signal amplitude.

SIGNIFICANCE

Overall, the model suggests single-unit surface recording is limited to neurons in layer 1 and further improvement in electrode design is needed.

A rodent brain-machine interface paradigm to study the impact of paraplegia on BMI performance

BACKGROUND

Most brain machine interfaces (BMI) focus on upper body function in non-injured animals, not addressing the lower limb functional needs of those with paraplegia. A need exists for a novel BMI task that engages the lower body and takes advantage of well-established rodent spinal cord injury (SCI) models to study methods to improve BMI performance.

NEW METHOD

A tilt BMI task was designed that randomly applies different types of tilts to a platform, decodes the tilt type applied and rights the platform if the decoder correctly classifies the tilt type. The task was tested on female rats and is relatively natural such that it does not require the animal to learn a new skill. It is self-rewarding such that there is no need for additional rewards, eliminating food or water restriction, which can be especially hard on spinalized rats. Finally, task difficulty can be adjusted by making the tilt parameters.

RESULTS

This novel BMI task bilaterally engages the cortex without visual feedback regarding limb position in space and animals learn to improve their performance both pre and post-SCI.Comparison with Existing Methods: Most BMI tasks primarily engage one hemisphere, are upper-body, rely heavily on visual feedback, do not perform investigations in animal models of SCI, and require nonnaturalistic extrinsic motivation such as water rewarding for performance improvement. Our task addresses these gaps.

CONCLUSIONS

The BMI paradigm presented here will enable researchers to investigate the interaction of plasticity after SCI and plasticity during BMI training on performance.

Real-time decoding of covert attention in higher-order visual areas

Brain-computer-interfaces (BCI) provide a means of using human brain activations to control devices for communication. Until now this has only been demonstrated in primary motor and sensory brain regions, using surgical implants or non-invasive neuroimaging techniques. Here, we provide proof-of-principle for the use of higher-order brain regions involved in complex cognitive processes such as attention. Using realtime fMRI, we implemented an online 'winner-takes-all approach' with quadrant-specific parameter estimates, to achieve single-block classification of brain activations. These were linked to the covert allocation of attention to real-world images presented at 4-quadrant locations. Accuracies in three target regions were significantly above chance, with individual decoding accuracies reaching upto 70%. By utilising higher order mental processes, 'cognitive BCIs' access varied and therefore more versatile information, potentially providing a platform for communication in patients who are unable to speak or move due to brain injury.

Closed-loop intracranial stimulation alters movement timing in humans

BACKGROUND

A prime objective driving the recent development of human neural prosthetics is to stimulate neural circuits in a manner time-locked to ongoing brain activity. The human supplementary motor area (SMA) is a particularly useful target for this objective because it displays characteristic neural activity just prior to voluntary movement.

OBJECTIVE

Here, we tested a method that detected activity in the human SMA related to impending movement and then delivered cortical stimulation with intracranial electrodes to influence the timing of movement.

METHODS

We conducted experiments in nine patients with electrodes implanted for epilepsy localization: five patients with SMA electrodes and four control patients with electrodes outside the SMA. In the first experiment, electrocorticographic (ECoG) recordings were used to localize the electrode of interest during a task involving bimanual finger movements. In the second experiment, a real-time sense-and-stimulate (SAS) system was implemented that delivered an electrical stimulus when pre-movement gamma power exceeded a threshold.

RESULTS

Stimulation based on real-time detection of this supra-threshold activity resulted in significant slowing of motor behavior in all of the cases where stimulation was carried out in the SMA patients but in none of the patients where stimulation was performed at the control site.

CONCLUSIONS

The neurophysiological correlates of impending movement can be used to trigger a closed loop stimulation device and influence ongoing motor behavior in a manner imperceptible to the subject. This is the first report of a human closed loop system designed to alter movement using direct cortical recordings and direct stimulation.

A Systems Neuroscience Approach to Migraine

Migraine is an extremely common but poorly understood nervous system disorder. We conceptualize migraine as a disorder of sensory network gain and plasticity, and we propose that this framing makes it amenable to the tools of current systems neuroscience.

Factors Affecting Volume Changes of the Somatosensory Cortex in Patients with Spinal Cord Injury: To Be Considered for Future Neuroprosthetic Design

Spinal cord injury (SCI) leads to severe chronic disability, but also to secondary adaptive changes upstream to the injury in the brain which are most likely induced due to the lack of afferent information. These neuroplastic changes are a potential target for innovative therapies such as neuroprostheses, e.g., by stimulation in order to evoke sensation or in order to suppress phantom limb pain. Diverging results on gray matter atrophy have been reported in patients with SCI. Detectability of atrophy seems to depend on the selection of the regions of interest, while whole-brain approaches are not sensitive enough. In this study, we discussed previous research approaches and analyzed differential atrophic changes in incomplete SCI using manual segmentation of the somatosensory cortex. Patients with incomplete SCI (ASIA C-D), with cervical (N = 5) and thoracic (N = 6) injury were included. Time since injury was ≤12 months in 7 patients, and 144, 152, 216, and 312 months in the other patients. Age at the injury was ≤26 years in 4 patients and ≥50 years in 7 patients. A sample of 12 healthy controls was included in the study. In contrast to all previous studies that used voxel-based morphometry, we performed manual segmentation of the somatosensory cortex in the postcentral gyrus from structural magnetic resonance images and normalized the calculated volumes against the sum of volumes of an automated whole-head segmentation. Volumes were smaller in patients than in controls (p = 0.011), and as a tendency, female patients had smaller volumes than male patients (p = 0.017, uncorrected). No effects of duration (subacute vs. chronic), level of lesion (cervical vs. thoracic), region (left vs. right S1), and age at onset (≤26 vs. ≥50 years) was found. Our results demonstrate volume loss of S1 in incomplete SCI and encourage further research with larger sample sizes on volumetric changes in the acute and chronic stage of SCI, in order to document the moderating effect of type and location of injury on neuroplastic changes. A better understanding of neuroplastic changes in the sensorimotor cortex after SCI and its interaction with sex is needed in order to develop efficient rehabilitative interventions and neuroprosthetic technologies.

Cortex-dependent recovery of unassisted hindlimb locomotion after complete spinal cord injury in adult rats

After paralyzing spinal cord injury the adult nervous system has little ability to ‘heal’ spinal connections, and it is assumed to be unable to develop extra-spinal recovery strategies to bypass the lesion. We challenge this assumption, showing that completely spinalized adult rats can recover unassisted hindlimb weight support and locomotion without explicit spinal transmission of motor commands through the lesion. This is achieved with combinations of pharmacological and physical therapies that maximize cortical reorganization, inducing an expansion of trunk motor cortex and forepaw sensory cortex into the deafferented hindlimb cortex, associated with sprouting of corticospinal axons. Lesioning the reorganized cortex reverses the recovery. Adult rats can thus develop a novel cortical sensorimotor circuit that bypasses the lesion, probably through biomechanical coupling, to partly recover unassisted hindlimb locomotion after complete spinal cord injury.

DOI:http://dx.doi.org/10.7554/eLife.23532.001

State-Dependent Decoding Algorithms Improve the Performance of a Bidirectional BMI in Anesthetized Rats

Brain-machine interfaces (BMIs) promise to improve the quality of life of patients suffering from sensory and motor disabilities by creating a direct communication channel between the brain and the external world. Yet, their performance is currently limited by the relatively small amount of information that can be decoded from neural activity recorded form the brain. We have recently proposed that such decoding performance may be improved when using state-dependent decoding algorithms that predict and discount the large component of the trial-to-trial variability of neural activity which is due to the dependence of neural responses on the network's current internal state. Here we tested this idea by using a bidirectional BMI to investigate the gain in performance arising from using a state-dependent decoding algorithm. This BMI, implemented in anesthetized rats, controlled the movement of a dynamical system using neural activity decoded from motor cortex and fed back to the brain the dynamical system's position by electrically microstimulating somatosensory cortex. We found that using state-dependent algorithms that tracked the dynamics of ongoing activity led to an increase in the amount of information extracted form neural activity by 22%, with a consequently increase in all of the indices measuring the BMI's performance in controlling the dynamical system. This suggests that state-dependent decoding algorithms may be used to enhance BMIs at moderate computational cost.

Beta band oscillations in motor cortex reflect neural population signals that delay movement onset

Motor cortical beta oscillations have been reported for decades, yet their behavioral correlates remain unresolved. Some studies link beta oscillations to changes in underlying neural activity, but the specific behavioral manifestations of these reported changes remain elusive. To investigate how changes in population neural activity, beta oscillations, and behavior are linked, we recorded multi-scale neural activity from motor cortex while three macaques performed a novel neurofeedback task. Subjects volitionally brought their beta oscillatory power to an instructed state and subsequently executed an arm reach. Reaches preceded by a reduction in beta power exhibited significantly faster movement onset times than reaches preceded by an increase in beta power. Further, population neural activity was found to shift farther from a movement onset state during beta oscillations that were neurofeedback-induced or naturally occurring during reaching tasks. This finding establishes a population neural basis for slowed movement onset following periods of beta oscillatory activity.

A network engineering perspective on probing and perturbing cognition with neurofeedback

Network science and engineering provide a flexible and generalizable tool set to describe and manipulate complex systems characterized by heterogeneous interaction patterns among component parts. While classically applied to social systems, these tools have recently proven to be particularly useful in the study of the brain. In this review, we describe the nascent use of these tools to understand human cognition, and we discuss their utility in informing the meaningful and predictable perturbation of cognition in combination with the emerging capabilities of neurofeedback. To blend these disparate strands of research, we build on emerging conceptualizations of how the brain functions (as a complex network) and how we can develop and target interventions or modulations (as a form of network control). We close with an outline of current frontiers that bridge neurofeedback, connectomics, and network control theory to better understand human cognition.

The future of psychiatry: brain devices

Recent advances in deep brain stimulators and brain-machine interfaces have greatly expanded the possibilities of neuroprosthetics and neuromodulation. Together with advances in neuroengineering, nanotechnology, molecular biology and material sciences, it is now possible to address fundamental questions in neuroscience in new, more powerful ways. It is now possible to apply these new technologies in ways that range from augmenting and restoring function to neuromodulation modalities that treat neuropsychiatric disorders. Recent developments in neuromodulation methods offer significant advantages and potential clinical benefits for a variety of disorders. Here we describe the current state of the art in neuromodulation methods, and some advances in brain-machine interfaces, describing the advantages and limitations of the clinical applications of each method. The future applications of these new methods and how they will shape the future of psychiatry and medicine, along with safety and ethical implications, are also discussed.