Neural Manifolds for the Control of Movement

Decoding thoughts, encoding ethics: A narrative review of the BCI-AI revolution

OBJECTIVES

This narrative review aims to analyze mechanisms underlying Brain-Computer Interface (BCI) and Artificial Intelligence (AI) integration, evaluate recent advances in signal acquisition and processing techniques, and assess AI-enhanced neural decoding strategies. The review identifies critical research gaps and examines emerging solutions across multiple domains of BCI-AI integration.

METHODS

A narrative review was conducted using major biomedical and scientific databases including PubMed, Web of Science, IEEE Xplore, and Scopus (2014-2024). Literature was analyzed to identify key developments in BCI-AI integration, with particular emphasis on recent advances (2019-2024). The review process involved thematic analysis of selected publications focusing on practical applications, technical innovations, and emerging challenges.

RESULTS

Recent advances demonstrate significant improvements in BCI-AI systems: 1) High-density electrode arrays achieve spatial resolution up to 5 mm, with stable recordings over 15 months; 2) Deep learning decoders show 40 % improvement in information transfer rates compared to traditional methods; 3) Adaptive algorithms maintain >90 % success rates in motor control tasks over 200-day periods without recalibration; 4) Novel closed-loop optimization frameworks reduce user training time by 55 % while improving accuracy. Latest developments in flexible neural interfaces and self-supervised learning approaches show promise in addressing long-term stability and cross-user generalization challenges.

CONCLUSIONS

BCI-AI integration shows remarkable progress in improving signal quality, decoding accuracy, and user adaptation. While challenges remain in long-term stability and user training, advances in adaptive algorithms and feedback mechanisms demonstrate the technology's growing viability for clinical applications. Recent innovations in electrode technology, AI architectures, and closed-loop systems, combined with emerging standardization frameworks, suggest accelerating progress toward widespread therapeutic use and human augmentation applications.

Lattice physics approaches for neural networks

Modern neuroscience has evolved into a frontier field that draws on numerous disciplines, resulting in the flourishing of novel conceptual frames primarily inspired by physics and complex systems science. Contributing in this direction, we recently introduced a mathematical framework to describe the spatiotemporal interactions of systems of neurons using lattice field theory, the reference paradigm for theoretical particle physics. In this note, we provide a concise summary of the basics of the theory, aiming to be intuitive to the interdisciplinary neuroscience community. We contextualize our methods, illustrating how to readily connect the parameters of our formulation to experimental variables using well-known renormalization procedures. This synopsis yields the key concepts needed to describe neural networks using lattice physics. Such classes of methods are attention-worthy in an era of blistering improvements in numerical computations, as they can facilitate relating the observation of neural activity to generative models underpinned by physical principles.

Accurate neural control of a hand prosthesis by posture-related activity in the primate grasping circuit

Brain-computer interfaces (BCIs) have the potential to restore hand movement for people with paralysis, but current devices still lack the fine control required to interact with objects of daily living. Following our understanding of cortical activity during arm reaches, hand BCI studies have focused primarily on velocity control. However, mounting evidence suggests that posture, and not velocity, dominates in hand-related areas. To explore whether this signal can causally control a prosthesis, we developed a BCI training paradigm centered on the reproduction of posture transitions. Monkeys trained with this protocol were able to control a multidimensional hand prosthesis with high accuracy, including execution of the very intricate precision grip. Analysis revealed that the posture signal in the target grasping areas was the main contributor to control. We present, for the first time, neural posture control of a multidimensional hand prosthesis, opening the door for future interfaces to leverage this additional information channel.

Neuro-cognitive multilevel causal modeling: A framework that bridges the explanatory gap between neuronal activity and cognition

Explaining how neuronal activity gives rise to cognition arguably remains the most significant challenge in cognitive neuroscience. We introduce neuro-cognitive multilevel causal modeling (NC-MCM), a framework that bridges the explanatory gap between neuronal activity and cognition by construing cognitive states as (behaviorally and dynamically) causally consistent abstractions of neuronal states. Multilevel causal modeling allows us to interchangeably reason about the neuronal- and cognitive causes of behavior while maintaining a physicalist (in contrast to a strong dualist) position. We introduce an algorithm for learning cognitive-level causal models from neuronal activation patterns and demonstrate its ability to learn cognitive states of the nematode C. elegans from calcium imaging data. We show that the cognitive-level model of the NC-MCM framework provides a concise representation of the neuronal manifold of C. elegans and its relation to behavior as a graph, which, in contrast to other neuronal manifold learning algorithms, supports causal reasoning. We conclude the article by arguing that the ability of the NC-MCM framework to learn causally interpretable abstractions of neuronal dynamics and their relation to behavior in a purely data-driven fashion is essential for understanding biological systems whose complexity prohibits the development of hand-crafted computational models.

The unbearable slowness of being: Why do we live at 10 bits/s?

This article is about the neural conundrum behind the slowness of human behavior. The information throughput of a human being is about 10 bits/s. In comparison, our sensory systems gather data at ∼109 bits/s. The stark contrast between these numbers remains unexplained and touches on fundamental aspects of brain function: what neural substrate sets this speed limit on the pace of our existence? Why does the brain need billions of neurons to process 10 bits/s? Why can we only think about one thing at a time? The brain seems to operate in two distinct modes: the "outer" brain handles fast high-dimensional sensory and motor signals, whereas the "inner" brain processes the reduced few bits needed to control behavior. Plausible explanations exist for the large neuron numbers in the outer brain, but not for the inner brain, and we propose new research directions to remedy this.

The Dimensions of dimensionality

Cognitive scientists often infer multidimensional representations from data. Whether the data involve text, neuroimaging, neural networks, or human judgments, researchers frequently infer and analyze latent representational spaces (i.e., embeddings). However, the properties of a latent representation (e.g., prediction performance, interpretability, compactness) depend on the inference procedure, which can vary widely across endeavors. For example, dimensions are not always globally interpretable and the dimensionality of different embeddings may not be readily comparable. Moreover, the dichotomy between multidimensional spaces and purportedly richer representational formats, such as graph representations, is misleading. We review what the different notions of dimension in cognitive science imply for how these latent representations should be used and interpreted.

Aligned and oblique dynamics in recurrent neural networks

The relation between neural activity and behaviorally relevant variables is at the heart of neuroscience research. When strong, this relation is termed a neural representation. There is increasing evidence, however, for partial dissociations between activity in an area and relevant external variables. While many explanations have been proposed, a theoretical framework for the relationship between external and internal variables is lacking. Here, we utilize recurrent neural networks (RNNs) to explore the question of when and how neural dynamics and the network's output are related from a geometrical point of view. We find that training RNNs can lead to two dynamical regimes: dynamics can either be aligned with the directions that generate output variables, or oblique to them. We show that the choice of readout weight magnitude before training can serve as a control knob between the regimes, similar to recent findings in feedforward networks. These regimes are functionally distinct. Oblique networks are more heterogeneous and suppress noise in their output directions. They are furthermore more robust to perturbations along the output directions. Crucially, the oblique regime is specific to recurrent (but not feedforward) networks, arising from dynamical stability considerations. Finally, we show that tendencies toward the aligned or the oblique regime can be dissociated in neural recordings. Altogether, our results open a new perspective for interpreting neural activity by relating network dynamics and their output.

A reinforcement learning based software simulator for motor brain-computer interfaces

Intracortical motor brain-computer interfaces (BCIs) are expensive and time-consuming to design because accurate evaluation traditionally requires real-time experiments. In a BCI system, a user interacts with an imperfect decoder and continuously changes motor commands in response to unexpected decoded movements. This "closed-loop" nature of BCI leads to emergent interactions between the user and decoder that are challenging to model. The gold standard for BCI evaluation is therefore real-time experiments, which significantly limits the speed and community of BCI research. We present a new BCI simulator that enables researchers to accurately and quickly design BCIs for cursor control entirely in software. Our simulator replaces the BCI user with a deep reinforcement learning (RL) agent that interacts with a simulated BCI system and learns to optimally control it. We demonstrate that our simulator is accurate and versatile, reproducing the published results of three distinct types of BCI decoders: (1) a state-of-the-art linear decoder (FIT-KF), (2) a "two-stage" BCI decoder requiring closed-loop decoder adaptation (ReFIT-KF), and (3) a nonlinear recurrent neural network decoder (FORCE).

Representational learning by optimization of neural manifolds in an olfactory memory network

Higher brain functions depend on experience-dependent representations of relevant information that may be organized by attractor dynamics or by geometrical modifications of continuous "neural manifolds". To explore these scenarios we analyzed odor-evoked activity in telencephalic area pDp of juvenile and adult zebrafish, the homolog of piriform cortex. No obvious signatures of attractor dynamics were detected. Rather, olfactory discrimination training selectively enhanced the separation of neural manifolds representing task-relevant odors from other representations, consistent with predictions of autoassociative network models endowed with precise synaptic balance. Analytical approaches using the framework of manifold capacity revealed multiple geometrical modifications of representational manifolds that supported the classification of task-relevant sensory information. Manifold capacity predicted odor discrimination across individuals, indicating a close link between manifold geometry and behavior. Hence, pDp and possibly related recurrent networks store information in the geometry of representational manifolds, resulting in joint sensory and semantic maps that may support distributed learning processes.

A combinatorial neural code for long-term motor memory

Motor skill repertoire can be stably retained over long periods, but the neural mechanism that underlies stable memory storage remains poorly understood1-8. Moreover, it is unknown how existing motor memories are maintained as new motor skills are continuously acquired. Here we tracked neural representation of learned actions throughout a significant portion of the lifespan of a mouse and show that learned actions are stably retained in combination with context, which protects existing memories from erasure during new motor learning. We established a continual learning paradigm in which mice learned to perform directional licking in different task contexts while we tracked motor cortex activity for up to six months using two-photon imaging. Within the same task context, activity driving directional licking was stable over time with little representational drift. When learning new task contexts, new preparatory activity emerged to drive the same licking actions. Learning created parallel new motor memories instead of modifying existing representations. Re-learning to make the same actions in the previous task context re-activated the previous preparatory activity, even months later. Continual learning of new task contexts kept creating new preparatory activity patterns. Context-specific memories, as we observed in the motor system, may provide a solution for stable memory storage throughout continual learning.

Pupillary manifolds: uncovering the latent geometrical structures behind phasic changes in pupil size

The size of the pupils reflects directly the balance of different branches of the autonomic nervous system. This measure is inexpensive, non-invasive, and has provided invaluable insights on a wide range of mental processes, from attention to emotion and executive functions. Two outstanding limitations of current pupillometry research are the lack of consensus in the analytical approaches, which vary wildly across research groups and disciplines, and the fact that, unlike other neuroimaging techniques, pupillometry lacks the dimensionality to shed light on the different sources of the observed effects. In other words, pupillometry provides an integrated readout of several distinct networks, but it is unclear whether each has a specific fingerprint, stemming from its function or physiological substrate. Here we show that phasic changes in pupil size are inherently low-dimensional, with modes that are highly consistent across behavioral tasks of very different nature, suggesting that these changes occur along pupillary manifolds that are highly constrained by the underlying physiological structures rather than functions. These results provide not only a unified approach to analyze pupillary data, but also the opportunity for physiology and psychology to refer to the same processes by tracing the sources of the reported changes in pupil size in the underlying biology.

Specific inhibition and disinhibition in the higher-order structure of a cortical connectome

Neurons are thought to act as parts of assemblies with strong internal excitatory connectivity. Conversely, inhibition is often reduced to blanket inhibition with no targeting specificity. We analyzed the structure of excitation and inhibition in the MICrONS $mm^{3}$ dataset, an electron microscopic reconstruction of a piece of cortical tissue. We found that excitation was structured around a feed-forward flow in large non-random neuron motifs with a structure of information flow from a small number of sources to a larger number of potential targets. Inhibitory neurons connected with neurons in specific sequential positions of these motifs, implementing targeted and symmetrical competition between them. None of these trends are detectable in only pairwise connectivity, demonstrating that inhibition is structured by these large motifs. While descriptions of inhibition in cortical circuits range from non-specific blanket-inhibition to targeted, our results describe a form of targeting specificity existing in the higher-order structure of the connectome. These findings have important implications for the role of inhibition in learning and synaptic plasticity.

Semi-orthogonal subspaces for value mediate a binding and generalization trade-off

When choosing between options, we must associate their values with the actions needed to select them. We hypothesize that the brain solves this binding problem through neural population subspaces. Here, in macaques performing a choice task, we show that neural populations in five reward-sensitive regions encode the values of offers presented on the left and right in distinct subspaces. This encoding is sufficient to bind offer values to their locations while preserving abstract value information. After offer presentation, all areas encode the value of the first and second offers in orthogonal subspaces; this orthogonalization also affords binding. Our binding-by-subspace hypothesis makes two new predictions confirmed by the data. First, behavioral errors should correlate with spatial, but not temporal, neural misbinding. Second, behavioral errors should increase when offers have low or high values, compared to medium values, even when controlling for value difference. Together, these results support the idea that the brain uses semi-orthogonal subspaces to bind features.

Fine-grained descending control of steering in walking Drosophila

Locomotion involves rhythmic limb movement patterns that originate in circuits outside the brain. Purposeful locomotion requires descending commands from the brain, but we do not understand how these commands are structured. Here, we investigate this issue, focusing on the control of steering in walking Drosophila. First, we describe different limb "gestures" associated with different steering maneuvers. Next, we identify a set of descending neurons whose activity predicts steering. Focusing on two descending cell types downstream of distinct brain networks, we show that they evoke specific limb gestures: one lengthens strides on the outside of a turn, while the other attenuates strides on the inside of a turn. Our results suggest that a single descending neuron can have opposite effects during different locomotor rhythm phases, and we identify networks positioned to implement this phase-specific gating. Together, our results show how purposeful locomotion emerges from specific, coordinated modulations of low-level patterns.

Cortico-thalamic communication for action coordination in a skilled motor sequence

The coordination of forelimb and orofacial movements to compose an ethological reach-to-consume behavior likely involves neural communication across brain regions. Leveraging wide-field imaging and photo-inhibition to survey across the cortex, we identified a cortical network and a high-order motor area (MOs-c), which coordinate action progression in a mouse reach-and-withdraw-to-drink (RWD) behavior. Electrophysiology and photo-inhibition across multiple projection neuron types within the MOs-c revealed differential contributions of pyramidal tract and corticothalamic (CTMOs) output channels to action progression and hand-mouth coordination. Notably, CTMOs display sustained firing throughout RWD sequence and selectively enhance RWD-relevant activity in postsynaptic thalamus neurons, which also contribute to action coordination. CTMOs receive converging monosynaptic inputs from forelimb and orofacial sensorimotor areas and are reciprocally connected to thalamic neurons, which project back to the cortical network. Therefore, motor cortex corticothalamic channel may selectively amplify the thalamic integration of cortical and subcortical sensorimotor streams to coordinate a skilled motor sequence.

Motor cortex is responsible for motoric dynamics in striatum and the execution of both skilled and unskilled actions

Striatum and its predominant input, motor cortex, are responsible for the selection and performance of purposive movement, but how their interaction guides these processes is not understood. To establish its neural and behavioral contributions, we bilaterally lesioned motor cortex and recorded striatal activity and reaching performance daily, capturing the lesion's direct ramifications within hours of the intervention. We observed reaching impairment and an absence of striatal motoric activity following lesion of motor cortex, but not parietal cortex control lesions. Although some aspects of performance began to recover after 8-10 days, striatal projection and interneuronal dynamics did not-eventually entering a non-motor encoding state that aligned with persisting kinematic control deficits. Lesioned mice also exhibited a profound inability to switch motor plans while locomoting, reminiscent of clinical freezing of gait (FOG). Our results demonstrate the necessity of motor cortex in generating trained and untrained actions as well as striatal motoric dynamics.

Global motor dynamics - Invariant neural representations of motor behavior in distributed brain-wide recordings

Objective.Motor-related neural activity is more widespread than previously thought, as pervasive brain-wide neural correlates of motor behavior have been reported in various animal species. Brain-wide movement-related neural activity have been observed in individual brain areas in humans as well, but it is unknown to what extent global patterns exist.Approach.Here, we use a decoding approach to capture and characterize brain-wide neural correlates of movement. We recorded invasive electrophysiological data from stereotactic electroencephalographic electrodes implanted in eight epilepsy patients who performed both an executed and imagined grasping task. Combined, these electrodes cover the whole brain, including deeper structures such as the hippocampus, insula and basal ganglia. We extract a low-dimensional representation and classify movement from rest trials using a Riemannian decoder.Main results.We reveal global neural dynamics that are predictive across tasks and participants. Using an ablation analysis, we demonstrate that these dynamics remain remarkably stable under loss of information. Similarly, the dynamics remain stable across participants, as we were able to predict movement across participants using transfer learning.Significance.Our results show that decodable global motor-related neural dynamics exist within a low-dimensional space. The dynamics are predictive of movement, nearly brain-wide and present in all our participants. The results broaden the scope to brain-wide investigations, and may allow combining datasets of multiple participants with varying electrode locations or calibrationless neural decoder.

Large-Scale Mechanistic Models of Brain Circuits with Biophysically and Morphologically Detailed Neurons

Understanding the brain requires studying its multiscale interactions from molecules to networks. The increasing availability of large-scale datasets detailing brain circuit composition, connectivity, and activity is transforming neuroscience. However, integrating and interpreting this data remains challenging. Concurrently, advances in supercomputing and sophisticated modeling tools now enable the development of highly detailed, large-scale biophysical circuit models. These mechanistic multiscale models offer a method to systematically integrate experimental data, facilitating investigations into brain structure, function, and disease. This review, based on a Society for Neuroscience 2024 MiniSymposium, aims to disseminate recent advances in large-scale mechanistic modeling to the broader community. It highlights (1) examples of current models for various brain regions developed through experimental data integration; (2) their predictive capabilities regarding cellular and circuit mechanisms underlying experimental recordings (e.g., membrane voltage, spikes, local-field potential, electroencephalography/magnetoencephalography) and brain function; and (3) their use in simulating biomarkers for brain diseases like epilepsy, depression, schizophrenia, and Parkinson's, aiding in understanding their biophysical underpinnings and developing novel treatments. The review showcases state-of-the-art models covering hippocampus, somatosensory, visual, motor, auditory cortical, and thalamic circuits across species. These models predict neural activity at multiple scales and provide insights into the biophysical mechanisms underlying sensation, motor behavior, brain signals, neural coding, disease, pharmacological interventions, and neural stimulation. Collaboration with experimental neuroscientists and clinicians is essential for the development and validation of these models, particularly as datasets grow. Hence, this review aims to foster interest in detailed brain circuit models, leading to cross-disciplinary collaborations that accelerate brain research.

De novo sensorimotor learning through reuse of movement components

From tying one's shoelaces to driving a car, complex skills involving the coordination of multiple muscles are common in everyday life; yet relatively little is known about how these skills are learned. Recent studies have shown that new sensorimotor skills involving re-mapping familiar body movements to unfamiliar outputs cannot be learned by adjusting pre-existing controllers, and that new task-specific controllers must instead be learned "de novo". To date, however, few studies have investigated de novo learning in scenarios requiring continuous and coordinated control of relatively unpractised body movements. In this study, we used a myoelectric interface to investigate how a novel controller is learned when the task involves an unpractised combination of relatively untrained continuous muscle contractions. Over five sessions on five consecutive days, participants learned to trace a series of trajectories using a computer cursor controlled by the activation of two muscles. The timing of the generated cursor trajectory and its shape relative to the target improved for conditions trained with post-trial visual feedback. Improvements in timing transferred to all untrained conditions, but improvements in shape transferred less robustly to untrained conditions requiring the trained order of muscle activation. All muscle outputs in the final session could already be generated during the first session, suggesting that participants learned the new task by improving the selection of existing motor commands. These results suggest that the novel controllers acquired during de novo learning can, in some circumstances, be constructed from components of existing controllers.

Unsupervised, piecewise linear decoding enables an accurate prediction of muscle activity in a multi-task brain computer interface

Objective

Creating an intracortical brain-computer interface (iBCI) capable of seamless transitions between tasks and contexts would greatly enhance user experience. However, the nonlinearity in neural activity presents challenges to computing a global iBCI decoder. We aimed to develop a method that differs from a globally optimized decoder to address this issue.

Approach

We devised an unsupervised approach that relies on the structure of a low-dimensional neural manifold to implement a piecewise linear decoder. We created a distinctive dataset in which monkeys performed a diverse set of tasks, some trained, others innate, while we recorded neural signals from the motor cortex (M1) and electromyographs (EMGs) from upper limb muscles. We used both linear and nonlinear dimensionality reduction techniques to discover neural manifolds and applied unsupervised algorithms to identify clusters within those spaces. Finally, we fit a linear decoder of EMG for each cluster. A specific decoder was activated corresponding to the cluster each new neural data point belonged to.

Main results

We found clusters in the neural manifolds corresponding with the different tasks or task sub-phases. The performance of piecewise decoding improved as the number of clusters increased and plateaued gradually. With only two clusters it already outperformed a global linear decoder, and unexpectedly, it outperformed even a global recurrent neural network (RNN) decoder with 10-12 clusters.

Significance

This study introduced a computationally lightweight solution for creating iBCI decoders that can function effectively across a broad range of tasks. EMG decoding is particularly challenging, as muscle activity is used, under varying contexts, to control interaction forces and limb stiffness, as well as motion. The results suggest that a piecewise linear decoder can provide a good approximation to the nonlinearity between neural activity and motor outputs, a result of our increased understanding of the structure of neural manifolds in motor cortex.

Motor Cortex Latent Dynamics Encode Spatial and Temporal Arm Movement Parameters Independently

The fluid movement of an arm requires multiple spatiotemporal parameters to be set independently. Recent studies have argued that arm movements are generated by the collective dynamics of neurons in motor cortex. An untested prediction of this hypothesis is that independent parameters of movement must map to independent components of the neural dynamics. Using a task where three male monkeys made a sequence of reaching movements to randomly placed targets, we show that the spatial and temporal parameters of arm movements are independently encoded in the low-dimensional trajectories of population activity in motor cortex: each movement's direction corresponds to a fixed neural trajectory through neural state space and its speed to how quickly that trajectory is traversed. Recurrent neural network models show that this coding allows independent control over the spatial and temporal parameters of movement by separate network parameters. Our results support a key prediction of the dynamical systems view of motor cortex, and also argue that not all parameters of movement are defined by different trajectories of population activity.

The Volitional Control of Individual Motor Units Is Constrained within Low-Dimensional Neural Manifolds by Common Inputs

The implementation of low-dimensional movement control by the central nervous system has been debated for decades. In this study, we investigated the dimensionality of the control signals received by spinal motor neurons when controlling either the ankle or knee joint torque. We first identified the low-dimensional latent factors underlying motor unit activity during torque-matched isometric contractions in male participants. Subsequently, we evaluated the extent to which motor units could be independently controlled. To this aim, we used an online control paradigm in which participants received the corresponding motor unit firing rates as visual feedback. We identified two main latent factors, regardless of the muscle group (vastus lateralis-medialis and gastrocnemius lateralis-medialis). The motor units of the gastrocnemius lateralis could be controlled largely independently from those of the gastrocnemius medialis during ankle plantarflexion. This dissociation of motor unit activity imposed similar behavior to the motor units that were not displayed in the feedback. Conversely, it was not possible to dissociate the activity of the motor units between the vastus lateralis and medialis muscles during the knee extension tasks. These results demonstrate that the number of latent factors estimated from linear dimensionality reduction algorithms does not necessarily reflect the dimensionality of volitional control of motor units. Overall, individual motor units were never controlled independently of all others but rather belonged to synergistic groups. Together, these findings provide evidence for a low-dimensional control of motor units constrained by common inputs, with notable differences between muscle groups.

Selective direct motor cortical influence during naturalistic climbing

It remains poorly resolved when and how motor cortical output directly influences limb muscle activity through descending projections, which impedes mechanistic understanding of cortical movement control. Here we addressed this in mice performing an ethologically inspired all-limb climbing behavior. We quantified the direct influence of forelimb primary motor cortex (caudal forelimb area, CFA) on muscle activity comprehensively across the muscle activity states that occur during climbing. We found that CFA informs muscle activity pattern, mainly by selectively activating certain muscles while exerting much smaller, bidirectional effects on their antagonists. From Neuropixel recordings, we identified linear combinations (components) of motor cortical activity that covary with these effects, finding that these components differ from those that covary with muscle activity or kinematics. Collectively, our results reveal an instructive direct motor cortical influence on limb muscles that is selective within a motor behavior and reliant on a new type of neural activity subspace.

Reach-dependent reorientation of rotational dynamics in motor cortex

During reaching, neurons in motor cortex exhibit complex, time-varying activity patterns. Though single-neuron activity correlates with movement parameters, movement correlations explain neural activity only partially. Neural responses also reflect population-level dynamics thought to generate outputs. These dynamics have previously been described as "rotational," such that activity orbits in neural state space. Here, we reanalyze reaching datasets from male Rhesus macaques and find two essential features that cannot be accounted for with standard dynamics models. First, the planes in which rotations occur differ for different reaches. Second, this variation in planes reflects the overall location of activity in neural state space. Our "location-dependent rotations" model fits nearly all motor cortex activity during reaching, and high-quality decoding of reach kinematics reveals a quasilinear relationship with spiking. Varying rotational planes allows motor cortex to produce richer outputs than possible under previous models. Finally, our model links representational and dynamical ideas: representation is present in the state space location, which dynamics then convert into time-varying command signals.

Revealing unexpected complex encoding but simple decoding mechanisms in motor cortex via separating behaviorally relevant neural signals

In motor cortex, behaviorally relevant neural responses are entangled with irrelevant signals, which complicates the study of encoding and decoding mechanisms. It remains unclear whether behaviorally irrelevant signals could conceal some critical truth. One solution is to accurately separate behaviorally relevant and irrelevant signals at both single-neuron and single-trial levels, but this approach remains elusive due to the unknown ground truth of behaviorally relevant signals. Therefore, we propose a framework to define, extract, and validate behaviorally relevant signals. Analyzing separated signals in three monkeys performing different reaching tasks, we found neural responses previously considered to contain little information actually encode rich behavioral information in complex nonlinear ways. These responses are critical for neuronal redundancy and reveal movement behaviors occupy a higher-dimensional neural space than previously expected. Surprisingly, when incorporating often-ignored neural dimensions, behaviorally relevant signals can be decoded linearly with comparable performance to nonlinear decoding, suggesting linear readout may be performed in motor cortex. Our findings prompt that separating behaviorally relevant signals may help uncover more hidden cortical mechanisms.

Neural Manifold Constraint for Spike Prediction Models Under Behavioral Reinforcement

Spike prediction models effectively predict downstream spike trains from upstream neural activity for neural prostheses. Such prostheses could potentially restore damaged neural communication pathways using predicted patterns to guide electrical stimulations on downstream. Since the ground truth of downstream neural activity is unavailable for subjects with the damage, reinforcement learning (RL) with behavior-level rewards becomes necessary for model training. However, existing models do not involve any constraint on the generated firing patterns and neglect the correlations among neural activities. Thus, the model outputs can greatly deviate from the natural range of neural activities, causing concerns for clinical usage. This study proposes the neural manifold constraint to solve this problem, shaping RL-generated spike trains in the feature space. The constraint terms describe the first and second order statistics of the neural manifold estimated from neural recordings during subjects' freely moving period. Then, the models can be optimized within the neural manifold by behavioral reinforcement. We test the method to predict primary motor cortex (M1) spikes from medial prefrontal (mPFC) spikes when rats perform the two-lever discrimination task. Results show that the neural activity generated by constrained models resembles the real M1 recordings. Compared with models without constraints, our approach achieves similar behavioral success rates, but reduces the mean squared error of neural firing by 61%. The constraints also increase the model's robustness across data segments and induce realistic neural correlations. Our method provides a promising tool to restore transregional communication with high behavioral performance and more realistic microscopic patterns.

Hippocampal-prefrontal communication subspaces align with behavioral and network patterns in a spatial memory task

Rhythmic network states have been theorized to facilitate communication between brain regions, but how these oscillations influence communication subspaces, i.e, the low-dimensional neural activity patterns that mediate inter-regional communication, and in turn how subspaces impact behavior remains unclear. Using a spatial memory task in rats, we simultaneously recorded ensembles from hippocampal CA1 and the prefrontal cortex (PFC) to address this question. We found that task behaviors best aligned with low-dimensional, shared subspaces between these regions, rather than local activity in either region. Critically, both network oscillations and speed modulated the structure and performance of this communication subspace. Contrary to expectations, theta coherence did not better predict CA1-PFC shared activity, while theta power played a more significant role. To understand the communication space, we visualized shared CA1-PFC communication geometry using manifold techniques and found ring-like structures. We hypothesize that these shared activity manifolds are utilized to mediate the task behavior. These findings suggest that memory-guided behaviors are driven by shared CA1-PFC interactions that are dynamically modulated by oscillatory states, offering a novel perspective on the interplay between rhythms and behaviorally relevant neural communication.

Thalamocortical architectures for flexible cognition and efficient learning

The brain exhibits a remarkable ability to learn and execute context-appropriate behaviors. How it achieves such flexibility, without sacrificing learning efficiency, is an important open question. Neuroscience, psychology, and engineering suggest that reusing and repurposing computations are part of the answer. Here, we review evidence that thalamocortical architectures may have evolved to facilitate these objectives of flexibility and efficiency by coordinating distributed computations. Recent work suggests that distributed prefrontal cortical networks compute with flexible codes, and that the mediodorsal thalamus provides regularization to promote efficient reuse. Thalamocortical interactions resemble hierarchical Bayesian computations, and their network implementation can be related to existing gating, synchronization, and hub theories of thalamic function. By reviewing recent findings and providing a novel synthesis, we highlight key research horizons integrating computation, cognition, and systems neuroscience.

Neural manifolds in V1 change with top-down signals from V4 targeting the foveal region

High-dimensional brain activity is often organized into lower-dimensional neural manifolds. However, the neural manifolds of the visual cortex remain understudied. Here, we study large-scale multi-electrode electrophysiological recordings of macaque (Macaca mulatta) areas V1, V4, and DP with a high spatiotemporal resolution. We find that the population activity of V1 contains two separate neural manifolds, which correlate strongly with eye closure (eyes open/closed) and have distinct dimensionalities. Moreover, we find strong top-down signals from V4 to V1, particularly to the foveal region of V1, which are significantly stronger during the eyes-open periods. Finally, in silico simulations of a balanced spiking neuron network qualitatively reproduce the experimental findings. Taken together, our analyses and simulations suggest that top-down signals modulate the population activity of V1. We postulate that the top-down modulation during the eyes-open periods prepares V1 for fast and efficient visual responses, resulting in a type of visual stand-by state.

Neural population dynamics in songbird RA and HVC during learned motor-vocal behavior

Complex, learned motor behaviors involve the coordination of large-scale neural activity across multiple brain regions, but our understanding of the population-level dynamics within different regions tied to the same behavior remains limited. Here, we investigate the neural population dynamics underlying learned vocal production in awake-singing songbirds. We use Neuropixels probes to record the simultaneous extracellular activity of populations of neurons in two regions of the vocal motor pathway. In line with observations made in non-human primates during limb-based motor tasks, we show that the population-level activity in both the premotor nucleus HVC and the motor nucleus RA is organized on low-dimensional neural manifolds upon which coordinated neural activity is well described by temporally structured trajectories during singing behavior. Both the HVC and RA latent trajectories provide relevant information to predict vocal sequence transitions between song syllables. However, the dynamics of these latent trajectories differ between regions. Our state-space models suggest a unique and continuous-over-time correspondence between the latent space of RA and vocal output, whereas the corresponding relationship for HVC exhibits a higher degree of neural variability. We then demonstrate that comparable high-fidelity reconstruction of continuous vocal outputs can be achieved from HVC and RA neural latents and spiking activity. Unlike those that use spiking activity, however, decoding models using neural latents generalize to novel sub-populations in each region, consistent with the existence of preserved manifolds that confine vocal-motor activity in HVC and RA.

Computational role of structure in neural activity and connectivity

One major challenge of neuroscience is identifying structure in seemingly disorganized neural activity. Different types of structure have different computational implications that can help neuroscientists understand the functional role of a particular brain area. Here, we outline a unified approach to characterize structure by inspecting the representational geometry and the modularity properties of the recorded activity and show that a similar approach can also reveal structure in connectivity. We start by setting up a general framework for determining geometry and modularity in activity and connectivity and relating these properties with computations performed by the network. We then use this framework to review the types of structure found in recent studies of model networks performing three classes of computations.

Similar neural states, but dissimilar decoding patterns for motor control in parietal cortex

Discrete neural states are associated with reaching movements across the fronto-parietal network. Here, the Hidden Markov Model (HMM) applied to spiking activity of the somato-motor parietal area PE revealed a sequence of states similar to those of the contiguous visuomotor areas PEc and V6A. Using a coupled clustering and decoding approach, we proved that these neural states carried spatiotemporal information regarding behaviour in all three posterior parietal areas. However, comparing decoding accuracy, PE was less informative than V6A and PEc. In addition, V6A outperformed PEc in target inference, indicating functional differences among the parietal areas. To check the consistency of these differences, we used both a supervised and an unsupervised variant of the HMM, and compared its performance with two more common classifiers, Support Vector Machine and Long-Short Term Memory. The differences in decoding between areas were invariant to the algorithm used, still showing the dissimilarities found with HMM, thus indicating that these dissimilarities are intrinsic in the information encoded by parietal neurons. These results highlight that, when decoding from the parietal cortex, for example, in brain machine interface implementations, attention should be paid in selecting the most suitable source of neural signals, given the great heterogeneity of this cortical sector.

Reconfigurations of cortical manifold structure during reward-based motor learning

Adaptive motor behavior depends on the coordinated activity of multiple neural systems distributed across the brain. While the role of sensorimotor cortex in motor learning has been well established, how higher-order brain systems interact with sensorimotor cortex to guide learning is less well understood. Using functional MRI, we examined human brain activity during a reward-based motor task where subjects learned to shape their hand trajectories through reinforcement feedback. We projected patterns of cortical and striatal functional connectivity onto a low-dimensional manifold space and examined how regions expanded and contracted along the manifold during learning. During early learning, we found that several sensorimotor areas in the dorsal attention network exhibited increased covariance with areas of the salience/ventral attention network and reduced covariance with areas of the default mode network (DMN). During late learning, these effects reversed, with sensorimotor areas now exhibiting increased covariance with DMN areas. However, areas in posteromedial cortex showed the opposite pattern across learning phases, with its connectivity suggesting a role in coordinating activity across different networks over time. Our results establish the neural changes that support reward-based motor learning and identify distinct transitions in the functional coupling of sensorimotor to transmodal cortex when adapting behavior.

Neural Activity in Quarks Language: Lattice Field Theory for a Network of Real Neurons

Brain-computer interfaces have seen extraordinary surges in developments in recent years, and a significant discrepancy now exists between the abundance of available data and the limited headway made in achieving a unified theoretical framework. This discrepancy becomes particularly pronounced when examining the collective neural activity at the micro and meso scale, where a coherent formalization that adequately describes neural interactions is still lacking. Here, we introduce a mathematical framework to analyze systems of natural neurons and interpret the related empirical observations in terms of lattice field theory, an established paradigm from theoretical particle physics and statistical mechanics. Our methods are tailored to interpret data from chronic neural interfaces, especially spike rasters from measurements of single neuron activity, and generalize the maximum entropy model for neural networks so that the time evolution of the system is also taken into account. This is obtained by bridging particle physics and neuroscience, paving the way for particle physics-inspired models of the neocortex.

A combinatorial neural code for long-term motor memory

Motor skill repertoire can be stably retained over long periods, but the neural mechanism underlying stable memory storage remains poorly understood. Moreover, it is unknown how existing motor memories are maintained as new motor skills are continuously acquired. Here we tracked neural representation of learned actions throughout a significant portion of a mouse's lifespan, and we show that learned actions are stably retained in motor memory in combination with context, which protects existing memories from erasure during new motor learning. We used automated home-cage training to establish a continual learning paradigm in which mice learned to perform directional licking in different task contexts. We combined this paradigm with chronic two-photon imaging of motor cortex activity for up to 6 months. Within the same task context, activity driving directional licking was stable over time with little representational drift. When learning new task contexts, new preparatory activity emerged to drive the same licking actions. Learning created parallel new motor memories while retaining the previous memories. Re-learning to make the same actions in the previous task context re-activated the previous preparatory activity, even months later. At the same time, continual learning of new task contexts kept creating new preparatory activity patterns. Context-specific memories, as we observed in the motor system, may provide a solution for stable memory storage throughout continual learning. Learning in new contexts produces parallel new representations instead of modifying existing representations, thus protecting existing motor repertoire from erasure.

A scalable spiking amygdala model that explains fear conditioning, extinction, renewal and generalization

The amygdala (AMY) is widely implicated in fear learning and fear behaviour, but it remains unclear how the many biological components present within AMY interact to achieve these abilities. Building on previous work, we hypothesize that individual AMY nuclei represent different quantities and that fear conditioning arises from error-driven learning on the synapses between AMY nuclei. We present a computational model of AMY that (a) recreates the divisions and connections between AMY nuclei and their constituent pyramidal and inhibitory neurons; (b) accommodates scalable high-dimensional representations of external stimuli; (c) learns to associate complex stimuli with the presence (or absence) of an aversive stimulus; (d) preserves feature information when mapping inputs to salience estimates, such that these estimates generalize to similar stimuli; and (e) induces a diverse profile of neural responses within each nucleus. Our model predicts (1) defensive responses and neural activities in several experimental conditions, (2) the consequence of artificially ablating particular nuclei and (3) the tendency to generalize defensive responses to novel stimuli. We test these predictions by comparing model outputs to neural and behavioural data from animals and humans. Despite the relative simplicity of our model, we find significant overlap between simulated and empirical data, which supports our claim that the model captures many of the neural mechanisms that support fear conditioning. We conclude by comparing our model to other computational models and by characterizing the theoretical relationship between pattern separation and fear generalization in healthy versus anxious individuals.

Increasing Robustness of Intracortical Brain-Computer Interfaces for Recording Condition Changes via Data Augmentation

BACKGROUND AND OBJECTIVE

Intracortical brain-computer interfaces (iBCIs) aim to help paralyzed individuals restore their motor functions by decoding neural activity into intended movement. However, changes in neural recording conditions hinder the decoding performance of iBCIs, mainly because the neural-to-kinematic mappings shift. Conventional approaches involve either training the neural decoders using large datasets before deploying the iBCI or conducting frequent calibrations during its operation. However, collecting data for extended periods can cause user fatigue, negatively impacting the quality and consistency of neural signals. Furthermore, frequent calibration imposes a substantial computational load.

METHODS

This study proposes a novel approach to increase iBCIs' robustness against changing recording conditions. The approach uses three neural augmentation operators to generate augmented neural activity that mimics common recording conditions. Then, contrastive learning is used to learn latent factors by maximizing the similarity between the augmented neural activities. The learned factors are expected to remain stable despite varying recording conditions and maintain a consistent correlation with the intended movement.

RESULTS

Experimental results demonstrate that the proposed iBCI outperformed the state-of-the-art iBCIs and was robust to changing recording conditions across days for long-term use on one publicly available nonhuman primate dataset. It achieved satisfactory offline decoding performance, even when a large training dataset was unavailable.

CONCLUSIONS

This study paves the way for reducing the need for frequent calibration of iBCIs and collecting a large amount of annotated training data. Potential future works aim to improve offline decoding performance with an ultra-small training dataset and improve the iBCIs' robustness to severely disabled electrodes.

Motor cortex latent dynamics encode spatial and temporal arm movement parameters independently

The fluid movement of an arm requires multiple spatiotemporal parameters to be set independently. Recent studies have argued that arm movements are generated by the collective dynamics of neurons in motor cortex. An untested prediction of this hypothesis is that independent parameters of movement must map to independent components of the neural dynamics. Using a task where monkeys made a sequence of reaching movements to randomly placed targets, we show that the spatial and temporal parameters of arm movements are independently encoded in the low-dimensional trajectories of population activity in motor cortex: Each movement's direction corresponds to a fixed neural trajectory through neural state space and its speed to how quickly that trajectory is traversed. Recurrent neural network models show this coding allows independent control over the spatial and temporal parameters of movement by separate network parameters. Our results support a key prediction of the dynamical systems view of motor cortex, but also argue that not all parameters of movement are defined by different trajectories of population activity.

Simultaneous, cortex-wide dynamics of up to 1 million neurons reveal unbounded scaling of dimensionality with neuron number

The brain's remarkable properties arise from the collective activity of millions of neurons. Widespread application of dimensionality reduction to multi-neuron recordings implies that neural dynamics can be approximated by low-dimensional "latent" signals reflecting neural computations. However, can such low-dimensional representations truly explain the vast range of brain activity, and if not, what is the appropriate resolution and scale of recording to capture them? Imaging neural activity at cellular resolution and near-simultaneously across the mouse cortex, we demonstrate an unbounded scaling of dimensionality with neuron number in populations up to 1 million neurons. Although half of the neural variance is contained within sixteen dimensions correlated with behavior, our discovered scaling of dimensionality corresponds to an ever-increasing number of neuronal ensembles without immediate behavioral or sensory correlates. The activity patterns underlying these higher dimensions are fine grained and cortex wide, highlighting that large-scale, cellular-resolution recording is required to uncover the full substrates of neuronal computations.

De novo motor learning creates structure in neural activity that shapes adaptation

Animals can quickly adapt learned movements to external perturbations, and their existing motor repertoire likely influences their ease of adaptation. Long-term learning causes lasting changes in neural connectivity, which shapes the activity patterns that can be produced during adaptation. Here, we examined how a neural population's existing activity patterns, acquired through de novo learning, affect subsequent adaptation by modeling motor cortical neural population dynamics with recurrent neural networks. We trained networks on different motor repertoires comprising varying numbers of movements, which they acquired following various learning experiences. Networks with multiple movements had more constrained and robust dynamics, which were associated with more defined neural 'structure'-organization in the available population activity patterns. This structure facilitated adaptation, but only when the changes imposed by the perturbation were congruent with the organization of the inputs and the structure in neural activity acquired during de novo learning. These results highlight trade-offs in skill acquisition and demonstrate how different learning experiences can shape the geometrical properties of neural population activity and subsequent adaptation.

A brain machine interface framework for exploring proactive control of smart environments

Brain machine interfaces (BMIs) can substantially improve the quality of life of elderly or disabled people. However, performing complex action sequences with a BMI system is onerous because it requires issuing commands sequentially. Fundamentally different from this, we have designed a BMI system that reads out mental planning activity and issues commands in a proactive manner. To demonstrate this, we recorded brain activity from freely-moving monkeys performing an instructed task and decoded it with an energy-efficient, small and mobile field-programmable gate array hardware decoder triggering real-time action execution on smart devices. Core of this is an adaptive decoding algorithm that can compensate for the day-by-day neuronal signal fluctuations with minimal re-calibration effort. We show that open-loop planning-ahead control is possible using signals from primary and pre-motor areas leading to significant time-gain in the execution of action sequences. This novel approach provides, thus, a stepping stone towards improved and more humane control of different smart environments with mobile brain machine interfaces.

Nonlinear manifolds underlie neural population activity during behaviour

There is rich variety in the activity of single neurons recorded during behaviour. Yet, these diverse single neuron responses can be well described by relatively few patterns of neural co-modulation. The study of such low-dimensional structure of neural population activity has provided important insights into how the brain generates behaviour. Virtually all of these studies have used linear dimensionality reduction techniques to estimate these population-wide co-modulation patterns, constraining them to a flat "neural manifold". Here, we hypothesised that since neurons have nonlinear responses and make thousands of distributed and recurrent connections that likely amplify such nonlinearities, neural manifolds should be intrinsically nonlinear. Combining neural population recordings from monkey, mouse, and human motor cortex, and mouse striatum, we show that: 1) neural manifolds are intrinsically nonlinear; 2) their nonlinearity becomes more evident during complex tasks that require more varied activity patterns; and 3) manifold nonlinearity varies across architecturally distinct brain regions. Simulations using recurrent neural network models confirmed the proposed relationship between circuit connectivity and manifold nonlinearity, including the differences across architecturally distinct regions. Thus, neural manifolds underlying the generation of behaviour are inherently nonlinear, and properly accounting for such nonlinearities will be critical as neuroscientists move towards studying numerous brain regions involved in increasingly complex and naturalistic behaviours.

Approximating Nonlinear Functions With Latent Boundaries in Low-Rank Excitatory-Inhibitory Spiking Networks

Deep feedforward and recurrent neural networks have become successful functional models of the brain, but they neglect obvious biological details such as spikes and Dale's law. Here we argue that these details are crucial in order to understand how real neural circuits operate. Towards this aim, we put forth a new framework for spike-based computation in low-rank excitatory-inhibitory spiking networks. By considering populations with rank-1 connectivity, we cast each neuron's spiking threshold as a boundary in a low-dimensional input-output space. We then show how the combined thresholds of a population of inhibitory neurons form a stable boundary in this space, and those of a population of excitatory neurons form an unstable boundary. Combining the two boundaries results in a rank-2 excitatory-inhibitory (EI) network with inhibition-stabilized dynamics at the intersection of the two boundaries. The computation of the resulting networks can be understood as the difference of two convex functions and is thereby capable of approximating arbitrary non-linear input-output mappings. We demonstrate several properties of these networks, including noise suppression and amplification, irregular activity and synaptic balance, as well as how they relate to rate network dynamics in the limit that the boundary becomes soft. Finally, while our work focuses on small networks (5-50 neurons), we discuss potential avenues for scaling up to much larger networks. Overall, our work proposes a new perspective on spiking networks that may serve as a starting point for a mechanistic understanding of biological spike-based computation.

A theory of brain-computer interface learning via low-dimensional control

A remarkable demonstration of the flexibility of mammalian motor systems is primates' ability to learn to control brain-computer interfaces (BCIs). This constitutes a completely novel motor behavior, yet primates are capable of learning to control BCIs under a wide range of conditions. BCIs with carefully calibrated decoders, for example, can be learned with only minutes to hours of practice. With a few weeks of practice, even BCIs with randomly constructed decoders can be learned. What are the biological substrates of this learning process? Here, we develop a theory based on a re-aiming strategy, whereby learning operates within a low-dimensional subspace of task-relevant inputs driving the local population of recorded neurons. Through comprehensive numerical and formal analysis, we demonstrate that this theory can provide a unifying explanation for disparate phenomena previously reported in three different BCI learning tasks, and we derive a novel experimental prediction that we verify with previously published data. By explicitly modeling the underlying neural circuitry, the theory reveals an interpretation of these phenomena in terms of biological constraints on neural activity.

Topological structure of population activity in mouse visual cortex encodes densely sampled stimulus rotations

The primary visual cortex is one of the most well understood regions supporting the processing involved in sensory computation. Following the popularization of high-density neural recordings, it has been observed that the activity of large neural populations is often constrained to low dimensional manifolds. In this work, we quantify the structure of such neural manifolds in the visual cortex. We do this by analyzing publicly available two-photon optical recordings of mouse primary visual cortex in response to visual stimuli with a densely sampled rotation angle. Using a geodesic metric along with persistent homology, we discover that population activity in response to such stimuli generates a circular manifold, encoding the angle of rotation. Furthermore, we observe that this circular manifold is expressed differently in subpopulations of neurons with differing orientation and direction selectivity. Finally, we discuss some of the obstacles to reliably retrieving the truthful topology generated by a neural population.

Encoding of Visual Objects in the Human Medial Temporal Lobe

The human medial temporal lobe (MTL) plays a crucial role in recognizing visual objects, a key cognitive function that relies on the formation of semantic representations. Nonetheless, it remains unknown how visual information of general objects is translated into semantic representations in the MTL. Furthermore, the debate about whether the human MTL is involved in perception has endured for a long time. To address these questions, we investigated three distinct models of neural object coding-semantic coding, axis-based feature coding, and region-based feature coding-in each subregion of the human MTL, using high-resolution fMRI in two male and six female participants. Our findings revealed the presence of semantic coding throughout the MTL, with a higher prevalence observed in the parahippocampal cortex (PHC) and perirhinal cortex (PRC), while axis coding and region coding were primarily observed in the earlier regions of the MTL. Moreover, we demonstrated that voxels exhibiting axis coding supported the transition to region coding and contained information relevant to semantic coding. Together, by providing a detailed characterization of neural object coding schemes and offering a comprehensive summary of visual coding information for each MTL subregion, our results not only emphasize a clear role of the MTL in perceptual processing but also shed light on the translation of perception-driven representations of visual features into memory-driven representations of semantics along the MTL processing pathway.

What does the mean mean? A simple test for neuroscience

Trial-averaged metrics, e.g. tuning curves or population response vectors, are a ubiquitous way of characterizing neuronal activity. But how relevant are such trial-averaged responses to neuronal computation itself? Here we present a simple test to estimate whether average responses reflect aspects of neuronal activity that contribute to neuronal processing. The test probes two assumptions implicitly made whenever average metrics are treated as meaningful representations of neuronal activity: Reliability: Neuronal responses repeat consistently enough across trials that they convey a recognizable reflection of the average response to downstream regions.Behavioural relevance: If a single-trial response is more similar to the average template, it is more likely to evoke correct behavioural responses. We apply this test to two data sets: (1) Two-photon recordings in primary somatosensory cortices (S1 and S2) of mice trained to detect optogenetic stimulation in S1; and (2) Electrophysiological recordings from 71 brain areas in mice performing a contrast discrimination task. Under the highly controlled settings of Data set 1, both assumptions were largely fulfilled. In contrast, the less restrictive paradigm of Data set 2 met neither assumption. Simulations predict that the larger diversity of neuronal response preferences, rather than higher cross-trial reliability, drives the better performance of Data set 1. We conclude that when behaviour is less tightly restricted, average responses do not seem particularly relevant to neuronal computation, potentially because information is encoded more dynamically. Most importantly, we encourage researchers to apply this simple test of computational relevance whenever using trial-averaged neuronal metrics, in order to gauge how representative cross-trial averages are in a given context.

High-density transparent graphene arrays for predicting cellular calcium activity at depth from surface potential recordings

Optically transparent neural microelectrodes have facilitated simultaneous electrophysiological recordings from the brain surface with the optical imaging and stimulation of neural activity. A remaining challenge is to scale down the electrode dimensions to the single-cell size and increase the density to record neural activity with high spatial resolution across large areas to capture nonlinear neural dynamics. Here we developed transparent graphene microelectrodes with ultrasmall openings and a large, transparent recording area without any gold extensions in the field of view with high-density microelectrode arrays up to 256 channels. We used platinum nanoparticles to overcome the quantum capacitance limit of graphene and to scale down the microelectrode diameter to 20 µm. An interlayer-doped double-layer graphene was introduced to prevent open-circuit failures. We conducted multimodal experiments, combining the recordings of cortical potentials of microelectrode arrays with two-photon calcium imaging of the mouse visual cortex. Our results revealed that visually evoked responses are spatially localized for high-frequency bands, particularly for the multiunit activity band. The multiunit activity power was found to be correlated with cellular calcium activity. Leveraging this, we employed dimensionality reduction techniques and neural networks to demonstrate that single-cell and average calcium activities can be decoded from surface potentials recorded by high-density transparent graphene arrays.

Motor cortex retains and reorients neural dynamics during motor imagery

The most prominent characteristic of motor cortex is its activation during movement execution, but it is also active when we simply imagine movements in the absence of actual motor output. Despite decades of behavioural and imaging studies, it is unknown how the specific activity patterns and temporal dynamics in motor cortex during covert motor imagery relate to those during motor execution. Here we recorded intracortical activity from the motor cortex of two people who retain some residual wrist function following incomplete spinal cord injury as they performed both actual and imagined isometric wrist extensions. We found that we could decompose the population activity into three orthogonal subspaces, where one was similarly active during both action and imagery, and the others were active only during a single task type-action or imagery. Although they inhabited orthogonal neural dimensions, the action-unique and imagery-unique subspaces contained a strikingly similar set of dynamic features. Our results suggest that during motor imagery, motor cortex maintains the same overall population dynamics as during execution by reorienting the components related to motor output and/or feedback into a unique, output-null imagery subspace.

Preparatory activity and the expansive null-space

The study of the cortical control of movement experienced a conceptual shift over recent decades, as the basic currency of understanding shifted from single-neuron tuning towards population-level factors and their dynamics. This transition was informed by a maturing understanding of recurrent networks, where mechanism is often characterized in terms of population-level factors. By estimating factors from data, experimenters could test network-inspired hypotheses. Central to such hypotheses are 'output-null' factors that do not directly drive motor outputs yet are essential to the overall computation. In this Review, we highlight how the hypothesis of output-null factors was motivated by the venerable observation that motor-cortex neurons are active during movement preparation, well before movement begins. We discuss how output-null factors then became similarly central to understanding neural activity during movement. We discuss how this conceptual framework provided key analysis tools, making it possible for experimenters to address long-standing questions regarding motor control. We highlight an intriguing trend: as experimental and theoretical discoveries accumulate, the range of computational roles hypothesized to be subserved by output-null factors continues to expand.