Neural population dynamics during reaching

Contrastive learning of shared spatiotemporal EEG representations across individuals for naturalistic neuroscience

Neural representations induced by naturalistic stimuli offer insights into how humans respond to stimuli in daily life. Understanding neural mechanisms underlying naturalistic stimuli processing hinges on the precise identification and extraction of the shared neural patterns that are consistently present across individuals. Targeting the Electroencephalogram (EEG) technique, known for its rich spatial and temporal information, this study presents a framework for Contrastive Learning of Shared SpatioTemporal EEG Representations across individuals (CL-SSTER). CL-SSTER utilizes contrastive learning to maximize the similarity of EEG representations across individuals for identical stimuli, contrasting with those for varied stimuli. The network employs spatial and temporal convolutions to simultaneously learn the spatial and temporal patterns inherent in EEG. The versatility of CL-SSTER was demonstrated on three EEG datasets, including a synthetic dataset, a natural speech comprehension EEG dataset, and an emotional video watching EEG dataset. CL-SSTER attained the highest inter-subject correlation (ISC) values compared to the state-of-the-art ISC methods. The latent representations generated by CL-SSTER exhibited reliable spatiotemporal EEG patterns, which can be explained by properties of the naturalistic stimuli. CL-SSTER serves as an interpretable and scalable framework for the identification of inter-subject shared neural representations in naturalistic neuroscience.

Convergent representation of values from tactile and visual inputs for efficient goal-directed behavior in the primate putamen

Animals can discriminate diverse sensory values with a limited number of neurons, raising questions about how the brain utilizes neural resources to efficiently process multi-dimensional inputs for decision-making. Here, we demonstrate that this efficiency is achieved by reducing sensory dimensions and converging towards the value dimension essential for goal-directed behavior in the putamen. Humans and monkeys performed tactile and visual value discrimination tasks while their neural responses were examined. Value information, whether originating from tactile or visual stimuli, was found to be processed within the human putamen using fMRI. Notably, at the single-neuron level in the macaque putamen, half of the individual neurons encode values independently of sensory inputs, while the other half selectively encode tactile or visual value. The responses of bimodal value neurons correlate with value-guided finger insertion behavior in both tasks, whereas modality-selective value neurons show task-specific correlations. Simulation using these neurons reveals that the presence of bimodal value neurons enables value discrimination with a significantly reduced number of neurons compared to simulations without them. Our data indicate that individual neurons in the primate putamen process different values in a convergent manner, thereby facilitating the efficient use of constrained neural resources for value-guided behavior.

Global motor dynamics - Invariant neural representations of motor behavior in distributed brain-wide recordings

Objective.Motor-related neural activity is more widespread than previously thought, as pervasive brain-wide neural correlates of motor behavior have been reported in various animal species. Brain-wide movement-related neural activity have been observed in individual brain areas in humans as well, but it is unknown to what extent global patterns exist.Approach.Here, we use a decoding approach to capture and characterize brain-wide neural correlates of movement. We recorded invasive electrophysiological data from stereotactic electroencephalographic electrodes implanted in eight epilepsy patients who performed both an executed and imagined grasping task. Combined, these electrodes cover the whole brain, including deeper structures such as the hippocampus, insula and basal ganglia. We extract a low-dimensional representation and classify movement from rest trials using a Riemannian decoder.Main results.We reveal global neural dynamics that are predictive across tasks and participants. Using an ablation analysis, we demonstrate that these dynamics remain remarkably stable under loss of information. Similarly, the dynamics remain stable across participants, as we were able to predict movement across participants using transfer learning.Significance.Our results show that decodable global motor-related neural dynamics exist within a low-dimensional space. The dynamics are predictive of movement, nearly brain-wide and present in all our participants. The results broaden the scope to brain-wide investigations, and may allow combining datasets of multiple participants with varying electrode locations or calibrationless neural decoder.

Changes in cortical manifold structure following stroke and its relation to behavioral recovery in the male macaque

Stroke, a major cause of disability, disrupts brain function and motor skills. Previous research has mainly focused on reorganization of the motor system post-stroke, but the effects on other brain areas and their influence on recovery is poorly understood. Here, we use functional neuroimaging in a nonhuman primate model (23 male Cynomolgus Macaques), we explore how ischemic stroke affects whole-brain cortical architecture and its relation to spontaneous behavioral recovery. By projecting patterns of cortical functional connectivity onto a low-dimensional manifold space, we find that several regions in both sensorimotor cortex and higher-order transmodal cortex exhibit significant shifts in their manifold embedding from pre- to post-stroke. Furthermore, we observe that changes in default mode and limbic network regions, and not preserved sensorimotor cortical regions, are associated with animal behavioral recovery post-stroke. These results establish the whole-brain functional changes associated with stroke, and suggest an important role for higher-order transmodal cortex in post-stroke outcomes.

Decoding the brain: From neural representations to mechanistic models

A central principle in neuroscience is that neurons within the brain act in concert to produce perception, cognition, and adaptive behavior. Neurons are organized into specialized brain areas, dedicated to different functions to varying extents, and their function relies on distributed circuits to continuously encode relevant environmental and body-state features, enabling other areas to decode (interpret) these representations for computing meaningful decisions and executing precise movements. Thus, the distributed brain can be thought of as a series of computations that act to encode and decode information. In this perspective, we detail important concepts of neural encoding and decoding and highlight the mathematical tools used to measure them, including deep learning methods. We provide case studies where decoding concepts enable foundational and translational science in motor, visual, and language processing.

Accurate neural control of a hand prosthesis by posture-related activity in the primate grasping circuit

Brain-computer interfaces (BCIs) have the potential to restore hand movement for people with paralysis, but current devices still lack the fine control required to interact with objects of daily living. Following our understanding of cortical activity during arm reaches, hand BCI studies have focused primarily on velocity control. However, mounting evidence suggests that posture, and not velocity, dominates in hand-related areas. To explore whether this signal can causally control a prosthesis, we developed a BCI training paradigm centered on the reproduction of posture transitions. Monkeys trained with this protocol were able to control a multidimensional hand prosthesis with high accuracy, including execution of the very intricate precision grip. Analysis revealed that the posture signal in the target grasping areas was the main contributor to control. We present, for the first time, neural posture control of a multidimensional hand prosthesis, opening the door for future interfaces to leverage this additional information channel.

Emerging perspectives for the study of the neural basis of motor behaviour

The 33rd Annual Meeting of the Society for the Neural Control of Movement (NCM) brought together over 500 experts to discuss recent advancements in motor control. This article highlights key topics from the conference, including the foundational mechanisms of motor control, the ongoing debate over the context-dependency of feedforward and feedback processes, and the interplay between motor and cognitive functions in learning, memory, and decision-making. It also presents innovative methods for studying movement in complex, real-world environments.

When and why does motor preparation arise in recurrent neural network models of motor control?

During delayed ballistic reaches, motor areas consistently display movement-specific activity patterns prior to movement onset. It is unclear why these patterns arise: while they have been proposed to seed an initial neural state from which the movement unfolds, recent experiments have uncovered the presence and necessity of ongoing inputs during movement, which may lessen the need for careful initialization. Here, we modeled the motor cortex as an input-driven dynamical system, and we asked what the optimal way to control this system to perform fast delayed reaches is. We find that delay-period inputs consistently arise in an optimally controlled model of M1. By studying a variety of network architectures, we could dissect and predict the situations in which it is beneficial for a network to prepare. Finally, we show that optimal input-driven control of neural dynamics gives rise to multiple phases of preparation during reach sequences, providing a novel explanation for experimentally observed features of monkey M1 activity in double reaching.

When preparation pays off

Computational principles shed light on why movement is preceded by preparatory activity within the neural networks that control muscles.

Comparison of orientation encoding across layers within single columns of primate V1 revealed by high-density recordings

Primary visual cortex (V1) has been the focus of extensive neurophysiological investigations, with its laminar organization serving as a crucial model for understanding the functional logic of neocortical microcircuits. Utilizing newly developed high-density, Neuropixels probes, we measured visual responses from large populations of simultaneously recorded neurons distributed across layers of macaque V1. Within single recordings, myriad differences in the functional properties of neuronal subpopulations could be observed. Notably, while standard measurements of orientation selectivity showed only minor differences between laminar compartments, decoding stimulus orientation from layer 4C responses outperformed both superficial and deep layers within the same cortical column. The superior orientation discrimination within layer 4C was associated with greater response reliability of individual neurons rather than lower correlated activity within neuronal populations. Our results underscore the efficacy of high-density electrophysiology in revealing the functional organization and network properties of neocortical microcircuits within single experiments.

High-dimensional encoding of movement by single neurons in basal ganglia output

The substantia nigra pars reticulata (SNpr), an output structure of the basal ganglia, is hypothesized to gate movement execution. Previous studies in the eye movement system focusing mostly on saccades have reported that SNpr neurons are tonically active and either pause or increase their firing during movements, consistent with the gating role. We recorded activity in the SNpr of two monkeys during smooth pursuit and saccadic eye movements. SNpr neurons exhibited highly diverse reaction patterns during pursuit, including frequent increases and decreases in firing rate, uncorrelated responses in different movement directions and in reward conditions that resulted in the high dimensional activity of single neurons. These diverse temporal patterns surpassed those in other oculomotor areas in the medial-temporal cortex, frontal cortex, basal ganglia, and cerebellum. These findings suggest that temporal properties of the responses enrich the coding capacity of the basal ganglia output beyond gating or permitting movement.

A Perspective on Neuroscience Data Standardization with Neurodata Without Borders

Neuroscience research has evolved to generate increasingly large and complex experimental data sets, and advanced data science tools are taking on central roles in neuroscience research. Neurodata Without Borders (NWB), a standard language for neurophysiology data, has recently emerged as a powerful solution for data management, analysis, and sharing. We here discuss our labs' efforts to implement NWB data science pipelines. We describe general principles and specific use cases that illustrate successes, challenges, and non-trivial decisions in software engineering. We hope that our experience can provide guidance for the neuroscience community and help bridge the gap between experimental neuroscience and data science. Key takeaways from this article are that (1) standardization with NWB requires non-trivial design choices; (2) the general practice of standardization in the lab promotes data awareness and literacy, and improves transparency, rigor, and reproducibility in our science; (3) we offer several feature suggestions to ease the extensibility, publishing/sharing, and usability for NWB standard and users of NWB data.

Electrical Signaling Beyond Neurons

Neural action potentials (APs) are difficult to interpret as signal encoders and/or computational primitives. Their relationships with stimuli and behaviors are obscured by the staggering complexity of nervous systems themselves. We can reduce this complexity by observing that "simpler" neuron-less organisms also transduce stimuli into transient electrical pulses that affect their behaviors. Without a complicated nervous system, APs are often easier to understand as signal/response mechanisms. We review examples of nonneural stimulus transductions in domains of life largely neglected by theoretical neuroscience: bacteria, protozoans, plants, fungi, and neuron-less animals. We report properties of those electrical signals-for example, amplitudes, durations, ionic bases, refractory periods, and particularly their ecological purposes. We compare those properties with those of neurons to infer the tasks and selection pressures that neurons satisfy. Throughout the tree of life, nonneural stimulus transductions time behavioral responses to environmental changes. Nonneural organisms represent the presence or absence of a stimulus with the presence or absence of an electrical signal. Their transductions usually exhibit high sensitivity and specificity to a stimulus, but are often slow compared to neurons. Neurons appear to be sacrificing the specificity of their stimulus transductions for sensitivity and speed. We interpret cellular stimulus transductions as a cell's assertion that it detected something important at that moment in time. In particular, we consider neural APs as fast but noisy detection assertions. We infer that a principal goal of nervous systems is to detect extremely weak signals from noisy sensory spikes under enormous time pressure. We discuss neural computation proposals that address this goal by casting neurons as devices that implement online, analog, probabilistic computations with their membrane potentials. Those proposals imply a measurable relationship between afferent neural spiking statistics and efferent neural membrane electrophysiology.

A computational mechanism of cue-stimulus integration for pain in the brain

The brain integrates information from pain-predictive cues and noxious inputs to construct the pain experience. Although previous studies have identified neural encodings of individual pain components, how they are integrated remains elusive. Here, using a cue-induced pain task, we examined temporal functional magnetic resonance imaging activities within the state space, where axes represent individual voxel activities. By analyzing the features of these activities at the large-scale network level, we demonstrated that overall brain networks preserve both cue and stimulus information in their respective subspaces within the state space. However, only higher-order brain networks, including limbic and default mode networks, could reconstruct the pattern of participants' reported pain by linear summation of subspace activities, providing evidence for the integration of cue and stimulus information. These results suggest a hierarchical organization of the brain for processing pain components and elucidate the mechanism for their integration underlying our pain perception.

Neuroelectrophysiology-compatible electrolytic lesioning

Lesion studies have historically been instrumental for establishing causal connections between brain and behavior. They stand to provide additional insight if integrated with multielectrode techniques common in systems neuroscience. Here, we present and test a platform for creating electrolytic lesions through chronically implanted, intracortical multielectrode probes without compromising the ability to acquire neuroelectrophysiology. A custom-built current source provides stable current and allows for controlled, repeatable lesions in awake-behaving animals. Performance of this novel lesioning technique was validated using histology from ex vivo and in vivo testing, current and voltage traces from the device, and measurements of spiking activity before and after lesioning. This electrolytic lesioning method avoids disruptive procedures, provides millimeter precision over the extent and submillimeter precision over the location of the injury, and permits electrophysiological recording of single-unit activity from the remaining neuronal population after lesioning. This technique can be used in many areas of cortex, in several species, and theoretically with any multielectrode probe. The low-cost, external lesioning device can also easily be adopted into an existing electrophysiology recording setup. This technique is expected to enable future causal investigations of the recorded neuronal population's role in neuronal circuit function, while simultaneously providing new insight into local reorganization after neuron loss.

Automated customization of large-scale spiking network models to neuronal population activity

Understanding brain function is facilitated by constructing computational models that accurately reproduce aspects of brain activity. Networks of spiking neurons capture the underlying biophysics of neuronal circuits, yet their activity's dependence on model parameters is notoriously complex. As a result, heuristic methods have been used to configure spiking network models, which can lead to an inability to discover activity regimes complex enough to match large-scale neuronal recordings. Here we propose an automatic procedure, Spiking Network Optimization using Population Statistics (SNOPS), to customize spiking network models that reproduce the population-wide covariability of large-scale neuronal recordings. We first confirmed that SNOPS accurately recovers simulated neural activity statistics. Then, we applied SNOPS to recordings in macaque visual and prefrontal cortices and discovered previously unknown limitations of spiking network models. Taken together, SNOPS can guide the development of network models, thereby enabling deeper insight into how networks of neurons give rise to brain function.

Dynamic control of sequential retrieval speed in networks with heterogeneous learning rules

Temporal rescaling of sequential neural activity has been observed in multiple brain areas during behaviors involving time estimation and motor execution at variable speeds. Temporally asymmetric Hebbian rules have been used in network models to learn and retrieve sequential activity, with characteristics that are qualitatively consistent with experimental observations. However, in these models sequential activity is retrieved at a fixed speed. Here, we investigate the effects of a heterogeneity of plasticity rules on network dynamics. In a model in which neurons differ by the degree of temporal symmetry of their plasticity rule, we find that retrieval speed can be controlled by varying external inputs to the network. Neurons with temporally symmetric plasticity rules act as brakes and tend to slow down the dynamics, while neurons with temporally asymmetric rules act as accelerators of the dynamics. We also find that such networks can naturally generate separate 'preparatory' and 'execution' activity patterns with appropriate external inputs.

Motor Cortex Latent Dynamics Encode Spatial and Temporal Arm Movement Parameters Independently

The fluid movement of an arm requires multiple spatiotemporal parameters to be set independently. Recent studies have argued that arm movements are generated by the collective dynamics of neurons in motor cortex. An untested prediction of this hypothesis is that independent parameters of movement must map to independent components of the neural dynamics. Using a task where three male monkeys made a sequence of reaching movements to randomly placed targets, we show that the spatial and temporal parameters of arm movements are independently encoded in the low-dimensional trajectories of population activity in motor cortex: each movement's direction corresponds to a fixed neural trajectory through neural state space and its speed to how quickly that trajectory is traversed. Recurrent neural network models show that this coding allows independent control over the spatial and temporal parameters of movement by separate network parameters. Our results support a key prediction of the dynamical systems view of motor cortex, and also argue that not all parameters of movement are defined by different trajectories of population activity.

Transitions in dynamical regime and neural mode underlie perceptual decision-making

Perceptual decision-making is the process by which an animal uses sensory stimuli to choose an action or mental proposition. This process is thought to be mediated by neurons organized as attractor networks 1,2 . However, whether attractor dynamics underlie decision behavior and the complex neuronal responses remains unclear. Here we use an unsupervised, deep learning-based method to discover decision-related dynamics from the simultaneous activity of neurons in frontal cortex and striatum of rats while they accumulate pulsatile auditory evidence. We found that trajectories evolved along two sequential regimes, the first dominated by sensory inputs, and the second dominated by the autonomous dynamics, with flow in a direction (i.e., "neural mode") largely orthogonal to that in the first regime. We propose that the second regime corresponds to decision commitment. We developed a simplified model that approximates the coupled transition in dynamics and neural mode and allows precise inference, from each trial's neural activity, of a putative internal decision commitment time in that trial. The simplified model captures diverse and complex single-neuron temporal profiles, such as ramping and stepping 3-5 . It also captures trial-averaged curved trajectories 6-8 , and reveals distinctions between brain regions. The putative neurally-inferred commitment times ("nTc") occurred at times broadly distributed across trials, and not time-locked to stimulus onset, offset, or response onset. Nevertheless, when trials were aligned to nTc, behavioral analysis showed that, as predicted by a decision commitment time, sensory evidence before nTc affected the subjects' decision, but evidence after nTc did not. Our results show that the formation of a perceptual choice involves a rapid, coordinated transition in both the dynamical regime and the neural mode of the decision process, and suggest the moment of commitment to be a useful entry point for dissecting mechanisms underlying rapid changes in internal state.

An output-null signature of inertial load in motor cortex

Coordinated movement requires the nervous system to continuously compensate for changes in mechanical load across different conditions. For voluntary movements like reaching, the motor cortex is a critical hub that generates commands to move the limbs and counteract loads. How does cortex contribute to load compensation when rhythmic movements are sequenced by a spinal pattern generator? Here, we address this question by manipulating the mass of the forelimb in unrestrained mice during locomotion. While load produces changes in motor output that are robust to inactivation of motor cortex, it also induces a profound shift in cortical dynamics. This shift is minimally affected by cerebellar perturbation and significantly larger than the load response in the spinal motoneuron population. This latent representation may enable motor cortex to generate appropriate commands when a voluntary movement must be integrated with an ongoing, spinally-generated rhythm.

Selective direct motor cortical influence during naturalistic climbing

It remains poorly resolved when and how motor cortical output directly influences limb muscle activity through descending projections, which impedes mechanistic understanding of cortical movement control. Here we addressed this in mice performing an ethologically inspired all-limb climbing behavior. We quantified the direct influence of forelimb primary motor cortex (caudal forelimb area, CFA) on muscle activity comprehensively across the muscle activity states that occur during climbing. We found that CFA informs muscle activity pattern, mainly by selectively activating certain muscles while exerting much smaller, bidirectional effects on their antagonists. From Neuropixel recordings, we identified linear combinations (components) of motor cortical activity that covary with these effects, finding that these components differ from those that covary with muscle activity or kinematics. Collectively, our results reveal an instructive direct motor cortical influence on limb muscles that is selective within a motor behavior and reliant on a new type of neural activity subspace.

Propofol anesthesia destabilizes neural dynamics across cortex

Every day, hundreds of thousands of people undergo general anesthesia. One hypothesis is that anesthesia disrupts dynamic stability-the ability of the brain to balance excitability with the need to be stable and controllable. To test this hypothesis, we developed a method for quantifying changes in population-level dynamic stability in complex systems: delayed linear analysis for stability estimation (DeLASE). Propofol was used to transition animals between the awake state and anesthetized unconsciousness. DeLASE was applied to macaque cortex local field potentials (LFPs). We found that neural dynamics were more unstable in unconsciousness compared with the awake state. Cortical trajectories mirrored predictions from destabilized linear systems. We mimicked the effect of propofol in simulated neural networks by increasing inhibitory tone. This in turn destabilized the networks, as observed in the neural data. Our results suggest that anesthesia disrupts dynamical stability that is required for consciousness.

Reach-dependent reorientation of rotational dynamics in motor cortex

During reaching, neurons in motor cortex exhibit complex, time-varying activity patterns. Though single-neuron activity correlates with movement parameters, movement correlations explain neural activity only partially. Neural responses also reflect population-level dynamics thought to generate outputs. These dynamics have previously been described as "rotational," such that activity orbits in neural state space. Here, we reanalyze reaching datasets from male Rhesus macaques and find two essential features that cannot be accounted for with standard dynamics models. First, the planes in which rotations occur differ for different reaches. Second, this variation in planes reflects the overall location of activity in neural state space. Our "location-dependent rotations" model fits nearly all motor cortex activity during reaching, and high-quality decoding of reach kinematics reveals a quasilinear relationship with spiking. Varying rotational planes allows motor cortex to produce richer outputs than possible under previous models. Finally, our model links representational and dynamical ideas: representation is present in the state space location, which dynamics then convert into time-varying command signals.

Revealing unexpected complex encoding but simple decoding mechanisms in motor cortex via separating behaviorally relevant neural signals

In motor cortex, behaviorally relevant neural responses are entangled with irrelevant signals, which complicates the study of encoding and decoding mechanisms. It remains unclear whether behaviorally irrelevant signals could conceal some critical truth. One solution is to accurately separate behaviorally relevant and irrelevant signals at both single-neuron and single-trial levels, but this approach remains elusive due to the unknown ground truth of behaviorally relevant signals. Therefore, we propose a framework to define, extract, and validate behaviorally relevant signals. Analyzing separated signals in three monkeys performing different reaching tasks, we found neural responses previously considered to contain little information actually encode rich behavioral information in complex nonlinear ways. These responses are critical for neuronal redundancy and reveal movement behaviors occupy a higher-dimensional neural space than previously expected. Surprisingly, when incorporating often-ignored neural dimensions, behaviorally relevant signals can be decoded linearly with comparable performance to nonlinear decoding, suggesting linear readout may be performed in motor cortex. Our findings prompt that separating behaviorally relevant signals may help uncover more hidden cortical mechanisms.

Flexible gating between subspaces in a neural network model of internally guided task switching

Behavioral flexibility relies on the brain's ability to switch rapidly between multiple tasks, even when the task rule is not explicitly cued but must be inferred through trial and error. The underlying neural circuit mechanism remains poorly understood. We investigated recurrent neural networks (RNNs) trained to perform an analog of the classic Wisconsin Card Sorting Test. The networks consist of two modules responsible for rule representation and sensorimotor mapping, respectively, where each module is comprised of a circuit with excitatory neurons and three major types of inhibitory neurons. We found that rule representation by self-sustained persistent activity across trials, error monitoring and gated sensorimotor mapping emerged from training. Systematic dissection of trained RNNs revealed a detailed circuit mechanism that is consistent across networks trained with different hyperparameters. The networks' dynamical trajectories for different rules resided in separate subspaces of population activity; the subspaces collapsed and performance was reduced to chance level when dendrite-targeting somatostatin-expressing interneurons were silenced, illustrating how a phenomenological description of representational subspaces is explained by a specific circuit mechanism.

Initial and corrective submovement encoding differences within primary motor cortex during precision reaching

Precision reaching often requires corrective submovements to obtain the desired goal. Most studies of reaching have focused on single initial movements, and implied the cortical encoding model was the same for all submovements. However, corrective submovements may show different encoding patterns from the initial submovement with distinct patterns of activation across the population. Two rhesus macaques performed a precision center-out-task with small targets. Neural activity from single units in the primary motor cortex and associated behavioral data were recorded to evaluate movement characteristics. Neural population data and individual neuronal firing rates identified with a peak finding algorithm to identify peaks in hand speed were examined for encoding differences between initial and corrective submovements. Individual neurons were fitted with a regression model that included the reach vector, position, and speed to predict firing rate. For both initial and corrective submovements, the largest effect remained movement direction. We observed a large subset changed their preferred direction greater than 45° between initial and corrective submovements. Neuronal depth of modulation also showed considerable variation when adjusted for movement speed. By using principal component analysis, neural trajectories of initial and corrective submovements progressed through different neural subspaces. These findings all suggest that different neural encoding patterns exist for initial and corrective submovements within the cortex. We hypothesize that this variation in how neurons change to encode small, corrective submovements might allow for a larger portion of the neural space being used to encode a greater range of movements with varying amplitudes and levels of precision.NEW & NOTEWORTHY Neuronal recordings matched with kinematic behavior were collected in a precision center-out task that often required corrective movements. We reveal large differences in preferred direction and depth of modulation between initial and corrective submovements across the neural population. We then present a model of the neural population describing how these shifts in tuning create different subspaces for signaling initial and corrective movements likely to improve motor precision.

Multiunit Frontal Eye Field Activity Codes the Visuomotor Transformation, But Not Gaze Prediction or Retrospective Target Memory, in a Delayed Saccade Task

Single-unit (SU) activity-action potentials isolated from one neuron-has traditionally been employed to relate neuronal activity to behavior. However, recent investigations have shown that multiunit (MU) activity-ensemble neural activity recorded within the vicinity of one microelectrode-may also contain accurate estimations of task-related neural population dynamics. Here, using an established model-fitting approach, we compared the spatial codes of SU response fields with corresponding MU response fields recorded from the frontal eye fields (FEFs) in head-unrestrained monkeys (Macaca mulatta) during a memory-guided saccade task. Overall, both SU and MU populations showed a simple visuomotor transformation: the visual response coded target-in-eye coordinates, transitioning progressively during the delay toward a future gaze-in-eye code in the saccade motor response. However, the SU population showed additional secondary codes, including a predictive gaze code in the visual response and retention of a target code in the motor response. Further, when SUs were separated into regular/fast spiking neurons, these cell types showed different spatial code progressions during the late delay period, only converging toward gaze coding during the final saccade motor response. Finally, reconstructing MU populations (by summing SU data within the same sites) failed to replicate either the SU or MU pattern. These results confirm the theoretical and practical potential of MU activity recordings as a biomarker for fundamental sensorimotor transformations (e.g., target-to-gaze coding in the oculomotor system), while also highlighting the importance of SU activity for coding more subtle (e.g., predictive/memory) aspects of sensorimotor behavior.

Dissociable roles of thalamic nuclei in the refinement of reaches to spatial targets

Reaches are complex movements that are critical for survival, and encompass the control of different aspects such as direction, speed, and endpoint precision. Complex movements have been postulated to be learned and controlled through distributed motor networks, of which the thalamus is a highly connected node. Still, the role of different thalamic circuits in learning and controlling specific aspects of reaches has not been investigated. We report dissociable roles of two distinct thalamic nuclei - the parafascicular (Pf) and ventroanterior/ventrolateral (VAL) nuclei - in the refinement of spatial target reaches in mice. Using 2-photon calcium imaging in a head-fixed joystick task where mice learned to reach to a target in space, we found that glutamatergic neurons in both areas were most active during reaches early in learning. Reach-related activity in both areas decreased late in learning, as movement direction was refined and reaches increased in accuracy. Furthermore, the population dynamics of Pf, but not VAL, covaried in different subspaces in early and late learning, but eventually stabilized in late learning. The neural activity in Pf, but not VAL, encoded the direction of reaches in early but not late learning. Accordingly, bilateral lesions of Pf before, but not after learning, strongly and specifically impaired the refinement of reach direction. VAL lesions did not impact direction refinement, but instead resulted in increased speed and target overshoot. Our findings provide new evidence that the thalamus is a critical motor node in the learning and control of reaching movements, with specific subnuclei controlling distinct aspects of the reach early in learning.

Irrational choices via a curvilinear representational geometry for value

We make decisions by comparing values, but it is not yet clear how value is represented in the brain. Many models assume, if only implicitly, that the representational geometry of value is linear. However, in part due to a historical focus on noisy single neurons, rather than neuronal populations, this hypothesis has not been rigorously tested. Here, we examine the representational geometry of value in the ventromedial prefrontal cortex (vmPFC), a part of the brain linked to economic decision-making, in two male rhesus macaques. We find that values are encoded along a curved manifold in vmPFC. This curvilinear geometry predicts a specific pattern of irrational decision-making: that decision-makers will make worse choices when an irrelevant, decoy option is worse in value, compared to when it is better. We observe this type of irrational choices in behavior. Together, these results not only suggest that the representational geometry of value is nonlinear, but that this nonlinearity could impose bounds on rational decision-making.

Flexible neural population dynamics govern the speed and stability of sensory encoding in mouse visual cortex

Time courses of neural responses underlie real-time sensory processing and perception. How these temporal dynamics change may be fundamental to how sensory systems adapt to different perceptual demands. By simultaneously recording from hundreds of neurons in mouse primary visual cortex, we examined neural population responses to visual stimuli at sub-second timescales, during different behavioural states. We discovered that during active behavioural states characterised by locomotion, single-neurons shift from transient to sustained response modes, facilitating rapid emergence of visual stimulus tuning. Differences in single-neuron response dynamics were associated with changes in temporal dynamics of neural correlations, including faster stabilisation of stimulus-evoked changes in the structure of correlations during locomotion. Using Factor Analysis, we examined temporal dynamics of latent population responses and discovered that trajectories of population activity make more direct transitions between baseline and stimulus-encoding neural states during locomotion. This could be partly explained by dampening of oscillatory dynamics present during stationary behavioural states. Functionally, changes in temporal response dynamics collectively enabled faster, more stable and more efficient encoding of new visual information during locomotion. These findings reveal a principle of how sensory systems adapt to perceptual demands, where flexible neural population dynamics govern the speed and stability of sensory encoding.

Context-dependence of deterministic and nondeterministic contributions to closed-loop steering control

In natural circumstances, sensory systems operate in a closed loop with motor output, whereby actions shape subsequent sensory experiences. A prime example of this is the sensorimotor processing required to align one's direction of travel, or heading, with one's goal, a behavior we refer to as steering. In steering, motor outputs work to eliminate errors between the direction of heading and the goal, modifying subsequent errors in the process. The closed-loop nature of the behavior makes it challenging to determine how deterministic and nondeterministic processes contribute to behavior. We overcome this by applying a nonparametric, linear kernel-based analysis to behavioral data of monkeys steering through a virtual environment in two experimental contexts. In a given context, the results were consistent with previous work that described the transformation as a second-order linear system. Classically, the parameters of such second-order models are associated with physical properties of the limb such as viscosity and stiffness that are commonly assumed to be approximately constant. By contrast, we found that the fit kernels differed strongly across tasks in these and other parameters, suggesting context-dependent changes in neural and biomechanical processes. We additionally fit residuals to a simple noise model and found that the form of the noise was highly conserved across both contexts and animals. Strikingly, the fitted noise also closely matched that found previously in a human steering task. Altogether, this work presents a kernel-based analysis that characterizes the context-dependence of deterministic and non-deterministic components of a closed-loop sensorimotor task.

Mixed Representations of Sound and Action in the Auditory Midbrain

Linking sensory input and its consequences is a fundamental brain operation. During behavior, the neural activity of neocortical and limbic systems often reflects dynamic combinations of sensory and task-dependent variables, and these "mixed representations" are suggested to be important for perception, learning, and plasticity. However, the extent to which such integrative computations might occur outside of the forebrain is less clear. Here, we conduct cellular-resolution two-photon Ca2+ imaging in the superficial "shell" layers of the inferior colliculus (IC), as head-fixed mice of either sex perform a reward-based psychometric auditory task. We find that the activity of individual shell IC neurons jointly reflects auditory cues, mice's actions, and behavioral trial outcomes, such that trajectories of neural population activity diverge depending on mice's behavioral choice. Consequently, simple classifier models trained on shell IC neuron activity can predict trial-by-trial outcomes, even when training data are restricted to neural activity occurring prior to mice's instrumental actions. Thus, in behaving mice, auditory midbrain neurons transmit a population code that reflects a joint representation of sound, actions, and task-dependent variables.

Neural manifolds in V1 change with top-down signals from V4 targeting the foveal region

High-dimensional brain activity is often organized into lower-dimensional neural manifolds. However, the neural manifolds of the visual cortex remain understudied. Here, we study large-scale multi-electrode electrophysiological recordings of macaque (Macaca mulatta) areas V1, V4, and DP with a high spatiotemporal resolution. We find that the population activity of V1 contains two separate neural manifolds, which correlate strongly with eye closure (eyes open/closed) and have distinct dimensionalities. Moreover, we find strong top-down signals from V4 to V1, particularly to the foveal region of V1, which are significantly stronger during the eyes-open periods. Finally, in silico simulations of a balanced spiking neuron network qualitatively reproduce the experimental findings. Taken together, our analyses and simulations suggest that top-down signals modulate the population activity of V1. We postulate that the top-down modulation during the eyes-open periods prepares V1 for fast and efficient visual responses, resulting in a type of visual stand-by state.

Hyperpolarization-activated currents drive neuronal activation sequences in sleep

Sequential neuronal patterns are believed to support information processing in the cortex, yet their origin is still a matter of debate. We report that neuronal activity in the mouse postsubiculum (PoSub), where a majority of neurons are modulated by the animal's head direction, was sequentially activated along the dorsoventral axis during sleep at the transition from hyperpolarized "DOWN" to activated "UP" states, while representing a stable direction. Computational modeling suggested that these dynamics could be attributed to a spatial gradient of hyperpolarization-activated currents (Ih), which we confirmed in ex vivo slice experiments and corroborated in other cortical structures. These findings open up the possibility that varying amounts of Ih across cortical neurons could result in sequential neuronal patterns and that traveling activity upstream of the entorhinal-hippocampal circuit organizes large-scale neuronal activity supporting learning and memory during sleep.

Bridging model and experiment in systems neuroscience with Cleo: the Closed-Loop, Electrophysiology, and Optophysiology simulation testbed

Systems neuroscience has experienced an explosion of new tools for reading and writing neural activity, enabling exciting new experiments such as all-optical or closed-loop control that effect powerful causal interventions. At the same time, improved computational models are capable of reproducing behavior and neural activity with increasing fidelity. Unfortunately, these advances have drastically increased the complexity of integrating different lines of research, resulting in the missed opportunities and untapped potential of suboptimal experiments. Experiment simulation can help bridge this gap, allowing model and experiment to better inform each other by providing a low-cost testbed for experiment design, model validation, and methods engineering. Specifically, this can be achieved by incorporating the simulation of the experimental interface into our models, but no existing tool integrates optogenetics, two-photon calcium imaging, electrode recording, and flexible closed-loop processing with neural population simulations. To address this need, we have developed Cleo: the Closed-Loop, Electrophysiology, and Optophysiology experiment simulation testbed. Cleo is a Python package enabling injection of recording and stimulation devices as well as closed-loop control with realistic latency into a Brian spiking neural network model. It is the only publicly available tool currently supporting two-photon and multi-opsin/wavelength optogenetics. To facilitate adoption and extension by the community, Cleo is open-source, modular, tested, and documented, and can export results to various data formats. Here we describe the design and features of Cleo, validate output of individual components and integrated experiments, and demonstrate its utility for advancing optogenetic techniques in prospective experiments using previously published systems neuroscience models.

Neural population dynamics in songbird RA and HVC during learned motor-vocal behavior

Complex, learned motor behaviors involve the coordination of large-scale neural activity across multiple brain regions, but our understanding of the population-level dynamics within different regions tied to the same behavior remains limited. Here, we investigate the neural population dynamics underlying learned vocal production in awake-singing songbirds. We use Neuropixels probes to record the simultaneous extracellular activity of populations of neurons in two regions of the vocal motor pathway. In line with observations made in non-human primates during limb-based motor tasks, we show that the population-level activity in both the premotor nucleus HVC and the motor nucleus RA is organized on low-dimensional neural manifolds upon which coordinated neural activity is well described by temporally structured trajectories during singing behavior. Both the HVC and RA latent trajectories provide relevant information to predict vocal sequence transitions between song syllables. However, the dynamics of these latent trajectories differ between regions. Our state-space models suggest a unique and continuous-over-time correspondence between the latent space of RA and vocal output, whereas the corresponding relationship for HVC exhibits a higher degree of neural variability. We then demonstrate that comparable high-fidelity reconstruction of continuous vocal outputs can be achieved from HVC and RA neural latents and spiking activity. Unlike those that use spiking activity, however, decoding models using neural latents generalize to novel sub-populations in each region, consistent with the existence of preserved manifolds that confine vocal-motor activity in HVC and RA.

Similar neural states, but dissimilar decoding patterns for motor control in parietal cortex

Discrete neural states are associated with reaching movements across the fronto-parietal network. Here, the Hidden Markov Model (HMM) applied to spiking activity of the somato-motor parietal area PE revealed a sequence of states similar to those of the contiguous visuomotor areas PEc and V6A. Using a coupled clustering and decoding approach, we proved that these neural states carried spatiotemporal information regarding behaviour in all three posterior parietal areas. However, comparing decoding accuracy, PE was less informative than V6A and PEc. In addition, V6A outperformed PEc in target inference, indicating functional differences among the parietal areas. To check the consistency of these differences, we used both a supervised and an unsupervised variant of the HMM, and compared its performance with two more common classifiers, Support Vector Machine and Long-Short Term Memory. The differences in decoding between areas were invariant to the algorithm used, still showing the dissimilarities found with HMM, thus indicating that these dissimilarities are intrinsic in the information encoded by parietal neurons. These results highlight that, when decoding from the parietal cortex, for example, in brain machine interface implementations, attention should be paid in selecting the most suitable source of neural signals, given the great heterogeneity of this cortical sector.

System-level time computation and representation in the suprachiasmatic nucleus revealed by large-scale calcium imaging and machine learning

The suprachiasmatic nucleus (SCN) is the mammalian central circadian pacemaker with heterogeneous neurons acting in concert while each neuron harbors a self-sustained molecular clockwork. Nevertheless, how system-level SCN signals encode time of the day remains enigmatic. Here we show that population-level Ca2+ signals predict hourly time, via a group decision-making mechanism coupled with a spatially modular time feature representation in the SCN. Specifically, we developed a high-speed dual-view two-photon microscope for volumetric Ca2+ imaging of up to 9000 GABAergic neurons in adult SCN slices, and leveraged machine learning methods to capture emergent properties from multiscale Ca2+ signals as a whole. We achieved hourly time prediction by polling random cohorts of SCN neurons, reaching 99.0% accuracy at a cohort size of 900. Further, we revealed that functional neuron subtypes identified by contrastive learning tend to aggregate separately in the SCN space, giving rise to bilaterally symmetrical ripple-like modular patterns. Individual modules represent distinctive time features, such that a module-specifically learned time predictor can also accurately decode hourly time from random polling of the same module. These findings open a new paradigm in deciphering the design principle of the biological clock at the system level.

A sensory-motor theory of the neocortex

Recent neurophysiological and neuroanatomical studies suggest a close interaction between sensory and motor processes across the neocortex. Here, I propose that the neocortex implements active predictive coding (APC): each cortical area estimates both latent sensory states and actions (including potentially abstract actions internal to the cortex), and the cortex as a whole predicts the consequences of actions at multiple hierarchical levels. Feedback from higher areas modulates the dynamics of state and action networks in lower areas. I show how the same APC architecture can explain (1) how we recognize an object and its parts using eye movements, (2) why perception seems stable despite eye movements, (3) how we learn compositional representations, for example, part-whole hierarchies, (4) how complex actions can be planned using simpler actions, and (5) how we form episodic memories of sensory-motor experiences and learn abstract concepts such as a family tree. I postulate a mapping of the APC model to the laminar architecture of the cortex and suggest possible roles for cortico-cortical and cortico-subcortical pathways.

Nondifferentiable activity in the brain

Spike raster plots of numerous neurons show vertical stripes, indicating that neurons exhibit synchronous activity in the brain. We seek to determine whether these coherent dynamics are caused by smooth brainwave activity or by something else. By analyzing biological data, we find that their cross-correlograms exhibit not only slow undulation but also a cusp at the origin, in addition to possible signs of monosynaptic connectivity. Here we show that undulation emerges if neurons are subject to smooth brainwave oscillations while a cusp results from nondifferentiable fluctuations. While modern analysis methods have achieved good connectivity estimation by adapting the models to slow undulation, they still make false inferences due to the cusp. We devise a new analysis method that may solve both problems. We also demonstrate that oscillations and nondifferentiable fluctuations may emerge in simulations of large-scale neural networks.

Response inhibition in premotor cortex corresponds to a complex reshuffle of the mesoscopic information network

Recent studies have explored functional and effective neural networks in animal models; however, the dynamics of information propagation among functional modules under cognitive control remain largely unknown. Here, we addressed the issue using transfer entropy and graph theory methods on mesoscopic neural activities recorded in the dorsal premotor cortex of rhesus monkeys. We focused our study on the decision time of a Stop-signal task, looking for patterns in the network configuration that could influence motor plan maturation when the Stop signal is provided. When comparing trials with successful inhibition to those with generated movement, the nodes of the network resulted organized into four clusters, hierarchically arranged, and distinctly involved in information transfer. Interestingly, the hierarchies and the strength of information transmission between clusters varied throughout the task, distinguishing between generated movements and canceled ones and corresponding to measurable levels of network complexity. Our results suggest a putative mechanism for motor inhibition in premotor cortex: a topological reshuffle of the information exchanged among ensembles of neurons.

The computational foundations of dynamic coding in working memory

Working memory (WM) is a fundamental aspect of cognition. WM maintenance is classically thought to rely on stable patterns of neural activities. However, recent evidence shows that neural population activities during WM maintenance undergo dynamic variations before settling into a stable pattern. Although this has been difficult to explain theoretically, neural network models optimized for WM typically also exhibit such dynamics. Here, we examine stable versus dynamic coding in neural data, classical models, and task-optimized networks. We review principled mathematical reasons for why classical models do not, while task-optimized models naturally do exhibit dynamic coding. We suggest an update to our understanding of WM maintenance, in which dynamic coding is a fundamental computational feature rather than an epiphenomenon.

The relativity of reaching: Motion of the touched surface alters the trajectory of hand movements

For dexterous control of the hand, humans integrate sensory information and prior knowledge regarding their bodies and the world. We studied the role of touch in hand motor control by challenging a fundamental prior assumption-that self-motion of inanimate objects is unlikely upon contact. In a reaching task, participants slid their fingertips across a robotic interface, with their hand hidden from sight. Unbeknownst to the participants, the robotic interface remained static, followed hand movement, or moved in opposition to it. We considered two hypotheses. Either participants were able to account for surface motion or, if the stationarity assumption held, they would integrate the biased tactile cues and proprioception. Motor errors consistent with the latter hypothesis were observed. The role of visual feedback, tactile sensitivity, and friction was also investigated. Our study carries profound implications for human-machine collaboration in a world where objects may no longer conform to the stationarity assumption.

Flexible gating between subspaces in a neural network model of internally guided task switching

Behavioral flexibility relies on the brain's ability to switch rapidly between multiple tasks, even when the task rule is not explicitly cued but must be inferred through trial and error. The underlying neural circuit mechanism remains poorly understood. We investigated recurrent neural networks (RNNs) trained to perform an analog of the classic Wisconsin Card Sorting Test. The networks consist of two modules responsible for rule representation and sensorimotor mapping, respectively, where each module is comprised of a circuit with excitatory neurons and three major types of inhibitory neurons. We found that rule representation by self-sustained persistent activity across trials, error monitoring and gated sensorimotor mapping emerged from training. Systematic dissection of trained RNNs revealed a detailed circuit mechanism that is consistent across networks trained with different hyperparameters. The networks' dynamical trajectories for different rules resided in separate subspaces of population activity; the subspaces collapsed and performance was reduced to chance level when dendrite-targeting somatostatin-expressing interneurons were silenced, illustrating how a phenomenological description of representational subspaces is explained by a specific circuit mechanism.

Rule-based modulation of a sensorimotor transformation across cortical areas

Flexible responses to sensory stimuli based on changing rules are critical for adapting to a dynamic environment. However, it remains unclear how the brain encodes and uses rule information to guide behavior. Here, we made single-unit recordings while head-fixed mice performed a cross-modal sensory selection task where they switched between two rules: licking in response to tactile stimuli while rejecting visual stimuli, or vice versa. Along a cortical sensorimotor processing stream including the primary (S1) and secondary (S2) somatosensory areas, and the medial (MM) and anterolateral (ALM) motor areas, single-neuron activity distinguished between the two rules both prior to and in response to the tactile stimulus. We hypothesized that neural populations in these areas would show rule-dependent preparatory states, which would shape the subsequent sensory processing and behavior. This hypothesis was supported for the motor cortical areas (MM and ALM) by findings that (1) the current task rule could be decoded from pre-stimulus population activity; (2) neural subspaces containing the population activity differed between the two rules; and (3) optogenetic disruption of pre-stimulus states impaired task performance. Our findings indicate that flexible action selection in response to sensory input can occur via configuration of preparatory states in the motor cortex.

Neural Activity in Quarks Language: Lattice Field Theory for a Network of Real Neurons

Brain-computer interfaces have seen extraordinary surges in developments in recent years, and a significant discrepancy now exists between the abundance of available data and the limited headway made in achieving a unified theoretical framework. This discrepancy becomes particularly pronounced when examining the collective neural activity at the micro and meso scale, where a coherent formalization that adequately describes neural interactions is still lacking. Here, we introduce a mathematical framework to analyze systems of natural neurons and interpret the related empirical observations in terms of lattice field theory, an established paradigm from theoretical particle physics and statistical mechanics. Our methods are tailored to interpret data from chronic neural interfaces, especially spike rasters from measurements of single neuron activity, and generalize the maximum entropy model for neural networks so that the time evolution of the system is also taken into account. This is obtained by bridging particle physics and neuroscience, paving the way for particle physics-inspired models of the neocortex.

A combinatorial neural code for long-term motor memory

Motor skill repertoire can be stably retained over long periods, but the neural mechanism underlying stable memory storage remains poorly understood. Moreover, it is unknown how existing motor memories are maintained as new motor skills are continuously acquired. Here we tracked neural representation of learned actions throughout a significant portion of a mouse's lifespan, and we show that learned actions are stably retained in motor memory in combination with context, which protects existing memories from erasure during new motor learning. We used automated home-cage training to establish a continual learning paradigm in which mice learned to perform directional licking in different task contexts. We combined this paradigm with chronic two-photon imaging of motor cortex activity for up to 6 months. Within the same task context, activity driving directional licking was stable over time with little representational drift. When learning new task contexts, new preparatory activity emerged to drive the same licking actions. Learning created parallel new motor memories while retaining the previous memories. Re-learning to make the same actions in the previous task context re-activated the previous preparatory activity, even months later. At the same time, continual learning of new task contexts kept creating new preparatory activity patterns. Context-specific memories, as we observed in the motor system, may provide a solution for stable memory storage throughout continual learning. Learning in new contexts produces parallel new representations instead of modifying existing representations, thus protecting existing motor repertoire from erasure.

Probing latent brain dynamics in Alzheimer's disease via recurrent neural network

The impairment of cognitive function in Alzheimer's disease (AD) is clearly correlated to abnormal changes in cortical rhythm. However, the mechanisms underlying this correlation are still poorly understood. Here, we investigate how network structure and dynamical characteristics alter their abnormal changes in cortical rhythm. To that end, biological data of AD and normal participates are collected. By extracting the energy characteristics of different sub-bands in EEG signals, we find that the rhythm of AD patients is special particularly in theta and alpha bands. The cortical rhythm of normal state is mainly at alpha band, while that of AD state shift to the theta band. Furthermore, recurrent neural network (RNN) is trained to explore the rhythm formation and transformation between two neural states from the perspective view of neurocomputation. It is found that the neural coupling strength decreases significantly under AD state when compared with normal state, which weakens the ability of information transmission in AD state. Besides, the low-dimensional properties of RNN are obtained. By analyzing the relationship between the cortical rhythm transition and the low-dimensional trajectory, it is concluded that the low-dimensional trajectory update is slower and the communication cost is higher in AD state, which explains the abnormal synchronization of AD brain network. Our work reveals the causes for the formation of abnormal brain synchronous functional network status, which may expand our understanding of the mechanism of cognitive impairment in AD and provide an EEG biomarker for early AD.

Effect in the spectra of eigenvalues and dynamics of RNNs trained with excitatory-inhibitory constraint

In order to comprehend and enhance models that describes various brain regions it is important to study the dynamics of trained recurrent neural networks. Including Dale's law in such models usually presents several challenges. However, this is an important aspect that allows computational models to better capture the characteristics of the brain. Here we present a framework to train networks using such constraint. Then we have used it to train them in simple decision making tasks. We characterized the eigenvalue distributions of the recurrent weight matrices of such networks. Interestingly, we discovered that the non-dominant eigenvalues of the recurrent weight matrix are distributed in a circle with a radius less than 1 for those whose initial condition before training was random normal and in a ring for those whose initial condition was random orthogonal. In both cases, the radius does not depend on the fraction of excitatory and inhibitory units nor the size of the network. Diminution of the radius, compared to networks trained without the constraint, has implications on the activity and dynamics that we discussed here.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-023-09956-w.

Dimensionality reduction beyond neural subspaces with slice tensor component analysis

Recent work has argued that large-scale neural recordings are often well described by patterns of coactivation across neurons. Yet the view that neural variability is constrained to a fixed, low-dimensional subspace may overlook higher-dimensional structure, including stereotyped neural sequences or slowly evolving latent spaces. Here we argue that task-relevant variability in neural data can also cofluctuate over trials or time, defining distinct 'covariability classes' that may co-occur within the same dataset. To demix these covariability classes, we develop sliceTCA (slice tensor component analysis), a new unsupervised dimensionality reduction method for neural data tensors. In three example datasets, including motor cortical activity during a classic reaching task in primates and recent multiregion recordings in mice, we show that sliceTCA can capture more task-relevant structure in neural data using fewer components than traditional methods. Overall, our theoretical framework extends the classic view of low-dimensional population activity by incorporating additional classes of latent variables capturing higher-dimensional structure.