Task representations in neural networks trained to perform many cognitive tasks

A geometric framework for understanding dynamic information integration in context-dependent computation

The prefrontal cortex (PFC) plays a prominent role in performing flexible cognitive functions and working memory, yet the underlying computational principle remains poorly understood. Here, we trained a rate-based recurrent neural network (RNN) to explore how the context rules are encoded, maintained across seconds-long mnemonic delay, and subsequently used in a context-dependent decision-making task. The trained networks replicated key experimentally observed features in the PFC of rodent and monkey experiments, such as mixed selectivity, neuronal sequential activity, and rotation dynamics. To uncover the high-dimensional neural dynamical system, we further proposed a geometric framework to quantify and visualize population coding and sensory integration in a temporally defined manner. We employed dynamic epoch-wise principal component analysis (PCA) to define multiple task-specific subspaces and task-related axes, and computed the angles between task-related axes and these subspaces. In low-dimensional neural representations, the trained RNN first encoded the context cues in a cue-specific subspace, and then maintained the cue information with a stable low-activity state persisting during the delay epoch, and further formed line attractors for sensor integration through low-dimensional neural trajectories to guide decision-making. We demonstrated via intensive computer simulations that the geometric manifolds encoding the context information were robust to varying degrees of weight perturbation in both space and time. Overall, our analysis framework provides clear geometric interpretations and quantification of information coding, maintenance, and integration, yielding new insight into the computational mechanisms of context-dependent computation.

Computation Through Neural Population Dynamics

Significant experimental, computational, and theoretical work has identified rich structure within the coordinated activity of interconnected neural populations. An emerging challenge now is to uncover the nature of the associated computations, how they are implemented, and what role they play in driving behavior. We term this computation through neural population dynamics. If successful, this framework will reveal general motifs of neural population activity and quantitatively describe how neural population dynamics implement computations necessary for driving goal-directed behavior. Here, we start with a mathematical primer on dynamical systems theory and analytical tools necessary to apply this perspective to experimental data. Next, we highlight some recent discoveries resulting from successful application of dynamical systems. We focus on studies spanning motor control, timing, decision-making, and working memory. Finally, we briefly discuss promising recent lines of investigation and future directions for the computation through neural population dynamics framework.

Bio-instantiated recurrent neural networks: Integrating neurobiology-based network topology in artificial networks

Biological neuronal networks (BNNs) are a source of inspiration and analogy making for researchers that focus on artificial neuronal networks (ANNs). Moreover, neuroscientists increasingly use ANNs as a model for the brain. Despite certain similarities between these two types of networks, important differences can be discerned. First, biological neural networks are sculpted by evolution and the constraints that it entails, whereas artificial neural networks are engineered to solve particular tasks. Second, the network topology of these systems, apart from some analogies that can be drawn, exhibits pronounced differences. Here, we examine strategies to construct recurrent neural networks (RNNs) that instantiate the network topology of brains of different species. We refer to such RNNs as bio-instantiated. We investigate the performance of bio-instantiated RNNs in terms of: (i) the prediction performance itself, that is, the capacity of the network to minimize the cost function at hand in test data, and (ii) speed of training, that is, how fast during training the network reaches its optimal performance. We examine bio-instantiated RNNs in working memory tasks where task-relevant information must be tracked as a sequence of events unfolds in time. We highlight the strategies that can be used to construct RNNs with the network topology found in BNNs, without sacrificing performance. Despite that we observe no enhancement of performance when compared to randomly wired RNNs, our approach demonstrates how empirical neural network data can be used for constructing RNNs, thus, facilitating further experimentation with biologically realistic network topologies, in contexts where such aspect is desired.

Multitask learning over shared subspaces

This paper uses constructs from machine learning to define pairs of learning tasks that either shared or did not share a common subspace. Human subjects then learnt these tasks using a feedback-based approach and we hypothesised that learning would be boosted for shared subspaces. Our findings broadly supported this hypothesis with either better performance on the second task if it shared the same subspace as the first, or positive correlations over task performance for shared subspaces. These empirical findings were compared to the behaviour of a Neural Network model trained using sequential Bayesian learning and human performance was found to be consistent with a minimal capacity variant of this model. Networks with an increased representational capacity, and networks without Bayesian learning, did not show these transfer effects. We propose that the concept of shared subspaces provides a useful framework for the experimental study of human multitask and transfer learning.

How does knowledge gained from previous experience affect learning of new tasks? This question of “Transfer Learning” has been addressed by teachers, psychologists, and more recently by researchers in the fields of neural networks and machine learning. Leveraging constructs from machine learning, we designed pairs of learning tasks that either shared or did not share a common subspace. We compared the dynamics of transfer learning in humans with those of a multitask neural network model, finding that human performance was consistent with a minimal capacity variant of the model. Learning was boosted in the second task if the same subspace was shared between tasks. Additionally, accuracy between tasks was positively correlated but only when they shared the same subspace. Our results highlight the roles of subspaces, showing how they could act as a learning boost if shared, and be detrimental if not.

From synapse to network: models of information storage and retrieval in neural circuits

The mechanisms of information storage and retrieval in brain circuits are still the subject of debate. It is widely believed that information is stored at least in part through changes in synaptic connectivity in networks that encode this information and that these changes lead in turn to modifications of network dynamics, such that the stored information can be retrieved at a later time. Here, we review recent progress in deriving synaptic plasticity rules from experimental data and in understanding how plasticity rules affect the dynamics of recurrent networks. We show that the dynamics generated by such networks exhibit a large degree of diversity, depending on parameters, similar to experimental observations in vivo during delayed response tasks.

Frontal circuit specialisations for decision making

There is widespread consensus that distributed circuits across prefrontal and anterior cingulate cortex (PFC/ACC) are critical for reward-based decision making. The circuit specialisations of these areas in primates were likely shaped by their foraging niche, in which decision making is typically sequential, attention-guided and temporally extended. Here, I argue that in humans and other primates, PFC/ACC circuits are functionally specialised in two ways. First, microcircuits found across PFC/ACC are highly recurrent in nature and have synaptic properties that support persistent activity across temporally extended cognitive tasks. These properties provide the basis of a computational account of time-varying neural activity within PFC/ACC as a decision is being made. Second, the macrocircuit connections (to other brain areas) differ between distinct PFC/ACC cytoarchitectonic subregions. This variation in macrocircuit connections explains why PFC/ACC subregions make unique contributions to reward-based decision tasks and how these contributions are shaped by attention. They predict dissociable neural representations to emerge in orbitofrontal, anterior cingulate and dorsolateral prefrontal cortex during sequential attention-guided choice, as recently confirmed in neurophysiological recordings.

Inferring the function performed by a recurrent neural network

A central goal in systems neuroscience is to understand the functions performed by neural circuits. Previous top-down models addressed this question by comparing the behaviour of an ideal model circuit, optimised to perform a given function, with neural recordings. However, this requires guessing in advance what function is being performed, which may not be possible for many neural systems. To address this, we propose an inverse reinforcement learning (RL) framework for inferring the function performed by a neural network from data. We assume that the responses of each neuron in a network are optimised so as to drive the network towards 'rewarded' states, that are desirable for performing a given function. We then show how one can use inverse RL to infer the reward function optimised by the network from observing its responses. This inferred reward function can be used to predict how the neural network should adapt its dynamics to perform the same function when the external environment or network structure changes. This could lead to theoretical predictions about how neural network dynamics adapt to deal with cell death and/or varying sensory stimulus statistics.

A continuum of invariant sensory and behavioral-context perceptual coding in secondary somatosensory cortex

A crucial role of cortical networks is the conversion of sensory inputs into perception. In the cortical somatosensory network, neurons of the primary somatosensory cortex (S1) show invariant sensory responses, while frontal lobe neuronal activity correlates with the animal's perceptual behavior. Here, we report that in the secondary somatosensory cortex (S2), neurons with invariant sensory responses coexist with neurons whose responses correlate with perceptual behavior. Importantly, the vast majority of the neurons fall along a continuum of combined sensory and categorical dynamics. Furthermore, during a non-demanding control task, the sensory responses remain unaltered while the sensory information exhibits an increase. However, perceptual responses and the associated categorical information decrease, implicating a task context-dependent processing mechanism. Conclusively, S2 neurons exhibit intriguing dynamics that are intermediate between those of S1 and frontal lobe. Our results contribute relevant evidence about the role that S2 plays in the conversion of touch into perception.

Predictive learning as a network mechanism for extracting low-dimensional latent space representations

Artificial neural networks have recently achieved many successes in solving sequential processing and planning tasks. Their success is often ascribed to the emergence of the task’s low-dimensional latent structure in the network activity – i.e., in the learned neural representations. Here, we investigate the hypothesis that a means for generating representations with easily accessed low-dimensional latent structure, possibly reflecting an underlying semantic organization, is through learning to predict observations about the world. Specifically, we ask whether and when network mechanisms for sensory prediction coincide with those for extracting the underlying latent variables. Using a recurrent neural network model trained to predict a sequence of observations we show that network dynamics exhibit low-dimensional but nonlinearly transformed representations of sensory inputs that map the latent structure of the sensory environment. We quantify these results using nonlinear measures of intrinsic dimensionality and linear decodability of latent variables, and provide mathematical arguments for why such useful predictive representations emerge. We focus throughout on how our results can aid the analysis and interpretation of experimental data.

Neural networks trained using predictive models generate representations that recover the underlying low-dimensional latent structure in the data. Here, the authors demonstrate that a network trained on a spatial navigation task generates place-related neural activations similar to those observed in the hippocampus and show that these are related to the latent structure.

Semantic Relatedness Emerges in Deep Convolutional Neural Networks Designed for Object Recognition

Human not only can effortlessly recognize objects, but also characterize object categories into semantic concepts with a nested hierarchical structure. One dominant view is that top-down conceptual guidance is necessary to form such hierarchy. Here we challenged this idea by examining whether deep convolutional neural networks (DCNNs) could learn relations among objects purely based on bottom-up perceptual experience of objects through training for object categorization. Specifically, we explored representational similarity among objects in a typical DCNN (e.g., AlexNet), and found that representations of object categories were organized in a hierarchical fashion, suggesting that the relatedness among objects emerged automatically when learning to recognize them. Critically, the emerged relatedness of objects in the DCNN was highly similar to the WordNet in human, implying that top-down conceptual guidance may not be a prerequisite for human learning the relatedness among objects. In addition, the developmental trajectory of the relatedness among objects during training revealed that the hierarchical structure was constructed in a coarse-to-fine fashion, and evolved into maturity before the establishment of object recognition ability. Finally, the fineness of the relatedness was greatly shaped by the demand of tasks that the DCNN performed, as the higher superordinate level of object classification was, the coarser the hierarchical structure of the relatedness emerged. Taken together, our study provides the first empirical evidence that semantic relatedness of objects emerged as a by-product of object recognition in DCNNs, implying that human may acquire semantic knowledge on objects without explicit top-down conceptual guidance.

Probing the structure-function relationship with neural networks constructed by solving a system of linear equations

Neural network models are an invaluable tool to understand brain function since they allow us to connect the cellular and circuit levels with behaviour. Neural networks usually comprise a huge number of parameters, which must be chosen carefully such that networks reproduce anatomical, behavioural, and neurophysiological data. These parameters are usually fitted with off-the-shelf optimization algorithms that iteratively change network parameters and simulate the network to evaluate its performance and improve fitting. Here we propose to invert the fitting process by proceeding from the network dynamics towards network parameters. Firing state transitions are chosen according to the transition graph associated with the solution of a task. Then, a system of linear equations is constructed from the network firing states and membrane potentials, in a way that guarantees the consistency of the system. This allows us to uncouple the dynamical features of the model, like its neurons firing rate and correlation, from the structural features, and the task-solving algorithm implemented by the network. We employed our method to probe the structure–function relationship in a sequence memory task. The networks obtained showed connectivity and firing statistics that recapitulated experimental observations. We argue that the proposed method is a complementary and needed alternative to the way neural networks are constructed to model brain function.

The Best Laid Plans: Computational Principles of Anterior Cingulate Cortex

Despite continual debate for the past 30 years about the function of anterior cingulate cortex (ACC), its key contribution to neurocognition remains unknown. However, recent computational modeling work has provided insight into this question. Here we review computational models that illustrate three core principles of ACC function, related to hierarchy, world models, and cost. We also discuss four constraints on the neural implementation of these principles, related to modularity, binding, encoding, and learning and regulation. These observations suggest a role for ACC in hierarchical model-based hierarchical reinforcement learning (HMB-HRL), which instantiates a mechanism motivating the execution of high-level plans.

Multi-task reinforcement learning in humans

The ability to transfer knowledge across tasks and generalize to novel ones is an important hallmark of human intelligence. Yet not much is known about human multitask reinforcement learning. We study participants' behaviour in a two-step decision-making task with multiple features and changing reward functions. We compare their behaviour with two algorithms for multitask reinforcement learning, one that maps previous policies and encountered features to new reward functions and one that approximates value functions across tasks, as well as to standard model-based and model-free algorithms. Across three exploratory experiments and a large preregistered confirmatory experiment, our results provide evidence that participants who are able to learn the task use a strategy that maps previously learned policies to novel scenarios. These results enrich our understanding of human reinforcement learning in complex environments with changing task demands.

Strong inhibitory signaling underlies stable temporal dynamics and working memory in spiking neural networks

Cortical neurons process information on multiple timescales, and areas important for working memory (WM) contain neurons capable of integrating information over a long timescale. However, the underlying mechanisms for the emergence of neuronal timescales stable enough to support WM are unclear. By analyzing a spiking recurrent neural network model trained on a WM task and activity of single neurons in the primate prefrontal cortex, we show that the temporal properties of our model and the neural data are remarkably similar. Dissecting our recurrent neural network model revealed strong inhibitory-to-inhibitory connections underlying a disinhibitory microcircuit as a critical component for long neuronal timescales and WM maintenance. We also found that enhancing inhibitory-to-inhibitory connections led to more stable temporal dynamics and improved task performance. Finally, we show that a network with such microcircuitry can perform other tasks without disrupting its pre-existing timescale architecture, suggesting that strong inhibitory signaling underlies a flexible WM network.

Themes of advanced information processing in the primate brain

Here is a review of several empirical examples of information processing that occur in the primate cerebral cortex. These include visual processing, object identification and perception, information encoding, and memory. Also, there is a discussion of the higher scale neural organization, mainly theoretical, which suggests hypotheses on how the brain internally represents objects. Altogether they support the general attributes of the mechanisms of brain computation, such as efficiency, resiliency, data compression, and a modularization of neural function and their pathways. Moreover, the specific neural encoding schemes are expectedly stochastic, abstract and not easily decoded by theoretical or empirical approaches.

Learning to select actions shapes recurrent dynamics in the corticostriatal system

Learning to select appropriate actions based on their values is fundamental to adaptive behavior. This form of learning is supported by fronto-striatal systems. The dorsal-lateral prefrontal cortex (dlPFC) and the dorsal striatum (dSTR), which are strongly interconnected, are key nodes in this circuitry. Substantial experimental evidence, including neurophysiological recordings, have shown that neurons in these structures represent key aspects of learning. The computational mechanisms that shape the neurophysiological responses, however, are not clear. To examine this, we developed a recurrent neural network (RNN) model of the dlPFC-dSTR circuit and trained it on an oculomotor sequence learning task. We compared the activity generated by the model to activity recorded from monkey dlPFC and dSTR in the same task. This network consisted of a striatal component which encoded action values, and a prefrontal component which selected appropriate actions. After training, this system was able to autonomously represent and update action values and select actions, thus being able to closely approximate the representational structure in corticostriatal recordings. We found that learning to select the correct actions drove action-sequence representations further apart in activity space, both in the model and in the neural data. The model revealed that learning proceeds by increasing the distance between sequence-specific representations. This makes it more likely that the model will select the appropriate action sequence as learning develops. Our model thus supports the hypothesis that learning in networks drives the neural representations of actions further apart, increasing the probability that the network generates correct actions as learning proceeds. Altogether, this study advances our understanding of how neural circuit dynamics are involved in neural computation, revealing how dynamics in the corticostriatal system support task learning.

Theoretical principles for illuminating sensorimotor processing with brain-wide neuronal recordings

Modern recording techniques now permit brain-wide sensorimotor circuits to be observed at single neuron resolution in small animals. Extracting theoretical understanding from these recordings requires principles that organize findings and guide future experiments. Here we review theoretical principles that shed light onto brain-wide sensorimotor processing. We begin with an analogy that conceptualizes principles as streetlamps that illuminate the empirical terrain, and we illustrate the analogy by showing how two familiar principles apply in new ways to brain-wide phenomena. We then focus the bulk of the review on describing three more principles that have wide utility for mapping brain-wide neural activity, making testable predictions from highly parameterized mechanistic models, and investigating the computational determinants of neuronal response patterns across the brain.

Engrams of Fast Learning

Fast learning designates the behavioral and neuronal mechanisms underlying the acquisition of a long-term memory trace after a unique and brief experience. As such it is opposed to incremental, slower reinforcement or procedural learning requiring repetitive training. This learning process, found in most animal species, exists in a large spectrum of natural behaviors, such as one-shot associative, spatial, or perceptual learning, and is a core principle of human episodic memory. We review here the neuronal and synaptic long-term changes associated with fast learning in mammals and discuss some hypotheses related to their underlying mechanisms. We first describe the variety of behavioral paradigms used to test fast learning memories: those preferentially involve a single and brief (from few hundred milliseconds to few minutes) exposures to salient stimuli, sufficient to trigger a long-lasting memory trace and new adaptive responses. We then focus on neuronal activity patterns observed during fast learning and the emergence of long-term selective responses, before documenting the physiological correlates of fast learning. In the search for the engrams of fast learning, a growing body of evidence highlights long-term changes in gene expression, structural, intrinsic, and synaptic plasticities. Finally, we discuss the potential role of the sparse and bursting nature of neuronal activity observed during the fast learning, especially in the induction plasticity mechanisms leading to the rapid establishment of long-term synaptic modifications. We conclude with more theoretical perspectives on network dynamics that could enable fast learning, with an overview of some theoretical approaches in cognitive neuroscience and artificial intelligence.

A goal-driven modular neural network predicts parietofrontal neural dynamics during grasping

One of the primary ways we interact with the world is using our hands. In macaques, the circuit spanning the anterior intraparietal area, the hand area of the ventral premotor cortex, and the primary motor cortex is necessary for transforming visual information into grasping movements. However, no comprehensive model exists that links all steps of processing from vision to action. We hypothesized that a recurrent neural network mimicking the modular structure of the anatomical circuit and trained to use visual features of objects to generate the required muscle dynamics used by primates to grasp objects would give insight into the computations of the grasping circuit. Internal activity of modular networks trained with these constraints strongly resembled neural activity recorded from the grasping circuit during grasping and paralleled the similarities between brain regions. Network activity during the different phases of the task could be explained by linear dynamics for maintaining a distributed movement plan across the network in the absence of visual stimulus and then generating the required muscle kinematics based on these initial conditions in a module-specific way. These modular models also outperformed alternative models at explaining neural data, despite the absence of neural data during training, suggesting that the inputs, outputs, and architectural constraints imposed were sufficient for recapitulating processing in the grasping circuit. Finally, targeted lesioning of modules produced deficits similar to those observed in lesion studies of the grasping circuit, providing a potential model for how brain regions may coordinate during the visually guided grasping of objects.

A modeling framework for adaptive lifelong learning with transfer and savings through gating in the prefrontal cortex

The prefrontal cortex encodes and stores numerous, often disparate, schemas and flexibly switches between them. Recent research on artificial neural networks trained by reinforcement learning has made it possible to model fundamental processes underlying schema encoding and storage. Yet how the brain is able to create new schemas while preserving and utilizing old schemas remains unclear. Here we propose a simple neural network framework that incorporates hierarchical gating to model the prefrontal cortex's ability to flexibly encode and use multiple disparate schemas. We show how gating naturally leads to transfer learning and robust memory savings. We then show how neuropsychological impairments observed in patients with prefrontal damage are mimicked by lesions of our network. Our architecture, which we call DynaMoE, provides a fundamental framework for how the prefrontal cortex may handle the abundance of schemas necessary to navigate the real world.

Complementary Inhibitory Weight Profiles Emerge from Plasticity and Allow Flexible Switching of Receptive Fields

Cortical areas comprise multiple types of inhibitory interneurons, with stereotypical connectivity motifs that may follow specific plasticity rules. Yet, their combined effect on postsynaptic dynamics has been largely unexplored. Here, we analyze the response of a single postsynaptic model neuron receiving tuned excitatory connections alongside inhibition from two plastic populations. Synapses from each inhibitory population change according to distinct plasticity rules. We tested different combinations of three rules: Hebbian, anti-Hebbian, and homeostatic scaling. Depending on the inhibitory plasticity rule, synapses become unspecific (flat), anticorrelated to, or correlated with excitatory synapses. Crucially, the neuron's receptive field (i.e., its response to presynaptic stimuli) depends on the modulatory state of inhibition. When both inhibitory populations are active, inhibition balances excitation, resulting in uncorrelated postsynaptic responses regardless of the inhibitory tuning profiles. Modulating the activity of a given inhibitory population produces strong correlations to either preferred or nonpreferred inputs, in line with recent experimental findings that show dramatic context-dependent changes of neurons' receptive fields. We thus confirm that a neuron's receptive field does not follow directly from the weight profiles of its presynaptic afferents. Our results show how plasticity rules in various cell types can interact to shape cortical circuit motifs and their dynamics.SIGNIFICANCE STATEMENT Neurons in sensory areas of the cortex are known to respond to specific features of a given input (e.g., specific sound frequencies), but recent experimental studies show that such responses (i.e., their receptive fields) depend on context. Inspired by the cortical connectivity, we built models of excitatory and inhibitory inputs onto a single neuron, to study how receptive fields may change on short and long time scales. We show how various synaptic plasticity rules allow for the emergence of diverse connectivity profiles and, moreover, how their dynamic interaction creates a mechanism by which postsynaptic responses can quickly change. Our work emphasizes multiple roles of inhibition in cortical processing and provides a first mechanistic model for flexible receptive fields.

A recurrent neural network framework for flexible and adaptive decision making based on sequence learning

The brain makes flexible and adaptive responses in a complicated and ever-changing environment for an organism's survival. To achieve this, the brain needs to understand the contingencies between its sensory inputs, actions, and rewards. This is analogous to the statistical inference that has been extensively studied in the natural language processing field, where recent developments of recurrent neural networks have found many successes. We wonder whether these neural networks, the gated recurrent unit (GRU) networks in particular, reflect how the brain solves the contingency problem. Therefore, we build a GRU network framework inspired by the statistical learning approach of NLP and test it with four exemplar behavior tasks previously used in empirical studies. The network models are trained to predict future events based on past events, both comprising sensory, action, and reward events. We show the networks can successfully reproduce animal and human behavior. The networks generalize the training, perform Bayesian inference in novel conditions, and adapt their choices when event contingencies vary. Importantly, units in the network encode task variables and exhibit activity patterns that match previous neurophysiology findings. Our results suggest that the neural network approach based on statistical sequence learning may reflect the brain's computational principle underlying flexible and adaptive behaviors and serve as a useful approach to understand the brain.

Stability of motor cortex network states during learning-associated neural reorganizations

A substantial reorganization of neural activity and neuron-to-movement relationship in motor cortical circuits accompanies the emergence of reproducible movement patterns during motor learning. Little is known about how this tempest of neural activity restructuring impacts the stability of network states in recurrent cortical circuits. To investigate this issue, we reanalyzed data in which we recorded for 14 days via population calcium imaging the activity of the same neural populations of pyramidal neurons in layer 2/3 and layer 5 of forelimb motor and premotor cortex in mice during the daily learning of a lever-press task. We found that motor cortex network states remained stable with respect to the critical network state during the extensive reorganization of both neural population activity and its relation to lever movement throughout learning. Specifically, layer 2/3 cortical circuits unceasingly displayed robust evidence for operating at the critical network state, a regime that maximizes information capacity and transmission and provides a balance between network robustness and flexibility. In contrast, layer 5 circuits operated away from the critical network state for all 14 days of recording and learning. In conclusion, this result indicates that the wide-ranging malleability of synapses, neurons, and neural connectivity during learning operates within the constraint of a stable and layer-specific network state regarding dynamic criticality, and suggests that different cortical layers operate under distinct constraints because of their specialized goals.NEW & NOTEWORTHY The neural activity reorganizes throughout motor learning, but how this reorganization impacts the stability of network states is unclear. We used two-photon calcium imaging to investigate how the network states in layer 2/3 and layer 5 of forelimb motor and premotor cortex are modulated by motor learning. We show that motor cortex network states are layer-specific and constant regarding criticality during neural activity reorganization, and suggests that layer-specific constraints could be motivated by different functions.

Low-dimensional dynamics for working memory and time encoding

Our decisions often depend on multiple sensory experiences separated by time delays. The brain can remember these experiences and, simultaneously, estimate the timing between events. To understand the mechanisms underlying working memory and time encoding, we analyze neural activity recorded during delays in four experiments on nonhuman primates. To disambiguate potential mechanisms, we propose two analyses, namely, decoding the passage of time from neural data and computing the cumulative dimensionality of the neural trajectory over time. Time can be decoded with high precision in tasks where timing information is relevant and with lower precision when irrelevant for performing the task. Neural trajectories are always observed to be low-dimensional. In addition, our results further constrain the mechanisms underlying time encoding as we find that the linear "ramping" component of each neuron's firing rate strongly contributes to the slow timescale variations that make decoding time possible. These constraints rule out working memory models that rely on constant, sustained activity and neural networks with high-dimensional trajectories, like reservoir networks. Instead, recurrent networks trained with backpropagation capture the time-encoding properties and the dimensionality observed in the data.

Analog Resonance Computation: A New Model for Human Cognition

Early models of human cognition appeared to posit the brain as a collection of discrete digital computing modules with specific data processing functions. More recent theories such as the Hierarchically Mechanistic Mind characterize the brain as a massive hierarchy of interconnected and adaptive circuits whose primary aim is to reduce entropy. However, studies in high workload/stress situations show that human behavior is often error prone and seemingly irrational. Rather than regarding such behavior to be uncharacteristic, this paper suggest that such "atypical" behavior provides the best information on which to base theories of human cognition. Rather than using a digital paradigm, human cognition should be seen as an analog computer based on resonating circuits whose primary driver is to constantly extract information from the massively complex and rapidly changing world around us to construct an internal model of reality that allows us to rapidly respond to the threats and opportunities.

Engineering recurrent neural networks from task-relevant manifolds and dynamics

Many cognitive processes involve transformations of distributed representations in neural populations, creating a need for population-level models. Recurrent neural network models fulfill this need, but there are many open questions about how their connectivity gives rise to dynamics that solve a task. Here, we present a method for finding the connectivity of networks for which the dynamics are specified to solve a task in an interpretable way. We apply our method to a working memory task by synthesizing a network that implements a drift-diffusion process over a ring-shaped manifold. We also use our method to demonstrate how inputs can be used to control network dynamics for cognitive flexibility and explore the relationship between representation geometry and network capacity. Our work fits within the broader context of understanding neural computations as dynamics over relatively low-dimensional manifolds formed by correlated patterns of neurons.

On Simulating Neural Damage in Connectionist Networks

A key strength of connectionist modelling is its ability to simulate both intact cognition and the behavioural effects of neural damage. We survey the literature, showing that models have been damaged in a variety of ways, e.g. by removing connections, by adding noise to connection weights, by scaling weights, by removing units and by adding noise to unit activations. While these different implementations of damage have often been assumed to be behaviourally equivalent, some theorists have made aetiological claims that rest on nonequivalence. They suggest that related deficits with different aetiologies might be accounted for by different forms of damage within a single model. We present two case studies that explore the effects of different forms of damage in two influential connectionist models, each of which has been applied to explain neuropsychological deficits. Our results indicate that the effect of simulated damage can indeed be sensitive to the way in which damage is implemented, particularly when the environment comprises subsets of items that differ in their statistical properties, but such effects are sensitive to relatively subtle aspects of the model's training environment. We argue that, as a consequence, substantial methodological care is required if aetiological claims about simulated neural damage are to be justified, and conclude more generally that implementation assumptions, including those concerning simulated damage, must be fully explored when evaluating models of neurological deficits, both to avoid over-extending the explanatory power of specific implementations and to ensure that reported results are replicable.

ELECTRONIC SUPPLEMENTARY MATERIAL

The online version of this article (10.1007/s42113-020-00081-z) contains supplementary material, which is available to authorized users.

Dynamic representations in networked neural systems

A group of neurons can generate patterns of activity that represent information about stimuli; subsequently, the group can transform and transmit activity patterns across synapses to spatially distributed areas. Recent studies in neuroscience have begun to independently address the two components of information processing: the representation of stimuli in neural activity and the transmission of information in networks that model neural interactions. Yet only recently are studies seeking to link these two types of approaches. Here we briefly review the two separate bodies of literature; we then review the recent strides made to address this gap. We continue with a discussion of how patterns of activity evolve from one representation to another, forming dynamic representations that unfold on the underlying network. Our goal is to offer a holistic framework for understanding and describing neural information representation and transmission while revealing exciting frontiers for future research.

Deciphering image contrast in object classification deep networks

The ultimate goal of neuroscience is to explain how complex behaviour arises from neuronal activity. A comparable level of complexity also emerges in deep neural networks (DNNs) while exhibiting human-level performance in demanding visual tasks. Unlike in biological systems, all parameters and operations of DNNs are accessible. Therefore, in theory, it should be possible to decipher the exact mechanisms learnt by these artificial networks. Here, we investigate the concept of contrast invariance within the framework of DNNs. We start by discussing how a network can achieve robustness to changes in local and global image contrast. We used a technique from neuroscience-"kernel lesion"-to measure the degree of performance degradation when individual kernels are eliminated from a network. We further compared contrast normalisation, a mechanism used in biological systems, to the strategies that DNNs learn to cope with changes of contrast. The results of our analysis suggest that (i) contrast is a low-level feature for these networks, and it is encoded in the shallow layers; (ii) a handful of kernels appear to have a greater impact on this feature, and their removal causes a substantially larger accuracy loss for low-contrast images; (iii) edges are a distinct visual feature within the internal representation of object classification DNNs.

Understanding the computation of time using neural network models

Perceiving, maintaining, and using time intervals in working memory are crucial for animals to anticipate or act correctly at the right time in the ever-changing world. Here, we systematically study the underlying neural mechanisms by training recurrent neural networks to perform temporal tasks or complex tasks in combination with spatial information processing and decision making. We found that neural networks perceive time through state evolution along stereotypical trajectories and produce time intervals by scaling evolution speed. Temporal and nontemporal information is jointly coded in a way that facilitates decoding generalizability. We also provided potential sources for the temporal signals observed in nontiming tasks. Our study revealed the computational principles of a number of experimental phenomena and provided several predictions.

To maximize future rewards in this ever-changing world, animals must be able to discover the temporal structure of stimuli and then anticipate or act correctly at the right time. How do animals perceive, maintain, and use time intervals ranging from hundreds of milliseconds to multiseconds in working memory? How is temporal information processed concurrently with spatial information and decision making? Why are there strong neuronal temporal signals in tasks in which temporal information is not required? A systematic understanding of the underlying neural mechanisms is still lacking. Here, we addressed these problems using supervised training of recurrent neural network models. We revealed that neural networks perceive elapsed time through state evolution along stereotypical trajectory, maintain time intervals in working memory in the monotonic increase or decrease of the firing rates of interval-tuned neurons, and compare or produce time intervals by scaling state evolution speed. Temporal and nontemporal information is coded in subspaces orthogonal with each other, and the state trajectories with time at different nontemporal information are quasiparallel and isomorphic. Such coding geometry facilitates the decoding generalizability of temporal and nontemporal information across each other. The network structure exhibits multiple feedforward sequences that mutually excite or inhibit depending on whether their preferences of nontemporal information are similar or not. We identified four factors that facilitate strong temporal signals in nontiming tasks, including the anticipation of coming events. Our work discloses fundamental computational principles of temporal processing, and it is supported by and gives predictions to a number of experimental phenomena.

Hand Knob Area of Premotor Cortex Represents the Whole Body in a Compositional Way

Decades after the motor homunculus was first proposed, it is still unknown how different body parts are intermixed and interrelated in human motor cortical areas at single-neuron resolution. Using multi-unit recordings, we studied how face, head, arm, and leg movements are represented in the hand knob area of premotor cortex (precentral gyrus) in people with tetraplegia. Contrary to traditional expectations, we found strong representation of all movements and a partially "compositional" neural code that linked together all four limbs. The code consisted of (1) a limb-coding component representing the limb to be moved and (2) a movement-coding component where analogous movements from each limb (e.g., hand grasp and toe curl) were represented similarly. Compositional coding might facilitate skill transfer across limbs, and it provides a useful framework for thinking about how the motor system constructs movement. Finally, we leveraged these results to create a whole-body intracortical brain-computer interface that spreads targets across all limbs.

Temporal chunking as a mechanism for unsupervised learning of task-sets

Depending on environmental demands, humans can learn and exploit multiple concurrent sets of stimulus-response associations. Mechanisms underlying the learning of such task-sets remain unknown. Here we investigate the hypothesis that task-set learning relies on unsupervised chunking of stimulus-response associations that occur in temporal proximity. We examine behavioral and neural data from a task-set learning experiment using a network model. We first show that task-set learning can be achieved provided the timescale of chunking is slower than the timescale of stimulus-response learning. Fitting the model to behavioral data on a subject-by-subject basis confirmed this expectation and led to specific predictions linking chunking and task-set retrieval that were borne out by behavioral performance and reaction times. Comparing the model activity with BOLD signal allowed us to identify neural correlates of task-set retrieval in a functional network involving ventral and dorsal prefrontal cortex, with the dorsal system preferentially engaged when retrievals are used to improve performance.

Reevaluating the Role of Persistent Neural Activity in Short-Term Memory

A traditional view of short-term working memory (STM) is that task-relevant information is maintained 'online' in persistent spiking activity. However, recent experimental and modeling studies have begun to question this long-held belief. In this review, we discuss new evidence demonstrating that information can be 'silently' maintained via short-term synaptic plasticity (STSP) without the need for persistent activity. We discuss how the neural mechanisms underlying STM are inextricably linked with the cognitive demands of the task, such that the passive maintenance and the active manipulation of information are subserved differently in the brain. Together, these recent findings point towards a more nuanced view of STM in which multiple substrates work in concert to support our ability to temporarily maintain and manipulate information.

Discovering the Computational Relevance of Brain Network Organization

Understanding neurocognitive computations will require not just localizing cognitive information distributed throughout the brain but also determining how that information got there. We review recent advances in linking empirical and simulated brain network organization with cognitive information processing. Building on these advances, we offer a new framework for understanding the role of connectivity in cognition: network coding (encoding/decoding) models. These models utilize connectivity to specify the transfer of information via neural activity flow processes, successfully predicting the formation of cognitive representations in empirical neural data. The success of these models supports the possibility that localized neural functions mechanistically emerge (are computed) from distributed activity flow processes that are specified primarily by connectivity patterns.

Passing the Message: Representation Transfer in Modular Balanced Networks

Neurobiological systems rely on hierarchical and modular architectures to carry out intricate computations using minimal resources. A prerequisite for such systems to operate adequately is the capability to reliably and efficiently transfer information across multiple modules. Here, we study the features enabling a robust transfer of stimulus representations in modular networks of spiking neurons, tuned to operate in a balanced regime. To capitalize on the complex, transient dynamics that such networks exhibit during active processing, we apply reservoir computing principles and probe the systems' computational efficacy with specific tasks. Focusing on the comparison of random feed-forward connectivity and biologically inspired topographic maps, we find that, in a sequential set-up, structured projections between the modules are strictly necessary for information to propagate accurately to deeper modules. Such mappings not only improve computational performance and efficiency, they also reduce response variability, increase robustness against interference effects, and boost memory capacity. We further investigate how information from two separate input streams is integrated and demonstrate that it is more advantageous to perform non-linear computations on the input locally, within a given module, and subsequently transfer the result downstream, rather than transferring intermediate information and performing the computation downstream. Depending on how information is integrated early on in the system, the networks achieve similar task-performance using different strategies, indicating that the dimensionality of the neural responses does not necessarily correlate with nonlinear integration, as predicted by previous studies. These findings highlight a key role of topographic maps in supporting fast, robust, and accurate neural communication over longer distances. Given the prevalence of such structural feature, particularly in the sensory systems, elucidating their functional purpose remains an important challenge toward which this work provides relevant, new insights. At the same time, these results shed new light on important requirements for designing functional hierarchical spiking networks.

Considerations in using recurrent neural networks to probe neural dynamics

Recurrent neural networks (RNNs) are increasingly being used to model complex cognitive and motor tasks performed by behaving animals. RNNs are trained to reproduce animal behavior while also capturing key statistics of empirically recorded neural activity. In this manner, the RNN can be viewed as an in silico circuit whose computational elements share similar motifs with the cortical area it is modeling. Furthermore, because the RNN's governing equations and parameters are fully known, they can be analyzed to propose hypotheses for how neural populations compute. In this context, we present important considerations when using RNNs to model motor behavior in a delayed reach task. First, by varying the network's nonlinear activation and rate regularization, we show that RNNs reproducing single-neuron firing rate motifs may not adequately capture important population motifs. Second, we find that even when RNNs reproduce key neurophysiological features on both the single neuron and population levels, they can do so through distinctly different dynamical mechanisms. To distinguish between these mechanisms, we show that an RNN consistent with a previously proposed dynamical mechanism is more robust to input noise. Finally, we show that these dynamics are sufficient for the RNN to generalize to tasks it was not trained on. Together, these results emphasize important considerations when using RNN models to probe neural dynamics.NEW & NOTEWORTHY Artificial neurons in a recurrent neural network (RNN) may resemble empirical single-unit activity but not adequately capture important features on the neural population level. Dynamics of RNNs can be visualized in low-dimensional projections to provide insight into the RNN's dynamical mechanism. RNNs trained in different ways may reproduce neurophysiological motifs but do so with distinctly different mechanisms. RNNs trained to only perform a delayed reach task can generalize to perform tasks where the target is switched or the target location is changed.

Behavioural analysis of single-cell aneural ciliate, Stentor roeseli, using machine learning approaches

There is still a significant gap between our understanding of neural circuits and the behaviours they compute-i.e. the computations performed by these neural networks (Carandini 2012 Nat. Neurosci.15, 507-509. (doi:10.1038/nn.3043)). Cellular decision-making processes, learning, behaviour and memory formation-all that have been only associated with animals with neural systems-have also been observed in many unicellular aneural organisms, namely Physarum, Paramecium and Stentor (Tang & Marshall2018 Curr. Biol.28, R1180-R1184. (doi:10.1016/j.cub.2018.09.015)). As these are fully functioning organisms, yet being unicellular, there is a much better chance to elucidate the detailed mechanisms underlying these learning processes in these organisms without the complications of highly interconnected neural circuits. An intriguing learning behaviour observed in Stentor roeseli (Jennings 1902 Am. J. Physiol. Legacy Content8, 23-60. (doi:10.1152/ajplegacy.1902.8.1.23)) when stimulated with carmine has left scientists puzzled for more than a century. So far, none of the existing learning paradigm can fully encapsulate this particular series of five characteristic avoidance reactions. Although we were able to observe all responses described in the literature and in a previous study (Dexter et al. 2019), they do not conform to any particular learning model. We then investigated whether models inferred from machine learning approaches, including decision tree, random forest and feed-forward artificial neural networks could infer and predict the behaviour of S. roeseli. Our results showed that an artificial neural network with multiple 'computational' neurons is inefficient at modelling the single-celled ciliate's avoidance reactions. This has highlighted the complexity of behaviours in aneural organisms. Additionally, this report will also discuss the significance of elucidating molecular details underlying learning and decision-making processes in these unicellular organisms, which could offer valuable insights that are applicable to higher animals.

Task-Dependent Changes in the Large-Scale Dynamics and Necessity of Cortical Regions

Neural activity throughout the cortex is correlated with perceptual decisions, but inactivation studies suggest that only a small number of areas are necessary for these behaviors. Here we show that the number of required cortical areas and their dynamics vary across related tasks with different cognitive computations. In a visually guided virtual T-maze task, bilateral inactivation of only a few dorsal cortical regions impaired performance. In contrast, in tasks requiring evidence accumulation and/or post-stimulus memory, performance was impaired by inactivation of widespread cortical areas with diverse patterns of behavioral deficits across areas and tasks. Wide-field imaging revealed widespread ramps of Ca2+ activity during the accumulation and visually guided tasks. Additionally, during accumulation, different regions had more diverse activity profiles, leading to reduced inter-area correlations. Using a modular recurrent neural network model trained to perform analogous tasks, we argue that differences in computational strategies alone could explain these findings.

Neural representation of newly instructed rule identities during early implementation trials

By following explicit instructions, humans instantaneously get the hang of tasks they have never performed before. We used a specially calibrated multivariate analysis technique to uncover the elusive representational states during the first few implementations of arbitrary rules such as ‘for coffee, press red button’ following their first-time instruction. Distributed activity patterns within the ventrolateral prefrontal cortex (VLPFC) indicated the presence of neural representations specific of individual stimulus-response (S-R) rule identities, preferentially for conditions requiring the memorization of instructed S-R rules for correct performance. Identity-specific representations were detectable starting from the first implementation trial and continued to be present across early implementation trials. The increasingly fluent application of novel rule representations was channelled through increasing cooperation between VLPFC and anterior striatum. These findings inform representational theories on how the prefrontal cortex supports behavioral flexibility specifically by enabling the ad-hoc coding of newly instructed individual rule identities during their first-time implementation.

Time-invariant working memory representations in the presence of code-morphing in the lateral prefrontal cortex

Maintenance of working memory is thought to involve the activity of prefrontal neuronal populations with strong recurrent connections. However, it was recently shown that distractors evoke a morphing of the prefrontal population code, even when memories are maintained throughout the delay. How can a morphing code maintain time-invariant memory information? We hypothesized that dynamic prefrontal activity contains time-invariant memory information within a subspace of neural activity. Using an optimization algorithm, we found a low-dimensional subspace that contains time-invariant memory information. This information was reduced in trials where the animals made errors in the task, and was also found in periods of the trial not used to find the subspace. A bump attractor model replicated these properties, and provided predictions that were confirmed in the neural data. Our results suggest that the high-dimensional responses of prefrontal cortex contain subspaces where different types of information can be simultaneously encoded with minimal interference.

One Step Back, Two Steps Forward: Interference and Learning in Recurrent Neural Networks

Artificial neural networks, trained to perform cognitive tasks, have recently been used as models for neural recordings from animals performing these tasks. While some progress has been made in performing such comparisons, the evolution of network dynamics throughout learning remains unexplored. This is paralleled by an experimental focus on recording from trained animals, with few studies following neural activity throughout training.

In this work, we address this gap in the realm of artificial networks by analyzing networks that are trained to perform memory and pattern generation tasks. The functional aspect of these tasks corresponds to dynamical objects in the fully trained network—a line attractor or a set of limit cycles for the two respective tasks. We use these dynamical objects as anchors to study the effect of learning on their emergence. We find that the sequential nature of learning—one trial at a time—has major consequences for the learning trajectory and its final outcome. Specifically, we show that least mean squares (LMS), a simple gradient descent suggested as a biologically plausible version of the FORCE algorithm, is constantly obstructed by forgetting, which is manifested as the destruction of dynamical objects from previous trials. The degree of interference is determined by the correlation between different trials. We show which specific ingredients of FORCE avoid this phenomenon. Overall, this difference results in convergence that is orders of magnitude slower for LMS.

Learning implies accumulating information across multiple trials to form the overall concept of the task. Our results show that interference between trials can greatly affect learning in a learning-rule-dependent manner. These insights can help design experimental protocols that minimize such interference, and possibly infer underlying learning rules by observing behavior and neural activity throughout learning.

Simultaneous multi-area recordings suggest that attention improves performance by reshaping stimulus representations

Visual attention dramatically improves individuals' ability to see and modulates the responses of neurons in every known visual and oculomotor area, but whether such modulations can account for perceptual improvements is unclear. We measured the relationship between populations of visual neurons, oculomotor neurons and behavior during detection and discrimination tasks. We found that neither of the two prominent hypothesized neuronal mechanisms underlying attention (which concern changes in information coding and the way sensory information is read out) provide a satisfying account of the observed behavioral improvements. Instead, our results are more consistent with the hypothesis that attention reshapes the representation of attended stimuli to more effectively influence behavior. Our results suggest a path toward understanding the neural underpinnings of perception and cognition in health and disease by analyzing neuronal responses in ways that are constrained by behavior and interactions between brain areas.

Harnessing behavioral diversity to understand neural computations for cognition

With the increasing acquisition of large-scale neural recordings comes the challenge of inferring the computations they perform and understanding how these give rise to behavior. Here, we review emerging conceptual and technological advances that begin to address this challenge, garnering insights from both biological and artificial neural networks. We argue that neural data should be recorded during rich behavioral tasks, to model cognitive processes and estimate latent behavioral variables. Careful quantification of animal movements can also provide a more complete picture of how movements shape neural dynamics and reflect changes in brain state, such as arousal or stress. Artificial neural networks (ANNs) could serve as artificial model organisms to connect neural dynamics and rich behavioral data. ANNs have already begun to reveal how a wide range of different behaviors can be implemented, generating hypotheses about how observed neural activity might drive behavior and explaining diversity in behavioral strategies.

How to study the neural mechanisms of multiple tasks

Most biological and artificial neural systems are capable of completing multiple tasks. However, the neural mechanism by which multiple tasks are accomplished within the same system is largely unclear. We start by discussing how different tasks can be related, and methods to generate large sets of inter-related tasks to study how neural networks and animals perform multiple tasks. We then argue that there are mechanisms that emphasize either specialization or flexibility. We will review two such neural mechanisms underlying multiple tasks at the neuronal level (modularity and mixed selectivity), and discuss how different mechanisms can emerge depending on training methods in neural networks.