Coding and learning of behavioral sequences

Hey, look over there: Distraction effects on rapid sequence recall

In the course of everyday life, the brain must store and recall a huge variety of representations of stimuli which are presented in an ordered or sequential way. The processes by which the ordering of these various things is stored and recalled are moderately well understood. We use here a computational model of a cortex-like recurrent neural network adapted by a multitude of plasticity mechanisms. We first demonstrate the learning of a sequence. Then, we examine the influence of different types of distractors on the network dynamics during the recall of the encoded ordered information being ordered in a sequence. We are able to broadly arrive at two distinct effect-categories for distractors, arrive at a basic understanding of why this is so, and predict what distractors will fall into each category.

How single neuron properties shape chaotic dynamics and signal transmission in random neural networks

While most models of randomly connected neural networks assume single-neuron models with simple dynamics, neurons in the brain exhibit complex intrinsic dynamics over multiple timescales. We analyze how the dynamical properties of single neurons and recurrent connections interact to shape the effective dynamics in large randomly connected networks. A novel dynamical mean-field theory for strongly connected networks of multi-dimensional rate neurons shows that the power spectrum of the network activity in the chaotic phase emerges from a nonlinear sharpening of the frequency response function of single neurons. For the case of two-dimensional rate neurons with strong adaptation, we find that the network exhibits a state of "resonant chaos", characterized by robust, narrow-band stochastic oscillations. The coherence of stochastic oscillations is maximal at the onset of chaos and their correlation time scales with the adaptation timescale of single units. Surprisingly, the resonance frequency can be predicted from the properties of isolated neurons, even in the presence of heterogeneity in the adaptation parameters. In the presence of these internally-generated chaotic fluctuations, the transmission of weak, low-frequency signals is strongly enhanced by adaptation, whereas signal transmission is not influenced by adaptation in the non-chaotic regime. Our theoretical framework can be applied to other mechanisms at the level of single neurons, such as synaptic filtering, refractoriness or spike synchronization. These results advance our understanding of the interaction between the dynamics of single units and recurrent connectivity, which is a fundamental step toward the description of biologically realistic neural networks.

Hippocampal CA1 replay becomes less prominent but more rigid without inputs from medial entorhinal cortex

The hippocampus is an essential brain area for learning and memory. However, the network mechanisms underlying memory storage, consolidation and retrieval remain incompletely understood. Place cell sequences during theta oscillations are thought to be replayed during non-theta states to support consolidation and route planning. In animals with medial entorhinal cortex (MEC) lesions, the temporal organization of theta-related hippocampal activity is disrupted, which allows us to test whether replay is also compromised. Two different analyses—comparison of co-activation patterns between running and rest epochs and analysis of the recurrence of place cell sequences—reveal that the enhancement of replay by behavior is reduced in MEC-lesioned versus control rats. In contrast, the degree of intrinsic network structure prior and subsequent to behavior remains unaffected by MEC lesions. The MEC-dependent temporal coordination during theta states therefore appears to facilitate behavior-related plasticity, but does not disrupt pre-existing functional connectivity.

Medial entorhinal cortex (MEC) is involved in memory processes that entail the replay of sequential firing of hippocampal place cells during rest periods and during behaviour. Here, the authors show that MEC lesioned animals show intact replay after an epoch of rats running on a linear track, while replay during the behavioral epoch is reduced.

Remembering rewarding futures: A simulation-selection model of the hippocampus

Despite tremendous progress, the neural circuit dynamics underlying hippocampal mnemonic processing remain poorly understood. We propose a new model for hippocampal function-the simulation-selection model-based on recent experimental findings and neuroecological considerations. Under this model, the mammalian hippocampus evolved to simulate and evaluate arbitrary navigation sequences. Specifically, we suggest that CA3 simulates unexperienced navigation sequences in addition to remembering experienced ones, and CA1 selects from among these CA3-generated sequences, reinforcing those that are likely to maximize reward during offline idling states. High-value sequences reinforced in CA1 may allow flexible navigation toward a potential rewarding location during subsequent navigation. We argue that the simulation-selection functions of the hippocampus have evolved in mammals mostly because of the unique navigational needs of land mammals. Our model may account for why the mammalian hippocampus has evolved not only to remember, but also to imagine episodes, and how this might be implemented in its neural circuits.

Traveling Theta Waves and the Hippocampal Phase Code

Hippocampal place fields form a neuronal map of the spatial environment. In addition, the distance between two place field centers is proportional to the firing phase difference of two place cells with respect to the local theta rhythm. This consistency between spatial distance and theta phase is generally assumed to result from hippocampal phase precession: The firing phase of a place cell decreases with distance traveled in the place field. The rate of phase precession depends on place field width such that the phase range covered in a traversal of a place field is independent of field width. Width-dependent precession rates, however, generally disrupt the consistency between distance and phase differences. In this paper we provide a mathematical theory suggesting that this consistency can only be secured for different place field widths if phase precession starts at a width-dependent phase offset. These offsets are in accordance with the experimentally observed theta wave traveling from the dorsal to the ventral pole of the hippocampus. Furthermore the theory predicts that sequences of place cells with different widths should be ordered according to the end of the place field. The results also hold for considerably nonlinear phase precession profiles.

Flexible theta sequence compression mediated via phase precessing interneurons

Encoding of behavioral episodes as spike sequences during hippocampal theta oscillations provides a neural substrate for computations on events extended across time and space. However, the mechanisms underlying the numerous and diverse experimentally observed properties of theta sequences remain poorly understood. Here we account for theta sequences using a novel model constrained by the septo-hippocampal circuitry. We show that when spontaneously active interneurons integrate spatial signals and theta frequency pacemaker inputs, they generate phase precessing action potentials that can coordinate theta sequences in place cell populations. We reveal novel constraints on sequence generation, predict cellular properties and neural dynamics that characterize sequence compression, identify circuit organization principles for high capacity sequential representation, and show that theta sequences can be used as substrates for association of conditioned stimuli with recent and upcoming events. Our results suggest mechanisms for flexible sequence compression that are suited to associative learning across an animal’s lifespan.

DOI:http://dx.doi.org/10.7554/eLife.20349.001

Nerve cells in the brain exchange information via electrical impulses. In a given brain area, the electrical impulses at any particular moment can be thought of as forming a code that represents an aspect of the outside world. For example, groups of nerve cells in the hippocampus generate a type of code called a theta sequence, which represents a series of recent and upcoming events. The specific timing of electrical impulses within a theta sequence is crucial in creating certain types of memory.

There are two major classes of nerve cell in the brain: excitatory cells activate impulses in neighbouring cells, while inhibitory cells act to temporarily block impulses from other nerve cells. Many groups, or “circuits”, of nerve cells contain combinations of both cell types to control how and when they communicate. Previous studies show that both types of cell are active within theta sequences, but it is not known precisely how they contribute to creating the sequences.

Chadwick et al. developed a new mathematical model that simulates how theta sequences can emerge from circuits of both excitatory and inhibitory nerve cells. The connections between these simulated cells are based on experimental data from real nerve cells in the hippocampus. The model predicts that inhibitory cells play an important role in generating theta sequences by interacting with groups of excitatory cells to coordinate the timing of electrical impulses. Furthermore, the model shows how memory capacity depends on these connections.

The next step following on from this work is to carry out experiments to test the model’s predictions. This will include monitoring the same group of nerve cells in multiple different situations to find out how their theta sequences change, and recording electrical events in individual nerve cells during theta sequences. If the theory’s predictions are confirmed this would lead to a deeper understanding of how our brains remember sequences of events.

DOI:http://dx.doi.org/10.7554/eLife.20349.002

The medial entorhinal cortex is necessary for temporal organization of hippocampal neuronal activity

The superficial layers of the medial entorhinal cortex (MEC) are the major input to the hippocampus. The high proportion of spatially modulated cells, including grid cells and border cells, in these layers suggests that the MEC inputs to the hippocampus are critical for the representation of space in the hippocampus. However, selective manipulations of the MEC do not completely abolish hippocampal spatial firing. To therefore determine whether other hippocampal firing characteristics depend more critically on MEC inputs, we recorded from hippocampal CA1 cells in rats with MEC lesions. Strikingly, theta phase precession was substantially disrupted, even during periods of stable spatial firing. Our findings indicate that MEC inputs to the hippocampus are required for the temporal organization of hippocampal firing patterns and suggest that cognitive functions that depend on precise neuronal sequences within the hippocampal theta cycle are particularly dependent on the MEC.

Multiple forms of metaplasticity at a single hippocampal synapse during late postnatal development

Metaplasticity refers to adjustment in the requirements for induction of synaptic plasticity based on the prior history of activity. Numerous forms of developmental metaplasticity are observed at Schaffer collateral synapses in the rat hippocampus at the end of the third postnatal week. Emergence of spatial learning and memory at this developmental stage suggests possible involvement of metaplasticity in the final maturation of the hippocampus. Three distinct metaplastic phenomena are apparent. (1) As transmitter release probability increases with increasing age, presynaptic potentiation is reduced. (2) Alterations in the composition and channel conductance properties of AMPARs facilitate the induction of postsynaptic potentiation with increasing age. (3) Low frequency stimulation inhibits subsequent induction of potentiation in animals older but not younger than 3 weeks of age. Thus, many forms of plasticity expressed at SC-CA1 synapses are different in rats younger and older than 3 weeks of age, illustrating the complex orchestration of physiological modifications that underlie the maturation of hippocampal excitatory synaptic transmission. This review paper describes three late postnatal modifications to synaptic plasticity induction in the hippocampus and attempts to relate these metaplastic changes to developmental alterations in hippocampal network activity and the maturation of contextual learning.

Learning a Sparse Code for Temporal Sequences Using STDP and Sequence Compression

A spiking neural network that learns temporal sequences is described. A sparse code in which individual neurons represent sequences and subsequences enables multiple sequences to be stored without interference. The network is founded on a model of sequence compression in the hippocampus that is robust to variation in sequence element duration and well suited to learn sequences through spike-timing dependent plasticity (STDP). Three additions to the sequence compression model underlie the sparse representation: synapses connecting the neurons of the network that are subject to STDP, a competitive plasticity rule so that neurons specialize to individual sequences, and neural depolarization after spiking so that neurons have a memory. The response to new sequence elements is determined by the neurons that have responded to the previous subsequence, according to the competitively learned synaptic connections. Numerical simulations show that the model can learn sets of intersecting sequences, presented with widely differing frequencies, with elements of varying duration.

Spiking Neuron Model for Temporal Sequence Recognition

A biologically inspired neuronal network that stores and recognizes temporal sequences of symbols is described. Each symbol is represented by excitatory input to distinct groups of neurons (symbol pools). Unambiguous storage of multiple sequences with common subsequences is ensured by partitioning each symbol pool into subpools that respond only when the current symbol has been preceded by a particular sequence of symbols. We describe synaptic structure and neural dynamics that permit the selective activation of subpools by the correct sequence. Symbols may have varying durations of the order of hundreds of milliseconds. Physiologically plausible plasticity mechanisms operate on a time scale of tens of milliseconds; an interaction of the excitatory input with periodic global inhibition bridges this gap so that neural events representing successive symbols occur on this much faster timescale. The network is shown to store multiple overlapping sequences of events. It is robust to variation in symbol duration, it is scalable, and its performance degrades gracefully with perturbation of its parameters.

Single-trial phase precession in the hippocampus

During the crossing of the place field of a pyramidal cell in the rat hippocampus, the firing phase of the cell decreases with respect to the local theta rhythm. This phase precession is usually studied on the basis of data in which many place field traversals are pooled together. Here we study properties of phase precession in single trials. We found that single-trial and pooled-trial phase precession were different with respect to phase-position correlation, phase-time correlation, and phase range. Whereas pooled-trial phase precession may span 360 degrees , the most frequent single-trial phase range was only approximately 180 degrees. In pooled trials, the correlation between phase and position (r = -0.58) was stronger than the correlation between phase and time (r = -0.27), whereas in single trials these correlations (r = -0.61 for both) were not significantly different. Next, we demonstrated that phase precession exhibited a large trial-to-trial variability. Overall, only a small fraction of the trial-to-trial variability in measures of phase precession (e.g., slope or offset) could be explained by other single-trial properties (such as running speed or firing rate), whereas the larger part of the variability remains to be explained. Finally, we found that surrogate single trials, created by randomly drawing spikes from the pooled data, are not equivalent to experimental single trials: pooling over trials therefore changes basic measures of phase precession. These findings indicate that single trials may be better suited for encoding temporally structured events than is suggested by the pooled data.

Differing presynaptic contributions to LTP and associative learning in behaving mice

The hippocampal CA3-CA1 synapse is an excellent experimental model for studying the interactions between short- and long-term plastic changes taking place following high-frequency stimulation (HFS) of Schaffer collaterals and during the acquisition and extinction of a classical eyeblink conditioning in behaving mice. Input/output curves and a full-range paired-pulse study enabled determining the optimal intensities and inter-stimulus intervals for evoking paired-pulse facilitation (PPF) or depression (PPD) at the CA3-CA1 synapse. Long-term potentiation (LTP) induced by HFS lasted ≈10 days. HFS-induced LTP evoked an initial depression of basal PPF. Recovery of PPF baseline values was a steady and progressive process lasting ≈20 days, i.e., longer than the total duration of the LTP. In a subsequent series of experiments, we checked whether PPF was affected similarly during activity-dependent synaptic changes. Animals were conditioned using a trace paradigm, with a tone as a conditioned stimulus (CS) and an electrical shock to the trigeminal nerve as an unconditioned stimulus (US). A pair of pulses (40 ms interval) was presented to the Schaffer collateral-commissural pathway to evoke field EPSPs (fEPSPs) during the CS-US interval. Basal PPF decreased steadily across conditioning sessions (i.e., in the opposite direction to that during LTP), reaching a minimum value during the 10th conditioning session. Thus, LTP and classical eyeblink conditioning share some presynaptic mechanisms, but with an opposite evolution. Furthermore, PPF and PPD might play a homeostatic role during long-term plastic changes at the CA3-CA1 synapse.

Transient dynamics for sequence-processing neural networks: effect of degree distributions

We derive an analytic evolution equation for overlap parameters, including the effect of degree distribution on the transient dynamics of sequence processing neural networks. In the special case of globally coupled networks, the precisely retrieved critical loading ratio alpha_{c}=N;{-12} is obtained, where N is the network size. In the presence of random networks, our theoretical predictions agree quantitatively with the numerical experiments for delta, binomial, and power-law degree distributions.

Generation of theta oscillations by weakly coupled neural oscillators in the presence of noise

Neuronal oscillations are a robust phenomenon occurring in a variety of brain regions despite considerable amounts of noise. In this article classical phase-response theory is generalized to the case of noisy weak-coupling regimes by deriving an iterated map for the asynchrony of spikes in an oscillation cycle. Two criteria are introduced to check the validity of our approximations: One criterion tests the assumption that all neurons fire exactly once per cycle, the other criterion tests for linearity. The framework is applied to stellate cells of the medial entorhinal cortex layer II. We find that rhythmogenesis is more robust in the case of excitatory noise as compared to inhibitory noise. It is shown that a network of stellate cells can also act as a generator of theta if the neurons are connected via a fast-oscillating network of inhibitory interneurons.

Neural dynamics of the cognitive map in the hippocampus

The rodent hippocampus has been thought to represent the spatial environment as a cognitive map. In the classical theory, the cognitive map has been explained as a consequence of the fact that different spatial regions are assigned to different cell populations in the framework of rate coding. Recently, the relation between place cell firing and local field oscillation theta in terms of theta phase precession was experimentally discovered and suggested as a temporal coding mechanism leading to memory formation of behavioral sequences accompanied with asymmetric Hebbian plasticity. The cognitive map theory is apparently outside of the sequence memory view. Therefore, theoretical analysis is necessary to consider the biological neural dynamics for the sequence encoding of the memory of behavioral sequences, providing the cognitive map formation. In this article, we summarize the theoretical neural dynamics of the real-time sequence encoding by theta phase precession, called theta phase coding, and review a series of theoretical models with the theta phase coding that we previously reported. With respect to memory encoding functions, instantaneous memory formation of one-time experience was first demonstrated, and then the ability of integration of memories of behavioral sequences into a network of the cognitive map was shown. In terms of memory retrieval functions, theta phase coding enables the hippocampus to represent the spatial location in the current behavioral context even with ambiguous sensory input when multiple sequences were coded. Finally, for utilization, retrieved temporal sequences in the hippocampus can be available for action selection, through the process of reverting theta rhythm-dependent activities to information in the behavioral time scale. This theoretical approach allows us to investigate how the behavioral sequences are encoded, updated, retrieved and used in the hippocampus, as the real-time interaction with the external environment. It may indeed be the bridge to the episodic memory function in human hippocampus.

Generation and reshaping of sequences in neural systems

The generation of informational sequences and their reorganization or reshaping is one of the most intriguing subjects for both neuroscience and the theory of autonomous intelligent systems. In spite of the diversity of sequential activities of sensory, motor, and cognitive neural systems, they have many similarities from the dynamical point of view. In this review we discus the ideas, models, and mathematical image of sequence generation and reshaping on different levels of the neural hierarchy, i.e., the role of a sensory network dynamics in the generation of a motor program (hunting swimming of marine mollusk Clione), olfactory dynamical coding, and sequential learning and decision making. Analysis of these phenomena is based on the winnerless competition principle. The considered models can be a basis for the design of biologically inspired autonomous intelligent systems.

Experimental study of electrical FitzHugh–Nagumo neurons with modified excitability

We present an electronical circuit modelling a FitzHugh-Nagumo neuron with a modified excitability. To characterize this basic cell, the bifurcation curves between stability with excitation threshold, bistability and oscillations are investigated. An electrical circuit is then proposed to realize a unidirectional coupling between two cells, mimicking an inter-neuron synaptic coupling. In such a master-slave configuration, we show experimentally how the coupling strength controls the dynamics of the slave neuron, leading to frequency locking, chaotic behavior and synchronization. These phenomena are then studied by phase map analysis. The architecture of a possible neural network is described introducing different kinds of coupling between neurons.

Microstimulation of frontal cortex can reorder a remembered spatial sequence

Complex goal-directed behaviors extend over time and thus depend on the ability to serially order memories and assemble compound, temporally coordinated movements. Theories of sequential processing range from simple associative chaining to hierarchical models in which order is encoded explicitly and separately from sequence components. To examine how short-term memory and planning for sequences might be coded, we used microstimulation to perturb neural activity in the supplementary eye field (SEF) while animals held a sequence of two cued locations in memory over a short delay. We found that stimulation affected the order in which animals saccaded to the locations, but not the memory for which locations were cued. These results imply that memory for sequential order can be dissociated from that of its components. Furthermore, stimulation of the SEF appeared to bias sequence endpoints to converge toward a location in contralateral space, suggesting that this area encodes sequences in terms of their endpoints rather than their individual components.

Conserving total synaptic weight ensures one-trial sequence learning of place fields in the hippocampus

The hippocampus plays a critical role in the rapid acquisition of information from a novel experience. Recent theoretical studies on the rat hippocampus have shown the possibility of behavioral sequence learning in a single traversal experience by theta phase coding. Specifically, previous work using computer simulations demonstrated that the extent of overlap among individual events of sequence and rat running velocity should be quantitatively incorporated into the learning rule to ensure one-trial sequence learning. These extents of overlap- and running velocity-dependent properties in the learning rule are called the input-dependent regulation of the learning rule. However, the biological meaning of such learning properties remains poorly understood. In this study, we quantitatively derive these learning properties with mathematical analyses. We further find that the input-dependent regulation of the learning rule allows maintenance of total synaptic weight over a given neuron during one-trial learning. Our results predict that a homeostatic plasticity mechanism should exist for conserving total synaptic weight on a rapid timescale.

Spiking dynamics of interacting oscillatory neurons

Spiking sequences emerging from dynamical interaction in a pair of oscillatory neurons are investigated theoretically and experimentally. The model comprises two unidirectionally coupled FitzHugh-Nagumo units with modified excitability (MFHN). The first (master) unit exhibits a periodic spike sequence with a certain frequency. The second (slave) unit is in its excitable mode and responds on the input signal with a complex (chaotic) spike trains. We analyze the dynamic mechanisms underlying different response behavior depending on interaction strength. Spiking phase maps describing the response dynamics are obtained. Complex phase locking and chaotic sequences are investigated. We show how the response spike trains can be effectively controlled by the interaction parameter and discuss the problem of neuronal information encoding.

Temporal sequence learning, prediction, and control: a review of different models and their relation to biological mechanisms

In this review, we compare methods for temporal sequence learning (TSL) across the disciplines machine-control, classical conditioning, neuronal models for TSL as well as spike-timing-dependent plasticity (STDP). This review introduces the most influential models and focuses on two questions: To what degree are reward-based (e.g., TD learning) and correlation-based (Hebbian) learning related? and How do the different models correspond to possibly underlying biological mechanisms of synaptic plasticity? We first compare the different models in an open-loop condition, where behavioral feedback does not alter the learning. Here we observe that reward-based and correlation-based learning are indeed very similar. Machine control is then used to introduce the problem of closed-loop control (e.g., actor-critic architectures). Here the problem of evaluative (rewards) versus nonevaluative (correlations) feedback from the environment will be discussed, showing that both learning approaches are fundamentally different in the closed-loop condition. In trying to answer the second question, we compare neuronal versions of the different learning architectures to the anatomy of the involved brain structures (basal-ganglia, thalamus, and cortex) and the molecular biophysics of glutamatergic and dopaminergic synapses. Finally, we discuss the different algorithms used to model STDP and compare them to reward-based learning rules. Certain similarities are found in spite of the strongly different timescales. Here we focus on the biophysics of the different calcium-release mechanisms known to be involved in STDP.

Hippocampal sequence-encoding driven by a cortical multi-item working memory buffer

Encoding and recall of memory sequences is an important process. Memory encoding is thought to occur by long-term potentiation (LTP) in the hippocampus; however, it remains unclear how LTP, which has a time window for induction of approximately 100 ms, could encode the linkage between sequential items that arrive with a temporal separation >100 ms. Here, we argue that LTP can underlie the learning of such memory sequences, provided the input to the hippocampus is from a cortical multi-item working memory buffer in which theta and gamma oscillations have an important role. In such a buffer, memory items that occurred seconds apart are represented with a temporal separation of 20-30 ms, thereby bringing them within the LTP window. The physiological and behavioral evidence for such a buffer will be reviewed.

Local learning rules: predicted influence of dendritic location on synaptic modification in spike-timing-dependent plasticity

Recent indirect experimental evidence suggests that synaptic plasticity changes along the dendrites of a neuron. Here we present a synaptic plasticity rule which is controlled by the properties of the pre- and postsynaptic signals. Using recorded membrane traces of back-propagating and dendritic spikes we demonstrate that LTP and LTD will depend specifically on the shape of the postsynaptic depolarization at a given dendritic site. We find that asymmetrical spike-timing-dependent plasticity (STDP) can be replaced by temporally symmetrical plasticity within physiologically relevant time windows if the postsynaptic depolarization rises shallow. Presynaptically the rule depends on the NMDA channel characteristic, and the model predicts that an increase in Mg(2+) will attenuate the STDP curve without changing its shape. Furthermore, the model suggests that the profile of LTD should be governed by the postsynaptic signal while that of LTP mainly depends on the presynaptic signal shape.

Reproducible sequence generation in random neural ensembles

Little is known about the conditions that neural circuits have to satisfy to generate reproducible sequences. Evidently, the genetic code cannot control all the details of the complex circuits in the brain. In this Letter, we give the conditions on the connectivity degree that lead to reproducible and robust sequences in a neural population of randomly coupled excitatory and inhibitory neurons. In contrast to the traditional theoretical view we show that the sequences do not need to be learned. In the framework proposed here just the averaged characteristics of the random circuits have to be under genetic control. We found that rhythmic sequences can be generated if random networks are in the vicinity of an excitatory-inhibitory synaptic balance. Reproducible transient sequences, on the other hand, are found far from a synaptic balance.

Synaptic computation

Neurons are often considered to be the computational engines of the brain, with synapses acting solely as conveyers of information. But the diverse types of synaptic plasticity and the range of timescales over which they operate suggest that synapses have a more active role in information processing. Long-term changes in the transmission properties of synapses provide a physiological substrate for learning and memory, whereas short-term changes support a variety of computations. By expressing several forms of synaptic plasticity, a single neuron can convey an array of different signals to the neural circuit in which it operates.