A Spiking Working Memory Model Based on Hebbian Short-Term Potentiation

Promiscuous involvement of metabotropic glutamate receptors in the storage of N-methyl-d-aspartate receptor-dependent short-term potentiation

Short- and long-term forms of N-methyl-d-aspartate receptor (NMDAR)-dependent potentiation (most commonly termed short-term potentiation (STP) and long-term potentiation (LTP)) are co-induced in hippocampal slices by theta-burst stimulation, which mimics naturally occurring patterns of neuronal activity. While NMDAR-dependent LTP (NMDAR-LTP) is said to be the cellular correlate of long-term memory storage, NMDAR-dependent STP (NMDAR-STP) is thought to underlie the encoding of shorter-lasting memories. The mechanisms of NMDAR-LTP have been researched much more extensively than those of NMDAR-STP, which is characterized by its extreme stimulation dependence. Thus, in the absence of low-frequency test stimulation, which is used to test the magnitude of potentiation, NMDAR-STP does not decline until the stimulation is resumed. NMDAR-STP represents, therefore, an inverse variant of Hebbian synaptic plasticity, illustrating that inactive synapses can retain their strength unchanged until they become active again. The mechanisms, by which NMDAR-STP is stored in synapses without a decrement, are unknown and we report here that activation of metabotropic glutamate receptors may be critical in maintaining the potentiated state of synaptic transmission. This article is part of a discussion meeting issue 'Long-term potentiation: 50 years on'.

Presynaptic Rac1 in the hippocampus selectively regulates working memory

One of the most extensively studied members of the Ras superfamily of small GTPases, Rac1 is an intracellular signal transducer that remodels actin and phosphorylation signaling networks. Previous studies have shown that Rac1-mediated signaling is associated with hippocampal-dependent working memory and longer-term forms of learning and memory and that Rac1 can modulate forms of both pre- and postsynaptic plasticity. How these different cognitive functions and forms of plasticity mediated by Rac1 are linked, however, is unclear. Here, we show that spatial working memory in mice is selectively impaired following the expression of a genetically encoded Rac1 inhibitor at presynaptic terminals, while longer-term cognitive processes are affected by Rac1 inhibition at postsynaptic sites. To investigate the regulatory mechanisms of this presynaptic process, we leveraged new advances in mass spectrometry to identify the proteomic and post-translational landscape of presynaptic Rac1 signaling. We identified serine/threonine kinases and phosphorylated cytoskeletal signaling and synaptic vesicle proteins enriched with active Rac1. The phosphorylated sites in these proteins are at positions likely to have regulatory effects on synaptic vesicles. Consistent with this, we also report changes in the distribution and morphology of synaptic vesicles and in postsynaptic ultrastructure following presynaptic Rac1 inhibition. Overall, this study reveals a previously unrecognized presynaptic role of Rac1 signaling in cognitive processes and provides insights into its potential regulatory mechanisms.

N-Methyl-D-Aspartate Receptor-Antibody Encephalitis Impairs Maintenance of Attention to Items in Working Memory

NMDA receptors (NMDARs) may be crucial to working memory (WM). Computational models predict that they sustain neural firing and produce associative memory, which may underpin maintaining and binding information, respectively. We test this in patients with antibodies to NMDAR (n = 10, female) and compare them with healthy control participants (n = 55, 20 male, 35 female). Patients were tested after recovery with a task that separates two aspects of WM: sustaining attention and feature binding. Participants had to remember two colored arrows. Then attention was directed to one of them. After a variable delay, they reported the direction of either the same arrow (congruent cue) or of the other arrow (incongruent cue). We asked how congruency affected recall precision and measured types of error. Patients had difficulty in both sustaining attention to an item over time and feature binding. Controls were less precise after longer delays and incongruent cues. In contrast, patients did not benefit from congruent cues at longer delays [group × congruency (long condition); p = 0.041], indicating they could not sustain attention. Additionally, patients reported the wrong item (misbinding errors) more than controls after congruent cues [group × delay (congruent condition), main effect of group; p ≤ 0.001]. Our results suggest NMDARs are critical for both maintaining attention and feature binding.

Beta: bursts of cognition

Beta oscillations are linked to the control of goal-directed processing of sensory information and the timing of motor output. Recent evidence demonstrates they are not sustained but organized into intermittent high-power bursts mediating timely functional inhibition. This implies there is a considerable moment-to-moment variation in the neural dynamics supporting cognition. Beta bursts thus offer new opportunities for studying how sensory inputs are selectively processed, reshaped by inhibitory cognitive operations and ultimately result in motor actions. Recent method advances reveal diversity in beta bursts that provide deeper insights into their function and the underlying neural circuit activity motifs. We propose that brain-wide, spatiotemporal patterns of beta bursting reflect various cognitive operations and that their dynamics reveal nonlinear aspects of cortical processing.

Representation and computation in visual working memory

The ability to sustain internal representations of the sensory environment beyond immediate perception is a fundamental requirement of cognitive processing. In recent years, debates regarding the capacity and fidelity of the working memory (WM) system have advanced our understanding of the nature of these representations. In particular, there is growing recognition that WM representations are not merely imperfect copies of a perceived object or event. New experimental tools have revealed that observers possess richer information about the uncertainty in their memories and take advantage of environmental regularities to use limited memory resources optimally. Meanwhile, computational models of visuospatial WM formulated at different levels of implementation have converged on common principles relating capacity to variability and uncertainty. Here we review recent research on human WM from a computational perspective, including the neural mechanisms that support it.

The importance of covert memory consolidation in schizophrenia: Dysfunctional network profiles of the hippocampus and the dorsolateral prefrontal cortex

Altered brain network profiles in schizophrenia (SCZ) during memory consolidation are typically observed during task-active periods such as encoding or retrieval. However active processes are also sub served by covert periods of memory consolidation. These periods are active in that they allow memories to be recapitulated even in the absence of overt sensorimotor processing. It is plausible that regions central to memory formation like the dlPFC and the hippocampus, exert network signatures during covert periods. Are these signatures altered in patients? The question is clinically relevant because real world learning and memory is facilitated by covert processing, and may be impaired in schizophrenia. Here, we compared network signatures of the dlPFC and the hippocampus during covert periods of a learning and memory task. Because behavioral proficiency increased non-linearly, functional connectivity of the dlPFC and hippocampus [psychophysiological interaction (PPI)] was estimated for each of the Early (linear increases in performance) and Late (asymptotic performance) covert periods. During Early periods, we observed hypo-modulation by the hippocampus but hyper-modulation by dlPFC. Conversely, during Late periods, we observed hypo-modulation by both the dlPFC and the hippocampus. We stitch these results into a conceptual model of network deficits during covert periods of memory consolidation.

Stochastic attractor models of visual working memory

This paper investigates models of working memory in which memory traces evolve according to stochastic attractor dynamics. These models have previously been shown to account for response-biases that are manifest across multiple trials of a visual working memory task. Here we adapt this approach by making the stable fixed points correspond to the multiple items to be remembered within a single-trial, in accordance with standard dynamical perspectives of memory, and find evidence that this multi-item model can provide a better account of behavioural data from continuous-report tasks. Additionally, the multi-item model proposes a simple mechanism by which swap-errors arise: memory traces diffuse away from their initial state and are captured by the attractors of other items. Swap-error curves reveal the evolution of this process as a continuous function of time throughout the maintenance interval and can be inferred from experimental data. Consistent with previous findings, we find that empirical memory performance is not well characterised by a purely-diffusive process but rather by a stochastic process that also embodies error-correcting dynamics.

Neurobiological Causal Models of Language Processing

The language faculty is physically realized in the neurobiological infrastructure of the human brain. Despite significant efforts, an integrated understanding of this system remains a formidable challenge. What is missing from most theoretical accounts is a specification of the neural mechanisms that implement language function. Computational models that have been put forward generally lack an explicit neurobiological foundation. We propose a neurobiologically informed causal modeling approach which offers a framework for how to bridge this gap. A neurobiological causal model is a mechanistic description of language processing that is grounded in, and constrained by, the characteristics of the neurobiological substrate. It intends to model the generators of language behavior at the level of implementational causality. We describe key features and neurobiological component parts from which causal models can be built and provide guidelines on how to implement them in model simulations. Then we outline how this approach can shed new light on the core computational machinery for language, the long-term storage of words in the mental lexicon and combinatorial processing in sentence comprehension. In contrast to cognitive theories of behavior, causal models are formulated in the "machine language" of neurobiology which is universal to human cognition. We argue that neurobiological causal modeling should be pursued in addition to existing approaches. Eventually, this approach will allow us to develop an explicit computational neurobiology of language.

Presynaptic Rac1 in the hippocampus selectively regulates working memory

One of the most extensively studied members of the Ras superfamily of small GTPases, Rac1 is an intracellular signal transducer that remodels actin and phosphorylation signaling networks. Previous studies have shown that Rac1-mediated signaling is associated with hippocampal-dependent working memory and longer-term forms of learning and memory and that Rac1 can modulate forms of both pre- and postsynaptic plasticity. How these different cognitive functions and forms of plasticity mediated by Rac1 are linked, however, is unclear. Here, we show that spatial working memory is selectively impaired following the expression of a genetically encoded Rac1-inhibitor at presynaptic terminals, while longer-term cognitive processes are affected by Rac1 inhibition at postsynaptic sites. To investigate the regulatory mechanisms of this presynaptic process, we leveraged new advances in mass spectrometry to identify the proteomic and post-translational landscape of presynaptic Rac1 signaling. We identified serine/threonine kinases and phosphorylated cytoskeletal signaling and synaptic vesicle proteins enriched with active Rac1. The phosphorylated sites in these proteins are at positions likely to have regulatory effects on synaptic vesicles. Consistent with this, we also report changes in the distribution and morphology of synaptic vesicles and in postsynaptic ultrastructure following presynaptic Rac1 inhibition. Overall, this study reveals a previously unrecognized presynaptic role of Rac1 signaling in cognitive processes and provides insights into its potential regulatory mechanisms.

Dynamic fading memory and expectancy effects in the monkey primary visual cortex

In order to investigate the involvement of the primary visual cortex (V1) in working memory (WM), parallel, multisite recordings of multi-unit activity were obtained from monkey V1 while the animals performed a delayed match-to-sample (DMS) task. During the delay period, V1 population firing rate vectors maintained a lingering trace of the sample stimulus that could be reactivated by intervening impulse stimuli that enhanced neuronal firing. This fading trace of the sample did not require active engagement of the monkeys in the DMS task and likely reflects the intrinsic dynamics of recurrent cortical networks in lower visual areas. This renders an active, attention-dependent involvement of V1 in the maintenance of WM contents unlikely. By contrast, population responses to the test stimulus depended on the probabilistic contingencies between sample and test stimuli. Responses to tests that matched expectations were reduced which agrees with concepts of predictive coding.

Fast Hebbian plasticity and working memory

Theories and models of working memory (WM) were at least since the mid-1990s dominated by the persistent activity hypothesis. The past decade has seen rising concerns about the shortcomings of sustained activity as the mechanism for short-term maintenance of WM information in the light of accumulating experimental evidence for so-called activity-silent WM and the fundamental difficulty in explaining robust multi-item WM. In consequence, alternative theories are now explored mostly in the direction of fast synaptic plasticity as the underlying mechanism. The question of non-Hebbian vs Hebbian synaptic plasticity emerges naturally in this context. In this review, we focus on fast Hebbian plasticity and trace the origins of WM theories and models building on this form of associative learning.

Competitive learning to generate sparse representations for associative memory

One of the most well established brain principles, Hebbian learning, has led to the theoretical concept of neural assemblies. Based on it, many interesting brain theories have spawned. Palm's work implements this concept through multiple binary Willshaw associative memories, in a model that not only has a wide cognitive explanatory power but also makes neuroscientific predictions. Yet, Willshaw's associative memory can only achieve top capacity when the stored vectors are extremely sparse (number of active bits can grow logarithmically with the vector's length). This strict requirement makes it difficult to apply any model that uses this associative memory, like Palm's, to real data. Hence the fact that most works apply the memory to optimal randomly generated codes that do not represent any information. This issue creates the need for encoders that can take real data, and produce sparse representations - a problem which is also raised following Barlow's efficient coding principle. In this work, we propose a biologically-constrained network that encodes images into codes that are suitable for Willshaw's associative memory. The network is organized into groups of neurons that specialize on local receptive fields, and learn through a competitive scheme. After conducting auto- and hetero-association experiments on two visual data sets, we can conclude that our network not only beats sparse coding baselines, but also that it comes close to the performance achieved using optimal random codes.

"Leader-Follower" Dynamic Perturbation Manipulates Multi-Item Working Memory in Humans

Manipulating working memory (WM) is a central yet challenging notion. Previous studies suggest that WM items with varied memory strengths reactivate at different latencies, supporting a time-based mechanism. Motivated by this view, here we developed a purely bottom-up "Leader-Follower" behavioral approach to manipulate WM in humans. Specifically, task-irrelevant flickering color disks that are bound to each of the memorized items are presented during the delay period, and the ongoing luminance sequences of the color disks follow a Leader-Follower relationship, that is, a hundreds of milliseconds temporal lag. We show that this dynamic behavioral approach leads to better memory performance for the item associated with the temporally advanced luminance sequence (Leader) than the item with the temporally lagged luminance sequence (Follower), yet with limited effectiveness. Together, our findings constitute evidence for the essential role of temporal dynamics in WM operation and offer a promising, noninvasive WM manipulation approach.

Working memory depends on the excitatory-inhibitory balance in neuron-astrocyte network

Previous studies have shown that astrocytes are involved in information processing and working memory (WM) in the central nervous system. Here, the neuron-astrocyte network model with biological properties is built to study the effects of excitatory-inhibitory balance and neural network structures on WM tasks. It is found that the performance metrics of WM tasks under the scale-free network are higher than other network structures, and the WM task can be successfully completed when the proportion of excitatory neurons in the network exceeds 30%. There exists an optimal region for the proportion of excitatory neurons and synaptic weight that the memory performance metrics of the WM tasks are higher. The multi-item WM task shows that the spatial calcium patterns for different items overlap significantly in the astrocyte network, which is consistent with the formation of cognitive memory in the brain. Moreover, complex image tasks show that cued recall can significantly reduce systematic noise and maintain the stability of the WM tasks. The results may contribute to understand the mechanisms of WM formation and provide some inspirations into the dynamic storage and recall of memory.

Rapid astrocyte-dependent facilitation amplifies multi-vesicular release in hippocampal synapses

Synaptic facilitation is a major form of short-term plasticity typically driven by an increase in residual presynaptic calcium. Using near-total internal reflection fluorescence (near-TIRF) imaging of single vesicle release in cultured hippocampal synapses, we demonstrate a distinctive, release-dependent form of facilitation in which probability of vesicle release is higher following a successful glutamate release event than following a failure. This phenomenon has an onset of ≤500 ms and lasts several seconds, resulting in clusters of successful release events. The release-dependent facilitation requires neuronal contact with astrocytes and astrocytic glutamate uptake by EAAT1. It is not observed in neurons grown alone or in the presence of astrocyte-conditioned media. This form of facilitation dynamically amplifies multi-vesicular release. Facilitation-evoked release events exhibit spatial clustering and have a preferential localization toward the active zone center. These results uncover a rapid astrocyte-dependent form of facilitation acting via modulation of multi-vesicular release and displaying distinctive spatiotemporal properties.

Neural signature of flexible coding in prefrontal cortex

The ability of prefrontal cortex to quickly encode novel associations is crucial for adaptive behavior and central to working memory. Fast Hebbian changes in synaptic strength permit forming new associations, but neuronal signatures of this have been elusive. We devised a trialwise index of pattern similarity to look for rapid changes in population codes. Based on a computational model of working memory, we hypothesized that synaptic strength-and consequently, the tuning of neurons-could change if features of a subsequent stimulus need to be "reassociated," i.e., if bindings between features need to be broken to encode the new item. As a result, identical stimuli might elicit different neural responses. As predicted, neural response similarity dropped following rebinding, but only in prefrontal cortex. The history-dependent changes were expressed on top of traditional, fixed selectivity and were not explainable by carryover of previous firing into the current trial or by neural adaptation.

Simulations of working memory spiking networks driven by short-term plasticity

Working Memory (WM) is a cognitive mechanism that enables temporary holding and manipulation of information in the human brain. This mechanism is mainly characterized by a neuronal activity during which neuron populations are able to maintain an enhanced spiking activity after being triggered by a short external cue. In this study, we implement, using the NEST simulator, a spiking neural network model in which the WM activity is sustained by a mechanism of short-term synaptic facilitation related to presynaptic calcium kinetics. The model, which is characterized by leaky integrate-and-fire neurons with exponential postsynaptic currents, is able to autonomously show an activity regime in which the memory information can be stored in a synaptic form as a result of synaptic facilitation, with spiking activity functional to facilitation maintenance. The network is able to simultaneously keep multiple memories by showing an alternated synchronous activity which preserves the synaptic facilitation within the neuron populations holding memory information. The results shown in this study confirm that a WM mechanism can be sustained by synaptic facilitation.

Modulation of working memory duration by synaptic and astrocytic mechanisms

Short-term synaptic plasticity and modulations of the presynaptic vesicle release rate are key components of many working memory models. At the same time, an increasing number of studies suggests a potential role of astrocytes in modulating higher cognitive function such as WM through their influence on synaptic transmission. Which influence astrocytic signaling could have on the stability and duration of WM representations, however, is still unclear. Here, we introduce a slow, activity-dependent astrocytic regulation of the presynaptic release probability in a synaptic attractor model of WM. We compare and analyze simulations of a simple WM protocol in firing rate and spiking networks with and without astrocytic regulation, and underpin our observations with analyses of the phase space dynamics in the rate network. We find that the duration and stability of working memory representations are altered by astrocytic signaling and by noise. We show that astrocytic signaling modulates the mean duration of WM representations. Moreover, if the astrocytic regulation is strong, a slow presynaptic timescale introduces a ‘window of vulnerability’, during which WM representations are easily disruptable by noise before being stabilized. We identify two mechanisms through which noise from different sources in the network can either stabilize or destabilize WM representations. Our findings suggest that (i) astrocytic regulation can act as a crucial determinant for the duration of WM representations in synaptic attractor models of WM, and (ii) that astrocytic signaling could facilitate different mechanisms for volitional top-down control of WM representations and their duration.

The ability to form memories and recall them is one of the fascinating features of our brain. Working memory operates like a memory scratch pad storing ongoing information for further processing. Here, we present a computational model dissecting the influence of astrocytes on the stability and duration of working memories. We find that a long astrocytic time constant can influence the mean duration of working memory representations and generate a “window of vulnerability”, during which some memories are tagged for long-term survival while some are terminated. The fraction of memories in the survival and termination groups could be regulated by adjusting the strength of astrocytic feedback or its time constant. This indicates that astrocytic signaling can be viewed as a candidate mechanism for top-down control of working memory representations and their duration.

Reduced variability of bursting activity during working memory

Working memories have long been thought to be maintained by persistent spiking. However, mounting evidence from multiple-electrode recording (and single-trial analyses) shows that the underlying spiking is better characterized by intermittent bursts of activity. A counterargument suggested this intermittent activity is at odds with observations that spike-time variability reduces during task performance. However, this counterargument rests on assumptions, such as randomness in the timing of the bursts, which may not be correct. Thus, we analyzed spiking and LFPs from monkeys’ prefrontal cortex (PFC) to determine if task-related reductions in variability can co-exist with intermittent spiking. We found that it does because both spiking and associated gamma bursts were task-modulated, not random. In fact, the task-related reduction in spike variability could largely be explained by a related reduction in gamma burst variability. Our results provide further support for the intermittent activity models of working memory as well as novel mechanistic insights into how spike variability is reduced during cognitive tasks.

Traces of semantization - from episodic to semantic memory in a spiking cortical network model

Episodic memory is a recollection of past personal experiences associated with particular times and places. This kind of memory is commonly subject to loss of contextual information or" semantization", which gradually decouples the encoded memory items from their associated contexts while transforming them into semantic or gist-like representations. Novel extensions to the classical Remember/Know behavioral paradigm attribute the loss of episodicity to multiple exposures of an item in different contexts. Despite recent advancements explaining semantization at a behavioral level, the underlying neural mechanisms remain poorly understood. In this study, we suggest and evaluate a novel hypothesis proposing that Bayesian-Hebbian synaptic plasticity mechanisms might cause semantization of episodic memory. We implement a cortical spiking neural network model with a Bayesian-Hebbian learning rule called Bayesian Confidence Propagation Neural Network (BCPNN), which captures the semantization phenomenon and offers a mechanistic explanation for it. Encoding items across multiple contexts leads to item-context decoupling akin to semantization. We compare BCPNN plasticity with the more commonly used spike-timing dependent plasticity (STDP) learning rule in the same episodic memory task. Unlike BCPNN, STDP does not explain the decontextualization process. We further examine how selective plasticity modulation of isolated salient events may enhance preferential retention and resistance to semantization. Our model reproduces important features of episodicity on behavioral timescales under various biological constraints whilst also offering a novel neural and synaptic explanation for semantization, thereby casting new light on the interplay between episodic and semantic memory processes.Significance StatementRemembering single episodes is a fundamental attribute of cognition. Difficulties recollecting contextual information is a key sign of episodic memory loss or semantization. Behavioral studies demonstrate that semantization of episodic memory can occur rapidly, yet the neural mechanisms underlying this effect are insufficiently investigated. In line with recent behavioral findings, we show that multiple stimulus exposures in different contexts may advance item-context decoupling. We suggest a Bayesian-Hebbian synaptic plasticity hypothesis of memory semantization and further show that a transient modulation of plasticity during salient events may disrupt the decontextualization process by strengthening memory traces, and thus, enhancing preferential retention. The proposed cortical network-of-networks model thus bridges micro and mesoscale synaptic effects with network dynamics and behavior.

Phase-change memtransistive synapses for mixed-plasticity neural computations

In the mammalian nervous system, various synaptic plasticity rules act, either individually or synergistically, over wide-ranging timescales to enable learning and memory formation. Hence, in neuromorphic computing platforms, there is a significant need for artificial synapses that can faithfully express such multi-timescale plasticity mechanisms. Although some plasticity rules have been emulated with elaborate complementary metal oxide semiconductor and memristive circuitry, device-level hardware realizations of long-term and short-term plasticity with tunable dynamics are lacking. Here we introduce a phase-change memtransistive synapse that leverages both the non-volatility of the phase configurations and the volatility of field-effect modulation for implementing tunable plasticities. We show that these mixed-plasticity synapses can enable plasticity rules such as short-term spike-timing-dependent plasticity that helps with the modelling of dynamic environments. Further, we demonstrate the efficacy of the memtransistive synapses in realizing accelerators for Hopfield neural networks for solving combinatorial optimization problems.

Astrocytes mediate analogous memory in a multi-layer neuron–astrocyte network

Modeling the neuronal processes underlying short-term working memory remains the focus of many theoretical studies in neuroscience. In this paper, we propose a mathematical model of a spiking neural network (SNN) which simulates the way a fragment of information is maintained as a robust activity pattern for several seconds and the way it completely disappears if no other stimuli are fed to the system. Such short-term memory traces are preserved due to the activation of astrocytes accompanying the SNN. The astrocytes exhibit calcium transients at a time scale of seconds. These transients further modulate the efficiency of synaptic transmission and, hence, the firing rate of neighboring neurons at diverse timescales through gliotransmitter release. We demonstrate how such transients continuously encode frequencies of neuronal discharges and provide robust short-term storage of analogous information. This kind of short-term memory can store relevant information for seconds and then completely forget it to avoid overlapping with forthcoming patterns. The SNN is inter-connected with the astrocytic layer by local inter-cellular diffusive connections. The astrocytes are activated only when the neighboring neurons fire synchronously, e.g., when an information pattern is loaded. For illustration, we took grayscale photographs of people’s faces where the shades of gray correspond to the level of applied current which stimulates the neurons. The astrocyte feedback modulates (facilitates) synaptic transmission by varying the frequency of neuronal firing. We show how arbitrary patterns can be loaded, then stored for a certain interval of time, and retrieved if the appropriate clue pattern is applied to the input.

Mapping the BCPNN Learning Rule to a Memristor Model

The Bayesian Confidence Propagation Neural Network (BCPNN) has been implemented in a way that allows mapping to neural and synaptic processes in the human cortexandhas been used extensively in detailed spiking models of cortical associative memory function and recently also for machine learning applications. In conventional digital implementations of BCPNN, the von Neumann bottleneck is a major challenge with synaptic storage and access to it as the dominant cost. The memristor is a non-volatile device ideal for artificial synapses that fuses computation and storage and thus fundamentally overcomes the von Neumann bottleneck. While the implementation of other neural networks like Spiking Neural Network (SNN) and even Convolutional Neural Network (CNN) on memristor has been studied, the implementation of BCPNN has not. In this paper, the BCPNN learning rule is mapped to a memristor model and implemented with a memristor-based architecture. The implementation of the BCPNN learning rule is a mixed-signal design with the main computation and storage happening in the analog domain. In particular, the nonlinear dopant drift phenomenon of the memristor is exploited to simulate the exponential decay of the synaptic state variables in the BCPNN learning rule. The consistency between the memristor-based solution and the BCPNN learning rule is simulated and verified in Matlab, with a correlation coefficient as high as 0.99. The analog circuit is designed and implemented in the SPICE simulation environment, demonstrating a good emulation effect for the BCPNN learning rule with a correlation coefficient as high as 0.98. This work focuses on demonstrating the feasibility of mapping the BCPNN learning rule to in-circuit computation in memristor. The feasibility of the memristor-based implementation is evaluated and validated in the paper, to pave the way for a more efficient BCPNN implementation, toward a real-time brain emulation engine.

Distributed Phase Oscillatory Excitation Efficiently Produces Attractors Using Spike-Timing-Dependent Plasticity

The brain is thought to represent information in the form of activity in distributed groups of neurons known as attractors. We show here that in a randomly connected network of simulated spiking neurons, periodic stimulation of neurons with distributed phase offsets, along with standard spike-timing-dependent plasticity (STDP), efficiently creates distributed attractors. These attractors may have a consistent ordered firing pattern or become irregular, depending on the conditions. We also show that when two such attractors are stimulated in sequence, the same STDP mechanism can create a directed association between them, forming the basis of an associative network. We find that for an STDP time constant of 20 ms, the dependence of the efficiency of attractor creation on the driving frequency has a broad peak centered around 8 Hz. Upon restimulation, the attractors self-oscillate, but with an oscillation frequency that is higher than the driving frequency, ranging from 10 to 100 Hz.

Trial-to-Trial Variability of Spiking Delay Activity in Prefrontal Cortex Constrains Burst-Coding Models of Working Memory

A hallmark neuronal correlate of working memory (WM) is stimulus-selective spiking activity of neurons in PFC during mnemonic delays. These observations have motivated an influential computational modeling framework in which WM is supported by persistent activity. Recently, this framework has been challenged by arguments that observed persistent activity may be an artifact of trial-averaging, which potentially masks high variability of delay activity at the single-trial level. In an alternative scenario, WM delay activity could be encoded in bursts of selective neuronal firing which occur intermittently across trials. However, this alternative proposal has not been tested on single-neuron spike-train data. Here, we developed a framework for addressing this issue by characterizing the trial-to-trial variability of neuronal spiking quantified by Fano factor (FF). By building a doubly stochastic Poisson spiking model, we first demonstrated that the burst-coding proposal implies a significant increase in FF positively correlated with firing rate, and thus loss of stability across trials during the delay. Simulation of spiking cortical circuit WM models further confirmed that FF is a sensitive measure that can well dissociate distinct WM mechanisms. We then tested these predictions on datasets of single-neuron recordings from macaque PFC during three WM tasks. In sharp contrast to the burst-coding model predictions, we only found a small fraction of neurons showing increased WM-dependent burstiness, and stability across trials during delay was strengthened in empirical data. Therefore, reduced trial-to-trial variability during delay provides strong constraints on the contribution of single-neuron intermittent bursting to WM maintenance.SIGNIFICANCE STATEMENT There are diverging classes of theoretical models explaining how information is maintained in working memory by cortical circuits. In an influential model class, neurons exhibit persistent elevated memorandum-selective firing, whereas a recently developed class of burst-coding models suggests that persistent activity is an artifact of trial-averaging, and spiking is sparse in each single trial, subserved by brief intermittent bursts. However, this alternative picture has not been characterized or tested on empirical spike-train data. Here we combine mathematical analysis, computational model simulation, and experimental data analysis to test empirically these two classes of models and show that the trial-to-trial variability of empirical spike trains is not consistent with burst coding. These findings provide constraints for theoretical models of working memory.

Natural and Artificial Intelligence: A brief introduction to the interplay between AI and neuroscience research

Neuroscience and artificial intelligence (AI) share a long history of collaboration. Advances in neuroscience, alongside huge leaps in computer processing power over the last few decades, have given rise to a new generation of in silico neural networks inspired by the architecture of the brain. These AI systems are now capable of many of the advanced perceptual and cognitive abilities of biological systems, including object recognition and decision making. Moreover, AI is now increasingly being employed as a tool for neuroscience research and is transforming our understanding of brain functions. In particular, deep learning has been used to model how convolutional layers and recurrent connections in the brain's cerebral cortex control important functions, including visual processing, memory, and motor control. Excitingly, the use of neuroscience-inspired AI also holds great promise for understanding how changes in brain networks result in psychopathologies, and could even be utilized in treatment regimes. Here we discuss recent advancements in four areas in which the relationship between neuroscience and AI has led to major advancements in the field; (1) AI models of working memory, (2) AI visual processing, (3) AI analysis of big neuroscience datasets, and (4) computational psychiatry.

Working Memory: From Neural Activity to the Sentient Mind

Working memory (WM) is the ability to maintain and manipulate information in the conscious mind over a timescale of seconds. This ability is thought to be maintained through the persistent discharges of neurons in a network of brain areas centered on the prefrontal cortex, as evidenced by neurophysiological recordings in nonhuman primates, though both the localization and the neural basis of WM has been a matter of debate in recent years. Neural correlates of WM are evident in species other than primates, including rodents and corvids. A specialized network of excitatory and inhibitory neurons, aided by neuromodulatory influences of dopamine, is critical for the maintenance of neuronal activity. Limitations in WM capacity and duration, as well as its enhancement during development, can be attributed to properties of neural activity and circuits. Changes in these factors can be observed through training-induced improvements and in pathological impairments. WM thus provides a prototypical cognitive function whose properties can be tied to the spiking activity of brain neurons. © 2021 American Physiological Society. Compr Physiol 11:1-41, 2021.

Lack of Sez6 Family Proteins Impairs Motor Functions, Short-Term Memory, and Cognitive Flexibility and Alters Dendritic Spine Properties

Seizure-related gene 6 (Sez6), Sez6-Like (Sez6L), and Sez6-Like 2 (Sez6L2) comprise a family of homologous proteins widely expressed throughout the brain that have been linked to neurodevelopmental and psychiatric disorders. Here, we use Sez6 triple knockout (TKO) mice, which lack all three Sez6 family proteins, to demonstrate that Sez6 family proteins regulate dendritic spine structure and cognitive functions, motor learning, and maintenance of motor functions across the lifespan. Compared to WT controls, we found that Sez6 TKO mice had impaired motor learning and their motor coordination was negatively affected from 6 weeks old and declined more rapidly as they aged. Sez6 TKO mice had reduced spine density in the hippocampus and dendritic spines were shifted to more immature morphologies in the somatosensory cortex. Cognitive testing revealed that they had enhanced stress responsiveness, impaired working, and spatial short-term memory but intact spatial long-term memory in the Morris water maze albeit accompanied by a reversal deficit. Our study demonstrates that the lack of Sez6 family proteins results in phenotypes commonly associated with neuropsychiatric disorders making it likely that Sez6 family proteins contribute to the complex etiologies of these disorders.

Biological constraints on neural network models of cognitive function

Neural network models are potential tools for improving our understanding of complex brain functions. To address this goal, these models need to be neurobiologically realistic. However, although neural networks have advanced dramatically in recent years and even achieve human-like performance on complex perceptual and cognitive tasks, their similarity to aspects of brain anatomy and physiology is imperfect. Here, we discuss different types of neural models, including localist, auto-associative, hetero-associative, deep and whole-brain networks, and identify aspects under which their biological plausibility can be improved. These aspects range from the choice of model neurons and of mechanisms of synaptic plasticity and learning to implementation of inhibition and control, along with neuroanatomical properties including areal structure and local and long-range connectivity. We highlight recent advances in developing biologically grounded cognitive theories and in mechanistically explaining, on the basis of these brain-constrained neural models, hitherto unaddressed issues regarding the nature, localization and ontogenetic and phylogenetic development of higher brain functions. In closing, we point to possible future clinical applications of brain-constrained modelling.

Transsynaptic modulation of presynaptic short-term plasticity in hippocampal mossy fiber synapses

The hippocampal mossy fiber synapse is a key synapse of the trisynaptic circuit. Post-tetanic potentiation (PTP) is the most powerful form of plasticity at this synaptic connection. It is widely believed that mossy fiber PTP is an entirely presynaptic phenomenon, implying that PTP induction is input-specific, and requires neither activity of multiple inputs nor stimulation of postsynaptic neurons. To directly test cooperativity and associativity, we made paired recordings between single mossy fiber terminals and postsynaptic CA3 pyramidal neurons in rat brain slices. By stimulating non-overlapping mossy fiber inputs converging onto single CA3 neurons, we confirm that PTP is input-specific and non-cooperative. Unexpectedly, mossy fiber PTP exhibits anti-associative induction properties. EPSCs show only minimal PTP after combined pre- and postsynaptic high-frequency stimulation with intact postsynaptic Ca2+ signaling, but marked PTP in the absence of postsynaptic spiking and after suppression of postsynaptic Ca2+ signaling (10 mM EGTA). PTP is largely recovered by inhibitors of voltage-gated R- and L-type Ca2+ channels, group II mGluRs, and vacuolar-type H+-ATPase, suggesting the involvement of retrograde vesicular glutamate signaling. Transsynaptic regulation of PTP extends the repertoire of synaptic computations, implementing a brake on mossy fiber detonation and a "smart teacher" function of hippocampal mossy fiber synapses.

A Computational Model of Working Memory Based on Spike-Timing-Dependent Plasticity

Working memory is closely involved in various cognitive activities, but its neural mechanism is still under exploration. The mainstream view has long been that persistent activity is the neural basis of working memory, but recent experiments have observed that activity-silent memory can also be correctly recalled. The underlying mechanism of activity-silent memory is considered to be an alternative scheme that rejects the theory of persistent activity. We propose a working memory model based on spike-timing-dependent plasticity (STDP). Different from models based on spike-rate coding, our model adopts temporal patterns of action potentials to represent information, so it can flexibly encode new memory representation. The model can work in both persistent and silent states, i.e., it is compatible with both of these seemingly conflicting neural mechanisms. We conducted a simulation experiment, and the results are similar to the real experimental results, which suggests that our model is plausible in biology.

Modeling Working Memory in a Spiking Neuron Network Accompanied by Astrocytes

We propose a novel biologically plausible computational model of working memory (WM) implemented by a spiking neuron network (SNN) interacting with a network of astrocytes. The SNN is modeled by synaptically coupled Izhikevich neurons with a non-specific architecture connection topology. Astrocytes generating calcium signals are connected by local gap junction diffusive couplings and interact with neurons via chemicals diffused in the extracellular space. Calcium elevations occur in response to the increased concentration of the neurotransmitter released by spiking neurons when a group of them fire coherently. In turn, gliotransmitters are released by activated astrocytes modulating the strength of the synaptic connections in the corresponding neuronal group. Input information is encoded as two-dimensional patterns of short applied current pulses stimulating neurons. The output is taken from frequencies of transient discharges of corresponding neurons. We show how a set of information patterns with quite significant overlapping areas can be uploaded into the neuron-astrocyte network and stored for several seconds. Information retrieval is organized by the application of a cue pattern representing one from the memory set distorted by noise. We found that successful retrieval with the level of the correlation between the recalled pattern and ideal pattern exceeding 90% is possible for the multi-item WM task. Having analyzed the dynamical mechanism of WM formation, we discovered that astrocytes operating at a time scale of a dozen of seconds can successfully store traces of neuronal activations corresponding to information patterns. In the retrieval stage, the astrocytic network selectively modulates synaptic connections in the SNN leading to successful recall. Information and dynamical characteristics of the proposed WM model agrees with classical concepts and other WM models.

Temporally coherent perturbation of neural dynamics during retention alters human multi-item working memory

Temporarily storing a list of items in working memory (WM), a fundamental ability in cognition, has been posited to rely on the temporal dynamics of multi-item neural representations during retention. However, the causal evidence, particularly in human subjects, is still lacking, let alone WM manipulation. Here, we develop a novel "dynamic perturbation" approach to manipulate the relative memory strength of WM items held in human brain, by presenting temporally correlated luminance sequences during retention to interfere with the multi-item neural dynamics. Six experiments on more than 150 subjects confirm the effectiveness of this WM manipulation approach. A computational model combining continuous attractor neural network (CANN) and short-term synaptic plasticity (STP) principles further reproduces all the empirical findings. The model reveals that the "dynamic perturbation" modifies the synaptic efficacies of WM items through STP principles, eventually leading to changes in their relative memory strengths. Our results support the causal role of temporal dynamics of neural network in mediating multi-item WM, and offer a promising, purely bottom-up approach to manipulate WM.

Embodied working memory during ongoing input streams

Sensory stimuli endow animals with the ability to generate an internal representation. This representation can be maintained for a certain duration in the absence of previously elicited inputs. The reliance on an internal representation rather than purely on the basis of external stimuli is a hallmark feature of higher-order functions such as working memory. Patterns of neural activity produced in response to sensory inputs can continue long after the disappearance of previous inputs. Experimental and theoretical studies have largely invested in understanding how animals faithfully maintain sensory representations during ongoing reverberations of neural activity. However, these studies have focused on preassigned protocols of stimulus presentation, leaving out by default the possibility of exploring how the content of working memory interacts with ongoing input streams. Here, we study working memory using a network of spiking neurons with dynamic synapses subject to short-term and long-term synaptic plasticity. The formal model is embodied in a physical robot as a companion approach under which neuronal activity is directly linked to motor output. The artificial agent is used as a methodological tool for studying the formation of working memory capacity. To this end, we devise a keyboard listening framework to delineate the context under which working memory content is (1) refined, (2) overwritten or (3) resisted by ongoing new input streams. Ultimately, this study takes a neurorobotic perspective to resurface the long-standing implication of working memory in flexible cognition.

Flexible Working Memory Through Selective Gating and Attentional Tagging

Working memory is essential: it serves to guide intelligent behavior of humans and nonhuman primates when task-relevant stimuli are no longer present to the senses. Moreover, complex tasks often require that multiple working memory representations can be flexibly and independently maintained, prioritized, and updated according to changing task demands. Thus far, neural network models of working memory have been unable to offer an integrative account of how such control mechanisms can be acquired in a biologically plausible manner. Here, we present WorkMATe, a neural network architecture that models cognitive control over working memory content and learns the appropriate control operations needed to solve complex working memory tasks. Key components of the model include a gated memory circuit that is controlled by internal actions, encoding sensory information through untrained connections, and a neural circuit that matches sensory inputs to memory content. The network is trained by means of a biologically plausible reinforcement learning rule that relies on attentional feedback and reward prediction errors to guide synaptic updates. We demonstrate that the model successfully acquires policies to solve classical working memory tasks, such as delayed recognition and delayed pro-saccade/anti-saccade tasks. In addition, the model solves much more complex tasks, including the hierarchical 12-AX task or the ABAB ordered recognition task, both of which demand an agent to independently store and updated multiple items separately in memory. Furthermore, the control strategies that the model acquires for these tasks subsequently generalize to new task contexts with novel stimuli, thus bringing symbolic production rule qualities to a neural network architecture. As such, WorkMATe provides a new solution for the neural implementation of flexible memory control.

Neuronal spike-rate adaptation supports working memory in language processing

Language processing involves the ability to store and integrate pieces of information in working memory over short periods of time. According to the dominant view, information is maintained through sustained, elevated neural activity. Other work has argued that short-term synaptic facilitation can serve as a substrate of memory. Here we propose an account where memory is supported by intrinsic plasticity that downregulates neuronal firing rates. Single neuron responses are dependent on experience, and we show through simulations that these adaptive changes in excitability provide memory on timescales ranging from milliseconds to seconds. On this account, spiking activity writes information into coupled dynamic variables that control adaptation and move at slower timescales than the membrane potential. From these variables, information is continuously read back into the active membrane state for processing. This neuronal memory mechanism does not rely on persistent activity, excitatory feedback, or synaptic plasticity for storage. Instead, information is maintained in adaptive conductances that reduce firing rates and can be accessed directly without cued retrieval. Memory span is systematically related to both the time constant of adaptation and baseline levels of neuronal excitability. Interference effects within memory arise when adaptation is long lasting. We demonstrate that this mechanism is sensitive to context and serial order which makes it suitable for temporal integration in sequence processing within the language domain. We also show that it enables the binding of linguistic features over time within dynamic memory registers. This work provides a step toward a computational neurobiology of language.

Interplay between persistent activity and activity-silent dynamics in the prefrontal cortex underlies serial biases in working memory

Persistent neuronal spiking has long been considered the mechanism underlying working memory, but recent proposals argue for alternative 'activity-silent' substrates. Using monkey and human electrophysiology data, we show here that attractor dynamics that control neural spiking during mnemonic periods interact with activity-silent mechanisms in the prefrontal cortex (PFC). This interaction allows memory reactivations, which enhance serial biases in spatial working memory. Stimulus information was not decodable between trials, but remained present in activity-silent traces inferred from spiking synchrony in the PFC. Just before the new stimulus, this latent trace was reignited into activity that recapitulated the previous stimulus representation. Importantly, the reactivation strength correlated with the strength of serial biases in both monkeys and humans, as predicted by a computational model that integrates activity-based and activity-silent mechanisms. Finally, single-pulse transcranial magnetic stimulation applied to the human PFC between successive trials enhanced serial biases, thus demonstrating the causal role of prefrontal reactivations in determining working-memory behavior.

A Model of Memory Linking Time to Space

The storage of temporally precise spike patterns can be realized by a single neuron. A spiking neural network (SNN) model is utilized to demonstrate the ability to precisely recall a spike pattern after presenting a single input. We show by using a simulation study that the temporal properties of input patterns can be transformed into spatial patterns of local dendritic spikes. The localization of time-points of spikes is facilitated by phase-shift of the subthreshold membrane potential oscillations (SMO) in the dendritic branches, which modifies their excitability. In reference to the points in time of the arriving input, the dendritic spikes are triggered in different branches. To store spatially distributed patterns, two unsupervised learning mechanisms are utilized. Either synaptic weights to the branches, spatial representation of the temporal input pattern, are enhanced by spike-timing-dependent plasticity (STDP) or the oscillation power of SMOs in spiking branches is increased by dendritic spikes. For retrieval, spike bursts activate stored spatiotemporal patterns in dendritic branches, which reactivate the original somatic spike patterns. The simulation of the prototypical model demonstrates the principle, how linking time to space enables the storage of temporal features of an input. Plausibility, advantages, and some variations of the proposed model are also discussed.

An Indexing Theory for Working Memory Based on Fast Hebbian Plasticity

Working memory (WM) is a key component of human memory and cognition. Computational models have been used to study the underlying neural mechanisms, but neglected the important role of short-term memory (STM) and long-term memory (LTM) interactions for WM. Here, we investigate these using a novel multiarea spiking neural network model of prefrontal cortex (PFC) and two parietotemporal cortical areas based on macaque data. We propose a WM indexing theory that explains how PFC could associate, maintain, and update multimodal LTM representations. Our simulations demonstrate how simultaneous, brief multimodal memory cues could build a temporary joint memory representation as an “index” in PFC by means of fast Hebbian synaptic plasticity. This index can then reactivate spontaneously and thereby also the associated LTM representations. Cueing one LTM item rapidly pattern completes the associated uncued item via PFC. The PFC–STM network updates flexibly as new stimuli arrive, thereby gradually overwriting older representations.

Mesoscopic population equations for spiking neural networks with synaptic short-term plasticity

Coarse-graining microscopic models of biological neural networks to obtain mesoscopic models of neural activities is an essential step towards multi-scale models of the brain. Here, we extend a recent theory for mesoscopic population dynamics with static synapses to the case of dynamic synapses exhibiting short-term plasticity (STP). The extended theory offers an approximate mean-field dynamics for the synaptic input currents arising from populations of spiking neurons and synapses undergoing Tsodyks-Markram STP. The approximate mean-field dynamics accounts for both finite number of synapses and correlation between the two synaptic variables of the model (utilization and available resources) and its numerical implementation is simple. Comparisons with Monte Carlo simulations of the microscopic model show that in both feedforward and recurrent networks, the mesoscopic mean-field model accurately reproduces the first- and second-order statistics of the total synaptic input into a postsynaptic neuron and accounts for stochastic switches between Up and Down states and for population spikes. The extended mesoscopic population theory of spiking neural networks with STP may be useful for a systematic reduction of detailed biophysical models of cortical microcircuits to numerically efficient and mathematically tractable mean-field models.

The what, where and how of delay activity

Working memory is characterized by neural activity that persists during the retention interval of delay tasks. Despite the ubiquity of this delay activity across tasks, species and experimental techniques, our understanding of this phenomenon remains incomplete. Although initially there was a narrow focus on sustained activation in a small number of brain regions, methodological and analytical advances have allowed researchers to uncover previously unobserved forms of delay activity various parts of the brain. In light of these new findings, this Review reconsiders what delay activity is, where in the brain it is found, what roles it serves and how it may be generated.

Reevaluating the Role of Persistent Neural Activity in Short-Term Memory

A traditional view of short-term working memory (STM) is that task-relevant information is maintained 'online' in persistent spiking activity. However, recent experimental and modeling studies have begun to question this long-held belief. In this review, we discuss new evidence demonstrating that information can be 'silently' maintained via short-term synaptic plasticity (STSP) without the need for persistent activity. We discuss how the neural mechanisms underlying STM are inextricably linked with the cognitive demands of the task, such that the passive maintenance and the active manipulation of information are subserved differently in the brain. Together, these recent findings point towards a more nuanced view of STM in which multiple substrates work in concert to support our ability to temporarily maintain and manipulate information.

Persistent Spiking Activity Underlies Working Memory

Persistent activity generated in the PFC during the delay period of working memory tasks represents information about stimuli held in memory and determines working memory performance. Alternative models of working memory, depending on the rhythmicity of discharges or exclusively on short-term synaptic plasticity, are inconsistent with the neurophysiological data.Dual Perspectives Companion Paper:Working Memory: Delay Activity, Yes! Persistent Activity? Maybe Not, by Mikael Lundqvist, Pawel Herman, and Earl K. Miller.

Introducing double bouquet cells into a modular cortical associative memory model

We present an electrophysiological model of double bouquet cells and integrate them into an established cortical columnar microcircuit model that has previously been used as a spiking attractor model for memory. Learning in that model relies on a Hebbian-Bayesian learning rule to condition recurrent connectivity between pyramidal cells. We here demonstrate that the inclusion of a biophysically plausible double bouquet cell model can solve earlier concerns about learning rules that simultaneously learn excitation and inhibition and might thus violate Dale’s principle. We show that learning ability and resulting effective connectivity between functional columns of previous network models is preserved when pyramidal synapses onto double bouquet cells are plastic under the same Hebbian-Bayesian learning rule. The proposed architecture draws on experimental evidence on double bouquet cells and effectively solves the problem of duplexed learning of inhibition and excitation by replacing recurrent inhibition between pyramidal cells in functional columns of different stimulus selectivity with a plastic disynaptic pathway. We thus show that the resulting change to the microcircuit architecture improves the model’s biological plausibility without otherwise impacting the model’s spiking activity, basic operation, and learning abilities.

Between persistently active and activity-silent frameworks: novel vistas on the cellular basis of working memory

Recent work has revealed important new discoveries on the cellular mechanisms of working memory (WM). These findings have motivated several seemingly conflicting theories on the mechanisms of short‐term memory maintenance. Here, we summarize the key insights gained from these new experiments and critically evaluate them in light of three hypotheses: classical persistent activity, activity‐silent, and dynamic coding. The experiments discussed include the first direct demonstration of persistently active neurons in the human medial temporal lobe that form static attractors with relevance to WM, single‐neuron recordings in the macaque prefrontal cortex that show evidence for both persistent and more dynamic types of WM representations, and noninvasive neuroimaging in humans that argues for activity‐silent representations. A key insight that emerges from these new results is that there are several neural mechanisms that support the maintenance of information in WM. Finally, based on established cognitive theories of WM, we propose a coherent model that encompasses these seemingly contradictory results. We propose that the three neuronal mechanisms of persistent activity, activity‐silent, and dynamic coding map well onto the cognitive levels of information processing (within focus of attention, activated long‐term memory, and central executive) that Cowan's WM model proposes.

Probabilistic associative learning suffices for learning the temporal structure of multiple sequences

From memorizing a musical tune to navigating a well known route, many of our underlying behaviors have a strong temporal component. While the mechanisms behind the sequential nature of the underlying brain activity are likely multifarious and multi-scale, in this work we attempt to characterize to what degree some of this properties can be explained as a consequence of simple associative learning. To this end, we employ a parsimonious firing-rate attractor network equipped with the Hebbian-like Bayesian Confidence Propagating Neural Network (BCPNN) learning rule relying on synaptic traces with asymmetric temporal characteristics. The proposed network model is able to encode and reproduce temporal aspects of the input, and offers internal control of the recall dynamics by gain modulation. We provide an analytical characterisation of the relationship between the structure of the weight matrix, the dynamical network parameters and the temporal aspects of sequence recall. We also present a computational study of the performance of the system under the effects of noise for an extensive region of the parameter space. Finally, we show how the inclusion of modularity in our network structure facilitates the learning and recall of multiple overlapping sequences even in a noisy regime.