Warming up recurrent neural networks to maximise reachable multistability greatly improves learning

Training recurrent neural networks is known to be difficult when time dependencies become long. In this work, we show that most standard cells only have one stable equilibrium at initialisation, and that learning on tasks with long time dependencies generally occurs once the number of network stable equilibria increases; a property known as multistability. Multistability is often not easily attained by initially monostable networks, making learning of long time dependencies between inputs and outputs difficult. This insight leads to the design of a novel way to initialise any recurrent cell connectivity through a procedure called "warmup" to improve its capability to learn arbitrarily long time dependencies. This initialisation procedure is designed to maximise network reachable multistability, i.e., the number of equilibria within the network that can be reached through relevant input trajectories, in few gradient steps. We show on several information restitution, sequence classification, and reinforcement learning benchmarks that warming up greatly improves learning speed and performance, for multiple recurrent cells, but sometimes impedes precision. We therefore introduce a double-layer architecture initialised with a partial warmup that is shown to greatly improve learning of long time dependencies while maintaining high levels of precision. This approach provides a general framework for improving learning abilities of any recurrent cell when long time dependencies are present. We also show empirically that other initialisation and pretraining procedures from the literature implicitly foster reachable multistability of recurrent cells.

A bio-inspired bistable recurrent cell allows for long-lasting memory

Recurrent neural networks (RNNs) provide state-of-the-art performances in a wide variety of tasks that require memory. These performances can often be achieved thanks to gated recurrent cells such as gated recurrent units (GRU) and long short-term memory (LSTM). Standard gated cells share a layer internal state to store information at the network level, and long term memory is shaped by network-wide recurrent connection weights. Biological neurons on the other hand are capable of holding information at the cellular level for an arbitrary long amount of time through a process called bistability. Through bistability, cells can stabilize to different stable states depending on their own past state and inputs, which permits the durable storing of past information in neuron state. In this work, we take inspiration from biological neuron bistability to embed RNNs with long-lasting memory at the cellular level. This leads to the introduction of a new bistable biologically-inspired recurrent cell that is shown to strongly improves RNN performance on time-series which require very long memory, despite using only cellular connections (all recurrent connections are from neurons to themselves, i.e. a neuron state is not influenced by the state of other neurons). Furthermore, equipping this cell with recurrent neuromodulation permits to link them to standard GRU cells, taking a step towards the biological plausibility of GRU. With this link, this work paves the way for studying more complex and biologically plausible neuromodulation schemes as gating mechanisms in RNNs.

Robust switches in thalamic network activity require a timescale separation between sodium and T-type calcium channel activations

Switches in brain states, synaptic plasticity and neuromodulation are fundamental processes in our brain that take place concomitantly across several spatial and timescales. All these processes target neuron intrinsic properties and connectivity to achieve specific physiological goals, raising the question of how they can operate without interfering with each other. Here, we highlight the central importance of a timescale separation in the activation of sodium and T-type calcium channels to sustain robust switches in brain states in thalamic neurons that are compatible with synaptic plasticity and neuromodulation. We quantify the role of this timescale separation by comparing the robustness of rhythms of six published conductance-based models at the cellular, circuit and network levels. We show that robust rhythm generation requires a T-type calcium channel activation whose kinetics are situated between sodium channel activation and T-type calcium channel inactivation in all models despite their quantitative differences.

Our brain is constantly processing information either from the environment to quickly react to incoming events or learning from experience to shape our memory. These brain states translate a collective activity of neurons interconnected via synaptic connections. Here, we focus on the thalamic network showing a transition from an active to an oscillatory mode at the population level, reverberating a switch from tonic to bursting mode at the cellular level. We are questioning how these activity fluctuations can be robustly modeled despite synaptic plasticity affecting the network configuration and the presence of neuromodulators affecting neuron intrinsic properties. To do so, we investigate six conductance-based models and their ability to reproduce activity switches at the cellular, circuit and population levels. We highlight that the robustness requires the timescale separation between the fast activation of sodium channels compared to the slow activation of T-type calcium channels. Our results show that this kinetics difference is not a computational detail but rather makes a model suitable and robust to study the interaction between switches in brain states, synaptic plasticity and neuromodulation.

From Biophysical to Integrate-and-Fire Modeling

This article proposes a methodology to extract a low-dimensional integrate-and-fire model from an arbitrarily detailed single-compartment biophysical model. The method aims at relating the modulation of maximal conductance parameters in the biophysical model to the modulation of parameters in the proposed integrate-and-fire model. The approach is illustrated on two well-documented examples of cellular neuromodulation: the transition between type I and type II excitability and the transition between spiking and bursting.

Introducing neuromodulation in deep neural networks to learn adaptive behaviours

Animals excel at adapting their intentions, attention, and actions to the environment, making them remarkably efficient at interacting with a rich, unpredictable and ever-changing external world, a property that intelligent machines currently lack. Such an adaptation property relies heavily on cellular neuromodulation, the biological mechanism that dynamically controls intrinsic properties of neurons and their response to external stimuli in a context-dependent manner. In this paper, we take inspiration from cellular neuromodulation to construct a new deep neural network architecture that is specifically designed to learn adaptive behaviours. The network adaptation capabilities are tested on navigation benchmarks in a meta-reinforcement learning context and compared with state-of-the-art approaches. Results show that neuromodulation is capable of adapting an agent to different tasks and that neuromodulation-based approaches provide a promising way of improving adaptation of artificial systems.

Cellular switches orchestrate rhythmic circuits

Small inhibitory neuronal circuits have long been identified as key neuronal motifs to generate and modulate the coexisting rhythms of various motor functions. Our paper highlights the role of a cellular switching mechanism to orchestrate such circuits. The cellular switch makes the circuits reconfigurable, robust, adaptable, and externally controllable. Without this cellular mechanism, the circuit rhythms entirely rely on specific tunings of the synaptic connectivity, which makes them rigid, fragile, and difficult to control externally. We illustrate those properties on the much studied architecture of a small network controlling both the pyloric and gastric rhythms of crabs. The cellular switch is provided by a slow negative conductance often neglected in mathematical modeling of central pattern generators. We propose that this conductance is simple to model and key to computational studies of rhythmic circuit neuromodulation.

Switchable slow cellular conductances determine robustness and tunability of network states

Neuronal information processing is regulated by fast and localized fluctuations of brain states. Brain states reliably switch between distinct spatiotemporal signatures at a network scale even though they are composed of heterogeneous and variable rhythms at a cellular scale. We investigated the mechanisms of this network control in a conductance-based population model that reliably switches between active and oscillatory mean-fields. Robust control of the mean-field properties relies critically on a switchable negative intrinsic conductance at the cellular level. This conductance endows circuits with a shared cellular positive feedback that can switch population rhythms on and off at a cellular resolution. The switch is largely independent from other intrinsic neuronal properties, network size and synaptic connectivity. It is therefore compatible with the temporal variability and spatial heterogeneity induced by slower regulatory functions such as neuromodulation, synaptic plasticity and homeostasis. Strikingly, the required cellular mechanism is available in all cell types that possess T-type calcium channels but unavailable in computational models that neglect the slow kinetics of their activation.

Brain information processing involves electrophysiological signals at multiple temporal and spatial timescales, from the single neuron level to whole brain areas. A fast and local control of these signals by neurochemicals called neuromodulators is essential in complex tasks such as movement initiation and attentional focus. The neuromodulators act at the cellular scale to control signals that propagate at potentially much larger scales. The present paper highlights the critical role of a cellular switch of excitability for the fast and localized control of cellular and network states. By turning ON and OFF the cellular switch, neuromodulators can robustly switch large populations between distinct network states. We stress the importance of controlling the switch at a cellular level and independently of the connectivity to allow for tunable spatiotemporal signatures of the network states.

Robust Modulation of Integrate-and-Fire Models

By controlling the state of neuronal populations, neuromodulators ultimately affect behavior. A key neuromodulation mechanism is the alteration of neuronal excitability via the modulation of ion channel expression. This type of neuromodulation is normally studied with conductance-based models, but those models are computationally challenging for large-scale network simulations needed in population studies. This article studies the modulation properties of the multiquadratic integrate-and-fire model, a generalization of the classical quadratic integrate-and-fire model. The model is shown to combine the computational economy of integrate-and-fire modeling and the physiological interpretability of conductance-based modeling. It is therefore a good candidate for affordable computational studies of neuromodulation in large networks.

Robust and tunable bursting requires slow positive feedback

We highlight that the robustness and tunability of a bursting model critically rely on currents that provide slow positive feedback to the membrane potential. Such currents have the ability to make the total conductance of the circuit negative in a timescale that is termed "slow" because it is intermediate between the fast timescale of the spike upstroke and the ultraslow timescale of even slower adaptation currents. We discuss how such currents can be assessed either in voltage-clamp experiments or in computational models. We show that, while frequent in the literature, mathematical and computational models of bursting that lack the slow negative conductance are fragile and rigid. Our results suggest that modeling the slow negative conductance of cellular models is important when studying the neuromodulation of rhythmic circuits at any broader scale. NEW & NOTEWORTHY Nervous system functions rely on the modulation of neuronal activity between different rhythmic patterns. The mechanisms of this modulation are still poorly understood. Using computational modeling, we show the critical role of currents that provide slow negative conductance, distinct from the fast negative conductance necessary for spike generation. The significance of the slow negative conductance for neuromodulation is often overlooked, leading to computational models that are rigid and fragile.

Implications of cellular models of dopamine neurons for disease

This review addresses the present state of single-cell models of the firing pattern of midbrain dopamine neurons and the insights that can be gained from these models into the underlying mechanisms for diseases such as Parkinson's, addiction, and schizophrenia. We will explain the analytical technique of separation of time scales and show how it can produce insights into mechanisms using simplified single-compartment models. We also use morphologically realistic multicompartmental models to address spatially heterogeneous aspects of neural signaling and neural metabolism. Separation of time scale analyses are applied to pacemaking, bursting, and depolarization block in dopamine neurons. Differences in subpopulations with respect to metabolic load are addressed using multicompartmental models.

Computational implications of biophysical diversity and multiple timescales in neurons and synapses for circuit performance

Despite advances in experimental and theoretical neuroscience, we are still trying to identify key biophysical details that are important for characterizing the operation of brain circuits. Biological mechanisms at the level of single neurons and synapses can be combined as 'building blocks' to generate circuit function. We focus on the importance of capturing multiple timescales when describing these intrinsic and synaptic components. Whether inherent in the ionic currents, the neuron's complex morphology, or the neurotransmitter composition of synapses, these multiple timescales prove crucial for capturing the variability and richness of circuit output and enhancing the information-carrying capacity observed across nervous systems.

A positive feedback at the cellular level promotes robustness and modulation at the circuit level

This article highlights the role of a positive feedback gating mechanism at the cellular level in the robustness and modulation properties of rhythmic activities at the circuit level. The results are presented in the context of half-center oscillators, which are simple rhythmic circuits composed of two reciprocally connected inhibitory neuronal populations. Specifically, we focus on rhythms that rely on a particular excitability property, the postinhibitory rebound, an intrinsic cellular property that elicits transient membrane depolarization when released from hyperpolarization. Two distinct ionic currents can evoke this transient depolarization: a hyperpolarization-activated cation current and a low-threshold T-type calcium current. The presence of a slow activation is specific to the T-type calcium current and provides a slow positive feedback at the cellular level that is absent in the cation current. We show that this slow positive feedback is required to endow the network rhythm with physiological modulation and robustness properties. This study thereby identifies an essential cellular property to be retained at the network level in modeling network robustness and modulation.

A novel phase portrait for neuronal excitability

Fifty years ago, FitzHugh introduced a phase portrait that became famous for a twofold reason: it captured in a physiological way the qualitative behavior of Hodgkin-Huxley model and it revealed the power of simple dynamical models to unfold complex firing patterns. To date, in spite of the enormous progresses in qualitative and quantitative neural modeling, this phase portrait has remained a core picture of neuronal excitability. Yet, a major difference between the neurophysiology of 1961 and of 2011 is the recognition of the prominent role of calcium channels in firing mechanisms. We show that including this extra current in Hodgkin-Huxley dynamics leads to a revision of FitzHugh-Nagumo phase portrait that affects in a fundamental way the reduced modeling of neural excitability. The revisited model considerably enlarges the modeling power of the original one. In particular, it captures essential electrophysiological signatures that otherwise require non-physiological alteration or considerable complexification of the classical model. As a basic illustration, the new model is shown to highlight a core dynamical mechanism by which calcium channels control the two distinct firing modes of thalamocortical neurons.

How modeling can reconcile apparently discrepant experimental results: the case of pacemaking in dopaminergic neurons

Midbrain dopaminergic neurons are endowed with endogenous slow pacemaking properties. In recent years, many different groups have studied the basis for this phenomenon, often with conflicting conclusions. In particular, the role of a slowly-inactivating L-type calcium channel in the depolarizing phase between spikes is controversial, and the analysis of slow oscillatory potential (SOP) recordings during the blockade of sodium channels has led to conflicting conclusions. Based on a minimal model of a dopaminergic neuron, our analysis suggests that the same experimental protocol may lead to drastically different observations in almost identical neurons. For example, complete L-type calcium channel blockade eliminates spontaneous firing or has almost no effect in two neurons differing by less than 1% in their maximal sodium conductance. The same prediction can be reproduced in a state of the art detailed model of a dopaminergic neuron. Some of these predictions are confirmed experimentally using single-cell recordings in brain slices. Our minimal model exhibits SOPs when sodium channels are blocked, these SOPs being uncorrelated with the spiking activity, as has been shown experimentally. We also show that block of a specific conductance (in this case, the SK conductance) can have a different effect on these two oscillatory behaviors (pacemaking and SOPs), despite the fact that they have the same initiating mechanism. These results highlight the fact that computational approaches, besides their well known confirmatory and predictive interests in neurophysiology, may also be useful to resolve apparent discrepancies between experimental results.

Dopamine is a neurotransmitter which plays important roles in the control of voluntary movement, motivation and reward, attention, and learning. Dysfunction of midbrain dopaminergic systems is involved in various diseases such as Parkinson's disease, schizophrenia and drug abuse. This underlines the importance of a tight regulation of dopamine levels in the brain. At the cellular level, the release of dopamine is directly correlated to the type of electrical activity (the firing pattern) of nerve cells that produce it, the so-called “dopaminergic neurons”. Therefore, an in depth understanding of the mechanisms underlying the electrical behavior of dopaminergic neurons is of critical importance to find new strategies for the treatment of diseases that result from dysfunction of this system.

M-type channels selectively control bursting in rat dopaminergic neurons

Midbrain dopaminergic neurons in the substantia nigra, pars compacta and ventral tegmental area are critically important in many physiological functions. These neurons exhibit firing patterns that include tonic slow pacemaking, irregular firing and bursting, and the amount of dopamine that is present in the synaptic cleft is much increased during bursting. The mechanisms responsible for the switch between these spiking patterns remain unclear. Using both in-vivo recordings combined with microiontophoretic or intraperitoneal drug applications and in-vitro experiments, we have found that M-type channels, which are present in midbrain dopaminergic cells, modulate the firing during bursting without affecting the background low-frequency pacemaker firing. Thus, a selective blocker of these channels, 10,10-bis(4-pyridinylmethyl)-9(10H)-anthracenone dihydrochloride, specifically potentiated burst firing. Computer modeling of the dopamine neuron confirmed the possibility of a differential influence of M-type channels on excitability during various firing patterns. Therefore, these channels may provide a novel target for the treatment of dopamine-related diseases, including Parkinson’s disease and drug addiction. Moreover, our results demonstrate that the influence of M-type channels on the excitability of these slow pacemaker neurons is conditional upon their firing pattern.