Synergies between synaptic and intrinsic plasticity in echo state networks

Exploiting Signal Propagation Delays to Match Task Memory Requirements in Reservoir Computing

Recurrent neural networks (RNNs) transmit information over time through recurrent connections. In contrast, biological neural networks use many other temporal processing mechanisms. One of these mechanisms is the inter-neuron delays caused by varying axon properties. Recently, this feature was implemented in echo state networks (ESNs), a type of RNN, by assigning spatial locations to neurons and introducing distance-dependent inter-neuron delays. These delays were shown to significantly improve ESN task performance. However, thus far, it is still unclear why distance-based delay networks (DDNs) perform better than ESNs. In this paper, we show that by optimizing inter-node delays, the memory capacity of the network matches the memory requirements of the task. As such, networks concentrate their memory capabilities to the points in the past which contain the most information for the task at hand. Moreover, we show that DDNs have a greater total linear memory capacity, with the same amount of non-linear processing power.

Evolving Dual-Threshold Bienenstock-Cooper-Munro Learning Rules in Echo State Networks

The strengthening and the weakening of synaptic strength in existing Bienenstock-Cooper-Munro (BCM) learning rule are determined by a long-term potentiation (LTP) sliding modification threshold and the afferent synaptic activities. However, synaptic long-term depression (LTD) even affects low-active synapses during the induction of synaptic plasticity, which may lead to information loss. Biological experiments have found another LTD threshold that can induce either potentiation or depression or no change, even at the activated synapses. In addition, existing BCM learning rules can only select a set of fixed rule parameters, which is biologically implausible and practically inflexible to learn the structural information of input signals. In this article, an evolved dual-threshold BCM learning rule is proposed to regulate the reservoir internal connection weights of the echo-state-network (ESN), which can contribute to alleviating information loss and enhancing learning performance by introducing different optimal LTD thresholds for different postsynaptic neurons. Our experimental results show that the evolved dual-threshold BCM learning rule can result in the synergistic learning of different plasticity rules, effectively improving the learning performance of an ESN in comparison with existing neural plasticity learning rules and some state-of-the-art ESN variants on three widely used benchmark tasks and the prediction of an esterification process.

Intrinsic threshold plasticity: cholinergic activation and role in the neuronal recognition of incomplete input patterns

Activity-dependent changes in membrane excitability are observed in neurons across brain areas and represent a cell-autonomous form of plasticity (intrinsic plasticity; IP) that in itself does not involve alterations in synaptic strength (synaptic plasticity; SP). Non-homeostatic IP may play an essential role in learning, e.g. by changing the action potential threshold near the soma. A computational problem, however, arises from the implication that such amplification does not discriminate between synaptic inputs and therefore may reduce the resolution of input representation. Here, we investigate consequences of IP for the performance of an artificial neural network in (a) the discrimination of unknown input patterns and (b) the recognition of known/learned patterns. While negative changes in threshold potentials in the output layer indeed reduce its ability to discriminate patterns, they benefit the recognition of known but incompletely presented patterns. An analysis of thresholds and IP-induced threshold changes in published sets of physiological data obtained from whole-cell patch-clamp recordings from L2/3 pyramidal neurons in (a) the primary visual cortex (V1) of awake macaques and (b) the primary somatosensory cortex (S1) of mice in vitro, respectively, reveals a difference between resting and threshold potentials of ∼15 mV for V1 and ∼25 mV for S1, and a total plasticity range of ∼10 mV (S1). The most efficient activity pattern to lower threshold is paired cholinergic and electric activation. Our findings show that threshold reduction promotes a shift in neural coding strategies from accurate faithful representation to interpretative assignment of input patterns to learned object categories. KEY POINTS: Intrinsic plasticity may change the action potential threshold near the soma of neurons (threshold plasticity), thus altering the input-output function for all synaptic inputs 'upstream' of the plasticity location. A potential problem arising from this shared amplification is that it may reduce the ability to discriminate between different input patterns. Here, we assess the performance of an artificial neural network in the discrimination of unknown input patterns as well as the recognition of known patterns subsequent to changes in the spike threshold. We observe that negative changes in threshold potentials do reduce discrimination performance, but at the same time improve performance in an object recognition task, in particular when patterns are incompletely presented. Analysis of whole-cell patch-clamp recordings from pyramidal neurons in the primary somatosensory cortex (S1) of mice reveals that negative threshold changes preferentially result from electric stimulation of neurons paired with the activation of muscarinic acetylcholine receptors.

An integrated measure of GABA to characterize post-stroke plasticity

Stroke is a major cause of death and chronic neurological disability. Despite the improvements in stroke care, the number of patients affected by stroke keeps increasing and many stroke survivors are left permanently disabled. Current therapies are limited in efficacy. Understanding the neurobiological mechanisms underlying post-stroke recovery is therefore crucial to find new therapeutic options to address this medical burden. Long-lasting and widespread alterations of γ-aminobutyric acid (GABA) neurotransmission seem to play a key role in stroke recovery. In this review we first discuss a possible model of GABAergic modulation of post-stroke plasticity. We then overview the techniques currently available to non-invasively assess GABA in patients and the conclusions drawn from this limited body of work. Finally, we address the remaining open questions to clarify GABAergic changes underlying post-stroke recovery, we briefly review possible ways to modulate GABA post stroke and propose a novel approach to thoroughly quantify GABA in stroke patients, by integrating its concentration, the activity of its receptors and its link with microstructural changes.