The Interplay of Synaptic Plasticity and Scaling Enables Self-Organized Formation and Allocation of Multiple Memory Representations

Computational models of Idling brain activity for memory processing

Studying the underlying neural mechanisms of cognitive functions of the brain is one of the central questions in modern biology. Moreover, it has significantly impacted the development of novel technologies in artificial intelligence. Spontaneous activity is a unique feature of the brain and is currently lacking in many artificially constructed intelligent machines. Spontaneous activity may represent the brain's idling states, which are internally driven by neuronal networks and possibly participate in offline processing during awake, sleep, and resting states. Evidence is accumulating that the brain's spontaneous activity is not mere noise but part of the mechanisms to process information about previous experiences. A bunch of literature has shown how previous sensory and behavioral experiences influence the subsequent patterns of brain activity with various methods in various animals. It seems, however, that the patterns of neural activity and their computational roles differ significantly from area to area and from function to function. In this article, I review the various forms of the brain's spontaneous activity, especially those observed during memory processing, and some attempts to model the generation mechanisms and computational roles of such activities.

Neural Control and Online Learning for Speed Adaptation of Unmanned Aerial Vehicles

Unmanned aerial vehicles (UAVs) are involved in critical tasks such as inspection and exploration. Thus, they have to perform several intelligent functions. Various control approaches have been proposed to implement these functions. Most classical UAV control approaches, such as model predictive control, require a dynamic model to determine the optimal control parameters. Other control approaches use machine learning techniques that require multiple learning trials to obtain the proper control parameters. All these approaches are computationally expensive. Our goal is to develop an efficient control system for UAVs that does not require a dynamic model and allows them to learn control parameters online with only a few trials and inexpensive computations. To achieve this, we developed a neural control method with fast online learning. Neural control is based on a three-neuron network, whereas the online learning algorithm is derived from a neural correlation-based learning principle with predictive and reflexive sensory information. This neural control technique is used here for the speed adaptation of the UAV. The control technique relies on a simple input signal from a compact optical distance measurement sensor that can be converted into predictive and reflexive sensory information for the learning algorithm. Such speed adaptation is a fundamental function that can be used as part of other complex control functions, such as obstacle avoidance. The proposed technique was implemented on a real UAV system. Consequently, the UAV can quickly learn within 3–4 trials to proactively adapt its flying speed to brake at a safe distance from the obstacle or target in the horizontal and vertical planes. This speed adaptation is also robust against wind perturbation. We also demonstrated a combination of speed adaptation and obstacle avoidance for UAV navigations, which is an important intelligent function toward inspection and exploration.

Rational designing of oscillatory rhythmicity for memory rescue in plasticity-impaired learning networks

In the brain, oscillatory strength embedded in network rhythmicity is important for processing experiences, and this process is disrupted in certain psychiatric disorders. The use of rhythmic network stimuli can change these oscillations and has shown promise in terms of improving cognitive function, although the underlying mechanisms are poorly understood. Here, we combine a two-layer learning model, with experiments involving genetically modified mice, that provides precise control of experience-driven oscillations by manipulating long-term potentiation of excitatory synapses onto inhibitory interneurons (LTP_E→I). We find that, in the absence of LTP_E→I, impaired network dynamics and memory are rescued by activating inhibitory neurons to augment the power in theta and gamma frequencies, which prevents network overexcitation with less inhibitory rebound. In contrast, increasing either theta or gamma power alone was less effective. Thus, inducing network changes at dual frequencies is involved in memory encoding, indicating a potentially feasible strategy for optimizing network-stimulating therapies.

From Biological Synapses to “Intelligent” Robots

This selective review explores biologically inspired learning as a model for intelligent robot control and sensing technology on the basis of specific examples. Hebbian synaptic learning is discussed as a functionally relevant model for machine learning and intelligence, as explained on the basis of examples from the highly plastic biological neural networks of invertebrates and vertebrates. Its potential for adaptive learning and control without supervision, the generation of functional complexity, and control architectures based on self-organization is brought forward. Learning without prior knowledge based on excitatory and inhibitory neural mechanisms accounts for the process through which survival-relevant or task-relevant representations are either reinforced or suppressed. The basic mechanisms of unsupervised biological learning drive synaptic plasticity and adaptation for behavioral success in living brains with different levels of complexity. The insights collected here point toward the Hebbian model as a choice solution for “intelligent” robotics and sensor systems.

Transection of the ventral hippocampal commissure impairs spatial reference but not contextual or spatial working memory

Plasticity is a neural phenomenon in which experience induces long-lasting changes to neuronal circuits and is at the center of most neurobiological theories of learning and memory. However, too much plasticity is maladaptive and must be balanced with substrate stability. Area CA3 of the hippocampus provides such a balance via hemispheric lateralization, with the left hemisphere dominant in providing plasticity and the right specialized for stability. Left and right CA3 project bilaterally to CA1; however, it is not known whether this downstream merging of lateralized plasticity and stability is functional. We hypothesized that interhemispheric convergence of input from these pathways is essential for integrating spatial memory stored in the left CA3 with navigational working memory facilitated by the right CA3. To test this, we severed interhemispheric connections between the left and right hippocampi in mice and assessed learning and memory. Despite damage to this major hippocampal fiber tract, hippocampus-dependent navigational working memory and short- and long-term memory were both spared. However, tasks that required the integration of information retrieved from memory with ongoing navigational working memory and navigation were impaired. We propose that one function of interhemispheric communication in the mouse hippocampus is to integrate lateralized processing of plastic and stable circuits to facilitate memory-guided spatial navigation.

Memory destabilization during reconsolidation: a consequence of homeostatic plasticity?

Remembering is not a static process: When retrieved, a memory can be destabilized and become prone to modifications. This phenomenon has been demonstrated in a number of brain regions, but the neuronal mechanisms that rule memory destabilization and its boundary conditions remain elusive. Using two distinct computational models that combine Hebbian plasticity and synaptic downscaling, we show that homeostatic plasticity can function as a destabilization mechanism, accounting for behavioral results of protein synthesis inhibition upon reactivation with different re-exposure times. Furthermore, by performing systematic reviews, we identify a series of overlapping molecular mechanisms between memory destabilization and synaptic downscaling, although direct experimental links between both phenomena remain scarce. In light of these results, we propose a theoretical framework where memory destabilization can emerge as an epiphenomenon of homeostatic adaptations prompted by memory retrieval.

Homeostatic synaptic scaling establishes the specificity of an associative memory

Correlation-based (Hebbian) forms of synaptic plasticity are crucial for the initial encoding of associative memories but likely insufficient to enable the stable storage of multiple specific memories within neural circuits. Theoretical studies have suggested that homeostatic synaptic normalization rules provide an essential countervailing force that can stabilize and expand memory storage capacity. Although such homeostatic mechanisms have been identified and studied for decades, experimental evidence that they play an important role in associative memory is lacking. Here, we show that synaptic scaling, a widely studied form of homeostatic synaptic plasticity that globally renormalizes synaptic strengths, is dispensable for initial associative memory formation but crucial for the establishment of memory specificity. We used conditioned taste aversion (CTA) learning, a form of associative learning that relies on Hebbian mechanisms within gustatory cortex (GC), to show that animals conditioned to avoid saccharin initially generalized this aversion to other novel tastants. Specificity of the aversion to saccharin emerged slowly over a time course of many hours and was associated with synaptic scaling down of excitatory synapses onto conditioning-active neuronal ensembles within gustatory cortex. Blocking synaptic scaling down in the gustatory cortex enhanced the persistence of synaptic strength increases induced by conditioning and prolonged the duration of memory generalization. Taken together, these findings demonstrate that synaptic scaling is crucial for sculpting the specificity of an associative memory and suggest that the relative strengths of Hebbian and homeostatic plasticity can modulate the balance between stable memory formation and memory generalization.

Unsupervised Learning and Clustered Connectivity Enhance Reinforcement Learning in Spiking Neural Networks

Reinforcement learning is a paradigm that can account for how organisms learn to adapt their behavior in complex environments with sparse rewards. To partition an environment into discrete states, implementations in spiking neuronal networks typically rely on input architectures involving place cells or receptive fields specified ad hoc by the researcher. This is problematic as a model for how an organism can learn appropriate behavioral sequences in unknown environments, as it fails to account for the unsupervised and self-organized nature of the required representations. Additionally, this approach presupposes knowledge on the part of the researcher on how the environment should be partitioned and represented and scales poorly with the size or complexity of the environment. To address these issues and gain insights into how the brain generates its own task-relevant mappings, we propose a learning architecture that combines unsupervised learning on the input projections with biologically motivated clustered connectivity within the representation layer. This combination allows input features to be mapped to clusters; thus the network self-organizes to produce clearly distinguishable activity patterns that can serve as the basis for reinforcement learning on the output projections. On the basis of the MNIST and Mountain Car tasks, we show that our proposed model performs better than either a comparable unclustered network or a clustered network with static input projections. We conclude that the combination of unsupervised learning and clustered connectivity provides a generic representational substrate suitable for further computation.

Homeostatic mechanisms regulate distinct aspects of cortical circuit dynamics

Homeostasis is indispensable to counteract the destabilizing effects of Hebbian plasticity. Although it is commonly assumed that homeostasis modulates synaptic strength, membrane excitability, and firing rates, its role at the neural circuit and network level is unknown. Here, we identify changes in higher-order network properties of freely behaving rodents during prolonged visual deprivation. Strikingly, our data reveal that functional pairwise correlations and their structure are subject to homeostatic regulation. Using a computational model, we demonstrate that the interplay of different plasticity and homeostatic mechanisms can capture the initial drop and delayed recovery of firing rates and correlations observed experimentally. Moreover, our model indicates that synaptic scaling is crucial for the recovery of correlations and network structure, while intrinsic plasticity is essential for the rebound of firing rates, suggesting that synaptic scaling and intrinsic plasticity can serve distinct functions in homeostatically regulating network dynamics.