Continual Learning in a Multi-Layer Network of an Electric Fish

Active sensing: Eavesdropping on your neighbor to locate prey

When animals using active sensing, e.g., sonar or an electric organ discharge, cooperate while foraging, the emitted sound or electric field is available to neighboring conspecifics. Experimental and modelling studies have shown that an electric fish can use the discharge of neighbors to extend their own electrosensory prey detection range.

Silences, spikes and bursts: Three-part knot of the neural code

When a neuron breaks silence, it can emit action potentials in a number of patterns. Some responses are so sudden and intense that electrophysiologists felt the need to single them out, labelling action potentials emitted at a particularly high frequency with a metonym - bursts. Is there more to bursts than a figure of speech? After all, sudden bouts of high-frequency firing are expected to occur whenever inputs surge. The burst coding hypothesis advances that the neural code has three syllables: silences, spikes and bursts. We review evidence supporting this ternary code in terms of devoted mechanisms for burst generation, synaptic transmission and synaptic plasticity. We also review the learning and attention theories for which such a triad is beneficial.

Task-dependent optimal representations for cerebellar learning

The cerebellar granule cell layer has inspired numerous theoretical models of neural representations that support learned behaviors, beginning with the work of Marr and Albus. In these models, granule cells form a sparse, combinatorial encoding of diverse sensorimotor inputs. Such sparse representations are optimal for learning to discriminate random stimuli. However, recent observations of dense, low-dimensional activity across granule cells have called into question the role of sparse coding in these neurons. Here, we generalize theories of cerebellar learning to determine the optimal granule cell representation for tasks beyond random stimulus discrimination, including continuous input-output transformations as required for smooth motor control. We show that for such tasks, the optimal granule cell representation is substantially denser than predicted by classical theories. Our results provide a general theory of learning in cerebellum-like systems and suggest that optimal cerebellar representations are task-dependent.

Hormonal coordination of motor output and internal prediction of sensory consequences in an electric fish

Steroid hormones remodel neural networks to induce seasonal or developmental changes in behavior. Hormonal changes in behavior likely require coordinated changes in sensorimotor integration. Here, we investigate hormonal effects on a predictive motor signal, termed corollary discharge, that modulates sensory processing in weakly electric mormyrid fish. In the electrosensory pathway mediating communication behavior, inhibition activated by a corollary discharge blocks sensory responses to self-generated electric pulses, allowing the downstream circuit to selectively analyze communication signals from nearby fish. These pulses are elongated by increasing testosterone levels in males during the breeding season. We induced electric-pulse elongation using testosterone treatment and found that the timing of electroreceptor responses to self-generated pulses was delayed as electric-pulse duration increased. Simultaneous recordings from an electrosensory nucleus and electromotor neurons revealed that the timing of corollary discharge inhibition was delayed and elongated by testosterone. Furthermore, this shift in the timing of corollary discharge inhibition was precisely matched to the shift in timing of receptor responses to self-generated pulses. We then asked whether the shift in inhibition timing was caused by direct action of testosterone on the corollary discharge circuit or by plasticity acting on the circuit in response to altered sensory feedback. We surgically silenced the electric organ of fish and found similar hormonal modulation of corollary discharge timing between intact and silent fish, suggesting that sensory feedback was not required for this shift. Our findings demonstrate that testosterone directly regulates motor output and internal prediction of the resulting sensory consequences in a coordinated manner.

An internal model for canceling self-generated sensory input in freely behaving electric fish

Internal models that predict the sensory consequences of motor actions are vital for sensory, motor, and cognitive functions. However, the relationship between motor action and sensory input is complex, often varying from one moment to another depending on the state of the animal and the environment. The neural mechanisms for generating predictions under such challenging, real-world conditions remain largely unknown. Using novel methods for underwater neural recording, a quantitative analysis of unconstrained behavior, and computational modeling, we provide evidence for an unexpectedly sophisticated internal model at the first stage of active electrosensory processing in mormyrid fish. Closed-loop manipulations reveal that electrosensory lobe neurons are capable of simultaneously learning and storing multiple predictions of the sensory consequences of motor commands specific to different sensory states. These results provide mechanistic insights into how internal motor signals and information about the sensory environment are combined within a cerebellum-like circuitry to predict the sensory consequences of natural behavior.

A mechanism for differential control of axonal and dendritic spiking underlying learning in a cerebellum-like circuit

In addition to the action potentials used for axonal signaling, many neurons generate dendritic "spikes" associated with synaptic plasticity. However, in order to control both plasticity and signaling, synaptic inputs must be able to differentially modulate the firing of these two spike types. Here, we investigate this issue in the electrosensory lobe (ELL) of weakly electric mormyrid fish, where separate control over axonal and dendritic spikes is essential for the transmission of learned predictive signals from inhibitory interneurons to the output stage of the circuit. Through a combination of experimental and modeling studies, we uncover a novel mechanism by which sensory input selectively modulates the rate of dendritic spiking by adjusting the amplitude of backpropagating axonal action potentials. Interestingly, this mechanism does not require spatially segregated synaptic inputs or dendritic compartmentalization but relies instead on an electrotonically distant spike initiation site in the axon-a common biophysical feature of neurons.

Statistical mechanics of continual learning: Variational principle and mean-field potential

An obstacle to artificial general intelligence is set by continual learning of multiple tasks of a different nature. Recently, various heuristic tricks, both from machine learning and from neuroscience angles, were proposed, but they lack a unified theory foundation. Here, we focus on continual learning in single-layered and multilayered neural networks of binary weights. A variational Bayesian learning setting is thus proposed in which the neural networks are trained in a field-space, rather than a gradient-ill-defined discrete-weight space, and furthermore, weight uncertainty is naturally incorporated, and it modulates synaptic resources among tasks. From a physics perspective, we translate variational continual learning into a Franz-Parisi thermodynamic potential framework, where previous task knowledge serves as a prior probability and a reference as well. We thus interpret the continual learning of the binary perceptron in a teacher-student setting as a Franz-Parisi potential computation. The learning performance can then be analytically studied with mean-field order parameters, whose predictions coincide with numerical experiments using stochastic gradient descent methods. Based on the variational principle and Gaussian field approximation of internal preactivations in hidden layers, we also derive the learning algorithm considering weight uncertainty, which solves the continual learning with binary weights using multilayered neural networks, and performs better than the currently available metaplasticity algorithm in which binary synapses bear hidden continuous states and the synaptic plasticity is modulated by a heuristic regularization function. Our proposed principled frameworks also connect to elastic weight consolidation, weight-uncertainty modulated learning, and neuroscience-inspired metaplasticity, providing a theoretically grounded method for real-world multitask learning with deep networks.

Bidirectional synaptic plasticity rapidly modifies hippocampal representations

Learning requires neural adaptations thought to be mediated by activity-dependent synaptic plasticity. A relatively non-standard form of synaptic plasticity driven by dendritic calcium spikes, or plateau potentials, has been reported to underlie place field formation in rodent hippocampal CA1 neurons. Here, we found that this behavioral timescale synaptic plasticity (BTSP) can also reshape existing place fields via bidirectional synaptic weight changes that depend on the temporal proximity of plateau potentials to pre-existing place fields. When evoked near an existing place field, plateau potentials induced less synaptic potentiation and more depression, suggesting BTSP might depend inversely on postsynaptic activation. However, manipulations of place cell membrane potential and computational modeling indicated that this anti-correlation actually results from a dependence on current synaptic weight such that weak inputs potentiate and strong inputs depress. A network model implementing this bidirectional synaptic learning rule suggested that BTSP enables population activity, rather than pairwise neuronal correlations, to drive neural adaptations to experience.

Self-backpropagation of synaptic modifications elevates the efficiency of spiking and artificial neural networks

Many synaptic plasticity rules found in natural circuits have not been incorporated into artificial neural networks (ANNs). We showed that incorporating a nonlocal feature of synaptic plasticity found in natural neural networks, whereby synaptic modification at output synapses of a neuron backpropagates to its input synapses made by upstream neurons, markedly reduced the computational cost without affecting the accuracy of spiking neural networks (SNNs) and ANNs in supervised learning for three benchmark tasks. For SNNs, synaptic modification at output neurons generated by spike timing–dependent plasticity was allowed to self-propagate to limited upstream synapses. For ANNs, modified synaptic weights via conventional backpropagation algorithm at output neurons self-backpropagated to limited upstream synapses. Such self-propagating plasticity may produce coordinated synaptic modifications across neuronal layers that reduce computational cost.

Population coding in the cerebellum: a machine learning perspective

The cere resembles a feedforward, three-layer network of neurons in which the "hidden layer" consists of Purkinje cells (P-cells) and the output layer consists of deep cerebellar nucleus (DCN) neurons. In this analogy, the output of each DCN neuron is a prediction that is compared with the actual observation, resulting in an error signal that originates in the inferior olive. Efficient learning requires that the error signal reach the DCN neurons, as well as the P-cells that project onto them. However, this basic rule of learning is violated in the cerebellum: the olivary projections to the DCN are weak, particularly in adulthood. Instead, an extraordinarily strong signal is sent from the olive to the P-cells, producing complex spikes. Curiously, P-cells are grouped into small populations that converge onto single DCN neurons. Why are the P-cells organized in this way, and what is the membership criterion of each population? Here, I apply elementary mathematics from machine learning and consider the fact that P-cells that form a population exhibit a special property: they can synchronize their complex spikes, which in turn suppress activity of DCN neuron they project to. Thus complex spikes cannot only act as a teaching signal for a P-cell, but through complex spike synchrony, a P-cell population may act as a surrogate teacher for the DCN neuron that produced the erroneous output. It appears that grouping of P-cells into small populations that share a preference for error satisfies a critical requirement of efficient learning: providing error information to the output layer neuron (DCN) that was responsible for the error, as well as the hidden layer neurons (P-cells) that contributed to it. This population coding may account for several remarkable features of behavior during learning, including multiple timescales, protection from erasure, and spontaneous recovery of memory.

Neural Networks: How a Multi-Layer Network Learns to Disentangle Exogenous from Self-Generated Signals

Artificial multi-layer networks can learn difficult tasks, such as recognizing faces, but their architecture and learning rules appear to be very different from those of biological neural networks. Experimental and computational studies of a two-layered biological neural network have revealed how the learning rules used in artificial neural networks can be efficiently implemented by neurons with complex dynamics and precisely organized connectivity.

A cerebellum-like circuit in the lateral line system of fish cancels mechanosensory input associated with its own movements

An animal's own movement exerts a profound impact on sensory input to its nervous system. Peripheral sensory receptors do not distinguish externally generated stimuli from stimuli generated by an animal's own behavior (reafference) - although the animal often must. One way that nervous systems can solve this problem is to provide movement-related signals (copies of motor commands and sensory feedback) to sensory systems, which can then be used to generate predictions that oppose or cancel out sensory responses to reafference. Here, we studied the use of movement-related signals to generate sensory predictions in the lateral line medial octavolateralis nucleus (MON) of the little skate. In the MON, mechanoreceptive afferents synapse on output neurons that also receive movement-related signals from central sources, via a granule cell parallel fiber system. This parallel fiber system organization is characteristic of a set of so-called cerebellum-like structures. Cerebellum-like structures have been shown to support predictive cancellation of reafference in the electrosensory systems of fish and the auditory system of mice. Here, we provide evidence that the parallel fiber system in the MON can generate predictions that are negative images of (and therefore cancel) sensory input associated with respiratory and fin movements. The MON, found in most aquatic vertebrates, is probably one of the most primitive cerebellum-like structures and a starting point for cerebellar evolution. The results of this study contribute to a growing body of work that uses an evolutionary perspective on the vertebrate cerebellum to understand its functional diversity in animal behavior.

Synaptic Plasticity Forms and Functions

Synaptic plasticity, the activity-dependent change in neuronal connection strength, has long been considered an important component of learning and memory. Computational and engineering work corroborate the power of learning through the directed adjustment of connection weights. Here we review the fundamental elements of four broadly categorized forms of synaptic plasticity and discuss their functional capabilities and limitations. Although standard, correlation-based, Hebbian synaptic plasticity has been the primary focus of neuroscientists for decades, it is inherently limited. Three-factor plasticity rules supplement Hebbian forms with neuromodulation and eligibility traces, while true supervised types go even further by adding objectives and instructive signals. Finally, a recently discovered hippocampal form of synaptic plasticity combines the above elements, while leaving behind the primary Hebbian requirement. We suggest that the effort to determine the neural basis of adaptive behavior could benefit from renewed experimental and theoretical investigation of more powerful directed types of synaptic plasticity.