Modifiable synapses necessary for learning

The limits of counting accuracy in distributed neural representations

Learning about a causal or statistical association depends on comparing frequencies of joint occurrence with frequencies expected from separate occurrences, and to do this, events must somehow be counted. Physiological mechanisms can easily generate the necessary measures if there is a direct, one-to-one relationship between significant events and neural activity, but if the events are represented across cell populations in a distributed manner, the counting of one event will be interfered with by the occurrence of others. Although the mean interference can be allowed for, there is inevitably an increase in the variance of frequency estimates that results in the need for extra data to achieve reliable learning. This lowering of statistical efficiency (Fisher, 1925) is calculated as the ratio of the minimum to actual variance of the estimates. We define two neural models, based on presynaptic and Hebbian synaptic modification, and explore the effects of sparse coding and the relative frequencies of events on the efficiency of frequency estimates. High counting efficiency must be a desirable feature of biological representations, but the results show that the number of events that can be counted simultaneously with 50% efficiency is fewer than the number of cells or 0.1-0.25 of the number of synapses (on the two models) - many fewer than can be unambiguously represented. Direct representations would lead to greater counting efficiency, but distributed representations have the versatility of detecting and counting many unforeseen or rare events. Efficient counting of rare but important events requires that they engage more active cells than common or unimportant ones. The results suggest reasons that representations in the cerebral cortex appear to use extravagant numbers of cells and modular organization, and they emphasize the importance of neuronal trigger features and the phenomena of habituation and attention.

Possible mechanisms for reducing memory confusion during sleep

Confusion readily occurs in memory processes between patterns that overlap substantially. Possible mechanisms are considered that might operate in an automatic manner to reduce such confusion. One such mechanism is the recall of patterns in a distorted way, so that they are enriched with greater activity in distinctive elements of experienced patterns than in overlapping elements. Selective consolidation based on such enriched patterns would reduce confusion in long-term recall and might benefit discrimination learning. It is shown how automatic algorithms could achieve this through a process with two phases. In the first phase, somewhat analogous to slow-wave sleep, it is necessary for the normal tendency of the nervous system to learn correlations of associated activity to be disabled. The second phase must occur while the cells most active in the first phase are relatively inexcitable. Enriched patterns would be generated during this phase through recall, which might be triggered by bursts of activity such as occur in rapid eye movement sleep. Selective consolidation would take place during the second phase. If such processes do occur in the nervous system, it seems likely that they would have evolved to occur during sleep.

Doubly modifiable synapses: a model of short and long term auto-associative memory

Synapses that can be strengthened in temporary and persistent manners by two separate mechanisms are shown to have powerful advantages in neural networks that perform auto-associative recall and recognition. A multiplicative relation between the two weights allows the same set of connections to be used in a closely interactive way for short-term and long-term memory. Algorithms and simulations are described for the storage, consolidation and recall of patterns that have been presented only once to a network. With double modifiability, the short-term performance is dramatically improved, becoming almost independent of the amount of long-term experience. The high quality of short-term recall allows consolidation to take place, with benefits from the selection and optimization of long term engrams to take account of relations between stored patterns. Long-term capacity is greater than short-term capacity, with little or no deficit compared with that obtained with singly modifiable synapses. Long-term recall requires special, simply implemented, procedures for increasing the temporary weights of the synapses being used to initiate recall. A consolidation algorithm is described for improving long-term recall when there is overlap between patterns. Confusional errors are reduced by strengthening the associations between non-overlapping elements in the patterns, in a two-stage process that has several of the characteristics of sleep.

Effect of minaprine on synaptic transmission in the neocortex of the rat in vivo

The effects of minaprine and/or excitatory amino acid antagonists on transcallosal responses were examined in urethane-anesthetized rats. The transcallosal response was recorded from the surface of the anterior neocortex, following electrical stimulation of the contralateral corpus callosum. The transcallosal response consisted of a biphasic positive-negative waveform. Intravenously-administered minaprine increased the amplitude of the positive- and negative-waves, in a dose-dependent manner. Intracortical injection of (+/-)-2-amino-5-phosphonovalerate (APV) and gamma-D-glutamylglycine (DGG) reduced the amplitude of the negative-wave, with no effects on the amplitude of the positive-wave. L-Glutamate diethylester (GDEE) had no effect on the transcallosal response. The minaprine-induced increase in the amplitude of the negative-wave was completely antagonized by simultaneous intracortical injections of APV and DGG which, per se, did not affect that transcallosal response. In contrast, APV and DGG had no effect on the increase in the amplitude of the positive-wave induced by minaprine. The enhancing effect of minaprine on the transcallosal response remained unaltered in case of an intracortical injection of GDEE. These findings indicate that the negative-wave of the transcallosal response may be related to receptors for excitatory amino acids. The possibility that the pharmacological action of minaprine on synaptic transmission in the neocortex may be linked to the excitatory amino acid receptors warrants further attention.

Enhanced long-term potentiation induced in rat dentate gyrus by coactivation of septal and entorhinal inputs: temporal constraints

High-frequency activation of the entorhinal cortical (perforant path) inputs to the rat dentate gyrus can produce a long-term potentiation (LTP) of perforant path-dentate evoked responses. In this paper we examined the enhanced LTP effects produced by coactivation of septal and entorhinal inputs to the dentate gyrus. Trains of electrical stimulation applied to the two inputs were found to increase the magnitude of LTP to a level above that produced by trains applied to the perforant path alone. The largest LTP increments were observed when the septal trains were applied less than 100 ms prior to the perforant path trains. If the septal trains followed the perforant path trains there was no additional increment in LTP magnitude, regardless of the intertrain interval. The relationship of this cooperativity effect to mechanisms of associative learning is discussed.

A comparison of the postnatal development of post-activation potentiation in the neocortex and dentate gyrus of the rat

The postnatal development of short-term potentiation (STP) and long-term potentiation (LTP) was examined in the neocortex and dentate gyrus of rats aged 7 days to adult. STP and LTP of the transcallosal response in the neocortex could not be demonstrated until the third postnatal week. STP and LTP of the perforant path-dentate response could not be demonstrated until the second postnatal week. In both cases, STP appeared several days before LTP. Structural and neurochemical correlates of STP/LTP development, and their implications for possible STP/LTP mechanisms, are discussed.

The postnatal development of post-activation potentiation in the rat neocortex

The postnatal development of short-term potentiation (STP) and long-term potentiation (LTP) of the transcallosal response (TCR) was examined in anesthetized rats, aged 7 days (PN7) to adult (greater than or equal to PN180). Stimulation of callosal fibers produced a biphasic, positive-negative TCR recorded at the surface of the anterior neocortex in all age groups. The TCR showed a decrease in threshold, latency and halfwidth, and an increase in peak amplitude with age. High frequency callosal stimulation produced marked STP and LTP of the mature TCR. STP in young animals, however, could not be reliably detected until after PN16. LTP was not detected until PN18. STP and LTP approached adult levels rapidly after their initial appearance. The correlations between STP/LTP onset and development and known structural developments such as initial synaptogenesis, spine formation and myelinogenesis were not particularly strong. The implications of these results for specific hypotheses of STP/LTP mechanisms were discussed.

Storage of temporal pattern sequence in a network

Learning of single patterns and a temporal pattern sequence in a network when the coupling coefficients between the network elements change their values according to a definite coupling function is described. In contrast to technical systems (e.g. film, tape) where temporal sequences are often encoded in the storage location, the network stores information only by changing the values of the coupling coefficients. A network of 100 elements was stimulated on an UNIVAC 1100/80 computer. Eight single patterns and a sequence of these patterns were offered at the input of the network. After the learning process the network reproduces every stored pattern as an output signal when only parts of it are fed in. The activity, that is the sum of all output signals, is regulated by an external control signal. By setting that control signal to a suitable value the network is able to reproduce the stored pattern sequence starting from any arbitrary pattern. Lowering the external control signal during that process causes the network to hold the last presented pattern until the external control signal is changed again. It is speculated that the coupling function implemented in the stimulation may be analogous to a characteristic describing the chemical process of cooperative binding.

The recall of events through the learning of associations between their parts

A form of recall is discussed in which some elements of an event serve to initiate recall of the whole set of elements which comprised an event. The conditions which enable learned associations between pairs of elements to be used for such recall are quantitatively assessed for recall procedures which use only direct associations from the initial elements (‘simple recall’) and which use both direct and indirect associations (‘progressive recall’). The theory is equivalent to the analysis of a neural model in which randomly arranged excitatory connections between a set of cells can be strengthened by simultaneous activity in their presynaptic and postsynaptic elements. The problems involved in a direct neural implementation of the model are discussed, and it is seen that some of the mechanisms resemble known physiological mechanisms in the cerebral cortex. The relationship between recall and recognition learning is discussed and the evidence relating these functions to the hippocampal formation is assessed.