The predictive brain: temporal coincidence and temporal order in synaptic learning mechanisms

Some forms of synaptic plasticity depend on the temporal coincidence of presynaptic activity and postsynaptic response. This requirement is consistent with the Hebbian, or correlational, type of learning rule used in many neural network models. Recent evidence suggests that synaptic plasticity may depend in part on the production of a membrane permeant-diffusible signal so that spatial volume may also be involved in correlational learning rules. This latter form of synaptic change has been called volume learning. In both Hebbian and volume learning rules, interaction among synaptic inputs depends on the degree of coincidence of the inputs and is otherwise insensitive to their exact temporal order. Conditioning experiments and psychophysical studies have shown, however, that most animals are highly sensitive to the temporal order of the sensory inputs. Although these experiments assay the behavior of the entire animal or perceptual system, they raise the possibility that nervous systems may be sensitive to temporally ordered events at many spatial and temporal scales. We suggest here the existence of a new class of learning rule, called a predictive Hebbian learning rule, that is sensitive to the temporal ordering of synaptic inputs. We show how this predictive learning rule could act at single synaptic connections and through diffuse neuromodulatory systems.

Protein secondary structure prediction using two-level case-based reasoning

We have developed a two-level case-based reasoning architecture for predicting protein secondary structure. The central idea is to break the problem into two levels: (i) reasoning at the object (protein) level and using the global information from this level to focus on a more restricted problem space; (ii) decomposing objects into pieces (segments) and reasoning at the level of internal structures. As a last step to the procedure, inferences from the parts of the internal structure are synthesized into predictions about global structure. The architecture has been developed and tested on a commonly used data set with 69.5% predictive accuracy. It was then tested on a new data set with 68.2% accuracy. With additional tuning, over 70% accuracy was achieved. In addition, a series of experiments were conducted to test various aspects of the method and the results are informative.

Learning Finite State Machines With Self-Clustering Recurrent Networks

Recent work has shown that recurrent neural networks have the ability to learn finite state automata from examples. In particular, networks using second-order units have been successful at this task. In studying the performance and learning behavior of such networks we have found that the second-order network model attempts to form clusters in activation space as its internal representation of states. However, these learned states become unstable as longer and longer test input strings are presented to the network. In essence, the network “forgets” where the individual states are in activation space. In this paper we propose a new method to force such a network to learn stable states by introducing discretization into the network and using a pseudo-gradient learning rule to perform training. The essence of the learning rule is that in doing gradient descent, it makes use of the gradient of a sigmoid function as a heuristic hint in place of that of the hard-limiting function, while still using the discretized value in the feedback update path. The new structure uses isolated points in activation space instead of vague clusters as its internal representation of states. It is shown to have similar capabilities in learning finite state automata as the original network, but without the instability problem. The proposed pseudo-gradient learning rule may also be used as a basis for training other types of networks that have hard-limiting threshold activation functions.

Genetic algorithms: principles of natural selection applied to computation

A genetic algorithm is a form of evolution that occurs on a computer. Genetic algorithms are a search method that can be used for both solving problems and modeling evolutionary systems. With various mapping techniques and an appropriate measure of fitness, a genetic algorithm can be tailored to evolve a solution for many types of problems, including optimization of a function of determination of the proper order of a sequence. Mathematical analysis has begun to explain how genetic algorithms work and how best to use them. Recently, genetic algorithms have been used to model several natural evolutionary systems, including immune systems.

Improving Generalization for Temporal Difference Learning: The Successor Representation

Estimation of returns over time, the focus of temporal difference (TD) algorithms, imposes particular constraints on good function approximators or representations. Appropriate generalization between states is determined by how similar their successors are, and representations should follow suit. This paper shows how TD machinery can be used to learn such representations, and illustrates, using a navigation task, the appropriately distributed nature of the result.

Learning and development in neural networks: the importance of starting small

It is a striking fact that in humans the greatest learning occurs precisely at that point in time--childhood--when the most dramatic maturational changes also occur. This report describes possible synergistic interactions between maturational change and the ability to learn a complex domain (language), as investigated in connectionist networks. The networks are trained to process complex sentences involving relative clauses, number agreement, and several types of verb argument structure. Training fails in the case of networks which are fully formed and 'adultlike' in their capacity. Training succeeds only when networks begin with limited working memory and gradually 'mature' to the adult state. This result suggests that rather than being a limitation, developmental restrictions on resources may constitute a necessary prerequisite for mastering certain complex domains. Specifically, successful learning may depend on starting small.

Computational approaches to cognition: top-down approaches

Computational models are useful tools for exploring the nature of human cognitive processes. In particular, connectionist models are providing researchers with new ways of thinking about the basic nature of cognition and its implementation in the brain. They support novel explanations of important aspects of perception, memory, language, thought and cognitive development, and allow cognitive processes to be linked with the underlying physiological mechanisms. The models also aid our understanding of how disorders of brain function lead to disorders of cognition.