Generalization and chaos in a layered neural network

Instabilities in attractor networks with fast synaptic fluctuations and partial updating of the neurons activity

We present and study a probabilistic neural automaton in which the fraction of simultaneously-updated neurons is a parameter, rhoin(0,1). For small rho, there is relaxation towards one of the attractors and a great sensibility to external stimuli and, for rho > or = rho(c), itinerancy among attractors. Tuning rho in this regime, oscillations may abruptly change from regular to chaotic and vice versa, which allows one to control the efficiency of the searching process. We argue on the similarity of the model behavior with recent observations, and on the possible role of chaos in neurobiology.

Chaotic hopping between attractors in neural networks

We present a neurobiologically-inspired stochastic cellular automaton whose state jumps with time between the attractors corresponding to a series of stored patterns. The jumping varies from regular to chaotic as the model parameters are modified. The resulting irregular behavior, which mimics the state of attention in which a system shows a great adaptability to changing stimulus, is a consequence in the model of short-time presynaptic noise which induces synaptic depression. We discuss results from both a mean-field analysis and Monte Carlo simulations.

Categorization in a Hopfield network trained with weighted examples: extensive number of concepts

We consider the categorization problem in a Hopfield network with an extensive number of concepts p = alpha N and trained with s examples of weight lambda tau, tau = 1,...,s in the presence of synaptic noise represented by a dimensionless "temperature" T. We find that the retrieval capacity of an example with weight lambda 1, and the corresponding categorization error, depend also on the arithmetic mean lambda m of the other weights. The categorization process is similar to that in a network trained with Hebb's rule, but for lambda 1/lambda m > 1 the retrieval phase is enhanced. We present the phase diagram in the T-alpha plane, together with the de Almeida-Thouless line of instability. The phase diagrams in the alpha-s plane are discussed in the absence of synaptic noise and several values of the correlation parameter b.

Categorization in fully connected multistate neural network models

The categorization ability of fully connected neural network models, with either discrete or continuous Q-state units, is studied in this work in replica symmetric mean-field theory. Hierarchically correlated multistate patterns in a two level structure of ancestors and descendents (examples) are embedded in the network and the categorization task consists in recognizing the ancestors when the network is trained exclusively with their descendents. Explicit results for the dependence of the equilibrium properties of a Q=3-state model and a Q=infinity-state model are obtained in the form of phase diagrams and categorization curves. A strong improvement of the categorization ability is found when the network is trained with examples of low activity. The categorization ability is found to be robust to finite threshold and synaptic noise. The Almeida-Thouless lines that limit the validity of the replica-symmetric results, are also obtained.

Categorization in the symmetrically dilute Hopfield network

A symmetrically dilute Hopfield model with a Hebbian learning rule is used to study the effects of gradual dilution and of synaptic noise on the categorization ability of an attractor neural network with hierarchically correlated patterns in a two-level structure of ancestors and descendants. Categorization consists in recognizing the ancestors when the network has been trained exclusively with the descendants. We consider a macroscopic number of ancestors, each with a finite number of descendants, and take into account the stochastic noise produced by the former in an equilibrium study of the network, by means of replica-symmetric mean-field theory. Phase diagrams are obtained that exhibit a categorization, a spin-glass, and a paramagnetic phase, as well as the dependence of the order parameters on the relevant quantities. The de Almeida-Thouless lines that limit the validity of the replica-symmetric results are also obtained. It is shown that gradual dilution increases considerably the region where a stable categorization phase may be found.