Mutual information of sparsely coded associative memory with self-control and ternary neurons

ADAPTIVE THRESHOLDS FOR NEURAL NETWORKS WITH SYNAPTIC NOISE

The inclusion of a macroscopic adaptive threshold is studied for the retrieval dynamics of both layered feedforward and fully connected neural network models with synaptic noise. These two types of architectures require a different method to be solved numerically. In both cases it is shown that, if the threshold is chosen appropriately as a function of the cross-talk noise and of the activity of the stored patterns, adapting itself automatically in the course of the recall process, an autonomous functioning of the network is guaranteed. This self-control mechanism considerably improves the quality of retrieval, in particular the storage capacity, the basins of attraction and the mutual information content.

Information and topology in attractor neural networks

A wide range of networks, including those with small-world topology, can be modeled by the connectivity ratio and randomness of the links. Both learning and attractor abilities of a neural network can be measured by the mutual information (MI) as a function of the load and the overlap between patterns and retrieval states. In this letter, we use MI to search for the optimal topology with regard to the storage and attractor properties of the network in an Amari-Hopfield model. We find that while an optimal storage implies an extremely diluted topology, a large basin of attraction leads to moderate levels of connectivity. This optimal topology is related to the clustering and path length of the network. We also build a diagram for the dynamical phases with random or local initial overlap and show that very diluted networks lose their attractor ability.

Time evolution of the extremely diluted Blume-Emery-Griffiths neural network

A study of the time evolution and a stability analysis of the phases in the extremely diluted Blume-Emery-Griffiths neural network model are shown to yield new phase diagrams in which fluctuation retrieval may drive pattern retrieval. It is shown that saddle-point solutions associated with fluctuation overlaps slow down the flow of the network states towards the retrieval fixed points. A comparison of the performance with other three-state networks is also presented.