Results from the Gardner-Derrida-Mottishaw theory of associative memory

Retrieval-time properties of the Little-Hopfield model and their physiological relevance

We perform an extensive numerical investigation on the retrieval dynamics of the synchronous Hopfield model, also known as Little-Hopfield model, up to sizes of 2(18) neurons. Our results correct and extend much of the early simulations on the model. We find that the average convergence time has a power law behavior for a wide range of system sizes, whose exponent depends both on the network loading and the initial overlap with the memory to be retrieved. Surprisingly, we also find that the variance of the convergence time grows as fast as its average, making it a non-self-averaging quantity. Based on the simulation data we differentiate between two definitions for memory retrieval time, one that is mathematically strict, tau(c), the number of updates needed to reach the attractor whose properties we just described, and a second definition correspondent to the time tau(eta) when the network stabilizes within a tolerance threshold eta such that the difference of two consecutive overlaps with a stored memory is smaller that eta. We show that the scaling relationships between tau(c) and tau(eta) and the typical network parameters as the memory load alpha or the size of the network N vary greatly, being tau(eta) relatively insensitive to system sizes and loading. We propose tau(eta) as the physiological realistic measure for the typical attractor network response.