A study of pattern recovery in recurrent correlation associative memories

Complex-Valued Recurrent Correlation Neural Networks

In this paper, we generalize the bipolar recurrent correlation neural networks (RCNNs) of Chiueh and Goodman for patterns whose components are in the complex unit circle. The novel networks, referred to as complex-valued RCNNs (CV-RCNNs), are characterized by a possible nonlinear function, which is applied on the real part of the scalar product of the current state and the original patterns. We show that the CV-RCNNs always converge to a stationary state. Thus, they have potential application as associative memories. In this context, we provide sufficient conditions for the retrieval of a memorized vector. Furthermore, computational experiments concerning the reconstruction of corrupted grayscale images reveal that certain CV-RCNNs exhibit an excellent noise tolerance.

Recurrent correlation associative memories: a feature space perspective

In this paper, we analyze a model of recurrent kernel associative memory (RKAM) recently proposed by Garcia and Moreno. We show that this model consists in a kernelization of the recurrent correlation associative memory (RCAM) of Chiueh and Goodman. In particular, using an exponential kernel, we obtain a generalization of the well-known exponential correlation associative memory (ECAM), while using a polynomial kernel, we obtain a generalization of higher order Hopfield networks with Hebbian weights. We show that the RKAM can outperform the aforementioned associative memory models, becoming equivalent to them when a dominance condition is fulfilled by the kernel matrix. To ascertain the dominance condition, we propose a statistical measure which can be easily computed from the probability distribution of the interpattern Hamming distance or directly estimated from the memory vectors. The RKAM can be used below saturation to realize associative memories with reduced dynamic range with respect to the ECAM and with reduced number of synaptic coefficients with respect to higher order Hopfield networks.

A weighted voting model of associative memory

This paper presents an analysis of a random access memory (RAM)-based associative memory which uses a weighted voting scheme for information retrieval. This weighted voting memory can operate in heteroassociative or autoassociative mode, can store both real-valued and binary-valued patterns and, unlike memory models, is equipped with a rejection mechanism. A theoretical analysis of the performance of the weighted voting memory is given for the case of binary and random memory sets. Performance measures are derived as a function of the model parameters: pattern size, window size, and number of patterns in the memory set. It is shown that the weighted voting model has large capacity and error correction. The results show that the weighted voting model can successfully achieve high-detection and -identification rates and, simultaneously, low-false-acceptance rates.

The self-trapping attractor neural network--part II: properties of a sparsely connected model storing multiple memories

In a previous paper, the self-trapping network (STN) was introduced as more biologically realistic than attractor neural networks (ANNs) based on the Ising model. This paper extends the previous analysis of a one-dimensional (1-D) STN storing a single memory to a model that stores multiple memories and that possesses generalized sparse connnectivity. The energy, Lyapunov function, and partition function derived for the 1-D model are generalized to the case of an attractor network with only near-neighbor synapses, coupled to a system that computes memory overlaps. Simulations reveal that 1) the STN dramatically reduces intra-ANN connectivity without severly affecting the size of basins of attraction, with fast self-trapping able to sustain attractors even in the absence of intra-ANN synapses; 2) the basins of attraction can be controlled by a single free parameter, providing natural attention-like effects; 3) the same parameter determines the memory capacity of the network, and the latter is much less dependent than a standard ANN on the noise level of the system; 4) the STN serves as a useful memory for some correlated memory patterns for which the standard ANN totally fails; 5) the STN can store a large number of sparse patterns; and 6) a Monte Carlo procedure, a competitive neural network, and binary neurons with thresholds can be used to induce self-trapping.

Sensitivity to noise in bidirectional associative memory (BAM)

Original Hebbian encoding scheme of bidirectional associative memory (BAM) provides a poor pattern capacity and recall performance. Based on Rosenblatt's perceptron learning algorithm, the pattern capacity of BAM is enlarged, and perfect recall of all training pattern pairs is guaranteed. However, these methods put their emphases on pattern capacity, rather than error correction capability which is another critical point of BAM. This paper analyzes the sensitivity to noise in BAM and obtains an interesting idea to improve noise immunity of BAM. Some researchers have found that the noise sensitivity of BAM relates to the minimum absolute value of net inputs (MAV). However, in this paper, the analysis on failure association shows that it is related not only to MAV but also to the variance of weights associated with synapse connections. In fact, it is a positive monotone increasing function of the quotient of MAV divided by the variance of weights. This idea provides an useful principle of improving error correction capability of BAM. Some revised encoding schemes, such as small variance learning for BAM (SVBAM), evolutionary pseudorelaxation learning for BAM (EPRLAB) and evolutionary bidirectional learning (EBL), have been introduced to illustrate the performance of this principle. All these methods perform better than their original versions in noise immunity. Moreover, these methods have no negative effect on the pattern capacity of BAM. The convergence of these methods is also discussed in this paper. If there exist solutions, EPRLAB and EBL always converge to a global optimal solution in the senses of both pattern capacity and noise immunity. However, the convergence of SVBAM may be affected by a preset function.