Weighted learning of bidirectional associative memories by global minimization

A bidirectional heteroassociative memory for binary and grey-level patterns

Typical bidirectional associative memories (BAM) use an offline, one-shot learning rule, have poor memory storage capacity, are sensitive to noise, and are subject to spurious steady states during recall. Recent work on BAM has improved network performance in relation to noisy recall and the number of spurious attractors, but at the cost of an increase in BAM complexity. In all cases, the networks can only recall bipolar stimuli and, thus, are of limited use for grey-level pattern recall. In this paper, we introduce a new bidirectional heteroassociative memory model that uses a simple self-convergent iterative learning rule and a new nonlinear output function. As a result, the model can learn online without being subject to overlearning. Our simulation results show that this new model causes fewer spurious attractors when compared to others popular BAM networks, for a comparable performance in terms of tolerance to noise and storage capacity. In addition, the novel output function enables it to learn and recall grey-level patterns in a bidirectional way.

Adaptive associative memories capable of pattern segmentation

This paper presents an adaptive type of associative memory (AAM) that can separate patterns from composite inputs which might be degraded by deficiency or noise and that can recover incomplete or noisy single patterns. The behavior of AAM is analyzed in terms of stability, giving the stable solutions (results of recall), and the recall of spurious memories (the undesired solutions) is shown to be greatly reduced compared with earlier types of associative memory that can perform pattern segmentation. Two conditions that guarantee the nonexistence of undesired solutions are also given. Results of computer experiments show that the performance of AAM is much better than that of the earlier types of associative memory in terms of pattern segmentation and pattern recovery.

Capacity analysis of the asymptotically stable multi-valued exponential bidirectional associative memory

The exponential bidirectional associative memory (eBAM) has been proposed and proved to be a stable and high capacity associative neural network. However, the intrinsic structure and the evolution functions of this network restrict the representation of patterns to be either bipolar or binary vectors. We consider the promising development of multi-valued systems and then design a multi-valued discrete eBAM (MV-eBAM). The multi-valued eBAM has been proved to be asymptotically stable under certain constraints. Although MV-eBAM is also verified to possess high capacity by thorough simulations, there are important characteristics to be explored, including the absolute lower bound of the radix, and the approximate capacity. In order to estimate the capacity of the MV-eBAM, a modified evolution equation is also proposed. Hence, an analytic solution is derived. Besides, a radix searching algorithm is presented such that the absolute lower bound of the radix for this MV-eBAM can be found.

Neighbor-layer updating in MBDS for the recall of pure bipolar patterns in gray-scale noise

To improve bidirectional associative memory (BAM), a modified bidirectional decoding strategy (MBDS) network has been proposed. The former is a two-layer structure in which stored associations are recalled by directionally updating the neuron state through the connecting weights M and M(T). The latter is an extension of the former in which two hidden layers are augmented and the corresponding extra connection weights-M(x), M(y), Tx, and T(y)-are encoded. The authors introduce a new updating rule for MBDS networks, called neighbor-layer updating (NLU), which gathers all weighted activations of all neighbor layers. The neighbor layers are defined as the layers in which there are direct synaptic weights connected to each other. Because of modification of the connection weights-M(x), M(y), Tx, and T(y)-and the constant bias inputs of MBDS, all stored associations are guaranteed to be recalled using NLU. Furthermore, with the aid of the Cohen-Grossberg theorem, all discrete MBDS results can be extended to continuous MBDS (CMBDS). The authors also give stability proofs of both discrete MBDS and CMBDS, Computer simulations demonstrate that the proposed CMBDS can be applied to recall pure bipolar patterns in the presence of gray-scale noise. The authors show that by removing BAM connections (matrix M) from the MBDS structure, a bidirectional holographic memory (BHM) is obtained. Both derivation and simulation indicate that one can remove the matrix M from the MBDS structure if the dimension of the training associations is larger than 16.

A synthesis procedure for brain-state-in-a-box neural networks

In this paper, some new qualitative properties of discrete-time neural networks based on the "brain-state-in-a-box" model are presented. These properties concern both the characterization of equilibrium points and the global dynamical behavior. Next, the analysis results are used as guidelines in developing an efficient synthesis procedure for networks that function as associative memories. A constrained design algorithm is presented that gives completely stable dynamical neural networks sharing some interesting features. It is guaranteed the absence of nonbinary stable equilibria, that is stable states with nonsaturated components. It is guaranteed that in close proximity (Hamming distance one) of the stored patterns there is no other binary equilibrium point. Moreover, the presented method allows one to optimize a design parameter that controls the size of the attraction basins of the stored vectors and the accuracy needed in a digital realization of the network.