An entropic associative memory

Imagery in the entropic associative memory

The Entropic Associative Memory is a novel declarative and distributed computational model of associative memory. The model is general, conceptually simple, and offers an alternative to models developed within the artificial neural networks paradigm. The memory uses a standard table as its medium, where the information is stored in an indeterminate form, and the entropy plays a functional and operation role. The memory register operation abstracts the input cue with the current memory content and is productive; memory recognition is performed through a logical test; and memory retrieval is constructive. The three operations can be performed in parallel using very few computing resources. In our previous work we explored the auto-associative properties of the memory and performed experiments to store, recognize and retrieve manuscript digits and letters with complete and incomplete cues, and also to recognize and learn phones, with satisfactory results. In such experiments a designated memory register was used to store all the objects of the same class, whereas in the present study we remove such restriction and use a single memory register to store all the objects in the domain. In this novel setting we explore the production of emerging objects and relations, such that cues are used not only to retrieve remembered objects, but also related and imaged objects, and to produce association chains. The present model supports the view that memory and classification are independent functions both conceptually and architecturally. The memory system can store images of the different modalities of perception and action, possibly multimodal, and offers a novel perspective on the imagery debate and computational models of declarative memory.

Weighted entropic associative memory and phonetic learning

The Entropic Associative Memory (EAM) holds declarative but distributed representations of remembered objects. These are characterized as functions from features to discrete values in an abstract amodal space. Memory objects are registered or remembered through a declarative operation; memory recognition is defined as a logical test and cues of objects not contained in the memory are rejected directly without search; and memory retrieval is a constructive operation. In its original formulation, the content of basic memory units or cells was either on or off, hence all stored objects had the same weight or strength. In the present weighted version (W-EAM) we introduce a basic learning mechanism to the effect that the values of the cells used in the representation of an object are reinforced by the memory register operation. As memory cells are shared by different representations, the corresponding associations are reinforced too. The memory system supports a second form of learning: the distributed representation generalizes and renders a large set of potential or latent units that can used for recognizing novel inputs, which can in turn be used for improving the performance of both the deep neural networks used for modelling perception and action, and of the memory operations. This process can be performed recurrently in open-ended fashion and can be used in long term learning. An experiment in the phonetic domain using the Mexican Spanish DIMEx100 Corpus was carried out. This corpus was collected in a controlled noise-free environment, and was transcribed manually by human trained phoneticians, but consists of a relatively small number of utterances. DIMEx100 was used to produced the initial state of the perceptual and motor modules, and for testing the performance of the memory system at such state. Then the incremental learning cycle was modelled using the Spanish CIEMPIESS Corpus, consisting of a very large number of noisy untagged speech utterances collected from radio and TV. The results support the viability of the Weighted Entropic Associative Memory for modelling cognitive processes, such as phonetic representation and learning, for the construction of applications, such as speech recognition and synthesis, and as a computational model of natural memory.

Entropic associative memory for manuscript symbols

Manuscript symbols can be stored, recognized and retrieved from an entropic digital memory that is associative and distributed but yet declarative; memory retrieval is a constructive operation, memory cues to objects not contained in the memory are rejected directly without search, and memory operations can be performed through parallel computations. Manuscript symbols, both letters and numerals, are represented in Associative Memory Registers that have an associated entropy. The memory recognition operation obeys an entropy trade-off between precision and recall, and the entropy level impacts on the quality of the objects recovered through the memory retrieval operation. The present proposal is contrasted in several dimensions with neural networks models of associative memory. We discuss the operational characteristics of the entropic associative memory for retrieving objects with both complete and incomplete information, such as severe occlusions. The experiments reported in this paper add evidence on the potential of this framework for developing practical applications and computational models of natural memory.