Dynamic searching in the brain

A memory access gate controlled by dynamic contexts

Temporary difficulties in accessing the contents of memories are a common experience in everyday life, for example, when we try to recognize a known person in an unusual context. In addition, recent experiments seem to indicate that retrograde amnesia in the early stages of Alzheimer's disease is due to disorders in accessing memories that were installed normally. These facts suggest the existence of an intermediate step between the stimulus arrival and the associative recognition. In this work, a multimodular neurocomputational model is presented postulating the existence of a neural gate that controls the access of the stimulus with its context to the consolidated memory. If recognition is not achieved, a random search is initiated in a contextual network aroused by the initial context. The search continues until the appropriate context that allows for recognition is found or until the process is turned off because the initial stimulus is no longer maintained in the working memory. The model is based on vector patterns of neural activity and context-dependent matrix memories. Simple Markov chain simulations are presented to exemplify possible search scenarios in the contextual network. Finally, we discuss some of the characteristics of the model and the phenomenon under study.

Multiplicative processing in the modeling of cognitive activities in large neural networks

Explaining the foundation of cognitive abilities in the processing of information by neural systems has been in the beginnings of biophysics since McCulloch and Pitts pioneered work within the biophysics school of Chicago in the 1940s and the interdisciplinary cybernetists meetings in the 1950s, inseparable from the birth of computing and artificial intelligence. Since then, neural network models have traveled a long path, both in the biophysical and the computational disciplines. The biological, neurocomputational aspect reached its representational maturity with the Distributed Associative Memory models developed in the early 70 s. In this framework, the inclusion of signal-signal multiplication within neural network models was presented as a necessity to provide matrix associative memories with adaptive, context-sensitive associations, while greatly enhancing their computational capabilities. In this review, we show that several of the most successful neural network models use a form of multiplication of signals. We present several classical models that included such kind of multiplication and the computational reasons for the inclusion. We then turn to the different proposals about the possible biophysical implementation that underlies these computational capacities. We pinpoint the important ideas put forth by different theoretical models using a tensor product representation and show that these models endow memories with the context-dependent adaptive capabilities necessary to allow for evolutionary adaptation to changing and unpredictable environments. Finally, we show how the powerful abilities of contemporary computationally deep-learning models, inspired in neural networks, also depend on multiplications, and discuss some perspectives in view of the wide panorama unfolded. The computational relevance of multiplications calls for the development of new avenues of research that uncover the mechanisms our nervous system uses to achieve multiplication.

A neurocomputational model for the processing of conflicting information in context-dependent decision tasks

Context-dependent computation is a relevant characteristic of neural systems, endowing them with the capacity of adaptively modifying behavioral responses and flexibly discriminating between relevant and irrelevant information in a stimulus. This ability is particularly highlighted in solving conflicting tasks. A long-standing problem in computational neuroscience, flexible routing of information, is also closely linked with the ability to perform context-dependent associations. Here we present an extension of a context-dependent associative memory model to achieve context-dependent decision-making in the presence of conflicting and noisy multi-attribute stimuli. In these models, the input vectors are multiplied by context vectors via the Kronecker tensor product. To outfit the model with a noisy dynamic, we embedded the context-dependent associative memory in a leaky competing accumulator model, and, finally, we proved the power of the model in the reproduction of a behavioral experiment with monkeys in a context-dependent conflicting decision-making task. At the end, we discuss the neural feasibility of the tensor product and made the suggestive observation that the capacities of tensor context models are surprisingly in alignment with the more recent experimental findings about functional flexibility at different levels of brain organization.

The biological Maxwell's demons: exploring ideas about the information processing in biological systems

This work is based on ideas supported by some of the biologists who discovered foundational facts of twentieth-century biology and who argued that Maxwell's demons are physically implemented by biological devices. In particular, JBS Haldane first, and later J. Monod, A, Lwoff and F. Jacob argued that enzymes and molecular receptors implemented Maxwell's demons that operate in systems far removed from thermodynamic equilibrium and that were responsible for creating the biological order. Later, these ideas were extended to other biological processes. In this article, we argue that these biological Maxwell's demons (BMD) are systems that have information processing capabilities that allow them to select their inputs and direct their outputs toward targets. In this context, we propose the idea that these BMD are information catalysts in which the processed information has broad thermodynamic consequences.

Tensor Representation of Topographically Organized Semantic Spaces

Human brains seem to represent categories of objects and actions as locations in a continuous semantic space across the cortical surface that reflects the similarity among categories. This vision of the semantic organization of information in the brain, suggested by recent experimental findings, is in harmony with the well-known topographically organized somatotopic, retinotopic, and tonotopic maps in the cerebral cortex. Here we show that these topographies can be operationally represented with context-dependent associative memories. In these models, the input vectors and, eventually also, the associated output vectors are multiplied by context vectors via the Kronecker tensor product, which allows a spatial organization of memories. Input and output tensor contexts localize matrices of semantic categories into a neural layer or slice and, at the same time, direct the flow of information arriving at the layer to a specific address, and then forward the output information toward the corresponding targets. Given a neural topographic pattern, the tensor representation will place a set of associative matrix memories within a topographic regionalized host matrix in such way that they reproduce the empirical pattern of patches in the actual neural layer. Progressive approximations to this goal are accomplished by avoiding excessive overlap of memories or the existence of empty regions within the host matrix.

Semantic integration by pattern priming: experiment and cortical network model

Neural network models describe semantic priming effects by way of mechanisms of activation of neurons coding for words that rely strongly on synaptic efficacies between pairs of neurons. Biologically inspired Hebbian learning defines efficacy values as a function of the activity of pre- and post-synaptic neurons only. It generates only pair associations between words in the semantic network. However, the statistical analysis of large text databases points to the frequent occurrence not only of pairs of words (e.g., "the way") but also of patterns of more than two words (e.g., "by the way"). The learning of these frequent patterns of words is not reducible to associations between pairs of words but must take into account the higher level of coding of three-word patterns. The processing and learning of pattern of words challenges classical Hebbian learning algorithms used in biologically inspired models of priming. The aim of the present study was to test the effects of patterns on the semantic processing of words and to investigate how an inter-synaptic learning algorithm succeeds at reproducing the experimental data. The experiment manipulates the frequency of occurrence of patterns of three words in a multiple-paradigm protocol. Results show for the first time that target words benefit more priming when embedded in a pattern with the two primes than when only associated with each prime in pairs. A biologically inspired inter-synaptic learning algorithm is tested that potentiates synapses as a function of the activation of more than two pre- and post-synaptic neurons. Simulations show that the network can learn patterns of three words to reproduce the experimental results.

A simplified computational memory model from information processing

This paper is intended to propose a computational model for memory from the view of information processing. The model, called simplified memory information retrieval network (SMIRN), is a bi-modular hierarchical functional memory network by abstracting memory function and simulating memory information processing. At first meta-memory is defined to express the neuron or brain cortices based on the biology and graph theories, and we develop an intra-modular network with the modeling algorithm by mapping the node and edge, and then the bi-modular network is delineated with intra-modular and inter-modular. At last a polynomial retrieval algorithm is introduced. In this paper we simulate the memory phenomena and functions of memorization and strengthening by information processing algorithms. The theoretical analysis and the simulation results show that the model is in accordance with the memory phenomena from information processing view.

A Possible Neural Representation of Mathematical Group Structures

Every cognitive activity has a neural representation in the brain. When humans deal with abstract mathematical structures, for instance finite groups, certain patterns of activity are occurring in the brain that constitute their neural representation. A formal neurocognitive theory must account for all the activities developed by our brain and provide a possible neural representation for them. Associative memories are neural network models that have a good chance of achieving a universal representation of cognitive phenomena. In this work, we present a possible neural representation of mathematical group structures based on associative memory models that store finite groups through their Cayley graphs. A context-dependent associative memory stores the transitions between elements of the group when multiplied by each generator of a given presentation of the group. Under a convenient election of the vector basis mapping the elements of the group in the neural activity, the input of a vector corresponding to a generator of the group collapses the context-dependent rectangular matrix into a virtual square permutation matrix that is the matrix representation of the generator. This neural representation corresponds to the regular representation of the group, in which to each element is assigned a permutation matrix. This action of the generator on the memory matrix can also be seen as the dissection of the corresponding monochromatic subgraph of the Cayley graph of the group, and the adjacency matrix of this subgraph is the permutation matrix corresponding to the generator.

On the time course of short-term forgetting: a human experimental model for the sense of balance

The primary aim of this study was to establish whether the decline of the memory of an angular displacement, detected by the semicircular canals, is best characterized by an exponential function or by a power function. In 27 subjects a conflict was created between the semicircular canals and the graviceptive systems. Subjects were seated, facing forwards, in the gondola of a large centrifuge. The centrifuge was accelerated from stationary to 2.5Gz. While the swing out of the gondola (66°) during acceleration constitutes a frontal plane angular-displacement stimulus to the semicircular canals, the graviceptive systems persistently signal that the subject is upright. During 6 min at 2.5Gz the perceived head and body position was recorded; in darkness the subject repeatedly adjusted the orientation of a luminous line so that it appeared to be horizontal. Acceleration of the centrifuge induced a sensation of tilt which declined with time in a characteristic way. A three-parameter exponential function (Y = Ae(-bt) + C) and a power function (Y = At(-b) + C) were fitted to the data points. The inter-individual variability was considerable. In the vast majority of cases, however, the exponential function provided a better fit (in terms of RMS error) than the power function. The mean exponential function was: y = 27.8e(-0.018t) + 0.5°, where t is time in seconds. Findings are discussed with connection to possible underlying neural mechanisms; in particular, the head-direction system and short-term potentiation and persistent action potential firing in the hippocampus are considered.

Modeling spatial-temporal operations with context-dependent associative memories

We organize our behavior and store structured information with many procedures that require the coding of spatial and temporal order in specific neural modules. In the simplest cases, spatial and temporal relations are condensed in prepositions like "below" and "above", "behind" and "in front of", or "before" and "after", etc. Neural operators lie beneath these words, sharing some similarities with logical gates that compute spatial and temporal asymmetric relations. We show how these operators can be modeled by means of neural matrix memories acting on Kronecker tensor products of vectors. The complexity of these memories is further enhanced by their ability to store episodes unfolding in space and time. How does the brain scale up from the raw plasticity of contingent episodic memories to the apparent stable connectivity of large neural networks? We clarify this transition by analyzing a model that flexibly codes episodic spatial and temporal structures into contextual markers capable of linking different memory modules.

A modular approach to language production: models and facts

Numerous cortical disorders affect language. We explore the connection between the observed language behavior and the underlying substrates by adopting a neurocomputational approach. To represent the observed trajectories of the discourse in patients with disorganized speech and in healthy participants, we design a graphical representation for the discourse as a trajectory that allows us to visualize and measure the degree of order in the discourse as a function of the disorder of the trajectories. Our work assumes that many of the properties of language production and comprehension can be understood in terms of the dynamics of modular networks of neural associative memories. Based upon this assumption, we connect three theoretical and empirical domains: (1) neural models of language processing and production, (2) statistical methods used in the construction of functional brain images, and (3) corpus linguistic tools, such as Latent Semantic Analysis (henceforth LSA), that are used to discover the topic organization of language. We show how the neurocomputational models intertwine with LSA and the mathematical basis of functional neuroimaging. Within this framework we describe the properties of a context-dependent neural model, based on matrix associative memories, that performs goal-oriented linguistic behavior. We link these matrix associative memory models with the mathematics that underlie functional neuroimaging techniques and present the "functional brain images" emerging from the model. This provides us with a completely "transparent box" with which to analyze the implication of some statistical images. Finally, we use these models to explore the possibility that functional synaptic disconnection can lead to an increase in connectivity between the representations of concepts that could explain some of the alterations in discourse displayed by patients with schizophrenia.

A hybrid model for the neural representation of complex mental processing in the human brain

In the present conceptual review several theoretical and empirical sources of information were integrated, and a hybrid model of the neural representation of complex mental processing in the human brain was proposed. Based on empirical evidence for strategy-related and inter-individually different task-related brain activation networks, and further based on empirical evidence for a remarkable overlap of fronto-parietal activation networks across different complex mental processes, it was concluded by the author that there might be innate and modular organized neuro-developmental starting regions, for example, in intra-parietal, and both medial and middle frontal brain regions, from which the neural organization of different kinds of complex mental processes emerge differently during individually shaped learning histories. Thus, the here proposed model provides a hybrid of both massive modular and holistic concepts of idiosyncratic brain physiological elaboration of complex mental processing. It is further concluded that 3-D information, obtained by respective methodological approaches, are not appropriate to identify the non-linear spatio-temporal dynamics of complex mental process-related brain activity in a sufficient way. How different participating network parts communicate with each other seems to be an indispensable aspect, which has to be considered in particular to improve our understanding of the neural organization of complex cognition.

Detecting order-disorder transitions in discourse: implications for schizophrenia

Several psychiatric and neurological conditions affect the semantic organization and content of a patient's speech. Specifically, the discourse of patients with schizophrenia is frequently characterized as lacking coherence. The evaluation of disturbances in discourse is often used in diagnosis and in assessing treatment efficacy, and is an important factor in prognosis. Measuring these deviations, such as "loss of meaning" and incoherence, is difficult and requires substantial human effort. Computational procedures can be employed to characterize the nature of the anomalies in discourse. We present a set of new tools derived from network theory and information science that may assist in empirical and clinical studies of communication patterns in patients, and provide the foundation for future automatic procedures. First we review information science and complex network approaches to measuring semantic coherence, and then we introduce a representation of discourse that allows for the computation of measures of disorganization. Finally we apply these tools to speech transcriptions from patients and a healthy participant, illustrating the implications and potential of this novel framework.

Memory reconsolidation for natural language processing

We propose a model of memory reconsolidation that can output new sentences with additional meaning after refining information from input sentences and integrating them with related prior experience. Our model uses available technology to first disambiguate the meanings of words and extracts information from the sentences into a structure that is an extension to semantic networks. Within our long-term memory we introduce an action relationships database reminiscent of the way symbols are associated in brain, and propose an adaptive mechanism for linking these actions with the different scenarios. The model then fills in the implicit context of the input and predicts relevant activities that could occur in the context based on a statistical action relationship database. The new data both of the more complete scenario and of the statistical relationships of the activities are reconsolidated into memory. Experiments show that our model improves upon the existing reasoning tool suggested by MIT Media lab, known as ConceptNet.