Semantic graphs and associative memories

A memory access gate controlled by dynamic contexts

Temporary difficulties in accessing the contents of memories are a common experience in everyday life, for example, when we try to recognize a known person in an unusual context. In addition, recent experiments seem to indicate that retrograde amnesia in the early stages of Alzheimer's disease is due to disorders in accessing memories that were installed normally. These facts suggest the existence of an intermediate step between the stimulus arrival and the associative recognition. In this work, a multimodular neurocomputational model is presented postulating the existence of a neural gate that controls the access of the stimulus with its context to the consolidated memory. If recognition is not achieved, a random search is initiated in a contextual network aroused by the initial context. The search continues until the appropriate context that allows for recognition is found or until the process is turned off because the initial stimulus is no longer maintained in the working memory. The model is based on vector patterns of neural activity and context-dependent matrix memories. Simple Markov chain simulations are presented to exemplify possible search scenarios in the contextual network. Finally, we discuss some of the characteristics of the model and the phenomenon under study.

Multiplicative processing in the modeling of cognitive activities in large neural networks

Explaining the foundation of cognitive abilities in the processing of information by neural systems has been in the beginnings of biophysics since McCulloch and Pitts pioneered work within the biophysics school of Chicago in the 1940s and the interdisciplinary cybernetists meetings in the 1950s, inseparable from the birth of computing and artificial intelligence. Since then, neural network models have traveled a long path, both in the biophysical and the computational disciplines. The biological, neurocomputational aspect reached its representational maturity with the Distributed Associative Memory models developed in the early 70 s. In this framework, the inclusion of signal-signal multiplication within neural network models was presented as a necessity to provide matrix associative memories with adaptive, context-sensitive associations, while greatly enhancing their computational capabilities. In this review, we show that several of the most successful neural network models use a form of multiplication of signals. We present several classical models that included such kind of multiplication and the computational reasons for the inclusion. We then turn to the different proposals about the possible biophysical implementation that underlies these computational capacities. We pinpoint the important ideas put forth by different theoretical models using a tensor product representation and show that these models endow memories with the context-dependent adaptive capabilities necessary to allow for evolutionary adaptation to changing and unpredictable environments. Finally, we show how the powerful abilities of contemporary computationally deep-learning models, inspired in neural networks, also depend on multiplications, and discuss some perspectives in view of the wide panorama unfolded. The computational relevance of multiplications calls for the development of new avenues of research that uncover the mechanisms our nervous system uses to achieve multiplication.

Text Authorship Identified Using the Dynamics of Word Co-Occurrence Networks

Automatic identification of authorship in disputed documents has benefited from complex network theory as this approach does not require human expertise or detailed semantic knowledge. Networks modeling entire books can be used to discriminate texts from different sources and understand network growth mechanisms, but only a few studies have probed the suitability of networks in modeling small chunks of text to grasp stylistic features. In this study, we introduce a methodology based on the dynamics of word co-occurrence networks representing written texts to classify a corpus of 80 texts by 8 authors. The texts were divided into sections with equal number of linguistic tokens, from which time series were created for 12 topological metrics. Since 73% of all series were stationary (ARIMA(p, 0, q)) and the remaining were integrable of first order (ARIMA(p, 1, q)), probability distributions could be obtained for the global network metrics. The metrics exhibit bell-shaped non-Gaussian distributions, and therefore distribution moments were used as learning attributes. With an optimized supervised learning procedure based on a nonlinear transformation performed by Isomap, 71 out of 80 texts were correctly classified using the K-nearest neighbors algorithm, i.e. a remarkable 88.75% author matching success rate was achieved. Hence, purely dynamic fluctuations in network metrics can characterize authorship, thus paving the way for a robust description of large texts in terms of small evolving networks.

A Possible Neural Representation of Mathematical Group Structures

Every cognitive activity has a neural representation in the brain. When humans deal with abstract mathematical structures, for instance finite groups, certain patterns of activity are occurring in the brain that constitute their neural representation. A formal neurocognitive theory must account for all the activities developed by our brain and provide a possible neural representation for them. Associative memories are neural network models that have a good chance of achieving a universal representation of cognitive phenomena. In this work, we present a possible neural representation of mathematical group structures based on associative memory models that store finite groups through their Cayley graphs. A context-dependent associative memory stores the transitions between elements of the group when multiplied by each generator of a given presentation of the group. Under a convenient election of the vector basis mapping the elements of the group in the neural activity, the input of a vector corresponding to a generator of the group collapses the context-dependent rectangular matrix into a virtual square permutation matrix that is the matrix representation of the generator. This neural representation corresponds to the regular representation of the group, in which to each element is assigned a permutation matrix. This action of the generator on the memory matrix can also be seen as the dissection of the corresponding monochromatic subgraph of the Cayley graph of the group, and the adjacency matrix of this subgraph is the permutation matrix corresponding to the generator.

A modular approach to language production: models and facts

Numerous cortical disorders affect language. We explore the connection between the observed language behavior and the underlying substrates by adopting a neurocomputational approach. To represent the observed trajectories of the discourse in patients with disorganized speech and in healthy participants, we design a graphical representation for the discourse as a trajectory that allows us to visualize and measure the degree of order in the discourse as a function of the disorder of the trajectories. Our work assumes that many of the properties of language production and comprehension can be understood in terms of the dynamics of modular networks of neural associative memories. Based upon this assumption, we connect three theoretical and empirical domains: (1) neural models of language processing and production, (2) statistical methods used in the construction of functional brain images, and (3) corpus linguistic tools, such as Latent Semantic Analysis (henceforth LSA), that are used to discover the topic organization of language. We show how the neurocomputational models intertwine with LSA and the mathematical basis of functional neuroimaging. Within this framework we describe the properties of a context-dependent neural model, based on matrix associative memories, that performs goal-oriented linguistic behavior. We link these matrix associative memory models with the mathematics that underlie functional neuroimaging techniques and present the "functional brain images" emerging from the model. This provides us with a completely "transparent box" with which to analyze the implication of some statistical images. Finally, we use these models to explore the possibility that functional synaptic disconnection can lead to an increase in connectivity between the representations of concepts that could explain some of the alterations in discourse displayed by patients with schizophrenia.

Dynamic searching in the brain

Cognitive functions rely on the extensive use of information stored in the brain, and the searching for the relevant information for solving some problem is a very complex task. Human cognition largely uses biological search engines, and we assume that to study cognitive function we need to understand the way these brain search engines work. The approach we favor is to study multi-modular network models, able to solve particular problems that involve searching for information. The building blocks of these multimodular networks are the context dependent memory models we have been using for almost 20 years. These models work by associating an output to the Kronecker product of an input and a context. Input, context and output are vectors that represent cognitive variables. Our models constitute a natural extension of the traditional linear associator. We show that coding the information in vectors that are processed through association matrices, allows for a direct contact between these memory models and some procedures that are now classical in the Information Retrieval field. One essential feature of context-dependent models is that they are based on the thematic packing of information, whereby each context points to a particular set of related concepts. The thematic packing can be extended to multimodular networks involving input-output contexts, in order to accomplish more complex tasks. Contexts act as passwords that elicit the appropriate memory to deal with a query. We also show toy versions of several 'neuromimetic' devices that solve cognitive tasks as diverse as decision making or word sense disambiguation. The functioning of these multimodular networks can be described as dynamical systems at the level of cognitive variables.

Context-sensitive autoassociative memories as expert systems in medical diagnosis

Background

The complexity of our contemporary medical practice has impelled the development of different decision-support aids based on artificial intelligence and neural networks. Distributed associative memories are neural network models that fit perfectly well to the vision of cognition emerging from current neurosciences.

Methods

We present the context-dependent autoassociative memory model. The sets of diseases and symptoms are mapped onto a pair of basis of orthogonal vectors. A matrix memory stores the associations between the signs and symptoms, and their corresponding diseases. A minimal numerical example is presented to show how to instruct the memory and how the system works. In order to provide a quick appreciation of the validity of the model and its potential clinical relevance we implemented an application with real data. A memory was trained with published data of neonates with suspected late-onset sepsis in a neonatal intensive care unit (NICU). A set of personal clinical observations was used as a test set to evaluate the capacity of the model to discriminate between septic and non-septic neonates on the basis of clinical and laboratory findings.

Results

We show here that matrix memory models with associations modulated by context can perform automatic medical diagnosis. The sequential availability of new information over time makes the system progress in a narrowing process that reduces the range of diagnostic possibilities. At each step the system provides a probabilistic map of the different possible diagnoses to that moment. The system can incorporate the clinical experience, building in that way a representative database of historical data that captures geo-demographical differences between patient populations. The trained model succeeds in diagnosing late-onset sepsis within the test set of infants in the NICU: sensitivity 100%; specificity 80%; percentage of true positives 91%; percentage of true negatives 100%; accuracy (true positives plus true negatives over the totality of patients) 93,3%; and Cohen's kappa index 0,84.

Conclusion

Context-dependent associative memories can operate as medical expert systems. The model is presented in a simple and tutorial way to encourage straightforward implementations by medical groups. An application with real data, presented as a primary evaluation of the validity and potentiality of the model in medical diagnosis, shows that the model is a highly promising alternative in the development of accuracy diagnostic tools.

Schizophrenic speech as a disordered trajectory in a collapsed cognitive "Small-World"

New theoretical instruments, as goal-directed neural networks models and geometric representations based on semantic graphs, open new approaches for our understanding of the schizophrenic speech. The neuropathologic disorders of the schizophrenia can be simulated using neural models, and these models can eventually explain the origin of goal confusion and incoherence in the schizophrenic discourse trajectory. Moreover, these models are useful to evaluate the different hypothesis about the pathogenic mechanisms of the disease. At the same time, a geometric representation of the trajectory of the speech can be obtained from real data. Our conjecture is that a context-dependent graph can be constructed in order to explore if, when the disease became more severe, a transition from a quasi ordered graph to a nearly completely random graph occurs. Plausibly, there exists a wide region where the graph has the properties of a "small-world". This kind of analyses could be potentially carried out using data coming from the spontaneous speech of schizophrenic patients, and can help to evaluate the progress of the disease. At the same time, these geometrical representations could help to evaluate the effect of treatments.

Graph theoretic modeling of large-scale semantic networks

During the past several years, social network analysis methods have been used to model many complex real-world phenomena, including social networks, transportation networks, and the Internet. Graph theoretic methods, based on an elegant representation of entities and relationships, have been used in computational biology to study biological networks; however they have not yet been adopted widely by the greater informatics community. The graphs produced are generally large, sparse, and complex, and share common global topological properties. In this review of research (1998-2005) on large-scale semantic networks, we used a tailored search strategy to identify articles involving both a graph theoretic perspective and semantic information. Thirty-one relevant articles were retrieved. The majority (28, 90.3%) involved an investigation of a real-world network. These included corpora, thesauri, dictionaries, large computer programs, biological neuronal networks, word association networks, and files on the Internet. Twenty-two of the 28 (78.6%) involved a graph comprised of words or phrases. Fifteen of the 28 (53.6%) mentioned evidence of small-world characteristics in the network investigated. Eleven (39.3%) reported a scale-free topology, which tends to have a similar appearance when examined at varying scales. The results of this review indicate that networks generated from natural language have topological properties common to other natural phenomena. It has not yet been determined whether artificial human-curated terminology systems in biomedicine share these properties. Large network analysis methods have potential application in a variety of areas of informatics, such as in development of controlled vocabularies and for characterizing a given domain.