Recognition of Abstract Objects Via Neural Oscillators: Interaction Among Topological Organization, Associative Memory and Gamma Band Synchronization

A model of working memory for encoding multiple items and ordered sequences exploiting the theta-gamma code

Recent experimental evidence suggests that oscillatory activity plays a pivotal role in the maintenance of information in working memory, both in rodents and humans. In particular, cross-frequency coupling between theta and gamma oscillations has been suggested as a core mechanism for multi-item memory. The aim of this work is to present an original neural network model, based on oscillating neural masses, to investigate mechanisms at the basis of working memory in different conditions. We show that this model, with different synapse values, can be used to address different problems, such as the reconstruction of an item from partial information, the maintenance of multiple items simultaneously in memory, without any sequential order, and the reconstruction of an ordered sequence starting from an initial cue. The model consists of four interconnected layers; synapses are trained using Hebbian and anti-Hebbian mechanisms, in order to synchronize features in the same items, and desynchronize features in different items. Simulations show that the trained network is able to desynchronize up to nine items without a fixed order using the gamma rhythm. Moreover, the network can replicate a sequence of items using a gamma rhythm nested inside a theta rhythm. The reduction in some parameters, mainly concerning the strength of GABAergic synapses, induce memory alterations which mimic neurological deficits. Finally, the network, isolated from the external environment (“imagination phase”) and stimulated with high uniform noise, can randomly recover sequences previously learned, and link them together by exploiting the similarity among items.

EEG-based image classification via a region-level stacked bi-directional deep learning framework

BACKGROUND

As a physiological signal, EEG data cannot be subjectively changed or hidden. Compared with other physiological signals, EEG signals are directly related to human cortical activities with excellent temporal resolution. After the rapid development of machine learning and artificial intelligence, the analysis and calculation of EEGs has made great progress, leading to a significant boost in performances for content understanding and pattern recognition of brain activities across the areas of both neural science and computer vision. While such an enormous advance has attracted wide range of interests among relevant research communities, EEG-based classification of brain activities evoked by images still demands efforts for further improvement with respect to its accuracy, generalization, and interpretation, yet some characters of human brains have been relatively unexplored.

METHODS

We propose a region-level stacked bi-directional deep learning framework for EEG-based image classification. Inspired by the hemispheric lateralization of human brains, we propose to extract additional information at regional level to strengthen and emphasize the differences between two hemispheres. The stacked bi-directional long short-term memories are used to capture the dynamic correlations hidden from both the past and the future to the current state in EEG sequences.

RESULTS

Extensive experiments are carried out and our results demonstrate the effectiveness of our proposed framework. Compared with the existing state-of-the-arts, our framework achieves outstanding performances in EEG-based classification of brain activities evoked by images. In addition, we find that the signals of Gamma band are not only useful for achieving good performances for EEG-based image classification, but also play a significant role in capturing relationships between the neural activations and the specific emotional states.

CONCLUSIONS

Our proposed framework provides an improved solution for the problem that, given an image used to stimulate brain activities, we should be able to identify which class the stimuli image comes from by analyzing the EEG signals. The region-level information is extracted to preserve and emphasize the hemispheric lateralization for neural functions or cognitive processes of human brains. Further, stacked bi-directional LSTMs are used to capture the dynamic correlations hidden in EEG data. Extensive experiments on standard EEG-based image classification dataset validate that our framework outperforms the existing state-of-the-arts under various contexts and experimental setups.

A biologically inspired neurocomputational model for audiovisual integration and causal inference

Recently, experimental and theoretical research has focused on the brain's abilities to extract information from a noisy sensory environment and how cross-modal inputs are processed to solve the causal inference problem to provide the best estimate of external events. Despite the empirical evidence suggesting that the nervous system uses a statistically optimal and probabilistic approach in addressing these problems, little is known about the brain's architecture needed to implement these computations. The aim of this work was to realize a mathematical model, based on physiologically plausible hypotheses, to analyze the neural mechanisms underlying multisensory perception and causal inference. The model consists of three layers topologically organized: two encode auditory and visual stimuli, separately, and are reciprocally connected via excitatory synapses and send excitatory connections to the third downstream layer. This synaptic organization realizes two mechanisms of cross-modal interactions: the first is responsible for the sensory representation of the external stimuli, while the second solves the causal inference problem. We tested the network by comparing its results to behavioral data reported in the literature. Among others, the network can account for the ventriloquism illusion, the pattern of sensory bias and the percept of unity as a function of the spatial auditory-visual distance, and the dependence of the auditory error on the causal inference. Finally, simulations results are consistent with probability matching as the perceptual strategy used in auditory-visual spatial localization tasks, agreeing with the behavioral data. The model makes untested predictions that can be investigated in future behavioral experiments.

Syntax meets semantics during brain logical computations

The discrepancy between syntax and semantics is a painstaking issue that hinders a better comprehension of the underlying neuronal processes in the human brain. In order to tackle the issue, we at first describe a striking correlation between Wittgenstein's Tractatus, that assesses the syntactic relationships between language and world, and Perlovsky's joint language-cognitive computational model, that assesses the semantic relationships between emotions and "knowledge instinct". Once established a correlation between a purely logical approach to the language and computable psychological activities, we aim to find the neural correlates of syntax and semantics in the human brain. Starting from topological arguments, we suggest that the semantic properties of a proposition are processed in higher brain's functional dimensions than the syntactic ones. In a fully reversible process, the syntactic elements embedded in Broca's area project into multiple scattered semantic cortical zones. The presence of higher functional dimensions gives rise to the increase in informational content that takes place in semantic expressions. Therefore, diverse features of human language and cognitive world can be assessed in terms of both the logic armor described by the Tractatus, and the neurocomputational techniques at hand. One of our motivations is to build a neuro-computational framework able to provide a feasible explanation for brain's semantic processing, in preparation for novel computers with nodes built into higher dimensions.

Intellectual disabilities and power spectra analysis during sleep: a new perspective on borderline intellectual functioning

BACKGROUND

The role of sleep in cognitive processes has been confirmed by a growing number of reports for all ages of life. Analysing sleep electroencephalogram (EEG) spectra may be useful to study cortical organisation in individuals with Borderline Intellectual Functioning (BIF), as seen in other disturbances even if it is not considered a disease. The aim of this study was to determine if the sleep EEG power spectra in children with BIF could be different from typically developing children.

METHODS

Eighteen BIF (12 males) (mean age 11.04; SD ± 1.07) and 24 typical developing children (14 men) (mean age 10.98; SD ± 1.76; P = 0.899) underwent an overnight polysomnography (PSG) recording in the Sleep Laboratory of the Clinic of Child and Adolescent Neuropsychiatry, after one adaptation night. Sleep was subdivided into 30-s epochs and sleep stages were scored according to the standard criteria and the power spectra were calculated for the Cz-A2 channel using the sleep analysis software Hypnolab 1.2 (SWS Soft, Italy) by means of the Fast Fourier Transform and the power spectrum was calculated for frequencies between 0.5 and 60 Hz with a frequency step of 1 Hz and then averaged across the following bands delta (0.5-4 Hz), theta (5-7 Hz), alpha (8-11 Hz), sigma (11-15 Hz), and beta (16-30 Hz), gamma (30-60 Hz) for S2, SWS and REM (Rapid Eye Movement) sleep stages.

RESULTS

BIF have a reduced sleep duration (total sleep time; P < 0.001), and an increased rate of stage shifts (P < 0.001) and awakenings (P < 0.001) and WASO (wakefulness after sleep onset) % (P < 0.001); the stage 2% (P < 0.001), and REM% (P < 0.001) were lower and slow-wave sleep percentage was slightly higher (P < 0.001). All children with BIF had an AHI (apnoea-hypopnea index) less than 1 (mean AHI = 0.691 ± 0.236) with a mean oxygen saturation of 97.6% and a periodic leg movement index (PLMI) less than 5 (mean PLMI = 2.94 ± 1.56). All sleep stages had a significant reduction in gamma frequency (30-60 Hz) (P < 0.001) and an increased delta frequency (0.5-4.0 Hz) (P < 0.001) power in BIF subjects compared with typically developing children.

CONCLUSION

Our findings shed light on the importance of sleep for cognition processes particularly in cognitive borderline dysfunction and the role of EEG spectral power analysis to recognize sleep characteristics in BIF children.

An ordered-patch-based image classification approach on the image Grassmannian manifold

This paper presents an ordered-patch-based image classification framework integrating the image Grassmannian manifold to address handwritten digit recognition, face recognition, and scene recognition problems. Typical image classification methods explore image appearances without considering the spatial causality among distinctive domains in an image. To address the issue, we introduce an ordered-patch-based image representation and use the autoregressive moving average (ARMA) model to characterize the representation. First, each image is encoded as a sequence of ordered patches, integrating both the local appearance information and spatial relationships of the image. Second, the sequence of these ordered patches is described by an ARMA model, which can be further identified as a point on the image Grassmannian manifold. Then, image classification can be conducted on such a manifold under this manifold representation. Furthermore, an appropriate Grassmannian kernel for support vector machine classification is developed based on a distance metric of the image Grassmannian manifold. Finally, the experiments are conducted on several image data sets to demonstrate that the proposed algorithm outperforms other existing image classification methods.

Perceptual grouping by entrainment in coupled Kuramoto oscillator networks

In this article we present a network composed of coupled Kuramoto oscillators, which is able to solve a broad spectrum of perceptual grouping tasks. Based on attracting and repelling interactions between these oscillators, the network dynamics forms various phase-synchronized clusters of oscillators corresponding to individual groups of similar input features. The degree of similarity between features is determined by a set of underlying receptive fields, which are learned directly from the feature domain. After illustrating the theoretical principles of the network, the approach is evaluated in an image segmentation task. Furthermore, the influence of a varying degree of sparse couplings is evaluated.

Binding and segmentation via a neural mass model trained with Hebbian and anti-Hebbian mechanisms

Synchronization of neural activity in the gamma band, modulated by a slower theta rhythm, is assumed to play a significant role in binding and segmentation of multiple objects. In the present work, a recent neural mass model of a single cortical column is used to analyze the synaptic mechanisms which can warrant synchronization and desynchronization of cortical columns, during an autoassociation memory task. The model considers two distinct layers communicating via feedforward connections. The first layer receives the external input and works as an autoassociative network in the theta band, to recover a previously memorized object from incomplete information. The second realizes segmentation of different objects in the gamma band. To this end, units within both layers are connected with synapses trained on the basis of previous experience to store objects. The main model assumptions are: (i) recovery of incomplete objects is realized by excitatory synapses from pyramidal to pyramidal neurons in the same object; (ii) binding in the gamma range is realized by excitatory synapses from pyramidal neurons to fast inhibitory interneurons in the same object. These synapses (both at points i and ii) have a few ms dynamics and are trained with a Hebbian mechanism. (iii) Segmentation is realized with faster AMPA synapses, with rise times smaller than 1 ms, trained with an anti-Hebbian mechanism. Results show that the model, with the previous assumptions, can correctly reconstruct and segment three simultaneous objects, starting from incomplete knowledge. Segmentation of more objects is possible but requires an increased ratio between the theta and gamma periods.

The formation of categories and the representation of feature saliency: analysis with a computational model trained with an Hebbian paradigm

An important issue in semantic memory models is the formation of categories and taxonomies, and the different role played by shared vs. distinctive and salient vs. marginal features. Aim of this work is to extend our previous model to critically discuss the mechanisms leading to the formation of categories, and to investigate how feature saliency can be learned from past experience. The model assumes that an object is represented as a collection of features, which belong to different cortical areas and are topologically organized. Excitatory synapses among features are created on the basis of past experience of object presentation, with a Hebbian paradigm, including the use of potentiation and depression of synapses, and thresholding for the presynaptic and postsynaptic. The model was trained using simple schematic objects as input (i.e., vector of features) having some shared features (so as to realize a simple category) and some distinctive features with different frequency. Three different taxonomies of objects were separately trained and tested, which differ as to the number of correlated features and the structure of categories. Results show that categories can be formed from past experience, using Hebbian rules with a different threshold for postsynaptic and presynaptic activity. Furthermore, features have a different saliency, as a consequence of their different frequency during training. The trained network is able to solve simple object recognition tasks, by maintaining a distinction between categories and individual members in the category, and providing a different role for salient features vs. not-salient features. In particular, not-salient features are not evoked in memory when thinking about the object, but they facilitate the reconstruction of objects when provided as input to the model. The results can provide indications on which neural mechanisms can be exploited to form robust categories among objects and on which mechanisms could be implemented in artificial connectionist systems to extract concepts and categories from a continuous stream of input objects (each represented as a vector of features).

A MULTI-LAYER NEURAL-MASS MODEL FOR LEARNING SEQUENCES USING THETA/GAMMA OSCILLATIONS

A neural mass model for the memorization of sequences is presented. It exploits three layers of cortical columns that generate a theta/gamma rhythm. The first layer implements an auto-associative memory working in the theta range; the second segments objects in the gamma range; finally, the feedback interactions between the third and the second layers realize a hetero-associative memory for learning a sequence. After training with Hebbian and anti-Hebbian rules, the network recovers sequences and accounts for the phase-precession phenomenon.

H∞ state estimation for discrete-time complex networks with randomly occurring sensor saturations and randomly varying sensor delays

In this paper, the state estimation problem is investigated for a class of discrete time-delay nonlinear complex networks with randomly occurring phenomena from sensor measurements. The randomly occurring phenomena include randomly occurring sensor saturations (ROSSs) and randomly varying sensor delays (RVSDs) that result typically from networked environments. A novel sensor model is proposed to describe the ROSSs and the RVSDs within a unified framework via two sets of Bernoulli-distributed white sequences with known conditional probabilities. Rather than employing the commonly used Lipschitz-type function, a more general sector-like nonlinear function is used to describe the nonlinearities existing in the network. The purpose of the addressed problem is to design a state estimator to estimate the network states through available output measurements such that, for all probabilistic sensor saturations and sensor delays, the dynamics of the estimation error is guaranteed to be exponentially mean-square stable and the effect from the exogenous disturbances to the estimation accuracy is attenuated at a given level by means of an H∞-norm. In terms of a novel Lyapunov-Krasovskii functional and the Kronecker product, sufficient conditions are established under which the addressed state estimation problem is recast as solving a convex optimization problem via the semidefinite programming method. A simulation example is provided to show the usefulness of the proposed state estimation conditions.

View-invariant action recognition based on artificial neural networks

In this paper, a novel view invariant action recognition method based on neural network representation and recognition is proposed. The novel representation of action videos is based on learning spatially related human body posture prototypes using self organizing maps. Fuzzy distances from human body posture prototypes are used to produce a time invariant action representation. Multilayer perceptrons are used for action classification. The algorithm is trained using data from a multi-camera setup. An arbitrary number of cameras can be used in order to recognize actions using a Bayesian framework. The proposed method can also be applied to videos depicting interactions between humans, without any modification. The use of information captured from different viewing angles leads to high classification performance. The proposed method is the first one that has been tested in challenging experimental setups, a fact that denotes its effectiveness to deal with most of the open issues in action recognition.

Oscillatory neural network for image segmentation with biased competition for attention

We study the emergent properties of an artificial neural network which combines segmentation by oscillations and biased competition for perceptual processing. The aim is to progress in image segmentation by mimicking abstractly the way how the cerebral cortex works. In our model, the neurons associated with features belonging to an object start to oscillate synchronously, while competing objects oscillate with an opposing phase. The emergent properties of the network are confirmed by experiments with artificial image data.

Fast and robust image segmentation by small-world neural oscillator networks

Inspired by the temporal correlation theory of brain functions, researchers have presented a number of neural oscillator networks to implement visual scene segmentation problems. Recently, it is shown that many biological neural networks are typical small-world networks. In this paper, we propose and investigate two small-world models derived from the well-known LEGION (locally excitatory and globally inhibitory oscillator network) model. To form a small-world network, we add a proper proportion of unidirectional shortcuts (random long-range connections) to the original LEGION model. With local connections and shortcuts, the neural oscillators can not only communicate with neighbors but also exchange phase information with remote partners. Model 1 introduces excitatory shortcuts to enhance the synchronization within an oscillator group representing the same object. Model 2 goes further to replace the global inhibitor with a sparse set of inhibitory shortcuts. Simulation results indicate that the proposed small-world models could achieve synchronization faster than the original LEGION model and are more likely to bind disconnected image regions belonging together. In addition, we argue that these two models are more biologically plausible.

An integrated neural model of semantic memory, lexical retrieval and category formation, based on a distributed feature representation

This work presents a connectionist model of the semantic-lexical system. Model assumes that the lexical and semantic aspects of language are memorized in two distinct stores, and are then linked together on the basis of previous experience, using physiological learning mechanisms. Particular characteristics of the model are: (1) the semantic aspects of an object are described by a collection of features, whose number may vary between objects. (2) Individual features are topologically organized to implement a similarity principle. (3) Gamma-band synchronization is used to segment different objects simultaneously. (4) The model is able to simulate the formation of categories, assuming that objects belong to the same category if they share some features. (5) Homosynaptic potentiation and homosynaptic depression are used within the semantic network, to create an asymmetric pattern of synapses; this allows a different role to be assigned to shared and distinctive features during object reconstruction. (6) Features which frequently occurred together, and the corresponding word-forms, become linked via reciprocal excitatory synapses. (7) Features in the semantic network tend to inhibit words not associated with them during the previous learning phase. Simulations show that, after learning, presentation of a cue can evoke the overall object and the corresponding word in the lexical area. Word presentation, in turn, activates the corresponding features in the sensory-motor areas, recreating the same conditions occurred during learning, according to a grounded cognition viewpoint. Several words and their conceptual description can coexist in the lexical-semantic system exploiting gamma-band time division. Schematic exempla are shown, to illustrate the possibility to distinguish between words representing a category, and words representing individual members and to evaluate the role of gamma-band synchronization in priming. Finally, the model is used to simulate patients with focalized lesions, assuming a damage of synaptic strength in specific feature areas. Results are critically discussed in view of future model extensions and application to real objects. The model represents an original effort to incorporate many basic ideas, found in recent conceptual theories, within a single quantitative scaffold.

Resting frontal gamma power at 16, 24 and 36 months predicts individual differences in language and cognition at 4 and 5 years

Gamma activity has been linked to a variety of different cognitive processes and exists in both transient and persistent forms. Across studies, different brain regions have been suggested to contribute to gamma activity. Multiple studies have shown that the function of gamma oscillations may be related to temporal binding of early sensory information to relevant top-down processes. Given this hypothesis, we expected gamma oscillations to subserve general brain mechanisms that contribute to the development of cognitive and linguistic systems. The present study aims to examine the predictive relations between resting-state cortical gamma power density at a critical point in language and cognitive acquisition (i.e. 16, 24 and 36 months), and cognitive and language output at ages 4 and 5 years. Our findings show that both 24- and 36-month gamma power are significantly correlated with later language scores, notably Non-Word Repetition. Further, 16-, 24- and 36-month gamma were all significantly correlated with 4-year PLS-3 and CELF-P sentence structure scores. Although associations reported here do not reflect a direct cause and effect of early resting gamma power on later language outcomes, capacity to generate higher power in the gamma range at crucial developmental periods may index better modulation of attention and allow easier access to working memory, thus providing an advantage for overall development, particularly in the linguistic domain. Moreover, measuring abilities at times when these abilities are still emergent may allow better prediction of later outcomes.

A computational model of the lexical-semantic system based on a grounded cognition approach

This work presents a connectionist model of the semantic-lexical system based on grounded cognition. The model assumes that the lexical and semantic aspects of language are memorized in two distinct stores. The semantic properties of objects are represented as a collection of features, whose number may vary among objects. Features are described as activation of neural oscillators in different sensory-motor areas (one area for each feature) topographically organized to implement a similarity principle. Lexical items are represented as activation of neural groups in a different layer. Lexical and semantic aspects are then linked together on the basis of previous experience, using physiological learning mechanisms. After training, features which frequently occurred together, and the corresponding word-forms, become linked via reciprocal excitatory synapses. The model also includes some inhibitory synapses: features in the semantic network tend to inhibit words not associated with them during the previous learning phase. Simulations show that after learning, presentation of a cue can evoke the overall object and the corresponding word in the lexical area. Moreover, different objects and the corresponding words can be simultaneously retrieved and segmented via a time division in the gamma-band. Word presentation, in turn, activates the corresponding features in the sensory-motor areas, recreating the same conditions occurring during learning. The model simulates the formation of categories, assuming that objects belong to the same category if they share some features. Simple exempla are shown to illustrate how words representing a category can be distinguished from words representing individual members. Finally, the model can be used to simulate patients with focalized lesions, assuming an impairment of synaptic strength in specific feature areas.

A semantic model to study neural organization of language in bilingualism

A neural network model of object semantic representation is used to simulate learning of new words from a foreign language. The network consists of feature areas, devoted to description of object properties, and a lexical area, devoted to words representation. Neurons in the feature areas are implemented as Wilson-Cowan oscillators, to allow segmentation of different simultaneous objects via gamma-band synchronization. Excitatory synapses among neurons in the feature and lexical areas are learned, during a training phase, via a Hebbian rule. In this work, we first assume that some words in the first language (L1) and the corresponding object representations are initially learned during a preliminary training phase. Subsequently, second-language (L2) words are learned by simultaneously presenting the new word together with the L1 one. A competitive mechanism between the two words is also implemented by the use of inhibitory interneurons. Simulations show that, after a weak training, the L2 word allows retrieval of the object properties but requires engagement of the first language. Conversely, after a prolonged training, the L2 word becomes able to retrieve object per se. In this case, a conflict between words can occur, requiring a higher-level decision mechanism.

A neural network model of semantic memory linking feature-based object representation and words

Recent theories in cognitive neuroscience suggest that semantic memory is a distributed process, which involves many cortical areas and is based on a multimodal representation of objects. The aim of this work is to extend a previous model of object representation to realize a semantic memory, in which sensory-motor representations of objects are linked with words. The model assumes that each object is described as a collection of features, coded in different cortical areas via a topological organization. Features in different objects are segmented via gamma-band synchronization of neural oscillators. The feature areas are further connected with a lexical area, devoted to the representation of words. Synapses among the feature areas, and among the lexical area and the feature areas are trained via a time-dependent Hebbian rule, during a period in which individual objects are presented together with the corresponding words. Simulation results demonstrate that, during the retrieval phase, the network can deal with the simultaneous presence of objects (from sensory-motor inputs) and words (from acoustic inputs), can correctly associate objects with words and segment objects even in the presence of incomplete information. Moreover, the network can realize some semantic links among words representing objects with shared features. These results support the idea that semantic memory can be described as an integrated process, whose content is retrieved by the co-activation of different multimodal regions. In perspective, extended versions of this model may be used to test conceptual theories, and to provide a quantitative assessment of existing data (for instance concerning patients with neural deficits).