Dynamics of the semantic priming shift: behavioral experiments and cortical network model

Dynamic branching in a neural network model for probabilistic prediction of sequences

An important function of the brain is to predict which stimulus is likely to occur based on the perceived cues. The present research studied the branching behavior of a computational network model of populations of excitatory and inhibitory neurons, both analytically and through simulations. Results show how synaptic efficacy, retroactive inhibition and short-term synaptic depression determine the dynamics of selection between different branches predicting sequences of stimuli of different probabilities. Further results show that changes in the probability of the different predictions depend on variations of neuronal gain. Such variations allow the network to optimize the probability of its predictions to changing probabilities of the sequences without changing synaptic efficacy.

Statistical learning of unbalanced exclusive-or temporal sequences in humans

A pervasive issue in statistical learning has been to determine the parameters of regularity extraction. Our hypothesis was that the extraction of transitional probabilities can prevail over frequency if the task involves prediction. Participants were exposed to four repeated sequences of three stimuli (XYZ) with each stimulus corresponding to the position of a red dot on a touch screen that participants were required to touch sequentially. The temporal and spatial structure of the positions corresponded to a serial version of the exclusive-or (XOR) that allowed testing of the respective effect of frequency and first- and second-order transitional probabilities. The XOR allowed the first-order transitional probability to vary while being not completely related to frequency and to vary while the second-order transitional probability was fixed (p(Z|X, Y) = 1). The findings show that first-order transitional probability prevails over frequency to predict the second stimulus from the first and that it also influences the prediction of the third item despite the presence of second-order transitional probability that could have offered a certain prediction of the third item. These results are particularly informative in light of statistical learning models.

Meaning before grammar: A review of ERP experiments on the neurodevelopmental origins of semantic processing

According to traditional linguistic theories, the construction of complex meanings relies firmly on syntactic structure-building operations. Recently, however, new models have been proposed in which semantics is viewed as being partly autonomous from syntax. In this paper, we discuss some of the developmental implications of syntax-based and autonomous models of semantics. We review event-related brain potential (ERP) studies on semantic processing in infants and toddlers, focusing on experiments reporting modulations of N400 amplitudes using visual or auditory stimuli and different temporal structures of trials. Our review suggests that infants can relate or integrate semantic information from temporally overlapping stimuli across modalities by 6 months of age. The ability to relate or integrate semantic information over time, within and across modalities, emerges by 9 months. The capacity to relate or integrate information from spoken words in sequences and sentences appears by 18 months. We also review behavioral and ERP studies showing that grammatical and syntactic processing skills develop only later, between 18 and 32 months. These results provide preliminary evidence for the availability of some semantic processes prior to the full developmental emergence of syntax: non-syntactic meaning-building operations are available to infants, albeit in restricted ways, months before the abstract machinery of grammar is in place. We discuss this hypothesis in light of research on early language acquisition and human brain development.

Latching dynamics in neural networks with synaptic depression

Prediction is the ability of the brain to quickly activate a target concept in response to a related stimulus (prime). Experiments point to the existence of an overlap between the populations of the neurons coding for different stimuli, and other experiments show that prime-target relations arise in the process of long term memory formation. The classical modelling paradigm is that long term memories correspond to stable steady states of a Hopfield network with Hebbian connectivity. Experiments show that short term synaptic depression plays an important role in the processing of memories. This leads naturally to a computational model of priming, called latching dynamics; a stable state (prime) can become unstable and the system may converge to another transiently stable steady state (target). Hopfield network models of latching dynamics have been studied by means of numerical simulation, however the conditions for the existence of this dynamics have not been elucidated. In this work we use a combination of analytic and numerical approaches to confirm that latching dynamics can exist in the context of a symmetric Hebbian learning rule, however lacks robustness and imposes a number of biologically unrealistic restrictions on the model. In particular our work shows that the symmetry of the Hebbian rule is not an obstruction to the existence of latching dynamics, however fine tuning of the parameters of the model is needed.

Semantic integration by pattern priming: experiment and cortical network model

Neural network models describe semantic priming effects by way of mechanisms of activation of neurons coding for words that rely strongly on synaptic efficacies between pairs of neurons. Biologically inspired Hebbian learning defines efficacy values as a function of the activity of pre- and post-synaptic neurons only. It generates only pair associations between words in the semantic network. However, the statistical analysis of large text databases points to the frequent occurrence not only of pairs of words (e.g., "the way") but also of patterns of more than two words (e.g., "by the way"). The learning of these frequent patterns of words is not reducible to associations between pairs of words but must take into account the higher level of coding of three-word patterns. The processing and learning of pattern of words challenges classical Hebbian learning algorithms used in biologically inspired models of priming. The aim of the present study was to test the effects of patterns on the semantic processing of words and to investigate how an inter-synaptic learning algorithm succeeds at reproducing the experimental data. The experiment manipulates the frequency of occurrence of patterns of three words in a multiple-paradigm protocol. Results show for the first time that target words benefit more priming when embedded in a pattern with the two primes than when only associated with each prime in pairs. A biologically inspired inter-synaptic learning algorithm is tested that potentiates synapses as a function of the activation of more than two pre- and post-synaptic neurons. Simulations show that the network can learn patterns of three words to reproduce the experimental results.

Inter-synaptic learning of combination rules in a cortical network model

Selecting responses in working memory while processing combinations of stimuli depends strongly on their relations stored in long-term memory. However, the learning of XOR-like combinations of stimuli and responses according to complex rules raises the issue of the non-linear separability of the responses within the space of stimuli. One proposed solution is to add neurons that perform a stage of non-linear processing between the stimuli and responses, at the cost of increasing the network size. Based on the non-linear integration of synaptic inputs within dendritic compartments, we propose here an inter-synaptic (IS) learning algorithm that determines the probability of potentiating/depressing each synapse as a function of the co-activity of the other synapses within the same dendrite. The IS learning is effective with random connectivity and without either a priori wiring or additional neurons. Our results show that IS learning generates efficacy values that are sufficient for the processing of XOR-like combinations, on the basis of the sole correlational structure of the stimuli and responses. We analyze the types of dendrites involved in terms of the number of synapses from pre-synaptic neurons coding for the stimuli and responses. The synaptic efficacy values obtained show that different dendrites specialize in the detection of different combinations of stimuli. The resulting behavior of the cortical network model is analyzed as a function of inter-synaptic vs. Hebbian learning. Combinatorial priming effects show that the retrospective activity of neurons coding for the stimuli trigger XOR-like combination-selective prospective activity of neurons coding for the expected response. The synergistic effects of inter-synaptic learning and of mixed-coding neurons are simulated. The results show that, although each mechanism is sufficient by itself, their combined effects improve the performance of the network.

Signal integration on the dendrites of a pyramidal neuron model

This paper studied the synaptic and dendritic integration with different spatial distributions of synapses on the dendrites of a biophysically-detailed layer 5 pyramidal neuron model. It has been observed that temporally synchronous and spatially clustered synaptic inputs make dendrites perform a highly nonlinear integration. The effect of clustering degree of synaptic distribution on neuronal responsiveness is investigated by changing the number of top apical dendrites where active synapses are allocated. The neuron shows maximum responsiveness to synaptic inputs which have an intermediate clustering degree of spatial distribution, indicating complex interactions among dendrites with the existence of nonlinear synaptic and dendritic integrations.

Early dynamics of the semantic priming shift

Semantic processing of sequences of words requires the cognitive system to keep several word meanings simultaneously activated in working memory with limited capacity. The real- time updating of the sequence of word meanings relies on dynamic changes in the associates to the words that are activated. Protocols involving two sequential primes report a semantic priming shift from larger priming of associates to the first prime to larger priming of associates to the second prime, in a range of long SOAs (stimulus-onset asynchronies) between the second prime and the target. However, the possibility for an early semantic priming shift is still to be tested, and its dynamics as a function of association strength remain unknown. Three multiple priming experiments are proposed that cross-manipulate association strength between each of two successive primes and a target, for different values of short SOAs and prime durations. Results show an early priming shift ranging from priming of associates to the first prime only to priming of strong associates to the first prime and all of the associates to the second prime. We investigated the neural basis of the early priming shift by using a network model of spike frequency adaptive cortical neurons (e.g., Deco & Rolls, 2005), able to code different association strengths between the primes and the target. The cortical network model provides a description of the early dynamics of the priming shift in terms of pro-active and retro-active interferences within populations of excitatory neurons regulated by fast and unselective inhibitory feedback.