Recurrent temporal networks and language acquisition-from corticostriatal neurophysiology to reservoir computing

Biological computation through recurrence

One of the defining features of living systems is their adaptability to changing environmental conditions. This requires organisms to extract temporal and spatial features of their environment, and use that information to compute the appropriate response. In the last two decades, a growing body of work, mainly coming from the machine learning and computational neuroscience fields, has shown that such complex information processing can be performed by recurrent networks. Temporal computations arise in these networks through the interplay between the external stimuli and the network's internal state. In this article we review our current understanding of how recurrent networks can be used by biological systems, from cells to brains, for complex information processing. Rather than focusing on sophisticated, artificial recurrent architectures such as long short-term memory (LSTM) networks, here we concentrate on simpler network structures and learning algorithms that can be expected to have been found by evolution. We also review studies showing evidence of naturally occurring recurrent networks in living organisms. Lastly, we discuss some relevant evolutionary aspects concerning the emergence of this natural computation paradigm.

Narrative event segmentation in the cortical reservoir

Recent research has revealed that during continuous perception of movies or stories, humans display cortical activity patterns that reveal hierarchical segmentation of event structure. Thus, sensory areas like auditory cortex display high frequency segmentation related to the stimulus, while semantic areas like posterior middle cortex display a lower frequency segmentation related to transitions between events. These hierarchical levels of segmentation are associated with different time constants for processing. Likewise, when two groups of participants heard the same sentence in a narrative, preceded by different contexts, neural responses for the groups were initially different and then gradually aligned. The time constant for alignment followed the segmentation hierarchy: sensory cortices aligned most quickly, followed by mid-level regions, while some higher-order cortical regions took more than 10 seconds to align. These hierarchical segmentation phenomena can be considered in the context of processing related to comprehension. In a recently described model of discourse comprehension word meanings are modeled by a language model pre-trained on a billion word corpus. During discourse comprehension, word meanings are continuously integrated in a recurrent cortical network. The model demonstrates novel discourse and inference processing, in part because of two fundamental characteristics: real-world event semantics are represented in the word embeddings, and these are integrated in a reservoir network which has an inherent gradient of functional time constants due to the recurrent connections. Here we demonstrate how this model displays hierarchical narrative event segmentation properties beyond the embeddings alone, or their linear integration. The reservoir produces activation patterns that are segmented by a hidden Markov model (HMM) in a manner that is comparable to that of humans. Context construction displays a continuum of time constants across reservoir neuron subsets, while context forgetting has a fixed time constant across these subsets. Importantly, virtual areas formed by subgroups of reservoir neurons with faster time constants segmented with shorter events, while those with longer time constants preferred longer events. This neurocomputational recurrent neural network simulates narrative event processing as revealed by the fMRI event segmentation algorithm provides a novel explanation of the asymmetry in narrative forgetting and construction. The model extends the characterization of online integration processes in discourse to more extended narrative, and demonstrates how reservoir computing provides a useful model of cortical processing of narrative structure.

Language is not a gadget

Heyes does well to argue that some of the apparently innate human capabilities for cultural learning can be considered in terms of more general-purpose mechanisms. In the application of this to language, she overlooks some of its most interesting properties. I review three, and then illustrate how mindreading can come from general-purpose mechanism via language.

Recent advances in physical reservoir computing: A review

Reservoir computing is a computational framework suited for temporal/sequential data processing. It is derived from several recurrent neural network models, including echo state networks and liquid state machines. A reservoir computing system consists of a reservoir for mapping inputs into a high-dimensional space and a readout for pattern analysis from the high-dimensional states in the reservoir. The reservoir is fixed and only the readout is trained with a simple method such as linear regression and classification. Thus, the major advantage of reservoir computing compared to other recurrent neural networks is fast learning, resulting in low training cost. Another advantage is that the reservoir without adaptive updating is amenable to hardware implementation using a variety of physical systems, substrates, and devices. In fact, such physical reservoir computing has attracted increasing attention in diverse fields of research. The purpose of this review is to provide an overview of recent advances in physical reservoir computing by classifying them according to the type of the reservoir. We discuss the current issues and perspectives related to physical reservoir computing, in order to further expand its practical applications and develop next-generation machine learning systems.

Propagation delays determine neuronal activity and synaptic connectivity patterns emerging in plastic neuronal networks

In plastic neuronal networks, the synaptic strengths are adapted to the neuronal activity. Specifically, spike-timing-dependent plasticity (STDP) is a fundamental mechanism that modifies the synaptic strengths based on the relative timing of pre- and postsynaptic spikes, taking into account the spikes' temporal order. In many studies, propagation delays were neglected to avoid additional dynamic complexity or computational costs. So far, networks equipped with a classic STDP rule typically rule out bidirectional couplings (i.e., either loops or uncoupled states) and are, hence, not able to reproduce fundamental experimental findings. In this review paper, we consider additional features, e.g., extensions of the classic STDP rule or additional aspects like noise, in order to overcome the contradictions between theory and experiment. In addition, we review in detail recent studies showing that a classic STDP rule combined with realistic propagation patterns is able to capture relevant experimental findings. In two coupled oscillatory neurons with propagation delays, bidirectional synapses can be preserved and potentiated. This result also holds for large networks of type-II phase oscillators. In addition, not only the mean of the initial distribution of synaptic weights, but also its standard deviation crucially determines the emergent structural connectivity, i.e., the mean final synaptic weight, the number of two-neuron loops, and the symmetry of the final connectivity pattern. The latter is affected by the firing rates, where more symmetric synaptic configurations emerge at higher firing rates. Finally, we discuss these findings in the context of the computational neuroscience-based development of desynchronizing brain stimulation techniques.

Sequential processing deficit as a shared persisting biomarker in dyslexia and childhood apraxia of speech

The purpose of this study was to investigate the hypothesis that individuals with dyslexia and individuals with childhood apraxia of speech share an underlying persisting deficit in processing sequential information. Levels of impairment (sensory encoding, memory, retrieval, and motor planning/programming) were also investigated. Participants were 22 adults with dyslexia, 10 adults with a probable history of childhood apraxia of speech (phCAS), and 22 typical controls. All participants completed nonword repetition, multisyllabic real word repetition, and nonword decoding tasks. Using phonological process analysis, errors were classified as sequence or substitution errors. Adults with dyslexia and adults with phCAS showed evidence of persisting nonword repetition deficits. In all three tasks, the adults in the two disorder groups produced more errors of both classes than the controls, but disproportionally more sequencing than substitution errors during the nonword repetition task. During the real word repetition task, the phCAS produced the most sequencing errors, whereas during the nonword decoding task, the dyslexia group produced the most sequencing errors. Performance during multisyllabic motor speech tasks, relative to monosyllabic conditions, was correlated with the sequencing error component during nonword repetition. The results provide evidence for a shared persisting sequential processing deficit in the dyslexia and phCAS groups during linguistic and motor speech tasks. Evidence for impairments in sensory encoding, short-term memory, and motor planning/programming was found in both disorder groups. Future studies should investigate clinical applications regarding preventative and targeted interventions towards cross-modal treatment effects.

Narrative Constructions for the Organization of Self Experience: Proof of Concept via Embodied Robotics

It has been proposed that starting from meaning that the child derives directly from shared experience with others, adult narrative enriches this meaning and its structure, providing causal links between unseen intentional states and actions. This would require a means for representing meaning from experience-a situation model-and a mechanism that allows information to be extracted from sentences and mapped onto the situation model that has been derived from experience, thus enriching that representation. We present a hypothesis and theory concerning how the language processing infrastructure for grammatical constructions can naturally be extended to narrative constructions to provide a mechanism for using language to enrich meaning derived from physical experience. Toward this aim, the grammatical construction models are augmented with additional structures for representing relations between events across sentences. Simulation results demonstrate proof of concept for how the narrative construction model supports multiple successive levels of meaning creation which allows the system to learn about the intentionality of mental states, and argument substitution which allows extensions to metaphorical language and analogical problem solving. Cross-linguistic validity of the system is demonstrated in Japanese. The narrative construction model is then integrated into the cognitive system of a humanoid robot that provides the memory systems and world-interaction required for representing meaning in a situation model. In this context proof of concept is demonstrated for how the system enriches meaning in the situation model that has been directly derived from experience. In terms of links to empirical data, the model predicts strong usage based effects: that is, that the narrative constructions used by children will be highly correlated with those that they experience. It also relies on the notion of narrative or discourse function words. Both of these are validated in the experimental literature.

Mario Becomes Cognitive

In line with Allen Newell's challenge to develop complete cognitive architectures, and motivated by a recent proposal for a unifying subsymbolic computational theory of cognition, we introduce the cognitive control architecture SEMLINCS. SEMLINCS models the development of an embodied cognitive agent that learns discrete production rule-like structures from its own, autonomously gathered, continuous sensorimotor experiences. Moreover, the agent uses the developing knowledge to plan and control environmental interactions in a versatile, goal-directed, and self-motivated manner. Thus, in contrast to several well-known symbolic cognitive architectures, SEMLINCS is not provided with production rules and the involved symbols, but it learns them. In this paper, the actual implementation of SEMLINCS causes learning and self-motivated, autonomous behavioral control of the game figure Mario in a clone of the computer game Super Mario Bros. Our evaluations highlight the successful development of behavioral versatility as well as the learning of suitable production rules and the involved symbols from sensorimotor experiences. Moreover, knowledge- and motivation-dependent individualizations of the agents' behavioral tendencies are shown. Finally, interaction sequences can be planned on the sensorimotor-grounded production rule level. Current limitations directly point toward the need for several further enhancements, which may be integrated into SEMLINCS in the near future. Overall, SEMLINCS may be viewed as an architecture that allows the functional and computational modeling of embodied cognitive development, whereby the current main focus lies on the development of production rules from sensorimotor experiences.

Toward a Unified Sub-symbolic Computational Theory of Cognition

This paper proposes how various disciplinary theories of cognition may be combined into a unifying, sub-symbolic, computational theory of cognition. The following theories are considered for integration: psychological theories, including the theory of event coding, event segmentation theory, the theory of anticipatory behavioral control, and concept development; artificial intelligence and machine learning theories, including reinforcement learning and generative artificial neural networks; and theories from theoretical and computational neuroscience, including predictive coding and free energy-based inference. In the light of such a potential unification, it is discussed how abstract cognitive, conceptualized knowledge and understanding may be learned from actively gathered sensorimotor experiences. The unification rests on the free energy-based inference principle, which essentially implies that the brain builds a predictive, generative model of its environment. Neural activity-oriented inference causes the continuous adaptation of the currently active predictive encodings. Neural structure-oriented inference causes the longer term adaptation of the developing generative model as a whole. Finally, active inference strives for maintaining internal homeostasis, causing goal-directed motor behavior. To learn abstract, hierarchical encodings, however, it is proposed that free energy-based inference needs to be enhanced with structural priors, which bias cognitive development toward the formation of particular, behaviorally suitable encoding structures. As a result, it is hypothesized how abstract concepts can develop from, and thus how they are structured by and grounded in, sensorimotor experiences. Moreover, it is sketched-out how symbol-like thought can be generated by a temporarily active set of predictive encodings, which constitute a distributed neural attractor in the form of an interactive free-energy minimum. The activated, interactive network attractor essentially characterizes the semantics of a concept or a concept composition, such as an actual or imagined situation in our environment. Temporal successions of attractors then encode unfolding semantics, which may be generated by a behavioral or mental interaction with an actual or imagined situation in our environment. Implications, further predictions, possible verification, and falsifications, as well as potential enhancements into a fully spelled-out unified theory of cognition are discussed at the end of the paper.

Seeking Synthesis: The Integrative Problem in Understanding Language and Its Evolution

We discuss two problems for a general scientific understanding of language, sequences and synergies: how language is an intricately sequenced behavior and how language is manifested as a multidimensionally structured behavior. Though both are central in our understanding, we observe that the former tends to be studied more than the latter. We consider very general conditions that hold in human brain evolution and its computational implications, and identify multimodal and multiscale organization as two key characteristics of emerging cognitive function in our species. This suggests that human brains, and cognitive function specifically, became more adept at integrating diverse information sources and operating at multiple levels for linguistic performance. We argue that framing language evolution, learning, and use in terms of synergies suggests new research questions, and it may be a fruitful direction for new developments in theory and modeling of language as an integrated system.

Selection of cortical dynamics for motor behaviour by the basal ganglia

The basal ganglia and cortex are strongly implicated in the control of motor preparation and execution. Re-entrant loops between these two brain areas are thought to determine the selection of motor repertoires for instrumental action. The nature of neural encoding and processing in the motor cortex as well as the way in which selection by the basal ganglia acts on them is currently debated. The classic view of the motor cortex implementing a direct mapping of information from perception to muscular responses is challenged by proposals viewing it as a set of dynamical systems controlling muscles. Consequently, the common idea that a competition between relatively segregated cortico-striato-nigro-thalamo-cortical channels selects patterns of activity in the motor cortex is no more sufficient to explain how action selection works. Here, we contribute to develop the dynamical view of the basal ganglia-cortical system by proposing a computational model in which a thalamo-cortical dynamical neural reservoir is modulated by disinhibitory selection of the basal ganglia guided by top-down information, so that it responds with different dynamics to the same bottom-up input. The model shows how different motor trajectories can so be produced by controlling the same set of joint actuators. Furthermore, the model shows how the basal ganglia might modulate cortical dynamics by preserving coarse-grained spatiotemporal information throughout cortico-cortical pathways.

Learning of Chunking Sequences in Cognition and Behavior

We often learn and recall long sequences in smaller segments, such as a phone number 858 534 22 30 memorized as four segments. Behavioral experiments suggest that humans and some animals employ this strategy of breaking down cognitive or behavioral sequences into chunks in a wide variety of tasks, but the dynamical principles of how this is achieved remains unknown. Here, we study the temporal dynamics of chunking for learning cognitive sequences in a chunking representation using a dynamical model of competing modes arranged to evoke hierarchical Winnerless Competition (WLC) dynamics. Sequential memory is represented as trajectories along a chain of metastable fixed points at each level of the hierarchy, and bistable Hebbian dynamics enables the learning of such trajectories in an unsupervised fashion. Using computer simulations, we demonstrate the learning of a chunking representation of sequences and their robust recall. During learning, the dynamics associates a set of modes to each information-carrying item in the sequence and encodes their relative order. During recall, hierarchical WLC guarantees the robustness of the sequence order when the sequence is not too long. The resulting patterns of activities share several features observed in behavioral experiments, such as the pauses between boundaries of chunks, their size and their duration. Failures in learning chunking sequences provide new insights into the dynamical causes of neurological disorders such as Parkinson’s disease and Schizophrenia.

Because chunking is a hallmark of the brain’s organization, efforts to understand its dynamics can provide valuable insights into the brain and its disorders. For identifying the dynamical principles of chunking learning, we hypothesize that perceptual sequences can be learned and stored as a chain of metastable fixed points in a low-dimensional dynamical system, similar to the trajectory of a ball rolling down a pinball machine. During a learning phase, the interactions in the network evolve such that the network learns a chunking representation of the sequence, as when memorizing a phone number in segments. In the example of the pinball machine, learning can be identified with the gradual placement of the pins. After learning, the pins are placed in a way that, at each run, the ball follows the same trajectory (recall of the same sequence) that encodes the perceptual sequence. Simulations show that the dynamics are endowed with the hallmarks of chunking observed in behavioral experiments, such as increased delays observed before loading new chunks.

A Cognitive Neural Architecture Able to Learn and Communicate through Natural Language

Communicative interactions involve a kind of procedural knowledge that is used by the human brain for processing verbal and nonverbal inputs and for language production. Although considerable work has been done on modeling human language abilities, it has been difficult to bring them together to a comprehensive tabula rasa system compatible with current knowledge of how verbal information is processed in the brain. This work presents a cognitive system, entirely based on a large-scale neural architecture, which was developed to shed light on the procedural knowledge involved in language elaboration. The main component of this system is the central executive, which is a supervising system that coordinates the other components of the working memory. In our model, the central executive is a neural network that takes as input the neural activation states of the short-term memory and yields as output mental actions, which control the flow of information among the working memory components through neural gating mechanisms. The proposed system is capable of learning to communicate through natural language starting from tabula rasa, without any a priori knowledge of the structure of phrases, meaning of words, role of the different classes of words, only by interacting with a human through a text-based interface, using an open-ended incremental learning process. It is able to learn nouns, verbs, adjectives, pronouns and other word classes, and to use them in expressive language. The model was validated on a corpus of 1587 input sentences, based on literature on early language assessment, at the level of about 4-years old child, and produced 521 output sentences, expressing a broad range of language processing functionalities.

Corticostriatal response selection in sentence production: Insights from neural network simulation with reservoir computing

Language production requires selection of the appropriate sentence structure to accommodate the communication goal of the speaker - the transmission of a particular meaning. Here we consider event meanings, in terms of predicates and thematic roles, and we address the problem that a given event can be described from multiple perspectives, which poses a problem of response selection. We present a model of response selection in sentence production that is inspired by the primate corticostriatal system. The model is implemented in the context of reservoir computing where the reservoir - a recurrent neural network with fixed connections - corresponds to cortex, and the readout corresponds to the striatum. We demonstrate robust learning, and generalization properties of the model, and demonstrate its cross linguistic capabilities in English and Japanese. The results contribute to the argument that the corticostriatal system plays a role in response selection in language production, and to the stance that reservoir computing is a valid potential model of corticostriatal processing.

Degree of automaticity and the prefrontal cortex

The dorsolateral prefrontal cortex (PFC), with more anterior areas [Brodmann area (BA) 45, 47, and 10], has been known to be activated as cognitive hierarchy increases. However, this does not hold for highly automatic processes such as first language (L1), where the posterior region (BA 44) is known as the key area for the processing of complex linguistic hierarchy. Discussing this disparity, we propose that the degree of automaticity (DoA) is a crucial factor for the framework of functional mapping in the PFC: the posterior-to-anterior gradient system for more controlled processes and the posterior-confined system for automatic processes. We support this view with previous findings and provide a new perspective on the functional organization of the PFC.