Unifying Physical Interaction, Linguistic Communication, and Language Acquisition of Cognitive Agents by Minimalist Grammars

Cognitive agents that act independently and solve problems in their environment on behalf of a user are referred to as autonomous. In order to increase the degree of autonomy, advanced cognitive architectures also contain higher-level psychological modules with which needs and motives of the agent are also taken into account and with which the behavior of the agent can be controlled. Regardless of the level of autonomy, successful behavior is based on interacting with the environment and being able to communicate with other agents or users. The agent can use these skills to learn a truthful knowledge model of the environment and thus predict the consequences of its own actions. For this purpose, the symbolic information received during the interaction and communication must be converted into representational data structures so that they can be stored in the knowledge model, processed logically and retrieved from there. Here, we firstly outline a grammar-based transformation mechanism that unifies the description of physical interaction and linguistic communication and on which the language acquisition is based. Specifically, we use minimalist grammar (MG) for this aim, which is a recent computational implementation of generative linguistics. In order to develop proper cognitive information and communication technologies, we are using utterance meaning transducers (UMT) that are based on semantic parsers and a mental lexicon, comprising syntactic and semantic features of the language under consideration. This lexicon must be acquired by a cognitive agent during interaction with its users. To this aim we outline a reinforcement learning algorithm for the acquisition of syntax and semantics of English utterances. English declarative sentences are presented to the agent by a teacher in form of utterance meaning pairs (UMP) where the meanings are encoded as formulas of predicate logic. Since MG codifies universal linguistic competence through inference rules, thereby separating innate linguistic knowledge from the contingently acquired lexicon, our approach unifies generative grammar and reinforcement learning, hence potentially resolving the still pending Chomsky-Skinner controversy.

Metastable Resting State Brain Dynamics

Metastability refers to the fact that the state of a dynamical system spends a large amount of time in a restricted region of its available phase space before a transition takes place, bringing the system into another state from where it might recur into the previous one. beim Graben and Hutt (2013) suggested to use the recurrence plot (RP) technique introduced by Eckmann et al. (1987) for the segmentation of system's trajectories into metastable states using recurrence grammars. Here, we apply this recurrence structure analysis (RSA) for the first time to resting-state brain dynamics obtained from functional magnetic resonance imaging (fMRI). Brain regions are defined according to the brain hierarchical atlas (BHA) developed by Diez et al. (2015), and as a consequence, regions present high-connectivity in both structure (obtained from diffusion tensor imaging) and function (from the blood-level dependent-oxygenation-BOLD-signal). Remarkably, regions observed by Diez et al. were completely time-invariant. Here, in order to compare this static picture with the metastable systems dynamics obtained from the RSA segmentation, we determine the number of metastable states as a measure of complexity for all subjects and for region numbers varying from 3 to 100. We find RSA convergence toward an optimal segmentation of 40 metastable states for normalized BOLD signals, averaged over BHA modules. Next, we build a bistable dynamics at population level by pooling 30 subjects after Hausdorff clustering. In link with this finding, we reflect on the different modeling frameworks that can allow for such scenarios: heteroclinic dynamics, dynamics with riddled basins of attraction, multiple-timescale dynamics. Finally, we characterize the metastable states both functionally and structurally, using templates for resting state networks (RSNs) and the automated anatomical labeling (AAL) atlas, respectively.

Detecting recurrence domains of dynamical systems by symbolic dynamics

We propose an algorithm for the detection of recurrence domains of complex dynamical systems from time series. Our approach exploits the characteristic checkerboard texture of recurrence domains exhibited in recurrence plots. In phase space, recurrence plots yield intersecting balls around sampling points that could be merged into cells of a phase space partition. We construct this partition by a rewriting grammar applied to the symbolic dynamics of time indices. A maximum entropy principle defines the optimal size of intersecting balls. The final application to high-dimensional brain signals yields an optimal symbolic recurrence plot revealing functional components of the signal.

Computationelle Neurolinguistik

Computational neurolinguistics integrates methods from computational (psycho-)linguistics and computational neuroscience in order to model neural correlates of linguistic behavior. We illustrate these techniques using an example of the language processing of German negative polarity items (NPI) in the event-related brain potential (ERP) paradigm. To that aim, we first describe the syntactic and semantic licensing conditions of NPIs by means of slightly modified minimalist grammars. In a second step we use dynamic cognitive modeling (DCM) to map the state descriptions of a minimalist parser onto activation patterns of a neural network. Thirdly, the network’s synaptic weights are trained with the correct parse of NPI constructions. Using these weights we calculate neural harmony measures for correct and for ungrammatical NPI constructions. In a final step we correlate the harmonies of the dynamical model with experimentally obtained ERP amplitudes by means of a simple statistical model.

Inverse problems in dynamic cognitive modeling

Inverse problems for dynamical system models of cognitive processes comprise the determination of synaptic weight matrices or kernel functions for neural networks or neural/dynamic field models, respectively. We introduce dynamic cognitive modeling as a three tier top-down approach where cognitive processes are first described as algorithms that operate on complex symbolic data structures. Second, symbolic expressions and operations are represented by states and transformations in abstract vector spaces. Third, prescribed trajectories through representation space are implemented in neurodynamical systems. We discuss the Amari equation for a neural/dynamic field theory as a special case and show that the kernel construction problem is particularly ill-posed. We suggest a Tikhonov-Hebbian learning method as regularization technique and demonstrate its validity and robustness for basic examples of cognitive computations.

Stability criteria for the contextual emergence of macrostates in neural networks

More than thirty years ago, Amari and colleagues proposed a statistical framework for identifying structurally stable macrostates of neural networks from observations of their microstates. We compare their stochastic stability criterion with a deterministic stability criterion based on the ergodic theory of dynamical systems, recently proposed for the scheme of contextual emergence and applied to particular inter-level relations in neuroscience. Stochastic and deterministic stability criteria for macrostates rely on macro-level contexts, which make them sensitive to differences between different macro-levels.

Enhancing dominant modes in nonstationary time series by means of the symbolic resonance analysis

We present the symbolic resonance analysis (SRA) as a viable method for addressing the problem of enhancing a weakly dominant mode in a mixture of impulse responses obtained from a nonlinear dynamical system. We demonstrate this using results from a numerical simulation with Duffing oscillators in different domains of their parameter space, and by analyzing event-related brain potentials (ERPs) from a language processing experiment in German as a representative application. In this paradigm, the averaged ERPs exhibit an N400 followed by a sentence final negativity. Contemporary sentence processing models predict a late positivity (P600) as well. We show that the SRA is able to unveil the P600 evoked by the critical stimuli as a weakly dominant mode from the covering sentence final negativity.

Topographic voltage and coherence mapping of brain potentials by means of the symbolic resonance analysis

We apply the recently developed symbolic resonance analysis to electroencephalographic measurements of event-related brain potentials (ERPs) in a language processing experiment by using a three-symbol static encoding with varying thresholds for analyzing the ERP epochs, followed by a spin-flip transformation as a nonlinear filter. We compute an estimator of the signal-to-noise ratio (SNR) for the symbolic dynamics measuring the coherence of threshold-crossing events. Hence, we utilize the inherent noise of the EEG for sweeping the underlying ERP components beyond the encoding thresholds. Plotting the SNR computed within the time window of a particular ERP component (the N400) against the encoding thresholds, we find different resonance curves for the experimental conditions. The maximal differences of the SNR lead to the estimation of optimal encoding thresholds. We show that topographic brain maps of the optimal threshold voltages and of their associated coherence differences are able to dissociate the underlying physiological processes, while corresponding maps gained from the customary voltage averaging technique are unable to do so.

Is it Positive or Negative? On Determining ERP Components

In most experiments using event-related brain potentials (ERPs), there is a straightforward way to define--on theoretical grounds-which of the conditions tested is the experimental condition and which is the control condition. If, however, theoretical assumptions do not give sufficient and unambiguous information to decide this question, then the interpretation of an ERP effect becomes difficult, especially if one takes into account that certain effects can be both a positivity or a negativity on the basis of the morphology of the pattern as well as with respect to peak latency (regard for example, N400 and P345). Exemplified with an ERP experiment on language processing, we present such a critical case and offer a possible solution on the basis of nonlinear data analysis. We show that a generalized polarity histogram, the word statistics of symbolic dynamics, is in principle able to distinguish negative going ERP components from positive ones when an appropriate encoding strategy, the half wave encoding is employed. We propose statistical criteria which allow to determine ERP components on purely methodological grounds.