The CABB dataset: A multimodal corpus of communicative interactions for behavioural and neural analyses

We present a dataset of behavioural and fMRI observations acquired in the context of humans involved in multimodal referential communication. The dataset contains audio/video and motion-tracking recordings of face-to-face, task-based communicative interactions in Dutch, as well as behavioural and neural correlates of participants' representations of dialogue referents. Seventy-one pairs of unacquainted participants performed two interleaved interactional tasks in which they described and located 16 novel geometrical objects (i.e., Fribbles) yielding spontaneous interactions of about one hour. We share high-quality video (from three cameras), audio (from head-mounted microphones), and motion-tracking (Kinect) data, as well as speech transcripts of the interactions. Before and after engaging in the face-to-face communicative interactions, participants' individual representations of the 16 Fribbles were estimated. Behaviourally, participants provided a written description (one to three words) for each Fribble and positioned them along 29 independent conceptual dimensions (e.g., rounded, human, audible). Neurally, fMRI signal evoked by each Fribble was measured during a one-back working-memory task. To enable functional hyperalignment across participants, the dataset also includes fMRI measurements obtained during visual presentation of eight animated movies (35 min total). We present analyses for the various types of data demonstrating their quality and consistency with earlier research. Besides high-resolution multimodal interactional data, this dataset includes different correlates of communicative referents, obtained before and after face-to-face dialogue, allowing for novel investigations into the relation between communicative behaviours and the representational space shared by communicators. This unique combination of data can be used for research in neuroscience, psychology, linguistics, and beyond.

How Intractability Spans the Cognitive and Evolutionary Levels of Explanation

The challenge of explaining how cognition can be tractably realized is widely recognized. Classical rationality is thought to be intractable due to its assumptions of optimization and/or domain generality, and proposed solutions therefore drop one or both of these assumptions. We consider three such proposals: Resource-Rationality, the Adaptive Toolbox theory, and Massive Modularity. All three seek to ensure the tractability of cognition by shifting part of the explanation from the cognitive to the evolutionary level: Evolution is responsible for producing the tractable architecture. We consider the three proposals and show that, in each case, the intractability challenge is not thereby resolved, but only relocated from the cognitive level to the evolutionary level. We explain how non-classical accounts do not currently have the upper hand on the new playing field.

Formalizing Verbal Theories

Abstract. We present a tutorial for formalizing verbal theories of psychological phenomena – social or otherwise. The approach builds on concepts and tools from the mathematics of computation. We use intuitive examples and illustrate the intrinsic dialectical nature of the formalization process by presenting dialogues between two fictive characters, called Verbal and Formal. These characters’ conversations and thought experiments serve to highlight important lessons in theoretical modeling.

Quantifying free behaviour in an open field using k-motif approach

Quantification and parametrisation of movement are widely used in animal behavioural paradigms. In particular, free movement in controlled conditions (e.g., open field paradigm) is used as a “proxy for indices of baseline and drug-induced behavioural changes. However, the analysis of this is often time- and labour-intensive and existing algorithms do not always classify the behaviour correctly. Here, we propose a new approach to quantify behaviour in an unconstrained environment: searching for frequent patterns (k-motifs) in the time series representing the position of the subject over time. Validation of this method was performed using subchronic quinpirole-induced changes in open field experiment behaviours in rodents. Analysis of this data was performed using k-motifs as features to better classify subjects into experimental groups on the basis of behaviour in the open field. Our classifier using k-motifs gives as high as 94% accuracy in classifying repetitive behaviour versus controls which is a substantial improvement compared to currently available methods including using standard feature definitions (depending on the choice of feature set and classification strategy, accuracy up to 88%). Furthermore, visualisation of the movement/time patterns is highly predictive of these behaviours. By using machine learning, this can be applied to behavioural analysis across experimental paradigms.

Naturalism, tractability and the adaptive toolbox

Many compelling examples have recently been provided in which people can achieve impressive epistemic success, e.g. draw highly accurate inferences, by using simple heuristics and very little information. This is possible by taking advantage of the features of the environment. The examples suggest an easy and appealing naturalization of rationality: on the one hand, people clearly can apply simple heuristics, and on the other hand, they intuitively ought do so when this brings them high accuracy at little cost.. The 'ought-can' principle is satisfied, and rationality is meaningfully normative. We show, however, that this naturalization program is endangered by a computational wrinkle in the adaptation process taken to be responsible for this heuristics-based ('ecological') rationality: for the adaptation process to guarantee even minimal rationality, it requires astronomical computational resources, making the problem intractable. We consider various plausible auxiliary assumptions in attempt to remove this obstacle, and show that they do not succeed; intractability is a robust property of adaptation. We discuss the implications of our findings for the project of naturalizing rationality.

Neural mechanisms of communicative innovation

Human referential communication is often thought as coding-decoding a set of symbols, neglecting that establishing shared meanings requires a computational mechanism powerful enough to mutually negotiate them. Sharing the meaning of a novel symbol might rely on similar conceptual inferences across communicators or on statistical similarities in their sensorimotor behaviors. Using magnetoencephalography, we assess spectral, temporal, and spatial characteristics of neural activity evoked when people generate and understand novel shared symbols during live communicative interactions. Solving those communicative problems induced comparable changes in the spectral profile of neural activity of both communicators and addressees. This shared neuronal up-regulation was spatially localized to the right temporal lobe and the ventromedial prefrontal cortex and emerged already before the occurrence of a specific communicative problem. Communicative innovation relies on neuronal computations that are shared across generating and understanding novel shared symbols, operating over temporal scales independent from transient sensorimotor behavior.

Recipient design in human communication: simple heuristics or perspective taking?

Humans have a remarkable capacity for tuning their communicative behaviors to different addressees, a phenomenon also known as recipient design. It remains unclear how this tuning of communicative behavior is implemented during live human interactions. Classical theories of communication postulate that recipient design involves perspective taking, i.e., the communicator selects her behavior based on her hypotheses about beliefs and knowledge of the recipient. More recently, researchers have argued that perspective taking is computationally too costly to be a plausible mechanism in everyday human communication. These researchers propose that computationally simple mechanisms, or heuristics, are exploited to perform recipient design. Such heuristics may be able to adapt communicative behavior to an addressee with no consideration for the addressee's beliefs and knowledge. To test whether the simpler of the two mechanisms is sufficient for explaining the “how” of recipient design we studied communicators' behaviors in the context of a non-verbal communicative task (the Tacit Communication Game, TCG). We found that the specificity of the observed trial-by-trial adjustments made by communicators is parsimoniously explained by perspective taking, but not by simple heuristics. This finding is important as it suggests that humans do have a computationally efficient way of taking beliefs and knowledge of a recipient into account.

Intentional communication: computationally easy or difficult?

Human intentional communication is marked by its flexibility and context sensitivity. Hypothesized brain mechanisms can provide convincing and complete explanations of the human capacity for intentional communication only insofar as they can match the computational power required for displaying that capacity. It is thus of importance for cognitive neuroscience to know how computationally complex intentional communication actually is. Though the subject of considerable debate, the computational complexity of communication remains so far unknown. In this paper we defend the position that the computational complexity of communication is not a constant, as some views of communication seem to hold, but rather a function of situational factors. We present a methodology for studying and characterizing the computational complexity of communication under different situational constraints. We illustrate our methodology for a model of the problems solved by receivers and senders during a communicative exchange. This approach opens the way to a principled identification of putative model parameters that control cognitive processes supporting intentional communication.

Rapid typing of group A streptococci by the use of DNA amplification and non-radioactive allele-specific oligonucleotide probes

Because of the allelic variations within the M protein gene (emm gene) of group A streptococci, reliable typing of this important human pathogen can be accomplished by the use of emm gene-specific oligonucleotide probes. Two technical modifications (a reverse dot blot and a reverse line blot hybridization assay) of a novel approach for the type-specific identification of emm genes have been developed. Both procedures involved amplification of an emm gene by polymerase chain reaction. The non-radioactively labeled amplicon was subsequently hybridized to a membrane carrying an array of immobilized emm gene-specific oligonucleotide probes, thus allowing the simultaneous analysis of the gene polymorphism in a single hybridization reaction. The feasibility of these rapid and easy to perform methods was shown for the unequivocal identification of reference strains and clinical isolates belonging to 16 different M serotypes.