Neural correlates of intentional communication

You talkin' to me? Communicative talker gaze activates left-lateralized superior temporal cortex during perception of degraded speech

Neuroimaging studies of speech perception have consistently indicated a left-hemisphere dominance in the temporal lobes’ responses to intelligible auditory speech signals (McGettigan and Scott, 2012). However, there are important communicative cues that cannot be extracted from auditory signals alone, including the direction of the talker's gaze. Previous work has implicated the superior temporal cortices in processing gaze direction, with evidence for predominantly right-lateralized responses (Carlin & Calder, 2013). The aim of the current study was to investigate whether the lateralization of responses to talker gaze differs in an auditory communicative context. Participants in a functional MRI experiment watched and listened to videos of spoken sentences in which the auditory intelligibility and talker gaze direction were manipulated factorially. We observed a left-dominant temporal lobe sensitivity to the talker's gaze direction, in which the left anterior superior temporal sulcus/gyrus and temporal pole showed an enhanced response to direct gaze – further investigation revealed that this pattern of lateralization was modulated by auditory intelligibility. Our results suggest flexibility in the distribution of neural responses to social cues in the face within the context of a challenging speech perception task.

•

Talker gaze is an important social cue during speech comprehension.

•

Neural responses to gaze were measured during perception of degraded sentences.

•

Gaze direction modulated activation in left-lateralized superior temporal cortex.

•

Left lateralization became stronger when speech was less intelligible.

•

Results suggest task-dependent flexibility in cortical responses to gaze.

You cannot speak and listen at the same time: a probabilistic model of turn-taking

Turn-taking is a preverbal skill whose mastering constitutes an important precondition for many social interactions and joint actions. However, the cognitive mechanisms supporting turn-taking abilities are still poorly understood. Here, we propose a computational analysis of turn-taking in terms of two general mechanisms supporting joint actions: action prediction (e.g., recognizing the interlocutor's message and predicting the end of turn) and signaling (e.g., modifying one's own speech to make it more predictable and discriminable). We test the hypothesis that in a simulated conversational scenario dyads using these two mechanisms can recognize the utterances of their co-actors faster, which in turn permits them to give and take turns more efficiently. Furthermore, we discuss how turn-taking dynamics depend on the fact that agents cannot simultaneously use their internal models for both action (or messages) prediction and production, as these have different requirements-or, in other words, they cannot speak and listen at the same time with the same level of accuracy. Our results provide a computational-level characterization of turn-taking in terms of cognitive mechanisms of action prediction and signaling that are shared across various interaction and joint action domains.

Brain basis of communicative actions in language

Although language is a key tool for communication in social interaction, most studies in the neuroscience of language have focused on language structures such as words and sentences. Here, the neural correlates of speech acts, that is, the actions performed by using language, were investigated with functional magnetic resonance imaging (fMRI). Participants were shown videos, in which the same critical utterances were used in different communicative contexts, to Name objects, or to Request them from communication partners. Understanding of critical utterances as Requests was accompanied by activation in bilateral premotor, left inferior frontal and temporo-parietal cortical areas known to support action-related and social interactive knowledge. Naming, however, activated the left angular gyrus implicated in linking information about word forms and related reference objects mentioned in critical utterances. These findings show that understanding of utterances as different communicative actions is reflected in distinct brain activation patterns, and thus suggest different neural substrates for different speech act types.

Animal communication

Animal communication is first and foremost about signal transmission and aims to understand how communication occurs. It is a field that has contributed to and been inspired by other fields, from information technology to neuroscience, in finding ever better methods to eavesdrop on the actual 'message' that forms the basis of communication. Much of this review deals with vocal communication as an example of the questions that research on communication has tried to answer and it provides an historical overview of the theoretical arguments proposed. Topics covered include signal transmission in different environments and different species, referential signaling, and intentionality. The contention is that animal communication may reveal significant thought processes that enable some individuals in a small number of species so far investigated to anticipate what conspecifics might do, although some researchers think of such behavior as adaptive or worth dismissing as anthropomorphizing. The review further points out that some species are more likely than others to develop more complex communication patterns. It is a matter of asking how animals categorize their world and which concepts require cognitive processes and which are adaptive. The review concludes with questions of life history, social learning, and decision making, all criteria that have remained relatively unexplored in communication research. Long-lived, cooperative social animals have so far offered especially exciting prospects for investigation. There are ample opportunities and now very advanced technologies as well to tap further into expressions of memory of signals, be they vocal or expressed in other modalities. WIREs Cogn Sci 2014, 5:661-677. doi: 10.1002/wcs.1321 For further resources related to this article, please visit the WIREs website.

CONFLICT OF INTEREST

The author has declared no conflicts of interest for this article.

Neural dynamics of speech act comprehension: an MEG study of naming and requesting

The neurobiological basis and temporal dynamics of communicative language processing pose important yet unresolved questions. It has previously been suggested that comprehension of the communicative function of an utterance, i.e. the so-called speech act, is supported by an ensemble of neural networks, comprising lexico-semantic, action and mirror neuron as well as theory of mind circuits, all activated in concert. It has also been demonstrated that recognition of the speech act type occurs extremely rapidly. These findings however, were obtained in experiments with insufficient spatio-temporal resolution, thus possibly concealing important facets of the neural dynamics of the speech act comprehension process. Here, we used magnetoencephalography to investigate the comprehension of Naming and Request actions performed with utterances controlled for physical features, psycholinguistic properties and the probability of occurrence in variable contexts. The results show that different communicative actions are underpinned by a dynamic neural network, which differentiates between speech act types very early after the speech act onset. Within 50–90 ms, Requests engaged mirror-neuron action-comprehension systems in sensorimotor cortex, possibly for processing action knowledge and intentions. Still, within the first 200 ms of stimulus onset (100–150 ms), Naming activated brain areas involved in referential semantic retrieval. Subsequently (200–300 ms), theory of mind and mentalising circuits were activated in medial prefrontal and temporo-parietal areas, possibly indexing processing of intentions and assumptions of both communication partners. This cascade of stages of processing information about actions and intentions, referential semantics, and theory of mind may underlie dynamic and interactive speech act comprehension.

Human sensorimotor communication: a theory of signaling in online social interactions

Although the importance of communication is recognized in several disciplines, it is rarely studied in the context of online social interactions and joint actions. During online joint actions, language and gesture are often insufficient and humans typically use non-verbal, sensorimotor forms of communication to send coordination signals. For example, when playing volleyball, an athlete can exaggerate her movements to signal her intentions to her teammates (say, a pass to the right) or to feint an adversary. Similarly, a person who is transporting a table together with a co-actor can push the table in a certain direction to signal where and when he intends to place it. Other examples of "signaling" are over-articulating in noisy environments and over-emphasizing vowels in child-directed speech. In all these examples, humans intentionally modify their action kinematics to make their goals easier to disambiguate. At the moment no formal theory exists of these forms of sensorimotor communication and signaling. We present one such theory that describes signaling as a combination of a pragmatic and a communicative action, and explains how it simplifies coordination in online social interactions. We cast signaling within a "joint action optimization" framework in which co-actors optimize the success of their interaction and joint goals rather than only their part of the joint action. The decision of whether and how much to signal requires solving a trade-off between the costs of modifying one's behavior and the benefits in terms of interaction success. Signaling is thus an intentional strategy that supports social interactions; it acts in concert with automatic mechanisms of resonance, prediction, and imitation, especially when the context makes actions and intentions ambiguous and difficult to read. Our theory suggests that communication dynamics should be studied within theories of coordination and interaction rather than only in terms of the maximization of information transmission.

Neural mechanisms of communicative innovation

Human referential communication is often thought as coding-decoding a set of symbols, neglecting that establishing shared meanings requires a computational mechanism powerful enough to mutually negotiate them. Sharing the meaning of a novel symbol might rely on similar conceptual inferences across communicators or on statistical similarities in their sensorimotor behaviors. Using magnetoencephalography, we assess spectral, temporal, and spatial characteristics of neural activity evoked when people generate and understand novel shared symbols during live communicative interactions. Solving those communicative problems induced comparable changes in the spectral profile of neural activity of both communicators and addressees. This shared neuronal up-regulation was spatially localized to the right temporal lobe and the ventromedial prefrontal cortex and emerged already before the occurrence of a specific communicative problem. Communicative innovation relies on neuronal computations that are shared across generating and understanding novel shared symbols, operating over temporal scales independent from transient sensorimotor behavior.

On the antiquity of language: the reinterpretation of Neandertal linguistic capacities and its consequences

It is usually assumed that modern language is a recent phenomenon, coinciding with the emergence of modern humans themselves. Many assume as well that this is the result of a single, sudden mutation giving rise to the full "modern package." However, we argue here that recognizably modern language is likely an ancient feature of our genus pre-dating at least the common ancestor of modern humans and Neandertals about half a million years ago. To this end, we adduce a broad range of evidence from linguistics, genetics, paleontology, and archaeology clearly suggesting that Neandertals shared with us something like modern speech and language. This reassessment of the antiquity of modern language, from the usually quoted 50,000-100,000 years to half a million years, has profound consequences for our understanding of our own evolution in general and especially for the sciences of speech and language. As such, it argues against a saltationist scenario for the evolution of language, and toward a gradual process of culture-gene co-evolution extending to the present day. Another consequence is that the present-day linguistic diversity might better reflect the properties of the design space for language and not just the vagaries of history, and could also contain traces of the languages spoken by other human forms such as the Neandertals.

Early and parallel processing of pragmatic and semantic information in speech acts: neurophysiological evidence

Although language is a tool for communication, most research in the neuroscience of language has focused on studying words and sentences, while little is known about the brain mechanisms of speech acts, or communicative functions, for which words and sentences are used as tools. Here the neural processing of two types of speech acts, Naming and Requesting, was addressed using the time-resolved event-related potential (ERP) technique. The brain responses for Naming and Request diverged as early as ~120 ms after the onset of the critical words, at the same time as, or even before, the earliest brain manifestations of semantic word properties could be detected. Request-evoked potentials were generally larger in amplitude than those for Naming. The use of identical words in closely matched settings for both speech acts rules out explanation of the difference in terms of phonological, lexical, semantic properties, or word expectancy. The cortical sources underlying the ERP enhancement for Requests were found in the fronto-central cortex, consistent with the activation of action knowledge, as well as in the right temporo-parietal junction (TPJ), possibly reflecting additional implications of speech acts for social interaction and theory of mind. These results provide the first evidence for surprisingly early access to pragmatic and social interactive knowledge, which possibly occurs in parallel with other types of linguistic processing, and thus supports the near-simultaneous access to different subtypes of psycholinguistic information.

Developmental Differences in the Control of Action Selection by Social Information

Our everyday actions are often performed in the context of a social interaction. We previously showed that, in adults, selecting an action on the basis of either social or symbolic cues was associated with activations in the fronto-parietal cognitive control network, whereas the presence and use of social versus symbolic cues was in addition associated with activations in the temporal and medial prefrontal cortex (MPFC) social brain network. Here we investigated developmental changes in these two networks. Fourteen adults (21–30 years of age) and 14 adolescents (11–16 years) followed instructions to move objects in a set of shelves. Interpretation of the instructions was conditional on the point of view of a visible “director” or the meaning of a symbolic cue (Director Present vs. Director Absent) and the number of potential referent objects in the shelves (3-object vs. 1-object). 3-object trials elicited increased fronto-parietal and temporal activations, with greater left lateral prefrontal cortex and parietal activations in adults than adolescents. Social versus symbolic information led to activations in superior dorsal MPFC, precuneus, and along the superior/middle temporal sulci. Both dorsal MPFC and left temporal clusters exhibited a Director × Object interaction, with greater activation when participants needed to consider the directors' viewpoints. This effect differed with age in dorsal MPFC. Adolescents showed greater activation whenever social information was present, whereas adults showed greater activation only when the directors' viewpoints were relevant to task performance. This study thus shows developmental differences in domain-general and domain-specific PFC activations associated with action selection in a social interaction context.