A job interview in the MRI scanner: How does indirectness affect addressees and overhearers?

tDCS of right-hemispheric Wernicke's area homologue affects contextual learning of novel lexicon

Numerous studies have shown robust evidence of the right hemisphere's involvement in the language function, for instance in the processing of intonation, grammar, word meanings, metaphors, etc. However, its role in lexicon acquisition remains obscure. We applied transcranial direct current stimulation (tDCS) over the right-hemispheric homologue of Wernicke's area to assess its putative involvement in the processing of different types of novel semantics. After receiving 15 min of anodal, cathodal, or sham (placebo) tDCS, three groups of healthy participants learnt novel concrete and abstract words in the context of short stories. Learning outcomes were assessed using a battery of tests immediately after this contextual learning session and 24 h later. As a result, an inhibitory effect of cathodal tDCS and a facilitatory effect of anodal tDCS were found for abstract word acquisition only. We also found a significant drop in task performance on the second day of the assessment for both word types in all the stimulation groups, suggesting no significant influence of tDCS on the post-learning consolidation of new memory traces. The results suggest an involvement of Wernicke's right-hemispheric counterpart in initial encoding (but not consolidation) of abstract semantics, which may be explained either by the right hemispheres direct role in processing lexical semantics or by an indirect impact of tDCS on contralateral (left-hemispheric) cortical areas through cross-callosal connections.

Conversational production and comprehension: fMRI-evidence reminiscent of but deviant from the classical Broca-Wernicke model

A key question in research on the neurobiology of language is to which extent the language production and comprehension systems share neural infrastructure, but this question has not been addressed in the context of conversation. We utilized a public fMRI dataset where 24 participants engaged in unscripted conversations with a confederate outside the scanner, via an audio-video link. We provide evidence indicating that the two systems share neural infrastructure in the left-lateralized perisylvian language network, but diverge regarding the level of activation in regions within the network. Activity in the left inferior frontal gyrus was stronger in production compared to comprehension, while comprehension showed stronger recruitment of the left anterior middle temporal gyrus and superior temporal sulcus, compared to production. Although our results are reminiscent of the classical Broca-Wernicke model, the anterior (rather than posterior) temporal activation is a notable difference from that model. This is one of the findings that may be a consequence of the conversational setting, another being that conversational production activated what we interpret as higher-level socio-pragmatic processes. In conclusion, we present evidence for partial overlap and functional asymmetry of the neural infrastructure of production and comprehension, in the above-mentioned frontal vs temporal regions during conversation.

Does the right temporo-parietal junction play a role in processing indirect speech acts? A transcranial magnetic stimulation study

In communication, much information is conveyed not explicitly but rather covertly, based on shared assumptions and common knowledge. For instance, when asked "Did you bring your cat to the vet?" a person could reply "It got hurt jumping down the table", thereby implicating that, indeed, the cat was brought to the vet. The assumption that getting hurt jumping down a table motivates a vet visit is tacitly attributed to the speaker by the listener, which implies Theory of Mind (ToM) processes. In the present study, we apply repetitive transcranial magnetic stimulation to the right temporo-parietal junction (rTPJ), a key brain region underlying ToM, with the aim to disrupt ToM processes necessary for language understanding. We then assess effects on the comprehension of indirect speech acts and their matched direct controls. In one set of conditions, the direct and indirect stimuli where not matched for speech act type, whereas, in the other, these were matched, therefore providing an unconfounded test case for in/directness. When indirect speech acts and direct controls were matched for speech act type (both statements), the indirect ones took longer to process both following sham and verum TMS. However, when the indirect and direct speech acts were not matched for communicative function (accept/decline offer vs. descriptive statement respectively), then a delay was detected for the indirect ones following sham TMS but, crucially, not following verum TMS. Additionally, TMS affected behavior in a ToM task. We therefore do not find evidence that the rTPJ is causally involved in comprehending of indirectness per se, but conclude that it could be involved instead in the processing of specific social communicative activity of rejecting of accepting offers, or to a combination of differing in/directness and communicative function. Our findings are consistent with the view that ToM processing in rTPJ is more important and/or more pronounced for offer acceptance/rejection than for descriptive answers.

The neurobiological map of theory of mind and pragmatic communication in autism

Children with autism often have difficulty with Theory of Mind (ToM), the ability to infer mental states, and pragmatic skills, the contextual use of language. Neuroimaging research suggests ToM and pragmatic skills overlap, as the ability to understand another's mental state is a prerequisite to interpersonal communication. To our knowledge, no study in the last decade has examined this overlap further. To assess the emerging consensus across neuroimaging studies of ToM and pragmatic skills in autism, we used coordinate-based activation likelihood estimation (ALE) analysis of 35 functional magnetic resonance imaging (MRI) studies (13 pragmatic skills, 22 ToM), resulting in a meta-analysis of 1,295 participants (647 autistic, 648 non-autistic) aged 7 to 49 years. Group difference analysis revealed decreased left inferior frontal gyrus (LIFG) activation in autistic participants during pragmatic skills tasks. For ToM tasks, we found reduced anterior cingulate cortex (ACC), medial prefrontal cortex (MPFC), and temporoparietal junction (TPJ) activation in autistic participants. Collectively, both ToM and pragmatic tasks showed activation in IFG and superior temporal gyrus (STG) and a reduction in left hemispheric activation in autistic participants. Overall, the findings underscore the cognitive and neural processing similarities between ToM and pragmatic skills, and their underlying neurobiological differences in autism.

Individual Differences in Emotion Attenuation Brought by Indirect Replies Is Related to Resting-State Brain Activity

During daily conversations, people prefer indirect replies in face-threatening situations. Existent studies have indicated that recipients tend to perceive the information conveyed by indirect replies as negative and emotion regions are engaged in indirect replies processing in face-threatening situations. In this study, we examined whether indirect replies can reduce recipients' experience of negative emotion and what are the underlying cerebral structures that may give rise to individual differences in the effectiveness of such replies in attenuating negative emotion. Behavior ratings and resting-stating functional magnetic resonance imaging (rs-fMRI) techniques were combined to explore these questions. We created dialogues expressing refusal or negative opinion with direct/indirect replies. Participants were asked to rate their emotional valence and arousal when they received such replies. The rating scores were used to correlate with spontaneous brain activity. Results showed that indirect replies indeed attenuated recipients' negative emotion experience. Moreover, the left caudate, the right anterior cingulate cortex (rACC), and the connectivity of rACC and left medial prefrontal cortex (lmPFC) were found to be positively correlated to individual differences in such emotion attenuation. Our findings provide direct empirical evidence for the face-saving function of indirect replies and reveal that the intrinsic brain activities of emotion network and theory of mind (ToM) network are related to individual differences in such emotion attenuation.

Non-literal language processing is jointly supported by the language and theory of mind networks: Evidence from a novel meta-analytic fMRI approach

Going beyond the literal meaning of language is key to communicative success. However, the mechanisms that support non-literal inferences remain debated. Using a novel meta-analytic approach, we evaluate the contribution of linguistic, social-cognitive, and executive mechanisms to non-literal interpretation. We identified 74 fMRI experiments (n = 1,430 participants) from 2001 to 2021 that contrasted non-literal language comprehension with a literal control condition, spanning ten phenomena (e.g., metaphor, irony, indirect speech). Applying the activation likelihood estimation approach to the 825 activation peaks yielded six left-lateralized clusters. We then evaluated the locations of both the individual-study peaks and the clusters against probabilistic functional atlases (cf. anatomical locations, as is typically done) for three candidate brain networks-the language-selective network (Fedorenko, Behr, & Kanwisher, 2011), which supports language processing, the Theory of Mind (ToM) network (Saxe & Kanwisher, 2003), which supports social inferences, and the domain-general Multiple-Demand (MD) network (Duncan, 2010), which supports executive control. These atlases were created by overlaying individual activation maps of participants who performed robust and extensively validated 'localizer' tasks that selectively target each network in question (n = 806 for language; n = 198 for ToM; n = 691 for MD). We found that both the individual-study peaks and the ALE clusters fell primarily within the language network and the ToM network. These results suggest that non-literal processing is supported by both i) mechanisms that process literal linguistic meaning, and ii) mechanisms that support general social inference. They thus undermine a strong divide between literal and non-literal aspects of language and challenge the claim that non-literal processing requires additional executive resources.

Face2face: advancing the science of social interaction

Face-to-face interaction is core to human sociality and its evolution, and provides the environment in which most of human communication occurs. Research into the full complexities that define face-to-face interaction requires a multi-disciplinary, multi-level approach, illuminating from different perspectives how we and other species interact. This special issue showcases a wide range of approaches, bringing together detailed studies of naturalistic social-interactional behaviour with larger scale analyses for generalization, and investigations of socially contextualized cognitive and neural processes that underpin the behaviour we observe. We suggest that this integrative approach will allow us to propel forwards the science of face-to-face interaction by leading us to new paradigms and novel, more ecologically grounded and comprehensive insights into how we interact with one another and with artificial agents, how differences in psychological profiles might affect interaction, and how the capacity to socially interact develops and has evolved in the human and other species. This theme issue makes a first step into this direction, with the aim to break down disciplinary boundaries and emphasizing the value of illuminating the many facets of face-to-face interaction. This article is part of a discussion meeting issue 'Face2face: advancing the science of social interaction'.

Linguistic signs in action: The neuropragmatics of speech acts

What makes human communication exceptional is the ability to grasp speaker's intentions beyond what is said verbally. How the brain processes communicative functions is one of the central concerns of the neurobiology of language and pragmatics. Linguistic-pragmatic theories define these functions as speech acts, and various pragmatic traits characterise them at the levels of propositional content, action sequence structure, related commitments and social aspects. Here I discuss recent neurocognitive studies, which have shown that the use of identical linguistic signs in conveying different communicative functions elicits distinct and ultra-rapid neural responses. Interestingly, cortical areas show differential involvement underlying various pragmatic features related to theory-of-mind, emotion and action for specific speech acts expressed with the same utterances. Drawing on a neurocognitive model, I posit that understanding speech acts involves the expectation of typical partner follow-up actions and that this predictive knowledge is immediately reflected in mind and brain.

“We Will Let You Know”: An Assessment of Digital vs. Face-to-Face Job Interviews via EEG Connectivity Analysis

We focused on job interviews as critical examples of complex social interaction in organizational contexts. We aimed at investigating the effect of face-to-face vs. computer-mediated interaction, of role (candidate, recruiter), and of the interview phase (introductory, attitudinal, technical, conclusive) on intra-brain and inter-brain connectivity measures and autonomic synchronization. Twenty expert recruiters and potential candidates took part in a hyperscanning investigation. Namely, electroencephalography (delta, theta, alpha, beta bands) and autonomic (skin-conductance, heart-rate) data were collected in candidate-recruiter dyads during a simulated job interview and then concurrently analyzed. Analyses highlighted a link between face-to-face condition and greater intra-/inter-brain connectivity indices in delta and theta bands. Furthermore, intra-brain and inter-brain connectivity measures were higher for delta and theta bands in the final interview phases compared to the first ones. Consistently, autonomic synchronization was higher during the final interview phases, specifically in the face-to-face condition. Finally, recruiters showed higher intra-brain connectivity in the delta range over frontal and temporoparietal areas, while candidates showed higher intra-brain connectivity in the theta range over frontal areas. Findings highlight the value of hyperscanning investigations in exploring social attunement in professional contexts and hint at their potential to foster neuroscience-informed practices in human resource management processes.

Social signalling as a framework for second-person neuroscience

Despite the recent increase in second-person neuroscience research, it is still hard to understand which neurocognitive mechanisms underlie real-time social behaviours. Here, we propose that social signalling can help us understand social interactions both at the single- and two-brain level in terms of social signal exchanges between senders and receivers. First, we show how subtle manipulations of being watched provide an important tool to dissect meaningful social signals. We then focus on how social signalling can help us build testable hypotheses for second-person neuroscience with the example of imitation and gaze behaviour. Finally, we suggest that linking neural activity to specific social signals will be key to fully understand the neurocognitive systems engaged during face-to-face interactions.

"We will be in touch". A neuroscientific assessment of remote vs. face-to-face job interviews via EEG hyperscanning

In the last decades, improving remote communications in companies has been a compelling issue. With the outspread of SARS-CoV-2 pandemic, this phenomenon has undergone an acceleration. Despite this, little to no research, considering neurocognitive and emotional systems, was conducted on job interview, a critical organizational phase which significantly contributes to a company long-term success.In this study, we aimed at exploring the emotional and cognitive processes related to different phases of a job interview (introductory, attitudinal, technical and conclusion), when considering two conditions: face-to-face and remote, by simultaneously gathering EEG (frequency bands: alpha, beta, delta, and theta) and autonomic data (skin-conductance-level, SCL, skin-conductance-response, SCR, and heart rate, HR) in both candidates and recruiters. Data highlighted a generalized alpha desynchronization during the job interview interaction. Recruiters showed increased frontal theta activity, which is connected to socio-emotional situations and emotional processing. In addition, results showed how face-to-face condition is related to increased SCL and theta power in the central-brain area, associated with learning processes, via the mid-brain dopamine system and the anterior cingulate cortex. Furthermore, we found higher HR in the candidates. Present results call to re-examine the impact of information-technology in the organization, opening to translational opportunities.

Face-to-face vs. remote digital settings in job assessment interviews: A multilevel hyperscanning protocol for the investigation of interpersonal attunement

The digitalization process for organizations, which was inevitably accelerated by the COVID-19 pandemic, raises relevant challenges for Human Resource Management (HRM) because every technological implementation has a certain impact on human beings. Between many organizational HRM practices, recruitment and assessment interviews represent a significant moment where a social interaction provides the context for evaluating candidates’ skills. It is therefore relevant to investigate how different interaction frames and relational conditions affect such task, with a specific focus on the differences between face-to-face (FTF) and remote computer-mediated (RCM) interaction settings. In particular, the possibility of qualifying and quantifying the mechanisms shaping the efficiency of interaction in the recruiter-candidate dyad—i.e. interpersonal attunement—is potentially insightful. We here present a neuroscientific protocol aimed at elucidating the impact of FTF vs. RCM modalities on social dynamics within assessment interviews. Specifically, the hyperscanning approach, understood as the concurrent recording and integrated analysis of behavioural-physiological responses of interacting agents, will be used to evaluate recruiter-candidate dyads while they are involved in either FTF or RCM conditions. Specifically, the protocol has been designed to collect self-report, oculometric, autonomic (electrodermal activity, heart rate, heart rate variability), and neurophysiological (electroencephalography) metrics from both inter-agents to explore the perceived quality of the interaction, automatic visual-attentional patterns of inter-agents, as well as their cognitive workload and emotional engagement. The proposed protocol will provide a theoretical evidence-based framework to assess possible differences between FTF vs. RMC settings in complex social interactions, with a specific focus on job interviews.

EXPRESS: An eye-tracking investigation of the cognitive processes involved in the comprehension of simple and complex communicative acts

Indirect speech acts communicate more than their literal meaning, and their comprehension relies on the listener's ability to draw the appropriate inferences in a given context. We used eye-tracking to investigate the cognitive processing involved in the comprehension of simple (direct) and complex (unconventional indirect) communicative acts, a more general distinction that applies not only to sincere speech acts, but also to irony and deceit. We recorded the eye movements of 40 participants while they read 60 stories (20 sincere, 20 deceitful, 20 ironic) consisting of a context and a target answer. For each story, we created two different contexts so that the same identical target answer was a simple (direct) and a complex (unconventional indirect) communicative act, respectively. We also assessed the indirectness of simple and complex communicative acts, as well as participants' working memory (WM) and Theory of Mind (ToM). Eye-pattern analysis showed that complex communicative acts were more difficult to understand than simple acts; differences between simple and complex acts held for all the pragmatic phenomena investigated, though processing differences were greater for sincere acts than for irony and deceit. We found a role of indirectness and ToM in the pragmatic processing of simple and complex acts, whereas the role of WM was modest. The present findings underscore the importance of adopting an encompassing theory that can account for different types of indirect speech acts, such as sincere, deceitful and ironic acts; they also suggest the importance of assessing individual differences in inferential and cognitive abilities.

OUP accepted manuscript

During conversations, speech prosody provides important clues about the speaker’s communicative intentions. In many languages, a rising vocal pitch at the end of a sentence typically expresses a question function, whereas a falling pitch suggests a statement. Here, the neurophysiological basis of intonation and speech act understanding were investigated with high-density electroencephalography (EEG) to determine whether prosodic features are reflected at the neurophysiological level. Already approximately 100 ms after the sentence-final word differing in prosody, questions, and statements expressed with the same sentences led to different neurophysiological activity recorded in the event-related potential. Interestingly, low-pass filtered sentences and acoustically matched nonvocal musical signals failed to show any neurophysiological dissociations, thus suggesting that the physical intonation alone cannot explain this modulation. Our results show rapid neurophysiological indexes of prosodic communicative information processing that emerge only when pragmatic and lexico-semantic information are fully expressed. The early enhancement of question-related activity compared with statements was due to sources in the articulatory-motor region, which may reflect the richer action knowledge immanent to questions, namely the expectation of the partner action of answering the question. The present findings demonstrate a neurophysiological correlate of prosodic communicative information processing, which enables humans to rapidly detect and understand speaker intentions in linguistic interactions.

The Neural Substrate of Speech Act Recognition

Pragmatic competence demands linguistic, but also communicative, social and cognitive competence. Successful use of language in social interaction requires mutual understanding of the speaker's intentions; without it, a conversation cannot proceed. The term speech act refers to what a speaker intends to accomplish when saying something. The purpose of this study was to contribute to the identification of the neural substrate of speech act recognition and to the characterization of the cognitive processes that may be involved. The recognition of speech acts resulted in greater activation of frontal regions, precuneus and posterior cingulate gyrus. From all cognitive and behavioral measures obtained, only the scores in mental flexibility predicted the change in blood oxygen level dependent (BOLD) signal in the precuneus. These results, support the idea that speech act recognition requires the inference of intention, executive functions, including memory and entails the activation of areas of social cognition that participate in several brain networks i.e., the Intention Processing, the Default Mode and Theory of Mind networks, and areas involved in planning and guiding behavior.

Brain signatures predict communicative function of speech production in interaction

People normally know what they want to communicate before they start speaking. However, brain indicators of communication are typically observed only after speech act onset, and it is unclear when any anticipatory brain activity prior to speaking might first emerge, along with the communicative intentions it possibly reflects. Here, we investigated brain activity prior to the production of different speech act types, request and naming actions performed by uttering single words embedded into language games with a partner, similar to natural communication. Starting ca. 600 msec before speech onset, an event-related potential maximal at fronto-central electrodes, which resembled the Readiness Potential, was larger when preparing requests compared to naming actions. Analysis of the cortical sources of this anticipatory brain potential suggests a relatively stronger involvement of fronto-central motor regions for requests, which may reflect the speaker's expectation of the partner actions typically following requests, e.g., the handing over of a requested object. Our results indicate that different neuronal circuits underlying the processing of different speech act types activate already before speaking. Results are discussed in light of previous work addressing the neural basis of speech act understanding and predictive brain indexes of language comprehension.

Emoji can facilitate recognition of conveyed indirect meaning

In face-to-face communication there are multiple paralinguistic and gestural features that facilitate recognition of a speaker’s intended meaning, features that are lacking when people communicate digitally (e.g., texting). As a result, substitutes have emerged (expressive punctuation, capitalization, etc.) to facilitate communication in these situations. However, little is known about the comprehension processes involved in digital communication. In this research we examined the role of emoji in the comprehension of face-threatening, indirect replies. Participants in two experiments read question–reply sequences and then judged the accuracy of interpretations of the replies. On critical trials the reply violated the relation maxim and conveyed a negative, face-threatening response. On one-third of the trials the reply contained only text, on one-third of the trials the reply contained text and an emoji, and on one-third of the trials the reply contained only an emoji. When the question requested potentially negative information about one of the interactants (disclosures and opinions), participants were more likely to endorse the indirect meaning of the reply, and did so faster, when the reply contained an emoji than when it did not. This effect did not occur when the question was a request for action, a more conventional type of indirect reply. Overall, then, this research demonstrates that emoji can sometimes facilitate the comprehension of meaning. Future research is needed to examine the boundary conditions for this effect.

The neurobiology of language beyond single-word processing

In this Review, I propose a multiple-network view for the neurobiological basis of distinctly human language skills. A much more complex picture of interacting brain areas emerges than in the classical neurobiological model of language. This is because using language is more than single-word processing, and much goes on beyond the information given in the acoustic or orthographic tokens that enter primary sensory cortices. This requires the involvement of multiple networks with functionally nonoverlapping contributions.

Uncovering cortical activations of discourse comprehension and their overlaps with common large-scale neural networks

We conducted a meta-analysis of 78 task-based functional magnetic resonance imaging (fMRI) studies (1976 total participants) to reveal underlying brain activations and their overlap with large-scale neural networks in the brain during general discourse comprehension and its sub-processes. We found that discourse comprehension involved a neural system consisting of widely distributed brain regions that comprised not only the bilateral perisylvian language zones, but also regions in the superior and medial frontal cortex and the medial temporal lobe. Moreover, this neural system can be categorized into several sub-systems representing various sub-processes of discourse comprehension, with the left inferior frontal gyrus and middle temporal gyrus serving as core regions across all sub-processes. At a large-scale network level, we found that discourse comprehension relied most heavily on the default network, particularly on its dorsal medial subsystem. The pattern associated with large-scale network cooperation varied according to the respective sub-processes required. Our results reveal the functional dissociation within the discourse comprehension neural system and highlight the flexible involvements of large-scale networks.

Brains in dialogue: decoding neural preparation of speaking to a conversational partner

In dialogue, language processing is adapted to the conversational partner. We hypothesize that the brain facilitates partner-adapted language processing through preparatory neural configurations (task sets) that are tailored to the conversational partner. In this experiment, we measured neural activity with functional magnetic resonance imaging (fMRI) while healthy participants in the scanner (a) engaged in a verbal communication task with a conversational partner outside of the scanner, or (b) spoke outside of a conversational context (to test the microphone). Using multivariate searchlight analysis, we identify cortical regions that represent information on whether speakers plan to speak to a conversational partner or without having a partner. Most notably a region that has been associated with processing social-affective information and perspective taking, the ventromedial prefrontal cortex, as well as regions that have been associated with prospective task representation, the bilateral ventral prefrontal cortex, are involved in encoding the speaking condition. Our results suggest that speakers prepare, in advance of speaking, for the social context in which they will speak.

Addressee Identity and Morphosyntactic Processing in Basque Allocutive Agreement

Information about interlocutor identity is pragmatic in nature and has traditionally been distinguished from explicitly coded linguistic information, including mophosyntax. Study of speaker identity in language processing has questioned this distinction, but addressee identity has been less considered. We used Basque to explore how addressee identity is processed during morphosyntactic analysis. In the familiar register hika, Basque has obligatory allocutive agreement, where verbal morphology represents the gender of a non-argument addressee. We manipulated the gender of the allocutive verb and the congruence of addressee gender in conversations between two interlocutors. Items with person agreement manipulations were included as a control comparison. Basque speakers familiar with hika completed speeded acceptability judgments and unspeeded, offline naturalness ratings for each conversation. Results showed a main effect of addressee identity congruence for naturalness ratings, but there was no main effect for addressee identity congruence for reaction times or accuracy in the acceptability judgment. Interactions and correlations with biographical data showed that the effect of congruence was modulated by the gender of the allocutive verb and that hika proficiency was related to participants' performance for the acceptability judgment. These results show an interaction between morphosyntactic and pragmatic information and are the first experimental data of allocutive processing. In comparison, clear effects were seen for the person agreement condition, indicating that person disagreement is more disruptive to processing than addressee identity incongruence. This study has implications for investigation of the role of extralinguistic information in morphosyntactic processing, and suggests that not all such information plays an equal role.

Effects of contextual relevance on pragmatic inference during conversation: An fMRI study

Contextual relevance, which is vital for understanding conversational implicatures (CI), engages both the frontal-temporal language and theory-of-mind networks. Here we investigate how contextual relevance affects CI processing and regulates the connectivity between CI-processing-related brain regions. Participants listened to dialogues in which the level of contextual relevance to dialogue-final utterance (reply) was manipulated. This utterance was either direct, indirect but relevant, irrelevant with contextual hint, or irrelevant with no contextual hint. Results indicated that compared with direct replies, indirect replies showed increased activations in bilateral IFG, bilateral MTG, bilateral TPJ, dmPFC, and precuneus, and increased connectivity between rTPJ/dmPFC and both IFG and MTG. Moreover, irrelevant replies activated right MTG along an anterior-posterior gradient as a function of the level of irrelevance. Our study provides novel evidence concerning how the language and theory-of-mind networks interact for pragmatic inference and how the processing of CI is modulated by level of contextual relevance.

Temporal Preparation for Speaking in Question-Answer Sequences

In every-day conversations, the gap between turns of conversational partners is most frequently between 0 and 200 ms. We were interested how speakers achieve such fast transitions. We designed an experiment in which participants listened to pre-recorded questions about images presented on a screen and were asked to answer these questions. We tested whether speakers already prepare their answers while they listen to questions and whether they can prepare for the time of articulation by anticipating when questions end. In the experiment, it was possible to guess the answer at the beginning of the questions in half of the experimental trials. We also manipulated whether it was possible to predict the length of the last word of the questions. The results suggest when listeners know the answer early they start speech production already during the questions. Speakers can also time when to speak by predicting the duration of turns. These temporal predictions can be based on the length of anticipated words and on the overall probability of turn durations.

The N400 Effect during Speaker-Switch-Towards a Conversational Approach of Measuring Neural Correlates of Language

Language occurs naturally in conversations. However, the study of the neural underpinnings of language has mainly taken place in single individuals using controlled language material. The interactive elements of a conversation (e.g., turn-taking) are often not part of neurolinguistic setups. The prime reason is the difficulty to combine open unrestricted conversations with the requirements of neuroimaging. It is necessary to find a trade-off between the naturalness of a conversation and the restrictions imposed by neuroscientific methods to allow for ecologically more valid studies. Here, we make an attempt to study the effects of a conversational element, namely turn-taking, on linguistic neural correlates, specifically the N400 effect. We focus on the physiological aspect of turn-taking, the speaker-switch, and its effect on the detectability of the N400 effect. The N400 event-related potential reflects expectation violations in a semantic context; the N400 effect describes the difference of the N400 amplitude between semantically expected and unexpected items. Sentences with semantically congruent and incongruent final words were presented in two turn-taking modes: (1) reading aloud first part of the sentence and listening to speaker-switch for the final word, and (2) listening to first part of the sentence and speaker-switch for the final word. A significant N400 effect was found for both turn-taking modes, which was not influenced by the mode itself. However, the mode significantly affected the P200, which was increased for the reading aloud mode compared to the listening mode. Our results show that an N400 effect can be detected during a speaker-switch. Speech articulation (reading aloud) before the analyzed sentence fragment did also not impede the N400 effect detection for the final word. The speaker-switch, however, seems to influence earlier components of the electroencephalogram, related to processing of salient stimuli. We conclude that the N400 can effectively be used to study neural correlates of language in conversational approaches including speaker-switches.