Situating language in a minimal social context: how seeing a picture of the speaker's face affects language comprehension

The bigger your pupils, the better my comprehension: an ERP study of how pupil size and gaze of the speaker affect syntactic processing

Gaze direction and pupil dilation play a critical role in communication and social interaction due to their ability to redirect and capture our attention and their relevance for emotional information. The present study aimed to explore whether the pupil size and gaze direction of the speaker affect language comprehension. Participants listened to sentences that could be correct or contain a syntactic anomaly, while the static face of a speaker was manipulated in terms of gaze direction (direct, averted) and pupil size (mydriasis, miosis). Left anterior negativity (LAN) and P600 linguistic event-related potential components were observed in response to syntactic anomalies across all conditions. The speaker's gaze did not impact syntactic comprehension. However, the amplitude of the LAN component for mydriasis (dilated pupil) was larger than for miosis (constricted pupil) condition. Larger pupils are generally associated with care, trust, interest, and attention, which might facilitate syntactic processing at early automatic stages. The result also supports the permeable and context-dependent nature of syntax. Previous studies also support an automatic nature of syntax (fast and efficient), which combined with the permeability to relevant sources of communicative information, such as pupil size and emotions, is highly adaptive for language comprehension and social interaction.

The Self-reference Effect Can Modulate Language Syntactic Processing Even Without Explicit Awareness: An Electroencephalography Study

Although it is well established that self-related information can rapidly capture our attention and bias cognitive functioning, whether this self-bias can affect language processing remains largely unknown. In addition, there is an ongoing debate as to the functional independence of language processes, notably regarding the syntactic domain. Hence, this study investigated the influence of self-related content on syntactic speech processing. Participants listened to sentences that could contain morphosyntactic anomalies while the masked face identity (self, friend, or unknown faces) was presented for 16 msec preceding the critical word. The language-related ERP components (left anterior negativity [LAN] and P600) appeared for all identity conditions. However, the largest LAN effect followed by a reduced P600 effect was observed for self-faces, whereas a larger LAN with no reduction of the P600 was found for friend faces compared with unknown faces. These data suggest that both early and late syntactic processes can be modulated by self-related content. In addition, alpha power was more suppressed over the left inferior frontal gyrus only when self-faces appeared before the critical word. This may reflect higher semantic demands concomitant to early syntactic operations (around 150-550 msec). Our data also provide further evidence of self-specific response, as reflected by the N250 component. Collectively, our results suggest that identity-related information is rapidly decoded from facial stimuli and may impact core linguistic processes, supporting an interactive view of syntactic processing. This study provides evidence that the self-reference effect can be extended to syntactic processing.

Corneal reflections and skin contrast yield better memory of human and virtual faces

Virtual faces have been found to be rated less human-like and remembered worse than photographic images of humans. What it is in virtual faces that yields reduced memory has so far remained unclear. The current study investigated face memory in the context of virtual agent faces and human faces, real and manipulated, considering two factors of predicted influence, i.e., corneal reflections and skin contrast. Corneal reflections referred to the bright points in each eye that occur when the ambient light reflects from the surface of the cornea. Skin contrast referred to the degree to which skin surface is rough versus smooth. We conducted two memory experiments, one with high-quality virtual agent faces (Experiment 1) and the other with the photographs of human faces that were manipulated (Experiment 2). Experiment 1 showed better memory for virtual faces with increased corneal reflections and skin contrast (rougher rather than smoother skin). Experiment 2 replicated these findings, showing that removing the corneal reflections and smoothening the skin reduced memory recognition of manipulated faces, with a stronger effect exerted by the eyes than the skin. This study highlights specific features of the eyes and skin that can help explain memory discrepancies between real and virtual faces and in turn elucidates the factors that play a role in the cognitive processing of faces.

Pragmatic and knowledge range lenience towards foreigners

Foreign-accented speech categorizes the speaker as an outgroup individual with a lower linguistic competence and a different knowledge heritage from a native speaker. Here we explore whether the identification of an individual as a native or a foreign speaker has an impact on trivia statement judgments, regardless of her foreign-accented speech. Italian native participants first read a bio description of a native and of a foreign speaker and then rate to what degree a series of statements associated with each of the speakers makes sense (Studies 1 and 2) or are true (Study 3). Importantly, the fluency processing between native and foreign speakers was kept constant by using a written presentation of the materials. Under-informative statements such as 'Some frogs are amphibians' were tested in Study 1. The results of Study 1 show more acceptable judgments when the sentences were associated with the foreign speaker. Unknown facts about world knowledge such as 'Butterflies do not see gray' were tested in Studies 2 and 3. The results show more acceptable (Study 2) and more true (Study 3) judgments when the sentences were associated with the foreign speaker. In addition, in Study 3 the foreign speaker was considered more trustworthy than the native speaker in a rating test at the end of the main judgment-sentence task. Our findings show that linguistic identity per se has an impact on evaluation judgments, suggesting that message interpretation cannot be dissociated from who is communicating the message.

Who speaks next? Adaptations to speaker identity in processing spoken sentences

When listening to a speaker, we need to adapt to her individual speaking characteristics, such as error proneness, accent, etc. The present study investigated two aspects of adaptation to speaker identity during processing spoken sentences in multi-speaker situations: the effect of speaker sequence across sentences and the effect of learning speaker-specific error probability. Spoken sentences were presented, cued, and accompanied by one of three portraits that were labeled as the speakers' faces. In Block 1 speaker-specific probabilities of syntax errors were 10%, 50%, or 90%; in Block 2 they were uniformly 50%. In both blocks, speech errors elicited P600 effects in the scalp recorded ERP. We found a speaker sequence effect only in Block 1: the P600 to target words was larger after speaker switches than after speaker repetitions, independent of sentence correctness. In Block 1, listeners showed higher accuracy in judging sentence correctness spoken by speakers with lower error proportions. No speaker-specific differences in target word P600 and accuracy were found in Block 2. When speakers differ in error proneness, listeners seem to flexibly adapt their speech processing for the upcoming sentence through attention reorientation and resource reallocation if the speaker is about to change, and through proactive maintenance of neural resources if the speaker remains the same.