The Immersed Experiencer: Toward An Embodied Theory Of Language Comprehension

The influence of sentence focus on mental simulation: A possible cause of ACE instability

Recent studies have revealed the instability of the action-sentence compatibility effect (ACE). The current study was designed to demonstrate the hypothesis that the instability of the ACE may be attributed to the instability of focused information in a sentence. A pilot study indicated that the focused information of sentences was relatively stable in the sentence-picture verification task but exhibited significant interindividual variability in the action-sentence compatibility paradigm in previous studies. Experiments 1 and 2 examined the effect of sentence focus on the shape match effect and the ACE by manipulating the focused information of sentences using the focus marker word "" (is). Experiment 1 found that the shape match effect occurred in the original sentence, while it disappeared when the word "" (is) was used to make an object noun no longer the focus of a sentence. Experiment 2 failed to observe the ACE regardless of whether the sentence focus was on the action information. Experiment 3 modified the focus manipulation to observe its impact on the ACE using different fonts and underlines to highlight the focused information. The results indicated that the ACE only occurred when the action information was the sentence focus. These findings suggest that sentence focus influences mental simulation, and the instability of the ACE is likely to be associated with the instability of sentence focus in previous studies. This outcome highlights the crucial role of identifying specific information as the critical element expressed in the current linguistic context for successful simulation.

Modal and amodal cognition: an overarching principle in various domains of psychology

Accounting for how the human mind represents the internal and external world is a crucial feature of many theories of human cognition. Central to this question is the distinction between modal as opposed to amodal representational formats. It has often been assumed that one but not both of these two types of representations underlie processing in specific domains of cognition (e.g., perception, mental imagery, and language). However, in this paper, we suggest that both formats play a major role in most cognitive domains. We believe that a comprehensive theory of cognition requires a solid understanding of these representational formats and their functional roles within and across different domains of cognition, the developmental trajectory of these representational formats, and their role in dysfunctional behavior. Here we sketch such an overarching perspective that brings together research from diverse subdisciplines of psychology on modal and amodal representational formats so as to unravel their functional principles and their interactions.

Neural specialization for 'visual' concepts emerges in the absence of vision

Vision provides a key source of information about many concepts, including 'living things' (e.g., tiger) and visual events (e.g., sparkle). According to a prominent theoretical framework, neural specialization for different conceptual categories is shaped by sensory features, e.g., living things are neurally dissociable from navigable places because living things concepts depend more on visual features. We tested this framework by comparing the neural basis of 'visual' concepts across sighted (n=22) and congenitally blind (n=21) adults. Participants judged the similarity of words varying in their reliance on vision while undergoing fMRI. We compared neural responses to living things nouns (birds, mammals) and place nouns (natural, manmade). In addition, we compared visual event verbs (e.g., 'sparkle') to non-visual events (sound emission, hand motion, mouth motion). People born blind exhibited distinctive univariate and multivariate responses to living things in a temporo-parietal semantic network activated by nouns, including the precuneus (PC). To our knowledge, this is the first demonstration that neural selectivity for living things does not require vision. We additionally observed preserved neural signatures of 'visual' light events in the left middle temporal gyrus (LMTG+). Across a wide range of semantic types, neural representations of sensory concepts develop independent of sensory experience.

Word-association norms for 1,100 French words with varying levels of concreteness

The organisation of concepts in the mental lexicon is a widely studied research topic in experimental psychology. For instance, several studies have shown that whereas concrete concepts are organised according to semantic similarity, abstract concepts are organised according to verbal association. However, these results are not systematically replicated, mainly due to a lack of normative database especially in French. To that end, we introduce a French word-association database for 1,100 cues with varying levels of concreteness from abstract to concrete concepts. Analyses from the word-association task revealed stronger association strength for concrete concepts compared with abstract concepts. Additional results showed that cues tend to elicit responses of a similar level of concreteness. The database will be useful for investigators interested in French verbal associations for abstract and concrete concepts. The data (available on OSF https://osf.io/dhuqs/) introduce responses organised according to association strength and provides cue concreteness.

Modeling Brain Representations of Words' Concreteness in Context Using GPT-2 and Human Ratings

The meaning of most words in language depends on their context. Understanding how the human brain extracts contextualized meaning, and identifying where in the brain this takes place, remain important scientific challenges. But technological and computational advances in neuroscience and artificial intelligence now provide unprecedented opportunities to study the human brain in action as language is read and understood. Recent contextualized language models seem to be able to capture homonymic meaning variation ("bat", in a baseball vs. a vampire context), as well as more nuanced differences of meaning-for example, polysemous words such as "book", which can be interpreted in distinct but related senses ("explain a book", information, vs. "open a book", object) whose differences are fine-grained. We study these subtle differences in lexical meaning along the concrete/abstract dimension, as they are triggered by verb-noun semantic composition. We analyze functional magnetic resonance imaging (fMRI) activations elicited by Italian verb phrases containing nouns whose interpretation is affected by the verb to different degrees. By using a contextualized language model and human concreteness ratings, we shed light on where in the brain such fine-grained meaning variation takes place and how it is coded. Our results show that phrase concreteness judgments and the contextualized model can predict BOLD activation associated with semantic composition within the language network. Importantly, representations derived from a complex, nonlinear composition process consistently outperform simpler composition approaches. This is compatible with a holistic view of semantic composition in the brain, where semantic representations are modified by the process of composition itself. When looking at individual brain areas, we find that encoding performance is statistically significant, although with differing patterns of results, suggesting differential involvement, in the posterior superior temporal sulcus, inferior frontal gyrus and anterior temporal lobe, and in motor areas previously associated with processing of concreteness/abstractness.

The experiential basis of compatibility effects in reading-by-rotating paradigms

The current study originates from inconsistent findings within the framework of embodied language processing, specifically in the reading-by-rotating literature: whereas some studies report a match advantage (e.g., Zwaan and Taylor (J Exp Psychol 135:1, 2006)), i.e., shorter reading times when the direction of a linguistically conveyed manual rotation matched rather than mismatched the direction of an actually to be performed manual rotation Claus (Acta Psychol 156:104-113, 2015) found a mismatch advantage. The current study addresses two explanations that were previously discussed as potentially responsible for this inconsistency: on the one hand, differences in the knob devices employed; on the other hand, differences in the perspectives adopted by the readers depending on the number of characters involved in the narratives. Concurrently, the study exploits individual differences in motoric experience to explore the experiential basis of action-sentence compatibility effects. The results are inconclusive with respect to the two explanations. However, in their overall picture, they contribute suggestive considerations for the ongoing debate on action-simulation effects by pointing to the potential role of interindividual variation in motoric experience.

"The great mixing machine": multisensory integration and brain-breath coupling in the cerebral cortex

It is common to distinguish between "holist" and "reductionist" views of brain function, where the former envisions the brain as functioning as an indivisible unit and the latter as a collection of distinct units that serve different functions. Opposing reductionism, a number of researchers have pointed out that cortical network architecture does not respect functional boundaries, and the neuroanatomist V. Braitenberg proposed to understand the cerebral cortex as a "great mixing machine" of neuronal activity from sensory inputs, motor commands, and intrinsically generated processes. In this paper, we offer a contextualization of Braitenberg's point, and we review evidence for the interactions of neuronal activity from multiple sensory inputs and intrinsic neuronal processes in the cerebral cortex. We focus on new insights from studies on audiovisual interactions and on the influence of respiration on brain functions, which do not seem to align well with "reductionist" views of areal functional boundaries. Instead, they indicate that functional boundaries are fuzzy and context dependent. In addition, we discuss the relevance of the influence of sensory, proprioceptive, and interoceptive signals on cortical activity for understanding brain-body interactions, highlight some of the consequences of these new insights for debates on embodied cognition, and offer some suggestions for future studies.

Mnemonic scaffolds vary in effectiveness for serial recall

Memory champions remember vast amounts of information in order and at first encounter by associating each study item to an anchor within a scaffold - a pre-learned, structured memory. The scaffold provides direct-access retrieval cues. Dominated by the familiar-route scaffold (Method of Loci), researchers have little insight into what characteristics of scaffolds make them effective, nor whether individual differences might play a role. We compared participant-generated mnemonic scaffolds: (a) familiar routes (Loci), (b) autobiographical stories (Story), (c) parts of the human body (Body), and (d) routine activities (Routine Activity). Loci, Body, and Story Scaffolds benefited serial recall over Control (no scaffold). The Body and Loci Scaffold were equally superior to the other scaffolds. Measures of visual imagery aptitude and vividness and body responsiveness did not predict accuracy. A second experiment tested whether embodiment could be responsible for the high level of effectiveness of the Body Scaffold; this was not supported. In short, mnemonic scaffolds are not equally effective and embodied cognition may not directly contribute to memory success. The Body Scaffold may be a strong alternative to the Method of Loci and may enhance learning for most learners, including those who do not find the Method of Loci useful.

Verifying Negative Sentences

In the long history of psycholinguistic research on verifying negative sentences, an often-reported finding is that participants take longer to correctly judge negative sentences true than false, while being faster to judge their positive counterparts true (e.g. Clark & Chase, Cogn Psychol 3(3):472-517, 1972; Carpenter & Just, Psychol Rev 82(1):45-73, 1975). While many linguists and psycholinguists have strongly advocated the idea that the costs and complexity of negation can be explained by appeal to context, context-based approaches have not been able to provide a satisfying account of this polarity*truth-value interaction. By contrast, the alternative theory of negation processing, which says that negation is processed by separately representing the positive, does provide a plausible account. Our proposals provide a means for reconciliation between the two views since we argue that negation is a strong cue to a positive context. Here we present our account of why and when negation is often apparently processed via the positive. We review many of the factors that are seen to be at play in sentence verification involving negation. We present evidence that participants' adoption of the positive-first procedure in sentence-picture verification tasks is conditioned by context.

Personality Traits Modulate the Effect of tDCS on Reading Speed of Social Sentences

In this case, 62 university students participated in the study, in which a between-subjects design was adopted. Participants were also given the behavioral approach system (BAS) and behavioral inhibition system (BIS) scales. Participants had to read a list of 60 sentences with interpersonal and neutral content: 20 approach (“Pedro accepted Rosa in Whatsapp”), 20 avoidance (“Pedro Blocked Rosa in Whatsapp”) and 20 neutral (“Marta thought about the causes of the problem”). After reading them, they were subjected to 20 min of transcranial direct current stimulation (tDCS) in one of the two conditions: anodal (31) or sham (31). After tDCS, they had to read other list of 60 sentences matched in approach, avoidance and neutral contents with the former list. We found significant improvement in reading speed after anodal stimulation for social and neutral sentences. Regarding affective traits, we found that anodal stimulation benefitted reading speed in low-BIS and low-BAS participants and had no effect in either high BAS or high BIS participants. In addition, tDCS improvement in reading speed was significantly lower in avoidance sentences in low-BIS (avoidance) participants. We discuss these results at the light of previous research and highlight the importance of approach and avoidance traits as moderators of tDCS effects.

Mapping Directional Mid-Air Unistroke Gestures to Interaction Commands: A User Elicitation and Evaluation Study

A stroke is the basic limb movement that both humans and animals naturally and repetitiously perform. Having been introduced into gestural interaction, mid-air stroke gestures saw a wide application range and quite intuitive use. In this paper, we present an approach for building command-to-gesture mapping that exploits the semantic association between interactive commands and the directions of mid-air unistroke gestures. Directional unistroke gestures make use of the symmetry of the semantics of commands, which makes a more systematic gesture set for users’ cognition and reduces the number of gestures users need to learn. However, the learnability of the directional unistroke gestures is varying with different commands. Through a user elicitation study, a gesture set containing eight directional mid-air unistroke gestures was selected by subjective ratings of the direction in respect to its association degree with the corresponding command. We evaluated this gesture set in a following study to investigate the learnability issue, and the directional mid-air unistroke gestures and user-preferred freehand gestures were compared. Our findings can offer preliminary evidence that “return”, “save”, “turn-off” and “mute” are the interaction commands more applicable to using directional mid-air unistrokes, which may have implication for the design of mid-air gestures in human–computer interaction.

Stimulus-independent neural coding of event semantics: Evidence from cross-sentence fMRI decoding

Multivariate neuroimaging studies indicate that the brain represents word and object concepts in a format that readily generalises across stimuli. Here we investigated whether this was true for neural representations of simple events described using sentences. Participants viewed sentences describing four events in different ways. Multivariate classifiers were trained to discriminate the four events using a subset of sentences, allowing us to test generalisation to novel sentences. We found that neural patterns in a left-lateralised network of frontal, temporal and parietal regions discriminated events in a way that generalised successfully over changes in the syntactic and lexical properties of the sentences used to describe them. In contrast, decoding in visual areas was sentence-specific and failed to generalise to novel sentences. In the reverse analysis, we tested for decoding of syntactic and lexical structure, independent of the event being described. Regions displaying this coding were limited and largely fell outside the canonical semantic network. Our results indicate that a distributed neural network represents the meaning of event sentences in a way that is robust to changes in their structure and form. They suggest that the semantic system disregards the surface properties of stimuli in order to represent their underlying conceptual significance.

Exposure to Literary Fiction Is Associated With Lower Psychological Essentialism

We investigated the impact of exposure to literary and popular fiction on psychological essentialism. Exposure to fiction was measured by using the Author Recognition Test, which allows us to separate exposure to authors of literary and popular fiction. Psychological essentialism was assessed by the discreteness subscale of the psychological essentialism scale in Study 1, and by the three subscales of the same scale (such as discreteness, informativeness, and biological basis) in Study 2 that was pre-registered. Results showed that exposure to literary fiction negatively predicts the three subscales. The results emerged controlling for political ideology, a variable that is commonly associated with psychological essentialism, and level of education.

Intangible features extraction in the processing of abstract concepts: Evidence from picture-word priming

Over the last decade, hypotheses ranging from linguistic symbol processing to embodiment have been formulated to account for the content and mechanisms responsible for the representation of abstract concepts. Results of recent studies have suggested that abstract concepts, just like concrete ones, can benefit from knowledge of real-world situational context, but that they can also be processed based on abstract pictures devoid of such situational features. This paper presents two semantic priming experiments to explore such mechanisms further. The first experiment replicates Kuipers, Jones, and Thierry (2018) in a cross-linguistic setting which shows that abstract concepts can be processed from abstract pictures devoid of tangible features. In the second experiment, we studied extraction mechanisms that come into play when participants are presented with abstract and concrete pictures that provide situational information to illustrate target abstract concepts. We expected this facilitatory effect to be limited to concrete picture primes. Our data analysed with both Bayesian and Frequentist tests showed however that even when presented with tangible situational information, the extraction of features still occurred for abstract pictures. We discuss the implications of this with respect to future avenues for studying the processing of abstract concepts.

Storytelling on Oral Grounds: Viewpoint Alignment and Perspective Taking in Narrative Discourse

In this paper, we seek to explain the power of perspective taking in narrative discourse by turning to research on the oral foundations of storytelling in human communication and language. We argue that narratives function through a central process of alignment between the viewpoints of narrator, hearer/reader, and character and develop an analytical framework that is capable of generating general claims about the processes and outcomes of narrative discourse while flexibly accounting for the great linguistic variability both across and within stories. The central propositions of this viewpoint alignment framework are that the distance between the viewpoints of participants in the narrative construal – narrator, character, reader – is dynamic and regulated by linguistic choices as well as contextual factors. Fundamentally, viewpoint alignment is grounded in oral narrative interaction and, from this conversation, transferred to the written narrative situation, varying between demonstration and invasion of the narrative subjects and guiding readers’ route of processing the narrative (experiential versus reflective). Our claim is that variations in viewpoint alignment are functional to the communicative context and intended outcomes of narratives. This is illustrated with the analysis of a corporate journalistic narrative that comprises both interactional and non-interactional aspects of storytelling. The concept of viewpoint alignment further explains the oral fundaments of narrative discourse in conversational storytelling and poses new questions on the relation between the dynamic processing of stories on the one hand and their static outcomes on the other.

Explicit (Not Implicit) Attitudes Mediate the Focus of Attention During Sentence Processing

Many studies showed that comprehenders monitor changes in protagonists' emotions and actions. This article reports two experiments that explored how focusing comprehenders' attention on a particular property of the protagonist dimension (e.g., emotional or action state) affects the accessibility of information about target objects mentioned in the sentence. Furthermore, the present research examined whether participants' attitudes toward the issues described in the sentence can modulate comprehension processes. To this end, we asked participants to read sentences about environmental issues that focused comprehenders' attention on different mental and physical attributes of the same entities (protagonists and objects) and then self-report their own thoughts on the topic of environment by responding to the items assessing their environmental awareness. Importantly, we manipulated the task requirements across two experiments by administering a self-report task (Experiment 1), which required the participants to rate the seriousness and the frequency of the problem mentioned in a sentence; and administering a sentence-picture verification paradigm (Experiment 2), which required the participants to merely indicate if the object depicted in the picture (related to a certain environmental problem) was mentioned in the preceding sentence. The results of these experiments suggest that the focus of a sentence on the environmental problem (rather than the protagonist's emotion and action) enhances the accessibility of information about environmental issues (e.g., plastic garbage); that the comprehender's level of environmental awareness influences one's attention during sentence processing; and that comprehender characteristics significantly modulate comprehension processes only when the measures tap into explicit (and not implicit) processes.

Semantic similarity and associated abstractness norms for 630 French word pairs

The representation of abstract concepts remains a challenge, justifying the need for further experimental investigation. To that end, we introduce a normative database for 630 semantically similar French word pairs and associated levels of abstractness for 1260 isolated words based on data from 900 subjects. The semantic similarity and abstractness norms were obtained in two studies using 7-point scales. The database is organised according to word-pair semantic similarity, abstractness, and associated lexical variables such as word length (in number of letters), word frequency, and other lexical variables to allow for matching of experimental material. The associated variables were obtained by cross-referencing our database with other known psycholinguistic databases including Lexique (New et al., 2004), the French Lexicon Project (Ferrand et al., 2010), Wordlex (Gimenes & New, 2016), and MEGALEX (Ferrand et al., 2018). We introduced sufficient diversity to allow researchers to select pairs with varying levels of semantic similarity and abstractness. In addition, it is possible to use these data as continuous or discrete variables. The full data are available in the supplementary materials as well as on OSF ( https://osf.io/qsd4v/ ).

Somatic and visceral effects of word valence, arousal and concreteness in a continuum lexical space

Although affective and semantic word properties are known to independently influence our sensorimotor system, less is known about their interaction. We investigated this issue applying a data-driven mixed-effects regression approach, evaluating the impact of lexical-semantic properties on electrophysiological parameters, namely facial muscles activity (left corrugator supercilii, zygomaticus major, levator labii superioris) and heartbeat, during word processing. 500 Italian words were acoustically presented to 20 native-speakers, while electrophysiological signals were continuously recorded. Stimuli varied for affective properties, namely valence (the degree of word positivity), arousal (the amount of emotional activation brought by the word), and semantic ones, namely concreteness. Results showed that the three variables interacted in predicting both heartbeat and muscular activity. Specifically, valence influenced activation for lower levels of arousal. This pattern was further modulated by concreteness: the lower the word concreteness, the larger affective-variable impact. Taken together, our results provide evidence for bodily responses during word comprehension. Crucially, such responses were found not only for voluntary muscles, but also for the heartbeat, providing evidence to the idea of a common emotional motor system. The higher impact of affective properties for abstract words supports proposals suggesting that emotions play a central role in the grounding of abstract concepts.

Affective iconic words benefit from additional sound-meaning integration in the left amygdala

Recent studies have shown that a similarity between sound and meaning of a word (i.e., iconicity) can help more readily access the meaning of that word, but the neural mechanisms underlying this beneficial role of iconicity in semantic processing remain largely unknown. In an fMRI study, we focused on the affective domain and examined whether affective iconic words (e.g., high arousal in both sound and meaning) activate additional brain regions that integrate emotional information from different domains (i.e., sound and meaning). In line with our hypothesis, affective iconic words, compared to their non‐iconic counterparts, elicited additional BOLD responses in the left amygdala known for its role in multimodal representation of emotions. Functional connectivity analyses revealed that the observed amygdalar activity was modulated by an interaction of iconic condition and activations in two hubs representative for processing sound (left superior temporal gyrus) and meaning (left inferior frontal gyrus) of words. These results provide a neural explanation for the facilitative role of iconicity in language processing and indicate that language users are sensitive to the interaction between sound and meaning aspect of words, suggesting the existence of iconicity as a general property of human language.

Affordance Compatibility Effect for Word Learning in Virtual Reality

Rich sensorimotor interaction facilitates language learning and is presumed to ground conceptual representations. Yet empirical support for early stages of embodied word learning is currently lacking. Finding evidence that sensorimotor interaction shapes learned linguistic representations would provide crucial support for embodied language theories. We developed a gamified word learning experiment in virtual reality in which participants learned the names of six novel objects by grasping and manipulating objects with either their left or right hand. Participants then completed a word-color match task in which they were tested on the same six words and objects. Participants were faster to respond to stimuli in the match task when the response hand was compatible with the hand used to interact with the named object, an effect we refer to as affordance compatibility. In two follow up experiments, we found that merely observing virtual hands interact with the objects was sufficient to acquire a smaller affordance compatibility effect, and we found that the compatibility effect was driven primarily by responses with a compatible hand and not by responses in a compatible spatial location. Our results support theoretical views of language which ground word representations in sensorimotor experiences, and they suggest promising future routes to explore the sensorimotor foundations of higher cognition through immersive virtual experiments.

Concurrent emotional response and semantic unification: An event-related potential study

Using event-related potentials, in this study we examined how implied emotion is derived from sentences. In the same sentential context, different emotionally neutral words rendered the whole sentence emotionally neutral and semantically congruent, emotionally negative and semantically congruent, or emotionally neutral and semantically incongruent. Relative to the words in the neutral-congruent condition, the words in the neutral-incongruent condition elicited a larger N400, indicating increased semantic processing, whereas the words in the negative-congruent condition elicited a long-lasting positivity between 300 and 1,000 ms, indicating an emotional response. The overlapping time windows of semantic processing and the emotional response suggest that the construction of emotional meaning operates concurrently with semantic unification. The results indicate that the implied emotional processing of sentences may be a result of unification operations but does not necessarily involve causal appraisal of a sentence's mental representation.

Are visual processes causally involved in "perceptual simulation" effects in the sentence-picture verification task?

Many studies have shown that sentences implying an object to have a certain shape produce a robust reaction time advantage for shape-matching pictures in the sentence-picture verification task. Typically, this finding has been interpreted as evidence for perceptual simulation, i.e., that access to implicit shape information involves the activation of modality-specific visual processes. It follows from this proposal that disrupting visual processing during sentence comprehension should interfere with perceptual simulation and obliterate the match effect. Here we directly test this hypothesis. Participants listened to sentences while seeing either visual noise that was previously shown to strongly interfere with basic visual processing or a blank screen. Experiments 1 and 2 replicated the match effect but crucially visual noise did not modulate it. When an interference technique was used that targeted high-level semantic processing (Experiment 3) however the match effect vanished. Visual noise specifically targeting high-level visual processes (Experiment 4) only had a minimal effect on the match effect. We conclude that the shape match effect in the sentence-picture verification paradigm is unlikely to rely on perceptual simulation.

An Investigation of Spatial Stimulus-Response Compatibility Effects Based on German Particles

In the current study, we tested if stimulus-response (SR) compatibility effects of spatially ambiguous words depend on a semantic priming context. Although many words, including spatial words, can take on several meanings, this is an open question. From Experiments 1 to 3, we manipulated the likelihood that the vertical meaning of the German particles auf and ab was processed by (1) instructing the processing of vertical meaning in Experiment 1, but not in Experiments 2 and 3, and (2) by using verbs that either primed (Experiments 1 and 2) or did not prime (Experiments 1-3) the targets' vertical meanings. Spatial SR compatibility effects resulted, regardless of whether or not the processing of the vertical meaning was instructed and the vertical meaning was primed. Results suggest that the selection between vertically discriminated responses could be sufficient to elicit the participants' extraction of the vertical meaning of the ambiguous particles.

Extending Situated Language Comprehension (Accounts) with Speaker and Comprehender Characteristics: Toward Socially Situated Interpretation

More and more findings suggest a tight temporal coupling between (non-linguistic) socially interpreted context and language processing. Still, real-time language processing accounts remain largely elusive with respect to the influence of biological (e.g., age) and experiential (e.g., world and moral knowledge) comprehender characteristics and the influence of the 'socially interpreted' context, as for instance provided by the speaker. This context could include actions, facial expressions, a speaker's voice or gaze, and gestures among others. We review findings from social psychology, sociolinguistics and psycholinguistics to highlight the relevance of (the interplay between) the socially interpreted context and comprehender characteristics for language processing. The review informs the extension of an extant real-time processing account (already featuring a coordinated interplay between language comprehension and the non-linguistic visual context) with a variable ('ProCom') that captures characteristics of the language user and with a first approximation of the comprehender's speaker representation. Extending the CIA to the sCIA (social Coordinated Interplay Account) is the first step toward a real-time language comprehension account which might eventually accommodate the socially situated communicative interplay between comprehenders and speakers.

Experiential simulations of negated text information

We investigated the question of whether comprehenders mentally simulate a described situation even when this situation is explicitly negated in the sentence. In two experiments, participants read negative sentences such as There was no eagle in the sky, and subsequently responded to pictures of mentioned entities in the context of a recognition task. Participants’ responses following negative sentences were faster when the depicted entity matched rather than mismatched the negated situation. These results suggest that comprehenders simulate the negated situation when processing a negated sentence. The results thereby provide further support for the experiential-simulations view of language comprehension.

Identification of opposites and intermediates by eye and by hand

In this eye-tracking and drawing study, we investigate the perceptual grounding of different types of spatial dimensions such as dense-sparse and top-bottom, focusing both on the participants' experiences of the opposite regions, e.g., O1: dense; O2: sparse, and the region that is experienced as intermediate, e.g., INT: neither dense nor sparse. Six spatial dimensions expected to have three different perceptual structures in terms of the point and range nature of O1, INT and O2 were analysed. Presented with images, the participants were instructed to identify each region (O1, INT, O2), first by looking at the region, and then circumscribing it using the computer mouse. We measured the eye movements, identification times and various characteristics of the drawings such as the relative size of the three regions, overlaps and gaps. Three main results emerged. Firstly, generally speaking, intermediate regions were not different from the poles on any of the indicators: overall identification times, number of fixations, and locations. Some differences emerged with regard to the duration of fixations for point INTs and the number of fixations for range INTs between two range poles (O1, O2). Secondly, the analyses of the fixation locations showed that the poles support the identification of the intermediate region as much as the intermediate region supports the identification of the poles. Finally, the relative size of the three areas selected in the drawing task were consistent with the classification of the regions as points or ranges. The analyses of the gaps and the overlaps between the three areas showed that the intermediate is neither O1 nor O2, but an entity in its own right.

Walking during the encoding of described environments enhances a heading-independent spatial representation

Previous studies demonstrated that physical movement enhanced spatial updating in described environments. However, those movements were executed only after the encoding of the environment, minimally affecting the development of the spatial representation. Thus, we investigated whether and how participants could benefit from the execution of physical movement during the encoding of described environments, in terms of enhanced spatial updating. Using the judgement of relative directions task, we compared the effects of walking both during and after the description of the environment, and walking only after the description on spatial updating. Spatial updating was evaluated in terms of accuracy and response times in different headings. We found that the distribution of response times across Headings seemed not to be related to the physical movement executed, whereas the distribution of accuracy scores seemed to significantly change with the action executed. Indeed, when no movement occurred during the encoding of the environment, a preference for the learning heading was found, which did not emerge when walking during encoding occurred. Therefore, the results seem to suggest that physical movement during encoding supports the development of a heading-independent representation of described environments, reducing the anchoring for a preferred heading in favor of a global representation.

Readers select a comprehension mode independent of pronoun: Evidence from fMRI during narrative comprehension

Perspective is a crucial feature for communicating about events. Yet it is unclear how linguistically encoded perspective relates to cognitive perspective taking. Here, we tested the effect of perspective taking with short literary stories. Participants listened to stories with 1st or 3rd person pronouns referring to the protagonist, while undergoing fMRI. When comparing action events with 1st and 3rd person pronouns, we found no evidence for a neural dissociation depending on the pronoun. A split sample approach based on the self-reported experience of perspective taking revealed 3 comprehension preferences. One group showed a strong 1st person preference, another a strong 3rd person preference, while a third group engaged in 1st and 3rd person perspective taking simultaneously. Comparing brain activations of the groups revealed different neural networks. Our results suggest that comprehension is perspective dependent, but not on the perspective suggested by the text, but on the reader's (situational) preference.

Numerical Congruency Effect in the Sentence-Picture Verification Task

In two experiments, we showed that irrelevant numerical information influenced the speed of sentence-picture verification. Participants were asked to verify whether the concept mentioned in a sentence matched the object presented in a subsequent picture. Concurrently, the number word attached to the concept in the sentence and the quantity of objects presented in the picture were manipulated (numerical congruency). The number of objects varied from one to four. In Experiment 1, participants read statements such as three dogs. In Experiment 2, they read sentences such as three dogs were wandering in the street. In both experiments, the verification speed revealed the interaction between response and numerical congruency. The verification times for concept-object match were faster when there was also numerical congruence (compared with incongruence) between the number word and quantity. On the other hand, there was no difference between numerical congruence and incongruence when the concept and object mismatched. The results are interpreted as evidence for the symbol grounding of number words in perceptual representation of small quantities, that is, quantities falling in the subitization range.

Uncharted features and dynamics of reading: Voices, characters, and crossing of experiences

•

Vivid experiences of characters implicate simulation processes during reading.

•

Reading imagery is related to features of inner speech and hallucination-proneness.

•

Qualitative analysis highlights involuntary experiences of characters and voices.

Readers often describe vivid experiences of voices and characters in a manner that has been likened to hallucination. Little is known, however, of how common such experiences are, nor the individual differences they may reflect. Here we present the results of a 2014 survey conducted in collaboration with a national UK newspaper and an international book festival. Participants (n = 1566) completed measures of reading imagery, inner speech, and hallucination-proneness, including 413 participants who provided detailed free-text descriptions of their reading experiences. Hierarchical regression analysis indicated that reading imagery was related to phenomenological characteristics of inner speech and proneness to hallucination-like experiences. However, qualitative analysis of reader’s accounts suggested that vivid reading experiences were marked not just by auditory phenomenology, but also their tendency to cross over into non-reading contexts. This supports social-cognitive accounts of reading while highlighting a role for involuntary and uncontrolled personality models in the experience of fictional characters.

Experience-Based Probabilities Modulate Expectations in a Gender-Coded Artificial Language

The current study combines artificial language learning with visual world eyetracking to investigate acquisition of representations associating spoken words and visual referents using morphologically complex pseudowords. Pseudowords were constructed to consistently encode referential gender by means of suffixation for a set of imaginary figures that could be either male or female. During training, the frequency of exposure to pseudowords and their imaginary figure referents were manipulated such that a given word and its referent would be more likely to occur in either the masculine form or the feminine form, or both forms would be equally likely. Results show that these experience-based probabilities affect the formation of new representations to the extent that participants were faster at recognizing a referent whose gender was consistent with the induced expectation than a referent whose gender was inconsistent with this expectation. Disambiguating gender information available from the suffix did not mask the induced expectations. Eyetracking data provide additional evidence that such expectations surface during online lexical processing. Taken together, these findings indicate that experience-based information is accessible during the earliest stages of processing, and are consistent with the view that language comprehension depends on the activation of perceptual memory traces.

Arguments about the nature of concepts: Symbols, embodiment, and beyond

How are the meanings of words, events, and objects represented and organized in the brain? This question, perhaps more than any other in the field, probes some of the deepest and most foundational puzzles regarding the structure of the mind and brain. Accordingly, it has spawned a field of inquiry that is diverse and multidisciplinary, has led to the discovery of numerous empirical phenomena, and has spurred the development of a wide range of theoretical positions. This special issue brings together the most recent theoretical developments from the leaders in the field, representing a range of viewpoints on issues of fundamental significance to a theory of meaning representation. Here we introduce the special issue by way of pulling out some key themes that cut across the contributions that form this issue and situating those themes in the broader literature. The core issues around which research on conceptual representation can be organized are representational format, representational content, the organization of concepts in the brain, and the processing dynamics that govern interactions between the conceptual system and sensorimotor representations. We highlight areas in which consensus has formed; for those areas in which opinion is divided, we seek to clarify the relation of theory and evidence and to set in relief the bridging assumptions that undergird current discussions.

Is Accessing of Words Affected by Affective Valence Only? A Discrete Emotion View on the Emotional Congruency Effect

This paper advances the discussion on which emotion information affects word accessing. Emotion information, which is formed as a result of repeated experiences, is primary and necessary in learning and representing word meanings. Previous findings suggested that valence (i.e., positive or negative) denoted by words can be automatically activated and plays a role in many significant cognitive processes. However, there has been a lack of discussion about whether discrete emotion information (i.e., happiness, anger, sadness, and fear) is also involved in these processes. According to the hierarchy model, emotions are considered organized within an abstract-to-concrete hierarchy, in which emotion prototypes are organized following affective valence. By controlling different congruencies of emotion relations (i.e., matches or mismatches between valences and prototypes of emotion), the present study showed both an evaluative congruency effect (Experiment 1) and a discrete emotional congruency effect (Experiment 2). These findings indicate that not only affective valences but also discrete emotions can be activated under the present priming lexical decision task. However, the present findings also suggest that discrete emotions might be activated at the later priming stage as compared to valences. The present work provides evidence that information about discrete emotion could be involved in word processing. This might be a result of subjects' embodied experiences.

Taking Perspective: Personal Pronouns Affect Experiential Aspects of Literary Reading

Personal pronouns have been shown to influence cognitive perspective taking during comprehension. Studies using single sentences found that 3^rd person pronouns facilitate the construction of a mental model from an observer’s perspective, whereas 2^nd person pronouns support an actor’s perspective. The direction of the effect for 1^st person pronouns seems to depend on the situational context. In the present study, we investigated how personal pronouns influence discourse comprehension when people read fiction stories and if this has consequences for affective components like emotion during reading or appreciation of the story. We wanted to find out if personal pronouns affect immersion and arousal, as well as appreciation of fiction. In a natural reading paradigm, we measured electrodermal activity and story immersion, while participants read literary stories with 1^st and 3^rd person pronouns referring to the protagonist. In addition, participants rated and ranked the stories for appreciation. Our results show that stories with 1^st person pronouns lead to higher immersion. Two factors—transportation into the story world and mental imagery during reading—in particular showed higher scores for 1^st person as compared to 3^rd person pronoun stories. In contrast, arousal as measured by electrodermal activity seemed tentatively higher for 3^rd person pronoun stories. The two measures of appreciation were not affected by the pronoun manipulation. Our findings underscore the importance of perspective for language processing, and additionally show which aspects of the narrative experience are influenced by a change in perspective.

Neuronal interactions between mentalising and action systems during indirect request processing

Human communication relies on the ability to process linguistic structure and to map words and utterances onto our environment. Furthermore, as what we communicate is often not directly encoded in our language (e.g. in the case of irony, jokes or indirect requests), we need to extract additional cues to infer the beliefs and desires of our conversational partners. Although the functional interplay between language and the ability to mentalise has been discussed in theoretical accounts in the past, the neurobiological underpinnings of these dynamics are currently not well understood. Here, we address this issue using functional imaging (fMRI). Participants listened to question-reply dialogues. In these dialogues, a reply is interpreted as a direct reply, an indirect reply or a request for action, depending on the question. We show that inferring meaning from indirect replies engages parts of the mentalising network (mPFC) while requests for action also activate the cortical motor system (IPL). Subsequent connectivity analysis using Dynamic Causal Modelling (DCM) revealed that this pattern of activation is best explained by an increase in effective connectivity from the mentalising network (mPFC) to the action system (IPL). These results are an important step towards a more integrative understanding of the neurobiological basis of indirect speech processing.

Readers of narratives take the protagonist's geographical perspective. Evidence from an event-related potential study

This ERP study explores how the reader's brain is sensitive to the protagonist's perspective in the fictitious environment of narratives. Participants initially received narratives describing a protagonist living in a given geographical place. Later on they were given short paragraphs describing another character as "coming" or "going" to a place either close to or distant from the protagonist. Paragraphs referring to distant places elicited larger negative waves than those with places close to the protagonist. Moreover, narratives with the verb to come incoherent with the protagonist's perspective (e.g., "she came to the distant place") elicited larger negative-going waves in the 320-400ms time window than coherent paragraphs (e.g., "she came to the close place"). These results indicate that readers of narratives were able to take the protagonist's geographical perspective, showing discourse-level coherence effects when they read motion sentences with the marked deictic verb to come.

Facial Action and Emotional Language: ERP Evidence that Blocking Facial Feedback Selectively Impairs Sentence Comprehension

There is a lively and theoretically important debate about whether, how, and when embodiment contributes to language comprehension. This study addressed these questions by testing how interference with facial action impacts the brain's real-time response to emotional language. Participants read sentences about positive and negative events (e.g., “She reached inside the pocket of her coat from last winter and found some (cash/bugs) inside it.”) while ERPs were recorded. Facial action was manipulated within participants by asking participants to hold chopsticks in their mouths using a position that allowed or blocked smiling, as confirmed by EMG. Blocking smiling did not influence ERPs to the valenced words (e.g., cash, bugs) but did influence ERPs to final words of sentences describing positive events. Results show that affectively positive sentences can evoke smiles and that such facial action can facilitate the semantic processing indexed by the N400 component. Overall, this study offers causal evidence that embodiment impacts some aspects of high-level comprehension, presumably involving the construction of the situation model.

Iconicity in the lab: a review of behavioral, developmental, and neuroimaging research into sound-symbolism

This review covers experimental approaches to sound-symbolism—from infants to adults, and from Sapir’s foundational studies to twenty-first century product naming. It synthesizes recent behavioral, developmental, and neuroimaging work into a systematic overview of the cross-modal correspondences that underpin iconic links between form and meaning. It also identifies open questions and opportunities, showing how the future course of experimental iconicity research can benefit from an integrated interdisciplinary perspective. Combining insights from psychology and neuroscience with evidence from natural languages provides us with opportunities for the experimental investigation of the role of sound-symbolism in language learning, language processing, and communication. The review finishes by describing how hypothesis-testing and model-building will help contribute to a cumulative science of sound-symbolism in human language.

Feature activation during word recognition: action, visual, and associative-semantic priming effects

Embodied theories of language postulate that language meaning is stored in modality-specific brain areas generally involved in perception and action in the real world. However, the temporal dynamics of the interaction between modality-specific information and lexical-semantic processing remain unclear. We investigated the relative timing at which two types of modality-specific information (action-based and visual-form information) contribute to lexical-semantic comprehension. To this end, we applied a behavioral priming paradigm in which prime and target words were related with respect to (1) action features, (2) visual features, or (3) semantically associative information. Using a Go/No-Go lexical decision task, priming effects were measured across four different inter-stimulus intervals (ISI = 100, 250, 400, and 1000 ms) to determine the relative time course of the different features. Notably, action priming effects were found in ISIs of 100, 250, and 1000 ms whereas a visual priming effect was seen only in the ISI of 1000 ms. Importantly, our data suggest that features follow different time courses of activation during word recognition. In this regard, feature activation is dynamic, measurable in specific time windows but not in others. Thus the current study (1) demonstrates how multiple ISIs can be used within an experiment to help chart the time course of feature activation and (2) provides new evidence for embodied theories of language.

Neuroelectric Correlates of Pragmatic Emotional Incongruence Processing: Empathy Matters

The emotions people feel can be simulated internally based on emotional situational contexts. In the present study, we assessed the behavioral and neuroelectric effects of seeing an unexpected emotional facial expression. We investigated the correct answer rate, response times and Event-Related Potential (ERP) effects during an incongruence paradigm between emotional faces and sentential contexts allowing emotional inferences. Most of the 36 healthy participants were recruited from a larger population (1 463 subjects), based on their scores on the Empathy Questionnaire (EQ). Regression analyses were conducted on these ratings using EQ factors as predictors (cognitive empathy, emotional reactivity and social skills). Recognition of pragmatic emotional incongruence was less accurate (P < .05) and slower (P < .05) than recognition of congruence. The incongruence effect on response times was inversely predicted by social skills. A significant N400 incongruence effect was found at the centro-parietal (P < .001) and centro-posterior midline (P < .01) electrodes. Cognitive empathy predicted the incongruence effect in the left occipital region, in the N400 time window. Finally, incongruence effects were also found on the LPP wave, in frontal midline and dorso-frontal regions, (P < .05), with no modulation by empathy. Processing pragmatic emotional incongruence is more cognitively demanding than congruence (as reflected by both behavioral and ERP data). This processing shows modulation by personality factors at the behavioral (through self-reported social skills) and neuroelectric levels (through self-reported cognitive empathy).

Starting off on the right foot: strong right-footers respond faster with the right foot to positive words and with the left foot to negative words

Recent studies have provided evidence for an association between valence and left/right modulated by handedness, which is predicted by the body-specificity hypothesis (Casasanto, 2009) and also reflected in response times. We investigated whether such a response facilitation can also be observed with foot responses. Right-footed participants classified positive and negative words according to their valence by pressing a key with their left or right foot. A significant interaction between valence and foot only emerged in the by-items analysis. However, when dividing participants into two groups depending on the strength of their footedness, an interaction between valence and left/right was observed for strong right-footers, who responded faster with the right foot to positive words, and with the left foot to negative words. No interaction emerged for weak right-footers. The results strongly support the assumption that fluency lies at the core of the association between valence and left/right.