Reading "sun" and looking up: the influence of language on saccadic eye movements in the vertical dimension

How workload and availability of spatial reference shape eye movement coupling in visuospatial working memory

Eyes are active in memory recall and visual imagination, yet our grasp of the underlying qualities and factors of these internally coupled eye movements is limited. To explore this, we studied 50 participants, examining how workload, spatial reference availability, and imagined movement direction influence internal coupling of eye movements. We designed a visuospatial working memory task in which participants mentally moved a black patch along a path within a matrix and each trial involved one step along this path (presented via speakers: up, down, left, or right). We varied workload by adjusting matrix size (3 × 3 vs. 5 × 5), manipulated availability of a spatial frame of reference by presenting either a blank screen (requiring participants to rely solely on their mental representation of the matrix) or spatial reference in the form of an empty matrix, and contrasted active task performance to two control conditions involving only active or passive listening. Our findings show that eye movements consistently matched the imagined movement of the patch in the matrix, not driven solely by auditory or semantic cues. While workload influenced pupil diameter, perceived demand, and performance, it had no observable impact on internal coupling. The availability of spatial reference enhanced coupling of eye movements, leading more frequent, precise, and resilient saccades against noise and bias. The absence of workload effects on coupled saccades in our study, in combination with the relatively high degree of coupling observed even in the invisible matrix condition, indicates that eye movements align with shifts in attention across both visually and internally represented information. This suggests that coupled eye movements are not merely strategic efforts to reduce workload, but rather a natural response to where attention is directed.

Experiential traces first: Does holding a location in visuospatial working memory affect the processing of space-associated words?

This study aimed to systematically examine whether actively maintaining a visual location in working memory can influence the processing of spatially related words. In five experiments, we asked participants to maintain either the location or the shape of a visually presented stimulus in working memory so that it could later be compared with a test stimulus concerning the relevant target features. In between, we presented participants with words that refer to objects typically encountered in the upper or lower vertical space (roof vs. root, respectively). The task participants performed as a response to these words differed between experiments. In Experiments 1-3, participants performed a lexical decision task, in Experiment 4 they performed a semantic task (deciding whether the word refers to an occupation), and in Experiment 5 they performed a spatial task (deciding whether the word refers to something in the upper or lower visual field.) Only in Experiment 5 did we observe an interaction between the position of the visual stimulus held in working memory (up vs. down) and the meaning of the spatial words (associated with up vs. down). Our results therefore suggest that actively maintaining a stimulus location in working memory does not automatically affect the processing of spatially related words, but does so if the relevant spatial dimension is made highly salient by the task. The results are thus in line with studies showing a strong context-dependency of embodiment effects and thus allow the conclusion that language processing proper is not operating on a sensorimotor representational format.

Raising the Roof: Situating Verbs in Symbolic and Embodied Language Processing

Recent investigations on how people derive meaning from language have focused on task-dependent shifts between two cognitive systems. The symbolic (amodal) system represents meaning as the statistical relationships between words. The embodied (modal) system represents meaning through neurocognitive simulation of perceptual or sensorimotor systems associated with a word's referent. A primary finding of literature in this field is that the embodied system is only dominant when a task necessitates it, but in certain paradigms, this has only been demonstrated using nouns and adjectives. The purpose of this paper is to study whether similar effects hold with verbs. Experiment 1 evaluated a novel task in which participants rated a selection of verbs on their implied vertical movement. Ratings correlated well with distributional semantic models, establishing convergent validity, though some variance was unexplained by language statistics alone. Experiment 2 replicated previous noun-based location-cue congruency experimental paradigms with verbs and showed that the ratings obtained in Experiment 1 predicted reaction times more strongly than language statistics. Experiment 3 modified the location-cue paradigm by adding movement to create an animated, temporally decoupled, movement-verb judgment task designed to examine the relative influence of symbolic and embodied processing for verbs. Results were generally consistent with linguistic shortcut hypotheses of symbolic-embodied integrated language processing; location-cue congruence elicited processing facilitation in some conditions, and perceptual information accounted for reaction times and accuracy better than language statistics alone. These studies demonstrate novel ways in which embodied and linguistic information can be examined while using verbs as stimuli.

Strong versus weak embodiment: Spatial iconicity in physical, abstract, and social semantic categories

BACKGROUND

Perceptual and action systems seem to be related to complex cognitive processes, but the scope of grounded or embodied cognition has been questioned. Zwaan and Yaxley (2003) proposed that cognitive processes of making semantic relatedness judgments can be facilitated when word pairs are presented in ways that their referents maintain their iconic configuration rather than their reverse-iconic configuration (the spatial iconicity effect). This effect has been observed in different semantic categories using specific experiments, but it is known that embodiment is highly dependent on task demands.

METHOD

The present study analyzed the spatial iconicity effect in three semantic categories (physical, abstract, and social) using the same experimental criteria to determine the scope of embodied cognition. In this reaction-time experiment, 75 participants judged the semantic relatedness of 384 word pairs whose experimental items were presented in their iconic or reverse-iconic configurations.

RESULTS

Two mixed-effects models with crossed random effects revealed that the interaction between word meaning and spatial position was present only for physical concepts but neither for abstract nor social concepts.

CONCLUSIONS

Within the framework of strong and weak embodiment theories, the data support weak embodiment theory as the most explicative one.

Sentence-based mental simulations: Evidence from behavioral experiments using garden-path sentences

Language comprehenders activate mental representations of sensorimotor experiences related to the content of utterances they process. However, it is still unclear whether these sensorimotor simulations are driven by associations with words or by a more complex process of meaning composition into larger linguistic expressions, such as sentences. In two experiments, we investigated whether comprehenders indeed create sentence-based simulations. Materials were constructed such that simulation effects could only emerge from sentence meaning and not from word-based associations alone. We additionally asked when during sentence processing these simulations are constructed, using a garden-path paradigm. Participants read either a garden-path sentence (e.g., "As Mary ate the egg was in the fridge") or a corresponding unambiguous control with the same meaning and words (e.g., "The egg was in the fridge as Mary ate"). Participants then judged whether a depicted entity was mentioned in the sentence or not. In both experiments, picture response times were faster when the picture was compatible (vs. incompatible) with the sentence-based interpretation of the target entity (e.g., both for garden-path and control sentence: an unpeeled egg), suggesting that participants created simulations based on the sentence content and only operating over the sentence as a whole.

The action-sentence compatibility effect (ACE): Meta-analysis of a benchmark finding for embodiment

The embodied account of language comprehension has been one of the most influential theoretical developments in the recent decades addressing the question how humans comprehend and represent language. To examine its assumptions, many studies have made use of behavioral paradigms involving basic compatibility effects. The action-sentence compatibility effect (ACE) is one of the most influential of these compatibility effects and is the most widely cited evidence for the embodied account of language comprehension. However, recently there have been difficulties in extending or even in reliably replicating the ACE. The conflicting findings concerning the ACE and its extensions lead to the discussion of whether the ACE is indeed a reliable effect. In a first step we conducted a meta-analysis using a random-effects model. This analysis revealed a small but significant effect size of the ACE. Furthermore, the task-parameter Delay occurred as a factor of interest in whether the ACE appears with positive or negative effect direction. A second meta-analytic approach (Fisher's method) supports these findings. Additionally, an analysis of publication bias suggests that there is bias in the ACE literature. In post-hoc analyses of the recent multi-lab investigation of the ACE (Morey et al., 2021), evidence for individual differences in the ACE was found. However, further analyses indicate that these differences are likely due to item-specific variability and the specific way in which items were assigned to conditions in the counterbalancing lists.

Catch the star! Spatial information activates the manual motor system

Previous research demonstrated a close bidirectional relationship between spatial attention and the manual motor system. However, it is unclear whether an explicit hand movement is necessary for this relationship to appear. A novel method with high temporal resolution–bimanual grip force registration–sheds light on this issue. Participants held two grip force sensors while being presented with lateralized stimuli (exogenous attentional shifts, Experiment 1), left- or right-pointing central arrows (endogenous attentional shifts, Experiment 2), or the words "left" or "right" (endogenous attentional shifts, Experiment 3). There was an early interaction between the presentation side or arrow direction and grip force: lateralized objects and central arrows led to a larger increase of the ipsilateral force and a smaller increase of the contralateral force. Surprisingly, words led to the opposite pattern: larger force increase in the contralateral hand and smaller force increase in the ipsilateral hand. The effect was stronger and appeared earlier for lateralized objects (60 ms after stimulus presentation) than for arrows (100 ms) or words (250 ms). Thus, processing visuospatial information automatically activates the manual motor system, but the timing and direction of this effect vary depending on the type of stimulus.

How head and visual movements affect evaluations of food products

Many studies suggest that specific movements or postures with shared social meaning can influence mainly verbal stimuli evaluation. On the other hand, several visuospatial biases can interact with this influence. Thus, we tested whether both head and stimuli movements can influence individual attitude towards food pictures. In two experiments, we used images of common foods with a weak positive valence in association with two kinds of movements. In Experiment 1, head movement was induced by presenting food pictures with a vertical or horizontal continuous movement on a computer screen. Conversely, Experiment 2 was conducted to test the effects of participants' own head movements with respect to the same food pictures presented in a fixed position. In neither case did head movements influence product evaluation. However, Experiment 1 revealed that the continuous movement left-right-left in the horizontal condition improved the desire to buy and eat, as well as the willingness to pay for the product shown. Two further experiments, the Experiments 3 and 4 demonstrated, respectively, that this effect disappears if the stimulus does not make the return direction, and that it does not depend on the starting or final placement of the images on the screen. These findings are discussed in the context of embodied cognition and visuospatial bias theories.

Remembering spatial words: Sensorimotor simulation affects verbal recognition memory

Previous evidence shows that words with implicit spatial meaning or metaphorical spatial associations are perceptually simulated and can guide attention to associated locations (e.g., bird-upward location). In turn, simulated representations interfere with visual perception at an associated location. The present study investigates the effect of spatial associations on short-term verbal recognition memory to disambiguate between modal and amodal accounts of spatial interference effects across two experiments. Participants in both experiments encoded words presented in congruent and incongruent locations. Congruent and incongruent locations were based on an independent norming task. In Experiment 1, an auditorily presented word probed participants' memory as they were visually cued to either the original location of the probe word or a diagonal location at retrieval. In Experiment 2, there was no cue at retrieval but a neutral encoding condition in which words normed to central locations were shown. Results show that spatial associations affected memory performance although spatial information was neither relevant nor necessary for successful retrieval: Words in Experiment 1 were retrieved more accurately when there was a visual cue in the congruent location at retrieval but only if they were encoded in a non-canonical position. A visual cue in the congruent location slowed down memory performance when retrieving highly imageable words. With no cue at retrieval (Experiment 2), participants were better at remembering spatially congruent words as opposed to neutral words. Results provide evidence in support of sensorimotor simulation in verbal memory and a perceptual competition account of spatial interference effect.

Motor Cortex Causally Contributes to Vocabulary Translation following Sensorimotor-Enriched Training

The role of the motor cortex in perceptual and cognitive functions is highly controversial. Here, we investigated the hypothesis that the motor cortex can be instrumental for translating foreign language vocabulary. Human participants of both sexes were trained on foreign language (L2) words and their native language translations over 4 consecutive days. L2 words were accompanied by complementary gestures (sensorimotor enrichment) or pictures (sensory enrichment). Following training, participants translated the auditorily presented L2 words that they had learned. During translation, repetitive transcranial magnetic stimulation was applied bilaterally to a site within the primary motor cortex (Brodmann area 4) located in the vicinity of the arm functional compartment. Responses within the stimulated motor region have previously been found to correlate with behavioral benefits of sensorimotor-enriched L2 vocabulary learning. Compared to sham stimulation, effective perturbation by repetitive transcranial magnetic stimulation slowed down the translation of sensorimotor-enriched L2 words, but not sensory-enriched L2 words. This finding suggests that sensorimotor-enriched training induced changes in L2 representations within the motor cortex, which in turn facilitated the translation of L2 words. The motor cortex may play a causal role in precipitating sensorimotor-based learning benefits, and may directly aid in remembering the native language translations of foreign language words following sensorimotor-enriched training. These findings support multisensory theories of learning while challenging reactivation-based theories.SIGNIFICANCE STATEMENT Despite the potential for sensorimotor enrichment to serve as a powerful tool for learning in many domains, its underlying brain mechanisms remain largely unexplored. Using transcranial magnetic stimulation and a foreign language (L2) learning paradigm, we found that sensorimotor-enriched training can induce changes in L2 representations within the motor cortex, which in turn causally facilitate the translation of L2 words. The translation of recently acquired L2 words may therefore rely not only on auditory information stored in memory or on modality-independent L2 representations, but also on the sensorimotor context in which the words have been experienced.

Early is left and up: Saccadic responses reveal horizontal and vertical spatial associations of serial order in working memory

Maintaining serial order in working memory is crucial for cognition. Recent theories propose that serial information is achieved by positional coding of items on a spatial frame of reference. In line with this, an early-left and late-right spatial-positional association of response code (SPoARC) effect has been established. Various theoretical accounts have been put forward to explain the SPoARC effect (the mental whiteboard hypothesis, conceptual metaphor theory, polarity correspondence, or the indirect spatial-numerical association effect). Crucially, while all these accounts predict a left-to-right orientation of the SPoARC effect, they make different predictions regarding the direction of a possible vertical SPoARC effect. In this study, we therefore investigated SPoARC effects along the horizontal and vertical spatial dimension by means of saccadic responses. We replicated the left-to-right horizontal SPoARC effect and established for the first time an up-to-down vertical SPoARC effect. The direction of the vertical SPoARC effect was in contrast to that predicted by metaphor theory, polarity correspondence, or by the indirect spatial-numerical association effect. Rather, our results support the mental whiteboard-hypothesis, according to which positions can be flexibly coded on an internal space depending on the task demands. We also found that the strengths of the horizontal and vertical SPoARC effects were correlated, showing that some people are more prone than others to use spatial references for position coding. Our results therefore suggest that context templates used for position marking are not necessarily spatial in nature but depend on individual strategy preferences.

Strategic Spatial Anchoring as Cognitive Compensation During Word Categorization in Parkinson's Disease: Evidence from Eye Movements

The association between a word and typical location (e.g., cloud-up) appears to modulate healthy individuals' response times and visual attention. This study examined whether similar effects can be observed in a clinical population characterized by difficulties in both spatial representation and lexical processing. In an eye-tracking experiment, participants categorized spoken words as either up-associated or down-associated. Parkinson's disease patients exhibited a tendency to maintain their visual attention in the upper half of the screen, however, this tendency was significantly lower when participants categorized concepts as down-associated. Instead, the control group showed no preference for either the upper or lower half of the screen. We argue that Parkinson's disease patients present an over-reliance on space during word categorization as a form of cognitive compensation. Such compensation reveals that this clinical population may use spatial anchoring when categorizing words with a spatial association, even in the absence of explicit spatial cues.

How words ripple through bilingual hands: Motor-language coupling during L1 and L2 writing

The speed of our hand movements can be affected by concurrent processing of manual action verbs (MaVs). Whereas this phenomenon is well established for native languages (L1s), it remains underexplored in late foreign languages (L2s), especially during highly automatized tasks. Here we timed keystroke activity while Spanish-English bilinguals copied MaVs, non-manual action verbs, and non-action verbs in their L1 and L2. Motor planning and execution dynamics were indexed by first-letter lag (the time-lapse between word presentation and first keystroke) and whole-word lag (the time-lapse between first and last keystroke), respectively. Despite yielding no effects on motor planning, MaVs facilitated typing execution in L1 but delayed it in L2, irrespective of the subjects' typing skills, age of L2 learning, and L2 competence. Therefore, motor-language coupling effects seem to be present in both languages though they can arise differently in each. These results extend language grounding models, illuminating the role of embodied mechanisms throughout life.

Nodding and shaking of the head as simulated approach and avoidance responses

Our recent study within the embodiment perspective showed that the evaluation of true and false information activates the simulation of vertical and horizontal head movements involved in nodding and shaking of the head (Moretti & Greco, 2018). This result was found in an explicit evaluation task where motion detection software was deployed to enable participants to assess a series of objectively true or false statements by moving them with the head vertically and horizontally on a computer screen, under conditions of compatibility and incompatibility between simulated and performed action. This study replicated that experiment, but with subjective statements about liked and disliked food, in both explicit and implicit evaluation tasks. Two experiments, plus one control experiment, were devised to test the presence of a motor-affective compatibility effect (vertical-liked; horizontal-disliked) and whether the motor-semantic compatibility found with objective statements (vertical-true; horizontal-false) could be a sub-effect of a more general and automatic association (vertical-accepted; horizontal-refused). As expected, response times were shorter when statements about liked foods and disliked foods were moved vertically and horizontally respectively by making head movements, even when participants were not explicitly required to evaluate them. In contrast, the truth compatibility effect only occurred in the explicit evaluation task. Overall results support the idea that head-nodding and shaking are simulated approach-avoidance responses. Different aspects of the meaning of these gestures and the practical implications of the study for cognitive and social research are discussed.

Too late to be grounded? Motor resonance for action words acquired after middle childhood

Though well established for languages acquired in infancy, the role of embodied mechanisms remains poorly understood for languages learned in middle childhood and adulthood. To bridge this gap, we examined 34 experiments that assessed sensorimotor resonance during processing of action-related words in real and artificial languages acquired since age 7 and into adulthood. Evidence from late bilinguals indicates that foreign-language action words modulate neural activity in motor circuits and predictably facilitate or delay physical movements (even in an effector-specific fashion), with outcomes that prove partly sensitive to language proficiency. Also, data from newly learned vocabularies suggest that embodied effects emerge after brief periods of adult language exposure, remain stable through time, and hinge on the performance of bodily movements (and, seemingly, on action observation, too). In sum, our work shows that infant language exposure is not indispensable for the recruitment of embodied mechanisms during language processing, a finding that carries non-trivial theoretical, pedagogical, and clinical implications for neurolinguistics, in general, and bilingualism research, in particular.

Hands show where things are: The close similarity between sign and natural space

Many of the signs produced across sign languages are iconic, in the sense that they resemble the concepts they represent. We examined whether location, one of basic sign parameters along with handshape and movement, is systematically used for purposes of iconicity. Our findings revealed a mapping of vertical sign space that is exploited in its entirety for encoding typical locations in natural space. In all of the twenty sign languages we analyzed, signs were more likely to have high locations with concepts typically occurring in high vs. low regions of natural space (e.g., cloud vs. root). Furthermore, the height of signs produced to identify a visual object varied depending on object position (e.g., it was higher for basketball in the basket than basketball on the floor). It thus appears that signing space is permeable to semantic and episodic features, and iconicity plays a crucial role in making signing space so adaptable.

When words are upside down: Language-space associations in children and adults

The nonlinguistic sensorimotor experiences and contexts that accompany language learning are often assumed to play an integral role in meaning representation. However, despite embodied models of language comprehension being well established in the literature, evidence is mainly derived from adult studies. For example, it has been shown that for adult comprehenders there is a close link between language and space, resulting in automatic reactivation of a referent's spatial location during word processing. In the current study, we investigated whether this link can also be found in young children (4;8-7;5 years;months). In a Stroop-like paradigm, our participants responded to a colored circle on the screen with an upward or downward arm movement after auditorily perceiving a task-irrelevant noun. We chose noun's with referents that are typically located either in upper or lower space (e.g., "sun" vs. "shoe"). Although it was not necessary to process the meaning of the nouns in order to fulfill the task, we found a spatial compatibility effect that was similar to language-space associations previously reported for adult participants. We conclude that sensorimotor experiences in the spatial domain seem to play a role during meaning processing from early childhood onward, a finding crucial for embodied models of language comprehension.

Perceptual Experience Norms for 506 Russian Nouns: Modality Rating, Spatial Localization, Manipulability, Imageability and Other Variables

A number of new psycholinguistic variables has been proposed during the last years within embodied cognition framework: modality experience rating (i.e., relationship between words and images of a particular perceptive modality-visual, auditory, haptic etc.), manipulability (the necessity for an object to interact with human hands in order to perform its function), vertical spatial localization. However, it is not clear how these new variables are related to each other and to such traditional variables as imageability, AoA and word frequency. In this article, normative data on the modality (visual, auditory, haptic, olfactory, and gustatory) ratings, vertical spatial localization of the object, manipulability, imageability, age of acquisition, and subjective frequency for 506 Russian nouns are presented. Strongest correlations were observed between olfactory and gustatory modalities (.81), visual modality and imageability (.78), haptic modality and manipulability (.7). Other modalities also significantly correlate with imageability: olfactory (.35), gustatory (.24), and haptic (.67). Factor analysis divided variables into four groups where visual and haptic modality ratings were combined with imageability, manipulability and AoA (the first factor); word length, frequency and AoA formed the second factor; olfactory modality was united with gustatory (the third factor); spatial localization only is included in the fourth factor. Present norms of imageability and AoA are consistent with previous as correlation analysis has revealed. The complete database can be downloaded from supplementary material.

Magnitude or Multitude - What Counts?

Recent studies revealed an association of low or high numbers (e.g., 1 vs. 9) and word semantics referring to entities typically found in upper or lower space (e.g., roof vs. root) indicating overlapping spatial representations. Another line of research revealed a similar association of grammatical number as a syntactic aspect of language and physical space: singular words were associated with left and plural words with right - resembling spatial-numerical associations of low numbers with left and high numbers with right. The present study aimed at integrating these lines of research by evaluating both types of spatial relations in one experiment. In a lexical decision task, pairs of a numerical cue and a subsequent plural noun were presented. For word with spatial associations (e.g., roofs vs. roots) number magnitude was expected to serve as a spatial cue. For spatially neutral words (e.g., tables) numbers were expected to cue multitude. Results showed the expected congruency-effect between the numbers and words with spatial associations (i.e., small numbers facilitate responses to down-words and high numbers to up-words). However, no effect was found for numbers and spatially neutral words. This seems to indicate that spatial aspects of word meaning may be related more closely to the magnitude of numbers than grammatical number is to the multitude reflected by numbers - at least in the current experimental setting, where only plural words were presented.

Neuroticism, schizotypy, and scale anchors influence eye movement behaviour in the visual exploration of abstract art: An exploratory study

The same piece of artwork can attract both admiration and rejection from different people. One potential explanation for this effect is individual differences in perceptual biases, which influence the way in which we see different aspects of the same image. We explored the relationship between individual differences (i.e., personality) and eye movements for examinations of abstract art. Images were presented for 5000ms, after which participants judged aesthetic appeal and perceived value using visual analogue scales. Scale anchor labels (Looks Good/Looks Bad; $0/$5000) were counterbalanced between participants such that positive labels were on the left half of the time and on the right half of the time. Overall, more fixations occurred to the right and upper visual fields. Neuroticism significantly predicted the proportion of fixations to the left, whereas cognitive disorganisation negatively predicted the proportion of fixations to upper space. Participants found images more aesthetically pleasing and more valuable when positive anchors were on the left. Findings demonstrate that personality traits influence fixation patterns. Further, the positioning of positive anchor labels on the left leads to higher ratings of visual stimuli.

Activating concepts by activating experiential traces: investigations with a series of anagram solution tasks

According to the experiential-simulations view of language, words automatically activate experiential traces that stem from the reader’s interactions with their referents. Here, we focus on the corresponding influence in the opposite direction. By means of an anagram-solving task we investigated whether activating spatial experiential traces would activate the corresponding concepts, which in turn facilitates access to associated words. Participants solved anagrams of nouns associated with the ocean or the sky (e.g. dolphin = “ dplhion” or cloud = “ cdulo”). In six experiments we provided additional context information such as positional information (presenting the anagram at the top or the bottom of the screen), or pictorial information that either matched the ocean and sky theme or not, or both positional and pictorial information. Anagrams were solved faster when the position of the anagram was congruent with the location of the noun’s referent in the real world, but only when presented on the background of an ocean-sky picture. Thus, activating experiential traces indeed seems to activate related concepts but positional information alone is not enough to find facilitation in an anagram solving task. Rather what is needed is a whole set of traces that sufficiently narrow down the number of related concepts.

Do I need to have my hands free to understand hand-related language? Investigating the functional relevance of experiential simulations

Theories of embodiment state that people mentally simulate the described situations and events during language comprehension. While several studies have provided evidence that these simulations exist, it is still unclear whether they are functionally relevant for comprehension. To investigate this question, we studied the effects of a secondary task on the processing of hand- and foot-related nouns. The secondary task occupied either the hand or the foot system, thereby impeding hand- or foot-related simulations, respectively. Participants performed a lexical decision task by responding to the presented nouns with their left hand or foot, depending on the color of the words, while withholding their response to pseudowords. In half of the experimental blocks, participants performed a simultaneous tapping task with their right hand (Experiment 1) or foot (Experiment 2). If simulations are functionally relevant for comprehension, the secondary task should affect the processing of hand words to a larger degree than the processing foot words in Experiment 1 and vice versa in Experiment 2. In both experiments, hand responses were faster for hand words than foot words, whereas the opposite was true for foot responses. This finding indicates that participants indeed simulated the words' meanings. Importantly, there was no difference between the influence of the hand tapping and the foot tapping task on lexical decision times to hand and foot words, indicating that experiential simulation might just be an optional by-product of language processing.

Objects but not concepts modulate the size of the attended region

Here we investigated the types of stimuli that modulate the size of the attentional spotlight. In particular, it has been previously shown that conceptual cues that either directly refer to or are semantically related to particular spatial locations can shift attention to that location (known as “ conceptual cueing”). For example, reading the word sun or joy can shift attention upward whereas the word boot or hostile can shift attention downward. Here, therefore, we tested whether words could modulate the size of the attended area. Across five experiments, we found that words that either directly referred to, or were abstractly associated with, particular sizes (small versus large) did not change the size of the attentional spotlight, whereas the presence of differently sized stimuli did, as evidenced by faster responses to targets when the spotlight is small than when it is large. This suggests that physical but not conceptual inducers can modulate the size of the attentional spotlight. This highlights an important difference between the regulation of spotlight size and shifts of attention, supporting the notion that they are subserved by distinct mechanisms.

Language use statistics and prototypical grapheme colours predict synaesthetes' and non-synaesthetes' word-colour associations

Synaesthesia is the neuropsychological phenomenon in which individuals experience unusual sensory associations, such as experiencing particular colours in response to particular words. While it was once thought the particular pairings between stimuli were arbitrary and idiosyncratic to particular synaesthetes, there is now growing evidence for a systematic psycholinguistic basis to the associations. Here we sought to assess the explanatory value of quantifiable lexical association measures (via latent semantic analysis; LSA) in the pairings observed between words and colours in synaesthesia. To test this, we had synaesthetes report the particular colours they experienced in response to given concept words, and found that language association between the concept and colour words provided highly reliable predictors of the reported pairings. These results provide convergent evidence for a psycholinguistic basis to synaesthesia, but in a novel way, showing that exposure to particular patterns of associations in language can predict the formation of particular synaesthetic lexical-colour associations. Consistent with previous research, the prototypical synaesthetic colour for the first letter of the word also played a role in shaping the colour for the whole word, and this effect also interacted with language association, such that the effect of the colour for the first letter was stronger as the association between the concept word and the colour word in language increased. Moreover, when a group of non-synaesthetes were asked what colours they associated with the concept words, they produced very similar reports to the synaesthetes that were predicted by both language association and prototypical synaesthetic colour for the first letter of the word. This points to a shared linguistic experience generating the associations for both groups.

Implied Spatial Meaning and Visuospatial Bias: Conceptual Processing Influences Processing of Visual Targets and Distractors

Concepts with implicit spatial meaning (e.g., "hat", "boots") can bias visual attention in space. This result is typically found in experiments with a single visual target per trial, which can appear at one of two locations (e.g., above vs. below). Furthermore, the interaction is typically found in the form of speeded responses to targets appearing at the compatible location (e.g., faster responses to a target above fixation, after reading "hat"). It has been argued that these concept-space interactions could also result from experimentally-induced associations between the binary set of locations and the conceptual categories with upward and downward meaning. Thus, rather than reflecting a conceptually driven spatial bias, the effect could reflect a benefit for compatible cue-target sequences that occurs only after target onset. We addressed these concerns by going beyond a binary set of locations and employing a search display consisting of four items (above, below, left, and right). Within each search trial, before performing a visual search task, participants performed a conceptual task involving concepts with implicit upward or downward meaning. The search display, in addition to including a target, could also include a salient distractor. Assuming a conceptually driven visual bias, we expected to observe, first, a benefit for target processing at the compatible location and, second, an increase in the cost of the salient distractor. The findings confirmed both predictions, suggesting that concepts do indeed generate a spatial bias. Finally, results from a control experiment, without the conceptual task, suggest the presence of an axis-specific effect, in addition to the location-specific effect, suggesting that concepts might cause both location-specific and axis-specific spatial bias. Taken together, our findings provide additional support for the involvement of spatial processing in conceptual understanding.

Bow Your Head in Shame, or, Hold Your Head Up with Pride: Semantic Processing of Self-Esteem Concepts Orients Attention Vertically

Embodied cognition holds that abstract concepts are grounded in perceptual-motor simulations. If a given embodied metaphor maps onto a spatial representation, then thinking of that concept should bias the allocation of attention. In this study, we used positive and negative self-esteem words to examine two properties of conceptual cueing. First, we tested the orientation-specificity hypothesis, which predicts that conceptual cues should selectively activate certain spatial axes (in this case, valenced self-esteem concepts should activate vertical space), instead of any spatial continuum. Second, we tested whether conceptual cueing requires semantic processing, or if it can be achieved with shallow visual processing of the cue words. Participants viewed centrally presented words consisting of high or low self-esteem traits (e.g., brave, timid) before detecting a target above or below the cue in the vertical condition, or on the left or right of the word in the horizontal condition. Participants were faster to detect targets when their location was compatible with the valence of the word cues, but only in the vertical condition. Moreover, this effect was observed when participants processed the semantics of the word, but not when processing its orthography. The results show that conceptual cueing by spatial metaphors is orientation-specific, and that an explicit consideration of the word cues' semantics is required for conceptual cueing to occur.

What's up? Emotion-specific activation of vertical space during language processing

The relationship between language processing and vertical space has been shown for various groups of words including valence words, implicit location words, and words referring to religious concepts. However, it remains unclear whether these are single phenomena or whether there is an underlying common mechanism. Here, we show that the evaluation of word valence interacts with motor responses in the vertical dimension, with positive (negative) evaluations facilitating upward (downward) responses. When valence evaluation was not required, implicit location words (e.g., bird, shoe) influenced motor responses whereas valence words (e.g., kiss, hate) did not. Importantly, a subset of specific emotional valence words that are commonly associated with particular bodily postures (e.g., proud→upright; sad→slouched) did automatically influence motor responses. Together, this suggests that while the vertical spatial dimension is not directly activated by word valence, it is activated when processing words referring to emotional states with stereotypical bodily-postures. These results provide strong evidence that the activation of spatial associations during language processing is experience-specific in nature and cannot be explained with reference to a general mapping between all valence words and space (i.e., all positive and negative words generally relate to spatial processing). These findings support the experiential view of language comprehension, suggesting that the automatic reactivation of bodily experiences is limited to word groups referring to emotions or entities directly associated with spatial experiences (e.g., posture or location in the world).

Musical metaphors: evidence for a spatial grounding of non-literal sentences describing auditory events

This study investigated whether the spatial terms high and low, when used in sentence contexts implying a non-literal interpretation, trigger similar spatial associations as would have been expected from the literal meaning of the words. In three experiments, participants read sentences describing either a high or a low auditory event (e.g., The soprano sings a high aria vs. The pianist plays a low note). In all Experiments, participants were asked to judge (yes/no) whether the sentences were meaningful by means of up/down (Experiments 1 and 2) or left/right (Experiment 3) key press responses. Contrary to previous studies reporting that metaphorical language understanding differs from literal language understanding with regard to simulation effects, the results show compatibility effects between sentence implied pitch height and response location. The results are in line with grounded models of language comprehension proposing that sensory motor experiences are being elicited when processing literal as well as non-literal sentences.

Relating numeric cognition and language processing: do numbers and words share a common representational platform?

Numerical processing and language processing are both grounded in space. In the present study we investigated whether these are fully independent phenomena, or whether they share a common basis. If number processing activates spatial dimensions that are also relevant for understanding words, then we can expect that processing numbers may influence subsequent lexical access to words. Specifically, if high numbers relate to upper space, then they can be expected to facilitate understanding of words such as bird that are having referents typically found in the upper vertical space. The opposite should hold for low numbers. These should facilitate the understanding of words such as ground referring to entities with referents in the lower vertical space. Indeed, in two experiments we found evidence for such an interaction between number and word processing. By eliminating a contribution of linguistic factors gained from additional investigations on large text corpora, this strongly suggests that understanding numbers and language is based on similar modal representations in the brain. The implications of these findings for a broader perspective on grounded cognition will be discussed.

Embodiment and second-language: automatic activation of motor responses during processing spatially associated L2 words and emotion L2 words in a vertical Stroop paradigm

Converging evidence suggests that understanding our first-language (L1) results in reactivation of experiential sensorimotor traces in the brain. Surprisingly, little is known regarding the involvement of these processes during second-language (L2) processing. Participants saw L1 or L2 words referring to entities with a typical location (e.g., star, mole) (Experiment 1 & 2) or to an emotion (e.g., happy, sad) (Experiment 3). Participants responded to the words' ink color with an upward or downward arm movement. Despite word meaning being fully task-irrelevant, L2 automatically activated motor responses similar to L1 even when L2 was acquired rather late in life (age >11). Specifically, words such as star facilitated upward, and words such as root facilitated downward responses. Additionally, words referring to positive emotions facilitated upward, and words referring to negative emotions facilitated downward responses. In summary our study suggests that reactivation of experiential traces is not limited to L1 processing.

Where numbers meet words: a common ventral network for semantic classification

Recent research has shown that both language and number processing are clear examples of distributed and connected processing in the human brain, emphasizing the importance of white matter connections between the associated cortex sites. Against this background we hypothesized joint cognitive processes and functions in a cross-domain manner to be reflected by the involvement of specific white matter tracts. Therefore, we evaluated white matter connectivity for the specific cognitive process of semantic classification, which is an integral part of tasks commonly employed to investigate the neural correlates of language and number processing. In line with our expectations, fiber tracking results clearly indicated a common ventral network for semantic classification for the domains of language and number processing. Thereby, the present data are hard to reconcile with a localizationalist view on processing characteristics of the human brain, but strongly suggest that white matter connectivity should be considered when investigating the neural underpinnings of human cognition.