Towards Understanding the Task Dependency of Embodied Language Processing: The Influence of Colour During Language-Vision Interactions

The influence of sentence focus on mental simulation: A possible cause of ACE instability

Recent studies have revealed the instability of the action-sentence compatibility effect (ACE). The current study was designed to demonstrate the hypothesis that the instability of the ACE may be attributed to the instability of focused information in a sentence. A pilot study indicated that the focused information of sentences was relatively stable in the sentence-picture verification task but exhibited significant interindividual variability in the action-sentence compatibility paradigm in previous studies. Experiments 1 and 2 examined the effect of sentence focus on the shape match effect and the ACE by manipulating the focused information of sentences using the focus marker word "" (is). Experiment 1 found that the shape match effect occurred in the original sentence, while it disappeared when the word "" (is) was used to make an object noun no longer the focus of a sentence. Experiment 2 failed to observe the ACE regardless of whether the sentence focus was on the action information. Experiment 3 modified the focus manipulation to observe its impact on the ACE using different fonts and underlines to highlight the focused information. The results indicated that the ACE only occurred when the action information was the sentence focus. These findings suggest that sentence focus influences mental simulation, and the instability of the ACE is likely to be associated with the instability of sentence focus in previous studies. This outcome highlights the crucial role of identifying specific information as the critical element expressed in the current linguistic context for successful simulation.

Flexible Conceptual Representations

A view that has been gaining prevalence over the past decade is that the human conceptual system is malleable, dynamic, context-dependent, and task-dependent, that is, flexible. Within the flexible conceptual representation framework, conceptual representations are constructed ad hoc, forming a different, idiosyncratic instantiation upon each occurrence. In this review, we scrutinize the neurocognitive literature to better understand the nature of this flexibility. First, we identify some key characteristics of these representations. Next, we consider how these flexible representations are constructed by addressing some of the open questions in this framework: We review the age-old question of how to reconcile flexibility with the apparent need for shareable stable definitions to anchor meaning and come to mutual understanding, as well as some newer questions we find critical, namely, the nature of relations among flexible representations, the role of feature saliency in activation, and the viability of all-or-none feature activations. We suggest replacing the debate about the existence of a definitional stable core that is obligatorily activated with a question of the degree and probability of activation of the information constituting a conceptual representation. We rely on published works to suggest that (1) prior featural salience matters, (2) feature activation may be graded, and (3) Bayesian updating of prior information according to current demands offers a viable account of how flexible representations are constructed. This proposal provides a theoretical mechanism for incorporating a changing momentary context into a constructed representation, while still preserving some of the concept's constituent meaning.

Experiential traces first: Does holding a location in visuospatial working memory affect the processing of space-associated words?

This study aimed to systematically examine whether actively maintaining a visual location in working memory can influence the processing of spatially related words. In five experiments, we asked participants to maintain either the location or the shape of a visually presented stimulus in working memory so that it could later be compared with a test stimulus concerning the relevant target features. In between, we presented participants with words that refer to objects typically encountered in the upper or lower vertical space (roof vs. root, respectively). The task participants performed as a response to these words differed between experiments. In Experiments 1-3, participants performed a lexical decision task, in Experiment 4 they performed a semantic task (deciding whether the word refers to an occupation), and in Experiment 5 they performed a spatial task (deciding whether the word refers to something in the upper or lower visual field.) Only in Experiment 5 did we observe an interaction between the position of the visual stimulus held in working memory (up vs. down) and the meaning of the spatial words (associated with up vs. down). Our results therefore suggest that actively maintaining a stimulus location in working memory does not automatically affect the processing of spatially related words, but does so if the relevant spatial dimension is made highly salient by the task. The results are thus in line with studies showing a strong context-dependency of embodiment effects and thus allow the conclusion that language processing proper is not operating on a sensorimotor representational format.

Do linguistic stimuli activate experiential colour traces related to the entities they refer to and, if so, under what circumstances?

The simulation view of language comprehension holds that lexical-semantic access prompts the re-enactment of sensorimotor experiences that regularly accompany word use. For the colour domain, this suggests that reading about a stop sign reactivates experiences involving the perception of the stop sign and hence experiences involving the colour red. However, it is still not clear what circumstances would limit reactivation of colour experiences during comprehension, if the activation takes place. To address this question, we varied in our study the conditions in which the target colour stimuli appeared. The experimental stimuli were individual words (Experiment (Exp.) 1, Exp. 7, 8) or sentences (Exp. 2-6) referring to objects with a typical colour of either green or red (e.g., cucumber or raspberry). Across experiments, we manipulated the presence of fillers (present or not), and whether fillers referred to objects with other colour (e.g., honey) or objects without any particular colour (e.g., car). The stimuli were presented along with two clickable "yes" and "no" buttons, one of which was red and the other green. Location and button colour varied from trial to trial. The tasks were lexical decision (Exp. 1, Exp. 7-8) and sensibility judgement (Exp. 2-6). We observed faster response times in the match vs mismatch condition in all word-based experiments, but only in those sentence-based experiments that did not have fillers. This suggests that comprehenders indeed reactivate colour experiences when processing linguistic stimuli referring to objects with a typical colour, but this activation seems to occur only under certain circumstances.

Good enough processing: what have we learned in the 20 years since Ferreira et al. (2002)?

Traditionally, language processing has been thought of in terms of complete processing of the input. In contrast to this, Ferreira and colleagues put forth the idea of good enough processing. The proposal was that during everyday processing, ambiguities remain unresolved, we rely on heuristics instead of full analyses, and we carry out deep processing only if we need to for the task at hand. This idea has gathered substantial traction since its conception. In the current work, I review the papers that have tested the three key claims of good enough processing: ambiguities remain unresolved and underspecified, we use heuristics to parse sentences, and deep processing is only carried out if required by the task. I find mixed evidence for these claims and conclude with an appeal to further refinement of the claims and predictions of the theory.

Linguistic labels cue biological motion perception and misperception

Linguistic labels exert a particularly strong top-down influence on perception. The potency of this influence has been ascribed to their ability to evoke category-diagnostic features of concepts. In doing this, they facilitate the formation of a perceptual template concordant with those features, effectively biasing perceptual activation towards the labelled category. In this study, we employ a cueing paradigm with moving, point-light stimuli across three experiments, in order to examine how the number of biological motion features (form and kinematics) encoded in lexical cues modulates the efficacy of lexical top-down influence on perception. We find that the magnitude of lexical influence on biological motion perception rises as a function of the number of biological motion-relevant features carried by both cue and target. When lexical cues encode multiple biological motion features, this influence is robust enough to mislead participants into reporting erroneous percepts, even when a masking level yielding high performance is used.