Integration and Prediction in Language Processing: A Synthesis of Old and New

Current theories of language processing emphasize prediction as a mechanism to facilitate comprehension, which contrasts with the state of the field a few decades ago, when prediction was rarely mentioned. We argue that the field of psycholinguistics would benefit from revisiting these earlier theories of comprehension which attempted to explain integration and the processes underlying the formation of rich representations of linguistic input, and that emphasized informational newness over redundancy. We suggest further that integration and anticipation may be complementary mechanisms that operate on elaborated, coherent discourse representations, supporting enhanced comprehension and memory. In addition, the traditional emphasis on language as a tool for communication implies that much linguistic content will be nonredundant; moreover, the purpose of anticipation is probably not to permit the prediction of exact lexical or syntactic forms, but instead to induce a state of preparedness that allows the comprehender to be receptive to new information, thus facilitating its processing.

Shared neural and cognitive mechanisms in action and language: The multiscale information transfer framework

This review compares how humans process action and language sequences produced by other humans. On the one hand, we identify commonalities between action and language processing in terms of cognitive mechanisms (e.g., perceptual segmentation, predictive processing, integration across multiple temporal scales), neural resources (e.g., the left inferior frontal cortex), and processing algorithms (e.g., comprehension based on changes in signal entropy). On the other hand, drawing on sign language with its particularly strong motor component, we also highlight what differentiates (both oral and signed) linguistic communication from nonlinguistic action sequences. We propose the multiscale information transfer framework (MSIT) as a way of integrating these insights and highlight directions into which future empirical research inspired by the MSIT framework might fruitfully evolve. This article is categorized under: Psychology > Language Linguistics > Language in Mind and Brain Psychology > Motor Skill and Performance Psychology > Prediction.

Abstract concepts, language and sociality: from acquisition to inner speech

The problem of representation of abstract concepts, such as 'freedom' and 'justice', has become particularly crucial in recent years, owing to the increased success of embodied and grounded views of cognition. We will present a novel view on abstract concepts and abstract words. Since abstract concepts do not have single objects as referents, children and adults might rely more on input from others to learn them; we, therefore, suggest that linguistic and social experience play an important role for abstract concepts. We will discuss evidence obtained in our and other laboratories showing that processing of abstract concepts evokes linguistic interaction and social experiences, leading to the activation of the mouth motor system. We will discuss the possible mechanisms that underlie this activation. Mouth motor system activation can be due to re-enactment of the experience of conceptual acquisition, which occurred through the mediation of language. Alternatively, it could be due to the re-explanation of the word meaning, possibly through inner speech. Finally, it can be due to a metacognitive process revealing low confidence in the meaning of our concepts. This process induces in us the need to rely on others to ask/negotiate conceptual meaning. We conclude that with abstract concepts language works as a social tool: it extends our thinking abilities and pushes us to rely on others to integrate our knowledge.This article is part of the theme issue 'Varieties of abstract concepts: development, use, and representation in the brain'.

Effects of meaningfulness on perception: Alpha-band oscillations carry perceptual expectations and influence early visual responses

Perceptual experience results from a complex interplay of bottom-up input and prior knowledge about the world, yet the extent to which knowledge affects perception, the neural mechanisms underlying these effects, and the stages of processing at which these two sources of information converge, are still unclear. In several experiments we show that language, in the form of verbal labels, both aids recognition of ambiguous “Mooney” images and improves objective visual discrimination performance in a match/non-match task. We then used electroencephalography (EEG) to better understand the mechanisms of this effect. The improved discrimination of images previously labeled was accompanied by a larger occipital-parietal P1 evoked response to the meaningful versus meaningless target stimuli. Time-frequency analysis of the interval between the cue and the target stimulus revealed increases in the power of posterior alpha-band (8–14 Hz) oscillations when the meaning of the stimuli to be compared was trained. The magnitude of the pre-target alpha difference and the P1 amplitude difference were positively correlated across individuals. These results suggest that prior knowledge prepares the brain for upcoming perception via the modulation of alpha-band oscillations, and that this preparatory state influences early (~120 ms) stages of visual processing.

Grammatical gender affects gender perception: Evidence for the structural-feedback hypothesis

Two experiments assessed the extent to which grammatical gender provides a predictive basis for bilinguals' judgments about perceptual gender. In both experiments, French-English bilinguals and native English monolinguals were consecutively presented with images of objects manipulated for their (i) conceptual gender association and (ii) grammatical gender category and were instructed to make a decision on a subsequent target face. The experiments differed in the implicitness of the association between the object primes and target faces. Results revealed that when prior knowledge sources such as conceptual gender can be strategically used to resolve the immediate task (Experiment 1), this information was readily extracted and employed. However, grammatical gender demonstrated a more robust and persisting effect on the bilinguals' judgments, indicating that the retrieval of obligatory grammatical information is automatic and modulates perceptual judgments (Experiment 2). These results suggest that grammar enables an effective and robust means to access prior knowledge which may be independent of task requirements.

Individual differences in nonverbal prediction and vocabulary size in infancy

Children who generate and update verbal predictions have larger vocabularies, suggesting that prediction may be a mechanism that supports language learning. We hypothesize that this relation is not confined to the domain of language, but instead signals a broader individual difference in information processing. To investigate this possibility, we tested infants (n = 50) in the early stages of vocabulary development (12-24 months) on their ability to generate and update nonverbal, visual predictions. In an eye-tracking task, a central fixation reliably preceded a peripheral target. Then, halfway through the experiment, the peripheral target began appearing on the opposite side. We assessed infants' proficiency in initiating anticipatory eye movements before and after the switch, and found that infants with larger vocabularies did not generate more predictions overall, but were more efficient in updating predictions to the new target side. These findings establish a link between nonverbal prediction and vocabulary in infancy, and suggest a promising means of addressing whether or not prediction abilities are causally related to language learning.

Emotion Perception as Conceptual Synchrony

Psychological research on emotion perception anchors heavily on an object perception analogy. We present static “cues,” such as facial expressions, as objects for perceivers to categorize. Yet in the real world, emotions play out as dynamic multidimensional events. Current theoretical approaches and research methods are limited in their ability to capture this complexity. We draw on insights from a predictive coding account of neural activity and a grounded cognition account of concept representation to conceive of emotion perception as a stream of synchronized conceptualizations between two individuals, which is supported and shaped by language. We articulate how this framework can illuminate the fundamental need to study culture, as well as other sources of conceptual variation, in unpacking conceptual synchrony in emotion. We close by suggesting that the conceptual system provides the necessary flexibility to overcome gaps in emotional synchrony.

The Active Inference Approach to Ecological Perception: General Information Dynamics for Natural and Artificial Embodied Cognition

The emerging neurocomputational vision of humans as embodied, ecologically embedded, social agents—who shape and are shaped by their environment—offers a golden opportunity to revisit and revise ideas about the physical and information-theoretic underpinnings of life, mind, and consciousness itself. In particular, the active inference framework (AIF) makes it possible to bridge connections from computational neuroscience and robotics/AI to ecological psychology and phenomenology, revealing common underpinnings and overcoming key limitations. AIF opposes the mechanistic to the reductive, while staying fully grounded in a naturalistic and information-theoretic foundation, using the principle of free energy minimization. The latter provides a theoretical basis for a unified treatment of particles, organisms, and interactive machines, spanning from the inorganic to organic, non-life to life, and natural to artificial agents. We provide a brief introduction to AIF, then explore its implications for evolutionary theory, ecological psychology, embodied phenomenology, and robotics/AI research. We conclude the paper by considering implications for machine consciousness.

Understanding environmental sounds in sentence context

There is debate about how individuals use context to successfully predict and recognize words. One view argues that context supports neural predictions that make use of the speech motor system, whereas other views argue for a sensory or conceptual level of prediction. While environmental sounds can convey clear referential meaning, they are not linguistic signals, and are thus neither produced with the vocal tract nor typically encountered in sentence context. We compared the effect of spoken sentence context on recognition and comprehension of spoken words versus nonspeech, environmental sounds. In Experiment 1, sentence context decreased the amount of signal needed for recognition of spoken words and environmental sounds in similar fashion. In Experiment 2, listeners judged sentence meaning in both high and low contextually constraining sentence frames, when the final word was present or replaced with a matching environmental sound. Results showed that sentence constraint affected decision time similarly for speech and nonspeech, such that high constraint sentences (i.e., frame plus completion) were processed faster than low constraint sentences for speech and nonspeech. Linguistic context facilitates the recognition and understanding of nonspeech sounds in much the same way as for spoken words. This argues against a simple form of a speech-motor explanation of predictive coding in spoken language understanding, and suggests support for conceptual-level predictions.

Using Neural Networks to Generate Inferential Roles for Natural Language

Neural networks have long been used to study linguistic phenomena spanning the domains of phonology, morphology, syntax, and semantics. Of these domains, semantics is somewhat unique in that there is little clarity concerning what a model needs to be able to do in order to provide an account of how the meanings of complex linguistic expressions, such as sentences, are understood. We argue that one thing such models need to be able to do is generate predictions about which further sentences are likely to follow from a given sentence; these define the sentence's "inferential role." We then show that it is possible to train a tree-structured neural network model to generate very simple examples of such inferential roles using the recently released Stanford Natural Language Inference (SNLI) dataset. On an empirical front, we evaluate the performance of this model by reporting entailment prediction accuracies on a set of test sentences not present in the training data. We also report the results of a simple study that compares human plausibility ratings for both human-generated and model-generated entailments for a random selection of sentences in this test set. On a more theoretical front, we argue in favor of a revision to some common assumptions about semantics: understanding a linguistic expression is not only a matter of mapping it onto a representation that somehow constitutes its meaning; rather, understanding a linguistic expression is mainly a matter of being able to draw certain inferences. Inference should accordingly be at the core of any model of semantic cognition.

The Two-Systems Account of Theory of Mind: Testing the Links to Social- Perceptual and Cognitive Abilities

According to the two-systems account of theory of mind (ToM), understanding mental states of others involves both fast social-perceptual processes, as well as slower, reflexive cognitive operations (Frith and Frith, 2008; Apperly and Butterfill, 2009). To test the respective roles of specific abilities in either of these processes we administered 15 experimental procedures to a large sample of 343 participants, testing ability in face recognition and holistic perception, language, and reasoning. ToM was measured by a set of tasks requiring ability to track and to infer complex emotional and mental states of others from faces, eyes, spoken language, and prosody. We used structural equation modeling to test the relative strengths of a social-perceptual (face processing related) and reflexive-cognitive (language and reasoning related) path in predicting ToM ability. The two paths accounted for 58% of ToM variance, thus validating a general two-systems framework. Testing specific predictor paths revealed language and face recognition as strong and significant predictors of ToM. For reasoning, there were neither direct nor mediated effects, albeit reasoning was strongly associated with language. Holistic face perception also failed to show a direct link with ToM ability, while there was a mediated effect via face recognition. These results highlight the respective roles of face recognition and language for the social brain, and contribute closer empirical specification of the general two-systems account.

The influence of communication mode on written language processing and beyond

Empirical evidence suggests a broad impact of communication mode on cognition at large, beyond language processing. Using a sign language since infancy might shape the representation of words and other linguistic stimuli - for example, incorporating in it the movements and signs used to express them. Once integrated into linguistic representations, this visuo-motor content can affect deaf signers' linguistic and cognitive processing.

Modeling the Predictive Social Mind

The social mind is tailored to the problem of predicting the mental states and actions of other people. However, social cognition researchers have only scratched the surface of the predictive social mind. We discuss here a new framework for explaining how people organize social knowledge and use it for social prediction. Specifically, we propose a multilayered framework of social cognition in which two hidden layers - the mental states and traits of others - support predictions about the observable layer - the actions of others. A parsimonious set of psychological dimensions structures each layer, and proximity within and across layers guides social prediction. This simple framework formalizes longstanding intuitions from social cognition, and in doing so offers a generative model for deriving new hypotheses about predictive social cognition.

Pacifier Overuse and Conceptual Relations of Abstract and Emotional Concepts

This study explores the impact of the extensive use of an oral device since infancy (pacifier) on the acquisition of concrete, abstract, and emotional concepts. While recent evidence showed a negative relation between pacifier use and children's emotional competence (Niedenthal et al., 2012), the possible interaction between use of pacifier and processing of emotional and abstract language has not been investigated. According to recent theories, while all concepts are grounded in sensorimotor experience, abstract concepts activate linguistic and social information more than concrete ones. Specifically, the Words As Social Tools (WAT) proposal predicts that the simulation of their meaning leads to an activation of the mouth (Borghi and Binkofski, 2014; Borghi and Zarcone, 2016). Since the pacifier affects facial mimicry forcing mouth muscles into a static position, we hypothesize its possible interference on acquisition/consolidation of abstract emotional and abstract not-emotional concepts, which are mainly conveyed during social and linguistic interactions, than of concrete concepts. Fifty-nine first grade children, with a history of different frequency of pacifier use, provided oral definitions of the meaning of abstract not-emotional, abstract emotional, and concrete words. Main effect of concept type emerged, with higher accuracy in defining concrete and abstract emotional concepts with respect to abstract not-emotional concepts, independently from pacifier use. Accuracy in definitions was not influenced by the use of pacifier, but correspondence and hierarchical clustering analyses suggest that the use of pacifier differently modulates the conceptual relations elicited by abstract emotional and abstract not-emotional. While the majority of the children produced a similar pattern of conceptual relations, analyses on the few (6) children who overused the pacifier (for more than 3 years) showed that they tend to distinguish less clearly between concrete and abstract emotional concepts and between concrete and abstract not-emotional concepts than children who did not use it (5) or used it for short (17). As to the conceptual relations they produced, children who overused the pacifier tended to refer less to their experience and to social and emotional situations, use more exemplifications and functional relations, and less free associations.

Social and Contextual Constraints on Embodied Perception

A number of papers have challenged research on physiological and psychological influences on perception by claiming to show that such findings can be explained by nonperceptual factors such as demand characteristics. Relatedly, calls for separating perception from judgment have been issued. However, such efforts fail to consider key processes known to shape judgment processes: people's inability to report accurately on their judgments, conversational dynamics of experimental research contexts, and misattribution and discounting processes. Indeed, the fact that initially observed effects of embodied influences disappear is predicted by an extensive amount of literature on judgments studied within social psychology. Thus, findings from such studies suggest that the initially presumed underlying processes are at work-namely, functional considerations that are informative in the context of preparing the body for action. In this article, I provide suggestions on how to conduct research on perception within the social constraints of experimental contexts.

Constructing emotion through simulation

Evidence increasingly suggests that simulations implement patterns of prior experience to construct one's current experience, whether that experience is oriented in the past, in the here and now, or in the future. Simulation is the mechanism by which the brain capitalizes on prior learning to efficiently navigate the situation at hand. This review examines the latest developments in theory and empirical research that address simulation during emotional phenomena. Integration of evidence across multiple literatures suggests that simulation accounts provide a unifying framework across many different emotional phenomena and highlights the importance of investigating dynamics, complexity, and variation in emotional experiences moving forward.

Predictive Processing and the Representation Wars

Clark has recently suggested that predictive processing advances a theory of neural function with the resources to put an ecumenical end to the "representation wars" of recent cognitive science. In this paper I defend and develop this suggestion. First, I broaden the representation wars to include three foundational challenges to representational cognitive science. Second, I articulate three features of predictive processing's account of internal representation that distinguish it from more orthodox representationalist frameworks. Specifically, I argue that it posits a resemblance-based representational architecture with organism-relative contents that functions in the service of pragmatic success, not veridical representation. Finally, I argue that internal representation so understood is either impervious to the three anti-representationalist challenges I outline or can actively embrace them.

Corticospinal excitability during the processing of handwritten and typed words and non-words

A number of studies have suggested that perception of actions is accompanied by motor simulation of those actions. To further explore this proposal, we applied Transcranial magnetic stimulation (TMS) to the left primary motor cortex during the observation of handwritten and typed language stimuli, including words and non-word consonant clusters. We recorded motor-evoked potentials (MEPs) from the right first dorsal interosseous (FDI) muscle to measure cortico-spinal excitability during written text perception. We observed a facilitation in MEPs for handwritten stimuli, regardless of whether the stimuli were words or non-words, suggesting potential motor simulation during observation. We did not observe a similar facilitation for the typed stimuli, suggesting that motor simulation was not occurring during observation of typed text. By demonstrating potential simulation of written language text during observation, these findings add to a growing literature suggesting that the motor system plays a strong role in the perception of written language.

Mixed emotions in the predictive brain

Understanding complex or mixed emotions first requires an exploration of the human nervous system underlying emotions, and indeed all experience. We review current research in neuroscience, which describes the brain as a predictive, internal model of the world that flexibly combines features from past experience to construct emotions. We argue that "mixed emotions" result when these features of past experience correspond to multiple emotion categories. Integrating event perception and cognitive linguistic theories, we propose that "mixed emotions" are perceived as an episode of distinct, linked emotional events due to attentional shifts which update the predicted model of experience. These proposed mechanisms have profound implications for the study of emotion; we conclude by suggesting methodological improvements for future research.

Changing What You See by Changing What You Know: The Role of Attention

Attending is a cognitive process that incorporates a person's knowledge, goals, and expectations. What we perceive when we attend to one thing is different from what we perceive when we attend to something else. Yet, it is often argued that attentional effects do not count as evidence that perception is influenced by cognition. I investigate two arguments often given to justify excluding attention. The first is arguing that attention is a post-perceptual process reflecting selection between fully constructed perceptual representations. The second is arguing that attention as a pre-perceptual process that simply changes the input to encapsulated perceptual systems. Both of these arguments are highly problematic. Although some attentional effects can indeed be construed as post-perceptual, others operate by changing perceptual content across the entire visual hierarchy. Although there is a natural analogy between spatial attention and a change of input, the analogy falls apart when we consider other forms of attention. After dispelling these arguments, I make a case for thinking of attention not as a confound, but as one of the mechanisms by which cognitive states affect perception by going through cases in which the same or similar visual inputs are perceived differently depending on the observer's cognitive state, and instances where cuing an observer using language affects what one sees. Lastly, I provide two compelling counter-examples to the critique that although cognitive influences on perception can be demonstrated in the laboratory, it is impossible to really experience them for oneself in a phenomenologically compelling way. Taken together, the current evidence strongly supports the thesis that what we know routinely influences what we see, that the same sensory input can be perceived differently depending on the current cognitive state of the viewer, and that phenomenologically salient demonstrations are possible if certain conditions are met.

The bilingual brain turns a blind eye to negative statements in the second language

Neurobilingualism research has failed to reveal significant language differences in the processing of affective content. However, the evidence to date derives mostly from studies in which affective stimuli are presented out of context, which is unnatural and fails to capture the complexity of everyday sentence-based communication. Here we investigated semantic integration of affectively salient stimuli in sentential context in the first- and second-language (L2) of late fluent Polish–English bilinguals living in the UK. The 19 participants indicated whether Polish and English sentences ending with a semantically and affectively congruent or incongruent adjective of controlled affective valence made sense while undergoing behavioral and electrophysiological recordings. We focused on the N400, a wave of event-related potentials known to index semantic integration. We expected N400 amplitude to index increased processing demands in L2 English comprehension and potential language–valence interactions to reveal differences in affective processing between languages. Contrary to our initial expectation, we found increased N400 for sentences in L1 Polish, possibly driven by greater affective salience of sentences in the native language. Critically, language interacted with affective valence, such that N400 amplitudes were reduced for English sentences ending in a negative fashion as compared to all other conditions. We interpreted this as a sign that bilinguals suppress L2 content embedded in naturalistic L2 sentences when it has negative valence, thus extending the findings of previous research on single words in clinical and linguistic research.

Implied motion language can influence visual spatial memory

How do language and vision interact? Specifically, what impact can language have on visual processing, especially related to spatial memory? What are typically considered errors in visual processing, such as remembering the location of an object to be farther along its motion trajectory than it actually is, can be explained as perceptual achievements that are driven by our ability to anticipate future events. In two experiments, we tested whether the prior presentation of motion language influences visual spatial memory in ways that afford greater perceptual prediction. Experiment 1 showed that motion language influenced judgments for the spatial memory of an object beyond the known effects of implied motion present in the image itself. Experiment 2 replicated this finding. Our findings support a theory of perception as prediction.

The Paradox of the Universal Triangle: Concepts, Language, and Prototypes

For over 300 years, the humble triangle has served as the paradigmatic example of the problem of abstraction. How can we have the idea of a general triangle even though every experience with triangles is with specific ones? Classical cognitive science seemed to provide an answer in symbolic representation. With its easily enumerated necessary and sufficient conditions, the triangle would appear to be an ideal candidate for being represented in a symbolic form. I show that it is not. Across a variety of tasks—drawing, speeded recognition, unspeeded visual judgments, and inference—representations of triangles appear to be graded and context dependent. I show that using the category name “triangle” activates a more prototypical representation than using an arguably coextensive cue, “three-sided polygon”. For example, when asked to draw “triangles” people draw more typical triangles than when asked to draw “three-sided polygons”. Altogether, the results support the view that (even formal) concepts have a graded and flexible structure, which takes on a more prototypical and stable form when activated by category labels.

Not even wrong: The “it's just X” fallacy

I applaud Firestone & Scholl (F&S) in calling for more rigor. But, although F&S are correct that some published work on top-down effects suffers from confounds, their sweeping claim that there are no top-down effects on perception is premised on incorrect assumptions. F&S's thesis is wrong. Perception is richly and interestingly influenced by cognition.

Predictions penetrate perception: Converging insights from brain, behaviour and disorder

It is argued that during ongoing visual perception, the brain is generating top-down predictions to facilitate, guide and constrain the processing of incoming sensory input. Here we demonstrate that these predictions are drawn from a diverse range of cognitive processes, in order to generate the richest and most informative prediction signals. This is consistent with a central role for cognitive penetrability in visual perception. We review behavioural and mechanistic evidence that indicate a wide spectrum of domains-including object recognition, contextual associations, cognitive biases and affective state-that can directly influence visual perception. We combine these insights from the healthy brain with novel observations from neuropsychiatric disorders involving visual hallucinations, which highlight the consequences of imbalance between top-down signals and incoming sensory information. Together, these lines of evidence converge to indicate that predictive penetration, be it cognitive, social or emotional, should be considered a fundamental framework that supports visual perception.

Building machines that learn and think like people

Recent progress in artificial intelligence has renewed interest in building systems that learn and think like people. Many advances have come from using deep neural networks trained end-to-end in tasks such as object recognition, video games, and board games, achieving performance that equals or even beats that of humans in some respects. Despite their biological inspiration and performance achievements, these systems differ from human intelligence in crucial ways. We review progress in cognitive science suggesting that truly human-like learning and thinking machines will have to reach beyond current engineering trends in both what they learn and how they learn it. Specifically, we argue that these machines should (1) build causal models of the world that support explanation and understanding, rather than merely solving pattern recognition problems; (2) ground learning in intuitive theories of physics and psychology to support and enrich the knowledge that is learned; and (3) harness compositionality and learning-to-learn to rapidly acquire and generalize knowledge to new tasks and situations. We suggest concrete challenges and promising routes toward these goals that can combine the strengths of recent neural network advances with more structured cognitive models.

Context-dependent plasticity in the subcortical encoding of linguistic pitch patterns

We examined the mechanics of online experience-dependent auditory plasticity by assessing the influence of prior context on the frequency-following responses (FFRs), which reflect phase-locked responses from neural ensembles within the subcortical auditory system. FFRs were elicited to a Cantonese falling lexical pitch pattern from 24 native speakers of Cantonese in a variable context, wherein the falling pitch pattern randomly occurred in the context of two other linguistic pitch patterns; in a patterned context, wherein, the falling pitch pattern was presented in a predictable sequence along with two other pitch patterns, and in a repetitive context, wherein the falling pitch pattern was presented with 100% probability. We found that neural tracking of the stimulus pitch contour was most faithful and accurate when listening context was patterned and least faithful when the listening context was variable. The patterned context elicited more robust pitch tracking relative to the repetitive context, suggesting that context-dependent plasticity is most robust when the context is predictable but not repetitive. Our study demonstrates a robust influence of prior listening context that works to enhance online neural encoding of linguistic pitch patterns. We interpret these results as indicative of an interplay between contextual processes that are responsive to predictability as well as novelty in the presentation context.

NEW & NOTEWORTHY

Human auditory perception in dynamic listening environments requires fine-tuning of sensory signal based on behaviorally relevant regularities in listening context, i.e., online experience-dependent plasticity. Our finding suggests what partly underlie online experience-dependent plasticity are interplaying contextual processes in the subcortical auditory system that are responsive to predictability as well as novelty in listening context. These findings add to the literature that looks to establish the neurophysiological bases of auditory system plasticity, a central issue in auditory neuroscience.

Grounding Abstractness: Abstract Concepts and the Activation of the Mouth

One key issue for theories of cognition is how abstract concepts, such as freedom, are represented. According to the WAT (Words As social Tools) proposal, abstract concepts activate both sensorimotor and linguistic/social information, and their acquisition modality involves the linguistic experience more than the acquisition of concrete concepts. We report an experiment in which participants were presented with abstract and concrete definitions followed by concrete and abstract target-words. When the definition and the word matched, participants were required to press a key, either with the hand or with the mouth. Response times and accuracy were recorded. As predicted, we found that abstract definitions and abstract words yielded slower responses and more errors compared to concrete definitions and concrete words. More crucially, there was an interaction between the target-words and the effector used to respond (hand, mouth). While responses with the mouth were overall slower, the advantage of the hand over the mouth responses was more marked with concrete than with abstract concepts. The results are in keeping with grounded and embodied theories of cognition and support the WAT proposal, according to which abstract concepts evoke linguistic-social information, hence activate the mouth. The mechanisms underlying the mouth activation with abstract concepts (re-enactment of acquisition experience, or re-explanation of the word meaning, possibly through inner talk) are discussed. To our knowledge this is the first behavioral study demonstrating with real words that the advantage of the hand over the mouth is more marked with concrete than with abstract concepts, likely because of the activation of linguistic information with abstract concepts.

Rapid Release From Listening Effort Resulting From Semantic Context, and Effects of Spectral Degradation and Cochlear Implants

People with hearing impairment are thought to rely heavily on context to compensate for reduced audibility. Here, we explore the resulting cost of this compensatory behavior, in terms of effort and the efficiency of ongoing predictive language processing. The listening task featured predictable or unpredictable sentences, and participants included people with cochlear implants as well as people with normal hearing who heard full-spectrum/unprocessed or vocoded speech. The crucial metric was the growth of the pupillary response and the reduction of this response for predictable versus unpredictable sentences, which would suggest reduced cognitive load resulting from predictive processing. Semantic context led to rapid reduction of listening effort for people with normal hearing; the reductions were observed well before the offset of the stimuli. Effort reduction was slightly delayed for people with cochlear implants and considerably more delayed for normal-hearing listeners exposed to spectrally degraded noise-vocoded signals; this pattern of results was maintained even when intelligibility was perfect. Results suggest that speed of sentence processing can still be disrupted, and exertion of effort can be elevated, even when intelligibility remains high. We discuss implications for experimental and clinical assessment of speech recognition, in which good performance can arise because of cognitive processes that occur after a stimulus, during a period of silence. Because silent gaps are not common in continuous flowing speech, the cognitive/linguistic restorative processes observed after sentences in such studies might not be available to listeners in everyday conversations, meaning that speech recognition in conventional tests might overestimate sentence-processing capability.

Hallucinations as top-down effects on perception

The problem of whether and how information is integrated across hierarchical brain networks embodies a fundamental tension in contemporary cognitive neuroscience, and by extension, cognitive neuropsychiatry. Indeed, the penetrability of perceptual processes in a 'top-down' manner by higher-level cognition-a natural extension of hierarchical models of perception-may contradict a strictly modular view of mental organization. Furthermore, some in the cognitive science community have challenged cognitive penetration as an unlikely, if not impossible, process. We review the evidence for and against top-down influences in perception, informed by a predictive coding model of perception and drawing heavily upon the literature of computational neuroimaging. We extend these findings to propose a way in which these processes may be altered in mental illness. We propose that hallucinations - perceptions without stimulus - can be understood as top-down effects on perception, mediated by inappropriate perceptual priors.

Why Do We Need Emotion Words in the First Place? Commentary on Lakoff (2015)

George Lakoff (2016) discusses how emotion metaphors reflect the discrete bodily states associated with each emotion. The analysis raises questions about the context for and frequency of use of emotion metaphors and, indeed, emotion labels (e.g., “angry”), per se. An assumption implicit to most theories of emotion is that emotion language is just another channel through which people express ongoing emotion states. Drawing from recent evidence that labeling ongoing emotions reduces their intensity, we propose that a primary function of emotion language is regulatory rather than expressive.

When expectancies collide: Action dynamics reveal the interaction between stimulus plausibility and congruency

The cognitive architecture routinely relies on expectancy mechanisms to process the plausibility of stimuli and establish their sequential congruency. In two computer mouse-tracking experiments, we use a cross-modal verification task to uncover the interaction between plausibility and congruency by examining their temporal signatures of activation competition as expressed in a computer- mouse movement decision response. In this task, participants verified the content congruency of sentence and scene pairs that varied in plausibility. The order of presentation (sentence-scene, scene-sentence) was varied between participants to uncover any differential processing. Our results show that implausible but congruent stimuli triggered less accurate and slower responses than implausible and incongruent stimuli, and were associated with more complex angular mouse trajectories independent of the order of presentation. This study provides novel evidence of a disassociation between the temporal signatures of plausibility and congruency detection on decision responses.

Encouraging prediction during production facilitates subsequent comprehension: Evidence from interleaved object naming in sentence context and sentence reading

Many studies have shown that a supportive context facilitates language comprehension. A currently influential view is that language production may support prediction in language comprehension. Experimental evidence for this, however, is relatively sparse. Here we explored whether encouraging prediction in a language production task encourages the use of predictive contexts in an interleaved comprehension task. In Experiment 1a, participants listened to the first part of a sentence and provided the final word by naming aloud a picture. The picture name was predictable or not predictable from the sentence context. Pictures were named faster when they could be predicted than when this was not the case. In Experiment 1b the same sentences, augmented by a final spill-over region, were presented in a self-paced reading task. No difference in reading times for predictive versus non-predictive sentences was found. In Experiment 2, reading and naming trials were intermixed. In the naming task, the advantage for predictable picture names was replicated. More importantly, now reading times for the spill-over region were considerable faster for predictive than for non-predictive sentences. We conjecture that these findings fit best with the notion that prediction in the service of language production encourages the use of predictive contexts in comprehension. Further research is required to identify the exact mechanisms by which production exerts its influence on comprehension.