Comprehending how visual context influences incremental sentence processing: Insights from ERPs and picture-sentence verification

Using Event-Related Potentials to Evidence the Visual and Semantic Impact: A Pilot Study with N400 Effect and Food Packaging

Packaging design is pivotal in motivating consumer decisions, as a key communication tool from creation to purchase. Currently, the interpretation and evaluation of packaging's impact are shifting toward non-traditional methods. This pilot study evaluated the packaging perception of York Ham and Turkey Breast products. The event-related potential (ERP) technique, the methodology priming words (positive and negative), and target images (original and modified packaging) were applied. A total of 23 participants were sampled using a 32-channels scalp elastic electrode cap and viewed 200 trials of word-image matching. Participants responded whether the images and adjectives matched or not, using the two groups of images. The results demonstrate an N400 effect in the parietal area. This region was observed to show evidence of cognitive processing related to congruency or incongruency, by contrasting the priming and target of this study. The evaluation positioned the York Ham packaging as the best rated. The findings show a relevant contribution to ERPs and research related to the food packaging perception.

Combining EEG and 3D-eye-tracking to study the prediction of upcoming speech in naturalistic virtual environments: A proof of principle

EEG and eye-tracking provide complementary information when investigating language comprehension. Evidence that speech processing may be facilitated by speech prediction comes from the observation that a listener's eye gaze moves towards a referent before it is mentioned if the remainder of the spoken sentence is predictable. However, changes to the trajectory of anticipatory fixations could result from a change in prediction or an attention shift. Conversely, N400 amplitudes and concurrent spectral power provide information about the ease of word processing the moment the word is perceived. In a proof-of-principle investigation, we combined EEG and eye-tracking to study linguistic prediction in naturalistic, virtual environments. We observed increased processing, reflected in theta band power, either during verb processing - when the verb was predictive of the noun - or during noun processing - when the verb was not predictive of the noun. Alpha power was higher in response to the predictive verb and unpredictable nouns. We replicated typical effects of noun congruence but not predictability on the N400 in response to the noun. Thus, the rich visual context that accompanied speech in virtual reality influenced language processing compared to previous reports, where the visual context may have facilitated processing of unpredictable nouns. Finally, anticipatory fixations were predictive of spectral power during noun processing and the length of time fixating the target could be predicted by spectral power at verb onset, conditional on the object having been fixated. Overall, we show that combining EEG and eye-tracking provides a promising new method to answer novel research questions about the prediction of upcoming linguistic input, for example, regarding the role of extralinguistic cues in prediction during language comprehension.

Causal inference: relating language to event representations and events in the world

Events are not isolated but rather linked to one another in various dimensions. In language processing, various sources of information-including real-world knowledge, (representations of) current linguistic input and non-linguistic visual context-help establish causal connections between events. In this review, we discuss causal inference in relation to events and event knowledge as one aspect of world knowledge, and their representations in language comprehension. To evaluate the mechanism and time course of causal inference, we gather insights from studies on (1) implicit causality/consequentiality as a specific form of causal inference regarding the protagonists of cause/consequence events, and (2) the processing of causal relations. We highlight the importance of methodology in measuring causal inference, compare the results from different research methods, and emphasize the contribution of the visual-world paradigm to achieve a better understanding of causal inference. We recommend that further investigations of causal inference consider temporally sensitive measures and more detailed contexts.

Electrophysiological evidence for a subject-first strategy in visually situated auditory sentence processing in Korean

This study investigated a subject-first strategy in prediction mechanism in visually situated sentence processing in Korean, using event-related potentials (ERPs). According to the subject-first strategy, parsers tend to generate sentences conforming to canonical sentence word order (i.e., SOV in Korean), subject-first sentence, mapping conceptually more prominent referent such as agent of the event on the subject position of the sentence. Therefore, in the predictive mechanism of language comprehension, the subject is pre-activated and anticipated for the first NP of the sentence at the initial phase of bottom-up language processing. This study tested this subject-first strategy in Korean by examining brain responses to object-initial sentences (OV) compared with subject-initial sentences (SV) under the context of clear thematic role relations set by a visual image. The results of an ERP experiment with 30 native Korean speakers identified neural effects for object-initial sentences compared with subject-initial sentences at the NP and Verb, reflecting a conflict between the pre-activated representation in the parser's mind and the encountered bottom-up input. An N400 effect was elicited at the NP, as early as at the noun, not at the following object case marker. Late frontal positivity (LFP) was also found in the sentence-final verb, proving the processing difficulty of non-canonical object-initial sentences compared with canonical subject-initial sentences. These results indicate that Korean native speakers build linguistic representation conforming to a canonical sentence in SOV language in the predictive mechanism supporting subject-first strategy but revise the predicted event structure rapidly upon newly encountering input.

Referential vs. Non-referential World-Language Relations: How Do They Modulate Language Comprehension in 4 to 5-Year-Olds, Younger, and Older Adults?

Age has been shown to influence language comprehension, with delays, for instance, in older adults' expectations about upcoming information. We examined to what extent expectations about upcoming event information (who-does-what-to-whom) change across the lifespan (in 4- to 5-year-old children, younger, and older adults) and as a function of different world-language relations. In a visual-world paradigm, participants in all three age groups inspected a speaker whose facial expression was either smiling or sad. Next they inspected two clipart agents (e.g., a smiling cat and a grumpy rat) depicted as acting upon a patient (e.g., a ladybug tickled by the cat and arrested by the rat). Control scenes featured the same three characters without the action depictions. While inspecting the depictions, comprehenders listened to a German sentence [e.g., Den Marienkäfer kitzelt vergnügt der Kater; literally: “The ladybug (object/patient) tickles happily the cat (subject/agent)”]. Referential verb-action relations (i.e., when the actions were present) could, in principle, cue the cat-agent and so could non-referential relations via links from the speaker's smile to “happily” and the cat's smile. We examined variation in participants' visual anticipation of the agent (the cat) before it was mentioned depending on (a) participant age and (b) whether the referentially mediated action depiction or the non-referentially associated speaker smile cued the agent. The action depictions rapidly boosted participants' visual anticipation of the agent, facilitating thematic role assignment in all age groups. By contrast, effects of the non-referentially cued speaker smile emerged in the younger adults only. We outline implications of these findings for processing accounts of the temporally coordinated interplay between listeners' age-dependent language comprehension, their interrogation of the visual context, and visual context influences.

Using eye-tracking and EEG to study the mental processing demands during learning of text-picture combinations

Using and combining eye-tracking and EEG frequency band power as process measures, in the current study we were interested in the mental processing demands during learning of text-picture combinations that either enabled or prohibited text-picture integration (TPI). In the mismatch condition, the textual and pictorial information being dissimilar, TPI was not possible, whereas in the match and the partial-match condition, the textual and pictorial information being identical respective complementary, TPI was possible. We expected mental processing demands to be higher in the mismatch condition, when pictorial and textual information had to be processed and memorized as separate representations, compared to the match and partial-match conditions when TPI was possible. As expected, on virtually all process measures we observed increased mental processing demands when two mental representations had to be processed and memorized compared to the two conditions where TPI was possible. The EEG alpha and theta frequency band power data corroborated and extended the eye-tracking measures of mental processing demands. In addition, we performed a fixation-related EEG frequency band power analysis that also corroborated the results of the classic stimulus-locked EEG frequency band power analysis, exemplifying the use of this former methodology in the context of complex multimedia task materials.

Using the Visual World Paradigm to Study Sentence Comprehension in Mandarin-Speaking Children with Autism

Sentence comprehension relies on the ability to rapidly integrate different types of linguistic and non-linguistic information. However, there is currently a paucity of research exploring how preschool children with autism understand sentences using different types of cues. The mechanisms underlying sentence comprehension remains largely unclear. The present study presents a protocol to examine the sentence comprehension abilities of preschool children with autism. More specifically, a visual world paradigm of eye-tracking is used to explore the moment-to-moment sentence comprehension in the children. The paradigm has multiple advantages. First, it is sensitive to the time course of sentence comprehension and thus can provide rich information about how sentence comprehension unfolds over time. Second, it requires minimal task and communication demands, so it is ideal for testing children with autism. To further minimize the computational burden of children, the present study measures eye movements that arise as automatic responses to linguistic input rather than measuring eye movements that accompany conscious responses to spoken instructions.

Harry Potter and the Chamber of What?: The impact of what individuals know on word processing during reading

During reading, effects of contextual support indexed by N400-a brain potential sensitive to semantic activation/retrieval-amplitude are presumably mediated by comprehenders' world knowledge. Moreover, variability in knowledge may influence the contents, timing, and mechanisms of what is brought to mind during real-time sentence processing. Since it is infeasible to assess the entirety of each individual's knowledge, we investigated a limited domain-the narrative world of Harry Potter (HP). We recorded event-related brain potentials while participants read sentences ending in words more/less contextually supported. For sentences about HP, but not about general topics, contextual N400 effects were graded according to individual participants' HP knowledge. Our results not only confirm that context affects semantic processing by ~250 ms or earlier, on average, but empirically demonstrate what has until now been assumed-that N400 context effects are a function of each individual's knowledge, which here is highly correlated with their reading experience.

Semantic Interference and Facilitation: Understanding the Integration of Spatial Distance and Conceptual Similarity During Sentence Reading

Existing evidence has shown a processing advantage (or facilitation) when representations derived from a non-linguistic context (spatial proximity depicted by gambling cards moving together) match the semantic content of an ensuing sentence. A match, inspired by conceptual metaphors such as 'similarity is closeness' would, for instance, involve cards moving closer together and the sentence relates similarity between abstract concepts such as war and battle. However, other studies have reported a disadvantage (or interference) for congruence between the semantic content of a sentence and representations of spatial distance derived from this sort of non-linguistic context. In the present article, we investigate the cognitive mechanisms underlying the interaction between the representations of spatial distance and sentence processing. In two eye-tracking experiments, we tested the predictions of a mechanism that considers the competition, activation, and decay of visually and linguistically derived representations as key aspects in determining the qualitative pattern and time course of that interaction. Critical trials presented two playing cards, each showing a written abstract noun; the cards turned around, obscuring the nouns, and moved either farther apart or closer together. Participants then read a sentence expressing either semantic similarity or difference between these two nouns. When instructed to attend to the nouns on the cards (Experiment 1), participants' total reading times revealed interference between spatial distance (e.g., closeness) and semantic relations (similarity) as soon as the sentence explicitly conveyed similarity. But when instructed to attend to the cards (Experiment 2), cards approaching (vs. moving apart) elicited first interference (when similarity was implicit) and then facilitation (when similarity was made explicit) during sentence reading. We discuss these findings in the context of a competition mechanism of interference and facilitation effects.

Validating presupposed versus focused text information

There is extensive evidence that readers continually validate discourse accuracy and congruence, but that they may also overlook conspicuous text contradictions. Validation may be thwarted when the inaccurate ideas are embedded sentence presuppositions. In four experiments, we examined readers' validation of presupposed ("given") versus new text information. Throughout, a critical concept, such as a truck versus a bus, was introduced early in a narrative. Later, a character stated or thought something about the truck, which therefore matched or mismatched its antecedent. Furthermore, truck was presented as either given or new information. Mismatch target reading times uniformly exceeded the matching ones by similar magnitudes for given and new concepts. We obtained this outcome using different grammatical constructions and with different antecedent-target distances. In Experiment 4, we examined only given critical ideas, but varied both their matching and the main verb's factivity (e.g., factive know vs. nonfactive think). The Match × Factivity interaction closely resembled that previously observed for new target information (Singer, 2006). Thus, readers can successfully validate given target information. Although contemporary theories tend to emphasize either deficient or successful validation, both types of theory can accommodate the discourse and reader variables that may regulate validation.

Gestalt Reasoning with Conjunctions and Disjunctions

Reasoning, solving mathematical equations, or planning written and spoken sentences all must factor in stimuli perceptual properties. Indeed, thinking processes are inspired by and subsequently fitted to concrete objects and situations. It is therefore reasonable to expect that the mental representations evoked when people solve these seemingly abstract tasks should interact with the properties of the manipulated stimuli. Here, we investigated the mental representations evoked by conjunction and disjunction expressions in language-picture matching tasks. We hypothesised that, if these representations have been derived using key Gestalt principles, reasoners should use perceptual compatibility to gauge the goodness of fit between conjunction/disjunction descriptions (e.g., the purple and/ or the green) and corresponding binary visual displays. Indeed, the results of three experimental studies demonstrate that reasoners associate conjunction descriptions with perceptually-dependent stimuli and disjunction descriptions with perceptually-independent stimuli, where visual dependency status follows the key Gestalt principles of common fate, proximity, and similarity.

Separate streams or probabilistic inference? What the N400 can tell us about the comprehension of events

Since the early 2000s, several ERP studies have challenged the assumption that we always use syntactic contextual information to influence semantic processing of incoming words, as reflected by the N400 component. One approach for explaining these findings is to posit distinct semantic and syntactic processing mechanisms, each with distinct time courses. While this approach can explain specific datasets, it cannot account for the wider body of findings. I propose an alternative explanation: a dynamic generative framework in which our goal is to infer the underlying event that best explains the set of inputs encountered at any given time. Within this framework, combinations of semantic and syntactic cues with varying reliabilities are used as evidence to weight probabilistic hypotheses about this event. I further argue that the computational principles of this framework can be extended to understand how we infer situation models during discourse comprehension, and intended messages during spoken communication.

Visual attention during spatial language comprehension

Spatial terms such as “above”, “in front of”, and “on the left of” are all essential for describing the location of one object relative to another object in everyday communication. Apprehending such spatial relations involves relating linguistic to object representations by means of attention. This requires at least one attentional shift, and models such as the Attentional Vector Sum (AVS) predict the direction of that attention shift, from the sausage to the box for spatial utterances such as “The box is above the sausage”. To the extent that this prediction generalizes to overt gaze shifts, a listener’s visual attention should shift from the sausage to the box. However, listeners tend to rapidly look at referents in their order of mention and even anticipate them based on linguistic cues, a behavior that predicts a converse attentional shift from the box to the sausage. Four eye-tracking experiments assessed the role of overt attention in spatial language comprehension by examining to which extent visual attention is guided by words in the utterance and to which extent it also shifts “against the grain” of the unfolding sentence. The outcome suggests that comprehenders’ visual attention is predominantly guided by their interpretation of the spatial description. Visual shifts against the grain occurred only when comprehenders had some extra time, and their absence did not affect comprehension accuracy. However, the timing of this reverse gaze shift on a trial correlated with that trial’s verification time. Thus, while the timing of these gaze shifts is subtly related to the verification time, their presence is not necessary for successful verification of spatial relations.

Spatial distance effects on incremental semantic interpretation of abstract sentences: evidence from eye tracking

A large body of evidence has shown that visual context information can rapidly modulate language comprehension for concrete sentences and when it is mediated by a referential or a lexical-semantic link. What has not yet been examined is whether visual context can also modulate comprehension of abstract sentences incrementally when it is neither referenced by, nor lexically associated with, the sentence. Three eye-tracking reading experiments examined the effects of spatial distance between words (Experiment 1) and objects (Experiment 2 and 3) on participants' reading times for sentences that convey similarity or difference between two abstract nouns (e.g., 'Peace and war are certainly different...'). Before reading the sentence, participants inspected a visual context with two playing cards that moved either far apart or close together. In Experiment 1, the cards turned and showed the first two nouns of the sentence (e.g., 'peace', 'war'). In Experiments 2 and 3, they turned but remained blank. Participants' reading times at the adjective (Experiment 1: first-pass reading time; Experiment 2: total times) and at the second noun phrase (Experiment 3: first-pass times) were faster for sentences that expressed similarity when the preceding words/objects were close together (vs. far apart) and for sentences that expressed dissimilarity when the preceding words/objects were far apart (vs. close together). Thus, spatial distance between words or entirely unrelated objects can rapidly and incrementally modulate the semantic interpretation of abstract sentences.

Event-related potentials to intact and disrupted actions in children and adults

The current research used event-related potentials (ERPs) to investigate neurophysiological responses to intact and disrupted actions embedded within an event in children and adults. Responses were recorded as children (24-month-olds) and adults observed a relatively novel event composed of three actions. In one condition pauses were inserted at intact boundaries (i.e., at the endpoint of each action), whereas in the other condition they were inserted at breakpoints that disrupted the action (i.e., in the middle of each action). Evoked responses revealed differences across conditions in both groups; disrupted actions elicited a prolonged negative slow wave from 100 to 700 ms in children, whereas adults demonstrated two distinct negative peaks between 50-150 and 250-350 ms. These findings contribute the first electrophysiological evidence that children readily detect disruptions to ongoing events by the end of the second year, even with limited exposure to the event itself. Furthermore, they suggest that adults rely on two distinct mechanisms when processing novel events. Results are discussed in relation to the role of perceptual and conceptual levels of analysis in the development of action processing.

Validation in Reading Comprehension

Language comprehension involves analysis at the level of the word, sentence, and message and the integration of message meaning with the prior discourse and world knowledge. Contemporary research converges on another facet of comprehension: the validation of message consistency. Existing evidence already favors several principles in validation of reading and listening. Validation is initiated immediately and is routine rather than requiring intentional strategies. Successful validation is a precondition to updating the situational representation of the message. Validation applies to discourse inferences as well as explicit assertions. Finally, the memory-retrieval processes that enable validation closely resemble those of intentional discourse memory. Competing observations of people’s validation failures are proposed to systematically stem from features of the message, understander, and comprehension task. Therefore, theoretical analysis that accommodates both successful and deficient language validation ought to be attainable.

Effects of speaker emotional facial expression and listener age on incremental sentence processing

We report two visual-world eye-tracking experiments that investigated how and with which time course emotional information from a speaker's face affects younger (N = 32, Mean age  = 23) and older (N = 32, Mean age  = 64) listeners’ visual attention and language comprehension as they processed emotional sentences in a visual context. The age manipulation tested predictions by socio-emotional selectivity theory of a positivity effect in older adults. After viewing the emotional face of a speaker (happy or sad) on a computer display, participants were presented simultaneously with two pictures depicting opposite-valence events (positive and negative; IAPS database) while they listened to a sentence referring to one of the events. Participants' eye fixations on the pictures while processing the sentence were increased when the speaker's face was (vs. wasn't) emotionally congruent with the sentence. The enhancement occurred from the early stages of referential disambiguation and was modulated by age. For the older adults it was more pronounced with positive faces, and for the younger ones with negative faces. These findings demonstrate for the first time that emotional facial expressions, similarly to previously-studied speaker cues such as eye gaze and gestures, are rapidly integrated into sentence processing. They also provide new evidence for positivity effects in older adults during situated sentence processing.

Throwing the banana away and keeping the peel: neuroelectric responses to unexpected but physically feasible action endings

Integrative semantic processing and mirror-neuron system activation in response to incongruence detection during action observation have so far been investigated only through paradigms that involve unfeasible action endings, introduction of extraneous objects or synthetic pictures. The ecological validity of the reported effects, particularly modulation of the N400 event-related potential (ERP), therefore remains questionable. To address this shortcoming, we investigated the ERPs evoked by physically feasible and naturalistic but unexpected action endings. We hypothesized that in comparison to existing work our more realistic scenarios would elicit slower processing, due to engagement of deeper analysis as opposed to fast detection based on automatic processes. ERPs were recorded in 14 healthy participants in response to 60 sequences, each containing three pictures, the last of which could be an expected or unexpected action ending. The N400 was larger for the first image compared to the others, and the LPP was enhanced for unexpected endings. Source localization implicated anterior fronto-temporal and temporo-parietal junction regions in generation of these potentials. Our findings challenge the view that the N400 universally indices verbal and action-perception incongruence detection, and point to situation-contingent effects which, in the case of subtle violations, are reflected in later activity components, speculatively reflecting recourse to the "mentalizing" system.

Perception and bias in the processing of compound versus phrasal stress: evidence from event-related brain potentials

Previous research using picture/word matching tasks has demonstrated a tendency to incorrectly interpret phrasally stressed strings as compounds. Using event-related potentials, we sought to determine whether this pattern stems from poor perceptual sensitivity to the compound/phrasal stress distinction, or from a post-perceptual bias in behavioral response selection. A secondary aim was to gain insight into the role played by contrastive stress patterns in online sentence comprehension. The behavioral results replicated previous findings of a preference for compounds, but the electrophysiological data suggested a robust sensitivity to both stress patterns. When incongruent with the context, both compound and phrasal stress elicited a sustained left-lateralized negativity. Moreover, incongruent compound stress elicited a centro-parietal negativity (N400), while incongruent phrasal stress elicited a late posterior positivity (P600). We conclude that the previous findings of a preference for compounds are due to response selection bias, and not a lack of perceptual sensitivity. The present results complement previous evidence for the immediate use of meter in semantic processing, as well as evidence for late interactions between prosodic and syntactic information.

Syntactic priming and the lexical boost effect during sentence production and sentence comprehension: an fMRI study

Behavioral syntactic priming effects during sentence comprehension are typically observed only if both the syntactic structure and lexical head are repeated. In contrast, during production syntactic priming occurs with structure repetition alone, but the effect is boosted by repetition of the lexical head. We used fMRI to investigate the neuronal correlates of syntactic priming and lexical boost effects during sentence production and comprehension. The critical measure was the magnitude of fMRI adaptation to repetition of sentences in active or passive voice, with or without verb repetition. In conditions with repeated verbs, we observed adaptation to structure repetition in the left IFG and MTG, for active and passive voice. However, in the absence of repeated verbs, adaptation occurred only for passive sentences. None of the fMRI adaptation effects yielded differential effects for production versus comprehension, suggesting that sentence comprehension and production are subserved by the same neuronal infrastructure for syntactic processing.

Distinct neural correlates for pragmatic and semantic meaning processing: an event-related potential investigation of scalar implicature processing using picture-sentence verification

The present study examines the brain-level representation and composition of meaning in scalar quantifiers (e.g., some), which have both a semantic meaning (at least one) and a pragmatic meaning (not all). We adopted a picture-sentence verification design to examine event-related potential (ERP) effects of reading infelicitous quantifiers for which the semantic meaning was correct with respect to the context but the pragmatic meaning was not, compared to quantifiers for which the semantic meaning was inconsistent with the context and no additional pragmatic meaning is available. In the first experiment, only pragmatically inconsistent quantifiers, not semantically inconsistent quantifiers, elicited a sustained posterior negative component. This late negativity contrasts with the N400 effect typically elicited by nouns that are incongruent with their context, suggesting that the recognition of scalar implicature errors elicits a qualitatively different ERP signature than the recognition of lexico-semantic errors. We hypothesize that the sustained negativity reflects cancellation of the pragmatic inference and retrieval of the semantic meaning. In our second experiment, we found that the process of re-interpreting the quantifier was independent from lexico-semantic processing: the N400 elicited by lexico-semantic violations was not modulated by the presence of a pragmatic inconsistency. These findings suggest that inferential pragmatic aspects of meaning are processed using different mechanisms than lexical or combinatorial semantic aspects of meaning, that inferential pragmatic meaning can be realized rapidly, and that the computation of meaning involves continuous negotiation between different aspects of meaning.

Incidental picture exposure affects later reading: evidence from the N400

Language comprehenders form a mental representation of the implied shape of objects mentioned in the text. In the present study, the influence of prior visual experience on subsequent reading was assessed. In two separate phases, participants saw a picture of an object and read a text about the object, suggesting the same or a different shape. When the shapes in the two phases mismatched, ERPs during reading showed a larger N400 amplitude than when the shapes matched, suggesting that a picture presented incidentally 15 min earlier affected reading. These results further strengthen the case for the interaction of language and visual experience during language comprehension.

Shared syntax in language production and language comprehension--an FMRI study

During speaking and listening syntactic processing is a crucial step. It involves specifying syntactic relations between words in a sentence. If the production and comprehension modality share the neuronal substrate for syntactic processing then processing syntax in one modality should lead to adaptation effects in the other modality. In the present functional magnetic resonance imaging experiment, participants either overtly produced or heard descriptions of pictures. We looked for brain regions showing adaptation effects to the repetition of syntactic structures. In order to ensure that not just the same brain regions but also the same neuronal populations within these regions are involved in syntactic processing in speaking and listening, we compared syntactic adaptation effects within processing modalities (syntactic production-to-production and comprehension-to-comprehension priming) with syntactic adaptation effects between processing modalities (syntactic comprehension-to-production and production-to-comprehension priming). We found syntactic adaptation effects in left inferior frontal gyrus (Brodmann's area [BA] 45), left middle temporal gyrus (BA 21), and bilateral supplementary motor area (BA 6) which were equally strong within and between processing modalities. Thus, syntactic repetition facilitates syntactic processing in the brain within and across processing modalities to the same extent. We conclude that that the same neurobiological system seems to subserve syntactic processing in speaking and listening.

Shared language: overlap and segregation of the neuronal infrastructure for speaking and listening revealed by functional MRI

Whether the brain's speech-production system is also involved in speech comprehension is a topic of much debate. Research has focused on whether motor areas are involved in listening, but overlap between speaking and listening might occur not only at primary sensory and motor levels, but also at linguistic levels (where semantic, lexical, and syntactic processes occur). Using functional MRI adaptation during speech comprehension and production, we found that the brain areas involved in semantic, lexical, and syntactic processing are mostly the same for speaking and for listening. Effects of primary processing load (indicative of sensory and motor processes) overlapped in auditory cortex and left inferior frontal cortex, but not in motor cortex, where processing load affected activity only in speaking. These results indicate that the linguistic parts of the language system are used for both speaking and listening, but that the motor system does not seem to provide a crucial contribution to listening.