Understanding counterfactuals in discourse modulates ERP and oscillatory gamma rhythms in the EEG

Similarities and differences in understanding negative and affirmative counterfactuals and causal assertions: Evidence from eye-tracking

Two eye-tracking experiments compared affirmative and negative counterfactuals, "if she had (not) arrived early, she would (not) have bought roses" and affirmative and negative causal assertions, "Because she arrived (did not arrive) early, she bought (did not buy) roses." When participants heard a counterfactual, they looked on screen at words corresponding to its conjecture ("roses"), and its presupposed facts ("no roses"), whereas for a causal assertion, they looked only at words corresponding to the facts. For counterfactuals, they looked at the conjecture first, and later the presupposed facts, and at the latter more than the former. The effect was more pronounced for negative counterfactuals than affirmative ones because the negative counterfactual's presupposed facts identify a specific item ("she bought roses"), whereas the affirmative counterfactual's presupposed facts do not ("she did not buy roses"). Hence, when participants were given a binary context, "she did not know whether to buy roses or carnations," they looked primarily at the presupposed facts for both sorts of counterfactuals. We discuss the implications for theories of negation, the dual meaning of counterfactuals, and their relation to causal assertions.

Autistic Adults are Not Impaired at Maintaining or Switching Between Counterfactual and Factual Worlds: An ERP Study

We report an event-related brain potential (ERP) experiment that tests whether autistic adults are able to maintain and switch between counterfactual and factual worlds. Participants (N = 48) read scenarios that set up a factual or counterfactual scenario, then either maintained the counterfactual world or switched back to the factual world. When the context maintained the world, participants showed appropriate detection of the inconsistent critical word. In contrast, when participants had to switch from a counterfactual to factual world, they initially experienced interference from the counterfactual context, then favoured the factual interpretation of events. None of these effects were modulated by group, despite group-level impairments in Theory of Mind and cognitive flexibility among the autistic adults. These results demonstrate that autistic adults can appropriately use complex contextual cues to maintain and/or update mental representations of counterfactual and factual events.

THE POWER OF NEURAL OSCILLATIONS TO INFORM SENTENCE COMPREHENSION: A LINGUISTIC PERSPECTIVE

The field of psycholinguistics is currently experiencing an explosion of interest in the analysis of neural oscillations - rhythmic brain activity synchronized at different temporal and spatial levels. Given that language comprehension relies on a myriad of processes, which are carried out in parallel in distributed brain networks, there is hope that this methodology might bring the field closer to understanding some of the more basic (spatially and temporally distributed, yet at the same time often overlapping) neural computations that support language function. In this review we discuss existing proposals linking oscillatory dynamics in different frequency bands to basic neural computations, and review relevant theories suggesting associations between band-specific oscillations and higher-level cognitive processes. More or less consistent patterns of oscillatory activity related to certain types of linguistic processing can already be derived from the evidence that has accumulated over the past few decades. The centerpiece of the current review is a synthesis of such patterns grouped by linguistic phenomenon. We restrict our review to evidence linking measures of oscillatory power to the comprehension of sentences, as well as linguistically (and/or pragmatically) more complex structures. For each grouping, we provide a brief summary and a table of associated oscillatory signatures that a psycholinguist might expect to find when employing a particular linguistic task. Summarizing across different paradigms, we conclude that a handful of basic neural oscillatory mechanisms are likely recruited in different ways and at different times for carrying out a variety of linguistic computations.

The Comprehension of Counterfactual Conditionals: Evidence From Eye-Tracking in the Visual World Paradigm

Three experiments tracked participants' eye-movements to examine the time course of comprehension of the dual meaning of counterfactuals, such as "if there had been oranges then there would have been pears." Participants listened to conditionals while looking at images in the visual world paradigm, including an image of oranges and pears that corresponds to the counterfactual's conjecture, and one of no oranges and no pears that corresponds to its presumed facts, to establish at what point in time they consider each one. The results revealed striking individual differences: some participants looked at the negative image and the affirmative one, and some only at the affirmative image. The first experiment showed that participants who looked at the negative image increased their fixation on it within half a second. The second experiment showed they do so even without explicit instructions, and the third showed they do so even for printed words.

How meaning unfolds in neural time: Embodied reactivations can precede multimodal semantic effects during language processing

Research on how the brain construes meaning during language use has prompted two conflicting accounts. According to the 'grounded view', word understanding involves quick reactivations of sensorimotor (embodied) experiences evoked by the stimuli, with simultaneous or later engagement of multimodal (conceptual) systems integrating information from various sensory streams. Contrariwise, for the 'symbolic view', this capacity depends crucially on multimodal operations, with embodied systems playing epiphenomenal roles after comprehension. To test these contradictory hypotheses, the present magnetoencephalography study assessed implicit semantic access to grammatically constrained action and non-action verbs (n = 100 per category) while measuring spatiotemporally precise signals from the primary motor cortex (M1, a core region subserving bodily movements) and the anterior temporal lobe (ATL, a putative multimodal semantic hub). Convergent evidence from sensor- and source-level analyses revealed that increased modulations for action verbs occurred earlier in M1 (∼130-190 ms) than in specific ATL hubs (∼250-410 ms). Moreover, machine-learning decoding showed that trial-by-trial classification peaks emerged faster in M1 (∼100-175 ms) than in the ATL (∼345-500 ms), with over 71% accuracy in both cases. Considering their latencies, these results challenge the 'symbolic view' and its implication that sensorimotor mechanisms play only secondary roles in semantic processing. Instead, our findings support the 'grounded view', showing that early semantic effects are critically driven by embodied reactivations and that these cannot be reduced to post-comprehension epiphenomena, even when words are individually classified. Briefly, our study offers non-trivial insights to constrain fine-grained models of language and understand how meaning unfolds in neural time.

Pragmatic skills predict online counterfactual comprehension: Evidence from the N400

Counterfactual thought allows people to consider alternative worlds they know to be false. Communicating these thoughts through language poses a social-communicative challenge because listeners typically expect a speaker to produce true utterances, but counterfactuals per definition convey information that is false. Listeners must therefore incorporate overt linguistic cues (subjunctive mood, such as in If I loved you then) in a rapid way to infer the intended counterfactual meaning. The present EEG study focused on the comprehension of such counterfactual antecedents and investigated if pragmatic ability—the ability to apply knowledge of the social-communicative use of language in daily life—predicts the online generation of counterfactual worlds. This yielded two novel findings: (1) Words that are consistent with factual knowledge incur a semantic processing cost, as reflected in larger N400 amplitude, in counterfactual antecedents compared to hypothetical antecedents (If sweets were/are made of sugar). We take this to suggest that counterfactuality is quickly incorporated during language comprehension and reduces online expectations based on factual knowledge. (2) Individual scores on the Autism Quotient Communication subscale modulated this effect, suggesting that individuals who are better at understanding the communicative intentions of other people are more likely to reduce knowledge-based expectations in counterfactuals. These results are the first demonstration of the real-time pragmatic processes involved in creating possible worlds.

Understanding Counterfactuality: A Review of Experimental Evidence for the Dual Meaning of Counterfactuals

Cognitive and linguistic theories of counterfactual language comprehension assume that counterfactuals convey a dual meaning. Subjunctive-counterfactual conditionals (e.g., 'If Tom had studied hard, he would have passed the test') express a supposition while implying the factual state of affairs (Tom has not studied hard and failed). The question of how counterfactual dual meaning plays out during language processing is currently gaining interest in psycholinguistics. Whereas numerous studies using offline measures of language processing consistently support counterfactual dual meaning, evidence coming from online studies is less conclusive. Here, we review the available studies that examine online counterfactual language comprehension through behavioural measurement (self-paced reading times, eye-tracking) and neuroimaging (electroencephalography, functional magnetic resonance imaging). While we argue that these studies do not offer direct evidence for the online computation of counterfactual dual meaning, they provide valuable information about the way counterfactual meaning unfolds in time and influences successive information processing. Further advances in research on counterfactual comprehension require more specific predictions about how counterfactual dual meaning impacts incremental sentence processing.

Examining the cognitive costs of counterfactual language comprehension: Evidence from ERPs

Recent empirical research suggests that understanding a counterfactual event (e.g. 'If Josie had revised, she would have passed her exams') activates mental representations of both the factual and counterfactual versions of events. However, it remains unclear when readers switch between these models during comprehension, and whether representing multiple 'worlds' is cognitively effortful. This paper reports two ERP studies where participants read contexts that set up a factual or counterfactual scenario, followed by a second sentence describing a consequence of this event. Critically, this sentence included a noun that was either consistent or inconsistent with the preceding context, and either included a modal verb to indicate reference to the counterfactual-world or not (thus referring to the factual-world). Experiment 2 used adapted versions of the materials used in Experiment 1 to examine the degree to which representing multiple versions of a counterfactual situation makes heavy demands on cognitive resources by measuring individuals' working memory capacity. Results showed that when reference to the counterfactual-world was maintained by the ongoing discourse, readers correctly interpreted events according to the counterfactual-world (i.e. showed larger N400 for inconsistent than consistent words). In contrast, when cues referred back to the factual-world, readers showed no difference between consistent and inconsistent critical words, suggesting that they simultaneously compared information against both possible worlds. These results support previous dual-representation accounts for counterfactuals, and provide new evidence that linguistic cues can guide the reader in selecting which world model to evaluate incoming information against. Crucially, we reveal evidence that maintaining and updating a hypothetical model over time relies upon the availability of cognitive resources.

Marking the counterfactual: ERP evidence for pragmatic processing of German subjunctives

Counterfactual conditionals are frequently used in language to express potentially valid reasoning from factually false suppositions. Counterfactuals provide two pieces of information: their literal meaning expresses a suppositional dependency between an antecedent (If the dice had been rigged…) and a consequent (… then the game would have been unfair). Their second, backgrounded meaning refers to the opposite state of affairs and suggests that, in fact, the dice were not rigged and the game was fair. Counterfactual antecedents are particularly intriguing because they set up a counterfactual world which is known to be false, but which is nevertheless kept to when evaluating the conditional's consequent. In the last years several event-related potential (ERP) studies have targeted the processing of counterfactual consequents, yet counterfactual antecedents have remained unstudied. We present an EEG/ERP investigation which employed German conditionals to compare subjunctive mood (which marks counterfactuality) with indicative mood at the critical point of mood disambiguation via auxiliary introduction in the conditional's antecedent. Conditional sentences were presented visually one word at a time. Participants completed an acceptability judgment and probe detection task which was not related to the critical manipulation of linguistic mood. ERPs at the point of mood disambiguation in the antecedent were compared between indicative and subjunctive. Our main finding is a transient negative deflection in frontal regions for subjunctive compared to indicative mood in a time-window of 450–600 ms. We discuss this novel finding in respect to working memory requirements for rule application and increased referential processing demands for the representation of counterfactuals' dual meaning. Our result suggests that the counterfactually implied dual meaning is processed without any delay at the earliest point where counterfactuality is marked by subjunctive mood.

Context-dependent Semantic Processing in the Human Brain: Evidence from Idiom Comprehension

Language comprehension involves activating word meanings and integrating them with the sentence context. This study examined whether these routines are carried out even when they are theoretically unnecessary, namely, in the case of opaque idiomatic expressions, for which the literal word meanings are unrelated to the overall meaning of the expression. Predictable words in sentences were replaced by a semantically related or unrelated word. In literal sentences, this yielded previously established behavioral and electrophysiological signatures of semantic processing: semantic facilitation in lexical decision, a reduced N400 for semantically related relative to unrelated words, and a power increase in the gamma frequency band that was disrupted by semantic violations. However, the same manipulations in idioms yielded none of these effects. Instead, semantic violations elicited a late positivity in idioms. Moreover, gamma band power was lower in correct idioms than in correct literal sentences. It is argued that the brain's semantic expectancy and literal word meaning integration operations can, to some extent, be “switched off” when the context renders them unnecessary. Furthermore, the results lend support to models of idiom comprehension that involve unitary idiom representations.

Processing counterfactual and hypothetical conditionals: an fMRI investigation

Counterfactual thinking is ubiquitous in everyday life and an important aspect of cognition and emotion. Although counterfactual thought has been argued to differ from processing factual or hypothetical information, imaging data which elucidate these differences on a neural level are still scarce. We investigated the neural correlates of processing counterfactual sentences under visual and aural presentation. We compared conditionals in subjunctive mood which explicitly contradicted previously presented facts (i.e. counterfactuals) to conditionals framed in indicative mood which did not contradict factual world knowledge and thus conveyed a hypothetical supposition. Our results show activation in right occipital cortex (cuneus) and right basal ganglia (caudate nucleus) during counterfactual sentence processing. Importantly the occipital activation is not only present under visual presentation but also with purely auditory stimulus presentation, precluding a visual processing artifact. Thus our results can be interpreted as reflecting the fact that counterfactual conditionals pragmatically imply the relevance of keeping in mind both factual and supposed information whereas the hypothetical conditionals imply that real world information is irrelevant for processing the conditional and can be omitted. The need to sustain representations of factual and suppositional events during counterfactual sentence processing requires increased mental imagery and integration efforts. Our findings are compatible with predictions based on mental model theory.