Failure to detect function word repetitions and omissions in reading: Are eye movements to blame?

Lexical-Semantic Content, Not Syntactic Structure, Is the Main Contributor to ANN-Brain Similarity of fMRI Responses in the Language Network

Representations from artificial neural network (ANN) language models have been shown to predict human brain activity in the language network. To understand what aspects of linguistic stimuli contribute to ANN-to-brain similarity, we used an fMRI data set of responses to n = 627 naturalistic English sentences (Pereira et al., 2018) and systematically manipulated the stimuli for which ANN representations were extracted. In particular, we (i) perturbed sentences' word order, (ii) removed different subsets of words, or (iii) replaced sentences with other sentences of varying semantic similarity. We found that the lexical-semantic content of the sentence (largely carried by content words) rather than the sentence's syntactic form (conveyed via word order or function words) is primarily responsible for the ANN-to-brain similarity. In follow-up analyses, we found that perturbation manipulations that adversely affect brain predictivity also lead to more divergent representations in the ANN's embedding space and decrease the ANN's ability to predict upcoming tokens in those stimuli. Further, results are robust as to whether the mapping model is trained on intact or perturbed stimuli and whether the ANN sentence representations are conditioned on the same linguistic context that humans saw. The critical result-that lexical-semantic content is the main contributor to the similarity between ANN representations and neural ones-aligns with the idea that the goal of the human language system is to extract meaning from linguistic strings. Finally, this work highlights the strength of systematic experimental manipulations for evaluating how close we are to accurate and generalizable models of the human language network.

The transposed-word effect provides no unequivocal evidence for parallel processing

Studies using a grammaticality decision task have revealed surprising flexibility in the processing of word order during sentence reading in both alphabetic and non-alphabetic scripts. Participants in these studies typically exhibit a transposed-word effect, in which they make more errors and slower correct responses for stimuli that contain a word transposition and are derived from grammatical as compared to ungrammatical base sentences. Some researchers have used this finding to argue that words are encoded in parallel during reading, such that multiple words can be processed simultaneously and might be recognised out of order. This contrasts with an alternative account of the reading process, which argues that words must be encoded serially, one at a time. We examined, in English, whether the transposed-word effect provides evidence for a parallel-processing account, employing the same grammaticality decision task used in previous research and display procedures that either allowed for parallel word encoding or permitted only the serial encoding of words. Our results replicate and extend recent findings by showing that relative word order can be processed flexibly even when parallel processing is not possible (i.e., within displays requiring serial word encoding). Accordingly, while the present findings provide further evidence for flexibility in the processing of relative word order during reading, they add to converging evidence that the transposed-word effect does not provide unequivocal evidence for a parallel-processing account of reading. We consider how the present findings may be accounted for by both serial and parallel accounts of word recognition in reading.

Lexical semantic content, not syntactic structure, is the main contributor to ANN-brain similarity of fMRI responses in the language network

Representations from artificial neural network (ANN) language models have been shown to predict human brain activity in the language network. To understand what aspects of linguistic stimuli contribute to ANN-to-brain similarity, we used an fMRI dataset of responses to n=627 naturalistic English sentences (Pereira et al., 2018) and systematically manipulated the stimuli for which ANN representations were extracted. In particular, we i) perturbed sentences' word order, ii) removed different subsets of words, or iii) replaced sentences with other sentences of varying semantic similarity. We found that the lexical semantic content of the sentence (largely carried by content words) rather than the sentence's syntactic form (conveyed via word order or function words) is primarily responsible for the ANN-to-brain similarity. In follow-up analyses, we found that perturbation manipulations that adversely affect brain predictivity also lead to more divergent representations in the ANN's embedding space and decrease the ANN's ability to predict upcoming tokens in those stimuli. Further, results are robust to whether the mapping model is trained on intact or perturbed stimuli, and whether the ANN sentence representations are conditioned on the same linguistic context that humans saw. The critical result-that lexical-semantic content is the main contributor to the similarity between ANN representations and neural ones-aligns with the idea that the goal of the human language system is to extract meaning from linguistic strings. Finally, this work highlights the strength of systematic experimental manipulations for evaluating how close we are to accurate and generalizable models of the human language network.

The transposed-word effect does not require parallel word processing: Failure to notice transpositions with serial presentation of words

Readers sometimes fail to notice word transposition errors, reporting a sentence with two transposed words to be grammatical (the transposed-word effect). It has been suggested that this effect implicates parallel word processing during sentence reading. The current study directly assessed the role of parallel word processing in failure to notice word transposition errors, by comparing error detection under normal sentence presentation conditions and when words are presented serially at 250 ms/word. Extending recent results obtained with serial presentation of Chinese sentences (Liu, Li, Cutter, Paterson, & Wang, Cognition 218: 104922, 2022), in Experiment 1 we found a transposed-word effect with serial presentation of English sentences. In Experiment 2, we replicated this finding with task instructions that allowed responding at any time during the presentation of the sentence; this result indicates that the transposed-word effect that appears with serial word presentation is not due to a late process of reconstruction of short-term memory. Thus, parallel word processing is not necessary for a transposed-word effect in English. Like Liu et al. (2022), we did find that the transposed-word effect was statistically larger with parallel presentation than with serial presentation; we consider several explanations as to why this is so.

Using eye tracking to investigate failure to notice word transpositions in reading

Previous research (Mirault, Snell, & Grainger, 2018) has demonstrated that subjects sometimes incorrectly judge an ungrammatical sentence as grammatical when it is created by the transposition of two words in a grammatical sentence (e.g., The white was cat big). Here we present two eye-tracking experiments designed to assess the prevalence of this phenomenon in a more natural reading task, and to explore theoretical explanations. Readers failed to notice transpositions at about the same rate as in Mirault et al. (2018). Failure to notice the transposition was more common when both words were short, and when readers' eyes skipped, rather than directly fixated, one of the two words. The status of the transposed words as open- or closed-class did not have a reliable effect. The transposed words caused disruption in the eye movement record only on trials when participants ultimately judged the sentence to be ungrammatical, not when they judged the sentence to be grammatical. We argue that the results are not entirely consistent with the account offered by Mirault et al. (2018), which attributes failure to notice transpositions to parallel processing of adjacent words, or with a late, post-perceptual rational inference account (Gibson, Bergen, & Piantadosi, 2013). We propose that word recognition is serial, but post-lexical integration of each word into its context may not be perfectly incremental.

Failure to stop autocorrect errors in reading aloud increases in aging especially with a positive biomarker for Alzheimer's disease

The present study examined the effects of aging and CSF biomarkers of Alzheimer's disease (AD) on the ability to control production of unexpected words in connected speech elicited by reading aloud. Fifty-two cognitively healthy participants aged 66-86 read aloud 6 paragraphs with 10 malapropisms including 5 on content words (e.g., "window cartons" that elicited autocorrect errors to "window curtains") and 5 on function words (e.g., "thus concept" that elicited autocorrections to "this concept") and completed a battery of neuropsychological tests including a standardized Stroop task. Reading aloud elicited more autocorrect errors on function than content words, but these were equally correlated with age and Aβ1-42 levels. The ability to stop autocorrect errors declined in aging and with lower (more AD-like) levels of Aβ1-42, and multiplicatively so, such that autocorrect errors were highest in the oldest-old with the lowest Aβ1-42 levels. Critically, aging effects were significant even when controlling statistically for Aβ1-42. Finally, both autocorrect and Stroop errors were correlated with Aβ1-42, but only autocorrect errors captured unique variance in predicting Aβ1-42 levels. Reading aloud requires simultaneous planning and monitoring of upcoming speech. These results suggest that healthy aging leads to decline in the ability to intermittently monitor for and detect conflict during speech planning and that subtle cognitive changes in preclinical AD magnify this aging deficit. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Cognitive control regions are recruited in bilinguals' silent reading of mixed-language paragraphs

When switching languages, bilinguals recruit a language control network that overlaps with brain regions known to support general cognitive control, but it is unclear whether these same regions are recruited in passive comprehension of language switches. Using fMRI with a blocked design, 24 Spanish-English bilinguals silently read 36 paragraphs in which the default language was Spanish or English, and that had either (1) no switches, (2) function word switches or (3) content word switches. Relative to no switches, function switches activated the right IFG, bilateral MFG, and left IPL/SMG. In contrast, switching on content words produced limited neural switching costs observed only in the left IFG. Switching into the dominant language was more costly in the right SMG than switching into the nondominant language, and neural switching costs were correlated with switching costs in the dominant language in cued picture-naming. Seemingly passive reading comprehension involves brain regions known to support cognitive control in active switching during production, possibly reflecting the operation of a modality-general switch mechanism.

Intact reversed language-dominance but exaggerated cognate effects in reading aloud of language switches in bilingual Alzheimer's disease

OBJECTIVE

The current study investigated how Alzheimer's disease (AD) affects production of speech errors in reading-aloud of mixed-language passages with language switches on cognates (e.g., family/familia), noncognates (e.g., people/gente), and function words (the/la).

METHOD

Twelve Spanish-English bilinguals with AD and 22 controls read-aloud 8 paragraphs in 4 conditions: (a) English-default content switches, (b) English-default function switches, (c) Spanish-default content switches, and (d) Spanish-default function switches.

RESULTS

Reading elicited language intrusions (e.g., saying la instead of the), and several types of within-language errors (e.g., saying their instead of the). Reversed language-dominance effects were intact in AD; both patients and controls produced many intrusions on dominant language targets, and relatively fewer intrusions on nondominant language targets. The opposite held for within-language errors, which were more common with nondominant than dominant targets. Patients produced the most intrusion errors with cognate switch words (which best distinguished patients from controls in ROC curves of all speech error types), while controls had equal difficulty switching on cognate and function word targets.

CONCLUSIONS

Reversed language-dominance effects appear to illustrate automatic inhibitory control over the dominant language, but could instead reflect limited resources available for monitoring when completing a task in the nondominant language. The greater sensitivity of intrusion errors with cognate than with function word targets for distinguishing patients from controls implies that language control may be aided by relatively intact knowledge of grammatical constraints over code-switching in bilinguals with AD. (PsycINFO Database Record (c) 2020 APA, all rights reserved).