Orthographic processing: A ‘mid-level’ vision of reading: The 44th Sir Frederic Bartlett Lecture

Do Grading Gray Stimuli Help to Encode Letter Position?

Numerous experiments in the past decades recurrently showed that a transposed-letter pseudoword (e.g., JUGDE) is much more wordlike than a replacement-letter control (e.g., JUPTE). Critically, there is an ongoing debate as to whether this effect arises at a perceptual level (e.g., perceptual uncertainty at assigning letter position of an array of visual objects) or at an abstract language-specific level (e.g., via a level of "open bigrams" between the letter and word levels). Here, we designed an experiment to test the limits of perceptual accounts of letter position coding. The stimuli in a lexical decision task were presented either with a homogeneous letter intensity or with a graded gray intensity, which indicated an unambiguous letter order. The pseudowords were either transposed-letter pseudowords or replaced-letter pseudowords (e.g., jugde vs. jupte). The results showed much longer response times and substantially more errors in the transposed-letter pseudowords than in the replacement-letter pseudowords, regardless of visual format. These findings favor the idea that language-specific orthographic element factors play an essential role when encoding letter position during word recognition.

Does the cowl make the monk? Detecting counterfeits in brand names versus logos

Companies and products are identified by their brand names, which are typically written with a specific letter style, color, and design (i.e., logos). This graphical information offers a distinctive image that facilitates their recognition. However, the uniqueness of these configuration cues may make brand names more vulnerable to counterfeiting via misspelling. We examined whether the confusability at detecting misspelled brand names is higher when embedded in the full logo than when presented in plain format (Experiment 1), when removing all graphical information of the logo other than typeface (Experiment 2), and when only modifying the typeface (Experiment 3). Participants had to decide whether the presented item was a correctly spelled brand name. The misspelled stimuli were created by either transposing or replacing two internal letters of popular brand names (amazon → amzaon vs. amceon), thus allowing us to have a measure of the transposed-letter confusability effect. Results showed a sizeable transposed-letter confusability effect for all types of brand names, but the effect was greatest for the misspelled full logos. Thus, the distinctiveness of the graphical information in logos has a deleterious side effect: logos are quite vulnerable to counterfeiting via misspelling branding.

An ERP investigation of transposed-word effects in same-different matching

Can several words be read in parallel, and if so, how is information about word order encoded under such circumstances? Here we focused on the bottom-up mechanisms involved in word-order encoding under the hypothesis of parallel word processing. We recorded EEG while participants performed a visual same-different matching task with sequences of five words (reference sequence followed by a target sequence each presented for 400 ms). The reference sequence could be grammatically correct or an ungrammatical scrambling of the same words (e.g., he wants these green apples/green wants these he apples). Target sequences for 'different' responses were created by either transposing two words in the reference (e.g., he these wants green apples/green these wants he apples), or by changing two words (e.g., he talks their green apples/green talks their he apples). Different responses were harder to make in the transposition condition, and this transposed-word effect started to emerge around 250 ms post-target onset. The transposed-word effect was first seen on an early onsetting N400 component, with reduced amplitudes (i.e., less negative ERPs) in the transposed condition relative to a two-word replacement condition. A later transposed-word effect was seen on a more temporally widespread positive-going component. Converging behavioral and EEG results showed no effects of reference grammaticality on 'different' responses nor an interaction with transposed-word effects. Our results point to the noisy, bottom-up association of word identities to spatiotopic locations as one means of encoding word order information, and one key source of transposed-word effects.

Is letter position coding when reading in L2 affected by the nature of position coding used when bilinguals read in their L1?

Using four-character Chinese word targets, Yang, Chen, Spinelli, and Lupker (Journal of Experimental Psychology: Learning, Memory, and Cognition, 45(8), 1511-1526, 2019) and Yang, Hino et al. (Journal of Memory and Language, 113, 104017, 2020) demonstrated that backward primes (Roman alphabet example-dcba priming ABCD) produce large masked priming effects. This result suggests that character position information is quite imprecisely coded by Chinese readers when reading in their native language. The present question was, If Chinese readers have evolved a reading system not requiring precise position information, would Chinese-English bilinguals show more extreme transposed letter priming effects when processing English words than both English monolinguals and other types of bilinguals whose L2 is English? In Experiment 1, Chinese-English bilinguals, but not English monolinguals, showed a clear backward priming effect in a lexical decision task. In Experiment 2, the parallel backward priming effect was absent for both Spanish-English and Arabic-English bilinguals. Apparently, the orthographic coding system that Chinese-English bilinguals use when reading in their L2 leans heavily on the flexible/imprecise position coding process that they develop for reading in their L1.

A general-purpose mechanism of visual feature association in visual word identification and beyond

As writing systems are a relatively novel invention (slightly over 5 kya),¹ they could not have influenced the evolution of our species. Instead, reading might recycle evolutionary older mechanisms that originally supported other tasks^2,3 and preceded the emergence of written language. Accordingly, it has been shown that baboons and pigeons can be trained to distinguish words from nonwords based on orthographic regularities in letter co-occurrence.^4,5 This suggests that part of what is usually considered reading-specific processing could be performed by domain-general visual mechanisms. Here, we tested this hypothesis in humans: if the reading system relies on domain-general visual mechanisms, some of the effects that are often found with orthographic material should also be observable with non-orthographic visual stimuli. We performed three experiments using the same exact design but with visual stimuli that progressively departed from orthographic material. Subjects were passively familiarized with a set of composite visual items and tested in an oddball paradigm for their ability to detect novel stimuli. Participants showed robust sensitivity to the co-occurrence of features ("bigram" coding) with strings of letter-like symbols but also with made-up 3D objects and sinusoidal gratings. This suggests that the processing mechanisms involved in the visual recognition of novel words also support the recognition of other novel visual objects. These mechanisms would allow the visual system to capture statistical regularities in the visual environment.^6-9 We hope that this work will inspire models of reading that, although addressing its unique aspects, place it within the broader context of vision.

On the noisy spatiotopic encoding of word positions during reading: Evidence from the change-detection task

The present study builds on our prior work showing evidence for noisy word-position coding in an immediate same-different matching task. In that research, participants found it harder to judge that two successive brief presentations of five-word sequences were different when the difference was caused by transposing two adjacent words compared with different word replacements – a transposition effect. Here we used the change-detection task with a 1-s delay introduced between sequences – a task thought to tap into visual short-term memory. Concurrent articulation was used to limit the contribution of active rehearsal. We used standard response-time (RT) and error-rate analyses plus signal detection theory (SDT) measures of discriminability (d’) and bias (c). We compared the transposition effects for ungrammatical word sequences and nonword sequences observed with these different measures. Although there was some evidence for transposition effects with nonwords, the effects were much larger with word sequences. These findings provide further support for the hypothesized noisy assignment of word identities to spatiotopic locations along a line of text during reading.

Orthographic and phonological contributions to flanker effects

Does phonology contribute to effects of orthographically related flankers in the flankers task? In order to answer this question, we implemented the flanker equivalent of a pseudohomophone priming manipulation that has been widely used to demonstrate automatic phonological processing during visual word recognition. In Experiment 1, central target words were flanked on each side by either a pseudohomophone of the target (e.g., roze rose roze), an orthographic control pseudoword (rone rose rone), or an unrelated pseudoword (mirt rose mirt). Both the pseudohomophone and the orthographic control conditions produced faster and more accurate responses to central targets, but performance in these two conditions did not differ significantly. Experiment 2 tested the same stimuli in a masked priming paradigm and replicated the standard finding in French that pseudohomophone primes produce significantly faster responses to target words than orthographic control primes. Therefore, contrary to its impact on masked priming, phonology does not contribute to effects of flanker relatedness, which would appear to be driven primarily by orthographic overlap.

A transposed-word effect in Chinese reading

Studies using a grammaticality decision task suggest surprising flexibility in the processing of the relative order of words in sentences when reading alphabetic scripts like French. In these studies, participants made rapid grammaticality decisions for ungrammatical stimuli created by transposing two adjacent words in either a grammatical or an ungrammatical base sentence, which were intermixed with equal numbers of grammatically correct stimuli. The key finding was that participants made more errors and were slower to reject transposed-word stimuli created from grammatical than ungrammatical base sentences. This suggested that flexibility in the processing of word order allowed participants to access representations of the base grammatical sentences, interfering with their decisions to correctly reject transposed-word stimuli. With the present research, we investigated if a similar transposed-word effect is observed for a non-alphabetic script (Chinese) that uses few grammatical markers and primarily conveys grammatical structure via word order. Such scripts may require stricter processing of word order during reading and so provide a strong test of the cross-linguistic generality of the transposed-word effect. We report three experiments using the same design and procedure as previous research, while varying the length of the transposed words across experiments. In all three experiments, participants made more errors and were slower to reject transposed-word stimuli derived from grammatical than ungrammatical base sentences. This replicates previous findings with alphabetic scripts and provides novel evidence for a transposed-word effect in Chinese reading. We consider the implications for models of reading in alphabetic and non-alphabetic scripts.

Does a mark make a difference? Visual similarity effects with accented vowels

Visual similarity effects are pervasive in masked priming (e.g., T4BLE→TABLE; obiect→OBJECT; docurnent→DOCUMENT) and can be easily explained in terms of uncertainty regarding letter identity. However, recent research failed to show visual similarity effects for primes containing accented vowels (e.g., féliz-FELIZ behaves as fáliz-FELIZ [happy in Spanish]). This null effect has been taken to suggest that accented and non-accented vowels (e.g., é and e) activate completely distinct representations. However, priming effects are reinstated for non-accented vowels (e.g., facil-FÁCIL < fecil-FÁCIL [easy in Spanish]). Here we tested the hypothesis that the lack of priming effects for primes containing accented vowels is a simple consequence of the saliency of the accent marks. To investigate this issue, we conducted a masked priming lexical decision experiment in which we minimized the saliency of the diacritical marks by using primes containing the letter i (i.e., a letter that contains itself a glyph over the letter). We manipulated prime-target visual similarity and the presence/absence of an accented vowel in the prime (e.g., obieto-OBJETO vs. obaeto-OBJETO; obíeto-OBJETO vs. obáeto-OBJETO [object in Spanish]). Results showed a sizeable visual similarity effect regardless of whether the prime was accented or not. Therefore, these findings suggest that, at least in scripts like Spanish, there is nothing special about the processing of accented vs. unaccented vowels once the saliency of the diacritical marks is reduced.

Does orthographic processing emerge rapidly after learning a new script?

Orthographic processing is characterized by location-invariant and location-specific processing (Grainger, 2018): (1) strings of letters are more vulnerable to transposition effects than the strings of symbols in same-different tasks (location-invariant processing); and (2) strings of letters, but not strings of symbols, show an initial position advantage in target-in-string identification tasks (location-specific processing). To examine the emergence of these two markers of orthographic processing, we conducted a same-different task and a target-in-string identification task with two unfamiliar scripts (pre-training experiments). Across six training sessions, participants learned to fluently read and write one of these scripts. The post-training experiments were parallel to the pre-training experiments. Results showed that the magnitude of the transposed-letter effect in the same-different task and the serial function in the target-in-string identification tasks were remarkably similar for the trained and untrained scripts. Thus, location-invariant and location-specific processing does not emerge rapidly after learning a new script; instead, they may require thorough experience with specific orthographic structures.

Literacy skill and intra-individual variability in eye-fixation durations during reading: Evidence from a diverse community-based adult sample

To understand the effects of literacy on fundamental processes involved in reading, we report a secondary data analysis examining individual differences in global eye-movement measures and first-pass eye-movement distributions in a diverse sample of community-dwelling adults aged 16 to 64. Participants (n = 80) completed an assessment battery probing verbal and non-verbal cognitive abilities and read simple two-sentence passages while their eye movements were recorded. Analyses were focused on characterising the effects of literacy skill on both global indices of eye-fixation distributions and distributional differences in the sensitivity to lexical features. Global reading measures showed that lower literate adults read more slowly on average. However, distributional analyses of fixation durations revealed that the first-pass fixation durations of adults with lower literacy skill were not slower in general (i.e., there was no shift in the fixation duration distribution among lower literate adults). Instead, lower literacy was associated with greater intra-individual variability in first-pass fixation durations, including an increased proportion of extremely long fixations, differentially skewing the distribution of both first-fixation and gaze durations. Exploratory repeated-measures quantile regression analyses of gaze duration revealed differentially greater influences of word length among lower literate readers and greater activation of phonological and orthographic neighbours among higher literate readers, particularly in the tail of the distribution. Collectively, these findings suggest that literacy skill in adulthood is associated with systematic differences in both global and lexically driven eye-movement control during reading.

A compositional neural code in high-level visual cortex can explain jumbled word reading

We read jubmled wrods effortlessly, but the neural correlates of this remarkable ability remain poorly understood. We hypothesized that viewing a jumbled word activates a visual representation that is compared to known words. To test this hypothesis, we devised a purely visual model in which neurons tuned to letter shape respond to longer strings in a compositional manner by linearly summing letter responses. We found that dissimilarities between letter strings in this model can explain human performance on visual search, and responses to jumbled words in word reading tasks. Brain imaging revealed that viewing a string activates this letter-based code in the lateral occipital (LO) region and that subsequent comparisons to stored words are consistent with activations of the visual word form area (VWFA). Thus, a compositional neural code potentially contributes to efficient reading.

Holistic word processing is correlated with efficiency in visual word recognition

Holistic processing of visual words (i.e., obligatory encoding of/attending to all letters of a word) could be a marker of expert word recognition. In the present study, we thus examined for the first time whether there is a direct relation between the word-composite effect (i.e., all parts of a visual word are fully processed when observers perform a task on a word part) and fast access to the orthographic lexicon by visual word experts (i.e., fluent adult readers). We adopted an individual differences approach and used the word-frequency effect (i.e., faster recognition of high- than low-frequency words) in an independent lexical decision task as a proxy of fast access to lexical orthographic representations. Fluent readers with larger word-composite effect showed smaller word-frequency effect. This correlation was mainly driven by an association between a larger composite effect and faster lexical decision on low-frequency words, probably because these lexical representations are less stable and integrated/unitized, hence allowing differentiating among fluent readers. We thus showed that holistic processing of visual words is indeed related to higher efficiency in visual word recognition by skilled readers.

Mapping visual symbols onto spoken language along the ventral visual stream

Learning to read is the most important milestone in a child’s education. However, controversies remain regarding how readers’ brains transform written words into sounds and meanings. We address these by combining artificial language learning with neuroimaging to reveal how the brain represents written words. Participants learned to read new words written in 2 different alphabets. Following 2 wk of training, we found a hierarchy of brain areas that support reading. Letter position is represented more flexibly from lower to higher visual regions. Furthermore, higher visual regions encode information about word sounds and meanings. These findings advance our understanding of how the brain comprehends language from arbitrary visual symbols.

Reading involves transforming arbitrary visual symbols into sounds and meanings. This study interrogated the neural representations in ventral occipitotemporal cortex (vOT) that support this transformation process. Twenty-four adults learned to read 2 sets of 24 novel words that shared phonemes and semantic categories but were written in different artificial orthographies. Following 2 wk of training, participants read the trained words while neural activity was measured with functional MRI. Representational similarity analysis on item pairs from the same orthography revealed that right vOT and posterior regions of left vOT were sensitive to basic visual similarity. Left vOT encoded letter identity and representations became more invariant to position along a posterior-to-anterior hierarchy. Item pairs that shared sounds or meanings, but were written in different orthographies with no letters in common, evoked similar neural patterns in anterior left vOT. These results reveal a hierarchical, posterior-to-anterior gradient in vOT, in which representations of letters become increasingly invariant to position and are transformed to convey spoken language information.

Reading without spaces revisited: The role of word identification and sentence-level constraints

The present study examined the relative contribution of bottom-up word identification and top-down sentence-level constraints in facilitating the reading of text printed without between-word spacing. We compared reading of grammatically correct sentences and shuffled versions of the same words presented both with normal spacing and without spaces. We found that reading was hampered by removing sentence structure as well as by removing spaces. A significantly greater impact of sentence structure when reading unspaced text was found in probe word identification accuracies and total viewing times per word, whereas the impact of sentence structure on the probability of making a regressive eye movement was greater when reading normally spaced text. Crucially, we also found that the length of the currently fixated word determined the amplitude of forward saccades leaving that word during the reading of unspaced text. We conclude that the relative ease with which skilled readers can read unspaced text is due to a combination of an increased use of bottom-up word identification in guiding the timing and targeting of eye movements, plus an increased interactivity between word identification and sentence-level processing.

Reading without spaces: The role of precise letter order

Prior research points to efficient identification of embedded words as a key factor in facilitating the reading of text printed without spacing between words. Here we further tested the primary role of bottom-up word identification by altering this process with a letter transposition manipulation. In two experiments, we examined silent reading and reading aloud of normal sentences and sentences containing words with letter transpositions, in both normally spaced and unspaced conditions. We predicted that letter transpositions should be particularly harmful for reading unspaced text. In line with our prediction, the majority of our measures of reading fluency showed that unspaced text with letter transpositions was disproportionately difficult to read. These findings provide further support for the claim that reading text without between-word spacing relies principally on efficient bottom-up processing, enabling accurate word identification in the absence of visual cues to identify word boundaries.

Reading by extracting invariant line junctions in typical and atypical young readers

We aimed at investigating whether typical and atypical young readers extract vertices (viewpoint-invariant line junctions) in reading, as has been shown for fluent adult readers. In an identification task, we presented partly deleted printed letters, words, and pseudowords, preserving either the vertices or the midsegments of the letters. This allowed assessing the occurrence of a vertex effect, that is, more errors when vertices are partly removed, keeping the midsegments intact, than in the reverse situation. In Experiment 1, the vertex effect was observed on words and pseudowords in three groups of typical readers: 48 adults, 56 beginning readers (Grades 2 and 3), and 42 more advanced readers (Grades 4 and 5). Yet, the effect was smaller in the beginning readers, in relation to their irregular word reading skills. In Experiment 2, we compared 40 children with dyslexia with children selected from Experiment 1 to match them on either chronological age (30 CA controls) or reading level (42 RL controls). Although all groups displayed a vertex effect on words and pseudowords, dyslexic children presented a smaller effect than CA controls without differing from RL controls. The whole result pattern suggests that vertices play an important role in the recognition of written strings not only in skilled adult readers but also in young readers, in relation to their actual reading skills rather than to a specific reading deficit.

MEGALEX: A megastudy of visual and auditory word recognition

Using the megastudy approach, we report a new database (MEGALEX) of visual and auditory lexical decision times and accuracy rates for tens of thousands of words. We collected visual lexical decision data for 28,466 French words and the same number of pseudowords, and auditory lexical decision data for 17,876 French words and the same number of pseudowords (synthesized tokens were used for the auditory modality). This constitutes the first large-scale database for auditory lexical decision, and the first database to enable a direct comparison of word recognition in different modalities. Different regression analyses were conducted to illustrate potential ways to exploit this megastudy database. First, we compared the proportions of variance accounted for by five word frequency measures. Second, we conducted item-level regression analyses to examine the relative importance of the lexical variables influencing performance in the different modalities (visual and auditory). Finally, we compared the similarities and differences between the two modalities. All data are freely available on our website ( https://sedufau.shinyapps.io/megalex/ ) and are searchable at www.lexique.org , inside the Open Lexique search engine.

You That Read Wrong Again! A Transposed-Word Effect in Grammaticality Judgments

We report a novel transposed-word effect in speeded grammaticality judgments made about five-word sequences. The critical ungrammatical test sequences were formed by transposing two adjacent words from either a grammatical base sequence (e.g., “The white cat was big” became “The white was cat big”) or an ungrammatical base sequence (e.g., “The white cat was slowly” became “The white was cat slowly”). These were intermixed with an equal number of correct sentences for the purpose of the grammaticality judgment task. In a laboratory experiment ( N = 57) and an online experiment ( N = 94), we found that ungrammatical decisions were harder to make when the ungrammatical sequence originated from a grammatically correct base sequence. This provides the first demonstration that the encoding of word order retains a certain amount of uncertainty. We further argue that the novel transposed-word effect reflects parallel processing of words during written sentence comprehension combined with top-down constraints from sentence-level structures.

Visual letter similarity effects during sentence reading: Evidence from the boundary technique

The study of how the cognitive system encodes letter identities from the visual input has received much attention in models of visual word recognition but it has typically been overlooked in models of eye movement control in reading. Here we examined how visual letter similarity affects early word processing during reading using Rayner's (1975) boundary change technique in which the parafoveal preview of the target word was either identical (e.g., frito-frito [fried]) or a one-letter-different nonword (e.g., frjto-frito vs. frgto-frito). Critically, the substituted letter in the nonword was visually similar (based on letter confusability norms) or visually dissimilar. Results showed shorter viewing times on the target word when the parafoveal preview was visually similar than when it was visually dissimilar. Thus, visual letter similarity modulates the integration of parafoveal and foveal information during sentence reading. Future implementations of models of eye movement control in reading should incorporate a more developed orthographic-lexical module to capture these effects.

Does letter rotation slow down orthographic processing in word recognition?

Leading neural models of visual word recognition assume that letter rotation slows down the conversion of the visual input to a stable orthographic representation (e.g., local detectors combination model; Dehaene, Cohen, Sigman, & Vinckier, 2005, Trends in Cognitive Sciences, 9, 335-341). If this premise is true, briefly presented rotated primes should be less effective at activating word representations than those primes with upright letters. To test this question, we conducted a masked priming lexical decision experiment with vertically presented words either rotated 90° or in marquee format (i.e., vertically but with upright letters). We examined the impact of the format on both letter identity (masked identity priming: identity vs. unrelated) and letter position (masked transposed-letter priming: transposed-letter prime vs. replacement-letter prime). Results revealed sizeable masked identity and transposed-letter priming effects that were similar in magnitude for rotated and marquee words. Therefore, the reading cost from letter rotation does not arise in the initial access to orthographic/lexical representations.

Stimulus orientation and the first-letter advantage

A post-cued partial report target-in-string identification experiment examined the influence of stimulus orientation on the serial position functions for strings of five consonants or five symbols, with an aim to test different accounts of the first-letter advantage observed in prior research. Under one account, this phenomenon is driven by processing that is specific to horizontally arranged letter (and digit) strings. An alternative account explains the first-letter advantage in terms of attentional biases towards the beginning of letter strings. We observed a significant three-way interaction between stimulus type (letters vs. symbols), serial position (1–5), and orientation (horizontal vs. vertical) that was driven by a greater first-position advantage for letters than symbols when stimuli were presented horizontally compared with vertical presentation. These results provide support for the letter-specific processing account of the first-letter advantage, and further suggest that differences in visual complexity between letters and symbols play a minor role. Nevertheless, a first-position advantage for letters was observed in the vertical presentation condition, thus pointing to some role for attentional biases that operate independently of string orientation.

•

We examined serial position functions for strings of letters and symbols.

•

Strings were displayed horizontally and vertically for a partial-report task.

•

Letters yielded a first-position advantage, larger for horizontal displays.

•

Results are discussed with reference to orthographic and attentional accounts.