Position sensitivity in the visual word form area

Orthographic and phonological processing in the left ventral occipitotemporal cortex during Chinese word reading

The left ventral occipitotemporal cortex (lvOT) has been consistently identified as a crucial structure in word reading, and its function varies across subregions. Nevertheless, the specific function of the lvOT and its subregions remains controversial because the obvious grapheme-to-phoneme correspondence rules of alphabetic languages make it difficult to disentangle the contributions of orthography and phonology to neural activations. To explore information processing in lvOT subregions, the present study manipulated the orthography and phonology in a factorial design and used the fMRI rapid adaptation paradigm. The results revealed a posterior-to-anterior functional gradient in lvOT in Chinese word reading and specified that the functional transition from sublexical to lexical processing occurred in the middle subregion close to the classic VWFA. More importantly, we found that the middle and posterior subregions of lvOT are responsible for processing both orthographic and phonological information during Chinese word reading. These results elaborated the function of the lvOT in Chinese word reading.

Cracking the neural code for word recognition in convolutional neural networks

Learning to read places a strong challenge on the visual system. Years of expertise lead to a remarkable capacity to separate similar letters and encode their relative positions, thus distinguishing words such as FORM and FROM, invariantly over a large range of positions, sizes and fonts. How neural circuits achieve invariant word recognition remains unknown. Here, we address this issue by recycling deep neural network models initially trained for image recognition. We retrain them to recognize written words and then analyze how reading-specialized units emerge and operate across the successive layers. With literacy, a small subset of units becomes specialized for word recognition in the learned script, similar to the visual word form area (VWFA) in the human brain. We show that these units are sensitive to specific letter identities and their ordinal position from the left or the right of a word. The transition from retinotopic to ordinal position coding is achieved by a hierarchy of "space bigram" unit that detect the position of a letter relative to a blank space and that pool across low- and high-frequency-sensitive units from early layers of the network. The proposed scheme provides a plausible neural code for written words in the VWFA, and leads to predictions for reading behavior, error patterns, and the neurophysiology of reading.

Reading Reshapes Stimulus Selectivity in the Visual Word Form Area

Reading depends on a brain region known as the "visual word form area" (VWFA) in the left ventral occipitotemporal cortex. This region's function is debated because its stimulus selectivity is not absolute, it is modulated by a variety of task demands, and it is inconsistently localized. We used fMRI to characterize the combination of sensory and cognitive factors that activate word-responsive regions that we precisely localized in 16 adult humans (4 male). We then presented three types of character strings: English words, pseudowords, and unfamiliar characters with matched visual features. Participants performed three different tasks while viewing those stimuli: detecting real words, detecting color in the characters, and detecting color in the fixation mark. There were three primary findings about the VWFA's response: (1) It preferred letter strings over unfamiliar characters even when the stimuli were ignored during the fixation task. (2) Compared with those baseline responses, engaging in the word reading task enhanced the response to words but suppressed the response to unfamiliar characters. (3) Attending to the stimuli to judge their color had little effect on the response magnitudes. Thus, the VWFA is uniquely modulated by a cognitive signal that is specific to voluntary linguistic processing and is not additive. Functional connectivity analyses revealed that communication between the VWFA and a left frontal language area increased when the participant engaged in the linguistic task. We conclude that the VWFA is inherently selective for familiar orthography, but it falls under control of the language network when the task demands it.

Reading reshapes stimulus selectivity in the visual word form area

Reading depends on a brain region known as the "visual word form area" (VWFA) in left ventral occipito-temporal cortex. This region's function is debated because its stimulus selectivity is not absolute, it is modulated by a variety of task demands, and it is inconsistently localized. We used fMRI to characterize the combination of sensory and cognitive factors that activate word-responsive regions that we precisely localized in 16 adult humans (4 male). We then presented three types of character strings: English words, pseudowords, and unfamiliar characters with matched visual features. Participants performed three different tasks while viewing those stimuli: detecting real words, detecting color in the characters, and detecting color in the fixation mark. There were three primary findings about the VWFA's response: (1) It preferred letter strings over unfamiliar characters even when the stimuli were ignored during the fixation task; (2) Compared to those baseline responses, engaging in the word reading task enhanced the response to words but suppressed the response to unfamiliar characters. (3) Attending to the stimuli to judge their font color had little effect on the response magnitudes. Thus, the VWFA is uniquely modulated by a cognitive signal that is specific to voluntary linguistic processing and is not additive. Functional connectivity analyses revealed that communication between the VWFA and a left frontal language area increased when the participant engaged in the linguistic task. We conclude that the VWFA is inherently selective for familiar orthography, but it falls under control of the language network when the task demands it.

Perception and Memory Reinstatement Engage Overlapping Face-Selective Regions within Human Ventral Temporal Cortex

Humans have the remarkable ability to vividly retrieve sensory details of past events. According to the theory of sensory reinstatement, during remembering, brain regions specialized for processing specific sensory stimuli are reactivated to support content-specific retrieval. Recently, several studies have emphasized transformations in the spatial organization of these reinstated activity patterns. Specifically, studies of scene stimuli suggest a clear anterior shift in the location of retrieval activations compared with the activity observed during perception. However, it is not clear that such transformations occur universally, with inconsistent evidence for other important stimulus categories, particularly faces. One challenge in addressing this question is the careful delineation of face-selective cortices, which are interdigitated with other selective regions, in configurations that spatially differ across individuals. Therefore, we conducted a multisession neuroimaging study to first carefully map individual participants' (nine males and seven females) face-selective regions within ventral temporal cortex (VTC), followed by a second session to examine the activity patterns within these regions during face memory encoding and retrieval. While face-selective regions were expectedly engaged during face perception at encoding, memory retrieval engagement exhibited a more selective and constricted reinstatement pattern within these regions, but did not show any consistent direction of spatial transformation (e.g., anteriorization). We also report on unique human intracranial recordings from VTC under the same experimental conditions. These findings highlight the importance of considering the complex configuration of category-selective cortex in elucidating principles shaping the neural transformations that occur from perception to memory.

Engaging in word recognition elicits highly specific modulations in visual cortex

A person's cognitive state determines how their brain responds to visual stimuli. The most common such effect is a response enhancement when stimuli are task relevant and attended rather than ignored. In this fMRI study, we report a surprising twist on such attention effects in the visual word form area (VWFA), a region that plays a key role in reading. We presented participants with strings of letters and visually similar shapes, which were either relevant for a specific task (lexical decision or gap localization) or ignored (during a fixation dot color task). In the VWFA, the enhancement of responses to attended stimuli occurred only for letter strings, whereas non-letter shapes evoked smaller responses when attended than when ignored. The enhancement of VWFA activity was accompanied by strengthened functional connectivity with higher-level language regions. These task-dependent modulations of response magnitude and functional connectivity were specific to the VWFA and absent in the rest of visual cortex. We suggest that language regions send targeted excitatory feedback into the VWFA only when the observer is trying to read. This feedback enables the discrimination of familiar and nonsense words and is distinct from generic effects of visual attention.

Task modulates the orthographic and phonological representations in the bilateral ventral Occipitotemporal cortex

As a key area in word reading, the left ventral occipitotemporal cortex is proposed for abstract orthographic processing, and its middle part has even been labeled as the visual word form area. Because the definition of the VWFA largely varies and the reading task differs across studies, the function of the left ventral occipitotemporal cortex in word reading is continuingly debated on whether this region is specific for orthographic processing or be involved in an interactive framework. By using representational similarity analysis (RSA), this study examined information representation in the VWFA at the individual level and the modulatory effect of reading task. Twenty-four subjects were scanned while performing the explicit (i.e., the naming task) and implicit (i.e., the perceptual task) reading tasks. Activation analysis showed that the naming task elicited greater activation in regions related to phonological processing (e.g., the bilateral prefrontal cortex and temporoparietal cortex), while the perceptual task recruited greater activation in visual cortex and default mode network (e.g., the bilateral middle frontal gyrus, angular gyrus, and the right middle temporal gyrus). More importantly, RSA also showed that task modulated information representation in the bilateral anterior occipitotemporal cortex and VWFA. Specifically, ROI-based RSA revealed enhanced orthographic and phonological representations in the bilateral anterior fusiform cortex and VWFA in the naming task relative to the perceptual task. These results suggest that lexical representation in the VWFA is influenced by the demand of phonological processing, which supports the interactive account of the VWFA.

Asymmetries around the visual field: From retina to cortex to behavior

Visual performance varies around the visual field. It is best near the fovea compared to the periphery, and at iso-eccentric locations it is best on the horizontal, intermediate on the lower, and poorest on the upper meridian. The fovea-to-periphery performance decline is linked to the decreases in cone density, retinal ganglion cell (RGC) density, and V1 cortical magnification factor (CMF) as eccentricity increases. The origins of polar angle asymmetries are not well understood. Optical quality and cone density vary across the retina, but recent computational modeling has shown that these factors can only account for a small percentage of behavior. Here, we investigate how visual processing beyond the cone photon absorptions contributes to polar angle asymmetries in performance. First, we quantify the extent of asymmetries in cone density, midget RGC density, and V1 CMF. We find that both polar angle asymmetries and eccentricity gradients increase from cones to mRGCs, and from mRGCs to cortex. Second, we extend our previously published computational observer model to quantify the contribution of phototransduction by the cones and spatial filtering by mRGCs to behavioral asymmetries. Starting with photons emitted by a visual display, the model simulates the effect of human optics, cone isomerizations, phototransduction, and mRGC spatial filtering. The model performs a forced choice orientation discrimination task on mRGC responses using a linear support vector machine classifier. The model shows that asymmetries in a decision maker's performance across polar angle are greater when assessing the photocurrents than when assessing isomerizations and are greater still when assessing mRGC signals. Nonetheless, the polar angle asymmetries of the mRGC outputs are still considerably smaller than those observed from human performance. We conclude that cone isomerizations, phototransduction, and the spatial filtering properties of mRGCs contribute to polar angle performance differences, but that a full account of these differences will entail additional contribution from cortical representations.

Anatomy and physiology of word-selective visual cortex: from visual features to lexical processing

Over the past 2 decades, researchers have tried to uncover how the human brain can extract linguistic information from a sequence of visual symbols. The description of how the brain's visual system processes words and enables reading has improved with the progressive refinement of experimental methodologies and neuroimaging techniques. This review provides a brief overview of this research journey. We start by describing classical models of object recognition in non-human primates, which represent the foundation for many of the early models of visual word recognition in humans. We then review functional neuroimaging studies investigating the word-selective regions in visual cortex. This research led to the differentiation of highly specialized areas, which are involved in the analysis of different aspects of written language. We then consider the corresponding anatomical measurements and provide a description of the main white matter pathways carrying neural signals crucial to word recognition. Finally, in an attempt to integrate structural, functional, and electrophysiological findings, we propose a view of visual word recognition, accounting for spatial and temporal facets of word-selective neural processes. This multi-modal perspective on the neural circuitry of literacy highlights the relevance of a posterior-anterior differentiation in ventral occipitotemporal cortex for visual processing of written language and lexical features. It also highlights unanswered questions that can guide us towards future research directions. Bridging measures of brain structure and function will help us reach a more precise understanding of the transformation from vision to language.

Visuospatial coding as ubiquitous scaffolding for human cognition

For more than 100 years we have known that the visual field is mapped onto the surface of visual cortex, imposing an inherently spatial reference frame on visual information processing. Recent studies highlight visuospatial coding not only throughout visual cortex, but also brain areas not typically considered visual. Such widespread access to visuospatial coding raises important questions about its role in wider cognitive functioning. Here, we synthesise these recent developments and propose that visuospatial coding scaffolds human cognition by providing a reference frame through which neural computations interface with environmental statistics and task demands via perception-action loops.

Emergence of a compositional neural code for written words: Recycling of a convolutional neural network for reading

The visual word form area (VWFA) is a region of human inferotemporal cortex that emerges at a fixed location in the occipitotemporal cortex during reading acquisition and systematically responds to written words in literate individuals. According to the neuronal recycling hypothesis, this region arises through the repurposing, for letter recognition, of a subpart of the ventral visual pathway initially involved in face and object recognition. Furthermore, according to the biased connectivity hypothesis, its reproducible localization is due to preexisting connections from this subregion to areas involved in spoken-language processing. Here, we evaluate those hypotheses in an explicit computational model. We trained a deep convolutional neural network of the ventral visual pathway, first to categorize pictures and then to recognize written words invariantly for case, font, and size. We show that the model can account for many properties of the VWFA, particularly when a subset of units possesses a biased connectivity to word output units. The network develops a sparse, invariant representation of written words, based on a restricted set of reading-selective units. Their activation mimics several properties of the VWFA, and their lesioning causes a reading-specific deficit. The model predicts that, in literate brains, written words are encoded by a compositional neural code with neurons tuned either to individual letters and their ordinal position relative to word start or word ending or to pairs of letters (bigrams).

Distinct neural sources underlying visual word form processing as revealed by steady state visual evoked potentials (SSVEP)

EEG has been central to investigations of the time course of various neural functions underpinning visual word recognition. Recently the steady-state visual evoked potential (SSVEP) paradigm has been increasingly adopted for word recognition studies due to its high signal-to-noise ratio. Such studies, however, have been typically framed around a single source in the left ventral occipitotemporal cortex (vOT). Here, we combine SSVEP recorded from 16 adult native English speakers with a data-driven spatial filtering approach—Reliable Components Analysis (RCA)—to elucidate distinct functional sources with overlapping yet separable time courses and topographies that emerge when contrasting words with pseudofont visual controls. The first component topography was maximal over left vOT regions with a shorter latency (approximately 180 ms). A second component was maximal over more dorsal parietal regions with a longer latency (approximately 260 ms). Both components consistently emerged across a range of parameter manipulations including changes in the spatial overlap between successive stimuli, and changes in both base and deviation frequency. We then contrasted word-in-nonword and word-in-pseudoword to test the hierarchical processing mechanisms underlying visual word recognition. Results suggest that these hierarchical contrasts fail to evoke a unitary component that might be reasonably associated with lexical access.

Reading: The Confluence of Vision and Language

The scientific study of reading has a rich history that spans disciplines from vision science to linguistics, psychology, cognitive neuroscience, neurology, and education. The study of reading can elucidate important general mechanisms in spatial vision, attentional control, object recognition, and perceptual learning, as well as the principles of plasticity and cortical topography. However, literacy also prompts the development of specific neural circuits to process a unique and artificial stimulus. In this review, we describe the sequence of operations that transforms visual features into language, how the key neural circuits are sculpted by experience during development, and what goes awry in children for whom learning to read is a struggle. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Is there a serial bottleneck in visual object recognition?

Divided attention has little effect for simple tasks, such as luminance detection, but it has large effects for complex tasks, such as semantic categorization of masked words. Here, we asked whether the semantic categorization of visual objects shows divided attention effects as large as those observed for words, or as small as those observed for simple feature judgments. Using a dual-task paradigm with nameable object stimuli, performance was compared with the predictions of serial and parallel models. At the extreme, parallel processes with unlimited capacity predict no effect of divided attention; alternatively, an all-or-none serial process makes two predictions: a large divided attention effect (lower accuracy for dual-task trials, compared to single-task trials) and a negative response correlation in dual-task trials (a given response is more likely to be incorrect when the response about the other stimulus is correct). These predictions were tested in two experiments examining object judgments. In both experiments, there was a large divided attention effect and a small negative correlation in responses. The magnitude of these effects was larger than for simple features, but smaller than for words. These effects were consistent with serial models, and rule out some but not all parallel models. More broadly, the results help establish one of the first examples of likely serial processing in perception.

Automaticity in the reading circuitry

Skilled reading requires years of practice associating visual symbols with speech sounds. Over the course of the learning process, this association becomes effortless and automatic. Here we test whether automatic activation of spoken-language circuits in response to visual words is a hallmark of skilled reading. Magnetoencephalography was used to measure word-selective responses under multiple cognitive tasks (N = 42, 7-12 years of age). Even when attention was drawn away from the words by performing an attention-demanding fixation task, strong word-selective responses were found in a language region (i.e., superior temporal gyrus) starting at ~300 ms after stimulus onset. Critically, this automatic word-selective response was indicative of reading skill: the magnitude of word-selective responses correlated with individual reading skill. Our results suggest that automatic recruitment of spoken-language circuits is a hallmark of skilled reading; with practice, reading becomes effortless as the brain learns to automatically translate letters into sounds and meaning.

The impact of emotional content on pseudoword recognition

The present study investigates the influence of emotional information on language processing. To this aim, we measured behavioral responses and event-related brain potentials (ERPs) during four Italian lexical decision experiments in which we used emotionally intense and neutral pseudowords—i.e., pseudowords derived from changing one letter in a word (e.g., cammelto, derived from cammello ‘camel’ vs. copezzolo, from capezzolo ‘nipple’)—as stimuli. In Experiment 1 and 2, half of the pseudowords were emotionally intense and half were neutral, and were mixed with neutral words. In Experiment 3, the list composition was manipulated, with ¼ of the pseudowords being derived from emotionally intense words and ¾ derived from neutral words. Experiment 4 was identical to Experiment 1, but ERPs were recorded. Emotionally intense pseudowords were categorized more slowly than neutral pseudowords, with the difference emerging both in the mean and at the leading edge of the response times distribution. Moreover, emotionally intense pseudowords elicited smaller N170 and N400 than neutral pseudowords. These results speak in favor of a fast and multi-level infiltration of the emotional information into the linguistic process of word recognition.

Can the right hemisphere read? A behavioral and disconnectome study on implicit reading in a patient with pure alexia

Patients with pure alexia have major difficulties in reading aloud. However, they often perform above chance level in reading tasks that do not require overt articulation of the target word - like lexical decision or semantic judgment - a phenomenon usually known as "implicit reading." There is no agreement in the literature on whether implicit reading should be attributed to relative sparing of some left hemisphere (LH) reading centers or rather to signs of compensatory endeavors by the right hemisphere (RH). We report the case of an 81-year-old patient (AA) with pure alexia due to a lesion involving the left occipital lobe and the temporal infero-mesial areas, as well as the posterior callosal pathways. Although AA's reading was severely impaired and proceeded letter by letter, she showed an above-chance-level performance for frequent concrete words in a tachistoscopic lexical decision task. A structural disconnectome analysis revealed that AA's lesion not only affected the left occipital cortex and the splenium: it also disconnected white-matter tracts meant to connect the visual word-form system to decision-related frontal areas within the LH. We suggest that the RH, rather than the LH, may be responsible for patient AA's implicit reading.

Neural sources of letter and Vernier acuity

Visual acuity can be measured in many different ways, including with letters and Vernier offsets. Prior psychophysical work has suggested that the two acuities are strongly linked given that they both depend strongly on retinal eccentricity and both are similarly affected in amblyopia. Here we used high-density EEG recordings to ask whether the underlying neural sources are common as suggested by the psychophysics or distinct. To measure visual acuity for letters, we recorded evoked potentials to 3 Hz alternations between intact and scrambled text comprised of letters of varying size. To measure visual acuity for Vernier offsets, we recorded evoked potentials to 3 Hz alternations between bar gratings with and without a set of Vernier offsets. Both alternation types elicited robust activity at the 3 Hz stimulus frequency that scaled in amplitude with both letter and offset size, starting near threshold. Letter and Vernier offset responses differed in both their scalp topography and temporal dynamics. The earliest evoked responses to letters occurred on lateral occipital visual areas, predominantly over the left hemisphere. Later responses were measured at electrodes over early visual cortex, suggesting that letter structure is first extracted in second-tier extra-striate areas and that responses over early visual areas are due to feedback. Responses to Vernier offsets, by contrast, occurred first at medial occipital electrodes, with responses at later time-points being more broadly distributed—consistent with feedforward pathway mediation. The previously observed commonalities between letter and Vernier acuity may be due to common bottlenecks in early visual cortex but not because the two tasks are subserved by a common network of visual areas.

Dissociating the functions of superior and inferior parts of the left ventral occipito-temporal cortex during visual word and object processing

During word and object recognition, extensive activation has consistently been observed in the left ventral occipito-temporal cortex (vOT), focused around the occipito-temporal sulcus (OTs). Previous studies have shown that there is a hierarchy of responses from posterior to anterior vOT regions (along the y-axis) that corresponds with increasing levels of recognition - from perceptual to semantic processing, respectively. In contrast, the functional differences between superior and inferior vOT responses (i.e. along the z-axis) have not yet been elucidated. To investigate, we conducted an extensive review of the literature and found that peak activation for reading varies by more than 1 cm in the z-axis. In addition, we investigated functional differences between superior and inferior parts of left vOT by analysing functional MRI data from 58 neurologically normal skilled readers performing 8 different visual processing tasks. We found that group activation in superior vOT was significantly more sensitive than inferior vOT to the type of task, with more superior vOT activation when participants were matching visual stimuli for their semantic or perceptual content than producing speech to the same stimuli. This functional difference along the z-axis was compared to existing boundaries between cytoarchitectonic areas around the OTs. In addition, using dynamic causal modelling, we show that connectivity from superior vOT to anterior vOT increased with semantic content during matching tasks but not during speaking tasks whereas connectivity from inferior vOT to anterior vOT was sensitive to semantic content for matching and speaking tasks. The finding of a functional dissociation between superior and inferior parts of vOT has implications for predicting deficits and response to rehabilitation for patients with partial damage to vOT following stroke or neurosurgery.

How are visual words represented? Insights from EEG-based visual word decoding, feature derivation and image reconstruction

Investigations into the neural basis of reading have shed light on the cortical locus and the functional role of visual‐orthographic processing. Yet, the fine‐grained structure of neural representations subserving reading remains to be clarified. Here, we capitalize on the spatiotemporal structure of electroencephalography (EEG) data to examine if and how EEG patterns can serve to decode and reconstruct the internal representation of visually presented words in healthy adults. Our results show that word classification and image reconstruction were accurate well above chance, that their temporal profile exhibited an early onset, soon after 100 ms, and peaked around 170 ms. Further, reconstruction results were well explained by a combination of visual‐orthographic word properties. Last, systematic individual differences were detected in orthographic representations across participants. Collectively, our results establish the feasibility of EEG‐based word decoding and image reconstruction. More generally, they help to elucidate the specific features, dynamics, and neurocomputational principles underlying word recognition.

Functionally Separable Font-invariant and Font-sensitive Neural Populations in Occipitotemporal Cortex

Reading relies on the rapid visual recognition of words viewed in a wide variety of fonts. We used fMRI to identify neural populations showing reduced fMRI responses to repeated words displayed in different fonts ("font-invariant" repetition suppression). We also identified neural populations showing greater fMRI responses to words repeated in a changing font as compared with words repeated in the same font ("font-sensitive" release from repetition suppression). We observed font-invariant repetition suppression in two anatomically distinct regions of the left occipitotemporal cortex (OT), a "visual word form area" in mid-fusiform cortex, and a more posterior region in the middle occipital gyrus. In contrast, bilateral shape-selective lateral occipital cortex and posterior fusiform showed considerable sensitivity to font changes during the viewing of repeated words. Although the visual word form area and the left middle occipital gyrus showed some evidence of font sensitivity, both regions showed a relatively greater degree of font invariance than font sensitivity. Our results show that the neural mechanisms in the left OT involved in font-invariant word recognition are anatomically distinct from those sensitive to font-related shape changes. We conclude that font-invariant representation of visual word form is instantiated at multiple levels by anatomically distinct neural mechanisms within the left OT.

The accessory Simon effect within and across visual dimensions

The Dimension-Action model maintains that response selection in the visual system is modular, such that response selection based on a target’s feature occurs within modules. This study suggests that response selection processes based on a target’s spatial location occur within modules as well, where spatial locations are coded along with the feature information. From this perspective, the typical Simon effect, in which interference occurs between a target’s feature and its spatial location, occurs within modules. This study explored whether the unique characteristic of the spatial Simon, namely, its reduction with increased reaction time is typical of spatial intra-dimension but not of spatial cross-dimension Simon effects, using the accessory Simon task. Experiment 1 demonstrated that intra-dimension Simon effects were reduced with increased reaction time, a reduction that was modulated by the task relevance of the distractor. In contrast, cross-dimension accessory Simon effects were positive and increased with reaction time. Experiment 2 demonstrated that intra-dimension Simon effects were not reduced when space was conveyed symbolically by arrows. Overall, the study suggests that interference in the accessory Simon task is influenced not only by the nature of the irrelevant spatial information but also by the modular locus of the targets and distractors.

Parallel spatial channels converge at a bottleneck in anterior word-selective cortex

In most environments, the visual system is confronted with many relevant objects simultaneously. That is especially true during reading. However, behavioral data demonstrate that a serial bottleneck prevents recognition of more than one word at a time. We used fMRI to investigate how parallel spatial channels of visual processing converge into a serial bottleneck for word recognition. Participants viewed pairs of words presented simultaneously. We found that retinotopic cortex processed the two words in parallel spatial channels, one in each contralateral hemisphere. Responses were higher for attended than for ignored words but were not reduced when attention was divided. We then analyzed two word-selective regions along the occipitotemporal sulcus (OTS) of both hemispheres (subregions of the visual word form area, VWFA). Unlike retinotopic regions, each word-selective region responded to words on both sides of fixation. Nonetheless, a single region in the left hemisphere (posterior OTS) contained spatial channels for both hemifields that were independently modulated by selective attention. Thus, the left posterior VWFA supports parallel processing of multiple words. In contrast, activity in a more anterior word-selective region in the left hemisphere (mid OTS) was consistent with a single channel, showing (i) limited spatial selectivity, (ii) no effect of spatial attention on mean response amplitudes, and (iii) sensitivity to lexical properties of only one attended word. Therefore, the visual system can process two words in parallel up to a late stage in the ventral stream. The transition to a single channel is consistent with the observed bottleneck in behavior.

Reading with letter transpositions in central and peripheral vision

We used a letter transposition (LT) technique to investigate letter position coding during reading in central and peripheral vision. Eighteen subjects read aloud sentences in a rapid serial visual presentation task. The tests contained a baseline and three LT conditions with initial, internal, and final transpositions (e.g., “reading” to “erading”, “raeding”, and “readign”). The four reading conditions were tested in separate blocks. We found that LT had a smaller cost on peripheral (10° lower field) than on central reading speed, possibly due to the higher intrinsic position uncertainty of letters in the periphery. The pattern of cost (initial > final > internal) was the same for central and peripheral vision, indicating a similar lexical route for both. In the periphery, LT only affected transposed words, while in central vision it also affected untransposed words. This spread of the LT effect in central vision could not be accounted for by increased attention or memory load, or by decreased sentence context.

Embedded word priming elicits enhanced fMRI responses in the visual word form area

Lexical embedding is common in all languages and elicits mutual orthographic interference between an embedded word and its carrier. The neural basis of such interference remains unknown. We employed a novel fMRI prime-target embedded word paradigm to test for involvement of a visual word form area (VWFA) in left ventral occipitotemporal cortex in co-activation of embedded words and their carriers. Based on the results of related fMRI studies we predicted either enhancement or suppression of fMRI responses to embedded words initially viewed as primes, and repeated in the context of target carrier words. Our results clearly showed enhancement of fMRI responses in the VWFA to embedded-carrier word pairs as compared to unrelated prime-target pairs. In contrast to non-visual language-related areas (e.g., left inferior frontal gyrus), enhanced fMRI responses did not occur in the VWFA when embedded-carrier word pairs were restricted to the left visual hemifield. Our finding of fMRI enhancement in the VWFA is novel evidence of its involvement in representational rivalry between orthographically similar words, and the co-activation of embedded words and their carriers.

Interhemispheric connectivity during lateralized lexical decision

The well-established right visual field (RVF-lh) advantage in word recognition is commonly attributed to the typical left hemisphere dominance in language; words presented to the LVF-rh are processed less efficiently due to the need for transcallosal transfer from the right to left hemisphere. The exact stage for this hemispheric transfer is currently unsettled. Some studies suggest that transfer occurs at very early stages between primary visual regions, whereas other studies suggest that transfer occurs between the left visual word form area and its right hemisphere homolog. This study explores these conflicting accounts and finds evidence for both. Participants conducted a lateralized lexical decision task with both unilateral and bilateral display conditions. Connectivity analyses were conducted from magnetoencephalography signals that were localized to the left middle occipital gyrus (LMOG), right middle occipital gyrus (RMOG), left visual word form area (LVWFA), and right visual word form area (RVWA). Results from unilateral trials showed asymmetrical interhemispheric connectivity from the RMOG to LMOG and symmetrical interhemispheric connectivity between the LVWFA and RVWFA. Furthermore, bilateral presentations led to reduced interhemispheric connectivity between both homologous region of interest pairs. Together, these results suggest that lateralized word recognition involves multiple stages of interhemispheric interactions and that these interactions are reduced with bilateral displays.

Orthographic and Phonological Representations in the Fusiform Cortex

Mental and neural representations of words are at the core of understanding the cognitive and neural mechanisms of reading. Despite extensive studies, the nature of visual word representation remains highly controversial due to methodological limitations. In particular, it is unclear whether the fusiform cortex contains only abstract orthographic representation, or represents both lower and higher level orthography as well as phonology. Using representational similarity analysis, we integrated behavioral ratings, computational models of reading and visual object recognition, and neuroimaging data to examine the nature of visual word representations in the fusiform cortex. Our results provided clear evidence that the middle and anterior fusiform represented both phonological and orthographic information. Whereas lower level orthographic information was represented at every stage of the ventral visual stream, abstract orthographic information was increasingly represented along the posterior-to-anterior axis. Furthermore, the left and right hemispheres were tuned to high- and low-frequency orthographic information, respectively. These results help to resolve the long-standing debates regarding the role of the fusiform in reading, and have significant implications for the development of psychological, neural, and computational theories of reading.

Localization and Functional Characterization of an Occipital Visual Word form Sensitive Area

In human occipitotemporal cortex, category-specific processing for visual objects seems to involve pairs of cortical regions, often with one located in the occipital cortex and another more anteriorly. We investigated whether such an arrangement might be the case for visual word processing. In addition to the Visual Word Form Area (VWFA) located in the occipitotemporal sulcus, we observed that another region in occipital lobe with robust responses to written words (Chinese characters). The current fMRI study investigated this area’s precise location and its functional selectivity using Chinese characters and other categories of visual images (cars, chairs and insects). In all the 13 subjects we could identify a cluster of voxels near the inferior occipital gyrus or middle occipital gyrus with stronger responses to Chinese characters than scrambled objects. We tentatively label this area as the Occipital Word Form Sensitive Area (OWA). The OWA’s response amplitudes showed similar preference to written words as the VWFA, with the VWFA showing a higher degree of word selectivity, which was confirmed by the result from spatial patterns of response. These results indicate that the OWA, together with the VWFA, are critical parts of the network for processing and representing the category information for word.

Blind readers break mirror invariance as sighted do

Mirror invariance refers to a predisposition of humans, including infants and animals, which urge them to consider mirrored images as corresponding to the same object. Yet in order to learn to read a written system that incorporates mirrored letters (e.g., <b> vs. <d> in the Latin alphabet), humans learn to break this perceptual bias. Here we examined the role visual experience and input modality play in the emergence of this bias. To this end, we tested congenital blind (CB) participants in two same-different tactile comparison tasks including pairs of mirrored and non-mirrored Braille letters as well as embossed unfamiliar geometric shapes and Latin letters, and compared their results to those of age-matched sighted participants involved in similar but visually-presented tasks. Sighted participants showed a classical pattern of results for their material of expertise, Latin letters. CB's results signed for their expertise with the Braille script compared to the other two materials that they processed according to an internal frame of reference. They also evidenced that they automatically break mirror invariance for different materials explored through the tactile modality, including Braille letters. Altogether, these results demonstrate that learning to read Braille through the tactile modality allows breaking mirror invariance in a comparable way to what is observed in sighted individuals for the mirrored letters of the Latin alphabet.

Diagnosing the Neural Circuitry of Reading

We summarize the current state of knowledge of the brain's reading circuits, and then we describe opportunities to use quantitative and reproducible methods for diagnosing these circuits. Neural circuit diagnostics-by which we mean identifying the locations and responses in an individual that differ significantly from measurements in good readers-can help parents and educators select the best remediation strategy. A sustained effort to develop and share diagnostic methods can support the societal goal of improving literacy.

Plastic reorganization of neural systems for perception of others in the congenitally blind

Recent evidence suggests that the function of the core system for face perception might extend beyond visual face-perception to a broader role in person perception. To critically test the broader role of core face-system in person perception, we examined the role of the core system during the perception of others in 7 congenitally blind individuals and 15 sighted subjects by measuring their neural responses using fMRI while they listened to voices and performed identity and emotion recognition tasks. We hypothesised that in people who have had no visual experience of faces, core face-system areas may assume a role in the perception of others via voices. Results showed that emotions conveyed by voices can be decoded in homologues of the core face system only in the blind. Moreover, there was a specific enhancement of response to verbal as compared to non-verbal stimuli in bilateral fusiform face areas and the right posterior superior temporal sulcus showing that the core system also assumes some language-related functions in the blind. These results indicate that, in individuals with no history of visual experience, areas of the core system for face perception may assume a role in aspects of voice perception that are relevant to social cognition and perception of others' emotions.

An fMRI study of visual hemifield integration and cerebral lateralization

The human brain integrates hemifield-split visual information via interhemispheric transfer. The degree to which neural circuits involved in this process behave differently during word recognition as compared to object recognition is not known. Evidence from neuroimaging (fMRI) suggests that interhemispheric transfer during word viewing converges in the left hemisphere, in two distinct brain areas, an "occipital word form area" (OWFA) and a more anterior occipitotemporal "visual word form area" (VWFA). We used a novel fMRI half-field repetition technique to test whether or not these areas also integrate nonverbal hemifield-split string stimuli of similar visual complexity. We found that the fMRI responses of both the OWFA and VWFA while viewing nonverbal stimuli were strikingly different than those measured during word viewing, especially with respect to half-stimulus changes restricted to a single hemifield. We conclude that normal reading relies on left-lateralized neural mechanisms, which integrate hemifield-split visual information for words but not for nonverbal stimuli.

Characterizing object- and position-dependent response profiles to uni- and bilateral stimulus configurations in human higher visual cortex: a 7T fMRI study

Visual scenes are initially processed via segregated neural pathways dedicated to either of the two visual hemifields. Although higher-order visual areas are generally believed to utilize invariant object representations (abstracted away from features such as stimulus position), recent findings suggest they retain more spatial information than previously thought. Here, we assessed the nature of such higher-order object representations in human cortex using high-resolution fMRI at 7T, supported by corroborative 3T data. We show that multi-voxel activation patterns in both the contra- and ipsilateral hemisphere can be exploited to successfully classify the object category of unilaterally presented stimuli. Moreover, robustly identified rank order-based response profiles demonstrated a strong contralateral bias which frequently outweighed object category preferences. Finally, we contrasted different combinatorial operations to predict the responses during bilateral stimulation conditions based on responses to their constituent unilateral elements. Results favored a max operation predominantly reflecting the contralateral stimuli. The current findings extend previous work by showing that configuration-dependent modulations in higher-order visual cortex responses as observed in single unit activity have a counterpart in human neural population coding. They furthermore corroborate the emerging view that position coding is a fundamental functional characteristic of ventral visual stream processing.

An individual differences analysis of the neurocognitive architecture of the semantic system at rest

Efficient semantic cognition depends on accessing and selecting conceptual knowledge relevant to the current task or context. This study explored the neurocognitive architecture that supports this function by examining how individual variation in functional brain organisation predicts comprehension and semantic generation. Participants underwent resting state functional magnetic resonance imaging (fMRI) and, on separate days, performed written synonym judgement, and letter and category fluency tasks. We found that better synonym judgement for high frequency items was linked to greater functional coupling between posterior fusiform and anterior superior temporal cortex (aSTG), which might index orthographic-to-semantic access. However, stronger coupling between aSTG and ventromedial prefrontal cortex was associated with poor performance on the same trials, potentially reflecting greater difficulty in focussing retrieval on relevant features for high frequency items that appear in a greater range of contexts. Fluency performance was instead linked to variations in the functional coupling of the inferior frontal gyrus (IFG); anterior IFG was more coupled to regions of primary visual cortex for individuals who were good at category fluency, while poor letter fluency was predicted by stronger coupling between posterior IFG and retrosplenial cortex. These results show that individual differences in functional connectivity at rest predict semantic performance and are consistent with a component process account of semantic cognition in which representational information is shaped by control processes to fit the current requirements, in both comprehension and fluency tasks.

Locating the cortical bottleneck for slow reading in peripheral vision

Yu, Legge, Park, Gage, and Chung (2010) suggested that the neural bottleneck for slow peripheral reading is located in nonretinotopic areas. We investigated the potential rate-limiting neural site for peripheral reading using fMRI, and contrasted peripheral reading with recognition of peripherally presented line drawings of common objects. We measured the BOLD responses to both text (three-letter words/nonwords) and line-drawing objects presented either in foveal or peripheral vision (10° lower right visual field) at three presentation rates (2, 4, and 8/second). The statistically significant interaction effect of visual field × presentation rate on the BOLD response for text but not for line drawings provides evidence for distinctive processing of peripheral text. This pattern of results was obtained in all five regions of interest (ROIs). At the early retinotopic cortical areas, the BOLD signal slightly increased with increasing presentation rate for foveal text, and remained fairly constant for peripheral text. In the Occipital Word-Responsive Area (OWRA), Visual Word Form Area (VWFA), and object sensitive areas (LO and PHA), the BOLD responses to text decreased with increasing presentation rate for peripheral but not foveal presentation. In contrast, there was no rate-dependent reduction in BOLD response for line-drawing objects in all the ROIs for either foveal or peripheral presentation. Only peripherally presented text showed a distinctive rate-dependence pattern. Although it is possible that the differentiation starts to emerge at the early retinotopic cortical representation, the neural bottleneck for slower reading of peripherally presented text may be a special property of peripheral text processing in object category selective cortex.

The Visual Word Form Area remains in the dominant hemisphere for language in late-onset left occipital lobe epilepsies: A postsurgery analysis of two cases

Automatic recognition of words from letter strings is a critical processing step in reading that is lateralized to the left-hemisphere middle fusiform gyrus in the so-called Visual Word Form Area (VWFA). Surgical lesions in this location can lead to irreversible alexia. Very early left hemispheric lesions can lead to transfer of the VWFA to the nondominant hemisphere, but it is currently unknown if this capability is preserved in epilepsies developing after reading acquisition. In this study, we aimed to determine the lateralization of the VWFA in late-onset left inferior occipital lobe epilepsies and also the effect of surgical disconnection from the adjacent secondary visual areas. Two patients with focal epilepsies with onset near the VWFA underwent to surgery for epilepsy, with sparing of this area. Neuropsychology evaluations were performed before and after surgery, as well as quantitative evaluation of the speed of word reading. Comparison of the surgical localization of the lesion, with the BOLD activation associated with the contrast of words-strings, was performed, as well as a study of the associated main white fiber pathways using diffusion-weighted imaging. Neither of the patients developed alexia after surgery (similar word reading speed before and after surgery) despite the fact that the inferior occipital surgical lesions reached the neighborhood (less than 1cm) of the VWFA. Surgeries partly disconnected the VWFA from left secondary visual areas, suggesting that pathways connecting to the posterior visual ventral stream were severely affected but did not induce alexia. The anterior and superior limits of the resection suggest that the critical connection between the VWFA and the Wernicke's Angular Gyrus cortex was not affected, which is supported by the detection of this tract with probabilistic tractography. Left occipital lobe epilepsies developing after reading acquisition did not produce atypical localizations of the VWFA, even with foci in the close neighborhood. Surgery for occipital lobe epilepsy should take this into consideration, as well as the fact that disconnection from the left secondary visual areas may not produce alexia.

Visual Similarity of Words Alone Can Modulate Hemispheric Lateralization in Visual Word Recognition: Evidence From Modeling Chinese Character Recognition

In Chinese orthography, the most common character structure consists of a semantic radical on the left and a phonetic radical on the right (SP characters); the minority, opposite arrangement also exists (PS characters). Recent studies showed that SP character processing is more left hemisphere (LH) lateralized than PS character processing. Nevertheless, it remains unclear whether this is due to phonetic radical position or character type frequency. Through computational modeling with artificial lexicons, in which we implement a theory of hemispheric asymmetry in perception but do not assume phonological processing being LH lateralized, we show that the difference in character type frequency alone is sufficient to exhibit the effect that the dominant type has a stronger LH lateralization than the minority type. This effect is due to higher visual similarity among characters in the dominant type than the minority type, demonstrating the modulation of visual similarity of words on hemispheric lateralization.

Target sites for transcallosal fibers in human visual cortex - A combined diffusion and polarized light imaging study

Transcallosal fibers of the visual system have preferential target sites within the occipital cortex of monkeys. These target sites coincide with vertical meridian representations of the visual field at borders of retinotopically defined visual areas. The existence of preferential target sites of transcallosal fibers in the human brain at the borders of early visual areas was claimed, but controversially discussed. Hence, we studied the distribution of transcallosal fibers in human visual cortex, searching for an organizational principle across early and higher visual areas. In-vivo high angular resolution diffusion imaging data of 28 subjects were used for probabilistic fiber tracking using a constrained spherical deconvolution approach. The fiber architecture within the target sites was analyzed at microscopic resolution using 3D polarized light imaging in a post-mortem human hemisphere. Fibers through a seed in the splenium of the corpus callosum reached the occipital cortex via the forceps major and the tapetum. We found target sites of these transcallosal fibers at borders of cytoarchitectonically defined occipital areas not only between early visual areas V1 and V2, V3d and V3A, and V3v and V4, but also between higher extrastriate areas, namely V4 (ventral) and posterior fusiform area FG1 as well as posterior fusiform area FG2 and lateral occipital cortex. In early visual areas, the target sites coincided with the vertical meridian representations of retinotopic maps. The spatial arrangement of the fibers in the 'border tuft' region at the V1/V2 border was found to be more complex than previously observed in myeloarchitectonic studies. In higher visual areas, our results provided additional evidence for a hemi-field representation in human area V4. The fiber topography in posterior fusiform gyrus indicated that additional retinotopic areas might exist, located between the recently identified retinotopic representations phPITv/phPITd and PHC-1/PHC-2 in lateral occipital cortex and parahippocampal gyrus.

Preserved mid-fusiform activation for visual words in a patient with a visual word recognition impairment

Previous functional imaging studies have highlighted the role of left ventral temporal cortex in processing written word forms. We explored activation and anatomical connectivity of this region in HE, a professional writer with alexia as a result of stroke affecting primarily white matter in the left inferior temporal lobe. We used a one-back visual recognition task and functional Magnetic Resonance Imaging to elicit automatic activation to various orthographic and non-orthographic stimuli. Surprisingly, HE showed cortical activation in the left mid-fusiform area during the presentation of words and word-like stimuli, suggesting that this region׳s role in processing visual words is intact despite his severely impaired reading. Diffusion Tensor Imaging data further suggest that HE shows an anatomical disconnection between the ventral temporal cortex and posterior middle temporal cortex. Together, these findings suggest that activation of word-specific regions of mid-fusiform gyrus is not sufficient to yield the conscious experience of reading in the absence of activity in downstream regions of the classical reading network.

Decoding letter position in word reading

A fundamental computation underlying visual word recognition is the ability to transform a set of letters into a visual word form. Neuropsychological data suggest that letter position within a word may be independently affected by brain damage, resulting in a dissociable subtype of peripheral dyslexia. Here we used functional magnetic resonance imaging and supervised machine learning techniques to classify letter position based on activation patterns evoked during reading Hebrew words. Across the entire brain, activity patterns in the left intraparietal sulcus provided the best classification accuracy (80%) with respect to letter position. Importantly, the same set of voxels that showed highest classification performance of letter position using one letter-of-interest also showed highest classification performance using a different letter-of-interest. A functional connectivity analysis revealed that activity in these voxels co-varied with activity in the Visual Word Form Area, confirming cross-talk between these regions during covert reading. The results converge with reports of patients with acquired letter position dyslexia, who suffer from left occipito-parietal lesions. These findings provide direct and novel evidence for the role of left IPS within the reading network in processing relative letter positions.

Neural correlates of letter reversal in children and adults

Children often make letter reversal errors when first learning to read and write, even for letters whose reversed forms do not appear in normal print. However, the brain basis of such letter reversal in children learning to read is unknown. The present study compared the neuroanatomical correlates (via functional magnetic resonance imaging) and the electrophysiological correlates (via event-related potentials or ERPs) of this phenomenon in children, ages 5-12, relative to young adults. When viewing reversed letters relative to typically oriented letters, adults exhibited widespread occipital, parietal, and temporal lobe activations, including activation in the functionally localized visual word form area (VWFA) in left occipito-temporal cortex. Adults exhibited significantly greater activation than children in all of these regions; children only exhibited such activation in a limited frontal region. Similarly, on the P1 and N170 ERP components, adults exhibited significantly greater differences between typical and reversed letters than children, who failed to exhibit significant differences between typical and reversed letters. These findings indicate that adults distinguish typical and reversed letters in the early stages of specialized brain processing of print, but that children do not recognize this distinction during the early stages of processing. Specialized brain processes responsible for early stages of letter perception that distinguish between typical and reversed letters may develop slowly and remain immature even in older children who no longer produce letter reversals in their writing.

Deep learning of orthographic representations in baboons

What is the origin of our ability to learn orthographic knowledge? We use deep convolutional networks to emulate the primate's ventral visual stream and explore the recent finding that baboons can be trained to discriminate English words from nonwords. The networks were exposed to the exact same sequence of stimuli and reinforcement signals as the baboons in the experiment, and learned to map real visual inputs (pixels) of letter strings onto binary word/nonword responses. We show that the networks' highest levels of representations were indeed sensitive to letter combinations as postulated in our previous research. The model also captured the key empirical findings, such as generalization to novel words, along with some intriguing inter-individual differences. The present work shows the merits of deep learning networks that can simulate the whole processing chain all the way from the visual input to the response while allowing researchers to analyze the complex representations that emerge during the learning process.

Wernicke's area revisited: parallel streams and word processing

Auditory word-form recognition was originally proposed by Wernicke to occur within left superior temporal gyrus (STG), later further specified to be in posterior STG. To account for clinical observations (specifically paraphasia), Wernicke proposed his sensory speech center was also essential for correcting output from frontal speech-motor regions. Recent work, in contrast, has established a role for anterior STG, part of the auditory ventral stream, in the recognition of species-specific vocalizations in nonhuman primates and word-form recognition in humans. Recent work also suggests monitoring self-produced speech and motor control are associated with posterior STG, part of the auditory dorsal stream. Working without quantitative methods or evidence of sensory cortex' hierarchical organization, Wernicke co-localized functions that today appear dissociable. "Wernicke's area" thus may be better construed as two cortical modules, an auditory word-form area (AWFA) in the auditory ventral stream and an "inner speech area" in the auditory dorsal stream.

The lazy visual word form area: computational insights into location-sensitivity

In a recent study, Rauschecker et al. convincingly demonstrate that visual words evoke neural activation signals in the Visual Word Form Area that can be classified based on where they were presented in the visual fields. This result goes against the prevailing consensus, and begs an explanation. We show that one of the simplest possible models for word recognition, a multilayer feedforward network, will exhibit precisely the same behavior when trained to recognize words at different locations. The model suggests that the VWFA initially starts with information about location, which is not being suppressed during reading acquisition more than is needed to meet the requirements of location-invariant word recognition. Some new interpretations of Rauschecker et al.'s results are proposed, and three specific predictions are derived to be tested in further studies.

Spoken word recognition without a TRACE

How do we map the rapid input of spoken language onto phonological and lexical representations over time? Attempts at psychologically-tractable computational models of spoken word recognition tend either to ignore time or to transform the temporal input into a spatial representation. TRACE, a connectionist model with broad and deep coverage of speech perception and spoken word recognition phenomena, takes the latter approach, using exclusively time-specific units at every level of representation. TRACE reduplicates featural, phonemic, and lexical inputs at every time step in a large memory trace, with rich interconnections (excitatory forward and backward connections between levels and inhibitory links within levels). As the length of the memory trace is increased, or as the phoneme and lexical inventory of the model is increased to a realistic size, this reduplication of time- (temporal position) specific units leads to a dramatic proliferation of units and connections, begging the question of whether a more efficient approach is possible. Our starting point is the observation that models of visual object recognition-including visual word recognition-have grappled with the problem of spatial invariance, and arrived at solutions other than a fully-reduplicative strategy like that of TRACE. This inspires a new model of spoken word recognition that combines time-specific phoneme representations similar to those in TRACE with higher-level representations based on string kernels: temporally independent (time invariant) diphone and lexical units. This reduces the number of necessary units and connections by several orders of magnitude relative to TRACE. Critically, we compare the new model to TRACE on a set of key phenomena, demonstrating that the new model inherits much of the behavior of TRACE and that the drastic computational savings do not come at the cost of explanatory power.