Orthographic and Phonological Representations in the Fusiform Cortex

The neural representations underlying asymmetric cross-modal prediction of words

Cross-modal prediction serves a crucial adaptive role in the multisensory world, yet the neural mechanisms underlying this prediction are poorly understood. The present study addressed this important question by combining a novel audiovisual sequence memory task, functional magnetic resonance imaging (fMRI), and multivariate neural representational analyses. Our behavioral results revealed a reliable asymmetric cross-modal predictive effect, with a stronger prediction from visual to auditory (VA) modality than auditory to visual (AV) modality. Mirroring the behavioral pattern, we found the superior parietal lobe (SPL) showed higher pattern similarity for VA than AV pairs, and the strength of the predictive coding in the SPL was positively correlated with the behavioral predictive effect in the VA condition. Representational connectivity analyses further revealed that the SPL mediated the neural pathway from the visual to the auditory cortex in the VA condition but was not involved in the auditory to visual cortex pathway in the AV condition. Direct neural pathways within the unimodal regions were found for the visual-to-visual and auditory-to-auditory predictions. Together, these results provide novel insights into the neural mechanisms underlying cross-modal sequence prediction.

The depth of semantic processing modulates cross-language pattern similarity in Chinese-English bilinguals

Previous studies have investigated factors related to the degree of cross-language overlap in brain activations in bilinguals/multilinguals. However, it is still unclear whether and how the depth of semantic processing (a critical task-related factor) affects the neural pattern similarity between native and second languages. To address this question, 26 Chinese-English bilinguals were scanned with fMRI while performing a word naming task (i.e., a task with shallow semantic processing) and a semantic judgment task (i.e., a task with deep semantic processing) in both native and second languages. Based on three sets of representational similarity analysis (whole brain, ROI-based, and within-language vs. cross-language semantic representation), we found that select regions in the reading brain network showed higher cross-language pattern similarity and higher cross-language semantic representations during deep semantic processing than during shallow semantic processing. These results suggest that compared to shallow semantic processing, deep semantic processing may lead to greater language-independent processing (i.e., cross-language semantic representation) and cross-language pattern similarity, and provide direct quantitative neuroimaging evidence for cognitive models of bilingual lexical memory.

Individuals' preference on reading pathways influences the involvement of neural pathways in phonological learning

Introduction

Existing behavioral and neuroimaging studies revealed inter-individual variability in the selection of the two phonological routes in word reading. However, it is not clear how individuals' preferred reading pathways/strategies modulate the involvement of a certain brain region for phonological learning in a new language, and consequently affect their behavioral performance on phonological access.

Methods

To address this question, the present study recruited a group of native Chinese speakers to learn two sets of artificial language characters, respectively, in addressed-phonology training (i.e., whole-word mapping) and assembled-phonology training conditions (i.e., grapheme-to-phoneme mapping).

Results

Behavioral results showed that the more lexical pathways participants preferred, the better they performed on newly-acquired addressed characters relative to assembled characters. More importantly, neuroimaging results showed that participants who preferred lexical pathway in phonological access show less involvement of brain regions for addressed phonology (e.g., the bilateral orbitofrontal cortex and right pars triangularis) in the processing of newly-acquired addressed characters.

Conclusion

These results indicated that phonological access via the preferred pathway required less neural resources to achieve better behavioral performance. These above results provide direct neuroimaging evidence for the influence of reading pathway preference on phonological learning.

Reduced pattern similarity in brain activation during orthographic processing in children with developmental dyslexia

Intra-individual variability of neural response has been found to be negatively associated with cognitive proficiency and automaticity. However, whether developmental dyslexia (DD) is marked by greater intra-individual neural variability remains unclear. Using a multivariate approach and dual-control group design, the current study aims to examine whether the pattern similarity of brain activation during a visual spelling task is abnormal in children with DD compared to age control and reading control children. We found that there was reduced intra-subject pattern similarity at the left occipito-temporal regions in children with DD than both control groups, suggesting a neural signature of DD. Furthermore, we found that pattern similarity was positively associated with stability of reaction time and reading fluency in both children with DD and typical control children, suggesting that neural stability supports behavioral stability and automaticity during reading.

Neural representation of phonological information during Chinese character reading

Previous studies have revealed that phonological processing of Chinese characters elicited activation in the left prefrontal cortex, bilateral parietal cortex, and occipitotemporal regions. However, it is controversial what role the left middle frontal gyrus plays in Chinese character reading, and whether the core regions (e.g., the left superior temporal gyrus and supramarginal gyrus) for phonological processing of alphabetic languages are also involved in Chinese character reading. To address these questions, the present study used both univariate and multivariate analysis (i.e., representational similarity analysis, RSA) to explore neural representations of phonological information during Chinese character reading. Participants were scanned while performing a reading aloud task. Univariate activation analysis revealed a widely distributed network for word reading, including the bilateral inferior frontal gyrus, middle frontal gyrus, lateral temporal cortex, and occipitotemporal cortex. More importantly, RSA showed that the left prefrontal (i.e., the left middle frontal gyrus and left inferior frontal gyrus) and bilateral occipitotemporal areas (i.e., the left inferior and middle temporal gyrus and bilateral fusiform gyrus) represented phonological information of Chinese characters. These results confirmed the importance of the left middle frontal gyrus and regions in ventral pathway in representing phonological information of Chinese characters.

Task modulates the orthographic and phonological representations in the bilateral ventral Occipitotemporal cortex

As a key area in word reading, the left ventral occipitotemporal cortex is proposed for abstract orthographic processing, and its middle part has even been labeled as the visual word form area. Because the definition of the VWFA largely varies and the reading task differs across studies, the function of the left ventral occipitotemporal cortex in word reading is continuingly debated on whether this region is specific for orthographic processing or be involved in an interactive framework. By using representational similarity analysis (RSA), this study examined information representation in the VWFA at the individual level and the modulatory effect of reading task. Twenty-four subjects were scanned while performing the explicit (i.e., the naming task) and implicit (i.e., the perceptual task) reading tasks. Activation analysis showed that the naming task elicited greater activation in regions related to phonological processing (e.g., the bilateral prefrontal cortex and temporoparietal cortex), while the perceptual task recruited greater activation in visual cortex and default mode network (e.g., the bilateral middle frontal gyrus, angular gyrus, and the right middle temporal gyrus). More importantly, RSA also showed that task modulated information representation in the bilateral anterior occipitotemporal cortex and VWFA. Specifically, ROI-based RSA revealed enhanced orthographic and phonological representations in the bilateral anterior fusiform cortex and VWFA in the naming task relative to the perceptual task. These results suggest that lexical representation in the VWFA is influenced by the demand of phonological processing, which supports the interactive account of the VWFA.

Brain responses during auditory word recognition vary with reading ability in Chinese school-age children

While the close relationship between the brain system for speech processing and reading development is well-documented in alphabetic languages, whether and how such a link exists in children in a language without systematic grapheme-phoneme correspondence has not been directly investigated. In the present study, we measured Chinese children's brain activation during an auditory lexical decision task with functional magnetic resonance imaging. The results showed that brain areas distributed across the temporal and frontal lobes activated during spoken word recognition. In addition, the left occipitotemporal cortex (OTC) was recruited, especially under the real word condition, thus confirming the involvement of this orthographic-related area in spoken language processing in Chinese children. Importantly, activation of the left temporoparietal cortex (TPC) in response to words and pseudowords was positively correlated with children's reading ability, thus supporting the salient role phonological processing plays in Chinese reading in the developing brain. Furthermore, children with higher reading scores also increasingly recruited the left anterior OTC to make decisions on the lexical status of pseudowords, indicating that higher-skill children tend to search abstract lexical representations more deeply than lower-skill children in deciding whether spoken syllables are real. In contrast, the precuneus was more related to trial-by-trial reaction time in lower-skill children, suggesting that effort-related neural systems differ among pupils with varying reading abilities. Taken together, these findings suggest a strong link between the neural correlates of speech processing and reading ability in Chinese children, thus supporting a universal basis underlying reading development across languages.

Neural representations of the amount and the delay time of reward in intertemporal decision making

Numerous studies have examined the neural substrates of intertemporal decision-making, but few have systematically investigated separate neural representations of the two attributes of future rewards (i.e., the amount of the reward and the delay time). More importantly, no study has used the novel analytical method of representational connectivity analysis (RCA) to map the two dimensions' functional brain networks at the level of multivariate neural representations. This study independently manipulated the amount and delay time of rewards during an intertemporal decision task. Both univariate and multivariate pattern analyses showed that brain activity in the dorsomedial prefrontal cortex (DMPFC) and lateral frontal pole cortex (LFPC) was modulated by the amount of rewards, whereas brain activity in the DMPFC and dorsolateral prefrontal cortex (DLPFC) was modulated by the length of delay. Moreover, representational similarity analysis (RSA) revealed that even for the regions of the DMPFC that overlapped between the two dimensions, they manifested distinct neural activity patterns. In terms of individual differences, those with large delay discounting rates (k) showed greater DMPFC and LFPC activity as the amount of rewards increased but showed lower DMPFC and DLPFC activity as the delay time increased. Lastly, RCA suggested that the topological metrics (i.e., global and local efficiency) of the functional connectome subserving the delay time dimension inversely predicted individual discounting rate. These findings provide novel insights into neural representations of the two attributes in intertemporal decisions, and offer a new approach to construct task-based functional brain networks whose topological properties are related to impulsivity.

A Heteromodal Word-Meaning Binding Site in the Visual Word Form Area under Top-Down Frontoparietal Control

The integral capacity of human language together with semantic memory drives the linkage of words and their meaning, which theoretically is subject to cognitive control. However, it remains unknown whether, across different language modalities and input/output formats, there is a shared system in the human brain for word-meaning binding and how this system interacts with cognitive control. Here, we conducted a functional magnetic resonance imaging experiment based on a large cohort of subjects (50 females, 50 males) to comprehensively measure the brain responses evoked by semantic processing in spoken and written word comprehension and production tasks (listening, speaking, reading, and writing). We found that heteromodal word input and output tasks involved distributed brain regions within a frontal-parietal-temporal network and focally coactivated the anterior lateral visual word form area (VWFA), which is located in the basal occipitotemporal area. Directed connectivity analysis revealed that the VWFA was invariably under significant top-down modulation of the frontoparietal control network and interacts with regions related to attention and semantic representation. This study reveals that the VWFA is a key site subserving general semantic processes linking words and meaning, challenging the predominant emphasis on this area's specific role in reading or more general visual processes. Our findings also suggest that the dynamics between semantic memory and cognitive control mechanisms during word processing are largely independent of the modalities of input or output.SIGNIFICANCE STATEMENT Binding words and their meaning into a coherent whole during retrieval requires accessing semantic memory and cognitive control, allowing our thoughts to be expressed and comprehended through mind-external tokens in multiple modalities, such as written or spoken forms. However, it is still unknown whether multimodal language comprehension and production share a common word-meaning binding system in human brains and how this system is connected to a cognitive control mechanism. By systematically measuring brain activity evoked by spoken and written verbal input and output tasks tagging word-meaning binding processes, we demonstrate a general word-meaning binding site within the visual word form area (VWFA) and how this site is modulated by the frontal-parietal control network.

Shared Neural Substrates Underlying Reading and Visual Matching: A Longitudinal Investigation

The role of visual skills in reading acquisition has long been debated and whether there is shared neurobiological basis between visual skills and reading is not clear. This study investigated the relationship between Visual Matching and reading and their shared neuroanatomical basis. Two hundred and ninety-three typically developing Mandarin-speaking children were followed in a longitudinal study from ages 4 to 11 years old. A subsample of 79 children was further followed up at 14 years old when the MRI data were collected. Results showed that the development of Visual Matching from ages 6 to 8 predicted reading accuracy at age 11. In addition, both the development of Visual Matching and reading accuracy were associated with cortical surface area of a cluster located in fusiform gyrus. These findings suggested that the mapping from visual codes to phonological codes is important in learning to read and that left fusiform gyrus provided neural basis for such mapping. Implications of these findings in light of a new approach toward the neurocognitive mechanisms underlying reading development are discussed.

Neural Components of Reading Revealed by Distributed and Symbolic Computational Models

Determining how the cognitive components of reading - orthographic, phonological, and semantic representations - are instantiated in the brain has been a longstanding goal of psychology and human cognitive neuroscience. The two most prominent computational models of reading instantiate different cognitive processes, implying different neural processes. Artificial neural network (ANN) models of reading posit non-symbolic, distributed representations. The dual-route cascaded (DRC) model instead suggests two routes of processing, one representing symbolic rules of spelling-sound correspondence, the other representing orthographic and phonological lexicons. These models are not adjudicated by behavioral data and have never before been directly compared in terms of neural plausibility. We used representational similarity analysis to compare the predictions of these models to neural data from participants reading aloud. Both the ANN and DRC model representations corresponded with neural activity. However, ANN model representations correlated to more reading-relevant areas of cortex. When contributions from the DRC model were statistically controlled, partial correlations revealed that the ANN model accounted for significant variance in the neural data. The opposite analysis, examining the variance explained by the DRC model with contributions from the ANN model factored out, revealed no correspondence to neural activity. Our results suggest that ANNs trained using distributed representations provide a better correspondence between cognitive and neural coding. Additionally, this framework provides a principled approach for comparing computational models of cognitive function to gain insight into neural representations.

Mapping visual symbols onto spoken language along the ventral visual stream

Learning to read is the most important milestone in a child’s education. However, controversies remain regarding how readers’ brains transform written words into sounds and meanings. We address these by combining artificial language learning with neuroimaging to reveal how the brain represents written words. Participants learned to read new words written in 2 different alphabets. Following 2 wk of training, we found a hierarchy of brain areas that support reading. Letter position is represented more flexibly from lower to higher visual regions. Furthermore, higher visual regions encode information about word sounds and meanings. These findings advance our understanding of how the brain comprehends language from arbitrary visual symbols.

Reading involves transforming arbitrary visual symbols into sounds and meanings. This study interrogated the neural representations in ventral occipitotemporal cortex (vOT) that support this transformation process. Twenty-four adults learned to read 2 sets of 24 novel words that shared phonemes and semantic categories but were written in different artificial orthographies. Following 2 wk of training, participants read the trained words while neural activity was measured with functional MRI. Representational similarity analysis on item pairs from the same orthography revealed that right vOT and posterior regions of left vOT were sensitive to basic visual similarity. Left vOT encoded letter identity and representations became more invariant to position along a posterior-to-anterior hierarchy. Item pairs that shared sounds or meanings, but were written in different orthographies with no letters in common, evoked similar neural patterns in anterior left vOT. These results reveal a hierarchical, posterior-to-anterior gradient in vOT, in which representations of letters become increasingly invariant to position and are transformed to convey spoken language information.

Neurobiological signatures of L2 proficiency: Evidence from a bi-directional cross-linguistic study

Recent evidence has shown that convergence of print and speech processing across a network of primarily left-hemisphere regions of the brain is a predictor of future reading skills in children, and a marker of fluent reading ability in adults. The present study extends these findings into the domain of second-language (L2) literacy, through brain imaging data of English and Hebrew L2 learners. Participants received an fMRI brain scan, while performing a semantic judgement task on spoken and written words and pseudowords in both their L1 and L2, alongside a battery of L1 and L2 behavioural measures. Imaging results show, overall, show a similar network of activation for reading across the two languages, alongside significant convergence of print and speech processing across a network of left-hemisphere regions in both L1 and L2 and in both cohorts. Importantly, convergence is greater for L1 in occipito-temporal regions tied to automatic skilled reading processes including the visual word-form area, but greater for L2 in frontal regions of the reading network, tied to more effortful, active processing. The main groupwise brain effects tell a similar story, with greater L2 than L1 activation across frontal, temporal and parietal regions, but greater L1 than L2 activation in parieto-occipital regions tied to automatic mapping processes in skilled reading. These results provide evidence for the shifting of the reading networks towards more automatic processing as reading proficiency rises and the mappings and statistics of the new orthography are learned and incorporated into the reading system.

Individual Differences in the Neural and Cognitive Mechanisms of Single Word Reading

Written language is a human invention that our brains did not evolve for. Yet, most research has focused on finding a single theory of reading, identifying the common set of cognitive and neural processes shared across individuals, neglecting individual differences. In contrast, we investigated variation in single word reading. Using a novel statistical method for analyzing heterogeneity in multi-subject task-based functional magnetic resonance imaging (fMRI), we clustered readers based on their brain's response to written stimuli. Separate behavioral testing and neuroimaging analysis shows that these clusters differed in the role of the sublexical pathway in processing written language, but not in reading skill. Taken together, these results suggest that individuals vary in the cognitive and neural mechanisms involved in word reading. In general, neurocognitive theories need to account not only for what tends to be true of the population, but also the types of variation that exist, even within a neurotypical population.

Representational similarity analysis reveals task-dependent semantic influence of the visual word form area

Access to semantic information of visual word forms is a key component of reading comprehension. In this study, we examined the involvement of the visual word form area (VWFA) in this process by investigating whether and how the activity patterns of the VWFA are influenced by semantic information during semantic tasks. We asked participants to perform two semantic tasks - taxonomic or thematic categorization - on visual words while obtaining the blood-oxygen-level-dependent (BOLD) fMRI responses to each word. Representational similarity analysis with four types of semantic relations (taxonomic, thematic, subjective semantic rating and word2vec) revealed that neural activity patterns of the VWFA were associated with taxonomic information only in the taxonomic task, with thematic information only in the thematic task and with the composite semantic information measured by word2vec in both semantic tasks. Furthermore, the semantic information in the VWFA cannot be explained by confounding factors including orthographic, low-level visual and phonological information. These findings provide positive evidence for the presence of both orthographic and task-relevant semantic information in the VWFA and have significant implications for the neurobiological basis of reading.

Neurofunctionally dissecting the reading system in children

The reading system can be broken down into four basic subcomponents in charge of prelexical, orthographic, phonological, and lexico-semantic processes. These processes need to jointly work together to become a fluent and efficient reader. Using functional magnetic resonance imaging (fMRI), we systematically analyzed differences in neural activation patterns of these four basic subcomponents in children (N=41, 9-13 years) using tasks specifically tapping each component (letter identification, orthographic decision, phonological decision, and semantic categorization). Regions of interest (ROI) were selected based on a meta-analysis of child reading and included the left ventral occipito-temporal cortex (vOT), left posterior parietal cortex (PPC), left inferior frontal gyrus (IFG), and bilateral supplementary motor area (SMA). Compared to a visual baseline task, enhanced activation in vOT and IFG was observed for all tasks with very little differences between tasks. Activity in the dorsal PPC system was confined to prelexical and phonological processing. Activity in the SMA was found in orthographic, phonological, and lexico-semantic tasks. Our results are consistent with the idea of an early engagement of the vOT accompanied by executive control functions in the frontal system, including the bilateral SMA.

Neurobiological bases of reading disorder Part I: Etiological investigations

While many studies have focused on identifying the neural and behavioral characteristics of decoding-based reading disorder (RD, aka developmental dyslexia), the etiology of RD remains largely unknown and understudied. Because the brain plays an intermediate role between genetic factors and behavioral outcomes, it is promising to address causality from a neural perspective. In the current, Part I of the two-part review, we discuss neuroimaging approaches to addressing the causality issue and review the results of studies that have employed these approaches. We assume that if a neural signature were associated with RD etiology, it would (a) manifest across comparisons in different languages, (b) be experience independent and appear in comparisons between RD and reading-matched controls, (c) be present both pre- and post-intervention, (d) be found in at-risk, pre-reading children and (e) be associated with genetic risk. We discuss each of these five characteristics in turn and summarize the studies that have examined each of them. The available literature provides evidence that anomalies in left temporo-parietal cortex, and possibly occipito-temporal cortex, may be closely related to the etiology of RD. Improved understanding of the etiology of RD can help improve the accuracy of early detection and enable targeted intervention of cognitive processes that are amenable to change, leading to improved outcomes in at-risk or affected populations.