Modality-independent neural mechanisms for novel phonetic processing

The neural representations underlying asymmetric cross-modal prediction of words

Cross-modal prediction serves a crucial adaptive role in the multisensory world, yet the neural mechanisms underlying this prediction are poorly understood. The present study addressed this important question by combining a novel audiovisual sequence memory task, functional magnetic resonance imaging (fMRI), and multivariate neural representational analyses. Our behavioral results revealed a reliable asymmetric cross-modal predictive effect, with a stronger prediction from visual to auditory (VA) modality than auditory to visual (AV) modality. Mirroring the behavioral pattern, we found the superior parietal lobe (SPL) showed higher pattern similarity for VA than AV pairs, and the strength of the predictive coding in the SPL was positively correlated with the behavioral predictive effect in the VA condition. Representational connectivity analyses further revealed that the SPL mediated the neural pathway from the visual to the auditory cortex in the VA condition but was not involved in the auditory to visual cortex pathway in the AV condition. Direct neural pathways within the unimodal regions were found for the visual-to-visual and auditory-to-auditory predictions. Together, these results provide novel insights into the neural mechanisms underlying cross-modal sequence prediction.

Neural pathways of phonological and semantic processing and its relations to children's reading skills

Behavioral research shows that children's phonological ability is strongly associated with better word reading skills, whereas semantic knowledge is strongly related to better reading comprehension. However, most neuroscience research has investigated how brain activation during phonological and semantic processing is related to word reading skill. This study examines if connectivity during phonological processing in the dorsal inferior frontal gyrus (dIFG) to posterior superior temporal gyrus (pSTG) pathway is related to word reading skill, whereas connectivity during semantic processing in the ventral inferior frontal gyrus (vIFG) to posterior middle temporal gyrus (pMTG) pathway is related to reading comprehension skill. We used behavioral and functional magnetic resonance imaging (fMRI) data from a publicly accessible dataset on OpenNeuro.org. The research hypotheses and analytical plan were pre-registered on the Open Science Framework. Forty-six children ages 8-15 years old were included in the final analyses. Participants completed an in-scanner reading task tapping into phonology (i.e., word rhyming) and semantics (i.e., word meaning) as well as standardized measures of word reading and reading comprehension skill. In a series of registered and exploratory analyses, we correlated connectivity coefficients from generalized psychophysiological interactions (gPPI) with behavioral measures and used z-scores to test the equality of two correlation coefficients. Results from the preregistered and exploratory analyses indicated weak evidence that functional connectivity of dIFG to pSTG during phonological processing is positively correlated with better word reading skill, but no evidence that connectivity in the vIFG-pMTG pathway during semantic processing is related to better reading comprehension skill. Moreover, there was no evidence to support the differentiation between the dorsal pathway's relation to word reading and the ventral pathway's relation to reading comprehension skills. Our finding suggesting the importance of phonological processing to word reading is in line with prior behavioral and neurodevelopmental models.

Measuring Receptive ASL Skills in Novice Signers and Nonsigners

We tested the utility of two standardized measures of receptive skills in American Sign Language (ASL) in hearing adults who are novice signers: the ASL Comprehension Test (ASL-CT; Hauser, P. C., Paludneviciene, R., Riddle, W., Kurz, K. B., Emmorey, K., & Contreras, J. (2016). American Sign Language Comprehension Test: A tool for sign language researchers. Journal of Deaf Studies and Deaf Education, 21(1), 64-69. https://doi.org/10.1093/deafed/env051) and the ASL Receptive Skills Test (ASL-RST; Allen, T. E., & Enns, C. (2013). A psychometric study of the ASL Receptive Skills Test when administered to deaf 3-, 4-, and 5-year-old children. Sign Language Studies, 14(1), 58-79. https://doi.org/10.1353/sls.2015.0006; Enns, C. J., & Herman, R. C. (2011). Adapting the Assessing British Sign Language Development: Receptive Skills Test into American Sign Language. Journal of Deaf Studies and Deaf Education, 16(3), 362-374. https://doi.org/10.1093/deafed/enr004). We also administered these tests to hearing nonsigners, with the goal of identifying items where the correct answer can be readily deduced through iconicity and/or test-taking skills, even by people with no prior ASL knowledge. We predicted that removing such items would strengthen the correlation between test score and ASL experience (as measured by semesters of instruction). We found that this relationship was stronger for the ASL-CT than the ASL-RST in this population, but still weaker than previously reported, and not substantially improved by any item selection strategy that we employed.

Association Between Resting-State Functional Connectivity and Reading in Two Writing Systems in Japanese Children With and Without Developmental Dyslexia

Introduction: Japanese is unique, as it features two distinct writing systems that share the same sound and meaning: syllabic Hiragana and logographic Kanji scripts. Acquired reading difficulties in Hiragana and Kanji have been examined in older patients with brain lesions. However, the precise mechanisms underlying deficits in developmental dyslexia (DD) remain unclear.

Materials and Methods: The neural signatures of Japanese children with DD were examined by using resting-state functional magnetic resonance imaging. We examined 22 dyslexic and 46 typically developing (TD) children, aged 7–14 years.

Results: Reading performance in each writing system was correlated with neural connectivity in TD children. In contrast, in children with DD, weak associations between neural connectivity and reading performance were observed. In TD children, Hiragana-reading fluency was positively correlated with the left fusiform gyrus network. No significant correlations between Hiragana fluency and neural connectivity were observed in children with DD. Correspondingly, there were fewer correlations between Kanji accuracy and strength of reading-related connectivity in children with DD, whereas positive correlations with the bilateral fronto-parietal network and negative correlations with the left fusiform network were found in TD children.

Discussion: These data suggest that positive and negative coupling with neural connectivity is associated with developing Japanese reading skills. Further, different neural connectivity correlations between Hiragana fluency and Kanji accuracy were detected in TD children but less in children with DD.

Conclusion: The two writing systems may exert differential effects and deficits on reading in healthy children and in children with DD, respectively.

Individual differences in lexical learning across two language modalities: Sign learning, word learning, and their relationship in hearing non-signing adults

A considerable amount of research has been devoted to understanding individual differences in lexical learning, however, the majority of this research has been conducted with spoken languages rather than signed languages and thus we know very little about the cognitive processes involved in sign learning or the extent to which lexical learning processes are specific to word learning. The present study was conducted to address this gap. Two-hundred thirty-six non-signing adults completed 25 tasks assessing word learning and sign learning (via associative learning paradigms) as well as modality-specific phonological short-term memory, working memory capacity, crystallized intelligence, and fluid intelligence. Latent variable analyses indicated that, when other variables were held constant, fluid intelligence was predictive of both word and sign learning, however, modality-specific phonological short-term memory factors were only predictive of lexical learning within modality-none of the other variables made significant independent contributions. It was further observed that sign and word learning were strongly correlated. Exploratory analyses revealed that all lexical learning tasks loaded onto a general factor, however, sign learning tasks loaded onto an additional specific factor. As such, this study provides insight into the cognitive components that are common to associative L2 lexical learning regardless of language modality and those that are unique to either signed or spoken languages. Results are further discussed in light of established and more recent theories of intelligence, short-term memory, and working memory.

Visual cortex entrains to sign language

Despite immense variability across languages, people can learn to understand any human language, spoken or signed. What neural mechanisms allow people to comprehend language across sensory modalities? When people listen to speech, electrophysiological oscillations in auditory cortex entrain to slow ([Formula: see text]8 Hz) fluctuations in the acoustic envelope. Entrainment to the speech envelope may reflect mechanisms specialized for auditory perception. Alternatively, flexible entrainment may be a general-purpose cortical mechanism that optimizes sensitivity to rhythmic information regardless of modality. Here, we test these proposals by examining cortical coherence to visual information in sign language. First, we develop a metric to quantify visual change over time. We find quasiperiodic fluctuations in sign language, characterized by lower frequencies than fluctuations in speech. Next, we test for entrainment of neural oscillations to visual change in sign language, using electroencephalography (EEG) in fluent speakers of American Sign Language (ASL) as they watch videos in ASL. We find significant cortical entrainment to visual oscillations in sign language <5 Hz, peaking at [Formula: see text]1 Hz. Coherence to sign is strongest over occipital and parietal cortex, in contrast to speech, where coherence is strongest over the auditory cortex. Nonsigners also show coherence to sign language, but entrainment at frontal sites is reduced relative to fluent signers. These results demonstrate that flexible cortical entrainment to language does not depend on neural processes that are specific to auditory speech perception. Low-frequency oscillatory entrainment may reflect a general cortical mechanism that maximizes sensitivity to informational peaks in time-varying signals.