Signed words in the congenitally deaf evoke typical late lexicosemantic responses with no early visual responses in left superior temporal cortex

Mapping the unique neural engagement in deaf individuals during picture, word, and sign language processing: fMRI study

Employing functional magnetic resonance imaging (fMRI) techniques, we conducted a comprehensive analysis of neural responses during sign language, picture, and word processing tasks in a cohort of 35 deaf participants and contrasted these responses with those of 35 hearing counterparts. Our voxel-based analysis unveiled distinct patterns of brain activation during language processing tasks. Deaf individuals exhibited robust bilateral activation in the superior temporal regions during sign language processing, signifying the profound neural adaptations associated with sign comprehension. Similarly, during picture processing, the deaf cohort displayed activation in the right angular, right calcarine, right middle temporal, and left angular gyrus regions, elucidating the neural dynamics engaged in visual processing tasks. Intriguingly, during word processing, the deaf group engaged the right insula and right fusiform gyrus, suggesting compensatory mechanisms at play during linguistic tasks. Notably, the control group failed to manifest additional or distinctive regions in any of the tasks when compared to the deaf cohort, underscoring the unique neural signatures within the deaf population. Multivariate Pattern Analysis (MVPA) of functional connectivity provided a more nuanced perspective on connectivity patterns across tasks. Deaf participants exhibited significant activation in a myriad of brain regions, including bilateral planum temporale (PT), postcentral gyrus, insula, and inferior frontal regions, among others. These findings underscore the intricate neural adaptations in response to auditory deprivation. Seed-based connectivity analysis, utilizing the PT as a seed region, revealed unique connectivity pattern across tasks. These connectivity dynamics provide valuable insights into the neural interplay associated with cross-modal plasticity.

Resting-state functional connectivity in deaf and hearing individuals and its link to executive processing

Sensory experience shapes brain structure and function, and it is likely to influence the organisation of functional networks of the brain, including those involved in cognitive processing. Here we investigated the influence of early deafness on the organisation of resting-state networks of the brain and its relation to executive processing. We compared resting-state connectivity between deaf and hearing individuals across 18 functional networks and 400 ROIs. Our results showed significant group differences in connectivity between seeds of the auditory network and most large-scale networks of the brain, in particular the somatomotor and salience/ventral attention networks. When we investigated group differences in resting-state fMRI and their link to behavioural performance in executive function tasks (working memory, inhibition and switching), differences between groups were found in the connectivity of association networks of the brain, such as the salience/ventral attention and default-mode networks. These findings indicate that sensory experience influences not only the organisation of sensory networks, but that it also has a measurable impact on the organisation of association networks supporting cognitive processing. Overall, our findings suggest that different developmental pathways and functional organisation can support executive processing in the adult brain.

Restricted language access during childhood affects adult brain structure in selective language regions

Due to the ubiquitous nature of language in the environment of infants, how it affects the anatomical structure of the brain language system over the lifespan is not well understood. In this study, we investigated the effects of early language experience on the adult brain by examining anatomical features of individuals born deaf with typical or restricted language experience in early childhood. Twenty-two deaf adults whose primary language was American Sign Language and were first immersed in it at ages ranging from birth to 14 y participated. The control group was 21 hearing non-signers. We acquired T1-weighted magnetic resonance images and used FreeSurfer [B. Fischl, Neuroimage 62, 774-781(2012)] to reconstruct the brain surface. Using an a priori regions of interest (ROI) approach, we identified 17 language and 19 somatomotor ROIs in each hemisphere from the Human Connectome Project parcellation map [M. F. Glasser et al., Nature 536, 171-178 (2016)]. Restricted language experience in early childhood was associated with negative changes in adjusted grey matter volume and/or cortical thickness in bilateral fronto-temporal regions. No evidence of anatomical differences was observed in any of these regions when deaf signers with infant sign language experience were compared with hearing speakers with infant spoken language experience, showing that the effects of early language experience on the brain language system are supramodal.

The Cortical Organization of Syntactic Processing Is Supramodal: Evidence from American Sign Language

Areas within the left-lateralized neural network for language have been found to be sensitive to syntactic complexity in spoken and written language. Previous research has revealed that these areas are active for sign language as well, but whether these areas are specifically responsive to syntactic complexity in sign language independent of lexical processing has yet to be found. To investigate the question, we used fMRI to neuroimage deaf native signers' comprehension of 180 sign strings in American Sign Language (ASL) with a picture-probe recognition task. The ASL strings were all six signs in length but varied at three levels of syntactic complexity: sign lists, two-word sentences, and complex sentences. Syntactic complexity significantly affected comprehension and memory, both behaviorally and neurally, by facilitating accuracy and response time on the picture-probe recognition task and eliciting a left lateralized activation response pattern in anterior and posterior superior temporal sulcus (aSTS and pSTS). Minimal or absent syntactic structure reduced picture-probe recognition and elicited activation in bilateral pSTS and occipital-temporal cortex. These results provide evidence from a sign language, ASL, that the combinatorial processing of anterior STS and pSTS is supramodal in nature. The results further suggest that the neurolinguistic processing of ASL is characterized by overlapping and separable neural systems for syntactic and lexical processing.

New Perspectives on the Neurobiology of Sign Languages

The first 40 years of research on the neurobiology of sign languages (1960-2000) established that the same key left hemisphere brain regions support both signed and spoken languages, based primarily on evidence from signers with brain injury and at the end of the 20th century, based on evidence from emerging functional neuroimaging technologies (positron emission tomography and fMRI). Building on this earlier work, this review focuses on what we have learned about the neurobiology of sign languages in the last 15-20 years, what controversies remain unresolved, and directions for future research. Production and comprehension processes are addressed separately in order to capture whether and how output and input differences between sign and speech impact the neural substrates supporting language. In addition, the review includes aspects of language that are unique to sign languages, such as pervasive lexical iconicity, fingerspelling, linguistic facial expressions, and depictive classifier constructions. Summary sketches of the neural networks supporting sign language production and comprehension are provided with the hope that these will inspire future research as we begin to develop a more complete neurobiological model of sign language processing.

Language Without Speech: Segregating Distinct Circuits in the Human Brain

Language is a fundamental part of human cognition. The question of whether language is processed independently of speech, however, is still heavily discussed. The absence of speech in deaf signers offers the opportunity to disentangle language from speech in the human brain. Using probabilistic tractography, we compared brain structural connectivity of adult deaf signers who had learned sign language early in life to that of matched hearing controls. Quantitative comparison of the connectivity profiles revealed that the core language tracts did not differ between signers and controls, confirming that language is independent of speech. In contrast, pathways involved in the production and perception of speech displayed lower connectivity in deaf signers compared to hearing controls. These differences were located in tracts towards the left pre-supplementary motor area and the thalamus when seeding in Broca's area, and in ipsilateral parietal areas and the precuneus with seeds in left posterior temporal regions. Furthermore, the interhemispheric connectivity between the auditory cortices was lower in the deaf than in the hearing group, underlining the importance of the transcallosal connection for early auditory processes. The present results provide evidence for a functional segregation of the neural pathways for language and speech.

Numerical Magnitude Processing in Deaf Adolescents and Its Contribution to Arithmetical Ability

Although most deaf individuals could use sign language or sign/spoken language mix, hearing loss would still affect their language acquisition. Compensatory plasticity holds that the lack of auditory stimulation experienced by deaf individuals, such as congenital deafness, can be met by enhancements in visual cognition. And the studies of hearing individuals have showed that visual form perception is the cognitive mechanism that could explain the association between numerical magnitude processing and arithmetic computation. Therefore, we examined numerical magnitude processing and its contribution to arithmetical ability in deaf adolescents, and explored the differences between the congenital and acquired deafness. 112 deaf adolescents (58 congenital deafness) and 58 hearing adolescents performed a series of cognitive and mathematical tests, and it was found there was no significant differences between the congenital group and the hearing group, but congenital group outperformed acquired group in numerical magnitude processing (reaction time) and arithmetic computation. It was also found there was a close association between numerical magnitude processing and arithmetic computation in all deaf adolescents, and after controlling for the demographic variables (age, gender, onset of hearing loss) and general cognitive abilities (non-verbal IQ, processing speed, reading comprehension), numerical magnitude processing could predict arithmetic computation in all deaf adolescents but not in congenital group. The role of numerical magnitude processing (symbolic and non-symbolic) in deaf adolescents' mathematical performance should be paid attention in the training of arithmetical ability.

Lexical Recognition in Deaf Children Learning American Sign Language: Activation of Semantic and Phonological Features of Signs

Children learning language efficiently process single words, and activate semantic, phonological, and other features of words during recognition. We investigated lexical recognition in deaf children acquiring American Sign Language (ASL) to determine how perceiving language in the visual-spatial modality affects lexical recognition. Twenty native- or early-exposed signing deaf children (ages 4 to 8 years) participated in a visual world eye-tracking study. Children were presented with a single ASL sign, target picture, and three competitor pictures that varied in their phonological and semantic relationship to the target. Children shifted gaze to the target picture shortly after sign offset. Children showed robust evidence for activation of semantic but not phonological features of signs, however in their behavioral responses children were most susceptible to phonological competitors. Results demonstrate that single word recognition in ASL is largely parallel to spoken language recognition among children who are developing a mature lexicon.

Cortical Encoding of Manual Articulatory and Linguistic Features in American Sign Language

The fluent production of a signed language requires exquisite coordination of sensory, motor, and cognitive processes. Similar to speech production, language produced with the hands by fluent signers appears effortless but reflects the precise coordination of both large-scale and local cortical networks. The organization and representational structure of sensorimotor features underlying sign language phonology in these networks remains unknown. Here, we present a unique case study of high-density electrocorticography (ECoG) recordings from the cortical surface of profoundly deaf signer during awake craniotomy. While neural activity was recorded from sensorimotor cortex, the participant produced a large variety of movements in linguistic and transitional movement contexts. We found that at both single electrode and neural population levels, high-gamma activity reflected tuning for particular hand, arm, and face movements, which were organized along dimensions that are relevant for phonology in sign language. Decoding of manual articulatory features revealed a clear functional organization and population dynamics for these highly practiced movements. Furthermore, neural activity clearly differentiated linguistic and transitional movements, demonstrating encoding of language-relevant articulatory features. These results provide a novel and unique view of the fine-scale dynamics of complex and meaningful sensorimotor actions.

Syntax-Sensitive Regions of the Posterior Inferior Frontal Gyrus and the Posterior Temporal Lobe Are Differentially Recruited by Production and Perception

Matchin and Hickok (2020) proposed that the left posterior inferior frontal gyrus (PIFG) and the left posterior temporal lobe (PTL) both play a role in syntactic processing, broadly construed, attributing distinct functions to these regions with respect to production and perception. Consistent with this hypothesis, functional dissociations between these regions have been demonstrated with respect to lesion-symptom mapping in aphasia. However, neuroimaging studies of syntactic comprehension typically show similar activations in these regions. In order to identify whether these regions show distinct activation patterns with respect to syntactic perception and production, we performed an fMRI study contrasting the subvocal articulation and perception of structured jabberwocky phrases (syntactic), sequences of real words (lexical), and sequences of pseudowords (phonological). We defined two sets of language-selective regions of interest (ROIs) in individual subjects for the PIFG and the PTL using the contrasts [syntactic > lexical] and [syntactic > phonological]. We found robust significant interactions of comprehension and production between these 2 regions at the syntactic level, for both sets of language-selective ROIs. This suggests a core difference in the function of these regions with respect to production and perception, consistent with the lesion literature.

The Cortical Organization of Syntax

Syntax, the structure of sentences, enables humans to express an infinite range of meanings through finite means. The neurobiology of syntax has been intensely studied but with little consensus. Two main candidate regions have been identified: the posterior inferior frontal gyrus (pIFG) and the posterior middle temporal gyrus (pMTG). Integrating research in linguistics, psycholinguistics, and neuroscience, we propose a neuroanatomical framework for syntax that attributes distinct syntactic computations to these regions in a unified model. The key theoretical advances are adopting a modern lexicalized view of syntax in which the lexicon and syntactic rules are intertwined, and recognizing a computational asymmetry in the role of syntax during comprehension and production. Our model postulates a hierarchical lexical-syntactic function to the pMTG, which interconnects previously identified speech perception and conceptual-semantic systems in the temporal and inferior parietal lobes, crucial for both sentence production and comprehension. These relational hierarchies are transformed via the pIFG into morpho-syntactic sequences, primarily tied to production. We show how this architecture provides a better account of the full range of data and is consistent with recent proposals regarding the organization of phonological processes in the brain.

Age of acquisition effects differ across linguistic domains in sign language: EEG evidence

One of the key questions in the study of human language acquisition is the extent to which the development of neural processing networks for different components of language are modulated by exposure to linguistic stimuli. Sign languages offer a unique perspective on this issue, because prelingually Deaf children who receive access to complex linguistic input later in life provide a window into brain maturation in the absence of language, and subsequent neuroplasticity of neurolinguistic networks during late language learning. While the duration of sensitive periods of acquisition of linguistic subsystems (sound, vocabulary, and syntactic structure) is well established on the basis of L2 acquisition in spoken language, for sign languages, the relative timelines for development of neural processing networks for linguistic sub-domains are unknown. We examined neural responses of a group of Deaf signers who received access to signed input at varying ages to three linguistic phenomena at the levels of classifier signs, syntactic structure, and information structure. The amplitude of the N400 response to the marked word order condition negatively correlated with the age of acquisition for syntax and information structure, indicating increased cognitive load in these conditions. Additionally, the combination of behavioral and neural data suggested that late learners preferentially relied on classifiers over word order for meaning extraction. This suggests that late acquisition of sign language significantly increases cognitive load during analysis of syntax and information structure, but not word-level meaning.

Effects of Early Language Deprivation on Brain Connectivity: Language Pathways in Deaf Native and Late First-Language Learners of American Sign Language

Previous research has identified ventral and dorsal white matter tracts as being crucial for language processing; their maturation correlates with increased language processing capacity. Unknown is whether the growth or maintenance of these language-relevant pathways is shaped by language experience in early life. To investigate the effects of early language deprivation and the sensory-motor modality of language on white matter tracts, we examined the white matter connectivity of language-relevant pathways in congenitally deaf people with or without early access to language. We acquired diffusion tensor imaging (DTI) data from two groups of individuals who experienced language from birth, twelve deaf native signers of American Sign Language, and twelve hearing L2 signers of ASL (native English speakers), and from three, well-studied individual cases who experienced minimal language during childhood. The results indicate that the sensory-motor modality of early language experience does not affect the white matter microstructure between crucial language regions. Both groups with early language experience, deaf and hearing, show leftward laterality in the two language-related tracts. However, all three cases with early language deprivation showed altered white matter microstructure, especially in the left dorsal arcuate fasciculus (AF) pathway.

Neural correlates of semantic and syntactic processing in German Sign Language

The study of deaf and hearing native users of signed languages can offer unique insights into how biological constraints and environmental input interact to shape the neural bases of language processing. Here, we use functional magnetic resonance imaging (fMRI) to address two questions: (1) Do semantic and syntactic processing in a signed language rely on anatomically and functionally distinct neural substrates as it has been shown for spoken languages? and (2) Does hearing status affect the neural correlates of these two types of linguistic processing? Deaf and hearing native signers performed a sentence judgement task on German Sign Language (Deutsche Gebärdensprache: DGS) sentences which were correct or contained either syntactic or semantic violations. We hypothesized that processing of semantic and syntactic violations in DGS relies on distinct neural substrates as it has been shown for spoken languages. Moreover, we hypothesized that effects of hearing status are observed within auditory regions, as deaf native signers have been shown to activate auditory areas to a greater extent than hearing native signers when processing a signed language. Semantic processing activated low-level visual areas and the left inferior frontal gyrus (IFG), suggesting both modality-dependent and independent processing mechanisms. Syntactic processing elicited increased activation in the right supramarginal gyrus (SMG). Moreover, psychophysiological interaction (PPI) analyses revealed a cluster in left middle occipital regions showing increased functional coupling with the right SMG during syntactic relative to semantic processing, possibly indicating spatial processing mechanisms that are specific to signed syntax. Effects of hearing status were observed in the right superior temporal cortex (STC): deaf but not hearing native signers showed greater activation for semantic violations than for syntactic violations in this region. Taken together, the present findings suggest that the neural correlates of language processing are partly determined by biological constraints, but that they may additionally be influenced by the unique processing demands of the language modality and different sensory experiences.

Rethinking the critical period for language: New insights into an old question from American Sign Language

The hypothesis that children surpass adults in long-term second-language proficiency is accepted as evidence for a critical period for language. However, the scope and nature of a critical period for language has been the subject of considerable debate. The controversy centers on whether the age-related decline in ultimate second-language proficiency is evidence for a critical period or something else. Here we argue that age-onset effects for first vs. second language outcome are largely different. We show this by examining psycholinguistic studies of ultimate attainment in L2 vs. L1 learners, longitudinal studies of adolescent L1 acquisition, and neurolinguistic studies of late L2 and L1 learners. This research indicates that L1 acquisition arises from post-natal brain development interacting with environmental linguistic experience. By contrast, L2 learning after early childhood is scaffolded by prior childhood L1 acquisition, both linguistically and neurally, making it a less clear test of the critical period for language.

Language switching decomposed through MEG and evidence from bimodal bilinguals

A defining feature of human cognition is the ability to quickly and accurately alternate between complex behaviors. One striking example of such an ability is bilinguals' capacity to rapidly switch between languages. This switching process minimally comprises disengagement from the previous language and engagement in a new language. Previous studies have associated language switching with increased prefrontal activity. However, it is unknown how the subcomputations of language switching individually contribute to these activities, because few natural situations enable full separation of disengagement and engagement processes during switching. We recorded magnetoencephalography (MEG) from American Sign Language-English bilinguals who often sign and speak simultaneously, which allows to dissociate engagement and disengagement. MEG data showed that turning a language "off" (switching from simultaneous to single language production) led to increased activity in the anterior cingulate cortex (ACC) and dorsolateral prefrontal cortex (dlPFC), while turning a language "on" (switching from one language to two simultaneously) did not. The distinct representational nature of these on and off processes was also supported by multivariate decoding analyses. Additionally, Granger causality analyses revealed that (i) compared with "turning on" a language, "turning off" required stronger connectivity between left and right dlPFC, and (ii) dlPFC activity predicted ACC activity, consistent with models in which the dlPFC is a top-down modulator of the ACC. These results suggest that the burden of language switching lies in disengagement from the previous language as opposed to engaging a new language and that, in the absence of motor constraints, producing two languages simultaneously is not necessarily more cognitively costly than producing one.

Language and Sensory Neural Plasticity in the Superior Temporal Cortex of the Deaf

Visual stimuli are known to activate the auditory cortex of deaf people, presenting evidence of cross-modal plasticity. However, the mechanisms underlying such plasticity are poorly understood. In this functional MRI study, we presented two types of visual stimuli, language stimuli (words, sign language, and lip-reading) and a general stimulus (checkerboard) to investigate neural reorganization in the superior temporal cortex (STC) of deaf subjects and hearing controls. We found that only in the deaf subjects, all visual stimuli activated the STC. The cross-modal activation induced by the checkerboard was mainly due to a sensory component via a feed-forward pathway from the thalamus and primary visual cortex, positively correlated with duration of deafness, indicating a consequence of pure sensory deprivation. In contrast, the STC activity evoked by language stimuli was functionally connected to both the visual cortex and the frontotemporal areas, which were highly correlated with the learning of sign language, suggesting a strong language component via a possible feedback modulation. While the sensory component exhibited specificity to features of a visual stimulus (e.g., selective to the form of words, bodies, or faces) and the language (semantic) component appeared to recruit a common frontotemporal neural network, the two components converged to the STC and caused plasticity with different multivoxel activity patterns. In summary, the present study showed plausible neural pathways for auditory reorganization and correlations of activations of the reorganized cortical areas with developmental factors and provided unique evidence towards the understanding of neural circuits involved in cross-modal plasticity.

Shared neural correlates for building phrases in signed and spoken language

Research on the mental representation of human language has convincingly shown that sign languages are structured similarly to spoken languages. However, whether the same neurobiology underlies the online construction of complex linguistic structures in sign and speech remains unknown. To investigate this question with maximally controlled stimuli, we studied the production of minimal two-word phrases in sign and speech. Signers and speakers viewed the same pictures during magnetoencephalography recording and named them with semantically identical expressions. For both signers and speakers, phrase building engaged left anterior temporal and ventromedial cortices with similar timing, despite different linguistic articulators. Thus the neurobiological similarity of sign and speech goes beyond gross measures such as lateralization: the same fronto-temporal network achieves the planning of structured linguistic expressions.

Phonological and semantic priming in American Sign Language: N300 and N400 effects

This study investigated the electrophysiological signatures of phonological and semantic priming in American Sign Language (ASL). Deaf signers made semantic relatedness judgments to pairs of ASL signs separated by a 1300 ms prime-target SOA. Phonologically related sign pairs shared two of three phonological parameters (handshape, location, and movement). Target signs preceded by phonologically related and semantically related prime signs elicited smaller negativities within the N300 and N400 windows than those preceded by unrelated primes. N300 effects, typically reported in studies of picture processing, are interpreted to reflect the mapping from the visual features of the signs to more abstract linguistic representations. N400 effects, consistent with rhyme priming effects in the spoken language literature, are taken to index lexico-semantic processes that appear to be largely modality independent. Together, these results highlight both the unique visual-manual nature of sign languages and the linguistic processing characteristics they share with spoken languages.

Neurolinguistic processing when the brain matures without language

The extent to which development of the brain language system is modulated by the temporal onset of linguistic experience relative to post-natal brain maturation is unknown. This crucial question cannot be investigated with the hearing population because spoken language is ubiquitous in the environment of newborns. Deafness blocks infants' language experience in a spoken form, and in a signed form when it is absent from the environment. Using anatomically constrained magnetoencephalography, aMEG, we neuroimaged lexico-semantic processing in a deaf adult whose linguistic experience began in young adulthood. Despite using language for 30 years after initially learning it, this individual exhibited limited neural response in the perisylvian language areas to signed words during the 300-400 ms temporal window, suggesting that the brain language system requires linguistic experience during brain growth to achieve functionality. The present case study primarily exhibited neural activations in response to signed words in dorsolateral superior parietal and occipital areas bilaterally, replicating the neural patterns exhibited by two previously case studies who matured without language until early adolescence (Ferjan Ramirez N, Leonard MK, Torres C, Hatrak M, Halgren E, Mayberry RI. 2014). The dorsal pathway appears to assume the task of processing words when the brain matures without experiencing the form-meaning network of a language.

Congenital deafness affects deep layers in primary and secondary auditory cortex

Congenital deafness leads to functional deficits in the auditory cortex for which early cochlear implantation can effectively compensate. Most of these deficits have been demonstrated functionally. Furthermore, the majority of previous studies on deafness have involved the primary auditory cortex; knowledge of higher-order areas is limited to effects of cross-modal reorganization. In this study, we compared the cortical cytoarchitecture of four cortical areas in adult hearing and congenitally deaf cats (CDCs): the primary auditory field A1, two secondary auditory fields, namely the dorsal zone and second auditory field (A2); and a reference visual association field (area 7) in the same section stained either using Nissl or SMI-32 antibodies. The general cytoarchitectonic pattern and the area-specific characteristics in the auditory cortex remained unchanged in animals with congenital deafness. Whereas area 7 did not differ between the groups investigated, all auditory fields were slightly thinner in CDCs, this being caused by reduced thickness of layers IV-VI. The study documents that, while the cytoarchitectonic patterns are in general independent of sensory experience, reduced layer thickness is observed in both primary and higher-order auditory fields in layer IV and infragranular layers. The study demonstrates differences in effects of congenital deafness between supragranular and other cortical layers, but similar dystrophic effects in all investigated auditory fields.

Cross-Modal Plasticity in Higher-Order Auditory Cortex of Congenitally Deaf Cats Does Not Limit Auditory Responsiveness to Cochlear Implants

Congenital sensory deprivation can lead to reorganization of the deprived cortical regions by another sensory system. Such cross-modal reorganization may either compete with or complement the "original" inputs to the deprived area after sensory restoration and can thus be either adverse or beneficial for sensory restoration. In congenital deafness, a previous inactivation study documented that supranormal visual behavior was mediated by higher-order auditory fields in congenitally deaf cats (CDCs). However, both the auditory responsiveness of "deaf" higher-order fields and interactions between the reorganized and the original sensory input remain unknown. Here, we studied a higher-order auditory field responsible for the supranormal visual function in CDCs, the auditory dorsal zone (DZ). Hearing cats and visual cortical areas served as a control. Using mapping with microelectrode arrays, we demonstrate spatially scattered visual (cross-modal) responsiveness in the DZ, but show that this did not interfere substantially with robust auditory responsiveness elicited through cochlear implants. Visually responsive and auditory-responsive neurons in the deaf auditory cortex formed two distinct populations that did not show bimodal interactions. Therefore, cross-modal plasticity in the deaf higher-order auditory cortex had limited effects on auditory inputs. The moderate number of scattered cross-modally responsive neurons could be the consequence of exuberant connections formed during development that were not pruned postnatally in deaf cats. Although juvenile brain circuits are modified extensively by experience, the main driving input to the cross-modally (visually) reorganized higher-order auditory cortex remained auditory in congenital deafness.

SIGNIFICANCE STATEMENT

In a common view, the "unused" auditory cortex of deaf individuals is reorganized to a compensatory sensory function during development. According to this view, cross-modal plasticity takes over the unused cortex and reassigns it to the remaining senses. Therefore, cross-modal plasticity might conflict with restoration of auditory function with cochlear implants. It is unclear whether the cross-modally reorganized auditory areas lose auditory responsiveness. We show that the presence of cross-modal plasticity in a higher-order auditory area does not reduce auditory responsiveness of that area. Visual reorganization was moderate, spatially scattered and there were no interactions between cross-modally reorganized visual and auditory inputs. These results indicate that cross-modal reorganization is less detrimental for neurosensory restoration than previously thought.

Functional selectivity for face processing in the temporal voice area of early deaf individuals

Brain systems supporting face and voice processing both contribute to the extraction of important information for social interaction (e.g., person identity). How does the brain reorganize when one of these channels is absent? Here, we explore this question by combining behavioral and multimodal neuroimaging measures (magneto-encephalography and functional imaging) in a group of early deaf humans. We show enhanced selective neural response for faces and for individual face coding in a specific region of the auditory cortex that is typically specialized for voice perception in hearing individuals. In this region, selectivity to face signals emerges early in the visual processing hierarchy, shortly after typical face-selective responses in the ventral visual pathway. Functional and effective connectivity analyses suggest reorganization in long-range connections from early visual areas to the face-selective temporal area in individuals with early and profound deafness. Altogether, these observations demonstrate that regions that typically specialize for voice processing in the hearing brain preferentially reorganize for face processing in born-deaf people. Our results support the idea that cross-modal plasticity in the case of early sensory deprivation relates to the original functional specialization of the reorganized brain regions.

Visual cortex entrains to sign language

Despite immense variability across languages, people can learn to understand any human language, spoken or signed. What neural mechanisms allow people to comprehend language across sensory modalities? When people listen to speech, electrophysiological oscillations in auditory cortex entrain to slow ([Formula: see text]8 Hz) fluctuations in the acoustic envelope. Entrainment to the speech envelope may reflect mechanisms specialized for auditory perception. Alternatively, flexible entrainment may be a general-purpose cortical mechanism that optimizes sensitivity to rhythmic information regardless of modality. Here, we test these proposals by examining cortical coherence to visual information in sign language. First, we develop a metric to quantify visual change over time. We find quasiperiodic fluctuations in sign language, characterized by lower frequencies than fluctuations in speech. Next, we test for entrainment of neural oscillations to visual change in sign language, using electroencephalography (EEG) in fluent speakers of American Sign Language (ASL) as they watch videos in ASL. We find significant cortical entrainment to visual oscillations in sign language <5 Hz, peaking at [Formula: see text]1 Hz. Coherence to sign is strongest over occipital and parietal cortex, in contrast to speech, where coherence is strongest over the auditory cortex. Nonsigners also show coherence to sign language, but entrainment at frontal sites is reduced relative to fluent signers. These results demonstrate that flexible cortical entrainment to language does not depend on neural processes that are specific to auditory speech perception. Low-frequency oscillatory entrainment may reflect a general cortical mechanism that maximizes sensitivity to informational peaks in time-varying signals.

Neural Language Processing in Adolescent First-Language Learners: Longitudinal Case Studies in American Sign Language

One key question in neurolinguistics is the extent to which the neural processing system for language requires linguistic experience during early life to develop fully. We conducted a longitudinal anatomically constrained magnetoencephalography (aMEG) analysis of lexico-semantic processing in 2 deaf adolescents who had no sustained language input until 14 years of age, when they became fully immersed in American Sign Language. After 2 to 3 years of language, the adolescents' neural responses to signed words were highly atypical, localizing mainly to right dorsal frontoparietal regions and often responding more strongly to semantically primed words (Ferjan Ramirez N, Leonard MK, Torres C, Hatrak M, Halgren E, Mayberry RI. 2014. Neural language processing in adolescent first-language learners. Cereb Cortex. 24 (10): 2772-2783). Here, we show that after an additional 15 months of language experience, the adolescents' neural responses remained atypical in terms of polarity. While their responses to less familiar signed words still showed atypical localization patterns, the localization of responses to highly familiar signed words became more concentrated in the left perisylvian language network. Our findings suggest that the timing of language experience affects the organization of neural language processing; however, even in adolescence, language representation in the human brain continues to evolve with experience.

Deaf Students as a Linguistic and Cultural Minority: Shifting Perspectives and Implications for Teaching and Learning

Deaf children have traditionally been perceived and educated as a special needs population. Over the past several decades, several factors have converged to enable a shift in perspective to one in which deaf children are viewed as a cultural and linguistic minority, and the education of deaf children is approached from a bilingual framework. In this article, we present the historical context in which such shifts in perspective have taken place and describe the linguistic, social, and cultural factors that shape a bilingual approach to deaf education. We further discuss the implications of a linguistic and cultural minority perspective of deaf children on language development, teacher preparation, and educational policy.

Differential activity in Heschl's gyrus between deaf and hearing individuals is due to auditory deprivation rather than language modality

Sensory cortices undergo crossmodal reorganisation as a consequence of sensory deprivation. Congenital deafness in humans represents a particular case with respect to other types of sensory deprivation, because cortical reorganisation is not only a consequence of auditory deprivation, but also of language-driven mechanisms. Visual crossmodal plasticity has been found in secondary auditory cortices of deaf individuals, but it is still unclear if reorganisation also takes place in primary auditory areas, and how this relates to language modality and auditory deprivation. Here, we dissociated the effects of language modality and auditory deprivation on crossmodal plasticity in Heschl's gyrus as a whole, and in cytoarchitectonic region Te1.0 (likely to contain the core auditory cortex). Using fMRI, we measured the BOLD response to viewing sign language in congenitally or early deaf individuals with and without sign language knowledge, and in hearing controls. Results show that differences between hearing and deaf individuals are due to a reduction in activation caused by visual stimulation in the hearing group, which is more significant in Te1.0 than in Heschl's gyrus as a whole. Furthermore, differences between deaf and hearing groups are due to auditory deprivation, and there is no evidence that the modality of language used by deaf individuals contributes to crossmodal plasticity in Heschl's gyrus.

Laminar profile of spontaneous and evoked theta: Rhythmic modulation of cortical processing during word integration

Theta may play a central role during language understanding and other extended cognitive processing, providing an envelope for widespread integration of participating cortical areas. We used linear microelectrode arrays in epileptics to define the circuits generating theta in inferotemporal, perirhinal, entorhinal, prefrontal and anterior cingulate cortices. In all locations, theta was generated by excitatory current sinks in middle layers which receive predominantly feedforward inputs, alternating with sinks in superficial layers which receive mainly feedback/associative inputs. Baseline and event-related theta were generated by indistinguishable laminar profiles of transmembrane currents and unit-firing. Word presentation could reset theta phase, permitting theta to contribute to late event-related potentials, even when theta power decreases relative to baseline. Limited recordings during sentence reading are consistent with rhythmic theta activity entrained by a given word modulating the neural background for the following word. These findings show that theta occurs spontaneously, and can be momentarily suppressed, reset and synchronized by words. Theta represents an alternation between feedforward/divergent and associative/convergent processing modes that may temporally organize sustained processing and optimize the timing of memory formation. We suggest that words are initially encoded via a ventral feedforward stream which is lexicosemantic in the anteroventral temporal lobe; its arrival may trigger a widespread theta rhythm which integrates the word within a larger context.

Simultaneous perception of a spoken and a signed language: The brain basis of ASL-English code-blends

Code-blends (simultaneous words and signs) are a unique characteristic of bimodal bilingual communication. Using fMRI, we investigated code-blend comprehension in hearing native ASL-English bilinguals who made a semantic decision (edible?) about signs, audiovisual words, and semantically equivalent code-blends. English and ASL recruited a similar fronto-temporal network with expected modality differences: stronger activation for English in auditory regions of bilateral superior temporal cortex, and stronger activation for ASL in bilateral occipitotemporal visual regions and left parietal cortex. Code-blend comprehension elicited activity in a combination of these regions, and no cognitive control regions were additionally recruited. Furthermore, code-blends elicited reduced activation relative to ASL presented alone in bilateral prefrontal and visual extrastriate cortices, and relative to English alone in auditory association cortex. Consistent with behavioral facilitation observed during semantic decisions, the findings suggest that redundant semantic content induces more efficient neural processing in language and sensory regions during bimodal language integration.

On the temporal dynamics of sign production: An ERP study in Catalan Sign Language (LSC)

This study investigates the temporal dynamics of sign production and how particular aspects of the signed modality influence the early stages of lexical access. To that end, we explored the electrophysiological correlates associated to sign frequency and iconicity in a picture signing task in a group of bimodal bilinguals. Moreover, a subset of the same participants was tested in the same task but naming the pictures instead. Our results revealed that both frequency and iconicity influenced lexical access in sign production. At the ERP level, iconicity effects originated very early in the course of signing (while absent in the spoken modality), suggesting a stronger activation of the semantic properties for iconic signs. Moreover, frequency effects were modulated by iconicity, suggesting that lexical access in signed language is determined by the iconic properties of the signs. These results support the idea that lexical access is sensitive to the same phenomena in word and sign production, but its time-course is modulated by particular aspects of the modality in which a lexical item will be finally articulated.

Cochlear implantation (CI) for prelingual deafness: the relevance of studies of brain organization and the role of first language acquisition in considering outcome success

Cochlear implantation (CI) for profound congenital hearing impairment, while often successful in restoring hearing to the deaf child, does not always result in effective speech processing. Exposure to non-auditory signals during the pre-implantation period is widely held to be responsible for such failures. Here, we question the inference that such exposure irreparably distorts the function of auditory cortex, negatively impacting the efficacy of CI. Animal studies suggest that in congenital early deafness there is a disconnection between (disordered) activation in primary auditory cortex (A1) and activation in secondary auditory cortex (A2). In humans, one factor contributing to this functional decoupling is assumed to be abnormal activation of A1 by visual projections-including exposure to sign language. In this paper we show that that this abnormal activation of A1 does not routinely occur, while A2 functions effectively supramodally and multimodally to deliver spoken language irrespective of hearing status. What, then, is responsible for poor outcomes for some individuals with CI and for apparent abnormalities in cortical organization in these people? Since infancy is a critical period for the acquisition of language, deaf children born to hearing parents are at risk of developing inefficient neural structures to support skilled language processing. A sign language, acquired by a deaf child as a first language in a signing environment, is cortically organized like a heard spoken language in terms of specialization of the dominant perisylvian system. However, very few deaf children are exposed to sign language in early infancy. Moreover, no studies to date have examined sign language proficiency in relation to cortical organization in individuals with CI. Given the paucity of such relevant findings, we suggest that the best guarantee of good language outcome after CI is the establishment of a secure first language pre-implant-however that may be achieved, and whatever the success of auditory restoration.

Magnetoencephalography in the study of brain dynamics

To progress toward understanding of the mechanisms underlying the functional organization of the human brain, either a bottom-up or a top-down approach may be adopted. The former starts from the study of the detailed functioning of a small number of neuronal assemblies, while the latter tries to decode brain functioning by considering the brain as a whole. This review discusses the top-down approach and the use of magnetoencephalography (MEG) to describe global brain properties. The main idea behind this approach is that the concurrence of several areas is required for the brain to instantiate a specific behavior/functioning. A central issue is therefore the study of brain functional connectivity and the concept of brain networks as ensembles of distant brain areas that preferentially exchange information. Importantly, the human brain is a dynamic device, and MEG is ideally suited to investigate phenomena on behaviorally relevant timescales, also offering the possibility of capturing behaviorally-related brain connectivity dynamics.

Neural language processing in adolescent first-language learners

The relation between the timing of language input and development of neural organization for language processing in adulthood has been difficult to tease apart because language is ubiquitous in the environment of nearly all infants. However, within the congenitally deaf population are individuals who do not experience language until after early childhood. Here, we investigated the neural underpinnings of American Sign Language (ASL) in 2 adolescents who had no sustained language input until they were approximately 14 years old. Using anatomically constrained magnetoencephalography, we found that recently learned signed words mainly activated right superior parietal, anterior occipital, and dorsolateral prefrontal areas in these 2 individuals. This spatiotemporal activity pattern was significantly different from the left fronto-temporal pattern observed in young deaf adults who acquired ASL from birth, and from that of hearing young adults learning ASL as a second language for a similar length of time as the cases. These results provide direct evidence that the timing of language experience over human development affects the organization of neural language processing.

Neuroanatomical profiles of deafness in the context of native language experience

The study of congenitally deaf adult humans provides an opportunity to examine neuroanatomical plasticity resulting from altered sensory experience. However, attributing the source of the brain's structural variance in the deaf is complicated by the fact that deaf individuals also differ in their language experiences (e.g., sign vs spoken), which likely influence brain anatomy independently. Although the majority of deaf individuals in the United States are born to hearing parents and are exposed to English, not American Sign Language (ASL) as their first language, most studies on deafness have been conducted with deaf native users of ASL (deaf signers). This raises the question of whether observations made in deaf signers can be generalized. Using a factorial design, we compared gray (GMV) and white (WMV) matter volume in deaf and hearing native users of ASL, as well as deaf and hearing native users of English. Main effects analysis of sensory experience revealed less GMV in the deaf groups combined (compared with hearing groups combined) in early visual areas and less WMV in a left early auditory region. The interaction of sensory experience and language experience revealed that deaf native users of English had fewer areas of anatomical differences than did deaf native users of ASL (each compared with their hearing counterparts). For deaf users of ASL specifically, WMV differences resided in language areas such as the left superior temporal and inferior frontal regions. Our results demonstrate that cortical plasticity resulting from deafness depends on language experience and that findings from native signers cannot be generalized.

How sensory-motor systems impact the neural organization for language: direct contrasts between spoken and signed language

To investigate the impact of sensory-motor systems on the neural organization for language, we conducted an H₂¹⁵O-PET study of sign and spoken word production (picture-naming) and an fMRI study of sign and audio-visual spoken language comprehension (detection of a semantically anomalous sentence) with hearing bilinguals who are native users of American Sign Language (ASL) and English. Directly contrasting speech and sign production revealed greater activation in bilateral parietal cortex for signing, while speaking resulted in greater activation in bilateral superior temporal cortex (STC) and right frontal cortex, likely reflecting auditory feedback control. Surprisingly, the language production contrast revealed a relative increase in activation in bilateral occipital cortex for speaking. We speculate that greater activation in visual cortex for speaking may actually reflect cortical attenuation when signing, which functions to distinguish self-produced from externally generated visual input. Directly contrasting speech and sign comprehension revealed greater activation in bilateral STC for speech and greater activation in bilateral occipital-temporal cortex for sign. Sign comprehension, like sign production, engaged bilateral parietal cortex to a greater extent than spoken language. We hypothesize that posterior parietal activation in part reflects processing related to spatial classifier constructions in ASL and that anterior parietal activation may reflect covert imitation that functions as a predictive model during sign comprehension. The conjunction analysis for comprehension revealed that both speech and sign bilaterally engaged the inferior frontal gyrus (with more extensive activation on the left) and the superior temporal sulcus, suggesting an invariant bilateral perisylvian language system. We conclude that surface level differences between sign and spoken languages should not be dismissed and are critical for understanding the neurobiology of language.

How does visual language affect crossmodal plasticity and cochlear implant success?

Cochlear implants (CI) are the most successful intervention for ameliorating hearing loss in severely or profoundly deaf children. Despite this, educational performance in children with CI continues to lag behind their hearing peers. From animal models and human neuroimaging studies it has been proposed the integrative functions of auditory cortex are compromised by crossmodal plasticity. This has been argued to result partly from the use of a visual language. Here we argue that 'cochlear implant sensitive periods' comprise both auditory and language sensitive periods, and thus cannot be fully described with animal models. Despite prevailing assumptions, there is no evidence to link the use of a visual language to poorer CI outcome. Crossmodal reorganisation of auditory cortex occurs regardless of compensatory strategies, such as sign language, used by the deaf person. In contrast, language deprivation during early sensitive periods has been repeatedly linked to poor language outcomes. Language sensitive periods have largely been ignored when considering variation in CI outcome, leading to ill-founded recommendations concerning visual language in CI habilitation.

Functional and anatomical correlates of word-, sentence-, and discourse-level integration in sign language

In both vocal and sign languages, we can distinguish word-, sentence-, and discourse-level integration in terms of hierarchical processes, which integrate various elements into another higher level of constructs. In the present study, we used magnetic resonance imaging and voxel-based morphometry (VBM) to test three language tasks in Japanese Sign Language (JSL): word-level (Word), sentence-level (Sent), and discourse-level (Disc) decision tasks. We analyzed cortical activity and gray matter (GM) volumes of Deaf signers, and clarified three major points. First, we found that the activated regions in the frontal language areas gradually expanded in the dorso-ventral axis, corresponding to a difference in linguistic units for the three tasks. Moreover, the activations in each region of the frontal language areas were incrementally modulated with the level of linguistic integration. These dual mechanisms of the frontal language areas may reflect a basic organization principle of hierarchically integrating linguistic information. Secondly, activations in the lateral premotor cortex and inferior frontal gyrus were left-lateralized. Direct comparisons among the language tasks exhibited more focal activation in these regions, suggesting their functional localization. Thirdly, we found significantly positive correlations between individual task performances and GM volumes in localized regions, even when the ages of acquisition (AOAs) of JSL and Japanese were factored out. More specifically, correlations with the performances of the Word and Sent tasks were found in the left precentral/postcentral gyrus and insula, respectively, while correlations with those of the Disc task were found in the left ventral inferior frontal gyrus and precuneus. The unification of functional and anatomical studies would thus be fruitful for understanding human language systems from the aspects of both universality and individuality.

Visual activity predicts auditory recovery from deafness after adult cochlear implantation

Modern cochlear implantation technologies allow deaf patients to understand auditory speech; however, the implants deliver only a coarse auditory input and patients must use long-term adaptive processes to achieve coherent percepts. In adults with post-lingual deafness, the high progress of speech recovery is observed during the first year after cochlear implantation, but there is a large range of variability in the level of cochlear implant outcomes and the temporal evolution of recovery. It has been proposed that when profoundly deaf subjects receive a cochlear implant, the visual cross-modal reorganization of the brain is deleterious for auditory speech recovery. We tested this hypothesis in post-lingually deaf adults by analysing whether brain activity shortly after implantation correlated with the level of auditory recovery 6 months later. Based on brain activity induced by a speech-processing task, we found strong positive correlations in areas outside the auditory cortex. The highest positive correlations were found in the occipital cortex involved in visual processing, as well as in the posterior-temporal cortex known for audio-visual integration. The other area, which positively correlated with auditory speech recovery, was localized in the left inferior frontal area known for speech processing. Our results demonstrate that the visual modality's functional level is related to the proficiency level of auditory recovery. Based on the positive correlation of visual activity with auditory speech recovery, we suggest that visual modality may facilitate the perception of the word's auditory counterpart in communicative situations. The link demonstrated between visual activity and auditory speech perception indicates that visuoauditory synergy is crucial for cross-modal plasticity and fostering speech-comprehension recovery in adult cochlear-implanted deaf patients.

Neural stages of spoken, written, and signed word processing in beginning second language learners

We combined magnetoencephalography (MEG) and magnetic resonance imaging (MRI) to examine how sensory modality, language type, and language proficiency interact during two fundamental stages of word processing: (1) an early word encoding stage, and (2) a later supramodal lexico-semantic stage. Adult native English speakers who were learning American Sign Language (ASL) performed a semantic task for spoken and written English words, and ASL signs. During the early time window, written words evoked responses in left ventral occipitotemporal cortex, and spoken words in left superior temporal cortex. Signed words evoked activity in right intraparietal sulcus that was marginally greater than for written words. During the later time window, all three types of words showed significant activity in the classical left fronto-temporal language network, the first demonstration of such activity in individuals with so little second language (L2) instruction in sign. In addition, a dissociation between semantic congruity effects and overall MEG response magnitude for ASL responses suggested shallower and more effortful processing, presumably reflecting novice L2 learning. Consistent with previous research on non-dominant language processing in spoken languages, the L2 ASL learners also showed recruitment of right hemisphere and lateral occipital cortex. These results demonstrate that late lexico-semantic processing utilizes a common substrate, independent of modality, and that proficiency effects in sign language are comparable to those in spoken language.

Spatial and temporal features of superordinate semantic processing studied with fMRI and EEG

The relationships between the anatomical representation of semantic knowledge in the human brain and the timing of neurophysiological mechanisms involved in manipulating such information remain unclear. This is the case for superordinate semantic categorization-the extraction of general features shared by broad classes of exemplars (e.g., living vs. non-living semantic categories). We proposed that, because of the abstract nature of this information, input from diverse input modalities (visual or auditory, lexical or non-lexical) should converge and be processed in the same regions of the brain, at similar time scales during superordinate categorization-specifically in a network of heteromodal regions, and late in the course of the categorization process. In order to test this hypothesis, we utilized electroencephalography and event related potentials (EEG/ERP) with functional magnetic resonance imaging (fMRI) to characterize subjects' responses as they made superordinate categorical decisions (living vs. non-living) about objects presented as visual pictures or auditory words. Our results reveal that, consistent with our hypothesis, during the course of superordinate categorization, information provided by these diverse inputs appears to converge in both time and space: fMRI showed that heteromodal areas of the parietal and temporal cortices are active during categorization of both classes of stimuli. The ERP results suggest that superordinate categorization is reflected as a late positive component (LPC) with a parietal distribution and long latencies for both stimulus types. Within the areas and times in which modality independent responses were identified, some differences between living and non-living categories were observed, with a more widespread spatial extent and longer latency responses for categorization of non-living items.

Multisensory integration and neuroplasticity in the human cerebral cortex

There is a strong interaction between multisensory processing and the neuroplasticity of the human brain. On one hand, recent research demonstrates that experience and training in various domains modifies how information from the different senses is integrated; and, on the other hand multisensory training paradigms seem to be particularly effective in driving functional and structural plasticity. Multisensory training affects early sensory processing within separate sensory domains, as well as the functional and structural connectivity between uni- and multisensory brain regions. In this review, we discuss the evidence for interactions of multisensory processes and brain plasticity and give an outlook on promising clinical applications and open questions.