Functional selectivity for face processing in the temporal voice area of early deaf individuals

Face detection mechanisms: Nature vs. nurture

For many animals, faces are a vitally important visual stimulus. Hence, it is not surprising that face perception has become a very popular research topic in neuroscience, with ca. 2000 papers published every year. As a result, significant progress has been made in understanding the intricate mechanisms underlying this phenomenon. However, the ontogeny of face perception, in particular the role of innate predispositions, remains largely unexplored at the neural level. Several influential studies in monkeys have suggested that seeing faces is necessary for the development of the face-selective brain domains. At the same time, behavioural experiments with newborn human babies and newly-hatched domestic chicks demonstrate that a spontaneous preference towards faces emerges early in life without pre-existing experience. Moreover, we were recently able to record face-selective neural responses in the brain of young, face-naïve chicks, thus demonstrating the existence of an innate face detection mechanism. In this review, we discuss these seemingly contradictory results and propose potential experimental approaches to resolve some of the open questions.

Research of visual attention networks in deaf individuals: a systematic review

The impact of deafness on visual attention has been widely discussed in previous research. It has been noted that deficiencies and strengths of previous research can be attributed to temporal or spatial aspects of attention, as well as variations in development and clinical characteristics. Visual attention is categorized into three networks: orienting (exogenous and endogenous), alerting (phasic and tonic), and executive control. This study aims to contribute new neuroscientific evidence supporting this hypothesis. This paper presents a systematic review of the international literature from the past 15 years focused on visual attention in the deaf population. The final review included 24 articles. The function of the orienting network is found to be enhanced in deaf adults and children, primarily observed in native signers without cochlear implants, while endogenous orienting is observed only in the context of gaze cues in children, with no differences found in adults. Results regarding alerting and executive function vary depending on clinical characteristics and paradigms used. Implications for future research on visual attention in the deaf population are discussed.

Connectome alterations following perinatal deafness in the cat

Following sensory deprivation, areas and networks in the brain may adapt and reorganize to compensate for the loss of input. These adaptations are manifestations of compensatory crossmodal plasticity, which has been documented in both human and animal models of deafness-including the domestic cat. Although there are abundant examples of structural plasticity in deaf felines from retrograde tracer-based studies, there is a lack of diffusion-based knowledge involving this model compared to the current breadth of human research. The purpose of this study was to explore white matter structural adaptations in the perinatally-deafened cat via tractography, increasing the methodological overlap between species. Plasticity was examined by identifying unique group connections and assessing altered connectional strength throughout the entirety of the brain. Results revealed a largely preserved connectome containing a limited number of group-specific or altered connections focused within and between sensory networks, which is generally corroborated by deaf feline anatomical tracer literature. Furthermore, five hubs of cortical plasticity and altered communication following perinatal deafness were observed. The limited differences found in the present study suggest that deafness-induced crossmodal plasticity is largely built upon intrinsic structural connections, with limited remodeling of underlying white matter.

Functional to structural plasticity in unilateral sudden sensorineural hearing loss: neuroimaging evidence

A cortical plasticity after long-duration single side deafness (SSD) is advocated with neuroimaging evidence while little is known about the short-duration SSDs. In this case-cohort study, we recruited unilateral sudden sensorineural hearing loss (SSNHL) patients and age-, gender-matched health controls (HC), followed by comprehensive neuroimaging analyses. The primary outcome measures were temporal alterations of varied dynamic functional network connectivity (dFNC) states, neurovascular coupling (NVC) and brain region volume at different stages of SSNHL. The secondary outcome measures were pure-tone audiograms of SSNHL patients before and after treatment. A total of 38 SSNHL patients (21 [55%] male; mean [standard deviation] age, 45.05 [15.83] years) and 44 HC (28 [64%] male; mean [standard deviation] age, 43.55 [12.80] years) were enrolled. SSNHL patients were categorized into subgroups based on the time from disease onset to the initial magnetic resonance imaging scan: early- (n = 16; 1-6 days), intermediate- (n = 9; 7-13 days), and late- stage (n = 13; 14-30 days) groups. We first identified slow state transitions between varied dFNC states at early-stage SSNHL, then revealed the decreased NVC restricted to the auditory cortex at the intermediate- and late-stage SSNHL. Finally, a significantly decreased volume of the left medial superior frontal gyrus (SFGmed) was observed only in the late-stage SSNHL cohort. Furthermore, the volume of the left SFGmed is robustly correlated with both disease duration and patient prognosis. Our study offered neuroimaging evidence for the evolvement from functional to structural brain alterations of SSNHL patients with disease duration less than 1 month, which may explain, from a neuroimaging perspective, why early-stage SSNHL patients have better therapeutic responses and hearing recovery.

Increased functional connectivity between the auditory cortex and the frontoparietal network compensates for impaired visuomotor transformation after early auditory deprivation

Early auditory deprivation leads to a reorganization of large-scale brain networks involving and extending beyond the auditory system. It has been documented that visuomotor transformation is impaired after early deafness, associated with a hyper-crosstalk between the task-critical frontoparietal network and the default-mode network. However, it remains unknown whether and how the reorganized large-scale brain networks involving the auditory cortex contribute to impaired visuomotor transformation after early deafness. Here, we asked deaf and early hard of hearing participants and normal hearing controls to judge the spatial location of a visual target. Compared with normal hearing controls, the superior temporal gyrus showed significantly increased functional connectivity with the frontoparietal network and the default-mode network in deaf and early hard of hearing participants, specifically during egocentric judgments. However, increased superior temporal gyrus-frontoparietal network and superior temporal gyrus-default-mode network coupling showed antagonistic effects on egocentric judgments. In deaf and early hard of hearing participants, increased superior temporal gyrus-frontoparietal network connectivity was associated with improved egocentric judgments, whereas increased superior temporal gyrus-default-mode network connectivity was associated with deteriorated performance in the egocentric task. Therefore, the data suggest that the auditory cortex exhibits compensatory neuroplasticity (i.e. increased functional connectivity with the task-critical frontoparietal network) to mitigate impaired visuomotor transformation after early auditory deprivation.

Social perception in deaf individuals: A meta-analysis of neuroimaging studies

Deaf individuals may report difficulties in social interactions. However, whether these difficulties depend on deafness affecting social brain circuits is controversial. Here, we report the first meta-analysis comparing brain activations of hearing and (prelingually) deaf individuals during social perception. Our findings showed that deafness does not impact on the functional mechanisms supporting social perception. Indeed, both deaf and hearing control participants recruited regions of the action observation network during performance of different social tasks employing visual stimuli, and including biological motion perception, face identification, action observation, viewing, identification and memory for signs and lip reading. Moreover, we found increased recruitment of the superior-middle temporal cortex in deaf individuals compared with hearing participants, suggesting a preserved and augmented function during social communication based on signs and lip movements. Overall, our meta-analysis suggests that social difficulties experienced by deaf individuals are unlikely to be associated with brain alterations but may rather depend on non-supportive environments.

Does age-related hearing loss deteriorate attentional resources?

Recent work suggests that age-related hearing loss (HL) is a possible risk factor for cognitive decline in older adults. Resulting poor speech recognition negatively impacts cognitive, social and emotional functioning and may relate to dementia. However, little is known about the consequences of hearing loss on other non-linguistic domains of cognition. The aim of this study was to investigate the role of HL on covert orienting of attention, selective attention and executive control. We compared older adults with and without mild to moderate hearing loss (26-60 dB) performing (1) a spatial cueing task with uninformative central cues (social vs. nonsocial cues), (2) a flanker task and (3) a neuropsychological assessment of attention. The results showed that overall response times and flanker interference effects were comparable across groups. However, in spatial cueing of attention using social and nonsocial cues, hearing impaired individuals were characterized by reduced validity effects, though no additional group differences were found between social and nonsocial cues. Hearing impaired individuals also demonstrated diminished performance on the Montreal Cognitive Assessment (MoCA) and on tasks requiring divided attention and flexibility. This work indicates that while response speed and response inhibition appear to be preserved following mild-to-moderate acquired hearing loss, orienting of attention, divided attention and the ability to flexibly allocate attentional resources are more deteriorated in older adults with HL. This work suggests that hearing loss might exacerbate the detrimental influences of aging on visual attention.

Resting-state functional connectivity in deaf and hearing individuals and its link to executive processing

Sensory experience shapes brain structure and function, and it is likely to influence the organisation of functional networks of the brain, including those involved in cognitive processing. Here we investigated the influence of early deafness on the organisation of resting-state networks of the brain and its relation to executive processing. We compared resting-state connectivity between deaf and hearing individuals across 18 functional networks and 400 ROIs. Our results showed significant group differences in connectivity between seeds of the auditory network and most large-scale networks of the brain, in particular the somatomotor and salience/ventral attention networks. When we investigated group differences in resting-state fMRI and their link to behavioural performance in executive function tasks (working memory, inhibition and switching), differences between groups were found in the connectivity of association networks of the brain, such as the salience/ventral attention and default-mode networks. These findings indicate that sensory experience influences not only the organisation of sensory networks, but that it also has a measurable impact on the organisation of association networks supporting cognitive processing. Overall, our findings suggest that different developmental pathways and functional organisation can support executive processing in the adult brain.

Crossmodal plasticity in hearing loss

Crossmodal plasticity is a textbook example of the ability of the brain to reorganize based on use. We review evidence from the auditory system showing that such reorganization has significant limits, is dependent on pre-existing circuitry and top-down interactions, and that extensive reorganization is often absent. We argue that the evidence does not support the hypothesis that crossmodal reorganization is responsible for closing critical periods in deafness, and crossmodal plasticity instead represents a neuronal process that is dynamically adaptable. We evaluate the evidence for crossmodal changes in both developmental and adult-onset deafness, which start as early as mild-moderate hearing loss and show reversibility when hearing is restored. Finally, crossmodal plasticity does not appear to affect the neuronal preconditions for successful hearing restoration. Given its dynamic and versatile nature, we describe how this plasticity can be exploited for improving clinical outcomes after neurosensory restoration.

Temporal visual representation elicits early auditory-like responses in hearing but not in deaf individuals

It is evident that the brain is capable of large-scale reorganization following sensory deprivation, but the extent of such reorganization is to date, not clear. The auditory modality is the most accurate to represent temporal information, and deafness is an ideal clinical condition to study the reorganization of temporal representation when the audio signal is not available. Here we show that hearing, but not deaf individuals, show a strong ERP response to visual stimuli in temporal areas during a time-bisection task. This ERP response appears 50-90 ms after the flash and recalls some aspects of the N1 ERP component usually elicited by auditory stimuli. The same ERP is not evident for a visual space-bisection task, suggesting that the early recruitment of temporal cortex is specific for building a highly resolved temporal representation within the visual modality. These findings provide evidence that the lack of auditory input can interfere with typical development of complex visual temporal representations.

Somatosensory processing in deaf and deafblind individuals: How does the brain adapt as a function of sensory and linguistic experience? A critical review

How do deaf and deafblind individuals process touch? This question offers a unique model to understand the prospects and constraints of neural plasticity. Our brain constantly receives and processes signals from the environment and combines them into the most reliable information content. The nervous system adapts its functional and structural organization according to the input, and perceptual processing develops as a function of individual experience. However, there are still many unresolved questions regarding the deciding factors for these changes in deaf and deafblind individuals, and so far, findings are not consistent. To date, most studies have not taken the sensory and linguistic experiences of the included participants into account. As a result, the impact of sensory deprivation vs. language experience on somatosensory processing remains inconclusive. Even less is known about the impact of deafblindness on brain development. The resulting neural adaptations could be even more substantial, but no clear patterns have yet been identified. How do deafblind individuals process sensory input? Studies on deafblindness have mostly focused on single cases or groups of late-blind individuals. Importantly, the language backgrounds of deafblind communities are highly variable and include the usage of tactile languages. So far, this kind of linguistic experience and its consequences have not been considered in studies on basic perceptual functions. Here, we will provide a critical review of the literature, aiming at identifying determinants for neuroplasticity and gaps in our current knowledge of somatosensory processing in deaf and deafblind individuals.

Face shape processing via visual-to-auditory sensory substitution activates regions within the face processing networks in the absence of visual experience

Previous evidence suggests that visual experience is crucial for the emergence and tuning of the typical neural system for face recognition. To challenge this conclusion, we trained congenitally blind adults to recognize faces via visual-to-auditory sensory-substitution (SDD). Our results showed a preference for trained faces over other SSD-conveyed visual categories in the fusiform gyrus and in other known face-responsive-regions of the deprived ventral visual stream. We also observed a parametric modulation in the same cortical regions, for face orientation (upright vs. inverted) and face novelty (trained vs. untrained). Our results strengthen the conclusion that there is a predisposition for sensory-independent and computation-specific processing in specific cortical regions that can be retained in life-long sensory deprivation, independently of previous perceptual experience. They also highlight that if the right training is provided, such cortical preference maintains its tuning to what were considered visual-specific face features.

Impact of blindness onset on the representation of sound categories in occipital and temporal cortices

The ventral occipito-temporal cortex (VOTC) reliably encodes auditory categories in people born blind using a representational structure partially similar to the one found in vision (Mattioni et al.,2020). Here, using a combination of uni- and multivoxel analyses applied to fMRI data, we extend our previous findings, comprehensively investigating how early and late acquired blindness impact on the cortical regions coding for the deprived and the remaining senses. First, we show enhanced univariate response to sounds in part of the occipital cortex of both blind groups that is concomitant to reduced auditory responses in temporal regions. We then reveal that the representation of the sound categories in the occipital and temporal regions is more similar in blind subjects compared to sighted subjects. What could drive this enhanced similarity? The multivoxel encoding of the ‘human voice’ category that we observed in the temporal cortex of all sighted and blind groups is enhanced in occipital regions in blind groups , suggesting that the representation of vocal information is more similar between the occipital and temporal regions in blind compared to sighted individuals. We additionally show that blindness does not affect the encoding of the acoustic properties of our sounds (e.g. pitch, harmonicity) in occipital and in temporal regions but instead selectively alter the categorical coding of the voice category itself. These results suggest a functionally congruent interplay between the reorganization of occipital and temporal regions following visual deprivation, across the lifespan.

Working Memory for Faces among Individuals with Congenital Deafness

BACKGROUND

 Studies examining face processing among individuals with congenital deafness show inconsistent results that are often accounted for by sign language skill. However, working memory for faces as an aspect of face processing has not yet been examined in congenital deafness.

PURPOSE

 To explore working memory for faces among individuals with congenital deafness who are skilled in sign language.

RESEARCH DESIGN

 A quasi-experimental study of individuals with congenital deafness and a control group.

STUDY SAMPLE

 Sixteen individuals with congenital deafness who are skilled in sign language and 18 participants with intact hearing, matched for age, and education.

INTERVENTION

 The participants performed two conditions of the N-back test in ascending difficulty (i.e., 1-back and 2-back).

DATA COLLECTION AND ANALYSIS

 Levene's and Shapiro-Wilk tests were used to assess group homoscedasticity and normality, respectively. A two-way repeated measures analysis of variance was applied to compare the groups in response time and accuracy of the N-back test, as well as Pearson correlation between response time and accuracy, and sign language skill duration.

RESULTS

 The congenital deafness group performed better than controls, as was found in the response time but not in the accuracy variables. However, an interaction effect showed that this pattern was significant for the 1-back but not for the 2-back condition in the response time but not the accuracy. Further, there was a marginal effect in response time but a significant one in accuracy showing the 2-back was performed worse than the 1-back. No significant correlation was found between response time and accuracy, and sign language skill duration.

CONCLUSION

 Face processing advantage associated with congenital deafness is dependent on cognitive load, but sign language duration does not affect this trend. In addition, response time and accuracy are not equally sensitive to performance differences in the N-back test.

Central auditory deficits associated with genetic forms of peripheral deafness

Since the 1990s, the study of inherited hearing disorders, mostly those detected at birth, in the prelingual period or in young adults, has led to the identification of their causal genes. The genes responsible for more than 140 isolated (non-syndromic) and about 400 syndromic forms of deafness have already been discovered. Studies of mouse models of these monogenic forms of deafness have provided considerable insight into the molecular mechanisms of hearing, particularly those involved in the development and/or physiology of the auditory sensory organ, the cochlea. In parallel, studies of these models have also made it possible to decipher the pathophysiological mechanisms underlying hearing impairment. This has led a number of laboratories to investigate the potential of gene therapy for curing these forms of deafness. Proof-of-concept has now been obtained for the treatment of several forms of deafness in mouse models, paving the way for clinical trials of cochlear gene therapy in patients in the near future. Nevertheless, peripheral deafness may also be associated with central auditory dysfunctions and may extend well beyond the auditory system itself, as a consequence of alterations to the encoded sensory inputs or involvement of the causal deafness genes in the development and/or functioning of central auditory circuits. Investigating the diversity, causes and underlying mechanisms of these central dysfunctions, the ways in which they could impede the expected benefits of hearing restoration by peripheral gene therapy, and determining how these problems could be remedied is becoming a research field in its own right. Here, we provide an overview of the current knowledge about the central deficits associated with genetic forms of deafness.

Developmental experiences alter the temporal processing characteristics of the visual cortex: Evidence from deaf and hearing native signers

To date, the extent to which early experience shapes the functional characteristics of neural circuits is still a matter of debate. In the present study, we tested whether congenital deafness and/or the acquisition of a sign language alter the temporal processing characteristics of the visual system. Moreover, we investigated whether, assuming cross-modal plasticity in deaf individuals, the temporal processing characteristics of possibly reorganised auditory areas resemble those of the visual cortex. Steady-state visual evoked potentials (SSVEPs) were recorded in congenitally deaf native signers, hearing native signers, and hearing nonsigners. The luminance of the visual stimuli was periodically modulated at 12, 21, and 40 Hz. For hearing nonsigners, the optimal driving rate was 12 Hz. By contrast, for the group of hearing signers the optimal driving rate was 12 and 21 Hz, whereas for the group of deaf signers the optimal driving rate was 21 Hz. We did not observe evidence for cross-modal recruitment of auditory cortex in the group of deaf signers. These results suggest a higher preferred neural processing rate as a consequence of the acquisition of a sign language.

Cross-modal integration and plasticity in the superior temporal cortex

In congenitally deaf people, temporal regions typically believed to be primarily auditory enhance their response to nonauditory information. The neural mechanisms and functional principles underlying this phenomenon, as well as its impact on auditory recovery after sensory restoration, yet remain debated. In this chapter, we demonstrate that the cross-modal recruitment of temporal regions by visual inputs in congenitally deaf people follows organizational principles known to be present in the hearing brain. We propose that the functional and structural mechanisms allowing optimal convergence of multisensory information in the temporal cortex of hearing people also provide the neural scaffolding for feeding visual or tactile information into the deafened temporal areas. Innate in their nature, such anatomo-functional links between the auditory and other sensory systems would represent the common substrate of both early multisensory integration and expression of selective cross-modal plasticity in the superior temporal cortex.

Reduced flash lag illusion in early deaf individuals

When a brief flash is quickly presented aligned with a moving target, the flash typically appears to lag behind the moving stimulus. This effect is widely known in the literature as a flash-lag illusion (FLI). The flash-lag is an example of a motion-induced position shift. Since auditory deprivation leads to both enhanced visual skills and impaired temporal abilities, both crucial for the perception of the flash-lag effect, here we hypothesized that lack of audition could influence the FLI. 13 early deaf and 18 hearing individuals were tested in a visual FLI paradigm to investigate this hypothesis. As expected, results demonstrated a reduction of the flash-lag effect following early deafness, both in the central and peripheral visual fields. Moreover, only for deaf individuals, there is a positive correlation between the flash-lag effect in the peripheral and central visual field, suggesting that the mechanisms underlying the effect in the center of the visual field expand to the periphery following deafness. Overall, these findings reveal that lack of audition early in life profoundly impacts early visual processing underlying the flash-lag effect.

The Extent of Task Specificity for Visual and Tactile Sequences in the Auditory Cortex of the Deaf and Hard of Hearing

It has been proposed that the auditory cortex in the deaf humans might undergo task-specific reorganization. However, evidence remains scarce as previous experiments used only two very specific tasks (temporal processing and face perception) in visual modality. Here, congenitally deaf/hard of hearing and hearing women and men were enrolled in an fMRI experiment as we sought to fill this evidence gap in two ways. First, we compared activation evoked by a temporal processing task performed in two different modalities, visual and tactile. Second, we contrasted this task with a perceptually similar task that focuses on the spatial dimension. Additional control conditions consisted of passive stimulus observation. In line with the task specificity hypothesis, the auditory cortex in the deaf was activated by temporal processing in both visual and tactile modalities. This effect was selective for temporal processing relative to spatial discrimination. However, spatial processing also led to significant auditory cortex recruitment which, unlike temporal processing, occurred even during passive stimulus observation. We conclude that auditory cortex recruitment in the deaf and hard of hearing might involve interplay between task-selective and pluripotential mechanisms of cross-modal reorganization. Our results open several avenues for the investigation of the full complexity of the cross-modal plasticity phenomenon.SIGNIFICANCE STATEMENT Previous studies suggested that the auditory cortex in the deaf may change input modality (sound to vision) while keeping its function (e.g., rhythm processing). We investigated this hypothesis by asking deaf or hard of hearing and hearing adults to discriminate between temporally and spatially complex sequences in visual and tactile modalities. The results show that such function-specific brain reorganization, as has previously been demonstrated in the visual modality, also occurs for tactile processing. On the other hand, they also show that for some stimuli (spatial) the auditory cortex activates automatically, which is suggestive of a take-over by a different kind of cognitive function. The observed differences in processing of sequences might thus result from an interplay of task-specific and pluripotent plasticity.

Altered regional activity and connectivity of functional brain networks in congenital unilateral conductive hearing loss

•

Focal brain function and seed-based functional connectivity were first analyzed in congenital unilateral CHL.

•

Auditory and visual networks showed altered regional activity and functional connectivity.

•

Abnormally high activity in the left ITG was positively correlated with duration of disease.

•

Higher-order networks including frontoparietal regions and DMN demonstrated abnormal functional connectivity.

•

GSR may produce spurious signals in SBFC analyses.

Neuroimaging studies have shown marked alterations in brain function after auditory deprivation, with these alterations mainly caused by sensorineural hearing loss. To date, however, little is known about the patterns of functional brain reorganization in conductive hearing loss (CHL). The effects of congenital unilateral CHL on human brain were assessed by resting-state functional magnetic resonance imaging in 24 patients with unilateral microtia (UM) and 25 healthy controls. Focal brain function and seed-based functional connectivity were analyzed to characterize spontaneous activity and network changes in UM. Patients with UM showed common alterations in focal brain activities in the left inferior temporal gyrus across different measurements, with these alterations significantly associated with the duration of hearing loss. Additionally, focal brain activities were decreased in the auditory system and increased in the visual system, with a disassociated pattern shown in the default-mode system. Using the left inferior temporal gyrus as the seed region, patients with UM showed lower connectivity with the default-mode system and right visual regions but higher connectivity with the left frontoparietal regions when compared with controls. These results indicate that congenital partial hearing deprivation, despite normal bone conduction hearing, can induce widespread reorganizations that continue into adolescence and adulthood.

'Visual' cortices of congenitally blind adults are sensitive to response selection demands in a go/no-go task

Studies of occipital cortex plasticity in blindness provide insight into how intrinsic constraints interact with experience to determine cortical specialization. We tested the cognitive nature and anatomical origins of occipital responses during non-verbal, non-spatial auditory tasks. In a go/no-go task, congenitally blind (N=23) and sighted (N=24) individuals heard rapidly occurring (<1/s) non-verbal sounds and made one of two button presses (frequent-go 50%, infrequent-go 25%) or withheld a response (no-go, 25%). Rapid and frequent button presses heighten response selection/inhibition demands on the no-go trials: In sighted and blind adults a right-lateralized prefrontal (PFC) network responded most to no-go trials, followed by infrequent-go and finally frequent-go trials. In the blind group only, a right-lateralized occipital network showed the same response profile and the laterality of occipital and PFC responses was correlated across blind individuals. A second experiment with spoken sentences and equations (N=16) found that no-go responses in occipital cortex are distinct from previously identified occipital responses to spoken language. Finally, in resting-state data (N=30 blind, N=31 blindfolded sighted), no-go responsive 'visual' cortex of blind relative to sighted participants was more synchronized with PFC and less synchronized with primary auditory and sensory-motor cortices. No-go responsive occipital cortex showed higher resting-state correlations with no-go responsive PFC than language responsive inferior frontal cortex. We conclude that in blindness, a right-lateralized occipital network responds to non-verbal executive processes, including response selection. These results suggest that connectivity with fronto-parietal executive networks is a key mechanism for plasticity in blindness.

Visual Influences on Auditory Behavioral, Neural, and Perceptual Processes: A Review

In a naturalistic environment, auditory cues are often accompanied by information from other senses, which can be redundant with or complementary to the auditory information. Although the multisensory interactions derived from this combination of information and that shape auditory function are seen across all sensory modalities, our greatest body of knowledge to date centers on how vision influences audition. In this review, we attempt to capture the state of our understanding at this point in time regarding this topic. Following a general introduction, the review is divided into 5 sections. In the first section, we review the psychophysical evidence in humans regarding vision's influence in audition, making the distinction between vision's ability to enhance versus alter auditory performance and perception. Three examples are then described that serve to highlight vision's ability to modulate auditory processes: spatial ventriloquism, cross-modal dynamic capture, and the McGurk effect. The final part of this section discusses models that have been built based on available psychophysical data and that seek to provide greater mechanistic insights into how vision can impact audition. The second section reviews the extant neuroimaging and far-field imaging work on this topic, with a strong emphasis on the roles of feedforward and feedback processes, on imaging insights into the causal nature of audiovisual interactions, and on the limitations of current imaging-based approaches. These limitations point to a greater need for machine-learning-based decoding approaches toward understanding how auditory representations are shaped by vision. The third section reviews the wealth of neuroanatomical and neurophysiological data from animal models that highlights audiovisual interactions at the neuronal and circuit level in both subcortical and cortical structures. It also speaks to the functional significance of audiovisual interactions for two critically important facets of auditory perception-scene analysis and communication. The fourth section presents current evidence for alterations in audiovisual processes in three clinical conditions: autism, schizophrenia, and sensorineural hearing loss. These changes in audiovisual interactions are postulated to have cascading effects on higher-order domains of dysfunction in these conditions. The final section highlights ongoing work seeking to leverage our knowledge of audiovisual interactions to develop better remediation approaches to these sensory-based disorders, founded in concepts of perceptual plasticity in which vision has been shown to have the capacity to facilitate auditory learning.

Direct Structural Connections between Auditory and Visual Motion-Selective Regions in Humans

In humans, the occipital middle-temporal region (hMT⁺/V5) specializes in the processing of visual motion, while the planum temporale (hPT) specializes in auditory motion processing. It has been hypothesized that these regions might communicate directly to achieve fast and optimal exchange of multisensory motion information. Here we investigated, for the first time in humans (male and female), the presence of direct white matter connections between visual and auditory motion-selective regions using a combined fMRI and diffusion MRI approach. We found evidence supporting the potential existence of direct white matter connections between individually and functionally defined hMT⁺/V5 and hPT. We show that projections between hMT⁺/V5 and hPT do not overlap with large white matter bundles, such as the inferior longitudinal fasciculus and the inferior frontal occipital fasciculus. Moreover, we did not find evidence suggesting the presence of projections between the fusiform face area and hPT, supporting the functional specificity of hMT⁺/V5-hPT connections. Finally, the potential presence of hMT⁺/V5-hPT connections was corroborated in a large sample of participants (n = 114) from the human connectome project. Together, this study provides a first indication for potential direct occipitotemporal projections between hMT⁺/V5 and hPT, which may support the exchange of motion information between functionally specialized auditory and visual regions.SIGNIFICANCE STATEMENT Perceiving and integrating moving signal across the senses is arguably one of the most important perceptual skills for the survival of living organisms. In order to create a unified representation of movement, the brain must therefore integrate motion information from separate senses. Our study provides support for the potential existence of direct connections between motion-selective regions in the occipital/visual (hMT⁺/V5) and temporal/auditory (hPT) cortices in humans. This connection could represent the structural scaffolding for the rapid and optimal exchange and integration of multisensory motion information. These findings suggest the existence of computationally specific pathways that allow information flow between areas that share a similar computational goal.

Cortical Activity Linked to Clocking in Deaf Adults: fNIRS Insights with Static and Animated Stimuli Presentation

The question of the possible impact of deafness on temporal processing remains unanswered. Different findings, based on behavioral measures, show contradictory results. The goal of the present study is to analyze the brain activity underlying time estimation by using functional near infrared spectroscopy (fNIRS) techniques, which allow examination of the frontal, central and occipital cortical areas. A total of 37 participants (19 deaf) were recruited. The experimental task involved processing a road scene to determine whether the driver had time to safely execute a driving task, such as overtaking. The road scenes were presented in animated format, or in sequences of 3 static images showing the beginning, mid-point, and end of a situation. The latter presentation required a clocking mechanism to estimate the time between the samples to evaluate vehicle speed. The results show greater frontal region activity in deaf people, which suggests that more cognitive effort is needed to process these scenes. The central region, which is involved in clocking according to several studies, is particularly activated by the static presentation in deaf people during the estimation of time lapses. Exploration of the occipital region yielded no conclusive results. Our results on the frontal and central regions encourage further study of the neural basis of time processing and its links with auditory capacity.

Visual motion processing recruits regions selective for auditory motion in early deaf individuals

In early deaf individuals, the auditory deprived temporal brain regions become engaged in visual processing. In our study we tested further the hypothesis that intrinsic functional specialization guides the expression of cross-modal responses in the deprived auditory cortex. We used functional MRI to characterize the brain response to horizontal, radial and stochastic visual motion in early deaf and hearing individuals matched for the use of oral or sign language. Visual motion showed enhanced response in the 'deaf' mid-lateral planum temporale, a region selective to auditory motion as demonstrated by a separate auditory motion localizer in hearing people. Moreover, multivariate pattern analysis revealed that this reorganized temporal region showed enhanced decoding of motion categories in the deaf group, while visual motion-selective region hMT+/V5 showed reduced decoding when compared to hearing people. Dynamic Causal Modelling revealed that the 'deaf' motion-selective temporal region shows a specific increase of its functional interactions with hMT+/V5 and is now part of a large-scale visual motion selective network. In addition, we observed preferential responses to radial, compared to horizontal, visual motion in the 'deaf' right superior temporal cortex region that also show preferential response to approaching/receding sounds in the hearing brain. Overall, our results suggest that the early experience of auditory deprivation interacts with intrinsic constraints and triggers a large-scale reallocation of computational load between auditory and visual brain regions that typically support the multisensory processing of motion information.

Arterial spin labeling brain MRI study to evaluate the impact of deafness on cerebral perfusion in 79 children before cochlear implantation

Age at implantation is considered to be a major factor, influencing outcomes after pediatric cochlear implantation. In the absence of acoustic input, it has been proposed that cross-modal reorganization can be detrimental for adaptation to the new electrical input provided by a cochlear implant. Here, through a retrospective study, we aimed to investigate differences in cerebral blood flow (CBF) at rest prior to implantation in children with congenital deafness compared to normally hearing children. In addition, we looked at the putative link between pre-operative rest-CBF and the oral intelligibility scores at 12 months post-implantation. Finally, we observed the evolution of perfusion with age, within brain areas showing abnormal rest-CBF associated to deafness, in deaf children and in normally hearing children. In children older than 5 years old, results showed a significant bilateral hypoperfusion in temporal regions in deaf children, particularly in Heschl's gyrus, and a significant hyperperfusion of occipital regions. Furthermore, in children older than 5 years old, whole brain voxel-by-voxel correlation analysis between pre-operative rest-CBF and oral intelligibility scores at 12 months post-implantation, showed significant negative correlation localized in the occipital regions: children who performed worse in the speech perception test one year after implantation were those presenting higher preoperative CBF values in these occipital regions. Finally, when comparing mean relative perfusion (extracted from the temporal regions found abnormal on whole-brain voxel-based analysis) across ages in patients and controls, we observed that the temporal perfusion evolution was significantly different in deaf children than in normally hearing children. Indeed, while temporal perfusion increased with age in normally hearing children, it remained stable in deaf children. We showed a critical period around 4 years old, where in the context of auditory deprivation, there is a lack of synaptic activity in auditory regions. These results support the benefits of early cochlear implantation to maximize the effectiveness of auditory rehabilitation and to avoid cross-modal reorganization.

Nonverbal auditory communication - Evidence for integrated neural systems for voice signal production and perception

While humans have developed a sophisticated and unique system of verbal auditory communication, they also share a more common and evolutionarily important nonverbal channel of voice signaling with many other mammalian and vertebrate species. This nonverbal communication is mediated and modulated by the acoustic properties of a voice signal, and is a powerful - yet often neglected - means of sending and perceiving socially relevant information. From the viewpoint of dyadic (involving a sender and a signal receiver) voice signal communication, we discuss the integrated neural dynamics in primate nonverbal voice signal production and perception. Most previous neurobiological models of voice communication modelled these neural dynamics from the limited perspective of either voice production or perception, largely disregarding the neural and cognitive commonalities of both functions. Taking a dyadic perspective on nonverbal communication, however, it turns out that the neural systems for voice production and perception are surprisingly similar. Based on the interdependence of both production and perception functions in communication, we first propose a re-grouping of the neural mechanisms of communication into auditory, limbic, and paramotor systems, with special consideration for a subsidiary basal-ganglia-centered system. Second, we propose that the similarity in the neural systems involved in voice signal production and perception is the result of the co-evolution of nonverbal voice production and perception systems promoted by their strong interdependence in dyadic interactions.

Minor hallucinations reflect early gray matter loss and predict subjective cognitive decline in Parkinson's disease

BACKGROUND AND PURPOSE

Well-structured hallucinations in Parkinson's disease (PD) are associated with poor prognosis and dementia. However, the predictive value of minor psychotic phenomena in cognitive deterioration is not well known. Cross-sectional studies have shown that PD patients with minor hallucinations have more severe cortical atrophy than non-hallucinators, but baseline and longitudinal studies addressing the evolution of these brain differences are lacking. The impact of developing minor hallucinations on cognitive impairment and cortical atrophy progression in early PD was explored.

METHODS

One hundred and thirty-one de novo PD patients from the Parkinson's Progression Marker Initiative for whom brain magnetic resonance imaging scans were available were included. Cognitive outcome at 5 years was compared between patients with and without minor hallucinations during follow-up. Additionally, using gray matter volume (GMV) voxel-based morphometry, cross-sectional (at baseline) and longitudinal (1- and 2-year GMV loss) structural brain differences between groups were studied.

RESULTS

During follow-up, 35.1% of patients developed minor hallucinations. At 5 years, these patients showed an increased prevalence of subjective cognitive decline compared to non-hallucinators (44.1% vs. 13.9%; p < 0.001), but not formal cognitive impairment. Additionally, compared to non-hallucinators, they exhibited reduced GMV at baseline in visuoperceptive areas and increased GMV loss in left temporal areas (p < 0.05 corrected).

CONCLUSIONS

Minor hallucinations seem to be an early clinical marker of increased neurodegeneration and are associated with mid-term subjective cognitive decline. Longer follow-up analyses would be needed to further define if these findings could reflect a higher risk of future cognitive deterioration.

Evidence for an effector-independent action system from people born without hands

What are the principles of brain organization? In the motor domain, separate pathways were found for reaching and grasping actions performed by the hand. To what extent is this organization specific to the hand or based on abstract action types, regardless of which body part performs them? We tested people born without hands who perform actions with their feet. Activity in frontoparietal association motor areas showed preference for an action type (reaching or grasping), regardless of whether it was performed by the foot in people born without hands or by the hand in typically-developed controls. These findings provide evidence that some association areas are organized based on abstract functions of action types, independent of specific sensorimotor experience and parameters of specific body parts.

Many parts of the visuomotor system guide daily hand actions, like reaching for and grasping objects. Do these regions depend exclusively on the hand as a specific body part whose movement they guide, or are they organized for the reaching task per se, for any body part used as an effector? To address this question, we conducted a neuroimaging study with people born without upper limbs—individuals with dysplasia—who use the feet to act, as they and typically developed controls performed reaching and grasping actions with their dominant effector. Individuals with dysplasia have no prior experience acting with hands, allowing us to control for hand motor imagery when acting with another effector (i.e., foot). Primary sensorimotor cortices showed selectivity for the hand in controls and foot in individuals with dysplasia. Importantly, we found a preference based on action type (reaching/grasping) regardless of the effector used in the association sensorimotor cortex, in the left intraparietal sulcus and dorsal premotor cortex, as well as in the basal ganglia and anterior cerebellum. These areas also showed differential response patterns between action types for both groups. Intermediate areas along a posterior–anterior gradient in the left dorsal premotor cortex gradually transitioned from selectivity based on the body part to selectivity based on the action type. These findings indicate that some visuomotor association areas are organized based on abstract action functions independent of specific sensorimotor parameters, paralleling sensory feature-independence in visual and auditory cortices in people born blind and deaf. Together, they suggest association cortices across action and perception may support specific computations, abstracted from low-level sensorimotor elements.

Review article: Structural brain alterations in prelingually deaf

Functional studies show that our brain has a remarkable ability to reorganize itself in the absence of one or more sensory modalities. In this review, we gathered all the available articles investigating structural alterations in congenitally deaf subjects. Some concentrated only on specific regions of interest (e.g., auditory areas), while others examined the whole brain. The majority of structural alterations were observed in the auditory white matter and were more pronounced in the right hemisphere. A decreased white matter volume or fractional anisotropy in the auditory areas were the most common findings in congenitally deaf subjects. Only a few studies observed alterations in the auditory grey matter. Preservation of the grey matter might be due to the cross-modal plasticity as well as due to the lack of sensitivity of methods used for microstructural alterations of grey matter. Structural alterations were also observed in the frontal, visual, and other cerebral regions as well as in the cerebellum. The observed structural brain alterations in the deaf can probably be attributed mainly to the cross-modal plasticity in the absence of sound input and use of sign instead of spoken language.

Categorical representation from sound and sight in the ventral occipito-temporal cortex of sighted and blind

Is vision necessary for the development of the categorical organization of the Ventral Occipito-Temporal Cortex (VOTC)? We used fMRI to characterize VOTC responses to eight categories presented acoustically in sighted and early blind individuals, and visually in a separate sighted group. We observed that VOTC reliably encodes sound categories in sighted and blind people using a representational structure and connectivity partially similar to the one found in vision. Sound categories were, however, more reliably encoded in the blind than the sighted group, using a representational format closer to the one found in vision. Crucially, VOTC in blind represents the categorical membership of sounds rather than their acoustic features. Our results suggest that sounds trigger categorical responses in the VOTC of congenitally blind and sighted people that partially match the topography and functional profile of the visual response, despite qualitative nuances in the categorical organization of VOTC between modalities and groups.

The world is full of rich and dynamic visual information. To avoid information overload, the human brain groups inputs into categories such as faces, houses, or tools. A part of the brain called the ventral occipito-temporal cortex (VOTC) helps categorize visual information. Specific parts of the VOTC prefer different types of visual input; for example, one part may tend to respond more to faces, whilst another may prefer houses. However, it is not clear how the VOTC characterizes information.

One idea is that similarities between certain types of visual information may drive how information is organized in the VOTC. For example, looking at faces requires using central vision, while looking at houses requires using peripheral vision. Furthermore, all faces have a roundish shape while houses tend to have a more rectangular shape. Another possibility, however, is that the categorization of different inputs cannot be explained just by vision, and is also be driven by higher-level aspects of each category. For instance, how humans use or interact with something may also influence how an input is categorized. If categories are established depending (at least partially) on these higher-level aspects, rather than purely through visual likeness, it is likely that the VOTC would respond similarly to both sounds and images representing these categories.

Now, Mattioni et al. have tested how individuals with and without sight respond to eight different categories of information to find out whether or not categorization is driven purely by visual likeness. Each category was presented to participants using sounds while measuring their brain activity. In addition, a group of participants who could see were also presented with the categories visually. Mattioni et al. then compared what happened in the VOTC of the three groups – sighted people presented with sounds, blind people presented with sounds, and sighted people presented with images – in response to each category.

The experiment revealed that the VOTC organizes both auditory and visual information in a similar way. However, there were more similarities between the way blind people categorized auditory information and how sighted people categorized visual information than between how sighted people categorized each type of input. Mattioni et al. also found that the region of the VOTC that responds to inanimate objects massively overlapped across the three groups, whereas the part of the VOTC that responds to living things was more variable.

These findings suggest that the way that the VOTC organizes information is, at least partly, independent from vision. The experiments also provide some information about how the brain reorganizes in people who are born blind. Further studies may reveal how differences in the VOTC of people with and without sight affect regions typically associated with auditory categorization, and potentially explain how the brain reorganizes in people who become blind later in life.

Stimulus arousal drives amygdalar responses to emotional expressions across sensory modalities

The factors that drive amygdalar responses to emotionally significant stimuli are still a matter of debate - particularly the proneness of the amygdala to respond to negatively-valenced stimuli has been discussed controversially. Furthermore, it is uncertain whether the amygdala responds in a modality-general fashion or whether modality-specific idiosyncrasies exist. Therefore, the present functional magnetic resonance imaging (fMRI) study systematically investigated amygdalar responding to stimulus valence and arousal of emotional expressions across visual and auditory modalities. During scanning, participants performed a gender judgment task while prosodic and facial emotional expressions were presented. The stimuli varied in stimulus valence and arousal by including neutral, happy and angry expressions of high and low emotional intensity. Results demonstrate amygdalar activation as a function of stimulus arousal and accordingly associated emotional intensity regardless of stimulus valence. Furthermore, arousal-driven amygdalar responding did not depend on the visual and auditory modalities of emotional expressions. Thus, the current results are consistent with the notion that the amygdala codes general stimulus relevance across visual and auditory modalities irrespective of valence. In addition, whole brain analyses revealed that effects in visual and auditory areas were driven mainly by high intense emotional facial and vocal stimuli, respectively, suggesting modality-specific representations of emotional expressions in auditory and visual cortices.

Neural Signatures of Working Memory in Age-related Hearing Loss

Age-related hearing loss affects the ability to hear high frequencies and therefore leads to difficulties in understanding speech, particularly under adverse listening conditions. This decrease in hearing can be partly compensated by the recruitment of executive functions, such as working memory. The compensatory effort may, however, lead to a decrease in available neural resources compromising cognitive abilities. We here aim to investigate whether mild to moderate hearing loss impacts prefrontal functions and related executive processes and whether these are related to speech-in-noise perception abilities. Nineteen hard of hearing and nineteen age-matched normal-hearing participants performed a working memory task to drive prefrontal activity, which was gauged with functional magnetic resonance imaging. In addition, speech-in-noise understanding, cognitive flexibility and inhibition control were assessed. Our results showed no differences in frontoparietal activation patterns and working memory performance between normal-hearing and hard of hearing participants. The behavioral assessment of further executive functions, however, provided evidence of lower cognitive flexibility in hard of hearing participants. Cognitive flexibility and hearing abilities further predicted speech-in-noise perception. We conclude that neural and behavioral signatures of working memory are intact in mild to moderate hearing loss. Moreover, cognitive flexibility seems to be closely related to hearing impairment and speech-in-noise perception and should, therefore, be investigated in future studies assessing age-related hearing loss and its implications on prefrontal functions.

The Cross-Modal Effects of Sensory Deprivation on Spatial and Temporal Processes in Vision and Audition: A Systematic Review on Behavioral and Neuroimaging Research since 2000

One of the most significant effects of neural plasticity manifests in the case of sensory deprivation when cortical areas that were originally specialized for the functions of the deprived sense take over the processing of another modality. Vision and audition represent two important senses needed to navigate through space and time. Therefore, the current systematic review discusses the cross-modal behavioral and neural consequences of deafness and blindness by focusing on spatial and temporal processing abilities, respectively. In addition, movement processing is evaluated as compiling both spatial and temporal information. We examine whether the sense that is not primarily affected changes in its own properties or in the properties of the deprived modality (i.e., temporal processing as the main specialization of audition and spatial processing as the main specialization of vision). References to the metamodal organization, supramodal functioning, and the revised neural recycling theory are made to address global brain organization and plasticity principles. Generally, according to the reviewed studies, behavioral performance is enhanced in those aspects for which both the deprived and the overtaking senses provide adequate processing resources. Furthermore, the behavioral enhancements observed in the overtaking sense (i.e., vision in the case of deafness and audition in the case of blindness) are clearly limited by the processing resources of the overtaking modality. Thus, the brain regions that were previously recruited during the behavioral performance of the deprived sense now support a similar behavioral performance for the overtaking sense. This finding suggests a more input-unspecific and processing principle-based organization of the brain. Finally, we highlight the importance of controlling for and stating factors that might impact neural plasticity and the need for further research into visual temporal processing in deaf subjects.

Increased overt attention to objects in early deaf adults: An eye-tracking study of complex naturalistic scenes

The study of selective attention in people with profound deafness has repeatedly documented enhanced attention to the peripheral regions of the visual field compared to hearing controls. This finding emerged from covert attention studies (i.e., without eye-movements) involving extremely simplified visual scenes and comprising few visual items. In this study, we aimed to test whether this key finding extends also to overt attention, using a more ecologically valid experimental context in which complex naturalistic images were presented for 3 s. In Experiment 1 (N = 35), all images contained a single central object superimposed on a congruent naturalistic background (e.g., a tiger in the woods). At the end of the visual exploration phase, an incidental memory task probed the participants' recollection of the seen central objects and image backgrounds. Results showed that hearing controls explored and remembered the image backgrounds more than deaf participants, who lingered on the central object to a greater extent. In Experiment 2 we aimed to disentangle if this behaviour of deaf participants reflected a bias in overt space-based attention towards the centre of the image, or instead, enhanced object-centred attention. We tested new participants (N = 42) in the visual exploration task adding images with lateralized objects, as well as images with multiple object or images without any object. Results confirmed increased exploration of objects in deaf participants. Taken together our novel findings show limitations of the well-known peripheral attention bias of deaf people and suggest that visual object-centred attention may also change after prolonged auditory deprivation.

Functional hierarchy for tactile processing in the visual cortex of sighted adults

Perception via different sensory modalities was traditionally believed to be supported by largely separate brain systems. However, a growing number of studies demonstrate that the visual cortices of typical, sighted adults are involved in tactile and auditory perceptual processing. Here, we investigated the spatiotemporal dynamics of the visual cortex's involvement in a complex tactile task: Braille letter recognition. Sighted subjects underwent Braille training and then participated in a transcranial magnetic stimulation (TMS) study in which they tactually identified single Braille letters. During this task, TMS was applied to their left early visual cortex, visual word form area (VWFA), and left early somatosensory cortex at five time windows from 20 to 520 ms following the Braille letter presentation's onset. The subjects' response accuracy decreased when TMS was applied to the early visual cortex at the 120-220 ms time window and when TMS was applied to the VWFA at the 320-420 ms time window. Stimulation of the early somatosensory cortex did not have a time-specific effect on the accuracy of the subjects' Braille letter recognition, but rather caused a general slowdown during this task. Our results indicate that the involvement of sighted people's visual cortices in tactile perception respects the canonical visual hierarchy-the early tactile processing stages involve the early visual cortex, whereas more advanced tactile computations involve high-level visual areas. Our findings are compatible with the metamodal account of brain organization and suggest that the whole visual cortex may potentially support spatial perception in a task-specific, sensory-independent manner.

What and How the Deaf Brain Sees

Over the past decade, there has been an unprecedented level of interest and progress into understanding visual processing in the brain of the deaf. Specifically, when the brain is deprived of input from one sensory modality (such as hearing), it often compensates with supranormal performance in one or more of the intact sensory systems (such as vision). Recent psychophysical, functional imaging, and reversible deactivation studies have converged to define the specific visual abilities that are enhanced in the deaf, as well as the cortical loci that undergo crossmodal plasticity in the deaf and are responsible for mediating these superior visual functions. Examination of these investigations reveals that central visual functions, such as object and facial discrimination, and peripheral visual functions, such as motion detection, visual localization, visuomotor synchronization, and Vernier acuity (measured in the periphery), are specifically enhanced in the deaf, compared with hearing participants. Furthermore, the cortical loci identified to mediate these functions reside in deaf auditory cortex: BA 41, BA 42, and BA 22, in addition to the rostral area, planum temporale, Te3, and temporal voice area in humans; primary auditory cortex, anterior auditory field, dorsal zone of auditory cortex, auditory field of the anterior ectosylvian sulcus, and posterior auditory field in cats; and primary auditory cortex and anterior auditory field in both ferrets and mice. Overall, the findings from these studies show that crossmodal reorganization in auditory cortex of the deaf is responsible for the superior visual abilities of the deaf.

Spatial Cues Influence Time Estimations in Deaf Individuals

Recent studies have reported a strong interaction between spatial and temporal representation when visual experience is missing: blind people use temporal representation of events to represent spatial metrics. Given the superiority of audition on time perception, we hypothesized that when audition is not available complex temporal representations could be impaired, and spatial representation of events could be used to build temporal metrics. To test this hypothesis, deaf and hearing subjects were tested with a visual temporal task where conflicting and not conflicting spatiotemporal information was delivered. As predicted, we observed a strong deficit of deaf participants when only temporal cues were useful and space was uninformative with respect to time. However, the deficit disappeared when coherent spatiotemporal cues were presented and increased for conflicting spatiotemporal stimuli. These results highlight that spatial cues influence time estimations in deaf participants, suggesting that deaf individuals use spatial information to infer temporal environmental coordinates.

Neuronal Development of Hearing and Language: Cochlear Implants and Critical Periods

The modern cochlear implant (CI) is the most successful neural prosthesis developed to date. CIs provide hearing to the profoundly hearing impaired and allow the acquisition of spoken language in children born deaf. Results from studies enabled by the CI have provided new insights into ( a) minimal representations at the periphery for speech reception, ( b) brain mechanisms for decoding speech presented in quiet and in acoustically adverse conditions, ( c) the developmental neuroscience of language and hearing, and ( d) the mechanisms and time courses of intramodal and cross-modal plasticity. Additionally, the results have underscored the interconnectedness of brain functions and the importance of top-down processes in perception and learning. The findings are described in this review with emphasis on the developing brain and the acquisition of hearing and spoken language.

Deaf Children as ‘English Learners’: The Psycholinguistic Turn in Deaf Education

The purpose of this literature review is to present the arguments in support of conceptualizing deaf children as ‘English Learners’, to explore the educational implications of such conceptualizations, and to suggest directions for future inquiry. Three ways of interpreting the label ‘English Learner’ in relationship to deaf children are explored: (1) as applied to deaf children whose native language is American Sign Language; (2) as applied to deaf children whose parents speak a language other than English; and (3) as applied to deaf children who have limited access to the spoken English used by their parents. Recent research from the fields of linguistics and neuroscience on the effects of language deprivation is presented and conceptualized within a framework that we refer to as the psycholinguistic turn in deaf education. The implications for developing the literacy skills of signing deaf children are explored, particularly around the theoretical construct of a ‘bridge’ between sign language proficiency and print-based literacy. Finally, promising directions for future inquiry are presented.

Cross-modal representations in early visual and auditory cortices revealed by multi-voxel pattern analysis

Primary sensory cortices can respond not only to their defined sensory modality but also to cross-modal information. In addition to the observed cross-modal phenomenon, it is valuable to research further whether cross-modal information can be valuable for categorizing stimuli and what effect other factors, such as experience and imagination, may have on cross-modal processing. In this study, we researched cross-modal information processing in the early visual cortex (EVC, including the visual area 1, 2, and 3 (V1, V2, and V3)) and auditory cortex (primary (A1) and secondary (A2) auditory cortex). Images and sound clips were presented to participants separately in two experiments in which participants' imagination and expectations were restricted by an orthogonal fixation task and the data were collected by functional magnetic resonance imaging (fMRI). We successfully decoded categories of the cross-modal stimuli in the ROIs except for V1 by multi-voxel pattern analysis (MVPA). It was further shown that familiar sounds had the advantage of classification accuracies in V2 and V3 when compared with unfamiliar sounds. The results of the cross-classification analysis showed that there was no significant similarity between the activity patterns induced by different stimulus modalities. Even though the cross-modal representation is robust when considering the restriction of top-down expectations and mental imagery in our experiments, the sound experience showed effects on cross-modal representation in V2 and V3. In addition, primary sensory cortices may receive information from different modalities in different ways, so the activity patterns between two modalities were not similar enough to complete the cross-classification successfully.

Orthographic Priming in Braille Reading as Evidence for Task-specific Reorganization in the Ventral Visual Cortex of the Congenitally Blind

The task-specific principle asserts that, following deafness or blindness, the deprived cortex is reorganized in a manner such that the task of a given area is preserved even though its input modality has been switched. Accordingly, tactile reading engages the ventral occipitotemporal cortex (vOT) in the blind in a similar way to regular reading in the sighted. Others, however, show that the vOT of the blind processes spoken sentence structure, which suggests that the task-specific principle might not apply to vOT. The strongest evidence for the vOT's engagement in sighted reading comes from orthographic repetition-suppression studies. Here, congenitally blind adults were tested in an fMRI repetition-suppression paradigm. Results reveal a double dissociation, with tactile orthographic priming in the vOT and auditory priming in general language areas. Reconciling our finding with other evidence, we propose that the vOT in the blind serves multiple functions, one of which, orthographic processing, overlaps with its function in the sighted.

Dissociation between Cerebellar and Cerebral Neural Activities in Humans with Long-Term Bilateral Sensorineural Hearing Loss

Abnormal neural activity in the cerebellum has been implicated in hearing impairments, but the effects of long-term hearing loss on cerebellar function are poorly understood. To further explore the role of long-term bilateral sensorineural hearing loss on cerebellar function, we investigated hearing loss-induced changes among neural networks within cerebellar subregions and the changes in cerebellar-cerebral connectivity patterns using resting-state functional MRI. Twenty-one subjects with long-term bilateral moderate-to-severe sensorineural hearing loss and 21 matched controls with clinically normal hearing underwent MRI scanning and a series of neuropsychological tests targeting cognition and emotion. Voxel-wise functional connectivity (FC) analysis demonstrated decreased couplings between the cerebellum and other cerebral areas, including the temporal pole (TP), insula, supramarginal gyrus, inferior frontal gyrus (IFG), medial frontal gyrus, and thalamus, in long-term bilateral sensorineural hearing loss patients. An ROI-wise FC analysis found weakened interregional connections within cerebellar subdivisions. Moreover, there was a negative correlation between anxiety and FC between the left cerebellar lobe VI and left insula. Hearing ability and anxiety scores were also correlated with FC between the left cerebellar lobe VI and left TP, as well as the right cerebellar lobule VI and left IFG. Our results suggest that sensorineural hearing loss disrupts cerebellar-cerebral circuits, some potentially linked to anxiety, and interregional cerebellar connectivity. The findings contribute to a growing body showing that auditory deprivation caused by cochlear hearing loss disrupts not only activity with the classical auditory pathway but also portions of the cerebellum that communicates with other cortical networks.

Differences in Emotion Recognition From Body and Face Cues Between Deaf and Hearing Individuals

Deaf individuals may compensate for the lack of the auditory input by showing enhanced capacities in certain visual tasks. Here we assessed whether this also applies to recognition of emotions expressed by bodily and facial cues. In Experiment 1, we compared deaf participants and hearing controls in a task measuring recognition of the six basic emotions expressed by actors in a series of video-clips in which either the face, the body, or both the face and body were visible. In Experiment 2, we measured the weight of body and face cues in conveying emotional information when intense genuine emotions are expressed, a situation in which face expressions alone may have ambiguous valence. We found that deaf individuals were better at identifying disgust and fear from body cues (Experiment 1) and in integrating face and body cues in case of intense negative genuine emotions (Experiment 2). Our findings support the capacity of deaf individuals to compensate for the lack of the auditory input enhancing perceptual and attentional capacities in the spared modalities, showing that this capacity extends to the affective domain.