Memory for faces and objects by deaf and hearing signers and hearing nonsigners

Sign language experience has little effect on face and biomotion perception in bimodal bilinguals

Sensory and language experience can affect brain organization and domain-general abilities. For example, D/deaf individuals show superior visual perception compared to hearing controls in several domains, including the perception of faces and peripheral motion. While these enhancements may result from sensory loss and subsequent neural plasticity, they may also reflect experience using a visual-manual language, like American Sign Language (ASL), where signers must process moving hand signs and facial cues simultaneously. In an effort to disentangle these concurrent sensory experiences, we examined how learning sign language influences visual abilities by comparing bimodal bilinguals (i.e., sign language users with typical hearing) and hearing non-signers. Bimodal bilinguals and hearing non-signers completed online psychophysical measures of face matching and biological motion discrimination. No significant group differences were observed across these two tasks, suggesting that sign language experience is insufficient to induce perceptual advantages in typical-hearing adults. However, ASL proficiency (but not years of experience or age of acquisition) was found to predict performance on the motion perception task among bimodal bilinguals. Overall, the results presented here highlight a need for more nuanced study of how linguistic environments, sensory experience, and cognitive functions impact broad perceptual processes and underlying neural correlates.

Working Memory for Faces among Individuals with Congenital Deafness

BACKGROUND

 Studies examining face processing among individuals with congenital deafness show inconsistent results that are often accounted for by sign language skill. However, working memory for faces as an aspect of face processing has not yet been examined in congenital deafness.

PURPOSE

 To explore working memory for faces among individuals with congenital deafness who are skilled in sign language.

RESEARCH DESIGN

 A quasi-experimental study of individuals with congenital deafness and a control group.

STUDY SAMPLE

 Sixteen individuals with congenital deafness who are skilled in sign language and 18 participants with intact hearing, matched for age, and education.

INTERVENTION

 The participants performed two conditions of the N-back test in ascending difficulty (i.e., 1-back and 2-back).

DATA COLLECTION AND ANALYSIS

 Levene's and Shapiro-Wilk tests were used to assess group homoscedasticity and normality, respectively. A two-way repeated measures analysis of variance was applied to compare the groups in response time and accuracy of the N-back test, as well as Pearson correlation between response time and accuracy, and sign language skill duration.

RESULTS

 The congenital deafness group performed better than controls, as was found in the response time but not in the accuracy variables. However, an interaction effect showed that this pattern was significant for the 1-back but not for the 2-back condition in the response time but not the accuracy. Further, there was a marginal effect in response time but a significant one in accuracy showing the 2-back was performed worse than the 1-back. No significant correlation was found between response time and accuracy, and sign language skill duration.

CONCLUSION

 Face processing advantage associated with congenital deafness is dependent on cognitive load, but sign language duration does not affect this trend. In addition, response time and accuracy are not equally sensitive to performance differences in the N-back test.

Evidence for superior encoding of detailed visual memories in deaf signers

Recent evidence shows that deaf signers outperform hearing non-signers in some tests of visual attention and discrimination. Furthermore, they can retain visual information better over short periods, i.e., seconds. However, it is unknown if deaf signers’ retention of detailed visual information is superior following more extended periods. We report a study investigating this possibility. Our data revealed that deaf individuals outperformed hearing people in a visual long-term memory test that probed the fine detail of new memories. Deaf individuals also performed better in a scene-discrimination test, which correlated positively with performance on the long-term memory test. Our findings provide evidence that deaf signers can demonstrate superior visual long-term memory, possibly because of enhanced visual attention during encoding. The relative contributions of factors including sign language fluency, protracted practice, and neural plasticity are still to be established. Our findings add to evidence showing that deaf signers are at an advantage in some respects, including the retention of detailed visual memories over the longer term.

Brain Morphological Modifications in Congenital and Acquired Auditory Deprivation: A Systematic Review and Coordinate-Based Meta-Analysis

Neuroplasticity following deafness has been widely demonstrated in both humans and animals, but the anatomical substrate of these changes is not yet clear in human brain. However, it is of high importance since hearing loss is a growing problem due to aging population. Moreover, knowing these brain changes could help to understand some disappointing results with cochlear implant, and therefore could improve hearing rehabilitation. A systematic review and a coordinate-based meta-analysis were realized about the morphological brain changes highlighted by MRI in severe to profound hearing loss, congenital and acquired before or after language onset. 25 papers were included in our review, concerning more than 400 deaf subjects, most of them presenting prelingual deafness. The most consistent finding is a volumetric decrease in gray matter around bilateral auditory cortex. This change was confirmed by the coordinate-based meta-analysis which shows three converging clusters in this region. The visual areas of deaf children is also significantly impacted, with a decrease of the volume of both gray and white matters. Finally, deafness is responsible of a gray matter increase within the cerebellum, especially at the right side. These results are largely discussed and compared with those from deaf animal models and blind humans, which demonstrate for example a much more consistent gray matter decrease along their respective primary sensory pathway. In human deafness, a lot of other factors than deafness could interact on the brain plasticity. One of the most important is the use of sign language and its age of acquisition, which induce among others changes within the hand motor region and the visual cortex. But other confounding factors exist which have been too little considered in the current literature, such as the etiology of the hearing impairment, the speech-reading ability, the hearing aid use, the frequent associated vestibular dysfunction or neurocognitive impairment. Another important weakness highlighted by this review concern the lack of papers about postlingual deafness, whereas it represents most of the deaf population. Further studies are needed to better understand these issues, and finally try to improve deafness rehabilitation.

Deprivation of Auditory Experience Influences Numerosity Discrimination, but Not Numerosity Estimation

Number sense is the ability to estimate the number of items, and it is common to many species. Despite the numerous studies dedicated to unveiling how numerosity is processed in the human brain, to date, it is not clear whether the representation of numerosity is supported by a single general mechanism or by multiple mechanisms. Since it is known that deafness entails a selective impairment in the processing of temporal information, we assessed the approximate numerical abilities of deaf individuals to disentangle these two hypotheses. We used a numerosity discrimination task (2AFC) and an estimation task, in both cases using sequential (temporal) or simultaneous (spatial) stimuli. The results showed a selective impairment of the deaf participants compared with the controls (hearing) in the temporal numerosity discrimination task, while no difference was found to discriminate spatial numerosity. Interestingly, the deaf and hearing participants did not differ in spatial or temporal numerosity estimation. Overall, our results suggest that the deficit in temporal processing induced by deafness also impacts perception in other domains such as numerosity, where sensory information is conveyed in a temporal format, which further suggests the existence of separate mechanisms subserving the processing of temporal and spatial numerosity.

Cross-modal connectivity effects in age-related hearing loss

Age-related sensorineural hearing loss (HL) leads to localized brain changes in the primary auditory cortex, long-range functional alterations, and is considered a risk factor for dementia. Nonhuman studies have repeatedly highlighted cross-modal brain plasticity in sensorial brain networks other than those primarily involved in the peripheral damage, thus in this study, the possible cortical alterations associated with HL have been analyzed using a whole-brain multimodal connectomic approach. Fifty-two HL and 30 normal hearing participants were examined in a 3T MRI study along with audiological and neurological assessments. Between-regions functional connectivity and whole-brain probabilistic tractography were calculated in a connectome-based manner and graph theory was used to obtain low-dimensional features for the analysis of brain connectivity at global and local levels. The HL condition was associated with a different functional organization of the visual subnetwork as revealed by a significant increase in global efficiency, density, and clustering coefficient. These functional effects were mirrored by similar (but more subtle) structural effects suggesting that a functional repurposing of visual cortical centers occurs to compensate for age-related loss of hearing abilities.

Supra-normal skills in processing of visuo-auditory prosodic information by cochlear-implanted deaf patients

Cochlear implanted (CI) adults with acquired deafness are known to depend on multisensory integration skills (MSI) for speech comprehension through the fusion of speech reading skills and their deficient auditory perception. But, little is known on how CI patients perceive prosodic information relating to speech content. Our study aimed to identify how CI patients use MSI between visual and auditory information to process paralinguistic prosodic information of multimodal speech and the visual strategies employed. A psychophysics assessment was developed, in which CI patients and hearing controls (NH) had to distinguish between a question and a statement. The controls were separated into two age groups (young and aged-matched) to dissociate any effect of aging. In addition, the oculomotor strategies used when facing a speaker in this prosodic decision task were recorded using an eye-tracking device and compared to controls. This study confirmed that prosodic processing is multisensory but it revealed that CI patients showed significant supra-normal audiovisual integration for prosodic information compared to hearing controls irrespective of age. This study clearly showed that CI patients had a visuo-auditory gain more than 3 times larger than that observed in hearing controls. Furthermore, CI participants performed better in the visuo-auditory situation through a specific oculomotor exploration of the face as they significantly fixate the mouth region more than young NH participants who fixate the eyes, whereas the aged-matched controls presented an intermediate exploration pattern equally reported between the eyes and mouth. To conclude, our study demonstrated that CI patients have supra-normal skills MSI when integrating visual and auditory linguistic prosodic information, and a specific adaptive strategy developed as it participates directly in speech content comprehension.

Cross-modal plasticity and central deficiencies: the case of deafness and the use of cochlear implants

The primary objective of this chapter is to describe the consequences of central deficiencies on the neurodevelopment of children. We approach this topic from the standpoint of congenital deafness. Thus we first present the current state of knowledge on cortical reorganization following congenital deafness. The allocation of auditory cortices to other sensory systems can enhance sensory processing and therefore the cognitive functions related to them. Second, we explore the linguistic development of deaf children. Given that the English written system is speech-based, its acquisition is complex and atypical for deaf children, usually leading to poorer achievements. Next, we explore the impact of a neural prosthesis named the cochlear implant on the neurocognitive and linguistic development of deaf children. In some cases, it allows the individuals to, at least partially, regain access to the lost sense. We also comment on the specific needs of the deaf population when it comes to neuropsychological assessment. Finally, we touch on the specific context of deaf children born of deaf parents, and therefore naturally exposed to sign language as the only means of communication.

The Impact of Early Deafness on Brain Plasticity: A Systematic Review of the White and Gray Matter Changes

Background: Auditory deprivation alters cortical and subcortical brain regions, primarily linked to auditory and language processing, resulting in behavioral consequences. Neuroimaging studies have reported various degrees of structural changes, yet multiple variables in deafness profiles need to be considered for proper interpretation of results. To date, many inconsistencies are reported in the gray and white matter alterations following early profound deafness. The purpose of this study was to provide the first systematic review synthesizing gray and white matter changes in deaf individuals.

Methods: We conducted a systematic review according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement in 27 studies comprising 626 deaf individuals.

Results: Evidence shows that auditory deprivation significantly alters the white matter across the primary and secondary auditory cortices. The most consistent alteration across studies was in the bilateral superior temporal gyri. Furthermore, reductions in the fractional anisotropy of white matter fibers comprising in inferior fronto-occipital fasciculus, the superior longitudinal fasciculus, and the subcortical auditory pathway are reported. The reviewed studies also suggest that gray and white matter integrity is sensitive to early sign language acquisition, attenuating the effect of auditory deprivation on neurocognitive development.

Conclusions: These findings suggest that understanding cortical reorganization through gray and white matter changes in auditory and non-auditory areas is an important factor in the development of auditory rehabilitation strategies in the deaf population.

Hemispheric Asymmetries in Deaf and Hearing During Sustained Peripheral Selective Attention

Previous studies have shown that compared to hearing individuals, early deaf individuals allocate relatively more attention to the periphery than central visual field. However, it is not clear whether these two groups also differ in their ability to selectively attend to specific peripheral locations. We examined deaf and hearing participants' selective attention using electroencephalography (EEG) and a frequency tagging paradigm, in which participants attended to one of two peripheral displays of moving dots that changed directions at different rates. Both participant groups showed similar amplifications and reductions in the EEG signal at the attended and unattended frequencies, indicating similar control over their peripheral attention for motion stimuli. However, for deaf participants these effects were larger in a right hemispheric region of interest (ROI), while for hearing participants these effects were larger in a left ROI. These results contribute to a growing body of evidence for a right hemispheric processing advantage in deaf populations when attending to motion.

Attention Dynamics During Emotion Recognition by Deaf and Hearing Individuals

The enhancement hypothesis suggests that deaf individuals are more vigilant to visual emotional cues than hearing individuals. The present eye-tracking study examined ambient-focal visual attention when encoding affect from dynamically changing emotional facial expressions. Deaf (n = 17) and hearing (n = 17) individuals watched emotional facial expressions that in 10-s animations morphed from a neutral expression to one of happiness, sadness, or anger. The task was to recognize emotion as quickly as possible. Deaf participants tended to be faster than hearing participants in affect recognition, but the groups did not differ in accuracy. In general, happy faces were more accurately and more quickly recognized than faces expressing anger or sadness. Both groups demonstrated longer average fixation duration when recognizing happiness in comparison to anger and sadness. Deaf individuals directed their first fixations less often to the mouth region than the hearing group. During the last stages of emotion recognition, deaf participants exhibited more focal viewing of happy faces than negative faces. This pattern was not observed among hearing individuals. The analysis of visual gaze dynamics, switching between ambient and focal attention, was useful in studying the depth of cognitive processing of emotional information among deaf and hearing individuals.

Visual field plasticity in hearing users of sign language

Studies have observed that deaf signers have a larger Visual Field (VF) than hearing non-signers with a particular large extension in the lower part of the VF. This increment could stem from early deafness or from the extensive use of sign language, since the lower VF is critical to perceive and understand linguistics gestures in sign language communication. The aim of the present study was to explore the potential impact of sign language experience without deafness on the VF sensitivity within its lower part. Using standard Humphrey Visual Field Analyzer, we compared luminance sensitivity in the fovea and between 3 and 27 degrees of visual eccentricity for the upper and lower VF, between hearing users of French Sign Language and age-matched hearing non-signers. The sensitivity in the fovea and in the upper VF were similar in both groups. Hearing signers had, however, higher luminance sensitivity than non-signers in the lower VF but only between 3 and 15°, the visual location for sign language perception. Sign language experience, no associated with deafness, may then be a modulating factor of VF sensitivity but restricted to the very specific location where signs are perceived.

Face Recognition is Shaped by the Use of Sign Language

Previous research has suggested that early deaf signers differ in face processing. Which aspects of face processing are changed and the role that sign language may have played in that change are however unclear. Here, we compared face categorization (human/non-human) and human face recognition performance in early profoundly deaf signers, hearing signers, and hearing non-signers. In the face categorization task, the three groups performed similarly in term of both response time and accuracy. However, in the face recognition task, signers (both deaf and hearing) were slower than hearing non-signers to accurately recognize faces, but had a higher accuracy rate. We conclude that sign language experience, but not deafness, drives a speed-accuracy trade-off in face recognition (but not face categorization). This suggests strategic differences in the processing of facial identity for individuals who use a sign language, regardless of their hearing status.

Functional selectivity for face processing in the temporal voice area of early deaf individuals

Brain systems supporting face and voice processing both contribute to the extraction of important information for social interaction (e.g., person identity). How does the brain reorganize when one of these channels is absent? Here, we explore this question by combining behavioral and multimodal neuroimaging measures (magneto-encephalography and functional imaging) in a group of early deaf humans. We show enhanced selective neural response for faces and for individual face coding in a specific region of the auditory cortex that is typically specialized for voice perception in hearing individuals. In this region, selectivity to face signals emerges early in the visual processing hierarchy, shortly after typical face-selective responses in the ventral visual pathway. Functional and effective connectivity analyses suggest reorganization in long-range connections from early visual areas to the face-selective temporal area in individuals with early and profound deafness. Altogether, these observations demonstrate that regions that typically specialize for voice processing in the hearing brain preferentially reorganize for face processing in born-deaf people. Our results support the idea that cross-modal plasticity in the case of early sensory deprivation relates to the original functional specialization of the reorganized brain regions.

Adaptive and maladaptive neural compensatory consequences of sensory deprivation-From a phantom percept perspective

It is suggested that the brain undergoes plastic changes in order to adapt to changing environmental needs. Sensory deprivation results in decreased input to the brain leading to adaptive or maladaptive changes. Although several theories hypothesize the mechanism of these adaptive and maladaptive changes, the course of action taken by the brain heavily depends on the age of incidence of damage. The growing body of literature on the topic proposes that maladaptive changes in the brain are instrumental in creating phantom percepts, defined as the perception of a sensory experience in the absence of a physical stimulus. The current article reviews the mechanisms of adaptive and maladaptive plasticity in the brain in congenital, early, and late-onset sensory deprivation in conjunction with the phantom percepts in the different sensory domains. We propose that the mechanisms of adaptive and maladaptive plasticity fall under a universal construct of updating hierarchical Bayesian prediction errors. This theory of the Bayesian brain hypothesizes that the brain constantly compares its internal milieu with changing environmental cues and either adjusts its predictions or discards the change, depending on the novelty or salience of the external stimulus. We propose that adaptive plasticity reflects both successful bottom-up compensation and top-down updating of the model while maladaptive plasticity reflects failure in one or both mechanisms, resulting in a constant prediction-error. Finally, we hypothesize that phantom percepts are generated by the brain as a solution to this prediction error and are thus a manifestation of unsuccessful adaptation to sensory deprivation.

Don't Assume Deaf Students are Visual Learners

In the education of deaf learners, from primary school to postsecondary settings, it frequently is suggested that deaf students are visual learners. That assumption appears to be based on the visual nature of signed languages-used by some but not all deaf individuals-and the fact that with greater hearing losses, deaf students will rely relatively more on vision than audition. However, the questions of whether individuals with hearing loss are more likely to be visual learners than verbal learners or more likely than hearing peers to be visual learners have not been empirically explored. Several recent studies, in fact, have indicated that hearing learners typically perform as well or better than deaf learners on a variety of visual-spatial tasks. The present study used two standardized instruments to examine learning styles among college deaf students who primarily rely on sign language or spoken language and their hearing peers. The visual-verbal dimension was of particular interest. Consistent with recent indirect findings, results indicated that deaf students are no more likely than hearing students to be visual learners and are no stronger in their visual skills and habits than their verbal skills and habits, nor are deaf students' visual orientations associated with sign language skills. The results clearly have specific implications for the educating of deaf learners.

A visual processing advantage for young-adolescent deaf observers: Evidence from face and object matching tasks

It is unresolved whether the permanent auditory deprivation that deaf people experience leads to the enhanced visual processing of faces. The current study explored this question with a matching task in which observers searched for a target face among a concurrent lineup of ten faces. This was compared with a control task in which the same stimuli were presented upside down, to disrupt typical face processing, and an object matching task. A sample of young-adolescent deaf observers performed with higher accuracy than hearing controls across all of these tasks. These results clarify previous findings and provide evidence for a general visual processing advantage in deaf observers rather than a face-specific effect.

Modifications of Visual Field Asymmetries for Face Categorization in Early Deaf Adults: A Study With Chimeric Faces

Right hemisphere lateralization for face processing is well documented in typical populations. At the behavioral level, this right hemisphere bias is often related to a left visual field (LVF) bias. A conventional mean to study this phenomenon consists of using chimeric faces that are composed of the left and right parts of two faces. In this paradigm, participants generally use the left part of the chimeric face, mostly processed through the right optic tract, to determine its identity, gender or age. To assess the impact of early auditory deprivation on face processing abilities, we tested the LVF bias in a group of early deaf participants and hearing controls. In two experiments, deaf and hearing participants performed a gender categorization task with chimeric and normal average faces. Over the two experiments the results confirmed the presence of a LVF bias in participants, which was less frequent in deaf participants. This result suggested modifications of hemispheric lateralization for face processing in deaf participants. In Experiment 2 we also recorded eye movements to examine whether the LVF bias could be related to face scanning behavior. In this second study, participants performed a similar task while we recorded eye movements using an eye tracking system. Using areas of interest analysis we observed that the proportion of fixations on the mouth relatively to the other areas was increased in deaf participants in comparison with the hearing group. This was associated with a decrease of the proportion of fixations on the eyes. In addition these measures were correlated to the LVF bias suggesting a relationship between the LVF bias and the patterns of facial exploration. Taken together, these results suggest that early auditory deprivation results in plasticity phenomenon affecting the perception of static faces through modifications of hemispheric lateralization and of gaze behavior.

Enhanced spontaneous functional connectivity of the superior temporal gyrus in early deafness

Early auditory deprivation may drive the auditory cortex into cross-modal processing of non-auditory sensory information. In a recent study, we had shown that early deaf subjects exhibited increased activation in the superior temporal gyrus (STG) bilaterally during visual spatial working memory; however, the changes in the organization of the STG related spontaneous functional network, and their cognitive relevance are still unknown. To clarify this issue, we applied resting state functional magnetic resonance imaging on 42 early deafness (ED) and 40 hearing controls (HC). We also acquired the visual spatial and numerical n-back working memory (WM) information in these subjects. Compared with hearing subjects, the ED exhibited faster reaction time of visual WM tasks in both spatial and numerical domains. Furthermore, ED subjects exhibited significantly increased functional connectivity between the STG (especially of the right hemisphere) and bilateral anterior insula and dorsal anterior cingulated cortex. Finally, the functional connectivity of STG could predict visual spatial WM performance, even after controlling for numerical WM performance. Our findings suggest that early auditory deprivation can strengthen the spontaneous functional connectivity of STG, which may contribute to the cross-modal involvement of this region in visual working memory.

Cross-modal reorganization in cochlear implant users: Auditory cortex contributes to visual face processing

There is converging evidence that the auditory cortex takes over visual functions during a period of auditory deprivation. A residual pattern of cross-modal take-over may prevent the auditory cortex to adapt to restored sensory input as delivered by a cochlear implant (CI) and limit speech intelligibility with a CI. The aim of the present study was to investigate whether visual face processing in CI users activates auditory cortex and whether this has adaptive or maladaptive consequences. High-density electroencephalogram data were recorded from CI users (n=21) and age-matched normal hearing controls (n=21) performing a face versus house discrimination task. Lip reading and face recognition abilities were measured as well as speech intelligibility. Evaluation of event-related potential (ERP) topographies revealed significant group differences over occipito-temporal scalp regions. Distributed source analysis identified significantly higher activation in the right auditory cortex for CI users compared to NH controls, confirming visual take-over. Lip reading skills were significantly enhanced in the CI group and appeared to be particularly better after a longer duration of deafness, while face recognition was not significantly different between groups. However, auditory cortex activation in CI users was positively related to face recognition abilities. Our results confirm a cross-modal reorganization for ecologically valid visual stimuli in CI users. Furthermore, they suggest that residual takeover, which can persist even after adaptation to a CI is not necessarily maladaptive.

Cross-modal activation of auditory regions during visuo-spatial working memory in early deafness

Early deafness can reshape deprived auditory regions to enable the processing of signals from the remaining intact sensory modalities. Cross-modal activation has been observed in auditory regions during non-auditory tasks in early deaf subjects. In hearing subjects, visual working memory can evoke activation of the visual cortex, which further contributes to behavioural performance. In early deaf subjects, however, whether and how auditory regions participate in visual working memory remains unclear. We hypothesized that auditory regions may be involved in visual working memory processing and activation of auditory regions may contribute to the superior behavioural performance of early deaf subjects. In this study, 41 early deaf subjects (22 females and 19 males, age range: 20-26 years, age of onset of deafness < 2 years) and 40 age- and gender-matched hearing controls underwent functional magnetic resonance imaging during a visuo-spatial delayed recognition task that consisted of encoding, maintenance and recognition stages. The early deaf subjects exhibited faster reaction times on the spatial working memory task than did the hearing controls. Compared with hearing controls, deaf subjects exhibited increased activation in the superior temporal gyrus bilaterally during the recognition stage. This increased activation amplitude predicted faster and more accurate working memory performance in deaf subjects. Deaf subjects also had increased activation in the superior temporal gyrus bilaterally during the maintenance stage and in the right superior temporal gyrus during the encoding stage. These increased activation amplitude also predicted faster reaction times on the spatial working memory task in deaf subjects. These findings suggest that cross-modal plasticity occurs in auditory association areas in early deaf subjects. These areas are involved in visuo-spatial working memory. Furthermore, amplitudes of cross-modal activation during the maintenance stage were positively correlated with the age of onset of hearing aid use and were negatively correlated with the percentage of lifetime hearing aid use in deaf subjects. These findings suggest that earlier and longer hearing aid use may inhibit cross-modal reorganization in early deaf subjects. Granger causality analysis revealed that, compared to the hearing controls, the deaf subjects had an enhanced net causal flow from the frontal eye field to the superior temporal gyrus. These findings indicate that a top-down mechanism may better account for the cross-modal activation of auditory regions in early deaf subjects.See MacSweeney and Cardin (doi:10/1093/awv197) for a scientific commentary on this article.

Interpreting text messages with graphic facial expression by deaf and hearing people

In interpreting verbal messages, humans use not only verbal information but also non-verbal signals such as facial expression. For example, when a person says “yes” with a troubled face, what he or she really means appears ambiguous. In the present study, we examined how deaf and hearing people differ in perceiving real meanings in texts accompanied by representations of facial expression. Deaf and hearing participants were asked to imagine that the face presented on the computer monitor was asked a question from another person (e.g., do you like her?). They observed either a realistic or a schematic face with a different magnitude of positive or negative expression on a computer monitor. A balloon that contained either a positive or negative text response to the question appeared at the same time as the face. Then, participants rated how much the individual on the monitor really meant it (i.e., perceived earnestness), using a 7-point scale. Results showed that the facial expression significantly modulated the perceived earnestness. The influence of positive expression on negative text responses was relatively weaker than that of negative expression on positive responses (i.e., “no” tended to mean “no” irrespective of facial expression) for both participant groups. However, this asymmetrical effect was stronger in the hearing group. These results suggest that the contribution of facial expression in perceiving real meanings from text messages is qualitatively similar but quantitatively different between deaf and hearing people.

The deaf utilize phonological representations in visually presented verbal memory tasks

The phonological abilities of congenitally deaf individuals are inferior to those of people who can hear. However, deaf individuals can acquire spoken languages by utilizing orthography and lip-reading. The present study used functional magnetic resonance imaging (fMRI) to show that deaf individuals utilize phonological representations via a mnemonic process. We compared the brain activation of deaf and hearing participants while they memorized serially visually presented Japanese kana letters (Kana), finger alphabets (Finger), and Arabic letters (Arabic). Hearing participants did not know which finger alphabets corresponded to which language sounds, whereas deaf participants did. All of the participants understood the correspondence between Kana and their language sounds. None of the participants knew the correspondence between Arabic and their language sounds, so this condition was used as a baseline. We found that the left superior temporal gyrus (STG) was activated by phonological representations in the deaf group when memorizing both Kana and Finger. Additionally, the brain areas associated with phonological representations for Finger in the deaf group were the same as the areas for Kana in the hearing group. Overall, despite the fact that they are superior in visual information processing, deaf individuals utilize phonological rather than visual representations in visually presented verbal memory.

Positive Clinical Neuroscience

Disorders of the brain and its sensory organs have traditionally been associated with deficits in movement, perception, cognition, emotion, and behavior. It is increasingly evident, however, that positive phenomena may also occur in such conditions, with implications for the individual, science, medicine, and for society. This article provides a selective review of such positive phenomena – enhanced function after brain lesions, better-than-normal performance in people with sensory loss, creativity associated with neurological disease, and enhanced performance associated with aging. We propose that, akin to the well-established field of positive psychology and the emerging field of positive clinical psychology, the nascent fields of positive neurology and positive neuropsychology offer new avenues to understand brain-behavior relationships, with both theoretical and therapeutic implications.

Early deafness increases the face inversion effect but does not modulate the composite face effect

Early deprivation in audition can have striking effects on the development of visual processing. Here we investigated whether early deafness induces changes in holistic/configural face processing. To this end, we compared the results of a group of early deaf participants to those of a group of hearing participants in an inversion-matching task (Experiment 1) and a composite face task (Experiment 2). We hypothesized that deaf individuals would show an enhanced inversion effect and/or an increased composite face effect compared to hearing controls in case of enhanced holistic/configural face processing. Conversely, these effects would be reduced if they rely more on facial features than hearing controls. As a result, we found that deaf individuals showed an increased inversion effect for faces, but not for non-face objects. They were also significantly slower than hearing controls to match inverted faces. However, the two populations did not differ regarding the overall size of their composite face effect. Altogether these results suggest that early deafness does not enhance or reduce the amount of holistic/configural processing devoted to faces but may increase the dependency on this mode of processing.

Using brain imaging to track problem solving in a complex state space

This paper describes how behavioral and imaging data can be combined with a Hidden Markov Model (HMM) to track participants' trajectories through a complex state space. Participants completed a problem-solving variant of a memory game that involved 625 distinct states, 24 operators, and an astronomical number of paths through the state space. Three sources of information were used for classification purposes. First, an Imperfect Memory Model was used to estimate transition probabilities for the HMM. Second, behavioral data provided information about the timing of different events. Third, multivoxel pattern analysis of the imaging data was used to identify features of the operators. By combining the three sources of information, an HMM algorithm was able to efficiently identify the most probable path that participants took through the state space, achieving over 80% accuracy. These results support the approach as a general methodology for tracking mental states that occur during individual problem-solving episodes.

The role of visual experience for the neural basis of spatial cognition

Blindness often results in the adaptive neural reorganization of the remaining modalities, producing sharper auditory and haptic behavioral performance. Yet, non-visual modalities might not be able to fully compensate for the lack of visual experience as in the case of congenital blindness. For example, developmental visual experience seems to be necessary for the maturation of multisensory neurons for spatial tasks. Additionally, the ability of vision to convey information in parallel might be taken into account as the main attribute that cannot be fully compensated by the spared modalities. Therefore, the lack of visual experience might impair all spatial tasks that require the integration of inputs from different modalities, such as having to represent a set of objects on the basis of the spatial relationships among the objects, rather than the spatial relationship that each object has with oneself. Here we integrate behavioral and neural evidence to conclude that visual experience is necessary for the neural development of normal spatial cognition.

Cortical plasticity for visuospatial processing and object recognition in deaf and hearing signers

Experience-dependent plasticity in deaf participants has been shown in a variety of studies focused on either the dorsal or ventral aspects of the visual system, but both systems have never been investigated in concert. Using functional magnetic resonance imaging (fMRI), we investigated functional plasticity for spatial processing (a dorsal visual pathway function) and for object processing (a ventral visual pathway function) concurrently, in the context of differing sensory (auditory deprivation) and language (use of a signed language) experience. During scanning, deaf native users of American Sign Language (ASL), hearing native ASL users, and hearing participants without ASL experience attended to either the spatial arrangement of frames containing objects or the identity of the objects themselves. These two tasks revealed the expected dorsal/ventral dichotomy for spatial versus object processing in all groups. In addition, the object identity matching task contained both face and house stimuli, allowing us to examine category-selectivity in the ventral pathway in all three participant groups. When contrasting the groups we found that deaf signers differed from the two hearing groups in dorsal pathway parietal regions involved in spatial cognition, suggesting sensory experience-driven plasticity. Group differences in the object processing system indicated that responses in the face-selective right lateral fusiform gyrus and anterior superior temporal cortex were sensitive to a combination of altered sensory and language experience, whereas responses in the amygdala were more closely tied to sensory experience. By selectively engaging the dorsal and ventral visual pathways within participants in groups with different sensory and language experiences, we have demonstrated that these experiences affect the function of both of these systems, and that certain changes are more closely tied to sensory experience, while others are driven by the combination of sensory and language experience.

Neural reorganization following sensory loss: the opportunity of change

There is growing evidence that sensory deprivation is associated with crossmodal neuroplastic changes in the brain. After visual or auditory deprivation, brain areas that are normally associated with the lost sense are recruited by spared sensory modalities. These changes underlie adaptive and compensatory behaviours in blind and deaf individuals. Although there are differences between these populations owing to the nature of the deprived sensory modality, there seem to be common principles regarding how the brain copes with sensory loss and the factors that influence neuroplastic changes. Here, we discuss crossmodal neuroplasticity with regards to behavioural adaptation after sensory deprivation and highlight the possibility of maladaptive consequences within the context of rehabilitation.

Categorical perception of affective and linguistic facial expressions

Two experiments investigated categorical perception (CP) effects for affective facial expressions and linguistic facial expressions from American Sign Language (ASL) for Deaf native signers and hearing non-signers. Facial expressions were presented in isolation (Experiment 1) or in an ASL verb context (Experiment 2). Participants performed ABX discrimination and identification tasks on morphed affective and linguistic facial expression continua. The continua were created by morphing end-point photo exemplars into 11 images, changing linearly from one expression to another in equal steps. For both affective and linguistic expressions, hearing non-signers exhibited better discrimination across category boundaries than within categories for both experiments, thus replicating previous results with affective expressions and demonstrating CP effects for non-canonical facial expressions. Deaf signers, however, showed significant CP effects only for linguistic facial expressions. Subsequent analyses indicated that order of presentation influenced signers' response time performance for affective facial expressions: viewing linguistic facial expressions first slowed response time for affective facial expressions. We conclude that CP effects for affective facial expressions can be influenced by language experience.

Cognition and functional outcome among deaf and hearing people with schizophrenia

Recent research has highlighted the relationships between impairments in cognitive functioning and poorer functional outcomes among people with schizophrenia (PWS). The purpose of this study was to replicate and extend this work by testing the relationships between cognition and functional outcome among deaf adults with schizophrenia. Empirical findings from deafness-oriented research reveals enhanced abilities in certain aspects of visual-spatial processing compared to hearing people. Sixty-five PWS (34 deaf, 31 hearing) were assessed using measures of verbal and visual memory, attention, and visual processing. The first hypothesis tested whether cognition predicted functional outcome in a similar fashion for both deaf and hearing subjects (n=63). For all subjects, higher levels of cognitive ability were associated with higher levels of functional outcome, and the strongest predictors of outcome were verbal memory and visual-spatial memory (recall condition) (VSM recall). However, the deaf and hearing groups did show different patterns of relationships between cognition and functioning when all cognitive variables were examined. The second hypothesis was that deaf subjects would display superior performance in early visual processing, visual-spatial memory (copy condition) (VSM copy), and VSM recall. Deaf subjects displayed superior performance on each task; however, no significant differences emerged. Deaf subjects outperformed hearing subjects in an unexpected domain (word memory/recognition). This study extends prior work in the area of cognition and schizophrenia and indicates that deaf and hearing subjects may benefit from interventions that address different domains of cognition.

Do deaf individuals see better?

The possibility that, following early auditory deprivation, the remaining senses such as vision are enhanced has been met with much excitement. However, deaf individuals exhibit both better and worse visual skills than hearing controls. We show that, when deafness is considered to the exclusion of other confounds, enhancements in visual cognition are noted. The changes are not, however, widespread but are selective, limited, as we propose, to those aspects of vision that are attentionally demanding and would normally benefit from auditory-visual convergence. The behavioral changes are accompanied by a reorganization of multisensory areas, ranging from higher-order cortex to early cortical areas, highlighting cross-modal interactions as a fundamental feature of brain organization and cognitive processing.

Neural organization for recognition of grammatical and emotional facial expressions in deaf ASL signers and hearing nonsigners

Recognition of emotional facial expressions is universal for all humans, but signed language users must also recognize certain non-affective facial expressions as linguistic markers. fMRI was used to investigate the neural systems underlying recognition of these functionally distinct expressions, comparing deaf ASL signers and hearing nonsigners. Within the superior temporal sulcus (STS), activation for emotional expressions was right lateralized for the hearing group and bilateral for the deaf group. In contrast, activation within STS for linguistic facial expressions was left lateralized only for signers and only when linguistic facial expressions co-occurred with verbs. Within the fusiform gyrus (FG), activation was left lateralized for ASL signers for both expression types, whereas activation was bilateral for both expression types for nonsigners. We propose that left lateralization in FG may be due to continuous analysis of local facial features during on-line sign language processing. The results indicate that function in part drives the lateralization of neural systems that process human facial expressions.

Tactile memory of deaf-blind adults on four tasks

The performance of ten deaf-blind and ten sighted-hearing participants on four tactile memory tasks was investigated. Recognition and recall memory tasks and a matching pairs game were used. It was hypothesized that deaf-blind participants would be superior on each task. Performance was measured in terms of the time taken, and the number of items correctly recalled. In Experiments 1 and 2, which measured recognition memory in terms of the time taken to remember target items, the hypothesis was supported, but not by the length of time taken to recognize the target items, or for the number of target items correctly identified. The hypothesis was supported by Experiment 3, which measured recall memory, with regard to time taken to complete some of the tasks but not for the number of correctly recalled positions. Experiment 4, which used the matching pairs game, supported the hypothesis in terms of both time taken and the number of moves required. It is concluded that the deaf-blind people's tactile encoding is more efficient than that of sighted-hearing people, and that it is probable that their storage and retrieval are normal.