Face Recognition is Shaped by the Use of Sign Language

Sign language experience has little effect on face and biomotion perception in bimodal bilinguals

Sensory and language experience can affect brain organization and domain-general abilities. For example, D/deaf individuals show superior visual perception compared to hearing controls in several domains, including the perception of faces and peripheral motion. While these enhancements may result from sensory loss and subsequent neural plasticity, they may also reflect experience using a visual-manual language, like American Sign Language (ASL), where signers must process moving hand signs and facial cues simultaneously. In an effort to disentangle these concurrent sensory experiences, we examined how learning sign language influences visual abilities by comparing bimodal bilinguals (i.e., sign language users with typical hearing) and hearing non-signers. Bimodal bilinguals and hearing non-signers completed online psychophysical measures of face matching and biological motion discrimination. No significant group differences were observed across these two tasks, suggesting that sign language experience is insufficient to induce perceptual advantages in typical-hearing adults. However, ASL proficiency (but not years of experience or age of acquisition) was found to predict performance on the motion perception task among bimodal bilinguals. Overall, the results presented here highlight a need for more nuanced study of how linguistic environments, sensory experience, and cognitive functions impact broad perceptual processes and underlying neural correlates.

Over-reliance on English hinders cognitive science

English is the dominant language in the study of human cognition and behavior: the individuals studied by cognitive scientists, as well as most of the scientists themselves, are frequently English speakers. However, English differs from other languages in ways that have consequences for the whole of the cognitive sciences, reaching far beyond the study of language itself. Here, we review an emerging body of evidence that highlights how the particular characteristics of English and the linguistic habits of English speakers bias the field by both warping research programs (e.g., overemphasizing features and mechanisms present in English over others) and overgeneralizing observations from English speakers' behaviors, brains, and cognition to our entire species. We propose mitigating strategies that could help avoid some of these pitfalls.

Visual attention for linguistic and non-linguistic body actions in non-signing and native signing children

Evidence from adult studies of deaf signers supports the dissociation between neural systems involved in processing visual linguistic and non-linguistic body actions. The question of how and when this specialization arises is poorly understood. Visual attention to these forms is likely to change with age and be affected by prior language experience. The present study used eye-tracking methodology with infants and children as they freely viewed alternating video sequences of lexical American sign language (ASL) signs and non-linguistic body actions (self-directed grooming action and object-directed pantomime). In Experiment 1, we quantified fixation patterns using an area of interest (AOI) approach and calculated face preference index (FPI) values to assess the developmental differences between 6 and 11-month-old hearing infants. Both groups were from monolingual English-speaking homes with no prior exposure to sign language. Six-month-olds attended the signer’s face for grooming; but for mimes and signs, they were drawn to attend to the “articulatory space” where the hands and arms primarily fall. Eleven-month-olds, on the other hand, showed a similar attention to the face for all body action types. We interpret this to reflect an early visual language sensitivity that diminishes with age, just before the child’s first birthday. In Experiment 2, we contrasted 18 hearing monolingual English-speaking children (mean age of 4.8 years) vs. 13 hearing children of deaf adults (CODAs; mean age of 5.7 years) whose primary language at home was ASL. Native signing children had a significantly greater face attentional bias than non-signing children for ASL signs, but not for grooming and mimes. The differences in the visual attention patterns that are contingent on age (in infants) and language experience (in children) may be related to both linguistic specialization over time and the emerging awareness of communicative gestural acts.

Face masks affect perception of happy faces in deaf people

The SARS-CoV-2 pandemic has led significant social repercussions and forced people to wear face masks. Recent research has demonstrated that the human ability to infer emotions from facial configurations is significantly reduced when face masks are worn. Since the mouth region is specifically crucial for deaf people who speak sign language, the current study assessed the impact of face masks on inferring emotional facial expressions in a population of adult deaf signers. A group of 34 congenitally deaf individuals and 34 normal-hearing individuals were asked to identify happiness, sadness, fear, anger, and neutral expression on static human pictures with and without facial masks presented through smartphones. For each emotion, the percentage of correct responses with and without face masks was calculated and compared between groups. Results indicated that face masks, such as those worn due to the SARS-CoV-2 pandemic, limit the ability of people to infer emotions from facial expressions. The negative impact of face masks is significantly pronounced when deaf people have to recognize low-intensity expressions of happiness. These findings are of essential importance because difficulties in recognizing emotions from facial expressions due to mask wearing may contribute to the communication challenges experienced by the deaf community during the SARS-CoV-2 pandemic, generating feelings of frustration and exclusion.

How sign language expertise can influence the effects of face masks on non-linguistic characteristics

Face masks occlude parts of the face which hinders social communication and emotion recognition. Since sign language users are known to process facial information not only perceptually but also linguistically, examining face processing in deaf signers may reveal how linguistic aspects add to perceptual information. In general, signers could be born deaf or acquire hearing loss later in life. For this study, we focused on signers who were born deaf. Specifically, we analyzed data from a sample of 59 signers who were born deaf and investigated the impacts of face masks on non-linguistic characteristics of the face. Signers rated still-image faces with and without face masks for the following characteristics: arousal and valence of three facial expressions (happy, neutral, sad), invariant characteristics (DV:sex, age), and trait-like characteristics (attractiveness, trustworthiness, approachability). Results indicated that, when compared to masked faces, signers rated no-masked faces with stronger valence intensity across all expressions. Masked faces also appeared older, albeit a tendency to look more approachable. This experiment was a repeat of a previous study conducted on hearing participants, and a post hoc comparison was performed to assess rating differences between signers and hearing people. From this comparison, signers exhibited a larger tendency to rate facial expressions more intensely than hearing people. This suggests that deaf people perceive more intense information from facial expressions and face masks are more inhibiting for deaf people than hearing people. We speculate that deaf people found face masks more approachable due to societal norms when interacting with people wearing masks. Other factors like age and face database’s legitimacy are discussed.

Working Memory for Faces among Individuals with Congenital Deafness

BACKGROUND

 Studies examining face processing among individuals with congenital deafness show inconsistent results that are often accounted for by sign language skill. However, working memory for faces as an aspect of face processing has not yet been examined in congenital deafness.

PURPOSE

 To explore working memory for faces among individuals with congenital deafness who are skilled in sign language.

RESEARCH DESIGN

 A quasi-experimental study of individuals with congenital deafness and a control group.

STUDY SAMPLE

 Sixteen individuals with congenital deafness who are skilled in sign language and 18 participants with intact hearing, matched for age, and education.

INTERVENTION

 The participants performed two conditions of the N-back test in ascending difficulty (i.e., 1-back and 2-back).

DATA COLLECTION AND ANALYSIS

 Levene's and Shapiro-Wilk tests were used to assess group homoscedasticity and normality, respectively. A two-way repeated measures analysis of variance was applied to compare the groups in response time and accuracy of the N-back test, as well as Pearson correlation between response time and accuracy, and sign language skill duration.

RESULTS

 The congenital deafness group performed better than controls, as was found in the response time but not in the accuracy variables. However, an interaction effect showed that this pattern was significant for the 1-back but not for the 2-back condition in the response time but not the accuracy. Further, there was a marginal effect in response time but a significant one in accuracy showing the 2-back was performed worse than the 1-back. No significant correlation was found between response time and accuracy, and sign language skill duration.

CONCLUSION

 Face processing advantage associated with congenital deafness is dependent on cognitive load, but sign language duration does not affect this trend. In addition, response time and accuracy are not equally sensitive to performance differences in the N-back test.

Evidence for superior encoding of detailed visual memories in deaf signers

Recent evidence shows that deaf signers outperform hearing non-signers in some tests of visual attention and discrimination. Furthermore, they can retain visual information better over short periods, i.e., seconds. However, it is unknown if deaf signers’ retention of detailed visual information is superior following more extended periods. We report a study investigating this possibility. Our data revealed that deaf individuals outperformed hearing people in a visual long-term memory test that probed the fine detail of new memories. Deaf individuals also performed better in a scene-discrimination test, which correlated positively with performance on the long-term memory test. Our findings provide evidence that deaf signers can demonstrate superior visual long-term memory, possibly because of enhanced visual attention during encoding. The relative contributions of factors including sign language fluency, protracted practice, and neural plasticity are still to be established. Our findings add to evidence showing that deaf signers are at an advantage in some respects, including the retention of detailed visual memories over the longer term.

Clinical utility of the ECLECTIC framework in providing culturally-informed autism spectrum disorder evaluations: a pediatric case-based approach

Objective: Social cognition does not exist within a vacuum. One's culture and surrounding social environment influence 1) development of social skills and behaviors, and 2) society's expectations regarding "normal" behavior versus behaviors consistent with the diagnosis of Autism Spectrum Disorder (ASD). Use of a comprehensive cultural framework such as Fujii's ECLECTIC model undergirds valid ASD testing by enhancing clinician awareness of potential biases during clinical decision-making and by supporting culturally relevant recommendations. Method: Four diverse pediatric patients presenting for concerns of ASD are described. Neuropsychological test data and salient cultural considerations are presented within the ECLECTIC framework. Results: The cases illustrate relevant cultural factors critical to the ASD assessment for youth with wide diversity (Southeast Asian, Deaf, Black, Hispanic/Latinx, and Chinese cultures) and varied contextual factors (adoption, underlying Down syndrome). The ECLETIC model better allows integration of salient factors such as cognition, family dynamics, behaviors, educational services, and language dominance. Conclusions: Unrecognized ethnocentric biases may shadow the complexities and nuances involved in ASD assessment across cultures. Such errors are minimized using a comprehensive cultural framework to guide equitable neuropsychological services. The ECLECTIC model's emphasis on cultural and contextual factors results in more accurate findings and more individualized planning for the patient. Recommendations for clinical application are provided.

Bridging ears and eyes when learning spoken words: On the effects of bilingual experience at 30 months

From the very first moments of their lives, infants selectively attend to the visible orofacial movements of their social partners and apply their exquisite speech perception skills to the service of lexical learning. Here we explore how early bilingual experience modulates children's ability to use visible speech as they form new lexical representations. Using a cross-modal word-learning task, bilingual children aged 30 months were tested on their ability to learn new lexical mappings in either the auditory or the visual modality. Lexical recognition was assessed either in the same modality as the one used at learning ('same modality' condition: auditory test after auditory learning, visual test after visual learning) or in the other modality ('cross-modality' condition: visual test after auditory learning, auditory test after visual learning). The results revealed that like their monolingual peers, bilingual children successfully learn new words in either the auditory or the visual modality and show cross-modal recognition of words following auditory learning. Interestingly, as opposed to monolinguals, they also demonstrate cross-modal recognition of words upon visual learning. Collectively, these findings indicate a bilingual edge in visual word learning, expressed in the capacity to form a recoverable cross-modal representation of visually learned words.

Language can shape the perception of oriented objects

Seeing an object is a natural source for learning about the object's configuration. We show that language can also shape our knowledge about visual objects. We investigated sign language that enables deaf individuals to communicate through hand movements with as much expressive power as any other natural language. A few signs represent objects in a specific orientation. Sign-language users (signers) recognized visual objects faster when oriented as in the sign, and this match in orientation elicited specific brain responses in signers, as measured by event-related potentials (ERPs). Further analyses suggested that signers' responsiveness to object orientation derived from changes in the visual object representations induced by the signs. Our results also show that language facilitates discrimination between objects of the same kind (e.g., different cars), an effect never reported before with spoken languages. By focusing on sign language we could better characterize the impact of language (a uniquely human ability) on object visual processing.

Unique N170 signatures to words and faces in deaf ASL signers reflect experience-specific adaptations during early visual processing

Previous studies with deaf adults reported reduced N170 waveform asymmetry to visual words, a finding attributed to reduced phonological mapping in left-hemisphere temporal regions compared to hearing adults. An open question remains whether this pattern indeed results from reduced phonological processing or from general neurobiological adaptations in visual processing of deaf individuals. Deaf ASL signers and hearing nonsigners performed a same-different discrimination task with visually presented words, faces, or cars, while scalp EEG time-locked to the onset of the first item in each pair was recorded. For word recognition, the typical left-lateralized N170 in hearing participants and reduced left-sided asymmetry in deaf participants were replicated. The groups did not differ on word discrimination but better orthographic skill was associated with larger N170 in the right hemisphere only for deaf participants. Face recognition was characterized by unique N170 signatures for both groups, and deaf individuals exhibited superior face discrimination performance. Laterality or discrimination performance effects did not generalize to the N170 responses to cars, confirming that deaf signers are not inherently less lateralized in their electrophysiological responses to words and critically, giving support to the phonological mapping hypothesis. P1 was attenuated for deaf participants compared to the hearing, but in both groups, P1 selectively discriminated between highly learned familiar objects - words and faces versus less familiar objects - cars. The distinct electrophysiological signatures to words and faces reflected experience-driven adaptations to words and faces that do not generalize to object recognition.

Hemispheric Asymmetries in Deaf and Hearing During Sustained Peripheral Selective Attention

Previous studies have shown that compared to hearing individuals, early deaf individuals allocate relatively more attention to the periphery than central visual field. However, it is not clear whether these two groups also differ in their ability to selectively attend to specific peripheral locations. We examined deaf and hearing participants' selective attention using electroencephalography (EEG) and a frequency tagging paradigm, in which participants attended to one of two peripheral displays of moving dots that changed directions at different rates. Both participant groups showed similar amplifications and reductions in the EEG signal at the attended and unattended frequencies, indicating similar control over their peripheral attention for motion stimuli. However, for deaf participants these effects were larger in a right hemispheric region of interest (ROI), while for hearing participants these effects were larger in a left ROI. These results contribute to a growing body of evidence for a right hemispheric processing advantage in deaf populations when attending to motion.

Language experience influences audiovisual speech integration in unimodal and bimodal bilingual infants

Infants as young as 2 months can integrate audio and visual aspects of speech articulation. A shift of attention from the eyes towards the mouth of talking faces occurs around 6 months of age in monolingual infants. However, it is unknown whether this pattern of attention during audiovisual speech processing is influenced by speech and language experience in infancy. The present study investigated this question by analysing audiovisual speech processing in three groups of 4- to 8-month-old infants who differed in their language experience: monolinguals, unimodal bilinguals (infants exposed to two or more spoken languages) and bimodal bilinguals (hearing infants with Deaf mothers). Eye-tracking was used to study patterns of face scanning while infants were viewing faces articulating syllables with congruent, incongruent and silent auditory tracks. Monolinguals and unimodal bilinguals increased their attention to the mouth of talking faces between 4 and 8 months, while bimodal bilinguals did not show any age difference in their scanning patterns. Moreover, older (6.6 to 8 months), but not younger, monolinguals (4 to 6.5 months) showed increased visual attention to the mouth of faces articulating audiovisually incongruent rather than congruent faces, indicating surprise or novelty. In contrast, no audiovisual congruency effect was found in unimodal or bimodal bilinguals. Results suggest that speech and language experience influences audiovisual integration in infancy. Specifically, reduced or more variable experience of audiovisual speech from the primary caregiver may lead to less sensitivity to the integration of audio and visual cues of speech articulation.

Impact of Language Experience on Attention to Faces in Infancy: Evidence From Unimodal and Bimodal Bilingual Infants

Faces capture and maintain infants' attention more than other visual stimuli. The present study addresses the impact of early language experience on attention to faces in infancy. It was hypothesized that infants learning two spoken languages (unimodal bilinguals) and hearing infants of Deaf mothers learning British Sign Language and spoken English (bimodal bilinguals) would show enhanced attention to faces compared to monolinguals. The comparison between unimodal and bimodal bilinguals allowed differentiation of the effects of learning two languages, from the effects of increased visual communication in hearing infants of Deaf mothers. Data are presented for two independent samples of infants: Sample 1 included 49 infants between 7 and 10 months (26 monolinguals and 23 unimodal bilinguals), and Sample 2 included 87 infants between 4 and 8 months (32 monolinguals, 25 unimodal bilinguals, and 30 bimodal bilingual infants with a Deaf mother). Eye-tracking was used to analyze infants' visual scanning of complex arrays including a face and four other stimulus categories. Infants from 4 to 10 months (all groups combined) directed their attention to faces faster than to non-face stimuli (i.e., attention capture), directed more fixations to, and looked longer at faces than non-face stimuli (i.e., attention maintenance). Unimodal bilinguals demonstrated increased attention capture and attention maintenance by faces compared to monolinguals. Contrary to predictions, bimodal bilinguals did not differ from monolinguals in attention capture and maintenance by face stimuli. These results are discussed in relation to the language experience of each group and the close association between face processing and language development in social communication.

Visual field plasticity in hearing users of sign language

Studies have observed that deaf signers have a larger Visual Field (VF) than hearing non-signers with a particular large extension in the lower part of the VF. This increment could stem from early deafness or from the extensive use of sign language, since the lower VF is critical to perceive and understand linguistics gestures in sign language communication. The aim of the present study was to explore the potential impact of sign language experience without deafness on the VF sensitivity within its lower part. Using standard Humphrey Visual Field Analyzer, we compared luminance sensitivity in the fovea and between 3 and 27 degrees of visual eccentricity for the upper and lower VF, between hearing users of French Sign Language and age-matched hearing non-signers. The sensitivity in the fovea and in the upper VF were similar in both groups. Hearing signers had, however, higher luminance sensitivity than non-signers in the lower VF but only between 3 and 15°, the visual location for sign language perception. Sign language experience, no associated with deafness, may then be a modulating factor of VF sensitivity but restricted to the very specific location where signs are perceived.