An Eye Tracking Study on the Perception and Comprehension of Unimodal and Bimodal Linguistic Inputs by Deaf Adolescents

Effects of age on face perception: Reduced eye region discrimination ability but intact holistic processing

While age-related decline in face recognition memory is well-established, the degree of decline in face perceptual abilities across the lifespan and the underlying mechanisms are incompletely characterized. In the present study, we used the part-whole task to examine lifespan changes in holistic and featural processing. After studying an intact face, participants are tested for memory of a face part (eyes, nose, mouth) with the target and foil part presented either in isolation or in the context of the whole face. To the extent that parts are encoded into a holistic face representation, an advantage is expected for part recognition when tested in the whole face condition. The task therefore provides measures of holistic processing (whole-over-isolated-part trial advantage) and featural processing for each part when tested in isolation. Using a large sample of 3,341 online participants aged 18-69 years, we found that while discrimination of the eye region decreased beginning in the 50s, both mouth discrimination accuracy and the holistic advantage of whole versus part trial discrimination were stable with age. In separate analyses by gender, we found that age-related declines in eye region accuracy were more pronounced in males than females. We discuss potential mechanistic explanations for this eye region-specific decline with age, including age-related hearing loss directing attention toward the mouth. Further, we discuss how this could be related to the age-related positivity effect, which is associated with reduced sensitivity to eye-related emotions (e.g., anger) but preserved mouth-related emotion sensitivity (e.g., happiness). (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Neural responses to naturalistic audiovisual speech are related to listening demand in cochlear implant users

There is a weak relationship between clinical and self-reported speech perception outcomes in cochlear implant (CI) listeners. Such poor correspondence may be due to differences in clinical and “real-world” listening environments and stimuli. Speech in the real world is often accompanied by visual cues, background environmental noise, and is generally in a conversational context, all factors that could affect listening demand. Thus, our objectives were to determine if brain responses to naturalistic speech could index speech perception and listening demand in CI users. Accordingly, we recorded high-density electroencephalogram (EEG) while CI users listened/watched a naturalistic stimulus (i.e., the television show, “The Office”). We used continuous EEG to quantify “speech neural tracking” (i.e., TRFs, temporal response functions) to the show’s soundtrack and 8–12 Hz (alpha) brain rhythms commonly related to listening effort. Background noise at three different signal-to-noise ratios (SNRs), +5, +10, and +15 dB were presented to vary the difficulty of following the television show, mimicking a natural noisy environment. The task also included an audio-only (no video) condition. After each condition, participants subjectively rated listening demand and the degree of words and conversations they felt they understood. Fifteen CI users reported progressively higher degrees of listening demand and less words and conversation with increasing background noise. Listening demand and conversation understanding in the audio-only condition was comparable to that of the highest noise condition (+5 dB). Increasing background noise affected speech neural tracking at a group level, in addition to eliciting strong individual differences. Mixed effect modeling showed that listening demand and conversation understanding were correlated to early cortical speech tracking, such that high demand and low conversation understanding occurred with lower amplitude TRFs. In the high noise condition, greater listening demand was negatively correlated to parietal alpha power, where higher demand was related to lower alpha power. No significant correlations were observed between TRF/alpha and clinical speech perception scores. These results are similar to previous findings showing little relationship between clinical speech perception and quality-of-life in CI users. However, physiological responses to complex natural speech may provide an objective measure of aspects of quality-of-life measures like self-perceived listening demand.

Impact of home literacy environment on literacy development of children with hearing loss: A mediation model

Reading presents an unsolved difficulty for children with hearing loss and research on factors influencing their literacy development is very limited. This work aimed to study the influence of home literacy environment (HLE) on literacy development of children with hearing loss and explore possible mediating effects of reading interest and parent-child relationship. 112 Chinese children with hearing loss were surveyed for scales of HLE, literacy development, reading interest, and parent-child relationship. Result analysis showed that HLE significantly predicted literacy development of children with hearing loss and this effect was no longer significant after including reading interest and parent-child relationship as variables. Further, HLE significantly predicted reading interest and parent-child relationship, each of which predicted literacy development and played a significant mediating role in HLE’s influence on literacy development. These findings provide educational tips for families of children with hearing loss.

Audio-visual integration in noise: Influence of auditory and visual stimulus degradation on eye movements and perception of the McGurk effect

Seeing a talker’s face can aid audiovisual (AV) integration when speech is presented in noise. However, few studies have simultaneously manipulated auditory and visual degradation. We aimed to establish how degrading the auditory and visual signal affected AV integration. Where people look on the face in this context is also of interest; Buchan, Paré and Munhall (Brain Research, 1242, 162–171, 2008) found fixations on the mouth increased in the presence of auditory noise whilst Wilson, Alsius, Paré and Munhall (Journal of Speech, Language, and Hearing Research, 59(4), 601–615, 2016) found mouth fixations decreased with decreasing visual resolution. In Condition 1, participants listened to clear speech, and in Condition 2, participants listened to vocoded speech designed to simulate the information provided by a cochlear implant. Speech was presented in three levels of auditory noise and three levels of visual blurring. Adding noise to the auditory signal increased McGurk responses, while blurring the visual signal decreased McGurk responses. Participants fixated the mouth more on trials when the McGurk effect was perceived. Adding auditory noise led to people fixating the mouth more, while visual degradation led to people fixating the mouth less. Combined, the results suggest that modality preference and where people look during AV integration of incongruent syllables varies according to the quality of information available.

The sign superiority effect: Lexical status facilitates peripheral handshape identification for deaf signers

Deaf signers exhibit an enhanced ability to process information in their peripheral visual field, particularly the motion of dots or orientation of lines. Does their experience processing sign language, which involves identifying meaningful visual forms across the visual field, contribute to this enhancement? We tested whether deaf signers recruit language knowledge to facilitate peripheral identification through a sign superiority effect (i.e., better handshape discrimination in a sign than a pseudosign) and whether such a superiority effect might be responsible for perceptual enhancements relative to hearing individuals (i.e., a decrease in the effect of eccentricity on perceptual identification). Deaf signers and hearing signers or nonsigners identified the handshape presented within a static ASL fingerspelling letter (Experiment 1), fingerspelled sequence (Experiment 2), or sign or pseudosign (Experiment 3) presented in the near or far periphery. Accuracy on all tasks was higher for deaf signers than hearing nonsigning participants and was higher in the near than the far periphery. Across experiments, there were different patterns of interactions between hearing status and eccentricity depending on the type of stimulus; deaf signers showed an effect of eccentricity for static fingerspelled letters, fingerspelled sequences, and pseudosigns but not for ASL signs. In contrast, hearing nonsigners showed an effect of eccentricity for all stimuli. Thus, deaf signers recruit lexical knowledge to facilitate peripheral perceptual identification, and this perceptual enhancement may derive from their extensive experience processing visual linguistic information in the periphery during sign comprehension. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

An Eye-Tracking Study on Audiovisual Speech Perception Strategies Adopted by Normal-Hearing and Deaf Adults Under Different Language Familiarities

Purpose The primary purpose of this study was to explore the audiovisual speech perception strategies.80.23.47 adopted by normal-hearing and deaf people in processing familiar and unfamiliar languages. Our primary hypothesis was that they would adopt different perception strategies due to different sensory experiences at an early age, limitations of the physical device, and the developmental gap of language, and others. Method Thirty normal-hearing adults and 33 prelingually deaf adults participated in the study. They were asked to perform judgment and listening tasks while watching videos of a Uygur-Mandarin bilingual speaker in a familiar language (Standard Chinese) or an unfamiliar language (Modern Uygur) while their eye movements were recorded by eye-tracking technology. Results Task had a slight influence on the distribution of selective attention, whereas subject and language had significant influences. To be specific, the normal-hearing and the d10eaf participants mainly gazed at the speaker's eyes and mouth, respectively, in the experiment; moreover, while the normal-hearing participants had to stare longer at the speaker's mouth when they confronted with the unfamiliar language Modern Uygur, the deaf participant did not change their attention allocation pattern when perceiving the two languages. Conclusions Normal-hearing and deaf adults adopt different audiovisual speech perception strategies: Normal-hearing adults mainly look at the eyes, and deaf adults mainly look at the mouth. Additionally, language and task can also modulate the speech perception strategy.

Effects of Video Reversal on Gaze Patterns during Signed Narrative Comprehension

Language knowledge, age of acquisition (AoA), and stimulus intelligibility all affect gaze behavior for reading print, but it is unknown how these factors affect "sign-watching" among signers. This study investigated how these factors affect gaze behavior during sign language comprehension in 52 adult signers who acquired American Sign Language (ASL) at different ages. We examined gaze patterns and story comprehension in four subject groups who differ in hearing status and when they learned ASL (i.e. Deaf Early, Deaf Late, Hearing Late, and Hearing Novice). Participants watched signed stories in normal (high intelligibility) and video-reversed (low intelligibility) conditions. This video manipulation was used because it distorts word order and thus disrupts the syntax and semantic content of narratives, while preserving most surface phonological features of individual signs. Video reversal decreased story comprehension accuracy, and this effect was greater for those who learned ASL later in life. Reversal also was associated with more dispersed gaze behavior. Although each subject group had unique gaze patterns, the effect of video reversal on gaze measures was similar across all groups. Among fluent signers, gaze behavior was not correlated with AoA, suggesting that "efficient" sign watching can be quickly learnt even among signers exposed to signed language later in life.

Sign language experience redistributes attentional resources to the inferior visual field

While a substantial body of work has suggested that deafness brings about an increased allocation of visual attention to the periphery there has been much less work on how using a signed language may also influence this attentional allocation. Signed languages are visual-gestural and produced using the body and perceived via the human visual system. Signers fixate upon the face of interlocutors and do not directly look at the hands moving in the inferior visual field. It is therefore reasonable to predict that signed languages require a redistribution of covert visual attention to the inferior visual field. Here we report a prospective and statistically powered assessment of the spatial distribution of attention to inferior and superior visual fields in signers - both deaf and hearing - in a visual search task. Using a Bayesian Hierarchical Drift Diffusion Model, we estimated decision making parameters for the superior and inferior visual field in deaf signers, hearing signers and hearing non-signers. Results indicated a greater attentional redistribution toward the inferior visual field in adult signers (both deaf and hearing) than in hearing sign-naïve adults. The effect was smaller for hearing signers than for deaf signers, suggestive of either a role for extent of exposure or greater plasticity of the visual system in the deaf. The data provide support for a process by which the demands of linguistic processing can influence the human attentional system.