Language/Culture Modulates Brain and Gaze Processes in Audiovisual Speech Perception

Age and Gender Differences in the Pseudo-Haptic Effect on Computer Mouse Operation in a Desktop Environment

Pseudo-haptics is a method that can provide a haptic sensation without requiring a physical haptic device. The effect of pseudo-haptics is known to depend on the individual, but it is unclear which factors cause individual differences. As the first study establishing a calibration method for these differences in future research, we examined the differences in the pseudo-haptic effect on mouse cursor operation in a desktop environment depending on the age and gender of the user. We conducted an online experiment and collected data from more than 400 participants. The participants performed a task of lifting a virtual object with a mouse pointer. We found that the effect of pseudo-haptics was greater in younger or male participants than in older or female participants. We also found that the effect of pseudo-haptics, which varied with age and gender, can be explained by habituation to the mouse in daily life and the accuracy of detecting the pointer position using vision or proprioception. Specifically, the pseudo-haptic effect was higher for those who used the mouse more frequently and had higher accuracy in identifying the pointer position using proprioception or vision. The results of the present study not only indicate the factors that cause age and gender differences but also provide hints for calibrating these differences.

Primacy of mouth over eyes to perceive audiovisual Mandarin lexical tones

The visual cues of lexical tones are more implicit and much less investigated than consonants and vowels, and it is still unclear what facial areas contribute to facial tones identification. This study investigated Chinese and English speakers' eye movements when they were asked to identify audiovisual Mandarin lexical tones. The Chinese and English speakers were presented with an audiovisual clip of Mandarin monosyllables (for instance, /ă/, /à/, /ĭ/, /ì/) and were asked to identify whether the syllables were a dipping tone (/ă/, / ĭ/) or a falling tone (/ à/, /ì/). These audiovisual syllables were presented in clear, noisy and silent (absence of audio signal) conditions. An eye-tracker recorded the participants' eye movements. Results showed that the participants gazed more at the mouth than the eyes. In addition, when acoustic conditions became adverse, both the Chinese and English speakers increased their gaze duration at the mouth rather than at the eyes. The findings suggested that the mouth is the primary area that listeners utilise in their perception of audiovisual lexical tones. The similar eye movements between the Chinese and English speakers imply that the mouth acts as a perceptual cue that provides articulatory information, as opposed to social and pragmatic information.

Atypical gaze patterns to facial feature areas in autism spectrum disorders reveal age and culture effects: A meta-analysis of eye-tracking studies

Children and adults with autism spectrum disorder (ASD) often present with atypical gaze patterns to others' faces, a finding substantiated throughout the literature. Yet, a quantification of atypical gaze patterns to different facial regions (e.g., eyes versus mouth) in ASD remains controversial. Also few study has investigated how age and culture impacted the pattern of gaze abnormalities in ASD. This research therefore conducted a meta-analysis of eye-tracking studies to evaluate age and culture effect on atypical gaze patterns of face processing in ASD. A total of 75 articles (91 studies) and 4209 individuals (ASD: 2027; controls: 2182) across all age ranges (i.e., childhood through to adulthood) from both Eastern and Western cultures were included in this meta-analysis. Individuals with ASD yielded shorter fixation durations to the eyes than individuals without ASD. Group differences in the time spent fixating on the eyes were not modulated by age, but affected by culture. Effect size in the eastern culture was larger than that in the western culture. In contrast, group differences on time spent looking at the mouth were not significant, but changed with age and modulated by culture. Relative to the neurotypical controls, Western individuals with ASD spent more time looking at the mouth from school age, whereas Eastern individuals with ASD did not gaze longer on mouth until adulthood. These results add to the body of evidence supporting atypical gaze behaviors to eyes in ASD and provide new insights into a potential mouth compensation strategy that develops with age in ASD. LAY SUMMARY: Individuals with autism spectrum disorder (ASD) often show atypical gaze patterns when looking at others' faces compared to neurotypical individuals. This paper examines the role of age and culture on pattern of gaze abnormalities in individuals with ASD. Results show that reduction of gaze on eyes in ASD is stable across all ages and cultures, while increase of gaze on mouth emerges as individuals with ASD get older. The findings provide a developmental insight to the gaze patterns on the autism spectrum across culture.

Selective attention to the mouth of a talker in Japanese-learning infants and toddlers: Its relationship with vocabulary and compensation for noise

Infants increasingly gaze at the mouth of talking faces during the latter half of the first postnatal year. This study investigated mouth-looking behavior of 120 full-term infants and toddlers (6 months-3 years) and 12 young adults (21-24 years) from Japanese monolingual families. The purpose of the study included: (1) Is such an attentional shift to the mouth in infancy similarly observed in Japanese environment where contribution of visual speech is known to be relatively weak? (2) Can noisy conditions increase mouth-looking behavior of Japanese young children? (3) Is the mouth-looking behavior related to language acquisition? To this end, movies of a talker speaking short phrases were presented while manipulating signal-to-noise ratio (SNR: Clear, SN+4, and SN-4). Expressive vocabulary of toddlers was obtained through their parents. The results indicated that Japanese infants initially have a strong preference for the eyes to mouth which is weakened toward 10 months, but the shift was later and in a milder fashion compared to known results for English-learning infants. Even after 10 months, no clear-cut preference for the mouth was observed even in linguistically challenging situations with strong noise until 3 years of age. In the Clear condition, there was a return of the gaze to the eyes as early as 3 years of age, where they showed increasing attention to the mouth with increasing noise level. In addition, multiple regression analyses revealed a tendency that 2- and 3-year-olds with larger vocabulary increasingly look at the eyes. Overall, the gaze of Japanese-learning infants and toddlers was more biased to the eyes in various aspects compared to known results of English-learning infants. The present findings shed new light on our understanding of the development of selective attention to the mouth in non-western populations.

A value-driven McGurk effect: Value-associated faces enhance the influence of visual information on audiovisual speech perception and its eye movement pattern

This study investigates whether and how value-associated faces affect audiovisual speech perception and its eye movement pattern. Participants were asked to learn to associate particular faces with or without monetary reward in the training phase, and, in the subsequent test phase, to identify syllables that the talkers had said in video clips in which the talkers' faces had or had not been associated with reward. The syllables were either congruent or incongruent with the talkers' mouth movements. Crucially, in some cases, the incongruent syllables could elicit the McGurk effect. Results showed that the McGurk effect occurred more often for reward-associated faces than for non-reward-associated faces. Moreover, the signal detection analysis revealed that participants had lower criterion and higher discriminability for reward-associated faces than for non-reward-associated faces. Surprisingly, eye movement data showed that participants spent more time looking at and fixated more often on the extraoral (nose/cheek) area for reward-associated faces than for non-reward-associated faces, while the opposite pattern was observed on the oral (mouth) area. The correlation analysis demonstrated that, over participants, the more they looked at the extraoral area in the training phase because of reward, the larger the increase of McGurk proportion (and the less they looked at the oral area) in the test phase. These findings not only demonstrate that value-associated faces enhance the influence of visual information on audiovisual speech perception but also highlight the importance of the extraoral facial area in the value-driven McGurk effect.