Impact of language on functional connectivity for audiovisual speech integration

A meta-analysis of letter-sound integration: Assimilation and accommodation in the superior temporal gyrus

Despite being a relatively new cultural phenomenon, the ability to perform letter-sound integration is readily acquired even though it has not had time to evolve in the brain. Leading theories of how the brain accommodates literacy acquisition include the neural recycling hypothesis and the assimilation-accommodation hypothesis. The neural recycling hypothesis proposes that a new cultural skill is developed by "invading" preexisting neural structures to support a similar cognitive function, while the assimilation-accommodation hypothesis holds that a new cognitive skill relies on direct invocation of preexisting systems (assimilation) and adds brain areas based on task requirements (accommodation). Both theories agree that letter-sound integration may be achieved by reusing pre-existing functionally similar neural bases, but differ in their proposals of how this occurs. We examined the evidence for each hypothesis by systematically comparing the similarities and differences between letter-sound integration and two other types of preexisting and functionally similar audiovisual (AV) processes, namely object-sound and speech-sound integration, by performing an activation likelihood estimation (ALE) meta-analysis. All three types of AV integration recruited the left posterior superior temporal gyrus (STG), while speech-sound integration additionally activated the bilateral middle STG and letter-sound integration directly invoked the AV areas involved in speech-sound integration. These findings suggest that letter-sound integration may reuse the STG for speech-sound and object-sound integration through an assimilation-accommodation mechanism.

Importance of Visual Support Through Lipreading in the Identification of Words in Spanish Language

We sought to examine the contribution of visual cues, such as lipreading, in the identification of familiar (words) and unfamiliar (phonemes) words in terms of percent accuracy. For that purpose, in this retrospective study, we presented lists of words and phonemes (adult female healthy voice) in auditory (A) and audiovisual (AV) modalities to 65 Spanish normal-hearing male and female listeners classified in four age groups. Our results showed a remarkable benefit of AV information in word and phoneme recognition. Regarding gender, women exhibited better performance than men in both A and AV modalities, although we only found significant differences for words but not for phonemes. Concerning age, significant differences were detected in word recognition in the A modality between the youngest (18-29 years old) and oldest (⩾50 years old) groups only. We conclude visual information enhances word and phoneme recognition and women are more influenced by visual signals than men in AV speech perception. On the contrary, it seems that, overall, age is not a limiting factor for word recognition, with no significant differences observed in the AV modality.

Selective attention to the mouth of a talker in Japanese-learning infants and toddlers: Its relationship with vocabulary and compensation for noise

Infants increasingly gaze at the mouth of talking faces during the latter half of the first postnatal year. This study investigated mouth-looking behavior of 120 full-term infants and toddlers (6 months-3 years) and 12 young adults (21-24 years) from Japanese monolingual families. The purpose of the study included: (1) Is such an attentional shift to the mouth in infancy similarly observed in Japanese environment where contribution of visual speech is known to be relatively weak? (2) Can noisy conditions increase mouth-looking behavior of Japanese young children? (3) Is the mouth-looking behavior related to language acquisition? To this end, movies of a talker speaking short phrases were presented while manipulating signal-to-noise ratio (SNR: Clear, SN+4, and SN-4). Expressive vocabulary of toddlers was obtained through their parents. The results indicated that Japanese infants initially have a strong preference for the eyes to mouth which is weakened toward 10 months, but the shift was later and in a milder fashion compared to known results for English-learning infants. Even after 10 months, no clear-cut preference for the mouth was observed even in linguistically challenging situations with strong noise until 3 years of age. In the Clear condition, there was a return of the gaze to the eyes as early as 3 years of age, where they showed increasing attention to the mouth with increasing noise level. In addition, multiple regression analyses revealed a tendency that 2- and 3-year-olds with larger vocabulary increasingly look at the eyes. Overall, the gaze of Japanese-learning infants and toddlers was more biased to the eyes in various aspects compared to known results of English-learning infants. The present findings shed new light on our understanding of the development of selective attention to the mouth in non-western populations.

Effects of Visual Speech on Early Auditory Evoked Fields - From the Viewpoint of Individual Variance

The effects of visual speech (the moving image of the speaker’s face uttering speech sound) on early auditory evoked fields (AEFs) were examined using a helmet-shaped magnetoencephalography system in 12 healthy volunteers (9 males, mean age 35.5 years). AEFs (N100m) in response to the monosyllabic sound /be/ were recorded and analyzed under three different visual stimulus conditions, the moving image of the same speaker’s face uttering /be/ (congruent visual stimuli) or uttering /ge/ (incongruent visual stimuli), and visual noise (still image processed from speaker’s face using a strong Gaussian filter: control condition). On average, latency of N100m was significantly shortened in the bilateral hemispheres for both congruent and incongruent auditory/visual (A/V) stimuli, compared to the control A/V condition. However, the degree of N100m shortening was not significantly different between the congruent and incongruent A/V conditions, despite the significant differences in psychophysical responses between these two A/V conditions. Moreover, analysis of the magnitudes of these visual effects on AEFs in individuals showed that the lip-reading effects on AEFs tended to be well correlated between the two different audio-visual conditions (congruent vs. incongruent visual stimuli) in the bilateral hemispheres but were not significantly correlated between right and left hemisphere. On the other hand, no significant correlation was observed between the magnitudes of visual speech effects and psychophysical responses. These results may indicate that the auditory-visual interaction observed on the N100m is a fundamental process which does not depend on the congruency of the visual information.