The Role of Audiovisual Speech in Fast-Mapping and Novel Word Retention in Monolingual and Bilingual 24-Month-Olds

Touch to learn: Multisensory input supports word learning and processing

Infants experience language in rich multisensory environments. For example, they may first be exposed to the word applesauce while touching, tasting, smelling, and seeing applesauce. In three experiments using different methods we asked whether the number of distinct senses linked with the semantic features of objects would impact word recognition and learning. Specifically, in Experiment 1 we asked whether words linked with more multisensory experiences were learned earlier than words linked fewer multisensory experiences. In Experiment 2, we asked whether 2-year-olds' known words linked with more multisensory experiences were better recognized than those linked with fewer. Finally, in Experiment 3, we taught 2-year-olds labels for novel objects that were linked with either just visual or visual and tactile experiences and asked whether this impacted their ability to learn the new label-to-object mappings. Results converge to support an account in which richer multisensory experiences better support word learning. We discuss two pathways through which rich multisensory experiences might support word learning.

Long-term phonemic representations become audiovisual by mid-childhood

In earlier work with adults, we showed that long-term phonemic representations are audiovisual, meaning that they contain information on typical mouth shape during articulation. Many aspects of audiovisual processing have a prolonged developmental course, often not reaching maturity until late adolescence. In this study, we examined the status of phonemic representations in two groups of children - 8-9-year-olds and 11-12-year-olds. We used the same audiovisual oddball paradigm as in the earlier study with adults (Kaganovich and Christ, 2021). On each trial, participants saw a face and heard one of two vowels. One vowel occurred frequently (standard), while another occurred rarely (deviant). In one condition (neutral), the face had a closed, non-articulating mouth. In the other condition (audiovisual violation), the mouth shape matched the frequent vowel. Although stimuli were audiovisual in both conditions, we hypothesized that identical auditory changes would be perceived differently by participants. Namely, in the neutral condition, deviants violated only the audiovisual pattern specific to each experimental block. By contrast, in the audiovisual violation condition, deviants additionally violated long-term representations for how a speaker's mouth looks during articulation. We compared the amplitude of MMN and P3 components elicited by deviants in the two conditions. In the 11-12-year-old group, the pattern of neural responses was similar to that in adults - namely, they had a larger MMN component in the audiovisual compared to neutral condition, with no major difference in the P3 amplitude. In contrast, in the 8-9-year-old group, we saw a posterior MMN in the neutral condition only and a larger P3 in the audiovisual violation compared to the neutral condition. The larger P3 in the audiovisual violation condition suggests that younger children did perceive deviants as being more attention-grabbing when they violated the typical combination of sound and mouth shape. Yet, at this age, the earlier, more automatic stages of phonemic processing indexed by the MMN component may not yet encode visual speech elements the same way they do in older children and adults. We conclude that phonemic representations do not become audiovisual until 11-12 years of age.

The Role of Talking Faces in Infant Language Learning: Mind the Gap between Screen-Based Settings and Real-Life Communicative Interactions

Over the last few decades, developmental (psycho) linguists have demonstrated that perceiving talking faces audio-visually is important for early language acquisition. Using mostly well-controlled and screen-based laboratory approaches, this line of research has shown that paying attention to talking faces is likely to be one of the powerful strategies infants use to learn their native(s) language(s). In this review, we combine evidence from these screen-based studies with another line of research that has studied how infants learn novel words and deploy their visual attention during naturalistic play. In our view, this is an important step toward developing an integrated account of how infants effectively extract audiovisual information from talkers' faces during early language learning. We identify three factors that have been understudied so far, despite the fact that they are likely to have an important impact on how infants deploy their attention (or not) toward talking faces during social interactions: social contingency, speaker characteristics, and task- dependencies. Last, we propose ideas to address these issues in future research, with the aim of reducing the existing knowledge gap between current experimental studies and the many ways infants can and do effectively rely upon the audiovisual information extracted from talking faces in their real-life language environment.

Learning Words in Two Languages: Manipulating Exemplar Variability for Within- and Cross-Language Generalization

PURPOSE

This article aims to describe how exemplar variability can manipulate the word learning environment to maximize within- and cross-language generalization in Spanish-English bilinguals. Furthermore, we examined sources of individual variability that predicted word learning.

METHOD

Nineteen Spanish-English bilingual children participated in a word learning task presenting words in both languages. Children learned words either in a high variability condition (in which multiple exemplars are introduced with the target word) or in a no variability condition (in which the same referent is used with the target word). Word learning was tracked over the course of the training, and retention was examined once the training was discontinued. Children's generalization of referents within and across languages was also examined.

RESULTS

The exemplar variability effect was observed in within-language generalization trials, whereas cross-language generalization was less robust. Nevertheless, cross-language associations emerged in examining the role of language proficiency, such that semantic skills in English predicted word retention across languages. Similarly, children's propensity to code-switch during language production was positively correlated with retention of words learned in the high variability condition.

CONCLUSIONS

The findings show that Spanish-English bilingual children may make use of exemplar variability to support word learning in different ways compared with monolinguals. The exemplar variability effect interacts with children's acquired language skills and word learning abilities at the start of the intervention. This study provides preliminary evidence from which future research can develop word learning interventions that are responsive to the needs of multilinguals.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.19241856.

Faces and Voices Processing in Human and Primate Brains: Rhythmic and Multimodal Mechanisms Underlying the Evolution and Development of Speech

While influential works since the 1970s have widely assumed that imitation is an innate skill in both human and non-human primate neonates, recent empirical studies and meta-analyses have challenged this view, indicating other forms of reward-based learning as relevant factors in the development of social behavior. The visual input translation into matching motor output that underlies imitation abilities instead seems to develop along with social interactions and sensorimotor experience during infancy and childhood. Recently, a new visual stream has been identified in both human and non-human primate brains, updating the dual visual stream model. This third pathway is thought to be specialized for dynamics aspects of social perceptions such as eye-gaze, facial expression and crucially for audio-visual integration of speech. Here, we review empirical studies addressing an understudied but crucial aspect of speech and communication, namely the processing of visual orofacial cues (i.e., the perception of a speaker's lips and tongue movements) and its integration with vocal auditory cues. Along this review, we offer new insights from our understanding of speech as the product of evolution and development of a rhythmic and multimodal organization of sensorimotor brain networks, supporting volitional motor control of the upper vocal tract and audio-visual voices-faces integration.

Bilingualism alters infants' cortical organization for attentional orienting mechanisms

A bilingual environment is associated with changes in the brain's structure and function. Some suggest that bilingualism also improves higher-cognitive functions in infants as young as 6-months, yet whether this effect is associated with changes in the infant brain remains unknown. In the present study, we measured brain activity using functional near-infrared spectroscopy in monolingual- and bilingual-raised 6- and 10-month-old infants. Infants completed an orienting attention task, in which a cue was presented prior to an object appearing on the same (Valid) or opposite (Invalid) side of a display. Task performance did not differ between the groups but neural activity did. At 6-months, both groups showed greater activity for Valid (> Invalid) trials in frontal regions (left hemisphere for bilinguals, right hemisphere for monolinguals). At 10-months, bilinguals showed greater activity for Invalid (> Valid) trials in bilateral frontal regions, while monolinguals showed greater brain activity for Valid (> Invalid) trials in left frontal regions. Bilinguals' brain activity trended with their parents' reporting of dual-language mixing when speaking to their child. These findings are the first to indicate how early (dual) language experience can alter the cortical organization underlying broader, non-linguistic cognitive functions during the first year of life.

Attentional orienting abilities in bilinguals: Evidence from a large infant sample

A key question in studies of cognitive development is whether bilingual environments impact higher-cognitive functions. Inconclusive evidence in search of a "bilingual cognitive advantage" has sparked debates on the reliability of these findings. Few studies with infants have examined this question, but most of them include small samples. The current study presents evidence from a large sample of 6- and 10-month-old monolingual- and bilingual-exposed infants (N = 152), which includes a longitudinal subset (n = 31), who completed a cueing attentional orienting task. The results suggest bilingual infants showed significant developmental gains in latency performance during the condition that was most cognitively demanding (Incongruent). The results also revealed bilingual infants' performance was associated with their parents' dual-language switching behavior. Taken together, these results provide support that bilingual experiences (i.e., dual-language mixing) influence infants' shifting and orienting of attention.

Infants' Mapping of New Faces to New Voices

The ability to identify individuals by voice is fundamental for communication. However, little is known about the expectations that infants hold when learning unfamiliar voices. Here, the voice-learning skills of 4- and 8-month-olds (N = 53; 29 girls, 14 boys of various ethnicities) were tested using a preferential-looking task that involved audiovisual stimuli of their mothers and other unfamiliar women. Findings reveal that the expectation that novel voices map on to novel faces emerges between 4 and 8 months of age, and that infants can retain learning of face-voice pairings via nonostensive cues by 8 months of age. This study provides new insights about infants' use of disambiguation and fast mapping in early voice learning.