Learning Spoken Words via the Ears and Eyes: Evidence from 30-Month-Old Children

Touch to learn: Multisensory input supports word learning and processing

Infants experience language in rich multisensory environments. For example, they may first be exposed to the word applesauce while touching, tasting, smelling, and seeing applesauce. In three experiments using different methods we asked whether the number of distinct senses linked with the semantic features of objects would impact word recognition and learning. Specifically, in Experiment 1 we asked whether words linked with more multisensory experiences were learned earlier than words linked fewer multisensory experiences. In Experiment 2, we asked whether 2-year-olds' known words linked with more multisensory experiences were better recognized than those linked with fewer. Finally, in Experiment 3, we taught 2-year-olds labels for novel objects that were linked with either just visual or visual and tactile experiences and asked whether this impacted their ability to learn the new label-to-object mappings. Results converge to support an account in which richer multisensory experiences better support word learning. We discuss two pathways through which rich multisensory experiences might support word learning.

Turn that music down! Affective musical bursts cause an auditory dominance in children recognizing bodily emotions

Previous work has shown that different sensory channels are prioritized across the life course, with children preferentially responding to auditory information. The aim of the current study was to investigate whether the mechanism that drives this auditory dominance in children occurs at the level of encoding (overshadowing) or when the information is integrated to form a response (response competition). Given that response competition is dependent on a modality integration attempt, a combination of stimuli that could not be integrated was used so that if children's auditory dominance persisted, this would provide evidence for the overshadowing over the response competition mechanism. Younger children (≤7 years), older children (8-11 years), and adults (18+ years) were asked to recognize the emotion (happy or fearful) in either nonvocal auditory musical emotional bursts or human visual bodily expressions of emotion in three conditions: unimodal, congruent bimodal, and incongruent bimodal. We found that children performed significantly worse at recognizing emotional bodies when they heard (and were told to ignore) musical emotional bursts. This provides the first evidence for auditory dominance in both younger and older children when presented with modally incongruent emotional stimuli. The continued presence of auditory dominance, despite the lack of modality integration, was taken as supportive evidence for the overshadowing explanation. These findings are discussed in relation to educational considerations, and future sensory dominance investigations and models are proposed.

Developmental change in children's speech processing of auditory and visual cues: An eyetracking study

This study investigates how children aged two to eight years (N = 129) and adults (N = 29) use auditory and visual speech for word recognition. The goal was to bridge the gap between apparent successes of visual speech processing in young children in visual-looking tasks, with apparent difficulties of speech processing in older children from explicit behavioural measures. Participants were presented with familiar words in audio-visual (AV), audio-only (A-only) or visual-only (V-only) speech modalities, then presented with target and distractor images, and looking to targets was measured. Adults showed high accuracy, with slightly less target-image looking in the V-only modality. Developmentally, looking was above chance for both AV and A-only modalities, but not in the V-only modality until 6 years of age (earlier on /k/-initial words). Flexible use of visual cues for lexical access develops throughout childhood.

Mechanisms of social evaluation in infancy: A preregistered exploration of infants' eye-movement and pupillary responses to prosocial and antisocial events

Past research shows infants selectively touch and look longer at characters who help versus hinder others (Social evaluation by preverbal infants. Nature, 2007, 450, 557; Three-month-olds show a negativity bias in their social evaluations. Developmental Science, 2010, 13, 923); however, the mechanisms underlying this tendency remain underspecified. The current preregistered experiment approaches this question by examining infants' real-time looking behaviors during prosocial and antisocial events, and exploring how individual infants' looking behaviors correlate with helper preferences. Using eye-tracking, 34 five-month-olds were familiarized with two blocks of the "hill" scenario originally developed by Kuhlmeier et al. (Attribution of dispositional states by 12-month-olds. Psychological Science, 2003, 14, 402), in which a climber tries unsuccessfully to reach the top of a hill and is alternately helped or hindered. Infants' visual preferences were assessed after each block of 6 helping and hindering events by proportional looking time to the helper versus hinderer in an image of the characters side by side. Results showed that, at the group level, infants looked longer at the helper after viewing 12 (but not after viewing 6) helping and hindering videos. Moreover, individual infants' average preference for the helper was predicted by their looking behaviors, particularly those suggestive of an understanding of the climber's unfulfilled goal. These results shed light on how infants process helping/hindering scenarios, and suggest that goal understanding is important for infants' helper preferences.

The Role of Audiovisual Speech in Fast-Mapping and Novel Word Retention in Monolingual and Bilingual 24-Month-Olds

Three experiments examined the role of audiovisual speech on 24-month-old monolingual and bilinguals’ performance in a fast-mapping task. In all three experiments, toddlers were exposed to familiar trials which tested their knowledge of known word–referent pairs, disambiguation trials in which novel word–referent pairs were indirectly learned, and retention trials which probed their recognition of the newly-learned word–referent pairs. In Experiment 1 (n = 48), lip movements were present during familiar and disambiguation trials, but not retention trials. In Experiment 2 (n = 48), lip movements were present during all three trial types. In Experiment 3 (bilinguals only, n = 24), a still face with no lip movements was present in all three trial types. While toddlers succeeded in the familiar and disambiguation trials of every experiment, success in the retention trials was only found in Experiment 2. This work suggests that the extra-linguistic support provided by lip movements improved the learning and recognition of the novel words.

Bridging ears and eyes when learning spoken words: On the effects of bilingual experience at 30 months

From the very first moments of their lives, infants selectively attend to the visible orofacial movements of their social partners and apply their exquisite speech perception skills to the service of lexical learning. Here we explore how early bilingual experience modulates children's ability to use visible speech as they form new lexical representations. Using a cross-modal word-learning task, bilingual children aged 30 months were tested on their ability to learn new lexical mappings in either the auditory or the visual modality. Lexical recognition was assessed either in the same modality as the one used at learning ('same modality' condition: auditory test after auditory learning, visual test after visual learning) or in the other modality ('cross-modality' condition: visual test after auditory learning, auditory test after visual learning). The results revealed that like their monolingual peers, bilingual children successfully learn new words in either the auditory or the visual modality and show cross-modal recognition of words following auditory learning. Interestingly, as opposed to monolinguals, they also demonstrate cross-modal recognition of words upon visual learning. Collectively, these findings indicate a bilingual edge in visual word learning, expressed in the capacity to form a recoverable cross-modal representation of visually learned words.

Bilingual phonological acquisition: the influence of language-internal, language-external, and lexical factors

This study examines the influence of language-internal (frequency and complexity of linguistic properties), language-external (percent French input, socioeconomic status (SES), and gender), and lexical factors (size of total and French vocabulary) on the phonological production abilities of monolingual and bilingual French-speaking children, aged 2;6. Children participated in an object and picture naming task in which they produced words selected to test different phonological properties. The bilinguals' first languages were coded in terms of the frequency and complexity of these phonological properties. Results indicated that bilinguals who spoke languages characterized by high frequency/complexity of codas and clusters had superior results in their coda and cluster accuracy in comparison to monolinguals. Bilinguals also had better coda and cluster accuracy scores than monolinguals. These findings provide evidence for cross-linguistic interaction in combination with a 'general bilingual effect'. In addition, percent French exposure, SES, total vocabulary, and gender influenced phonological production.