Seeing Iconic Gestures While Encoding Events Facilitates Children's Memory of These Events

Do 14-17-month-old infants use iconic speech and gesture cues to interpret word meanings?a)

This experimental study investigated whether infants use iconicity in speech and gesture cues to interpret word meanings. Specifically, we tested infants' sensitivity to size sound symbolism and iconic gesture cues and asked whether combining these cues in a multimodal fashion would enhance infants' sensitivity in a superadditive manner. Thirty-six 14-17-month-old infants participated in a preferential looking task in which they heard a spoken nonword (e.g., "zudzud") while observing a small and large object (e.g., a small and large square). All infants were presented with an iconic cue for object size (small or large) (1) in the pitch of the spoken non-word (high vs low), (2) in gesture (small or large), or (3) congruently in pitch and gesture (e.g., a high pitch and small gesture indicating a small square). Infants did not show a preference for congruently sized objects in any iconic cue condition. Bayes factor analyses showed moderate to strong support for the null hypotheses. In conclusion, 14-17-month-old infants did not use iconic pitch cues, iconic gesture cues, or iconic multimodal cues (pitch and gesture) to associate speech sounds with their referents. These findings challenge theories that emphasize the role of iconicity in early language development.

Preschool children's discourse competence in different genres and how it relates to iconic gestures

Based on the linguistic analysis of game explanations and retellings, the paper's goal is to investigate the relation of preschool children's situated discourse competence and iconic gestures in different communicative genres, focussing on reinforcing and supplementary speech-gesture-combinations. To this end, a method was developed to evaluate discourse competence as a context-sensitive and interactively embedded phenomenon. The so-called GLOBE-model was adapted to assess discourse competence in relation to interactive scaffolding. The findings show clear links between the children's competence and their parents' scaffolding. We suggest this to be evidence of a fine-tuned interactive support system. The results also indicate strong relations between higher discourse competence and increased frequency of iconic gestures. This applies in particular to reinforcing gestures. The results are interpreted as a confirmation that the speech-gesture system undergoes systematic changes during early childhood, and that gesturing becomes more iconic - and thus more communicative - when discourse competence is growing.

Embodying Similarity and Difference: The Effect of Listing and Contrasting Gestures During U.S. Political Speech

Public speakers like politicians carefully craft their words to maximize the clarity, impact, and persuasiveness of their messages. However, these messages can be shaped by more than words. Gestures play an important role in how spoken arguments are perceived, conceptualized, and remembered by audiences. Studies of political speech have explored the ways spoken arguments are used to persuade audiences and cue applause. Studies of politicians' gestures have explored the ways politicians illustrate different concepts with their hands, but have not focused on gesture's potential as a tool of persuasion. Our paper combines these traditions to ask first, how politicians gesture when using spoken rhetorical devices aimed at persuading audiences, and second, whether these gestures influence the ways their arguments are perceived. Study 1 examined two rhetorical devices-contrasts and lists-used by three politicians during U.S. presidential debates and asked whether the gestures produced during contrasts and lists differ. Gestures produced during contrasts were more likely to involve changes in hand location, and gestures produced during lists were more likely to involve changes in trajectory. Study 2 used footage from the same debates in an experiment to ask whether gesture influenced the way people perceived the politicians' arguments. When participants had access to gestural information, they perceived contrasted items as more different from one another and listed items as more similar to one another than they did when they only had access to speech. This was true even when participants had access to only gesture (in muted videos). We conclude that gesture is effective at communicating concepts of similarity and difference and that politicians (and likely other speakers) take advantage of gesture's persuasive potential.

Strategies for Teaching Verbs to Children with and without Language Impairment

The aim of this feasibility study was to extend the current evidence base on intransitive verb learning by evaluating and comparing three strategies (syntactic cues, semantic cues, combined cues) for teaching novel verbs to expand the vocabularies of children with and without language impairment. Twenty-three children with typical development, seven children with developmental language disorder, and eight children with Down syndrome participated in Studies 1, 2, and 3, respectively. They were taught novel, intransitive verbs using syntactic, semantic, and combined cues and then asked to receptively identify and expressively label the novel verbs. Across all conditions, participants learned novel verbs receptively with large effect sizes and participants with typical development and Down syndrome also learned the verbs expressively with large effect sizes. There were no significant differences between conditions. This study extends word-learning research by evaluating not only receptive but also expressive intransitive verb learning to expand one's vocabulary. The results provide positive evidence for three effective strategies for teaching intransitive verbs to children with and without language impairment.

The role of hand gestures in emotion communication: Do type and size of gestures matter?

We communicate emotions in a multimodal way, yet non-verbal emotion communication is a relatively understudied area of research. In three experiments, we investigated the role of gesture characteristics (e.g., type, size in space) on individuals' processing of emotional content. In Experiment 1, participants were asked to rate the emotional intensity of emotional narratives from the videoclips either with iconic or beat gestures. Participants in the iconic gesture condition rated the emotional intensity higher than participants in the beat gesture condition. In Experiment 2, the size of gestures and its interaction with gesture type were investigated in a within-subjects design. Participants again rated the emotional intensity of emotional narratives from the videoclips. Although individuals overall rated narrow gestures more emotionally intense than wider gestures, no effects of gesture type, or gesture size and type interaction were found. Experiment 3 was conducted to check whether findings of Experiment 2 were due to viewing gestures in all videoclips. We compared the gesture and no gesture (i.e., speech only) conditions and showed that there was not a difference between them on emotional ratings. However, we could not replicate the findings related to gesture size of Experiment 2. Overall, these findings indicate the importance of examining gesture's role in emotional contexts and that different gesture characteristics such as size of gestures can be considered in nonverbal communication.

The Gestural Misinformation Effect in Child Interviews in Switzerland

Gestures embody concepts in the form of universal representations. Researchers have highlighted that social communication often embodies nonverbal behavior. A forensic interviewer’s nonverbal behavior, such as gesturing during an interview, could communicate misleading information and may cause inaccuracies in the interviewees’ testimonies. The current study was conducted in Switzerland and included 108 child participants, in three age groups (a younger sample aged 6–9 years, n = 32) (a middle sample aged 10–11 years, n = 40) and an older sample aged 12–13 years, n = 36). Participants viewed a video and completed an interview about the video, individually, immediately after. During the questioning, the interviewer deliberately misled the interviewees with nonverbal gestures. The results showed that 95 children were misled by at least one gesture and that gestures led to a significant decrease in accuracy. Children also incorporated misleading gestures and reported false information; adding to existing evidence that misinformation can also be communicated through nonverbal gestures. Our findings demonstrate the negative influence of misleading gestures in child eyewitness interviews and provide more evidence for the robustness of the gestural misinformation effect, reported in previous research.

A Hands-On Approach to Learning: Gesture Production During Encoding and its Effect on Narrative Recall

Research has shown that gesture production supports learning across a number of tasks. It is unclear, however, whether gesture production during encoding can support narrative recall, who gesture production benefits most, and whether certain types of gestures are more beneficial than others. This study, therefore, investigated the effect of gesture production during the encoding of a narrative on subsequent narrative recall, and whether individuals' levels of verbal and nonverbal memory moderated this effect. Additionally, this study investigated whether producing certain types of gestures during encoding was more beneficial than others. Participants (N = 90, M_age = 20.43) read aloud a narrative while under instruction to produce gestures, under no specific instruction to produce gestures, or were required to keep their hands behind their back to prevent them from gesturing. Participants completed measures assessing verbal and nonverbal memory. While gesture production during encoding benefitted narrative recall (as measured through specific questions), verbal memory moderated the effect, such that gesture production was more beneficial for individuals with higher than lower verbal memory. Furthermore, producing representational gestures during encoding benefitted recall of points in the narrative at which those gestures were produced, while beat gestures had no effect. Findings have implications for understanding the mechanisms underlying the links between gesture and learning, as well as practical implications in instructional settings.

Encouraging pointing with the right hand, but not the left hand, gives right-handed 3-year-olds a linguistic advantage

Previous research has shown a strong positive association between right-handed gesturing and vocabulary development. However, the causal nature of this relationship remains unclear. In the current study, we tested whether gesturing with the right hand enhances linguistic processing in the left hemisphere, which is contralateral to the right hand. We manipulated the gesture hand children used in pointing tasks to test whether it would affect their performance. In either a linguistic task (verb learning) or a non-linguistic control task (memory), 131 typically developing right-handed 3-year-olds were encouraged to use either their right hand or left hand to respond. While encouraging children to use a specific hand to indicate their responses had no effect on memory performance, encouraging children to use the right hand to respond, compared to the left hand, significantly improved their verb learning performance. This study is the first to show that manipulating the hand with which children are encouraged to gesture gives them a linguistic advantage. Language lateralization in healthy right-handed children typically involves a dominant left hemisphere. Producing right-handed gestures may therefore lead to increased activation in the left hemisphere which may, in turn, facilitate forming and accessing lexical representations. It is important to note that this study manipulated gesture handedness among right-handers and does therefore not support the practice of encouraging children to become right-handed in manual activities. RESEARCH HIGHLIGHTS: Right-handed 3-year-olds were instructed to point to indicate their answers exclusively with their right or left hand in either a memory or verb learning task. Right-handed pointing was associated with improved verb generalization performance, but not improved memory performance. Thus, gesturing with the right hand, compared to the left hand, gives right-handed 3-year-olds an advantage in a linguistic but not a non-linguistic task. Right-handed pointing might lead to increased activation in the left hemisphere and facilitate forming and accessing lexical representations.

The Relation Between Cognitive Abilities and the Distribution of Semantic Features Across Speech and Gesture in 4-year-olds

When young children learn to use language, they start to use their hands in co-verbal gesturing. There are, however, considerable differences between children, and it is not completely understood what these individual differences are due to. We studied how children at 4 years of age employ speech and iconic gestures to convey meaning in different kinds of spatial event descriptions, and how this relates to their cognitive abilities. Focusing on spontaneous illustrations of actions, we applied a semantic feature (SF) analysis to characterize combinations of speech and gesture meaning and related them to the child's visual-spatial abilities or abstract/concrete reasoning abilities (measured using the standardized SON-R 2 1 2 - 7 test). Results show that children with higher cognitive abilities convey significantly more meaning via gesture and less solely via speech. These findings suggest that young children's use of cospeech representational gesturing is positively related to their mental representation and reasoning abilities.

The Role of Iconic Gestures in Speech Comprehension: An Overview of Various Methodologies

Iconic gesture-speech integration is a relatively recent field of investigation with numerous researchers studying its various aspects. The results obtained are just as diverse. The definition of iconic gestures is often overlooked in the interpretations of results. Furthermore, while most behavioral studies have demonstrated an advantage of bimodal presentation, brain activity studies show a diversity of results regarding the brain regions involved in the processing of this integration. Clinical studies also yield mixed results, some suggesting parallel processing channels, others a unique and integrated channel. This review aims to draw attention to the methodological variations in research on iconic gesture-speech integration and how they impact conclusions regarding the underlying phenomena. It will also attempt to draw together the findings from other relevant research and suggest potential areas for further investigation in order to better understand processes at play during speech integration process.

What Is the Buzz About Iconicity? How Iconicity in Caregiver Speech Supports Children's Word Learning

One cue that may facilitate children's word learning is iconicity, or the correspondence between a word's form and meaning. Some have even proposed that iconicity in the early lexicon may serve to help children learn how to learn words, supporting the acquisition of even noniconic, or arbitrary, word-referent associations. However, this proposal remains untested. Here, we investigate the iconicity of caregivers' speech to young children during a naturalistic free-play session with novel stimuli and ask whether the iconicity of caregivers' speech facilitates children's learning of the noniconic novel names of those stimuli. Thirty-four 1.5-2-year-olds (19 girls; half monolingual English learners and half bilingual English-Spanish learners) participated in a naturalistic free-play task with their caregivers followed by a test of word-referent retention. We found that caregivers' use of iconicity, particularly in utterances in which they named the novel stimuli, was associated with the likelihood that children learned that novel name. This result held even when controlling for other factors associated with word learning, such as the concreteness and frequency of words in caregiver speech. Together, the results demonstrate that iconicity not only can serve to help children identify the referent of novel words (as in previous research) but can also support their ability to retain even noniconic word-referent mappings.

The Value of Non-Referential Gestures: A Systematic Review of Their Cognitive and Linguistic Effects in Children's Language Development

Speakers produce both referential gestures, which depict properties of a referent, and non-referential gestures, which lack semantic content. While a large number of studies have demonstrated the cognitive and linguistic benefits of referential gestures as well as their precursor and predictive role in both typically developing (TD) and non-TD children, less is known about non-referential gestures in cognitive and complex linguistic domains, such as narrative development. This paper is a systematic review and narrative synthesis of the research concerned with assessing the effects of non-referential gestures in such domains. A search of the literature turned up 11 studies, collectively involving 898 2- to 8-year-old TD children. Although they yielded contradictory evidence, pointing to the need for further investigations, the results of the six studies-in which experimental tasks and materials were pragmatically based-revealed that non-referential gestures not only enhance information recall and narrative comprehension but also act as predictors and causal mechanisms for narrative performance. This suggests that their bootstrapping role in language development is due to the fact that they have important discourse-pragmatic functions that help frame discourse. These findings should be of particular interest to teachers and future studies could extend their impact to non-TD children.

Gesture Use and Processing: A Review on Individual Differences in Cognitive Resources

Speakers use spontaneous hand gestures as they speak and think. These gestures serve many functions for speakers who produce them as well as for listeners who observe them. To date, studies in the gesture literature mostly focused on group-comparisons or the external sources of variation to examine when people use, process, and benefit from using and observing gestures. However, there are also internal sources of variation in gesture use and processing. People differ in how frequently they use gestures, how salient their gestures are, for what purposes they produce gestures, and how much they benefit from using and seeing gestures during comprehension and learning depending on their cognitive dispositions. This review addresses how individual differences in different cognitive skills relate to how people employ gestures in production and comprehension across different ages (from infancy through adulthood to healthy aging) from a functionalist perspective. We conclude that speakers and listeners can use gestures as a compensation tool during communication and thinking that interacts with individuals' cognitive dispositions.

Learning From Gesture and Action: An Investigation of Memory for Where Objects Went and How They Got There

Speakers often use gesture to demonstrate how to perform actions-for example, they might show how to open the top of a jar by making a twisting motion above the jar. Yet it is unclear whether listeners learn as much from seeing such gestures as they learn from seeing actions that physically change the position of objects (i.e., actually opening the jar). Here, we examined participants' implicit and explicit understanding about a series of movements that demonstrated how to move a set of objects. The movements were either shown with actions that physically relocated each object or with gestures that represented the relocation without touching the objects. Further, the end location that was indicated for each object covaried with whether the object was grasped with one or two hands. We found that memory for the end location of each object was better after seeing the physical relocation of the objects, that is, after seeing action, than after seeing gesture, regardless of whether speech was absent (Experiment 1) or present (Experiment 2). However, gesture and action built similar implicit understanding of how a particular handgrasp corresponded with a particular end location. Although gestures miss the benefit of showing the end state of objects that have been acted upon, the data show that gestures are as good as action in building knowledge of how to perform an action.

Seeing Iconic Gesture Promotes First- and Second-Order Verb Generalization in Preschoolers

This study investigated whether seeing iconic gestures depicting verb referents promotes two types of generalization. We taught 3- to 4-year-olds novel locomotion verbs. Children who saw iconic manner gestures during training generalized more verbs to novel events (first-order generalization) than children who saw interactive gestures (Experiment 1, N = 48; Experiment 2, N = 48) and path-tracing gestures (Experiment 3, N = 48). Furthermore, immediately (Experiments 1 and 3) and after 1 week (Experiment 2), the iconic manner gesture group outperformed the control groups in subsequent generalization trials with different novel verbs (second-order generalization), although all groups saw interactive gestures. Thus, seeing iconic gestures that depict verb referents helps children (a) generalize individual verb meanings to novel events and (b) learn more verbs from the same subcategory.

The effects of observing different gestures during storytelling on the recall of path and event information in 5-year-olds and adults

This study examined (a) how observing different types of gestures while listening to a story affected the recall of path and event information in 5-year-old children (n = 71) and adults (n = 55) and (b) whether the effects of gesture type on children's recall of information were related to individual differences such as working memory, language abilities, spontaneous gesture use, and gesture production during the recall task. Participants were asked four questions to measure their spontaneous gesture frequency. They then listened to a story that included different path and event information. Depending on the assigned condition, participants listened to the story with the narrator producing iconic gestures (gestures having semantic meaning), beat gestures (rhythmic hand movements), or no gesture. We then asked participants to relate what happened in the story and administered a recognition task about the story. Children were given standardized tests to assess their language and working memory skills. Children and adults best recalled the story after observing iconic gestures as compared with children and adults presented with beat gestures or no gestures. Children who were exposed to iconic gestures during encoding better recalled event information than children in the other conditions. Children's language abilities, but not working memory, were related to their recall performance. More important, children with better expressive language abilities benefitted more from seeing iconic gestures. These results suggest that observing iconic gestures at encoding facilitates recall and that children's language skills could play a role in encoding and using specific information provided by gestures.

Sound Symbolism Facilitates Long-Term Retention of the Semantic Representation of Novel Verbs in Three-Year-Olds

Previous research has shown that sound symbolism facilitates action label learning when the test trial used to assess learning immediately followed the training trial in which the (novel) verb was taught. The current study investigated whether sound symbolism benefits verb learning in the long term. Forty-nine children were taught either sound-symbolically matching or mismatching pairs made up of a novel verb and an action video. The following day, the children were asked whether a verb can be used for a scene shown in a video. They were tested with four videos for each word they had been taught. The four videos differed as to whether they contained the same or different actions and actors as in the training video: (1) same-action, same-actor; (2) same-action, different-actor; (3) different-action, same-actor; and (4) different-action, different-actor. The results showed that sound symbolism significantly improved the childrens’ ability to encode the semantic representation of the novel verb and correctly generalise it to a new event the following day. A control experiment ruled out the possibility that children were generalising to the “same-action, different-actor” video because they did not recognize the actor change due to the memory decay. Nineteen children were presented with the stimulus videos that had also been shown to children in the sound symbolic match condition in Experiment 1, but this time the videos were not labeled. In the test session the following day, the experimenter tested the children’s recognition memory for the videos. The results indicated that the children could detect the actor change from the original training video a day later. The results of the main experiment and the control experiment support the idea that a motivated (iconic) link between form and meaning facilitates the symbolic development in children. The current study, along with recent related studies, provided further evidence for an iconic advantage in symbol development in the domain of verb learning. A motivated form-meaning relationship can help children learn new words and store them long term in the mental lexicon.