Natural reference: A phylo- and ontogenetic perspective on the comprehension of iconic gestures and vocalizations

Do 14-17-month-old infants use iconic speech and gesture cues to interpret word meanings?a)

This experimental study investigated whether infants use iconicity in speech and gesture cues to interpret word meanings. Specifically, we tested infants' sensitivity to size sound symbolism and iconic gesture cues and asked whether combining these cues in a multimodal fashion would enhance infants' sensitivity in a superadditive manner. Thirty-six 14-17-month-old infants participated in a preferential looking task in which they heard a spoken nonword (e.g., "zudzud") while observing a small and large object (e.g., a small and large square). All infants were presented with an iconic cue for object size (small or large) (1) in the pitch of the spoken non-word (high vs low), (2) in gesture (small or large), or (3) congruently in pitch and gesture (e.g., a high pitch and small gesture indicating a small square). Infants did not show a preference for congruently sized objects in any iconic cue condition. Bayes factor analyses showed moderate to strong support for the null hypotheses. In conclusion, 14-17-month-old infants did not use iconic pitch cues, iconic gesture cues, or iconic multimodal cues (pitch and gesture) to associate speech sounds with their referents. These findings challenge theories that emphasize the role of iconicity in early language development.

Meaning as mentalization

The way we establish meaning has been a profound question not only in language research but in developmental science as well. The relation between linguistic form and content has been loosened up in recent pragmatic approaches to communication, showing that code-based models of language comprehension must be augmented by context-sensitive, pragmatic-inferential mechanisms to recover the speaker's intended meaning. Language acquisition has traditionally been thought to involve building a mental lexicon and extracting syntactic rules from noisy linguistic input, while communicative-pragmatic inferences have also been argued to be indispensable. Recent research findings exploring the electrophysiological indicator of semantic processing, the N400, have raised serious questions about the traditional separation between semantic decoding and pragmatic inferential processes. The N400 appears to be sensitive to mentalization-the ability to attribute beliefs to social partners-already from its developmental onset. This finding raises the possibility that mentalization may not simply contribute to pragmatic inferences that enrich linguistic decoding processes but that the semantic system may be functioning in a fundamentally mentalistic manner. The present review first summarizes the key contributions of pragmatic models of communication to language comprehension. Then, it provides an overview of how communicative intentions are interpreted in developmental theories of communication, with a special emphasis on mentalization. Next, it discusses the sensitivity of infants to the information-transmitting potential of language, their ability to pick up its code-like features, and their capacity to track language comprehension of social partners using mentalization. In conclusion, I argue that the recovery of meaning during linguistic communication is not adequately modeled as a process of code-based semantic retrieval complemented by pragmatic inferences. Instead, the semantic system may establish meaning, as intended, during language comprehension and acquisition through mentalistic attribution of content to communicative partners.

Evidence for compositionality in baboons (Papio papio) through the test case of negation

Can non-human animals combine abstract representations much like humans do with language? In particular, can they entertain a compositional representation such as 'not blue'? Across two experiments, we demonstrate that baboons (Papio papio) show a capacity for compositionality. Experiment 1 showed that baboons can entertain negative, compositional, representations: they can learn to associate a cue with iconically related referents (e.g., a blue patch referring to all blue objects), but also to the complement set associated with it (e.g., a blue patch referring to all non-blue objects). Strikingly, Experiment 2 showed that baboons not only learn to associate a cue with iconically related referents, but can learn to associate complex cues (composed of the same cue and an additional visual element) with the complement object set. Thus, they can learn an operation, instantiated by this additional visual element, that can be compositionally combined with previously learned cues. These results significantly reduce any claim that would make the manipulation and combination of abstract representations a solely human privilege.

Great ape communication as contextual social inference: a computational modelling perspective

Human communication has been described as a contextual social inference process. Research into great ape communication has been inspired by this view to look for the evolutionary roots of the social, cognitive and interactional processes involved in human communication. This approach has been highly productive, yet it is partly compromised by the widespread focus on how great apes use and understand individual signals. This paper introduces a computational model that formalizes great ape communication as a multi-faceted social inference process that integrates (a) information contained in the signals that make up an utterance, (b) the relationship between communicative partners and (c) the social context. This model makes accurate qualitative and quantitative predictions about real-world communicative interactions between semi-wild-living chimpanzees. When enriched with a pragmatic reasoning process, the model explains repeatedly reported differences between humans and great apes in the interpretation of ambiguous signals (e.g. pointing or iconic gestures). This approach has direct implications for observational and experimental studies of great ape communication and provides a new tool for theorizing about the evolution of uniquely human communication. This article is part of the theme issue 'Revisiting the human 'interaction engine': comparative approaches to social action coordination'.

Bo-NO-bouba-kiki: picture-word mapping but no spontaneous sound symbolic speech-shape mapping in a language trained bonobo

Humans share the ability to intuitively map 'sharp' or 'round' pseudowords, such as 'bouba' versus 'kiki', to abstract edgy versus round shapes, respectively. This effect, known as sound symbolism, appears early in human development. The phylogenetic origin of this phenomenon, however, is unclear: are humans the only species capable of experiencing correspondences between speech sounds and shapes, or could similar effects be observed in other animals? Thus far, evidence from an implicit matching experiment failed to find evidence of this sound symbolic matching in great apes, suggesting its human uniqueness. However, explicit tests of sound symbolism have never been conducted with nonhuman great apes. In the present study, a language-competent bonobo completed a cross-modal matching-to-sample task in which he was asked to match spoken English words to pictures, as well as 'sharp' or 'round' pseudowords to shapes. Sound symbolic trials were interspersed among English words. The bonobo matched English words to pictures with high accuracy, but did not show any evidence of spontaneous sound symbolic matching. Our results suggest that speech exposure/comprehension alone cannot explain sound symbolism. This lends plausibility to the hypothesis that biological differences between human and nonhuman primates could account for the putative human specificity of this effect.

A Cross-Sectional Test of Sign Creation by Children in the Gesture and Vocal Modalities

Naturalistic studies show that children can create language-like communication systems in the absence of conventional language. However, experimental evidence is mixed. We address this discrepancy using an experimental paradigm that simulates naturalistic sign creation. Specifically, we tested if a sample of 6- to 12-year-old children (52 girls and 56 boys drawn from an urban, predominantly white population in Western Australia) can comprehend and create novel gestural and vocal signs. Experiment 1 tested children's ability to comprehend novel signs. Experiment 2 tested children's ability to create novel signs. Results show that children can comprehend and create gestural and vocal signs, that communication is more successful in the gesture modality, and that older children outperform younger children.

Constructing a protolanguage: reconstructing prehistoric languages in a usage-based construction grammar framework

Construction grammar is an approach to language that posits that units and structures in language can be exhaustively described as pairings between form and meaning. These pairings are called constructions and can have different degrees of abstraction, i.e. they span the entire range from very concrete (armadillo, avocado) to very abstract constructions such as the ditransitive construction (I gave her a book). This approach has been applied to a wide variety of different areas of research in linguistics, such as how new constructions emerge and change historically. It has also been applied to investigate the evolutionary emergence of modern fully fledged language, i.e. the question of how systems of constructions can arise out of prelinguistic communication. In this paper, we review the contribution of usage-based construction grammar approaches to language change and language evolution to the questions of (i) the structure and nature of prehistoric languages and (ii) how constructions in prehistoric languages emerged out of non-linguistic or protolinguistic communication. In particular, we discuss the possibilities of using constructions as the main unit of analysis both in reconstructing predecessors of existing languages (protolanguages) and in formulating theories of how a potential predecessor of human language in general (protolanguage) must have looked like. This article is part of the theme issue 'Reconstructing prehistoric languages'.

Spontaneous (minimal) ritual in non-human great apes?

The potential for rituals in non-human great apes (apes) is an understudied topic. We derive a minimal definition of ritual and then examine the currently available evidence for it in untrained and non-enculturated apes. First, we examine whether such apes show evidence for the two main components of our minimal definition of ritual: symbolism and copying. Second, we examine if there are actual cases already identifiable today that may fit all aspects of our minimal definition of ritual-or whether there are at least cases that fit some aspects (proto-ritual). We find that apes are not likely to spontaneously practise minimal ritual, but we claim that the highest expected likelihood of occurrence is in the results-copying domain. Yet, we did not find actual cases of minimal ritual in apes-including those involving environmental results. We did, however, find some cases that may match at least part of our minimal ritual definition-which we termed proto-ritual. At least two out of three potential cases of such proto-rituals that we identified (rain dance, object-in-ear and surplus nest-making procedures) do revolve around results. Overall, apes do not show much, or very clear, evidence for even minimal ritual, but may sometimes show proto-ritual. However, dedicated ape ritual studies are currently lacking, and future work may identify ape ritual (or clearer cases of proto-ritual). We discuss the implications of our preliminary finding for inferences of ritual in the last common ancestor of humans and apes. This article is part of the theme issue 'Ritual renaissance: new insights into the most human of behaviours'.

Learning Novel Skills From Iconic Gestures: A Developmental and Evolutionary Perspective

Cumulative cultural learning has been argued to rely on high-fidelity copying of other individuals' actions. Iconic gestures of actions have no physical effect on objects in the world but merely represent actions that would have an effect. Learning from iconic gestures thus requires paying close attention to the teacher's precise bodily movements-a prerequisite for high-fidelity copying. In three studies, we investigated whether 2- and 3-year-old children (N = 122) and great apes (N = 36) learn novel skills from iconic gestures. When faced with a novel apparatus, participants watched an experimenter perform either an iconic gesture depicting the action necessary to open the apparatus or a gesture depicting a different action. Children, but not great apes, profited from iconic gestures, with older children doing so to a larger extent. These results suggest that high-fidelity copying abilities are firmly in place in humans by at least 3 years of age.

Communicative roots of complex sociality and cognition

Mammals living in more complex social groups typically have large brains for their body size and many researchers have proposed that the primary driver of the increase in brain size through primate and hominin evolution was the selection pressures associated with sociality. Many mammals, and especially primates, use flexible signals that show a high degree of voluntary control and these signals may play an important role in forming and maintaining social relationships between group members. However, the specific role that cognitive skills play in this complex communication, and how in turn this relates to sociality, is still unclear. The hypothesis for the communicative roots of complex sociality and cognition posits that cognitive demands behind the communication needed to form and maintain bonded social relationships in complex social settings drives the link between brain size and sociality. We review the evidence in support of this hypothesis and why key features of cognitively complex communication such as intentionality and referentiality should be more effective in forming and maintaining bonded relationships as compared with less cognitively complex communication. Exploring the link between cognition, communication and sociality provides insights into how increasing flexibility in communication can facilitate the emergence of social systems characterised by bonded social relationships, such as those found in non-human primates and humans. To move the field forward and carry out both within- and among-species comparisons, we advocate the use of social network analysis, which provides a novel way to describe and compare social structure. Using this approach can lead to a new, systematic way of examining social and communicative complexity across species, something that is lacking in current comparative studies of social structure.

Multimodality and the origin of a novel communication system in face-to-face interaction

Face-to-face communication is multimodal at its core: it consists of a combination of vocal and visual signalling. However, current evidence suggests that, in the absence of an established communication system, visual signalling, especially in the form of visible gesture, is a more powerful form of communication than vocalization and therefore likely to have played a primary role in the emergence of human language. This argument is based on experimental evidence of how vocal and visual modalities (i.e. gesture) are employed to communicate about familiar concepts when participants cannot use their existing languages. To investigate this further, we introduce an experiment where pairs of participants performed a referential communication task in which they described unfamiliar stimuli in order to reduce reliance on conventional signals. Visual and auditory stimuli were described in three conditions: using visible gestures only, using non-linguistic vocalizations only and given the option to use both (multimodal communication). The results suggest that even in the absence of conventional signals, gesture is a more powerful mode of communication compared with vocalization, but that there are also advantages to multimodality compared to using gesture alone. Participants with an option to produce multimodal signals had comparable accuracy to those using only gesture, but gained an efficiency advantage. The analysis of the interactions between participants showed that interactants developed novel communication systems for unfamiliar stimuli by deploying different modalities flexibly to suit their needs and by taking advantage of multimodality when required.

The Pervasive Role of Pragmatics in Early Language

Language is a fundamentally social endeavor. Pragmatics is the study of how speakers and listeners use social reasoning to go beyond the literal meanings of words to interpret language in context. In this article, we take a pragmatic perspective on language development and argue for developmental continuity between early nonverbal communication, language learning, and linguistic pragmatics. We link phenomena from these different literatures by relating them to a computational framework (the rational speech act framework), which conceptualizes communication as fundamentally inferential and grounded in social cognition. The model specifies how different information sources (linguistic utterances, social cues, common ground) are combined when making pragmatic inferences. We present evidence in favor of this inferential view and review how pragmatic reasoning supports children's learning, comprehension, and use of language.

Young children spontaneously recreate core properties of language in a new modality

How the world's 6,000+ natural languages have arisen is mostly unknown. Yet, new sign languages have emerged recently among deaf people brought together in a community, offering insights into the dynamics of language evolution. However, documenting the emergence of these languages has mostly consisted of studying the end product; the process by which ad hoc signs are transformed into a structured communication system has not been directly observed. Here we show how young children create new communication systems that exhibit core features of natural languages in less than 30 min. In a controlled setting, we blocked the possibility of using spoken language. In order to communicate novel messages, including abstract concepts, dyads of children spontaneously created novel gestural signs. Over usage, these signs became increasingly arbitrary and conventionalized. When confronted with the need to communicate more complex meanings, children began to grammatically structure their gestures. Together with previous work, these results suggest that children have the basic skills necessary, not only to acquire a natural language, but also to spontaneously create a new one. The speed with which children create these structured systems has profound implications for theorizing about language evolution, a process which is generally thought to span across many generations, if not millennia.

Social and ecological complexity is associated with gestural repertoire size of wild chimpanzees

Increasing our understanding of primate gestural communication can provide new insights into language evolution. A key question in primate communication is the association between the social relationships of primates and their repertoire of gestures. Such analyses can reveal how primates use their repertoire of gestural communication to maintain their networks of family and friends, much as humans use language to maintain their social networks. In this study we examined the association between the repertoire of gestures (overall, manual and bodily gestures, and gestures of different modalities) and social bonds (presence of reciprocated grooming), coordinated behaviors (travel, resting, co‐feeding), and the complexity of ecology (e.g. noise, illumination) and sociality (party size, audience), in wild East African chimpanzees (Pan troglodytes schweinfurthii). A larger repertoire size of manual, visual gestures was associated with the presence of a relationship based on reciprocated grooming and increases in social complexity. A smaller repertoire of manual tactile gestures occurred when the relationship was based on reciprocated grooming. A smaller repertoire of bodily gestures occurred between partners who jointly traveled for longer. Whereas gesture repertoire size was associated with social complexity, complex ecology also influenced repertoire size. The evolution of a large repertoire of manual, visual gestures may have been a key factor that enabled larger social groups to emerge during evolution. Thus, the evolution of the larger brains in hominins may have co‐occurred with an increase in the cognitive complexity underpinning gestural communication and this, in turn, may have enabled hominins to live in more complex social groups.

Sound symbolic congruency detection in humans but not in great apes

Theories on the evolution of language highlight iconicity as one of the unique features of human language. One important manifestation of iconicity is sound symbolism, the intrinsic relationship between meaningless speech sounds and visual shapes, as exemplified by the famous correspondences between the pseudowords 'maluma' vs. 'takete' and abstract curved and angular shapes. Although sound symbolism has been studied extensively in humans including young children and infants, it has never been investigated in non-human primates lacking language. In the present study, we administered the classic "takete-maluma" paradigm in both humans (N = 24 and N = 31) and great apes (N = 8). In a forced choice matching task, humans but not great apes, showed crossmodal sound symbolic congruency effects, whereby effects were more pronounced for shape selections following round-sounding primes than following edgy-sounding primes. These results suggest that the ability to detect sound symbolic correspondences is the outcome of a phylogenetic process, whose underlying emerging mechanism may be relevant to symbolic ability more generally.

Multimodal communication and language origins: integrating gestures and vocalizations

The presence of divergent and independent research traditions in the gestural and vocal domains of primate communication has resulted in major discrepancies in the definition and operationalization of cognitive concepts. However, in recent years, accumulating evidence from behavioural and neurobiological research has shown that both human and non-human primate communication is inherently multimodal. It is therefore timely to integrate the study of gestural and vocal communication. Herein, we review evidence demonstrating that there is no clear difference between primate gestures and vocalizations in the extent to which they show evidence for the presence of key language properties: intentionality, reference, iconicity and turn-taking. We also find high overlap in the neurobiological mechanisms producing primate gestures and vocalizations, as well as in ontogenetic flexibility. These findings confirm that human language had multimodal origins. Nonetheless, we note that in great apes, gestures seem to fulfil a carrying (i.e. predominantly informative) role in close-range communication, whereas the opposite holds for face-to-face interactions of humans. This suggests an evolutionary shift in the carrying role from the gestural to the vocal stream, and we explore this transition in the carrying modality. Finally, we suggest that future studies should focus on the links between complex communication, sociality and cooperative tendency to strengthen the study of language origins.

Orangutans' Comprehension of Zoo Keepers' Communicative Signals

Zoological institutions often encourage cooperative interactions between keepers and animals so as to promote animals' welfare. One useful technique has been conditioning training, whereby animals learn to respond to keepers' requests, which facilitates a number of, otherwise sensitive, daily routines. As various media have been used to convey keepers' instructions, the question remains of which modality is best to promote mutual understanding. Here, we explored this question with two captive female orangutans. In the first experiment, we compared orangutans' understanding of previously acquired instructions when those were performed with verbal signals only, gazes only, gestures only, and when all those modalities were combined. Our results showed that gestures were sufficient for successful comprehension by these two apes. In the second experiment, we asked whether this preference could be driven by the non-arbitrary relationship that gestures bear to what they refer to, through iconicity or pointing. Our results revealed that neither iconicity nor pointing helped the subjects comprehend the keepers' instructions. Our results indicate a preference for instructions given through gestural signals in two captive female orangutans, although its cause remains elusive. Future practice may encourage the use of gestures in communication between keepers and orangutans in general or potentially other animals.