The relative contribution of acoustic signals versus movement cues in group coordination and collective decision-making

To benefit from group living, individuals need to maintain cohesion and coordinate their activities. Effective communication thus becomes critical, facilitating rapid coordination of behaviours and reducing consensus costs when group members have differing needs and information. In many bird and mammal species, collective decisions rely on acoustic signals in some contexts but on movement cues in others. Yet, to date, there is no clear conceptual framework that predicts when decisions should evolve to be based on acoustic signals versus movement cues. Here, we first review how acoustic signals and movement cues are used for coordinating activities. We then outline how information masking, discrimination ability (Weber's Law) and encoding limitations, as well as trade-offs between these, can identify which types of collective behaviours likely rely on acoustic signals or movement cues. Specifically, our framework proposes that behaviours involving the timing of events or expression of specific actions should rely more on acoustic signals, whereas decisions involving complex choices with multiple options (e.g. direction and destination) should generally use movement cues because sounds are more vulnerable to information masking and Weber's Law effects. We then discuss potential future avenues of enquiry, including multimodal communication and collective decision-making by mixed-species animal groups. This article is part of the theme issue 'The power of sound: unravelling how acoustic communication shapes group dynamic'.

Listening to your partner: serotonin increases male responsiveness to female vocal signals in mice

The context surrounding vocal communication can have a strong influence on how vocal signals are perceived. The serotonergic system is well-positioned for modulating the perception of communication signals according to context, because serotonergic neurons are responsive to social context, influence social behavior, and innervate auditory regions. Animals like lab mice can be excellent models for exploring how serotonin affects the primary neural systems involved in vocal perception, including within central auditory regions like the inferior colliculus (IC). Within the IC, serotonergic activity reflects not only the presence of a conspecific, but also the valence of a given social interaction. To assess whether serotonin can influence the perception of vocal signals in male mice, we manipulated serotonin systemically with an injection of its precursor 5-HTP, and locally in the IC with an infusion of fenfluramine, a serotonin reuptake blocker. Mice then participated in a behavioral assay in which males suppress their ultrasonic vocalizations (USVs) in response to the playback of female broadband vocalizations (BBVs), used in defensive aggression by females when interacting with males. Both 5-HTP and fenfluramine increased the suppression of USVs during BBV playback relative to controls. 5-HTP additionally decreased the baseline production of a specific type of USV and male investigation, but neither drug treatment strongly affected male digging or grooming. These findings show that serotonin modifies behavioral responses to vocal signals in mice, in part by acting in auditory brain regions, and suggest that mouse vocal behavior can serve as a useful model for exploring the mechanisms of context in human communication.

Alteration of the temporal association between courtship audio and visual components affects female sexual response

Some multimodal signals-that is, occurring in more than one sensory modality-appear to carry additional information which is not present when component signals are presented separately. To understand the function of male ring dove's (Streptopelia risoria) multimodal courtship, we used audiovisual playback of male displays to investigate female response to stimuli differing in their audiovisual timing. From natural courtship recordings, we created a shifted stimulus where audio was shifted relative to video by a fixed value and a jittered stimulus, where each call was moved randomly along the visual channel. We presented 3 groups of females with the same stimulus type, that is, control, shifted, and jittered, for 7 days. We recorded their behavior and assessed pre- and post-test blood estradiol concentration. We found that playback exposure increased estradiol levels, confirming that this technique can be efficiently used to study doves' sexual communication. Additionally, chasing behavior (indicating sexual stimulation) increased over experimental days only in the control condition, suggesting a role of multimodal timing on female response. This stresses the importance of signal configuration in multimodal communication, as additional information is likely to be contained in the temporal association between modalities.

Production of multimodal signals to assert social dominance in white-lipped peccary (Tayassu pecari)

In this study we aimed to examine whether the 'redundancy' (a backup function to ensure the signal transmission) or 'multiple messages' (sensory communication system in combination) hypothesis would explain the function of multimodal communication of white-lipped peccaries (Tayassu pecari-WLPs). We also aimed to assess the individual factors (the social rank and sex of the sender) influencing the production of, and responses to unimodal and multimodal signals. We determined the social rank of 21 WLPs living in two captive groups and quantified the production of unimodal and multimodal signals when displaying threatening and submissive behaviors. WLPs most often produce multimodal signals independent of a previous unimodal signal failure, which suggests that they were adding more information, such as the sender's size, rather than merely increasing efficacy by engaging a different receiver's sensory channel. There was no effect of the sender's sex in the production of, and responses to, multimodal signals. However, the higher the sender's social rank, the greater the production of multimodal signals when WLPs were displaying threatening behaviors; whereas the lower the sender's social rank, the greater the production of multimodal signals when displaying submission behaviors. Multimodal signals elicited more non-aggressive responses than did the unimodal signals when displaying a threat. Moreover, the higher the sender's social rank, the greater the occurrence of non-aggressive responses to multimodal signals when displaying a threat; whereas the opposite occurred when displaying submission. Our findings support the 'multiple messages' hypothesis to explain the function of multimodal signaling during agonistic interactions in WLPs. Additionally, both the production of, and responses to, multimodal signals are related to the sender's social rank. These results allow us to suggest that the production of multimodal signals may have a key role in mitigating conflict and thus promoting group cohesion among white-lipped peccaries.

Birds differentially prioritize visual and olfactory foraging cues depending on habitat of origin and sex

Animals interpret their environment by combining information from multiple senses. The relative usefulness of different senses may vary between species, habitats and sexes; yet, how multimodal stimuli are integrated and prioritized is unknown for most taxa. We experimentally assessed foraging preferences of great tits (Parus major) to test whether urban and forest individuals prioritize visual and olfactory cues differently during foraging. We trained 13 wild-caught birds to associate multimodal (colour + odour) cues with a food reward and assessed their foraging preferences in a cue-separation test. In this, the birds could choose between the multimodal training cue and its olfactory or visual components. Our results suggest that the birds did not perceive multimodal cues in an integrated way, as their response was not stronger than for unimodal cue components. Urban birds preferred olfactory cues, while forest birds preferred visual cues. Nevertheless, female birds preferred the multimodal cue, while males foraged more randomly with respect to which cue was present. These findings contribute to our understanding of the relative roles of vision and olfaction in bird foraging behaviour. Future work should focus on how habitat- and sex-specific sensory prioritization modifies bird foraging behaviour and foraging success in the context of urban adaptations across populations.

Multisensory modalities increase working memory for mating signals in a treefrog

Animal choruses, such as those found in insects and frogs, are often intermittent. Thus, females sampling males in the chorus might have to remember the location of the potential mates' calls during periods of silence. Although a number of studies have shown that frogs use and prefer multimodal mating signals, usually acoustic plus visual, it is not clear why they do so. Here we tested the hypothesis that preference for multimodal signals over unimodal signals might be due to multimodal signals instantiating longer memories than unimodal signals, particularly during the inter-chorus intervals. We tested this hypothesis in serrate-legged small treefrogs Kurixalus odontotarsus whose males produce advertisement calls accompanied by conspicuous vocal sac inflation. Females were tested with acoustic and acoustic + visual (video of inflating-deflating vocal sac) mating calls. We found that females prefer multimodal calls over unimodal, audio-only calls. Furthermore, multimodal calls are still preferred after a silent period of up to 30 s, a time that spans the average silent period of the chorus. This was not true of unimodal calls. Our results demonstrate that a multimodal signal can engage longer working memory than a unimodal signal, and thus female memory might favour the evolution of multimodal signals in males through sexual selection. Selection might also favour female preference for multimodal signals if longer memory facilitates mate searching and assessment. Our study does not allow us to elucidate the sequence of evolution of this trait and preference.

Optimal multisensory integration

Animals are often confronted with potentially informative stimuli from a variety of sensory modalities. Although there is a large proximate literature demonstrating multisensory integration, no general framework explains why animals integrate. We developed and tested a quantitative model that explains why multisensory integration is not always adaptive and explains why unimodal decision-making might be favored over multisensory integration. We present our model in terms of a prey that must determine the presence or absence of a predator. A greater chance of encountering a predator, a greater benefit of correctly responding to a predator, a lower benefit of correctly foraging, or a greater uncertainty of the second stimulus favors integration. Uncertainty of the first stimulus may either increase or decrease the favorability of integration. In three field studies, we demonstrate how our model can be empirically tested. We evaluated the model with field studies of yellow-bellied marmots (Marmota flaviventer) by presenting marmots with an olfactory-acoustic predator stimulus at a feed station. We found some support for the model's prediction that integration is favored when the second stimulus is less noisy. We hope additional predictions of the model will guide future empirical work that seeks to understand the extent to which multimodal integration might be situation dependent. We suggest that the model is generalizable beyond antipredator contexts and can be applied within or between individuals, populations, or species.

Multisensory integration is often studied from a very proximate view that simply describes the process of integration. We developed a model, the first of its kind, to investigate the situations under which multisensory integration is adaptive. We empirically evaluated the model by investigating the conditions under which yellow-bellied marmots integrated predatory scents and sounds. We found that integration can depend on an animal's situation at a given point in time.

Booming far: the long-range vocal strategy of a lekking bird

The pressures of selection acting on transmission of information by acoustic signals are particularly high in long-distance communication networks. Males of the North African houbara bustard (Chlamydotis undulata undulata) produce extremely low-frequency vocalizations called 'booms' as a component of their courtship displays. These displays are performed on sites separated by a distance of on average 550 m, constituting exploded leks. Here, we investigate the acoustic features of booms involved in species-specific identity. We first assessed the modifications of acoustic parameters during boom transmission at long range within the natural habitat of the species, finding that the frequency content of booms was reliably transmitted up to 600 m. Additionally, by testing males' behavioural responses to playbacks of modified signals, we found that the presence of the second harmonic and the frequency modulation are the key parameters for species identification, and also that a sequence of booms elicited stronger responses than a single boom. Thus, the coding-decoding process relies on redundant and propagation-resistant features, making the booms particularly well adapted for the long-range transmission of information between males. Moreover, by experimentally disentangling the presentation of visual and acoustic signals, we showed that during the booming phase of courtship, the two sensory modalities act in synergy. The acoustic component is dominant in the context of intra-sexual competition. While the visual component is not necessary to induce agonistic response, it acts as an amplifier and reduces the time of detection of the signaller. The utilization of these adaptive strategies allows houbara males to maximize the active space of vocalizations emitted in exploded leks.

Wild chimpanzees' use of single and combined vocal and gestural signals

ABSTRACT

We describe the individual and combined use of vocalizations and gestures in wild chimpanzees. The rate of gesturing peaked in infancy and, with the exception of the alpha male, decreased again in older age groups, while vocal signals showed the opposite pattern. Although gesture-vocal combinations were relatively rare, they were consistently found in all age groups, especially during affiliative and agonistic interactions. Within behavioural contexts rank (excluding alpha-rank) had no effect on the rate of male chimpanzees' use of vocal or gestural signals and only a small effect on their use of combination signals. The alpha male was an outlier, however, both as a prolific user of gestures and recipient of high levels of vocal and gesture-vocal signals. Persistence in signal use varied with signal type: chimpanzees persisted in use of gestures and gesture-vocal combinations after failure, but where their vocal signals failed they tended to add gestural signals to produce gesture-vocal combinations. Overall, chimpanzees employed signals with a sensitivity to the public/private nature of information, by adjusting their use of signal types according to social context and by taking into account potential out-of-sight audiences. We discuss these findings in relation to the various socio-ecological challenges that chimpanzees are exposed to in their natural forest habitats and the current discussion of multimodal communication in great apes.

SIGNIFICANCE STATEMENT

All animal communication combines different types of signals, including vocalizations, facial expressions, and gestures. However, the study of primate communication has typically focused on the use of signal types in isolation. As a result, we know little on how primates use the full repertoire of signals available to them. Here we present a systematic study on the individual and combined use of gestures and vocalizations in wild chimpanzees. We find that gesturing peaks in infancy and decreases in older age, while vocal signals show the opposite distribution, and patterns of persistence after failure suggest that gestural and vocal signals may encode different types of information. Overall, chimpanzees employed signals with a sensitivity to the public/private nature of information, by adjusting their use of signal types according to social context and by taking into account potential out-of-sight audiences.

Cross-Modal Correspondences in Non-human Mammal Communication

For both humans and other animals, the ability to combine information obtained through different senses is fundamental to the perception of the environment. It is well established that humans form systematic cross-modal correspondences between stimulus features that can facilitate the accurate combination of sensory percepts. However, the evolutionary origins of the perceptual and cognitive mechanisms involved in these cross-modal associations remain surprisingly underexplored. In this review we outline recent comparative studies investigating how non-human mammals naturally combine information encoded in different sensory modalities during communication. The results of these behavioural studies demonstrate that various mammalian species are able to combine signals from different sensory channels when they are perceived to share the same basic features, either because they can be redundantly sensed and/or because they are processed in the same way. Moreover, evidence that a wide range of mammals form complex cognitive representations about signallers, both within and across species, suggests that animals also learn to associate different sensory features which regularly co-occur. Further research is now necessary to determine how multisensory representations are formed in individual animals, including the relative importance of low level feature-related correspondences. Such investigations will generate important insights into how animals perceive and categorise their environment, as well as provide an essential basis for understanding the evolution of multisensory perception in humans.

The bark, the howl and the bark-howl: Identity cues in dingoes' multicomponent calls

Dingoes (genus Canis) produce a stereotyped bark-howl vocalisation, which is a unimodal complex signal formed by the concatenation of two call types (a bark and a howl). Bark-howls may function as alarm signals, although there has been no empirical investigation of this vocalisation's structure or function. We quantified the content and efficacy of the bark and howl segments separately and when combined, using 140 calls from 10 individuals. We found that both segments are individually distinctive, although howl segments are more accurately classified, suggesting a higher level of individuality. Furthermore, howls convey signature characteristics that are conserved across different contexts of production, and thus may act as 'identity signals'. The individual distinctiveness of full bark-howls increases above that of isolated segments, which may be a result of selection on improved signal discriminability. Propagation tests revealed that bark-howls are best described as medium-range signals, with both segments potentially allowing for individual discrimination up to 200m regardless of environmental conditions. We discuss our findings regarding the fitness benefits of encoding identity cues in a potential alarm call and propose additional hypotheses for the function(s) of bark and howl segments.

Aggressive Bimodal Communication in Domestic Dogs, Canis familiaris

Evidence of animal multimodal signalling is widespread and compelling. Dogs’ aggressive vocalisations (growls and barks) have been extensively studied, but without any consideration of the simultaneously produced visual displays. In this study we aimed to categorize dogs’ bimodal aggressive signals according to the redundant/non-redundant classification framework. We presented dogs with unimodal (audio or visual) or bimodal (audio-visual) stimuli and measured their gazing and motor behaviours. Responses did not qualitatively differ between the bimodal and two unimodal contexts, indicating that acoustic and visual signals provide redundant information. We could not further classify the signal as ‘equivalent’ or ‘enhancing’ as we found evidence for both subcategories. We discuss our findings in relation to the complex signal framework, and propose several hypotheses for this signal’s function.

An Evolutionary Comparison of the Handicap Principle and Hybrid Equilibrium Theories of Signaling

The handicap principle has come under significant challenge both from empirical studies and from theoretical work. As a result, a number of alternative explanations for honest signaling have been proposed. This paper compares the evolutionary plausibility of one such alternative, the “hybrid equilibrium,” to the handicap principle. We utilize computer simulations to compare these two theories as they are instantiated in Maynard Smith’s Sir Philip Sidney game. We conclude that, when both types of communication are possible, evolution is unlikely to lead to handicap signaling and is far more likely to result in the partially honest signaling predicted by hybrid equilibrium theory.

The temporal multimodal influence of optical and auditory cues on the repellent behavior of ring-billed gulls (Larus delewarensis)

Context A generation of new animal repellents is based on the premise that threat stimuli are best interpreted through multiple sensory pathways. Ring-billed gulls (RBG; Larus delawarensis) offer a unique opportunity to assess the efficacy of multimodal repellents over time. This pest species is repelled by both auditory and optical cues and persists in stable populations, often remaining in the same colony for life. This distinctive attribute makes it possible to assess colonies independently over time and space. Aims We assessed the unimodal (single-cue treatment) and multimodal (paired-cue) response by RBG to auditory (conspecific distress call) and optical (green or red laser) cues, along with a double-negative control (flashlight aimed at ground, background noise). Methods All stimuli were investigated separately and together within a 3 × 2 factorial design randomised by treatment and site. We predicted that paired stimuli would generate more pronounced (number of gulls fleeing from a roost) and faster (flight initiation time) responses than stimuli presented alone with a control. Key results The distress call was more effective than either visual signal and almost nullified our ability to detect a multimodal response. However, the multimodal influence was detected on two levels. Gulls were more likely to flee from either paired treatment (optical + auditory) than from unimodal stimuli (laser light only; P < 0.001) and gulls fled more quickly from multiple cues (P < 0.001). A more subtle, but important, benefit was observed in that – over time – gulls were more likely to flee from either paired treatment (optical or auditory), but not from unimodal treatments (P < 0.005). The latter response may have been due to a fear-conditioned generalisation. Conclusions We provide evidence and a causal mechanism to address why multimodal stimuli may be more efficacious as deterrents than single-mode treatments. This species may be more effectively managed, over longer periods of time, through the use of multimodal repellents. Implications A better understanding of how multimodal repellents function may help frame novel approaches to animal conservation and to assay better tools and repellents for wildlife management. Even modest multimodal benefits may justify their use, if they delay habituation over time.

Multimodal integration in behaving chickens

In everyday life we constantly perceive and discriminate between a large variety of sensory inputs, the far majority of which consists of more than one modality. We performed two experiments to investigate whether chickens use the information present in multimodal signals. To test whether audiovisual stimuli are better detected than visual or acoustic stimuli alone, we first measured the detection threshold with a staircase paradigm. We found that chickens were able to detect weaker stimuli using audiovisual stimuli. Next, we tested whether the multimodal nature of a stimulus also increases the discrimination between two stimuli by measuring the smallest difference that the animals could still distinguish from each other. We found that chickens can discriminate smaller differences using audiovisual stimuli in comparison to visual stimuli but not in comparison to acoustic stimuli. Thus, even in a generalist species such as the chicken, the benefits from multimodal integration are exploited for sensory processing.

Signaling in multiple modalities in male rhesus macaques: sex skin coloration and barks in relation to androgen levels, social status, and mating behavior

The past decade has seen an increasing shift in animal communication towards more studies that incorporate aspects of signaling in multiple modalities. Although nonhuman primates are an excellent group for studying the extent to which different aspects of condition may be signaled in different modalities, and how such information may be integrated during mate choice, very few studies of primate species have incorporated such analyses. Here, we present data from free-ranging male rhesus macaques on sex skin coloration (modeled to receiver perception), bark vocal signals, androgen levels, morphometric variables, dominance status, and female mate choice. We show that, consistent with data on females, most intra- and interindividual variation in sex skin appearance occurs in luminance rather than color. Sex skin luminance was significantly correlated across different skin regions. Sex skin luminance did not correlate with the majority of bark parameters, suggesting the potential for the two signals to convey different information. Sex skin appearance was not related to androgen levels although we found some evidence for links between androgen levels and bark parameters, several of which were also related to morphometric variables. We found no evidence that either signal was related to male dominance rank or used in female mate choice, though more direct measures of female proceptive behavior are needed. Overall, the function of male sex skin coloration in this species remains unclear. Our study is among the first nonhuman primate studies to incorporate measurements of multiple signals in multiple modalities, and we encourage other authors to incorporate such analyses into their work.

Lateralization of social cognition in the domestic chicken (Gallus gallus)

In this paper, we report on the ongoing work in our laboratories on the effect of lateralization produced by light exposure in the egg on social cognition in the domestic chick (Gallus gallus). The domestic chick possesses a lateralized visual system. This has effects on the chick's perception towards and interaction with its environment. This includes its ability to live successfully within a social group. We show that there is a tendency for right brain hemisphere dominance when performing social cognitive actions. As such, chicks show a left hemispatial bias for approaching a signalled target object, tend to perceive gaze and faces of human-like masks more effectively when using their left eye, are able to inhibit a pecking response more effectively when viewing a neighbour tasting a bitter substance with their left eye, and are better able to perform a transitive inference task when exposed to light in the egg and when forced to use their left eye only compared to dark-hatched or right eye chicks. Some of these effects were sex specific, with male chicks tending to show an increased effect of lateralization on their behaviours. These data are discussed in terms of overall social cognition in group living.

Silent tidbitting in male fowl, Gallus gallus: a referential visual signal with multiple functions

With the notable exception of bee dances, there are no established examples of multimodal referential signals. The food calls of male fowl, Gallus gallus, are functionally referential and the acoustic component of a multimodal display. However, the specificity of the receiver's response to the visual component (tidbitting) has never been tested. Here we provide the first detailed analysis of tidbitting, and test the hypothesis that these characteristic movements are functionally referential. We conducted a playback experiment with five high-definition video stimuli: Silent tidbit,Matched-frequency motion in the opposite direction, Silent crows, Inactive male and Empty cage. Females searched for food more during Silent tidbitting than under any other condition, suggesting that this visual display specifically predicts the presence of food and hence has similar functional properties to food calls. Silent tidbitting was also singularly effective at evoking approach and close inspection, which may enhance signal memorability. These social responses suggest that the visual component of the display has the unique function of triggering assessment of signaler identity and quality as a potential mate. The acoustic and visual components are hence redundant as a food signal, but synergistic when additional functions are considered. These findings emphasize the perceptual complexity of multimodal displays and provide the first demonstration of multimodal referential signaling in a vertebrate.