Who is calling? Optimizing source identification from marmoset vocalizations with hierarchical machine learning classifiers

With their highly social nature and complex vocal communication system, marmosets are important models for comparative studies of vocal communication and, eventually, language evolution. However, our knowledge about marmoset vocalizations predominantly originates from playback studies or vocal interactions between dyads, and there is a need to move towards studying group-level communication dynamics. Efficient source identification from marmoset vocalizations is essential for this challenge, and machine learning algorithms (MLAs) can aid it. Here we built a pipeline capable of plentiful feature extraction, meaningful feature selection, and supervised classification of vocalizations of up to 18 marmosets. We optimized the classifier by building a hierarchical MLA that first learned to determine the sex of the source, narrowed down the possible source individuals based on their sex and then determined the source identity. We were able to correctly identify the source individual with high precisions (87.21%-94.42%, depending on call type, and up to 97.79% after the removal of twins from the dataset). We also examine the robustness of identification across varying sample sizes. Our pipeline is a promising tool not only for source identification from marmoset vocalizations but also for analysing vocalizations of other species.

Measuring rhythms of vocal interactions: a proof of principle in harbour seal pups

Rhythmic patterns in interactive contexts characterize human behaviours such as conversational turn-taking. These timed patterns are also present in other animals, and often described as rhythm. Understanding fine-grained temporal adjustments in interaction requires complementary quantitative methodologies. Here, we showcase how vocal interactive rhythmicity in a non-human animal can be quantified using a multi-method approach. We record vocal interactions in harbour seal pups (Phoca vitulina) under controlled conditions. We analyse these data by combining analytical approaches, namely categorical rhythm analysis, circular statistics and time series analyses. We test whether pups' vocal rhythmicity varies across behavioural contexts depending on the absence or presence of a calling partner. Four research questions illustrate which analytical approaches are complementary versus orthogonal. For our data, circular statistics and categorical rhythms suggest that a calling partner affects a pup's call timing. Granger causality suggests that pups predictively adjust their call timing when interacting with a real partner. Lastly, the ADaptation and Anticipation Model estimates statistical parameters for a potential mechanism of temporal adaptation and anticipation. Our analytical complementary approach constitutes a proof of concept; it shows feasibility in applying typically unrelated techniques to seals to quantify vocal rhythmic interactivity across behavioural contexts. This article is part of a discussion meeting issue 'Face2face: advancing the science of social interaction'.

Spontaneous rhythm discrimination in a mammalian vocal learner

Rhythm and vocal production learning are building blocks of human music and speech. Vocal learning has been hypothesized as a prerequisite for rhythmic capacities. Yet, no mammalian vocal learner but humans have shown the capacity to flexibly and spontaneously discriminate rhythmic patterns. Here we tested untrained rhythm discrimination in a mammalian vocal learning species, the harbour seal (Phoca vitulina). Twenty wild-born seals were exposed to music-like playbacks of conspecific call sequences varying in basic rhythmic properties. These properties were called length, sequence regularity, and overall tempo. All three features significantly influenced seals' reaction (number of looks and their duration), demonstrating spontaneous rhythm discrimination in a vocal learning mammal. This finding supports the rhythm–vocal learning hypothesis and showcases pinnipeds as promising models for comparative research on rhythmic phylogenies.

Equivalence classification, learning by exclusion, and long-term memory in pinnipeds: cognitive mechanisms demonstrated through research with subjects under human care and in the field

Comparative cognition, as an interdisciplinary field, should utilize a holistic approach for studying cognitive mechanisms. We suggest that research with species of interest should employ both work with animals under human care and in the field. This complimentary approach allows for a better understanding of functional cognitive mechanisms themselves (i.e., comparative cognition regarding processes), and how these skill sets can relate to a particular species' ecological niche. We suggest that research evidence for equivalence classification, learning by exclusion, and long-term memory in pinnipeds can provide a foundation for discussion and implementation of a two-pronged methodological approach utilizing 'lab' and field' work. First, we describe evidence from research with pinnipeds under human care supporting each of these cognitive abilities, then follow this with evidence for implications of these mechanisms from complimentary field research. Lastly, we provide a brief discussion of implementation of a purposeful and two-pronged research approach as an understanding of pinnipeds' high levels of cognitive flexibility may underlie their success for navigating the ever-changing, and often human-altered, natural environment.

Mutual mother-pup vocal recognition in the highly colonial Cape fur seal: evidence of discrimination of calls with a high acoustic similarity

The Cape fur seal (Arctocephalus pusillus pusillus) is one of the most colonial mammal species in the world. Females exclusively nurse their pups for 9 to 11 months, during which they alternate frequent foraging trips at sea with suckling periods ashore. The survival of the pup thus depends on the ability of the mother-pup pair to relocate each other among thousands of individuals. Previous work has demonstrated identity information encoded in pup-attraction (PAC) and female-attraction (FAC) calls. Here, we investigated vocal recognition between mother and pup using playbacks of PAC and FAC performed during the breeding season at Pelican Point, Namibia. Both females and pups were able to specifically discriminate the voice of their pup or their mother from non-affiliated pup or mother. Females were able to memorize previous versions of their pup's calls (evidence of recognition up to 73 days after pup's calls recording). Vocal recognition was demonstrated in pups from 1- to 13-weeks old age. Females and pups did not respond differently to the non-filial or non-mother (for pups) stimulus even if it had a strong acoustic similarity with the filial or mother stimulus. This suggested that Cape fur seal mother-pup pairs have high perceptual and cognitive abilities, allowing individuals to identify kin's vocalizations in a very noisy and confusing environment.

Vocal tract allometry in a mammalian vocal learner

Acoustic allometry occurs when features of animal vocalisations can be predicted from body size measurements. Despite this being considered the norm, allometry sometimes breaks, resulting in species sounding smaller or larger than expected for their size. A recent hypothesis suggests that allometry-breaking mammals cluster into two groups: those with anatomical adaptations to their vocal tracts and those capable of learning new sounds (vocal learners). Here, we tested which mechanism is used to escape from acoustic allometry by probing vocal tract allometry in a proven mammalian vocal learner, the harbour seal (Phoca vitulina). We tested whether vocal tract structures and body size scale allometrically in 68 young individuals. We found that both body length and body mass accurately predict vocal tract length and one tracheal dimension. Independently, body length predicts vocal fold length while body mass predicts a second tracheal dimension. All vocal tract measures are larger in weaners than in pups and some structures are sexually dimorphic within age classes. We conclude that harbour seals do comply with anatomical allometric constraints. However, allometry between body size and vocal fold length seems to emerge after puppyhood, suggesting that ontogeny may modulate the anatomy–learning distinction previously hypothesised as clear cut. We suggest that seals, and perhaps other species producing signals that deviate from those expected from their vocal tract dimensions, may break allometry without morphological adaptations. In seals, and potentially other vocal learning mammals, advanced neural control over vocal organs may be the main mechanism for breaking acoustic allometry.

Summary: Harbour seals are vocal learners that can escape acoustic allometry despite complying with anatomical allometric constraints. Advanced neural control over their vocal organs may allow them to break acoustic allometry.

Individuality in houbara chick calls and its dynamics throughout ontogeny

In many taxa, breeding success depends heavily on reliable vocal recognition between parents and offspring. Although the acoustic basis of this recognition has been explored in several species, few studies have examined the evolution of acoustic cues to identity across development. Here, in a captive breeding program, we investigated for the first time the acoustic signals produced by North African houbara bustard Chlamydotis undulata undulata chicks. Two call types (contact and distress) were recorded from 15 chicks in 4 age classes. Acoustic analyses showed that the acoustic parameters of the calls varied systematically with age in both contact and distress calls. However, both call types remained highly stereotyped and individualized between chicks at every tested age, indicating that calls encode reliable information about individual identity throughout development, thus potentially enabling the mother to distinguish her own chicks through their development up to fledging. Playback experiments are now needed to verify such parent-chick recognition in houbara bustards and its efficiency across chick ontogeny.

Vocal plasticity in harbour seal pups

Vocal plasticity can occur in response to environmental and biological factors, including conspecifics' vocalizations and noise. Pinnipeds are one of the few mammalian groups capable of vocal learning, and are therefore relevant to understanding the evolution of vocal plasticity in humans and other animals. Here, we investigate the vocal plasticity of harbour seals (Phoca vitulina), a species with vocal learning abilities observed in adulthood but not puppyhood. To evaluate early mammalian vocal development, we tested 1-3 weeks-old seal pups. We tailored noise playbacks to this species and age to induce seal pups to shift their fundamental frequency (f0), rather than adapt call amplitude or temporal characteristics. We exposed individual pups to low- and high-intensity bandpass-filtered noise, which spanned-and masked-their typical range of f0; simultaneously, we recorded pups' spontaneous calls. Unlike most mammals, pups modified their vocalizations by lowering their f0 in response to increased noise. This modulation was precise and adapted to the particular experimental manipulation of the noise condition. In addition, higher levels of noise induced less dispersion around the mean f0, suggesting that pups may have actively focused their phonatory efforts to target lower frequencies. Noise did not seem to affect call amplitude. However, one seal showed two characteristics of the Lombard effect known for human speech in noise: significant increase in call amplitude and flattening of spectral tilt. Our relatively low noise levels may have favoured f0 modulation while inhibiting amplitude adjustments. This lowering of f0 is unusual, as most animals commonly display no such f0 shift. Our data represent a relatively rare case in mammalian neonates, and have implications for the evolution of vocal plasticity and vocal learning across species, including humans. This article is part of the theme issue 'Voice modulation: from origin and mechanism to social impact (Part I)'.

Can harbor seals (Phoca vitulina) discriminate familiar conspecific calls after long periods of separation?

The ability to discriminate between familiar and unfamiliar calls may play a key role in pinnipeds' communication and survival, as in the case of mother-pup interactions. Vocal discrimination abilities have been suggested to be more developed in pinniped species with the highest selective pressure such as the otariids; yet, in some group-living phocids, such as harbor seals (Phoca vitulina), mothers are also able to recognize their pup's voice. Conspecifics' vocal recognition in pups has never been investigated; however, the repeated interaction occurring between pups within the breeding season suggests that long-term vocal discrimination may occur. Here we explored this hypothesis by presenting three rehabilitated seal pups with playbacks of vocalizations from unfamiliar or familiar pups. It is uncommon for seals to come into rehabilitation for a second time in their lifespan, and this study took advantage of these rare cases. A simple visual inspection of the data plots seemed to show more reactions, and of longer duration, in response to familiar as compared to unfamiliar playbacks in two out of three pups. However, statistical analyses revealed no significant difference between the experimental conditions. We also found no significant asymmetry in orientation (left vs. right) towards familiar and unfamiliar sounds. While statistics do not support the hypothesis of an established ability to discriminate familiar vocalizations from unfamiliar ones in harbor seal pups, further investigations with a larger sample size are needed to confirm or refute this hypothesis.

Laryngeal and soft palate valving in the harbour seal (Phoca vitulina)

Effective 'valving' in the upper aerodigestive tract (UAT) is essential to temporarily separate the digestive and respiratory pathways. Marine mammals are largely dedicated to feeding underwater, and in many cases swallowing prey whole. In seals, little work has been done to explore the anatomy and function of the UAT in the context of valving mechanisms that function to separate food and air pathways. Here we use videofluoroscopy, gross dissection, histology and computed tomography (CT) renderings to explore the anatomy of the larynx and soft palate in the harbour seal (Phoca vitulina), and generate models for how valving mechanisms in the head and neck may function during breathing, phonating, diving and swallowing. Harbour seals have an elevated larynx and the epiglottis may rise above the level of the soft palate, particularly in pups when sucking. In addition, the corniculate and arytenoid cartilages with associated muscles form most of the lateral margins of the laryngeal inlet and vestibule, and move independently to facilitate airway closure. The corniculate cartilages flex over the laryngeal inlet beneath the epiglottis to completely close the laryngeal vestibule and inlet. The vocal folds are thick and muscular and the medial margin of the folds contains a small vocal ligament. The soft palate has well-defined levator veli palatini muscles that probably function to elevate the palate and close the pharyngeal isthmus during feeding. Our results support the conclusion that harbour seals have evolved UAT valving mechanisms as adaptations to a marine environment that are not seen in terrestrial carnivores.

Modelling Animal Interactive Rhythms in Communication

Time is one crucial dimension conveying information in animal communication. Evolution has shaped animals' nervous systems to produce signals with temporal properties fitting their socio-ecological niches. Many quantitative models of mechanisms underlying rhythmic behaviour exist, spanning insects, crustaceans, birds, amphibians, and mammals. However, these computational and mathematical models are often presented in isolation. Here, we provide an overview of the main mathematical models employed in the study of animal rhythmic communication among conspecifics. After presenting basic definitions and mathematical formalisms, we discuss each individual model. These computational models are then compared using simulated data to uncover similarities and key differences in the underlying mechanisms found across species. Our review of the empirical literature is admittedly limited. We stress the need of using comparative computer simulations - both before and after animal experiments - to better understand animal timing in interaction. We hope this article will serve as a potential first step towards a common computational framework to describe temporal interactions in animals, including humans.

Ontogeny of vocal rhythms in harbor seal pups: an exploratory study

Puppyhood is a very active social and vocal period in a harbor seal’s life Phoca vitulina. An important feature of vocalizations is their temporal and rhythmic structure, and understanding vocal timing and rhythms in harbor seals is critical to a cross-species hypothesis in evolutionary neuroscience that links vocal learning, rhythm perception, and synchronization. This study utilized analytical techniques that may best capture rhythmic structure in pup vocalizations with the goal of examining whether (1) harbor seal pups show rhythmic structure in their calls and (2) rhythms evolve over time. Calls of 3 wild-born seal pups were recorded daily over the course of 1–3 weeks; 3 temporal features were analyzed using 3 complementary techniques. We identified temporal and rhythmic structure in pup calls across different time windows. The calls of harbor seal pups exhibit some degree of temporal and rhythmic organization, which evolves over puppyhood and resembles that of other species’ interactive communication. We suggest next steps for investigating call structure in harbor seal pups and propose comparative hypotheses to test in other pinniped species.

Spontaneous rhythms in a harbor seal pup calls

Objectives

Timing and rhythm (i.e. temporal structure) are crucial, though historically neglected, dimensions of animal communication. When investigating these in non-human animals, it is often difficult to balance experimental control and ecological validity. Here I present the first step of an attempt to balance the two, focusing on the timing of vocal rhythms in a harbor seal pup (Phoca vitulina). Collection of this data had a clear aim: To find spontaneous vocal rhythms in this individual in order to design individually-adapted and ecologically-relevant stimuli for a later playback experiment.

Data description

The calls of one seal pup were recorded. The audio recordings were annotated using Praat, a free software to analyze vocalizations in humans and other animals. The annotated onsets and offsets of vocalizations were then imported in a Python script. The script extracted three types of timing information: the duration of calls, the intervals between calls’ onsets, and the intervals between calls’ maximum-intensity peaks. Based on the annotated data, available to download, I provide simple descriptive statistics for these temporal measures, and compare their distributions.

Comment on "Temporal and spatial variation in harbor seal (Phoca vitulina L.) roar calls from southern Scandinavia" [J. Acoust. Soc. Am. 141, 1824-1834 (2017)]

In their recent article, Sabinsky and colleagues investigated heterogeneity in harbor seals' vocalizations. The authors found seasonal and geographical variation in acoustic parameters, warning readers that recording conditions might account for some of their results. This paper expands on the temporal aspect of the encountered heterogeneity in harbor seals' vocalizations. Temporal information is the least susceptible to variable recording conditions. Hence geographical and seasonal variability in roar timing constitutes the most robust finding in the target article. In pinnipeds, evidence of timing and rhythm in the millisecond range-as opposed to circadian and seasonal rhythms-has theoretical and interdisciplinary relevance. In fact, the study of rhythm and timing in harbor seals is particularly decisive to support or confute a cross-species hypothesis, causally linking the evolution of vocal production learning and rhythm. The results by Sabinsky and colleagues can shed light on current scientific questions beyond pinniped bioacoustics, and help formulate empirically testable predictions.

How small could a pup sound? The physical bases of signaling body size in harbor seals

Vocal communication is a crucial aspect of animal behavior. The mechanism which most mammals use to vocalize relies on three anatomical components. First, air overpressure is generated inside the lower vocal tract. Second, as the airstream goes through the glottis, sound is produced via vocal fold vibration. Third, this sound is further filtered by the geometry and length of the upper vocal tract. Evidence from mammalian anatomy and bioacoustics suggests that some of these three components may covary with an animal's body size. The framework provided by acoustic allometry suggests that, because vocal tract length (VTL) is more strongly constrained by the growth of the body than vocal fold length (VFL), VTL generates more reliable acoustic cues to an animal's size. This hypothesis is often tested acoustically but rarely anatomically, especially in pinnipeds. Here, we test the anatomical bases of the acoustic allometry hypothesis in harbor seal pups Phoca vitulina. We dissected and measured vocal tract, vocal folds, and other anatomical features of 15 harbor seals post-mortem. We found that, while VTL correlates with body size, VFL does not. This suggests that, while body growth puts anatomical constraints on how vocalizations are filtered by harbor seals' vocal tract, no such constraints appear to exist on vocal folds, at least during puppyhood. It is particularly interesting to find anatomical constraints on harbor seals' vocal tracts, the same anatomical region partially enabling pups to produce individually distinctive vocalizations.

Quantitative classification of harbor seal breeding calls in Georgia Strait, Canada

During breeding season, male harbor seals (Phoca vitulina) produce underwater calls used in sexual competition and advertisement. Call characteristics vary among populations, and within-population differences are thought to represent individual variation. However, vocalizations have not been described for several populations of this widely-distributed and genetically diverse species. This study describes the vocal repertoire of harbor seals from British Columbia, Canada. Underwater recordings were made near Hornby Island during the summer of 2014 using a single hydrophone. A wide variability was detected in breeding vocalizations within this single breeding site. Four candidate call types were identified, containing six subtypes. Linear discriminant analysis showed 88% agreement with subjective classification of call types, and 74% agreement for call subtypes. Classification tree analysis gave a 92% agreement with candidate call types, with all splits made on the basis of call duration. Differences in duration may have reflected individual differences among seals. This study suggests that the vocal repertoire of harbor seals in this area comprises a vocal continuum rather than discrete call types. Further work with the ability to localize calls may help to determine whether this complexity represents variability due to propagation conditions, animal orientation, or differences among individual seals.

In-air vocal repertoires of spotted seals, Phoca largha

Spotted seals (Phoca largha) are thought to be less vocal than other phocids. However, acoustic communication behaviors of spotted seals have been reported several times. In this study, the vocal repertoires of spotted seals housed in Dalian Sun Aquarium, China were recorded and analyzed. The frequencies of the sounds made by the seals ranged from 139.3 to 2323.1 Hz, and the time durations lasted from 92.8 to 1208 ms, depending on age and gender (P < 0.01). The peak-to-peak sound source levels were 109-124 dB re 20μPa. In total, seven vocal types were identified: pup call, yearling call, bark, growl, grunt, moo, and throat guttural. The pups emitted sounds with high frequencies (F1: 972.4 ± 374.4 Hz, mean ± standard deviation) and medial time durations (564 ± 178 ms); when the pups grew older, the sounds became yearling calls, which had high frequencies with median (interquartile range) of 1198.0 (821.7-1385.5) Hz; and long time durations [902 (745-1080) ms]. The male adults emitted sounds with low frequencies [430.2 (388.2-486.7) Hz] and short time durations [334 (233-599) ms], while the female adults emitted sounds with medial frequencies [814.5 (592.6-1024.3) Hz] and medial time durations [531 (336-688) ms].

What Pinnipeds Have to Say about Human Speech, Music, and the Evolution of Rhythm

Research on the evolution of human speech and music benefits from hypotheses and data generated in a number of disciplines. The purpose of this article is to illustrate the high relevance of pinniped research for the study of speech, musical rhythm, and their origins, bridging and complementing current research on primates and birds. We briefly discuss speech, vocal learning, and rhythm from an evolutionary and comparative perspective. We review the current state of the art on pinniped communication and behavior relevant to the evolution of human speech and music, showing interesting parallels to hypotheses on rhythmic behavior in early hominids. We suggest future research directions in terms of species to test and empirical data needed.