Hierarchical temporal structure in music, speech and animal vocalizations: jazz is like a conversation, humpbacks sing like hermit thrushes

Isochrony as ancestral condition to call and song in a primate

Animal songs differ from calls in function and structure, and have comparative and translational value, showing similarities to human music. Rhythm in music is often distributed in quantized classes of intervals known as rhythmic categories. These classes have been found in the songs of a few nonhuman species but never in their calls. Are rhythmic categories song-specific, as in human music, or can they transcend the song-call boundary? We analyze the vocal displays of one of the few mammals producing both songs and call sequences: Indri indri. We test whether rhythmic categories (a) are conserved across songs produced in different contexts, (b) exist in call sequences, and (c) differ between songs and call sequences. We show that rhythmic categories occur across vocal displays. Vocalization type and function modulate deployment of categories. We find isochrony (1:1 ratio, like the rhythm of a ticking clock) in all song types, but only advertisement songs show three rhythmic categories (1:1, 1:2, 2:1 ratios). Like songs, some call types are also isochronous. Isochrony is the backbone of most indri vocalizations, unlike human speech, where it is rare. In indri, isochrony underlies both songs and hierarchy-less call sequences and might be ancestral to both.

Cetaceans are the next frontier for vocal rhythm research

While rhythm can facilitate and enhance many aspects of behavior, its evolutionary trajectory in vocal communication systems remains enigmatic. We can trace evolutionary processes by investigating rhythmic abilities in different species, but research to date has largely focused on songbirds and primates. We present evidence that cetaceans-whales, dolphins, and porpoises-are a missing piece of the puzzle for understanding why rhythm evolved in vocal communication systems. Cetaceans not only produce rhythmic vocalizations but also exhibit behaviors known or thought to play a role in the evolution of different features of rhythm. These behaviors include vocal learning abilities, advanced breathing control, sexually selected vocal displays, prolonged mother-infant bonds, and behavioral synchronization. The untapped comparative potential of cetaceans is further enhanced by high interspecific diversity, which generates natural ranges of vocal and social complexity for investigating various evolutionary hypotheses. We show that rhythm (particularly isochronous rhythm, when sounds are equally spaced in time) is prevalent in cetacean vocalizations but is used in different contexts by baleen and toothed whales. We also highlight key questions and research areas that will enhance understanding of vocal rhythms across taxa. By coupling an infraorder-level taxonomic assessment of vocal rhythm production with comparisons to other species, we illustrate how broadly comparative research can contribute to a more nuanced understanding of the prevalence, evolution, and possible functions of rhythm in animal communication.

Leadership dynamics in musical groups: Quantifying effects of musical structure on directionality of influence in concert performance videos

Music ensemble performance provides an ecologically valid context for investigating leadership dynamics in small group interactions. Musical texture, specifically the relative salience of simultaneously sounding ensemble parts, is a feature that can potentially alter leadership dynamics by introducing hierarchical relationships between individual parts. The present study extended previous work on quantifying interpersonal coupling in musical ensembles by examining the relationship between musical texture and leader-follower relations, operationalised as directionality of influence between co-performers' body motion in concert video recordings. It was hypothesised that the directionality of influence, indexed by Granger Causality, would be greater for 'homophonic' textures with a clear distinction between melody and accompaniment parts than for 'polyphonic' textures with less distinction between melody and accompaniment. This hypothesis was tested by using pose estimation algorithms to track instrumentalists' body movements in a string quartet and a clarinet quintet, and then applying Granger Causality analysis to their head motion to estimate directional influence between instrumentalist pairs for sections of the pieces that varied in texture. It was found that Granger Causality values were generally higher (indicating greater directionality of influence) for homophonic than polyphonic textures. Furthermore, considering melody and accompaniment instrument roles revealed more evidence for the melody instrument influencing accompanying instruments than vice versa, plus a high degree of directionality among accompanying instruments, in homophonic textures. These observed patterns of directional information flow in co-performer body motion are consistent with changing leader-follower relations depending on hierarchical relations between ensemble parts in terms of the relative salience of melodic material in the musical texture. The finding that automatic pose estimation can detect modulations of leadership dynamics in standard video recordings under naturalistic performance conditions has implications for investigating interpersonal coordination in large-scale music video datasets representing different cultural traditions, and for exploring nonverbal communication in group activities more generally.

Isochrony in barks of Cape fur seal (Arctocephalus pusillus pusillus) pups and adults

Animal vocal communication often relies on call sequences. The temporal patterns of such sequences can be adjusted to other callers, follow complex rhythmic structures or exhibit a metronome-like pattern (i.e., isochronous). How regular are the temporal patterns in animal signals, and what influences their precision? If present, are rhythms already there early in ontogeny? Here, we describe an exploratory study of Cape fur seal (Arctocephalus pusillus pusillus) barks-a vocalisation type produced across many pinniped species in rhythmic, percussive bouts. This study is the first quantitative description of barking in Cape fur seal pups. We analysed the rhythmic structures of spontaneous barking bouts of pups and adult females from the breeding colony in Cape Cross, Namibia. Barks of adult females exhibited isochrony, that is they were produced at fairly regular points in time. Instead, intervals between pup barks were more variable, that is skipping a bark in the isochronous series occasionally. In both age classes, beat precision, that is how well the barks followed a perfect template, was worse when barking at higher rates. Differences could be explained by physiological factors, such as respiration or arousal. Whether, and how, isochrony develops in this species remains an open question. This study provides evidence towards a rhythmic production of barks in Cape fur seal pups and lays the groundwork for future studies to investigate the development of rhythm using multidimensional metrics.

NONAN GaitPrint: An IMU gait database of healthy young adults

An ongoing thrust of research focused on human gait pertains to identifying individuals based on gait patterns. However, no existing gait database supports modeling efforts to assess gait patterns unique to individuals. Hence, we introduce the Nonlinear Analysis Core (NONAN) GaitPrint database containing whole body kinematics and foot placement during self-paced overground walking on a 200-meter looping indoor track. Noraxon Ultium MotionTM inertial measurement unit (IMU) sensors sampled the motion of 35 healthy young adults (19-35 years old; 18 men and 17 women; mean ± 1 s.d. age: 24.6 ± 2.7 years; height: 1.73 ± 0.78 m; body mass: 72.44 ± 15.04 kg) over 18 4-min trials across two days. Continuous variables include acceleration, velocity, position, and the acceleration, velocity, position, orientation, and rotational velocity of each corresponding body segment, and the angle of each respective joint. The discrete variables include an exhaustive set of gait parameters derived from the spatiotemporal dynamics of foot placement. We technically validate our data using continuous relative phase, Lyapunov exponent, and Hurst exponent-nonlinear metrics quantifying different aspects of healthy human gait.

Music as a window into real-world communication

Communication has been studied extensively in the context of speech and language. While speech is tremendously effective at transferring ideas between people, music is another communicative mode that has a unique power to bring people together and transmit a rich tapestry of emotions, through joint music-making and listening in a variety of everyday contexts. Research has begun to examine the behavioral and neural correlates of the joint action required for successful musical interactions, but it has yet to fully account for the rich, dynamic, multimodal nature of musical communication. We review the current literature in this area and propose that naturalistic musical paradigms will open up new ways to study communication more broadly.

Animal linguistics: a primer

The evolution of language has been investigated by several research communities, including biologists and linguists, striving to highlight similar linguistic capacities across species. To date, however, no consensus exists on the linguistic capacities of non-human species. Major controversies remain on the use of linguistic terminology, analysis methods and behavioural data collection. The field of 'animal linguistics' has emerged to overcome these difficulties and attempt to reach uniform methods and terminology. This primer is a tutorial review of 'animal linguistics'. It describes the linguistic concepts of semantics, pragmatics and syntax, and proposes minimal criteria to be fulfilled to claim that a given species displays a particular linguistic capacity. Second, it reviews relevant methods successfully applied to the study of communication in animals and proposes a list of useful references to detect and overcome major pitfalls commonly observed in the collection of animal behaviour data. This primer represents a step towards mutual understanding and fruitful collaborations between linguists and biologists.

1/f laws found in non-human music

A compelling question at the intersection of physics, neuroscience, and evolutionary biology concerns the extent to which the brains of various species evolved to encode regularities of the physical world. It would be parsimonious and adaptive, for example, for brains to evolve an innate understanding of gravity and the laws of motion, and to be able to detect, auditorily, those patterns of noises that ambulatory creatures make when moving about the world. One such physical regularity of the world is fractal structure, generally characterized by power-law correlations or 1/f ^β spectral distributions. Such laws are found broadly in nature and human artifacts, from noise in physical systems, to coastline topography (e.g., the Richardson effect), to neuronal spike patterns. These distributions have also been found to hold for the rhythm and power spectral density of a wide array of human music, suggesting that human music incorporates regularities of the physical world that our species evolved to recognize and produce. Here we show for the first time that 1/f^β laws also govern the spectral density of a wide range of animal vocalizations (music), from songbirds, to whales, to howling wolves. We discovered this 1/f^β power-law distribution in the vocalizations within all of the 17 diverse species examined. Our results demonstrate that such power laws are prevalent in the animal kingdom, evidence that their brains have evolved a sensitivity to them as an aid in processing sensory features of the natural world.

Child language and autism diagnosis impact hierarchical temporal structure of parent-child vocal interactions in early childhood

Timing is critical to successful social interactions. The temporal structure of dyadic vocal interactions emerges from the rhythm, timing, and frequency of each individuals' vocalizations and reflects how the dyad dynamically organizes and adapts during an interaction. This study investigated the temporal structure of vocal interactions longitudinally in parent-child dyads of typically developing (TD) infants (n = 49; 9-18 months; 48% male) and toddlers with ASD (n = 23; 27.2 ± 5.0 months; 91.3% male) to identify how developing language and social skills impact the temporal dynamics of the interaction. Acoustic hierarchical temporal structure (HTS), a measure of the nested clustering of acoustic events across multiple timescales, was measured in free play interactions using Allan Factor. HTS reflects a signal's temporal complexity and variability, with greater HTS indicating reduced flexibility of the dyadic system. Child expressive language significantly predicted HTS (ß = -0.2) longitudinally across TD infants, with greater dyadic HTS associated with lower child language skills. ASD dyads exhibited greater HTS (i.e., more rigid temporal structure) than nonverbal matched (d = 0.41) and expressive language matched TD dyads (d = 0.28). Increased HTS in ASD dyads occurred at timescales >1 s, suggesting greater structuring of pragmatic aspects of interaction. Results provide a new window into how language development and social reciprocity serve as constraints to shape parent-child interaction dynamics and showcase a novel automated approach to characterizing vocal interactions across multiple timescales during early childhood.

Hierarchical amplitude modulation structures and rhythm patterns: Comparing Western musical genres, song, and nature sounds to Babytalk

Statistical learning of physical stimulus characteristics is important for the development of cognitive systems like language and music. Rhythm patterns are a core component of both systems, and rhythm is key to language acquisition by infants. Accordingly, the physical stimulus characteristics that yield speech rhythm in "Babytalk" may also describe the hierarchical rhythmic relationships that characterize human music and song. Computational modelling of the amplitude envelope of "Babytalk" (infant-directed speech, IDS) using a demodulation approach (Spectral-Amplitude Modulation Phase Hierarchy model, S-AMPH) can describe these characteristics. S-AMPH modelling of Babytalk has shown previously that bands of amplitude modulations (AMs) at different temporal rates and their phase relations help to create its structured inherent rhythms. Additionally, S-AMPH modelling of children's nursery rhymes shows that different rhythm patterns (trochaic, iambic, dactylic) depend on the phase relations between AM bands centred on ~2 Hz and ~5 Hz. The importance of these AM phase relations was confirmed via a second demodulation approach (PAD, Probabilistic Amplitude Demodulation). Here we apply both S-AMPH and PAD to demodulate the amplitude envelopes of Western musical genres and songs. Quasi-rhythmic and non-human sounds found in nature (birdsong, rain, wind) were utilized for control analyses. We expected that the physical stimulus characteristics in human music and song from an AM perspective would match those of IDS. Given prior speech-based analyses, we also expected that AM cycles derived from the modelling may identify musical units like crotchets, quavers and demi-quavers. Both models revealed an hierarchically-nested AM modulation structure for music and song, but not nature sounds. This AM modulation structure for music and song matched IDS. Both models also generated systematic AM cycles yielding musical units like crotchets and quavers. Both music and language are created by humans and shaped by culture. Acoustic rhythm in IDS and music appears to depend on many of the same physical characteristics, facilitating learning.

Syntactic modulation of rhythm in Australian pied butcherbird song

The acoustic structure of birdsong is spectrally and temporally complex. Temporal complexity is often investigated in a syntactic framework focusing on the statistical features of symbolic song sequences. Alternatively, temporal patterns can be investigated in a rhythmic framework that focuses on the relative timing between song elements. Here, we investigate the merits of combining both frameworks by integrating syntactic and rhythmic analyses of Australian pied butcherbird (Cracticus nigrogularis) songs, which exhibit organized syntax and diverse rhythms. We show that rhythms of the pied butcherbird song bouts in our sample are categorically organized and predictable by the song's first-order sequential syntax. These song rhythms remain categorically distributed and strongly associated with the first-order sequential syntax even after controlling for variance in note length, suggesting that the silent intervals between notes induce a rhythmic structure on note sequences. We discuss the implication of syntactic-rhythmic relations as a relevant feature of song complexity with respect to signals such as human speech and music, and advocate for a broader conception of song complexity that takes into account syntax, rhythm, and their interaction with other acoustic and perceptual features.

Syntactic modulation of rhythm in Australian pied butcherbird song

The acoustic structure of birdsong is spectrally and temporally complex. Temporal complexity is often investigated in a syntactic framework focusing on the statistical features of symbolic song sequences. Alternatively, temporal patterns can be investigated in a rhythmic framework that focuses on the relative timing between song elements. Here, we investigate the merits of combining both frameworks by integrating syntactic and rhythmic analyses of Australian pied butcherbird (Cracticus nigrogularis) songs, which exhibit organized syntax and diverse rhythms. We show that rhythms of the pied butcherbird song bouts in our sample are categorically organized and predictable by the song's first-order sequential syntax. These song rhythms remain categorically distributed and strongly associated with the first-order sequential syntax even after controlling for variance in note length, suggesting that the silent intervals between notes induce a rhythmic structure on note sequences. We discuss the implication of syntactic-rhythmic relations as a relevant feature of song complexity with respect to signals such as human speech and music, and advocate for a broader conception of song complexity that takes into account syntax, rhythm, and their interaction with other acoustic and perceptual features.

Ergodic descriptors of non-ergodic stochastic processes

The stochastic processes underlying the growth and stability of biological and psychological systems reveal themselves when far-from-equilibrium. Far-from-equilibrium, non-ergodicity reigns. Non-ergodicity implies that the average outcome for a group/ensemble (i.e. of representative organisms/minds) is not necessarily a reliable estimate of the average outcome for an individual over time. However, the scientific interest in causal inference suggests that we somehow aim at stable estimates of the cause that will generalize to new individuals in the long run. Therefore, the valid analysis must extract an ergodic stationary measure from fluctuating physiological data. So the challenge is to extract statistical estimates that may describe or quantify some of this non-ergodicity (i.e. of the raw measured data) without themselves (i.e. the estimates) being non-ergodic. We show that traditional linear statistics such as the standard deviation, coefficient of variation and root mean square can break ergodicity. Time series of statistics addressing sequential structure and its potential nonlinearity: fractality and multi-fractality, change in a time-independent way and fulfil the ergodic assumption. Complementing traditional linear indices with fractal and multi-fractal indices would empower the study of stochastic far-from-equilibrium biological and psychological dynamics.

Collective Rhythm as an Emergent Property During Human Social Coordination

The literature on social interactions has shown that participants coordinate not only at the behavioral but also at the physiological and neural levels, and that this coordination gives a temporal structure to the individual and social dynamics. However, it has not been fully explored whether such temporal patterns emerge during interpersonal coordination beyond dyads, whether this phenomenon arises from complex cognitive mechanisms or from relatively simple rules of behavior, or which are the sociocultural processes that underlie this phenomenon. We review the evidence for the existence of group-level rhythmic patterns that result from social interactions and argue that the complexity of group dynamics can lead to temporal regularities that cannot be predicted from the individual periodicities: an emergent collective rhythm. Moreover, we use this interpretation of the literature to discuss how taking into account the sociocultural niche in which individuals develop can help explain the seemingly divergent results that have been reported on the social influences and consequences of interpersonal coordination. We make recommendations on further research to test these arguments and their relationship to the feeling of belonging and assimilation experienced during group dynamics.

Daylong Mobile Audio Recordings Reveal Multitimescale Dynamics in Infants' Vocal Productions and Auditory Experiences

The sounds of human infancy-baby babbling, adult talking, lullaby singing, and more-fluctuate over time. Infant-friendly wearable audio recorders can now capture very large quantities of these sounds throughout infants' everyday lives at home. Here, we review recent discoveries about how infants' soundscapes are organized over the course of a day based on analyses designed to detect patterns at multiple timescales. Analyses of infants' day-long audio have revealed that everyday vocalizations are clustered hierarchically in time, vocal explorations are consistent with foraging dynamics, and musical tunes are distributed such that some are much more available than others. This approach focusing on the multi-scale distributions of sounds heard and produced by infants provides new, fundamental insights on human communication development from a complex systems perspective.

Coordination dynamics of multi-agent interaction in a musical ensemble

Humans interact with other humans at a variety of timescales and in a variety of social contexts. We exhibit patterns of coordination that may differ depending on whether we are genuinely interacting as part of a coordinated group of individuals vs merely co-existing within the same physical space. Moreover, the local coordination dynamics of an interacting pair of individuals in an otherwise non-interacting group may spread, propagating change in the global coordination dynamics and interaction of an entire crowd. Dynamical systems analyses, such as Recurrence Quantification Analysis (RQA), can shed light on some of the underlying coordination dynamics of multi-agent human interaction. We used RQA to examine the coordination dynamics of a performance of "Welcome to the Imagination World", composed for wind orchestra. This performance enacts a real-life simulation of the transition from uncoordinated, non-interacting individuals to a coordinated, interacting multi-agent group. Unlike previous studies of social interaction in musical performance which rely on different aspects of video and/or acoustic data recorded from each individual, this project analyzes group-level coordination patterns solely from the group-level acoustic data of an audio recording of the performance. Recurrence and stability measures extracted from the audio recording increased when musicians coordinated as an interacting group. Variability in these measures also increased, indicating that the interacting ensemble of musicians were able to explore a greater variety of behavior than when they performed as non-interacting individuals. As an orchestrated (non-emergent) example of coordination, we believe these analyses provide an indication of approximate expected distributions for recurrence patterns that may be measurable before and after truly emergent coordination.

Encoding of acquired sound-sequence salience by auditory cortical offset responses

Behaviorally relevant sounds are often composed of distinct acoustic units organized into specific temporal sequences. The meaning of such sound sequences can therefore be fully recognized only when they have terminated. However, the neural mechanisms underlying the perception of sound sequences remain unclear. Here, we use two-photon calcium imaging in the auditory cortex of behaving mice to test the hypothesis that neural responses to termination of sound sequences ("Off-responses") encode their acoustic history and behavioral salience. We find that auditory cortical Off-responses encode preceding sound sequences and that learning to associate a sound sequence with a reward induces enhancement of Off-responses relative to responses during the sound sequence ("On-responses"). Furthermore, learning enhances network-level discriminability of sound sequences by Off-responses. Last, learning-induced plasticity of Off-responses but not On-responses lasts to the next day. These findings identify auditory cortical Off-responses as a key neural signature of acquired sound-sequence salience.

Multilevel rhythms in multimodal communication

It is now widely accepted that the brunt of animal communication is conducted via several modalities, e.g. acoustic and visual, either simultaneously or sequentially. This is a laudable multimodal turn relative to traditional accounts of temporal aspects of animal communication which have focused on a single modality at a time. However, the fields that are currently contributing to the study of multimodal communication are highly varied, and still largely disconnected given their sole focus on a particular level of description or their particular concern with human or non-human animals. Here, we provide an integrative overview of converging findings that show how multimodal processes occurring at neural, bodily, as well as social interactional levels each contribute uniquely to the complex rhythms that characterize communication in human and non-human animals. Though we address findings for each of these levels independently, we conclude that the most important challenge in this field is to identify how processes at these different levels connect. This article is part of the theme issue 'Synchrony and rhythm interaction: from the brain to behavioural ecology'.

Cross-frequency coupling explains the preference for simple ratios in rhythmic behaviour and the relative stability across non-synchronous patterns

Rhythms are important for understanding coordinated behaviours in ecological systems. The repetitive nature of rhythms affords prediction, planning of movements and coordination of processes within and between individuals. A major challenge is to understand complex forms of coordination when they differ from complete synchronization. By expressing phase as ratio of a cycle, we adapted levels of the Farey tree as a metric of complexity mapped to the range between in-phase and anti-phase synchronization. In a bimanual tapping task, this revealed an increase of variability with ratio complexity, a range of hidden and unstable yet measurable modes, and a rank-frequency scaling law across these modes. We use the phase-attractive circle map to propose an interpretation of these findings in terms of hierarchical cross-frequency coupling (CFC). We also consider the tendency for small-integer attractors in the single-hand repeated tapping of three-interval rhythms reported in the literature. The phase-attractive circle map has wider basins of attractions for such ratios. This work motivates the question whether CFC intrinsic to neural dynamics implements low-level priors for timing and coordination and thus becomes involved in phenomena as diverse as attractor states in bimanual coordination and the cross-cultural tendency for musical rhythms to have simple interval ratios. This article is part of the theme issue 'Synchrony and rhythm interaction: from the brain to behavioural ecology'.

Perceiving and remembering speech depend on multifractal nonlinearity in movements producing and exploring speech

Speech perception and memory for speech require active engagement. Gestural theories have emphasized mainly the effect of speaker's movements on speech perception. They fail to address the effects of listener movement, focusing on communication as a boundary condition constraining movement among interlocutors. The present work attempts to break new ground by using multifractal geometry of physical movement as a common currency for supporting both sides of the speaker-listener dyads. Participants self-paced their listening to a narrative, after which they completed a test of memory querying their narrative comprehension and their ability to recognize words from the story. The multifractal evidence of nonlinear interactions across timescales predicted the fluency of speech perception. Self-pacing movements that enabled listeners to control the presentation of speech sounds constituted a rich exploratory process. The multifractal nonlinearity of this exploration supported several aspects of memory for the perceived spoken language. These findings extend the role of multifractal geometry in the speaker's movements to the narrative case of speech perception. In addition to posing novel basic research questions, these findings make a compelling case for calibrating multifractal structure in text-to-speech synthesizers for better perception and memory of speech.

Evolutionary origins of music. Classical and recent hypotheses

The aim of this paper is to review recent hypotheses on the evolutionary origins of music in Homo sapiens, taking into account the most influential traditional hypotheses. To date, theories derived from evolution have focused primarily on the importance that music carries in solving detailed adaptive problems. The three most influential theoretical concepts have described the evolution of human music in terms of 1) sexual selection, 2) the formation of social bonds, or treated it 3) as a byproduct. According to recent proposals, traditional hypotheses are flawed or insufficient in fully explaining the complexity of music in Homo sapiens. This paper will critically discuss three traditional hypotheses of music evolution (music as an effect of sexual selection, a mechanism of social bonding, and a byproduct), as well as and two recent concepts of music evolution - music as a credible signal and Music and Social Bonding (MSB) hypothesis.

Analysis and Design of Social Presence in a Computer-Mediated Communication System

Social presence, or the subjective experience of being present with another existing person, varies with the interaction medium. In general, social presence research has mainly focused on uni-directional aspects of each exchanged message, not on bidirectional interactions. Our primary purpose is to introduce such bidirectional evaluation by quantifying the degree of social presence with a few statistical measures. To this end, we developed a software called “TypeTrace” that records all keystrokes of online chat interactants and reenacts their typing actions and analyzed the results from different chat conditions, mainly focusing on the characterization of bi-directional interactions. We also compared the chat interaction patterns with the patterns from phone call datasets to investigate the difference of live communication in different media. The hypothesis of the experiment was that either richness or concurrency of communication is important for organizing social presence. Richness is defined by the variety of information at a time in communication and the concurrency is the number of temporal thread being processed at the same time. Our results show that when we merely increase the richness of information by presenting the typing process, the cognition of others' presence does not significantly increase. However, when the information concurrency is augmented by introducing the transmission of realtime text, we found that the transfer entropy between the interactants becomes considerably higher, and the social presence and emotional arousal, intimacy increased. High transfer entropy was also observed in the phone call dataset. This result shows that the mere augmentation of information richness does not necessarily lead to increased social presence, and concurrent communication is another critical factor for fostering vivid conversation in digital environments.

Fractality and Variability in Canonical and Non-Canonical English Fiction and in Non-Fictional Texts

This study investigates global properties of three categories of English text: canonical fiction, non-canonical fiction, and non-fictional texts. The central hypothesis of the study is that there are systematic differences with respect to structural design features between canonical and non-canonical fiction, and between fictional and non-fictional texts. To investigate these differences, we compiled a corpus containing texts of the three categories of interest, the Jena Corpus of Expository and Fictional Prose (JEFP Corpus). Two aspects of global structure are investigated, variability and self-similar (fractal) patterns, which reflect long-range correlations along texts. We use four types of basic observations, (i) the frequency of POS-tags per sentence, (ii) sentence length, (iii) lexical diversity, and (iv) the distribution of topic probabilities in segments of texts. These basic observations are grouped into two more general categories, (a) the lower-level properties (i) and (ii), which are observed at the level of the sentence (reflecting linguistic decoding), and (b) the higher-level properties (iii) and (iv), which are observed at the textual level (reflecting comprehension/integration). The observations for each property are transformed into series, which are analyzed in terms of variance and subjected to Multi-Fractal Detrended Fluctuation Analysis (MFDFA), giving rise to three statistics: (i) the degree of fractality ( H ), (ii) the degree of multifractality ( D ), i.e., the width of the fractal spectrum, and (iii) the degree of asymmetry ( A ) of the fractal spectrum. The statistics thus obtained are compared individually across text categories and jointly fed into a classification model (Support Vector Machine). Our results show that there are in fact differences between the three text categories of interest. In general, lower-level text properties are better discriminators than higher-level text properties. Canonical fictional texts differ from non-canonical ones primarily in terms of variability in lower-level text properties. Fractality seems to be a universal feature of text, slightly more pronounced in non-fictional than in fictional texts. On the basis of our results obtained on the basis of corpus data we point out some avenues for future research leading toward a more comprehensive analysis of textual aesthetics, e.g., using experimental methodologies.

Rhythm in speech and animal vocalizations: a cross-species perspective

Why does human speech have rhythm? As we cannot travel back in time to witness how speech developed its rhythmic properties and why humans have the cognitive skills to process them, we rely on alternative methods to find out. One powerful tool is the comparative approach: studying the presence or absence of cognitive/behavioral traits in other species to determine which traits are shared between species and which are recent human inventions. Vocalizations of many species exhibit temporal structure, but little is known about how these rhythmic structures evolved, are perceived and produced, their biological and developmental bases, and communicative functions. We review the literature on rhythm in speech and animal vocalizations as a first step toward understanding similarities and differences across species. We extend this review to quantitative techniques that are useful for computing rhythmic structure in acoustic sequences and hence facilitate cross-species research. We report links between vocal perception and motor coordination and the differentiation of rhythm based on hierarchical temporal structure. While still far from a complete cross-species perspective of speech rhythm, our review puts some pieces of the puzzle together.

An in vitro experimental study on the relationship between pulsatile tinnitus and the dehiscence/thinness of sigmoid sinus cortical plate

Pulsatile tinnitus (PT), characterized as pulse-synchronous, is generally objective. Sigmoid sinus (SS) venous sound is widely suggested to be a possible sound source of PT. The dehiscence and thinness of SS cortical plate (CP) was commonly reported as PT pathology in previous studies, but lack quantitative or biomechanical analysis. In this study, it was aimed to quantify the relationship between venous sound and CP dehiscence/thinness using in vitro experiment. The in vitro models of SS and CP were established based on 3D-printing, with the developed pulsatile venous flow in the SS model. The generated sound signal and the vibration response at the dehiscent/thinned area were analyzed. The sound signal generated in the normal-sized dehiscence model was pulse-synchronous within 100--400 Hz, which had similar acoustic characteristics as the clinical PT sounds. It was concluded that the pulsatile venous sound is produced at TS-SS junction in case of CP dehiscence. The CP, even a thinned one can effectively diminish the venous sound and sound-generating pulsatile vibration at TS-SS junction. The CP dehiscence would induce pulse-synchronous and high pressure venous sound, as well as pulse-synchronous vibration above 20 Hz, regardless of the dehiscence size. On the contrary, the CP thinness would not induce obvious venous sound or pulsatile vibration above 20 Hz.

Bringing the Nonlinearity of the Movement System to Gestural Theories of Language Use: Multifractal Structure of Spoken English Supports the Compensation for Coarticulation in Human Speech Perception

Coarticulation is the tendency for speech vocalization and articulation even at the phonemic level to change with context, and compensation for coarticulation (CfC) reflects the striking human ability to perceive phonemic stability despite this variability. A current controversy centers on whether CfC depends on contrast between formants of a speech-signal spectrogram-specifically, contrast between offset formants concluding context stimuli and onset formants opening the target sound-or on speech-sound variability specific to the coordinative movement of speech articulators (e.g., vocal folds, postural muscles, lips, tongues). This manuscript aims to encode that coordinative-movement context in terms of speech-signal multifractal structure and to determine whether speech's multifractal structure might explain the crucial gestural support for any proposed spectral contrast. We asked human participants to categorize individual target stimuli drawn from an 11-step [ga]-to-[da] continuum as either phonemes "GA" or "DA." Three groups each heard a specific-type context stimulus preceding target stimuli: either real-speech [al] or [a], sine-wave tones at the third-formant offset frequency of either [al] or [aɹ], and either simulated-speech contexts [al] or [aɹ]. Here, simulating speech contexts involved randomizing the sequence of relatively homogeneous pitch periods within vowel-sound [a] of each [al] and [aɹ]. Crucially, simulated-speech contexts had the same offset and extremely similar vowel formants as and, to additional naïve participants, sounded identical to real-speech contexts. However, randomization distorted original speech-context multifractality, and effects of spectral contrast following speech only appeared after regression modeling of trial-by-trial "GA" judgments controlled for context-stimulus multifractality. Furthermore, simulated-speech contexts elicited faster responses (like tone contexts do) and weakened known biases in CfC, suggesting that spectral contrast depends on the nonlinear interactions across multiple scales that articulatory gestures express through the speech signal. Traditional mouse-tracking behaviors measured as participants moved their computer-mouse cursor to register their "GA"-or-"DA" decisions with mouse-clicks suggest that listening to speech leads the movement system to resonate with the multifractality of context stimuli. We interpret these results as shedding light on a new multifractal terrain upon which to found a better understanding in which movement systems play an important role in shaping how speech perception makes use of acoustic information.

Ontogeny of vocal rhythms in harbor seal pups: an exploratory study

Puppyhood is a very active social and vocal period in a harbor seal’s life Phoca vitulina. An important feature of vocalizations is their temporal and rhythmic structure, and understanding vocal timing and rhythms in harbor seals is critical to a cross-species hypothesis in evolutionary neuroscience that links vocal learning, rhythm perception, and synchronization. This study utilized analytical techniques that may best capture rhythmic structure in pup vocalizations with the goal of examining whether (1) harbor seal pups show rhythmic structure in their calls and (2) rhythms evolve over time. Calls of 3 wild-born seal pups were recorded daily over the course of 1–3 weeks; 3 temporal features were analyzed using 3 complementary techniques. We identified temporal and rhythmic structure in pup calls across different time windows. The calls of harbor seal pups exhibit some degree of temporal and rhythmic organization, which evolves over puppyhood and resembles that of other species’ interactive communication. We suggest next steps for investigating call structure in harbor seal pups and propose comparative hypotheses to test in other pinniped species.

The Bursts and Lulls of Multimodal Interaction: Temporal Distributions of Behavior Reveal Differences Between Verbal and Non-Verbal Communication

Recent studies of naturalistic face-to-face communication have demonstrated coordination patterns such as the temporal matching of verbal and non-verbal behavior, which provides evidence for the proposal that verbal and non-verbal communicative control derives from one system. In this study, we argue that the observed relationship between verbal and non-verbal behaviors depends on the level of analysis. In a reanalysis of a corpus of naturalistic multimodal communication (Louwerse, Dale, Bard, & Jeuniaux, ), we focus on measuring the temporal patterns of specific communicative behaviors in terms of their burstiness. We examined burstiness estimates across different roles of the speaker and different communicative modalities. We observed more burstiness for verbal versus non-verbal channels, and for more versus less informative language subchannels. Using this new method for analyzing temporal patterns in communicative behaviors, we show that there is a complex relationship between verbal and non-verbal channels. We propose a "temporal heterogeneity" hypothesis to explain how the language system adapts to the demands of dialog.

Spontaneous rhythms in a harbor seal pup calls

Objectives

Timing and rhythm (i.e. temporal structure) are crucial, though historically neglected, dimensions of animal communication. When investigating these in non-human animals, it is often difficult to balance experimental control and ecological validity. Here I present the first step of an attempt to balance the two, focusing on the timing of vocal rhythms in a harbor seal pup (Phoca vitulina). Collection of this data had a clear aim: To find spontaneous vocal rhythms in this individual in order to design individually-adapted and ecologically-relevant stimuli for a later playback experiment.

Data description

The calls of one seal pup were recorded. The audio recordings were annotated using Praat, a free software to analyze vocalizations in humans and other animals. The annotated onsets and offsets of vocalizations were then imported in a Python script. The script extracted three types of timing information: the duration of calls, the intervals between calls’ onsets, and the intervals between calls’ maximum-intensity peaks. Based on the annotated data, available to download, I provide simple descriptive statistics for these temporal measures, and compare their distributions.