1
|
Rathcke T, Smit E, Zheng Y, Canzi M. Perception of temporal structure in speech is influenced by body movement and individual beat perception ability. Atten Percept Psychophys 2024:10.3758/s13414-024-02893-8. [PMID: 38769276 DOI: 10.3758/s13414-024-02893-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/03/2024] [Indexed: 05/22/2024]
Abstract
The subjective experience of time flow in speech deviates from the sound acoustics in substantial ways. The present study focuses on the perceptual tendency to regularize time intervals found in speech but not in other types of sounds with a similar temporal structure. We investigate to what extent individual beat perception ability is responsible for perceptual regularization and if the effect can be eliminated through the involvement of body movement during listening. Participants performed a musical beat perception task and compared spoken sentences to their drumbeat-based versions either after passive listening or after listening and moving along with the beat of the sentences. The results show that the interval regularization prevails in listeners with a low beat perception ability performing a passive listening task and is eliminated in an active listening task involving body movement. Body movement also helped to promote a veridical percept of temporal structure in speech at the group level. We suggest that body movement engages an internal timekeeping mechanism, promoting the fidelity of auditory encoding even in sounds of high temporal complexity and irregularity such as natural speech.
Collapse
Affiliation(s)
- Tamara Rathcke
- Department of Linguistics, University of Konstanz, Konstanz, 78464, Baden-Württemberg, Germany.
| | - Eline Smit
- Department of Linguistics, University of Konstanz, Konstanz, 78464, Baden-Württemberg, Germany
- The MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Street, Penrith, 2751, NSW, Australia
| | - Yue Zheng
- Department of Psychology, University of York, York, YO10 5DD, UK
- Department of Hearing Sciences, University of Nottingham, Nottingham, NG7 2RD, UK
| | - Massimiliano Canzi
- Department of Linguistics, University of Konstanz, Konstanz, 78464, Baden-Württemberg, Germany
| |
Collapse
|
2
|
van der Werff J, Ravignani A, Jadoul Y. thebeat: A Python package for working with rhythms and other temporal sequences. Behav Res Methods 2024; 56:3725-3736. [PMID: 38308146 PMCID: PMC11133136 DOI: 10.3758/s13428-023-02334-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/27/2023] [Indexed: 02/04/2024]
Abstract
thebeat is a Python package for working with temporal sequences and rhythms in the behavioral and cognitive sciences, as well as in bioacoustics. It provides functionality for creating experimental stimuli, and for visualizing and analyzing temporal data. Sequences, sounds, and experimental trials can be generated using single lines of code. thebeat contains functions for calculating common rhythmic measures, such as interval ratios, and for producing plots, such as circular histograms. thebeat saves researchers time when creating experiments, and provides the first steps in collecting widely accepted methods for use in timing research. thebeat is an open-source, on-going, and collaborative project, and can be extended for use in specialized subfields. thebeat integrates easily with the existing Python ecosystem, allowing one to combine our tested code with custom-made scripts. The package was specifically designed to be useful for both skilled and novice programmers. thebeat provides a foundation for working with temporal sequences onto which additional functionality can be built. This combination of specificity and plasticity should facilitate research in multiple research contexts and fields of study.
Collapse
Affiliation(s)
- J van der Werff
- Comparative Bioacoustics Group, Max Planck Institute for Psycholinguistics, Wundtlaan 1, Nijmegen, The Netherlands.
- Department of Human Neurosciences, Sapienza University of Rome, Piazzale Aldo Moro, 5, Rome, Italy.
| | - Andrea Ravignani
- Comparative Bioacoustics Group, Max Planck Institute for Psycholinguistics, Wundtlaan 1, Nijmegen, The Netherlands
- Department of Human Neurosciences, Sapienza University of Rome, Piazzale Aldo Moro, 5, Rome, Italy
- Center for Music in the Brain, Aarhus University, Universitetsbyen 3, Aarhus, Denmark
| | - Yannick Jadoul
- Comparative Bioacoustics Group, Max Planck Institute for Psycholinguistics, Wundtlaan 1, Nijmegen, The Netherlands
- Department of Human Neurosciences, Sapienza University of Rome, Piazzale Aldo Moro, 5, Rome, Italy
| |
Collapse
|
3
|
Zoefel B, Kösem A. Neural tracking of continuous acoustics: properties, speech-specificity and open questions. Eur J Neurosci 2024; 59:394-414. [PMID: 38151889 DOI: 10.1111/ejn.16221] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Revised: 11/17/2023] [Accepted: 11/22/2023] [Indexed: 12/29/2023]
Abstract
Human speech is a particularly relevant acoustic stimulus for our species, due to its role of information transmission during communication. Speech is inherently a dynamic signal, and a recent line of research focused on neural activity following the temporal structure of speech. We review findings that characterise neural dynamics in the processing of continuous acoustics and that allow us to compare these dynamics with temporal aspects in human speech. We highlight properties and constraints that both neural and speech dynamics have, suggesting that auditory neural systems are optimised to process human speech. We then discuss the speech-specificity of neural dynamics and their potential mechanistic origins and summarise open questions in the field.
Collapse
Affiliation(s)
- Benedikt Zoefel
- Centre de Recherche Cerveau et Cognition (CerCo), CNRS UMR 5549, Toulouse, France
- Université de Toulouse III Paul Sabatier, Toulouse, France
| | - Anne Kösem
- Lyon Neuroscience Research Center (CRNL), INSERM U1028, Bron, France
| |
Collapse
|
4
|
Bonnet P, Bonnefond M, Kösem A. What is a Rhythm for the Brain? The Impact of Contextual Temporal Variability on Auditory Perception. J Cogn 2024; 7:15. [PMID: 38250558 PMCID: PMC10798173 DOI: 10.5334/joc.344] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2023] [Accepted: 01/02/2024] [Indexed: 01/23/2024] Open
Abstract
Temporal predictions can be formed and impact perception when sensory timing is fully predictable: for instance, the discrimination of a target sound is enhanced if it is presented on the beat of an isochronous rhythm. However, natural sensory stimuli, like speech or music, are not entirely predictable, but still possess statistical temporal regularities. We investigated whether temporal expectations can be formed in non-fully predictable contexts, and how the temporal variability of sensory contexts affects auditory perception. Specifically, we asked how "rhythmic" an auditory stimulation needs to be in order to observe temporal predictions effects on auditory discrimination performances. In this behavioral auditory oddball experiment, participants listened to auditory sound sequences where the temporal interval between each sound was drawn from gaussian distributions with distinct standard deviations. Participants were asked to discriminate sounds with a deviant pitch in the sequences. Auditory discrimination performances, as measured with deviant sound discrimination accuracy and response times, progressively declined as the temporal variability of the sound sequence increased. Moreover, both global and local temporal statistics impacted auditory perception, suggesting that temporal statistics are promptly integrated to optimize perception. Altogether, these results suggests that temporal predictions can be set up quickly based on the temporal statistics of past sensory events and are robust to a certain amount of temporal variability. Therefore, temporal predictions can be built on sensory stimulations that are not purely periodic nor temporally deterministic.
Collapse
Affiliation(s)
- Pierre Bonnet
- Lyon Neuroscience Research Center (CRNL), Computation, Cognition and Neurophysiology team (Cophy), Inserm U1028, Université Claude Bernard Lyon1, CNRS UMR 5292, 69000 Lyon, France
| | - Mathilde Bonnefond
- Lyon Neuroscience Research Center (CRNL), Computation, Cognition and Neurophysiology team (Cophy), Inserm U1028, Université Claude Bernard Lyon1, CNRS UMR 5292, 69000 Lyon, France
| | - Anne Kösem
- Lyon Neuroscience Research Center (CRNL), Computation, Cognition and Neurophysiology team (Cophy), Inserm U1028, Université Claude Bernard Lyon1, CNRS UMR 5292, 69000 Lyon, France
| |
Collapse
|
5
|
Yu CY, Cabildo A, Grahn JA, Vanden Bosch der Nederlanden CM. Perceived rhythmic regularity is greater for song than speech: examining acoustic correlates of rhythmic regularity in speech and song. Front Psychol 2023; 14:1167003. [PMID: 37303916 PMCID: PMC10250601 DOI: 10.3389/fpsyg.2023.1167003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Accepted: 05/09/2023] [Indexed: 06/13/2023] Open
Abstract
Rhythm is a key feature of music and language, but the way rhythm unfolds within each domain differs. Music induces perception of a beat, a regular repeating pulse spaced by roughly equal durations, whereas speech does not have the same isochronous framework. Although rhythmic regularity is a defining feature of music and language, it is difficult to derive acoustic indices of the differences in rhythmic regularity between domains. The current study examined whether participants could provide subjective ratings of rhythmic regularity for acoustically matched (syllable-, tempo-, and contour-matched) and acoustically unmatched (varying in tempo, syllable number, semantics, and contour) exemplars of speech and song. We used subjective ratings to index the presence or absence of an underlying beat and correlated ratings with stimulus features to identify acoustic metrics of regularity. Experiment 1 highlighted that ratings based on the term "rhythmic regularity" did not result in consistent definitions of regularity across participants, with opposite ratings for participants who adopted a beat-based definition (song greater than speech), a normal-prosody definition (speech greater than song), or an unclear definition (no difference). Experiment 2 defined rhythmic regularity as how easy it would be to tap or clap to the utterances. Participants rated song as easier to clap or tap to than speech for both acoustically matched and unmatched datasets. Subjective regularity ratings from Experiment 2 illustrated that stimuli with longer syllable durations and with less spectral flux were rated as more rhythmically regular across domains. Our findings demonstrate that rhythmic regularity distinguishes speech from song and several key acoustic features can be used to predict listeners' perception of rhythmic regularity within and across domains as well.
Collapse
Affiliation(s)
- Chu Yi Yu
- The Brain and Mind Institute, Western University, London, ON, Canada
- Department of Psychology, Western University, London, ON, Canada
| | - Anne Cabildo
- Department of Psychology, University of Toronto, Mississauga, ON, Canada
| | - Jessica A. Grahn
- The Brain and Mind Institute, Western University, London, ON, Canada
- Department of Psychology, Western University, London, ON, Canada
| | - Christina M. Vanden Bosch der Nederlanden
- The Brain and Mind Institute, Western University, London, ON, Canada
- Department of Psychology, Western University, London, ON, Canada
- Department of Psychology, University of Toronto, Mississauga, ON, Canada
| |
Collapse
|
6
|
Rimmele JM, Sun Y, Michalareas G, Ghitza O, Poeppel D. Dynamics of Functional Networks for Syllable and Word-Level Processing. NEUROBIOLOGY OF LANGUAGE (CAMBRIDGE, MASS.) 2023; 4:120-144. [PMID: 37229144 PMCID: PMC10205074 DOI: 10.1162/nol_a_00089] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/18/2021] [Accepted: 11/07/2022] [Indexed: 05/27/2023]
Abstract
Speech comprehension requires the ability to temporally segment the acoustic input for higher-level linguistic analysis. Oscillation-based approaches suggest that low-frequency auditory cortex oscillations track syllable-sized acoustic information and therefore emphasize the relevance of syllabic-level acoustic processing for speech segmentation. How syllabic processing interacts with higher levels of speech processing, beyond segmentation, including the anatomical and neurophysiological characteristics of the networks involved, is debated. In two MEG experiments, we investigate lexical and sublexical word-level processing and the interactions with (acoustic) syllable processing using a frequency-tagging paradigm. Participants listened to disyllabic words presented at a rate of 4 syllables/s. Lexical content (native language), sublexical syllable-to-syllable transitions (foreign language), or mere syllabic information (pseudo-words) were presented. Two conjectures were evaluated: (i) syllable-to-syllable transitions contribute to word-level processing; and (ii) processing of words activates brain areas that interact with acoustic syllable processing. We show that syllable-to-syllable transition information compared to mere syllable information, activated a bilateral superior, middle temporal and inferior frontal network. Lexical content resulted, additionally, in increased neural activity. Evidence for an interaction of word- and acoustic syllable-level processing was inconclusive. Decreases in syllable tracking (cerebroacoustic coherence) in auditory cortex and increases in cross-frequency coupling between right superior and middle temporal and frontal areas were found when lexical content was present compared to all other conditions; however, not when conditions were compared separately. The data provide experimental insight into how subtle and sensitive syllable-to-syllable transition information for word-level processing is.
Collapse
Affiliation(s)
- Johanna M. Rimmele
- Departments of Neuroscience and Cognitive Neuropsychology, Max-Planck-Institute for Empirical Aesthetics, Frankfurt am Main, Germany
- Max Planck NYU Center for Language, Music and Emotion, Frankfurt am Main, Germany; New York, NY, USA
| | - Yue Sun
- Departments of Neuroscience and Cognitive Neuropsychology, Max-Planck-Institute for Empirical Aesthetics, Frankfurt am Main, Germany
| | - Georgios Michalareas
- Departments of Neuroscience and Cognitive Neuropsychology, Max-Planck-Institute for Empirical Aesthetics, Frankfurt am Main, Germany
| | - Oded Ghitza
- Departments of Neuroscience and Cognitive Neuropsychology, Max-Planck-Institute for Empirical Aesthetics, Frankfurt am Main, Germany
- College of Biomedical Engineering & Hearing Research Center, Boston University, Boston, MA, USA
| | - David Poeppel
- Departments of Neuroscience and Cognitive Neuropsychology, Max-Planck-Institute for Empirical Aesthetics, Frankfurt am Main, Germany
- Department of Psychology and Center for Neural Science, New York University, New York, NY, USA
- Max Planck NYU Center for Language, Music and Emotion, Frankfurt am Main, Germany; New York, NY, USA
- Ernst Strüngmann Institute for Neuroscience, Frankfurt am Main, Germany
| |
Collapse
|
7
|
Giroud J, Lerousseau JP, Pellegrino F, Morillon B. The channel capacity of multilevel linguistic features constrains speech comprehension. Cognition 2023; 232:105345. [PMID: 36462227 DOI: 10.1016/j.cognition.2022.105345] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2022] [Revised: 09/28/2022] [Accepted: 11/22/2022] [Indexed: 12/05/2022]
Abstract
Humans are expert at processing speech but how this feat is accomplished remains a major question in cognitive neuroscience. Capitalizing on the concept of channel capacity, we developed a unified measurement framework to investigate the respective influence of seven acoustic and linguistic features on speech comprehension, encompassing acoustic, sub-lexical, lexical and supra-lexical levels of description. We show that comprehension is independently impacted by all these features, but at varying degrees and with a clear dominance of the syllabic rate. Comparing comprehension of French words and sentences further reveals that when supra-lexical contextual information is present, the impact of all other features is dramatically reduced. Finally, we estimated the channel capacity associated with each linguistic feature and compared them with their generic distribution in natural speech. Our data reveal that while acoustic modulation, syllabic and phonemic rates unfold respectively at 5, 5, and 12 Hz in natural speech, they are associated with independent processing bottlenecks whose channel capacity are of 15, 15 and 35 Hz, respectively, as suggested by neurophysiological theories. They moreover point towards supra-lexical contextual information as the feature limiting the flow of natural speech. Overall, this study reveals how multilevel linguistic features constrain speech comprehension.
Collapse
Affiliation(s)
- Jérémy Giroud
- Aix Marseille Univ, Inserm, INS, Inst Neurosci Syst, Marseille, France.
| | | | - François Pellegrino
- Laboratoire Dynamique du Langage UMR 5596, CNRS, University of Lyon, 14 Avenue Berthelot, 69007 Lyon, France
| | - Benjamin Morillon
- Aix Marseille Univ, Inserm, INS, Inst Neurosci Syst, Marseille, France
| |
Collapse
|
8
|
Gnanateja GN, Devaraju DS, Heyne M, Quique YM, Sitek KR, Tardif MC, Tessmer R, Dial HR. On the Role of Neural Oscillations Across Timescales in Speech and Music Processing. Front Comput Neurosci 2022; 16:872093. [PMID: 35814348 PMCID: PMC9260496 DOI: 10.3389/fncom.2022.872093] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2022] [Accepted: 05/24/2022] [Indexed: 11/25/2022] Open
Abstract
This mini review is aimed at a clinician-scientist seeking to understand the role of oscillations in neural processing and their functional relevance in speech and music perception. We present an overview of neural oscillations, methods used to study them, and their functional relevance with respect to music processing, aging, hearing loss, and disorders affecting speech and language. We first review the oscillatory frequency bands and their associations with speech and music processing. Next we describe commonly used metrics for quantifying neural oscillations, briefly touching upon the still-debated mechanisms underpinning oscillatory alignment. Following this, we highlight key findings from research on neural oscillations in speech and music perception, as well as contributions of this work to our understanding of disordered perception in clinical populations. Finally, we conclude with a look toward the future of oscillatory research in speech and music perception, including promising methods and potential avenues for future work. We note that the intention of this mini review is not to systematically review all literature on cortical tracking of speech and music. Rather, we seek to provide the clinician-scientist with foundational information that can be used to evaluate and design research studies targeting the functional role of oscillations in speech and music processing in typical and clinical populations.
Collapse
Affiliation(s)
- G Nike Gnanateja
- Department of Communication Science and Disorders, University of Pittsburgh, Pittsburgh, PA, United States
| | - Dhatri S Devaraju
- Department of Communication Science and Disorders, University of Pittsburgh, Pittsburgh, PA, United States
| | - Matthias Heyne
- Department of Communication Science and Disorders, University of Pittsburgh, Pittsburgh, PA, United States
| | - Yina M Quique
- Center for Education in Health Sciences, Northwestern University, Chicago, IL, United States
| | - Kevin R Sitek
- Department of Communication Science and Disorders, University of Pittsburgh, Pittsburgh, PA, United States
| | - Monique C Tardif
- Department of Communication Science and Disorders, University of Pittsburgh, Pittsburgh, PA, United States
| | - Rachel Tessmer
- Department of Speech, Language, and Hearing Sciences, The University of Texas at Austin, Austin, TX, United States
| | - Heather R Dial
- Department of Speech, Language, and Hearing Sciences, The University of Texas at Austin, Austin, TX, United States.,Department of Communication Sciences and Disorders, University of Houston, Houston, TX, United States
| |
Collapse
|
9
|
Ten Oever S, Martin AE. An oscillating computational model can track pseudo-rhythmic speech by using linguistic predictions. eLife 2021; 10:68066. [PMID: 34338196 PMCID: PMC8328513 DOI: 10.7554/elife.68066] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2021] [Accepted: 07/16/2021] [Indexed: 11/19/2022] Open
Abstract
Neuronal oscillations putatively track speech in order to optimize sensory processing. However, it is unclear how isochronous brain oscillations can track pseudo-rhythmic speech input. Here we propose that oscillations can track pseudo-rhythmic speech when considering that speech time is dependent on content-based predictions flowing from internal language models. We show that temporal dynamics of speech are dependent on the predictability of words in a sentence. A computational model including oscillations, feedback, and inhibition is able to track pseudo-rhythmic speech input. As the model processes, it generates temporal phase codes, which are a candidate mechanism for carrying information forward in time. The model is optimally sensitive to the natural temporal speech dynamics and can explain empirical data on temporal speech illusions. Our results suggest that speech tracking does not have to rely only on the acoustics but could also exploit ongoing interactions between oscillations and constraints flowing from internal language models.
Collapse
Affiliation(s)
- Sanne Ten Oever
- Language and Computation in Neural Systems group, Max Planck Institute for Psycholinguistics, Nijmegen, Netherlands.,Donders Centre for Cognitive Neuroimaging, Radboud University, Nijmegen, Netherlands.,Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
| | - Andrea E Martin
- Language and Computation in Neural Systems group, Max Planck Institute for Psycholinguistics, Nijmegen, Netherlands.,Donders Centre for Cognitive Neuroimaging, Radboud University, Nijmegen, Netherlands
| |
Collapse
|
10
|
|
11
|
Abstract
OBJECTIVES Children with hearing loss (HL), in spite of early cochlear implantation, often struggle considerably with language acquisition. Previous research has shown a benefit of rhythmic training on linguistic skills in children with HL, suggesting that improving rhythmic capacities could help attenuating language difficulties. However, little is known about general rhythmic skills of children with HL and how they relate to speech perception. The aim of this study is twofold: (1) to assess the abilities of children with HL in different rhythmic sensorimotor synchronization tasks compared to a normal-hearing control group and (2) to investigate a possible relation between sensorimotor synchronization abilities and speech perception abilities in children with HL. DESIGN A battery of sensorimotor synchronization tests with stimuli of varying acoustic and temporal complexity was used: a metronome, different musical excerpts, and complex rhythmic patterns. Synchronization abilities were assessed in 32 children (aged from 5 to 10 years) with a severe to profound HL mainly fitted with one or two cochlear implants (n = 28) or with hearing aids (n = 4). Working memory and sentence repetition abilities were also assessed. Performance was compared to an age-matched control group of 24 children with normal hearing. The comparison took into account variability in working memory capacities. For children with HL only, we computed linear regressions on speech, sensorimotor synchronization, and working memory abilities, including device-related variables such as onset of device use, type of device, and duration of use. RESULTS Compared to the normal-hearing group, children with HL performed poorly in all sensorimotor synchronization tasks, but the effect size was greater for complex as compared to simple stimuli. Group differences in working memory did not explain this result. Linear regression analysis revealed that working memory, synchronization to complex rhythms performances, age, and duration of device use predicted the number of correct syllables produced in a sentence repetition task. CONCLUSION Despite early cochlear implantation or hearing aid use, hearing impairment affects the quality of temporal processing of acoustic stimuli in congenitally deaf children. This deficit seems to be more severe with stimuli of increasing rhythmic complexity highlighting a difficulty in structuring sounds according to a temporal hierarchy.
Collapse
|
12
|
Ravignani A, Dalla Bella S, Falk S, Kello CT, Noriega F, Kotz SA. Rhythm in speech and animal vocalizations: a cross-species perspective. Ann N Y Acad Sci 2019; 1453:79-98. [PMID: 31237365 PMCID: PMC6851814 DOI: 10.1111/nyas.14166] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2019] [Revised: 05/14/2019] [Accepted: 05/24/2019] [Indexed: 12/31/2022]
Abstract
Why does human speech have rhythm? As we cannot travel back in time to witness how speech developed its rhythmic properties and why humans have the cognitive skills to process them, we rely on alternative methods to find out. One powerful tool is the comparative approach: studying the presence or absence of cognitive/behavioral traits in other species to determine which traits are shared between species and which are recent human inventions. Vocalizations of many species exhibit temporal structure, but little is known about how these rhythmic structures evolved, are perceived and produced, their biological and developmental bases, and communicative functions. We review the literature on rhythm in speech and animal vocalizations as a first step toward understanding similarities and differences across species. We extend this review to quantitative techniques that are useful for computing rhythmic structure in acoustic sequences and hence facilitate cross-species research. We report links between vocal perception and motor coordination and the differentiation of rhythm based on hierarchical temporal structure. While still far from a complete cross-species perspective of speech rhythm, our review puts some pieces of the puzzle together.
Collapse
Affiliation(s)
- Andrea Ravignani
- Artificial Intelligence LaboratoryVrije Universiteit BrusselBrusselsBelgium
- Institute for Advanced StudyUniversity of AmsterdamAmsterdamthe Netherlands
| | - Simone Dalla Bella
- International Laboratory for BrainMusic and Sound Research (BRAMS)MontréalQuebecCanada
- Department of PsychologyUniversity of MontrealMontréalQuebecCanada
- Department of Cognitive PsychologyWarsawPoland
| | - Simone Falk
- International Laboratory for BrainMusic and Sound Research (BRAMS)MontréalQuebecCanada
- Laboratoire de Phonétique et Phonologie, UMR 7018, CNRS/Université Sorbonne Nouvelle Paris‐3Institut de Linguistique et Phonétique générales et appliquéesParisFrance
| | | | - Florencia Noriega
- Chair for Network DynamicsCenter for Advancing Electronics Dresden (CFAED), TU DresdenDresdenGermany
- CODE University of Applied SciencesBerlinGermany
| | - Sonja A. Kotz
- International Laboratory for BrainMusic and Sound Research (BRAMS)MontréalQuebecCanada
- Basic and Applied NeuroDynamics Laboratory, Faculty of Psychology and Neuroscience, Department of Neuropsychology and PsychopharmacologyMaastricht UniversityMaastrichtthe Netherlands
- Department of NeuropsychologyMax‐Planck Institute for Human Cognitive and Brain SciencesLeipzigGermany
| |
Collapse
|
13
|
Coupé C, Oh YM, Dediu D, Pellegrino F. Different languages, similar encoding efficiency: Comparable information rates across the human communicative niche. SCIENCE ADVANCES 2019; 5:eaaw2594. [PMID: 32047854 PMCID: PMC6984970 DOI: 10.1126/sciadv.aaw2594] [Citation(s) in RCA: 48] [Impact Index Per Article: 9.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/21/2018] [Accepted: 08/02/2019] [Indexed: 05/18/2023]
Abstract
Language is universal, but it has few indisputably universal characteristics, with cross-linguistic variation being the norm. For example, languages differ greatly in the number of syllables they allow, resulting in large variation in the Shannon information per syllable. Nevertheless, all natural languages allow their speakers to efficiently encode and transmit information. We show here, using quantitative methods on a large cross-linguistic corpus of 17 languages, that the coupling between language-level (information per syllable) and speaker-level (speech rate) properties results in languages encoding similar information rates (~39 bits/s) despite wide differences in each property individually: Languages are more similar in information rates than in Shannon information or speech rate. These findings highlight the intimate feedback loops between languages' structural properties and their speakers' neurocognition and biology under communicative pressures. Thus, language is the product of a multiscale communicative niche construction process at the intersection of biology, environment, and culture.
Collapse
Affiliation(s)
- Christophe Coupé
- Laboratoire Dynamique du Langage UMR5596, CNRS, University of Lyon, 14 Avenue Berthelot, 69007 Lyon, France
- Department of Linguistics, The University of Hong Kong, Pokfulam Road, Hong Kong
| | - Yoon Mi Oh
- New Zealand Institute of Language, Brain and Behaviour, University of Canterbury, Private Bag 4800, Christchurch 8140, New Zealand
- Department of French Language and Literature, Ajou University, Suwon, 16499 Gyeonggi-do, South Korea
| | - Dan Dediu
- Laboratoire Dynamique du Langage UMR5596, CNRS, University of Lyon, 14 Avenue Berthelot, 69007 Lyon, France
- Collegium de Lyon, University of Lyon, 24 rue Baldassini, 69007 Lyon, France
| | - François Pellegrino
- Laboratoire Dynamique du Langage UMR5596, CNRS, University of Lyon, 14 Avenue Berthelot, 69007 Lyon, France
- Corresponding author.
| |
Collapse
|
14
|
Lagrois MÉ, Palmer C, Peretz I. Poor Synchronization to Musical Beat Generalizes to Speech. Brain Sci 2019; 9:E157. [PMID: 31277417 PMCID: PMC6680836 DOI: 10.3390/brainsci9070157] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2019] [Accepted: 07/01/2019] [Indexed: 12/24/2022] Open
Abstract
The rhythmic nature of speech may recruit entrainment mechanisms in a manner similar to music. In the current study, we tested the hypothesis that individuals who display a severe deficit in synchronizing their taps to a musical beat (called beat-deaf here) would also experience difficulties entraining to speech. The beat-deaf participants and their matched controls were required to align taps with the perceived regularity in the rhythm of naturally spoken, regularly spoken, and sung sentences. The results showed that beat-deaf individuals synchronized their taps less accurately than the control group across conditions. In addition, participants from both groups exhibited more inter-tap variability to natural speech than to regularly spoken and sung sentences. The findings support the idea that acoustic periodicity is a major factor in domain-general entrainment to both music and speech. Therefore, a beat-finding deficit may affect periodic auditory rhythms in general, not just those for music.
Collapse
Affiliation(s)
- Marie-Élaine Lagrois
- International Laboratory for Brain, Music, and Sound Research, Montreal, QC H3C 3J7, Canada.
- Department of Psychology, University of Montreal, Montreal, QC H3C 3J7, Canada.
| | - Caroline Palmer
- International Laboratory for Brain, Music, and Sound Research, Montreal, QC H3C 3J7, Canada
- Department of Psychology, McGill University, Montreal, QC H3A 1B1, Canada
| | - Isabelle Peretz
- International Laboratory for Brain, Music, and Sound Research, Montreal, QC H3C 3J7, Canada
- Department of Psychology, University of Montreal, Montreal, QC H3C 3J7, Canada
| |
Collapse
|
15
|
Ravignani A, de Reus K. Modelling Animal Interactive Rhythms in Communication. Evol Bioinform Online 2019; 15:1176934318823558. [PMID: 30733626 PMCID: PMC6343447 DOI: 10.1177/1176934318823558] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2018] [Accepted: 11/29/2018] [Indexed: 11/15/2022] Open
Abstract
Time is one crucial dimension conveying information in animal communication. Evolution has shaped animals' nervous systems to produce signals with temporal properties fitting their socio-ecological niches. Many quantitative models of mechanisms underlying rhythmic behaviour exist, spanning insects, crustaceans, birds, amphibians, and mammals. However, these computational and mathematical models are often presented in isolation. Here, we provide an overview of the main mathematical models employed in the study of animal rhythmic communication among conspecifics. After presenting basic definitions and mathematical formalisms, we discuss each individual model. These computational models are then compared using simulated data to uncover similarities and key differences in the underlying mechanisms found across species. Our review of the empirical literature is admittedly limited. We stress the need of using comparative computer simulations - both before and after animal experiments - to better understand animal timing in interaction. We hope this article will serve as a potential first step towards a common computational framework to describe temporal interactions in animals, including humans.
Collapse
Affiliation(s)
- Andrea Ravignani
- Artificial Intelligence Lab, Vrije Universiteit Brussel, Brussels, Belgium
| | - Koen de Reus
- Department Life Sciences, Erasmus University College, Erasmus University Rotterdam, Rotterdam, The Netherlands
| |
Collapse
|
16
|
Kotz S, Ravignani A, Fitch W. The Evolution of Rhythm Processing. Trends Cogn Sci 2018; 22:896-910. [DOI: 10.1016/j.tics.2018.08.002] [Citation(s) in RCA: 68] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2018] [Revised: 07/25/2018] [Accepted: 08/02/2018] [Indexed: 01/14/2023]
|
17
|
Proactive Sensing of Periodic and Aperiodic Auditory Patterns. Trends Cogn Sci 2018; 22:870-882. [DOI: 10.1016/j.tics.2018.08.003] [Citation(s) in RCA: 134] [Impact Index Per Article: 22.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2018] [Revised: 08/12/2018] [Accepted: 08/13/2018] [Indexed: 11/18/2022]
|
18
|
Ravignani A, Kello CT, de Reus K, Kotz SA, Dalla Bella S, Méndez-Aróstegui M, Rapado-Tamarit B, Rubio-Garcia A, de Boer B. Ontogeny of vocal rhythms in harbor seal pups: an exploratory study. Curr Zool 2018; 65:107-120. [PMID: 30697246 PMCID: PMC6347067 DOI: 10.1093/cz/zoy055] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2018] [Accepted: 07/02/2018] [Indexed: 11/15/2022] Open
Abstract
Puppyhood is a very active social and vocal period in a harbor seal’s life Phoca vitulina. An important feature of vocalizations is their temporal and rhythmic structure, and understanding vocal timing and rhythms in harbor seals is critical to a cross-species hypothesis in evolutionary neuroscience that links vocal learning, rhythm perception, and synchronization. This study utilized analytical techniques that may best capture rhythmic structure in pup vocalizations with the goal of examining whether (1) harbor seal pups show rhythmic structure in their calls and (2) rhythms evolve over time. Calls of 3 wild-born seal pups were recorded daily over the course of 1–3 weeks; 3 temporal features were analyzed using 3 complementary techniques. We identified temporal and rhythmic structure in pup calls across different time windows. The calls of harbor seal pups exhibit some degree of temporal and rhythmic organization, which evolves over puppyhood and resembles that of other species’ interactive communication. We suggest next steps for investigating call structure in harbor seal pups and propose comparative hypotheses to test in other pinniped species.
Collapse
Affiliation(s)
- Andrea Ravignani
- Research Department, Sealcentre Pieterburen, Pieterburen, The Netherlands.,Artificial Intelligence Lab, Vrije Universiteit Brussel, Brussels, Belgium
| | - Christopher T Kello
- Cognitive and Information Sciences, University of California, Merced, CA, USA
| | - Koen de Reus
- Research Department, Sealcentre Pieterburen, Pieterburen, The Netherlands
| | - Sonja A Kotz
- Basic and Applied NeuroDynamics Lab, Faculty of Psychology and Neuroscience, Department of Neuropsychology & Psychopharmacology, Maastricht University, Maastricht, The Netherlands.,Department of Neuropsychology, Max-Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.,International Laboratory for Brain, Music and Sound Research (BRAMS), Montréal, QC, Canada
| | - Simone Dalla Bella
- International Laboratory for Brain, Music and Sound Research (BRAMS), Montréal, QC, Canada.,Department of Psychology, University of Montreal, Montréal, QC, Canada.,Department of Cognitive Psychology, WSFiZ in Warsaw, Warsaw, Poland
| | | | | | - Ana Rubio-Garcia
- Research Department, Sealcentre Pieterburen, Pieterburen, The Netherlands
| | - Bart de Boer
- Artificial Intelligence Lab, Vrije Universiteit Brussel, Brussels, Belgium
| |
Collapse
|
19
|
Seifart F, Meyer J, Grawunder S, Dentel L. Reducing language to rhythm: Amazonian Bora drummed language exploits speech rhythm for long-distance communication. ROYAL SOCIETY OPEN SCIENCE 2018; 5:170354. [PMID: 29765620 PMCID: PMC5936885 DOI: 10.1098/rsos.170354] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/24/2017] [Accepted: 03/15/2018] [Indexed: 06/08/2023]
Abstract
Many drum communication systems around the world transmit information by emulating tonal and rhythmic patterns of spoken languages in sequences of drumbeats. Their rhythmic characteristics, in particular, have not been systematically studied so far, although understanding them represents a rare occasion for providing an original insight into the basic units of speech rhythm as selected by natural speech practices directly based on beats. Here, we analyse a corpus of Bora drum communication from the northwest Amazon, which is nowadays endangered with extinction. We show that four rhythmic units are encoded in the length of pauses between beats. We argue that these units correspond to vowel-to-vowel intervals with different numbers of consonants and vowel lengths. By contrast, aligning beats with syllables, mora or only vowel length yields inconsistent results. Moreover, we also show that Bora drummed messages conventionally select rhythmically distinct markers to further distinguish words. The two phonological tones represented in drummed speech encode only few lexical contrasts. Rhythm thus appears to crucially contribute to the intelligibility of drummed Bora. Our study provides novel evidence for the role of rhythmic structures composed of vowel-to-vowel intervals in the complex puzzle concerning the redundancy and distinctiveness of acoustic features embedded in speech.
Collapse
Affiliation(s)
- Frank Seifart
- Amsterdam Center for Language and Communication, University of Amsterdam, 1012 CX Amsterdam, The Netherlands
- Department of Linguistics, University of Cologne, 50923 Köln, Germany
| | - Julien Meyer
- Université Grenoble Alpes, CNRS, GIPSA-lab, 38000 Grenoble, France
- Linguistics Division, Museu Paraense Emilio Goeldi, Belém, Brazil
| | - Sven Grawunder
- Department of Human Behavior, Ecology and Culture, Max Planck Institute for Evolutionary Anthropology, 04103 Leipzig, Germany
- Department of Linguistics, Kiel University, 24118 Kiel, Germany
| | - Laure Dentel
- The World Whistles Research Association, Paris, France
| |
Collapse
|
20
|
Schneider JN, Mercado E. Characterizing the rhythm and tempo of sound production by singing whales. BIOACOUSTICS 2018. [DOI: 10.1080/09524622.2018.1428827] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Affiliation(s)
| | - Eduardo Mercado
- Department of Psychology, University at Buffalo, The State University of New York, Buffalo, NY, USA
| |
Collapse
|
21
|
Ravignani A. Spontaneous rhythms in a harbor seal pup calls. BMC Res Notes 2018; 11:3. [PMID: 29298731 PMCID: PMC5751680 DOI: 10.1186/s13104-017-3107-6] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2017] [Accepted: 12/19/2017] [Indexed: 12/25/2022] Open
Abstract
Objectives Timing and rhythm (i.e. temporal structure) are crucial, though historically neglected, dimensions of animal communication. When investigating these in non-human animals, it is often difficult to balance experimental control and ecological validity. Here I present the first step of an attempt to balance the two, focusing on the timing of vocal rhythms in a harbor seal pup (Phoca vitulina). Collection of this data had a clear aim: To find spontaneous vocal rhythms in this individual in order to design individually-adapted and ecologically-relevant stimuli for a later playback experiment. Data description The calls of one seal pup were recorded. The audio recordings were annotated using Praat, a free software to analyze vocalizations in humans and other animals. The annotated onsets and offsets of vocalizations were then imported in a Python script. The script extracted three types of timing information: the duration of calls, the intervals between calls’ onsets, and the intervals between calls’ maximum-intensity peaks. Based on the annotated data, available to download, I provide simple descriptive statistics for these temporal measures, and compare their distributions.
Collapse
Affiliation(s)
- Andrea Ravignani
- Research Department, Sealcentre Pieterburen, Hoofdstraat 94a, 9968 AG, Pieterburen, The Netherlands. .,Artificial Intelligence Lab, Vrije Universiteit Brussel, Pleinlaan 2, 1050, Brussels, Belgium. .,Language and Cognition Department, Max Planck Institute for Psycholinguistics, Wundtlaan 1, 6525 XD, Nijmegen, The Netherlands.
| |
Collapse
|
22
|
Ravignani A, Madison G. The Paradox of Isochrony in the Evolution of Human Rhythm. Front Psychol 2017; 8:1820. [PMID: 29163252 PMCID: PMC5681750 DOI: 10.3389/fpsyg.2017.01820] [Citation(s) in RCA: 44] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2017] [Accepted: 09/30/2017] [Indexed: 01/04/2023] Open
Abstract
Isochrony is crucial to the rhythm of human music. Some neural, behavioral and anatomical traits underlying rhythm perception and production are shared with a broad range of species. These may either have a common evolutionary origin, or have evolved into similar traits under different evolutionary pressures. Other traits underlying rhythm are rare across species, only found in humans and few other animals. Isochrony, or stable periodicity, is common to most human music, but isochronous behaviors are also found in many species. It appears paradoxical that humans are particularly good at producing and perceiving isochronous patterns, although this ability does not conceivably confer any evolutionary advantage to modern humans. This article will attempt to solve this conundrum. To this end, we define the concept of isochrony from the present functional perspective of physiology, cognitive neuroscience, signal processing, and interactive behavior, and review available evidence on isochrony in the signals of humans and other animals. We then attempt to resolve the paradox of isochrony by expanding an evolutionary hypothesis about the function that isochronous behavior may have had in early hominids. Finally, we propose avenues for empirical research to examine this hypothesis and to understand the evolutionary origin of isochrony in general.
Collapse
Affiliation(s)
- Andrea Ravignani
- Language and Cognition Department, Max Planck Institute for Psycholinguistics, Nijmegen, Netherlands.,Veterinary and Research Department, Sealcentre Pieterburen, Pieterburen, Netherlands.,Artificial Intelligence Lab, Vrije Universiteit Brussel, Brussels, Belgium
| | - Guy Madison
- Department of Psychology, Umeå University, Umeå, Sweden
| |
Collapse
|