1
|
Ross JM, Balasubramaniam R. Time Perception for Musical Rhythms: Sensorimotor Perspectives on Entrainment, Simulation, and Prediction. Front Integr Neurosci 2022; 16:916220. [PMID: 35865808 PMCID: PMC9294366 DOI: 10.3389/fnint.2022.916220] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2022] [Accepted: 06/16/2022] [Indexed: 11/19/2022] Open
Abstract
Neural mechanisms supporting time perception in continuously changing sensory environments may be relevant to a broader understanding of how the human brain utilizes time in cognition and action. In this review, we describe current theories of sensorimotor engagement in the support of subsecond timing. We focus on musical timing due to the extensive literature surrounding movement with and perception of musical rhythms. First, we define commonly used but ambiguous concepts including neural entrainment, simulation, and prediction in the context of musical timing. Next, we summarize the literature on sensorimotor timing during perception and performance and describe current theories of sensorimotor engagement in the support of subsecond timing. We review the evidence supporting that sensorimotor engagement is critical in accurate time perception. Finally, potential clinical implications for a sensorimotor perspective of timing are highlighted.
Collapse
Affiliation(s)
- Jessica M. Ross
- Veterans Affairs Palo Alto Healthcare System and the Sierra Pacific Mental Illness, Research, Education, and Clinical Center, Palo Alto, CA, United States
- Department of Psychiatry and Behavioral Sciences, Stanford University Medical Center, Stanford, CA, United States
- Berenson-Allen Center for Non-invasive Brain Stimulation, Beth Israel Deaconess Medical Center, Boston, MA, United States
- Department of Neurology, Harvard Medical School, Boston, MA, United States
- *Correspondence: Jessica M. Ross,
| | - Ramesh Balasubramaniam
- Cognitive and Information Sciences, University of California, Merced, Merced, CA, United States
| |
Collapse
|
2
|
Villegas J, Perkins J, Wilson I. Effects of task and language nativeness on the Lombard effect and on its onset and offset timing. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2021; 149:1855. [PMID: 33765802 DOI: 10.1121/10.0003772] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/17/2020] [Accepted: 02/24/2021] [Indexed: 06/12/2023]
Abstract
This study focuses on the differences in speech sound pressure levels (here, called speech loudness) of Lombard speech (i.e., speech produced in the presence of an energetic masker) associated with different tasks and language nativeness. Vocalizations were produced by native speakers of Japanese with normal hearing and limited English proficiency while performing four tasks: dialog, a competitive game (both communicative), soliloquy, and text passage reading (noncommunicative). Relative to the native language (L1), larger loudness increments were observed in the game and text reading when performed in the second language (L2). Communicative tasks yielded louder vocalizations and larger increments of speech loudness than did noncommunicative tasks regardless of the spoken language. The period in which speakers increased their loudness after the onset of the masker was about fourfold longer than the time in which they decreased their loudness after the offset of the masker. Results suggest that when relying on acoustic signals, speakers use similar vocalization strategies in L1 and L2, and these depend on the complexity of the task, the need for accurate pronunciation, and the presence of a listener. Results also suggest that speakers use different strategies depending on the onset or offset of an energetic masker.
Collapse
Affiliation(s)
- Julián Villegas
- Computer Arts Laboratory, University of Aizu, Aizu-Wakamatsu, Fukushima, 965-8580, Japan
| | - Jeremy Perkins
- CLR Phonetics Laboratory, University of Aizu, Aizu-Wakamatsu, Fukushima, 965-8580, Japan
| | - Ian Wilson
- CLR Phonetics Laboratory, University of Aizu, Aizu-Wakamatsu, Fukushima, 965-8580, Japan
| |
Collapse
|
3
|
Xu M, Tachibana RO, Okanoya K, Hagiwara H, Hashimoto RI, Homae F. Unconscious and Distinctive Control of Vocal Pitch and Timbre During Altered Auditory Feedback. Front Psychol 2020; 11:1224. [PMID: 32581975 PMCID: PMC7294928 DOI: 10.3389/fpsyg.2020.01224] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2019] [Accepted: 05/11/2020] [Indexed: 01/01/2023] Open
Abstract
Vocal control plays a critical role in smooth social communication. Speakers constantly monitor auditory feedback (AF) and make adjustments when their voices deviate from their intentions. Previous studies have shown that when certain acoustic features of the AF are artificially altered, speakers compensate for this alteration in the opposite direction. However, little is known about how the vocal control system implements compensations for alterations of different acoustic features, and associates them with subjective consciousness. The present study investigated whether compensations for the fundamental frequency (F0), which corresponds to perceived pitch, and formants, which contribute to perceived timbre, can be performed unconsciously and independently. Forty native Japanese speakers received two types of altered AF during vowel production that involved shifts of either only the formant frequencies (formant modification; Fm) or both the pitch and formant frequencies (pitch + formant modification; PFm). For each type, three levels of shift (slight, medium, and severe) in both directions (increase or decrease) were used. After the experiment, participants were tested for whether they had perceived a change in the F0 and/or formants. The results showed that (i) only formants were compensated for in the Fm condition, while both the F0 and formants were compensated for in the PFm condition; (ii) the F0 compensation exhibited greater precision than the formant compensation in PFm; and (iii) compensation occurred even when participants misperceived or could not explicitly perceive the alteration in AF. These findings indicate that non-experts can compensate for both formant and F0 modifications in the AF during vocal production, even when the modifications are not explicitly or correctly perceived, which provides further evidence for a dissociation between conscious perception and action in vocal control. We propose that such unconscious control of voice production may enhance rapid adaptation to changing speech environments and facilitate mutual communication.
Collapse
Affiliation(s)
- Mingdi Xu
- Department of Language Sciences, Graduate School of Humanities, Tokyo Metropolitan University, Tokyo, Japan
| | - Ryosuke O Tachibana
- Department of Life Sciences, Graduate School of Arts and Sciences, The University of Tokyo, Tokyo, Japan
| | - Kazuo Okanoya
- Department of Life Sciences, Graduate School of Arts and Sciences, The University of Tokyo, Tokyo, Japan
| | - Hiroko Hagiwara
- Department of Language Sciences, Graduate School of Humanities, Tokyo Metropolitan University, Tokyo, Japan.,Research Center for Language, Brain and Genetics, Tokyo Metropolitan University, Tokyo, Japan
| | - Ryu-Ichiro Hashimoto
- Department of Language Sciences, Graduate School of Humanities, Tokyo Metropolitan University, Tokyo, Japan.,Research Center for Language, Brain and Genetics, Tokyo Metropolitan University, Tokyo, Japan
| | - Fumitaka Homae
- Department of Language Sciences, Graduate School of Humanities, Tokyo Metropolitan University, Tokyo, Japan.,Research Center for Language, Brain and Genetics, Tokyo Metropolitan University, Tokyo, Japan
| |
Collapse
|
4
|
Pomberger T, Löschner J, Hage SR. Compensatory mechanisms affect sensorimotor integration during ongoing vocal motor acts in marmoset monkeys. Eur J Neurosci 2020; 52:3531-3544. [PMID: 32170972 DOI: 10.1111/ejn.14721] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2019] [Revised: 02/20/2020] [Accepted: 03/10/2020] [Indexed: 01/08/2023]
Abstract
Any transmission of vocal signals faces the challenge of acoustic interferences such as heavy rain, wind, animal or urban sounds. Consequently, several mechanisms and strategies have evolved to optimize signal-to-noise ratio. Examples to increase detectability are the Lombard effect, an involuntary rise in call amplitude in response to masking ambient noise, which is often associated with other vocal changes such as call frequency and duration, as well as the animals' capability of limiting calling to periods where noise perturbation is absent. Previous studies revealed vocal flexibility and various audio-vocal integration mechanisms in marmoset monkeys. Using acoustic perturbation triggered by vocal behaviour, we investigated whether marmosets are capable of exhibiting changes in call structure when perturbing noise starts after call onset or whether such effects only occur if noise perturbation starts prior to call onset. We show that marmosets are capable of rapidly modulating call amplitude and frequency in response to such noise perturbation. Vocalizations swiftly increased call frequency after noise onset indicating a rapid effect of perturbing noise on vocal motor production. Call amplitudes were also affected. Interestingly, however, the marmosets did not exhibit the Lombard effect as previously reported but decreased call intensity in response to noise. Our findings indicate that marmosets possess a general avoidance strategy to call in the presence of ambient noise and suggest that these animals are capable of counteracting a previously thought involuntary audio-vocal mechanism, the Lombard effect. These findings will pave the way to investigate the underlying audio-vocal integration mechanisms explaining these behaviours.
Collapse
Affiliation(s)
- Thomas Pomberger
- Neurobiology of Vocal Communication, Werner Reichardt Centre for Integrative Neuroscience, University of Tübingen, Tübingen, Germany.,Graduate School of Neural & Behavioural Sciences - International Max Planck Research School, University of Tübingen, Tübingen, Germany
| | - Julia Löschner
- Neurobiology of Vocal Communication, Werner Reichardt Centre for Integrative Neuroscience, University of Tübingen, Tübingen, Germany
| | - Steffen R Hage
- Neurobiology of Vocal Communication, Werner Reichardt Centre for Integrative Neuroscience, University of Tübingen, Tübingen, Germany.,Department of Otolaryngology - Head and Neck Surgery, Hearing Research Center, University of Tübingen, Medical Center, Tübingen, Germany
| |
Collapse
|
5
|
Correia JM, Caballero-Gaudes C, Guediche S, Carreiras M. Phonatory and articulatory representations of speech production in cortical and subcortical fMRI responses. Sci Rep 2020; 10:4529. [PMID: 32161310 PMCID: PMC7066132 DOI: 10.1038/s41598-020-61435-y] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2019] [Accepted: 02/24/2020] [Indexed: 11/25/2022] Open
Abstract
Speaking involves coordination of multiple neuromotor systems, including respiration, phonation and articulation. Developing non-invasive imaging methods to study how the brain controls these systems is critical for understanding the neurobiology of speech production. Recent models and animal research suggest that regions beyond the primary motor cortex (M1) help orchestrate the neuromotor control needed for speaking, including cortical and sub-cortical regions. Using contrasts between speech conditions with controlled respiratory behavior, this fMRI study investigates articulatory gestures involving the tongue, lips and velum (i.e., alveolars versus bilabials, and nasals versus orals), and phonatory gestures (i.e., voiced versus whispered speech). Multivariate pattern analysis (MVPA) was used to decode articulatory gestures in M1, cerebellum and basal ganglia. Furthermore, apart from confirming the role of a mid-M1 region for phonation, we found that a dorsal M1 region, linked to respiratory control, showed significant differences for voiced compared to whispered speech despite matched lung volume observations. This region was also functionally connected to tongue and lip M1 seed regions, underlying its importance in the coordination of speech. Our study confirms and extends current knowledge regarding the neural mechanisms underlying neuromotor speech control, which hold promise to study neural dysfunctions involved in motor-speech disorders non-invasively.
Collapse
Affiliation(s)
- Joao M Correia
- BCBL, Basque Center on Cognition Brain and Language, San Sebastian, Spain. .,Centre for Biomedical Research (CBMR)/Department of Psychology, University of Algarve, Faro, Portugal.
| | | | - Sara Guediche
- BCBL, Basque Center on Cognition Brain and Language, San Sebastian, Spain
| | - Manuel Carreiras
- BCBL, Basque Center on Cognition Brain and Language, San Sebastian, Spain.,Ikerbasque. Basque Foundation for Science, Bilbao, Spain.,University of the Basque Country. UPV/EHU, Bilbao, Spain
| |
Collapse
|
6
|
Luo J, Hage SR, Moss CF. The Lombard Effect: From Acoustics to Neural Mechanisms. Trends Neurosci 2018; 41:938-949. [DOI: 10.1016/j.tins.2018.07.011] [Citation(s) in RCA: 43] [Impact Index Per Article: 7.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2018] [Revised: 06/27/2018] [Accepted: 07/20/2018] [Indexed: 01/12/2023]
|
7
|
The impact of perilaryngeal vibration on the self-perception of loudness and the Lombard effect. Exp Brain Res 2018; 236:1713-1723. [PMID: 29623381 DOI: 10.1007/s00221-018-5248-9] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2017] [Accepted: 03/29/2018] [Indexed: 10/17/2022]
Abstract
The role of somatosensory feedback in speech and the perception of loudness was assessed in adults without speech or hearing disorders. Participants completed two tasks: loudness magnitude estimation of a short vowel and oral reading of a standard passage. Both tasks were carried out in each of three conditions: no-masking, auditory masking alone, and mixed auditory masking plus vibration of the perilaryngeal area. A Lombard effect was elicited in both masking conditions: speakers unconsciously increased vocal intensity. Perilaryngeal vibration further increased vocal intensity above what was observed for auditory masking alone. Both masking conditions affected fundamental frequency and the first formant frequency as well, but only vibration was associated with a significant change in the second formant frequency. An additional analysis of pure-tone thresholds found no difference in auditory thresholds between masking conditions. Taken together, these findings indicate that perilaryngeal vibration effectively masked somatosensory feedback, resulting in an enhanced Lombard effect (increased vocal intensity) that did not alter speakers' self-perception of loudness. This implies that the Lombard effect results from a general sensorimotor process, rather than from a specific audio-vocal mechanism, and that the conscious self-monitoring of speech intensity is not directly based on either auditory or somatosensory feedback.
Collapse
|
8
|
Balasubramaniam R, Hove MJ, Médé B. Factorization of Force and Timing in Sensorimotor Performance: Long-Range Correlation Properties of Two Different Task Goals. Top Cogn Sci 2017; 10:120-132. [DOI: 10.1111/tops.12301] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2016] [Revised: 08/10/2017] [Accepted: 08/23/2017] [Indexed: 10/18/2022]
Affiliation(s)
| | - Michael J. Hove
- Department of Psychological Science; Fitchburg State University
| | - Butovens Médé
- Cognitive & Information Sciences; University of California; Merced
| |
Collapse
|
9
|
Abstract
There is growing interest in whether the motor system plays an essential role in rhythm perception. The motor system is active during the perception of rhythms, but is such motor activity merely a sign of unexecuted motor planning, or does it play a causal role in shaping the perception of rhythm? We present evidence for a causal role of motor planning and simulation, and review theories of internal simulation for beat-based timing prediction. Brain stimulation studies have the potential to conclusively test if the motor system plays a causal role in beat perception and ground theories to their neural underpinnings.
Collapse
Affiliation(s)
- Jessica M Ross
- a Cognitive and Information Sciences , University of California , Merced , CA , USA
| | - John R Iversen
- b Swartz Center for Computational Neuroscience, Institute for Neural Computation , University of California , San Diego , CA , USA
| | | |
Collapse
|
10
|
Autophonic Loudness of Singers in Simulated Room Acoustic Environments. J Voice 2016; 31:388.e13-388.e25. [PMID: 27742495 DOI: 10.1016/j.jvoice.2016.09.016] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2016] [Revised: 09/14/2016] [Accepted: 09/15/2016] [Indexed: 11/24/2022]
Abstract
OBJECTIVES This paper aims to study the effect of room acoustics and phonemes on the perception of loudness of one's own voice (autophonic loudness) for a group of trained singers. METHODS For a set of five phonemes, 20 singers vocalized over several autophonic loudness ratios, while maintaining pitch constancy over extreme voice levels, within five simulated rooms. RESULTS There were statistically significant differences in the slope of the autophonic loudness function (logarithm of autophonic loudness as a function of voice sound pressure level) for the five phonemes, with slopes ranging from 1.3 (/a:/) to 2.0 (/z/). There was no significant variation in the autophonic loudness function slopes with variations in room acoustics. The autophonic room response, which represents a systematic decrease in voice levels with increasing levels of room reflections, was also studied, with some evidence found in support. Overall, the average slope of the autophonic room response for the three corner vowels (/a:/, /i:/, and /u:/) was -1.4 for medium autophonic loudness. CONCLUSIONS The findings relating to the slope of the autophonic loudness function are in agreement with the findings of previous studies where the sensorimotor mechanisms in regulating voice were shown to be more important in the perception of autophonic loudness than hearing of room acoustics. However, the role of room acoustics, in terms of the autophonic room response, is shown to be more complicated, requiring further inquiry. Overall, it is shown that autophonic loudness grows at more than twice the rate of loudness growth for sounds created outside the human body.
Collapse
|
11
|
Ross JM, Balasubramaniam R. Physical and neural entrainment to rhythm: human sensorimotor coordination across tasks and effector systems. Front Hum Neurosci 2014; 8:576. [PMID: 25136306 PMCID: PMC4118030 DOI: 10.3389/fnhum.2014.00576] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2014] [Accepted: 07/14/2014] [Indexed: 11/13/2022] Open
Abstract
The human sensorimotor system can be readily entrained to environmental rhythms, through multiple sensory modalities. In this review, we provide an overview of theories of timekeeping that make this neuroentrainment possible. First, we present recent evidence that contests the assumptions made in classic timekeeper models. The role of state estimation, sensory feedback and movement parameters on the organization of sensorimotor timing are discussed in the context of recent experiments that examined simultaneous timing and force control. This discussion is extended to the study of coordinated multi-effector movements and how they may be entrained.
Collapse
Affiliation(s)
- Jessica Marie Ross
- Sensorimotor Neuroscience Laboratory, Cognitive and Information Sciences, University of California Merced, CA, USA
| | - Ramesh Balasubramaniam
- Sensorimotor Neuroscience Laboratory, Cognitive and Information Sciences, University of California Merced, CA, USA
| |
Collapse
|