1
|
Pearson L, Pouw W. Gesture-vocal coupling in Karnatak music performance: A neuro-bodily distributed aesthetic entanglement. Ann N Y Acad Sci 2022; 1515:219-236. [PMID: 35730069 DOI: 10.1111/nyas.14806] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/30/2022]
Abstract
In many musical styles, vocalists manually gesture while they sing. Coupling between gesture kinematics and vocalization has been examined in speech contexts, but it is an open question how these couple in music making. We examine this in a corpus of South Indian, Karnatak vocal music that includes motion-capture data. Through peak magnitude analysis (linear mixed regression) and continuous time-series analyses (generalized additive modeling), we assessed whether vocal trajectories around peaks in vertical velocity, speed, or acceleration were coupling with changes in vocal acoustics (namely, F0 and amplitude). Kinematic coupling was stronger for F0 change versus amplitude, pointing to F0's musical significance. Acceleration was the most predictive for F0 change and had the most reliable magnitude coupling, showing a one-third power relation. That acceleration, rather than other kinematics, is maximally predictive for vocalization is interesting because acceleration entails force transfers onto the body. As a theoretical contribution, we argue that gesturing in musical contexts should be understood in relation to the physical connections between gesturing and vocal production that are brought into harmony with the vocalists' (enculturated) performance goals. Gesture-vocal coupling should, therefore, be viewed as a neuro-bodily distributed aesthetic entanglement.
Collapse
Affiliation(s)
- Lara Pearson
- Department of Music, Max Planck Institute for Empirical Aesthetics, Frankfurt am Main, Germany
| | - Wim Pouw
- Donders Institute for Brain, Cognition, and Behaviour, Radboud University, Nijmegen, The Netherlands.,Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands
| |
Collapse
|
2
|
The speed-curvature power law in tongue movements of repetitive speech. PLoS One 2019; 14:e0213851. [PMID: 30883586 PMCID: PMC6422270 DOI: 10.1371/journal.pone.0213851] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2018] [Accepted: 03/01/2019] [Indexed: 11/19/2022] Open
Abstract
The speed-curvature power law is a celebrated law of motor control expressing a relation between the kinematic property of speed and the geometric property of curvature. We aimed to assess whether speech movements obey this law just as movements from other domains do. We describe a metronome-driven speech elicitation paradigm designed to cover a wide range of speeds. We recorded via electromagnetic articulometry speech movements in sequences of the form /CV…/ from nine speakers (five German, four English) speaking at eight distinct rates. First, we demonstrate that the paradigm of metronome-driven manipulations results in speech movement data consistent with earlier reports on the kinematics of speech production. Second, analysis of our data in their full three-dimensions and using advanced numerical differentiation methods offers stronger evidence for the law than that reported in previous studies devoted to its assessment. Finally, we demonstrate the presence of a clear rate dependency of the power law's parameters. The robustness of the speed-curvature relation in our datasets lends further support to the hypothesis that the power law is a general feature of human movement. We place our results in the context of other work in movement control and consider implications for models of speech production.
Collapse
|
3
|
Parrell B, Lammert AC, Ciccarelli G, Quatieri TF. Current models of speech motor control: A control-theoretic overview of architectures and properties. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2019; 145:1456. [PMID: 31067944 DOI: 10.1121/1.5092807] [Citation(s) in RCA: 29] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/01/2018] [Accepted: 02/13/2019] [Indexed: 06/09/2023]
Abstract
This paper reviews the current state of several formal models of speech motor control, with particular focus on the low-level control of the speech articulators. Further development of speech motor control models may be aided by a comparison of model attributes. The review builds an understanding of existing models from first principles, before moving into a discussion of several models, showing how each is constructed out of the same basic domain-general ideas and components-e.g., generalized feedforward, feedback, and model predictive components. This approach allows for direct comparisons to be made in terms of where the models differ, and their points of agreement. Substantial differences among models can be observed in their use of feedforward control, process of estimating system state, and method of incorporating feedback signals into control. However, many commonalities exist among the models in terms of their reliance on higher-level motor planning, use of feedback signals, lack of time-variant adaptation, and focus on kinematic aspects of control and biomechanics. Ongoing research bridging hybrid feedforward/feedback pathways with forward dynamic control, as well as feedback/internal model-based state estimation, is discussed.
Collapse
Affiliation(s)
- Benjamin Parrell
- Department of Communication Sciences & Disorders, University of Wisconsin-Madison, Madison, Wisconsin 53706, USA
| | - Adam C Lammert
- Bioengineering Systems & Technologies, MIT Lincoln Laboratory, Lexington, Massachusetts 02421, USA
| | - Gregory Ciccarelli
- Bioengineering Systems & Technologies, MIT Lincoln Laboratory, Lexington, Massachusetts 02421, USA
| | - Thomas F Quatieri
- Bioengineering Systems & Technologies, MIT Lincoln Laboratory, Lexington, Massachusetts 02421, USA
| |
Collapse
|
4
|
Abstract
Speech motor actions are performed quickly, while simultaneously maintaining a high degree of accuracy. Are speed and accuracy in conflict during speech production? Speed-accuracy tradeoffs have been shown in many domains of human motor action, but have not been directly examined in the domain of speech production. The present work seeks evidence for Fitts' law, a rigorous formulation of this fundamental tradeoff, in speech articulation kinematics by analyzing USC-TIMIT, a real-time magnetic resonance imaging data set of speech production. A theoretical framework for considering Fitts' law with respect to models of speech motor control is elucidated. Methodological challenges in seeking relationships consistent with Fitts' law are addressed, including the operational definitions and measurement of key variables in real-time MRI data. Results suggest the presence of speed-accuracy tradeoffs for certain types of speech production actions, with wide variability across syllable position, and substantial variability also across subjects. Coda consonant targets immediately following the syllabic nucleus show the strongest evidence of this tradeoff, with correlations as high as 0.72 between speed and accuracy. A discussion is provided concerning the potentially limited applicability of Fitts' law in the context of speech production, as well as the theoretical context for interpreting the results.
Collapse
|
5
|
Marken RS, Shaffer DM. The power law of movement: an example of a behavioral illusion. Exp Brain Res 2017; 235:1835-1842. [PMID: 28299410 DOI: 10.1007/s00221-017-4939-y] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2016] [Accepted: 02/28/2017] [Indexed: 11/26/2022]
Abstract
The curved movements produced by living organisms follow a power law where the velocity of movement is a power function of the degree of curvature through which the movement is made. The exponent of the power function is close to either 1/3 or 2/3 depending on how velocity and curvature are measured. This power law is thought to reflect biological and/or kinematic constraints on how organisms produce movements. The present paper shows that the power law is actually a statistical artifact that results from mistaking a correlational for a causal relationship between variables. The power law implies that curvature influences the velocity of movement. In fact, the power law is a mathematical consequence of the way that these variables are calculated. The appearance that curvature affects the velocity of movement is shown to be an example of a "behavioral illusion" that results from ignoring the purpose of behavior.
Collapse
Affiliation(s)
| | - Dennis M Shaffer
- Department of Psychology, Ohio State University Mansfield, Mansfield, Ohio, USA
| |
Collapse
|
6
|
Catavitello G, Ivanenko YP, Lacquaniti F, Viviani P. Drawing ellipses in water: evidence for dynamic constraints in the relation between velocity and path curvature. Exp Brain Res 2016; 234:1649-57. [PMID: 26838360 DOI: 10.1007/s00221-016-4569-9] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2015] [Accepted: 01/20/2016] [Indexed: 11/26/2022]
Abstract
Several types of continuous human movements comply with the so-called Two-Thirds Power Law (2/3-PL) stating that velocity (V) is a power function of the radius of curvature (R) of the endpoint trajectory. The origin of the 2/3-PL has been the object of much debate. An experiment investigated further this issue by comparing two-dimensional drawing movements performed in air and water. In both conditions, participants traced continuously quasi-elliptic trajectories (period T = 1.5 s). Other experimental factors were the movement plane (horizontal/vertical), and whether the movement was performed free-hand, or by following the edge of a template. In all cases a power function provided a good approximation to the V-R relation. The main result was that the exponent of the power function in water was significantly smaller than in air. This appears incompatible with the idea that the power relationship depends only on kinematic constraints and suggests a significant contribution of dynamic factors. We argue that a satisfactory explanation of the observed behavior must take into account the interplay between the properties of the central motor commands and the visco-elastic nature of the mechanical plant that implements the commands.
Collapse
Affiliation(s)
- Giovanna Catavitello
- Laboratory of Neuromotor Physiology, Santa Lucia Foundation, via Ardeatina, 306-00179, Rome, Italy
- Centre of Space BioMedicine, University of Rome Tor Vergata, 00133, Rome, Italy
| | - Yuri P Ivanenko
- Laboratory of Neuromotor Physiology, Santa Lucia Foundation, via Ardeatina, 306-00179, Rome, Italy.
| | - Francesco Lacquaniti
- Laboratory of Neuromotor Physiology, Santa Lucia Foundation, via Ardeatina, 306-00179, Rome, Italy
- Centre of Space BioMedicine, University of Rome Tor Vergata, 00133, Rome, Italy
- Department of Systems Medicine, University of Rome Tor Vergata, 00133, Rome, Italy
| | - Paolo Viviani
- Laboratory of Neuromotor Physiology, Santa Lucia Foundation, via Ardeatina, 306-00179, Rome, Italy
- Centre of Space BioMedicine, University of Rome Tor Vergata, 00133, Rome, Italy
| |
Collapse
|
7
|
Pape D, Jesus LMT. Cue-weighting in the perception of intervocalic stop voicing in European Portuguese. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2014; 136:1334. [PMID: 25190406 DOI: 10.1121/1.4890639] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
This paper describes the perception of intervocalic stop voicing in European Portuguese (EP) stimuli without a stop burst, when varying three acoustic cues: Vowel duration, stop duration, and voicing maintenance during stop closure. Perceptual stimuli were generated using biomechanical modeling. First, a discrimination experiment was conducted to determine the listeners' perceptual sensitivity to the voicing maintenance cue. Second, an identification experiment was conducted to examine the effect and interaction of vowel duration, stop duration, and voicing maintenance during stop closure on the voiced/voiceless identification responses of EP listeners. The results of the discrimination test show that voicing maintenance differences have a significant effect as soon as they exceed a certain threshold. In the identification experiment, evidence was found that only the two factors vowel duration and voicing maintenance significantly influence the listeners' decisions, but not stop duration. The ratio between stop duration and vowel duration plays a major role in distinguishing stop voicing, but only for highly devoiced stimuli. It is shown that in stimuli without a stop burst, both voicing maintenance, as a major but not required cue, and vowel duration are important acoustic cues for stop voicing distinctions in EP.
Collapse
Affiliation(s)
- Daniel Pape
- Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro, 3830-193 Aveiro, Portugal
| | - Luis M T Jesus
- Institute of Electronics and Informatics Engineering of Aveiro (IEETA) and School of Health Sciences (ESSUA), University of Aveiro, 3830-193 Aveiro, Portugal
| |
Collapse
|
8
|
Neufeld C, van Lieshout P. Tongue kinematics in palate relative coordinate spaces for electro-magnetic articulography. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2014; 135:352-361. [PMID: 24437775 DOI: 10.1121/1.4836515] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
This paper describes a method for constructing a three-dimensional model of the hard palate using electro-magnetic articulography, and defines two algorithms to derive constriction degree and constriction location values from the trajectories of tongue coils using this model. The kinematics of tongue motion that have been transformed into constriction degree and constriction location values are investigated in detail to determine whether this type of representation obeys the constraints theorized to operate over higher level motor control. Results show that palate-relative coordinate spaces decouple mechanical dependencies present in the tongue, while maintaining low-level kinematic properties. They additionally preserve the 1/3 power law for speed and curvature observed across many motor systems. Finally, it is shown that tongue movements in a palate relative coordinate space more closely correspond to their optimal, jerk-minimized trajectories. These results suggest that this type of coordinate space provides a closer match to higher level motor-planning, in line with production models that specify control units in terms of vocal tract constriction parameters.
Collapse
Affiliation(s)
- Chris Neufeld
- Department of Linguistics, University of Maryland, 1401 Marie Mount Hall, College Park, Maryland 20742
| | - Pascal van Lieshout
- Department of Speech-Language Pathology, University of Toronto, 160-500 University Avenue, Toronto, Ontario, M5G 1V7, Canada
| |
Collapse
|
9
|
Nazari MA, Perrier P, Payan Y. The distributed lambda (λ) model (DLM): a 3-D, finite-element muscle model based on Feldman's λ model; assessment of orofacial gestures. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2013; 56:S1909-S1923. [PMID: 24687446 DOI: 10.1044/1092-4388(2013/12-0222)] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
PURPOSE The authors aimed to design a distributed lambda model (DLM), which is well adapted to implement three-dimensional (3-D), finite-element descriptions of muscles. METHOD A muscle element model was designed. Its stress-strain relationships included the active force-length characteristics of the λ model along the muscle fibers, together with the passive properties of muscle tissues in the 3-D space. The muscle element was first assessed using simple geometrical representations of muscles in the form of rectangular bars. It was then included in a 3-D face model, and its impact on lip protrusion was compared with the impact of a Hill-type muscle model. RESULTS The force-length characteristic associated with the muscle elements matched well with the invariant characteristics of the λ model. The impact of the passive properties was assessed. Isometric force variation and isotonic displacements were modeled. The comparison with a Hill-type model revealed strong similarities in terms of global stress and strain. CONCLUSION The DLM accounted for the characteristics of the λ model. Biomechanically, no clear differences were found between the DLM and a Hill-type model. Accurate evaluations of the λ model, based on the comparison between data and simulations, are now possible with 3-D biomechanical descriptions of the speech articulators because of the DLM.
Collapse
|
10
|
Weirich M, Lancia L, Brunner J. Inter-speaker articulatory variability during vowel-consonant-vowel sequences in twins and unrelated speakers. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2013; 134:3766-3780. [PMID: 24180787 DOI: 10.1121/1.4822480] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/02/2023]
Abstract
The purpose of this study is to examine and compare the amount of inter-speaker variability in the articulation of monozygotic twin pairs (MZ), dizygotic twin pairs (DZ), and pairs of unrelated twins with the goal of examining in greater depth the influence of physiology on articulation. Physiological parameters are assumed to be very similar in MZ twin pairs in contrast to DZ twin pairs or unrelated speakers, and it is hypothesized that the speaker specific shape of articulatory looping trajectories of the tongue is at least partly dependent on biomechanical properties and the speaker's individual physiology. By means of electromagnetic articulography (EMA), inter-speaker variability in the looping trajectories of the tongue back during /VCV/ sequences is analyzed. Results reveal similar looping patterns within MZ twin pairs but in DZ pairs differences in the shape of the loop, the direction of the upward and downward movement, and the amount of horizontal sliding movement at the palate are found.
Collapse
Affiliation(s)
- Melanie Weirich
- Zentrum für Allgemeine Sprachwissenschaft, Schützenstrasse 18, 10117 Berlin, Germany
| | | | | |
Collapse
|
11
|
Abstract
The two-thirds power law, postulating an inverse local relation between the velocity and cubed root of curvature of planar trajectories, is a long-established simplifying principle of human hand movements. In perception, the motion of a dot along a planar elliptical path appears most uniform for speed profiles closer to those predicted by the power law than to constant Euclidean speed, a kinetic-visual illusion. Mathematically, complying with this law is equivalent to moving at constant planar equi-affine speed, while unconstrained three-dimensional drawing movements generally follow constant spatial equi-affine speed. Here we test the generalization of this illusion to visual perception of spatial motion for a dot moving along five differently shaped paths, using stereoscopic projection. The movements appeared most uniform for speed profiles closer to constant spatial equi-affine speed than to constant Euclidean speed, with path torsion (i.e., local deviation from planarity) directly affecting the speed profiles perceived as most uniform, as predicted for constant spatial equi-affine speed. This demonstrates the dominance of equi-affine geometry in spatial motion perception. However, constant equi-affine speed did not fully account for the variability among the speed profiles selected as most uniform for different shapes. Moreover, in a followup experiment, we found that viewing distance affected the speed profile reported as most uniform for the extensively studied planar elliptical motion paths. These findings provide evidence for the critical role of equi-affine geometry in spatial motion perception and contribute to the mounting evidence for the role of non-Euclidean geometries in motion perception and production.
Collapse
Affiliation(s)
- Uri Maoz
- Department of Computer Science and Applied Mathematics, Weizmann Institute of Science, Rehovot, Israel
| | | |
Collapse
|
12
|
Endres D, Meirovitch Y, Flash T, Giese MA. Segmenting sign language into motor primitives with Bayesian binning. Front Comput Neurosci 2013; 7:68. [PMID: 23750135 PMCID: PMC3664315 DOI: 10.3389/fncom.2013.00068] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2013] [Accepted: 05/08/2013] [Indexed: 11/16/2022] Open
Abstract
The endpoint trajectories of human movements fulfill characteristic power laws linking velocity and curvature. The parameters of these power laws typically vary between different segments of longer action sequences. These parameters might thus be exploited for the unsupervised segmentation of actions into movement primitives. For the example of sign language we investigate whether such segments can be identified by Bayesian binning (BB), using a Gaussian observation model whose mean has a polynomial time dependence. We show that this method yields good segmentation and correctly models ground truth kinematics composed of consecutive segments derived from wrist trajectories recorded from users of Israeli Sign Language (ISL). Importantly, polynomial orders between 3 and 5 yield an optimal trade-off between complexity and accuracy of the trajectory approximation, in accordance with the minimum acceleration and minimum jerk models. Comparing the orders of the polynomials best approximating natural kinematics against those needed to fit the power law ground truth data suggests that kinematic properties not compatible with power laws are also not adequately represented by low order polynomials and require higher order polynomials for a good approximation.
Collapse
Affiliation(s)
- Dominik Endres
- Department of Cognitive Neurology, Section Computational Sensomotorics, CIN, HIH and University Clinic TübingenTübingen, Germany
| | - Yaron Meirovitch
- Department of Applied Mathematics and Computer Science, The Weizmann Institute of ScienceRehovot, Israel
| | - Tamar Flash
- Department of Applied Mathematics and Computer Science, The Weizmann Institute of ScienceRehovot, Israel
| | - Martin A. Giese
- Department of Cognitive Neurology, Section Computational Sensomotorics, CIN, HIH and University Clinic TübingenTübingen, Germany
| |
Collapse
|
13
|
Recasens D, Espinosa A. Lingual kinematics and coarticulation for alveolopalatal and velar consonants in Catalan. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2010; 127:3154-3165. [PMID: 21117764 DOI: 10.1121/1.3372631] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/30/2023]
Abstract
Vertical lingual movement data for the alveolopalatal consonants /ʃ/ and /ɲ/ and for the dorsovelar consonant /k/ in Catalan /aCa/ sequences produced by three speakers reveal that the tongue body travels a smaller distance at a slower speed and in a longer time during the lowering period extending from the consonant into the following vowel (CV) than during the rising period extending from the preceding vowel into the consonant (VC). For two speakers, two-phase trajectories characterized by two successive velocity peaks occur more frequently during the former period than during the latter, whether associated with tongue blade and dorsum (for alveolopalatals) or with the tongue dorsum articulator alone (for velars). Greater tongue dorsum involvement for /ɲ/ and /k/ than for /ʃ/ accounts for a different kinematic relationship between the four articulatory phases. The lingual gesture for alveolopalatals and, less so, that for velars may exert more prominent spatial and temporal effects on V2 than on V1 which is in agreement with the salience of the C-to-V carryover component associated with these consonants according to previous coarticulation studies. These kinematic and coarticulation data may be attributed to tongue dorsum biomechanics to a large extent.
Collapse
Affiliation(s)
- Daniel Recasens
- Departament de Filologia Catalana, Universitat Autònoma de Barcelona, Bellaterra, Barcelona 08193, Spain.
| | | |
Collapse
|
14
|
Maoz U, Berthoz A, Flash T. Complex Unconstrained Three-Dimensional Hand Movement and Constant Equi-Affine Speed. J Neurophysiol 2009; 101:1002-15. [DOI: 10.1152/jn.90702.2008] [Citation(s) in RCA: 36] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
One long-established simplifying principle behind the large repertoire and high versatility of human hand movements is the two-thirds power law—an empirical law stating a relationship between local geometry and kinematics of human hand trajectories during planar curved movements. It was further generalized not only to various types of human movements, but also to motion perception and prediction, although it was unsuccessful in explaining unconstrained three-dimensional (3D) movements. Recently, movement obeying the power law was proved to be equivalent to moving with constant planar equi-affine speed. Generalizing such motion to 3D space—i.e., to movement at constant spatial equi-affine speed—predicts the emergence of a new power law, whose utility for describing spatial scribbling movements we have previously demonstrated. In this empirical investigation of the new power law, subjects repetitively traced six different 3D geometrical shapes with their hand. We show that the 3D power law explains the data consistently better than both the two-thirds power law and an additional power law that was previously suggested for spatial hand movements. We also found small yet systematic modifications of the power-law's exponents across the various shapes, which further scrutiny suggested to be correlated with global geometric factors of the traced shape. Nevertheless, averaging over all subjects and shapes, the power-law exponents are generally in accordance with constant spatial equi-affine speed. Taken together, our findings provide evidence for the potential role of non-Euclidean geometry in motion planning and control. Moreover, these results seem to imply a relationship between geometry and kinematics that is more complex than the simple local one stipulated by the two-thirds power law and similar models.
Collapse
|