Adaptive control of vowel formant frequency: evidence from real-time formant manipulation

Divided Attention Has Limited Effects on Speech Sensorimotor Control

PURPOSE

When vowel formants are externally perturbed, speakers change their production to oppose that perturbation both during the ongoing production (compensation) and in future productions (adaptation). To date, attempts to explain the large variability across individuals in these responses have focused on trait-based characteristics such as auditory acuity, but evidence from other motor domains suggests that attention may modulate the motor response to sensory perturbations. Here, we test the extent to which divided attention impacts sensorimotor control for supralaryngeal articulation.

METHOD

Neurobiologically healthy speakers were exposed to random (Experiment 1) or consistent (Experiment 2) real-time auditory perturbation of vowel formants to measure online compensation and trial-to-trial adaptation, respectively. In both experiments, participants completed two conditions: one with a simultaneous visual distractor task to divide attention and one without this secondary task.

RESULTS

Divided visual attention slightly reduced online compensation, but only starting > 300 ms after vowel onset, well beyond the typical duration of vowels in speech. Divided attention had no effect on adaptation.

CONCLUSIONS

The results from both experiments suggest that the use of sensory feedback in typical speech motor control is a largely automatic process unaffected by divided visual attention, suggesting that the source of cross-speaker variability in response to formant perturbations likely lies within the speech production system rather than in higher-level cognitive processes. Methodologically, these results suggest that compensation for formant perturbations should be measured prior to 300 ms after vowel onset to avoid any potential impact of attention or other higher-order cognitive factors.

Sensorimotor Adaptation to Formant-Shifted Auditory Feedback Is Predicted by Language-Specific Factors in L1 and L2 Speech Production

Auditory feedback plays an important role in the long-term updating and maintenance of speech motor control; thus, the current study explored the unresolved question of how sensorimotor adaptation is predicted by language-specific and domain-general factors in first-language (L1) and second-language (L2) production. Eighteen English-L1 speakers and 22 English-L2 speakers performed the same sensorimotor adaptation experiments and tasks, which measured language-specific and domain-general abilities. The experiment manipulated the language groups (English-L1 and English-L2) and experimental conditions (baseline, early adaptation, late adaptation, and end). Linear mixed-effects model analyses indicated that auditory acuity was significantly associated with sensorimotor adaptation in L1 and L2 speakers. Analysis of vocal responses showed that L1 speakers exhibited significant sensorimotor adaptation under the early adaptation, late adaptation, and end conditions, whereas L2 speakers exhibited significant sensorimotor adaptation only under the late adaptation condition. Furthermore, the domain-general factors of working memory and executive control were not associated with adaptation/aftereffects in either L1 or L2 production, except for the role of working memory in aftereffects in L2 production. Overall, the study empirically supported the hypothesis that sensorimotor adaptation is predicted by language-specific factors such as auditory acuity and language experience, whereas general cognitive abilities do not play a major role in this process.

Speech sensorimotor relationships in francophone preschoolers and adults: Adaptation to real-time auditory feedback perturbations

PURPOSE

This study investigates the development of sensorimotor relationships by examining adaptation to real-time perturbations of auditory feedback.

METHOD

Acoustic signals were recorded while preschoolers and adult speakers of Canadian French produced several utterances of the front rounded vowel /ø/ for which F2 was gradually shifted up to a maximum of 40%.

RESULTS

The findings indicate that, although preschool-aged children produced overall similar responses to the perturbed feedback, they displayed significantly more trial-to-trial variability than adults. Furthermore, whereas the magnitude of the adaptation in adults was positively correlated with the slope of the perceptual categorical function, the amount of adaptation in children was linked to the variability of their productions in the baseline condition. These patterns suggest that the immature motor control observed in children, which contributes to increased variability in their speech production, plays a role in shaping adaptive behavior, as it allows children to explore articulatory/acoustic spaces and learn sensorimotor relationships.

Increased vowel contrast and intelligibility in connected speech induced by sensorimotor adaptation

Alterations to sensory feedback can drive robust adaptive changes to the production of consonants and vowels, but these changes often have no behavioral relevance or benefit to communication (e.g., making "head" more like "had"). This work aims to align the outcomes of adaptation with changes known to increase speech intelligibility - specifically, adaptations that increase the acoustic contrast between vowels in running speech. To this end, we implemented a vowel centralization feedback perturbation paradigm that pushes all vowels towards the center of vowel space, making them sound less distinct from one another. Speakers across the adult lifespan adapted to the centralization perturbation during sentence production, increasing the global acoustic contrast among vowels and the articulatory excursions for individual vowels. These changes persisted after the perturbation was removed, including after a silent delay, and showed robust transfer to words that were not present in the sentences. Control analyses demonstrated that these effects were unlikely to be due to explicit pronunciation strategies and occurred in the face of increasingly more rapid and less distinct production of familiar sentences. Finally, sentence transcription by crowd-sourced listeners showed that speakers' vowel contrast predicted their baseline intelligibility and that experimentally-induced increases in contrast predicted intelligibility gains. These findings establish the validity of a sensorimotor adaptation paradigm to implicitly increase vowel contrast and intelligibility in connected speech, an outcome that has the potential to enhance rehabilitation in individuals who present with a reduced vowel space due to motor speech disorders, such as the hypokinetic dysarthria associated with Parkinson's disease.

Sensorimotor learning during synchronous speech is modulated by the acoustics of the other voice

This study tested the hypothesis that speaking with other voices can influence sensorimotor predictions of one's own voice. Real-time manipulations of auditory feedback were used to drive sensorimotor adaptation in speech, while participants spoke sentences in synchrony with another voice, a task known to induce implicit imitation (phonetic convergence). The acoustic-phonetic properties of the other voice were manipulated between groups, such that convergence with it would either oppose (incongruent group, n = 15) or align with (congruent group, n = 16) speech motor adaptation. As predicted, significantly greater adaptation was seen in the congruent compared to the incongruent group. This suggests the use of shared sensory targets in speech for predicting the sensory outcomes of both the actions of others (speech perception) and the actions of the self (speech production). This finding has important implications for wider theories of shared predictive mechanisms across perception and action, such as active inference.

The Accompanying Effect in Responses to Auditory Perturbations: Unconscious Vocal Adjustments to Unperturbed Parameters

PURPOSE

The present study examined whether participants respond to unperturbed parameters while experiencing specific perturbations in auditory feedback. For instance, we aim to determine if speakers adjust voice loudness when only pitch is artificially altered in auditory feedback. This phenomenon is referred to as the "accompanying effect" in the present study.

METHOD

Thirty native Mandarin speakers were asked to sustain the vowel /ɛ/ for 3 s while their auditory feedback underwent single shifts in one of the three distinct ways: pitch shift (±100 cents; coded as PT), loudness shift (±6 dB; coded as LD), or first formant (F1) shift (±100 Hz; coded as FM). Participants were instructed to ignore the perturbations in their auditory feedback. Response types were categorized based on pitch, loudness, and F1 for each individual trial, such as Popp_Lopp_Fopp indicating opposing responses in all three domains.

RESULTS

The accompanying effect appeared 93% of the time. Bayesian Poisson regression models indicate that opposing responses in all three domains (Popp_Lopp_Fopp) were the most prevalent response type across the conditions (PT, LD, and FM). The more frequently used response types exhibited opposing responses and significantly larger response curves than the less frequently used response types. Following responses became more prevalent only when the perturbed stimuli were perceived as voices from someone else (external references), particularly in the FM condition. In terms of isotropy, loudness and F1 tended to change in the same direction rather than loudness and pitch.

CONCLUSION

The presence of the accompanying effect suggests that the motor systems responsible for regulating pitch, loudness, and formants are not entirely independent but rather interconnected to some degree.

Perceptual formant discrimination during speech movement planning

Evoked potential studies have shown that speech planning modulates auditory cortical responses. The phenomenon's functional relevance is unknown. We tested whether, during this time window of cortical auditory modulation, there is an effect on speakers' perceptual sensitivity for vowel formant discrimination. Participants made same/different judgments for pairs of stimuli consisting of a pre-recorded, self-produced vowel and a formant-shifted version of the same production. Stimuli were presented prior to a "go" signal for speaking, prior to passive listening, and during silent reading. The formant discrimination stimulus /uh/ was tested with a congruent productions list (words with /uh/) and an incongruent productions list (words without /uh/). Logistic curves were fitted to participants' responses, and the just-noticeable difference (JND) served as a measure of discrimination sensitivity. We found a statistically significant effect of condition (worst discrimination before speaking) without congruency effect. Post-hoc pairwise comparisons revealed that JND was significantly greater before speaking than during silent reading. Thus, formant discrimination sensitivity was reduced during speech planning regardless of the congruence between discrimination stimulus and predicted acoustic consequences of the planned speech movements. This finding may inform ongoing efforts to determine the functional relevance of the previously reported modulation of auditory processing during speech planning.

Using altered auditory feedback to study pitch compensation and adaptation in tonal language speakers

Human speech production is strongly influenced by the auditory feedback it generates. Auditory feedback-what we hear when we speak-enables us to learn and maintain speaking skills and to rapidly correct errors in our speech. Over the last three decades, the real-time altered auditory feedback (AAF) paradigm has gained popularity as a tool to study auditory feedback control during speech production. This method involves changing a speaker's speech and feeding it back to them in near real time. More than 50% of the world's population speak tonal languages, in which the pitch or tone used to pronounce a word can change its meaning. This review article aims to offer an overview of the progression of AAF paradigm as a method to study pitch motor control among speakers of tonal languages. Eighteen studies were included in the current mini review and were compared based on their methodologies and results. Overall, findings from these studies provide evidence that tonal language speakers can compensate and adapt when receiving inconsistent and consistent pitch perturbations. Response magnitude and latency are influenced by a range of factors. Moreover, by combining AAF with brain stimulation and neuroimaging techniques, the neural basis of pitch motor control in tonal language speakers has been investigated. To sum up, AAF has been demonstrated to be an emerging tool for studying pitch motor control in speakers of tonal languages.

On the interplay between speech perception and production: insights from research and theories

The study of spoken communication has long been entrenched in a debate surrounding the interdependence of speech production and perception. This mini review summarizes findings from prior studies to elucidate the reciprocal relationships between speech production and perception. We also discuss key theoretical perspectives relevant to speech perception-production loop, including hyper-articulation and hypo-articulation (H&H) theory, speech motor theory, direct realism theory, articulatory phonology, the Directions into Velocities of Articulators (DIVA) and Gradient Order DIVA (GODIVA) models, and predictive coding. Building on prior findings, we propose a revised auditory-motor integration model of speech and provide insights for future research in speech perception and production, focusing on the effects of impaired peripheral auditory systems.

Test-Retest Reliability of Behavioral Assays of Feedforward and Feedback Auditory-Motor Control of Voice and Articulation

PURPOSE

Behavioral assays of feedforward and feedback auditory-motor control of voice and articulation frequently are used to make inferences about underlying neural mechanisms and to study speech development and disorders. However, no studies have examined the test-retest reliability of such measures, which is critical for rigorous study of auditory-motor control. Thus, the purpose of the present study was to assess the reliability of assays of feedforward and feedback control in voice versus articulation domains.

METHOD

Twenty-eight participants (14 cisgender women, 12 cisgender men, one transgender man, one transmasculine/nonbinary) who denied any history of speech, hearing, or neurological impairment were measured for responses to predictable versus unexpected auditory feedback perturbations of vocal (fundamental frequency, fo) and articulatory (first formant, F1) acoustic parameters twice, with 3-6 weeks between sessions. Reliability was measured with intraclass correlations.

RESULTS

Opposite patterns of reliability were observed for fo and F1; fo reflexive responses showed good reliability and fo adaptive responses showed poor reliability, whereas F1 reflexive responses showed poor reliability and F1 adaptive responses showed moderate reliability. However, a criterion-referenced categorical measurement of fo adaptive responses as typical versus atypical showed substantial test-retest agreement.

CONCLUSIONS

Individual responses to some behavioral assays of auditory-motor control of speech should be interpreted with caution, which has implications for several fields of research. Additional research is needed to establish reliable criterion-referenced measures of F1 adaptive responses as well as fo and F1 reflexive responses. Furthermore, the opposite patterns of test-retest reliability observed for voice versus articulation add to growing evidence for differences in underlying neural control mechanisms.

Do Not Cut Off Your Tail: A Mega-Analysis of Responses to Auditory Perturbation Experiments

PURPOSE

The practice of removing "following" responses from speech perturbation analyses is increasingly common, despite no clear evidence as to whether these responses represent a unique response type. This study aimed to determine if the distribution of responses to auditory perturbation paradigms represents a bimodal distribution, consisting of two distinct response types, or a unimodal distribution.

METHOD

This mega-analysis pooled data from 22 previous studies to examine the distribution and magnitude of responses to auditory perturbations across four tasks: adaptive pitch, adaptive formant, reflexive pitch, and reflexive formant. Data included at least 150 unique participants for each task, with studies comprising younger adult, older adult, and Parkinson's disease populations. A Silverman's unimodality test followed by a smoothed bootstrap resampling technique was performed for each task to evaluate the number of modes in each distribution. Wilcoxon signed-ranks tests were also performed for each distribution to confirm significant compensation in response to the perturbation.

RESULTS

Modality analyses were not significant (p > .05) for any group or task, indicating unimodal distributions. Our analyses also confirmed compensatory reflexive responses to pitch and formant perturbations across all groups, as well as adaptive responses to sustained formant perturbations. However, analyses of sustained pitch perturbations only revealed evidence of adaptation in studies with younger adults.

CONCLUSION

The demonstration of a clear unimodal distribution across all tasks suggests that following responses do not represent a distinct response pattern, but rather the tail of a unimodal distribution.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.24282676.

Perceptual formant discrimination during speech movement planning

Evoked potential studies have shown that speech planning modulates auditory cortical responses. The phenomenon's functional relevance is unknown. We tested whether, during this time window of cortical auditory modulation, there is an effect on speakers' perceptual sensitivity for vowel formant discrimination. Participants made same/different judgments for pairs of stimuli consisting of a pre-recorded, self-produced vowel and a formant-shifted version of the same production. Stimuli were presented prior to a "go" signal for speaking, prior to passive listening, and during silent reading. The formant discrimination stimulus /uh/ was tested with a congruent productions list (words with /uh/) and an incongruent productions list (words without /uh/). Logistic curves were fitted to participants' responses, and the just-noticeable difference (JND) served as a measure of discrimination sensitivity. We found a statistically significant effect of condition (worst discrimination before speaking) without congruency effect. Post-hoc pairwise comparisons revealed that JND was significantly greater before speaking than during silent reading. Thus, formant discrimination sensitivity was reduced during speech planning regardless of the congruence between discrimination stimulus and predicted acoustic consequences of the planned speech movements. This finding may inform ongoing efforts to determine the functional relevance of the previously reported modulation of auditory processing during speech planning.

How to cut the pie is no piece of cake: Toward a process-oriented approach to assessment and diagnosis of speech sound disorders

BACKGROUND

'Speech sound disorder' is an umbrella term that encompasses dysarthria, articulation disorders, childhood apraxia of speech and phonological disorders. However, differential diagnosis between these disorders is a persistent challenge in speech pathology, as many diagnostic procedures use symptom clusters instead of identifying an origin of breakdown in the speech and language system.

AIMS

This article reviews typical and disordered speech through the lens of two well-developed models of production-one focused on phonological encoding and one focused on speech motor planning. We illustrate potential breakdown locations within these models that may relate to childhood apraxia of speech and phonological disorders.

MAIN CONTRIBUTION

This paper presents an overview of an approach to conceptualisation of speech sound disorders that is grounded in current models of speech production and emphasises consideration of underlying processes. The paper also sketches a research agenda for the development of valid, reliable and clinically feasible assessment protocols for children with speech sound disorders.

CONCLUSION

The process-oriented approach outlined here is in the early stages of development but holds promise for developing a more detailed and comprehensive understanding of, and assessment protocols for speech sound disorders that go beyond broad diagnostic labels based on error analysis. Directions for future research are discussed.

WHAT THIS PAPER ADDS

What is already known on the subject Speech sound disorders (SSD) are heterogeneous, and there is agreement that some children have a phonological impairment (phonological disorders, PD) whereas others have an impairment of speech motor planning (childhood apraxia of speech, CAS). There is also recognition that speech production involves multiple processes, and several approaches to the assessment and diagnosis of SSD have been proposed. What this paper adds to existing knowledge This paper provides a more detailed conceptualisation of potential impairments in children with SSD that is grounded in current models of speech production and encourages greater consideration of underlying processes. The paper illustrates this approach and provides guidance for further development. One consequence of this perspective is the notion that broad diagnostic category labels (PD, CAS) may each comprise different subtypes or profiles depending on the processes that are affected. What are the potential or actual clinical implications of this work? Although the approach is in the early stages of development and no comprehensive validated set of tasks and measures is available to assess all processes, clinicians may find the conceptualisation of different underlying processes and the notion of potential subtypes within PD and CAS informative when evaluating SSD. In addition, this perspective discourages either/or thinking (PD or CAS) and instead encourages consideration of the possibility that children may have different combinations of impairments at different processing stages.

Mechanisms of sensorimotor adaptation in a hierarchical state feedback control model of speech

Upon perceiving sensory errors during movements, the human sensorimotor system updates future movements to compensate for the errors, a phenomenon called sensorimotor adaptation. One component of this adaptation is thought to be driven by sensory prediction errors-discrepancies between predicted and actual sensory feedback. However, the mechanisms by which prediction errors drive adaptation remain unclear. Here, auditory prediction error-based mechanisms involved in speech auditory-motor adaptation were examined via the feedback aware control of tasks in speech (FACTS) model. Consistent with theoretical perspectives in both non-speech and speech motor control, the hierarchical architecture of FACTS relies on both the higher-level task (vocal tract constrictions) as well as lower-level articulatory state representations. Importantly, FACTS also computes sensory prediction errors as a part of its state feedback control mechanism, a well-established framework in the field of motor control. We explored potential adaptation mechanisms and found that adaptive behavior was present only when prediction errors updated the articulatory-to-task state transformation. In contrast, designs in which prediction errors updated forward sensory prediction models alone did not generate adaptation. Thus, FACTS demonstrated that 1) prediction errors can drive adaptation through task-level updates, and 2) adaptation is likely driven by updates to task-level control rather than (only) to forward predictive models. Additionally, simulating adaptation with FACTS generated a number of important hypotheses regarding previously reported phenomena such as identifying the source(s) of incomplete adaptation and driving factor(s) for changes in the second formant frequency during adaptation to the first formant perturbation. The proposed model design paves the way for a hierarchical state feedback control framework to be examined in the context of sensorimotor adaptation in both speech and non-speech effector systems.

Speaking in the presence of noise: Consistency of acoustic properties in clear-Lombard speech over time

Individual speakers are often able to modify their speech to facilitate communication in challenging conditions, such as speaking in a noisy environment. Such vocal "enrichments" might include reductions in speech rate or increases in acoustic contrasts. However, it is unclear how consistently speakers enrich their speech over time. This study examined inter-speaker variability in the speech enrichment modifications applied by speakers. The study compared a baseline habitual speaking style to a clear-Lombard style and measured changes in acoustic differences between the two styles over sentence trials. Seventy-eight young adult participants read out sentences in the habitual and clear-Lombard speaking styles. Acoustic differences between speaking styles generally increased nonlinearly over trials, suggesting that speakers require practice before realizing their full speech enrichment potential when speaking clearly in noise with reduced auditory feedback. Using a recent objective intelligibility metric based on glimpses, the study also found that predicted intelligibility increased over trials, highlighting that communicative benefits of the clear-Lombard style are not static. These findings underline the dynamic nature of speaking styles.

Identifying distinct latent classes of pitch-shift response consistency: Evidence from manipulating the predictability of shift direction

Auditory feedback plays an important role in regulating our vocal pitch. When pitch shifts suddenly appear in auditory feedback, the majority of the responses are opposing, correcting for the mismatch between perceived pitch and actual pitch. However, research has indicated that following responses to auditory perturbation could be common. This study attempts to explore the ways individual speakers would respond to pitch perturbation (using an opposing response or a following response) from trial to trial. Thirty-six native speakers of Mandarin produced the vowel /a/ while receiving perturbed pitch at a random time (500 ~ 700 ms) after vocal onset for a duration of 200 ms. Three blocks of 30 trials that differed in the pitch-shift stimulus direction were recorded in a randomized order: (a) the down-only condition where pitch was shifted downwards 250 cents; (b) the up-only condition where pitch was shifted upwards 250 cents; and (c) the random condition where downshifts and upshifts occurred randomly and were equally likely. The participants were instructed to ignore the pitch shifts. Results from the latent class analysis show that at the individual level across trials, 57% of participants were switchers, 28% were opposers, and 15% were followers. Our results support that speakers produce a mix of opposing and following responses when they respond to perturbed pitch. Specifically, the proportion of followers was conditional on the expectancy of pitch-shift stimulus direction: More followers were observed when the pitch-shift stimulus direction was predictable. Closer inspection of the levels of response consistency in different time phases shows that a particular mechanism (opposing or following) was initially implemented; the two mechanisms may alternate in the middle phase; and then finally, the pitch-shift response was featured as a particular mechanism near the end phase.

Movement variability can be modulated in speech production

Although movement variability is often attributed to unwanted noise in the motor system, recent work has demonstrated that variability may be actively controlled. To date, research on regulation of motor variability has relied on relatively simple, laboratory-specific reaching tasks. It is not clear how these results translate to complex, well-practiced tasks. Here, we test how variability is regulated during speech production, a complex, highly overpracticed, and natural motor behavior that relies on auditory and somatosensory feedback. Specifically, in a series of four experiments, we assessed the effects of auditory feedback manipulations that modulate perceived speech variability, shifting every production either toward (inward pushing) or away from (outward pushing) the center of the distribution for each vowel. Participants exposed to the inward-pushing perturbation (experiment 1) increased produced variability while the perturbation was applied as well as after it was removed. Unexpectedly, the outward-pushing perturbation (experiment 2) also increased produced variability during exposure, but variability returned to near-baseline levels when the perturbation was removed. Outward-pushing perturbations failed to reduce participants' produced variability both with larger perturbation magnitude (experiment 3) and after their variability had increased above baseline levels as a result of the inward-pushing perturbation (experiment 4). Simulations of the applied perturbations using a state-space model of motor behavior suggest that the increases in produced variability in response to the two types of perturbations may arise through distinct mechanisms. Together, these results suggest that motor variability is actively monitored and can be modulated even in complex and well-practiced behaviors such as speech.NEW & NOTEWORTHY By implementing a novel auditory feedback perturbation that modulates participants' perceived trial-to-trial variability without affecting their overall mean behavior, we show that variability in the speech motor system can be modulated. By assaying speech production, we expand our current understanding of variability to a well-practiced, complex behavior outside of the limb control system. Our results additionally highlight the need to incorporate the active control of variability in models of speech motor control.

Motor constellation theory: A model of infants' phonological development

Every normally developing human infant solves the difficult problem of mapping their native-language phonology, but the neural mechanisms underpinning this behavior remain poorly understood. Here, motor constellation theory, an integrative neurophonological model, is presented, with the goal of explicating this issue. It is assumed that infants' motor-auditory phonological mapping takes place through infants' orosensory "reaching" for phonological elements observed in the language-specific ambient phonology, via reference to kinesthetic feedback from motor systems (e.g., articulators), and auditory feedback from resulting speech and speech-like sounds. Attempts are regulated by basal ganglion-cerebellar speech neural circuitry, and successful attempts at reproduction are enforced through dopaminergic signaling. Early in life, the pace of anatomical development constrains mapping such that complete language-specific phonological mapping is prohibited by infants' undeveloped supralaryngeal vocal tract and undescended larynx; constraints gradually dissolve with age, enabling adult phonology. Where appropriate, reference is made to findings from animal and clinical models. Some implications for future modeling and simulation efforts, as well as clinical settings, are also discussed.

Individual sensorimotor adaptation characteristics are independent across orofacial speech movements and limb reaching movements

Sensorimotor adaptation is critical for human motor control but shows considerable interindividual variability. Efforts are underway to identify factors accounting for individual differences in specific adaptation tasks. However, a fundamental question has remained unaddressed: Is an individual's capability for adaptation effector system specific or does it reflect a generalized adaptation ability? We therefore tested the same participants in analogous adaptation paradigms focusing on distinct sensorimotor systems: speaking with perturbed auditory feedback and reaching with perturbed visual feedback. Each task was completed once with the perturbation introduced gradually (ramped up over 60 trials) and, on a different day, once with the perturbation introduced suddenly. Consistent with studies of each system separately, visuomotor reach adaptation was more complete than auditory-motor speech adaptation (80% vs. 29% of the perturbation). Adaptation was not significantly correlated between the speech and reach tasks. Moreover, considered within tasks, 1) adaptation extent was correlated between the gradual and sudden conditions for reaching but not for speaking, 2) adaptation extent was correlated with additional measures of performance (e.g., trial duration, within-trial corrections) only for reaching and not for speaking, and 3) fitting individual participant adaptation profiles with exponential rather than linear functions offered a larger benefit [lower root mean square error (RMSE)] for the reach task than for the speech task. Combined, results suggest that the ability for sensorimotor adaptation relies on neural plasticity mechanisms that are effector system specific rather than generalized. This finding has important implications for ongoing efforts seeking to identify cognitive, behavioral, and neurochemical predictors of individual sensorimotor adaptation.NEW & NOTEWORTHY This study provides the first detailed demonstration that individual sensorimotor adaptation characteristics are independent across articulatory speech movements and limb reaching movements. Thus, individual sensorimotor learning abilities are effector system specific rather than generalized. Findings regarding one effector system do not necessarily apply to other systems, different underlying mechanisms may be involved, and implications for clinical rehabilitation or performance training also cannot be generalized.

Perturbing the consistency of auditory feedback in speech

Sensory information, including auditory feedback, is used by talkers to maintain fluent speech articulation. Current models of speech motor control posit that speakers continually adjust their motor commands based on discrepancies between the sensory predictions made by a forward model and the sensory consequences of their speech movements. Here, in two within-subject design experiments, we used a real-time formant manipulation system to explore how reliant speech articulation is on the accuracy or predictability of auditory feedback information. This involved introducing random formant perturbations during vowel production that varied systematically in their spatial location in formant space (Experiment 1) and temporal consistency (Experiment 2). Our results indicate that, on average, speakers’ responses to auditory feedback manipulations varied based on the relevance and degree of the error that was introduced in the various feedback conditions. In Experiment 1, speakers’ average production was not reliably influenced by random perturbations that were introduced every utterance to the first (F1) and second (F2) formants in various locations of formant space that had an overall average of 0 Hz. However, when perturbations were applied that had a mean of +100 Hz in F1 and −125 Hz in F2, speakers demonstrated reliable compensatory responses that reflected the average magnitude of the applied perturbations. In Experiment 2, speakers did not significantly compensate for perturbations of varying magnitudes that were held constant for one and three trials at a time. Speakers’ average productions did, however, significantly deviate from a control condition when perturbations were held constant for six trials. Within the context of these conditions, our findings provide evidence that the control of speech movements is, at least in part, dependent upon the reliability and stability of the sensory information that it receives over time.

A single exposure to altered auditory feedback causes observable sensorimotor adaptation in speech

Sensory errors induce two types of behavioral changes: rapid compensation within a movement and longer-term adaptation of subsequent movements. Although adaptation is hypothesized to occur whenever a sensory error is perceived (including after a single exposure to altered feedback), adaptation of articulatory movements in speech has only been observed after repeated exposure to auditory perturbations, questioning both current theories of speech sensorimotor adaptation and the universality of more general theories of adaptation. We measured single-exposure or ‘one-shot’ learning in a large dataset in which participants were exposed to intermittent, unpredictable perturbations of their speech acoustics. On unperturbed trials immediately following these perturbed trials, participants adjusted their speech to oppose the preceding shift, demonstrating that learning occurs even after a single exposure to auditory error. These results provide critical support for current theories of sensorimotor adaptation in speech and align speech more closely with learning in other motor domains.

Inter-Trial Formant Variability in Speech Production Is Actively Controlled but Does Not Affect Subsequent Adaptation to a Predictable Formant Perturbation

Despite ample evidence that speech production is associated with extensive trial-to-trial variability, it remains unclear whether this variability represents merely unwanted system noise or an actively regulated mechanism that is fundamental for maintaining and adapting accurate speech movements. Recent work on upper limb movements suggest that inter-trial variability may be not only actively regulated based on sensory feedback, but also provide a type of workspace exploration that facilitates sensorimotor learning. We therefore investigated whether experimentally reducing or magnifying inter-trial formant variability in the real-time auditory feedback during speech production (a) leads to adjustments in formant production variability that compensate for the manipulation, (b) changes the temporal structure of formant adjustments across productions, and (c) enhances learning in a subsequent adaptation task in which a predictable formant-shift perturbation is applied to the feedback signal. Results show that subjects gradually increased formant variability in their productions when hearing auditory feedback with reduced variability, but subsequent formant-shift adaptation was not affected by either reducing or magnifying the perceived variability. Thus, findings provide evidence for speakers’ active control of inter-trial formant variability based on auditory feedback from previous trials, but–at least for the current short-term experimental manipulation of feedback variability–not for a role of this variability regulation mechanism in subsequent auditory-motor learning.

Auditory and somatosensory feedback mechanisms of laryngeal and articulatory speech motor control

PURPOSE

Speech production is a complex motor task involving multiple subsystems. The relationships between these subsystems need to be comprehensively investigated to understand the underlying mechanisms of speech production. The goal of this paper is to examine the differential contributions of 1) auditory and somatosensory feedback control mechanisms, and 2) laryngeal and articulatory speech production subsystems on speech motor control at an individual speaker level using altered auditory and somatosensory feedback paradigms.

METHODS

Twenty young adults completed speaking tasks in which sudden and unpredictable auditory and physical perturbations were applied to the laryngeal and articulatory speech production subsystems. Auditory perturbations were applied to laryngeal or articulatory acoustic features of speech. Physical perturbations were applied to the larynx and the jaw. Pearson-product moment correlation coefficients were calculated between 1) auditory and somatosensory reflexive responses to investigate relationships between auditory and somatosensory feedback control mechanisms, and 2) laryngeal and articulatory reflexive responses as well as acuity measures to investigate the relationship between auditory-motor features of laryngeal and articulatory subsystems.

RESULTS

No statistically significant correlations were found concerning the relationships between auditory and somatosensory feedback. No statistically significant correlations were found between auditory-motor features in the laryngeal and articulatory control subsystems.

CONCLUSION

Results suggest that the laryngeal and articulatory speech production subsystems operate with differential auditory and somatosensory feedback control mechanisms. The outcomes suggest that current models of speech motor control should consider decoupling laryngeal and articulatory domains to better model speech motor control processes.

Pediatric Responses to Fundamental and Formant Frequency Altered Auditory Feedback: A Scoping Review

Purpose

The ability to hear ourselves speak has been shown to play an important role in the development and maintenance of fluent and coherent speech. Despite this, little is known about the developing speech motor control system throughout childhood, in particular if and how vocal and articulatory control may differ throughout development. A scoping review was undertaken to identify and describe the full range of studies investigating responses to frequency altered auditory feedback in pediatric populations and their contributions to our understanding of the development of auditory feedback control and sensorimotor learning in childhood and adolescence.

Method

Relevant studies were identified through a comprehensive search strategy of six academic databases for studies that included (a) real-time perturbation of frequency in auditory input, (b) an analysis of immediate effects on speech, and (c) participants aged 18 years or younger.

Results

Twenty-three articles met inclusion criteria. Across studies, there was a wide variety of designs, outcomes and measures used. Manipulations included fundamental frequency (9 studies), formant frequency (12), frequency centroid of fricatives (1), and both fundamental and formant frequencies (1). Study designs included contrasts across childhood, between children and adults, and between typical, pediatric clinical and adult populations. Measures primarily explored acoustic properties of speech responses (latency, magnitude, and variability). Some studies additionally examined the association of these acoustic responses with clinical measures (e.g., stuttering severity and reading ability), and neural measures using electrophysiology and magnetic resonance imaging.

Conclusion

Findings indicated that children above 4 years generally compensated in the opposite direction of the manipulation, however, in several cases not as effectively as adults. Overall, results varied greatly due to the broad range of manipulations and designs used, making generalization challenging. Differences found between age groups in the features of the compensatory vocal responses, latency of responses, vocal variability and perceptual abilities, suggest that maturational changes may be occurring in the speech motor control system, affecting the extent to which auditory feedback is used to modify internal sensorimotor representations. Varied findings suggest vocal control develops prior to articulatory control. Future studies with multiple outcome measures, manipulations, and more expansive age ranges are needed to elucidate findings.

Repetitive Exposure to Orofacial Somatosensory Inputs in Speech Perceptual Training Modulates Vowel Categorization in Speech Perception

Orofacial somatosensory inputs may play a role in the link between speech perception and production. Given the fact that speech motor learning, which involves paired auditory and somatosensory inputs, results in changes to speech perceptual representations, somatosensory inputs may also be involved in learning or adaptive processes of speech perception. Here we show that repetitive pairing of somatosensory inputs and sounds, such as occurs during speech production and motor learning, can also induce a change of speech perception. We examined whether the category boundary between /ε/ and /a/ was changed as a result of perceptual training with orofacial somatosensory inputs. The experiment consisted of three phases: Baseline, Training, and Aftereffect. In all phases, a vowel identification test was used to identify the perceptual boundary between /ε/ and /a/. In the Baseline and the Aftereffect phase, an adaptive method based on the maximum-likelihood procedure was applied to detect the category boundary using a small number of trials. In the Training phase, we used the method of constant stimuli in order to expose participants to stimulus variants which covered the range between /ε/ and /a/ evenly. In this phase, to mimic the sensory input that accompanies speech production and learning in an experimental group, somatosensory stimulation was applied in the upward direction when the stimulus sound was presented. A control group (CTL) followed the same training procedure in the absence of somatosensory stimulation. When we compared category boundaries prior to and following paired auditory-somatosensory training, the boundary for participants in the experimental group reliably changed in the direction of /ε/, indicating that the participants perceived /a/ more than /ε/ as a consequence of training. In contrast, the CTL did not show any change. Although a limited number of participants were tested, the perceptual shift was reduced and almost eliminated 1 week later. Our data suggest that repetitive exposure of somatosensory inputs in a task that simulates the sensory pairing which occurs during speech production, changes perceptual system and supports the idea that somatosensory inputs play a role in speech perceptual adaptation, probably contributing to the formation of sound representations for speech perception.

Drifting pitch awareness after exposure to altered auditory feedback

Various studies have claimed that the sense of agency is based on a comparison between an internal estimate of an action's outcome and sensory feedback. With respect to speech, this presumes that speakers have a stable prearticulatory representation of their own speech. However, recent research suggests that the sense of agency is flexible and thus in some contexts we may feel like we produced speech that was not actually produced by us. The current study tested whether the estimated pitch of one's articulation (termed pitch awareness) is affected by manipulated auditory feedback. In four experiments, 56 participants produced isolated vowels while being exposed to pitch-shifted auditory feedback. After every vocalization, participants indicated whether they thought the feedback was higher or lower than their actual production. After exposure to a block of high-pitched auditory feedback (+500 cents pitch shift), participants were more likely to label subsequent auditory feedback as "lower than my actual production," suggesting that prolonged exposure to high-pitched auditory feedback led to a drift in participants' pitch awareness. The opposite pattern was found after exposure to a constant -500 cents pitch shift. This suggests that pitch awareness is not solely based on a prearticulatory representation of intended speech or on a sensory prediction, but also on sensory feedback. We propose that this drift in pitch awareness could be indicative of a sense of agency over the pitch-shifted auditory feedback in the exposure block. If so, this suggests that the sense of agency over vocal output is flexible.

Disruption of speech motor adaptation with repetitive transcranial magnetic stimulation of the articulatory representation in primary motor cortex

When auditory feedback perturbation is introduced in a predictable way over a number of utterances, speakers learn to compensate by adjusting their own productions, a process known as sensorimotor adaptation. Despite multiple lines of evidence indicating the role of primary motor cortex (M1) in motor learning and memory, whether M1 causally contributes to sensorimotor adaptation in the speech domain remains unclear. Here, we aimed to assay whether temporary disruption of the articulatory representation in left M1 by repetitive transcranial magnetic stimulation (rTMS) impairs speech adaptation. To induce sensorimotor adaptation, the frequencies of first formants (F1) were shifted up and played back to participants when they produced “head”, “bed”, and “dead” repeatedly (the learning phase). A low-frequency rTMS train (.6 Hz, subthreshold, 12 min) over either the tongue or the hand representation of M1 (between-subjects design) was applied before participants experienced altered auditory feedback in the learning phase. We found that the group who received rTMS over the hand representation showed the expected compensatory response for the upwards shift in F1 by significantly reducing F1 and increasing the second formant (F2) frequencies in their productions. In contrast, these expected compensatory changes in both F1 and F2 did not occur in the group that received rTMS over the tongue representation. Critically, rTMS (subthreshold) over the tongue representation did not affect vowel production, which was unchanged from baseline. These results provide direct evidence that the articulatory representation in left M1 causally contributes to sensorimotor learning in speech. Furthermore, these results also suggest that M1 is critical to the network supporting a more global adaptation that aims to move the altered speech production closer to a learnt pattern of speech production used to produce another vowel.

Auditory Feedback Is Used for Adaptation and Compensation in Speech Timing

Purpose Real-time altered feedback has demonstrated a key role for auditory feedback in both online feedback control and in updating feedforward control for future utterances. The aim of this study was to examine adaptation in response to temporal perturbation using real-time perturbation of ongoing speech. Method Twenty native English speakers with no reported history of speech or hearing disorders participated in this study. The study consisted of four word blocks, using the phrases "a capper," "a gapper," "a sapper," and "a zapper" (due to issues with the implementation of perturbation, "gapper" was excluded from analysis). In each block, participants completed a baseline phase (30 trials of veridical feedback), a ramp phase (feedback perturbation increasing to maximum over 30 trials), a hold phase (60 trials with perturbation held at maximum), and a washout phase (30 trials, feedback abruptly returned to veridical feedback). Word-initial consonant targets (voice onset time for /k, g/ and fricative duration for /s, z/) were lengthened, and the following stressed vowel (/æ/) was shortened. Results Overall, speakers did not adapt the production of their consonants but did lengthen their vowel production in response to shortening. Vowel lengthening showed continued aftereffects during the early portion of the washout phase. Although speakers did not adapt absolute consonant durations, consonant duration was reduced as a proportion of the total syllable duration. This is consistent with previous research that suggests that speakers attend to proportional durations rather than absolute durations. Conclusion These results indicate that speakers actively monitor proportional durations and update the temporal dynamics of planning units extending beyond a single segment.

Articulatory compensation for low-pass filtered formant-altered auditory feedback

Auditory feedback while speaking plays an important role in stably controlling speech articulation. Its importance has been verified in formant-altered auditory feedback (AAF) experiments where speakers utter while listening to speech with perturbed first (F1) and second (F2) formant frequencies. However, the contribution of the frequency components higher than F2 to the articulatory control under the perturbations of F1 and F2 has not yet been investigated. In this study, a formant-AAF experiment was conducted in which a low-pass filter was applied to speech. The experimental results showed that the deviation in the compensatory response was significantly larger when a low-pass filter with a cutoff frequency of 3 kHz was used compared to that when cutoff frequencies of 4 and 8 kHz were used. It was also found that the deviation in the 3-kHz condition correlated with the fundamental frequency and spectral tilt of the produced speech. Additional simulation results using a neurocomputational model of speech production (SimpleDIVA model) and the experimental data showed that the feedforward learning rate increased as the cutoff frequency decreased. These results suggest that high-frequency components of the auditory feedback would be involved in the determination of corrective motor commands from auditory errors.

Intact Correction for Self-Produced Vowel Formant Variability in Individuals With Cerebellar Ataxia Regardless of Auditory Feedback Availability

Purpose Individuals with cerebellar ataxia (CA) caused by cerebellar degeneration exhibit larger reactive compensatory responses to unexpected auditory feedback perturbations than neurobiologically typical speakers, suggesting they may rely more on feedback control during speech. We test this hypothesis by examining variability in unaltered speech. Previous studies of typical speakers have demonstrated a reduction in formant variability (centering) observed during the initial phase of vowel production from vowel onset to vowel midpoint. Centering is hypothesized to reflect feedback-based corrections for self-produced variability and thus may provide a behavioral assay of feedback control in unperturbed speech in the same manner as the compensatory response does for feedback perturbations. Method To comprehensively compare centering in individuals with CA and controls, we examine centering in two vowels (/i/ and /ɛ/) under two contexts (isolated words and connected speech). As a control, we examine speech produced both with and without noise to mask auditory feedback. Results Individuals with CA do not show increased centering compared to age-matched controls, regardless of vowel, context, or masking. Contrary to previous results in neurobiologically typical speakers, centering was not affected by the presence of masking noise in either group. Conclusions The similar magnitude of centering seen with and without masking noise questions whether centering is driven by auditory feedback. However, if centering is at least partially driven by auditory/somatosensory feedback, these results indicate that the larger compensatory response to altered auditory feedback observed in individuals with CA may not reflect typical motor control processes during normal, unaltered speech production.

Speech Intensity Response to Altered Intensity Feedback in Individuals With Parkinson's Disease

Purpose Hypophonia (low speech intensity) is the most common speech symptom experienced by individuals with Parkinson's disease (IWPD). Previous research suggests that, in IWPD, there may be abnormal integration of sensory information for motor production of speech intensity. In the current study, intensity of auditory feedback was systematically manipulated (altered in both positive and negative directions) during sensorimotor conditions that are known to modulate speech intensity in everyday contexts in order to better understand the role of auditory feedback for speech intensity regulation. Method Twenty-six IWPD and 24 neurologically healthy controls were asked to complete the following tasks: converse with the experimenter, start vowel production, and read sentences at a comfortable loudness, while hearing their own speech intensity randomly altered. Altered intensity feedback conditions included 5-, 10-, and 15-dB reductions and increases in the feedback intensity. Speech tasks were completed in no noise and in background noise. Results IWPD displayed a reduced response to the altered intensity feedback compared to control participants. This reduced response was most apparent when participants were speaking in background noise. Specific task-based differences in responses were observed such that the reduced response by IWPD was most pronounced during the conversation task. Conclusions The current study suggests that IWPD have abnormal processing of auditory information for speech intensity regulation, and this disruption particularly impacts their ability to regulate speech intensity in the context of speech tasks with clear communicative goals (i.e., conversational speech) and speaking in background noise.

Compensation to Altered Auditory Feedback in Children With Developmental Language Disorder and Typical Development

Purpose Developmental language disorder (DLD), an unexplained problem using and understanding spoken language, has been hypothesized to have an underlying auditory processing component. Auditory feedback plays a key role in speech motor control. The current study examined whether auditory feedback is used to regulate speech production in a similar way by children with DLD and their typically developing (TD) peers. Method Participants aged 6-11 years completed tasks measuring hearing, language, first formant (F1) discrimination thresholds, partial vowel space, and responses to altered auditory feedback with F1 perturbation. Results Children with DLD tended to compensate more than TD children for the positive F1 manipulation and compensated less than TD children in the negative shift condition. Conclusion Our findings suggest that children with DLD make atypical use of auditory feedback.

The Role of Sensory Feedback in Developmental Stuttering: A Review

Developmental stuttering is a neurodevelopmental disorder that severely affects speech fluency. Multiple lines of evidence point to a role of sensory feedback in the disorder; this has led to a number of theories proposing different disruptions to the use of sensory feedback during speech motor control in people who stutter. The purpose of this review was to bring together evidence from studies using altered auditory feedback paradigms with people who stutter, in order to evaluate the predictions of these different theories. This review highlights converging evidence for particular patterns of differences in the responses of people who stutter to feedback perturbations. The implications for hypotheses on the nature of the disruption to sensorimotor control of speech in the disorder are discussed, with reference to neurocomputational models of speech control (predominantly, the DIVA model; Guenther et al., 2006; Tourville et al., 2008). While some consistent patterns are emerging from this evidence, it is clear that more work in this area is needed with developmental samples in particular, in order to tease apart differences related to symptom onset from those related to compensatory strategies that develop with experience of stuttering.

Online Control of Voice Intensity in Late Bilinguals' First and Second Language Speech Production: Evidence From Unexpected and Brief Noise Masking

Purpose Speech production requires the combined efforts of feedforward control and feedback control subsystems. The primary purpose of this study is to explore whether the relative weighting of auditory feedback control is different between the first language (L1) and the second language (L2) production for late bilinguals. The authors also make an exploratory investigation into how bilinguals' speech fluency and speech perception relate to their auditory feedback control. Method Twenty Chinese-English bilinguals named Chinese or English bisyllabic words, while being exposed to 30- or 60-dB unexpected brief masking noise. Variables of language (L1 or L2) and noise condition (quiet, weak noise, or strong noise) were manipulated in the experiment. L1 and L2 speech fluency tests and an L2 perception test were also included to measure bilinguals' speech fluency and auditory acuity. Results Peak intensity analyses indicated that the intensity increases in the weak noise and strong noise conditions were larger in L2-English than L1-Chinese production. Intensity contour analysis showed that the intensity increases in both languages had an onset around 80-140 ms, a peak around 220-250 ms, and persisted till 400 ms post vocalization onset. Correlation analyses also revealed that poorer speech fluency or L2 auditory acuity was associated with larger Lombard effect. Conclusions For late bilinguals, the reliance on auditory feedback control is heavier in L2 than in L1 production. We empirically supported a relation between speech fluency and the relative weighting of auditory feedback control, and provided the first evidence for the production-perception link in L2 speech motor control.

Are Vowels Normalized After Maxillary Osteotomy? An Acoustic Study in Cleft Lip and Palate

BACKGROUND

Maxillary hypoplasia is a common skeletal condition in cleft lip and palate (CLP). Maxillary osteotomy is typically used to reposition the maxilla in CLP with maxillary hypoplasia. Previous studies have suggested that vowel articulations are adjusted postsurgically due to altered vocal tract configuration and articulatory reorganization. This acoustic study aims to investigate whether vowels are normalized postoperatively and to explore the nature of articulatory reorganization.

METHODS AND PROCEDURES

A prospective study was conducted to examine the vowel production of a group of individuals with CLP (N = 17) undergoing maxillary osteotomy and a group of normal controls (N = 20), using speech acoustic data. The data were collected at 0 to 3 months presurgery (T1), 3-months (T2), and 12-months (T3) postsurgery. General linear model repeated measures and independent t-tests were undertaken on F1, F2, and vowel space area.

RESULTS

General linear model repeated measures revealed no main effects of time for F1 (F [2, 22] = 1.094, P = 0.352), F2 (F [2, 22] = 1.269, P = 0.301), and vowel space area (F [2, 28] = 0.059, P = 0.943). Independent t-tests showed statistically significant differences (P < 0.05) for all acoustic parameters and all vowels between the CLP and the normal groups at all time points.

CONCLUSIONS

Vowels were not normalized after maxillary osteotomy despite positive anatomical changes within the oral cavity. Individuals with CLP tended to adjust their vowel articulatory gestures to match presurgical patterns. The nature of articulatory reorganization appears to be prompt, sensory-driven, complete, and permanent.

Increased speech contrast induced by sensorimotor adaptation to a nonuniform auditory perturbation

When auditory feedback is perturbed in a consistent way, speakers learn to adjust their speech to compensate, a process known as sensorimotor adaptation. Although this paradigm has been highly informative for our understanding of the role of sensory feedback in speech motor control, its ability to induce behaviorally relevant changes in speech that affect communication effectiveness remains unclear. Because reduced vowel contrast contributes to intelligibility deficits in many neurogenic speech disorders, we examine human speakers' ability to adapt to a nonuniform perturbation field that was designed to affect vowel distinctiveness, applying a shift that depended on the vowel being produced. Twenty-five participants were exposed to this "vowel centralization" feedback perturbation in which the first two formant frequencies were shifted toward the center of each participant's vowel space, making vowels less distinct from one another. Speakers adapted to this nonuniform shift, learning to produce corner vowels with increased vowel space area and vowel contrast to partially overcome the perceived centralization. The increase in vowel contrast occurred without a concomitant increase in duration and persisted after the feedback shift was removed, including after a 10-min silent period. These findings establish the validity of a sensorimotor adaptation paradigm to increase vowel contrast, showing that complex, nonuniform alterations to sensory feedback can successfully drive changes relevant to intelligible communication.NEW & NOTEWORTHY To date, the speech motor learning evoked in sensorimotor adaptation studies has had little ecological consequences for communication. By inducing complex, nonuniform acoustic errors, we show that adaptation can be leveraged to cause an increase in speech sound contrast, a change that has the capacity to improve intelligibility. This study is relevant for models of sensorimotor integration across motor domains, showing that complex alterations to sensory feedback can successfully drive changes relevant to ecological behavior.

Speech compensation responses and sensorimotor adaptation to formant feedback perturbations

Control of speech formants is important for the production of distinguishable speech sounds and is achieved with both feedback and learned feedforward control. However, it is unclear whether the learning of feedforward control involves the mechanisms of feedback control. Speakers have been shown to compensate for unpredictable transient mid-utterance perturbations of pitch and loudness feedback, demonstrating online feedback control of these speech features. To determine whether similar feedback control mechanisms exist in the production of formants, responses to unpredictable vowel formant feedback perturbations were examined. Results showed similar within-trial compensatory responses to formant perturbations that were presented at utterance onset and mid-utterance. The relationship between online feedback compensation to unpredictable formant perturbations and sensorimotor adaptation to consistent formant perturbations was further examined. Within-trial online compensation responses were not correlated with across-trial sensorimotor adaptation. A detailed analysis of within-trial time course dynamics across trials during sensorimotor adaptation revealed that across-trial sensorimotor adaptation responses did not result from an incorporation of within-trial compensation response. These findings suggest that online feedback compensation and sensorimotor adaptation are governed by distinct neural mechanisms. These findings have important implications for models of speech motor control in terms of how feedback and feedforward control mechanisms are implemented.

The Influence of Vowel Identity, Vowel Production Variability, and Consonant Environment on Envelope Following Responses

OBJECTIVES

The vowel-evoked envelope following response (EFR) is a useful tool for studying brainstem processing of speech in natural consonant-vowel productions. Previous work, however, demonstrates that the amplitude of EFRs is highly variable across vowels. To clarify factors contributing to the variability observed, the objectives of the present study were to evaluate: (1) the influence of vowel identity and the consonant context surrounding each vowel on EFR amplitude and (2) the effect of variations in repeated productions of a vowel on EFR amplitude while controlling for the consonant context.

DESIGN

In Experiment 1, EFRs were recorded in response to seven English vowels (/ij/, /Ι/, /ej/, /ε/, /æ/, /u/, and /JOURNAL/earher/04.03/00003446-202105000-00017/inline-graphic1/v/2021-04-30T105427Z/r/image-tiff/) embedded in each of four consonant contexts (/hVd/, /sVt/, /zVf/, and /JOURNAL/earher/04.03/00003446-202105000-00017/inline-graphic2/v/2021-04-30T105427Z/r/image-tiffVv/). In Experiment 2, EFRs were recorded in response to four different variants of one of the four possible vowels (/ij/, /ε/, /æ/, or /JOURNAL/earher/04.03/00003446-202105000-00017/inline-graphic3/v/2021-04-30T105427Z/r/image-tiff/), embedded in the same consonant-vowel-consonant environments used in Experiment 1. All vowels were edited to minimize formant transitions before embedding in a consonant context. Different talkers were used for the two experiments. Data from a total of 30 and 64 (16 listeners/vowel) young adults with normal hearing were included in Experiments 1 and 2, respectively. EFRs were recorded using a single-channel electrode montage between the vertex and nape of the neck while stimuli were presented monaurally.

RESULTS

In Experiment 1, vowel identity had a significant effect on EFR amplitude with the vowel /æ/ eliciting the highest amplitude EFRs (170 nV, on average), and the vowel /ej/ eliciting the lowest amplitude EFRs (106 nV, on average). The consonant context surrounding each vowel stimulus had no statistically significant effect on EFR amplitude. Similarly in Experiment 2, consonant context did not influence the amplitude of EFRs elicited by the vowel variants. Vowel identity significantly altered EFR amplitude with /ε/ eliciting the highest amplitude EFRs (104 nV, on average). Significant, albeit small, differences (<21 nV, on average) in EFR amplitude were evident between some variants of /ε/ and /u/.

CONCLUSION

Based on a comprehensive set of naturally produced vowel samples in carefully controlled consonant contexts, the present study provides additional evidence for the sensitivity of EFRs to vowel identity and variations in vowel production. The surrounding consonant context (after removal of formant transitions) has no measurable effect on EFRs, irrespective of vowel identity and variant. The sensitivity of EFRs to nuances in vowel acoustics emphasizes the need for adequate control and evaluation of stimuli proposed for clinical and research purposes.

Individual variability in auditory feedback processing: Responses to real-time formant perturbations and their relation to perceptual acuity

In this study, both between-subject and within-subject variability in speech perception and speech production were examined in the same set of speakers. Perceptual acuity was determined using an ABX auditory discrimination task, whereby speakers made judgments between pairs of syllables on a /ɛ/ to /æ/ acoustic continuum. Auditory feedback perturbations of the first two formants were implemented in a production task to obtain measures of compensation, normal speech production variability, and vowel spacing. Speakers repeated the word "head" 120 times under varying feedback conditions, with the final Hold phase involving the strongest perturbations of +240 Hz in F1 and -300 Hz in F2. Multiple regression analyses were conducted to determine whether individual differences in compensatory behavior in the Hold phase could be predicted by perceptual acuity, speech production variability, and vowel spacing. Perceptual acuity significantly predicted formant changes in F1, but not in F2. These results are discussed in consideration of the importance of using larger sample sizes in the field and developing new methods to explore feedback processing at the individual participant level. The potential positive role of variability in speech motor control is also considered.

Sensorimotor adaptation of speech depends on the direction of auditory feedback alteration

A hallmark feature of speech motor control is its ability to learn to anticipate and compensate for persistent feedback alterations, a process referred to as sensorimotor adaptation. Because this process involves adjusting articulation to counter the perceived effects of altering acoustic feedback, there are a number of factors that affect it, including the complex relationship between acoustics and articulation and non-uniformities of speech perception. As a consequence, sensorimotor adaptation is hypothesised to vary as a function of the direction of the applied auditory feedback alteration in vowel formant space. This hypothesis was tested in two experiments where auditory feedback was altered in real time, shifting the frequency values of the first and second formants (F1 and F2) of participants' speech. Shifts were designed on a subject-by-subject basis and sensorimotor adaptation was quantified with respect to the direction of applied shift, normalised for individual speakers. Adaptation was indeed found to depend on the direction of the applied shift in vowel formant space, independent of shift magnitude. These findings have implications for models of sensorimotor adaptation of speech.

The Relation of Articulatory and Vocal Auditory-Motor Control in Typical Speakers

Purpose The purpose of this study was to explore the relationship between feedback and feedforward control of articulation and voice by measuring reflexive and adaptive responses to first formant (F 1) and fundamental frequency (f o) perturbations. In addition, perception of F 1 and f o perturbation was estimated using passive (listening) and active (speaking) just noticeable difference paradigms to assess the relation of auditory acuity to reflexive and adaptive responses. Method Twenty healthy women produced single words and sustained vowels while the F 1 or f o of their auditory feedback was suddenly and unpredictably perturbed to assess reflexive responses or gradually and predictably perturbed to assess adaptive responses. Results Typical speakers' reflexive responses to sudden perturbation of F 1 were related to their adaptive responses to gradual perturbation of F 1. Specifically, speakers with larger reflexive responses to sudden perturbation of F 1 had larger adaptive responses to gradual perturbation of F 1. Furthermore, their reflexive responses to sudden perturbation of F 1 were associated with their passive auditory acuity to F 1 such that speakers with better auditory acuity to F 1 produced larger reflexive responses to sudden perturbations of F 1. Typical speakers' adaptive responses to gradual perturbation of F 1 were not associated with their auditory acuity to F 1. Speakers' reflexive and adaptive responses to perturbation of f o were not related, nor were their responses related to either measure of auditory acuity to f o. Conclusion These findings indicate that there may be disparate feedback and feedforward control mechanisms for articulatory and vocal error correction based on auditory feedback.

Adaptation to pitch-altered feedback is independent of one's own voice pitch sensitivity

Monitoring voice pitch is a fine-tuned process in daily conversations as conveying accurately the linguistic and affective cues in a given utterance depends on the precise control of phonation and intonation. This monitoring is thought to depend on whether the error is treated as self-generated or externally-generated, resulting in either a correction or inflation of errors. The present study reports on two separate paradigms of adaptation to altered feedback to explore whether participants could behave in a more cohesive manner once the error is of comparable size perceptually. The vocal behavior of normal-hearing and fluent speakers was recorded in response to a personalized size of pitch shift versus a non-specific size, one semitone. The personalized size of shift was determined based on the just-noticeable difference in fundamental frequency (F0) of each participant’s voice. Here we show that both tasks successfully demonstrated opposing responses to a constant and predictable F0 perturbation (on from the production onset) but these effects barely carried over once the feedback was back to normal, depicting a pattern that bears some resemblance to compensatory responses. Experiencing a F0 shift that is perceived as self-generated (because it was precisely just-noticeable) is not enough to force speakers to behave more consistently and more homogeneously in an opposing manner. On the contrary, our results suggest that the type of the response as well as the magnitude of the response do not depend in any trivial way on the sensitivity of participants to their own voice pitch. Based on this finding, we speculate that error correction could possibly occur even with a bionic ear, typically even when F0 cues are too subtle for cochlear implant users to detect accurately.

Noninvasive neurostimulation of left ventral motor cortex enhances sensorimotor adaptation in speech production

Sensorimotor adaptation-enduring changes to motor commands due to sensory feedback-allows speakers to match their articulations to intended speech acoustics. How the brain integrates auditory feedback to modify speech motor commands and what limits the degree of these modifications remain unknown. Here, we investigated the role of speech motor cortex in modifying stored speech motor plans. In a within-subjects design, participants underwent separate sessions of sham and anodal transcranial direct current stimulation (tDCS) over speech motor cortex while speaking and receiving altered auditory feedback of the first formant. Anodal tDCS increased the rate of sensorimotor adaptation for feedback perturbation. Computational modeling of our results using the Directions Into Velocities of Articulators (DIVA) framework of speech production suggested that tDCS primarily affected behavior by increasing the feedforward learning rate. This study demonstrates how focal noninvasive neurostimulation can enhance the integration of auditory feedback into speech motor plans.

Compensation to real-time temporal auditory feedback perturbation depends on syllable position

Auditory feedback perturbations involving spectral shifts indicated a crucial contribution of auditory feedback to planning and execution of speech. However, much less is known about the contribution of auditory feedback with respect to temporal properties of speech. The current study aimed at providing insight into the representation of temporal properties of speech and the relevance of auditory feedback for speech timing. Real-time auditory feedback perturbations were applied in the temporal domain, viz., stretching and compressing of consonant-consonant-vowel (CCV) durations in onset + nucleus vs vowel-consonant-consonant (VCC) durations in nucleus + coda. Since CCV forms a gesturally more cohesive and stable structure than VCC, greater articulatory adjustments to nucleus + coda (VCC) perturbation were expected. The results show that speakers compensate for focal temporal feedback alterations. Responses to VCC perturbation were greater than to CCV perturbation, suggesting less deformability of onsets when confronted with temporally perturbed auditory feedback. Further, responses to CCV perturbation rather reflected within-trial reactive compensation, whereas VCC compensation was more pronounced and indicative of adaptive behavior. Accordingly, planning and execution of temporal properties of speech are indeed guided by auditory feedback, but the precise nature of the reaction to perturbations is linked to the structural position in the syllable and the associated feedforward timing strategies.

Unconscious and Distinctive Control of Vocal Pitch and Timbre During Altered Auditory Feedback

Vocal control plays a critical role in smooth social communication. Speakers constantly monitor auditory feedback (AF) and make adjustments when their voices deviate from their intentions. Previous studies have shown that when certain acoustic features of the AF are artificially altered, speakers compensate for this alteration in the opposite direction. However, little is known about how the vocal control system implements compensations for alterations of different acoustic features, and associates them with subjective consciousness. The present study investigated whether compensations for the fundamental frequency (F0), which corresponds to perceived pitch, and formants, which contribute to perceived timbre, can be performed unconsciously and independently. Forty native Japanese speakers received two types of altered AF during vowel production that involved shifts of either only the formant frequencies (formant modification; Fm) or both the pitch and formant frequencies (pitch + formant modification; PFm). For each type, three levels of shift (slight, medium, and severe) in both directions (increase or decrease) were used. After the experiment, participants were tested for whether they had perceived a change in the F0 and/or formants. The results showed that (i) only formants were compensated for in the Fm condition, while both the F0 and formants were compensated for in the PFm condition; (ii) the F0 compensation exhibited greater precision than the formant compensation in PFm; and (iii) compensation occurred even when participants misperceived or could not explicitly perceive the alteration in AF. These findings indicate that non-experts can compensate for both formant and F0 modifications in the AF during vocal production, even when the modifications are not explicitly or correctly perceived, which provides further evidence for a dissociation between conscious perception and action in vocal control. We propose that such unconscious control of voice production may enhance rapid adaptation to changing speech environments and facilitate mutual communication.

The perils of learning to move while speaking: One-sided interference between speech and visuomotor adaptation

Our understanding of the adaptive processes that shape sensorimotor behavior is largely derived from studying isolated movements. Studies of visuomotor adaptation, in which participants adapt cursor movements to rotations of the cursor's screen position, have led to prominent theories of motor control. In response to changes in visual feedback of movements, explicit (cognitive) and implicit (automatic) learning processes adapt movements to counter errors. However, movements rarely occur in isolation. The extent to which explicit and implicit processes drive sensorimotor adaptation when multiple movements occur simultaneously, as in the real world, remains unclear. Here we address this problem in the context of speech and hand movements. Participants spoke in-time with rapid, hand-driven cursor movements. Using real-time alterations of vowel sound feedback, and visual rotations of the cursor's screen position, we induced sensorimotor adaptation in one or both movements simultaneously. Across three experiments (n = 60, n = 48 and n = 76, respectively), we demonstrate that visuomotor adaptation is markedly impaired by simultaneous speech adaptation, and the impairment is specific to the explicit learning process in visuomotor adaptation. In contrast, visuomotor adaptation had no impact on speech adaptation. The results demonstrate that the explicit learning process in visuomotor adaptation is sensitive to movements in other motor domains. They suggest that some forms of speech adaptation may lack an explicit learning process altogether.

Sensorimotor adaptation across the speech production workspace in response to a palatal perturbation

Talkers have been shown to adapt the production of multiple vowel sounds simultaneously in response to altered auditory feedback. The present study extends this work by exploring the adaptation of speech production to a physical alteration of the vocal tract involving a palatal prosthesis that impacts both somatosensory and auditory feedback during the production of a range of consonants and vowels. Acoustic and kinematic measures of the tongue were used to examine the impact of the physical perturbation across the various speech sounds, and to assess learned changes following 20 min of speech practice involving the production of complex, variable sentences. As in prior studies, acoustic analyses showed perturbation and adaptation effects primarily for sounds directly involving interaction with the palate. Analyses of tongue kinematics, however, revealed systematic, robust effects of the perturbation and subsequent motor learning across the full range of speech sounds. The results indicate that speakers are able to reconfigure oral motor patterns during the production of multiple speech sounds spanning the articulatory workspace following a physical alteration of the vocal tract.

A Simple 3-Parameter Model for Examining Adaptation in Speech and Voice Production

Sensorimotor adaptation experiments are commonly used to examine motor learning behavior and to uncover information about the underlying control mechanisms of many motor behaviors, including speech production. In the speech and voice domains, aspects of the acoustic signal are shifted/perturbed over time via auditory feedback manipulations. In response, speakers alter their production in the opposite direction of the shift so that their perceived production is closer to what they intended. This process relies on a combination of feedback and feedforward control mechanisms that are difficult to disentangle. The current study describes and tests a simple 3-parameter mathematical model that quantifies the relative contribution of feedback and feedforward control mechanisms to sensorimotor adaptation. The model is a simplified version of the DIVA model, an adaptive neural network model of speech motor control. The three fitting parameters of SimpleDIVA are associated with the three key subsystems involved in speech motor control, namely auditory feedback control, somatosensory feedback control, and feedforward control. The model is tested through computer simulations that identify optimal model fits to six existing sensorimotor adaptation datasets. We show its utility in (1) interpreting the results of adaptation experiments involving the first and second formant frequencies as well as fundamental frequency; (2) assessing the effects of masking noise in adaptation paradigms; (3) fitting more than one perturbation dimension simultaneously; (4) examining sensorimotor adaptation at different timepoints in the production signal; and (5) quantitatively predicting responses in one experiment using parameters derived from another experiment. The model simulations produce excellent fits to real data across different types of perturbations and experimental paradigms (mean correlation between data and model fits across all six studies = 0.95 ± 0.02). The model parameters provide a mechanistic explanation for the behavioral responses to the adaptation paradigm that are not readily available from the behavioral responses alone. Overall, SimpleDIVA offers new insights into speech and voice motor control and has the potential to inform future directions of speech rehabilitation research in disordered populations. Simulation software, including an easy-to-use graphical user interface, is publicly available to facilitate the use of the model in future studies.

Enriching footsteps sounds in gait rehabilitation in chronic stroke patients: a pilot study

In the context of neurorehabilitation, sound is being increasingly applied for facilitating sensorimotor learning. In this study, we aimed to test the potential value of auditory stimulation for improving gait in chronic stroke patients by inducing alterations of the frequency spectra of walking sounds via a sound system that selectively amplifies and equalizes the signal in order to produce distorted auditory feedback. Twenty-two patients with lower extremity paresis were exposed to real-time alterations of their footstep sounds while walking. Changes in body perception, emotion, and gait were quantified. Our results suggest that by altering footsteps sounds, several gait parameters can be modified in terms of left-right foot asymmetry. We observed that augmenting low-frequency bands or amplifying the natural walking sounds led to a reduction in the asymmetry index of stance and stride times, whereas it inverted the asymmetry pattern in heel-ground exerted force. By contrast, augmenting high-frequency bands led to opposite results. These gait changes might be related to updating of internal forward models, signaling the need for adjustment of the motor system to reduce the perceived discrepancies between predicted-actual sensory feedbacks. Our findings may have the potential to enhance gait awareness in stroke patients and other clinical conditions, supporting gait rehabilitation.