Talkers alter vowel production in response to real-time formant perturbation even when instructed not to compensate

Divided Attention Has Limited Effects on Speech Sensorimotor Control

PURPOSE

When vowel formants are externally perturbed, speakers change their production to oppose that perturbation both during the ongoing production (compensation) and in future productions (adaptation). To date, attempts to explain the large variability across individuals in these responses have focused on trait-based characteristics such as auditory acuity, but evidence from other motor domains suggests that attention may modulate the motor response to sensory perturbations. Here, we test the extent to which divided attention impacts sensorimotor control for supralaryngeal articulation.

METHOD

Neurobiologically healthy speakers were exposed to random (Experiment 1) or consistent (Experiment 2) real-time auditory perturbation of vowel formants to measure online compensation and trial-to-trial adaptation, respectively. In both experiments, participants completed two conditions: one with a simultaneous visual distractor task to divide attention and one without this secondary task.

RESULTS

Divided visual attention slightly reduced online compensation, but only starting > 300 ms after vowel onset, well beyond the typical duration of vowels in speech. Divided attention had no effect on adaptation.

CONCLUSIONS

The results from both experiments suggest that the use of sensory feedback in typical speech motor control is a largely automatic process unaffected by divided visual attention, suggesting that the source of cross-speaker variability in response to formant perturbations likely lies within the speech production system rather than in higher-level cognitive processes. Methodologically, these results suggest that compensation for formant perturbations should be measured prior to 300 ms after vowel onset to avoid any potential impact of attention or other higher-order cognitive factors.

Shaping linguistic input in parent-infant interactions: The influence of the Infant's temperament

Parent-infant interactions highlight the role of parental input, considering both the quality, infant-directed speech, and quantity of interactions, adult words and communicative turns, in these interactions. However, communication is bidirectional, yet little is known about the infant's role in these interactions. This study (n = 35 4-month-old infants) explores how infant-directed speech, the number of adult words and turn-taking (both measured by the LENA system) are correlated with infants' temperament. Our findings reveal that, while mothers use the typical characteristics of infant-directed speech, they are not correlated with the infant's temperament. However, we observe more adult-infant turn-taking in both introverted infants (with lower Surgency scores) and infants with lower attention regulation (with lower Regulatory/Orienting scores). The number of adult words was not correlated with infants' temperament. We suggest that infants with an introverted temperament prefer quieter exchanges that may lead to more turns and that infants with lower attention regulation might create more opportunities for interactions due to their lower level of self-regulation. These findings suggest that infants' temperament is associated with how adults talk with infants (communicative turns) rather than how adults talk to infants (infant-directed speech, number of adult words). Our results underscore the infant's role in parent-infant communication.

Neurophysiological evidence of sensory prediction errors driving speech sensorimotor adaptation

The human sensorimotor system has a remarkable ability to quickly and efficiently learn movements from sensory experience. A prominent example is sensorimotor adaptation, learning that characterizes the sensorimotor system's response to persistent sensory errors by adjusting future movements to compensate for those errors. Despite being essential for maintaining and fine-tuning motor control, mechanisms underlying sensorimotor adaptation remain unclear. A component of sensorimotor adaptation is implicit (i.e., the learner is unaware of the learning process) which has been suggested to result from sensory prediction errors-the discrepancies between predicted sensory consequences of motor commands and actual sensory feedback. However, to date no direct neurophysiological evidence that sensory prediction errors drive adaptation has been demonstrated. Here, we examined prediction errors via magnetoencephalography (MEG) imaging of the auditory cortex (n = 34) during sensorimotor adaptation of speech to altered auditory feedback, an entirely implicit adaptation task. Specifically, we measured how speaking-induced suppression (SIS)--a neural representation of auditory prediction errors--changed over the trials of the adaptation experiment. SIS refers to the suppression of auditory cortical response to speech onset (in particular, the M100 response) to self-produced speech when compared to the response to passive listening to identical playback of that speech. SIS was reduced (reflecting larger prediction errors) during the early learning phase compared to the initial unaltered feedback phase. Furthermore, reduction in SIS positively correlated with behavioral adaptation extents, suggesting that larger prediction errors were associated with more learning. In contrast, such a reduction in SIS was not found in a control experiment in which participants heard unaltered feedback and thus did not adapt. In addition, in some participants who reached a plateau in the late learning phase, SIS increased (reflecting smaller prediction errors), demonstrating that prediction errors were minimal when there was no further adaptation. Together, these findings provide the first neurophysiological evidence for the hypothesis that prediction errors drive human sensorimotor adaptation.

Speech-induced suppression and vocal feedback sensitivity in human cortex

Across the animal kingdom, neural responses in the auditory cortex are suppressed during vocalization, and humans are no exception. A common hypothesis is that suppression increases sensitivity to auditory feedback, enabling the detection of vocalization errors. This hypothesis has been previously confirmed in non-human primates, however a direct link between auditory suppression and sensitivity in human speech monitoring remains elusive. To address this issue, we obtained intracranial electroencephalography (iEEG) recordings from 35 neurosurgical participants during speech production. We first characterized the detailed topography of auditory suppression, which varied across superior temporal gyrus (STG). Next, we performed a delayed auditory feedback (DAF) task to determine whether the suppressed sites were also sensitive to auditory feedback alterations. Indeed, overlapping sites showed enhanced responses to feedback, indicating sensitivity. Importantly, there was a strong correlation between the degree of auditory suppression and feedback sensitivity, suggesting suppression might be a key mechanism that underlies speech monitoring. Further, we found that when participants produced speech with simultaneous auditory feedback, posterior STG was selectively activated if participants were engaged in a DAF paradigm, suggesting that increased attentional load can modulate auditory feedback sensitivity.

Speech sensorimotor relationships in francophone preschoolers and adults: Adaptation to real-time auditory feedback perturbations

PURPOSE

This study investigates the development of sensorimotor relationships by examining adaptation to real-time perturbations of auditory feedback.

METHOD

Acoustic signals were recorded while preschoolers and adult speakers of Canadian French produced several utterances of the front rounded vowel /ø/ for which F2 was gradually shifted up to a maximum of 40%.

RESULTS

The findings indicate that, although preschool-aged children produced overall similar responses to the perturbed feedback, they displayed significantly more trial-to-trial variability than adults. Furthermore, whereas the magnitude of the adaptation in adults was positively correlated with the slope of the perceptual categorical function, the amount of adaptation in children was linked to the variability of their productions in the baseline condition. These patterns suggest that the immature motor control observed in children, which contributes to increased variability in their speech production, plays a role in shaping adaptive behavior, as it allows children to explore articulatory/acoustic spaces and learn sensorimotor relationships.

Increased vowel contrast and intelligibility in connected speech induced by sensorimotor adaptation

Alterations to sensory feedback can drive robust adaptive changes to the production of consonants and vowels, but these changes often have no behavioral relevance or benefit to communication (e.g., making "head" more like "had"). This work aims to align the outcomes of adaptation with changes known to increase speech intelligibility - specifically, adaptations that increase the acoustic contrast between vowels in running speech. To this end, we implemented a vowel centralization feedback perturbation paradigm that pushes all vowels towards the center of vowel space, making them sound less distinct from one another. Speakers across the adult lifespan adapted to the centralization perturbation during sentence production, increasing the global acoustic contrast among vowels and the articulatory excursions for individual vowels. These changes persisted after the perturbation was removed, including after a silent delay, and showed robust transfer to words that were not present in the sentences. Control analyses demonstrated that these effects were unlikely to be due to explicit pronunciation strategies and occurred in the face of increasingly more rapid and less distinct production of familiar sentences. Finally, sentence transcription by crowd-sourced listeners showed that speakers' vowel contrast predicted their baseline intelligibility and that experimentally-induced increases in contrast predicted intelligibility gains. These findings establish the validity of a sensorimotor adaptation paradigm to implicitly increase vowel contrast and intelligibility in connected speech, an outcome that has the potential to enhance rehabilitation in individuals who present with a reduced vowel space due to motor speech disorders, such as the hypokinetic dysarthria associated with Parkinson's disease.

Speech-induced suppression and vocal feedback sensitivity in human cortex

Across the animal kingdom, neural responses in the auditory cortex are suppressed during vocalization, and humans are no exception. A common hypothesis is that suppression increases sensitivity to auditory feedback, enabling the detection of vocalization errors. This hypothesis has been previously confirmed in non-human primates, however a direct link between auditory suppression and sensitivity in human speech monitoring remains elusive. To address this issue, we obtained intracranial electroencephalography (iEEG) recordings from 35 neurosurgical participants during speech production. We first characterized the detailed topography of auditory suppression, which varied across superior temporal gyrus (STG). Next, we performed a delayed auditory feedback (DAF) task to determine whether the suppressed sites were also sensitive to auditory feedback alterations. Indeed, overlapping sites showed enhanced responses to feedback, indicating sensitivity. Importantly, there was a strong correlation between the degree of auditory suppression and feedback sensitivity, suggesting suppression might be a key mechanism that underlies speech monitoring. Further, we found that when participants produced speech with simultaneous auditory feedback, posterior STG was selectively activated if participants were engaged in a DAF paradigm, suggesting that increased attentional load can modulate auditory feedback sensitivity.

The Accompanying Effect in Responses to Auditory Perturbations: Unconscious Vocal Adjustments to Unperturbed Parameters

PURPOSE

The present study examined whether participants respond to unperturbed parameters while experiencing specific perturbations in auditory feedback. For instance, we aim to determine if speakers adjust voice loudness when only pitch is artificially altered in auditory feedback. This phenomenon is referred to as the "accompanying effect" in the present study.

METHOD

Thirty native Mandarin speakers were asked to sustain the vowel /ɛ/ for 3 s while their auditory feedback underwent single shifts in one of the three distinct ways: pitch shift (±100 cents; coded as PT), loudness shift (±6 dB; coded as LD), or first formant (F1) shift (±100 Hz; coded as FM). Participants were instructed to ignore the perturbations in their auditory feedback. Response types were categorized based on pitch, loudness, and F1 for each individual trial, such as Popp_Lopp_Fopp indicating opposing responses in all three domains.

RESULTS

The accompanying effect appeared 93% of the time. Bayesian Poisson regression models indicate that opposing responses in all three domains (Popp_Lopp_Fopp) were the most prevalent response type across the conditions (PT, LD, and FM). The more frequently used response types exhibited opposing responses and significantly larger response curves than the less frequently used response types. Following responses became more prevalent only when the perturbed stimuli were perceived as voices from someone else (external references), particularly in the FM condition. In terms of isotropy, loudness and F1 tended to change in the same direction rather than loudness and pitch.

CONCLUSION

The presence of the accompanying effect suggests that the motor systems responsible for regulating pitch, loudness, and formants are not entirely independent but rather interconnected to some degree.

Pars Opercularis Underlies Efferent Predictions and Successful Auditory Feedback Processing in Speech: Evidence From Left-Hemisphere Stroke

Hearing one's own speech allows for acoustic self-monitoring in real time. Left-hemisphere motor planning regions are thought to give rise to efferent predictions that can be compared to true feedback in sensory cortices, resulting in neural suppression commensurate with the degree of overlap between predicted and actual sensations. Sensory prediction errors thus serve as a possible mechanism of detection of deviant speech sounds, which can then feed back into corrective action, allowing for online control of speech acoustics. The goal of this study was to assess the integrity of this detection-correction circuit in persons with aphasia (PWA) whose left-hemisphere lesions may limit their ability to control variability in speech output. We recorded magnetoencephalography (MEG) while 15 PWA and age-matched controls spoke monosyllabic words and listened to playback of their utterances. From this, we measured speaking-induced suppression of the M100 neural response and related it to lesion profiles and speech behavior. Both speaking-induced suppression and cortical sensitivity to deviance were preserved at the group level in PWA. PWA with more spared tissue in pars opercularis had greater left-hemisphere neural suppression and greater behavioral correction of acoustically deviant pronunciations, whereas sparing of superior temporal gyrus was not related to neural suppression or acoustic behavior. In turn, PWA who made greater corrections had fewer overt speech errors in the MEG task. Thus, the motor planning regions that generate the efferent prediction are integral to performing corrections when that prediction is violated.

Discrimination and sensorimotor adaptation of self-produced vowels in cochlear implant users

Humans rely on auditory feedback to monitor and adjust their speech for clarity. Cochlear implants (CIs) have helped over a million people restore access to auditory feedback, which significantly improves speech production. However, there is substantial variability in outcomes. This study investigates the extent to which CI users can use their auditory feedback to detect self-produced sensory errors and make adjustments to their speech, given the coarse spectral resolution provided by their implants. First, we used an auditory discrimination task to assess the sensitivity of CI users to small differences in formant frequencies of their self-produced vowels. Then, CI users produced words with altered auditory feedback in order to assess sensorimotor adaptation to auditory error. Almost half of the CI users tested can detect small, within-channel differences in their self-produced vowels, and they can utilize this auditory feedback towards speech adaptation. An acoustic hearing control group showed better sensitivity to the shifts in vowels, even in CI-simulated speech, and elicited more robust speech adaptation behavior than the CI users. Nevertheless, this study confirms that CI users can compensate for sensory errors in their speech and supports the idea that sensitivity to these errors may relate to variability in production.

How to cut the pie is no piece of cake: Toward a process-oriented approach to assessment and diagnosis of speech sound disorders

BACKGROUND

'Speech sound disorder' is an umbrella term that encompasses dysarthria, articulation disorders, childhood apraxia of speech and phonological disorders. However, differential diagnosis between these disorders is a persistent challenge in speech pathology, as many diagnostic procedures use symptom clusters instead of identifying an origin of breakdown in the speech and language system.

AIMS

This article reviews typical and disordered speech through the lens of two well-developed models of production-one focused on phonological encoding and one focused on speech motor planning. We illustrate potential breakdown locations within these models that may relate to childhood apraxia of speech and phonological disorders.

MAIN CONTRIBUTION

This paper presents an overview of an approach to conceptualisation of speech sound disorders that is grounded in current models of speech production and emphasises consideration of underlying processes. The paper also sketches a research agenda for the development of valid, reliable and clinically feasible assessment protocols for children with speech sound disorders.

CONCLUSION

The process-oriented approach outlined here is in the early stages of development but holds promise for developing a more detailed and comprehensive understanding of, and assessment protocols for speech sound disorders that go beyond broad diagnostic labels based on error analysis. Directions for future research are discussed.

WHAT THIS PAPER ADDS

What is already known on the subject Speech sound disorders (SSD) are heterogeneous, and there is agreement that some children have a phonological impairment (phonological disorders, PD) whereas others have an impairment of speech motor planning (childhood apraxia of speech, CAS). There is also recognition that speech production involves multiple processes, and several approaches to the assessment and diagnosis of SSD have been proposed. What this paper adds to existing knowledge This paper provides a more detailed conceptualisation of potential impairments in children with SSD that is grounded in current models of speech production and encourages greater consideration of underlying processes. The paper illustrates this approach and provides guidance for further development. One consequence of this perspective is the notion that broad diagnostic category labels (PD, CAS) may each comprise different subtypes or profiles depending on the processes that are affected. What are the potential or actual clinical implications of this work? Although the approach is in the early stages of development and no comprehensive validated set of tasks and measures is available to assess all processes, clinicians may find the conceptualisation of different underlying processes and the notion of potential subtypes within PD and CAS informative when evaluating SSD. In addition, this perspective discourages either/or thinking (PD or CAS) and instead encourages consideration of the possibility that children may have different combinations of impairments at different processing stages.

Mechanisms of sensorimotor adaptation in a hierarchical state feedback control model of speech

Upon perceiving sensory errors during movements, the human sensorimotor system updates future movements to compensate for the errors, a phenomenon called sensorimotor adaptation. One component of this adaptation is thought to be driven by sensory prediction errors-discrepancies between predicted and actual sensory feedback. However, the mechanisms by which prediction errors drive adaptation remain unclear. Here, auditory prediction error-based mechanisms involved in speech auditory-motor adaptation were examined via the feedback aware control of tasks in speech (FACTS) model. Consistent with theoretical perspectives in both non-speech and speech motor control, the hierarchical architecture of FACTS relies on both the higher-level task (vocal tract constrictions) as well as lower-level articulatory state representations. Importantly, FACTS also computes sensory prediction errors as a part of its state feedback control mechanism, a well-established framework in the field of motor control. We explored potential adaptation mechanisms and found that adaptive behavior was present only when prediction errors updated the articulatory-to-task state transformation. In contrast, designs in which prediction errors updated forward sensory prediction models alone did not generate adaptation. Thus, FACTS demonstrated that 1) prediction errors can drive adaptation through task-level updates, and 2) adaptation is likely driven by updates to task-level control rather than (only) to forward predictive models. Additionally, simulating adaptation with FACTS generated a number of important hypotheses regarding previously reported phenomena such as identifying the source(s) of incomplete adaptation and driving factor(s) for changes in the second formant frequency during adaptation to the first formant perturbation. The proposed model design paves the way for a hierarchical state feedback control framework to be examined in the context of sensorimotor adaptation in both speech and non-speech effector systems.

Effects of Gradual and Sudden Introduction of Perturbations on Adaptive Responses to Formant-Shift and Formant-Clamp Perturbations

PURPOSE

When the speech motor system encounters errors, it generates adaptive responses to compensate for the errors. Unlike errors induced by formant-shift perturbations, errors induced by formant-clamp perturbations do not correspond with the speaker's speech (i.e., degraded motor-to-auditory correspondence). We previously showed that adaptive responses to formant-clamp perturbations are smaller than responses to formant-shift perturbations when perturbations are introduced gradually. This study examined responses to formant-clamp and formant-shift perturbations when perturbations are introduced suddenly.

METHOD

One group of participants (n = 30) experienced gradually introduced formant-clamp and formant-shift perturbations, and another group (n = 30) experienced suddenly introduced formant-clamp and formant-shift perturbations. We designed the perturbations based on participant-specific vowel configurations such that a participant's first and second formants of /ɛ/ were perturbed toward their /æ/. To estimate adaptive responses, we measured formant changes (0-100 ms of the vowel) in response to the formant perturbations.

RESULTS

We found that (a) the difference between responses to formant-clamp and formant-shift perturbations was smaller when the perturbations were introduced suddenly and (b) responses to suddenly introduced (but not gradually introduced) formant-shift perturbations positively correlated with responses to formant-clamp perturbations.

CONCLUSIONS

These results showed that the speech motor system responds to errors induced by formant-shift and formant-clamp perturbations more differently when perturbations are introduced gradually than suddenly. Overall, the quality of errors (formant-shift vs. formant-clamp) and the manner of introducing errors (gradually vs. suddenly) modulate the speech motor system's evaluations of and responses to errors.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.22406422.

Voice Biofeedback via Bone Conduction Headphones: Effects on Acoustic Voice Parameters and Self-Reported Vocal Effort in Individuals With Voice Disorders

PURPOSE

This study explores sidetone amplification (amplified playback of one's own voice) provided via bone conduction in participants with voice disorders. The effects of bone conduction feedback on acoustic voice parameters and vocal effort ratings are examined.

METHODS

Speech samples of 47 participants with voice disorders were recorded in three auditory feedback conditions: two with sidetone amplification delivered via bone conduction and one condition with no alteration of the feedback. After each task, the participants rated their vocal effort on a visual analog scale. The voice recordings were evaluated by a speech-language pathologist through the GRBAS scale and processed to calculate the within-participant centered sound pressure level (SPL) values, the mean pitch strength (PS), the time dose (Dt%), and cepstral peak prominence smoothed (CPPS). The effects of the feedback conditions on these acoustic parameters and vocal effort ratings were analyzed.

RESULTS

The high sidetone amplification condition resulted in a statistically significant decrease in the within-participant centered SPL values and mean pitch strength across all participants. The feedback conditions had no statistically significant effects on the vocal effort ratings, time dose (Dt%), or CPPS.

CONCLUSIONS

This study provides an evidence that bone conduction sidetone amplification contributes to a consistent adaptation in the within-participant centered SPL values (ΔSPL) in patients with vocal hyperfunction, glottal insufficiency, and organic/neurological laryngeal pathologies compared to conditions with no feedback.

Virtual reality head-mounted displays affect sidetone perception

The purpose of this study was to investigate whether head-mounted displays (HMDs) change the sidetone to an auditory perceivable extent. Impulse responses (IRs) were recorded using a dummy head wearing a HMD (IRtest) and compared to IRs measured without HMD (IRref). Ten naive listeners were tested on their ability to discriminate between the IRtest and IRref using convolved speech signals. The spectral analysis showed that the HMDs decreased the spectral energy of the sidetone around 2000-4500 Hz. Most listeners were able to discriminate between the IRs. It is concluded that HMDs change the sidetone to a small but perceivable extent.

Individual sensorimotor adaptation characteristics are independent across orofacial speech movements and limb reaching movements

Sensorimotor adaptation is critical for human motor control but shows considerable interindividual variability. Efforts are underway to identify factors accounting for individual differences in specific adaptation tasks. However, a fundamental question has remained unaddressed: Is an individual's capability for adaptation effector system specific or does it reflect a generalized adaptation ability? We therefore tested the same participants in analogous adaptation paradigms focusing on distinct sensorimotor systems: speaking with perturbed auditory feedback and reaching with perturbed visual feedback. Each task was completed once with the perturbation introduced gradually (ramped up over 60 trials) and, on a different day, once with the perturbation introduced suddenly. Consistent with studies of each system separately, visuomotor reach adaptation was more complete than auditory-motor speech adaptation (80% vs. 29% of the perturbation). Adaptation was not significantly correlated between the speech and reach tasks. Moreover, considered within tasks, 1) adaptation extent was correlated between the gradual and sudden conditions for reaching but not for speaking, 2) adaptation extent was correlated with additional measures of performance (e.g., trial duration, within-trial corrections) only for reaching and not for speaking, and 3) fitting individual participant adaptation profiles with exponential rather than linear functions offered a larger benefit [lower root mean square error (RMSE)] for the reach task than for the speech task. Combined, results suggest that the ability for sensorimotor adaptation relies on neural plasticity mechanisms that are effector system specific rather than generalized. This finding has important implications for ongoing efforts seeking to identify cognitive, behavioral, and neurochemical predictors of individual sensorimotor adaptation.NEW & NOTEWORTHY This study provides the first detailed demonstration that individual sensorimotor adaptation characteristics are independent across articulatory speech movements and limb reaching movements. Thus, individual sensorimotor learning abilities are effector system specific rather than generalized. Findings regarding one effector system do not necessarily apply to other systems, different underlying mechanisms may be involved, and implications for clinical rehabilitation or performance training also cannot be generalized.

Perturbing the consistency of auditory feedback in speech

Sensory information, including auditory feedback, is used by talkers to maintain fluent speech articulation. Current models of speech motor control posit that speakers continually adjust their motor commands based on discrepancies between the sensory predictions made by a forward model and the sensory consequences of their speech movements. Here, in two within-subject design experiments, we used a real-time formant manipulation system to explore how reliant speech articulation is on the accuracy or predictability of auditory feedback information. This involved introducing random formant perturbations during vowel production that varied systematically in their spatial location in formant space (Experiment 1) and temporal consistency (Experiment 2). Our results indicate that, on average, speakers’ responses to auditory feedback manipulations varied based on the relevance and degree of the error that was introduced in the various feedback conditions. In Experiment 1, speakers’ average production was not reliably influenced by random perturbations that were introduced every utterance to the first (F1) and second (F2) formants in various locations of formant space that had an overall average of 0 Hz. However, when perturbations were applied that had a mean of +100 Hz in F1 and −125 Hz in F2, speakers demonstrated reliable compensatory responses that reflected the average magnitude of the applied perturbations. In Experiment 2, speakers did not significantly compensate for perturbations of varying magnitudes that were held constant for one and three trials at a time. Speakers’ average productions did, however, significantly deviate from a control condition when perturbations were held constant for six trials. Within the context of these conditions, our findings provide evidence that the control of speech movements is, at least in part, dependent upon the reliability and stability of the sensory information that it receives over time.

A single exposure to altered auditory feedback causes observable sensorimotor adaptation in speech

Sensory errors induce two types of behavioral changes: rapid compensation within a movement and longer-term adaptation of subsequent movements. Although adaptation is hypothesized to occur whenever a sensory error is perceived (including after a single exposure to altered feedback), adaptation of articulatory movements in speech has only been observed after repeated exposure to auditory perturbations, questioning both current theories of speech sensorimotor adaptation and the universality of more general theories of adaptation. We measured single-exposure or ‘one-shot’ learning in a large dataset in which participants were exposed to intermittent, unpredictable perturbations of their speech acoustics. On unperturbed trials immediately following these perturbed trials, participants adjusted their speech to oppose the preceding shift, demonstrating that learning occurs even after a single exposure to auditory error. These results provide critical support for current theories of sensorimotor adaptation in speech and align speech more closely with learning in other motor domains.

Auditory and somatosensory feedback mechanisms of laryngeal and articulatory speech motor control

PURPOSE

Speech production is a complex motor task involving multiple subsystems. The relationships between these subsystems need to be comprehensively investigated to understand the underlying mechanisms of speech production. The goal of this paper is to examine the differential contributions of 1) auditory and somatosensory feedback control mechanisms, and 2) laryngeal and articulatory speech production subsystems on speech motor control at an individual speaker level using altered auditory and somatosensory feedback paradigms.

METHODS

Twenty young adults completed speaking tasks in which sudden and unpredictable auditory and physical perturbations were applied to the laryngeal and articulatory speech production subsystems. Auditory perturbations were applied to laryngeal or articulatory acoustic features of speech. Physical perturbations were applied to the larynx and the jaw. Pearson-product moment correlation coefficients were calculated between 1) auditory and somatosensory reflexive responses to investigate relationships between auditory and somatosensory feedback control mechanisms, and 2) laryngeal and articulatory reflexive responses as well as acuity measures to investigate the relationship between auditory-motor features of laryngeal and articulatory subsystems.

RESULTS

No statistically significant correlations were found concerning the relationships between auditory and somatosensory feedback. No statistically significant correlations were found between auditory-motor features in the laryngeal and articulatory control subsystems.

CONCLUSION

Results suggest that the laryngeal and articulatory speech production subsystems operate with differential auditory and somatosensory feedback control mechanisms. The outcomes suggest that current models of speech motor control should consider decoupling laryngeal and articulatory domains to better model speech motor control processes.

Pediatric Responses to Fundamental and Formant Frequency Altered Auditory Feedback: A Scoping Review

Purpose

The ability to hear ourselves speak has been shown to play an important role in the development and maintenance of fluent and coherent speech. Despite this, little is known about the developing speech motor control system throughout childhood, in particular if and how vocal and articulatory control may differ throughout development. A scoping review was undertaken to identify and describe the full range of studies investigating responses to frequency altered auditory feedback in pediatric populations and their contributions to our understanding of the development of auditory feedback control and sensorimotor learning in childhood and adolescence.

Method

Relevant studies were identified through a comprehensive search strategy of six academic databases for studies that included (a) real-time perturbation of frequency in auditory input, (b) an analysis of immediate effects on speech, and (c) participants aged 18 years or younger.

Results

Twenty-three articles met inclusion criteria. Across studies, there was a wide variety of designs, outcomes and measures used. Manipulations included fundamental frequency (9 studies), formant frequency (12), frequency centroid of fricatives (1), and both fundamental and formant frequencies (1). Study designs included contrasts across childhood, between children and adults, and between typical, pediatric clinical and adult populations. Measures primarily explored acoustic properties of speech responses (latency, magnitude, and variability). Some studies additionally examined the association of these acoustic responses with clinical measures (e.g., stuttering severity and reading ability), and neural measures using electrophysiology and magnetic resonance imaging.

Conclusion

Findings indicated that children above 4 years generally compensated in the opposite direction of the manipulation, however, in several cases not as effectively as adults. Overall, results varied greatly due to the broad range of manipulations and designs used, making generalization challenging. Differences found between age groups in the features of the compensatory vocal responses, latency of responses, vocal variability and perceptual abilities, suggest that maturational changes may be occurring in the speech motor control system, affecting the extent to which auditory feedback is used to modify internal sensorimotor representations. Varied findings suggest vocal control develops prior to articulatory control. Future studies with multiple outcome measures, manipulations, and more expansive age ranges are needed to elucidate findings.

Disruption of speech motor adaptation with repetitive transcranial magnetic stimulation of the articulatory representation in primary motor cortex

When auditory feedback perturbation is introduced in a predictable way over a number of utterances, speakers learn to compensate by adjusting their own productions, a process known as sensorimotor adaptation. Despite multiple lines of evidence indicating the role of primary motor cortex (M1) in motor learning and memory, whether M1 causally contributes to sensorimotor adaptation in the speech domain remains unclear. Here, we aimed to assay whether temporary disruption of the articulatory representation in left M1 by repetitive transcranial magnetic stimulation (rTMS) impairs speech adaptation. To induce sensorimotor adaptation, the frequencies of first formants (F1) were shifted up and played back to participants when they produced “head”, “bed”, and “dead” repeatedly (the learning phase). A low-frequency rTMS train (.6 Hz, subthreshold, 12 min) over either the tongue or the hand representation of M1 (between-subjects design) was applied before participants experienced altered auditory feedback in the learning phase. We found that the group who received rTMS over the hand representation showed the expected compensatory response for the upwards shift in F1 by significantly reducing F1 and increasing the second formant (F2) frequencies in their productions. In contrast, these expected compensatory changes in both F1 and F2 did not occur in the group that received rTMS over the tongue representation. Critically, rTMS (subthreshold) over the tongue representation did not affect vowel production, which was unchanged from baseline. These results provide direct evidence that the articulatory representation in left M1 causally contributes to sensorimotor learning in speech. Furthermore, these results also suggest that M1 is critical to the network supporting a more global adaptation that aims to move the altered speech production closer to a learnt pattern of speech production used to produce another vowel.

Differential Effects of Cerebellar Degeneration on Feedforward versus Feedback Control across Speech and Reaching Movements

Errors that result from a mismatch between predicted movement outcomes and sensory afference are used to correct ongoing movements through feedback control and to adapt feedforward control of future movements. The cerebellum has been identified as a critical part of the neural circuit underlying implicit adaptation across a wide variety of movements (reaching, gait, eye movements, and speech). The contribution of this structure to feedback control is less well understood. Although it has recently been shown in the speech domain that individuals with cerebellar degeneration produce larger online corrections for sensory perturbations than control participants, similar behavior has not been observed in other motor domains. Currently, comparisons across domains are limited by different population samples and potential ceiling effects in existing tasks. To assess the relationship between changes in feedforward and feedback control associated with cerebellar degeneration across motor domains, we evaluated adaptive (feedforward) and compensatory (feedback) responses to sensory perturbations in reaching and speech production in human participants of both sexes with cerebellar degeneration and neurobiologically healthy controls. As expected, the cerebellar group demonstrated impaired adaptation in both reaching and speech. In contrast, the groups did not differ in their compensatory response in either domain. Moreover, compensatory and adaptive responses in the cerebellar group were not correlated within or across motor domains. These results point to a general impairment in feedforward control with spared feedback control in cerebellar degeneration. However, the magnitude of feedforward impairments and potential changes in feedback-based control manifest in a domain-specific manner across individuals.SIGNIFICANCE STATEMENT The cerebellum contributes to feedforward updating of movement in response to sensory errors, but its role in feedback control is less understood. Here, we tested individuals with cerebellar degeneration (CD), using sensory perturbations to assess adaptation of feedforward control and feedback gains during reaching and speech production tasks. The results confirmed that CD leads to reduced adaption in both domains. However, feedback gains were unaffected by CD in either domain. Interestingly, measures of feedforward and feedback control were not correlated across individuals within or across motor domains. Together, these results indicate a general impairment in feedforward control with spared feedback control in CD. However, the magnitude of feedforward impairments manifests in a domain-specific manner across individuals.

Response patterns to vowel formant perturbations in children

Auditory feedback is an important component of speech motor control, but its precise role in developing speech is less understood. The role of auditory feedback in development was probed by perturbing the speech of children 4-9 years old. The vowel sound /ɛ/ was shifted to /æ/ in real time and presented to participants as their own auditory feedback. Analyses of the resultant formant magnitude changes in the participants' speech indicated that children compensated and adapted by adjusting their formants to oppose the perturbation. Older and younger children responded to perturbation differently in F₁ and F₂. The compensatory change in F₁ was greater for younger children, whereas the increase in F₂ was greater for older children. Adaptation aftereffects were observed in both groups. Exploratory directional analyses in the two-dimensional formant space indicated that older children responded more directly and less variably to the perturbation than younger children, shifting their vowels back toward the vowel sound /ɛ/ to oppose the perturbation. Findings support the hypothesis that auditory feedback integration continues to develop between the ages of 4 and 9 years old such that the differences in the adaptive and compensatory responses arise between younger and older children despite receiving the same auditory feedback perturbation.

Auditory Feedback Is Used for Adaptation and Compensation in Speech Timing

Purpose Real-time altered feedback has demonstrated a key role for auditory feedback in both online feedback control and in updating feedforward control for future utterances. The aim of this study was to examine adaptation in response to temporal perturbation using real-time perturbation of ongoing speech. Method Twenty native English speakers with no reported history of speech or hearing disorders participated in this study. The study consisted of four word blocks, using the phrases "a capper," "a gapper," "a sapper," and "a zapper" (due to issues with the implementation of perturbation, "gapper" was excluded from analysis). In each block, participants completed a baseline phase (30 trials of veridical feedback), a ramp phase (feedback perturbation increasing to maximum over 30 trials), a hold phase (60 trials with perturbation held at maximum), and a washout phase (30 trials, feedback abruptly returned to veridical feedback). Word-initial consonant targets (voice onset time for /k, g/ and fricative duration for /s, z/) were lengthened, and the following stressed vowel (/æ/) was shortened. Results Overall, speakers did not adapt the production of their consonants but did lengthen their vowel production in response to shortening. Vowel lengthening showed continued aftereffects during the early portion of the washout phase. Although speakers did not adapt absolute consonant durations, consonant duration was reduced as a proportion of the total syllable duration. This is consistent with previous research that suggests that speakers attend to proportional durations rather than absolute durations. Conclusion These results indicate that speakers actively monitor proportional durations and update the temporal dynamics of planning units extending beyond a single segment.

Articulatory compensation for low-pass filtered formant-altered auditory feedback

Auditory feedback while speaking plays an important role in stably controlling speech articulation. Its importance has been verified in formant-altered auditory feedback (AAF) experiments where speakers utter while listening to speech with perturbed first (F1) and second (F2) formant frequencies. However, the contribution of the frequency components higher than F2 to the articulatory control under the perturbations of F1 and F2 has not yet been investigated. In this study, a formant-AAF experiment was conducted in which a low-pass filter was applied to speech. The experimental results showed that the deviation in the compensatory response was significantly larger when a low-pass filter with a cutoff frequency of 3 kHz was used compared to that when cutoff frequencies of 4 and 8 kHz were used. It was also found that the deviation in the 3-kHz condition correlated with the fundamental frequency and spectral tilt of the produced speech. Additional simulation results using a neurocomputational model of speech production (SimpleDIVA model) and the experimental data showed that the feedforward learning rate increased as the cutoff frequency decreased. These results suggest that high-frequency components of the auditory feedback would be involved in the determination of corrective motor commands from auditory errors.

Intact Correction for Self-Produced Vowel Formant Variability in Individuals With Cerebellar Ataxia Regardless of Auditory Feedback Availability

Purpose Individuals with cerebellar ataxia (CA) caused by cerebellar degeneration exhibit larger reactive compensatory responses to unexpected auditory feedback perturbations than neurobiologically typical speakers, suggesting they may rely more on feedback control during speech. We test this hypothesis by examining variability in unaltered speech. Previous studies of typical speakers have demonstrated a reduction in formant variability (centering) observed during the initial phase of vowel production from vowel onset to vowel midpoint. Centering is hypothesized to reflect feedback-based corrections for self-produced variability and thus may provide a behavioral assay of feedback control in unperturbed speech in the same manner as the compensatory response does for feedback perturbations. Method To comprehensively compare centering in individuals with CA and controls, we examine centering in two vowels (/i/ and /ɛ/) under two contexts (isolated words and connected speech). As a control, we examine speech produced both with and without noise to mask auditory feedback. Results Individuals with CA do not show increased centering compared to age-matched controls, regardless of vowel, context, or masking. Contrary to previous results in neurobiologically typical speakers, centering was not affected by the presence of masking noise in either group. Conclusions The similar magnitude of centering seen with and without masking noise questions whether centering is driven by auditory feedback. However, if centering is at least partially driven by auditory/somatosensory feedback, these results indicate that the larger compensatory response to altered auditory feedback observed in individuals with CA may not reflect typical motor control processes during normal, unaltered speech production.

Compensation to Altered Auditory Feedback in Children With Developmental Language Disorder and Typical Development

Purpose Developmental language disorder (DLD), an unexplained problem using and understanding spoken language, has been hypothesized to have an underlying auditory processing component. Auditory feedback plays a key role in speech motor control. The current study examined whether auditory feedback is used to regulate speech production in a similar way by children with DLD and their typically developing (TD) peers. Method Participants aged 6-11 years completed tasks measuring hearing, language, first formant (F1) discrimination thresholds, partial vowel space, and responses to altered auditory feedback with F1 perturbation. Results Children with DLD tended to compensate more than TD children for the positive F1 manipulation and compensated less than TD children in the negative shift condition. Conclusion Our findings suggest that children with DLD make atypical use of auditory feedback.

The Role of Sensory Feedback in Developmental Stuttering: A Review

Developmental stuttering is a neurodevelopmental disorder that severely affects speech fluency. Multiple lines of evidence point to a role of sensory feedback in the disorder; this has led to a number of theories proposing different disruptions to the use of sensory feedback during speech motor control in people who stutter. The purpose of this review was to bring together evidence from studies using altered auditory feedback paradigms with people who stutter, in order to evaluate the predictions of these different theories. This review highlights converging evidence for particular patterns of differences in the responses of people who stutter to feedback perturbations. The implications for hypotheses on the nature of the disruption to sensorimotor control of speech in the disorder are discussed, with reference to neurocomputational models of speech control (predominantly, the DIVA model; Guenther et al., 2006; Tourville et al., 2008). While some consistent patterns are emerging from this evidence, it is clear that more work in this area is needed with developmental samples in particular, in order to tease apart differences related to symptom onset from those related to compensatory strategies that develop with experience of stuttering.

Linking Cortical Morphology to Interindividual Variability in Auditory Feedback Control of Vocal Production

Speakers regulate vocal motor behaviors in a compensatory manner when perceiving errors in auditory feedback. Little is known, however, about the source of interindividual variability that exists in the degree to which speakers compensate for perceived errors. The present study included 40 young adults to investigate whether individual differences in auditory integration for vocal pitch regulation, as indexed by vocal compensations for pitch perturbations in auditory feedback, can be predicted by cortical morphology as assessed by gray-matter volume, cortical thickness, and surface area in a whole-brain manner. The results showed that greater gray-matter volume in the left inferior parietal lobule and greater cortical thickness and surface area in the left superior/middle temporal gyrus, temporal pole, inferior/superior parietal lobule, and precuneus predicted larger vocal responses. Greater cortical thickness in the right inferior frontal gyrus and superior parietal lobule and surface area in the left precuneus and cuneus were significantly correlated with smaller magnitudes of vocal responses. These findings provide the first evidence that vocal compensations for feedback errors are predicted by the structural morphology of the frontal and tempo-parietal regions, and further our understanding of the neural basis that underlies interindividual variability in auditory-motor control of vocal production.

Neural Development of Speech Sensorimotor Learning

The development of the human brain continues through to early adulthood. It has been suggested that cortical plasticity during this protracted period of development shapes circuits in associative transmodal regions of the brain. Here we considered how cortical plasticity during development might contribute to the coordinated brain activity required for speech motor learning. Specifically, we examined patterns of brain functional connectivity (FC), whose strength covaried with the capacity for speech audio-motor adaptation in children ages 5-12 and in young adults of both sexes. Children and adults showed distinct patterns of the encoding of learning in the brain. Adult performance was associated with connectivity in transmodal regions that integrate auditory and somatosensory information, whereas children rely on basic somatosensory and motor circuits. A progressive reliance on transmodal regions is consistent with human cortical development and suggests that human speech motor adaptation abilities are built on cortical remodeling, which is observable in late childhood and is stabilized in adults.SIGNIFICANCE STATEMENT A protracted period of neuro plasticity during human development is associated with extensive reorganization of associative cortex. We examined how the relationship between FC and speech motor learning capacity are reconfigured in conjunction with this cortical reorganization. Young adults and children aged 5-12 years showed distinctly different patterns. Mature brain networks related to learning included associative cortex, which integrates auditory and somatosensory feedback in speech, whereas the immature networks in children included motor regions of the brain. These patterns are consistent with the cortical reorganization that is initiated in late childhood. The result provides insights into the human biology of speech as well as to the mature neural mechanisms for multisensory integration in motor learning.

Speech auditory-motor adaptation to formant-shifted feedback lacks an explicit component: Reduced adaptation in adults who stutter reflects limitations in implicit sensorimotor learning

The neural mechanisms underlying stuttering remain poorly understood. A large body of work has focused on sensorimotor integration difficulties in individuals who stutter, including recently the capacity for sensorimotor learning. Typically, sensorimotor learning is assessed with adaptation paradigms in which one or more sensory feedback modalities are experimentally perturbed in real time. Our own previous work on speech with perturbed auditory feedback revealed substantial auditory-motor learning limitations in both children and adults who stutter (AWS). It remains unknown, however, which subprocesses of sensorimotor learning are impaired. Indeed, new insights from research on upper limb motor control indicate that sensorimotor learning involves at least two distinct components: (a) an explicit component that includes intentional strategy use and presumably is driven by target error and (b) an implicit component that updates an internal model without awareness of the learner and presumably is driven by sensory prediction error. Here, we attempted to dissociate these components for speech auditory-motor learning in AWS versus adults who do not stutter (AWNS). Our formant-shift auditory-motor adaptation results replicated previous findings that such sensorimotor learning is limited in AWS. Novel findings are that neither control nor stuttering participants reported any awareness of changing their productions in response to the auditory perturbation and that neither group showed systematic drift in auditory target judgments made throughout the adaptation task. These results indicate that speech auditory-motor adaptation to formant-shifted feedback relies exclusively on implicit learning processes. Thus, limited adaptation in AWS reflects poor implicit sensorimotor learning. Speech auditory-motor adaptation to formant-shifted feedback lacks an explicit component: Reduced adaptation in adults who stutter reflects limitations in implicit sensorimotor learning.

Interlocutor accommodation of gradually altered nasal signal levels in a model speaker

BACKGROUND

Phonetic accommodation is observed when interacting speakers gradually converge (or diverge) on phonetic features over the course of a conversation. The present experiment investigated whether gradual changes in the nasal signal levels of a pre-recorded model speaker would lead to accommodation in the nasalance scores of the interlocutor in a speech-shadowing experiment.

METHODS

Twenty female speakers in two groups repeated sentences after a pre-recorded model speaker whose nasal signal level was gradually increased or decreased over the course of the experiment. Outcome measures were the mean nasalance scores at the initial baseline, maximum nasal signal level, minimum nasal signal level and final baseline conditions. The order of presentation of the maximum and minimum nasal signal levels was varied between the two groups.

RESULTS

The results showed a significant effect of condition in F(3) = 2.86, p = 0.045. Both groups of participants demonstrated lower nasalance scores in response to increased nasal signal levels in the model (phonetic divergence). The group that was first presented with the maximum nasal signal levels demonstrated lower nasalance scores for the minimum nasal signal level condition (phonetic convergence).

CONCLUSION

Speakers showed a consistent divergent reaction to a more nasal-sounding model speaker, but their response to a less nasal-sounding model may depend on the order of presentation of the manipulations. More research is needed to investigate the effects of increased versus decreased nasality in the speech of an interlocutor.

Increased speech contrast induced by sensorimotor adaptation to a nonuniform auditory perturbation

When auditory feedback is perturbed in a consistent way, speakers learn to adjust their speech to compensate, a process known as sensorimotor adaptation. Although this paradigm has been highly informative for our understanding of the role of sensory feedback in speech motor control, its ability to induce behaviorally relevant changes in speech that affect communication effectiveness remains unclear. Because reduced vowel contrast contributes to intelligibility deficits in many neurogenic speech disorders, we examine human speakers' ability to adapt to a nonuniform perturbation field that was designed to affect vowel distinctiveness, applying a shift that depended on the vowel being produced. Twenty-five participants were exposed to this "vowel centralization" feedback perturbation in which the first two formant frequencies were shifted toward the center of each participant's vowel space, making vowels less distinct from one another. Speakers adapted to this nonuniform shift, learning to produce corner vowels with increased vowel space area and vowel contrast to partially overcome the perceived centralization. The increase in vowel contrast occurred without a concomitant increase in duration and persisted after the feedback shift was removed, including after a 10-min silent period. These findings establish the validity of a sensorimotor adaptation paradigm to increase vowel contrast, showing that complex, nonuniform alterations to sensory feedback can successfully drive changes relevant to intelligible communication.NEW & NOTEWORTHY To date, the speech motor learning evoked in sensorimotor adaptation studies has had little ecological consequences for communication. By inducing complex, nonuniform acoustic errors, we show that adaptation can be leveraged to cause an increase in speech sound contrast, a change that has the capacity to improve intelligibility. This study is relevant for models of sensorimotor integration across motor domains, showing that complex alterations to sensory feedback can successfully drive changes relevant to ecological behavior.

Individual variability in auditory feedback processing: Responses to real-time formant perturbations and their relation to perceptual acuity

In this study, both between-subject and within-subject variability in speech perception and speech production were examined in the same set of speakers. Perceptual acuity was determined using an ABX auditory discrimination task, whereby speakers made judgments between pairs of syllables on a /ɛ/ to /æ/ acoustic continuum. Auditory feedback perturbations of the first two formants were implemented in a production task to obtain measures of compensation, normal speech production variability, and vowel spacing. Speakers repeated the word "head" 120 times under varying feedback conditions, with the final Hold phase involving the strongest perturbations of +240 Hz in F1 and -300 Hz in F2. Multiple regression analyses were conducted to determine whether individual differences in compensatory behavior in the Hold phase could be predicted by perceptual acuity, speech production variability, and vowel spacing. Perceptual acuity significantly predicted formant changes in F1, but not in F2. These results are discussed in consideration of the importance of using larger sample sizes in the field and developing new methods to explore feedback processing at the individual participant level. The potential positive role of variability in speech motor control is also considered.

Acoustic features of infant-directed speech to infants with hearing loss

This study investigated the effects of hearing loss and hearing experience on the acoustic features of infant-directed speech (IDS) to infants with hearing loss (HL) compared to controls with normal hearing (NH) matched by either chronological or hearing age (experiment 1) and across development in infants with hearing loss as well as the relation between IDS features and infants' developing lexical abilities (experiment 2). Both experiments included detailed acoustic analyses of mothers' productions of the three corner vowels /a, i, u/ and utterance-level pitch in IDS and in adult-directed speech. Experiment 1 demonstrated that IDS to infants with HL was acoustically more variable than IDS to hearing-age matched infants with NH. Experiment 2 yielded no changes in IDS features over development; however, the results did show a positive relationship between formant distances in mothers' speech and infants' concurrent receptive vocabulary size, as well as between vowel hyperarticulation and infants' expressive vocabulary. These findings suggest that despite infants' HL and thus diminished access to speech input, infants with HL are exposed to IDS with generally similar acoustic qualities as are infants with NH. However, some differences persist, indicating that infants with HL might receive less intelligible speech.

Exposure to Auditory Feedback Delay while Speaking Induces Perceptual Habituation but does not Mitigate the Disruptive Effect of Delay on Speech Auditory-motor Learning

Perceiving the sensory consequences of our actions with a delay alters the interpretation of these afferent signals and impacts motor learning. For reaching movements, delayed visual feedback of hand position reduces the rate and extent of visuomotor adaptation, but substantial adaptation still occurs. Moreover, the detrimental effect of visual feedback delay on reach motor learning-selectively affecting its implicit component-can be mitigated by prior habituation to the delay. Auditory-motor learning for speech has been reported to be more sensitive to feedback delay, and it remains unknown whether habituation to auditory delay reduces its negative impact on learning. We investigated whether 30 min of exposure to auditory delay during speaking (a) affects the subjective perception of delay, and (b) mitigates its disruptive effect on speech auditory-motor learning. During a speech adaptation task with real-time perturbation of vowel spectral properties, participants heard this frequency-shifted feedback with no delay, 75 ms delay, or 115 ms delay. In the delay groups, 50% of participants had been exposed to the delay throughout a preceding 30-minute block of speaking whereas the remaining participants completed this block without delay. Although habituation minimized awareness of the delay, no improvement in adaptation to the spectral perturbation was observed. Thus, short-term habituation to auditory feedback delays is not effective in reducing the negative impact of delay on speech auditory-motor adaptation. Combined with previous findings, the strong negative effect of delay and the absence of an influence of delay awareness suggest the involvement of predominantly implicit learning mechanisms in speech.

Contribution of sensory memory to speech motor learning

Speech learning requires precise motor control, but it likewise requires transient storage of information to enable the adjustment of upcoming movements based on the success or failure of previous attempts. The contribution of somatic sensory memory for limb position has been documented in work on arm movement; however, in speech, the sensory support for speech production comes from both somatosensory and auditory inputs, and accordingly sensory memory for either or both of sounds and somatic inputs might contribute to learning. In the present study, adaptation to altered auditory feedback was used as an experimental model of speech motor learning. Participants also underwent tests of both auditory and somatic sensory memory. We found that although auditory memory for speech sounds is better than somatic memory for speechlike facial skin deformations, somatic sensory memory predicts adaptation, whereas auditory sensory memory does not. Thus even though speech relies substantially on auditory inputs and in the present manipulation adaptation requires the minimization of auditory error, it is somatic inputs that provide the memory support for learning.NEW & NOTEWORTHY In speech production, almost everyone achieves an exceptionally high level of proficiency. This is remarkable because speech involves some of the smallest and most carefully timed movements of which we are capable. The present paper demonstrates that sensory memory contributes to speech motor learning. Moreover, we report the surprising result that somatic sensory memory predicts speech motor learning, whereas auditory memory does not.

The Effect of Pitch Auditory Feedback Perturbations on the Production of Anticipatory Phrasal Prominence and Boundary

Purpose In this study, we investigated how the direction and timing of a perturbation in voice pitch auditory feedback during phrasal production modulated the magnitude and latency of the pitch-shift reflex as well as the scaling of acoustic production of anticipatory intonation targets for phrasal prominence and boundary. Method Brief pitch auditory feedback perturbations (±200 cents for 200-ms duration) were applied during the production of a target phrase on the first or the second word of the phrase. To replicate previous work, we first measured the magnitude and latency of the pitch-shift reflex as a function of the direction and timing of the perturbation within the phrase. As a novel approach, we also measured the adjustment in the production of the phrase-final prominent word as a function of perturbation direction and timing by extracting the acoustic correlates of pitch, loudness, and duration. Results The pitch-shift reflex was greater in magnitude after perturbations on the first word of the phrase, replicating the results from Mandarin speakers in an American English-speaking population. Additionally, the production of the phrase-final prominent word was acoustically enhanced (lengthened vowel duration and increased intensity and fundamental frequency) after perturbations earlier in the phrase, but more so after perturbations on the first word in the phrase. Conclusion The timing of the pitch perturbation within the phrase modulated both the magnitude of the pitch-shift reflex and the production of the prominent word, supporting our hypothesis that speakers use auditory feedback to correct for immediate production errors and to scale anticipatory intonation targets during phrasal production.

Unconscious and Distinctive Control of Vocal Pitch and Timbre During Altered Auditory Feedback

Vocal control plays a critical role in smooth social communication. Speakers constantly monitor auditory feedback (AF) and make adjustments when their voices deviate from their intentions. Previous studies have shown that when certain acoustic features of the AF are artificially altered, speakers compensate for this alteration in the opposite direction. However, little is known about how the vocal control system implements compensations for alterations of different acoustic features, and associates them with subjective consciousness. The present study investigated whether compensations for the fundamental frequency (F0), which corresponds to perceived pitch, and formants, which contribute to perceived timbre, can be performed unconsciously and independently. Forty native Japanese speakers received two types of altered AF during vowel production that involved shifts of either only the formant frequencies (formant modification; Fm) or both the pitch and formant frequencies (pitch + formant modification; PFm). For each type, three levels of shift (slight, medium, and severe) in both directions (increase or decrease) were used. After the experiment, participants were tested for whether they had perceived a change in the F0 and/or formants. The results showed that (i) only formants were compensated for in the Fm condition, while both the F0 and formants were compensated for in the PFm condition; (ii) the F0 compensation exhibited greater precision than the formant compensation in PFm; and (iii) compensation occurred even when participants misperceived or could not explicitly perceive the alteration in AF. These findings indicate that non-experts can compensate for both formant and F0 modifications in the AF during vocal production, even when the modifications are not explicitly or correctly perceived, which provides further evidence for a dissociation between conscious perception and action in vocal control. We propose that such unconscious control of voice production may enhance rapid adaptation to changing speech environments and facilitate mutual communication.

Compensatory and adaptive responses to real-time formant shifts in adults and children

Auditory feedback plays an important role in speech motor learning, yet, little is known about the strength of motor learning and feedback control in speech development. This study investigated compensatory and adaptive responses to auditory feedback perturbation in children (aged 4-9 years old) and young adults (aged 18-29 years old). Auditory feedback was perturbed by near-real-time shifting F1 and F2 of the vowel /ɪː/ during the production of consonant-vowel-consonant words. Children were able to compensate and adapt in a similar or larger degree compared to young adults. Higher token-to-token variability was found in children compared to adults but not disproportionately higher during the perturbation phases compared to the unperturbed baseline. The added challenge to auditory-motor integration did not influence production variability in children, and compensation and adaptation effects were found to be strong and sustainable. Significant group differences were absent in the proportions of speakers displaying a compensatory or adaptive response, an amplifying response, or no consistent response. Within these categories, children produced significantly stronger compensatory, adaptive, or amplifying responses, which could be explained by less-ingrained existing representations. The results are interpreted as both auditory-motor integration and learning capacities are stronger in young children compared to adults.

The perils of learning to move while speaking: One-sided interference between speech and visuomotor adaptation

Our understanding of the adaptive processes that shape sensorimotor behavior is largely derived from studying isolated movements. Studies of visuomotor adaptation, in which participants adapt cursor movements to rotations of the cursor's screen position, have led to prominent theories of motor control. In response to changes in visual feedback of movements, explicit (cognitive) and implicit (automatic) learning processes adapt movements to counter errors. However, movements rarely occur in isolation. The extent to which explicit and implicit processes drive sensorimotor adaptation when multiple movements occur simultaneously, as in the real world, remains unclear. Here we address this problem in the context of speech and hand movements. Participants spoke in-time with rapid, hand-driven cursor movements. Using real-time alterations of vowel sound feedback, and visual rotations of the cursor's screen position, we induced sensorimotor adaptation in one or both movements simultaneously. Across three experiments (n = 60, n = 48 and n = 76, respectively), we demonstrate that visuomotor adaptation is markedly impaired by simultaneous speech adaptation, and the impairment is specific to the explicit learning process in visuomotor adaptation. In contrast, visuomotor adaptation had no impact on speech adaptation. The results demonstrate that the explicit learning process in visuomotor adaptation is sensitive to movements in other motor domains. They suggest that some forms of speech adaptation may lack an explicit learning process altogether.

Effects of Knowledge of Task on Control of Oral-Nasal Balance in Speech

<b><i>Introduction:</i></b> Previous research has shown that altering the nasal signal level auditory feedback changed the control of oral-nasal balance in normal speakers. The present study investigated whether knowledge of the task and the instruction not to compensate would change the participants’ response to the manipulation. <b><i>Methods:</i></b> Twenty participants (10 females) in 2 groups continuously repeated a sentence while their nasal signal level was increased or decreased and fed back to them via headphones, so the speakers heard themselves as more or less nasal, respectively. After the first recording session, participants were debriefed about the true nature of the experiment. They were instructed not to compensate in the second recording session. The outcome measures were the percentage changes of nasalance scores from the first baseline. <b><i>Results:</i></b> Statistical analysis using a repeated measures analysis of variance showed an effect of the nasal signal level, <i>F</i>(5,80) = 2.51, <i>p</i> = 0.049, and a nasal signal level by knowledge of task interaction effect, <i>F</i>(5,80) = 3.25, <i>p</i> = 0.019. Post hoc tests showed that the maximum nasal signal level auditory feedback resulted in a significant decrease of nasality from the initial baseline. <b><i>Conclusion:</i></b> Despite knowledge of the task, speakers were unable to resist compensating. As found in previous research, there was a numerically higher compensation response at the maximum than at the minimum nasal signal level auditory feedback condition.

Influence of Altered Auditory Feedback on Oral-Nasal Balance in Speakers of Brazilian Portuguese

Purpose This study explored the role of auditory feedback in the regulation of oral-nasal balance in speakers of Brazilian Portuguese. Method Twenty typical speakers of Brazilian Portuguese (10 male, 10 female) wore a Nasometer headset and headphones while continuously repeating stimuli with oral and nasal sounds. Oral-nasal balance was quantified with nasalance scores. The signals from 2 additional oral and nasal microphones were played back to the participants through the headphones. The relative loudness of the nasal channel in the mix was gradually changed, so that the speakers heard themselves as more or less nasal. Results A repeated-measures analysis of variance of the mean nasalance scores of the stimuli at baseline, minimum, and maximum nasal feedback conditions demonstrated significant effects of nasal feedback condition (p < .0001) and stimuli (p < .0001). Post hoc analyses demonstrated that the mean nasalance scores were lowest for the maximum nasal feedback condition. The scores of the minimum nasal feedback condition were significantly higher than 2 of 3 baseline feedback conditions. The speaking amplitude of the participants did not change between the nasal feedback conditions. Conclusions Increased nasal signal level feedback led to a compensatory adjustment in the opposite direction, confirming that oral-nasal balance is regulated by auditory feedback. However, reduced nasal signal level feedback resulted in a compensatory response that was lower in magnitude. This suggests that, even in Brazilian Portuguese, a language with phonetic and phonological vowel nasalization, decreased nasality was not perceived as critically as increased nasality by the speakers.

Modeling the Role of Sensory Feedback in Speech Motor Control and Learning

Purpose While the speech motor system is sensitive to feedback perturbations, sensory feedback does not seem to be critical to speech motor production. How the speech motor system is able to be so flexible in its use of sensory feedback remains an open question. Method We draw on evidence from a variety of disciplines to summarize current understanding of the sensory systems' role in speech motor control, including both online control and motor learning. We focus particularly on computational models of speech motor control that incorporate sensory feedback, as these models provide clear encapsulations of different theories of sensory systems' function in speech production. These computational models include the well-established directions into velocities of articulators model and computational models that we have been developing in our labs based on the domain-general theory of state feedback control (feedback aware control of tasks in speech model). Results After establishing the architecture of the models, we show that both the directions into velocities of articulators and state feedback control/feedback aware control of tasks models can replicate key behaviors related to sensory feedback in the speech motor system. Although the models agree on many points, the underlying architecture of the 2 models differs in a few key ways, leading to different predictions in certain areas. We cover key disagreements between the models to show the limits of our current understanding and point toward areas where future experimental studies can resolve these questions. Conclusions Understanding the role of sensory information in the speech motor system is critical to understanding speech motor production and sensorimotor learning in healthy speakers as well as in disordered populations. Computational models, with their concrete implementations and testable predictions, are an important tool to understand this process. Comparison of different models can highlight areas of agreement and disagreement in the field and point toward future experiments to resolve important outstanding questions about the speech motor control system.

Decreased Gray-Matter Volume in Insular Cortex as a Correlate of Singers' Enhanced Sensorimotor Control of Vocal Production

Accumulating evidence has shown enhanced sensorimotor control of vocal production as a consequence of extensive singing experience. The neural basis of this ability, however, is poorly understood. Given that the insula mediates motor aspects of vocal production, the present study investigated structural plasticity in insula induced by singing experience and its link to auditory feedback control of vocal production. Voxel-based morphometry (VBM) was used to examine the differences in gray matter (GM) volume in the insula of 21 singers and 21 non-singers. An auditory feedback perturbation paradigm was used to examine the differences in auditory-motor control of vocal production between singers and non-singers. Both groups vocalized sustained vowels while hearing their voice unexpectedly pitch-shifted −50 or −200 cents (200 ms duration). VBM analyses showed that singers exhibited significantly lower GM volumes in the bilateral insula than non-singers. When exposed to pitch perturbations in voice auditory feedback, singers involuntarily compensated for pitch perturbations in voice auditory feedback to a significantly lesser degree than non-singers. Moreover, across the two sizes of pitch perturbations, the magnitudes of vocal compensations were positively correlated with the total regional GM volumes in the bilateral insula. These results indicate that extensive singing training leads to decreased GM volumes in insula and suggest that morphometric plasticity in insula contributes to the enhanced sensorimotor control of vocal production observed in singers.

The influence of coarticulatory and phonemic relations on individual compensatory formant production

Previous auditory perturbation studies have shown that speakers are able to simultaneously use multiple compensatory strategies to produce a certain acoustic target. In the case of formant perturbation, these findings were obtained examining the compensatory production for low vowels /ɛ/ and /æ/. This raises some controversy as more recent research suggests that the contribution of the somatosensory feedback to the production of vowels might differ across phonemes. In particular, the compensatory magnitude to auditory perturbations is expected to be weaker for high vowels compared to low vowels since the former are characterized by larger linguopalatal contact. To investigate this hypothesis, this paper conducted a bidirectional auditory perturbation study in which F2 of the high central vowel /ɨ/ was perturbed in opposing directions depending on the preceding consonant (alveolar vs velar). The consonants were chosen such that speakers' usual coarticulatory patterns were either compatible or incompatible with the required compensatory strategy. The results demonstrate that speakers were able to compensate for applied perturbations even if speakers' compensatory movements resulted in unusual coarticulatory configurations. However, the results also suggest that individual compensatory patterns were influenced by additional perceptual factors attributable to the phonemic space surrounding the target vowel /ɨ/.

Different Responses to Altered Auditory Feedback in Younger and Older Adults Reflect Differences in Lexical Bias

Purpose Previous work has found that both young and older adults exhibit a lexical bias in categorizing speech stimuli. In young adults, this has been argued to be an automatic influence of the lexicon on perceptual category boundaries. Older adults exhibit more top-down biases than younger adults, including an increased lexical bias. We investigated the nature of the increased lexical bias using a sensorimotor adaptation task designed to evaluate whether automatic processes drive this bias in older adults. Method A group of older adults ( n = 27) and younger adults ( n = 35) participated in an altered auditory feedback production task. Participants produced target words and nonwords under altered feedback that affected the 1st formant of the vowel. There were 2 feedback conditions that affected the lexical status of the target, such that target words were shifted to sound more like nonwords (e.g., less-liss) and target nonwords to sound more like words (e.g., kess-kiss). Results A mixed-effects linear regression was used to investigate the magnitude of compensation to altered auditory feedback between age groups and lexical conditions. Over the course of the experiment, older adults compensated (by shifting their production of 1st formant) more to altered auditory feedback when producing words that were shifted toward nonwords ( less-liss) than when producing nonwords that were shifted toward words ( kess-kiss). This is in contrast to younger adults who compensated more to nonwords that were shifted toward words compared to words that were shifted toward nonwords. Conclusion We found no evidence that the increased lexical bias previously observed in older adults is driven by a greater sensitivity to top-down lexical influence on perceptual category boundaries. We suggest the increased lexical bias in older adults is driven by postperceptual processes that arise as a result of age-related cognitive and sensory changes.

Infant-directed speech from seven to nineteen months has similar acoustic properties but different functions

This longitudinal study assessed three acoustic components of maternal infant-directed speech (IDS) - pitch, affect, and vowel hyperarticulation - in relation to infants' age and their expressive vocabulary size. These three individual components were measured in IDS addressed to infants at 7, 9, 11, 15, and 19 months (N = 18). All three components were exaggerated at all ages in mothers' IDS compared to their adult-directed speech. Importantly, the only significant predictor of infants' expressive vocabulary size at 15 and 19 months was vowel hyperarticulation, but only at 9 months and beyond, not at 7 months, and not pitch or affect at any age. These results set apart vowel hyperarticulation in IDS to infants as the critical IDS component for vocabulary development. Thus IDS, specifically the degree of vowel hyperarticulation therein, is a vehicle by which parents can provide the most optimal speech quality for their infants' linguistic and communicative development.