Talkers alter vowel production in response to real-time formant perturbation even when instructed not to compensate

Supra-Segmental Changes in Speech Production as a Result of Spectral Feedback Degradation: Comparison with Lombard Speech

Perturbations to acoustic speech feedback have been typically localized to specific phonetic characteristics, for example, fundamental frequency (F0) or the first two formants (F1/F2), or affect all aspects of the speech signal equally, for example, via the addition of background noise. This paper examines the consequences of a more selective global perturbation: real-time cochlear implant (CI) simulation of acoustic speech feedback. Specifically, we examine the potential similarity between speakers' response to noise vocoding and the characteristics of Lombard speech. An acoustic analysis of supra-segmental characteristics in speaking rate, F0 production, and voice amplitude revealed changes that paralleled the Lombard effect in some domains but not others. Two studies of speech intelligibility complemented the acoustic analysis, finding that intelligibility significantly decreased as a result of CI simulation of speaker feedback. Together, the results point to differences in speakers' responses to these two superficially similar feedback manipulations. In both cases we see a complex, multi-faceted behavior on the part of talkers. We argue that more instances of global perturbation and broader response assessment are needed to determine whether such complexity is present in other feedback manipulations or if it represents a relatively rare exception to the typical compensatory feedback response.

Variability of articulator positions and formants across nine English vowels

Speech, though communicative, is quite variable both in articulation and acoustics, and it has often been claimed that articulation is more variable. Here we compared variability in articulation and acoustics for 32 speakers in the x-ray microbeam database (XRMB; Westbury, 1994). Variability in tongue, lip and jaw positions for nine English vowels (/u, ʊ, æ, ɑ, ʌ, ɔ, ε, ɪ, i/) was compared to that of the corresponding formant values. The domains were made comparable by creating three-dimensional spaces for each: the first three principal components from an analysis of a 14-dimensional space for articulation, and an F1xF2xF3 space for acoustics. More variability occurred in the articulation than the acoustics for half of the speakers, while the reverse was true for the other half. Individual tokens were further from the articulatory median than the acoustic median for 40-60% of tokens across speakers. A separate analysis of three non-low front vowels (/ε, ɪ, i/, for which the XRMB system provides the most direct articulatory evidence) did not differ from the omnibus analysis. Speakers tended to be either more or less variable consistently across vowels. Across speakers, there was a positive correlation between articulatory and acoustic variability, both for all vowels and for just the three non-low front vowels. Although the XRMB is an incomplete representation of articulation, it nonetheless provides data for direct comparisons between articulatory and acoustic variability that have not been reported previously. The results indicate that articulation is not more variable than acoustics, that speakers had relatively consistent variability across vowels, and that articulatory and acoustic variability were related for the vowels themselves.

Cortico-cerebellar Networks Drive Sensorimotor Learning in Speech

The motor cortex and cerebellum are thought to be critical for learning and maintaining motor behaviors. Here we use transcranial direct current stimulation (tDCS) to test the role of the motor cortex and cerebellum in sensorimotor learning in speech. During productions of “head,” “bed,” and “dead,” the first formant of the vowel sound was altered in real time toward the first formant of the vowel sound in “had,” “bad,” and “dad.” Compensatory changes in first and second formant production were used as a measure of motor adaptation. tDCS to either the motor cortex or the cerebellum improved sensorimotor learning in speech compared with sham stimulation ( n = 20 in each group). However, in the case of cerebellar tDCS, production changes were restricted to the source of the acoustical error (i.e., the first formant). Motor cortex tDCS drove production changes that offset errors in the first formant, but unlike cerebellar tDCS, adaptive changes in the second formant also occurred. The results suggest that motor cortex and cerebellar tDCS have both shared and dissociable effects on motor adaptation. The study provides initial causal evidence in speech production that the motor cortex and the cerebellum support different aspects of sensorimotor learning. We propose that motor cortex tDCS drives sensorimotor learning toward previously learned patterns of movement, whereas cerebellar tDCS focuses sensorimotor learning on error correction.

Online Adaptation to Altered Auditory Feedback Is Predicted by Auditory Acuity and Not by Domain-General Executive Control Resources

When a speaker's auditory feedback is altered, he adapts for the perturbation by altering his own production, which demonstrates the role of auditory feedback in speech motor control. In the present study, we explored the role of auditory acuity and executive control in this process. Based on the DIVA model and the major cognitive control models, we expected that higher auditory acuity, and better executive control skills would predict larger adaptation to the alteration. Thirty-six Spanish native speakers performed an altered auditory feedback experiment, executive control (numerical Stroop, Simon and Flanker) tasks, and auditory acuity tasks (loudness, pitch, and melody pattern discrimination). In the altered feedback experiment, participants had to produce the pseudoword “pep” (/pep/) while perceiving their auditory feedback in real time through earphones. The auditory feedback was first unaltered and then progressively altered in F1 and F2 dimensions until maximal alteration (F1 −150 Hz; F2 +300 Hz). The normalized distance of maximal adaptation ranged from 4 to 137 Hz (median of 75 ± 36). The different measures of auditory acuity were significant predictors of adaptation, while individual measures of cognitive function skills (obtained from the executive control tasks) were not. Better auditory discriminators adapted more to the alteration. We conclude that adaptation to altered auditory feedback is very well-predicted by general auditory acuity, as suggested by the DIVA model. In line with the framework of motor-control models, no specific claim on the implication of executive resources in speech motor control can be made.

Auditory-Motor Control of Vocal Production during Divided Attention: Behavioral and ERP Correlates

When people hear unexpected perturbations in auditory feedback, they produce rapid compensatory adjustments of their vocal behavior. Recent evidence has shown enhanced vocal compensations and cortical event-related potentials (ERPs) in response to attended pitch feedback perturbations, suggesting that this reflex-like behavior is influenced by selective attention. Less is known, however, about auditory-motor integration for voice control during divided attention. The present cross-modal study investigated the behavioral and ERP correlates of auditory feedback control of vocal pitch production during divided attention. During the production of sustained vowels, 32 young adults were instructed to simultaneously attend to both pitch feedback perturbations they heard and flashing red lights they saw. The presentation rate of the visual stimuli was varied to produce a low, intermediate, and high attentional load. The behavioral results showed that the low-load condition elicited significantly smaller vocal compensations for pitch perturbations than the intermediate-load and high-load conditions. As well, the cortical processing of vocal pitch feedback was also modulated as a function of divided attention. When compared to the low-load and intermediate-load conditions, the high-load condition elicited significantly larger N1 responses and smaller P2 responses to pitch perturbations. These findings provide the first neurobehavioral evidence that divided attention can modulate auditory feedback control of vocal pitch production.

Sensorimotor adaptation of voice fundamental frequency in Parkinson's disease

OBJECTIVE

This study examined adaptive responses to auditory perturbation of fundamental frequency (fo) in speakers with Parkinson's disease (PD) and control speakers.

METHOD

Sixteen speakers with PD and nineteen control speakers produced sustained vowels while they received perturbed auditory feedback (i.e., fo shifted upward or downward). Speakers' pitch acuity was quantified using a just-noticeable-difference (JND) paradigm. Twelve listeners provided estimates of the speech intelligibility for speakers with PD.

RESULTS

Fifteen responses from each speaker group for each shift direction were included in analyses. While control speakers generally showed consistent adaptive responses opposing the perturbation, speakers with PD showed no compensation on average, with individual PD speakers showing highly variable responses. In the PD group, the degree of compensation was not significantly correlated with age, disease progression, pitch acuity, or intelligibility.

CONCLUSIONS

These findings indicate reduced adaptation to sustained fo perturbation and higher variability in PD compared to control participants. No significant differences were seen in pitch acuity between groups, suggesting that the fo adaptation deficit in PD is not the result of purely perceptual mechanisms.

SIGNIFICANCE

These results suggest there is an impairment in vocal motor control in PD. Building on these results, contributions can be made to developing targeted voice treatments for PD.

Influence of Altered Auditory Feedback on Oral-Nasal Balance in Speech

PURPOSE

This study explored the role of auditory feedback in the regulation of oral-nasal balance in speech.

METHOD

Twenty typical female speakers wore a Nasometer 6450 (KayPentax) headset and headphones while continuously repeating a sentence with oral and nasal sounds. Oral-nasal balance was quantified with nasalance scores. The signals from 2 additional oral and nasal microphones were played back to the participants through the headphones. The relative loudness of the nasal channel in the mix was gradually changed so that the speakers heard themselves as more or less nasal. An additional amplitude control group of 9 female speakers completed the same task while hearing themselves louder or softer in the headphones.

RESULTS

A repeated-measures analysis of variance of the mean nasalance scores of the stimulus sentence at baseline, minimum, and maximum nasal feedback conditions demonstrated a significant effect of the nasal feedback condition. Post hoc analyses found that the mean nasalance scores were lowest for the maximum nasal feedback condition. The scores of the minimum nasal feedback condition were significantly higher than 2 of the 3 baseline feedback conditions. The amplitude control group did not show any effects of volume changes on nasalance scores.

CONCLUSIONS

Increased nasal feedback led to a compensatory adjustment in the opposite direction, confirming that oral-nasal balance is regulated by auditory feedback. However, a lack of nasal feedback did not lead to a consistent compensatory response of similar magnitude.

Vowel generalization and its relation to adaptation during perturbations of auditory feedback

Repeated perturbations of auditory feedback during vowel production elicit changes not only in the production of the perturbed vowel (adaptation) but also in the production of nearby vowels that were not perturbed (generalization). The finding that adaptation generalizes to other, nonperturbed vowels suggests that sensorimotor representations for vowels are not independent; instead, the goals for producing any one vowel may depend in part on the goals for other vowels. The present study investigated the dependence or independence of vowel representations by evaluating adaptation and generalization in two groups of speakers exposed to auditory perturbations of their first formant (F1) during different vowels. The speakers in both groups who adapted to the perturbation exhibited generalization in two nonperturbed vowels that were produced under masking noise. Correlation testing was performed to evaluate the relations between adaptation and generalization as well as between the generalization in the two nonperturbed vowels. These tests identified significant coupling between the F1 changes of adjacent vowels but not nonadjacent vowels. The pattern of correlation findings indicates that generalization was due in part to feedforward representations that are partly shared across adjacent vowels, possibly to maintain their acoustic contrast.NEW & NOTEWORTHY Speech adaptations to alterations, or perturbations, of auditory feedback have provided important insights into sensorimotor representations underlying speech. One finding from these studies that is yet to be accounted for is vowel generalization, which describes the effects of repeated perturbations to one vowel on the production of other vowels that were not perturbed. The present study used correlation testing to quantify the effects of changes in a perturbed vowel on neighboring (i.e., similar) nonperturbed vowels. The results identified significant correlations between the changes of adjacent, but not nonadjacent, vowel pairs. This finding suggests that generalization is partly a response to adaptation and not solely due to the auditory perturbation.

Top-Down Modulation of Auditory-Motor Integration during Speech Production: The Role of Working Memory

Although working memory (WM) is considered as an emergent property of the speech perception and production systems, the role of WM in sensorimotor integration during speech processing is largely unknown. We conducted two event-related potential experiments with female and male young adults to investigate the contribution of WM to the neurobehavioural processing of altered auditory feedback during vocal production. A delayed match-to-sample task that required participants to indicate whether the pitch feedback perturbations they heard during vocalizations in test and sample sequences matched, elicited significantly larger vocal compensations, larger N1 responses in the left middle and superior temporal gyrus, and smaller P2 responses in the left middle and superior temporal gyrus, inferior parietal lobule, somatosensory cortex, right inferior frontal gyrus, and insula compared with a control task that did not require memory retention of the sequence of pitch perturbations. On the other hand, participants who underwent extensive auditory WM training produced suppressed vocal compensations that were correlated with improved auditory WM capacity, and enhanced P2 responses in the left middle frontal gyrus, inferior parietal lobule, right inferior frontal gyrus, and insula that were predicted by pretraining auditory WM capacity. These findings indicate that WM can enhance the perception of voice auditory feedback errors while inhibiting compensatory vocal behavior to prevent voice control from being excessively influenced by auditory feedback. This study provides the first evidence that auditory-motor integration for voice control can be modulated by top-down influences arising from WM, rather than modulated exclusively by bottom-up and automatic processes.SIGNIFICANCE STATEMENT One outstanding question that remains unsolved in speech motor control is how the mismatch between predicted and actual voice auditory feedback is detected and corrected. The present study provides two lines of converging evidence, for the first time, that working memory cannot only enhance the perception of vocal feedback errors but also exert inhibitory control over vocal motor behavior. These findings represent a major advance in our understanding of the top-down modulatory mechanisms that support the detection and correction of prediction-feedback mismatches during sensorimotor control of speech production driven by working memory. Rather than being an exclusively bottom-up and automatic process, auditory-motor integration for voice control can be modulated by top-down influences arising from working memory.

Linguistic initiation signals increase auditory feedback error correction

Previous research has shown that speakers can adapt their speech in a flexible manner as a function of a variety of contextual and task factors. While it is known that speech tasks may play a role in speech motor behavior, it remains to be explored if the manner in which the speaking action is initiated can modify low-level, automatic control of vocal motor action. In this study, the nature (linguistic vs non-linguistic) and modality (auditory vs visual) of the go signal (i.e., the prompts) was manipulated in an otherwise identical vocal production task. Participants were instructed to produce the word "head" when prompted, and the auditory feedback they were receiving was altered by systematically changing the first formants of the vowel /ε/ in real time using a custom signal processing system. Linguistic prompts induced greater corrective behaviors to the acoustic perturbations than non-linguistic prompts. This suggests that the accepted variance for the intended speech sound decreases when external linguistic templates are provided to the speaker. Overall, this result shows that the automatic correction of vocal errors is influenced by flexible, context-dependant mechanisms.

Compensations to auditory feedback perturbations in congenitally blind and sighted speakers: Acoustic and articulatory data

This study investigated the effects of visual deprivation on the relationship between speech perception and production by examining compensatory responses to real-time perturbations in auditory feedback. Specifically, acoustic and articulatory data were recorded while sighted and congenitally blind French speakers produced several repetitions of the vowel /ø/. At the acoustic level, blind speakers produced larger compensatory responses to altered vowels than their sighted peers. At the articulatory level, blind speakers also produced larger displacements of the upper lip, the tongue tip, and the tongue dorsum in compensatory responses. These findings suggest that blind speakers tolerate less discrepancy between actual and expected auditory feedback than sighted speakers. The study also suggests that sighted speakers have acquired more constrained somatosensory goals through the influence of visual cues perceived in face-to-face conversation, leading them to tolerate less discrepancy between expected and altered articulatory positions compared to blind speakers and thus resulting in smaller observed compensatory responses.

Modulation of auditory-motor learning in response to formant perturbation as a function of delayed auditory feedback

The interaction of language production and perception has been substantiated by empirical studies where speakers compensate their speech articulation in response to the manipulated sound of their voice heard in real-time as auditory feedback. A recent study by Max and Maffett [(2015). Neurosci. Lett. 591, 25-29] reported an absence of compensation (i.e., auditory-motor learning) for frequency-shifted formants when auditory feedback was delayed by 100 ms. In the present study, the effect of auditory feedback delay was studied when only the first formant was manipulated while delaying auditory feedback systematically. In experiment 1, a small yet significant compensation was observed even with 100 ms of auditory delay unlike the past report. This result suggests that the tolerance of feedback delay depends on different types of auditory errors being processed. In experiment 2, it was revealed that the amount of formant compensation had an inverse linear relationship with the amount of auditory delay. One of the speculated mechanisms to account for these results is that as auditory delay increases, undelayed (and unperturbed) somatosensory feedback is given more preference for accuracy control of vowel formants.

Occlusion effect on compensatory formant production and voice amplitude in response to real-time perturbation

The importance of auditory feedback for controlling speech articulation has been substantiated by the use of the real-time auditory perturbation paradigm. With this paradigm, speakers receive their own manipulated voice signal in real-time while they produce a simple speech segment. In response, they spontaneously compensate for the manipulation. In the case of vowel formant control, various studies have reported behavioral and neural mechanisms of how auditory feedback is processed for compensatory behavior. However, due to technical limitations such as avoiding an electromagnetic artifact or metal transducers near a scanner, some studies require foam tip insert earphones. These earphones occlude the ear canal, and may cause more energy of the unmanipulated first formant to reach the cochlea through bone conduction and thus confound the effect of formant manipulation. Moreover, amplification of lower frequencies due to occluded ear canals may influence speakers' voice amplitude. The current study examined whether using circumaural headphones and insert earphones would elicit different compensatory speech production when speakers' first formant was manipulated in real-time. The results of the current study showed that different headphones did not elicit different compensatory formant production. Voice amplitude results were varied across different vowels examined; however, voice amplitude tended to decrease with the introduction of F1 perturbation.

Mothers speak differently to infants at-risk for dyslexia

Dyslexia is a neurodevelopmental disorder manifested in deficits in reading and spelling skills that is consistently associated with difficulties in phonological processing. Dyslexia is genetically transmitted, but its manifestation in a particular individual is thought to depend on the interaction of epigenetic and environmental factors. We adopt a novel interactional perspective on early linguistic environment and dyslexia by simultaneously studying two pre-existing factors, one maternal and one infant, that may contribute to these interactions; and two behaviours, one maternal and one infant, to index the effect of these factors. The maternal factor is whether mothers are themselves dyslexic or not (with/without dyslexia) and the infant factor is whether infants are at-/not-at family risk for dyslexia (due to their mother or father being dyslexic). The maternal behaviour is mothers' infant-directed speech (IDS), which typically involves vowel hyperarticulation, thought to benefit speech perception and language acquisition. The infant behaviour is auditory perception measured by infant sensitivity to amplitude envelope rise time, which has been found to be reduced in dyslexic children. Here, at-risk infants showed significantly poorer acoustic sensitivity than not-at-risk infants and mothers only hyperarticulated vowels to infants who were not at-risk for dyslexia. Mothers' own dyslexia status had no effect on IDS quality. Parental speech input is thus affected by infant risk status, with likely consequences for later linguistic development.

The Curious Incident of Attention in Multisensory Integration: Bottom-up vs. Top-down

The role attention plays in our experience of a coherent, multisensory world is still controversial. On the one hand, a subset of inputs may be selected for detailed processing and multisensory integration in a top-down manner, i.e., guidance of multisensory integration by attention. On the other hand, stimuli may be integrated in a bottom-up fashion according to low-level properties such as spatial coincidence, thereby capturing attention. Moreover, attention itself is multifaceted and can be describedviaboth top-down and bottom-up mechanisms. Thus, the interaction between attention and multisensory integration is complex and situation-dependent. The authors of this opinion paper are researchers who have contributed to this discussion from behavioural, computational and neurophysiological perspectives. We posed a series of questions, the goal of which was to illustrate the interplay between bottom-up and top-down processes in various multisensory scenarios in order to clarify the standpoint taken by each author and with the hope of reaching a consensus. Although divergence of viewpoint emerges in the current responses, there is also considerable overlap: In general, it can be concluded that the amount of influence that attention exerts on MSI depends on the current task as well as prior knowledge and expectations of the observer. Moreover stimulus properties such as the reliability and salience also determine how open the processing is to influences of attention.

Formant compensation for auditory feedback with English vowels

Past studies have shown that speakers spontaneously adjust their speech acoustics in response to their auditory feedback perturbed in real time. In the case of formant perturbation, the majority of studies have examined speaker's compensatory production using the English vowel /ɛ/ as in the word "head." Consistent behavioral observations have been reported, and there is lively discussion as to how the production system integrates auditory versus somatosensory feedback to control vowel production. However, different vowels have different oral sensation and proprioceptive information due to differences in the degree of lingual contact or jaw openness. This may in turn influence the ways in which speakers compensate for auditory feedback. The aim of the current study was to examine speakers' compensatory behavior with six English monophthongs. Specifically, the current study tested to see if "closed vowels" would show less compensatory production than "open vowels" because closed vowels' strong lingual sensation may richly specify production via somatosensory feedback. Results showed that, indeed, speakers exhibited less compensatory production with the closed vowels. Thus sensorimotor control of vowels is not fixed across all vowels; instead it exerts different influences across different vowels.

Effects of real-time cochlear implant simulation on speech production

Investigations using normal-hearing subjects listening to simulations of cochlear implant (CI) acoustic processing have provided substantial information about the impact of these distorted listening conditions on the accuracy of auditory perception, but extensions of this method to the domain of speech production have been limited. In the present study, a portable, real-time vocoder was used to simulate conditions of CI auditory feedback during speech production in NH subjects. Acoustic-phonetic characteristics of sibilant fricatives, aspirated stops, and F1/F2 vowel qualities were analyzed for changes as a result of CI simulation of acoustic speech feedback. Significant changes specific to F1 were observed; speakers reduced their phonological vowel height contrast, typically via talker-specific raising of the low vowels [æ] and [ɑ] or lowering of high vowels [i] and [u]. Comparisons to the results of both localized feedback perturbation procedures and investigations of speech production in deaf adults with CIs are discussed.

Precise feedback control underlies sensorimotor learning in speech

Acquiring the skill of speaking in another language, or for that matter a child's learning to talk, does not follow a single recipe. People learn by variable amounts. A major component of speech learnability seems to be sensing precise feedback errors to correct subsequent utterances that help maintain speech goals. We have tested this idea in a speech motor learning paradigm under altered auditory feedback, in which subjects repeated a word while their auditory feedback was changed online. Subjects learned the task to variable degrees, with some simply failing to learn. We assessed feedback contribution by computing one-lag covariance between formant trajectories of the current feedback and the following utterance that was found to be a significant predictor of learning. Our findings rely on a novel use of information-rich formant trajectories in evaluating speech motor learning and argue for their relevance in auditory speech goals of vowel sounds.

Auditory feedback perturbation in children with developmental speech sound disorders

BACKGROUND/PURPOSE

Several studies indicate a close relation between auditory and speech motor functions in children with speech sound disorders (SSD). The aim of this study was to investigate the ability to compensate and adapt for perturbed auditory feedback in children with SSD compared to age-matched normally developing children.

METHOD

17 normally developing children aged 4.1-8.7 years (mean=5.5, SD=1.4), and 11 children with SSD aged 3.9-7.5 years (mean=5.1, SD=1.0) participated in the study. Auditory feedback was perturbed by real-time shifting the first and second formant of the vowel /e/ during the production of CVC words in a five-step paradigm (practice/familiarization; start/baseline; ramp; hold; end/release).

RESULTS

At the group level, the normally developing children were better able to compensate and adapt, adjusting their formant frequencies in the direction opposite to the perturbation, while the group of children with SSD followed (amplifying) the perturbation. However, large individual differences lie underneath. Furthermore, strong correlations were found between the amount of compensation and performance on oral motor movement non-word repetition tasks.

CONCLUSIONS

Results suggested that while most children with SSD can detect incongruencies in auditory feedback and can adapt their target representations, they are unable to compensate for perturbed auditory feedback. These findings suggest that impaired auditory-motor integration may play a key role in SSD.

LEARNING OUTCOMES

The reader will be able to: (1) describe the potential role of auditory feedback control in developmental speech disorders (SSD); (2) identify the neural control subsystems involved in feedback based speech motor control; (3) describe the differences between compensation and adaptation for perturbed auditory feedback; (4) explain why auditory-motor integration may play a key role in SSD.

Multiple factors are involved in the dysarthria associated with Parkinson's disease: a review with implications for clinical practice and research

PURPOSE

Motor speech abnormalities are highly common and debilitating in individuals with idiopathic Parkinson's disease (IPD). These abnormalities, collectively termed hypokinetic dysarthria (HKD), have been traditionally attributed to hypokinesia and bradykinesia secondary to muscle rigidity and dopamine deficits. However, the role of rigidity and dopamine in the development of HKD is far from clear. The purpose of the present study was to offer an alternative view of the factors underlying HKD.

METHOD

The authors conducted an extensive, but not exhaustive, review of the literature to examine the evidence for the traditional view versus the alternative view.

RESULTS

The review suggests that HKD is a highly complex and variable phenomenon including multiple factors, such as scaling and maintaining movement amplitude and effort; preplanning and initiation of movements; internal cueing; sensory and temporal processing; automaticity; emotive vocalization; and attention to action (vocal vigilance). Although not part of the dysarthria, nonmotor factors, such as depression, aging, and cognitive-linguistic abnormalities, are likely to contribute to the overall speech symptomatology associated with IPD.

CONCLUSION

These findings have important implications for clinical practice and research.

Brief periods of auditory perceptual training can determine the sensory targets of speech motor learning

The perception of speech is notably malleable in adults, yet alterations in perception seem to have little impact on speech production. However, we hypothesized that speech perceptual training might immediately influence speech motor learning. To test this, we paired a speech perceptual-training task with a speech motor-learning task. Subjects performed a series of perceptual tests designed to measure and then manipulate the perceptual distinction between the words head and had. Subjects then produced head with the sound of the vowel altered in real time so that they heard themselves through headphones producing a word that sounded more like had. In support of our hypothesis, the amount of motor learning in response to the voice alterations depended on the perceptual boundary acquired through perceptual training. The studies show that plasticity in adults' speech perception can have immediate consequences for speech production in the context of speech learning.

Temporal control and compensation for perturbed voicing feedback

Previous research employing a real-time auditory perturbation paradigm has shown that talkers monitor their own speech attributes such as fundamental frequency, vowel intensity, vowel formants, and fricative noise as part of speech motor control. In the case of vowel formants or fricative noise, what was manipulated is spectral information about the filter function of the vocal tract. However, segments can be contrasted by parameters other than spectral configuration. It is possible that the feedback system monitors phonation timing in the way it does spectral information. This study examined whether talkers exhibit a compensatory behavior when manipulating information about voicing. When talkers received feedback of the cognate of the intended voicing category (saying "tipper" while hearing "dipper" or vice versa), they changed the voice onset time and in some cases the following vowel.

Auditory feedback of one's own voice is used for high-level semantic monitoring: the "self-comprehension" hypothesis

What would it be like if we said one thing, and heard ourselves saying something else? Would we notice something was wrong? Or would we believe we said the thing we heard? Is feedback of our own speech only used to detect errors, or does it also help to specify the meaning of what we say? Comparator models of self-monitoring favor the first alternative, and hold that our sense of agency is given by the comparison between intentions and outcomes, while inferential models argue that agency is a more fluent construct, dependent on contextual inferences about the most likely cause of an action. In this paper, we present a theory about the use of feedback during speech. Specifically, we discuss inferential models of speech production that question the standard comparator assumption that the meaning of our utterances is fully specified before articulation. We then argue that auditory feedback provides speakers with a channel for high-level, semantic “self-comprehension”. In support of this we discuss results using a method we recently developed called Real-time Speech Exchange (RSE). In our first study using RSE (Lind et al., in press) participants were fitted with headsets and performed a computerized Stroop task. We surreptitiously recorded words they said, and later in the test we played them back at the exact same time that the participants uttered something else, while blocking the actual feedback of their voice. Thus, participants said one thing, but heard themselves saying something else. The results showed that when timing conditions were ideal, more than two thirds of the manipulations went undetected. Crucially, in a large proportion of the non-detected manipulated trials, the inserted words were experienced as self-produced by the participants. This indicates that our sense of agency for speech has a strong inferential component, and that auditory feedback of our own voice acts as a pathway for semantic monitoring. We believe RSE holds great promise as a tool for investigating the role of auditory feedback during speech, and we suggest a number of future studies to serve this purpose.

Expert pianists do not listen: the expertise-dependent influence of temporal perturbation on the production of sequential movements

Auditory information plays an important role in fine motor control such as speech and musical performance. The purpose of this study was to assess expertise-dependent differences in the role of temporal information of auditory feedback in the production of sequential movements. Differences in motor responses to the transient delay of tone production during musical performance between expert pianists and non-musicians were evaluated. Compared to expert pianists, non-musicians showed more pronounced movement disruptions following the delayed auditory feedback. For example, in response to a perturbation the inter-keystroke interval was prolonged and the key-press was longer in non-musicians, while the expert pianist marginally shortened both measures. These distinct differences between groups suggest that extensive musical training influences feedback control in sequential finger movements. Furthermore, there was a significant positive correlation between the age at which the expert pianists commenced their musical training and the amount of disruption. Overall, these findings suggest that expert pianists have a higher level of robustness against perturbations and depend less on auditory feedback during the performance of sequential movements.

Language dependent vowel representation in speech production

The representation of speech goals was explored using an auditory feedback paradigm. When talkers produce vowels the formant structure of which is perturbed in real time, they compensate to preserve the intended goal. When vowel formants are shifted up or down in frequency, participants change the formant frequencies in the opposite direction to the feedback perturbation. In this experiment, the specificity of vowel representation was explored by examining the magnitude of vowel compensation when the second formant frequency of a vowel was perturbed for speakers of two different languages (English and French). Even though the target vowel was the same for both language groups, the pattern of compensation differed. French speakers compensated to smaller perturbations and made larger compensations overall. Moreover, French speakers modified the third formant in their vowels to strengthen the compensation even though the third formant was not perturbed. English speakers did not alter their third formant. Changes in the perceptual goodness ratings by the two groups of participants were consistent with the threshold to initiate vowel compensation in production. These results suggest that vowel goals not only specify the quality of the vowel but also the relationship of the vowel to the vowel space of the spoken language.

Multivoxel patterns reveal functionally differentiated networks underlying auditory feedback processing of speech

The everyday act of speaking involves the complex processes of speech motor control. An important component of control is monitoring, detection, and processing of errors when auditory feedback does not correspond to the intended motor gesture. Here we show, using fMRI and converging operations within a multivoxel pattern analysis framework, that this sensorimotor process is supported by functionally differentiated brain networks. During scanning, a real-time speech-tracking system was used to deliver two acoustically different types of distorted auditory feedback or unaltered feedback while human participants were vocalizing monosyllabic words, and to present the same auditory stimuli while participants were passively listening. Whole-brain analysis of neural-pattern similarity revealed three functional networks that were differentially sensitive to distorted auditory feedback during vocalization, compared with during passive listening. One network of regions appears to encode an "error signal" regardless of acoustic features of the error: this network, including right angular gyrus, right supplementary motor area, and bilateral cerebellum, yielded consistent neural patterns across acoustically different, distorted feedback types, only during articulation (not during passive listening). In contrast, a frontotemporal network appears sensitive to the speech features of auditory stimuli during passive listening; this preference for speech features was diminished when the same stimuli were presented as auditory concomitants of vocalization. A third network, showing a distinct functional pattern from the other two, appears to capture aspects of both neural response profiles. Together, our findings suggest that auditory feedback processing during speech motor control may rely on multiple, interactive, functionally differentiated neural systems.

Individual variability in delayed auditory feedback effects on speech fluency and rate in normally fluent adults

PURPOSE

Delayed auditory feedback (DAF) is known to induce stuttering-like disfluencies (SLDs) and cause speech rate reductions in normally fluent adults, but the reason for speech disruptions is not fully known, and individual variation has not been well characterized. Studying individual variation in susceptibility to DAF may identify factors that predispose an individual to be more or less dependent on auditory feedback.

METHOD

Participants were 62 normally fluent adults. Each participant performed a spontaneous speech task in 250-ms DAF and amplified nondelayed auditory feedback (NAF) conditions. SLDs, other disfluencies (ODs), speech errors (SEs), and articulation rate (AR) were measured under each condition.

RESULTS

In the DAF condition, SLDs and SEs significantly increased, and AR decreased. Sex had a limited effect in that men exhibited higher rates of ODs and faster AR than women. More important, parametric cluster analysis identified that 2- and 3-subgroup solutions reveal important variation that differentiates tendencies toward disfluency changes and rate reduction under DAF, which are theoretically and empirically preferred to a single-group solution.

CONCLUSION

Individual variability in response to DAF may be accounted for by subgroups of individuals. This suggests that certain normally fluent individuals could be more dependent on intact feedback to maintain fluency.

Auditory-motor adaptation to frequency-altered auditory feedback occurs when participants ignore feedback

Background

Auditory feedback is important for accurate control of voice fundamental frequency (F₀). The purpose of this study was to address whether task instructions could influence the compensatory responding and sensorimotor adaptation that has been previously found when participants are presented with a series of frequency-altered feedback (FAF) trials. Trained singers and musically untrained participants (nonsingers) were informed that their auditory feedback would be manipulated in pitch while they sang the target vowel [/ɑ /]. Participants were instructed to either ‘compensate’ for, or ‘ignore’ the changes in auditory feedback. Whole utterance auditory feedback manipulations were either gradually presented (‘ramp’) in -2 cent increments down to -100 cents (1 semitone) or were suddenly (’constant‘) shifted down by 1 semitone.

Results

Results indicated that singers and nonsingers could not suppress their compensatory responses to FAF, nor could they reduce the sensorimotor adaptation observed during both the ramp and constant FAF trials.

Conclusions

Compared to previous research, these data suggest that musical training is effective in suppressing compensatory responses only when FAF occurs after vocal onset (500-2500 ms). Moreover, our data suggest that compensation and adaptation are automatic and are influenced little by conscious control.

Opposing and following vocal responses to pitch-shifted auditory feedback: evidence for different mechanisms of voice pitch control

The present study describes a technique for analysis of vocal responses to auditory feedback pitch perturbations in which individual trials are first sorted according to response direction and then separately averaged in groups of upward or downward responses. In experiment 1, the stimulus direction was predictable (all upward) but magnitude was randomized between +100, +200, or +500 cents (unpredictable). Results showed that pitch-shift stimuli (PSS) of +100 and +200 cents elicited significantly larger opposing (compensatory) responses than +500 cent stimuli, but no such effect was observed for "following" responses. In experiment 2, subjects were tested in three blocks of trials where for the first two, PSS magnitude and direction were predictable (block 1+100 and block 2-100 cents), and in block 3, the magnitude was predictable (±100 cents) but direction was randomized (upward or downward). Results showed there were slightly more opposing than following responses for predictable PSS direction, but randomized directions led to significantly more opposing than following responses. Results suggest that predictability of stimulus direction and magnitude can modulate vocal responses to feedback pitch perturbations. The function and causes of the opposing and following responses are unknown, but there may be two different neural mechanisms involved in their production.

A cross-language study of compensation in response to real-time formant perturbation

Past studies have shown that when formants are perturbed in real time, speakers spontaneously compensate for the perturbation by changing their formant frequencies in the opposite direction to the perturbation. Further, the pattern of these results suggests that the processing of auditory feedback error operates at a purely acoustic level. This hypothesis was tested by comparing the response of three language groups to real-time formant perturbations, (1) native English speakers producing an English vowel /ε/, (2) native Japanese speakers producing a Japanese vowel (/e([inverted perpendicular])/), and (3) native Japanese speakers learning English, producing /ε/. All three groups showed similar production patterns when F1 was decreased; however, when F1 was increased, the Japanese groups did not compensate as much as the native English speakers. Due to this asymmetry, the hypothesis that the compensatory production for formant perturbation operates at a purely acoustic level was rejected. Rather, some level of phonological processing influences the feedback processing behavior.

Effects of voice harmonic complexity on ERP responses to pitch-shifted auditory feedback

OBJECTIVE

The present study investigated the neural mechanisms of voice pitch control for different levels of harmonic complexity in the auditory feedback.

METHODS

Event-related potentials (ERPs) were recorded in response to+200 cents pitch perturbations in the auditory feedback of self-produced natural human vocalizations, complex and pure tone stimuli during active vocalization and passive listening conditions.

RESULTS

During active vocal production, ERP amplitudes were largest in response to pitch shifts in the natural voice, moderately large for non-voice complex stimuli and smallest for the pure tones. However, during passive listening, neural responses were equally large for pitch shifts in voice and non-voice complex stimuli but still larger than that for pure tones.

CONCLUSIONS

These findings suggest that pitch change detection is facilitated for spectrally rich sounds such as natural human voice and non-voice complex stimuli compared with pure tones. Vocalization-induced increase in neural responses for voice feedback suggests that sensory processing of naturally-produced complex sounds such as human voice is enhanced by means of motor-driven mechanisms (e.g. efference copies) during vocal production.

SIGNIFICANCE

This enhancement may enable the audio-vocal system to more effectively detect and correct for vocal errors in the feedback of natural human vocalizations to maintain an intended vocal output for speaking.

Speaker compensation for local perturbation of fricative acoustic feedback

Feedback perturbation studies of speech acoustics have revealed a great deal about how speakers monitor and control their productions of segmental (e.g., formant frequencies) and non-segmental (e.g., pitch) linguistic elements. The majority of previous work, however, overlooks the role of acoustic feedback in consonant production and makes use of acoustic manipulations that effect either entire utterances or the entire acoustic signal, rather than more temporally and phonetically restricted alterations. This study, therefore, seeks to expand the feedback perturbation literature by examining perturbation of consonant acoustics that is applied in a time-restricted and phonetically specific manner. The spectral center of the alveopalatal fricative [∫] produced in vowel-fricative-vowel nonwords was incrementally raised until it reached the potential for [s]-like frequencies, but the characteristics of high-frequency energy outside the target fricative remained unaltered. An "offline," more widely accessible signal processing method was developed to perform this manipulation. The local feedback perturbation resulted in changes to speakers' fricative production that were more variable, idiosyncratic, and restricted than the compensation seen in more global acoustic manipulations reported in the literature. Implications and interpretations of the results, as well as future directions for research based on the findings, are discussed.

Probing the independence of formant control using altered auditory feedback

Two auditory feedback perturbation experiments were conducted to examine the nature of control of the first two formants in vowels. In the first experiment, talkers heard their auditory feedback with either F1 or F2 shifted in frequency. Talkers altered production of the perturbed formant by changing its frequency in the opposite direction to the perturbation but did not produce a correlated alteration of the unperturbed formant. Thus, the motor control system is capable of fine-grained independent control of F1 and F2. In the second experiment, a large meta-analysis was conducted on data from talkers who received feedback where both F1 and F2 had been perturbed. A moderate correlation was found between individual compensations in F1 and F2 suggesting that the control of F1 and F2 is processed in a common manner at some level. While a wide range of individual compensation magnitudes were observed, no significant correlations were found between individuals' compensations and vowel space differences. Similarly, no significant correlations were found between individuals' compensations and variability in normal vowel production. Further, when receiving normal auditory feedback, most of the population exhibited no significant correlation between the natural variation in production of F1 and F2.

Adaptive auditory feedback control of the production of formant trajectories in the Mandarin triphthong /iau/ and its pattern of generalization

In order to test whether auditory feedback is involved in the planning of complex articulatory gestures in time-varying phonemes, the current study examined native Mandarin speakers' responses to auditory perturbations of their auditory feedback of the trajectory of the first formant frequency during their production of the triphthong /iau/. On average, subjects adaptively adjusted their productions to partially compensate for the perturbations in auditory feedback. This result indicates that auditory feedback control of speech movements is not restricted to quasi-static gestures in monophthongs as found in previous studies, but also extends to time-varying gestures. To probe the internal structure of the mechanisms of auditory-motor transformations, the pattern of generalization of the adaptation learned on the triphthong /iau/ to other vowels with different temporal and spatial characteristics (produced only under masking noise) was tested. A broad but weak pattern of generalization was observed; the strength of the generalization diminished with increasing dissimilarity from /iau/. The details and implications of the pattern of generalization are examined and discussed in light of previous sensorimotor adaptation studies of both speech and limb motor control and a neurocomputational model of speech motor control.

Role of auditory feedback in the control of successive keystrokes during piano playing

The purpose of this study was to elucidate the role of auditory feedback derived from one keystroke in the control of the rhythmicity and velocity of successive keystrokes during piano playing. We examined the effects of transient auditory perturbations with respect to the pitch, loudness, and timing of one tone on subsequent keystrokes while six pianists played short excerpts from three simple musical pieces having different tempi ("event rates"). Immediately after a delay in tone production, the inter-keystroke interval became shorter. This compensatory action depended on the tempo, being most prominent at the medium tempo. This indicates that temporal information provided by auditory feedback is utilized to regulate the timing of movement elements produced in a sequence. We also found that the keystroke velocity changed after the timing, pitch, or loudness of a tone was altered, although the response differed depending on the type of perturbation. While delaying the timing or altering the pitch led to an increase in the velocity, altering the loudness changed the velocity in an inconsistent manner. Furthermore, perturbing a tone elicited by the right hand also affected the rhythmicity and velocity of keystrokes with the left hand, indicating that bimanual coordination of tone production was maintained. Finally, altering the pitch sometimes resulted in striking an incorrect key, mostly in the slow piece, emphasizing the importance of pitch information for accurate planning and execution of sequential piano keystrokes.

Compensations in response to real-time formant perturbations of different magnitudes

Previous auditory perturbation studies have demonstrated that talkers spontaneously compensate for real-time formant-shifts by altering formant production in a manner opposite to the perturbation. Here, two experiments were conducted to examine the effect of amplitude of perturbation on the compensatory behavior for the vowel /epsilon/. In the first experiment, 20 male talkers received three step-changes in acoustic feedback: F1 was increased by 50, 100, and 200 Hz, while F2 was simultaneously decreased by 75, 125, and 250 Hz. In the second experiment, 21 male talkers received acoustic feedback in which the shifts in F1 and F2 were incremented by +4 and -5 Hz on each utterance to a maximum of +350 and -450 Hz, respectively. In both experiments, talkers altered production of F1 and F2 in a manner opposite to that of the formant-shift perturbation. Compensation was approximately 25%-30% of the perturbation magnitude for shifts in F1 and F2 up to 200 and 250 Hz, respectively. As larger shifts were applied, compensation reached a plateau and then decreased. The similarity of results across experiments suggests that the compensatory response is dependent on the perturbation magnitude but not on the rate at which the perturbation is introduced.

Control of vocalization at utterance onset and mid-utterance: different mechanisms for different goals

A large body of evidence suggests that the motor system maintains a forward model that predicts the sensory outcome of movements. When sensory feedback does not match the predicted consequences, a compensatory response corrects for the motor error and the forward model is updated to prevent future errors. Like other motor behaviours, vocalization relies on sensory feedback for the maintenance of forward models. In this study, we used a frequency altered feedback (FAF) paradigm to study the role of auditory feedback in the control of vocal pitch (F0). We adapted subjects to a one semitone shift and induced a perturbation by briefly removing the altered feedback. This was compared to a control block in which a 1 semitone perturbation was introduced into an unshifted trial, or trials were randomly shifted up 1 semitone, and a perturbation was introduced by removing the feedback alteration. The compensation response to mid-utterance perturbations was identical in all conditions, and was always smaller than the compensation to a shift at utterance onset. These results are explained by a change in the control strategy at utterance onset and mid-utterance. At utterance onset, auditory feedback is compared to feedback predicted by a forward model to ensure the pitch goal is achieved. However, after utterance onset, the control strategy switches and stabilization is maintained by comparing feedback to previous F0 production.