Temporal control and compensation for perturbed voicing feedback

Auditory feedback control in adults who stutter during metronome-paced speech I. Timing Perturbation

PURPOSE

This study determined whether adults who stutter (AWS) exhibit deficits in responding to an auditory feedback timing perturbation, and whether external timing cues, which increase fluency, attenuate any disruptions due to altered temporal auditory feedback.

METHODS

Fifteen AWS and sixteen adults who do not stutter (ANS) read aloud a multisyllabic sentence either with normal pacing or with each syllable paced at the rate of a metronome. On random trials, an auditory feedback timing perturbation was applied, and timing responses were compared between groups and pacing conditions.

RESULTS

Both groups responded to the timing perturbation by delaying subsequent syllable boundaries, and there were no significant differences between groups in either pacing condition. Furthermore, no response differences were found between normally paced and metronome-paced conditions.

CONCLUSION

These findings are interpreted as showing that 1) AWS respond normally to pure timing perturbations, and 2) metronome-paced speech has no effect on online speech timing control as assessed in the present experiment.

Speakers monitor auditory feedback for temporal alignment and linguistically relevant duration

Recent altered auditory feedback studies suggest that speakers adapt to external perturbations to the duration of syllable nuclei and codas, but there is mixed evidence for adaptation of onsets. This study investigates this asymmetry, testing three hypotheses: (1) onsets adapt only if the perturbation produces a categorical error; (2) previously observed increases in vowel duration stem from feedback delays, rather than adaptation to durational perturbations; (3) gestural coordination between onsets and nuclei prevents independent adaptation of each segment. Word-initial consonant targets received shortening perturbations to approximate a different phoneme (cross-category; VOT of /t/ > /d/; duration of /s/ > /z/) or lengthening perturbations to generate a long version of the same phoneme (within-category; /k/ > [khh]; /ʃ/ > [ʃː]). Speakers adapted the duration of both consonants in the cross-category condition; in the within-category condition, only /k/ showed adaptive shortening. Speakers also lengthened all delayed segments while perturbation was active, even when segment duration was not perturbed. Finally, durational changes in syllable onsets and nuclei were not correlated, indicating that speakers can adjust each segment independently. The data suggest that speakers mainly attend to deviations from the predicted timing of motor states but do adjust for durational errors when linguistically relevant.

Temporal malleability to auditory feedback perturbation is modulated by rhythmic abilities and auditory acuity

Auditory feedback perturbation studies have indicated a link between feedback and feedforward mechanisms in speech production when participants compensate for applied shifts. In spectral perturbation studies, speakers with a higher perceptual auditory acuity typically compensate more than individuals with lower acuity. However, the reaction to feedback perturbation is unlikely to be merely a matter of perceptual acuity but also affected by the prediction and production of precise motor action. This interplay between prediction, perception, and motor execution seems to be crucial for the timing of speech and non-speech motor actions. In this study, to examine the relationship between the responses to temporally perturbed auditory feedback and rhythmic abilities, we tested 45 adult speakers on the one hand with a temporal auditory feedback perturbation paradigm, and on the other hand with rhythm perception and production tasks. The perturbation tasks temporally stretched and compressed segments (onset + vowel or vowel + coda) in fluent speech in real-time. This technique sheds light on the temporal representation and the production flexibility of timing mechanisms in fluent speech with respect to the structure of the syllable. The perception tasks contained staircase paradigms capturing duration discrimination abilities and beat-alignment judgments. The rhythm production tasks consisted of finger tapping tasks taken from the BAASTA tapping battery and additional speech tapping tasks. We found that both auditory acuity and motor stability in finger tapping affected responses to temporal auditory feedback perturbation. In general, speakers with higher auditory acuity and higher motor variability compensated more. However, we observed a different weighting of auditory acuity and motor stability dependent on the prosodic structure of the perturbed sequence and the nature of the response as purely online or adaptive. These findings shed light on the interplay of phonological structure with feedback and feedforward integration for timing mechanisms in speech.

Perturbing the consistency of auditory feedback in speech

Sensory information, including auditory feedback, is used by talkers to maintain fluent speech articulation. Current models of speech motor control posit that speakers continually adjust their motor commands based on discrepancies between the sensory predictions made by a forward model and the sensory consequences of their speech movements. Here, in two within-subject design experiments, we used a real-time formant manipulation system to explore how reliant speech articulation is on the accuracy or predictability of auditory feedback information. This involved introducing random formant perturbations during vowel production that varied systematically in their spatial location in formant space (Experiment 1) and temporal consistency (Experiment 2). Our results indicate that, on average, speakers’ responses to auditory feedback manipulations varied based on the relevance and degree of the error that was introduced in the various feedback conditions. In Experiment 1, speakers’ average production was not reliably influenced by random perturbations that were introduced every utterance to the first (F1) and second (F2) formants in various locations of formant space that had an overall average of 0 Hz. However, when perturbations were applied that had a mean of +100 Hz in F1 and −125 Hz in F2, speakers demonstrated reliable compensatory responses that reflected the average magnitude of the applied perturbations. In Experiment 2, speakers did not significantly compensate for perturbations of varying magnitudes that were held constant for one and three trials at a time. Speakers’ average productions did, however, significantly deviate from a control condition when perturbations were held constant for six trials. Within the context of these conditions, our findings provide evidence that the control of speech movements is, at least in part, dependent upon the reliability and stability of the sensory information that it receives over time.

Auditory Feedback Is Used for Adaptation and Compensation in Speech Timing

Purpose Real-time altered feedback has demonstrated a key role for auditory feedback in both online feedback control and in updating feedforward control for future utterances. The aim of this study was to examine adaptation in response to temporal perturbation using real-time perturbation of ongoing speech. Method Twenty native English speakers with no reported history of speech or hearing disorders participated in this study. The study consisted of four word blocks, using the phrases "a capper," "a gapper," "a sapper," and "a zapper" (due to issues with the implementation of perturbation, "gapper" was excluded from analysis). In each block, participants completed a baseline phase (30 trials of veridical feedback), a ramp phase (feedback perturbation increasing to maximum over 30 trials), a hold phase (60 trials with perturbation held at maximum), and a washout phase (30 trials, feedback abruptly returned to veridical feedback). Word-initial consonant targets (voice onset time for /k, g/ and fricative duration for /s, z/) were lengthened, and the following stressed vowel (/æ/) was shortened. Results Overall, speakers did not adapt the production of their consonants but did lengthen their vowel production in response to shortening. Vowel lengthening showed continued aftereffects during the early portion of the washout phase. Although speakers did not adapt absolute consonant durations, consonant duration was reduced as a proportion of the total syllable duration. This is consistent with previous research that suggests that speakers attend to proportional durations rather than absolute durations. Conclusion These results indicate that speakers actively monitor proportional durations and update the temporal dynamics of planning units extending beyond a single segment.

Compensation to real-time temporal auditory feedback perturbation depends on syllable position

Auditory feedback perturbations involving spectral shifts indicated a crucial contribution of auditory feedback to planning and execution of speech. However, much less is known about the contribution of auditory feedback with respect to temporal properties of speech. The current study aimed at providing insight into the representation of temporal properties of speech and the relevance of auditory feedback for speech timing. Real-time auditory feedback perturbations were applied in the temporal domain, viz., stretching and compressing of consonant-consonant-vowel (CCV) durations in onset + nucleus vs vowel-consonant-consonant (VCC) durations in nucleus + coda. Since CCV forms a gesturally more cohesive and stable structure than VCC, greater articulatory adjustments to nucleus + coda (VCC) perturbation were expected. The results show that speakers compensate for focal temporal feedback alterations. Responses to VCC perturbation were greater than to CCV perturbation, suggesting less deformability of onsets when confronted with temporally perturbed auditory feedback. Further, responses to CCV perturbation rather reflected within-trial reactive compensation, whereas VCC compensation was more pronounced and indicative of adaptive behavior. Accordingly, planning and execution of temporal properties of speech are indeed guided by auditory feedback, but the precise nature of the reaction to perturbations is linked to the structural position in the syllable and the associated feedforward timing strategies.

Differential contributions of the two cerebral hemispheres to temporal and spectral speech feedback control

Proper speech production requires auditory speech feedback control. Models of speech production associate this function with the right cerebral hemisphere while the left hemisphere is proposed to host speech motor programs. However, previous studies have investigated only spectral perturbations of the auditory speech feedback. Since auditory perception is known to be lateralized, with right-lateralized analysis of spectral features and left-lateralized processing of temporal features, it is unclear whether the observed right-lateralization of auditory speech feedback processing reflects a preference for speech feedback control or for spectral processing in general. Here we use a behavioral speech adaptation experiment with dichotically presented altered auditory feedback and an analogous fMRI experiment with binaurally presented altered feedback to confirm a right hemisphere preference for spectral feedback control and to reveal a left hemisphere preference for temporal feedback control during speaking. These results indicate that auditory feedback control involves both hemispheres with differential contributions along the spectro-temporal axis.

Supra-Segmental Changes in Speech Production as a Result of Spectral Feedback Degradation: Comparison with Lombard Speech

Perturbations to acoustic speech feedback have been typically localized to specific phonetic characteristics, for example, fundamental frequency (F0) or the first two formants (F1/F2), or affect all aspects of the speech signal equally, for example, via the addition of background noise. This paper examines the consequences of a more selective global perturbation: real-time cochlear implant (CI) simulation of acoustic speech feedback. Specifically, we examine the potential similarity between speakers' response to noise vocoding and the characteristics of Lombard speech. An acoustic analysis of supra-segmental characteristics in speaking rate, F0 production, and voice amplitude revealed changes that paralleled the Lombard effect in some domains but not others. Two studies of speech intelligibility complemented the acoustic analysis, finding that intelligibility significantly decreased as a result of CI simulation of speaker feedback. Together, the results point to differences in speakers' responses to these two superficially similar feedback manipulations. In both cases we see a complex, multi-faceted behavior on the part of talkers. We argue that more instances of global perturbation and broader response assessment are needed to determine whether such complexity is present in other feedback manipulations or if it represents a relatively rare exception to the typical compensatory feedback response.

Modulation of auditory-motor learning in response to formant perturbation as a function of delayed auditory feedback

The interaction of language production and perception has been substantiated by empirical studies where speakers compensate their speech articulation in response to the manipulated sound of their voice heard in real-time as auditory feedback. A recent study by Max and Maffett [(2015). Neurosci. Lett. 591, 25-29] reported an absence of compensation (i.e., auditory-motor learning) for frequency-shifted formants when auditory feedback was delayed by 100 ms. In the present study, the effect of auditory feedback delay was studied when only the first formant was manipulated while delaying auditory feedback systematically. In experiment 1, a small yet significant compensation was observed even with 100 ms of auditory delay unlike the past report. This result suggests that the tolerance of feedback delay depends on different types of auditory errors being processed. In experiment 2, it was revealed that the amount of formant compensation had an inverse linear relationship with the amount of auditory delay. One of the speculated mechanisms to account for these results is that as auditory delay increases, undelayed (and unperturbed) somatosensory feedback is given more preference for accuracy control of vowel formants.

Occlusion effect on compensatory formant production and voice amplitude in response to real-time perturbation

The importance of auditory feedback for controlling speech articulation has been substantiated by the use of the real-time auditory perturbation paradigm. With this paradigm, speakers receive their own manipulated voice signal in real-time while they produce a simple speech segment. In response, they spontaneously compensate for the manipulation. In the case of vowel formant control, various studies have reported behavioral and neural mechanisms of how auditory feedback is processed for compensatory behavior. However, due to technical limitations such as avoiding an electromagnetic artifact or metal transducers near a scanner, some studies require foam tip insert earphones. These earphones occlude the ear canal, and may cause more energy of the unmanipulated first formant to reach the cochlea through bone conduction and thus confound the effect of formant manipulation. Moreover, amplification of lower frequencies due to occluded ear canals may influence speakers' voice amplitude. The current study examined whether using circumaural headphones and insert earphones would elicit different compensatory speech production when speakers' first formant was manipulated in real-time. The results of the current study showed that different headphones did not elicit different compensatory formant production. Voice amplitude results were varied across different vowels examined; however, voice amplitude tended to decrease with the introduction of F1 perturbation.

Compensation for a lip-tube perturbation in 4-year-olds: Articulatory, acoustic, and perceptual data analyzed in comparison with adults

The nature of the speech goal in children was investigated in a study of compensation strategies for a lip-tube perturbation. Acoustic, articulatory, and perceptual analyses of the vowels /y/ and /u/ produced by ten 4-year-old French speakers and ten adult French speakers were conducted under two conditions: normal and with a large tube inserted between the lips. Ultrasound and acoustic recordings of isolated vowels were made in the normal condition before any perturbation, for each of the trials in the perturbed condition, and in the normal condition after the perturbed trials. Data revealed that adult participants moved their tongues in the perturbed condition more than children did. The perturbation was generally at least partly compensated for during the perturbed trials in adults, but children did not show a typical learning effect. In particular, unsystematic improvements were observed during the sequence of perturbed trials, and after-effects were not clear in the articulatory domain. This suggests that children may establish associative links between multisensory phonemic representations and articulatory maneuvers, but those links may mainly rely on trial-to-trial, error-based feedback correction mechanisms rather than on the internal model of the speech production apparatus, as they are in adults.

EFFECT OF DELAYED AUDITORY FEEDBACK, SPEECH RATE, AND SEX ON SPEECH PRODUCTION

Perturbations in Delayed Auditory Feedback (DAF) and speech rate were examined as sources of disruptions in speech between men and women. Fluent adult men (n = 16) and women (n = 16) spoke at a normal and an imposed fast rate of speech with 0, 25, 50, 100, and 200 msec. DAF. The syllable rate significantly increased when participants were instructed to speak at a fast rate, and the syllable rate decreased with increasing DAF delays. Men's speech rate was significantly faster during the fast speech rate condition with a 200 msec. DAF. Disfluencies increased with increasing DAF delay. Significantly more disfluency occurred at delays of 25 and 50 msec. at the fast rate condition, while more disfluency occurred at 100 and 200 msec. in normal rate conditions. Men and women did not display differences in the number of disfluencies. These findings demonstrate sex differences in susceptibility to perturbations in DAF and speech rate suggesting feedforward/feedback subsystems that monitor vocalizations may be different between sexes.