Modulation of Auditory Responses to Speech vs. Nonspeech Stimuli during Speech Movement Planning

Perceptual formant discrimination during speech movement planning

Evoked potential studies have shown that speech planning modulates auditory cortical responses. The phenomenon's functional relevance is unknown. We tested whether, during this time window of cortical auditory modulation, there is an effect on speakers' perceptual sensitivity for vowel formant discrimination. Participants made same/different judgments for pairs of stimuli consisting of a pre-recorded, self-produced vowel and a formant-shifted version of the same production. Stimuli were presented prior to a "go" signal for speaking, prior to passive listening, and during silent reading. The formant discrimination stimulus /uh/ was tested with a congruent productions list (words with /uh/) and an incongruent productions list (words without /uh/). Logistic curves were fitted to participants' responses, and the just-noticeable difference (JND) served as a measure of discrimination sensitivity. We found a statistically significant effect of condition (worst discrimination before speaking) without congruency effect. Post-hoc pairwise comparisons revealed that JND was significantly greater before speaking than during silent reading. Thus, formant discrimination sensitivity was reduced during speech planning regardless of the congruence between discrimination stimulus and predicted acoustic consequences of the planned speech movements. This finding may inform ongoing efforts to determine the functional relevance of the previously reported modulation of auditory processing during speech planning.

Neurophysiological evidence of motor preparation in inner speech and the effect of content predictability

Self-generated overt actions are preceded by a slow negativity as measured by electroencephalogram, which has been associated with motor preparation. Recent studies have shown that this neural activity is modulated by the predictability of action outcomes. It is unclear whether inner speech is also preceded by a motor-related negativity and influenced by the same factor. In three experiments, we compared the contingent negative variation elicited in a cue paradigm in an active vs. passive condition. In Experiment 1, participants produced an inner phoneme, at which an audible phoneme whose identity was unpredictable was concurrently presented. We found that while passive listening elicited a late contingent negative variation, inner speech production generated a more negative late contingent negative variation. In Experiment 2, the same pattern of results was found when participants were instead asked to overtly vocalize the phoneme. In Experiment 3, the identity of the audible phoneme was made predictable by establishing probabilistic expectations. We observed a smaller late contingent negative variation in the inner speech condition when the identity of the audible phoneme was predictable, but not in the passive condition. These findings suggest that inner speech is associated with motor preparatory activity that may also represent the predicted action-effects of covert actions.

The initiation of a hand grip is delayed by silently reading an incompatible syllable

The movements of phonation structures (e.g., tongue) have been shown to facilitate compatible hand movements. For example, reaction time (RT) of precision and power hand grips (made with tips of thumb and finger vs. whole hand) are shortened with the production of syllables that share similar action features (e.g., employing the proximal vs. dorsal portion of the tongue, respectively). This effect is coined the articulation-grip correspondence (AGC) effect. However, it is not known if the AGC effect is due to action facilitation vs. interference, and if such facilitation/ interference is due to covertly or overtly reading the syllable. To answer the associated empirical questions, the present experiment involved participants initiating a precision or power grip without the covert/ overt reading of a syllable, or while covertly or overtly reading the syllable /ti/ or /ka/. In both the covert and overt reading conditions, there were longer RTs for precision grips with the syllable /ka/ than /ti/, and there were longer RTs for power grips with the syllable /ti/. In contrast, the syllable /ti/ or /ka/ did not alter precision or power grip RTs, respectively. These findings support the notion of articulation-grip interference but not facilitation and that such interference can be observed with covert (silent) reading.

Perceptual formant discrimination during speech movement planning

Evoked potential studies have shown that speech planning modulates auditory cortical responses. The phenomenon's functional relevance is unknown. We tested whether, during this time window of cortical auditory modulation, there is an effect on speakers' perceptual sensitivity for vowel formant discrimination. Participants made same/different judgments for pairs of stimuli consisting of a pre-recorded, self-produced vowel and a formant-shifted version of the same production. Stimuli were presented prior to a "go" signal for speaking, prior to passive listening, and during silent reading. The formant discrimination stimulus /uh/ was tested with a congruent productions list (words with /uh/) and an incongruent productions list (words without /uh/). Logistic curves were fitted to participants' responses, and the just-noticeable difference (JND) served as a measure of discrimination sensitivity. We found a statistically significant effect of condition (worst discrimination before speaking) without congruency effect. Post-hoc pairwise comparisons revealed that JND was significantly greater before speaking than during silent reading. Thus, formant discrimination sensitivity was reduced during speech planning regardless of the congruence between discrimination stimulus and predicted acoustic consequences of the planned speech movements. This finding may inform ongoing efforts to determine the functional relevance of the previously reported modulation of auditory processing during speech planning.

A Computational Model for Estimating the Speech Motor System's Sensitivity to Auditory Prediction Errors

Purpose The speech motor system uses feedforward and feedback control mechanisms that are both reliant on prediction errors. Here, we developed a state-space model to estimate the error sensitivity of the control systems. We examined (a) whether the model accounts for the error sensitivity of the control systems and (b) whether the two systems have similar error sensitivity. Method Participants (N = 50) completed an adaptation paradigm, in which their first and second formants were perturbed such that a participant's /ε/ would sound like her /ӕ/. We measured adaptive responses to the perturbations at early (0-80 ms) and late (220-300 ms) time points relative to the onset of the perturbations. As data-driven correlates of the error sensitivity of the feedforward and feedback systems, we used the average early responses and difference responses (i.e., late minus early responses), respectively. We fitted the state-space model to participants' adaptive responses and used the model's parameters as model-based estimates of error sensitivity. Results We found that the late responses were larger than the early responses. Additionally, the model-based estimates of error sensitivity strongly correlated with the data-driven estimates. However, the data-driven and model-based estimates of error sensitivity of the feedforward system did not correlate with those of the feedback system. Conclusions Overall, our results suggested that the dynamics of adaptive responses as well as error sensitivity of the control systems can be accurately predicted by the model. Furthermore, our results suggested that the feedforward and feedback control systems function independently. Supplemental Material https://doi.org/10.23641/asha.14669808.

Compensatory Responses to Formant Perturbations Proportionally Decrease as Perturbations Increase

Purpose We continuously monitor our speech output to detect potential errors in our productions. When we encounter errors, we rapidly change our speech output to compensate for the errors. However, it remains unclear whether we adjust the magnitude of our compensatory responses based on the characteristics of errors. Method Participants (N = 30 adults) produced monosyllabic words containing /ɛ/ (/hɛp/, /hɛd/, /hɛk/) while receiving perturbed or unperturbed auditory feedback. In the perturbed trials, we applied two different types of formant perturbations: (a) the F1 shift, in which the first formant of /ɛ/ was increased, and (b) the F1-F2 shift, in which the first formant was increased and the second formant was decreased to make a participant's /ɛ/ sound like his or her /æ/. In each perturbation condition, we applied three participant-specific perturbation magnitudes (0.5, 1.0, and 1.5 ɛ-æ distance). Results Compensatory responses to perturbations with the magnitude of 1.5 ɛ-æ were proportionally smaller than responses to perturbation magnitudes of 0.5 ɛ-æ. Responses to the F1-F2 shift were larger than responses to the F1 shift regardless of the perturbation magnitude. Additionally, compensatory responses for /hɛd/ were smaller than responses for /hɛp/ and /hɛk/. Conclusions Overall, these results suggest that the brain uses its error evaluation to determine the extent of compensatory responses. The brain may also consider categorical errors and phonemic environments (e.g., articulatory configurations of the following phoneme) to determine the magnitude of its compensatory responses to auditory errors.

The Orofacial Somatosensory System Is Modulated During Speech Planning and Production

Purpose In our previous studies, we showed that the brain modulates the auditory system, and the modulation starts during speech planning. However, it remained unknown whether the brain uses similar mechanisms to modulate the orofacial somatosensory system. Here, we developed a novel behavioral paradigm to (a) examine whether the somatosensory system is modulated during speech planning and (b) determine the somatosensory modulation's time course during planning and production. Method Participants (N = 20) completed two experiments in which we applied electrical current stimulation to the lower lip to induce somatosensory sensation. In the first experiment, we used a staircase method (one-up, four-down) to determine each participant's perceptual threshold at rest (i.e., the stimulus that the participant detected on 85% of trials). In the second experiment, we estimated each participant's detection ratio of electrical stimuli (with a magnitude equivalent of their perceptual threshold) delivered at various time points before speaking and during a control condition (silent reading). Results We found that the overall detection ratio in the silent reading condition remained unchanged relative to the detection ratio at rest. Approximately 536 ms before speech onset, the detection ratio in the speaking condition was similar to that in the silent reading condition; however, the detection ratio in the speaking condition gradually started to decrease and reached its lowest level at 58 ms before speech onset. Conclusions Overall, we provided compelling behavioral evidence that, as the speech motor system prepares speech movements, it also modulates the orofacial somatosensory system in a temporally specific manner.

Changes of orofacial somatosensory attenuation during speech production

Modulation of auditory activity occurs before and during voluntary speech movement. However, it is unknown whether orofacial somatosensory input is modulated in the same manner. The current study examined whether or not the somatosensory event-related potentials (ERPs) in response to facial skin stretch are changed during speech and nonspeech production tasks. Specifically, we compared ERP changes to somatosensory stimulation for different orofacial postures and speech utterances. Participants produced three different vowel sounds (voicing) or non-speech oral tasks in which participants maintained a similar posture without voicing. ERP's were recorded from 64 scalp sites in response to the somatosensory stimulation under six task conditions (three vowels × voicing/posture) and compared to a resting baseline condition. The first negative peak for the vowel /u/ was reliably reduced from the baseline in both the voicing and posturing tasks, but the other conditions did not differ. The second positive peak was reduced for all voicing tasks compared to the posturing tasks. The results suggest that the sensitivity of somatosensory ERP to facial skin deformation is modulated by the task and that somatosensory processing during speaking may be modulated differently relative to phonetic identity.

Corollary Discharge Versus Efference Copy: Distinct Neural Signals in Speech Preparation Differentially Modulate Auditory Responses

Actions influence sensory processing in a complex way to shape behavior. For example, during actions, a copy of motor signals-termed "corollary discharge" (CD) or "efference copy" (EC)-can be transmitted to sensory regions and modulate perception. However, the sole inhibitory function of the motor copies is challenged by mixed empirical observations as well as multifaceted computational demands for behaviors. We hypothesized that the content in the motor signals available at distinct stages of actions determined the nature of signals (CD vs. EC) and constrained their modulatory functions on perceptual processing. We tested this hypothesis using speech in which we could precisely control and quantify the course of action. In three electroencephalography (EEG) experiments using a novel delayed articulation paradigm, we found that preparation without linguistic contents suppressed auditory responses to all speech sounds, whereas preparing to speak a syllable selectively enhanced the auditory responses to the prepared syllable. A computational model demonstrated that a bifurcation of motor signals could be a potential algorithm and neural implementation to achieve the distinct functions in the motor-to-sensory transformation. These results suggest that distinct motor signals are generated in the motor-to-sensory transformation and integrated with sensory input to modulate perception.

Limited Pre-Speech Auditory Modulation in Individuals Who Stutter: Data and Hypotheses

Purpose We review and interpret our recent series of studies investigating motor-to-auditory influences during speech movement planning in fluent speakers and speakers who stutter. In those studies, we recorded auditory evoked potentials in response to probe tones presented immediately prior to speaking or at the equivalent time in no-speaking control conditions. As a measure of pre-speech auditory modulation (PSAM), we calculated changes in auditory evoked potential amplitude in the speaking conditions relative to the no-speaking conditions. Whereas adults who do not stutter consistently showed PSAM, this phenomenon was greatly reduced or absent in adults who stutter. The same between-group difference was observed in conditions where participants expected to hear their prerecorded speech played back without actively producing it, suggesting that the speakers who stutter use inefficient forward modeling processes rather than inefficient motor command generation processes. Compared with fluent participants, adults who stutter showed both less PSAM and less auditory-motor adaptation when producing speech while exposed to formant-shifted auditory feedback. Across individual participants, however, PSAM and auditory-motor adaptation did not correlate in the typically fluent group, and they were negatively correlated in the stuttering group. Interestingly, speaking with a consistent 100-ms delay added to the auditory feedback signal-normalized PSAM in speakers who stutter, and there no longer was a between-group difference in this condition. Conclusions Combining our own data with human and animal neurophysiological evidence from other laboratories, we interpret the overall findings as suggesting that (a) speech movement planning modulates auditory processing in a manner that may optimize its tuning characteristics for monitoring feedback during speech production and, (b) in conditions with typical auditory feedback, adults who stutter do not appropriately modulate the auditory system prior to speech onset. Lack of modulation of speakers who stutter may lead to maladaptive feedback-driven movement corrections that manifest themselves as repetitive movements or postural fixations.

Speaking-Induced Suppression of the Auditory Cortex in Humans and Its Relevance to Schizophrenia

Speaking-induced suppression (SIS) is the phenomenon that the sounds one generates by overt speech elicit a smaller neurophysiological response in the auditory cortex than comparable sounds that are externally generated. SIS is a specific example of the more general phenomenon of self-suppression. SIS has been well established in nonhuman animals and is believed to involve the action of corollary discharges. This review summarizes, first, the evidence for SIS in heathy human participants, where it has been most commonly assessed with electroencephalography and/or magnetoencephalography using an experimental paradigm known as "Talk-Listen"; and second, the growing number of Talk-Listen studies that have reported subnormal levels of SIS in patients with schizophrenia. This result is theoretically significant, as it provides a plausible explanation for some of the most distinctive and characteristic symptoms of schizophrenia, namely the first-rank symptoms. In particular, while the failure to suppress the neural consequences of self-generated movements (such as those associated with overt speech) provides a prima facie explanation for delusions of control, the failure to suppress the neural consequences of self-generated inner speech provides a plausible explanation for certain classes of auditory-verbal hallucinations, such as audible thoughts. While the empirical evidence for a relationship between SIS and the first-rank symptoms is currently limited, I predict that future studies with more sensitive experimental designs will confirm its existence. Establishing the existence of a causal, mechanistic relationship would represent a major step forward in our understanding of schizophrenia, which is a necessary precursor to the development of novel treatments.

Interference in speaking while hearing and vice versa

Even when speakers are not actively doing another task, they can be interfered in their speech planning by concurrent auditory stimuli. In this study, we used picture naming with passive hearing, or active listening, combined to high-density electroencephalographic (EEG) recordings to investigate the locus and origin of interference on speech production. Participants named pictures while ignoring (or paying attention to) auditory syllables presented at different intervals (+150 ms, +300 ms or +450 ms). Interference of passive hearing was observed at all positive stimulus onset asynchronies (SOA) including when distractors appeared 450 ms after picture onset. Analyses of ERPs and microstates revealed modulations appearing in a time-window close to verbal response onset likely relating to post-lexical planning processes. A shift of latency of the N1 auditory component for syllables displayed 450 ms after picture onset relative to hearing in isolation was also observed. Data from picture naming with active listening to auditory syllables also pointed to post-lexical interference. The present study suggests that, beyond the lexical stage, post-lexical processes can be interfered and that the reciprocal interference between utterance planning and hearing relies on attentional demand and possibly competing neural substrates.

Production Variability and Categorical Perception of Vowels Are Strongly Linked

Theoretical models of speech production suggest that the speech motor system (SMS) uses auditory goals to determine errors in its auditory output during vowel production. This type of error calculation indicates that within-speaker production variability of a given vowel is related to the size of the vowel’s auditory goal. However, emerging evidence suggests that the SMS may also take into account perceptual knowledge of vowel categories (in addition to auditory goals) to estimate errors in auditory feedback. In this study, we examined how this mechanism influences within-speaker variability in vowel production. We conducted a study (n = 40 adults), consisting of a vowel categorization task and a vowel production task. The vowel categorization task was designed—based on participant-specific vowels—to estimate the categorical perceptual boundary (CPB) between two front vowels (/ε/ and /æ/). Using the vowel production data of each participant, we calculated a variability-based boundary (VBB) located at the “center of mass” of the two vowels. The inverse of the standard deviation of a vowel distribution was used as the “mass” of the vowel. We found that: (a) categorical boundary was located farther from more variable vowels; and (b) the calculated VBB (i.e., the center of mass of the vowels) significantly and positively correlated with the estimated categorical boundary (r = 0.912 for formants calculated in hertz; r = 0.854 for formants calculated in bark). Overall, our findings support a view that vowel production and vowel perception are strongly and bidirectionally linked.

Trait related sensorimotor deficits in people who stutter: An EEG investigation of μ rhythm dynamics during spontaneous fluency

Stuttering is associated with compromised sensorimotor control (i.e., internal modeling) across the dorsal stream and oscillations of EEG mu (μ) rhythms have been proposed as reliable indices of anterior dorsal stream processing. The purpose of this study was to compare μ rhythm oscillatory activity between (PWS) and matched typically fluent speakers (TFS) during spontaneously fluent overt and covert speech production tasks. Independent component analysis identified bilateral μ components from 24/27 PWS and matched TFS that localized over premotor cortex. Time-frequency analysis of the left hemisphere μ clusters demonstrated significantly reduced μ-α and μ-β ERD (pCLUSTER < 0.05) in PWS across the time course of overt and covert speech production, while no group differences were found in the right hemisphere in any condition. Results were interpreted through the framework of State Feedback Control. They suggest that weak forward modeling and evaluation of sensory feedback across the time course of speech production characterizes the trait related sensorimotor impairment in PWS. This weakness is proposed to represent an underlying sensorimotor instability that may predispose the speech of PWS to breakdown.

Role of the Cerebellum in Adaptation to Delayed Action Effects

Actions are typically associated with sensory consequences. For example, knocking at a door results in predictable sounds. These self-initiated sensory stimuli are known to elicit smaller cortical responses compared to passively presented stimuli, e.g., early auditory evoked magnetic fields known as M100 and M200 components are attenuated. Current models implicate the cerebellum in the prediction of the sensory consequences of our actions. However, causal evidence is largely missing. In this study, we introduced a constant delay (of 100 ms) between actions and action-associated sounds, and we recorded magnetoencephalography (MEG) data as participants adapted to the delay. We found an increase in the attenuation of the M100 component over time for self-generated sounds, which indicates cortical adaptation to the introduced delay. In contrast, no change in M200 attenuation was found. Interestingly, disrupting cerebellar activity via transcranial magnetic stimulation (TMS) abolished the adaptation of M100 attenuation, while the M200 attenuation reverses to an M200 enhancement. Our results provide causal evidence for the involvement of the cerebellum in adapting to delayed action effects, and thus in the prediction of the sensory consequences of our actions.

Bioelectrical brain effects of one's own voice identification in pitch of voice auditory feedback

Control of voice fundamental frequency (F0) relies in part on comparison of the intended F0 level and auditory feedback. This comparison impacts "sense of agency", or SoA, commonly defined as being the agent of one's own actions and plays a key role for self-awareness and social interactions. SoA is aberrant in several psychiatric disorders. Knowledge about brain activity reflecting SoA can be used in clinical practice for these disorders. It was shown that perception of voice feedback as one's own voice, reflecting the recognition of SoA, alters auditory sensory processing. Using a voice perturbation paradigm we contrasted vocal and bioelectrical brain responses to auditory stimuli that differed in magnitude: 100 and 400 cents. Results suggest the different magnitudes were perceived as a pitch error in self-vocalization (100 cents) or as a pitch shift generated externally (400 cents). Vocalizations and neural responses to changes in pitch of self-vocalization were defined as those made to small magnitude pitch-shifts (100 cents) and which did not show differential neural responses to upward versus downward changes in voice pitch auditory feedback. Vocal responses to large magnitude pitch shifts (400 cents) were smaller than those made to small pitch shifts, and neural responses differed according to upwards versus downward changes in pitch. Our results suggest that the presence of SoA for self-produced sounds may modify bioelectrical brain responses reflecting differences in auditory processing of the direction of a pitch shift. We suggest that this modification of bioelectrical response can be used as a biological index of SoA. Possible neuronal mechanisms of this modification of bioelectrical brain response are discussed.