Self-monitoring in the cerebral cortex: Neural responses to small pitch shifts in auditory feedback during speech production

Predictive language processing: integrating comprehension and production, and what atypical populations can tell us

Predictive processing, a crucial aspect of human cognition, is also relevant for language comprehension. In everyday situations, we exploit various sources of information to anticipate and therefore facilitate processing of upcoming linguistic input. In the literature, there are a variety of models that aim at accounting for such ability. One group of models propose a strict relationship between prediction and language production mechanisms. In this review, we first introduce very briefly the concept of predictive processing during language comprehension. Secondly, we focus on models that attribute a prominent role to language production and sensorimotor processing in language prediction ("prediction-by-production" models). Contextually, we provide a summary of studies that investigated the role of speech production and auditory perception on language comprehension/prediction tasks in healthy, typical participants. Then, we provide an overview of the limited existing literature on specific atypical/clinical populations that may represent suitable testing ground for such models-i.e., populations with impaired speech production and auditory perception mechanisms. Ultimately, we suggest a more widely and in-depth testing of prediction-by-production accounts, and the involvement of atypical populations both for model testing and as targets for possible novel speech/language treatment approaches.

Causal role of medial superior frontal cortex on enhancing neural information flow and self-agency judgments in the self-agency network

Self-agency is being aware of oneself as the agent of one's thoughts and actions. Self-agency is necessary for successful interactions with the outside world (reality-monitoring). Prior research has shown that the medial superior prefrontal gyri (mPFC/SFG) may represent one neural correlate underlying self-agency judgments. However, the causal relationship remains unknown. Here, we applied high-frequency 10Hz repetitive transcranial magnetic stimulation (rTMS) to modulate the excitability of the mPFC/SFG site that we have previously shown to mediate self-agency. For the first time, we delineate causal neural mechanisms, revealing precisely how rTMS modulates SFG excitability and impacts directional neural information flow in the self-agency network by implementing innovative magnetoencephalography (MEG) phase-transfer entropy (PTE) metrics, measured from pre-to-post rTMS. We found that, compared to control rTMS, enhancing SFG excitability by rTMS induced significant increases in information flow between SFG and specific cingulate and paracentral regions in the self-agency network in delta-theta, alpha, and gamma bands, which predicted improved self-agency judgments. This is the first multimodal imaging study in which we implement MEG PTE metrics of 5D imaging of space, frequency and time, to provide cutting-edge analyses of the causal neural mechanisms of how rTMS enhances SFG excitability and improves neural information flow between distinct regions in the self-agency network to potentiate improved self-agency judgments. Our findings provide a novel perspective for investigating causal neural mechanisms underlying self-agency and create a path towards developing novel neuromodulation interventions to improve self-agency that will be particularly useful for patients with psychosis who exhibit severe impairments in self-agency.

Global MEG Resting State Functional Connectivity in Children with Autism and Sensory Processing Dysfunction

Sensory processing dysfunction not only affects most individuals with autism spectrum disorder (ASD), but at least 5% of children without ASD also experience dysfunctional sensory processing. Our understanding of the relationship between sensory dysfunction and resting state brain activity is still emerging. This study compared long-range resting state functional connectivity of neural oscillatory behavior in children aged 8-12 years with autism spectrum disorder (ASD; N=18), those with sensory processing dysfunction (SPD; N=18) who do not meet ASD criteria, and typically developing control participants (TDC; N=24) using magnetoencephalography (MEG). Functional connectivity analyses were performed in the alpha and beta frequency bands, which are known to be implicated in sensory information processing. Group differences in functional connectivity and associations between sensory abilities and functional connectivity were examined. Distinct patterns of functional connectivity differences between ASD and SPD groups were found only in the beta band, but not in the alpha band. In both alpha and beta bands, ASD and SPD cohorts differed from the TDC cohort. Somatosensory cortical beta-band functional connectivity was associated with tactile processing abilities, while higher-order auditory cortical alpha-band functional connectivity was associated with auditory processing abilities. These findings demonstrate distinct long-range neural synchrony alterations in SPD and ASD that are associated with sensory processing abilities. Neural synchrony measures could serve as potential sensitive biomarkers for ASD and SPD.

Deep Learning for Neuromuscular Control of Vocal Source for Voice Production

A computational neuromuscular control system that generates lung pressure and three intrinsic laryngeal muscle activations (cricothyroid, thyroarytenoid, and lateral cricoarytenoid) to control the vocal source was developed. In the current study, LeTalker, a biophysical computational model of the vocal system was used as the physical plant. In the LeTalker, a three-mass vocal fold model was used to simulate self-sustained vocal fold oscillation. A constant/ǝ/vowel was used for the vocal tract shape. The trachea was modeled after MRI measurements. The neuromuscular control system generates control parameters to achieve four acoustic targets (fundamental frequency, sound pressure level, normalized spectral centroid, and signal-to-noise ratio) and four somatosensory targets (vocal fold length, and longitudinal fiber stress in the three vocal fold layers). The deep-learning-based control system comprises one acoustic feedforward controller and two feedback (acoustic and somatosensory) controllers. Fifty thousand steady speech signals were generated using the LeTalker for training the control system. The results demonstrated that the control system was able to generate the lung pressure and the three muscle activations such that the four acoustic and four somatosensory targets were reached with high accuracy. After training, the motor command corrections from the feedback controllers were minimal compared to the feedforward controller except for thyroarytenoid muscle activation.

Impaired Speaking-Induced Suppression in Alzheimer's Disease

Alzheimer's disease (AD) is a neurodegenerative disease involving cognitive impairment and abnormalities in speech and language. Here, we examine how AD affects the fidelity of auditory feedback predictions during speaking. We focus on the phenomenon of speaking-induced suppression (SIS), the auditory cortical responses' suppression during auditory feedback processing. SIS is determined by subtracting the magnitude of auditory cortical responses during speaking from listening to playback of the same speech. Our state feedback control (SFC) model of speech motor control explains SIS as arising from the onset of auditory feedback matching a prediction of that feedback onset during speaking, a prediction that is absent during passive listening to playback of the auditory feedback. Our model hypothesizes that the auditory cortical response to auditory feedback reflects the mismatch with the prediction: small during speaking, large during listening, with the difference being SIS. Normally, during speaking, auditory feedback matches its predictions, then SIS will be large. Any reductions in SIS will indicate inaccuracy in auditory feedback prediction not matching the actual feedback. We investigated SIS in AD patients [n = 20; mean (SD) age, 60.77 (10.04); female (%), 55.00] and healthy controls [n = 12; mean (SD) age, 63.68 (6.07); female (%), 83.33] through magnetoencephalography (MEG)-based functional imaging. We found a significant reduction in SIS at ∼100 ms in AD patients compared with healthy controls (linear mixed effects model, F (1,57.5) = 6.849, p = 0.011). The results suggest that AD patients generate inaccurate auditory feedback predictions, contributing to abnormalities in AD speech.

Aberrant neurophysiological signaling associated with speech impairments in Parkinson's disease

Difficulty producing intelligible speech is a debilitating symptom of Parkinson's disease (PD). Yet, both the robust evaluation of speech impairments and the identification of the affected brain systems are challenging. Using task-free magnetoencephalography, we examine the spectral and spatial definitions of the functional neuropathology underlying reduced speech quality in patients with PD using a new approach to characterize speech impairments and a novel brain-imaging marker. We found that the interactive scoring of speech impairments in PD (N = 59) is reliable across non-expert raters, and better related to the hallmark motor and cognitive impairments of PD than automatically-extracted acoustical features. By relating these speech impairment ratings to neurophysiological deviations from healthy adults (N = 65), we show that articulation impairments in patients with PD are associated with aberrant activity in the left inferior frontal cortex, and that functional connectivity of this region with somatomotor cortices mediates the influence of cognitive decline on speech deficits.

Echolocation-related reversal of information flow in a cortical vocalization network

The mammalian frontal and auditory cortices are important for vocal behavior. Here, using local-field potential recordings, we demonstrate that the timing and spatial patterns of oscillations in the fronto-auditory network of vocalizing bats (Carollia perspicillata) predict the purpose of vocalization: echolocation or communication. Transfer entropy analyses revealed predominant top-down (frontal-to-auditory cortex) information flow during spontaneous activity and pre-vocal periods. The dynamics of information flow depend on the behavioral role of the vocalization and on the timing relative to vocal onset. We observed the emergence of predominant bottom-up (auditory-to-frontal) information transfer during the post-vocal period specific to echolocation pulse emission, leading to self-directed acoustic feedback. Electrical stimulation of frontal areas selectively enhanced responses to sounds in auditory cortex. These results reveal unique changes in information flow across sensory and frontal cortices, potentially driven by the purpose of the vocalization in a highly vocal mammalian model.

EXPRESS: Don't blame yourself: Conscious source monitoring modulates feedback control during speech production

Sensory feedback plays an important role in speech motor control. One of the main sources of evidence for this are studies where online auditory feedback is perturbed during ongoing speech. In motor control, it is therefore crucial to distinguish between sensory feedback and externally generated sensory events. This is called source monitoring. Previous altered feedback studies have taken non-conscious source monitoring for granted, as automatic responses to altered sensory feedback imply that the feedback changes are processed as self-caused. However, the role of conscious source monitoring is unclear. The current study investigated whether conscious source monitoring modulates responses to unexpected pitch changes in auditory feedback. During a first block, some participants spontaneously attributed the pitch shifts to themselves (self-blamers) while others attributed them to an external source (other-blamers). Before block 2, all participants were informed that the pitch shifts were experimentally induced. The self-blamers then showed a reduction in response magnitude in block 2 compared with block 1, while the other-blamers did not. This suggests that conscious source monitoring modulates responses to altered auditory feedback, such that consciously ascribing feedback to oneself leads to larger compensation responses. These results can be accounted for within the dominant comparator framework, where conscious source monitoring could modulate the gain on sensory feedback. Alternatively, the results can be naturally explained from an inferential framework, where conscious knowledge may bias the priors in a Bayesian process to determine the most likely source of a sensory event.

The Role of the Medial Prefontal Cortex in Self-Agency in Schizophrenia

Schizophrenia is a disorder of the self. In particular, patients show cardinal deficits in self-agency (i.e., the experience and awareness of being the agent of one’s own thoughts and actions) that directly contribute to positive psychotic symptoms of hallucinations and delusions and distort reality monitoring (defined as distinguishing self-generated information from externally-derived information). Predictive coding models suggest that the experience of self-agency results from a minimal prediction error between the predicted sensory consequence of a self-generated action and the actual outcome. In other words, the experience of self-agency is thought to be driven by making reliable predictions about the expected outcomes of one’s own actions. Most of the agency literature has focused on the motor system; here we present a novel viewpoint that examines agency from a different lens using distinct tasks of reality monitoring and speech monitoring. The self-prediction mechanism that leads to self-agency is necessary for reality monitoring in that self-predictions represent a critical precursor for the successful encoding and memory retrieval of one’s own thoughts and actions during reality monitoring to enable accurate self-agency judgments (i.e., accurate identification of self-generated information). This self-prediction mechanism is also critical for speech monitoring where we continually compare auditory feedback (i.e., what we hear ourselves say) with what we expect to hear. Prior research has shown that the medial prefrontal cortex (mPFC) may represent one potential neural substrate of this self-prediction mechanism. Unfortunately, patients with schizophrenia (SZ) show mPFC hypoactivity associated with self-agency impairments on reality and speech monitoring tasks, as well as aberrant mPFC functional connectivity during intrinsic measures of agency during resting states that predicted worsening psychotic symptoms. Causal neurostimulation and neurofeedback techniques can move the frontiers of schizophrenia research into a new era where we implement techniques to manipulate excitability in key neural regions, such as the mPFC, to modulate patients’ reliance on self-prediction mechanisms on distinct tasks of reality and speech monitoring. We hypothesize these findings will show that mPFC provides a unitary basis for self-agency, driven by reliance on self-prediction mechanisms, which will facilitate the development of new targeted treatments in patients with schizophrenia.

Impairment of speech auditory feedback error detection and motor correction in post-stroke aphasia

INTRODUCTION

The present study investigated how damage to left-hemisphere brain networks affects the ability for speech auditory feedback error detection and motor correction in post-stroke aphasia.

METHODS

34 individuals with left-hemisphere stroke and 25 neurologically intact age-matched control participants performed two randomized experimental tasks in which their online speech auditory feedback was altered using externally induced pitch-shift stimuli: 1) vocalization of a steady speech vowel sound /a/, and 2) listening to the playback of the same self-produced vowel vocalizations. Randomized control condition trials were interleaved in between vocalization and listening tasks where no pitch-shift stimuli were delivered. Following each trial, participants pressed a button to indicate whether they detected a pitch-shift error in their speech auditory feedback during vocalization and listening tasks.

RESULTS

Our data analysis revealed that speech auditory feedback error detection accuracy rate was significantly lower in the stroke compared with control participants, irrespective of the experimental task (i.e. vocalization vs. listening) and trial condition (i.e. pitch-shifted vs. no-pitch-shift). We found that this effect was associated with the reduced magnitude of speech compensation in the early phase of responses at 150-200 ms following the onset of pitch-shift stimuli in stroke participants. In addition, motor speech compensation deficit in the stroke group was correlated with lower scores on speech repetition tasks as an index of language impairment resulting from aphasia.

CONCLUSIONS

These findings provide evidence that left-hemisphere stroke is associated with impaired speech auditory feedback error processing, and such deficits account for specific aspects of language impairment in aphasia.

Neural competition between concurrent speech production and other speech perception

Understanding others' speech while individuals simultaneously produce speech utterances implies neural competition and requires specific mechanisms for a neural resolution given that previous studies proposed opposing signal dynamics for both processes in the auditory cortex (AC). We here used neuroimaging in humans to investigate this neural competition by lateralized stimulations with other speech samples and ipsilateral or contralateral lateralized feedback of actively produced self speech utterances in the form of various speech vowels. In experiment 1, we show, first, that others' speech classifications during active self speech lead to activity in the planum temporale (PTe) when both self and other speech samples were presented together to only the left or right ear. The contralateral PTe also seemed to indifferently respond to single self and other speech samples. Second, specific activity in the left anterior superior temporal cortex (STC) was found during dichotic stimulations (i.e. self and other speech presented to separate ears). Unlike previous studies, this left anterior STC activity supported self speech rather than other speech processing. Furthermore, right mid and anterior STC was more involved in other speech processing. These results signify specific mechanisms for self and other speech processing in the left and right STC beyond a more general speech processing in PTe. Third, other speech recognition in the context of listening to recorded self speech in experiment 2 led to largely symmetric activity in STC and additionally in inferior frontal subregions. The latter was previously reported to be generally relevant for other speech perception and classification, but we found frontal activity only when other speech classification was challenged by recorded but not by active self speech samples. Altogether, unlike formerly established brain networks for uncompetitive other speech perception, active self speech during other speech perception seemingly leads to a neural reordering, functional reassignment, and unusual lateralization of AC and frontal brain activations.

It's About Time: Minimizing Hardware and Software Latencies in Speech Research With Real-Time Auditory Feedback

Purpose Various aspects of speech production related to auditory-motor integration and learning have been examined through auditory feedback perturbation paradigms in which participants' acoustic speech output is experimentally altered and played back via earphones/headphones "in real time." Scientific rigor requires high precision in determining and reporting the involved hardware and software latencies. Many reports in the literature, however, are not consistent with the minimum achievable latency for a given experimental setup. Here, we focus specifically on this methodological issue associated with implementing real-time auditory feedback perturbations, and we offer concrete suggestions for increased reproducibility in this particular line of work. Method Hardware and software latencies as well as total feedback loop latency were measured for formant perturbation studies with the Audapter software. Measurements were conducted for various audio interfaces, desktop and laptop computers, and audio drivers. An approach for lowering Audapter's software latency through nondefault parameter specification was also tested. Results Oft-overlooked hardware-specific latencies were not negligible for some of the tested audio interfaces (adding up to 15 ms). Total feedback loop latencies (including both hardware and software latency) were also generally larger than claimed in the literature. Nondefault parameter values can improve Audapter's own processing latency without negative impact on formant tracking. Conclusions Audio interface selection and software parameter optimization substantially affect total feedback loop latency. Thus, the actual total latency (hardware plus software) needs to be correctly measured and described in all published reports. Future speech research with "real-time" auditory feedback perturbations should increase scientific rigor by minimizing this latency.

Does passive sound attenuation affect responses to pitch-shifted auditory feedback?

The role of auditory feedback in vocal production has mainly been investigated by altered auditory feedback (AAF) in real time. In response, speakers compensate by shifting their speech output in the opposite direction. Current theory suggests this is caused by a mismatch between expected and observed feedback. A methodological issue is the difficulty to fully isolate the speaker's hearing so that only AAF is presented to their ears. As a result, participants may be presented with two simultaneous signals. If this is true, an alternative explanation is that responses to AAF depend on the contrast between the manipulated and the non-manipulated feedback. This hypothesis was tested by varying the passive sound attenuation (PSA). Participants vocalized while auditory feedback was unexpectedly pitch shifted. The feedback was played through three pairs of headphones with varying amounts of PSA. The participants' responses were not affected by the different levels of PSA. This suggests that across all three headphones, PSA is either good enough to make the manipulated feedback dominant, or differences in PSA are too small to affect the contribution of non-manipulated feedback. Overall, the results suggest that it is important to realize that non-manipulated auditory feedback could affect responses to AAF.

Attention/memory complaint is correlated with motor speech disorder in Parkinson's disease

BACKGROUND

The mechanisms underlying the online modulation of motor speech in Parkinson's disease (PD) have not been determined. Moreover, medical and rehabilitation interventions for PD-associated motor speech disorder (MSD) have a poor long-term prognosis.

METHODS

To compare risk factors in PD patients with MSD to those without MSD (non-MSD) and determine predictive independent risk factors correlated with the MSD phenotype, we enrolled 314 PD patients, including 250 with and 64 without MSD. We compared demographic, characteristic data, as well as PD-associated evaluations between the MSD group and non-MSD group.

RESULTS

Univariate analysis showed that demographic characteristics, including occupation, educational level, monthly income and speaking background; clinical characteristics, including lesions in the frontal and temporal lobes, and concurrent dysphagia; and PD-associated evaluations, including the activity of daily living (ADL) score, non-motor symptoms scale (NMSS) domain 4 score (perceptual problem), and NMSS domain 5 score (attention/memory) were all significantly different between the MSD and non-MSD group (all P < 0.05). Multivariate logistic regression analysis showed that educational level, frontal lesions, and NMSS domain 5 score (attention/memory) were independent risk factors for PD-associated MSD (all P < 0.005).

CONCLUSIONS

We determined an association between MSD phenotype and cognitive impairment, reflected by low-level education and related clinical profiles. Moreover, attention and memory dysfunction may play key roles in the progression of MSD in PD patients. Further studies are required to detail the mechanism underlying abnormal speech motor modulation in PD patients. Early cognitive intervention may enhance rehabilitation management and motor speech function in patients with PD-associated MSD.

Limited Pre-Speech Auditory Modulation in Individuals Who Stutter: Data and Hypotheses

Purpose We review and interpret our recent series of studies investigating motor-to-auditory influences during speech movement planning in fluent speakers and speakers who stutter. In those studies, we recorded auditory evoked potentials in response to probe tones presented immediately prior to speaking or at the equivalent time in no-speaking control conditions. As a measure of pre-speech auditory modulation (PSAM), we calculated changes in auditory evoked potential amplitude in the speaking conditions relative to the no-speaking conditions. Whereas adults who do not stutter consistently showed PSAM, this phenomenon was greatly reduced or absent in adults who stutter. The same between-group difference was observed in conditions where participants expected to hear their prerecorded speech played back without actively producing it, suggesting that the speakers who stutter use inefficient forward modeling processes rather than inefficient motor command generation processes. Compared with fluent participants, adults who stutter showed both less PSAM and less auditory-motor adaptation when producing speech while exposed to formant-shifted auditory feedback. Across individual participants, however, PSAM and auditory-motor adaptation did not correlate in the typically fluent group, and they were negatively correlated in the stuttering group. Interestingly, speaking with a consistent 100-ms delay added to the auditory feedback signal-normalized PSAM in speakers who stutter, and there no longer was a between-group difference in this condition. Conclusions Combining our own data with human and animal neurophysiological evidence from other laboratories, we interpret the overall findings as suggesting that (a) speech movement planning modulates auditory processing in a manner that may optimize its tuning characteristics for monitoring feedback during speech production and, (b) in conditions with typical auditory feedback, adults who stutter do not appropriately modulate the auditory system prior to speech onset. Lack of modulation of speakers who stutter may lead to maladaptive feedback-driven movement corrections that manifest themselves as repetitive movements or postural fixations.