Preliminary Evidence of Pre-Attentive Distinctions of Frequency-Modulated Tones that Convey Affect

Happy you, happy me: expressive changes on a stranger's voice recruit faster implicit processes than self-produced expressions

In social interactions, people have to pay attention both to the ‘what’ and ‘who’. In particular, expressive changes heard on speech signals have to be integrated with speaker identity, differentiating e.g. self- and other-produced signals. While previous research has shown that self-related visual information processing is facilitated compared to non-self stimuli, evidence in the auditory modality remains mixed. Here, we compared electroencephalography (EEG) responses to expressive changes in sequence of self- or other-produced speech sounds using a mismatch negativity (MMN) passive oddball paradigm. Critically, to control for speaker differences, we used programmable acoustic transformations to create voice deviants that differed from standards in exactly the same manner, making EEG responses to such deviations comparable between sequences. Our results indicate that expressive changes on a stranger’s voice are highly prioritized in auditory processing compared to identical changes on the self-voice. Other-voice deviants generate earlier MMN onset responses and involve stronger cortical activations in a left motor and somatosensory network suggestive of an increased recruitment of resources for less internally predictable, and therefore perhaps more socially relevant, signals.

Assisted and unassisted recession of functional anomalies associated with dysprosody in adults who stutter

PURPOSE

Speech in persons who stutter (PWS) is associated with disturbed prosody (speech melody and intonation), which may impact communication. The neural correlates of PWS' altered prosody during speaking are not known, neither is how a speech-restructuring therapy affects prosody at both a behavioral and a cerebral level.

METHODS

In this fMRI study, we explored group differences in brain activation associated with the production of different kinds of prosody in 13 male adults who stutter (AWS) before, directly after, and at least 1 year after an effective intensive fluency-shaping treatment, in 13 typically fluent-speaking control participants (CP), and in 13 males who had spontaneously recovered from stuttering during adulthood (RAWS), while sentences were read aloud with 'neutral', instructed emotional (happy), and linguistically driven (questioning) prosody. These activations were related to speech production acoustics.

RESULTS

During pre-treatment prosody generation, the pars orbitalis of the left inferior frontal gyrus and the left anterior insula were activated less in AWS than in CP. The degree of hypo-activation correlated with acoustic measures of dysprosody. Paralleling the near-normalization of free speech melody following fluency-shaping therapy, AWS normalized the inferior frontal hypo-activation, sooner after treatment for generating emotional than linguistic prosody. Unassisted recovery was associated with an additional recruitment of cerebellar resources.

CONCLUSIONS

Fluency shaping therapy may restructure prosody, which approaches that of typically fluent-speaking people. Such a process may benefit from additional training of instructed emotional and linguistic prosody by inducing plasticity in the inferior frontal region which has developed abnormally during childhood in PWS.

Neural Substrates of Auditory Emotion Recognition Deficits in Schizophrenia

Deficits in auditory emotion recognition (AER) are a core feature of schizophrenia and a key component of social cognitive impairment. AER deficits are tied behaviorally to impaired ability to interpret tonal ("prosodic") features of speech that normally convey emotion, such as modulations in base pitch (F0M) and pitch variability (F0SD). These modulations can be recreated using synthetic frequency modulated (FM) tones that mimic the prosodic contours of specific emotional stimuli. The present study investigates neural mechanisms underlying impaired AER using a combined event-related potential/resting-state functional connectivity (rsfMRI) approach in 84 schizophrenia/schizoaffective disorder patients and 66 healthy comparison subjects. Mismatch negativity (MMN) to FM tones was assessed in 43 patients/36 controls. rsfMRI between auditory cortex and medial temporal (insula) regions was assessed in 55 patients/51 controls. The relationship between AER, MMN to FM tones, and rsfMRI was assessed in the subset who performed all assessments (14 patients, 21 controls). As predicted, patients showed robust reductions in MMN across FM stimulus type (p = 0.005), particularly to modulations in F0M, along with impairments in AER and FM tone discrimination. MMN source analysis indicated dipoles in both auditory cortex and anterior insula, whereas rsfMRI analyses showed reduced auditory-insula connectivity. MMN to FM tones and functional connectivity together accounted for ∼50% of the variance in AER performance across individuals. These findings demonstrate that impaired preattentive processing of tonal information and reduced auditory-insula connectivity are critical determinants of social cognitive dysfunction in schizophrenia, and thus represent key targets for future research and clinical intervention.

SIGNIFICANCE STATEMENT

Schizophrenia patients show deficits in the ability to infer emotion based upon tone of voice [auditory emotion recognition (AER)] that drive impairments in social cognition and global functional outcome. This study evaluated neural substrates of impaired AER in schizophrenia using a combined event-related potential/resting-state fMRI approach. Patients showed impaired mismatch negativity response to emotionally relevant frequency modulated tones along with impaired functional connectivity between auditory and medial temporal (anterior insula) cortex. These deficits contributed in parallel to impaired AER and accounted for ∼50% of variance in AER performance. Overall, these findings demonstrate the importance of both auditory-level dysfunction and impaired auditory/insula connectivity in the pathophysiology of social cognitive dysfunction in schizophrenia.

The effects of stimulus complexity on the preattentive processing of self-generated and nonself voices: An ERP study

The ability to differentiate one's own voice from the voice of somebody else plays a critical role in successful verbal self-monitoring processes and in communication. However, most of the existing studies have only focused on the sensory correlates of self-generated voice processing, whereas the effects of attentional demands and stimulus complexity on self-generated voice processing remain largely unknown. In this study, we investigated the effects of stimulus complexity on the preattentive processing of self and nonself voice stimuli. Event-related potentials (ERPs) were recorded from 17 healthy males who watched a silent movie while ignoring prerecorded self-generated (SGV) and nonself (NSV) voice stimuli, consisting of a vocalization (vocalization category condition: VCC) or of a disyllabic word (word category condition: WCC). All voice stimuli were presented as standard and deviant events in four distinct oddball sequences. The mismatch negativity (MMN) ERP component peaked earlier for NSV than for SGV stimuli. Moreover, when compared with SGV stimuli, the P3a amplitude was increased for NSV stimuli in the VCC only, whereas in the WCC no significant differences were found between the two voice types. These findings suggest differences in the time course of automatic detection of a change in voice identity. In addition, they suggest that stimulus complexity modulates the magnitude of the orienting response to SGV and NSV stimuli, extending previous findings on self-voice processing.

Sex differences in preattentive perception of emotional voices and acoustic attributes

Sex stereotypes consider women to be superior in terms of voice sensitivity. However, whether such sex differences are driven by voice perception per se or by low-level acoustic attributes remains unclear. Using a passive auditory oddball paradigm, we studied the emotionally spoken meaningless syllables 'dada' (neutral, happy, fearful) along with corresponding nonvocal sounds in female and male adults. Mismatch negativity (MMN) and P3a were identified in the waveforms obtained by subtraction of the responses to standard stimuli from the deviant stimuli. Results showed that MMN in response to fearful syllables was stronger in women, whereas MMN elicited by nonvocal sounds was comparable between sexes. This sex effect was specifically found in MMN, but not in P3a. These findings suggest that sex differences in voice sensitivity are selectively driven by voice perception. The sex effect in preattentive processing of emotional voices may further implicate possible etiological pathways for mental disorders characterized by disturbance in emotional processes as well as disparities in the prevalence between sexes.

Relationship between auditory processing and affective prosody in schizophrenia

Patients with schizophrenia have well-established deficits in their ability to identify emotion from facial expression and tone of voice. In the visual modality, there is strong evidence that basic processing deficits contribute to impaired facial affect recognition in schizophrenia. However, few studies have examined the auditory modality for mechanisms underlying affective prosody identification. In this study, we explored links between different stages of auditory processing, using event-related potentials (ERPs), and affective prosody detection in schizophrenia. Thirty-six schizophrenia patients and 18 healthy control subjects received tasks of affective prosody, facial emotion identification, and tone matching, as well as two auditory oddball paradigms, one passive for mismatch negativity (MMN) and one active for P300. Patients had significantly reduced MMN and P300 amplitudes, impaired auditory and visual emotion recognition, and poorer tone matching performance, relative to healthy controls. Correlations between ERP and behavioral measures within the patient group revealed significant associations between affective prosody recognition and both MMN and P300 amplitudes. These relationships were modality specific, as MMN and P300 did not correlate with facial emotion recognition. The two ERP waves accounted for 49% of the variance in affective prosody in a regression analysis. Our results support previous suggestions of a relationship between basic auditory processing abnormalities and affective prosody dysfunction in schizophrenia, and indicate that both relatively automatic pre-attentive processes (MMN) and later attention-dependent processes (P300) are involved with accurate auditory emotion identification. These findings provide support for bottom-up (e.g., perceptually based) cognitive remediation approaches.

Reduction in tonal discriminations predicts receptive emotion processing deficits in schizophrenia and schizoaffective disorder

INTRODUCTION

Schizophrenia patients show decreased ability to identify emotion based upon tone of voice (voice emotion recognition), along with deficits in basic auditory processing. Interrelationship among these measures is poorly understood.

METHODS

Forty-one patients with schizophrenia/schizoaffective disorder and 41 controls were asked to identify the emotional valence (happy, sad, angry, fear, or neutral) of 38 synthesized frequency-modulated (FM) tones designed to mimic key acoustic features of human vocal expressions. The mean (F0M) and variability (F0SD) of fundamental frequency (pitch) and absence or presence of high frequency energy (HF500) of the tones were independently manipulated to assess contributions on emotion identification. Forty patients and 39 controls also completed tone-matching and voice emotion recognition tasks.

RESULTS

Both groups showed a nonrandom response pattern (P < .0001). Stimuli with highest and lowest F0M/F0SD were preferentially identified as happy and sad, respectively. Stimuli with low F0M and midrange F0SD values were identified as angry. Addition of HF500 increased rates of angry and decreased rates of sad identifications. Patients showed less differentiation of response across frequency changes, leading to a highly significant between-group difference in response pattern to maximally identifiable stimuli (d = 1.4). The differential identification pattern for FM tones correlated with deficits in basic tone-matching ability (P = .01), voice emotion recognition (P < .001), and negative symptoms (P < .001).

CONCLUSIONS

Specific FM tones conveyed reliable emotional percepts in both patients and controls and correlated highly with deficits in ability to recognize information based upon tone of voice, suggesting significant bottom-up contributions to social cognition and negative symptom impairments in schizophrenia.