The neural correlates of emotional prosody comprehension: disentangling simple from complex emotion

Neural correlates of individual differences in multimodal emotion recognition ability

Studies have reported substantial variability in emotion recognition ability (ERA) - an important social skill - but possible neural underpinnings for such individual differences are not well understood. This functional magnetic resonance imaging (fMRI) study investigated neural responses during emotion recognition in young adults (N = 49) who were selected for inclusion based on their performance (high or low) during previous testing of ERA. Participants were asked to judge brief video recordings in a forced-choice emotion recognition task, wherein stimuli were presented in visual, auditory and multimodal (audiovisual) blocks. Emotion recognition rates during brain scanning confirmed that individuals with high (vs low) ERA received higher accuracy for all presentation blocks. fMRI-analyses focused on key regions of interest (ROIs) involved in the processing of multimodal emotion expressions, based on previous meta-analyses. In neural response to emotional stimuli contrasted with neutral stimuli, individuals with high (vs low) ERA showed higher activation in the following ROIs during the multimodal condition: right middle superior temporal gyrus (mSTG), right posterior superior temporal sulcus (PSTS), and right inferior frontal cortex (IFC). Overall, results suggest that individual variability in ERA may be reflected across several stages of decisional processing, including extraction (mSTG), integration (PSTS) and evaluation (IFC) of emotional information.

Mentalizing in a movie for the assessment of social cognition (MASC): the validation in a taiwanese sample

BACKGROUND

The present study evaluated the psychometrics properties of a sensitive video-based test used in the evaluation of mentalizing skills, that is, the Movie for the Assessment of Social Cognition-Taiwanese version (MASC-TW).

METHODS

We recruited two independent samples of nonclinical participants (N = 167) and adult patients with schizophrenia (N = 41). The MASC-TW and two other social cognition measures, namely the Chinese version of Theory of Mind task (ToM) and the Taiwanese version of the Diagnostic Analysis of Nonverbal Accuracy-2 (DANAV-TW-2), and an executive function measure of the Wisconsin Card Sorting Test (WCST), were administered to both groups.

RESULTS

The MASC proved to be a reliable measure of mentalizing capacity, high Cronbach's α value of 0.87. The intraclass correlation coefficient for the MASC-TW total correct scores was 0.85 across three waves of data collection. Across the entire sample, the scores on the MASC-TW were significantly correlated with verbal and nonverbal scores for the ToM task and recognition of facial and prosodic emotion on the DANAV-TW-2. Both executive function and emotion recognition emerged as noteworthy predictors of mentalizing, indicating that these two variables might play crucial roles in the development of mentalizing capacities. Finally, a receiver operating characteristic analysis revealed that in patients with schizophrenia, the MASC was the most accurate discriminator of diagnostic groups, highlighting the validity of the MASC.

CONCLUSIONS

Overall, the MASC-TW is an ecologically valid and useful tool for assessing mentalizing abilities in a Taiwanese population.

The emotional component of inner speech: A pilot exploratory fMRI study

Inner speech is one of the most important human cognitive processes. Nevertheless, until now, many aspects of inner speech, particularly the emotional characteristics of inner speech, remain poorly understood. The main objectives of our study are to identify the neural substrate for the emotional (prosodic) dimension of inner speech and brain structures that control the suppression of expression in inner speech. To achieve these goals, a pilot exploratory fMRI study was carried out on 33 people. The subjects listened to pre-recorded phrases or individual words pronounced with different emotional connotations, after which they were internally spoken with the same emotion or with suppression of expression (neutral). The results show that there is an emotional component in inner speech, which is encoded by similar structures as in spoken speech. The unique role of the caudate nuclei in the suppression of expression in the inner speech was also shown.

Dysfunctional cerebello-cerebral network associated with vocal emotion recognition impairments

Vocal emotion recognition, a key determinant to analyzing a speaker's emotional state, is known to be impaired following cerebellar dysfunctions. Nevertheless, its possible functional integration in the large-scale brain network subtending emotional prosody recognition has yet to be explored. We administered an emotional prosody recognition task to patients with right versus left-hemispheric cerebellar lesions and a group of matched controls. We explored the lesional correlates of vocal emotion recognition in patients through a network-based analysis by combining a neuropsychological approach for lesion mapping with normative brain connectome data. Results revealed impaired recognition among patients for neutral or negative prosody, with poorer sadness recognition performances by patients with right cerebellar lesion. Network-based lesion-symptom mapping revealed that sadness recognition performances were linked to a network connecting the cerebellum with left frontal, temporal, and parietal cortices. Moreover, when focusing solely on a subgroup of patients with right cerebellar damage, sadness recognition performances were associated with a more restricted network connecting the cerebellum to the left parietal lobe. As the left hemisphere is known to be crucial for the processing of short segmental information, these results suggest that a corticocerebellar network operates on a fine temporal scale during vocal emotion decoding.

The hearing hippocampus

The hippocampus has a well-established role in spatial and episodic memory but a broader function has been proposed including aspects of perception and relational processing. Neural bases of sound analysis have been described in the pathway to auditory cortex, but wider networks supporting auditory cognition are still being established. We review what is known about the role of the hippocampus in processing auditory information, and how the hippocampus itself is shaped by sound. In examining imaging, recording, and lesion studies in species from rodents to humans, we uncover a hierarchy of hippocampal responses to sound including during passive exposure, active listening, and the learning of associations between sounds and other stimuli. We describe how the hippocampus' connectivity and computational architecture allow it to track and manipulate auditory information - whether in the form of speech, music, or environmental, emotional, or phantom sounds. Functional and structural correlates of auditory experience are also identified. The extent of auditory-hippocampal interactions is consistent with the view that the hippocampus makes broad contributions to perception and cognition, beyond spatial and episodic memory. More deeply understanding these interactions may unlock applications including entraining hippocampal rhythms to support cognition, and intervening in links between hearing loss and dementia.

Language and developmental plasticity after perinatal stroke

The mature human brain is lateralized for language, with the left hemisphere (LH) primarily responsible for sentence processing and the right hemisphere (RH) primarily responsible for processing suprasegmental aspects of language such as vocal emotion. However, it has long been hypothesized that in early life there is plasticity for language, allowing young children to acquire language in other cortical regions when LH areas are damaged. If true, what are the constraints on functional reorganization? Which areas of the brain can acquire language, and what happens to the functions these regions ordinarily perform? We address these questions by examining long-term outcomes in adolescents and young adults who, as infants, had a perinatal arterial ischemic stroke to the LH areas ordinarily subserving sentence processing. We compared them with their healthy age-matched siblings. All participants were tested on a battery of behavioral and functional imaging tasks. While stroke participants were impaired in some nonlinguistic cognitive abilities, their processing of sentences and of vocal emotion was normal and equal to that of their healthy siblings. In almost all, these abilities have both developed in the healthy RH. Our results provide insights into the remarkable ability of the young brain to reorganize language. Reorganization is highly constrained, with sentence processing almost always in the RH frontotemporal regions homotopic to their location in the healthy brain. This activation is somewhat segregated from RH emotion processing, suggesting that the two functions perform best when each has its own neural territory.

Crossed functional specialization between the basal ganglia and cerebellum during vocal emotion decoding: Insights from stroke and Parkinson’s disease

There is growing evidence that both the basal ganglia and the cerebellum play functional roles in emotion processing, either directly or indirectly, through their connections with cortical and subcortical structures. However, the lateralization of this complex processing in emotion recognition remains unclear. To address this issue, we investigated emotional prosody recognition in individuals with Parkinson’s disease (model of basal ganglia dysfunction) or cerebellar stroke patients, as well as in matched healthy controls (n = 24 in each group). We analysed performances according to the lateralization of the predominant brain degeneration/lesion. Results showed that a right (basal ganglia and cerebellar) hemispheric dysfunction was likely to induce greater deficits than a left one. Moreover, deficits following left hemispheric dysfunction were only observed in cerebellar stroke patients, and these deficits resembled those observed after degeneration of the right basal ganglia. Additional analyses taking disease duration / time since stroke into consideration revealed a worsening of performances in patients with predominantly right-sided lesions over time. These results point to the differential, but complementary, involvement of the cerebellum and basal ganglia in emotional prosody decoding, with a probable hemispheric specialization according to the level of cognitive integration.

Brain mapping of emotional prosody in patients with drug-resistant temporal epilepsy: An indicator of plasticity

INTRODUCTION

Emotional prosody, a suprasegmental component of language, is predominantly processed by right temporo-frontal areas of the cerebral cortex. In temporal lobe epilepsy (TLE), brain disturbances affecting prosody processing frequently occur. This research assesses compensatory brain mechanisms of prosody processing in refractory TLE using fMRI.

METHODS

Patients with focal unilateral epilepsy, right (RTLE) (N = 19), left (LTLE) (N = 19), and healthy controls (CTRL) (N = 20) were evaluated during a prosody decoding fMRI task. The stimuli consisted in spoken numbers with different tones of voice (joy, fear, anger, neutral and silent trials). Participants were instructed to label the emotion with a keypad. "Joy" was removed from the analysis due to a high degree of variability. A lateralization index (LI) was used to see individual differences in the interhemispheric activations of each participant.

RESULTS

Behaviorally, The LTLE and RTLE groups did not differ significantly from each other neither from CTRL. In Negative Emotions versus Baseline contrast, the whole sample analysis showed extensive activations in bilateral superior temporal gyrus, bilateral precentral and post-central gyrus, right putamen, and left cerebellar vermis. Compared to the LTLE and CTRL, RTLE activated similar areas, but to a lesser extent. The LI analysis revealed significant differences in hemispheric laterality of the temporal lobe and the parietal lobe between RTLE compared to LTLE and CTRL, being the RTLE group lateralized towards the left, unlike the other two groups.

DISCUSSION

The LI indicated that, since the CTRL and the LTLE groups recruited putative prosodic regions, the RTLE lateralized prosody processing towards the left, recruiting contralateral nodes, homotopic to the putative areas of the prosody. Considering that the groups did not differ in prosody task performance, the findings suggest that, in the RTLE group, alternative brain nodes were recruited for the task, demonstrating plasticity.

Functional patterns of neural activation during vocal emotion recognition in youth with and without refractory epilepsy

Epilepsy has been associated with deficits in the social cognitive ability to decode others' nonverbal cues to infer their emotional intent (emotion recognition). Studies have begun to identify potential neural correlates of these deficits, but have focused primarily on one type of nonverbal cue (facial expressions) to the detriment of other crucial social signals that inform the tenor of social interactions (e.g., tone of voice). Less is known about how individuals with epilepsy process these forms of social stimuli, with a particular gap in knowledge about representation of vocal cues in the developing brain. The current study compared vocal emotion recognition skills and functional patterns of neural activation to emotional voices in youth with and without refractory focal epilepsy. We made novel use of inter-subject pattern analysis to determine brain areas in which activation to emotional voices was predictive of epilepsy status. Results indicated that youth with epilepsy were comparatively less able to infer emotional intent in vocal expressions than their typically developing peers. Activation to vocal emotional expressions in regions of the mentalizing and/or default mode network (e.g., right temporo-parietal junction, right hippocampus, right medial prefrontal cortex, among others) differentiated youth with and without epilepsy. These results are consistent with emerging evidence that pediatric epilepsy is associated with altered function in neural networks subserving social cognitive abilities. Our results contribute to ongoing efforts to understand the neural markers of social cognitive deficits in pediatric epilepsy, in order to better tailor and funnel interventions to this group of youth at risk for poor social outcomes.

Longitudinal Change in Neural Response to Vocal Emotion in Adolescence

Adolescence is associated with maturation of function within neural networks supporting the processing of social information. Previous longitudinal studies have established developmental influences on youth’s neural response to facial displays of emotion. Given the increasing recognition of the importance of non-facial cues to social communication, we build on existing work by examining longitudinal change in neural response to vocal expressions of emotion in 8- to 19-year-old youth. Participants completed a vocal emotion recognition task at two timepoints (1 year apart) while undergoing functional magnetic resonance imaging. The right inferior frontal gyrus, right dorsal striatum and right precentral gyrus showed decreases in activation to emotional voices across timepoints, which may reflect focalization of response in these areas. Activation in the dorsomedial prefrontal cortex was positively associated with age but was stable across timepoints. In addition, the slope of change across visits varied as a function of participants’ age in the right temporo-parietal junction (TPJ): this pattern of activation across timepoints and age may reflect ongoing specialization of function across childhood and adolescence. Decreased activation in the striatum and TPJ across timepoints was associated with better emotion recognition accuracy. Findings suggest that specialization of function in social cognitive networks may support the growth of vocal emotion recognition skills across adolescence.

Abnormal singing can identify patients with right hemisphere cortical strokes at risk for impaired prosody

Despite lacking aphasia seen with left hemisphere (LH) infarcts involving the middle cerebral artery territory, right hemisphere (RH) strokes can result in significant difficulties in affective prosody. These impairments may be more difficult to identify but lead to significant communication problems.We determine if evaluation of singing can accurately identify stroke patients with cortical RH infarcts at risk for prosodic impairment who may benefit from rehabilitation.A prospective cohort of 36 patients evaluated with acute ischemic stroke was recruited. Participants underwent an experimental battery evaluating their singing, prosody comprehension, and prosody production. Singing samples were rated by 2 independent reviewers as subjectively "normal" or "abnormal," and analyzed for properties of the fundamental frequency. Relationships between infarct location, singing, and prosody performance were evaluated using t tests and chi-squared analysis.Eighty percent of participants with LH cortical strokes were unable to successfully complete any of the tasks due to severe aphasia. For the remainder, singing ratings corresponded to stroke location for 68% of patients. RH cortical strokes demonstrated a lower mean fundamental frequency while singing than those with subcortical infarcts (176.8 vs 130.4, P = 0.02). They also made more errors on tasks of prosody comprehension (28.6 vs 16.0, P < 0.001) and production (40.4 vs 18.4, P < 0.001).Patients with RH cortical infarcts are more likely to exhibit impaired prosody comprehension and production and demonstrate the poor variation of tone when singing compared to patients with subcortical infarcts. A simple singing screen is able to successfully identify patients with cortical lesions and potential prosodic deficits.

Functional anatomy of idiomatic expressions

Idiomatic expressions (IE) are groups of words whose meaning is different from the sum of its components. Neural mechanisms underlying their processing are still debated, especially regarding lateralization, main structures involved, and whether this neural network is independent from the spoken language. To investigate the neural correlates of IE processing in healthy Spanish speakers.Twenty one native speakers of Spanish were asked to select one of 4 possible meanings for IE or literal sentences. fMRI scans were performed in a 3.0T scanner and processed by SPM 12 comparing IE vs. literal sentences. Laterality indices were calculated at the group level. IE activated a bilateral, slightly right-sided network comprising the pars triangularis and areas 9 and 10. In the left hemisphere (LH): the pars orbitalis, superior frontal, angular and fusiform gyrus. In the right hemisphere (RH): anterior insula, middle frontal, and superior temporal gyrus. This network reveals the importance of the RH, besides traditional LH areas, to comprehend IE. This agrees with the semantic coding model: the LH activates narrow semantic fields choosing one single meaning and ignoring others, and the RH detects distant semantic relationships, activating diffuse semantic fields. It is also in line with the configuration hypothesis: both meanings, literal and figurative, are executed simultaneously, until the literal meaning is definitively rejected and the figurative one is accepted. Processing IE requires the activation of fronto-temporal networks in both hemispheres. The results concur with previous studies in other languages, so these networks are independent from the spoken language. Understanding these mechanisms sheds light on IE processing difficulties in different clinical populations and must be considered when planning resective surgery.

Sensory contribution to vocal emotion deficit in patients with cerebellar stroke

In recent years, there has been increasing evidence of cerebellar involvement in emotion processing. Difficulties in the recognition of emotion from voices (i.e., emotional prosody) have been observed following cerebellar stroke. However, the interplay between sensory and higher-order cognitive dysfunction in these deficits, as well as possible hemispheric specialization for emotional prosody processing, has yet to be elucidated. We investigated the emotional prosody recognition performances of patients with right versus left cerebellar lesions, as well as of matched controls, entering the acoustic features of the stimuli in our statistical model. We also explored the cerebellar lesion-behavior relationship, using voxel-based lesion-symptom mapping. Results revealed impairment of vocal emotion recognition in both patient subgroups, particularly for neutral or negative prosody, with a higher number of misattributions in patients with right-hemispheric stroke. Voxel-based lesion-symptom mapping showed that some emotional misattributions correlated with lesions in the right Lobules VIIb and VIII and right Crus I and II. Furthermore, a significant proportion of the variance in this misattribution was explained by acoustic features such as pitch, loudness, and spectral aspects. These results point to bilateral posterior cerebellar involvement in both the sensory and cognitive processing of emotions.

Nonverbal auditory communication - Evidence for integrated neural systems for voice signal production and perception

While humans have developed a sophisticated and unique system of verbal auditory communication, they also share a more common and evolutionarily important nonverbal channel of voice signaling with many other mammalian and vertebrate species. This nonverbal communication is mediated and modulated by the acoustic properties of a voice signal, and is a powerful - yet often neglected - means of sending and perceiving socially relevant information. From the viewpoint of dyadic (involving a sender and a signal receiver) voice signal communication, we discuss the integrated neural dynamics in primate nonverbal voice signal production and perception. Most previous neurobiological models of voice communication modelled these neural dynamics from the limited perspective of either voice production or perception, largely disregarding the neural and cognitive commonalities of both functions. Taking a dyadic perspective on nonverbal communication, however, it turns out that the neural systems for voice production and perception are surprisingly similar. Based on the interdependence of both production and perception functions in communication, we first propose a re-grouping of the neural mechanisms of communication into auditory, limbic, and paramotor systems, with special consideration for a subsidiary basal-ganglia-centered system. Second, we propose that the similarity in the neural systems involved in voice signal production and perception is the result of the co-evolution of nonverbal voice production and perception systems promoted by their strong interdependence in dyadic interactions.

A neurocognitive model of perceptual decision-making on emotional signals

Humans make various kinds of decisions about which emotions they perceive from others. Although it might seem like a split-second phenomenon, deliberating over which emotions we perceive unfolds across several stages of decisional processing. Neurocognitive models of general perception postulate that our brain first extracts sensory information about the world then integrates these data into a percept and lastly interprets it. The aim of the present study was to build an evidence-based neurocognitive model of perceptual decision-making on others' emotions. We conducted a series of meta-analyses of neuroimaging data spanning 30 years on the explicit evaluations of others' emotional expressions. We find that emotion perception is rather an umbrella term for various perception paradigms, each with distinct neural structures that underline task-related cognitive demands. Furthermore, the left amygdala was responsive across all classes of decisional paradigms, regardless of task-related demands. Based on these observations, we propose a neurocognitive model that outlines the information flow in the brain needed for a successful evaluation of and decisions on other individuals' emotions. HIGHLIGHTS: Emotion classification involves heterogeneous perception and decision-making tasks Decision-making processes on emotions rarely covered by existing emotions theories We propose an evidence-based neuro-cognitive model of decision-making on emotions Bilateral brain processes for nonverbal decisions, left brain processes for verbal decisions Left amygdala involved in any kind of decision on emotions.

What you say versus how you say it: Comparing sentence comprehension and emotional prosody processing using fMRI

While language processing is often described as lateralized to the left hemisphere (LH), the processing of emotion carried by vocal intonation is typically attributed to the right hemisphere (RH) and more specifically, to areas mirroring the LH language areas. However, the evidence base for this hypothesis is inconsistent, with some studies supporting right-lateralization but others favoring bilateral involvement in emotional prosody processing. Here we compared fMRI activations for an emotional prosody task with those for a sentence comprehension task in 20 neurologically healthy adults, quantifying lateralization using a lateralization index. We observed right-lateralized frontotemporal activations for emotional prosody that roughly mirrored the left-lateralized activations for sentence comprehension. In addition, emotional prosody also evoked bilateral activation in pars orbitalis (BA47), amygdala, and anterior insula. These findings are consistent with the idea that analysis of the auditory speech signal is split between the hemispheres, possibly according to their preferred temporal resolution, with the left preferentially encoding phonetic and the right encoding prosodic information. Once processed, emotional prosody information is fed to domain-general emotion processing areas and integrated with semantic information, resulting in additional bilateral activations.

Age-related differences in neural activation and functional connectivity during the processing of vocal prosody in adolescence

The ability to recognize others' emotions based on vocal emotional prosody follows a protracted developmental trajectory during adolescence. However, little is known about the neural mechanisms supporting this maturation. The current study investigated age-related differences in neural activation during a vocal emotion recognition (ER) task. Listeners aged 8 to 19 years old completed the vocal ER task while undergoing functional magnetic resonance imaging. The task of categorizing vocal emotional prosody elicited activation primarily in temporal and frontal areas. Age was associated with a) greater activation in regions in the superior, middle, and inferior frontal gyri, b) greater functional connectivity between the left precentral and inferior frontal gyri and regions in the bilateral insula and temporo-parietal junction, and c) greater fractional anisotropy in the superior longitudinal fasciculus, which connects frontal areas to posterior temporo-parietal regions. Many of these age-related differences in brain activation and connectivity were associated with better performance on the ER task. Increased activation in, and connectivity between, areas typically involved in language processing and social cognition may facilitate the development of vocal ER skills in adolescence.

Emotional prosody Stroop effect in Hindi: An event related potential study

Prosody processing is an important aspect of language comprehension. Previous research on emotional word-prosody conflict has shown that participants are worse when emotional prosody and word meaning are incongruent. Studies with event-related potentials have shown a congruency effect in N400 component. There has been no study on emotional processing in Hindi language in the context of conflict between emotional word meaning and prosody. We used happy and angry words spoken using happy and angry prosody. Participants had to identify whether the word had a happy or angry word meaning. The results showed a congruency effect with worse performance in incongruent trials indicating an emotional Stroop effect in Hindi. The ERP results showed that prosody information is detected very early, which can be seen in the N1 component. In addition, there was a congruency effect in N400. The results show that prosody is processed very early and emotional meaning-prosody congruency effect is obtained with Hindi. Further studies would be needed to investigate similarities and differences in cognitive control associated with language processing.

Is infant neural sensitivity to vocal emotion associated with mother-infant relational experience?

An early understanding of others' vocal emotions provides infants with a distinct advantage for eliciting appropriate care from caregivers and for navigating their social world. Consistent with this notion, an emerging literature suggests that a temporal cortical response to the prosody of emotional speech is observable in the first year of life. Furthermore, neural specialisation to vocal emotion in infancy may vary according to early experience. Neural sensitivity to emotional non-speech vocalisations was investigated in 29 six-month-old infants using near-infrared spectroscopy (fNIRS). Both angry and happy vocalisations evoked increased activation in the temporal cortices (relative to neutral and angry vocalisations respectively), and the strength of the angry minus neutral effect was positively associated with the degree of directiveness in the mothers' play interactions with their infant. This first fNIRS study of infant vocal emotion processing implicates bilateral temporal mechanisms similar to those found in adults and suggests that infants who experience more directive caregiving or social play may more strongly or preferentially process vocal anger by six months of age.

The Relation Between Vocal Pitch and Vocal Emotion Recognition Abilities in People with Autism Spectrum Disorder and Typical Development

We tested the relation between vocal emotion and vocal pitch perception abilities in adults with high-functioning autism spectrum disorder (ASD) and pairwise matched adults with typical development. The ASD group had impaired vocal but typical non-vocal pitch and vocal timbre perception abilities. The ASD group showed less accurate vocal emotion perception than the comparison group and vocal emotion perception abilities were correlated with traits and symptoms associated with ASD. Vocal pitch and vocal emotion perception abilities were significantly correlated in the comparison group only. Our results suggest that vocal emotion recognition difficulties in ASD might not only be based on difficulties with complex social tasks, but also on difficulties with processing of basic sensory features, such as vocal pitch.

The Functional Neuroanatomy of Lexical Tone Perception: An Activation Likelihood Estimation Meta-Analysis

In tonal language such as Chinese, lexical tone serves as a phonemic feature in determining word meaning. Meanwhile, it is close to prosody in terms of suprasegmental pitch variations and larynx-based articulation. The important yet mixed nature of lexical tone has evoked considerable studies, but no consensus has been reached on its functional neuroanatomy. This meta-analysis aimed at uncovering the neural network of lexical tone perception in comparison with that of phoneme and prosody in a unified framework. Independent Activation Likelihood Estimation meta-analyses were conducted for different linguistic elements: lexical tone by native tonal language speakers, lexical tone by non-tonal language speakers, phoneme, word-level prosody, and sentence-level prosody. Results showed that lexical tone and prosody studies demonstrated more extensive activations in the right than the left auditory cortex, whereas the opposite pattern was found for phoneme studies. Only tonal language speakers consistently recruited the left anterior superior temporal gyrus (STG) for processing lexical tone, an area implicated in phoneme processing and word-form recognition. Moreover, an anterior-lateral to posterior-medial gradient of activation as a function of element timescale was revealed in the right STG, in which the activation for lexical tone lied between that for phoneme and that for prosody. Another topological pattern was shown on the left precentral gyrus (preCG), with the activation for lexical tone overlapped with that for prosody but ventral to that for phoneme. These findings provide evidence that the neural network for lexical tone perception is hybrid with those for phoneme and prosody. That is, resembling prosody, lexical tone perception, regardless of language experience, involved right auditory cortex, with activation localized between sites engaged by phonemic and prosodic processing, suggesting a hierarchical organization of representations in the right auditory cortex. For tonal language speakers, lexical tone additionally engaged the left STG lexical mapping network, consistent with the phonemic representation. Similarly, when processing lexical tone, only tonal language speakers engaged the left preCG site implicated in prosody perception, consistent with tonal language speakers having stronger articulatory representations for lexical tone in the laryngeal sensorimotor network. A dynamic dual-stream model for lexical tone perception was proposed and discussed.

Emotional Prosody Processing in Epilepsy: Some Insights on Brain Reorganization

Drug resistant epilepsy is one of the most complex, multifactorial and polygenic neurological syndrome. Besides its dynamicity and variability, it still provides us with a model to study brain-behavior relationship, giving cues on the anatomy and functional representation of brain function. Given that onset zone of focal epileptic seizures often affects different anatomical areas, cortical but limited to one hemisphere, this condition also let us study the functional differences of the left and right cerebral hemispheres. One lateralized function in the human brain is emotional prosody, and it can be a useful ictal sign offering hints on the location of the epileptogenic zone. Besides its importance for effective communication, prosody is not considered an eloquent domain, making resective surgery on its neural correlates feasible. We performed an Electronic databases search (Medline and PsychINFO) from inception to July 2017 for studies about prosody in epilepsy. The search terms included “epilepsy,” “seizure,” “emotional prosody,” and “vocal affect.” This review focus on emotional prosody processing in epilepsy as it can give hints regarding plastic functional changes following seizures (preoperatively), resection (post operatively), and also as an ictal sign enabling the assessment of dynamic brain networks. Moreover, it is argued that such reorganization can help to preserve the expression and reception of emotional prosody as a central skill to develop appropriate social interactions.

Speech Prosodies of Different Emotional Categories Activate Different Brain Regions in Adult Cortex: an fNIRS Study

Emotional expressions of others embedded in speech prosodies are important for social interactions. This study used functional near-infrared spectroscopy to investigate how speech prosodies of different emotional categories are processed in the cortex. The results demonstrated several cerebral areas critical for emotional prosody processing. We confirmed that the superior temporal cortex, especially the right middle and posterior parts of superior temporal gyrus (BA 22/42), primarily works to discriminate between emotional and neutral prosodies. Furthermore, the results suggested that categorization of emotions occurs within a high-level brain region–the frontal cortex, since the brain activation patterns were distinct when positive (happy) were contrasted to negative (fearful and angry) prosody in the left middle part of inferior frontal gyrus (BA 45) and the frontal eye field (BA8), and when angry were contrasted to neutral prosody in bilateral orbital frontal regions (BA 10/11). These findings verified and extended previous fMRI findings in adult brain and also provided a “developed version” of brain activation for our following neonatal study.

Emotional prosody processing in autism spectrum disorder

Individuals with Autism Spectrum Disorder (ASD) are characterized by severe deficits in social communication, whereby the nature of their impairments in emotional prosody processing have yet to be specified. Here, we investigated emotional prosody processing in individuals with ASD and controls with novel, lifelike behavioral and neuroimaging paradigms. Compared to controls, individuals with ASD showed reduced emotional prosody recognition accuracy on a behavioral task. On the neural level, individuals with ASD displayed reduced activity of the STS, insula and amygdala for complex vs basic emotions compared to controls. Moreover, the coupling between the STS and amygdala for complex vs basic emotions was reduced in the ASD group. Finally, groups differed with respect to the relationship between brain activity and behavioral performance. Brain activity during emotional prosody processing was more strongly related to prosody recognition accuracy in ASD participants. In contrast, the coupling between STS and anterior cingulate cortex (ACC) activity predicted behavioral task performance more strongly in the control group. These results provide evidence for aberrant emotional prosody processing of individuals with ASD. They suggest that the differences in the relationship between the neural and behavioral level of individuals with ASD may account for their observed deficits in social communication.

Identifying disease foci from static and dynamic effective connectivity networks: Illustration in soldiers with trauma

Brain connectivity studies report group differences in pairwise connection strengths. While informative, such results are difficult to interpret since our understanding of the brain relies on region-based properties, rather than on connection information. Given that large disruptions in the brain are often caused by a few pivotal sources, we propose a novel framework to identify the sources of functional disruption from effective connectivity networks. Our approach integrates static and time-varying effective connectivity modeling in a probabilistic framework, to identify aberrant foci and the corresponding aberrant connectomics network. Using resting-state fMRI, we illustrate the utility of this novel approach in U.S. Army soldiers (N = 87) with posttraumatic stress disorder (PTSD), mild traumatic brain injury (mTBI) and combat controls. Additionally, we employed machine-learning classification to identify those significant connectivity features that possessed high predictive ability. We identified three disrupted foci (middle frontal gyrus [MFG], insula, hippocampus), and an aberrant prefrontal-subcortical-parietal network of information flow. We found the MFG to be the pivotal focus of network disruption, with aberrant strength and temporal-variability of effective connectivity to the insula, amygdala and hippocampus. These connectivities also possessed high predictive ability (giving a classification accuracy of 81%); and they exhibited significant associations with symptom severity and neurocognitive functioning. In summary, dysregulation originating in the MFG caused elevated and temporally less-variable connectivity in subcortical regions, followed by a similar effect on parietal memory-related regions. This mechanism likely contributes to the reduced control over traumatic memories leading to re-experiencing, hyperarousal and flashbacks observed in soldiers with PTSD and mTBI. Hum Brain Mapp 39:264-287, 2018. © 2017 Wiley Periodicals, Inc.

Mobile technology for mental health assessment

Assessment and outcome monitoring are critical for the effective detection and treatment of mental illness. Traditional methods of capturing social, functional, and behavioral data are limited to the information that patients report back to their health care provider at selected points in time. As a result, these data are not accurate accounts of day-to-day functioning, as they are often influenced by biases in self-report. Mobile technology (mobile applications on smartphones, activity bracelets) has the potential to overcome such problems with traditional assessment and provide information about patient symptoms, behavior, and functioning in real time. Although the use of sensors and apps are widespread, several questions remain in the field regarding the reliability of off-the-shelf apps and sensors, use of these tools by consumers, and provider use of these data in clinical decision-making.

Consensus Paper: Cerebellum and Emotion

Over the past three decades, insights into the role of the cerebellum in emotional processing have substantially increased. Indeed, methodological refinements in cerebellar lesion studies and major technological advancements in the field of neuroscience are in particular responsible to an exponential growth of knowledge on the topic. It is timely to review the available data and to critically evaluate the current status of the role of the cerebellum in emotion and related domains. The main aim of this article is to present an overview of current facts and ongoing debates relating to clinical, neuroimaging, and neurophysiological findings on the role of the cerebellum in key aspects of emotion. Experts in the field of cerebellar research discuss the range of cerebellar contributions to emotion in nine topics. Topics include the role of the cerebellum in perception and recognition, forwarding and encoding of emotional information, and the experience and regulation of emotional states in relation to motor, cognitive, and social behaviors. In addition, perspectives including cerebellar involvement in emotional learning, pain, emotional aspects of speech, and neuropsychiatric aspects of the cerebellum in mood disorders are briefly discussed. Results of this consensus paper illustrate how theory and empirical research have converged to produce a composite picture of brain topography, physiology, and function that establishes the role of the cerebellum in many aspects of emotional processing.

Neural systems for evaluating speaker (Un)believability

Our voice provides salient cues about how confident we sound, which promotes inferences about how believable we are. However, the neural mechanisms involved in these social inferences are largely unknown. Employing functional magnetic resonance imaging, we examined the brain networks and individual differences underlying the evaluation of speaker believability from vocal expressions. Participants (n = 26) listened to statements produced in a confident, unconfident, or "prosodically unmarked" (neutral) voice, and judged how believable the speaker was on a 4-point scale. We found frontal-temporal networks were activated for different levels of confidence, with the left superior and inferior frontal gyrus more activated for confident statements, the right superior temporal gyrus for unconfident expressions, and bilateral cerebellum for statements in a neutral voice. Based on listener's believability judgment, we observed increased activation in the right superior parietal lobule (SPL) associated with higher believability, while increased left posterior central gyrus (PoCG) was associated with less believability. A psychophysiological interaction analysis found that the anterior cingulate cortex and bilateral caudate were connected to the right SPL when higher believability judgments were made, while supplementary motor area was connected with the left PoCG when lower believability judgments were made. Personal characteristics, such as interpersonal reactivity and the individual tendency to trust others, modulated the brain activations and the functional connectivity when making believability judgments. In sum, our data pinpoint neural mechanisms that are involved when inferring one's believability from a speaker's voice and establish ways that these mechanisms are modulated by individual characteristics of a listener. Hum Brain Mapp 38:3732-3749, 2017. © 2017 Wiley Periodicals, Inc.

A comparative neurological approach to emotional expressions in primate vocalizations

Different approaches from different research domains have crystallized debate over primate emotional processing and vocalizations in recent decades. On one side, researchers disagree about whether emotional states or processes in animals truly compare to those in humans. On the other, a long-held assumption is that primate vocalizations are innate communicative signals over which nonhuman primates have limited control and a mirror of the emotional state of the individuals producing them, despite growing evidence of intentional production for some vocalizations. Our goal is to connect both sides of the discussion in deciphering how the emotional content of primate calls compares with emotional vocal signals in humans. We focus particularly on neural bases of primate emotions and vocalizations to identify cerebral structures underlying emotion, vocal production, and comprehension in primates, and discuss whether particular structures or neuronal networks solely evolved for specific functions in the human brain. Finally, we propose a model to classify emotional vocalizations in primates according to four dimensions (learning, control, emotional, meaning) to allow comparing calls across species.

Brain responses to emotional salience and reward in alcohol use disorder

Heightened neural responsiveness of alcoholics to alcohol cues and social emotion may impede sobriety. To test mesocorticolimbic network responsivity, 10 (8 men) alcohol use disorder (AUD) patients sober for 3 weeks to 10 months and 11 (8 men) controls underwent fMRI whilst viewing pictures of alcohol and non-alcohol beverages and of emotional faces (happy, sad, angry). AUD and controls showed similarities in mesocorticolimbic activity: both groups activated fusiform for emotional faces and hippocampal and pallidum regions during alcohol picture processing. In AUD, less fusiform activity to emotional faces and more pallidum activity to alcohol pictures were associated with longer sobriety. Using graph theory-based network efficiency measures to specify the role of the mesocorticolimbic network nodes for emotion and reward in sober AUD revealed that the left hippocampus was less efficiently connected with the other task-activated network regions in AUD than controls when viewing emotional faces, while the pallidum was more efficiently connected when viewing alcohol beverages. Together our findings identified lower occipito-temporal sensitivity to emotional faces and enhanced striatal sensitivity to alcohol stimuli in AUD than controls. Considering the role of the striatum in encoding reward, its activation enhancement with longer sobriety may reflect adaptive neural changes in the first year of drinking cessation and mesocorticolimbic system vulnerability for encoding emotional salience and reward potentially affecting executive control ability and relapse propensity during abstinence.

The sound of emotions-Towards a unifying neural network perspective of affective sound processing

Affective sounds are an integral part of the natural and social environment that shape and influence behavior across a multitude of species. In human primates, these affective sounds span a repertoire of environmental and human sounds when we vocalize or produce music. In terms of neural processing, cortical and subcortical brain areas constitute a distributed network that supports our listening experience to these affective sounds. Taking an exhaustive cross-domain view, we accordingly suggest a common neural network that facilitates the decoding of the emotional meaning from a wide source of sounds rather than a traditional view that postulates distinct neural systems for specific affective sound types. This new integrative neural network view unifies the decoding of affective valence in sounds, and ascribes differential as well as complementary functional roles to specific nodes within a common neural network. It also highlights the importance of an extended brain network beyond the central limbic and auditory brain systems engaged in the processing of affective sounds.

Generating an item pool for translational social cognition research: methodology and initial validation

Existing sets of social and emotional stimuli suitable for social cognition research are limited in many ways, including size, unimodal stimulus delivery, and restriction to major universal emotions. Existing measures of social cognition could be improved by taking advantage of item response theory and adaptive testing technology to develop instruments that obtain more efficient measures of multimodal social cognition. However, for this to be possible, large pools of emotional stimuli must be obtained and validated. We present the development of a large, high-quality multimedia stimulus set produced by professional adult and child actors (ages 5 to 74) containing both visual and vocal emotional expressions. We obtained over 74,000 audiovisual recordings of a wide array of emotional and social behaviors, including the main universal emotions (happiness, sadness, anger, fear, disgust, and surprise), as well as more complex social expressions (pride, affection, sarcasm, jealousy, and shame). The actors generated a high quantity of technically superior, ecologically valid stimuli that were digitized, archived, and rated for accuracy and intensity of expressions. A subset of these facial and vocal expressions of emotion and social behavior were submitted for quantitative ratings to generate parameters for validity and discriminability. These stimuli are suitable for affective neuroscience-based psychometric tests, functional neuroimaging, and social cognitive rehabilitation programs. The purposes of this report are to describe the method of obtaining and validating this database and to make it accessible to the scientific community. We invite all those interested in participating in the use and validation of these stimuli to access them at www.med.upenn.edu/bbl/actors/index.shtml .

A possible functional localizer for identifying brain regions sensitive to sentence-level prosody

Investigations of how we produce and perceive prosodic patterns are not only interesting in their own right but can inform fundamental questions in language research. We here argue that functional magnetic resonance imaging (fMRI) in general - and the functional localization approach in particular (e.g., Kanwisher et al., 1997; Saxe et al., 2006; Fedorenko et al., 2010; Nieto-Castañon & Fedorenko, 2012) - has the potential to help address open research questions in prosody research and at the intersection of prosody and other domains. Critically, this approach can go beyond questions like "where in the brain does mental process x produce activation" and toward questions that probe the nature of the representations and computations that subserve different mental abilities. We describe one way to functionally define regions sensitive to sentence-level prosody in individual subjects. This or similar "localizer" contrasts can be used in future studies to test hypotheses about the precise contributions of prosody-sensitive brain regions to prosodic processing and cognition more broadly.

Transcranial Magnetic Stimulation and Deep Brain Stimulation in the treatment of alcohol dependence

Alcohol dependence is a major social, economic, and public health problem. Alcoholism can lead to damage of the gastrointestinal, nervous, cardiovascular, and respiratory systems and it can be lethal, costing hundreds of billions to the health care system. Despite the existence of cognitive-behavioral therapy, psychosocial interventions, and spiritually integrated treatment to treat it, alcohol dependence has a high relapse rate and poor prognosis, albeit with high interindividual variability. In this review, we discuss the use of two neuromodulation techniques, namely repetitive transcranial magnetic stimulation (rTMS) and deep brain stimulation (DBS), and their advantages and disadvantages compared to first-line pharmacological treatment for alcohol dependence. We also discuss rTMS and DBS targets for alcohol dependence treatment, considering experimental animal and human evidence, with careful consideration of methodological issues preventing the identification of feasible targets for neuromodulation treatments, as well as inter-individual variability factors influencing alcoholism prognosis. Lastly, we anticipate future research aiming to tailor the treatment to each individual patient by combining neurofunctional, neuroanatomical and neurodisruptive techniques optimizing the outcome.

The role of the medial temporal limbic system in processing emotions in voice and music

Subcortical brain structures of the limbic system, such as the amygdala, are thought to decode the emotional value of sensory information. Recent neuroimaging studies, as well as lesion studies in patients, have shown that the amygdala is sensitive to emotions in voice and music. Similarly, the hippocampus, another part of the temporal limbic system (TLS), is responsive to vocal and musical emotions, but its specific roles in emotional processing from music and especially from voices have been largely neglected. Here we review recent research on vocal and musical emotions, and outline commonalities and differences in the neural processing of emotions in the TLS in terms of emotional valence, emotional intensity and arousal, as well as in terms of acoustic and structural features of voices and music. We summarize the findings in a neural framework including several subcortical and cortical functional pathways between the auditory system and the TLS. This framework proposes that some vocal expressions might already receive a fast emotional evaluation via a subcortical pathway to the amygdala, whereas cortical pathways to the TLS are thought to be equally used for vocal and musical emotions. While the amygdala might be specifically involved in a coarse decoding of the emotional value of voices and music, the hippocampus might process more complex vocal and musical emotions, and might have an important role especially for the decoding of musical emotions by providing memory-based and contextual associations.

The effect of criticism on functional brain connectivity and associations with neuroticism

Neuroticism is a robust personality trait that constitutes a risk factor for psychopathology, especially anxiety disorders and depression. High neurotic individuals tend to be more self-critical and are overly sensitive to criticism by others. Hence, we used a novel resting-state paradigm to investigate the effect of criticism on functional brain connectivity and associations with neuroticism. Forty-eight participants completed the NEO Personality Inventory Revised (NEO-PI-R) to assess neuroticism. Next, we recorded resting state functional magnetic resonance imaging (rsfMRI) during two sessions. We manipulated the second session before scanning by presenting three standardized critical remarks through headphones, in which the subject was urged to please lie still in the scanner. A seed-based functional connectivity method and subsequent clustering were used to analyse the resting state data. Based on the reviewed literature related to criticism, we selected brain regions associated with self-reflective processing and stress-regulation as regions of interest. The findings showed enhanced functional connectivity between the clustered seed regions and brain areas involved in emotion processing and social cognition during the processing of criticism. Concurrently, functional connectivity was reduced between these clusters and brain structures related to the default mode network and higher-order cognitive control. Furthermore, individuals scoring higher on neuroticism showed altered functional connectivity between the clustered seed regions and brain areas involved in the appraisal, expression and regulation of negative emotions. These results may suggest that the criticized person is attempting to understand the beliefs, perceptions and feelings of the critic in order to facilitate flexible and adaptive social behavior. Furthermore, multiple aspects of emotion processing were found to be affected in individuals scoring higher on neuroticism during the processing of criticism, which may increase their sensitivity to negative social-evaluation.

Valence, arousal, and task effects in emotional prosody processing

Previous research suggests that emotional prosody processing is a highly rapid and complex process. In particular, it has been shown that different basic emotions can be differentiated in an early event-related brain potential (ERP) component, the P200. Often, the P200 is followed by later long lasting ERPs such as the late positive complex. The current experiment set out to explore in how far emotionality and arousal can modulate these previously reported ERP components. In addition, we also investigated the influence of task demands (implicit vs. explicit evaluation of stimuli). Participants listened to pseudo-sentences (sentences with no lexical content) spoken in six different emotions or in a neutral tone of voice while they either rated the arousal level of the speaker or their own arousal level. Results confirm that different emotional intonations can first be differentiated in the P200 component, reflecting a first emotional encoding of the stimulus possibly including a valence tagging process. A marginal significant arousal effect was also found in this time-window with high arousing stimuli eliciting a stronger P200 than low arousing stimuli. The P200 component was followed by a long lasting positive ERP between 400 and 750 ms. In this late time-window, both emotion and arousal effects were found. No effects of task were observed in either time-window. Taken together, results suggest that emotion relevant details are robustly decoded during early processing and late processing stages while arousal information is only reliably taken into consideration at a later stage of processing.

Emotional prosody modulates attention in schizophrenia patients with hallucinations

Recent findings have demonstrated that emotional prosody (EP) attracts attention involuntarily (Grandjean et al., 2008). The automat shift of attention toward emotionally salient stimuli can be overcome by attentional control (Hahn et al., 2010). Attentional control is impaired in schizophrenia, especially in schizophrenic patients with hallucinations because the "voices" capture attention increasing the processing load and competing for top-down resources. The present study investigates how involuntary attention is driven by implicit EP in schizophrenia with auditory verbal hallucinations (AVH) and without (NAVH). Fifteen AVH patients, 12 NAVH patients and 16 healthy controls (HC) completed a dual-task dichotic listening paradigm, in which an emotional vocal outburst was paired with a neutral vocalization spoken in male and female voices. Participants were asked to report the speaker's gender while attending to either the left or right ear. NAVH patients and HC revealed shorter response times for stimuli presented to the attended left ear than the attended right ear. This laterality effect was not present in AVH patients. In addition, NAVH patients and HC showed faster responses when the EP stimulus was presented to the unattended ear, probably because of less interference between the attention-controlled gender voice identification task and involuntary EP processing. AVH patients did not benefit from presenting emotional stimuli to the unattended ear. The findings suggest that similar to HC, NAVH patients show a right hemispheric bias for EP processing. AVH patients seem to be less lateralized for EP and therefore might be more susceptible to interfering involuntary EP processing; regardless which ear/hemisphere receives the bottom up input.

Attitudinal prosody: what we know and directions for future study

Prosodic aspects of speech such as pitch, duration and amplitude constitute nonverbal cues that supplement or modify the meaning of the spoken word, to provide valuable clues as to a speakers' state of mind. It can thus indicate what emotion a person is feeling (emotional prosody), or their attitude towards an event, person or object (attitudinal prosody). Whilst the study of emotional prosody has gathered pace, attitudinal prosody now deserves equal attention. In social cognition, understanding attitudinal prosody is important in its own right, since it can convey powerful constructs such as confidence, persuasion, sarcasm and superiority. In this review, it is examined what prosody is, how it conveys attitudes, and which attitudes prosody can convey. The review finishes by considering the neuroanatomy associated with attitudinal prosody, and put forward the hypothesis that this cognition is mediated by the right cerebral hemisphere, particularly posterior superior lateral temporal cortex, with an additional role for the basal ganglia, and limbic regions such as the medial prefrontal cortex and amygdala. It is suggested that further exploration of its functional neuroanatomy is greatly needed, since it could provide valuable clues about the value of current prosody nomenclature and its separability from other types of prosody at the behavioural level.

Social cognitive deficits and their neural correlates in progressive supranuclear palsy

Although progressive supranuclear palsy is defined by its akinetic rigidity, vertical supranuclear gaze palsy and falls, cognitive impairments are an important determinant of patients’ and carers’ quality of life. Here, we investigate whether there is a broad deficit of modality-independent social cognition in progressive supranuclear palsy and explore the neural correlates for these. We recruited 23 patients with progressive supranuclear palsy (using clinical diagnostic criteria, nine with subsequent pathological confirmation) and 22 age- and education-matched controls. Participants performed an auditory (voice) emotion recognition test, and a visual and auditory theory of mind test. Twenty-two patients and 20 controls underwent structural magnetic resonance imaging to analyse neural correlates of social cognition deficits using voxel-based morphometry. Patients were impaired on the voice emotion recognition and theory of mind tests but not auditory and visual control conditions. Grey matter atrophy in patients correlated with both voice emotion recognition and theory of mind deficits in the right inferior frontal gyrus, a region associated with prosodic auditory emotion recognition. Theory of mind deficits also correlated with atrophy of the anterior rostral medial frontal cortex, a region associated with theory of mind in health. We conclude that patients with progressive supranuclear palsy have a multimodal deficit in social cognition. This deficit is due, in part, to progressive atrophy in a network of frontal cortical regions linked to the integration of socially relevant stimuli and interpretation of their social meaning. This impairment of social cognition is important to consider for those managing and caring for patients with progressive supranuclear palsy.

Contributions of emotional prosody comprehension deficits to the formation of auditory verbal hallucinations in schizophrenia

Deficits in emotional processing have been widely described in schizophrenia. Associations of positive symptoms with poor emotional prosody comprehension (EPC) have been reported at the phenomenological, behavioral, and neural levels. This review focuses on the relation between emotional processing deficits and auditory verbal hallucinations (AVH). We explore the possibility that the relation between AVH and EPC in schizophrenia might be mediated by the disruption of a common mechanism intrinsic to auditory processing, and that, moreover, prosodic feature processing deficits play a pivotal role in the formation of AVH. The review concludes with proposing a mechanism by which AVH are constituted and showing how different aspects of our neuropsychological model can explain the constellation of subjective experiences which occur in relation to AVH.