New tests to measure individual differences in matching and labelling facial expressions of emotion, and their association with ability to recognise vocal emotions and facial identity

Although good tests are available for diagnosing clinical impairments in face expression processing, there is a lack of strong tests for assessing "individual differences"--that is, differences in ability between individuals within the typical, nonclinical, range. Here, we develop two new tests, one for expression perception (an odd-man-out matching task in which participants select which one of three faces displays a different expression) and one additionally requiring explicit identification of the emotion (a labelling task in which participants select one of six verbal labels). We demonstrate validity (careful check of individual items, large inversion effects, independence from nonverbal IQ, convergent validity with a previous labelling task), reliability (Cronbach's alphas of.77 and.76 respectively), and wide individual differences across the typical population. We then demonstrate the usefulness of the tests by addressing theoretical questions regarding the structure of face processing, specifically the extent to which the following processes are common or distinct: (a) perceptual matching and explicit labelling of expression (modest correlation between matching and labelling supported partial independence); (b) judgement of expressions from faces and voices (results argued labelling tasks tap into a multi-modal system, while matching tasks tap distinct perceptual processes); and (c) expression and identity processing (results argued for a common first step of perceptual processing for expression and identity).

A unified coding strategy for processing faces and voices

Both faces and voices are rich in socially-relevant information, which humans are remarkably adept at extracting, including a person's identity, age, gender, affective state, personality, etc. Here, we review accumulating evidence from behavioral, neuropsychological, electrophysiological, and neuroimaging studies which suggest that the cognitive and neural processing mechanisms engaged by perceiving faces or voices are highly similar, despite the very different nature of their sensory input. The similarity between the two mechanisms likely facilitates the multi-modal integration of facial and vocal information during everyday social interactions. These findings emphasize a parsimonious principle of cerebral organization, where similar computational problems in different modalities are solved using similar solutions.

Norm-based coding of voice identity in human auditory cortex

Listeners exploit small interindividual variations around a generic acoustical structure to discriminate and identify individuals from their voice—a key requirement for social interactions. The human brain contains temporal voice areas (TVA) [1] involved in an acoustic-based representation of voice identity [2, 3, 4, 5, 6], but the underlying coding mechanisms remain unknown. Indirect evidence suggests that identity representation in these areas could rely on a norm-based coding mechanism [4, 7, 8, 9, 10, 11]. Here, we show by using fMRI that voice identity is coded in the TVA as a function of acoustical distance to two internal voice prototypes (one male, one female)—approximated here by averaging a large number of same-gender voices by using morphing [12]. Voices more distant from their prototype are perceived as more distinctive and elicit greater neuronal activity in voice-sensitive cortex than closer voices—a phenomenon not merely explained by neuronal adaptation [13, 14]. Moreover, explicit manipulations of distance-to-mean by morphing voices toward (or away from) their prototype elicit reduced (or enhanced) neuronal activity. These results indicate that voice-sensitive cortex integrates relevant acoustical features into a complex representation referenced to idealized male and female voice prototypes. More generally, they shed light on remarkable similarities in cerebral representations of facial and vocal identity.

•

Identity coding in temporal voice area in terms of acoustical distance to prototypes

•

Description of the “voice space” in terms of simple acoustical measures

•

Male and female prototypes are ideally smooth versions of the population means

•

Comparable coding mechanism for identity across sensory modalities

Sex matters: Neural correlates of voice gender perception

The basis for different neural activations in response to male and female voices as well as the question, whether men and women perceive male and female voices differently, has not been thoroughly investigated. Therefore, the aim of the present study was to examine the behavioral and neural correlates of gender-related voice perception in healthy male and female volunteers. fMRI data were collected while 39 participants (19 female) were asked to indicate the gender of 240 voice stimuli. These stimuli included recordings of 3-syllable nouns as well as the same recordings pitch-shifted in 2, 4 and 6 semitone steps in the direction of the other gender. Data analysis revealed a) equal voice discrimination sensitivity in men and women but better performance in the categorization of opposite-sex stimuli at least in men, b) increased responses to increasing gender ambiguity in the mid cingulate cortex and bilateral inferior frontal gyri, and c) stronger activation in a fronto-temporal neural network in response to voices of the opposite sex. Our results indicate a gender specific processing for male and female voices on a behavioral and neuronal level. We suggest that our results reflect higher sensitivity probably due to the evolutionary relevance of voice perception in mate selection.

Voices to reckon with: perceptions of voice identity in clinical and non-clinical voice hearers

The current review focuses on the perception of voice identity in clinical and non-clinical voice hearers. Identity perception in auditory verbal hallucinations (AVH) is grounded in the mechanisms of human (i.e., real, external) voice perception, and shapes the emotional (distress) and behavioral (help-seeking) response to the experience. Yet, the phenomenological assessment of voice identity is often limited, for example to the gender of the voice, and has failed to take advantage of recent models and evidence on human voice perception. In this paper we aim to synthesize the literature on identity in real and hallucinated voices and begin by providing a comprehensive overview of the features used to judge voice identity in healthy individuals and in people with schizophrenia. The findings suggest some subtle, but possibly systematic biases across different levels of voice identity in clinical hallucinators that are associated with higher levels of distress. Next we provide a critical evaluation of voice processing abilities in clinical and non-clinical voice hearers, including recent data collected in our laboratory. Our studies used diverse methods, assessing recognition and binding of words and voices in memory as well as multidimensional scaling of voice dissimilarity judgments. The findings overall point to significant difficulties recognizing familiar speakers and discriminating between unfamiliar speakers in people with schizophrenia, both with and without AVH. In contrast, these voice processing abilities appear to be generally intact in non-clinical hallucinators. The review highlights some important avenues for future research and treatment of AVH associated with a need for care, and suggests some novel insights into other symptoms of psychosis.

Voice identity discrimination in schizophrenia

Voices provide a wealth of socially-relevant information, including cues to a speaker's identity and emotion. Deficits recognising emotion from voice have been extensively described in schizophrenia, and linked specifically to auditory hallucinations (AH), but relatively little attention has been given to examining the ability to analyse speaker identity. Hence, the current study assessed the ability to discriminate between different speakers in people with schizophrenia (including 33 with and 32 without AH) compared to 32 healthy controls. Participants rated the degree of perceived identity similarity of pairs of unfamiliar voices pronouncing three-syllable words. Multidimensional scaling of the dissimilarity matrices was performed and the resulting dimensions were interpreted, a posteriori, via correlations with acoustic measures relevant to voice identity. A two-dimensional perceptual space was found to be appropriate for both schizophrenia patients and controls, with axes corresponding to the average fundamental frequency (F0) and formant dispersion (D(f)). Patients with schizophrenia did not differ from healthy controls in their reliance on F0 in differentiating voices, suggesting that the ability to use pitch-based cues for discriminating voice identity may be relatively preserved in schizophrenia. On the other hand, patients (both with and without AH) made less use of D(f) in discriminating voices compared to healthy controls. This distorted pattern of responses suggests some potentially important differences in voice identity processing in schizophrenia. Formant dispersion has been linked to perceptions of dominance, masculinity, size and age in healthy individuals. These findings open some interesting new directions for future research.

Cognitive mechanisms of auditory verbal hallucinations in psychotic and non-psychotic groups

The continuum model of psychosis has been extremely influential. It assumes that psychotic symptoms, such as auditory verbal hallucinations (AVH), are not limited to patients with psychosis but also occur in healthy, non-clinical individuals - suggesting similar mechanisms of origin. Recent debate surrounding this model has highlighted certain differences, as well as similarities, in the phenomenology of AVH in clinical and non-clinical populations. These findings imply that there may, in fact, be only partial overlap of the mechanism(s) involved in generating AVH in these groups. We review evidence of continuity or similarity, and dissimilarity, in cognitive, and related neural processes, underlying AVH in clinical and non-clinical samples. The results reveal some shared (intrusive cognitions, inhibitory deficits) and some distinct (aspects of source memory and cerebral lateralization) mechanisms in these groups. The evidence, therefore, supports both continuous and categorical models of positive psychotic symptoms. The review considers potential risks of uncritical acceptance of the continuum model and highlights some important methodological issues for future research.