Idiosyncratic and shared contributions shape impressions from voices and faces

Voices elicit rich first impressions of what the person we are hearing might be like. Research stresses that these impressions from voices are shared across different listeners, such that people on average agree which voices sound trustworthy or old and which do not. However, can impressions from voices also be shaped by the 'ear of the beholder'? We investigated whether - and how - listeners' idiosyncratic, personal preferences contribute to first impressions from voices. In two studies (993 participants, 156 voices), we find evidence for substantial idiosyncratic contributions to voice impressions using a variance portioning approach. Overall, idiosyncratic contributions were as important as shared contributions to impressions from voices for inferred person characteristics (e.g., trustworthiness, friendliness). Shared contributions were only more influential for impressions of more directly apparent person characteristics (e.g., gender, age). Both idiosyncratic and shared contributions were reduced when stimuli were limited in their (perceived) variability, suggesting that natural variation in voices is key to understanding this impression formation. When comparing voice impressions to face impressions, we found that idiosyncratic and shared contributions to impressions similarly across modality when stimulus properties are closely matched - although voice impressions were overall less consistent than face impressions. We thus reconceptualise impressions from voices as being formed not only based on shared but also idiosyncratic contributions. We use this new framing to suggest future directions of research, including understanding idiosyncratic mechanisms, development, and malleability of voice impression formation.

The time course of person perception from voices in the brain

When listeners hear a voice, they rapidly form a complex first impression of who the person behind that voice might be. We characterize how these multivariate first impressions from voices emerge over time across different levels of abstraction using electroencephalography and representational similarity analysis. We find that for eight perceived physical (gender, age, and health), trait (attractiveness, dominance, and trustworthiness), and social characteristics (educatedness and professionalism), representations emerge early (~80 ms after stimulus onset), with voice acoustics contributing to those representations between ~100 ms and 400 ms. While impressions of person characteristics are highly correlated, we can find evidence for highly abstracted, independent representations of individual person characteristics. These abstracted representationse merge gradually over time. That is, representations of physical characteristics (age, gender) arise early (from ~120 ms), while representations of some trait and social characteristics emerge later (~360 ms onward). The findings align with recent theoretical models and shed light on the computations underpinning person perception from voices.

Sounds Pleasantness Ratings in Autism: Interaction Between Social Information and Acoustical Noise Level

A lack of response to voices, and a great interest for music are part of the behavioral expressions, commonly (self-)reported in Autism Spectrum Disorder (ASD). These atypical interests for vocal and musical sounds could be attributable to different levels of acoustical noise, quantified in the harmonic-to-noise ratio (HNR). No previous study has investigated explicit auditory pleasantness in ASD comparing vocal and non-vocal sounds, in relation to acoustic noise level. The aim of this study is to objectively evaluate auditory pleasantness. 16 adults on the autism spectrum and 16 neuro-typical (NT) matched adults rated the likeability of vocal and non-vocal sounds, with varying harmonic-to-noise ratio levels. A group by category interaction in pleasantness judgements revealed that participants on the autism spectrum judged vocal sounds as less pleasant than non-vocal sounds; an effect not found for NT participants. A category by HNR level interaction revealed that participants of both groups rated sounds with a high HNR as more pleasant for non-vocal sounds. A significant group by HNR interaction revealed that people on the autism spectrum tended to judge as less pleasant sounds with high HNR and more pleasant those with low HNR than NT participants. Acoustical noise level of sounds alone does not appear to explain atypical interest for voices and greater interest in music in ASD.

No evidence that averaging voices influences attractiveness

Vocal attractiveness influences important social outcomes. While most research on the acoustic parameters that influence vocal attractiveness has focused on the possible roles of sexually dimorphic characteristics of voices, such as fundamental frequency (i.e., pitch) and formant frequencies (i.e., a correlate of body size), other work has reported that increasing vocal averageness increases attractiveness. Here we investigated the roles these three characteristics play in judgments of the attractiveness of male and female voices. In Study 1, we found that increasing vocal averageness significantly decreased distinctiveness ratings, demonstrating that participants could detect manipulations of vocal averageness in this stimulus set and using this testing paradigm. However, in Study 2, we found no evidence that increasing averageness significantly increased attractiveness ratings of voices. In Study 3, we found that fundamental frequency was negatively correlated with male vocal attractiveness and positively correlated with female vocal attractiveness. By contrast with these results for fundamental frequency, vocal attractiveness and formant frequencies were not significantly correlated. Collectively, our results suggest that averageness may not necessarily significantly increase attractiveness judgments of voices and are consistent with previous work reporting significant associations between attractiveness and voice pitch.

No clear evidence of a difference between individuals who self-report an absence of auditory imagery and typical imagers on auditory imagery tasks

Aphantasia is characterised by the inability to create mental images in one's mind. Studies investigating impairments in imagery typically focus on the visual domain. However, it is possible to generate many different forms of imagery including imagined auditory, kinesthetic, tactile, motor, taste and other experiences. Recent studies show that individuals with aphantasia report a lack of imagery in modalities, other than vision, including audition. However, to date, no research has examined whether these reductions in self-reported auditory imagery are associated with decrements in tasks that require auditory imagery. Understanding the extent to which visual and auditory imagery deficits co-occur can help to better characterise the core deficits of aphantasia and provide an alternative perspective on theoretical debates on the extent to which imagery draws on modality-specific or modality-general processes. In the current study, individuals that self-identified as being aphantasic and matched control participants with typical imagery performed two tasks: a musical pitch-based imagery and voice-based categorisation task. The majority of participants with aphantasia self-reported significant deficits in both auditory and visual imagery. However, we did not find a concomitant decrease in performance on tasks which require auditory imagery, either in the full sample or only when considering those participants that reported significant deficits in both domains. These findings are discussed in relation to the mechanisms that might obscure observation of imagery deficits in auditory imagery tasks in people that report reduced auditory imagery.

Mapping of facial and vocal processing in common marmosets with ultra-high field fMRI

Primate communication relies on multimodal cues, such as vision and audition, to facilitate the exchange of intentions, enable social interactions, avoid predators, and foster group cohesion during daily activities. Understanding the integration of facial and vocal signals is pivotal to comprehend social interaction. In this study, we acquire whole-brain ultra-high field (9.4 T) fMRI data from awake marmosets (Callithrix jacchus) to explore brain responses to unimodal and combined facial and vocal stimuli. Our findings reveal that the multisensory condition not only intensifies activations in the occipito-temporal face patches and auditory voice patches but also engages a more extensive network that includes additional parietal, prefrontal and cingulate areas, compared to the summed responses of the unimodal conditions. By uncovering the neural network underlying multisensory audiovisual integration in marmosets, this study highlights the efficiency and adaptability of the marmoset brain in processing facial and vocal social signals, providing significant insights into primate social communication.

Left-handed voices? Examining the perceptual learning of novel person characteristics from the voice

We regularly form impressions of who a person is from their voice, such that we can readily categorise people as being female or male, child or adult, trustworthy or not, and can furthermore recognise who specifically is speaking. How we establish mental representations for such categories of person characteristics has, however, only been explored in detail for voice identity learning. In a series of experiments, we therefore set out to examine whether and how listeners can learn to recognise a novel person characteristic. We specifically asked how diagnostic acoustic properties underpinning category distinctions inform perceptual judgements. We manipulated recordings of voices to create acoustic signatures for a person's handedness (left-handed vs. right-handed) in their voice. After training, we found that listeners were able to successfully learn to recognise handedness from voices with above-chance accuracy, although no significant differences in accuracy between the different types of manipulation emerged. Listeners were, furthermore, sensitive to the specific distributions of acoustic properties that underpinned the category distinctions. We, however, also find evidence for perceptual biases that may reflect long-term prior exposure to how voices vary in naturalistic settings. These biases shape how listeners use acoustic information in the voices when forming representations for distinguishing handedness from voices. This study is thus a first step to examine how representations for novel person characteristics are established, outside of voice identity perception. We discuss our findings in light of theoretical accounts of voice perception and speculate about potential mechanisms that may underpin our results.

Perception of threat and intent to harm from vocal and facial cues

What constitutes a "threatening tone of voice"? There is currently little research exploring how listeners infer threat, or the intention to cause harm, from speakers' voices. Here, we investigated the influence of key linguistic variables on these evaluations (Study 1). Results showed a trend for voices perceived to be lower in pitch, particularly those of male speakers, to be evaluated as sounding more threatening and conveying greater intent to harm. We next investigated the evaluation of multimodal stimuli comprising voices and faces varying in perceived dominance (Study 2). Visual information about the speaker's face had a significant effect on threat and intent ratings. In both experiments, we observed a relatively low level of agreement among individual listeners' evaluations, emphasising idiosyncrasy in the ways in which threat and intent-to-harm are perceived. This research provides a basis for the perceptual experience of a "threatening tone of voice," along with an exploration of vocal and facial cue integration in social evaluation.

Familiarity Is Key: Exploring the Effect of Familiarity on the Face-Voice Correlation

Recent research has examined the extent to which face and voice processing are associated by virtue of the fact that both tap into a common person perception system. However, existing findings do not yet fully clarify the role of familiarity in this association. Given this, two experiments are presented that examine face-voice correlations for unfamiliar stimuli (Experiment 1) and for familiar stimuli (Experiment 2). With care being taken to use tasks that avoid floor and ceiling effects and that use realistic speech-based voice clips, the results suggested a significant positive but small-sized correlation between face and voice processing when recognizing unfamiliar individuals. In contrast, the correlation when matching familiar individuals was significant and positive, but much larger. The results supported the existing literature suggesting that face and voice processing are aligned as constituents of an overarching person perception system. However, the difference in magnitude of their association here reinforced the view that familiar and unfamiliar stimuli are processed in different ways. This likely reflects the importance of a pre-existing mental representation and cross-talk within the neural architectures when processing familiar faces and voices, and yet the reliance on more superficial stimulus-based and modality-specific analysis when processing unfamiliar faces and voices.

Converging evidence that left extrastriate body area supports visual sensitivity to social interactions

Navigating our complex social world requires processing the interactions we observe. Recent psychophysical and neuroimaging studies provide parallel evidence that the human visual system may be attuned to efficiently perceive dyadic interactions. This work implies, but has not yet demonstrated, that activity in body-selective cortical regions causally supports efficient visual perception of interactions. We adopt a multi-method approach to close this important gap. First, using a large fMRI dataset (n = 92), we found that the left hemisphere extrastriate body area (EBA) responds more to face-to-face than non-facing dyads. Second, we replicated a behavioral marker of visual sensitivity to interactions: categorization of facing dyads is more impaired by inversion than non-facing dyads. Third, in a pre-registered experiment, we used fMRI-guided transcranial magnetic stimulation to show that online stimulation of the left EBA, but not a nearby control region, abolishes this selective inversion effect. Activity in left EBA, thus, causally supports the efficient perception of social interactions.

Is song processing distinct and special in the auditory cortex?

Is the singing voice processed distinctively in the human brain? In this Perspective, we discuss what might distinguish song processing from speech processing in light of recent work suggesting that some cortical neuronal populations respond selectively to song and we outline the implications for our understanding of auditory processing. We review the literature regarding the neural and physiological mechanisms of song production and perception and show that this provides evidence for key differences between song and speech processing. We conclude by discussing the significance of the notion that song processing is special in terms of how this might contribute to theories of the neurobiological origins of vocal communication and to our understanding of the neural circuitry underlying sound processing in the human cortex.

A model for person perception from familiar and unfamiliar voices

When hearing a voice, listeners can form a detailed impression of the person behind the voice. Existing models of voice processing focus primarily on one aspect of person perception - identity recognition from familiar voices - but do not account for the perception of other person characteristics (e.g., sex, age, personality traits). Here, we present a broader perspective, proposing that listeners have a common perceptual goal of perceiving who they are hearing, whether the voice is familiar or unfamiliar. We outline and discuss a model - the Person Perception from Voices (PPV) model - that achieves this goal via a common mechanism of recognising a familiar person, persona, or set of speaker characteristics. Our PPV model aims to provide a more comprehensive account of how listeners perceive the person they are listening to, using an approach that incorporates and builds on aspects of the hierarchical frameworks and prototype-based mechanisms proposed within existing models of voice identity recognition.

Oxytocinergic modulation of speech production-a double-blind placebo-controlled fMRI study

Many socio-affective behaviors, such as speech, are modulated by oxytocin. While oxytocin modulates speech perception, it is not known whether it also affects speech production. Here, we investigated effects of oxytocin administration and interactions with the functional rs53576 oxytocin receptor (OXTR) polymorphism on produced speech and its underlying brain activity. During functional magnetic resonance imaging, 52 healthy male participants read sentences out loud with either neutral or happy intonation, a covert reading condition served as a common baseline. Participants were studied once under the influence of intranasal oxytocin and in another session under placebo. Oxytocin administration increased the second formant of produced vowels. This acoustic feature has previously been associated with speech valence; however, the acoustic differences were not perceptually distinguishable in our experimental setting. When preparing to speak, oxytocin enhanced brain activity in sensorimotor cortices and regions of both dorsal and right ventral speech processing streams, as well as subcortical and cortical limbic and executive control regions. In some of these regions, the rs53576 OXTR polymorphism modulated oxytocin administration-related brain activity. Oxytocin also gated cortical-basal ganglia circuits involved in the generation of happy prosody. Our findings suggest that several neural processes underlying speech production are modulated by oxytocin, including control of not only affective intonation but also sensorimotor aspects during emotionally neutral speech.

Degree of Breathiness in a Synthesized Voice Signal as it Differentiates Masculine versus Feminine Voices

INTRODUCTION

Most studies determining speakers' perceived gender as binarily female or male are reliant on F0 perception, although other vocal parameters may also contribute to the perception of gender. The current study focused on the impact of breathiness on the perception of speakers' gender as a biological variable (feminine or masculine).

METHODS

n = 31 normal hearing, native English speakers, 18 female, 13 male, mean age 23 (SD = 3.54), were auditorily and visually trained in and then took part in a categorical perception task. A continuum of nine samples of the word "hello", was created in an airway modulation model of speech and voice production. Resting vocal fold length, resting vocal fold thickness, F0, and vocal tract length were fixed. Glottal width at the vocal process, posterior glottal gap, and bronchial pressure were continually modified for all stimuli. Each stimulus was randomly presented 30 times within each of the five blocks (150 presentations in total). Participants rated stimuli as binarily female or male.

RESULTS

Showed a sigmoidal shift in breathiness along the continuum between perceived feminine or masculine voicing. This shift was evident at stimuli four and five, indicating a nonlinear, discrete perception of breathiness among participants. Response times were also significantly slower in these two stimuli, suggesting a categorical perception of breathiness among participants.

CONCLUSION

Breathiness created by the change in glottal width of at least 0.21 cm may influence the perception of a speaker's perceived gender.

Perceived naturalness of emotional voice morphs

Research into voice perception benefits from manipulation software to gain experimental control over acoustic expression of social signals such as vocal emotions. Today, parameter-specific voice morphing allows a precise control of the emotional quality expressed by single vocal parameters, such as fundamental frequency (F0) and timbre. However, potential side effects, in particular reduced naturalness, could limit ecological validity of speech stimuli. To address this for the domain of emotion perception, we collected ratings of perceived naturalness and emotionality on voice morphs expressing different emotions either through F0 or Timbre only. In two experiments, we compared two different morphing approaches, using either neutral voices or emotional averages as emotionally non-informative reference stimuli. As expected, parameter-specific voice morphing reduced perceived naturalness. However, perceived naturalness of F0 and Timbre morphs were comparable with averaged emotions as reference, potentially making this approach more suitable for future research. Crucially, there was no relationship between ratings of emotionality and naturalness, suggesting that the perception of emotion was not substantially affected by a reduction of voice naturalness. We hold that while these findings advocate parameter-specific voice morphing as a suitable tool for research on vocal emotion perception, great care should be taken in producing ecologically valid stimuli.

Young children's attributes are better conveyed by voices than by faces

The purpose of this study was to explore how young children's vocal and facial cues contribute to conveying to adults important information about children's attributes when presented together. In particular, the study aimed to disentangle whether children's vocal or facial cues, if either, are more dominant when both types of cues are displayed in a contradictory mode. To do this, we assigned 127 college students to one of three between-participants conditions. In the Voices-Only condition, participants listened to four pairs of synthetized voices simulating the voices of 4-5-year-old and 9-10-year-old children verbalizing a neutral-content sentence. Participants needed to indicate which voice was better associated with a series of 14 attributes organized into four trait dimensions (Positive Affect, Negative Affect, Intelligence, and Helpless), potentially meaningful in young child-adult interactions. In the Consistent condition, the same four pairs of voices delivered in the Voices-Only condition were presented jointly with morphed photographs of children's faces of equivalent age. In the Inconsistent condition, the four pairs of voices and faces were paired in a contradictory manner (immature voices with mature faces vs. mature voices with immature faces). Results revealed that vocal cues were more effective than facial cues in conveying young children's attributes to adults and that women were more efficient (i.e., faster) than men in responding to children's cues. These results confirm and extend previous evidence on the relevance of children's vocal cues to signaling important information about children's attributes and needs during their first 6 years of life.

The Curious Case of Impersonators and Singers: Telling Voices Apart and Telling Voices Together under Naturally Challenging Listening Conditions

Vocal identity processing depends on the ability to tell apart two instances of different speakers whilst also being able to tell together two instances of the same speaker. Whilst previous research has examined these voice processing capabilities under relatively common listening conditions, it has not yet tested the limits of these capabilities. Here, two studies are presented that employ challenging listening tasks to determine just how good we are at these voice processing tasks. In Experiment 1, 54 university students were asked to distinguish between very similar sounding, yet different speakers (celebrity targets and their impersonators). Participants completed a 'Same/Different' task and a 'Which is the Celebrity?' task to pairs of speakers, and a 'Real or Not?' task to individual speakers. In Experiment 2, a separate group of 40 university students was asked to pair very different sounding instances of the same speakers (speaking and singing). Participants were presented with an array of voice clips and completed a 'Pairs Task' as a variant of the more traditional voice sorting task. The results of Experiment 1 suggested that significantly more mistakes were made when distinguishing celebrity targets from their impersonators than when distinguishing the same targets from control voices. Nevertheless, listeners were significantly better than chance in all three tasks despite the challenge. Similarly, the results of Experiment 2 suggested that it was significantly more difficult to pair singing and speaking clips than to pair two speaking clips, particularly when the speakers were unfamiliar. Again, however, the performance was significantly above zero, and was again better than chance in a cautious comparison. Taken together, the results suggest that vocal identity processing is a highly adaptable task, assisted by familiarity with the speaker. However, the fact that performance remained above chance in all tasks suggests that we had not reached the limit of our listeners' capability, despite the considerable listening challenges introduced. We conclude that voice processing is far better than previous research might have presumed.

Familiar and unfamiliar speaker recognition assessment and system emulation for cochlear implant users

In the area of speech processing, human speaker identification under naturalistic environments is a challenging task, especially for hearing-impaired individuals with cochlear implants (CIs) or hearing aids (HAs). Motivated by the fact that electrodograms reflect direct CI stimulation of input audio, this study proposes a speaker identification (ID) investigation using two-dimensional electrodograms constructed from the responses of a CI auditory system to emulate CI speaker ID capabilities. Features are extracted from electrodograms through an identity vector (i-vector) framework to train and generate identity models for each speaker using a Gaussian mixture model-universal background model followed by probabilistic linear discriminant analysis. To validate the proposed system, perceptual speaker ID for 20 normal hearing (NH) and seven CI listeners was evaluated with a total of 41 different speakers and compared with the scores from the proposed system. A one-way analysis of variance showed that the proposed system can reliably predict the speaker ID capability of CI (F[1,10] = 0.18, p = 0.68) and NH (F[1,20] = 0, p = 0.98) listeners in naturalistic environments. The impact of speaker familiarity is also addressed, and the results show a reduced performance for speaker recognition by CI subjects using their CI processor, highlighting limitations of current speech processing strategies used in CIs/HAs.

Electrocorticography reveals the dynamics of famous voice responses in human fusiform gyrus

Voice and face processing occur through convergent neural systems that facilitate speaker recognition. Neuroimaging studies suggest that familiar voice processing engages early visual cortex, including the bilateral fusiform gyrus (FG) on the basal temporal lobe. However, what role the FG plays in voice processing and whether it is driven by bottom-up or top-down mechanisms is unresolved. In this study we directly examined neural responses to famous voices and faces in human FG with direct cortical surface recordings (electrocorticography) in epilepsy surgery patients. We tested the hypothesis that neural populations in human FG respond to famous voices and investigated the temporal properties of voice responses in FG. Recordings were acquired from five adult participants during a person identification task using visual and auditory stimuli from famous speakers (U.S. Presidents Barack Obama, George W. Bush, and Bill Clinton). Patients were presented with images of presidents or clips of their voices and asked to identify the portrait/speaker. Our results demonstrate that a subset of face-responsive sites in and near FG also exhibit voice responses that are both lower in magnitude and delayed (300-600 ms) compared with visual responses. The dynamics of voice processing revealed by direct cortical recordings suggests a top-down feedback-mediated response to famous voices in FG that may facilitate speaker identification.NEW & NOTEWORTHY Interactions between auditory and visual cortices play an important role in person identification, but the dynamics of these interactions remain poorly understood. We performed direct brain recordings of fusiform face cortex in human epilepsy patients performing a famous voice naming task, revealing the dynamics of famous voice processing in human fusiform face cortex. The findings support a model of top-down interactions from auditory to visual cortex to facilitate famous voice recognition.

Talker and accent familiarity yield advantages for voice identity perception: A voice sorting study

In the current study, we examine and compare the effects of talker and accent familiarity in the context of a voice identity sorting task, using naturally varying voice recording samples from the TV show Derry Girls. Voice samples were thus all spoken with a regional accent of UK/Irish English (from [London]derry). We tested four listener groups: Listeners were either familiar or unfamiliar with the TV show (and therefore the talker identities) and were either highly familiar or relatively less familiar with Northern Irish accents. Both talker and accent familiarity significantly improved accuracy of voice identity sorting performance. However, the talker familiarity benefits were overall larger, and more consistent. We discuss the results in light of a possible hierarchy of familiarity effects and argue that our findings may provide additional evidence for interactions of speech and identity processing pathways in voice identity perception. We also identify some key limitations in the current work and provide suggestions for future studies to address these.

How do we describe other people from voices and faces?

When seeing someone's face or hearing their voice, perceivers routinely infer information about a person's age, sex and social traits. While many experiments have explored how individual person characteristics are perceived in isolation, less is known about which person characteristics are described spontaneously from voices and faces and how descriptions may differ across modalities. In Experiment 1, participants provided free descriptions for voices and faces. These free descriptions followed similar patterns for voices and faces - and for individual identities: Participants spontaneously referred to a wide range of descriptors. Psychological descriptors, such as character traits, were used most frequently; physical characteristics, such as age and sex, were notable as they were mentioned earlier than other types of descriptors. After finding primarily similarities between modalities when analysing person descriptions across identities, Experiment 2 asked whether free descriptions encode how individual identities differ. For this purpose, the measures derived from the free descriptions were linked to voice/face discrimination judgements that are known to describe differences in perceptual properties between identity pairs. Significant relationships emerged within and across modalities, showing that free descriptions indeed encode differences between identities - information that is shared with discrimination judgements. This suggests that the two tasks tap into similar, high-level person representations. These findings show that free description data can offer valuable insights into person perception and underline that person perception is a multivariate process during which perceivers rapidly and spontaneously infer many different person characteristics to form a holistic impression of a person.

Evaluating earwitness identification procedures: adapting pre-parade instructions and parade procedure

Voice identification parades can be unreliable, as earwitness responses are error-prone. In this paper we tested performance across serial and sequential procedures, and varied pre-parade instructions, with the aim of reducing errors. The participants heard a target voice and later attempted to identify it from a parade. In Experiment 1 they were either warned that the target may or may not be present (standard warning) or encouraged to consider responding "not present" because of the associated risk of a wrongful conviction (strong warning). Strong warnings prompted a conservative criterion shift, with participants less likely to make a positive identification regardless of whether the target was present. In contrast to previous findings, we found no statistically reliable difference in accuracy between serial and sequential parades. Experiment 2 ruled out a potential confound in Experiment 1. Taken together, our results suggest that adapting pre-parade instructions provides a simple way of reducing the risk of false identifications.

Brain mechanism of unfamiliar and familiar voice processing: an activation likelihood estimation meta-analysis

Interpersonal communication through vocal information is very important for human society. During verbal interactions, our vocal cord vibrations convey important information regarding voice identity, which allows us to decide how to respond to speakers (e.g., neither greeting a stranger too warmly or speaking too coldly to a friend). Numerous neural studies have shown that identifying familiar and unfamiliar voices may rely on different neural bases. However, the mechanism underlying voice identification of individuals of varying familiarity has not been determined due to vague definitions, confusion of terms, and differences in task design. To address this issue, the present study first categorized three kinds of voice identity processing (perception, recognition and identification) from speakers with different degrees of familiarity. We defined voice identity perception as passively listening to a voice or determining if the voice was human, voice identity recognition as determining if the sound heard was acoustically familiar, and voice identity identification as ascertaining whether a voice is associated with a name or face. Of these, voice identity perception involves processing unfamiliar voices, and voice identity recognition and identification involves processing familiar voices. According to these three definitions, we performed activation likelihood estimation (ALE) on 32 studies and revealed different brain mechanisms underlying processing of unfamiliar and familiar voice identities. The results were as follows: (1) familiar voice recognition/identification was supported by a network involving most regions in the temporal lobe, some regions in the frontal lobe, subcortical structures and regions around the marginal lobes; (2) the bilateral superior temporal gyrus was recruited for voice identity perception of an unfamiliar voice; (3) voice identity recognition/identification of familiar voices was more likely to activate the right frontal lobe than voice identity perception of unfamiliar voices, while voice identity perception of an unfamiliar voice was more likely to activate the bilateral temporal lobe and left frontal lobe; and (4) the bilateral superior temporal gyrus served as a shared neural basis of unfamiliar voice identity perception and familiar voice identity recognition/identification. In general, the results of the current study address gaps in the literature, provide clear definitions of concepts, and indicate brain mechanisms for subsequent investigations.

Auditory perceptual learning in autistic adults

The automatic retuning of phoneme categories to better adapt to the speech of a novel talker has been extensively documented across various (neurotypical) populations, including both adults and children. However, no studies have examined auditory perceptual learning effects in populations atypical in perceptual, social, and language processing for communication, such as populations with autism. Employing a classic lexically-guided perceptual learning paradigm, the present study investigated perceptual learning effects in Australian English autistic and non-autistic adults. The findings revealed that automatic attunement to existing phoneme categories was not activated in the autistic group in the same manner as for non-autistic control subjects. Specifically, autistic adults were able to both successfully discern lexical items and to categorize speech sounds; however, they did not show effects of perceptual retuning to talkers. These findings may have implications for the application of current sensory theories (e.g., Bayesian decision theory) to speech and language processing by autistic individuals. LAY SUMMARY: Lexically guided perceptual learning assists in the disambiguation of speech from a novel talker. The present study established that while Australian English autistic adult listeners were able to successfully discern lexical items and categorize speech sounds in their native language, perceptual flexibility in updating speaker-specific phonemic knowledge when exposed to a novel talker was not available. Implications for speech and language processing by autistic individuals as well as current sensory theories are discussed.

Gender and Context-Specific Effects of Vocal Dominance and Trustworthiness on Leadership Decisions

Objective

The evolutionary-contingency hypothesis, which suggests that preferences for leaders are context-dependent, has found relatively consistent support from research investigating leadership decisions based on facial pictures. Here, we test whether these results transfer to leadership decisions based on voice recordings. We examined how dominance and trustworthiness perceptions relate to leadership decisions in wartime and peacetime contexts and whether effects differ by a speaker’s gender. Further, we investigate two cues that might be related to leadership decisions, as well as dominance and trustworthiness perceptions: voice pitch and strength of regional accent.

Methods

We conducted a preregistered online study with 125 raters and recordings of 120 speakers (61 men, 59 women) from different parts in Germany. Raters were randomly distributed into four rating conditions: dominance, trustworthiness, hypothetical vote (wartime) and hypothetical vote (peacetime).

Results

We find that dominant speakers were more likely to be voted for in a wartime context while trustworthy speakers were more likely to be voted for in a peacetime context. Voice pitch functions as a main cue for dominance perceptions, while strength of regional accent functions as a main cue for trustworthiness perceptions.

Conclusions

This study adds to a stream of research that suggests that (a) people’s voices contain important information based on which we form social impressions and (b) we prefer different types of leaders across different contexts. Future research should disentangle effects of gender bias in leadership decisions and investigate underlying mechanisms that influence how people’s voices contribute to achieving social status.

Vocal Characteristics Influence Women's Perceptions of Infidelity and Relationship Investment in China

Vocal characteristics are important cues to form social impressions. Previous studies indicated that men with masculine voices are perceived as engaging in higher rates of infidelity and being less committed to their relationship. In the current study, we examined how women in China perceive information regarding infidelity and relationship investment conveyed by the voices (voice pitch and vocal tract length) of males, and whether different vocal characteristics play a similar role in driving these impressions. In addition, we examined whether these perceptions are consistent in Chinese and English language contexts. The results indicated that women perceived men with more masculine voices (lower voice pitch and longer vocal tract length) as showing a lower likelihood of infidelity and higher relationship investment; further, women who preferred more masculine voices in long-term relationships, but not in short-term relationships, were more likely to perceive men with masculine voices as less likely to engage in infidelity and more likely to invest in their relationship. Moreover, the participants formed very similar impressions irrespective of whether the voices spoke native (Chinese) or foreign (English) languages. These results provide new evidence for the role of the voice in women's choices in selecting long-term partners.

Rapid pre-attentive processing of a famous speaker: Electrophysiological effects of Angela Merkel's voice

The recognition of human speakers by their voices is a remarkable cognitive ability. Previous research has established a voice area in the right temporal cortex involved in the integration of speaker-specific acoustic features. This integration appears to occur rapidly, especially in case of familiar voices. However, the exact time course of this process is less well understood. To this end, we here investigated the automatic change detection response of the human brain while listening to the famous voice of German chancellor Angela Merkel, embedded in the context of acoustically matched voices. A classic passive oddball paradigm contrasted short word stimuli uttered by Merkel with word stimuli uttered by two unfamiliar female speakers. Electrophysiological voice processing indices from 21 participants were quantified as mismatch negativities (MMNs) and P3a differences. Cortical sources were approximated by variable resolution electromagnetic tomography. The results showed amplitude and latency effects for both MMN and P3a: The famous (familiar) voice elicited a smaller but earlier MMN than the unfamiliar voices. The P3a, by contrast, was both larger and later for the familiar than for the unfamiliar voices. Familiar-voice MMNs originated from right-hemispheric regions in temporal cortex, overlapping with the temporal voice area, while unfamiliar-voice MMNs stemmed from left superior temporal gyrus. These results suggest that the processing of a very famous voice relies on pre-attentive right temporal processing within the first 150 ms of the acoustic signal. The findings further our understanding of the neural dynamics underlying familiar voice processing.

Contributions of fundamental frequency and timbre to vocal emotion perception and their electrophysiological correlates

Our ability to infer a speaker's emotional state depends on the processing of acoustic parameters such as fundamental frequency (F0) and timbre. Yet, how these parameters are processed and integrated to inform emotion perception remains largely unknown. Here we pursued this issue using a novel parameter-specific voice morphing technique to create stimuli with emotion modulations in only F0 or only timbre. We used these stimuli together with fully modulated vocal stimuli in an event-related potential (ERP) study in which participants listened to and identified stimulus emotion. ERPs (P200 and N400) and behavioral data converged in showing that both F0 and timbre support emotion processing but do so differently for different emotions: Whereas F0 was most relevant for responses to happy, fearful and sad voices, timbre was most relevant for responses to voices expressing pleasure. Together, these findings offer original insights into the relative significance of different acoustic parameters for early neuronal representations of speaker emotion and show that such representations are predictive of subsequent evaluative judgments.

Voices as Cues to Children's Needs for Caregiving

The aim of this study was to explore the role of voices as cues to adults of children’s needs for potential caregiving during early childhood. To this purpose, 74 college students listened to pairs of 5-year-old versus 10-year-old children verbalizing neutral-content sentences and indicated which voice was better associated with each of 14 traits, potentially meaningful in interactions between young children and adults. Results indicated that children with immature voices were perceived more positively and as being more helpless than children with mature voices. Children’s voices, regardless of the content of speech, seem to be a powerful source of information about children’s need for caregiving for parents and others during the first six years of life.

Intra-individual Reliability of Voice- and Music-elicited Responses and their Modulation by Expertise

A growing number of functional neuroimaging studies have identified regions within the temporal lobe, particularly along the planum polare and planum temporale, that respond more strongly to music than other types of acoustic stimuli, including voice. This "music preferred" regions have been reported using a variety of stimulus sets, paradigms and analysis approaches and their consistency across studies confirmed through meta-analyses. However, the critical question of intra-subject reliability of these responses has received less attention. Here, we directly assessed this important issue by contrasting brain responses to musical vs. vocal stimuli in the same subjects across three consecutive fMRI runs, using different types of stimuli. Moreover, we investigated whether these music- and voice-preferred responses were reliably modulated by expertise. Results demonstrated that music-preferred activity previously reported in temporal regions, and its modulation by expertise, exhibits a high intra-subject reliability. However, we also found that activity in some extra-temporal regions, such as the precentral and middle frontal gyri, did depend on the particular stimuli employed, which may explain why these are less consistently reported in the literature. Taken together, our findings confirm and extend the notion that specific regions in the brain consistently respond more strongly to certain socially-relevant stimulus categories, such as faces, voices and music, but that some of these responses appear to depend, at least to some extent, on the specific features of the paradigm employed.

Auditory Perception of Roughness and Breathiness by Dysphonic Women

OBJECTIVE

To investigate the auditory perception of roughness and breathiness by dysphonic women.

METHODS

Twenty-two dysphonic native Brazilian Portuguese women participated in this research. All participants underwent audiological evaluation and laryngeal examination to confirm the diagnosis. During the tests, they recorded the sustained vowel /Ɛ/. A speech-language pathologist performed the auditory-perceptual judgment of voice quality for these vocal samples, categorizing the general degree of vocal deviation (mild, moderate, and severe degree) and the predominant type of deviation (roughness or breathiness). Thirty-two (32) stimuli were selected from a voice database, including twenty-four (24) dysphonic voice samples and eight (8) voice samples from vocally healthy women. The authors conducted five perception experiments, being three categorization tasks (normal vs. deviated, breathy vs. nonbreathy, rough vs. nonrough) and two tasks for discriminating the degree of deviation (roughness degree and breathiness degree).

RESULTS

The experiments showed a difference between the answers for presence/absence of deviation, presence/absence of breathiness, and presence/absence of roughness in the stimuli, and a difference in the proportion of similar answers of dysphonic women (P < 0.001) regarding the identification of the deviation. Participants classified a large part of the deviated (57.9%), breathy (63.13%), and rough (65.31%) voices as normal. The degree of vocal deviation (P = 0.008) and the degree of roughness in the stimuli correlated positively with the proportion of similar answers of the participants. As for the discrimination of breathiness degrees, less deviated (normal and mild) voices were less discriminated, and more deviated (moderate and severe) voices were better discriminated. Regarding the discrimination of roughness degrees, only the voices with severe deviations showed good discrimination.

CONCLUSION

Dysphonic women had a high rate of not similar answers in the identification of normal and deviated voices. They identified more than half of the deviated voices as normal. Samples with more severe deviations were proportionally more identified as deviated by the participants. The greater the vocal deviation of the participants' voices, the smallest the number of similar answers. Participants had a high rate of not similar answers in the identification of normal and breathy voices. Dysphonic women show less ability to perceive mildly and moderately breathy voices in the breathy category. Participants had a high rate of similar answers in the identification of normal and rough voices. Dysphonic women show less ability to perceive mildly and moderately breathy voices in the breathy category. Participants show less ability to perceive only mildly roughness voices with similar responses. Dysphonic women could discriminate between voices with adjacent degrees of roughness but had a low percentage of similar answers for discrimination between voices with adjacent degrees of breathiness.

Vocal emotion adaptation aftereffects within and across speaker genders: Roles of timbre and fundamental frequency

While the human perceptual system constantly adapts to the environment, some of the underlying mechanisms are still poorly understood. For instance, although previous research demonstrated perceptual aftereffects in emotional voice adaptation, the contribution of different vocal cues to these effects is unclear. In two experiments, we used parameter-specific morphing of adaptor voices to investigate the relative roles of fundamental frequency (F0) and timbre in vocal emotion adaptation, using angry and fearful utterances. Participants adapted to voices containing emotion-specific information in either F0 or timbre, with all other parameters kept constant at an intermediate 50% morph level. Full emotional voices and ambiguous voices were used as reference conditions. All adaptor stimuli were either of the same (Experiment 1) or opposite speaker gender (Experiment 2) of subsequently presented target voices. In Experiment 1, we found consistent aftereffects in all adaptation conditions. Crucially, aftereffects following timbre adaptation were much larger than following F0 adaptation and were only marginally smaller than those following full adaptation. In Experiment 2, adaptation aftereffects appeared massively and proportionally reduced, with differences between morph types being no longer significant. These results suggest that timbre plays a larger role than F0 in vocal emotion adaptation, and that vocal emotion adaptation is compromised by eliminating gender-correspondence between adaptor and target stimuli. Our findings also add to mounting evidence suggesting a major role of timbre in auditory adaptation.

Advancing the RDoC initiative through the assessment of caregiver social processes

The relationships infants and young children have with their caregivers are fundamental to their survival and well-being. Theorists and researchers across disciplines have attempted to describe and assess the variation in these relationships, leading to a general acceptance that caregiving is critical to understanding child functioning, including developmental psychopathology. At the same time, we lack consensus on how to assess these fundamental relationships. In the present paper, we first review research documenting the importance of the caregiver-child relationship in understanding environmental risk for psychopathology. Second, we propose that the National Institute of Mental Health's Research Domain Criteria (RDoC) initiative provides a useful framework for extending the study of children's risk for psychopathology by assessing their caregivers' social processes. Third, we describe the units of analysis for caregiver social processes, documenting how the specific subconstructs in the domain of social processes are relevant to the goal of enhancing knowledge of developmental psychopathology. Lastly, we highlight how past research can inform new directions in the study of caregiving and the parent-child relationship through this innovative extension of the RDoC initiative.

Audiovisual identity perception from naturally-varying stimuli is driven by visual information

Identity perception often takes place in multimodal settings, where perceivers have access to both visual (face) and auditory (voice) information. Despite this, identity perception is usually studied in unimodal contexts, where face and voice identity perception are modelled independently from one another. In this study, we asked whether and how much auditory and visual information contribute to audiovisual identity perception from naturally-varying stimuli. In a between-subjects design, participants completed an identity sorting task with either dynamic video-only, audio-only or dynamic audiovisual stimuli. In this task, participants were asked to sort multiple, naturally-varying stimuli from three different people by perceived identity. We found that identity perception was more accurate for video-only and audiovisual stimuli compared with audio-only stimuli. Interestingly, there was no difference in accuracy between video-only and audiovisual stimuli. Auditory information nonetheless played a role alongside visual information as audiovisual identity judgements per stimulus could be predicted from both auditory and visual identity judgements, respectively. While the relationship was stronger for visual information and audiovisual information, auditory information still uniquely explained a significant portion of the variance in audiovisual identity judgements. Our findings thus align with previous theoretical and empirical work that proposes that, compared with faces, voices are an important but relatively less salient and a weaker cue to identity perception. We expand on this work to show that, at least in the context of this study, having access to voices in addition to faces does not result in better identity perception accuracy.

Perceptual narrowing in face- and speech-perception domains in infancy: A longitudinal approach

During the first year of life, infants undergo a process known as perceptual narrowing, which reduces their sensitivity to classes of stimuli which the infants do not encounter in their environment. It has been proposed that perceptual narrowing for faces and speech may be driven by shared domain-general processes. To investigate this theory, our study longitudinally tested 50 German Caucasian infants with respect to these domains first at 6 months of age followed by a second testing at 9 months of age. We used an infant-controlled habituation-dishabituation paradigm to test the infants' ability to discriminate among other-race Asian faces and non-native Cantonese speech tones, as well as same-race Caucasian faces as a control. We found that while at 6 months of age infants could discriminate among all stimuli, by 9 months of age they could no longer discriminate among other-race faces or non-native tones. However, infants could discriminate among same-race stimuli both at 6 and at 9 months of age. These results demonstrate that the same infants undergo perceptual narrowing for both other-race faces and non-native speech tones between the ages of 6 and 9 months. This parallel development of perceptual narrowing occurring in both the face and speech perception modalities over the same period of time lends support to the domain-general theory of perceptual narrowing in face and speech perception.

People with larger social networks show poorer voice recognition

The way we process language is influenced by our experience. We are more likely to attend to features that proved to be useful in the past. Importantly, the size of individuals’ social network can influence their experience, and consequently, how they process language. In the case of voice recognition, having a larger social network might provide more variable input and thus enhance the ability to recognise new voices. On the other hand, learning to recognise voices is more demanding and less beneficial for people with a larger social network as they have more speakers to learn yet spend less time with each. This paper tests whether social network size influences voice recognition, and if so, in which direction. Native Dutch speakers listed their social network and performed a voice recognition task. Results showed that people with larger social networks were poorer at learning to recognise voices. Experiment 2 replicated the results with a British sample and English stimuli. Experiment 3 showed that the effect does not generalise to voice recognition in an unfamiliar language suggesting that social network size influences attention to the linguistic rather than non-linguistic markers that differentiate speakers. The studies thus show that our social network size influences our inclination to learn speaker-specific patterns in our environment, and consequently, the development of skills that rely on such learned patterns, such as voice recognition.

Fast Periodic Auditory Stimulation Reveals a Robust Categorical Response to Voices in the Human Brain

Voices are arguably among the most relevant sounds in humans' everyday life, and several studies have suggested the existence of voice-selective regions in the human brain. Despite two decades of research, defining the human brain regions supporting voice recognition remains challenging. Moreover, whether neural selectivity to voices is merely driven by acoustic properties specific to human voices (e.g., spectrogram, harmonicity), or whether it also reflects a higher-level categorization response is still under debate. Here, we objectively measured rapid automatic categorization responses to human voices with fast periodic auditory stimulation (FPAS) combined with electroencephalography (EEG). Participants were tested with stimulation sequences containing heterogeneous non-vocal sounds from different categories presented at 4 Hz (i.e., four stimuli/s), with vocal sounds appearing every three stimuli (1.333 Hz). A few minutes of stimulation are sufficient to elicit robust 1.333 Hz voice-selective focal brain responses over superior temporal regions of individual participants. This response is virtually absent for sequences using frequency-scrambled sounds, but is clearly observed when voices are presented among sounds from musical instruments matched for pitch and harmonicity-to-noise ratio (HNR). Overall, our FPAS paradigm demonstrates that the human brain seamlessly categorizes human voices when compared with other sounds including musical instruments' sounds matched for low level acoustic features and that voice-selective responses are at least partially independent from low-level acoustic features, making it a powerful and versatile tool to understand human auditory categorization in general.

When two is worse than one: The deleterious impact of multisensory stimulation on response inhibition

Multisensory facilitation is known to improve the perceptual performances and reaction times of participants in a wide range of tasks, from detection and discrimination to memorization. We asked whether a multimodal signal can similarly improve action inhibition using the stop-signal paradigm. Indeed, consistent with a crossmodal redundant signal effect that relies on multisensory neuronal integration, the threshold for initiating behavioral responses is known for being reached faster with multisensory stimuli. To evaluate whether this phenomenon also occurs for inhibition, we compared stop signals in unimodal (human faces or voices) versus audiovisual modalities in natural or degraded conditions. In contrast to the expected multisensory facilitation, we observed poorer inhibition efficiency in the audiovisual modality compared with the visual and auditory modalities. This result was corroborated by both response probabilities and stop-signal reaction times. The visual modality (faces) was the most effective. This is the first demonstration of an audiovisual impairment in the domain of perception and action. It suggests that when individuals are engaged in a high-level decisional conflict, bimodal stimulation is not processed as a simple multisensory object improving the performance but is perceived as concurrent visual and auditory information. This absence of unity increases task demand and thus impairs the ability to revise the response.

The impact of living with transfeminine vocal gender dysphoria: Health utility outcomes assessment

Background: The voice signals a tremendous amount of gender cues. Transfeminine individuals report debilitating quality-of-life deficits as a result of their vocal gender dysphoria.Aims: We aimed to quantify the potential impact of this dysphoria experienced with quality-adjusted life years (QALYs), as well as associated treatments, through validated health utility measures. Methods: Peri-operative phonometric audio recordings of a consented transfeminine patient volunteer with a history of vocal gender dysphoria aided in the description of two transfeminine health states, pre- and post-vocal feminization gender dysphoria; monocular and binocular blindness were health state controls. Survey responses from general population adults rate these four health states via visual analogue scale (VAS), standard gamble (SG), and time tradeoff (TTO). Results: Survey respondents totaled 206 with a mean age of 35.8 years. Through VAS measures, these general adult respondents on average perceived a year of life with transfeminine vocal gender dysphoria as approximately three-quarters of a life-year of perfect health. Respondents also on average would have risked a 15%-20% chance of death on SG analysis and would have sacrificed 10 years of their remaining life on TTO measures to cure the condition. The QALY scores for the post-gender affirming treatments for vocal gender dysphoria (+0.09 VAS, p < 0.01) were significantly higher compared to the pretreatment state. There were no differences in the severity of these QALY scores by survey respondent's political affiliation or gender identity. Conclusions: To our knowledge, this study is the first to quantify how the general population perceives the health burden of vocal gender dysphoria experienced by transfeminine patients. Feminization treatments including voice therapy with feminization laryngoplasty appear to significantly increase health utility scores.

Speech-evoked brain activity is more robust to competing speech when it is spoken by someone familiar

When speech is masked by competing sound, people are better at understanding what is said if the talker is familiar compared to unfamiliar. The benefit is robust, but how does processing of familiar voices facilitate intelligibility? We combined high-resolution fMRI with representational similarity analysis to quantify the difference in distributed activity between clear and masked speech. We demonstrate that brain representations of spoken sentences are less affected by a competing sentence when they are spoken by a friend or partner than by someone unfamiliar-effectively, showing a cortical signal-to-noise ratio (SNR) enhancement for familiar voices. This effect correlated with the familiar-voice intelligibility benefit. We functionally parcellated auditory cortex, and found that the most prominent familiar-voice advantage was manifest along the posterior superior and middle temporal gyri. Overall, our results demonstrate that experience-driven improvements in intelligibility are associated with enhanced multivariate pattern activity in posterior temporal cortex.

Female Voice-Related Sexual Attractiveness to Males: Does it Vary With Different Degrees of Conception Likelihood?

Previous investigations have found that female voice-related attractiveness to males increases when both conception likelihood (CL) and voice fundamental frequency (f_o) are elevated. To test this hypothesis, we conducted a perceptual experiment where 78 heterosexual males rated sexual attractiveness of 9 female voice samples, recorded at menstrual, follicular and luteal phases of the menstrual cycle under two double-blinded randomly allocated conditions: a natural menstrual cycle (placebo condition) and when using an oral contraceptive pill (OCP condition). The voice samples yielded a total of 54 stimuli that were visually sorted and rated using Visor software. Concentrations of estrogens, progesterone and testosterone were analyzed, and measurements of speaking fundamental frequency (sf_o) and its standard deviation (sf_oSD), f_o derivative (df_o) and f_o slope were made. A multilevel ordinal logistic regression model nested in listeners and in females, and adjusted by phase and condition, was carried out to assess the association between ratings and: (1) phases and conditions; (2) sex steroid hormonal concentrations; and (3) voice parameters. A high probability of obtaining high ratings of voice sexual attractiveness was found for: (1) menstrual phase of placebo use and follicular phase of OCP use; (2) for low estradiol to progesterone ratio and testosterone concentrations; and (3) for low df_o. The latter showed a moderate statistical association with ratings of high attractiveness, as compared with the small association found for the remaining variables. It seems that the voice is a weak cue for female CL. Female sexual attraction to males may be a consequence of what females do in order to regulate their extended sexuality across the menstrual cycle rather than of estrus cues, the use of paralinguistic speech patterns being an example.

Comparing unfamiliar voice and face identity perception using identity sorting tasks

Identity sorting tasks, in which participants sort multiple naturally varying stimuli of usually two identities into perceived identities, have recently gained popularity in voice and face processing research. In both modalities, participants who are unfamiliar with the identities tend to perceive multiple stimuli of the same identity as different people and thus fail to "tell people together." These similarities across modalities suggest that modality-general mechanisms may underpin sorting behaviour. In this study, participants completed a voice sorting and a face sorting task. Taking an individual differences approach, we asked whether participants' performance on voice and face sorting of unfamiliar identities is correlated. Participants additionally completed a voice discrimination (Bangor Voice Matching Test) and a face discrimination task (Glasgow Face Matching Test). Using these tasks, we tested whether performance on sorting related to explicit identity discrimination. Performance on voice sorting and face sorting tasks was correlated, suggesting that common modality-general processes underpin these tasks. However, no significant correlations were found between sorting and discrimination performance, with the exception of significant relationships for performance on "same identity" trials with "telling people together" for voices and faces. Overall, any reported relationships were however relatively weak, suggesting the presence of additional modality-specific and task-specific processes.

Semantics-weighted lexical surprisal modeling of naturalistic functional MRI time-series during spoken narrative listening

Probabilistic language models are increasingly used to provide neural representations of linguistic features under naturalistic settings. Word surprisal models can be applied to continuous fMRI recordings during task-free listening of narratives, to detect regions linked to language prediction and comprehension. Here, to this purpose, a novel semantics-weighted lexical surprisal is applied to naturalistic fMRI data. FMRI was performed at 3 Tesla in 31 subjects during task-free listening to a 12-minute audiobook played in both original and word-reversed (control) version. Lexical-only and semantics-weighted lexical surprisal models were estimated for the original and control word series. The two series were alternatively chosen to build the predictor of interest in the first-level general linear model and were compared in the second-level (group) analysis. The addition of the surprisal predictor to the stimulus-related predictors significantly improved the fitting of the neural signal. In average, the semantics-weighted model yielded lower surprisal values and, in some areas, better fitting of the fMRI data compared to the lexical-only model. The two models produced both overlapping and distinct activations: while lexical-only surprisal activated secondary auditory areas in the superior temporal gyri and the cerebellum, semantics-weighted surprisal additionally activated the left inferior frontal gyrus. These results confirm the usefulness of surprisal models in the naturalistic fMRI analysis of linguistic processes and suggest that the use of semantic information may increase the sensitivity of a probabilistic language model in higher-order language-related areas, with possible implications for future naturalistic fMRI studies of language under normal and (clinically or pharmacologically) modified conditions.

Intrinsic functional architecture of the human speech processing network

Speech engages distributed temporo-fronto-parietal brain regions, however a comprehensive understanding of its intrinsic functional network architecture is lacking. Here we investigate the human speech processing network using the largest sample to date, high temporal resolution resting-state fMRI data, network stability analysis, and theoretically informed models. Network consensus analysis revealed three stable functional modules encompassing: (1) superior temporal plane (STP) and Area Spt, (2) superior temporal sulcus (STS) + ventral frontoparietal cortex, and (3) dorsal frontoparietal cortex. The STS + ventral frontoparietal cortex module showed the highest participation coefficient, and a hub-like organization linking STP with frontoparietal cortical nodes. Node-wise analysis revealed key connectivity features underlying this modular architecture, including a leftward asymmetric connectivity profile, and differential connectivity of STS and STP, with frontoparietal cortex. Our findings, replicated across cohorts, reveal a tripartite functional network architecture supporting speech processing and provide a novel template for future studies.

Look up to the body: An eye-tracking investigation of 7-months-old infants' visual exploration of emotional body expressions

The human body is an important source of information to infer a person's emotional state. Research with adult observers indicate that the posture of the torso, arms and hands provide important perceptual cues for recognising anger, fear and happy expressions. Much less is known about whether infants process body regions differently for different body expressions. To address this issue, we used eye tracking to investigate whether infants' visual exploration patterns differed when viewing body expressions. Forty-eight 7-months-old infants were randomly presented with static images of adult female bodies expressing anger, fear and happiness, as well as an emotionally-neutral posture. Facial cues to emotional state were removed by masking the faces. We measured the proportion of looking time, proportion and number of fixations, and duration of fixations on the head, upper body and lower body regions for the different expressions. We showed that infants explored the upper body more than the lower body. Importantly, infants at this age fixated differently on different body regions depending on the expression of the body posture. In particular, infants spent a larger proportion of their looking times and had longer fixation durations on the upper body for fear relative to the other expressions. These results extend and replicate the information about infant processing of emotional expressions displayed by human bodies, and they support the hypothesis that infants' visual exploration of human bodies is driven by the upper body.

Unconscious and Distinctive Control of Vocal Pitch and Timbre During Altered Auditory Feedback

Vocal control plays a critical role in smooth social communication. Speakers constantly monitor auditory feedback (AF) and make adjustments when their voices deviate from their intentions. Previous studies have shown that when certain acoustic features of the AF are artificially altered, speakers compensate for this alteration in the opposite direction. However, little is known about how the vocal control system implements compensations for alterations of different acoustic features, and associates them with subjective consciousness. The present study investigated whether compensations for the fundamental frequency (F0), which corresponds to perceived pitch, and formants, which contribute to perceived timbre, can be performed unconsciously and independently. Forty native Japanese speakers received two types of altered AF during vowel production that involved shifts of either only the formant frequencies (formant modification; Fm) or both the pitch and formant frequencies (pitch + formant modification; PFm). For each type, three levels of shift (slight, medium, and severe) in both directions (increase or decrease) were used. After the experiment, participants were tested for whether they had perceived a change in the F0 and/or formants. The results showed that (i) only formants were compensated for in the Fm condition, while both the F0 and formants were compensated for in the PFm condition; (ii) the F0 compensation exhibited greater precision than the formant compensation in PFm; and (iii) compensation occurred even when participants misperceived or could not explicitly perceive the alteration in AF. These findings indicate that non-experts can compensate for both formant and F0 modifications in the AF during vocal production, even when the modifications are not explicitly or correctly perceived, which provides further evidence for a dissociation between conscious perception and action in vocal control. We propose that such unconscious control of voice production may enhance rapid adaptation to changing speech environments and facilitate mutual communication.