Vocal fundamental and formant frequencies affect perceptions of speaker cooperativeness

Voice-based judgments of sex, height, weight, attractiveness, health, and psychological traits based on free speech versus scripted speech

How do we perceive others based on their voices? This question has attracted research and media attention for decades, producing hundreds of studies showing that the voice is socially and biologically relevant, but these studies vary in methodology and ecological validity. Here we test whether vocalizers producing read versus free speech are judged similarly by listeners on ten biological and/or psychosocial traits. In perception experiments using speech from 208 men and women and ratings from 4,088 listeners, we show that listeners' assessments of vocalizer sex and age are highly accurate, regardless of speech type. Assessments of body size, femininity-masculinity and women's health also did not differ between free and read speech. In contrast, read speech elicited higher ratings of attractiveness, dominance and trustworthiness in both sexes and of health in males compared to free speech. Importantly, these differences were small, and we additionally show moderate to strong correlations between ratings of the same vocalizers based on their read and free speech for all ten traits, indicating that voice-based judgments are highly consistent within speakers, whether or not speech is spontaneous. Our results provide evidence that the human voice can communicate various biological and psychosocial traits via both read and free speech, with theoretical and practical implications.

A premium for positive social interest and attractive voices in the acceptability of unfair offers? An ERP study

Although the attractiveness of voices plays an important role in social interactions, it is unclear how voice attractiveness and social interest influence social decision-making. Here, we combined the ultimatum game with recording event-related brain potentials (ERPs) and examined the effect of attractive versus unattractive voices of the proposers, expressing positive versus negative social interest ("I like you" vs. "I don't like you"), on the acceptance of the proposal. Overall, fair offers were accepted at significantly higher rates than unfair offers, and high voice attractiveness increased acceptance rates for all proposals. In ERPs in response to the voices, their attractiveness and expressed social interests yielded early additive effects in the N1 component, followed by interactions in the subsequent P2, P3 and N400 components. More importantly, unfair offers elicited a larger Medial Frontal Negativity (MFN) than fair offers but only when the proposer's voice was unattractive or when the voice carried positive social interest. These results suggest that both voice attractiveness and social interest moderate social decision-making and there is a similar "beauty premium" for voices as for faces.

It's Not Only What You Say, But Also How You Say It: Machine Learning Approach to Estimate Trust from Conversation

OBJECTIVE

The objective of this study was to estimate trust from conversations using both lexical and acoustic data.

BACKGROUND

As NASA moves to long-duration space exploration operations, the increasing need for cooperation between humans and virtual agents requires real-time trust estimation by virtual agents. Measuring trust through conversation is a novel and unintrusive approach.

METHOD

A 2 (reliability) × 2 (cycles) × 3 (events) within-subject study with habitat system maintenance was designed to elicit various levels of trust in a conversational agent. Participants had trust-related conversations with the conversational agent at the end of each decision-making task. To estimate trust, subjective trust ratings were predicted using machine learning models trained on three types of conversational features (i.e., lexical, acoustic, and combined). After training, model explanation was performed using variable importance and partial dependence plots.

RESULTS

Results showed that a random forest algorithm, trained using the combined lexical and acoustic features, predicted trust in the conversational agent most accurately ( R a d j 2 = 0.71 ) . The most important predictors were a combination of lexical and acoustic cues: average sentiment considering valence shifters, the mean of formants, and Mel-frequency cepstral coefficients (MFCC). These conversational features were identified as partial mediators predicting people's trust.

CONCLUSION

Precise trust estimation from conversation requires lexical cues and acoustic cues.

APPLICATION

These results showed the possibility of using conversational data to measure trust, and potentially other dynamic mental states, unobtrusively and dynamically.

Effects of Voice Pitch on Social Perceptions Vary With Relational Mobility and Homicide Rate

Fundamental frequency ( fo) is the most perceptually salient vocal acoustic parameter, yet little is known about how its perceptual influence varies across societies. We examined how fo affects key social perceptions and how socioecological variables modulate these effects in 2,647 adult listeners sampled from 44 locations across 22 nations. Low male fo increased men's perceptions of formidability and prestige, especially in societies with higher homicide rates and greater relational mobility in which male intrasexual competition may be more intense and rapid identification of high-status competitors may be exigent. High female fo increased women's perceptions of flirtatiousness where relational mobility was lower and threats to mating relationships may be greater. These results indicate that the influence of fo on social perceptions depends on socioecological variables, including those related to competition for status and mates.

Speech With Pauses Sounds Deceptive to Listeners With and Without Hearing Impairment

PURPOSE

Communication is as much persuasion as it is the transfer of information. This creates a tension between the interests of the speaker and those of the listener, as dishonest speakers naturally attempt to hide deceptive speech and listeners are faced with the challenge of sorting truths from lies. Listeners with hearing impairment in particular may have differing levels of access to the acoustical cues that give away deceptive speech. A greater tendency toward speech pauses has been hypothesized to result from the cognitive demands of lying convincingly. Higher vocal pitch has also been hypothesized to mark the increased anxiety of a dishonest speaker.

METHOD

Listeners with or without hearing impairments heard short utterances from natural conversations, some of which had been digitally manipulated to contain either increased pausing or raised vocal pitch. Listeners were asked to guess whether each statement was a lie in a two-alternative forced-choice task. Participants were also asked explicitly which cues they believed had influenced their decisions.

RESULTS

Statements were more likely to be perceived as a lie when they contained pauses, but not when vocal pitch was raised. This pattern held regardless of hearing ability. In contrast, both groups of listeners self-reported using vocal pitch cues to identify deceptive statements, though at lower rates than pauses.

CONCLUSIONS

Listeners may have only partial awareness of the cues that influence their impression of dishonesty. Listeners with hearing impairment may place greater weight on acoustical cues according to the differing degrees of access provided by hearing aids.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.24052446.

You are how you speak: The roles of vocal pitch and semantic cues in shaping social perceptions

Social perceptions of speakers are influenced by their voice information, including vocal characteristics and semantic content. Our study investigated how individuals' warmth- and competence-related perceptions of speakers were affected by vocal pitch levels (i.e., high/low) and three kinds of semantic cues (i.e., prosocial, antisocial, and neutral) simultaneously. We have three key findings. First, antisocial cues negatively affected social perceptions, regardless of speakers' gender. However, prosocial cues did not have positive impacts on evaluations of speakers because ratings were similar between prosocial cues and neutral cues. Second, female vocal pitch mattered for warmth-related perceptions but not for competence-related perceptions. The role of semantic cues should be additionally considered when investigating the impact of male vocal pitch on these perceptions. For example, higher-pitched men in prosocial contexts were perceived as warmer, while low-pitched men in antisocial contexts were judged as more competent. Third, the connection between vocal pitch and two kinds of perceptions showed an opposite trend, in which high pitch was related to more warmth but less competence, while the low pitch was associated with less warmth but more competence. These findings extend the understanding of the role of vocal pitch in the formation of stereotypes of strangers in different semantic contexts.

Femme/Butch/Androgyne Identity and Preferences for Femininity Across Face, Voice, and Personality Traits in Chinese Lesbian and Bisexual Women

Studies indicate that individuals preferring masculinity/femininity in one domain also prefer it in other domains. Heterosexual men and women and gay men have reported consistent preferences for masculinity/femininity across the faces and voices of their preferred sex. This study explored the femininity preferences of 417 Chinese lesbian and bisexual women in terms of face, voice pitch, vocal tract length, and personality traits and explored the effect of sexual self-labels (femme, butch, and androgyne) on these preferences. We found that lesbian and bisexual women showed a stronger preference for feminized faces, voice pitch, vocal tract length, and personality traits than masculinized versions, and these preferences were highly consistent across the four domains. Moreover, femininity preference was moderated by sexual self-labels, with butches preferring more feminine voice pitch, vocal tract length, and personality traits than femmes and androgynes. However, no significant difference was found for facial femininity preferences among different sexual self-labels. These findings present evidence of consistent femininity preference across visual, auditory, and personality traits and suggest that, regardless of sexual orientation, multiple cues may be used together when determining the attractiveness of individuals. Furthermore, these results support the hypothesis that the partner preference of lesbian and bisexual women mirrors that of heterosexual men.

Vocal Characteristics Influence Women's Perceptions of Infidelity and Relationship Investment in China

Vocal characteristics are important cues to form social impressions. Previous studies indicated that men with masculine voices are perceived as engaging in higher rates of infidelity and being less committed to their relationship. In the current study, we examined how women in China perceive information regarding infidelity and relationship investment conveyed by the voices (voice pitch and vocal tract length) of males, and whether different vocal characteristics play a similar role in driving these impressions. In addition, we examined whether these perceptions are consistent in Chinese and English language contexts. The results indicated that women perceived men with more masculine voices (lower voice pitch and longer vocal tract length) as showing a lower likelihood of infidelity and higher relationship investment; further, women who preferred more masculine voices in long-term relationships, but not in short-term relationships, were more likely to perceive men with masculine voices as less likely to engage in infidelity and more likely to invest in their relationship. Moreover, the participants formed very similar impressions irrespective of whether the voices spoke native (Chinese) or foreign (English) languages. These results provide new evidence for the role of the voice in women's choices in selecting long-term partners.

Vocal Attractiveness Matters: Social Preferences in Cooperative Behavior

Research has shown the phenomenon that "what sounds beautiful is good" is a stereotype. It is not clear whether vocal attractiveness affects social decision-making in economic games. Using a modified trust game task, we investigated the neural mechanism of the influence of vocal attractiveness on cooperative decision-making. Participants first heard the voice (attractive or unattractive) of the partner. They had enough time to decide whether to cooperate with the partner for a chance to earn monetary rewards. The behavioral results showed that participants made more invest choices in the attractive partner condition, and they were more likely to cooperate with the female partners in the unattractive voice condition. The event-related potential (ERP) analysis for voice stimuli showed that attractive voices induced larger N1 amplitude than unattractive voices only in the male voice condition. And female voices elicited smaller N1 and larger P2 amplitudes than male voices in both the attractive and unattractive voices condition. A larger P3 amplitude was evoked by female voices and attractive voices. In addition, a more positive late positive complex (LPC) was induced by male voices and attractive voices. This study suggested that attractive voices facilitated cooperative behavior, providing evidence for the "beauty premium" effect of the attractive voices. Moreover, participants were more likely to cooperate with female partners. In the early stage, gender information and male vocal attractiveness were processed automatically, suggesting that male vocal attractiveness was processed preferentially than the female voice. In the late stage, participants allocated attention to both male and female vocal attractiveness.

Peer audience effects on children's vocal masculinity and femininity

Existing evidence suggests that children from around the age of 8 years strategically alter their public image in accordance with known values and preferences of peers, through the self-descriptive information they convey. However, an important but neglected aspect of this 'self-presentation' is the medium through which such information is communicated: the voice itself. The present study explored peer audience effects on children's vocal productions. Fifty-six children (26 females, aged 8-10 years) were presented with vignettes where a fictional child, matched to the participant's age and sex, is trying to make friends with a group of same-sex peers with stereotypically masculine or feminine interests (rugby and ballet, respectively). Participants were asked to impersonate the child in that situation and, as the child, to read out loud masculine, feminine and gender-neutral self-descriptive statements to these hypothetical audiences. They also had to decide which of those self-descriptive statements would be most helpful for making friends. In line with previous research, boys and girls preferentially selected masculine or feminine self-descriptive statements depending on the audience interests. Crucially, acoustic analyses of fundamental frequency and formant frequency spacing revealed that children also spontaneously altered their vocal productions: they feminized their voices when speaking to members of the ballet club, while they masculinized their voices when speaking to members of the rugby club. Both sexes also feminized their voices when uttering feminine sentences, compared to when uttering masculine and gender-neutral sentences. Implications for the hitherto neglected role of acoustic qualities of children's vocal behaviour in peer interactions are discussed. This article is part of the theme issue 'Voice modulation: from origin and mechanism to social impact (Part II)'.

The smell of cooperativeness: Do human body odours advertise cooperative behaviours?

Several physical features influence the perception of how cooperative a potential partner is. While previous work focused on face and voice, it remains unknown whether body odours influence judgements of cooperativeness and if odour-based judgements are accurate. Here, we first collected axillary odours of cooperative and uncooperative male donors through a public good game and used them as olfactory stimuli in a series of tasks examining whether and how they influence cooperative decision-making in an incentivized economic game and ratings of cooperativeness. Our results show that having access to the donor's body odours provided a strategic advantage to women during economic decisions (but not to men): with age, women were more likely to cooperate with cooperative men and to avoid interacting with uncooperative men. Ratings of cooperativeness were nonetheless unrelated to the donors' actual cooperativeness. Finally, while men with masculine and intense body odours were judged less cooperative, we found no evidence that donors' actual cooperativeness was associated with less masculine or less intense body odour. Overall, our findings suggest that, as faces and voices, body odours influence perceived cooperativeness and might be used accurately and in a non-aware manner as olfactory cues of cooperativeness, at least by women.

(Mis)perceiving cooperativeness

Cooperation is crucial for the success of social interactions. Given its importance, humans should readily be able to use available cues to predict how likely others are to cooperate. Here, we review the empirical literature on how accurate such predictions are. To this end, we distinguish between three classes of cues: behavioral (including past decisions), personal (including gender, attractiveness, and group membership) and situational (including the benefits to cooperation and the ability to communicate with each other). We discuss (i) how each cue correlates with future cooperative decisions and (ii) whether people correctly anticipate each cue's predictive value. We find that people are fairly accurate in interpreting behavioral and situational cues. However, they often misperceive the value of personal cues.

Masculine Voices Predict Attachment Style and Relationship Communication Patterns in Romantic Relationships

Humans exhibit sexually-based vocal dimorphisms, providing information about peoples' intrinsic states. Studies indicate that voice pitch predicts relationship quality; however, none have explored its effects on relationship maintenance. We explored the association between sexually dimorphic vocal properties [voice pitch, measured by fundamental frequency (F0) and F0 variation, the within-subject SD in F0 across the utterance (F0-SD)], attachment styles, and communication patterns among Chinese heterosexuals in romantic relationships. Men's F0-SD positively correlates with constructive communication pattern and negatively correlates with avoidant attachment style. No significant correlations are found for women. These findings suggest that men with masculine voices are more avoidantly attached, using avoidant communications. Furthermore, they show that voice may play a crucial role in and be an important morphological index of human mating relationships.

What do these sounds tell us about the therapeutic alliance: Acoustic markers as predictors of alliance

Predicting the trajectories of alliance formation that the patient is likely to establish with the therapist during treatment, even before their first meeting, can help prevent the potentially harmful consequences of deterioration in alliance, such as poor outcome and premature dropout. The present study aimed to examine the ability of four pretreatment acoustic markers to predict the alliance that is likely to be formed in the course of treatment: F0 span, speech rate, pause proportion and jitter. Data from 560 observations of 38 patients were collected as part of an ongoing randomized clinical trial of short-term psychotherapy for major depressive disorder. The acoustic markers were measured using high-quality recordings at baseline, before the patient and therapist ever met or had any type of communication. A multilevel model was used to examine the ability of the four acoustic markers to predict the slopes of alliance formation in the course of treatment, all markers being introduced in the same model. The clinical utility of the acoustic markers was explored in two case studies. The model explained 22% of the variance in alliance formation. Higher levels of both jitter and pause proportion at baseline predicted less strengthening of the alliance in the course of treatment. The findings, which should be replicated in larger samples, suggest that much of the therapeutic alliance can be predicted based on the acoustic characteristics of the patient's voice in the first 3 min of their intake, before they even meet their therapist.

Linking human male vocal parameters to perceptions, body morphology, strength and hormonal profiles in contexts of sexual selection

Sexual selection appears to have shaped the acoustic signals of diverse species, including humans. Deep, resonant vocalizations in particular may function in attracting mates and/or intimidating same-sex competitors. Evidence for these adaptive functions in human males derives predominantly from perception studies in which vocal acoustic parameters were manipulated using specialist software. This approach affords tight experimental control but provides little ecological validity, especially when the target acoustic parameters vary naturally with other parameters. Furthermore, such experimental studies provide no information about what acoustic variables indicate about the speaker-that is, why attention to vocal cues may be favored in intrasexual and intersexual contexts. Using voice recordings with high ecological validity from 160 male speakers and biomarkers of condition, including baseline cortisol and testosterone levels, body morphology and strength, we tested a series of pre-registered hypotheses relating to both perceptions and underlying condition of the speaker. We found negative curvilinear and negative linear relationships between male fundamental frequency (fo) and female perceptions of attractiveness and male perceptions of dominance. In addition, cortisol and testosterone negatively interacted in predicting fo, and strength and measures of body size negatively predicted formant frequencies (Pf). Meta-analyses of the present results and those from two previous samples confirmed that fonegatively predicted testosterone only among men with lower cortisol levels. This research offers empirical evidence of possible evolutionary functions for attention to men's vocal characteristics in contexts of sexual selection.

Does he sound cooperative? Acoustic correlates of cooperativeness

The sound of the voice has several acoustic features that influence the perception of how cooperative the speaker is. It remains unknown, however, whether these acoustic features are associated with actual cooperative behaviour. This issue is crucial to disentangle whether inferences of traits from voices are based on stereotypes, or facilitate the detection of cooperative partners. The latter is likely due to the pleiotropic effect that testosterone has on both cooperative behaviours and acoustic features. In the present study, we quantified the cooperativeness of native French-speaking men in a one-shot public good game. We also measured mean fundamental frequency, pitch variations, roughness, and breathiness from spontaneous speech recordings of the same men and collected saliva samples to measure their testosterone levels. Our results showed that men with lower-pitched voices and greater pitch variations were more cooperative. However, testosterone did not influence cooperative behaviours or acoustic features. Our finding provides the first evidence of the acoustic correlates of cooperative behaviour. When considered in combination with the literature on the detection of cooperativeness from faces, the results imply that assessment of cooperative behaviour would be improved by simultaneous consideration of visual and auditory cues.

Voice in different phases of menstrual cycle among naturally cycling women and users of hormonal contraceptives

Previous studies have shown changes in women's behavior and physical appearance between the non-fertile and fertile phases of the menstrual cycle. It is assumed that these changes are regulated by fluctuations in sex hormone levels across the cycle. Receptors for sex hormones have been found on the vocal folds, suggesting a link between hormone levels and vocal fold function, which might cause changes in voice production. However, attempts to identify changes in voice production across the menstrual cycle have produced mixed results. Therefore, the purpose of this study was to investigate changes in sexually dimorphic vocal characteristics and quality of women's voices in different phases of the cycle and to compare these with users of monophasic hormonal contraception. Voice samples (vowel phonation) of 44 naturally cycling women were obtained in the menstrual, late follicular (confirmed by LH surge) and luteal phases, and in 20 hormonal contraceptive users across equivalent stages of the monthly cycle. Results showed that voices of naturally cycling women had higher minimum pitch in the late follicular phase compared with the other phases. In addition, voice intensity was at its lowest in the luteal phase. In contrast, there were no voice changes across the cycle in hormonal contraceptive users. Comparison between the two groups of women revealed that the naturally cycling group had higher minimum pitch in the fertile phase and higher harmonics to noise ratio in the menstrual phase. In general, present results support the assumption that sex hormones might have an effect on voice function. These results, coupled with mixed findings in previous studies, suggest that vocal changes in relation to hormonal fluctuation are subtle, at least during vowel production. Future studies should explore voice changes in a defined social context and with more free-flowing speech.

Attractiveness Is Multimodal: Beauty Is Also in the Nose and Ear of the Beholder

Attractiveness plays a central role in human non-verbal communication and has been broadly examined in diverse subfields of contemporary psychology. Researchers have garnered compelling evidence in support of the evolutionary functions of physical attractiveness and its role in our daily lives, while at the same time, having largely ignored the significant contribution of non-visual modalities and the relationships among them. Acoustic and olfactory cues can, separately or in combination, strongly influence the perceived attractiveness of an individual and therefore attitudes and actions toward that person. Here, we discuss the relative importance of visual, auditory and olfactory traits in judgments of attractiveness, and review neural and behavioral studies that support the highly complex and multimodal nature of person perception. Further, we discuss three alternative evolutionary hypotheses aimed at explaining the function of multiple indices of attractiveness. In this review, we provide several lines of evidence supporting the importance of the voice, body odor, and facial and body appearance in the perception of attractiveness and mate preferences, and therefore the critical need to incorporate cross-modal perception and multisensory integration into future research on human physical attractiveness.