Quantifying idiosyncratic and shared contributions to judgment

Discrepancies in perceived humanness between spatially filtered and unfiltered faces and their associations with uncanny feelings

Human and artificial features that coexist in certain types of human-like robots create a discrepancy in perceived humanness and evoke uncanny feelings in human observers. However, whether this perceptual mismatch in humanness occurs for all faces, and whether it is related to the uncanny feelings toward them, is unknown. We investigated this by examining perceived humanness for a variety of natural images of robot and human faces with different spatial frequency (SF) information: that is, faces with only low SF, middle SF, and high SF information, and intact (spatially unfiltered) faces. Uncanny feelings elicited by these faces were also measured. The results showed perceptual mismatches that LSF, MSF, and HSF faces were perceived as more human than intact faces. This was particularly true for intact robot faces that looked slightly human, which tended to evoke strong uncanny feelings. Importantly, the mismatch in perceived humanness between the intact and spatially filtered faces was positively correlated with uncanny feelings toward intact faces. Given that the human visual system performs SF analysis when processing faces, the perceptual mismatches observed in this study likely occur in real life for all faces, and as such might be a ubiquitous source of uncanny feelings in real-life situations.

Consensus and contention in beauty judgment

Variance across participants is at the heart of the centuries-old debate about the universality of beauty. Beauty's belonging to the eye of the beholder implies large interindividual variance, while beauty as a universal object property implies the opposite. To characterize the variance at the center of this debate, we selected two quartets with either high- or low-variance images with high typicality and a given mean beauty. The quartets have high or low variance across 50 participants (group variance) and correspondingly high or low variance across images of a quartet for each participant (quartet variance). We asked 52 new participants to estimate their own mean and quartet variance. Participants successfully predicted their quartet mean but failed to predict their quartet variance. Though invisible, beauty variance is essential to prediction, both in theory and in practice. The quartets show that mean beauty is not the whole story - beauty variance is heterogeneous.

Idiosyncratic and shared contributions shape impressions from voices and faces

Voices elicit rich first impressions of what the person we are hearing might be like. Research stresses that these impressions from voices are shared across different listeners, such that people on average agree which voices sound trustworthy or old and which do not. However, can impressions from voices also be shaped by the 'ear of the beholder'? We investigated whether - and how - listeners' idiosyncratic, personal preferences contribute to first impressions from voices. In two studies (993 participants, 156 voices), we find evidence for substantial idiosyncratic contributions to voice impressions using a variance portioning approach. Overall, idiosyncratic contributions were as important as shared contributions to impressions from voices for inferred person characteristics (e.g., trustworthiness, friendliness). Shared contributions were only more influential for impressions of more directly apparent person characteristics (e.g., gender, age). Both idiosyncratic and shared contributions were reduced when stimuli were limited in their (perceived) variability, suggesting that natural variation in voices is key to understanding this impression formation. When comparing voice impressions to face impressions, we found that idiosyncratic and shared contributions to impressions similarly across modality when stimulus properties are closely matched - although voice impressions were overall less consistent than face impressions. We thus reconceptualise impressions from voices as being formed not only based on shared but also idiosyncratic contributions. We use this new framing to suggest future directions of research, including understanding idiosyncratic mechanisms, development, and malleability of voice impression formation.

Implicit bias and experience influence overall but not relative trustworthiness judgment of other-race faces

Impressions of trustworthiness are formed quickly from faces. To what extent are these impressions shared among observers of the same or different races? Although high consensus of trustworthiness evaluation has been consistently reported, recent studies suggested substantial individual differences. For instance, negative implicit racial bias and low contact experience towards individuals of the other race have been shown to be related to low trustworthiness judgments for other-race faces. This pre-registered study further examined the effects of implicit social bias and experience on trustworthiness judgments of other-race faces. A relatively large sample of White (N = 338) and Black (N = 299) participants completed three tasks: a trustworthiness rating task of faces, a race implicit association test, and a questionnaire of experience. Each participant rated trustworthiness of 100 White faces and 100 Black faces. We found that the overall trustworthiness ratings for other-race faces were influenced by both implicit bias and experience with individuals of the other-race. Nonetheless, when comparing to the own-race baseline ratings, high correlations were observed for the relative differences in trustworthiness ratings of other-race faces for participants with varied levels of implicit bias and experience. These results suggest differential impact of social concepts (e.g., implicit bias, experience) vs. instinct (e.g., decision of approach-vs-avoid) on trustworthiness impressions, as revealed by overall vs. relative ratings on other-race faces.

Modelling individual aesthetic judgements over time

Listening to music, watching a sunset-many sensory experiences are valuable to us, to a degree that differs significantly between individuals, and within an individual over time. We have theorized (Brielmann & Dayan 2022 Psychol. Rev. 129, 1319-1337 (doi:10.1037/rev0000337))) that these idiosyncratic values derive from the task of using experiences to tune the sensory-cognitive system to current and likely future input. We tested the theory using participants' (n = 59) ratings of a set of dog images (n = 55) created using the NeuralCrossbreed morphing algorithm. A full realization of our model that uses feature representations extracted from image-recognizing deep neural nets (e.g. VGG-16) is able to capture liking judgements on a trial-by-trial basis (median r = 0.65), outperforming predictions based on population averages (median r = 0.01). Furthermore, the model's learning component allows it to explain image sequence dependent rating changes, capturing on average 17% more variance in the ratings for the true trial order than for simulated random trial orders. This validation of our theory is the first step towards a comprehensive treatment of individual differences in evaluation. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectives'.

Genetic effects on variability in visual aesthetic evaluations are partially shared across visual domains

The aesthetic values that individuals place on visual images are formed and shaped over a lifetime. However, whether the formation of visual aesthetic value is solely influenced by environmental exposure is still a matter of debate. Here, we considered differences in aesthetic value emerging across three visual domains: abstract images, scenes, and faces. We examined variability in two major dimensions of ordinary aesthetic experiences: taste-typicality and evaluation-bias. We build on two samples from the Australian Twin Registry where 1547 and 1231 monozygotic and dizygotic twins originally rated visual images belonging to the three domains. Genetic influences explained 26% to 41% of the variance in taste-typicality and evaluation-bias. Multivariate analyses showed that genetic effects were partially shared across visual domains. Results indicate that the heritability of major dimensions of aesthetic evaluations is comparable to that of other complex social traits, albeit lower than for other complex cognitive traits. The exception was taste-typicality for abstract images, for which we found only shared and unique environmental influences. Our study reveals that diverse sources of genetic and environmental variation influence the formation of aesthetic value across distinct visual domains and provides improved metrics to assess inter-individual differences in aesthetic value.

Quantifying the contribution of subject and group factors in brain activation

Research in neuroscience often assumes universal neural mechanisms, but increasing evidence points toward sizeable individual differences in brain activations. What remains unclear is the extent of the idiosyncrasy and whether different types of analyses are associated with different levels of idiosyncrasy. Here we develop a new method for addressing these questions. The method consists of computing the within-subject reliability and subject-to-group similarity of brain activations and submitting these values to a computational model that quantifies the relative strength of group- and subject-level factors. We apply this method to a perceptual decision-making task (n = 50) and find that activations related to task, reaction time, and confidence are influenced equally strongly by group- and subject-level factors. Both group- and subject-level factors are dwarfed by a noise factor, though higher levels of smoothing increases their contributions relative to noise. Overall, our method allows for the quantification of group- and subject-level factors of brain activations and thus provides a more detailed understanding of the idiosyncrasy levels in brain activations.

Self-Relevance Predicts the Aesthetic Appeal of Real and Synthetic Artworks Generated via Neural Style Transfer

What determines the aesthetic appeal of artworks? Recent work suggests that aesthetic appeal can, to some extent, be predicted from a visual artwork's image features. Yet a large fraction of variance in aesthetic ratings remains unexplained and may relate to individual preferences. We hypothesized that an artwork's aesthetic appeal depends strongly on self-relevance. In a first study (N = 33 adults, online replication N = 208), rated aesthetic appeal for real artworks was positively predicted by rated self-relevance. In a second experiment (N = 45 online), we created synthetic, self-relevant artworks using deep neural networks that transferred the style of existing artworks to photographs. Style transfer was applied to self-relevant photographs selected to reflect participant-specific attributes such as autobiographical memories. Self-relevant, synthetic artworks were rated as more aesthetically appealing than matched control images, at a level similar to human-made artworks. Thus, self-relevance is a key determinant of aesthetic appeal, independent of artistic skill and image features.

The intrinsic variance of beauty judgment

Recall memory and sequential dependence threaten the independence of successive beauty ratings. Such independence is usually assumed when using repeated measures to estimate the intrinsic variance of a rating. We call "intrinsic" the variance of all possible responses that the participant could give on a trial. Variance arises within and across participants. In attributing the measured variance to sources, the first step is to assess how much is intrinsic. In seven experiments, we measure how much of the variability across beauty ratings can be attributed to recall memory and sequential dependence. With a set size of one, memory is a problem and contributes half the measured variance. However, we showed that for both beauty and ellipticity, with set size of nine or more, recall memory causes a mere 10% increase in the variance of repeated ratings. Moreover, we showed that as long as the stimuli are diverse (i.e., represent different object categories), sequential dependence does not affect the variance of beauty ratings. Lastly, the variance of beauty ratings increases in proportion to the 0.15 power of stimulus set size. We show that the beauty rating of a stimulus in a diverse set is affected by the stimulus set size and not the value of other stimuli. Overall, we conclude that the variance of repeated ratings is a good way to estimate the intrinsic variance of a beauty rating of a stimulus in a diverse set.

Ten simple rules for socially responsible science

Guidelines concerning the potentially harmful effects of scientific studies have historically focused on ethical considerations for minimizing risk for participants. However, studies can also indirectly inflict harm on individuals and social groups through how they are designed, reported, and disseminated. As evidenced by recent criticisms and retractions of high-profile studies dealing with a wide variety of social issues, there is a scarcity of resources and guidance on how one can conduct research in a socially responsible manner. As such, even motivated researchers might publish work that has negative social impacts due to a lack of awareness. To address this, we propose 10 simple rules for researchers who wish to conduct socially responsible science. These rules, which cover major considerations throughout the life cycle of a study from inception to dissemination, are not aimed as a prescriptive list or a deterministic code of conduct. Rather, they are meant to help motivated scientists to reflect on their social responsibility as researchers and actively engage with the potential social impact of their research.

Individual attractiveness preferences differentially modulate immediate and voluntary attention

Physical attractiveness plays a crucial role in mate choice for both men and women. This is reflected in visual attention: people immediately attend towards and look longer at attractive faces, especially when they are motivated to find a partner. However, previous studies did not incorporate real-life dating decisions. Here, we aimed to combine attentional tasks with individual attractiveness ratings and a real-life mate choice context, namely a speed-dating paradigm. We investigated whether heterosexual non-committed young adults showed biases in immediate and voluntary attention towards attractive faces and preferred dating partners. In line with previous research, we found considerable individual differences in individual attractiveness preferences. Furthermore, our results showed that men had a bias towards attractive faces and preferred dating partners in the immediate attention task, while results for women were mixed. In the voluntary attention task, however, both men and women had an attentional bias towards attractive faces and preferred dating partners. Our results suggest that individual attractiveness preferences are good predictors of especially voluntary attention. We discuss these findings from an evolutionary perspective and suggest directions for future research.

Generative models for visualizing idiosyncratic impressions

In their comprehensive review of research on impressions from faces, Sutherland and Young (this issue) highlight both the remarkable progress and the many challenges facing the field. We focus on two of the challenges: the need for generative, powerful models of impressions and the idiosyncratic nature of complex impressions.

Modern applications of cross-classified random effects models in social and behavioral research: Illustration with R package PLmixed

Cross-classified random effects models (CCREMs) have been developed for appropriately analyzing data with a cross-classified structure. Despite its flexibility and the prevalence of cross-classified data in social and behavioral research, CCREMs have been under-utilized in applied research. In this article, we present CCREMs as a general and flexible modeling framework, and present a wide range of existing models designed for different purposes as special instances of CCREMs. We also introduce several less well-known applications of CCREMs. The flexibility of CCREMs allows these models to be easily extended to address substantive questions. We use the free R package PLmixed to illustrate the estimation of these models, and show how the general language of the CCREM framework can be translated into specific modeling contexts.

A data-driven, hyper-realistic method for visualizing individual mental representations of faces

Research in person and face perception has broadly focused on group-level consensus that individuals hold when making judgments of others (e.g., “X type of face looks trustworthy”). However, a growing body of research demonstrates that individual variation is larger than shared, stimulus-level variation for many social trait judgments. Despite this insight, little research to date has focused on building and explaining individual models of face perception. Studies and methodologies that have examined individual models are limited in what visualizations they can reliably produce to either noisy and blurry or computer avatar representations. Methods that produce low-fidelity visual representations inhibit generalizability by being clearly computer manipulated and produced. In the present work, we introduce a novel paradigm to visualize individual models of face judgments by leveraging state-of-the-art computer vision methods. Our proposed method can produce a set of photorealistic face images that correspond to an individual's mental representation of a specific attribute across a variety of attribute intensities. We provide a proof-of-concept study which examines perceived trustworthiness/untrustworthiness and masculinity/femininity. We close with a discussion of future work to substantiate our proposed method.

Personality Across World Regions Predicts Variability in the Structure of Face Impressions

Research on face impressions has often focused on a fixed, universal architecture, treating regional variability as noise. Here, we demonstrated a crucial yet neglected role of cultural learning processes in forming face impressions. In Study 1, we found that variability in the structure of adult perceivers' face impressions across 42 world regions (N = 287,178) could be explained by variability in the actual personality structure of people living in those regions. In Study 2, data from 232 world regions (N = 307,136) revealed that adult perceivers use the actual personality structure learned from their local environment to form lay beliefs about personality, and these beliefs in turn support the structure of perceivers' face impressions. Together, these results suggest that people form face impressions on the basis of a conceptual understanding of personality structure that they have come to learn from their regional environment. The findings suggest a need for greater attention to the regional and cultural specificity of face impressions.

Self-perception of orofacial appearance: Brazil-Finland cross-national study

OBJECTIVE

(i) To study the measurement invariance of Orofacial Esthetic Scale (OES) and Psychosocial Impact of Dental Aesthetics Questionnaire (PIDAQ), (ii) to compare the perception of orofacial appearance (OA) and (iii) to study the frequency of individuals who have sought or received aesthetic dental treatment between Brazil and Finland.

METHODS

This was a cross-sectional observational study with snowball non-probability sample selection. Students and staff from universities in Finland and Brazil were invited to participate. Data were collected online using a demographic questionnaire, OES and PIDAQ. The samples consisted of 3636 Finns (75.0% female; age: 32.0 years) and 1468 Brazilians (72.6% female; age: 33.2 years). The frequency of receiving aesthetic dental treatment was estimated. If configurational invariance was observed, cross-national measurement invariance was verified by multigroup analysis. When measurement invariance was attested, factor scores were compared using Welch's t-test.

RESULTS

OES showed configurational and measurement invariance and no significant difference between the countries. Despite similarity in satisfaction with OA, 71.9% of Brazilians had received aesthetic dental treatment, while 59.4% of Finns had never sought such treatments. PIDAQ did not present configurational invariance between the countries.

CONCLUSION

Although there is no difference in satisfaction with OA, seeking and receiving aesthetic dental treatment is significantly greater for Brazilians. Psychosocial impact of OA is perceived differently in the studied countries.

Deep models of superficial face judgments

We quickly and irresistibly form impressions of what other people are like based solely on how their faces look. These impressions have real-life consequences ranging from hiring decisions to sentencing decisions. We model and visualize the perceptual bases of facial impressions in the most comprehensive fashion to date, producing photorealistic models of 34 perceived social and physical attributes (e.g., trustworthiness and age). These models leverage and demonstrate the utility of deep learning in face evaluation, allowing for 1) generation of an infinite number of faces that vary along these perceived attribute dimensions, 2) manipulation of any face photograph along these dimensions, and 3) prediction of the impressions any face image may evoke in the general (mostly White, North American) population.

The diversity of human faces and the contexts in which they appear gives rise to an expansive stimulus space over which people infer psychological traits (e.g., trustworthiness or alertness) and other attributes (e.g., age or adiposity). Machine learning methods, in particular deep neural networks, provide expressive feature representations of face stimuli, but the correspondence between these representations and various human attribute inferences is difficult to determine because the former are high-dimensional vectors produced via black-box optimization algorithms. Here we combine deep generative image models with over 1 million judgments to model inferences of more than 30 attributes over a comprehensive latent face space. The predictive accuracy of our model approaches human interrater reliability, which simulations suggest would not have been possible with fewer faces, fewer judgments, or lower-dimensional feature representations. Our model can be used to predict and manipulate inferences with respect to arbitrary face photographs or to generate synthetic photorealistic face stimuli that evoke impressions tuned along the modeled attributes.

Four dimensions characterize attributions from faces using a representative set of English trait words

People readily (but often inaccurately) attribute traits to others based on faces. While the details of attributions depend on the language available to describe social traits, psychological theories argue that two or three dimensions (such as valence and dominance) summarize social trait attributions from faces. However, prior work has used only a small number of trait words (12 to 18), limiting conclusions to date. In two large-scale, preregistered studies we ask participants to rate 100 faces (obtained from existing face stimuli sets), using a list of 100 English trait words that we derived using deep neural network analysis of words that have been used by other participants in prior studies to describe faces. In study 1 we find that these attributions are best described by four psychological dimensions, which we interpret as “warmth”, “competence”, “femininity”, and “youth”. In study 2 we partially reproduce these four dimensions using the same stimuli among additional participant raters from multiple regions around the world, in both aggregated and individual-level data. These results provide a comprehensive characterization of trait attributions from faces, although we note our conclusions are limited by the scope of our study (in particular we note only white faces and English trait words were included).

People form impressions about others from seeing their faces, and use many words to describe those impressions. Here, using ratings of 100 representatively sampled white adult faces on a large set of traits, the authors show that trait impressions from faces can be summarized by four psychological dimensions.

From Visual Perception to Aesthetic Appeal: Brain Responses to Aesthetically Appealing Natural Landscape Movies

During aesthetically appealing visual experiences, visual content provides a basis for computation of affectively tinged representations of aesthetic value. How this happens in the brain is largely unexplored. Using engaging video clips of natural landscapes, we tested whether cortical regions that respond to perceptual aspects of an environment (e.g., spatial layout, object content and motion) were directly modulated by rated aesthetic appeal. Twenty-four participants watched a series of videos of natural landscapes while being scanned using functional magnetic resonance imaging (fMRI) and reported both continuous ratings of enjoyment (during the videos) and overall aesthetic judgments (after each video). Although landscape videos engaged a greater expanse of high-level visual cortex compared to that observed for images of landscapes, independently localized category-selective visual regions (e.g., scene-selective parahippocampal place area and motion-selective hMT+) were not significantly modulated by aesthetic appeal. Rather, a whole-brain analysis revealed modulations by aesthetic appeal in ventral (collateral sulcus) and lateral (middle occipital sulcus, posterior middle temporal gyrus) clusters that were adjacent to scene and motion selective regions. These findings suggest that aesthetic appeal per se is not represented in well-characterized feature- and category-selective regions of visual cortex. Rather, we propose that the observed activations reflect a local transformation from a feature-based visual representation to a representation of "elemental affect," computed through information-processing mechanisms that detect deviations from an observer's expectations. Furthermore, we found modulation by aesthetic appeal in subcortical reward structures but not in regions of the default-mode network (DMN) nor orbitofrontal cortex, and only weak evidence for associated changes in functional connectivity. In contrast to other visual aesthetic domains, aesthetically appealing interactions with natural landscapes may rely more heavily on comparisons between ongoing stimulation and well-formed representations of the natural world, and less on top-down processes for resolving ambiguities or assessing self-relevance.

Individual differences in trust evaluations are shaped mostly by environments, not genes

People evaluate a stranger's trustworthiness from their facial features in a fraction of a second, despite common advice "not to judge a book by its cover." Evaluations of trustworthiness have critical and widespread social impact, predicting financial lending, mate selection, and even criminal justice outcomes. Consequently, understanding how people perceive trustworthiness from faces has been a major focus of scientific inquiry, and detailed models explain how consensus impressions of trustworthiness are driven by facial attributes. However, facial impression models do not consider variation between observers. Here, we develop a sensitive test of trustworthiness evaluation and use it to document substantial, stable individual differences in trustworthiness impressions. Via a twin study, we show that these individual differences are largely shaped by variation in personal experience, rather than genes or shared environments. Finally, using multivariate twin modeling, we show that variation in trustworthiness evaluation is specific, dissociating from other key facial evaluations of dominance and attractiveness. Our finding that variation in facial trustworthiness evaluation is driven mostly by personal experience represents a rare example of a core social perceptual capacity being predominantly shaped by a person's unique environment. Notably, it stands in sharp contrast to variation in facial recognition ability, which is driven mostly by genes. Our study provides insights into the development of the social brain, offers a different perspective on disagreement in trust in wider society, and motivates new research into the origins and potential malleability of face evaluation, a critical aspect of human social cognition.