Temporal and spatial ensemble statistics are formed by distinct mechanisms

Alcohol Use Predicts Face Perception Impairments and Difficulties in Face Recognition

Background: Risky alcohol use is related to a variety of cognitive impairments, including memory and visuo-perceptual difficulties. Remarkably, no prior work has assessed whether usage of alcohol can predict difficulties perceiving facial identity. Objectives: Therefore, this study aimed to investigate whether riskier alcohol consumption predicted impairments in face perception and self-reported difficulties in face recognition. Results: Participants (N = 239, male = 77) were over 18 years old and had normal or corrected-to-normal vision. Alcohol use was assessed using the Alcohol Use Disorder Identification Test (AUDIT), while face recognition difficulties were determined by the 20-item Prosopagnosia Index questionnaire (PI20). A subsample of participants (N = 126, male = 51) completed the Cambridge Face Perception task (CFPT) to assess their face perception ability. Multiple linear regressions showed significant models of prediction on both face perception and face recognition when considering AUDIT score and age as predictors. Conclusion: This study suggested, for the first time, that risky alcohol use predicts both poorer visuo-perceptual processing for faces and self-reported difficulties in face recognition.

The background assimilation effect: Facial emotional perception is affected by surrounding stimuli

The perception of facial emotion is not only determined by the physical features of the face itself but also be influenced by the emotional information of the background or surrounding information. However, the details of such effect are not fully understood. Here, the authors tested the perceived emotion of a target face surrounded by stimuli with different levels of emotional valence. In Experiment 1, four types of objects were divided into three groups (negative, unpleasant flowers and unpleasant animals; mildly negative (neutral), houses; positive, pleasant flowers). In Experiment 2, three groups of surrounding faces with different social-emotional valence (negative, neutral, and positive) were formed with the memory of affective personal knowledge. The data from two experiments showed that the perception of facial emotion can be influenced and modulated by the emotional valence of the surrounding stimuli, which can be explained by assimilation: the positive stimuli increased the valence of a target face, while the negative stimuli comparatively decreased it. Furthermore, the neutral stimuli also increased the valence of the target, which could be explained by the social positive effect. Therefore, the process of assimilation is likely to be a high-level emotional cognition rather than a low-level visual perception. The results of this study may help us better understand face perception in realistic scenarios.

Modeling facial perception in group context from a serial perception perspective

By utilizing statistical properties and summary statistics, the visual system can efficiently integrate perception of spatially and temporally adjacent stimuli into perception of a given target. For instance, perception of a target face can either be biased positively toward previous faces (e.g. the serial dependence effect) or be biased negatively by surrounding faces in the same trial/space (e.g. spatial ensemble averaging). However, both aspects were investigated separately. As spatial and temporal processing share the same purpose to reduce redundancy in visual processing, if one statistical processing occurs, would the statistical processing in the other domain still exist or be discarded? We investigated this question by exploring whether serial dependence of face perception (of attractiveness and averageness) survives when the changed face perception in the group context occurs. The results of Markov Chain modeling and conventional methods suggested that serial dependence (the temporal aspect) co-occurs with changed face perception in the group context (the spatial aspect). We also utilized the Hidden Markov modeling, as a new mathematical method, to model statistical processing from both domains. The results confirmed the co-occurrence of temporal effect and changed face perception in the group context for both attractiveness and averageness, suggesting potentially different spatial and temporal compression mechanisms in high-level vision. Further modeling and cluster analysis further revealed that the detailed computation of spatially and temporally adjacent faces in the attractiveness and averageness processing were similar yet different among different individuals. This work builds a bridge to understanding mathematical principles underlying changed face perception in the group context from the serial perspective.

Disrupted visual input unveils the computational details of artificial neural networks for face perception

Background

Convolutional Neural Network (DCNN), with its great performance, has attracted attention of researchers from many disciplines. The studies of the DCNN and that of biological neural systems have inspired each other reciprocally. The brain-inspired neural networks not only achieve great performance but also serve as a computational model of biological neural systems.

Methods

Here in this study, we trained and tested several typical DCNNs (AlexNet, VGG11, VGG13, VGG16, DenseNet, MobileNet, and EfficientNet) with a face ethnicity categorization task for experiment 1, and an emotion categorization task for experiment 2. We measured the performance of DCNNs by testing them with original and lossy visual inputs (various kinds of image occlusion) and compared their performance with human participants. Moreover, the class activation map (CAM) method allowed us to visualize the foci of the "attention" of these DCNNs.

Results

The results suggested that the VGG13 performed the best: Its performance closely resembled human participants in terms of psychophysics measurements, it utilized similar areas of visual inputs as humans, and it had the most consistent performance with inputs having various kinds of impairments.

Discussion

In general, we examined the processing mechanism of DCNNs using a new paradigm and found that VGG13 might be the most human-like DCNN in this task. This study also highlighted a possible paradigm to study and develop DCNNs using human perception as a benchmark.

Ensemble perception without phenomenal awareness of elements

Humans efficiently recognize complex scenes by grouping multiple features and objects into ensembles. It has been suggested that ensemble processing does not require, or even impairs, conscious discrimination of individual element properties. The present study examined whether ensemble perception requires phenomenal awareness of elements. We asked observers to judge the mean orientation of a line-based texture pattern whose central region was made invisible by backward masks. Masks were composed of either a Mondrian pattern (Exp. 1) or of an annular contour (Exp. 2) which, unlike the Mondrian, did not overlap spatially with elements in the central region. In the Mondrian-mask experiment, perceived mean orientation was determined only by visible elements outside the central region. However, in the annular-mask experiment, perceived mean orientation matched the mean orientation of all elements, including invisible elements within the central region. Results suggest that the visual system can compute spatial ensembles even without phenomenal awareness of stimuli.

The aftereffect of the ensemble average of facial expressions on subsequent facial expression recognition

An ensemble or statistical summary can be extracted from facial expressions presented in different spatial locations simultaneously. However, how such complicated objects are represented in the mind is not clear. It is known that the aftereffect of facial expressions, in which prolonged viewing of facial expressions biases the perception of subsequent facial expressions of the same category, occurs only when a visual representation is formed. Using this methodology, we examined whether an ensemble can be represented with visualized information. Experiment 1 revealed that the presentation of multiple facial expressions biased the perception of subsequent facial expressions to less happy as much as the presentation of a single face did. Experiment 2 compared the presentation of faces comprising strong and weak intensities of emotional expressions with an individual face as the adaptation stimulus. The results indicated that the perceptual biases were found after the presentation of four faces and a strong single face, but not after the weak single face presentation. Experiment 3 employed angry expressions, a distinct category from the test expression used as an adaptation stimulus; no aftereffect was observed. Finally, Experiment 4 clearly demonstrated the perceptual bias with a higher number of faces. Altogether, these results indicate that an ensemble average extracted from multiple faces leads to the perceptual bias, and this effect is similar in terms of its properties to that of a single face. This supports the idea that an ensemble of faces is represented with visualized information as a single face.

Modeling mean estimation tasks in within-trial and across-trial contexts

The mean estimation task, which explicitly asks observers to estimate the mean feature value of multiple stimuli, is a fundamental paradigm in research areas such as ensemble coding and cue integration. The current study uses computational models to formalize how observers summarize information in mean estimation tasks. We compare model predictions from our Fidelity-based Integration Model (FIM) and other models on their ability to simulate observed patterns in within-trial weight distribution, across-trial information integration, and set-size effects on mean estimation accuracy. Experiments show non-equal weighting within trials in both sequential and simultaneous mean estimation tasks. Observers implicitly overestimated trial means below the global mean and underestimated trial means above the global mean. Mean estimation performance declined and stabilized with increasing set sizes. FIM successfully simulated all observed patterns, while other models failed. FIM's information sampling structure provides a new way to interpret the capacity limit in visual working memory and sub-sampling strategies. As a model framework, FIM offers task-dependent modeling for various ensemble coding paradigms, facilitating research synthesis across different studies in the literature.

Attention Modulates the Ensemble Coding of Facial Expressions

Ensemble coding and attention are two mechanisms utilized by our visual system to overcome the limitation of visual processing when confronted with the overwhelming visual information. Recent evidence in ensemble coding of size suggests that the attended items contributed more to the averaging. On the other hand, some new evidence also indicates that reduced attention jeopardies the perceptual averaging of stimuli. What is the relationship between attention and ensemble coding? To answer this question, in the current study, we tested whether an exogenous attentional cue would influence the reported mean emotion of a crowd. We showed participants a group of four faces with different emotions. Participants' attention was guided to the happiest or saddest face (attention conditions), or not to any specific face (baseline condition). The results supported the notion that the attention alters the ensemble perception of the facial expression by elevating the weight of that face in the ensemble representation. This opens the question for the neural mechanisms of ensemble coding and its connection to visual attention.

A general serial dependence among various facial traits: Evidence from Markov Chain and derivative of Gaussian

The human vision system can extract a stable representation of the always-changing visual world. However, the mechanism underlying such perceptual continuity remains unclear. A possible candidate is the serial dependence: visual perception of an object is positively biased toward the visual input from the recent past. Does the visual system use one pattern of serial dependence for general purposes? Or different patterns of serial dependence for different visual tasks? Because different social facial traits (e.g., trustworthiness and dominance) are dissociable, it is reasonable to assume that the perception of different facial characteristics would require different patterns of serial dependences. In this study, we examine the existence and the similarities of the serial dependence(s) in the evaluation of seven facial characteristics (i.e., attractiveness, trustworthiness, confidence, dominance, intelligence, age, and aggressiveness). The convergent evidence from conventional Derivative of Gaussian fitting and Markov Chain modeling demonstrated that (1) serial dependence exists in judgments of all seven social facial characteristic, (2) the serial dependences of them are highly similar, and (3) the serial dependence follows the efficient coding. Thus it is highly possible that there exists a general serial dependence mechanism for (at least high-level) vision processing. Moreover, we used the Markov Chain modeling to better describe the transitional pattern of serial dependence, which is a kind of Markov process. These findings may shed light on future works regarding serial dependence, as well as face perception.

Change in Evaluation Mode Can Cause a Cheerleader Effect

The cheerleader effect describes the phenomenon whereby faces are perceived as being more attractive when flanked by other faces than when they are perceived in isolation. At least four theories predict the cheerleader effect. Two visual memory processes could cause a cheerleader effect. First, visual information will sometimes be averaged in the visual memory: the averaging of faces could increase the perceived attractiveness of all the faces flanked by other faces. Second, information will often be combined into a higher-order concept. This hierarchical encoding suggests that information processing causes faces to appear more attractive when flanked by highly attractive faces. Two further explanations posit that comparison processes cause the cheerleader effect. While contrast effects predict that a difference between the target face and the flanking faces causes the cheerleader effect due to comparison processes, a change in the evaluation mode, which alters the standard of comparison between joint and separate evaluation of faces, could be sufficient for producing a cheerleader effect. This leads to the prediction that even when there is no contrast between the attractiveness of the target face and the flanking faces, a cheerleader effect could occur. The results of one experiment support this prediction. The findings of this study have practical implications, such as for individuals who post selfies on social media. An individual’s face will appear more attractive in a selfie taken with people of low attractiveness than in a selfie without other people, even when all the faces have equally low levels of attractiveness.

Grasping of Real-World Objects Is Not Biased by Ensemble Perception

The visual system is known to extract summary representations of visually similar objects which bias the perception of individual objects toward the ensemble average. Although vision plays a large role in guiding action, less is known about whether ensemble representation is informative for action. Motor behavior is tuned to the veridical dimensions of objects and generally considered resistant to perceptual biases. However, when the relevant grasp dimension is not available or is unconstrained, ensemble perception may be informative to behavior by providing gist information about surrounding objects. In the present study, we examined if summary representations of a surrounding ensemble display influenced grip aperture and orientation when participants reached-to-grasp a central circular target which had an explicit size but importantly no explicit orientation that the visuomotor system could selectively attend to. Maximum grip aperture and grip orientation were not biased by ensemble statistics during grasping, although participants were able to perceive and provide manual estimations of the average size and orientation of the ensemble display. Support vector machine classification of ensemble statistics achieved above-chance classification accuracy when trained on kinematic and electromyography data of the perceptual but not grasping conditions, supporting our univariate findings. These results suggest that even along unconstrained grasping dimensions, visually-guided behaviors toward real-world objects are not biased by ensemble processing.

Friend effects framework: Contrastive and hierarchical processing in cheerleader effects

Cheerleader effects, group attractiveness effects, and divisive normalization are all characterized by faces appearing more attractive when seen within a group. However, it is possible that your friends could have a detrimental effect upon your attractiveness too: if these group effects arose partly as a contrastive process between your face and your friends, then highly attractive friends may diminish your attractiveness. We confirm this hypothesis across two experiments by showing that the presence of highly attractive friends can indeed make you appear less attractive (i.e., a reverse cheerleader effect), suggesting friend effects are driven in part by a contrastive process against the group. However, these effects are also influenced by your own attractiveness in a fashion that appears consistent with hierarchical encoding, where less attractive targets benefit more from being viewed in an increasingly unattractive group than attractive targets. Our final experiment demonstrates that the company of others not only alters our attractiveness, but also induces shifts in how average or distinctive a target face appears too, with these averageness effects associated with the friend effects observed in our first experiment. We present a Friend Effects Framework within which 'friend effects' is an umbrella term for the positive (e.g., cheerleader effects, group attractiveness effects) and negative (i.e., the reverse cheerleader effect) ways in which hierarchical encoding, group contrastive effects, and other influences of friends can have on your attractiveness.

Metacognition of average face perception

Individuals have the ability to extract summary statistics from multiple items presented simultaneously. However, it is unclear yet whether we have insight into the process of ensemble coding. The aim of this study was to investigate metacognition about average face perception. Participants saw a group of four faces presented for 2 s or 5 s, and then they were asked to judge whether the following test face was present in the previous set (Experiment 1), or whether the test face was the average of the four member faces (Experiment 2). After each response, participants rated their confidence. Replicating previous findings, there was substantial endorsement for the average face derived from the four member faces in Experiment 1, even though it was not present in the set. When judging faces that had been presented in the set, confidence correlated positively with accuracy, providing evidence for metacognitive awareness of previously studied faces. Importantly, there was a negative confidence-accuracy relationship for judging average faces when duration was 2 s, and a near-zero relationship when duration was 5 s. By contrast, when the average face had to be identified explicitly in Experiment 2, performance was above chance level and there was a positive correlation between confidence and accuracy. These results suggest that people have metacognitive awareness about average face perception when averaging is required explicitly, but they lack insight into the averaging process when member identification is required.

Contrast Effect of Facial Attractiveness in Groups

Research on facial attractiveness is an important part of aesthetics. Most relevant studies in the area have focused on the influence of individual perspectives on facial attractiveness, but it is necessary to consider the effect of contextual information on facial attractiveness. In this study, we examine the influence on attractiveness of special faces in a given group. We define a “special face” as one that is significantly different from other members of the same group in terms of facial attractiveness. We conducted three experiments to explore the influence of different modes of presentation and central positions in a group on the judgment of attractiveness of the special face. The results show the following: (1) When the special face was part of a given group, the subjects made more extreme judgments than without it: that is, they judged the most attractive face as more attractive and the least as less attractive than when faces were presented alone. (2) The subjects rated the most attractive faces lower and the least attractive faces higher when the target faces in the middle of the group than in other positions. The results favored the contrast effect: when the subjects judged the attractiveness of target stimulus, they always compared it with the environment, which then became a reference in this regard. Moreover, the greater the amount of contextual information perceived, the higher the likelihood that assimilation would occur.