Effect of attention on ensemble perception: Comparison between exogenous attention, endogenous attention, and depth

Ensemble perception is an important ability of human beings that allows one to extract summary information for scenes and environments that contain information that far exceeds the processing limit of the visual system. Although attention has been shown to bias ensemble perception, two important questions remain unclear: (1) whether direct manipulations on different types of spatial attention could produce similar effects on ensembles and (2) whether factors potentially influencing the attention distribution, such as depth perception, could evoke an indirect effect of attention on ensemble representation. This study aims to address these questions. In Experiments 1 and 2, two types of precues were used to evoke exogenous and endogenous attention, respectively, and the ensemble color perceptions were examined. We found that both exogenous and endogenous attention biased ensemble representation towards the attended items, and the latter produced a greater effect. In Experiments 3 and 4, we examined whether depth perception could affect color ensembles by indirectly influencing attention allocation in 3D space. The items were separated in two depth planes, and no explicit cues were applied. The results showed that color ensemble was biased to closer items when depth information was task relevant. This suggests that ensemble perception is naturally biased in 3D space, probably through the mechanism of attention. Computational modeling consistently showed that attention exerted a direct shift on the ensemble statistics rather than averaging the feature values over the cued and noncued items, providing evidence against an averaging process of individual perception.

The perceptual and mnemonic effects of ensemble representation on individual size representation

Our visual world consists of multiple objects, necessitating the identification of individual objects. Nevertheless, the representation of visual objects often exerts influence on each other. Even when we selectively attend to a subset of visual objects, the representations of surrounding items are encoded and influence the processing of the attended item(s). However, it remains unclear whether the effect of group ensemble representation on individual item representation occurs at the perceptual encoding phase, during the memory maintenance period, or both. Therefore, the current study conducted visual psychophysics experiments to investigate the contributions of perceptual and mnemonic bias on the observed effect of ensemble representation on individual size representation. Across five experiments, we found a consistent pattern of repulsive ensemble bias, such that the size of an individual target circle was consistently reported to be smaller than it actually was when presented alongside other circles with larger mean size, and vice versa. There was a perceptual component to the bias, but mnemonic factors also influenced its magnitude. Specifically, the repulsion bias was strongest with a short retention period (0-50 ms), then reduced within a second to a weaker magnitude that remained stable for a longer retention period (5,000 ms). Such patterns of results persisted when we facilitated the processing of ensemble representation by increasing the set size (Experiment 1B) or post-cueing the target circle so that attention was distributed across all items (Experiment 2B).

Examining the role of action-driven attention in ensemble processing

Ensemble processing allows the visual system to condense visual information into useful summary statistics (e.g., average size), thereby overcoming capacity limitations to visual working memory and attention. To examine the role of attention in ensemble processing, we conducted three experiments using a novel paradigm that merged the action effect (a manipulation of attention) and ensemble processing. Participants were instructed to make a simple action if the feature of a cue word corresponded to a subsequent shape. Immediately after, they were shown an ensemble display of eight ovals of varying sizes and were asked to report either the average size of all ovals or the size of a single oval from the set. In Experiments 1 and 2, participants were cued with a task-relevant feature, and in Experiment 3, participants were cued with a task-irrelevant feature. Overall, the task-relevant cues that elicited an action influenced reports of average size in the ensemble phase more than the cues that were passively viewed, whereas task-irrelevant cues did not bias the reports of average size. The results of this study suggest that attention influences ensemble processing only when it is directed toward a task-relevant feature.

Priming effect of individual similarity and ensemble perception in visual search and working memory

Perceptual priming is a well-known phenomenon showing that the repetition of an object's feature can facilitate subsequent detection of that item. Although the priming effect has been rigorously studied in visual search, less is known about its effect on working memory and it is unclear whether the repetition of similar features, and furthermore, ensemble perception created by a large set of similar features, can induce priming. In this study, we investigated the priming effects of individual similarity and ensemble perception in visual search and visual working memory (VWM). We replicated the classic perceptual priming effect (Experiment 1a) and found that visual search was enhanced when the current target had a similar color to the previous target (Experiment 1b), but not when the similar color had been shown as a distractor before (Experiment 1c). However, if the target and distractors of similar colors formed ensemble perception, the search efficiency was again promoted even when the current target shared the same color with the previous distractor (Experiment 1d). For VWM, repeating the ensembles of the target- and nontarget-color subsets did not significantly affect the memory capacity, while switching the two harmed the memory fidelity but not capacity (Experiment 2). We suggest different underlying mechanisms for priming in visual search and VWM: in the former, the perception history of individual similarity and stimuli ensemble exert their effects on through the priority map, by forming a gradient distribution of attentional weights that peak at the previous target feature and diminish as stimulus diverges from the previously selected one; while in the latter, perception history of memory ensemble may influence the deployment of existing memory resources across trials, thereby affecting the memory fidelity but not its capacity.

Perceiving the outlier in the crowd: The influence of facial identity

The accurate perception of groups with outliers can help us identify potential risks. However, it is unclear how outliers affect the perception of group emotion. To address this question, we conducted a study on group emotion perception in the context of facial identity. We presented 74 participants with pictures of crowds, and asked them to evaluate the valence ratios and intensity of the crowd by means of the Emotional Aperture Measure. The results revealed that outlier emotions were often overestimated within crowds. Moreover, we found that the emotional expression of a close friend modulated the perception of outliers. Specifically, when a close friend expressed the group emotion, participants overestimated the outlier less than when a close friend expressed the outlier emotion. These results suggest that people can detect outliers within groups, and that their perception of group emotion is influenced by close friends. Thus, we provide evidence that facial identity affects group emotion perception.

Americans weigh an attended emotion more than Koreans in overall mood judgments

Face ensemble coding is the perceptual ability to create a quick and overall impression of a group of faces, triggering social and behavioral motivations towards other people (approaching friendly people or avoiding an angry mob). Cultural differences in this ability have been reported, such that Easterners are better at face ensemble coding than Westerners are. The underlying mechanism has been attributed to differences in processing styles, with Easterners allocating attention globally, and Westerners focusing on local parts. However, the remaining question is how such default attention mode is influenced by salient information during ensemble perception. We created visual displays that resembled a real-world social setting in which one individual in a crowd of different faces drew the viewer's attention while the viewer judged the overall emotion of the crowd. In each trial, one face in the crowd was highlighted by a salient cue, capturing spatial attention before the participants viewed the entire group. American participants' judgment of group emotion more strongly weighed the attended individual face than Korean participants, suggesting a greater influence of local information on global perception. Our results showed that different attentional modes between cultural groups modulate social-emotional processing underlying people's perceptions and attributions.

Nonadditive integration of visual information in ensemble processing

Statistically summarizing information from a stimulus array into an ensemble representation (e.g., the mean) improves the efficiency of visual processing. However, little is known about how the brain computes the ensemble statistics. Here, we propose that ensemble processing is realized by nonadditive integration, rather than linear averaging, of individual items. We used a linear regression model approach to extract EEG responses to three levels of information: the individual items, their local interactions, and their global interaction. The local and global interactions, representing nonadditive integration of individual items, elicited rapid and independent neural responses. Critically, only the neural representation of the global interaction predicted the precision of the ensemble perception at the behavioral level. Furthermore, spreading attention over the global pattern to enhance ensemble processing directly promoted rapid neural representation of the global interaction. Taken together, these findings advocate a global, nonadditive mechanism of ensemble processing in the brain.

Subcortical encoding of summary statistics in humans

Statistical encoding compresses redundant information from multiple items into a single summary metric (e.g., mean). Such statistical representation has been suggested to be automatic, but at which stage it is extracted is unknown. Here, we examined the involvement of the subcortex in the processing of summary statistics. We presented an array of circles dichoptically or monocularly while matching the number of perceived circles after binocular fusion. Experiments 1 and 2 showed that interocularly suppressed, invisible circles were automatically involved in the summary statistical representation, but only when they were presented to the same eye as the visible circles. This same-eye effect was further observed for consciously processed circles in Experiment 3, in which the estimated mean size of the circles was biased toward the information transmitted by monocular channels. Together, we provide converging evidence that the processing of summary statistics, an assumed high-level cognitive process, is mediated by subcortical structures.

Foveal vision determines the perceived emotion of face ensembles

People can extract summary statistical information from groups of similar objects, an ability called ensemble perception. However, not every object in a group is weighted equally. For example, in ensemble emotion perception, faces far from fixation were weighted less than faces close to fixation. Yet the contribution of foveal input in ensemble emotion perception is still unclear. In two experiments, groups of faces with varying emotions were presented for 100 ms at three different eccentricities (0°, 3°, 8°). Observers reported the perceived average emotion of the group. In two conditions, stimuli consisted of a central face flanked by eight faces (flankers) (central-present condition) and eight faces without the central face (central-absent condition). In the central-present condition, the emotion of the central face was either congruent or incongruent with that of the flankers. In Experiment 1, flanker emotions were uniform (identical flankers); in Experiment 2 they were varied. In both experiments, performance in the central-present condition was superior at 3° compared to 0° and 8°. At 0°, performance was superior in the central-absent (i.e., no foveal input) compared to the central-present condition. Poor performance in the central-present condition was driven by the incongruent condition where the foveal face strongly biased responses. At 3° and 8°, performance was comparable between central-present and central-absent conditions. Our results showed how foveal input determined the perceived emotion of face ensembles, suggesting that ensemble perception fails when salient target information is available in central vision.

Ensemble size perception: Its neural signature and the role of global interaction over individual items

To efficiently process complex visual scenes, the visual system often summarizes statistical information across individual items and represents them as an ensemble. However, due to the lack of techniques to disentangle the representation of the ensemble from that of the individual items constituting the ensemble, whether there exists a specialized neural mechanism for ensemble processing and how ensemble perception is computed in the brain remain unknown. To address these issues, we used a frequency-tagging EEG approach to track brain responses to periodically updated ensemble sizes. Neural responses tracking the ensemble size were detected in parieto-occipital electrodes, revealing a global and specialized neural mechanism of ensemble size perception. We then used the temporal response function to isolate neural responses to the individual sizes and their interactions. Notably, while the individual sizes and their local and global interactions were encoded in the EEG signals, only the global interaction contributed directly to the ensemble size perception. Finally, distributed attention to the global stimulus pattern enhanced the neural signature of the ensemble size, mainly by modulating the neural representation of the global interaction between all individual sizes. These findings advocate a specialized, global neural mechanism of ensemble size perception and suggest that global interaction between individual items contributes to ensemble perception.

Value-driven effects on perceptual averaging

Perceptual averaging refers to a strategy of encoding the statistical properties of entire sets of objects rather than encoding individual object properties, potentially circumventing the visual system's strict capacity limitations. Prior work has shown that such average representations of set properties, such as its mean size, can be modulated by top-down and bottom-up attention. However, it is unclear to what extent attentional biases through selection history, in the form of value-driven attentional capture, influences this type of summary statistical representation. To investigate, we conducted two experiments in which participants estimated the mean size of a set of heterogeneously sized circles while a previously rewarded color singleton was part of the set. In Experiment 1, all circles were gray, except either the smallest or the largest circle, which was presented in a color previously associated with a reward. When the largest circle in the set was associated with the highest value (as a proxy of selection history), we observed the largest biases, such that perceived mean size scaled linearly with the increasing value of the attended color singleton. In Experiment 2, we introduced a dual-task component in the form of an attentional search task to ensure that the observed bias of reward on perceptual averaging was not fully explained by focusing attention solely on the reward-signaling color singleton. Collectively, findings support the proposal that selection history, like bottom-up and top-down attention, influences perceptual averaging, and that this happens in a flexible manner proportional to the extent to which attention is captured.

Perceived variability reflects the reliability of individual items

When confronted with many visual items, people can compute their variability accurately and rapidly, which facilitates efficient information processing and optimal decision making. However, how the visual system computes variability is still unclear. To investigate this, we implemented situations whereby estimates of variability based on several possible variability measures (e.g., range, standard deviation, and weighted standard deviation) could be differentiated, and then examined which best accounted for human variability perception. In three psychophysical experiments, participants watched two arrays of items with various orientations and judged which had more variable orientations. Results showed that perceived variability was most consistent with the weighted standard deviation based on the reliability of individual items. Specifically, participants gave less consideration to deviant orientations that were likely to be outliers, and greater consideration to salient orientations that were likely to be encoded precisely. This reliability-based weighted standard deviation suggests an efficient and flexible way of representing visual variability.