Ensemble Perception

The perception/cognition distincton: Challenging the representational account

A central goal for cognitive science and philosophy of mind is to distinguish between perception and cognition. The representational approach has emerged as a prominent candidate to draw such a distinction. The idea is that perception and cognition differ in the content and the format in which the information is represented -just as perceptual representations are nonconceptual in content and iconic in format, cognitive representations are conceptual in content and discursive in format. This paper argues against this view. I argue that both perception and cognition can use conceptual and nonconceptual contents and be vehiculated in iconic and discursive formats. If correct, the representational strategy to distinguish perception from cognition fails.

What kind of empirical evidence is needed for probabilistic mental representations? An example from visual perception

Recent accounts of perception and cognition propose that the brain represents information probabilistically. While this assumption is common, empirical support for such probabilistic representations in perception has recently been criticized. Here, we evaluate these criticisms and present an account based on a recently developed psychophysical methodology, Feature Distribution Learning (FDL), which provides promising evidence for probabilistic representations by avoiding these criticisms. The method uses priming and role-reversal effects in visual search. Observers' search times reveal the structure of perceptual representations, in which the probability distribution of distractor features is encoded. We explain how FDL results provide evidence for a stronger notion of representation that relies on structural correspondence between stimulus uncertainty and perceptual representations, rather than a mere co-variation between the two. Moreover, such an account allows us to demonstrate what kind of empirical evidence is needed to support probabilistic representations as posited in current probabilistic Bayesian theories of perception.

Multivariate summary of a complex scene

The current study investigated how people summarize and represent objects with multiple features to cope with the complexity due to the number of objects and feature dimensions. We presented a set of circles whose color and size were either correlated perfectly (r = 1) or not correlated at all (r = 0). Using a membership identification task, we found that participants formed a statistical representation that included information about conjunctions as well as each color and size dimensions. In addition, we found that participants represented different set boundaries depending on the correlation between features of a set. Lastly, a pair-matching task revealed that participants predicted one feature value from the other feature value based on the correlation between features of a set. Our findings suggest that people represent a multi-feature ensemble statistically as a multivariate feature distribution, which is an efficient strategy to cope with scene complexity.

Fitness Beats Truth in the Evolution of Perception

Does natural selection favor veridical percepts-those that accurately (if not exhaustively) depict objective reality? Perceptual and cognitive scientists standardly claim that it does. Here we formalize this claim using the tools of evolutionary game theory and Bayesian decision theory. We state and prove the "Fitness-Beats-Truth (FBT) Theorem" which shows that the claim is false: If one starts with the assumption that perception involves inference to states of the objective world, then the FBT Theorem shows that a strategy that simply seeks to maximize expected-fitness payoff, with no attempt to estimate the "true" world state, does consistently better. More precisely, the FBT Theorem provides a quantitative measure of the extent to which the fitness-only strategy dominates the truth strategy, and of how this dominance increases with the size of the perceptual space. The FBT Theorem supports the Interface Theory of Perception (e.g. Hoffman et al. in Psychon Bull Rev https://doi.org/10.3758/s13423-015-0890-8 , 2015), which proposes that our perceptual systems have evolved to provide a species-specific interface to guide adaptive behavior, and not to provide a veridical representation of objective reality.

Perception of the Potential for Interaction in Social Scenes

In urban environments, humans often encounter other people that may engage one in interaction. How do humans perceive such invitations to interact at a glance? We briefly presented participants with pictures of actors carrying out one of 11 behaviors (e.g., waving or looking at a phone) at four camera-actor distances. Participants were asked to describe what they might do in such a situation, how they decided, and what stood out most in the photograph. In addition, participants rated how likely they deemed interaction to take place. Participants formulated clear responses about how they might act. We show convincingly that what participants would do depended on the depicted behavior, but not the camera-actor distance. The likeliness to interact ratings depended both on the depicted behavior and the camera-actor distance. We conclude that humans perceive the "gist" of photographs and that various aspects of the actor, action, and context depicted in photographs are subjectively available at a glance. Our conclusions are discussed in the context of scene perception, social robotics, and intercultural differences.

Is the n-back task a measure of unstructured working memory capacity? Towards understanding its connection to other working memory tasks

Working memory is fundamental to human cognitive functioning, and it is often measured with the n-back task. However, it is not clear whether the n-back task is a valid measure of working memory. Importantly, previous studies have found poor correlations with measures of complex span, whereas a recent study (Frost et al., 2019) showed that n-back performance was correlated with a transsaccadic memory task but dissociated from performance on the change detection task, a well-accepted measure of working memory capacity. To test whether capacity is involved in the n-back task we correlated a spatial version of the test with different versions of the change detection task. Experiment 1 introduced perceptual and cognitive disruptions to the change detection task. This impacted task performance, however, all versions of the change detection task remained highly correlated with one another whereas there was no significant correlation with the n-back task. Experiment 2 removed spatial and non-spatial context from the change detection task. This produced a correlation with n-back. Our results indicate that the n-back task is supported by faculties similar to those that support change detection, but that this commonality is hidden when contextual information is available to be exploited in a change detection task such that structured representations can form. We suggest that n-back might be a valid measure of working memory, and that the ability to exploit contextual information is an important faculty captured by some versions of the change detection task.

Oversampling of minority categories drives misperceptions of group compositions

The ability to estimate proportions informs our immediate impressions of social environments (e.g., of the diversity of races or genders within a crowded room). This study examines how the distribution of attention during brief glances shapes estimates of group gender proportions. Performance-wise, subjects exhibit a canonical pattern of judgment errors: small proportions are overestimated while large values are underestimated. Subjects' eye movements at sub-second timescales reveal that these biases follow from a tendency to visually oversample members of the gender minority. Rates of oversampling dovetail with average levels of error magnitudes, response variability, and response times. Visual biases are thus associated with the inherent difficulty in estimating particular proportions. All results are replicated at a within-subjects level with non-human ensembles using natural scene stimuli; the observed attentional patterns and judgment biases are thus not exclusively guided by face-specific visual properties. Our results reveal the biased distribution of attention underlying typical judgment errors of group proportions.

Testing temporal integration of feature probability distributions using role-reversal effects in visual search

The visual system is sensitive to statistical properties of complex scenes and can encode feature probability distributions in detail. But does the brain use these statistics to build probabilistic models of the ever-changing visual input? To investigate this, we examined how observers temporally integrate two different orientation distributions from sequentially presented visual search trials. If the encoded probabilistic information is used in a Bayesian optimal way, observers should weigh more reliable information more strongly, such as feature distributions with low variance. We therefore manipulated the variance of the two feature distributions. Participants performed sequential odd-one-out visual search for an oddly oriented line among distractors. During successive learning trials, the distractor orientations were sampled from two different Gaussian distributions on alternating trials. Then, observers performed a 'test trial' where the orientations of the target and distractors were switched, allowing us to assess observer's internal representation of distractor distributions based on changes in response times. In three experiments we observed that observer's search times on test trials depended mainly on the very last learning trial, indicating a strong recency effect. Since temporal integration has been previously observed with this method, we conclude that when the input is unreliable, the visual system relies more on the most recent stimulus. This indicates that the visual system prefers to utilize sensory history when the statistical properties of the environment are relatively stable.

Average faces: How does the averaging process change faces physically and perceptually?

Average faces have been used frequently in face recognition studies, either as a theoretical concept (e.g., face norm) or as a tool to manipulate facial attributes (e.g., modifying identity strength). Nonetheless, how the face averaging process- the creation of average faces using an increasing number of faces -changes the resulting averaged faces and our ability to differentiate between them remains to be elucidated. Here we addressed these questions by combining 3D-face averaging, eye-movement tracking, and the computation of image-based face similarity. Participants judged whether two average faces showed the same person while we systematically increased their average level (i.e., number of faces being averaged). Our results showed, with increasing averaging, both a nonlinear increase of the computational similarity between the resulting average faces and a nonlinear decrease of face discrimination performance. Participants' performance dropped from near-ceiling level when two different faces had been averaged together to chance level when 80 faces were mixed. We also found a nonlinear relationship between face similarity and face discrimination performance, which was fitted nicely with an exponential function. Furthermore, when the comparison task became more challenging, participants performed more fixations onto the faces. Nonetheless, the distribution of fixations across facial features (eyes, nose, mouth, and the center area of a face) remained unchanged. These results not only set new constraints on the theoretical characterization of the average face and its role in establishing face norms but also offer practical guidance for creating approximated face norms to manipulate face identity.

Comparing person and people perception: Multiple group members do not increase stereotype priming

A characteristic feature of daily life is encountering people in groups. Surprisingly, however, at least during the initial stages of processing, research has focused almost exclusively on the construal of single individuals. As such, it remains unclear whether person and people (i.e., group) perception yield comparable or divergent outcomes. Addressing this issue, here we explored a core social-cognitive topic-stereotype activation-by presenting both single and multiple facial primes in a sequential-priming task. In addition, the processes underlying task performance were probed using a drift diffusion model analysis. Based on prior work, it was hypothesised that multiple (vs. single) primes would increase stereotype-based responding. Across two experiments, a consistent pattern of results emerged. First, stereotype priming was insensitive to the number of primes that were presented and occurred only at a short prime-target stimulus onset asynchrony (i.e., 250 ms). Second, priming was underpinned by a bias towards congruent (vs. incongruent) prime-target responses. Collectively these findings advance understanding of the emergence and origin of stereotype priming during person and people perception.

Guided Search 6.0: An updated model of visual search

This paper describes Guided Search 6.0 (GS6), a revised model of visual search. When we encounter a scene, we can see something everywhere. However, we cannot recognize more than a few items at a time. Attention is used to select items so that their features can be "bound" into recognizable objects. Attention is "guided" so that items can be processed in an intelligent order. In GS6, this guidance comes from five sources of preattentive information: (1) top-down and (2) bottom-up feature guidance, (3) prior history (e.g., priming), (4) reward, and (5) scene syntax and semantics. These sources are combined into a spatial "priority map," a dynamic attentional landscape that evolves over the course of search. Selective attention is guided to the most active location in the priority map approximately 20 times per second. Guidance will not be uniform across the visual field. It will favor items near the point of fixation. Three types of functional visual field (FVFs) describe the nature of these foveal biases. There is a resolution FVF, an FVF governing exploratory eye movements, and an FVF governing covert deployments of attention. To be identified as targets or rejected as distractors, items must be compared to target templates held in memory. The binding and recognition of an attended object is modeled as a diffusion process taking > 150 ms/item. Since selection occurs more frequently than that, it follows that multiple items are undergoing recognition at the same time, though asynchronously, making GS6 a hybrid of serial and parallel processes. In GS6, if a target is not found, search terminates when an accumulating quitting signal reaches a threshold. Setting of that threshold is adaptive, allowing feedback about performance to shape subsequent searches. Simulation shows that the combination of asynchronous diffusion and a quitting signal can produce the basic patterns of response time and error data from a range of search experiments.

Emotion matters: Face ensemble perception is affected by emotional states

Ensemble coding is the ability of the visual system to extract a summary statistic from a set of stimuli. For example, observers often spontaneously extract an average face identity from a set of faces. Ensemble coding is known to operate in the frame of a distributed/global attention model. Because both attention and holistic processing are modulated by emotion - where positive emotions broaden the scope of attention and facilitate global processing, whereas negative emotions narrow the scope of attention and promote local processing - the current research explored whether emotional states could affect visual averaging of multiple face identities. Participants completed an ensemble-coding task before and after their emotion was induced via film clips. In the ensemble-coding task, a set of four face identities was shown briefly, followed by a probe face. Participants judged whether the probe face was presented in the preceding set. Evidence for ensemble coding was indexed by responses that treated an average face of the preceding set as a member of that set. The results showed that the tendency to choose this average was modulated by emotional states. Visual averaging increased after seeing positive film clips, but decreased after seeing negative film clips. These results support Frederickson's broaden-and-built theory, and extended its application to ensemble perception.

Human Texture Vision as Multi-Order Spectral Analysis

Texture information plays a critical role in the rapid perception of scenes, objects, and materials. Here, we propose a novel model in which visual texture perception is essentially determined by the 1st-order (2D-luminance) and 2nd-order (4D-energy) spectra. This model is an extension of the dimensionality of the Filter-Rectify-Filter (FRF) model, and it also corresponds to the frequency representation of the Portilla-Simoncelli (PS) statistics. We show that preserving two spectra and randomizing phases of a natural texture image result in a perceptually similar texture, strongly supporting the model. Based on only two single spectral spaces, this model provides a simpler framework to describe and predict texture representations in the primate visual system. The idea of multi-order spectral analysis is consistent with the hierarchical processing principle of the visual cortex, which is approximated by a multi-layer convolutional network.

Procedural Control Versus Resources as Potential Origins of Human Hyper Selectivity

In the current review, we argue that experimental results usually interpreted as evidence for cognitive resource limitations could also reflect functional necessities of human information processing. First, we point out that selective processing of only specific features, objects, or locations at each moment in time allows humans to monitor the success and failure of their own overt actions and covert cognitive procedures. We then proceed to show how certain instances of selectivity are at odds with commonly assumed resource limitations. Next, we discuss examples of seemingly automatic, resource-free processing that challenge the resource view but can be easily understood from the functional perspective of monitoring cognitive procedures. Finally, we suggest that neurophysiological data supporting resource limitations might actually reflect mechanisms of how procedural control is implemented in the brain.

Contribution of a common ability in average and variability judgments

People can make judgments about statistical properties of visual features within groups of objects, such as the average size, size variability, or numerosity of circles. Emerging from recent work is the view that these kinds of visual estimations, collectively dubbed ensemble perception, rely on independent abilities that are specific to the statistical property being estimated (mean, variance, range, numerosity). Here we revisit evidence for the claim that different statistical judgments (i.e., average and variability) for the same object feature are based on independent abilities. We tested a large sample of people, using a pre-registered open-ended sequential design to avoid ending up with weak evidence. We estimated the shared variance in ability across different ensemble judgments, with common constraints for the different tasks. We found that the abilities to judge the average size and the size variability for an array of circles are positively correlated, even after controlling for the ability to discriminate the size of single circles. Our results refute the idea that judgments of average and variability for the same object feature rely on completely independent abilities.

Shared cognitive mechanisms involved in the processing of scene texture and scene shape

Recent research has demonstrated that the parahippocampal place area represents both the shape and texture features of scenes, with the importance of each feature varying according to perceived scene category. Namely, shape features are predominately more diagnostic to the processing of artificial human–made scenes, while shape and texture are equally diagnostic in natural scene processing. However, to date little is known regarding the degree of interactivity or independence observed in the processing of these scene features. Furthermore, manipulating the scope of visual attention (i.e., globally vs. locally) when processing ensembles of multiple objects—stimuli that share a functional neuroanatomical link with scenes—has been shown to affect their cognitive visual representation. It remains unknown whether manipulating the scope of attention impacts scene processing in a similar manner. Using the well-established Garner speeded-classification behavioral paradigm, we investigated the influence of both feature diagnosticity and the scope of visual attention on potential interactivity or independence in the shape and texture processing of artificial human–made scenes. The results revealed asymmetric interference between scene shape and texture processing, with the more diagnostic feature (i.e., shape) interfering with the less diagnostic feature (i.e., texture), but not vice versa. Furthermore, this interference was attenuated and enhanced with more local and global visual processing strategies, respectively. These findings suggest that the scene shape and texture processing are mediated by shared cognitive mechanisms and that, although these representations are governed primarily via feature diagnosticity, they can nevertheless be influenced by the scope of visual attention.

Tracking the relation between gist and item memory over the course of long-term memory consolidation

Our experiences in the world support memories not only of specific episodes but also of the generalities (the 'gist') across related experiences. It remains unclear how these two types of memories evolve and influence one another over time. In two experiments, 173 human participants encoded spatial locations from a distribution and reported both item memory (specific locations) and gist memory (center for the locations) across 1-2 months. Experiment 1 demonstrated that after 1 month, gist memory was preserved relative to item memory, despite a persistent positive correlation between them. Critically, item memories were biased toward the gist over time. Experiment 2 showed that a spatial outlier item changed this relationship and that the extraction of gist is sensitive to the regularities of items. Our results suggest that the gist starts to guide item memories over longer durations as their relative strengths change.

Ensemble coding of average length and average orientation are correlated

Viewers can summarize redundant features in groups of objects into an ensemble percept. There appears to be separate mechanisms underlying ensemble perception of low- and high-level visual features, but it is unclear whether ensemble perception of different low-level features is supported by common mechanisms. Yörük and Boduroglu, in Attention, Perception, & Psychophysics 82 (2020) 852-864, investigated whether length and orientation summarization tap common mechanisms by examining the correlation between errors on length- and orientation-averaging tasks and concluded that because they did not find any correlations, the two features are summarized by different, feature-specific mechanisms. However, their study was conducted with a small sample size and included sources of individual performance variance that may have diminished correlations. We report two studies that tested the correlation between performance in the length- and orientation-averaging tasks, with larger samples and modifications that sought to reduce the sources of variance. Study 1 used ensembles that varied in both feature dimensions and Study 2 used ensembles that only varied in the task-relevant dimension. Both studies showed that errors in length- and orientation-averaging are correlated, suggesting that ensemble perception of these low-level features is supported, at least to some extent, by a common ability.

When two faces are not better than one: Serial limited-capacity processing with redundant-target faces

Many researchers believe that faces-whether presented alone or as part of an ensemble-are processed automatically. According to this idea, (a) the detection of single or multiple faces is resource free and does not require allocation of attention, and (b) visual search for faces is held in parallel. The current study put these hypotheses under direct scrutiny. Participants performed in a redundant target-detection task, responding according to the presence or absence of a face (or faces) on the display. I used a rigorous methodology known as the system factorial technology (SFT). The SFT methodology afforded simultaneous assessment of (a) architecture (serial vs. parallel), (b) stopping rule (exhaustive vs. self-terminating), and (c) capacity (limited, unlimited, or supercapacity). SFT analyses were held at the level of the mean RTs and at the level of the RT distributions. The analyses pointed conclusively to a serial self-terminating architecture with limited capacity. These findings cast serious doubts on the alleged automaticity of face perception.

Replicating and reversing the group attractiveness effect: Relatively unattractive groups are perceived as less attractive than the average attractiveness of their members

The group attractiveness (GA) effect reveals that groups as a whole are perceived as more attractive than the average attractiveness of their members. This effect seems to be driven by selective attention to the most attractive members of a group. We hypothesized that when group members stand out because of their unattractiveness, they would make a group as a whole seem less attractive. In the current study (N = 621), we aimed to replicate the GA-effect as well as reverse it. Indeed, the data replicated a GA effect for relatively attractive groups, while in relatively unattractive groups the group as a whole was perceived as less attractive than the average attractiveness of their members.

Linguistic meanings as cognitive instructions

Natural languages like English connect pronunciations with meanings. Linguistic pronunciations can be described in ways that relate them to our motor system (e.g., to the movement of our lips and tongue). But how do linguistic meanings relate to our nonlinguistic cognitive systems? As a case study, we defend an explicit proposal about the meaning of most by comparing it to the closely related more: whereas more expresses a comparison between two independent subsets, most expresses a subset-superset comparison. Six experiments with adults and children demonstrate that these subtle differences between their meanings influence how participants organize and interrogate their visual world. In otherwise identical situations, changing the word from most to more affects preferences for picture-sentence matching (experiments 1-2), scene creation (experiments 3-4), memory for visual features (experiment 5), and accuracy on speeded truth judgments (experiment 6). These effects support the idea that the meanings of more and most are mental representations that provide detailed instructions to conceptual systems.

A review of interactions between peripheral and foveal vision

Visual processing varies dramatically across the visual field. These differences start in the retina and continue all the way to the visual cortex. Despite these differences in processing, the perceptual experience of humans is remarkably stable and continuous across the visual field. Research in the last decade has shown that processing in peripheral and foveal vision is not independent, but is more directly connected than previously thought. We address three core questions on how peripheral and foveal vision interact, and review recent findings on potentially related phenomena that could provide answers to these questions. First, how is the processing of peripheral and foveal signals related during fixation? Peripheral signals seem to be processed in foveal retinotopic areas to facilitate peripheral object recognition, and foveal information seems to be extrapolated toward the periphery to generate a homogeneous representation of the environment. Second, how are peripheral and foveal signals re-calibrated? Transsaccadic changes in object features lead to a reduction in the discrepancy between peripheral and foveal appearance. Third, how is peripheral and foveal information stitched together across saccades? Peripheral and foveal signals are integrated across saccadic eye movements to average percepts and to reduce uncertainty. Together, these findings illustrate that peripheral and foveal processing are closely connected, mastering the compromise between a large peripheral visual field and high resolution at the fovea.

The virtual loss function in the summary perception of motion and its limited adjustability

Humans can grasp the "average" feature of a visual ensemble quickly and effortlessly. However, it is largely unknown what is the exact form of the summary statistic humans perceive and it is even less known whether this form can be changed by feedback. Here we borrow the concept of loss function to characterize how the summary perception is related to the distribution of feature values in the ensemble, assuming that the summary statistic minimizes a virtual expected loss associated with its deviation from individual feature values. In two experiments, we investigated a random-dot motion estimation task to infer the virtual loss function implicit in ensemble perception and see whether it can be changed by feedback. On each trial, participants reported the average moving direction of an ensemble of moving dots whose distribution of moving directions was skewed. In Experiment 1, where no feedback was available, participants' estimates fell between the mean and the mode of the distribution and were closer to the mean. In particular, the deviation from the mean and toward the mode increased almost linearly with the mode-to-mean distance. The pattern was best modeled by an inverse Gaussian loss function, which punishes large errors less heavily than the quadratic loss function does. In Experiment 2, we tested whether this virtual loss function can be altered by feedback. Two groups of participants either received the mode or the mean as the correct answer. After extensive training up to five days, both groups' estimates moved slightly towards the mode. That is, feedback had no specific influence on participants' virtual loss function. To conclude, the virtual loss function in the summary perception of motion is close to inverse Gaussian, and it can hardly be changed by feedback.

Dual strategies in human confidence judgments

Although confidence is commonly believed to be an essential element in decision-making, it remains unclear what gives rise to one's sense of confidence. Recent Bayesian theories propose that confidence is computed, in part, from the degree of uncertainty in sensory evidence. Alternatively, observers can use physical properties of the stimulus as a heuristic to confidence. In the current study, we developed ideal observer models for either hypothesis and compared their predictions against human data obtained from psychophysical experiments. Participants reported the orientation of a stimulus, and their confidence in this estimate, under varying levels of internal and external noise. As predicted by the Bayesian model, we found a consistent link between confidence and behavioral variability for a given stimulus orientation. Confidence was higher when orientation estimates were more precise, for both internal and external sources of noise. However, we observed the inverse pattern when comparing between stimulus orientations: although observers gave more precise orientation estimates for cardinal orientations (a phenomenon known as the oblique effect), they were more confident about oblique orientations. We show that these results are well explained by a strategy to confidence that is based on the perceived amount of noise in the stimulus. Altogether, our results suggest that confidence is not always computed from the degree of uncertainty in one's perceptual evidence but can instead be based on visual cues that function as simple Heuristics to confidence.

Categorization of Vocal Emotion Cues Depends on Distributions of Input

Learners use the distributional properties of stimuli to identify environmentally relevant categories in a range of perceptual domains, including words, shapes, faces, and colors. We examined whether similar processes may also operate on affective information conveyed through the voice. In Experiment 1, we tested how adults (18–22-year-olds) and children (8–10-year-olds) categorized affective states communicated by vocalizations varying continuously from “calm” to “upset.” We found that the threshold for categorizing both verbal (i.e., spoken word) and nonverbal (i.e., a yell) vocalizations as “upset” depended on the statistical distribution of the stimuli participants encountered. In Experiment 2, we replicated and extended these findings in adults using vocalizations that conveyed multiple negative affect states. These results suggest perceivers’ flexibly and rapidly update their interpretation of affective vocal cues based upon context.

Ensemble perception without attention depends upon attentional control settings

People are able to rapidly extract summary statistical information about common patterns, or ensembles, that may exist in a scene, such as repeated textures or colors. Here we examined the extent to which such an ensemble perception can occur in the absence of focal visual attention using a method that has some advantages over methods previously used to study the issue. In particular, we assessed the extent to which ensembles can be processed without attention by measuring the indirect effect of a to-be-ignored ensemble on judgments of an attended ensemble. The results show that ensembles outside the focus of attention do influence judgments of attended ensembles when the to-be-ignored ensemble contains summary statistics that match a sought-for target category. Thus, an attentional control setting for specific summary statistical information permits the processing of ensembles outside of focal attention, facilitating the rapid perception of visual scenes.

The anisotropic field of ensemble coding

Human observers can accurately estimate statistical summaries from an ensemble of multiple stimuli, including the average size, hue, and direction of motion. The efficiency and speed with which statistical summaries are extracted suggest an automatic mechanism of ensemble coding that operates beyond the capacity limits of attention and memory. However, the extent to which ensemble coding reflects a truly parallel and holistic mode of processing or a non-uniform and biased integration of multiple items is still under debate. In the present work, we used a technique, based on a Spatial Weighted Average Model (SWM), to recover the spatial profile of weights with which individual stimuli contribute to the estimated average during mean size adjustment tasks. In a series of experiments, we derived two-dimensional SWM maps for ensembles presented at different retinal locations, with different degrees of dispersion and under different attentional demands. Our findings revealed strong spatial anisotropies and leftward biases in ensemble coding that were organized in retinotopic reference frames and persisted under attentional manipulations. These results demonstrate an anisotropic spatial contribution to ensemble coding that could be mediated by the differential activation of the two hemispheres during spatial processing and scene encoding.

Proximity model of perceived numerosity

The occupancy model (OM) was proposed to explain how the spatial arrangement of dots in sparse random patterns affects their perceived numerosity. The model's central thesis maintained that each dot seemingly fills or occupies its surrounding area within a fixed radius r_o and the total area collectively occupied by all the dots determines their apparent number. Because the perceptual system is not adapted for the precise estimation of area, it looks likely that the OM is just a convenient computational algorithm that does not necessarily correspond to the processes that actually take place in the perceptual system. As an alternative, the proximity model (PM) was proposed, which instead relies on a binomial function with the probability β characterizing the perceptual salience with which each element can be registered by the perceptual system. It was also assumed that the magnitude of β is proportional to the distance between a dot and its nearest neighbor. A simulation experiment demonstrated that the occupancy area computed according to the OM can almost perfectly be replicated by the mean nearest neighbor distance. It was concluded that proximity between elements is a critical factor in determining their perceived numerosity, but the exact algorithm that is used for the measure of proximities is yet to be established.

Grasping of Real-World Objects Is Not Biased by Ensemble Perception

The visual system is known to extract summary representations of visually similar objects which bias the perception of individual objects toward the ensemble average. Although vision plays a large role in guiding action, less is known about whether ensemble representation is informative for action. Motor behavior is tuned to the veridical dimensions of objects and generally considered resistant to perceptual biases. However, when the relevant grasp dimension is not available or is unconstrained, ensemble perception may be informative to behavior by providing gist information about surrounding objects. In the present study, we examined if summary representations of a surrounding ensemble display influenced grip aperture and orientation when participants reached-to-grasp a central circular target which had an explicit size but importantly no explicit orientation that the visuomotor system could selectively attend to. Maximum grip aperture and grip orientation were not biased by ensemble statistics during grasping, although participants were able to perceive and provide manual estimations of the average size and orientation of the ensemble display. Support vector machine classification of ensemble statistics achieved above-chance classification accuracy when trained on kinematic and electromyography data of the perceptual but not grasping conditions, supporting our univariate findings. These results suggest that even along unconstrained grasping dimensions, visually-guided behaviors toward real-world objects are not biased by ensemble processing.

Structural and contextual priors affect visual search in children with and without autism

Bayesian predictive coding theories of autism spectrum disorder propose that impaired acquisition or a broader shape of prior probability distributions lies at the core of the condition. However, we still know very little about how probability distributions are learned and encoded by children, let alone children with autism. Here, we take advantage of a recently developed distribution learning paradigm to characterize how children with and without autism acquire information about probability distributions. Twenty-four autistic and 25-matched neurotypical children searched for an odd-one-out target among a set of distractor lines with orientations sampled from a Gaussian distribution repeated across multiple trials to allow for learning of the parameters (mean and variance) of the distribution. We could measure the width (variance) of the participant's encoded distribution by introducing a target-distractor role-reversal while varying the similarity between target and previous distractor mean. Both groups performed similarly on the visual search task and learned the distractor distribution to a similar extent. However, the variance learned was much broader than the one presented, consistent with less informative priors in children irrespective of autism diagnosis. These findings have important implications for Bayesian accounts of perception throughout development, and Bayesian accounts of autism specifically. LAY SUMMARY: Recent theories about the underlying cognitive mechanisms of autism propose that the way autistic individuals estimate variability or uncertainty in their perceptual environment may differ from how typical individuals do so. Children had to search an oddly tilted line in a set of lines pointing in different directions, and based on their response times we examined how they learned about the variability in a set of objects. We found that autistic children learn variability as well as typical children, but both groups learn with less precision than typical adults do on the same task.

Enumerating the forest before the trees: The time courses of estimation-based and individuation-based numerical processing

Ensemble perception refers to the ability to report attributes of a group of objects, rather than focusing on only one or a few individuals. An everyday example of ensemble perception is the ability to estimate the numerosity of a large number of items. The time course of ensemble processing, including that of numerical estimation, remains a matter of debate, with some studies arguing for rapid, "preattentive" processing and other studies suggesting that ensemble perception improves with longer presentation durations. We used a forward-simultaneous masking procedure that effectively controls stimulus durations to directly measure the temporal dynamics of ensemble estimation and compared it with more precise enumeration of individual objects. Our main finding was that object individuation within the subitizing range (one to four items) took about 100-150 ms to reach its typical capacity limits, whereas estimation (six or more items) showed a temporal resolution of 50 ms or less. Estimation accuracy did not improve over time. Instead, there was an increasing tendency, with longer effective durations, to underestimate the number of targets for larger set sizes (11-35 items). Overall, the time course of enumeration for one or a few single items was dramatically different from that of estimating numerosity of six or more items. These results are consistent with the idea that the temporal resolution of ensemble processing may be as rapid as, or even faster than, individuation of individual items, and support a basic distinction between the mechanisms underlying exact enumeration of small sets (one to four items) from estimation.

Variability leads to overestimation of mean summaries

Research on ensemble perception has shown that people can extract both mean and variance information, but much less is understand how these two different types of summaries interact with one another. Some research has argued that people are more erroneous in extracting the mean of displays that have greater variability. In all three experiments, we manipulated the variability in the displays. Participants reported the mean size of a set of circles (Experiment 1) and mean length of horizontally placed (Experiment 2a) and randomly oriented lines (Experiment 2b). In all experiments, we found that mean size estimations were more erroneous for higher than smaller variance displays. More critically, there was a tendency to overestimate the mean, driven by variance in both task-relevant and task-irrelevant features. We discuss these findings in relation to limitations in concurrent summarization ability and outlier discounting in ensemble perception.

The dissociable influence of social context on judgements of facial attractiveness and trustworthiness

The cheerleader effect occurs when the same face is rated to be more attractive when it is seen in a group compared to when seen alone. We investigated whether this phenomenon also occurs for trustworthiness judgements, and examined how these effects are influenced by the characteristics of the individual being evaluated and those of the group they are seen in. Across three experiments, we reliably replicated the cheerleader effect. Most faces became more attractive in a group. Yet, the size of the cheerleader effect that each face experienced was not related to its own attractiveness, nor to the attractiveness of the group or the group's digitally averaged face. We discuss the implications of our findings for the hierarchical encoding and contrast mechanisms that have previously been used to explain the cheerleader effect. Surprisingly, judgements of facial trustworthiness did not experience a 'cheerleader effect'. Instead, we found that untrustworthy faces became significantly more trustworthy in all groups, while there was no change for faces that were already trustworthy alone. Taken together, our results demonstrate that social context can have a dissociable influence on our first impressions, depending on the trait being evaluated.

Contributions of ensemble perception to outlier representation precision

It is known that the visual system can efficiently extract mean and variance information, facilitating the detection of outliers. However, no research to date has directly investigated whether ensemble perception mechanisms contribute to outlier representation precision. We specifically were interested in how the distinctiveness of outliers impacts their precision. Across two experiments, we compared how accurately viewers represented the orientation of spatial outliers that varied in distinctiveness and found that increased outlier distinctiveness resulted in greater precision. Based on comparisons of our data to simulations reflecting particular selective strategies, we eliminated the possibility that participants were selectively processing the outlier, at the expense of the ensemble. Thus, we argued that participants separately represented distinct outliers along with ensemble summaries of the remaining items in a display. We also found that outlier distinctiveness moderated the precision of how the remaining items were summarized. We discuss these findings in relation to computational capacity and constraints of ensemble perception mechanisms.

Perceived variability reflects the reliability of individual items

When confronted with many visual items, people can compute their variability accurately and rapidly, which facilitates efficient information processing and optimal decision making. However, how the visual system computes variability is still unclear. To investigate this, we implemented situations whereby estimates of variability based on several possible variability measures (e.g., range, standard deviation, and weighted standard deviation) could be differentiated, and then examined which best accounted for human variability perception. In three psychophysical experiments, participants watched two arrays of items with various orientations and judged which had more variable orientations. Results showed that perceived variability was most consistent with the weighted standard deviation based on the reliability of individual items. Specifically, participants gave less consideration to deviant orientations that were likely to be outliers, and greater consideration to salient orientations that were likely to be encoded precisely. This reliability-based weighted standard deviation suggests an efficient and flexible way of representing visual variability.

A method for detection of inattentional feature blindness

In ensemble displays, two principal factors determine the precision with which the mean value of some perceptual attribute, such as size and orientation, can be discriminated: inefficiency and representational noise of each element. Inefficiency is mainly caused by biased inference, or by inattentional (feature) blindness (i.e., some elements or their features are not processed). Here, we define inattentional feature blindness as an inability to perceive the value(s) of certain feature(s) of an object while the presence of the object itself may be registered. Separation of the effects of inattentional (feature) blindness and perceptual noise has escaped traditional analytic methods because of their trade-off effects on the slope of the psychometric discrimination function. Here, we propose a method that can separate the effects of inattentional feature blindness from that of the representational noise. The basic idea is to display a set of elements from which only one contains information relevant for solving the task, while all other elements are "dummies" carrying no useful information because they do not differ from the reference. If the single informative element goes unprocessed, the correct answer can only be given by a random guess. The guess rate can be modeled similarly to the lapse rate, traditionally represented by λ. As an illustration, we present evidence that the presence versus lack of inattentional feature blindness in orientation pooling depends on the feature types present in the display.

Dissociating implicit and explicit ensemble representations reveals the limits of visual perception and the richness of behavior

Our senses provide us with a rich experience of a detailed visual world, yet the empirical results seem to suggest severe limitations on our ability to perceive and remember. In recent attempts to reconcile the contradiction between what is experienced and what can be reported, it has been argued that the visual world is condensed to a set of summary statistics, explaining both the rich experience and the sparse reports. Here, we show that explicit reports of summary statistics underestimate the richness of ensemble perception. Our observers searched for an odd-one-out target among heterogeneous distractors and their representation of distractor characteristics was tested explicitly or implicitly. Observers could explicitly distinguish distractor sets with different mean and variance, but not differently-shaped probability distributions. In contrast, the implicit assessment revealed that the visual system encodes the mean, the variance, and even the shape of feature distributions. Furthermore, explicit measures had common noise sources that distinguished them from implicit measures. This suggests that explicit judgments of stimulus ensembles underestimate the richness of visual representations. We conclude that feature distributions are encoded in rich detail and can guide behavior implicitly, even when the information available for explicit summary judgments is coarse and limited.

Size-distance rescaling in the ensemble representation of range: Study with binocular and monocular cues

According to numerous studies observers can rapidly and precisely evaluate mean or range of the set. Recent studies have shown that the mean size estimated based on sizes of objects rescaled to their distances (Tiurina & Utochkin, 2019). In the current study, we directly tested this rescaling mechanism on the perception of range using binocular and monocular cues. In Experiment 1, a sample set of circles with different angular sizes and in different apparent distances were stereoscopically presented. Participants had to adjust the range of the test set to match the range of the sample set. The main manipulation was the size-distance correlation for sample and test sets: in negative size-distance correlation, the apparent range had to decrease, while in positive correlation - increase. We found the highest underestimation in the condition with the negative sample correlation and positive test correlation, which could be explained only if ensemble summary statistics were estimated after the item's rescaling. In Experiment 2, we used Ponzo-like illusion and spatial positions as a depth cue. Sets were presented with positive, negative or without size-distance correlation on a grey background or the background with Ponzo-like illusion. We found that the range was underestimated in negative correlation and overestimated in positive correlation. Thus, items of ensemble could be automatically rescaled according to their distance, based on both binocular and monocular cues, and ensemble summary statistics estimation is based on perceived sizes.

A Design Space of Vision Science Methods for Visualization Research

A growing number of efforts aim to understand what people see when using a visualization. These efforts provide scientific grounding to complement design intuitions, leading to more effective visualization practice. However, published visualization research currently reflects a limited set of available methods for understanding how people process visualized data. Alternative methods from vision science offer a rich suite of tools for understanding visualizations, but no curated collection of these methods exists in either perception or visualization research. We introduce a design space of experimental methods for empirically investigating the perceptual processes involved with viewing data visualizations to ultimately inform visualization design guidelines. This paper provides a shared lexicon for facilitating experimental visualization research. We discuss popular experimental paradigms, adjustment types, response types, and dependent measures used in vision science research, rooting each in visualization examples. We then discuss the advantages and limitations of each technique. Researchers can use this design space to create innovative studies and progress scientific understanding of design choices and evaluations in visualization. We highlight a history of collaborative success between visualization and vision science research and advocate for a deeper relationship between the two fields that can elaborate on and extend the methodological design space for understanding visualization and vision.

Variance-dependent neural activity in an unvoluntary averaging task

Ensemble statistics of a visual scene can be estimated to provide a gist of the scene without detailed analysis of all individual items. The simplest and most widely studied ensemble statistic is mean estimation, which requires averaging an ensemble of elements. Averaging is useful to estimate the mean of an ensemble and discard the variance. The source of variance can be external (i.e., variance across the physical elements) or internal (i.e., imprecisions in the estimates of the elements by the visual system). The equivalent noise paradigm is often used to measure the impact of the internal variance (i.e., the equivalent input noise). This paradigm relies on the assumption that the averaging process is equally effective independently of the main source of variance, internal or external, so any difference between the processing when the main source of variance is internal or external must be assumed not to affect the averaging efficiency. The current fMRI study compared the neural activity when the main variance is caused by the stimulus (i.e., high variance) and when it is caused by imprecisions in the estimates of the elements by the visual system (i.e., low variance). The results showed that the right superior frontal and left middle frontal gyri can be significantly more activated when the variance in the orientation of the Gabors was high than when it was low. Consequently, the use of the equivalent noise paradigm requires the assumption that such additional neural activity in high variance does not affect the averaging efficiency.

Global and local interference effects in ensemble encoding are best explained by interactions between summary representations of the mean and the range

Through ensemble encoding, the visual system compresses redundant statistical properties from multiple items into a single summary metric (e.g., average size). Numerous studies have shown that global summary information is extracted quickly, does not require access to single-item representations, and often interferes with reports of single items from the set. Yet a thorough understanding of ensemble processing would benefit from a more extensive investigation at the local level. Thus, the purpose of this study was to provide a more critical inspection of global-local processing in ensemble perception. Taking inspiration from Navon (Cognitive Psychology, 9(3), 353-383, 1977), we employed a novel paradigm that independently manipulates the degree of interference at the global (mean) or local (single item) level of the ensemble. Initial results were consistent with reciprocal interference between global and local ensemble processing. However, further testing revealed that local interference effects were better explained by interference from another summary statistic, the range of the set. Furthermore, participants were unable to disambiguate single items from the ensemble display from other items that were within the ensemble range but, critically, were not actually present in the ensemble. Thus, it appears that local item values are likely inferred based on their relationship to higher-order summary statistics such as the range and the mean. These results conflict with claims that local information is captured alongside global information in summary representations. In such studies, successful identification of set members was not compared with misidentification of items within the range, but which were nevertheless not presented within the set.

Ensemble perception during multiple-object tracking

Multiple-object tracking studies consistently reveal attentive tracking limits of approximately three to five items. How do factors such as visual grouping and ensemble perception impact these capacity limits? Which heuristics lead to the perception of multiple objects as a group? This work investigates the role of grouping on multiple-object tracking ability, and more specifically, in identifying the heuristics that lead to the formation and perception of ensembles within dynamic contexts. First, we show that group tracking limits are approximately four groups of objects and are independent of the number of items that compose the groups. Further, we show that group tracking performance declines as inter-object spacing increases. We also demonstrate the role of group rigidity in tracking performance in that disruptions to common fate negatively impact ensemble tracking ability. The findings from this work contribute to our overall understanding of the perception of dynamic groups of objects. They characterize the properties that determine the formation and perception of dynamic object ensembles. In addition, they inform development and design decisions considering cognitive limitations involving tracking groups of objects.

Redundancy masking: The loss of repeated items in crowded peripheral vision

Crowding is the deterioration of target identification in the presence of neighboring objects. Recent studies using appearance-based methods showed that the perceived number of target elements is often diminished in crowding. Here we introduce a related type of diminishment in repeating patterns (sets of parallel lines), which we term “redundancy masking.” In four experiments, observers were presented with arrays of small numbers of lines centered at 10° eccentricity. The task was to indicate the number of lines. In Experiment 1, spatial characteristics of redundancy masking were examined by varying the inter-line spacing. We found that redundancy masking decreased with increasing inter-line spacing and ceased at spacings of approximately 0.25 times the eccentricity. In Experiment 2, we assessed whether the strength of redundancy masking differed between radial and tangential arrangements of elements as it does in crowding. Redundancy masking was strong with radially arranged lines (horizontally arranged vertical lines), and absent with tangentially arranged lines (vertically arranged horizontal lines). In Experiment 3, we investigated whether target size (line width and length) modulated redundancy masking. There was an effect of width: Thinner lines yielded stronger redundancy masking. We did not find any differences between the tested line lengths. In Experiment 4, we varied the regularity of the line arrays by vertically or horizontally jittering the positions of the lines. Redundancy masking was strongest with regular spacings and weakened with decreasing regularity. Our experiments show under which conditions whole items are lost in crowded displays, and how this redundancy masking resembles—and partly diverges from—crowded identification. We suggest that redundancy masking is a contributor to the deterioration of performance in crowded displays with redundant patterns.

A direct comparison of central tendency recall and temporal integration in the successive field iconic memory task

The ensemble coding literature suggests the existence of a fast, automatic formation of some ensemble codes. Can statistical representations, such as memory for the central tendency along a particular visual feature dimension, be extracted from information held in the sensory register? Furthermore, can knowledge of early, iconic memory processes be used to determine how central tendency is extracted? We focused on the potential role of visible persistence mechanisms that support temporal integration. We tested whether mean orientation could be accurately recalled from brief visual displays using the successive field task. On separate blocks of trials, participants were asked to report the location of a split element (requiring differentiation of frames), a missing element (requiring integration across frames), and the average orientation of elements pooled across both frames (central tendency recall). Results replicate the expected tradeoff between differentiation and integration performance across inter-frame interval (IFI). In contrast, precision of mean estimates was high and invariant across IFIs. A manipulation of within-frame distributional similarity coupled with simulations using 12 models supported 2-item subsampling. The results argue against the “strategic” interpretation of subsampling since 2-item readout was predicted by information theoretic estimates of STM encoding rate: the 2 items were not from a superset in STM. Most crucially, the results argue against the various early “preattentive/parallel/global pooling” accounts and instead suggest that non-selective readout of information from iconic memory supplies a relatively small amount of item information to STM, and it is only at this point that the computation of ensemble averages begins.

Set size and ensemble perception of numerical value

There is a growing body of research on ensemble perception, or our ability to form ensemble representations based on perceptual features for stimuli of varying levels of complexity, and more recently, on ensemble cognition, which refers to our ability to perceive higher-level properties of stimuli such as facial attractiveness or gaze direction. Less is known about our ability to form ensemble representations based on more abstract properties such as the semantic meaning associated with items in a scene. Previous work examining whether the meaning associated with digits can be incorporated into summary statistical representations suggests that numerical information from digit ensembles can be extracted rapidly, and likely using a parallel processing mechanism. Here, we further investigate whether participants can accurately generate summary representations of numerical value from digit sets and explore the effect of set size on their ability to do so, by comparing psychometric functions based on a numerical averaging task in which set size varied. Steeper slopes for ten- and seven-item compared to five-item digit sets provide evidence that displays with more digits yield more reliable discrimination between larger and smaller numerical averages. Additionally, consistent with previous reports, we observed a response bias such that participants were more likely to report that the numerical average was "greater than 5" for larger compared to smaller sets. Overall, our results contribute to evidence that ensemble representations for semantic attributes may be carried out via similar mechanisms as those reported for perceptual features.

Synergy between research on ensemble perception, data visualization, and statistics education: A tutorial review

In the age of big data, we are constantly inventing new data visualizations to consolidate massive amounts of numerical information into smaller and more digestible visual formats. These data visualizations use various visual features to convey quantitative information, such as spatial position in scatter plots, color saturation in heat maps, and area in dot maps. These data visualizations are typically composed of ensembles, or groups of related objects, that together convey information about a data set. Ensemble perception, or one's ability to perceive summary statistics from an ensemble, such as the mean, has been used as a foundation for understanding and explaining the effectiveness of certain data visualizations. However, research in data visualization has revealed some perceptual biases and conceptual difficulties people face when trying to utilize the information in these graphs. In this tutorial review, we will provide a broad overview of research conducted in ensemble perception, discuss how principles of ensemble encoding have been applied to the research in data visualization, and showcase the barriers graphs can pose to learning statistical concepts, using histograms as a specific example. The goal of this tutorial review is to highlight possible connections between three areas of research-ensemble perception, data visualization, and statistics education-and to encourage research in the practical applications of ensemble perception in solving real-world problems in statistics education.

Somatosensory evoked potentials that index lateral inhibition are modulated according to the mode of perceptual processing: comparing or combining multi-digit tactile motion

Many perceptual studies focus on the brain's capacity to discriminate between stimuli. However, our normal experience of the world also involves integrating multiple stimuli into a single perceptual event. Neural mechanisms such as lateral inhibition are believed to enhance local differences between sensory inputs from nearby regions of the receptor surface. However, this mechanism would seem dysfunctional when sensory inputs need to be combined rather than contrasted. Here, we investigated whether the brain can strategically regulate the strength of suppressive interactions that underlie lateral inhibition between finger representations in human somatosensory processing. To do this, we compared sensory processing between conditions that required either comparing or combining information. We delivered two simultaneous tactile motion trajectories to index and middle fingertips of the right hand. Participants had to either compare the directions of the two stimuli, or to combine them to form their average direction. To reveal preparatory tuning of somatosensory cortex, we used an established event-related potential design to measure the interaction between cortical representations evoked by digital nerve shocks immediately before each tactile stimulus. Consistent with previous studies, we found a clear suppression between cortical activations when participants were instructed to compare the tactile motion directions. Importantly, this suppression was significantly reduced when participants had to combine the same stimuli. These findings suggest that the brain can strategically switch between a comparative and a combinative mode of somatosensory processing, according to the perceptual goal, by preparatorily adjusting the strength of a process akin to lateral inhibition.