Parallel averaging of size is possible but range-limited: a reply to Marchant, Simons, and De Fockert

Similarity in feature space dictates the efficiency of attentional selection during ensemble processing

Humans can rapidly and accurately extract statistical information about features of the visual environment, an ability referred to as ensemble perception. However, little is known about how ensemble estimates are affected when task-irrelevant and distracting feature information is present. Here, we tested how effectively feature-based attention-when tuned to a specific color-can select a single item set out of two intermixed ensembles of colored lines. Participants were instructed to report the average orientation of a target-colored item set, while ignoring a second differently colored set. To assess how representational overlap between the two sets impacts color-based selection, we systematically varied the orientation similarity between the relevant and irrelevant items. Our results showed that participants' orientation reports were reliably biased towards the irrelevant items, but interestingly, these biases were only observed when the item sets overlapped in orientation space. In a second experiment, using a visual mask to disrupt access to color information at different time points, we found that these biases were stronger when less time was available to process the stimuli. Together, these results suggest that ensemble representations are rapidly formed based on all available information in the relevant feature dimension, regardless of task relevance, and that selective attention weights and separates these ensemble representations at a relatively later processing stage. This selection appears highly effective when the underlying population activity generated by the two sets is separable along the to-be-estimated feature dimension, but is dampened when relevant and irrelevant ensemble representations overlap in feature space.

Mean orientation discrimination based on proximal stimuli

Ensemble perception refers to the ability to accurately and rapidly perceive summary statistical representations of specific features from a group of similar objects. However, the specific type of representation involved in this perception within a three-dimensional (3-D) environment remains unclear. In the context of perspective viewing with stereopsis, distal stimuli can be projected onto the retina as different forms of proximal stimuli based on their distances, despite sharing similar properties, such as object size and spatial frequency. This study aimed to investigate the effects of distal and proximal stimuli on the perception of summary statistical information related to orientation. In our experiment, we presented multiple Gabor patches in a stereoscopic environment, allowing us to measure the discrimination threshold of the mean orientation. The object size and spatial frequency were fixed for all patches regardless of depth. However, the physical angular size and absolute spatial frequency covaried with the depth. The results revealed the threshold elevation with depth expansion, especially when the patches formed two clusters at near and far distances, leading to large variations in their retinotopic representations. This finding indicates a minor contribution of similarity of the distal stimuli. Subsequent experiments demonstrated that the variability in physical angular size of the patches significantly influenced the threshold elevation in contrast to that of binocular disparity and absolute spatial frequency. These findings highlight the critical role of physical angular size variability in perceiving mean orientations within the 3-D space.

What is the basis of ensemble subset selection?

The visual system can rapidly calculate the ensemble statistics of a set of objects; for example, people can easily estimate an average size of apples on a tree. To accomplish this, it is not always useful to summarize all the visual information. If there are various types of objects, the visual system should select a relevant subset: only apples, not leaves and branches. Here, we ask what kind of visual information makes a "good" ensemble that can be selectively attended to provide an accurate summary estimate. We tested three candidate representations: basic features, preattentive object files, and full-fledged bound objects. In four experiments, we presented a target and several distractors' sets of differently colored objects. We found that conditions where a target ensemble had at least one unique color (basic feature) provided ensemble averaging performance comparable to the baseline displays without distractors. When the target subset was defined as a conjunction of two colors or color-shape partly shared with distractors (so that they could be differentiated only as preattentive object files), subset averaging was also possible but less accurate than in the baseline and feature conditions. Finally, performance was very poor when the target subset was defined by an exact feature relationship, such as in the spatial conjunction of two colors (spatially bound object). Overall, these results suggest that distinguishable features and, to a lesser degree, preattentive object files can serve as the representational basis of ensemble selection, while bound objects cannot.

The functional role of spatial anisotropies in ensemble perception

BACKGROUND

The human brain can rapidly represent sets of similar stimuli by their ensemble summary statistics, like the average orientation or size. Classic models assume that ensemble statistics are computed by integrating all elements with equal weight. Challenging this view, here, we show that ensemble statistics are estimated by combining parafoveal and foveal statistics in proportion to their reliability. In a series of experiments, observers reproduced the average orientation of an ensemble of stimuli under varying levels of visual uncertainty.

RESULTS

Ensemble statistics were affected by multiple spatial biases, in particular, a strong and persistent bias towards the center of the visual field. This bias, evident in the majority of subjects and in all experiments, scaled with uncertainty: the higher the uncertainty in the ensemble statistics, the larger the bias towards the element shown at the fovea.

CONCLUSION

Our findings indicate that ensemble perception cannot be explained by simple uniform pooling. The visual system weights information anisotropically from both the parafovea and the fovea, taking the intrinsic spatial anisotropies of vision into account to compensate for visual uncertainty.

A signal-detection account of item-based and ensemble-based visual change detection: A reply to Harrison, McMaster, and Bays

Growing empirical evidence shows that ensemble information (e.g., the average feature or feature variance of a set of objects) affects visual working memory for individual items. Recently, Harrison, McMaster, and Bays (2021) used a change detection task to test whether observers explicitly rely on ensemble representations to improve their memory for individual objects. They found that sensitivity to simultaneous changes in all memorized items (which also globally changed set summary statistics) rarely exceeded a level predicted by the so-called optimal summation model within the signal-detection framework. This model implies simple integration of evidence for change from all individual items and no additional evidence coming from ensemble. Here, we argue that performance at the level of optimal summation does not rule out the use of ensemble information. First, in two experiments, we show that, even if evidence from only one item is available at test, the statistics of the whole memory set affect performance. Second, we argue that optimal summation itself can be conceptually interpreted as one of the strategies of holistic, ensemble-based decision. We also redefine the reference level for the item-based strategy as the so-called "minimum rule," which predicts performance far below the optimum. We found that that both our and Harrison et al. (2021)'s observers consistently outperformed this level. We conclude that observers can rely on ensemble information when performing visual change detection. Overall, our work clarifies and refines the use of signal-detection analysis in measuring and modeling working memory.

Limited memory for ensemble statistics in visual change detection

Accounts of working memory based on independent item representations may overlook a possible contribution of ensemble statistics, higher-order regularities of a scene such as the mean or variance of a visual attribute. Here we used change detection tasks to investigate the hypothesis that observers store ensemble statistics in working memory and use them to detect changes in the visual environment. We controlled changes to the ensemble mean or variance between memory and test displays across six experiments. We made specific predictions of observers' sensitivity using an optimal summation model that integrates evidence across separate items but does not detect changes in ensemble statistics. We found strong evidence that observers outperformed this model, but only when task difficulty was high, and only for changes in stimulus variance. Under these conditions, we estimated that the variance of items contributed to change detection sensitivity more strongly than any individual item in this case. In contrast, however, we found strong evidence against the hypothesis that the average feature value is stored in working memory: when the mean of memoranda changed, sensitivity did not differ from the optimal summation model, which was blind to the ensemble mean, in five out of six experiments. Our results reveal that change detection is primarily limited by uncertainty in the memory of individual features, but that memory for the variance of items can facilitate detection under a limited set of conditions that involve relatively high working memory demands.

An explicit investigation of the roles that feature distributions play in rapid visual categorization

Ensemble representations are often described as efficient tools when summarizing features of multiple similar objects as a group. However, it can sometimes be more useful not to compute a single summary description for all of the objects if they are substantially different, for example when they belong to entirely different categories. It was proposed that the visual system can efficiently use the distributional information of ensembles to decide whether simultaneously displayed items belong to single or several different categories. Here we directly tested how the feature distribution of items in a visual array affects an ability to discriminate individual items (Experiment 1) and sets (Experiments 2-3) when participants were instructed explicitly to categorize individual objects based on the median of size distribution. We varied the width (narrow or fat) as well as the shape (smooth or two-peaked) of distributions in order to manipulate the ease of ensemble extraction from the items. We found that observers unintentionally relied on the grand mean as a natural categorical boundary and that their categorization accuracy increased as a function of the size differences among individual items and a function of their separation from the grand mean. For ensembles drawn from two-peaked size distributions, participants showed better categorization performance. They were more accurate at judging within-category ensemble properties in other dimensions (centroid and orientation) and less biased by superset statistics. This finding corroborates the idea that the two-peaked feature distributions support the "segmentability" of spatially intermixed sets of objects. Our results emphasize important roles of ensemble statistics (mean, range, distribution shape) in explicit visual categorization.

Variability leads to overestimation of mean summaries

Research on ensemble perception has shown that people can extract both mean and variance information, but much less is understand how these two different types of summaries interact with one another. Some research has argued that people are more erroneous in extracting the mean of displays that have greater variability. In all three experiments, we manipulated the variability in the displays. Participants reported the mean size of a set of circles (Experiment 1) and mean length of horizontally placed (Experiment 2a) and randomly oriented lines (Experiment 2b). In all experiments, we found that mean size estimations were more erroneous for higher than smaller variance displays. More critically, there was a tendency to overestimate the mean, driven by variance in both task-relevant and task-irrelevant features. We discuss these findings in relation to limitations in concurrent summarization ability and outlier discounting in ensemble perception.

Contributions of ensemble perception to outlier representation precision

It is known that the visual system can efficiently extract mean and variance information, facilitating the detection of outliers. However, no research to date has directly investigated whether ensemble perception mechanisms contribute to outlier representation precision. We specifically were interested in how the distinctiveness of outliers impacts their precision. Across two experiments, we compared how accurately viewers represented the orientation of spatial outliers that varied in distinctiveness and found that increased outlier distinctiveness resulted in greater precision. Based on comparisons of our data to simulations reflecting particular selective strategies, we eliminated the possibility that participants were selectively processing the outlier, at the expense of the ensemble. Thus, we argued that participants separately represented distinct outliers along with ensemble summaries of the remaining items in a display. We also found that outlier distinctiveness moderated the precision of how the remaining items were summarized. We discuss these findings in relation to computational capacity and constraints of ensemble perception mechanisms.

Perceived variability reflects the reliability of individual items

When confronted with many visual items, people can compute their variability accurately and rapidly, which facilitates efficient information processing and optimal decision making. However, how the visual system computes variability is still unclear. To investigate this, we implemented situations whereby estimates of variability based on several possible variability measures (e.g., range, standard deviation, and weighted standard deviation) could be differentiated, and then examined which best accounted for human variability perception. In three psychophysical experiments, participants watched two arrays of items with various orientations and judged which had more variable orientations. Results showed that perceived variability was most consistent with the weighted standard deviation based on the reliability of individual items. Specifically, participants gave less consideration to deviant orientations that were likely to be outliers, and greater consideration to salient orientations that were likely to be encoded precisely. This reliability-based weighted standard deviation suggests an efficient and flexible way of representing visual variability.

Differential neurodynamics and connectivity in the dorsal and ventral visual pathways during perception of emotional crowds and individuals: a MEG study

Reading the prevailing emotion of groups of people ("crowd emotion") is critical to understanding their overall intention and disposition. It alerts us to potential dangers, such as angry mobs or panicked crowds, giving us time to escape. A critical aspect of processing crowd emotion is that it must occur rapidly, because delays often are costly. Although knowing the timing of neural events is crucial for understanding how the brain guides behaviors using coherent signals from a glimpse of multiple faces, this information is currently lacking in the literature on face ensemble coding. Therefore, we used magnetoencephalography to examine the neurodynamics in the dorsal and ventral visual streams and the periamygdaloid cortex to compare perception of groups of faces versus individual faces. Forty-six participants compared two groups of four faces or two individual faces with varying emotional expressions and chose which group or individual they would avoid. We found that the dorsal stream was activated as early as 68 msec after the onset of stimuli containing groups of faces. In contrast, the ventral stream was activated later and predominantly for individual face stimuli. The latencies of the dorsal stream activation peaks correlated with participants' response times for facial crowds. We also found enhanced connectivity earlier between the periamygdaloid cortex and the dorsal stream regions for crowd emotion perception. Our findings suggest that ensemble coding of facial crowds proceeds rapidly and in parallel by engaging the dorsal stream to mediate adaptive social behaviors, via a distinct route from single face perception.

Size-distance rescaling in the ensemble representation of range: Study with binocular and monocular cues

According to numerous studies observers can rapidly and precisely evaluate mean or range of the set. Recent studies have shown that the mean size estimated based on sizes of objects rescaled to their distances (Tiurina & Utochkin, 2019). In the current study, we directly tested this rescaling mechanism on the perception of range using binocular and monocular cues. In Experiment 1, a sample set of circles with different angular sizes and in different apparent distances were stereoscopically presented. Participants had to adjust the range of the test set to match the range of the sample set. The main manipulation was the size-distance correlation for sample and test sets: in negative size-distance correlation, the apparent range had to decrease, while in positive correlation - increase. We found the highest underestimation in the condition with the negative sample correlation and positive test correlation, which could be explained only if ensemble summary statistics were estimated after the item's rescaling. In Experiment 2, we used Ponzo-like illusion and spatial positions as a depth cue. Sets were presented with positive, negative or without size-distance correlation on a grey background or the background with Ponzo-like illusion. We found that the range was underestimated in negative correlation and overestimated in positive correlation. Thus, items of ensemble could be automatically rescaled according to their distance, based on both binocular and monocular cues, and ensemble summary statistics estimation is based on perceived sizes.

Global and local interference effects in ensemble encoding are best explained by interactions between summary representations of the mean and the range

Through ensemble encoding, the visual system compresses redundant statistical properties from multiple items into a single summary metric (e.g., average size). Numerous studies have shown that global summary information is extracted quickly, does not require access to single-item representations, and often interferes with reports of single items from the set. Yet a thorough understanding of ensemble processing would benefit from a more extensive investigation at the local level. Thus, the purpose of this study was to provide a more critical inspection of global-local processing in ensemble perception. Taking inspiration from Navon (Cognitive Psychology, 9(3), 353-383, 1977), we employed a novel paradigm that independently manipulates the degree of interference at the global (mean) or local (single item) level of the ensemble. Initial results were consistent with reciprocal interference between global and local ensemble processing. However, further testing revealed that local interference effects were better explained by interference from another summary statistic, the range of the set. Furthermore, participants were unable to disambiguate single items from the ensemble display from other items that were within the ensemble range but, critically, were not actually present in the ensemble. Thus, it appears that local item values are likely inferred based on their relationship to higher-order summary statistics such as the range and the mean. These results conflict with claims that local information is captured alongside global information in summary representations. In such studies, successful identification of set members was not compared with misidentification of items within the range, but which were nevertheless not presented within the set.

Set size and ensemble perception of numerical value

There is a growing body of research on ensemble perception, or our ability to form ensemble representations based on perceptual features for stimuli of varying levels of complexity, and more recently, on ensemble cognition, which refers to our ability to perceive higher-level properties of stimuli such as facial attractiveness or gaze direction. Less is known about our ability to form ensemble representations based on more abstract properties such as the semantic meaning associated with items in a scene. Previous work examining whether the meaning associated with digits can be incorporated into summary statistical representations suggests that numerical information from digit ensembles can be extracted rapidly, and likely using a parallel processing mechanism. Here, we further investigate whether participants can accurately generate summary representations of numerical value from digit sets and explore the effect of set size on their ability to do so, by comparing psychometric functions based on a numerical averaging task in which set size varied. Steeper slopes for ten- and seven-item compared to five-item digit sets provide evidence that displays with more digits yield more reliable discrimination between larger and smaller numerical averages. Additionally, consistent with previous reports, we observed a response bias such that participants were more likely to report that the numerical average was "greater than 5" for larger compared to smaller sets. Overall, our results contribute to evidence that ensemble representations for semantic attributes may be carried out via similar mechanisms as those reported for perceptual features.

Task-driven and flexible mean judgment for heterogeneous luminance ensembles

Spatial averaging of luminances over a variegated region has been assumed in visual processes such as light adaptation, texture segmentation, and lightness scaling. Despite the importance of these processes, how mean brightness can be computed remains largely unknown. We investigated how accurately and precisely mean brightness can be compared for two briefly presented heterogeneous luminance arrays composed of different numbers of disks. The results demonstrated that mean brightness judgments can be made in a task-dependent and flexible fashion. Mean brightness judgments measured via the point of subjective equality (PSE) exhibited a consistent bias, suggesting that observers relied strongly on a subset of the disks (e.g., the highest- or lowest-luminance disks) in making their judgments. Moreover, the direction of the bias flexibly changed with the task requirements, even when the stimuli were completely the same. When asked to choose the brighter array, observers relied more on the highest-luminance disks. However, when asked to choose the darker array, observers relied more on the lowest-luminance disks. In contrast, when the task was the same, observers' judgments were almost immune to substantial changes in apparent contrast caused by changing the background luminance. Despite the bias in PSE, the mean brightness judgments were precise. The just-noticeable differences measured for multiple disks were similar to or even smaller than those for single disks, which suggested a benefit of averaging. These findings implicated flexible weighted averaging; that is, mean brightness can be judged efficiently by flexibly relying more on a few items that are relevant to the task.

Statistical stability and set size exert distinct influences on visual search

Despite continuous retinal chaos, we perceive the world as stable and complete. This illusion is sustained over consecutive glances by reliance on statistical redundancies inherent in the visual environment. For instance, repeating the average size of a collection of differently sized items speeds visual search for a randomly located target regardless of trial-to-trial changes in local element size (Corbett & Melcher, 2014b). Here, we manipulate set size to investigate the potential role attention may play in these facilitative effects of statistical stability on visual search. Observers discriminated the left or right tilt of a Gabor target defined by a unique conjunction of orientation and spatial frequency in displays of Gabors with a stable or unstable mean size over successive trials. When set size was manipulated over sequences of successive trials, but held constant within a given sequence in Experiment 1, we observed distinct effects of statistical stability and attention, such that participants made faster correct responses as a function of stability and slower correct responses as a function of increasing set size. Replicating these main effects in Experiment 2, when set size was always unstable, provided converging evidence for discrete influences of statistical stability and attentional contributions to visual search. Overall, results support the proposal that our stable impressions of the surrounding environment and our abilities to attend salient events within that environment are distinctively governed by inherent statistical context and attentional processing demands.

Roles of saliency and set size in ensemble averaging

Ensemble statistics are often thought of as a reliable impression of numerous items despite limited capacities to consciously represent each individual. However, whether all items equally contribute to ensemble summaries (e.g., mean) and whether they might be affected by known limited-capacity processes, such as focused attention, is still debated. We addressed these questions via a recently described "amplification effect," a systematic bias of perceived mean (e.g., average size) towards the more salient "tail" of a feature distribution (e.g., larger items). In our experiments, observers adjusted the mean orientation of sets of items varying in set size. We made some of the items more salient or less salient by changing their size. While the whole orientation distribution was fixed, the more salient subset could be shifted relative to the set mean or differ in range. We measured the bias away from the set mean and the standard deviation (SD) of errors, as it is known to reflect the physical range from which ensemble information is sampled. We found that bias and SD changes followed the shifts and range changes in salient subsets, providing evidence for amplification. However, these changes were weaker than those expected from sampling only salient items, suggesting that less salient items were also sampled. Importantly, the SD decreased as a function of set size, which is only possible if the number of sampled elements increased with set size. Overall, we conclude that orientation summary statistics are sampled from an entire ensemble and modulated by the amplification effect of attention.

Independent and parallel visual processing of ensemble statistics: Evidence from dual tasks

The visual system can represent multiple objects in a compressed form of ensemble summary statistics (such as object numerosity, mean, and feature variance/range). Yet the relationships between the different types of visual statistics remain relatively unclear. Here, we tested whether two summaries (mean and numerosity, or mean and range) are calculated independently from each other and in parallel. Our participants performed dual tasks requiring a report about two summaries in each trial, and single tasks requiring a report about one of the summaries. We estimated trial-by-trial correlations between the precision of reports as well as correlations across observers. Both analyses showed the absence of correlations between different types of ensemble statistics, suggesting their independence. We also found no decrement (except that related to the order of report explained by memory retrieval) in performance in dual compared to single tasks, which suggests that two statistics of one ensemble can be processed in parallel.

Adaptation to mean and variance: Interrelationships between mean and variance representations in orientation perception

When there are many visual items, the visual system could represent their summary statistics (e.g., mean, variance) to process them efficiently. Although many previous studies have investigated the mean or variance representation itself, a relationship between these two ensemble representations has not been investigated much. In this study, we tested the potential interaction between mean and variance representations by using a visual adaptation method. We reasoned that if mean and variance representations interact with each other, an adaptation aftereffect to either mean or variance would influence the perception of the other. Participants watched a sequence of orientation arrays containing a specific statistical property during the adaptation period. To produce an adaptation aftereffect specific to variance or mean, one property of the adaptor arrays (variance or mean) had a fixed value while the other property was randomly varied. After the adaptation, participants were asked to discriminate the property of the test array that was randomly varied during the adaptation. We found that the adaptation aftereffect of orientation variance influenced the sensitivity of mean orientation discrimination (Experiment 1), and that the adaptation aftereffect of mean orientation influenced the bias of orientation variance discrimination (Experiment 2). These results suggest that mean and variance representations do closely interact with each other. Considering that mean and variance reflect the representative value and dispersion of multiple items respectively, the interactions between mean and variance representations may reflect their complementary roles to summarize complex visual information effectively.

Continuous to discrete: Ensemble-based segmentation in the perception of multiple feature conjunctions

Although objects around us vary in a number of continuous dimensions (color, size, orientation, etc.), we tend to perceive the objects using more discrete, categorical descriptions (e.g., berries and leaves). Previously, we described how continuous ensemble statistics of simple features are transformed into categorical classes: The visual system tests whether the feature distribution has one or several peaks, each representing a likely "category". Here, we tested the mechanism of segmentation for more complex conjunctions of features. Observers discriminated between two textures filled with lines of various lengths and orientations, which had same distributions between the textures, but opposite directions of correlations. Critically, feature distributions could be "segmentable" (only extreme feature values and a large gap between them) or "non-segmentable" (both extreme and middle values with smooth transition are present). Segmentable displays yielded steeper psychometric functions indicating better discrimination (Experiment 1). The effect of segmentability arises early in visual processing (Experiment 2) and is likely to be provided by global sampling of the entire field (Experiment 3). Also, rapid segmentation requires both feature dimensions having a "segmentable" distribution supporting division of the textures into categorical classes of conjunctions. We propose that observers select items from one side (peak) of one dimension and sample mean differences along a second dimension within the selected subset. In this scenario, subset selection is a limiting factor (Experiment 4) of texture discrimination. Yet, segmentability provided by the sharp feature distributions seems to facilitate both subset selection and mean comparison.

Ensemble coding remains accurate under object and spatial visual working memory load

A number of studies have provided evidence that the visual system statistically summarizes large amounts of information that would exceed the limitations of attention and working memory (ensemble coding). However the necessity of working memory resources for ensemble coding has not yet been tested directly. In the current study, we used a dual task design to test the effect of object and spatial visual working memory load on size averaging accuracy. In Experiment 1, we tested participants' accuracy in comparing the mean size of two sets under various levels of object visual working memory load. Although the accuracy of average size judgments depended on the difference in mean size between the two sets, we found no effect of working memory load. In Experiment 2, we tested the same average size judgment while participants were under spatial visual working memory load, again finding no effect of load on averaging accuracy. Overall our results reveal that ensemble coding can proceed unimpeded and highly accurately under both object and spatial visual working memory load, providing further evidence that ensemble coding reflects a basic perceptual process distinct from that of individual object processing.

Capacity limitations to extract the mean emotion from multiple facial expressions depend on emotion variance

We examined the processing capacity and the role of emotion variance in ensemble representation for multiple facial expressions shown concurrently. A standard set size manipulation was used, whereby the sets consisted of 4, 8, or 16 morphed faces each uniquely varying along a happy-angry continuum (Experiment 1) or a neutral-happy/angry continuum (Experiments 2 & 3). Across the three experiments, we reduced the amount of emotion variance in the sets to explore the boundaries of this process. Participants judged the perceived average emotion from each set on a continuous scale. We computed and compared objective and subjective difference scores, using the morph units and post-experiment ratings, respectively. Results of the subjective scores were more consistent than the objective ones across the first two experiments where the variance was relatively large, and revealed each time that increasing set size led to a poorer averaging ability, suggesting capacity limitations in establishing ensemble representations for multiple facial expressions. However, when the emotion variance in the sets was reduced in Experiment 3, both subjective and objective scores remained unaffected by set size, suggesting that the emotion averaging process was unlimited in these conditions. Collectively, these results suggest that extracting mean emotion from a set composed of multiple faces depends on both structural (attentional) and stimulus-related effects.

Averaging colors of multicolor mosaics

The present study investigated how color information was summarized in multicolor mosaics. The mosaics were composed of small elements of 17 colors that roughly belonged to a single color category. We manipulated the degree of color variation around the mean by varying the proportion of different color elements. Observers matched the mean color of the multicolor mosaic by adjusting the color of a spatially uniform matching stimulus. Results showed that when the color variation was large, the matched color deviated from the colorimetric mean toward the most-saturated color, although the hue of the matched color was almost the same as that of the colorimetric mean. These findings together suggested differential processing of hue and saturation. The deviation of the matched color decreased, but did not disappear, when the color variation was reduced. The analysis of color metric underlying color averaging revealed differential color scaling in nearly orthogonal blue-orange and green-purple directions, implying that the visual system does not solely rely on linear cone-opponent codes when summarizing color signals. The deviation itself was consistently found regardless of different color metrics tested. The robustness of the deviation indicated an inherent bias of mean color judgments favoring highly saturated colors.

The nature of correlation perception in scatterplots

For scatterplots with gaussian distributions of dots, the perception of Pearson correlation r can be described by two simple laws: a linear one for discrimination, and a logarithmic one for perceived magnitude (Rensink & Baldridge, 2010). The underlying perceptual mechanisms, however, remain poorly understood. To cast light on these, four different distributions of datapoints were examined. The first had 100 points with equal variance in both dimensions. Consistent with earlier results, just noticeable difference (JND) was a linear function of the distance away from r = 1, and the magnitude of perceived correlation a logarithmic function of this quantity. In addition, these laws were linked, with the intercept of the JND line being the inverse of the bias in perceived magnitude. Three other conditions were also examined: a dot cloud with 25 points, a horizontal compression of the cloud, and a cloud with a uniform distribution of dots. Performance was found to be similar in all conditions. The generality and form of these laws suggest that what underlies correlation perception is not a geometric structure such as the shape of the dot cloud, but the shape of the probability distribution of the dots, likely inferred via a form of ensemble coding. It is suggested that this reflects the ability of observers to perceive the information entropy in an image, with this quantity used as a proxy for Pearson correlation.

Summary statistics in the attentional blink

We used the attentional blink (AB) paradigm to investigate the processing stage at which extraction of summary statistics from visual stimuli ("ensemble coding") occurs. Experiment 1 examined whether ensemble coding requires attentional engagement with the items in the ensemble. Participants performed two sequential tasks on each trial: gender discrimination of a single face (T1) and estimating the average emotional expression of an ensemble of four faces (or of a single face, as a control condition) as T2. Ensemble coding was affected by the AB when the tasks were separated by a short temporal lag. In Experiment 2, the order of the tasks was reversed to test whether ensemble coding requires more working-memory resources, and therefore induces a larger AB, than estimating the expression of a single face. Each condition produced a similar magnitude AB in the subsequent gender-discrimination T2 task. Experiment 3 additionally investigated whether the previous results were due to participants adopting a subsampling strategy during the ensemble-coding task. Contrary to this explanation, we found different patterns of performance in the ensemble-coding condition and a condition in which participants were instructed to focus on only a single face within an ensemble. Taken together, these findings suggest that ensemble coding emerges automatically as a result of the deployment of attentional resources across the ensemble of stimuli, prior to information being consolidated in working memory.

Differential hemispheric and visual stream contributions to ensemble coding of crowd emotion

In crowds, where scrutinizing individual facial expressions is inefficient, humans can make snap judgments about the prevailing mood by reading "crowd emotion". We investigated how the brain accomplishes this feat in a set of behavioral and fMRI studies. Participants were asked to either avoid or approach one of two crowds of faces presented in the left and right visual hemifields. Perception of crowd emotion was improved when crowd stimuli contained goal-congruent cues and was highly lateralized to the right hemisphere. The dorsal visual stream was preferentially activated in crowd emotion processing, with activity in the intraparietal sulcus and superior frontal gyrus predicting perceptual accuracy for crowd emotion perception, whereas activity in the fusiform cortex in the ventral stream predicted better perception of individual facial expressions. Our findings thus reveal significant behavioral differences and differential involvement of the hemispheres and the major visual streams in reading crowd versus individual face expressions.

The numerosity and mean size of multiple objects are perceived independently and in parallel

It is well documented that people are good at the rapid representation of multiple objects in the form of ensemble summary statistics of different types (numerosity, the average feature, the variance of features, etc.). However, there is not enough clarity regarding the links between statistical domains. The relations between different-type summaries (numerosity and the mean) are of particular interest, since they can shed light on (1) a very general functional organization of ensemble processing and (2) mechanisms of statistical computations (whether averaging takes into account numerical information, as in regular statistics). Here, we show no correlation between the precision of estimated numerosity and that of the estimated mean. We also found that people are very good at dividing attention between numerosity and the mean size of a single set (Experiment 1); however, they show some cost of dividing attention between two same-type (two numerosities or two mean sizes, Experiment 2) and two different-type (one numerosity and one mean size, Experiment 3) summaries when each summary is ascribed to a different set. These results support the idea of domain specificity of numerosity and mean size perception, which also implies that, unlike regular statistics, computing the mean does not require numerosity information. We also conclude that computational capacity of ensemble statistics is more limited by encoding several ensembles than computing several summaries.

Ensemble perception of color in autistic adults

Dominant accounts of visual processing in autism posit that autistic individuals have an enhanced access to details of scenes [e.g., weak central coherence] which is reflected in a general bias toward local processing. Furthermore, the attenuated priors account of autism predicts that the updating and use of summary representations is reduced in autism. Ensemble perception describes the extraction of global summary statistics of a visual feature from a heterogeneous set (e.g., of faces, sizes, colors), often in the absence of local item representation. The present study investigated ensemble perception in autistic adults using a rapidly presented (500 msec) ensemble of four, eight, or sixteen elements representing four different colors. We predicted that autistic individuals would be less accurate when averaging the ensembles, but more accurate in recognizing individual ensemble colors. The results were consistent with the predictions. Averaging was impaired in autism, but only when ensembles contained four elements. Ensembles of eight or sixteen elements were averaged equally accurately across groups. The autistic group also showed a corresponding advantage in rejecting colors that were not originally seen in the ensemble. The results demonstrate the local processing bias in autism, but also suggest that the global perceptual averaging mechanism may be compromised under some conditions. The theoretical implications of the findings and future avenues for research on summary statistics in autism are discussed. Autism Res 2017, 10: 839-851. © 2016 International Society for Autism Research, Wiley Periodicals, Inc.

Ensemble perception in autism spectrum disorder: Member-identification versus mean-discrimination

To efficiently represent the outside world our brain compresses sets of similar items into a summarized representation, a phenomenon known as ensemble perception. While most studies on ensemble perception investigate this perceptual mechanism in typically developing (TD) adults, more recently, researchers studying perceptual organization in individuals with autism spectrum disorder (ASD) have turned their attention toward ensemble perception. The current study is the first to investigate the use of ensemble perception for size in children with and without ASD (N = 42, 8-16 years). We administered a pair of tasks pioneered by Ariely [2001] evaluating both member-identification and mean-discrimination. In addition, we varied the distribution types of our sets to allow a more detailed evaluation of task performance. Results show that, overall, both groups performed similarly in the member-identification task, a test of "local perception," and similarly in the mean identification task, a test of "gist perception." However, in both tasks performance of the TD group was affected more strongly by the degree of stimulus variability in the set, than performance of the ASD group. These findings indicate that both TD children and children with ASD use ensemble statistics to represent a set of similar items, illustrating the fundamental nature of ensemble coding in visual perception. Differences in sensitivity to stimulus variability between both groups are discussed in relation to recent theories of information processing in ASD (e.g., increased sampling, decreased priors, increased precision). Autism Res 2017. © 2017 International Society for Autism Research, Wiley Periodicals, Inc. Autism Res 2017, 10: 1291-1299. © 2017 International Society for Autism Research, Wiley Periodicals, Inc.

Processing of Individual Items during Ensemble Coding of Facial Expressions

There is growing evidence that human observers are able to extract the mean emotion or other type of information from a set of faces. The most intriguing aspect of this phenomenon is that observers often fail to identify or form a representation for individual faces in a face set. However, most of these results were based on judgments under limited processing resource. We examined a wider range of exposure time and observed how the relationship between the extraction of a mean and representation of individual facial expressions would change. The results showed that with an exposure time of 50 ms for the faces, observers were more sensitive to mean representation over individual representation, replicating the typical findings in the literature. With longer exposure time, however, observers were able to extract both individual and mean representation more accurately. Furthermore, diffusion model analysis revealed that the mean representation is also more prone to suffer from the noise accumulated in redundant processing time and leads to a more conservative decision bias, whereas individual representations seem more resistant to this noise. Results suggest that the encoding of emotional information from multiple faces may take two forms: single face processing and crowd face processing.

The time-limited visual statistician

The visual system can calculate summary statistics over time. For example, the multiple frames of a movie showing a dynamically changing disk can be collapsed to form a single representation of that disk's mean size. Summary representations of dynamic information may engage online updating processes that establish a running average of the mean by continuously adjusting the persisting representation of the average in tandem with the arrival of incoming information. Alternatively, summary representations may involve subsampling strategies that reflect limitations in the degree to which the visual system can integrate information over time. Observers watched movies of a disk that changed size smoothly at different rates and then reported the disk's average size by adjusting the diameter of a response disk. Critically, the movie varied in duration. Size estimates depended on the duration of the movie. They were constant and fairly accurate for movie durations up to approximately 600 ms, at which point accuracy decreased with increasing duration to imprecise levels by about 1,000 ms. Summary statistics established over time are unlikely to be updated continuously and may instead be restricted by subsampling processes, such as limited temporal windows of integration. (PsycINFO Database Record

Accurate rapid averaging of multihue ensembles is due to a limited capacity subsampling mechanism

It is claimed that the extraction of average features from rapidly presented ensembles is holistic, with attention distributed across the whole set. We investigated whether observers' extraction of mean hue is holistic or could reflect subsampling. Analysis of selections for the mean hue revealed a distribution that peaked at the expected mean hue. However, an ideal observer simulation suggested that a subsampling mechanism incorporating just two items from each ensemble would suffice to reproduce the precision of most observers. The results imply that hue may not be averaged as holistically and efficiently as other attributes.

The capacity limitations of orientation summary statistics

The simultaneous-sequential method was used to test the processing capacity of establishing mean orientation summaries. Four clusters of oriented Gabor patches were presented in the peripheral visual field. One of the clusters had a mean orientation that was tilted either left or right, whereas the mean orientations of the other three clusters were roughly vertical. All four clusters were presented at the same time in the simultaneous condition, whereas the clusters appeared in temporal subsets of two in the sequential condition. Performance was lower when the means of all four clusters had to be processed concurrently than when only two had to be processed in the same amount of time. The advantage for establishing fewer summaries at a given time indicates that the processing of mean orientation engages limited-capacity processes (Exp. 1). This limitation cannot be attributed to crowding, low target-distractor discriminability, or a limited-capacity comparison process (Exps. 2 and 3). In contrast to the limitations of establishing multiple summary representations, establishing a single summary representation unfolds without interference (Exp. 4). When interpreted in the context of recent work on the capacity of summary statistics, these findings encourage a reevaluation of the view that early visual perception consists of creating summary statistic representations that unfold independently across multiple areas of the visual field.