Ensemble Perception

Redundancy masking: The loss of repeated items in crowded peripheral vision

Crowding is the deterioration of target identification in the presence of neighboring objects. Recent studies using appearance-based methods showed that the perceived number of target elements is often diminished in crowding. Here we introduce a related type of diminishment in repeating patterns (sets of parallel lines), which we term “redundancy masking.” In four experiments, observers were presented with arrays of small numbers of lines centered at 10° eccentricity. The task was to indicate the number of lines. In Experiment 1, spatial characteristics of redundancy masking were examined by varying the inter-line spacing. We found that redundancy masking decreased with increasing inter-line spacing and ceased at spacings of approximately 0.25 times the eccentricity. In Experiment 2, we assessed whether the strength of redundancy masking differed between radial and tangential arrangements of elements as it does in crowding. Redundancy masking was strong with radially arranged lines (horizontally arranged vertical lines), and absent with tangentially arranged lines (vertically arranged horizontal lines). In Experiment 3, we investigated whether target size (line width and length) modulated redundancy masking. There was an effect of width: Thinner lines yielded stronger redundancy masking. We did not find any differences between the tested line lengths. In Experiment 4, we varied the regularity of the line arrays by vertically or horizontally jittering the positions of the lines. Redundancy masking was strongest with regular spacings and weakened with decreasing regularity. Our experiments show under which conditions whole items are lost in crowded displays, and how this redundancy masking resembles—and partly diverges from—crowded identification. We suggest that redundancy masking is a contributor to the deterioration of performance in crowded displays with redundant patterns.

A direct comparison of central tendency recall and temporal integration in the successive field iconic memory task

The ensemble coding literature suggests the existence of a fast, automatic formation of some ensemble codes. Can statistical representations, such as memory for the central tendency along a particular visual feature dimension, be extracted from information held in the sensory register? Furthermore, can knowledge of early, iconic memory processes be used to determine how central tendency is extracted? We focused on the potential role of visible persistence mechanisms that support temporal integration. We tested whether mean orientation could be accurately recalled from brief visual displays using the successive field task. On separate blocks of trials, participants were asked to report the location of a split element (requiring differentiation of frames), a missing element (requiring integration across frames), and the average orientation of elements pooled across both frames (central tendency recall). Results replicate the expected tradeoff between differentiation and integration performance across inter-frame interval (IFI). In contrast, precision of mean estimates was high and invariant across IFIs. A manipulation of within-frame distributional similarity coupled with simulations using 12 models supported 2-item subsampling. The results argue against the “strategic” interpretation of subsampling since 2-item readout was predicted by information theoretic estimates of STM encoding rate: the 2 items were not from a superset in STM. Most crucially, the results argue against the various early “preattentive/parallel/global pooling” accounts and instead suggest that non-selective readout of information from iconic memory supplies a relatively small amount of item information to STM, and it is only at this point that the computation of ensemble averages begins.

Set size and ensemble perception of numerical value

There is a growing body of research on ensemble perception, or our ability to form ensemble representations based on perceptual features for stimuli of varying levels of complexity, and more recently, on ensemble cognition, which refers to our ability to perceive higher-level properties of stimuli such as facial attractiveness or gaze direction. Less is known about our ability to form ensemble representations based on more abstract properties such as the semantic meaning associated with items in a scene. Previous work examining whether the meaning associated with digits can be incorporated into summary statistical representations suggests that numerical information from digit ensembles can be extracted rapidly, and likely using a parallel processing mechanism. Here, we further investigate whether participants can accurately generate summary representations of numerical value from digit sets and explore the effect of set size on their ability to do so, by comparing psychometric functions based on a numerical averaging task in which set size varied. Steeper slopes for ten- and seven-item compared to five-item digit sets provide evidence that displays with more digits yield more reliable discrimination between larger and smaller numerical averages. Additionally, consistent with previous reports, we observed a response bias such that participants were more likely to report that the numerical average was "greater than 5" for larger compared to smaller sets. Overall, our results contribute to evidence that ensemble representations for semantic attributes may be carried out via similar mechanisms as those reported for perceptual features.

Synergy between research on ensemble perception, data visualization, and statistics education: A tutorial review

In the age of big data, we are constantly inventing new data visualizations to consolidate massive amounts of numerical information into smaller and more digestible visual formats. These data visualizations use various visual features to convey quantitative information, such as spatial position in scatter plots, color saturation in heat maps, and area in dot maps. These data visualizations are typically composed of ensembles, or groups of related objects, that together convey information about a data set. Ensemble perception, or one's ability to perceive summary statistics from an ensemble, such as the mean, has been used as a foundation for understanding and explaining the effectiveness of certain data visualizations. However, research in data visualization has revealed some perceptual biases and conceptual difficulties people face when trying to utilize the information in these graphs. In this tutorial review, we will provide a broad overview of research conducted in ensemble perception, discuss how principles of ensemble encoding have been applied to the research in data visualization, and showcase the barriers graphs can pose to learning statistical concepts, using histograms as a specific example. The goal of this tutorial review is to highlight possible connections between three areas of research-ensemble perception, data visualization, and statistics education-and to encourage research in the practical applications of ensemble perception in solving real-world problems in statistics education.

Somatosensory evoked potentials that index lateral inhibition are modulated according to the mode of perceptual processing: comparing or combining multi-digit tactile motion

Many perceptual studies focus on the brain's capacity to discriminate between stimuli. However, our normal experience of the world also involves integrating multiple stimuli into a single perceptual event. Neural mechanisms such as lateral inhibition are believed to enhance local differences between sensory inputs from nearby regions of the receptor surface. However, this mechanism would seem dysfunctional when sensory inputs need to be combined rather than contrasted. Here, we investigated whether the brain can strategically regulate the strength of suppressive interactions that underlie lateral inhibition between finger representations in human somatosensory processing. To do this, we compared sensory processing between conditions that required either comparing or combining information. We delivered two simultaneous tactile motion trajectories to index and middle fingertips of the right hand. Participants had to either compare the directions of the two stimuli, or to combine them to form their average direction. To reveal preparatory tuning of somatosensory cortex, we used an established event-related potential design to measure the interaction between cortical representations evoked by digital nerve shocks immediately before each tactile stimulus. Consistent with previous studies, we found a clear suppression between cortical activations when participants were instructed to compare the tactile motion directions. Importantly, this suppression was significantly reduced when participants had to combine the same stimuli. These findings suggest that the brain can strategically switch between a comparative and a combinative mode of somatosensory processing, according to the perceptual goal, by preparatorily adjusting the strength of a process akin to lateral inhibition.

Uncertainty-based overestimation in the perception of group actions

Individuals are adept at estimating average properties of group visual stimuli, even following brief presentations. In estimating the directional heading of walking human figures, judgments are biased in a peculiar manner: groups facing intermediate directions are perceived to be more leftward- or rightward-facing than actual averages. This effect was previously explained as a repulsive bias away from a central category boundary; groups along this boundary (directly facing the observer) are estimated with lower variability and with relatively greater accuracy. Here we show that: (i) the original effect replicates and is constant over time in a novel estimation task with persistent directional states; and, (ii) novel patterns of response variability and durations align with the entire range of overestimation. A simple model of additive errors proportional to viewer uncertainty matches the observed bias magnitudes. We furthermore show that the bias generalizes beyond approaching walkers with the use of rearward-facing walkers presented at a nonparallel angle. Overall, the recurring relation between bias and uncertainty is also consistent with top-down and post-perceptual causes of misestimation.

Asymmetric contextual effects in age perception

Perception is context dependent. For example, the perceived orientation of a bar changes depending on the presence of oriented bars around it. Contextual effects have also been demonstrated for more complex judgements, such as facial attractiveness or expression, although it remains unclear how these contextual facial effects depend on the types of faces surrounding the target face. To examine this, we measured the perceived age (a quantifiable measure) of a target face in the presence of differently aged faces in the surround. Using a unique database of standardized passport photos, participants were asked to estimate the age of a target face which was viewed either on its own or surrounded by two different identity flanker faces. The flanker faces were either both younger or both older than the target face, with different age offsets between flankers and targets of ±5, ±10, ±15, ±20 years. We find that when a target face is surrounded by younger faces, it systematically appears younger than when viewed on its own, and when it is surrounded by older faces, it systematically appears older than when viewed on its own. Surprisingly, we find that the magnitude of the flanker effects on perceived age of the target is asymmetric with younger flankers having a greater influence than older flankers, a result that may reflect the participants' own-age bias, since all participants were young. This result holds irrespective of gender or race of the faces and is consistent with averaging.

Temporal bisection is influenced by ensemble statistics of the stimulus set

Although humans are well capable of precise time measurement, their duration judgments are nevertheless susceptible to temporal context. Previous research on temporal bisection has shown that duration comparisons are influenced by both stimulus spacing and ensemble statistics. However, theories proposed to account for bisection performance lack a plausible justification of how the effects of stimulus spacing and ensemble statistics are actually combined in temporal judgments. To explain the various contextual effects in temporal bisection, we develop a unified ensemble-distribution account (EDA), which assumes that the mean and variance of the duration set serve as a reference, rather than the short and long standards, in duration comparison. To validate this account, we conducted three experiments that varied the stimulus spacing (Experiment 1), the frequency of the probed durations (Experiment 2), and the variability of the probed durations (Experiment 3). The results revealed significant shifts of the bisection point in Experiments 1 and 2, and a change of the sensitivity of temporal judgments in Experiment 3-which were all well predicted by EDA. In fact, comparison of EDA to the extant prior accounts showed that using ensemble statistics can parsimoniously explain various stimulus set-related factors (e.g., spacing, frequency, variance) that influence temporal judgments.

Holistic ensemble perception

In a glance, observers can evaluate gist characteristics from crowds of faces, such as the average emotional tenor or the average family resemblance. Prior research suggests that high-level ensemble percepts rely on holistic and viewpoint-invariant information. However, it is also possible that feature-based analysis was sufficient to yield successful ensemble percepts in many situations. To confirm that ensemble percepts can be extracted holistically, we asked observers to report the average emotional valence of Mooney face crowds. Mooney faces are two-tone, shadow-defined images that cannot be recognized in a part-based manner. To recognize features in a Mooney face, one must first recognize the image as a face by processing it holistically. Across experiments, we demonstrated that observers successfully extracted the average emotional valence from crowds that were spatially distributed or viewed in a rapid temporal sequence. In a subsequent set of experiments, we maximized holistic processing by including only those Mooney faces that were difficult to recognize when inverted. Under these conditions, participants remained highly sensitive to the average emotional valence of Mooney face crowds. Taken together, these experiments provide evidence that ensemble perception can operate selectively on holistic representations of human faces, even when feature-based information is not readily available.

Ensemble coding of crowd speed using biological motion

The accurate perception of human crowds is integral to social understanding and interaction. Previous studies have shown that observers are sensitive to several crowd characteristics such as average facial expression, gender, identity, joint attention, and heading direction. In two experiments, we examined ensemble perception of crowd speed using standard point-light walkers (PLW). Participants were asked to estimate the average speed of a crowd consisting of 12 figures moving at different speeds. In Experiment 1, trials of intact PLWs alternated with trials of scrambled PLWs with a viewing duration of 3 seconds. We found that ensemble processing of crowd speed could rely on local motion alone, although a globally intact configuration enhanced performance. In Experiment 2, observers estimated the average speed of intact-PLW crowds that were displayed at reduced viewing durations across five blocks of trials (between 2500 ms and 500 ms). Estimation of fast crowds was precise and accurate regardless of viewing duration, and we estimated that three to four walkers could still be integrated at 500 ms. For slow crowds, we found a systematic deterioration in performance as viewing time reduced, and performance at 500 ms could not be distinguished from a single-walker response strategy. Overall, our results suggest that rapid and accurate ensemble perception of crowd speed is possible, although sensitive to the precise speed range examined.

Car expertise does not compete with face expertise during ensemble coding

When objects from two categories of expertise (e.g., faces and cars in dual car/face experts) are processed simultaneously, competition occurs across a variety of tasks. Here, we investigate whether competition between face and car processing also occurs during ensemble coding. The relationship between single object recognition and ensemble coding is debated, but if ensemble coding relies on the same ability as object recognition, we expect cars to interfere with ensemble coding of faces as a function of car expertise. We measured the ability to judge the variability in identity of arrays of faces, in the presence of task-irrelevant distractors (cars or novel objects). On each trial, participants viewed two sequential arrays containing four faces and four distractors, judging which array was the more diverse in terms of face identity. We measured participants' car expertise, object recognition ability, and face recognition ability. Using Bayesian statistics, we found evidence against competition as a function of car expertise during ensemble coding of faces. Face recognition ability predicted ensemble judgments for faces, regardless of the category of task-irrelevant distractors. The result suggests that ensemble coding is not susceptible to competition between different domains of similar expertise, unlike single-object recognition.

Logarithmic encoding of ensemble time intervals

Although time perception is based on the internal representation of time, whether the subjective timeline is scaled linearly or logarithmically remains an open issue. Evidence from previous research is mixed: while the classical internal-clock model assumes a linear scale with scalar variability, there is evidence that logarithmic timing provides a better fit to behavioral data. A major challenge for investigating the nature of the internal scale is that the retrieval process required for time judgments may involve a remapping of the subjective time back to the objective scale, complicating any direct interpretation of behavioral findings. Here, we used a novel approach, requiring rapid intuitive ‘ensemble’ averaging of a whole set of time intervals, to probe the subjective timeline. Specifically, observers’ task was to average a series of successively presented, auditory or visual, intervals in the time range 300–1300 ms. Importantly, the intervals were taken from three sets of durations, which were distributed such that the arithmetic mean (from the linear scale) and the geometric mean (from the logarithmic scale) were clearly distinguishable. Consistently across the three sets and the two presentation modalities, our results revealed subjective averaging to be close to the geometric mean, indicative of a logarithmic timeline underlying time perception.

Ensemble perception includes information from multiple spatial scales

Most visual scenes contain information at different spatial scales, including the local and global, or the detail and gist. Global processes have become increasingly implicated in research examining summary statistical perception, initially as the output of ensemble coding, and more recently as a gating mechanism for selecting which information is included in the averaging process itself. Yet local and global processing are known to be rapidly integrated by the visual system, and it is plausible that global-level information, like spatial organization, may be included as an input during ensemble coding. We tested this hypothesis using an ensemble shape-perception task in which observers evaluated the mean aspect ratios of sets of ellipses. In addition to varying the aspect ratios of the individual shapes, we independently varied the spatial arrangements of the sets so that they had either flat or tall organizations at the global level. We found that observers made precise summary judgments about the average aspect ratios of the sets by integrating information from multiple shapes. More importantly, global flat and tall organizations were incorporated into ensemble judgments about the sets; summary judgments were biased in the directions of the global spatial arrangements on each trial. This global-to-local integration even occurred when the global organizations were masked. Our results demonstrate that the process of summary representation can include information from both the local and global scales. The gist is not just an output of ensemble representation - it can be included as an input to the mechanism itself.

Implicit processing during inattentional blindness: A systematic review and meta-analysis

The occurrence of implicit processing of visual stimuli during inattentional blindness is still a matter of debate. To assess the evidence available in this debate, we conducted a systematic review of articles that explored whether unexpected visual stimuli presented during inattentional blindness are implicitly processed despite not being reported. Additionally, we employed meta-analysis to combine 59 behavioral experiments and investigate the statistical support for such implicit processing across experiments. Results showed that visual stimuli can be processed when unattended and unnoticed. Additionally, we reviewed the measures used to assess participants' awareness of the unexpected stimuli. We also employed meta-analysis to search for differences in awareness of the unexpected stimuli that may result from adopting distinct criteria to categorize participants as aware or unaware. The results showed that the overall effect of awareness changed depending on whether more demanding or less demanding measures of awareness were employed. This suggests that the choice of awareness measure may influence conclusions about whether processing of the US is implicit or explicit. We discuss the implications of these results for the study of implicit processing and the role of attention in visual cognition.

Ensemble size judgments account for size constancy

The natural environment is full of redundant information that the visual system compresses into an ensemble representation by averaging features of groups of items. Ensemble perception has been shown to operate with remarkable flexibility, efficiently integrating information across a variety of visual domains. In the current set of experiments, we tested whether average size representations reflect the physical size of objects displayed on a screen or perceptual transformations due to size constancy. We induced a perceptual change by presenting sets of triangles with linear perspective cues - lines converging at the horizon. Assuming a constant size, these cues cause individual objects "in the distance" to appear larger than objects without distance cues, due to size constancy heuristics. Observers viewed sets of triangles with and without linear perspective cues and judged whether a subsequently presented test triangle was larger or smaller than the average size of the preceding set. Results revealed ensemble size representations took size constancy into account, reflecting the perceived size of the triangles rather than their absolute size. Interestingly, the amount of bias exhibited was well characterized by the summed bias associated with each of the three triangles presented individually. Other pictorial cues to depth, such as occlusion and height-in-field, did not elicit the same bias when those were the only depth cues available. Overall, our results complement and extend other work showing that average size reflects the perceptual size of individual items in a set.

Statistical summary representations of bound features

The visual system can efficiently summarize various lower-level and higher-level features of ensembles. However, no research to date has directly investigated how different features interact with each other within a single summary and whether people can efficiently report an integrated summary of two feature dimensions. In the first two experiments, we specifically investigated whether individuals can integrate spatial and size information to report a bound spatial summary, the center of mass (CoM), as efficiently as the centroid, which is devoid of size information. Both experiments revealed that viewers were equally accurate in extracting the centroid and the CoM, with the size distribution inadvertently affecting the centroid estimates. In the final experiment, we investigated whether encouraging observers to attend to individual item size would cause the centroid estimates to be biased toward the CoM. When item size was task-irrelevant, as in the centroid task, observers were able to selectively focus on spatial location, eliminating any impact from the size distribution. Taken together, these findings demonstrate that viewers are capable of extracting integrated summaries, possibly through a mechanism that allows them to represent the spatial distribution of sizes. We discuss possible mechanisms that may support the extraction of integrated summaries, and highlight the need to consider multilevel mechanisms extending beyond simple feature- and object-based mechanisms.

The role of category- and exemplar-specific experience in ensemble processing of objects

People can relatively easily report summary properties for ensembles of objects, suggesting that this information can enrich visual experience and increase the efficiency of perceptual processing. Here, we ask whether the ability to judge diversity within object arrays improves with experience. We surmised that ensemble judgments would be more accurate for commonly experienced objects, and perhaps even more for objects of expertise like faces. We also expected improvements in ensemble processing with practice with a novel category, and perhaps even more with repeated experience with specific exemplars. We compared the effect of experience on diversity judgments for arrays of objects, with participants being tested with either a small number of repeated exemplars or with a large number of exemplars from the same object category. To explore the role of more prolonged experience, we tested participants with completely novel objects (random blobs), with objects familiar at the category level (cars), and with objects with which observers are experts at subordinate-level recognition (faces). For objects that are novel, participants showed evidence of improved ability to distribute attention. In contrast, for object categories with long-term experience, i.e., faces and cars, performance improved during the experiment but not necessarily due to improved ensemble processing. Practice with specific exemplars did not result in better diversity judgments for all object categories. Considered together, these results suggest that ensemble processing improves with experience. However, experience operates rapidly, the role of experience does not rely on exemplar-level knowledge and may not benefit from subordinate-level expertise.

Perception of means, sums, and areas

In this age of data visualization, it is important to understand our perception of the symbols that are used. For example, does the perceived size of a disc correspond most closely to its area, diameter, circumference, or some other measure? When multiple items are present, this becomes a question of ensemble perception. Here, we compare observers' performance across three different tasks: judgments of (i) the mean diameter, (ii) the total diameter, or (iii) the total area of (N = 1, 2, 3, or 7) test circles compared with a single reference circle. We draw a parallel between Anne Treisman's feature integration theory and Daniel Kahneman's cognitive systems, comparing the preattentive stage to System 1, and the focused attention stage to System 2. In accordance with Kahneman's prediction, average size (diameter) of the geometric figures can be judged with considerable accuracy, but the total diameter of the same figures cannot. Like the total length, the cumulative area covered by circles was also judged considerably less accurately than the mean diameter. Differences in efficiency between these three tasks illustrate powerful constraints upon visual processing: The visual system is well adapted for the perception of the mean size while there are no analogous mechanisms for the accurate perception of the total length or cumulative area. Thus, in visualizing data, using bubble charts proportional to area may be misleading as our visual system seems better adapted to perceive disc size by the radius rather than the area.

Feature-specificity in visual statistical summary processing

Visual statistical summary processing enables people to extract the average feature of a set of items rapidly and accurately. Previous studies have demonstrated independent mechanisms for summarizing low (e.g. color, orientation) and high-level (facial identity, emotion) visual information. However, no study to date has conclusively determined whether there are feature-specific summarization mechanisms for low-level features or whether there are low-level, feature agnostic summarization mechanisms. To address this issue, we asked participants to report either the average orientation or the average size from a set of lines where both features varied. Participants completed these tasks either in single-task or mixed-task conditions; in the latter, successful performance required extraction of both summaries concurrently. If there were feature-specific summarization mechanisms that could operate in parallel, then errors in mean size and mean orientation tasks should be independent, in both single and mixed task conditions. On the other hand, a central domain-general mechanism for low-level summarization would imply a correlation between errors for both features and greater error in the mixed than single task trials. In Experiment 1, we found that there was no correlation between the mean size and mean orientation errors and performance was similar across single and mixed-task conditions, suggesting that there may be independent summarization mechanisms for size and orientation features. To further test the feature-specificity account, in Experiment 2 and 3 (with mask), we manipulated the display duration to determine whether there were any differences in the summarization of earlier (orientation) vs. later (size) features. While these experiments replicated the pattern of results observed in Experiment 1, at shorter display durations, no differences emerged across features. We argue that our data is consistent with independent, multi-level feature-specific statistical summary mechanisms for low-level visual features.

Contrast Effect of Facial Attractiveness in Groups

Research on facial attractiveness is an important part of aesthetics. Most relevant studies in the area have focused on the influence of individual perspectives on facial attractiveness, but it is necessary to consider the effect of contextual information on facial attractiveness. In this study, we examine the influence on attractiveness of special faces in a given group. We define a “special face” as one that is significantly different from other members of the same group in terms of facial attractiveness. We conducted three experiments to explore the influence of different modes of presentation and central positions in a group on the judgment of attractiveness of the special face. The results show the following: (1) When the special face was part of a given group, the subjects made more extreme judgments than without it: that is, they judged the most attractive face as more attractive and the least as less attractive than when faces were presented alone. (2) The subjects rated the most attractive faces lower and the least attractive faces higher when the target faces in the middle of the group than in other positions. The results favored the contrast effect: when the subjects judged the attractiveness of target stimulus, they always compared it with the environment, which then became a reference in this regard. Moreover, the greater the amount of contextual information perceived, the higher the likelihood that assimilation would occur.

Ensemble coding of color and luminance contrast

Ensemble coding has been demonstrated for many attributes including color, but the metrics on which this coding is based remain uncertain. We examined ensemble percepts for stimulus sets that varied in chromatic contrast between complementary hues, or that varied in luminance contrast between increments and decrements, in both cases focusing on the ensemble percepts for the neutral gray stimulus defining the category boundary. Each ensemble was composed of 16 circles with four contrast levels. Observers saw the display for 0.5 s and then judged whether a target contrast was a member of the set. False alarms were high for intermediate contrasts (within the range of the ensemble) and fell for higher or lower values. However, for ensembles with complementary hues, gray was less likely to be reported as a member, even when it represented the mean chromaticity of the set. When the settings were repeated for luminance contrast, false alarms for gray were higher and fell off more gradually for out-of-range contrasts. This difference implies that opposite luminance polarities represent a more continuous perceptual dimension than opponent-color variations, and that "gray" is a stronger category boundary for chromatic than luminance contrasts. For color, our results suggest that ensemble percepts reflect pooling within rather than between large hue differences, perhaps because the visual system represents hue differences more like qualitatively different categories than like quantitative differences within an underlying color "space." The differences for luminance and color suggest more generally that ensemble coding for different visual attributes might depend on different processes that in turn depend on the format of the visual representation.

Convergent evidence for a theory of rapid, automatic, and accurate sex ratio tracking

It is presumed that people track the sex ratios in their environment (the number of males relative to number of females) in order to adaptively adjust their decisions and behaviors, but this actual tracking ability has not been established. The relevance of sex ratio information, drawn from evolutionary biology and studies of human relationship decision making, is integrated here with memory research (on frequency encoding), perception research (on ensemble coding), and neuroscience research. A series of four experiments provide empirical results to help fill research gaps and facilitate this theoretical integration. In particular, these studies connect details from memory research on relatively automatic frequency encoding of both items and categories, perception research on summary statistics from ensemble coding, and theoretical ideas about the function of these abilities (specifically applied to human sex ratios based on faces) from social and evolutionary approaches. Collectively this research demonstrates an evolved psychological mechanism for functional, fast, and relatively automatic human abilities to track experienced sex ratios in the social world. This sex ratio information is theorized to underpin documented facultative adjustments in relationship dynamics as well as perceptions of social group characteristics. This integrative approach highlights how the coding, memory, and judgments about population sex ratios can both account for a number of existing findings and point towards key further research.

Combining biological motion perception with optic flow analysis for self-motion in crowds

Heading estimation from optic flow relies on the assumption that the visual world is rigid. This assumption is violated when one moves through a crowd of people, a common and socially important situation. The motion of people in the crowd contains cues to their translation in the form of the articulation of their limbs, known as biological motion. We investigated how translation and articulation of biological motion influence heading estimation from optic flow for self-motion in a crowd. Participants had to estimate their heading during simulated self-motion toward a group of walkers who collectively walked in a single direction. We found that the natural combination of translation and articulation produces surprisingly small heading errors. In contrast, experimental conditions that either present only translation or only articulation produced strong idiosyncratic biases. The individual biases explained well the variance in the natural combination. A second experiment showed that the benefit of articulation and the bias produced by articulation were specific to biological motion. An analysis of the differences in biases between conditions and participants showed that different perceptual mechanisms contribute to heading perception in crowds. We suggest that coherent group motion affects the reference frame of heading perception from optic flow.

The Face of Image Reconstruction: Progress, Pitfalls, Prospects

Recent research has demonstrated that neural and behavioral data acquired in response to viewing face images can be used to reconstruct the images themselves. However, the theoretical implications, promises, and challenges of this direction of research remain unclear. We evaluate the potential of this research for elucidating the visual representations underlying face recognition. Specifically, we outline complementary and converging accounts of the visual content, the representational structure, and the neural dynamics of face processing. We illustrate how this research addresses fundamental questions in the study of normal and impaired face recognition, and how image reconstruction provides a powerful framework for uncovering face representations, for unifying multiple types of empirical data, and for facilitating both theoretical and methodological progress.

Oscillatory Bursts in Parietal Cortex Reflect Dynamic Attention between Multiple Objects and Ensembles

The visual system uses two complimentary strategies to process multiple objects simultaneously within a scene and update their spatial positions in real time. It either uses selective attention to individuate a complex, dynamic scene into a few focal objects (i.e., object individuation), or it represents multiple objects as an ensemble by distributing attention more globally across the scene (i.e., ensemble grouping). Neural oscillations may be a key signature for focal object individuation versus distributed ensemble grouping, because they are thought to regulate neural excitability over visual areas through inhibitory control mechanisms. We recorded whole-head MEG data during a multiple-object tracking paradigm, in which human participants (13 female, 11 male) switched between different instructions for object individuation and ensemble grouping on different trials. The stimuli, responses, and the demand to keep track of multiple spatial locations over time were held constant between the two conditions. We observed increased α-band power (9-13 Hz) packed into oscillatory bursts in bilateral inferior parietal cortex during multiple-object processing. Single-trial analysis revealed greater burst occurrences on object individuation versus ensemble grouping trials. By contrast, we found no differences using standard analyses on across-trials averaged α-band power. Moreover, the bursting effects occurred only below/at, but not above, the typical capacity limits for multiple-object processing (at ∼4 objects). Our findings reveal the real-time neural correlates underlying the dynamic processing of multiple-object scenarios, which are modulated by grouping strategies and capacity. They support a rhythmic, α-pulsed organization of dynamic attention to multiple objects and ensembles.SIGNIFICANCE STATEMENT Dynamic multiple-object scenarios are an important problem in real-world and computer vision. They require keeping track of multiple objects as they move through space and time. Such problems can be solved in two ways: One can individuate a scene object by object, or alternatively group objects into ensembles. We observed greater occurrences of α-oscillatory burst events in parietal cortex for processing objects versus ensembles and below/at versus above processing capacity. These results demonstrate a unique top-down mechanism by which the brain dynamically adjusts its computational level between objects and ensembles. They help to explain how the brain copes with its capacity limitations in real-time environments and may lead the way to technological innovations for time-critical video analysis in computer vision.

Perceptual learning of ensemble and outlier perception

Previous studies have demonstrated a complex relationship between ensemble perception and outlier detection. We presented two array of heterogeneously oriented stimulus bars and different mean orientations and/or a bar with an outlier orientation, asking participants to discriminate the mean orientations or detect the outlier. Perceptual learning was found in every case, with improved performance accuracy and speeded responses. Testing for improved accuracy through cross-task transfer, we found considerable transfer from training outlier detection to mean discrimination performance, and none in the opposite direction. Implicit learning in terms of increased accuracy was not found in either direction when participants performed one task, and the second task's stimulus features were present. Reaction time improvement was found to transfer in all cases. This study adds to the already broad knowledge concerning perceptual learning and cross-task transfer of training effects.

Encoding perceptual ensembles during visual search in peripheral vision

Observers can learn complex statistical properties of visual ensembles, such as their probability distributions. Even though ensemble encoding is considered critical for peripheral vision, whether observers learn such distributions in the periphery has not been studied. Here, we used a visual search task to investigate how the shape of distractor distributions influences search performance and ensemble encoding in peripheral and central vision. Observers looked for an oddly oriented bar among distractors taken from either uniform or Gaussian orientation distributions with the same mean and range. The search arrays were either presented in the foveal or peripheral visual fields. The repetition and role reversal effects on search times revealed observers' internal model of distractor distributions. Our results showed that the shape of the distractor distribution influenced search times only in foveal, but not in peripheral search. However, role reversal effects revealed that the shape of the distractor distribution could be encoded peripherally depending on the interitem spacing in the search array. Our results suggest that, although peripheral vision might rely heavily on summary statistical representations of feature distributions, it can also encode information about the distributions themselves.

Value associations bias ensemble perception

Ensemble perception refers to awareness of average properties, e.g. size, of “noisy” elements that often comprise visual arrays in natural scenes. Here, we asked how ensemble perception might be influenced when some but not all array elements are associated with monetary reward. Previous studies show that reward associations can speed object processing, facilitate selection, and enhance working-memory maintenance, suggesting they may bias ensemble judgments. To investigate, participants reported the average element size of brief arrays of different-sized circles. In the learning phase, all circles had the same color, but different colors produced high or low performance-contingent rewards. Then, in an unrewarded test phase, arrays comprised three spatially inter-mixed subsets, each with a different color, including the high-reward color. In different trials, the mean size of the subset with the high-reward color was smaller, larger, or the same as the ensemble mean. Ensemble size estimates were significantly biased by the high-reward-associated subset, showing that value associations modulate ensemble perception. In the test phase of a second experiment, a pattern mask appeared immediately after array presentation to limit top-down processing. Not only was value-biasing eliminated, ensemble accuracy improved, suggesting that value associations distort consciously available ensemble representation via late high-level processing.

Roles of saliency and set size in ensemble averaging

Ensemble statistics are often thought of as a reliable impression of numerous items despite limited capacities to consciously represent each individual. However, whether all items equally contribute to ensemble summaries (e.g., mean) and whether they might be affected by known limited-capacity processes, such as focused attention, is still debated. We addressed these questions via a recently described "amplification effect," a systematic bias of perceived mean (e.g., average size) towards the more salient "tail" of a feature distribution (e.g., larger items). In our experiments, observers adjusted the mean orientation of sets of items varying in set size. We made some of the items more salient or less salient by changing their size. While the whole orientation distribution was fixed, the more salient subset could be shifted relative to the set mean or differ in range. We measured the bias away from the set mean and the standard deviation (SD) of errors, as it is known to reflect the physical range from which ensemble information is sampled. We found that bias and SD changes followed the shifts and range changes in salient subsets, providing evidence for amplification. However, these changes were weaker than those expected from sampling only salient items, suggesting that less salient items were also sampled. Importantly, the SD decreased as a function of set size, which is only possible if the number of sampled elements increased with set size. Overall, we conclude that orientation summary statistics are sampled from an entire ensemble and modulated by the amplification effect of attention.

Emotional judgments of scenes are influenced by unintentional averaging

Background

The visual system uses ensemble perception to summarize visual input across a variety of domains. This heuristic operates at multiple levels of vision, compressing information as basic as oriented lines or as complex as emotional faces. Given its pervasiveness, the ensemble unsurprisingly can influence how an individual item is perceived, and vice versa.

Methods

In the current experiments, we tested whether the perceived emotional valence of a single scene could be influenced by surrounding, simultaneously presented scenes. Observers first rated the emotional valence of a series of individual scenes. They then saw ensembles of the original images, presented in sets of four, and were cued to rate, for a second time, one of four.

Results

Results confirmed that the perceived emotional valence of the cued image was pulled toward the mean emotion of the surrounding ensemble on the majority of trials, even though the ensemble was task-irrelevant. Control experiments and analyses confirmed that the pull was driven by high-level, ensemble information.

Conclusion

We conclude that high-level ensemble information can influence how we perceive individual items in a crowd, even when working memory demands are low and the ensemble information is not directly task-relevant.

Demystifying visual awareness: Peripheral encoding plus limited decision complexity resolve the paradox of rich visual experience and curious perceptual failures

Human beings subjectively experience a rich visual percept. However, when behavioral experiments probe the details of that percept, observers perform poorly, suggesting that vision is impoverished. What can explain this awareness puzzle? Is the rich percept a mere illusion? How does vision work as well as it does? This paper argues for two important pieces of the solution. First, peripheral vision encodes its inputs using a scheme that preserves a great deal of useful information, while losing the information necessary to perform certain tasks. The tasks rendered difficult by the peripheral encoding include many of those used to probe the details of visual experience. Second, many tasks used to probe attentional and working memory limits are, arguably, inherently difficult, and poor performance on these tasks may indicate limits on decision complexity. Two assumptions are critical to making sense of this hypothesis: (1) All visual perception, conscious or not, results from performing some visual task; and (2) all visual tasks face the same limit on decision complexity. Together, peripheral encoding plus decision complexity can explain a wide variety of phenomena, including vision's marvelous successes, its quirky failures, and our rich subjective impression of the visual world.

Color ensembles: Sampling and averaging spatial hue distributions

Color serves both to segment a scene into objects and background and to identify objects. Although objects and surfaces usually contain multiple colors, humans can readily extract a representative color description, for instance, that tomatoes are red and bananas yellow. The study of color discrimination and identification has a long history, yet we know little about the formation of summary representations of multicolored stimuli. Here, we characterize the human ability to integrate hue information over space for simple color stimuli varying in the amount of information, stimulus size, and spatial configuration of stimulus elements. We show that humans are efficient at integrating hue information over space beyond what has been shown before for color stimuli. Integration depends only on the amount of information in the display and not on spatial factors such as element size or spatial configuration in the range measured. Finally, we find that observers spontaneously prefer a simple averaging strategy even with skewed color distributions. These results shed light on how human observers form summary representations of color and make a link between the perception of polychromatic surfaces and the broader literature of ensemble perception.

One bad apple spoils the whole bushel: The neural basis of outlier processing

How are outliers in an otherwise homogeneous object ensemble represented by our visual system? Are outliers ignored because they are the minority? Or do outliers alter our perception of an otherwise homogeneous ensemble? We have previously demonstrated ensemble representation in human anterior-medial ventral visual cortex (overlapping the scene-selective parahippocampal place area; PPA). In this study we investigated how outliers impact object-ensemble representation in this human brain region as well as visual representation throughout posterior brain regions. We presented a homogeneous ensemble followed by an ensemble containing either identical elements or a majority of identical elements with a few outliers. Human participants ignored the outliers and made a same/different judgment between the two ensembles. In PPA, fMRI adaptation was observed when the outliers in the second ensemble matched the items in the first, even though the majority of the elements in the second ensemble were distinct from those in the first; conversely, release from fMRI adaptation was observed when the outliers in the second ensemble were distinct from the items in the first, even though the majority of the elements in the second ensemble were identical to those in the first. A similarly robust outlier effect was also found in other brain regions, including a shape-processing region in lateral occipital cortex (LO) and task-processing fronto-parietal regions. These brain regions likely work in concert to flag the presence of outliers during visual perception and then weigh the outliers appropriately in subsequent behavioral decisions. To our knowledge, this is the first time the neural mechanisms involved in outlier processing have been systematically documented in the human brain. Such an outlier effect could well provide the neural basis mediating our perceptual experience in situations like "one bad apple spoils the whole bushel".

Five dichotomies in the psychophysics of ensemble perception

Whereas psychophysicists may formulate hypotheses about appearance, they can only measure performance. Bias and imprecision in psychophysical data need not necessarily reflect bias and imprecision in perception. Sensory systems may exaggerate the differences between each item and its neighbors in an ensemble. Alternatively, sensory systems may homogenize the ensemble, thereby removing any apparent differences between neighboring items. Ensemble perception may be involuntary when observers attempt to report the identities of individual items. Conversely, when asked to make a (voluntary) decision about the ensemble as a whole, observers may find it very difficult to compute statistics that are based on more than a very small number of individual items. Modeling decisions about prothetic continua, such as size and contrast, can be tricky because sensory signals may be distorted before and/or after voluntarily computing ensemble statistics. With metathetic continua, such as spatial orientation, distortion is less problematic; physically vertical things necessarily appear close to vertical and physically horizontal things necessarily appear close to horizontal. Decision processes are corrupted by noise that, like distortion, may be added to sensory signals prior to and/or after voluntarily computing ensemble statistics.

How does the human visual system compare the speeds of spatially separated objects?

We measured psychophysical thresholds for discriminating the speeds of two arrays of moving dots. The arrays could be juxtaposed or could be spatially separated by up to 10 degrees of visual angle, eccentricity being held constant. We found that the precision of the judgments varied little with separation. Moreover, the function relating threshold to separation was similar whether the arrays moved in the same, in opposite or in orthogonal directions. And there was no significant difference in threshold whether the two stimuli were initially presented to the same cerebral hemisphere or to opposite ones. How are human observers able to compare stimuli that fall at well separated positions in the visual field? We consider two classes of explanation: (i) Observers' judgments might be based directly on the signals of dedicated 'comparator neurons', i.e. neurons drawing inputs of opposite sign from local regions of the visual field. (ii) Signals about local features might be transmitted to the site of comparison by a shared 'cerebral bus', where the same physical substrate carries different information from moment to moment. The minimal effects of proximity and direction (which might be expected to influence local detectors of relative motion), and the combinatorial explosion in the number of comparator neurons that would be required by (i), lead us to favor models of type (ii).

Color variance and achromatic settings

The average color in a scene is a potentially important cue to the illuminant and thus for color constancy, but it remains unknown how well and in what ways observers can estimate the mean chromaticity. We examined this by measuring the variability in "achromatic" settings for stimuli composed of different distributions of colors with varying contrast ranges along the luminance, SvsLM, and LvsM cardinal axes. Observers adjusted the mean chromaticity of the palette to set the average to gray. Variability in the settings increased as chromatic contrast or (to a lesser extent) luminance contrast increased. Signals along the cardinal axes are relatively independent in many detection and discrimination tasks, but showed strong interference in the white estimates. This "cross-masking" and the effects of chromatic variance in general may occur because observers cannot explicitly perceive or represent the mean of a set of qualitatively different hues (e.g., that red and green hues average to gray), and thus may infer the mean only indirectly (e.g., from the relative saturation of different hues).

The Verriest Lecture: Adventures in blue and yellow

Conventional models of color vision assume that blue and yellow (along with red and green) are the fundamental building blocks of color appearance, yet how these hues are represented in the brain and whether and why they might be special are questions that remain shrouded in mystery. Many studies have explored the visual encoding of color categories, from the statistics of the environment to neural processing to perceptual experience. Blue and yellow are tied to salient features of the natural color world, and these features have likely shaped several important aspects of color vision. However, it remains less certain that these dimensions are encoded as primary or "unique" in the visual representation of color. There are also striking differences between blue and yellow percepts that may reflect high-level inferences about the world, specifically about the colors of light and surfaces. Moreover, while the stimuli labeled as blue or yellow or other basic categories show a remarkable degree of constancy within the observer, they all vary independently of one another across observers. This pattern of variation again suggests that blue and yellow and red and green are not a primary or unitary dimension of color appearance, and instead suggests a representation in which different hues reflect qualitatively different categories rather than quantitative differences within an underlying low-dimensional "color space."

Ensemble perception and focused attention: Two different modes of visual processing to cope with limited capacity

The visual system has a limited capacity for dealing with complex and redundant information in a scene. Here, we propose that a distributed attention mode of processing is necessary for coping with this limit, together with a focused attention mode of processing. The distributed attention mode provides a statistical summary of a scene, whereas the focused attention mode provides relevant information for object recognition. In this paper, we claim that a distributed mode of processing is necessary because (1) averaging performance improves with increased set-sizes, (2) even unselected items are likely to contribute to averaging, and (3) the assumption of variable capacity limits in averaging over different set-sizes is not plausible. We then propose how the averaging process can access multiple items over the capacity limit of focused attention. The visual system can represent multiple items as population responses and read out relevant information using the two modes of attention. It can summarize population responses with a broad application of a Gaussian profile (i.e., distributed attention) and represent its peak as the mean. It can focus on relevant population responses with a narrow application of a Gaussian profile (i.e., focused attention) and select important information for object recognition. The two attention modes of processing provide a framework for incorporating two seemingly opposing fields of study (ensemble perception and selective attention) and a unified theory of a coping strategy with our limited capacity.

Extra-foveal Processing of Object Semantics Guides Early Overt Attention During Visual Search

Eye-tracking studies using arrays of objects have demonstrated that some high-level processing of object semantics can occur in extra-foveal vision, but its role on the allocation of early overt attention is still unclear. This eye-tracking visual search study contributes novel findings by examining the role of object-to-object semantic relatedness and visual saliency on search responses and eye-movement behaviour across arrays of increasing size (3, 5, 7). Our data show that a critical object was looked at earlier and for longer when it was semantically unrelated than related to the other objects in the display, both when it was the search target (target-present trials) and when it was a target's semantically related competitor (target-absent trials). Semantic relatedness effects manifested already during the very first fixation after array onset, were consistently found for increasing set sizes, and were independent of low-level visual saliency, which did not play any role. We conclude that object semantics can be extracted early in extra-foveal vision and capture overt attention from the very first fixation. These findings pose a challenge to models of visual attention which assume that overt attention is guided by the visual appearance of stimuli, rather than by their semantics.

Independent and parallel visual processing of ensemble statistics: Evidence from dual tasks

The visual system can represent multiple objects in a compressed form of ensemble summary statistics (such as object numerosity, mean, and feature variance/range). Yet the relationships between the different types of visual statistics remain relatively unclear. Here, we tested whether two summaries (mean and numerosity, or mean and range) are calculated independently from each other and in parallel. Our participants performed dual tasks requiring a report about two summaries in each trial, and single tasks requiring a report about one of the summaries. We estimated trial-by-trial correlations between the precision of reports as well as correlations across observers. Both analyses showed the absence of correlations between different types of ensemble statistics, suggesting their independence. We also found no decrement (except that related to the order of report explained by memory retrieval) in performance in dual compared to single tasks, which suggests that two statistics of one ensemble can be processed in parallel.

Perception and decision mechanisms involved in average estimation of spatiotemporal ensembles

A number of studies on texture and ensemble perception have shown that humans can immediately estimate the average of spatially distributed visual information. The present study characterized mechanisms involved in estimating averages for information distributed over both space and time. Observers viewed a rapid sequence of texture patterns in which elements' orientation were determined by dynamic Gaussian noise with variable spatial and temporal standard deviations (SDs). We found that discrimination thresholds increased beyond a certain spatial SD if temporal SD was small, but if temporal SD was large, thresholds remained nearly constant regardless of spatial SD. These data are at odds with predictions that threshold is uniquely determined by spatiotemporal SD. Moreover, a reverse correlation analysis revealed that observers judged the spatiotemporal average orientation largely depending on the spatial average orientation over the last few frames of the texture sequence - a recency effect widely observed in studies of perceptual decision making. Results are consistent with the notion that the visual system rapidly computes spatial ensembles and adaptively accumulates information over time to make a decision on spatiotemporal average. A simple computational model based on this notion successfully replicated observed data.

Adaptation to mean and variance: Interrelationships between mean and variance representations in orientation perception

When there are many visual items, the visual system could represent their summary statistics (e.g., mean, variance) to process them efficiently. Although many previous studies have investigated the mean or variance representation itself, a relationship between these two ensemble representations has not been investigated much. In this study, we tested the potential interaction between mean and variance representations by using a visual adaptation method. We reasoned that if mean and variance representations interact with each other, an adaptation aftereffect to either mean or variance would influence the perception of the other. Participants watched a sequence of orientation arrays containing a specific statistical property during the adaptation period. To produce an adaptation aftereffect specific to variance or mean, one property of the adaptor arrays (variance or mean) had a fixed value while the other property was randomly varied. After the adaptation, participants were asked to discriminate the property of the test array that was randomly varied during the adaptation. We found that the adaptation aftereffect of orientation variance influenced the sensitivity of mean orientation discrimination (Experiment 1), and that the adaptation aftereffect of mean orientation influenced the bias of orientation variance discrimination (Experiment 2). These results suggest that mean and variance representations do closely interact with each other. Considering that mean and variance reflect the representative value and dispersion of multiple items respectively, the interactions between mean and variance representations may reflect their complementary roles to summarize complex visual information effectively.

The Perceptual Proxies of Visual Comparison

Perceptual tasks in visualizations often involve comparisons. Of two sets of values depicted in two charts, which set had values that were the highest overall? Which had the widest range? Prior empirical work found that the performance on different visual comparison tasks (e.g., "biggest delta", "biggest correlation") varied widely across different combinations of marks and spatial arrangements. In this paper, we expand upon these combinations in an empirical evaluation of two new comparison tasks: the "biggest mean" and "biggest range" between two sets of values. We used a staircase procedure to titrate the difficulty of the data comparison to assess which arrangements produced the most precise comparisons for each task. We find visual comparisons of biggest mean and biggest range are supported by some chart arrangements more than others, and that this pattern is substantially different from the pattern for other tasks. To synthesize these dissonant findings, we argue that we must understand which features of a visualization are actually used by the human visual system to solve a given task. We call these perceptual proxies. For example, when comparing the means of two bar charts, the visual system might use a "Mean length" proxy that isolates the actual lengths of the bars and then constructs a true average across these lengths. Alternatively, it might use a "Hull Area" proxy that perceives an implied hull bounded by the bars of each chart and then compares the areas of these hulls. We propose a series of potential proxies across different tasks, marks, and spatial arrangements. Simple models of these proxies can be empirically evaluated for their explanatory power by matching their performance to human performance across these marks, arrangements, and tasks. We use this process to highlight candidates for perceptual proxies that might scale more broadly to explain performance in visual comparison.