Information limit on the spatial integration of local orientation signals

Consistent metacognitive efficiency and variable response biases in peripheral vision

Across the visual periphery, perceptual and metacognitive abilities differ depending on the locus of visual attention, the location of peripheral stimulus presentation, the task design, and many other factors. In this investigation, we aimed to illuminate the relationship between attention and eccentricity in the visual periphery by estimating perceptual sensitivity, metacognitive sensitivity, and response biases across the visual field. In a 2AFC detection task, participants were asked to determine whether a signal was present or absent at one of eight peripheral locations (±10°, 20°, 30°, and 40°), using either a valid or invalid attentional cue. As expected, results revealed that perceptual sensitivity declined with eccentricity and was modulated by attention, with higher sensitivity on validly cued trials. Furthermore, a significant main effect of eccentricity on response bias emerged, with variable (but relatively unbiased) c'a values from 10° to 30°, and conservative c'a values at 40°. Regarding metacognitive sensitivity, significant main effects of attention and eccentricity were found, with metacognitive sensitivity decreasing with eccentricity, and decreasing in the invalid cue condition. Interestingly, metacognitive efficiency, as measured by the ratio of meta-d'a/d'a, was not modulated by attention or eccentricity. Overall, these findings demonstrate (1) that in some circumstances, observers have surprisingly robust metacognitive insights into how performance changes across the visual field and (2) that the periphery may be subject to variable detection biases that are contingent on the exact location in peripheral space.

The Real-World Usability, Feasibility, and Performance Distributions of Deploying a Digital Toolbox of Computerized Assessments to Remotely Evaluate Brain Health: Development and Usability Study

BACKGROUND

An ongoing global challenge is managing brain health and understanding how performance changes across the lifespan.

OBJECTIVE

We developed and deployed a set of self-administrable, computerized assessments designed to measure key indexes of brain health across the visual and auditory sensory modalities. In this pilot study, we evaluated the usability, feasibility, and performance distributions of the assessments in a home-based, real-world setting without supervision.

METHODS

Potential participants were untrained users who self-registered on an existing brain training app called BrainHQ. Participants were contacted via a recruitment email and registered remotely to complete a demographics questionnaire and 29 unique assessments on their personal devices. We examined participant engagement, descriptive and psychometric properties of the assessments, associations between performance and self-reported demographic variables, cognitive profiles, and factor loadings.

RESULTS

Of the 365,782 potential participants contacted via a recruitment email, 414 (0.11%) registered, of whom 367 (88.6%) completed at least one assessment and 104 (25.1%) completed all 29 assessments. Registered participants were, on average, aged 63.6 (SD 14.8; range 13-107) years, mostly female (265/414, 64%), educated (329/414, 79.5% with a degree), and White (349/414, 84.3% White and 48/414, 11.6% people of color). A total of 72% (21/29) of the assessments showed no ceiling or floor effects or had easily modifiable score bounds to eliminate these effects. When correlating performance with self-reported demographic variables, 72% (21/29) of the assessments were sensitive to age, 72% (21/29) of the assessments were insensitive to gender, 93% (27/29) of the assessments were insensitive to race and ethnicity, and 93% (27/29) of the assessments were insensitive to education-based differences. Assessments were brief, with a mean duration of 3 (SD 1.0) minutes per task. The pattern of performance across the assessments revealed distinctive cognitive profiles and loaded onto 4 independent factors.

CONCLUSIONS

The assessments were both usable and feasible and warrant a full normative study. A digital toolbox of scalable and self-administrable assessments that can evaluate brain health at a glance (and longitudinally) may lead to novel future applications across clinical trials, diagnostics, and performance optimization.

Mean orientation discrimination based on proximal stimuli

Ensemble perception refers to the ability to accurately and rapidly perceive summary statistical representations of specific features from a group of similar objects. However, the specific type of representation involved in this perception within a three-dimensional (3-D) environment remains unclear. In the context of perspective viewing with stereopsis, distal stimuli can be projected onto the retina as different forms of proximal stimuli based on their distances, despite sharing similar properties, such as object size and spatial frequency. This study aimed to investigate the effects of distal and proximal stimuli on the perception of summary statistical information related to orientation. In our experiment, we presented multiple Gabor patches in a stereoscopic environment, allowing us to measure the discrimination threshold of the mean orientation. The object size and spatial frequency were fixed for all patches regardless of depth. However, the physical angular size and absolute spatial frequency covaried with the depth. The results revealed the threshold elevation with depth expansion, especially when the patches formed two clusters at near and far distances, leading to large variations in their retinotopic representations. This finding indicates a minor contribution of similarity of the distal stimuli. Subsequent experiments demonstrated that the variability in physical angular size of the patches significantly influenced the threshold elevation in contrast to that of binocular disparity and absolute spatial frequency. These findings highlight the critical role of physical angular size variability in perceiving mean orientations within the 3-D space.

What is the basis of ensemble subset selection?

The visual system can rapidly calculate the ensemble statistics of a set of objects; for example, people can easily estimate an average size of apples on a tree. To accomplish this, it is not always useful to summarize all the visual information. If there are various types of objects, the visual system should select a relevant subset: only apples, not leaves and branches. Here, we ask what kind of visual information makes a "good" ensemble that can be selectively attended to provide an accurate summary estimate. We tested three candidate representations: basic features, preattentive object files, and full-fledged bound objects. In four experiments, we presented a target and several distractors' sets of differently colored objects. We found that conditions where a target ensemble had at least one unique color (basic feature) provided ensemble averaging performance comparable to the baseline displays without distractors. When the target subset was defined as a conjunction of two colors or color-shape partly shared with distractors (so that they could be differentiated only as preattentive object files), subset averaging was also possible but less accurate than in the baseline and feature conditions. Finally, performance was very poor when the target subset was defined by an exact feature relationship, such as in the spatial conjunction of two colors (spatially bound object). Overall, these results suggest that distinguishable features and, to a lesser degree, preattentive object files can serve as the representational basis of ensemble selection, while bound objects cannot.

The functional role of spatial anisotropies in ensemble perception

BACKGROUND

The human brain can rapidly represent sets of similar stimuli by their ensemble summary statistics, like the average orientation or size. Classic models assume that ensemble statistics are computed by integrating all elements with equal weight. Challenging this view, here, we show that ensemble statistics are estimated by combining parafoveal and foveal statistics in proportion to their reliability. In a series of experiments, observers reproduced the average orientation of an ensemble of stimuli under varying levels of visual uncertainty.

RESULTS

Ensemble statistics were affected by multiple spatial biases, in particular, a strong and persistent bias towards the center of the visual field. This bias, evident in the majority of subjects and in all experiments, scaled with uncertainty: the higher the uncertainty in the ensemble statistics, the larger the bias towards the element shown at the fovea.

CONCLUSION

Our findings indicate that ensemble perception cannot be explained by simple uniform pooling. The visual system weights information anisotropically from both the parafovea and the fovea, taking the intrinsic spatial anisotropies of vision into account to compensate for visual uncertainty.

A signal-detection account of item-based and ensemble-based visual change detection: A reply to Harrison, McMaster, and Bays

Growing empirical evidence shows that ensemble information (e.g., the average feature or feature variance of a set of objects) affects visual working memory for individual items. Recently, Harrison, McMaster, and Bays (2021) used a change detection task to test whether observers explicitly rely on ensemble representations to improve their memory for individual objects. They found that sensitivity to simultaneous changes in all memorized items (which also globally changed set summary statistics) rarely exceeded a level predicted by the so-called optimal summation model within the signal-detection framework. This model implies simple integration of evidence for change from all individual items and no additional evidence coming from ensemble. Here, we argue that performance at the level of optimal summation does not rule out the use of ensemble information. First, in two experiments, we show that, even if evidence from only one item is available at test, the statistics of the whole memory set affect performance. Second, we argue that optimal summation itself can be conceptually interpreted as one of the strategies of holistic, ensemble-based decision. We also redefine the reference level for the item-based strategy as the so-called "minimum rule," which predicts performance far below the optimum. We found that that both our and Harrison et al. (2021)'s observers consistently outperformed this level. We conclude that observers can rely on ensemble information when performing visual change detection. Overall, our work clarifies and refines the use of signal-detection analysis in measuring and modeling working memory.

Estimating lighting direction in scenes with multiple objects

To recover the reflectance and shape of an object in a scene, the human visual system must account for the properties of the light illuminating the object. Here, we examine the extent to which multiple objects within a scene are utilised to estimate the direction of lighting in a scene. In Experiment 1, we presented participants with rendered scenes that contained 1, 9, or 25 unfamiliar blob-like objects and measured their capacity to discriminate whether a directional light source was left or right of the participants' vantage point. Trends reported for ensemble perception suggest that the number of utilised objects-and, consequently, discrimination sensitivity-would increase with set size. However, we find little indication that increasing the number of objects in a scene increased discrimination sensitivity. In Experiment 2, an equivalent noise analysis was used to measure participants' internal noise and the number of objects used to judge the average light source direction in a scene, finding that participants relied on 1 or 2 objects to make their judgement regardless of whether 9 or 25 objects were present. In Experiment 3, participants completed a shape identification task that required an implicit judgement of light source direction, rather than an explicit judgement as in Experiments 1 and 2. We find that sensitivity for identifying surface shape was comparable for scenes containing 1, 9, and 25 objects. Our results suggest that the visual system relied on a small number of objects to estimate the direction of lighting in our rendered scenes.

The influence of natural image statistics on upright orientation judgements

Humans have well-documented priors for many features present in nature that guide visual perception. Despite being putatively grounded in the statistical regularities of the environment, scene priors are frequently violated due to the inherent variability of visual features from one scene to the next. However, these repeated violations do not appreciably challenge visuo-cognitive function, necessitating the broad use of priors in conjunction with context-specific information. We investigated the trade-off between participants' internal expectations formed from both longer-term priors and those formed from immediate contextual information using a perceptual inference task and naturalistic stimuli. Notably, our task required participants to make perceptual inferences about naturalistic images using their own internal criteria, rather than making comparative judgements. Nonetheless, we show that observers' performance is well approximated by a model that makes inferences using a prior for low-level image statistics, aggregated over many images. We further show that the dependence on this prior is rapidly re-weighted against contextual information, even when misleading. Our results therefore provide insight into how apparent high-level interpretations of scene appearances follow from the most basic of perceptual processes, which are grounded in the statistics of natural images.

What's special about horizontal disparity

Horizontal disparity has been recognized as the primary signal driving stereoscopic depth since the invention of the stereoscope in the 1830s. It has a unique status in our understanding of binocular vision. The direction of offset of the eyes gives the disparities of corresponding image point locations across the two retinas a strong horizontal bias. Beyond the retina, other factors give shape to the effective disparity direction used by visual mechanisms. The influence of orientation is examined here. I argue that horizontal disparity is an inflection point along a continuum of effective directions, and its role in stereo vision can be reinterpreted. The pointwise geometric justification for its special status neglects the oriented structural elements of spatial vision, its physiological support is equivocal, and psychophysical support of its special status may partially reflect biased stimulus sampling. The literature shows that horizontal disparity plays no particular role in the processing of one-dimensional stimuli, a reflection of the stereo aperture problem. The resulting depth is non-veridical, even non-transitive. Although one-dimensional components contribute to the stereo depth of visual objects generally, two-dimensional stimuli appear not to inherit the aperture problem. However, a look at the two-dimensional stimuli that predominate in experimental studies shows regularities in orientation that give a new perspective on horizontal disparity.

Amblyopic stereo vision is efficient but noisy

People with amblyopia demonstrate a reduced ability to judge depth using stereopsis. Our understanding of this deficit is limited, as standard clinical stereo tests may not be suited to give a quantitative account of the residual stereo ability in amblyopia. In this study we used a stereo test designed specifically for that purpose. Participants identified the location of a disparity-defined odd-one-out target within a random-dot display. We tested 29 amblyopic (3 strabismic, 17 anisometropic, 9 mixed) participants and 17 control participants. We obtained stereoacuity thresholds from 59% of our amblyopic participants. There was a factor of two difference between the median stereoacuity of our amblyopic (103 arcsec) and control (56 arcsec) groups. We used the equivalent noise method to evaluate the role of equivalent internal noise and processing efficiency in amblyopic stereopsis. Using the linear amplifier model (LAM), we determined the threshold difference was due to a greater equivalent internal noise in the amblyopic group (238 vs 135 arcsec), with no significant difference in processing efficiency. A multiple linear regression determined 56% of the stereoacuity variance within the amblyopic group was predicted by the two LAM parameters, with equivalent internal noise predicting 46% alone. Analysis of control group data aligned with our previous work, finding that trade-offs between equivalent internal noise and efficiency play a greater role. Our results allow a better understanding of what is limiting amblyopic performance in our task. We find this to be a reduced quality of disparity signals in the input to the task-specific processing.

Perceptual history biases in serial ensemble representation

Ensemble perception refers to the visual system's ability to efficiently represent groups of similar objects as a unified percept using their summary statistical information. Most studies focused on extraction of current trial averages, giving little attention to prior experience effects, although a few recent studies found that ensemble mean estimations contract toward previously presented stimuli, with most of these focusing on explicit perceptual averaging of simultaneously presented item ensembles. Yet, the time element is crucial in real dynamic environments, where we encounter ensemble items over time, aggregating information until reaching summary representations. Moreover, statistical information of objects and scenes is learned over time and often implicitly and then used for predictions that shape perception, promoting environmental stability. Therefore, we now focus on temporal aspects of ensemble statistics and test whether prior information, beyond the current trial, biases implicit perceptual decisions. We designed methods to separate current trial biases from those of previously seen trial ensembles. In each trial, six circles of different sizes were presented serially, followed by two test items. Participants were asked to choose which was present in the sequence. Participants unconsciously rely on ensemble statistics, choosing stimuli closer to the ensemble mean. To isolate the influence of earlier trials, the two test items were sometimes equidistant from the current trial mean. Results showed membership judgment biases toward current trial mean, when informative (largest effect). On equidistant trials, judgments were biased toward previously experienced stimulus statistics. Comparison of similar conditions with a shifted stimulus distribution ruled out a bias toward an earlier, presession, prototypical diameter. We conclude that ensemble perception, even for temporally experienced ensembles, is influenced not only by current trial mean but also by means of recently seen ensembles and that these influences are somewhat correlated on a participant-by-participant basis.

Context consistency improves ensemble perception of facial expressions

Humans have developed the capacity to rapidly extract summary statistics from the facial expressions of a crowd, such as computing the average facial expression. Although dual-task paradigms involving memory and ensemble tasks have recently found that this ensemble coding ability is biased by visual working memory, few studies have examined whether the context-dependent nature of memory itself can influence the perceptual averaging process. In two experiments, participants made forced-choice judgments about mean facial expressions that were paired with task-irrelevant background images, and the background images either matched or mismatched across encoding and response phases. When the backgrounds matched, it was at either the perceptual level (uniformly oriented lines with the same orientation in encoding and response phases, in Experiment 1), or at the summary statistics level (uniformly oriented lines in the response phase that had the same orientation as the mean of randomly oriented lines that were seen in the encoding phase, in Experiment 2). Participants in Experiment 1 showed a higher ensemble precision and better discrimination sensitivity when the backgrounds matched than when they mismatched, which is consistent with the kind of robust contextual memory effect that has been seen in prior research. We further demonstrated that the context-matching facilitation effect occurred at both the perceptual level (Experiment 1) and at the summary statistics level (Experiment 2). These results demonstrate that the effects of visual working memory on perceptual averaging are obligatory, and they highlight the importance of memory-related context dependency in perceptual averaging.

Ensemble averaging: What can we learn from skewed feature distributions?

Many studies have shown that observers can accurately estimate the average feature of a group of objects. However, the way the visual system relies on the information from each individual item is still under debate. Some models suggest some or all items sampled and averaged arithmetically. Another strategy implies "robust averaging," when middle elements gain greater weight than outliers. One version of a robust averaging model was recently suggested by Teng et al. (2021), who studied motion direction averaging in skewed feature distributions and found systematic biases toward their modes. They interpreted these biases as evidence for robust averaging and suggested a probabilistic weighting model based on minimization of the virtual loss function. In four experiments, we replicated systematic skew-related biases in another feature domain, namely, orientation averaging. Importantly, we show that the magnitude of the bias is not determined by the locations of the mean or mode alone, but is substantially defined by the shape of the whole feature distribution. We test a model that accounts for such distribution-dependent biases and robust averaging in a biologically plausible way. The model is based on well-established mechanisms of spatial pooling and population encoding of local features by neurons with large receptive fields. Both the loss functions model and the population coding model with a winner-take-all decoding rule accurately predicted the observed patterns, suggesting that the pooled population response model can be considered a neural implementation of the computational algorithms of information sampling and robust averaging in ensemble perception.

Foveal vision determines the perceived emotion of face ensembles

People can extract summary statistical information from groups of similar objects, an ability called ensemble perception. However, not every object in a group is weighted equally. For example, in ensemble emotion perception, faces far from fixation were weighted less than faces close to fixation. Yet the contribution of foveal input in ensemble emotion perception is still unclear. In two experiments, groups of faces with varying emotions were presented for 100 ms at three different eccentricities (0°, 3°, 8°). Observers reported the perceived average emotion of the group. In two conditions, stimuli consisted of a central face flanked by eight faces (flankers) (central-present condition) and eight faces without the central face (central-absent condition). In the central-present condition, the emotion of the central face was either congruent or incongruent with that of the flankers. In Experiment 1, flanker emotions were uniform (identical flankers); in Experiment 2 they were varied. In both experiments, performance in the central-present condition was superior at 3° compared to 0° and 8°. At 0°, performance was superior in the central-absent (i.e., no foveal input) compared to the central-present condition. Poor performance in the central-present condition was driven by the incongruent condition where the foveal face strongly biased responses. At 3° and 8°, performance was comparable between central-present and central-absent conditions. Our results showed how foveal input determined the perceived emotion of face ensembles, suggesting that ensemble perception fails when salient target information is available in central vision.

On the flexibility of strategies for center estimation

When subjects are asked to indicate the center of a spatially distributed stimulus, the features that control their responses tend to vary (1) across subjects and (2) as stimulus properties are altered. Here we ask: can subjects bring these different response tendencies under top-down control? In each of three tasks, all using briefly displayed, Gaussian dot-clouds, subjects were trained to perform different center-estimation responses. In the "mass task," the target was the centroid of the dots. In the "hull task," the target was the centroid of the region circumscribed by the convex hull of the dot-cloud. In the "hull-vertex task," the target was the centroid of the vertices of the convex hull. Subjects were able to perform each of the mass- and hull-tasks accurately and reliably. However, they found the hull-vertex task more difficult; errors were substantially larger in this task, and responses tended to be closer to both of the hull- and mass-task centers. The finding that subjects can intentionally target either the centroid of the dots in the stimulus or the centroid of the stimulus convex hull suggests that individual differences in feedback-free experiments may reflect idiosyncratic decisions by different subjects about what combination of these statistics to use in responding.

Internal noise measures in coarse and fine motion direction discrimination tasks and the correlation with autism traits

Motion perception is essential for visual guidance of behavior and is known to be limited by both internal additive noise (i.e., a constant level of random fluctuations in neural activity independent of the stimulus) and motion pooling (global integration of local motion signals across space). People with autism spectrum disorder (ASD) display abnormalities in motion processing, which have been linked to both elevated noise and abnormal pooling. However, to date, the impact of a third limit—induced internal noise (internal noise that scales up with increases in external stimulus noise)—has not been investigated in motion perception of any group. Here, we describe an extension on the double-pass paradigm to quantify additive noise and induced noise in a motion paradigm. We also introduce a new way to experimentally estimate motion pooling. We measured the impact of induced noise on direction discrimination, which we ascribe to fluctuations in decision-related variables. Our results are suggestive of higher internal noise in individuals with high ASD traits only on coarse but not fine motion direction discrimination tasks. However, we report no significant correlations between autism traits and additive noise, induced noise, or motion pooling in either task. We conclude that, under some conditions, the internal noise may be higher in individuals with pronounced ASD traits and that the assessment of induced internal noise is a useful way of exploring decision-related limits on motion perception, irrespective of ASD traits.

Restriction of orientation variability and spatial frequency on the perception of average orientation

During the observation of a single object, orientation and spatial frequency are jointly coded in an early stage of visual processing, as is evident from studies on the aftereffects of specific combinations of both these features. However, they become independent in the decision-making stage because observers can identify one feature while ignoring the other. Does this separability expand into the perception of ensemble representations? This study investigated the effect of the spatial frequencies of Gabor patches on orientation averaging. In the experiment, the average orientations of all eight patches composed of either homogeneous (i.e., eight 3 cycles/degree or 0.8 cycles/degree patches) or heterogeneous (i.e., four 3 cycles/degree and four 0.8 cycles/degree patches) spatial frequencies were stably estimated if the orientation varied within the range of ±7.7° around the true mean. However, when the range was extended to ±14°, we found that the averaging performance was better in the homogeneous lower spatial frequency than in the homogeneous higher spatial frequency and heterogeneous spatial frequency conditions. These results suggest that an ensemble perception of orientation is modulated by spatial frequency components.

Modeling mean estimation tasks in within-trial and across-trial contexts

The mean estimation task, which explicitly asks observers to estimate the mean feature value of multiple stimuli, is a fundamental paradigm in research areas such as ensemble coding and cue integration. The current study uses computational models to formalize how observers summarize information in mean estimation tasks. We compare model predictions from our Fidelity-based Integration Model (FIM) and other models on their ability to simulate observed patterns in within-trial weight distribution, across-trial information integration, and set-size effects on mean estimation accuracy. Experiments show non-equal weighting within trials in both sequential and simultaneous mean estimation tasks. Observers implicitly overestimated trial means below the global mean and underestimated trial means above the global mean. Mean estimation performance declined and stabilized with increasing set sizes. FIM successfully simulated all observed patterns, while other models failed. FIM's information sampling structure provides a new way to interpret the capacity limit in visual working memory and sub-sampling strategies. As a model framework, FIM offers task-dependent modeling for various ensemble coding paradigms, facilitating research synthesis across different studies in the literature.

Extracting Summary Statistics of Rapid Numerical Sequences

We examine the ability of observers to extract summary statistics (such as the mean and the relative-variance) from rapid numerical sequences of two digit numbers presented at a rate of 4/s. In four experiments (total N = 100), we find that the participants show a remarkable ability to extract such summary statistics and that their precision in the estimation of the sequence-mean improves with the sequence-length (subject to individual differences). Using model selection for individual participants we find that, when only the sequence-average is estimated, most participants rely on a holistic process of frequency based estimation with a minority who rely on a (rule-based and capacity limited) mid-range strategy. When both the sequence-average and the relative variance are estimated, about half of the participants rely on these two strategies. Importantly, the holistic strategy appears more efficient in terms of its precision. We discuss implications for the domains of two pathways numerical processing and decision-making.

Limited memory for ensemble statistics in visual change detection

Accounts of working memory based on independent item representations may overlook a possible contribution of ensemble statistics, higher-order regularities of a scene such as the mean or variance of a visual attribute. Here we used change detection tasks to investigate the hypothesis that observers store ensemble statistics in working memory and use them to detect changes in the visual environment. We controlled changes to the ensemble mean or variance between memory and test displays across six experiments. We made specific predictions of observers' sensitivity using an optimal summation model that integrates evidence across separate items but does not detect changes in ensemble statistics. We found strong evidence that observers outperformed this model, but only when task difficulty was high, and only for changes in stimulus variance. Under these conditions, we estimated that the variance of items contributed to change detection sensitivity more strongly than any individual item in this case. In contrast, however, we found strong evidence against the hypothesis that the average feature value is stored in working memory: when the mean of memoranda changed, sensitivity did not differ from the optimal summation model, which was blind to the ensemble mean, in five out of six experiments. Our results reveal that change detection is primarily limited by uncertainty in the memory of individual features, but that memory for the variance of items can facilitate detection under a limited set of conditions that involve relatively high working memory demands.

The interaction of global motion and global form processing on the perception of implied motion: An equivalent noise approach

Global motion and global form are proposed to be processed through functionally differentiated independent channels along dorsal (motion) and ventral (form) pathways. However, more recent studies show significant interactions between these pathways by inducing the perception of motion (implied motion) from presenting the independent frames of static Glass patterns. The mechanisms behind such interaction are not adequately understood with studies showing a larger contribution of either a motion or form processing mechanism. In the current study, we adapted the equivalent noise paradigm to disentangle the effect of internal noise (local processing) and sampling efficiency (global processing) on global motion, global form, and the interaction of both on the perception of implied motion using physically equivalent stimuli. Six visually normal observers discriminated the direction or orientation of random dot kinematograms (RDK), static Glass patterns (Glass), and dynamic Glass patterns (dGlass) whose directions/orientations were determined by the means of normal distributions with a range of direction/orientation variances that served as external noise. Thresholds (τ) showed a consistent pattern across observers and external noise levels, where τ_Glass > τ_dGlass > τ_RDK. Nested model comparisons where the thresholds were related to the external noise, internal noise, and the sampling efficiency revealed that the difference in performance between the tasks was best described by the change in sampling efficiency with invariable internal noise. Our results showed that the higher thresholds for implied motion compared to real motion could be due to inefficient pooling of local dipole orientation cues at global processing stages involving motion mechanisms.

The virtual loss function in the summary perception of motion and its limited adjustability

Humans can grasp the "average" feature of a visual ensemble quickly and effortlessly. However, it is largely unknown what is the exact form of the summary statistic humans perceive and it is even less known whether this form can be changed by feedback. Here we borrow the concept of loss function to characterize how the summary perception is related to the distribution of feature values in the ensemble, assuming that the summary statistic minimizes a virtual expected loss associated with its deviation from individual feature values. In two experiments, we investigated a random-dot motion estimation task to infer the virtual loss function implicit in ensemble perception and see whether it can be changed by feedback. On each trial, participants reported the average moving direction of an ensemble of moving dots whose distribution of moving directions was skewed. In Experiment 1, where no feedback was available, participants' estimates fell between the mean and the mode of the distribution and were closer to the mean. In particular, the deviation from the mean and toward the mode increased almost linearly with the mode-to-mean distance. The pattern was best modeled by an inverse Gaussian loss function, which punishes large errors less heavily than the quadratic loss function does. In Experiment 2, we tested whether this virtual loss function can be altered by feedback. Two groups of participants either received the mode or the mean as the correct answer. After extensive training up to five days, both groups' estimates moved slightly towards the mode. That is, feedback had no specific influence on participants' virtual loss function. To conclude, the virtual loss function in the summary perception of motion is close to inverse Gaussian, and it can hardly be changed by feedback.

Dual strategies in human confidence judgments

Although confidence is commonly believed to be an essential element in decision-making, it remains unclear what gives rise to one's sense of confidence. Recent Bayesian theories propose that confidence is computed, in part, from the degree of uncertainty in sensory evidence. Alternatively, observers can use physical properties of the stimulus as a heuristic to confidence. In the current study, we developed ideal observer models for either hypothesis and compared their predictions against human data obtained from psychophysical experiments. Participants reported the orientation of a stimulus, and their confidence in this estimate, under varying levels of internal and external noise. As predicted by the Bayesian model, we found a consistent link between confidence and behavioral variability for a given stimulus orientation. Confidence was higher when orientation estimates were more precise, for both internal and external sources of noise. However, we observed the inverse pattern when comparing between stimulus orientations: although observers gave more precise orientation estimates for cardinal orientations (a phenomenon known as the oblique effect), they were more confident about oblique orientations. We show that these results are well explained by a strategy to confidence that is based on the perceived amount of noise in the stimulus. Altogether, our results suggest that confidence is not always computed from the degree of uncertainty in one's perceptual evidence but can instead be based on visual cues that function as simple Heuristics to confidence.

An explicit investigation of the roles that feature distributions play in rapid visual categorization

Ensemble representations are often described as efficient tools when summarizing features of multiple similar objects as a group. However, it can sometimes be more useful not to compute a single summary description for all of the objects if they are substantially different, for example when they belong to entirely different categories. It was proposed that the visual system can efficiently use the distributional information of ensembles to decide whether simultaneously displayed items belong to single or several different categories. Here we directly tested how the feature distribution of items in a visual array affects an ability to discriminate individual items (Experiment 1) and sets (Experiments 2-3) when participants were instructed explicitly to categorize individual objects based on the median of size distribution. We varied the width (narrow or fat) as well as the shape (smooth or two-peaked) of distributions in order to manipulate the ease of ensemble extraction from the items. We found that observers unintentionally relied on the grand mean as a natural categorical boundary and that their categorization accuracy increased as a function of the size differences among individual items and a function of their separation from the grand mean. For ensembles drawn from two-peaked size distributions, participants showed better categorization performance. They were more accurate at judging within-category ensemble properties in other dimensions (centroid and orientation) and less biased by superset statistics. This finding corroborates the idea that the two-peaked feature distributions support the "segmentability" of spatially intermixed sets of objects. Our results emphasize important roles of ensemble statistics (mean, range, distribution shape) in explicit visual categorization.

Contributions of ensemble perception to outlier representation precision

It is known that the visual system can efficiently extract mean and variance information, facilitating the detection of outliers. However, no research to date has directly investigated whether ensemble perception mechanisms contribute to outlier representation precision. We specifically were interested in how the distinctiveness of outliers impacts their precision. Across two experiments, we compared how accurately viewers represented the orientation of spatial outliers that varied in distinctiveness and found that increased outlier distinctiveness resulted in greater precision. Based on comparisons of our data to simulations reflecting particular selective strategies, we eliminated the possibility that participants were selectively processing the outlier, at the expense of the ensemble. Thus, we argued that participants separately represented distinct outliers along with ensemble summaries of the remaining items in a display. We also found that outlier distinctiveness moderated the precision of how the remaining items were summarized. We discuss these findings in relation to computational capacity and constraints of ensemble perception mechanisms.

Dissociating implicit and explicit ensemble representations reveals the limits of visual perception and the richness of behavior

Our senses provide us with a rich experience of a detailed visual world, yet the empirical results seem to suggest severe limitations on our ability to perceive and remember. In recent attempts to reconcile the contradiction between what is experienced and what can be reported, it has been argued that the visual world is condensed to a set of summary statistics, explaining both the rich experience and the sparse reports. Here, we show that explicit reports of summary statistics underestimate the richness of ensemble perception. Our observers searched for an odd-one-out target among heterogeneous distractors and their representation of distractor characteristics was tested explicitly or implicitly. Observers could explicitly distinguish distractor sets with different mean and variance, but not differently-shaped probability distributions. In contrast, the implicit assessment revealed that the visual system encodes the mean, the variance, and even the shape of feature distributions. Furthermore, explicit measures had common noise sources that distinguished them from implicit measures. This suggests that explicit judgments of stimulus ensembles underestimate the richness of visual representations. We conclude that feature distributions are encoded in rich detail and can guide behavior implicitly, even when the information available for explicit summary judgments is coarse and limited.

Variance-dependent neural activity in an unvoluntary averaging task

Ensemble statistics of a visual scene can be estimated to provide a gist of the scene without detailed analysis of all individual items. The simplest and most widely studied ensemble statistic is mean estimation, which requires averaging an ensemble of elements. Averaging is useful to estimate the mean of an ensemble and discard the variance. The source of variance can be external (i.e., variance across the physical elements) or internal (i.e., imprecisions in the estimates of the elements by the visual system). The equivalent noise paradigm is often used to measure the impact of the internal variance (i.e., the equivalent input noise). This paradigm relies on the assumption that the averaging process is equally effective independently of the main source of variance, internal or external, so any difference between the processing when the main source of variance is internal or external must be assumed not to affect the averaging efficiency. The current fMRI study compared the neural activity when the main variance is caused by the stimulus (i.e., high variance) and when it is caused by imprecisions in the estimates of the elements by the visual system (i.e., low variance). The results showed that the right superior frontal and left middle frontal gyri can be significantly more activated when the variance in the orientation of the Gabors was high than when it was low. Consequently, the use of the equivalent noise paradigm requires the assumption that such additional neural activity in high variance does not affect the averaging efficiency.

Ensemble perception: Extracting the average of perceptual versus numerical stimuli

Recent research has established that humans can extract the average perceptual feature over briefly presented arrays of visual elements or the average of a rapid temporal sequence of numbers. Here we compared the extraction of the average over briefly presented arrays, for a perceptual feature (orientations) and for numerical values (1-9 digits), using an identical experimental design for the two tasks. We hypothesized that the averaging of numbers, more than of orientations, would be constrained by capacity limitations. Arrays of Gabor elements or digits were simultaneously presented for 300 ms and observers were required to estimate the average on a continuous response scale. In each trial the elements were sampled from normal distributions (of various means) and we varied the set size (4-12). We found that while for orientation the averaging precision remained constant with set size, for numbers it decreased with set size. Using computational modeling we also extracted capacity parameters (the number of elements that are pooled in the average extraction). Despite marked heterogeneity between observers, the capacity for orientations (around eight items) was much larger than for numbers (around four items). The orientation task also had a larger fraction of participants relying on distributed attention to all elements. Our study thus supports the idea that numbers more than perceptual features are subject to capacity or attentional limitations when observers need to evaluate the average over an ensemble of stimuli.

Synergy between research on ensemble perception, data visualization, and statistics education: A tutorial review

In the age of big data, we are constantly inventing new data visualizations to consolidate massive amounts of numerical information into smaller and more digestible visual formats. These data visualizations use various visual features to convey quantitative information, such as spatial position in scatter plots, color saturation in heat maps, and area in dot maps. These data visualizations are typically composed of ensembles, or groups of related objects, that together convey information about a data set. Ensemble perception, or one's ability to perceive summary statistics from an ensemble, such as the mean, has been used as a foundation for understanding and explaining the effectiveness of certain data visualizations. However, research in data visualization has revealed some perceptual biases and conceptual difficulties people face when trying to utilize the information in these graphs. In this tutorial review, we will provide a broad overview of research conducted in ensemble perception, discuss how principles of ensemble encoding have been applied to the research in data visualization, and showcase the barriers graphs can pose to learning statistical concepts, using histograms as a specific example. The goal of this tutorial review is to highlight possible connections between three areas of research-ensemble perception, data visualization, and statistics education-and to encourage research in the practical applications of ensemble perception in solving real-world problems in statistics education.

Ensemble coding of crowd speed using biological motion

The accurate perception of human crowds is integral to social understanding and interaction. Previous studies have shown that observers are sensitive to several crowd characteristics such as average facial expression, gender, identity, joint attention, and heading direction. In two experiments, we examined ensemble perception of crowd speed using standard point-light walkers (PLW). Participants were asked to estimate the average speed of a crowd consisting of 12 figures moving at different speeds. In Experiment 1, trials of intact PLWs alternated with trials of scrambled PLWs with a viewing duration of 3 seconds. We found that ensemble processing of crowd speed could rely on local motion alone, although a globally intact configuration enhanced performance. In Experiment 2, observers estimated the average speed of intact-PLW crowds that were displayed at reduced viewing durations across five blocks of trials (between 2500 ms and 500 ms). Estimation of fast crowds was precise and accurate regardless of viewing duration, and we estimated that three to four walkers could still be integrated at 500 ms. For slow crowds, we found a systematic deterioration in performance as viewing time reduced, and performance at 500 ms could not be distinguished from a single-walker response strategy. Overall, our results suggest that rapid and accurate ensemble perception of crowd speed is possible, although sensitive to the precise speed range examined.

The averaging of numerosities: A psychometric investigation of the mental line

Humans and animals are capable of estimating and discriminating nonsymbolic numerosities via mental representation of magnitudes—the approximate number system (ANS). There are two models of the ANS system, which are similar in their prediction in numerosity discrimination tasks. The log-Gaussian model, which assumes numerosities are represented on a compressed logarithmic scale, and the scalar variability model, which assumes numerosities are represented on a linear scale. In the first experiment of this paper, we contrasted these models using averaging of numerosities. We examined whether participants generate a compressed mean (i.e., geometric mean) or a linear mean when averaging two numerosities. Our results demonstrated that half of the participants are linear and half are compressed; however, in general, the compression is milder than a logarithmic compression. In Experiments 2 and 3, we examined averaging of numerosities in sequences larger than two. We found that averaging precision increases with sequence length. These results are in line with previous findings, suggesting a mechanism in which the estimate is generated by population averaging of the responses each stimulus generates on the numerosity representation.

Peak-at-end rule: adaptive mechanism predicts time-dependent decision weighting

Humans make decisions under various natural circumstances, integrating multiple pieces of information that are distributed over space and time. Although psychophysical and physiological studies have investigated temporal dynamics underlying perceptual decision making, weighting profiles for inliers and outliers during temporal integration have yet to be fully investigated in most studies. Here, we examined the temporal weighting profile of a computational model characterized by a leaky integrator of sensory evidence. As a corollary of its leaky nature, the model predicts the recency effect and overweights outlying elements around the end of the stream. Moreover, we found that the model underweights outlying values occurring earlier in the stream (i.e., robust averaging). We also show that human observers exhibit exactly the same weighting profile in an average estimation task. These findings suggest that the adaptive decision process in the brain results in the time-dependent decision weighting, the “peak-at-end” rule, rather than the peak-end rule in behavioral economics.

Perception of means, sums, and areas

In this age of data visualization, it is important to understand our perception of the symbols that are used. For example, does the perceived size of a disc correspond most closely to its area, diameter, circumference, or some other measure? When multiple items are present, this becomes a question of ensemble perception. Here, we compare observers' performance across three different tasks: judgments of (i) the mean diameter, (ii) the total diameter, or (iii) the total area of (N = 1, 2, 3, or 7) test circles compared with a single reference circle. We draw a parallel between Anne Treisman's feature integration theory and Daniel Kahneman's cognitive systems, comparing the preattentive stage to System 1, and the focused attention stage to System 2. In accordance with Kahneman's prediction, average size (diameter) of the geometric figures can be judged with considerable accuracy, but the total diameter of the same figures cannot. Like the total length, the cumulative area covered by circles was also judged considerably less accurately than the mean diameter. Differences in efficiency between these three tasks illustrate powerful constraints upon visual processing: The visual system is well adapted for the perception of the mean size while there are no analogous mechanisms for the accurate perception of the total length or cumulative area. Thus, in visualizing data, using bubble charts proportional to area may be misleading as our visual system seems better adapted to perceive disc size by the radius rather than the area.

Feature-specificity in visual statistical summary processing

Visual statistical summary processing enables people to extract the average feature of a set of items rapidly and accurately. Previous studies have demonstrated independent mechanisms for summarizing low (e.g. color, orientation) and high-level (facial identity, emotion) visual information. However, no study to date has conclusively determined whether there are feature-specific summarization mechanisms for low-level features or whether there are low-level, feature agnostic summarization mechanisms. To address this issue, we asked participants to report either the average orientation or the average size from a set of lines where both features varied. Participants completed these tasks either in single-task or mixed-task conditions; in the latter, successful performance required extraction of both summaries concurrently. If there were feature-specific summarization mechanisms that could operate in parallel, then errors in mean size and mean orientation tasks should be independent, in both single and mixed task conditions. On the other hand, a central domain-general mechanism for low-level summarization would imply a correlation between errors for both features and greater error in the mixed than single task trials. In Experiment 1, we found that there was no correlation between the mean size and mean orientation errors and performance was similar across single and mixed-task conditions, suggesting that there may be independent summarization mechanisms for size and orientation features. To further test the feature-specificity account, in Experiment 2 and 3 (with mask), we manipulated the display duration to determine whether there were any differences in the summarization of earlier (orientation) vs. later (size) features. While these experiments replicated the pattern of results observed in Experiment 1, at shorter display durations, no differences emerged across features. We argue that our data is consistent with independent, multi-level feature-specific statistical summary mechanisms for low-level visual features.

Integration of contours defined by second-order contrast-modulation of texture

Boundaries in the visual world can be defined by changes in luminance and texture in the input image. A "contour integration" process joins together local changes into percepts of lines or edges. A previous study tested the integration of contours defined by second-order contrast-modulation. Their contours were placed in a background of random wavelets. Participants performed near chance. We re-visited second-order contour integration with a different task. Participants distinguished contours with "good continuation" from distractors. We measured thresholds in different amounts of external orientation or position noise. This gave two noise-masking functions. We also measured thresholds for contours with a baseline curvature to assess performance with more curvy targets. Our participants were able to discriminate the good continuation of second-order contours. Thresholds were higher than for first-order contours. In our modelling, we found this was due to multiple factors. There was a doubling of equivalent internal noise between first- and second-order contour integration. There was also a reduction in efficiency. The efficiency difference was only significant in our orientation noise condition. For both first- and second-order stimuli, participants were also able to perform our task with more curved contours. We conclude that humans can integrate second-order contours, even when they are curved. There is however reduced performance compared to first-order contours. We find both an impaired input to the integrating mechanism, and reduced efficiency seem responsible. Second-order contour integration may be more affected by the noise background used in the previous study. Difficulty segregating that background may explain their result.

Roles of saliency and set size in ensemble averaging

Ensemble statistics are often thought of as a reliable impression of numerous items despite limited capacities to consciously represent each individual. However, whether all items equally contribute to ensemble summaries (e.g., mean) and whether they might be affected by known limited-capacity processes, such as focused attention, is still debated. We addressed these questions via a recently described "amplification effect," a systematic bias of perceived mean (e.g., average size) towards the more salient "tail" of a feature distribution (e.g., larger items). In our experiments, observers adjusted the mean orientation of sets of items varying in set size. We made some of the items more salient or less salient by changing their size. While the whole orientation distribution was fixed, the more salient subset could be shifted relative to the set mean or differ in range. We measured the bias away from the set mean and the standard deviation (SD) of errors, as it is known to reflect the physical range from which ensemble information is sampled. We found that bias and SD changes followed the shifts and range changes in salient subsets, providing evidence for amplification. However, these changes were weaker than those expected from sampling only salient items, suggesting that less salient items were also sampled. Importantly, the SD decreased as a function of set size, which is only possible if the number of sampled elements increased with set size. Overall, we conclude that orientation summary statistics are sampled from an entire ensemble and modulated by the amplification effect of attention.

Distinct strategies for estimating the temporal average of numerical and perceptual information

Humans can estimate global trends in dynamic information presented either as perceptual features or as symbolic codes such as numbers. Previous studies on temporal statistics estimation have shown that observers judge the temporal average of visual attributes according to information from the last few frames of the presentation sequence (in what is referred to as the recency effect). Here, we investigated how humans estimate the temporal average of number vs. orientation using identical stimuli for the two tasks. In Experiment 1, a randomly-selected single-digit number was serially presented at orientations randomly varying over time. In Experiment 2, a texture comprising a random number of Gabor elements was shown at orientations randomly varying over time. In both experiments, observers judged the temporal averages of the numerical values and orientations in separate blocks. Results showed that observers judging the temporal average of orientation relied upon information from later frames as predicted by a typical model of perceptual decision making. By contrast, for the judgement of numerical values, we found that the impacts of each temporal frame were constant or varied little across temporal frames regardless of whether the numerical information was given as digits or by the number of texture elements. The results are interpreted as evidence that distinct computational strategies may be involved in estimating the temporal averages of perceptual features and numerical information.

Color ensembles: Sampling and averaging spatial hue distributions

Color serves both to segment a scene into objects and background and to identify objects. Although objects and surfaces usually contain multiple colors, humans can readily extract a representative color description, for instance, that tomatoes are red and bananas yellow. The study of color discrimination and identification has a long history, yet we know little about the formation of summary representations of multicolored stimuli. Here, we characterize the human ability to integrate hue information over space for simple color stimuli varying in the amount of information, stimulus size, and spatial configuration of stimulus elements. We show that humans are efficient at integrating hue information over space beyond what has been shown before for color stimuli. Integration depends only on the amount of information in the display and not on spatial factors such as element size or spatial configuration in the range measured. Finally, we find that observers spontaneously prefer a simple averaging strategy even with skewed color distributions. These results shed light on how human observers form summary representations of color and make a link between the perception of polychromatic surfaces and the broader literature of ensemble perception.

Five dichotomies in the psychophysics of ensemble perception

Whereas psychophysicists may formulate hypotheses about appearance, they can only measure performance. Bias and imprecision in psychophysical data need not necessarily reflect bias and imprecision in perception. Sensory systems may exaggerate the differences between each item and its neighbors in an ensemble. Alternatively, sensory systems may homogenize the ensemble, thereby removing any apparent differences between neighboring items. Ensemble perception may be involuntary when observers attempt to report the identities of individual items. Conversely, when asked to make a (voluntary) decision about the ensemble as a whole, observers may find it very difficult to compute statistics that are based on more than a very small number of individual items. Modeling decisions about prothetic continua, such as size and contrast, can be tricky because sensory signals may be distorted before and/or after voluntarily computing ensemble statistics. With metathetic continua, such as spatial orientation, distortion is less problematic; physically vertical things necessarily appear close to vertical and physically horizontal things necessarily appear close to horizontal. Decision processes are corrupted by noise that, like distortion, may be added to sensory signals prior to and/or after voluntarily computing ensemble statistics.

Factors limiting sensitivity to binocular disparity in human vision: Evidence from a noise-masking approach

Our visual system uses the disparity between the images received by the two eyes to judge three-dimensional distance to surfaces. We can measure this ability by having subjects discriminate the disparity of rendered surfaces. We wanted to know the basis of the individual differences in this ability. We tested 53 adults with normal vision using a relative disparity detection task. Targets were wedge-shaped surfaces formed from random dots. These were presented in either crossed or uncrossed disparity relative to a random dot background. The threshold disparity ranged from 24 arc seconds in the most-able subject to 275 arc seconds in the least-able subject. There was a small advantage for detecting crossed-disparity targets. We used the noise-masking paradigm to partition subject performance into two factors. These were the subject's equivalent internal noise and their processing efficiency. The parameters were estimated by fitting the linear amplifier model. We found both factors contributed to the individual differences in stereoacuity. Within subjects, those showing an advantage for one disparity direction had enhanced efficiency for that direction. Some subjects had a higher equivalent internal noise for one direction that was balanced out by an increased efficiency. Our approach provides a more thorough account of the stereo-ability of our subjects compared with measuring thresholds alone. We present a normative set of results that can be compared with clinical populations.

Ensemble perception and focused attention: Two different modes of visual processing to cope with limited capacity

The visual system has a limited capacity for dealing with complex and redundant information in a scene. Here, we propose that a distributed attention mode of processing is necessary for coping with this limit, together with a focused attention mode of processing. The distributed attention mode provides a statistical summary of a scene, whereas the focused attention mode provides relevant information for object recognition. In this paper, we claim that a distributed mode of processing is necessary because (1) averaging performance improves with increased set-sizes, (2) even unselected items are likely to contribute to averaging, and (3) the assumption of variable capacity limits in averaging over different set-sizes is not plausible. We then propose how the averaging process can access multiple items over the capacity limit of focused attention. The visual system can represent multiple items as population responses and read out relevant information using the two modes of attention. It can summarize population responses with a broad application of a Gaussian profile (i.e., distributed attention) and represent its peak as the mean. It can focus on relevant population responses with a narrow application of a Gaussian profile (i.e., focused attention) and select important information for object recognition. The two attention modes of processing provide a framework for incorporating two seemingly opposing fields of study (ensemble perception and selective attention) and a unified theory of a coping strategy with our limited capacity.

Perception and decision mechanisms involved in average estimation of spatiotemporal ensembles

A number of studies on texture and ensemble perception have shown that humans can immediately estimate the average of spatially distributed visual information. The present study characterized mechanisms involved in estimating averages for information distributed over both space and time. Observers viewed a rapid sequence of texture patterns in which elements' orientation were determined by dynamic Gaussian noise with variable spatial and temporal standard deviations (SDs). We found that discrimination thresholds increased beyond a certain spatial SD if temporal SD was small, but if temporal SD was large, thresholds remained nearly constant regardless of spatial SD. These data are at odds with predictions that threshold is uniquely determined by spatiotemporal SD. Moreover, a reverse correlation analysis revealed that observers judged the spatiotemporal average orientation largely depending on the spatial average orientation over the last few frames of the texture sequence - a recency effect widely observed in studies of perceptual decision making. Results are consistent with the notion that the visual system rapidly computes spatial ensembles and adaptively accumulates information over time to make a decision on spatiotemporal average. A simple computational model based on this notion successfully replicated observed data.

Adaptation to mean and variance: Interrelationships between mean and variance representations in orientation perception

When there are many visual items, the visual system could represent their summary statistics (e.g., mean, variance) to process them efficiently. Although many previous studies have investigated the mean or variance representation itself, a relationship between these two ensemble representations has not been investigated much. In this study, we tested the potential interaction between mean and variance representations by using a visual adaptation method. We reasoned that if mean and variance representations interact with each other, an adaptation aftereffect to either mean or variance would influence the perception of the other. Participants watched a sequence of orientation arrays containing a specific statistical property during the adaptation period. To produce an adaptation aftereffect specific to variance or mean, one property of the adaptor arrays (variance or mean) had a fixed value while the other property was randomly varied. After the adaptation, participants were asked to discriminate the property of the test array that was randomly varied during the adaptation. We found that the adaptation aftereffect of orientation variance influenced the sensitivity of mean orientation discrimination (Experiment 1), and that the adaptation aftereffect of mean orientation influenced the bias of orientation variance discrimination (Experiment 2). These results suggest that mean and variance representations do closely interact with each other. Considering that mean and variance reflect the representative value and dispersion of multiple items respectively, the interactions between mean and variance representations may reflect their complementary roles to summarize complex visual information effectively.

Inflation versus filling-in: why we feel we see more than we actually do in peripheral vision

Do we perceive fine details in the visual periphery? Here, we propose that phenomenology in the visual periphery can be characterized by an inflated sense of perceptual capacity, as observers overestimate the quality of their perceptual inputs. Distinct from the well-known perceptual phenomenon of 'filling-in' where perceptual content is generated or completed endogenously, inflation can be characterized by incorrect introspection at the subjective level. The perceptual content itself may be absent or weak (i.e. not necessarily filled-in), and yet such content is mistakenly regarded by the system as rich. Behaviourally, this can be reflected by metacognitive deficits in the degree to which confidence judgements track task accuracy, and decisional biases for observers to think particular items are present, even when they are not. In two experiments using paradigms that exploit unique attributes of peripheral vision (crowding and summary statistics), we provide evidence that both types of deficits are present in peripheral vision, as observers' reports are marked by overconfidence in discrimination judgements and high numbers of false alarms in detection judgements. We discuss potential mechanisms that may be the cause of inflation and propose future experiments to further explore this unique sensory phenomenon.This article is part of the theme issue 'Perceptual consciousness and cognitive access'.

Internal but not external noise frees working memory resources

The precision with which visual information can be recalled from working memory declines as the number of items in memory increases. This finding has been explained in terms of the distribution of a limited representational resource between items. Here we investigated how the sensory strength of memoranda affects resource allocation. We manipulated signal strength of an orientation stimulus in two ways: we varied the internal (sensory) noise by adjusting stimulus contrast, and varied the external (stimulus) noise by altering the within-stimulus variability. Both manipulations had similar effects on the precision with which the orientation could be recalled, but differed in their impact on memory for other stimuli. These results indicate that increasing internal noise released resources that could be used to store other stimuli more precisely; increasing external noise had no such effect. We show that these observations can be captured by a simple neural model of working memory encoding, in which spiking activity takes on the role of the limited resource.

Investigations of visual short-term memory typically involve memorising clearly visible objects with elementary features, such as monochromatic disks or oriented bars. Results of such studies indicate that memory is allocated like a limited resource, i.e. shared out between objects. However, in daily life we are often confronted with visual features that are difficult to make out, like when an object is in shadow, or poorly-defined, like the color of a variegated leaf. Here we asked whether these kinds of features occupy as much memory resource as simple highly-visible objects. Our results demonstrate that reducing the sensory strength of a stimulus makes the quality of recall worse, but also takes up less resource so other objects can be remembered more precisely. Increasing the variability within a stimulus worsens recall, but has no effect on how other objects are remembered. These findings can be explained by considering how visual information is stored in populations of neurons: only the manipulation of sensory strength changes the amount of spiking activity dedicated to a stimulus.

Ensemble perception of facial attractiveness

Ensemble perception, the extraction of a statistical summary of multiple instances of a feature, enables efficient processing of information. Here we investigated whether ensemble representations can be formed for facial attractiveness, a socially important complex feature. After verifying that our face stimuli produced by geometric morphing represented a valid continuum of attractiveness (Experiment 1), we asked participants to compare the average attractiveness of four faces with a single probe face. Whether the four faces were homogeneous or heterogeneous resulted in highly similar performance levels, suggesting the visual system could extract an ensemble representation of the attractiveness of a heterogeneous group of faces. Statistical simulations with human-level bias and noise indicated participants did not rely on subsampling one random face or the most/least attractive face from the array (Experiment 2). Ensemble perception of facial attractiveness was not affected by variance in the stimulus array (Experiment 3), did not depend on memory of individual faces in the array (Experiment 4), and could be extended to larger arrays with faces asymmetrically distributed around the set mean (Experiment 5). Our findings give further evidence to the prevalence of perception of statistical regularities in vision.

Capacity limit of ensemble perception of multiple spatially intermixed sets

The visual system is remarkably efficient at extracting summary statistics from the environment. Yet at any given time, the environment consists of many groups of objects distributed over space. Thus, the challenge for the visual system is to summarize over multiple groups. The current study investigates the capacity and computational efficiency of ensemble perception, in the context of perceiving mean sizes of multiple spatially intermixed groups of circles. In a series of experiments, participants viewed an array of one to eight sets of circles. Each set contained four circles in the same colors, but with different sizes. Participants estimated the mean size of a probed set. The set that would be probed was either known before onset of the array (pre-cue condition) or afterwards (post-cue condition). By comparing estimation error in the pre-cue and post-cue conditions, we found that participants could reliably estimate mean sizes for approximately two sets (Experiment 1). Importantly, this capacity was robust against attention bias toward individual objects in the sets (Experiment 2). Varying the exposure time to stimulus arrays did not increase the capacity limit, suggesting that ensemble perception could be limited by an internal resource constraint, rather than the speed of information encoding (Experiment 3). Moreover, we found that the visual system could not encode and hold more individual items than ensemble representations (Experiment 4). Taken together, these results suggest that ensemble perception provides an efficient way of information processing but with constraints.