The hierarchical sparse selection model of visual crowding

Neural Correlates of Crowding in Macaque Area V4

Visual crowding refers to the phenomenon where a target object that is easily identifiable in isolation becomes difficult to recognize when surrounded by other stimuli (distractors). Many psychophysical studies have investigated this phenomenon and proposed alternative models for the underlying mechanisms. One prominent hypothesis, albeit with mixed psychophysical support, posits that crowding arises from the loss of information due to pooled encoding of features from target and distractor stimuli in the early stages of cortical visual processing. However, neurophysiological studies have not rigorously tested this hypothesis. We studied the responses of single neurons in macaque (one male, one female) area V4, an intermediate stage of the object-processing pathway, to parametrically designed crowded displays and texture statistics-matched metameric counterparts. Our investigations reveal striking parallels between how crowding parameters-number, distance, and position of distractors-influence human psychophysical performance and V4 shape selectivity. Importantly, we also found that enhancing the salience of a target stimulus could alleviate crowding effects in highly cluttered scenes, and this could be temporally protracted reflecting a dynamical process. Thus, a pooled encoding of nearby stimuli cannot explain the observed responses, and we propose an alternative model where V4 neurons preferentially encode salient stimuli in crowded displays. Overall, we conclude that the magnitude of crowding effects is determined not just by the number of distractors and target-distractor separation but also by the relative salience of targets versus distractors based on their feature attributes-the similarity of distractors and the contrast between target and distractor stimuli.

Seeing in crowds: Averaging first, then max

Crowding, a fundamental limit in object recognition, is believed to result from excessive integration of nearby items in peripheral vision. To understand its pooling mechanisms, we measured subjects' internal response distributions in an orientation crowding task. Contrary to the prediction of an averaging model, we observed a pattern suggesting that the perceptual judgement is made based on choosing the largest response across the noise-perturbed items. A model featuring first-stage averaging and second-stage signed-max operation predicts the diverse errors made by human observers under various signal strength levels. These findings suggest that different rules operate to resolve the bottleneck at early and high-level stages of visual processing, implementing a combination of linear and nonlinear pooling strategies.

Abnormal basic visual processing functions in binocular fusion disorders

Heterophoria is a common type of binocular fusion disorder that consists of a latent eye misalignment with potential consequences on daily activities such as reading or working on a computer (with CVS). Crowding, a type of contextual modulation, can also impair reading. Our recent studies found an abnormal pattern of low-level visual processing with larger perceptive fields (PF) in heterophoria. The PF is the fundamental processing unit of human vision and both masking and crowding depend on its size. We investigated how heterophoria would impact the PF's size via a lateral masking experiment and consequently affect the foveal crowding at different letter-spacings (the crowding zone). More specifically, we explored the relationship between crowding, lateral masking, the PF's size, and the amount of heterophoria. The binocular horizontal PF's size was larger with heterophoric subjects, in agreement with our previous study. We found a stronger crowding and an extended crowding zone associated with slower response times; this shows that the processing of letter identification under both crowded and uncrowded conditions requires more processing effort in heterophoric individuals. In agreement with previous studies, we found a correlation between the crowding zone and the PF's size; each was strongly correlated with the amount of phoria. These findings resemble those involving the PF size and the extended crowding found at the fovea in amblyopia and young children. We suggest that these findings could help explain the inter-observers' variability found in the masking literature, and the reading difficulties often encountered in subjects with high heterophoria.

Differential modulation on neural activity related to flankers during face processing: A visual crowding study

In this visual crowding study, we manipulated the perceivability of a central crowded face (a fearful or a neutral face) by varying the similarity between the central face and the surrounding flanker stimuli. We presented participants with pairs of visual clutters and recorded their electroencephalography during an emotion judgement task. In an upright flanker condition where both the central target face and flanker faces were upright faces, participants were less likely to report seeing the target face, and their P300 was weakened, compared to a scrambled flanker condition where scrambled face images were used as flankers. Additionally, at ∼ 120 ms post-stimulus, a posterior negativity was found for the upright compared to scrambled flanker condition, however only for fearful face targets. We concluded that early neural responses seem to be affected by the perceptual characteristics of both target and flanker stimuli whereas later-stage neural activity is associated with post-perceptual evaluation of the stimuli in this visual crowding paradigm.

The impact of training on the inner-outer asymmetry in crowding

Inner-outer asymmetry, where the outer flanker induces stronger crowding than the inner flanker, is a hallmark property of visual crowding. It is unclear the contribution of inner-outer asymmetry to the pattern of crowding errors (biased predominantly toward the flanker identities) and the role of training on crowding errors. In a typical radial crowding display, 20 observers were asked to report the orientation of a target Gabor (7.5° eccentricity) flanked by either an inner or outer Gabor along the horizontal meridian. The results showed that outer flanker conditions induced stronger crowding, accompanied by assimilative errors to the outer flanker for similar target/flanker elements. In contrast, the inner flanker condition exhibited weaker crowding, with no significant patterns of crowding errors. A population coding model showed that the flanker weights in the outer flanker condition were significantly higher than those in the inner flanker condition. Nine observers continued to train the outer flanker condition for four sessions. Training reduced inner-outer asymmetry and reduced flanker weights to the outer flanker. The learning effects were retained over 4 to 6 months. Individual differences in the appearance of crowding errors, the strength of inner-outer asymmetry, and the training effects were evident. Nevertheless, our findings indicate that different crowding mechanisms may be responsible for the asymmetric crowding effects induced by inner and outer flankers, with the outer flankers dominating the appearance more than the inner ones. Training reduces inner-outer asymmetry by reducing target/flanker confusion, and learning is persistent over months, suggesting that perceptual learning has the potential to improve visual performance by promoting neural plasticity.

The role of spatial attention in crowding and feature binding

Crowding refers to the failure to identify a peripheral object due to nearby objects (flankers). A hallmark of crowding is inner-outer asymmetry; that is, the outer flanker (more peripheral) produces stronger interference than the inner one. Here, by manipulating attention, we tested the predictions of two competing accounts: the attentional account, which predicts a positive attentional effect on the inner-outer asymmetry (i.e., attention to the outer flanker will increase asymmetry) and the receptive field size account, which predicts a negative attentional effect. In Experiment 1, observers estimated a Gabor target orientation. A peripheral pre-cue drew attention to one of three locations: target, inner flanker, or outer flanker. Probabilistic mixture modeling demonstrated asymmetry by showing that observers often misreported the outer-flanker orientation as the target. Interestingly, the outer cue led to a higher misreport rate of the outer flanker, and the inner cue led to a lower misreport rate of the outer flanker. Experiment 2 tested the effect of crowding and attention on incoherent object reports (i.e., binding errors, reporting the tilt of one presented item with the color of another item). In each trial, observers estimated both the tilt and color of the target. Attention merely increased coherent target reports, but not coherent flanker reports. The results suggest that the locus of spatial attention plays an essential role in crowding, as well as inner-outer asymmetry, and demonstrate that crowding and feature binding are closely related. However, our findings are inconsistent with the view that covert attention automatically binds features together.

Ensemble perception: Stacking the hay to find the needle

The visual clutter we constantly encounter in the world limits object recognition, a phenomenon known as visual crowding. A new study shows that ensemble perception counters this by condensing redundant information into summary statistical representations, which thus releases visual crowding's effect on individual objects.

Theta activity in semantic priming under visual crowding as revealed by magnetoencephalography

Crowding refers to impaired object recognition of peripheral visual targets caused by nearby flankers. It has been shown that the response to a word was faster when it was preceded by a semantically related than unrelated crowded prime, demonstrating that semantic priming survives crowding. This study examines neural correlates of semantic priming under visual crowding using magnetoencephalography with four conditions: prime (isolated, crowded) x prime-target relationship (related, unrelated). Participants judged whether the target was a word or a nonword. We found significant differences in θ activity at the left inferior frontal gyrus (IFG) for both isolated and crowded primes when comparing the unrelated and related conditions, although the activation was delayed with the crowded prime compared to the isolated prime. The locations within the IFG were also different: theta-band activation was at BA 45 in the isolated condition and at BA 47 in the crowded condition. Phase-locking-value analysis revealed that bilateral IFG was more synchronized with unrelated prime-target pairs than related pairs regardless of whether the primes were isolated or crowded, indicating the recruitment of the right hemisphere when the prime-target semantic relationship was remote. Finally, the distinct waveform patterns found in the isolated and crowded conditions from both the source localization and PLV analysis suggest different neural mechanisms for processing semantic information with isolated primes versus crowded primes.

Morality extracted under crowding impairs face identification

We investigated whether morality associated with faces is perceptible even under less optimal visual conditions such as crowding. A facial image was paired with a sentence describing an immoral act or a neutral act. Participants imagined the person performing the actions described in the sentence during the learning phase. Then, in the crowding phase, the target face was briefly presented in the left or right peripheral visual fields. Participants were required to judge the gender or morality of the target face in Experiment 1 and to choose the target face from two faces in Experiment 2. In both experiments, flankers were presented around the target face in the flanker condition, whereas no flankers were presented in the no-flanker condition. Experiment 1 indicated that the accuracy of judgments about the morality of a crowded face was higher for immoral faces than for neutral faces. This demonstrates that morality is preferentially extracted even when conscious access to facial representations is limited. Experiment 2 showed that the accuracy of selecting the flanked face from two faces was higher for neutral faces than for immoral faces. These indicated that the morality processed under the crowding impaired the discrimination of the facial identity.

Spatial Attention Enhances Crowded Stimulus Encoding Across Modeled Receptive Fields by Increasing Redundancy of Feature Representations

Any visual system, biological or artificial, must make a trade-off between the number of units used to represent the visual environment and the spatial resolution of the sampling array. Humans and some other animals are able to allocate attention to spatial locations to reconfigure the sampling array of receptive fields (RFs), thereby enhancing the spatial resolution of representations without changing the overall number of sampling units. Here, we examine how representations of visual features in a fully convolutional neural network interact and interfere with each other in an eccentricity-dependent RF pooling array and how these interactions are influenced by dynamic changes in spatial resolution across the array. We study these feature interactions within the framework of visual crowding, a well-characterized perceptual phenomenon in which target objects in the visual periphery that are easily identified in isolation are much more difficult to identify when flanked by similar nearby objects. By separately simulating effects of spatial attention on RF size and on the density of the pooling array, we demonstrate that the increase in RF density due to attention is more beneficial than changes in RF size for enhancing target classification for crowded stimuli. Furthermore, by varying target/flanker spacing, as well as the spatial extent of attention, we find that feature redundancy across RFs has more influence on target classification than the fidelity of the feature representations themselves. Based on these findings, we propose a candidate mechanism by which spatial attention relieves visual crowding through enhanced feature redundancy that is mostly due to increased RF density.

Global and high-level effects in crowding cannot be predicted by either high-dimensional pooling or target cueing

In visual crowding, the perception of a target deteriorates in the presence of nearby flankers. Traditionally, target-flanker interactions have been considered as local, mostly deleterious, low-level, and feature specific, occurring when information is pooled along the visual processing hierarchy. Recently, a vast literature of high-level effects in crowding (grouping effects and face-holistic crowding in particular) led to a different understanding of crowding, as a global, complex, and multilevel phenomenon that cannot be captured or explained by simple pooling models. It was recently argued that these high-level effects may still be captured by more sophisticated pooling models, such as the Texture Tiling model (TTM). Unlike simple pooling models, the high-dimensional pooling stage of the TTM preserves rich information about a crowded stimulus and, in principle, this information may be sufficient to drive high-level and global aspects of crowding. In addition, it was proposed that grouping effects in crowding may be explained by post-perceptual target cueing. Here, we extensively tested the predictions of the TTM on the results of six different studies that highlighted high-level effects in crowding. Our results show that the TTM cannot explain any of these high-level effects, and that the behavior of the model is equivalent to a simple pooling model. In addition, we show that grouping effects in crowding cannot be predicted by post-perceptual factors, such as target cueing. Taken together, these results reinforce once more the idea that complex target-flanker interactions determine crowding and that crowding occurs at multiple levels of the visual hierarchy.

Unraveling brain interactions in vision: The example of crowding

Crowding, the impairment of target discrimination in clutter, is the standard situation in vision. Traditionally, crowding is explained with (feedforward) models, in which only neighboring elements interact, leading to a "bottleneck" at the earliest stages of vision. It is with this implicit prior that most functional magnetic resonance imaging (fMRI) studies approach the identification of the "neural locus" of crowding, searching for the earliest visual area in which the blood-oxygenation-level-dependent (BOLD) signal is suppressed under crowded conditions. Using this classic approach, we replicated previous findings of crowding-related BOLD suppression starting in V2 and increasing up the visual hierarchy. Surprisingly, under conditions of uncrowding, in which adding flankers improves performance, the BOLD signal was further suppressed. This suggests an important role for top-down connections, which is in line with global models of crowding. To discriminate between various possible models, we used dynamic causal modeling (DCM). We show that recurrent interactions between all visual areas, including higher-level areas like V4 and the lateral occipital complex (LOC), are crucial in crowding and uncrowding. Our results explain the discrepancies in previous findings: in a recurrent visual hierarchy, the crowding effect can theoretically be detected at any stage. Beyond crowding, we demonstrate the need for models like DCM to understand the complex recurrent processing which most likely underlies human perception in general.

Investigating Visual Crowding of Objects in Complex Real-World Scenes

Visual crowding, the impairment of object recognition in peripheral vision due to flanking objects, has generally been studied using simple stimuli on blank backgrounds. While crowding is widely assumed to occur in natural scenes, it has not been shown rigorously yet. Given that scene contexts can facilitate object recognition, crowding effects may be dampened in real-world scenes. Therefore, this study investigated crowding using objects in computer-generated real-world scenes. In two experiments, target objects were presented with four flanker objects placed uniformly around the target. Previous research indicates that crowding occurs when the distance between the target and flanker is approximately less than half the retinal eccentricity of the target. In each image, the spacing between the target and flanker objects was varied considerably above or below the standard (0.5) threshold to either suppress or facilitate the crowding effect. Experiment 1 cued the target location and then briefly flashed the scene image before participants could move their eyes. Participants then selected the target object’s category from a 15-alternative forced choice response set (including all objects shown in the scene). Experiment 2 used eye tracking to ensure participants were centrally fixating at the beginning of each trial and showed the image for the duration of the participant’s fixation. Both experiments found object recognition accuracy decreased with smaller spacing between targets and flanker objects. Thus, this study rigorously shows crowding of objects in semantically consistent real-world scenes.

Binocular summation is affected by crowding and tagging

In perceptual crowding, a letter easily recognized on its own, becomes unrecognizable if it is surrounded by other letters, an effect that confers a limit on the visual processing. Models assume that crowding is a hallmark of the periphery but that it is almost absent in the fovea. However, recently it was shown that crowding occurs in the fovea of people with an abnormal development of functional vision (amblyopia), when the stimulus is presented for a very short time. When targets and flankers are dissimilar, the crowding is reduced (tagging). Since a combination of binocular inputs increases the processing load, we investigated whether color tagging the target reduces crowding in the fovea of subjects with normal vision and determined how crowding is combined with binocular vision. The crowding effect at the fovea was significantly reduced by tagging with a color target. Interestingly, whereas binocular summation for a single letter was expected to be about 40%, it was significantly reduced and almost absent under crowding conditions. Our results are consistent with the notion that the crowding effect produces a high processing load on visual processing, which interferes with other processes such as binocular summation. We assume that the tagging effect in our experiment improved the subject's abilities (sensitivity and RT) by creating a "segmentation", i.e., a visual simulated separation between the target letter and the background. Interestingly, tagging the target with a distinct color can eliminate or reduce the crowding effect and consequently, binocular summation recovers.

Mixture model investigation of the inner-outer asymmetry in visual crowding reveals a heavier weight towards the visual periphery

Crowding, the failure to identify a peripheral item in clutter, is an essential bottleneck in visual information processing. A hallmark characteristic of crowding is the inner-outer asymmetry in which the outer flanker (more eccentric) produces stronger interference than the inner one (closer to the fovea). We tested the contribution of the inner-outer asymmetry to the pattern of crowding errors in a typical radial crowding display in which both flankers are presented simultaneously on the horizontal meridian. In two experiments, observers were asked to estimate the orientation of a Gabor target. Instead of the target, observers reported the outer flanker much more frequently than the inner one. When the target was the outer Gabor, crowding was reduced. Furthermore, when there were four flankers, two on each side of the target, observers misreported the outer flanker adjacent to the target, not the outermost flanker. Model comparisons suggested that orientation crowding reflects sampling over a weighted sum of the represented features, in which the outer flanker is more heavily weighted compared to the inner one. Our findings reveal a counterintuitive phenomenon: in a radial arrangement of orientation crowding, within a region of selection, the outer item dominates appearance more than the inner one.

Excessive visual crowding effects in developmental dyscalculia

Visual crowding refers to the inability to identify objects when surrounded by other similar items. Crowding-like mechanisms are thought to play a key role in numerical perception by determining the sensory mechanisms through which ensembles are perceived. Enhanced visual crowding might hence prevent the normal development of a system involved in segregating and perceiving discrete numbers of items and ultimately the acquisition of more abstract numerical skills. Here, we investigated whether excessive crowding occurs in developmental dyscalculia (DD), a neurodevelopmental disorder characterized by difficulty in learning the most basic numerical and arithmetical concepts, and whether it is found independently of associated major reading and attentional difficulties. We measured spatial crowding in two groups of adult individuals with DD and control subjects. In separate experiments, participants were asked to discriminate the orientation of a Gabor patch either in isolation or under spatial crowding. Orientation discrimination thresholds were comparable across groups when stimuli were shown in isolation, yet they were much higher for the DD group with respect to the control group when the target was crowded by closely neighbouring flanking gratings. The difficulty in discriminating orientation (as reflected by the combination of accuracy and reaction times) in the DD compared to the control group persisted over several larger target flanker distances. Finally, we found that the degree of such spatial crowding correlated with impairments in mathematical abilities even when controlling for visual attention and reading skills. These results suggest that excessive crowding effects might be a characteristic of DD, independent of other associated neurodevelopmental disorders.

Multi-level Crowding and the Paradox of Object Recognition in Clutter

In everyday life, we are constantly surrounded by complex and cluttered scenes. In such cluttered environments, visual perception is primarily limited by crowding, the deleterious influence of nearby objects on object recognition. For the past several decades, visual crowding was assumed to occur at a single stage, only between low-level features or object parts, thus dismantling, destroying, or filtering object information. A large and converging body of evidence has demonstrated that this assumption is false: crowding occurs at multiple stages of visual analysis, and information passes through crowding at each of these stages. This converging empirical evidence points to a seeming paradox: crowding happens at multiple levels, which would seem to impair object recognition, and yet visual information at each of those levels is maintained intact and influences subsequent higher-level visual processing. Thus, while crowding impairs the access we have to visual information at many levels, it does not impair the representation of that information. The resolution of this paradox reveals how the visual system strikes a balance between the limits of object selection and the desire to represent multiple levels of visual information throughout cluttered scenes. Understanding crowding is therefore key to resolving the relationship between the richness of object and scene representations and the limits of conscious object recognition.

Challenges to pooling models of crowding: Implications for visual mechanisms

A set of phenomena known as crowding reveal peripheral vision's vulnerability in the face of clutter. Crowding is important both because of its ubiquity, making it relevant for many real-world tasks and stimuli, and because of the window it provides onto mechanisms of visual processing. Here we focus on models of the underlying mechanisms. This review centers on a popular class of models known as pooling models, as well as the phenomenology that appears to challenge a pooling account. Using a candidate high-dimensional pooling model, we gain intuitions about whether a pooling model suffices and reexamine the logic behind the pooling challenges. We show that pooling mechanisms can yield substitution phenomena and therefore predict better performance judging the properties of a set versus a particular item. Pooling models can also exhibit some similarity effects without requiring mechanisms that pool at multiple levels of processing, and without constraining pooling to a particular perceptual group. Moreover, we argue that other similarity effects may in part be due to noncrowding influences like cuing. Unlike low-dimensional straw-man pooling models, high-dimensional pooling preserves rich information about the stimulus, which may be sufficient to support high-level processing. To gain insights into the implications for pooling mechanisms, one needs a candidate high-dimensional pooling model and cannot rely on intuitions from low-dimensional models. Furthermore, to uncover the mechanisms of crowding, experiments need to separate encoding from decision effects. While future work must quantitatively examine all of the challenges to a high-dimensional pooling account, insights from a candidate model allow us to conclude that a high-dimensional pooling mechanism remains viable as a model of the loss of information leading to crowding.

Running Large-Scale Simulations on the Neurorobotics Platform to Understand Vision - The Case of Visual Crowding

Traditionally, human vision research has focused on specific paradigms and proposed models to explain very specific properties of visual perception. However, the complexity and scope of modern psychophysical paradigms undermine the success of this approach. For example, perception of an element strongly deteriorates when neighboring elements are presented in addition (visual crowding). As it was shown recently, the magnitude of deterioration depends not only on the directly neighboring elements but on almost all elements and their specific configuration. Hence, to fully explain human visual perception, one needs to take large parts of the visual field into account and combine all the aspects of vision that become relevant at such scale. These efforts require sophisticated and collaborative modeling. The Neurorobotics Platform (NRP) of the Human Brain Project offers a unique opportunity to connect models of all sorts of visual functions, even those developed by different research groups, into a coherently functioning system. Here, we describe how we used the NRP to connect and simulate a segmentation model, a retina model, and a saliency model to explain complex results about visual perception. The combination of models highlights the versatility of the NRP and provides novel explanations for inward-outward anisotropy in visual crowding.

Beyond Bouma's window: How to explain global aspects of crowding?

In crowding, perception of an object deteriorates in the presence of nearby elements. Although crowding is a ubiquitous phenomenon, since elements are rarely seen in isolation, to date there exists no consensus on how to model it. Previous experiments showed that the global configuration of the entire stimulus must be taken into account. These findings rule out simple pooling or substitution models and favor models sensitive to global spatial aspects. In order to investigate how to incorporate global aspects into models, we tested a large number of models with a database of forty stimuli tailored for the global aspects of crowding. Our results show that incorporating grouping like components strongly improves model performance.

Two's company, three's a crowd: Individuation is necessary for object recognition

Object recognition is essential for navigating the real world. Despite decades of research on this topic, the processing steps necessary for recognition remain unclear. In this study, we examined the necessity and role of individuation, the ability to select a small number of spatially distinct objects irrespective of their identity, in the recognition process. More specifically, we tested if the ability to rapidly individuate and enumerate a small number of objects (subitizing) can be impaired by crowding. Crowding is flanker-induced interference that specifically impedes the recognition process. We found that subitizing is impaired when objects are close to each other (Experiment 1), and if the target objects are surrounded by irrelevant but perceptually similar flankers (Experiments 2-4). This impairment cannot be attributed to confusion between targets and flankers, wherein flankers are inadvertently included in or targets are excluded from enumeration (Experiments 3-4). Importantly, the flanker induced interference was comparable in both subitizing and crowding tasks (Experiment 4), suggesting that individuation and identification share a common processing pathway. We conclude that individuation is an essential stage in the object recognition pipeline and argue for a cohesive proposal that both crowding and subitizing are due to limitations of selective attention.

Contextual-Dependent Attention Effect on Crowded Orientation Signals in Human Visual Cortex

A target becomes hard to identify with nearby visual stimuli. This phenomenon, known as crowding, places a fundamental limit on conscious perception and object recognition. To understand the neural representation of crowded stimuli, we used fMRI and a forward encoding model to reconstruct the target-specific feature from multivoxel activation patterns evoked by orientation patches. Orientation-selective response profiles were constructed in V1-V4 for a target embedded in different contexts. Subjects of both sexes either directed their attention over all the orientation patches or selectively to the target. In the context with a weak crowding effect, attending to the target enhanced the orientation selectivity of the response profile; such effect increased along the visual pathway. In the context with a strong crowding effect, attending to the target enhanced the orientation selectivity of the response profile in the earlier visual area, but not in V4. The increase and decrease of orientation selectivity along the visual hierarchy demonstrate a contextual-dependent attention effect on crowded orientation signals: in the context with a weak crowding effect, selective attention gradually resolves the target from nearby distractors along the hierarchy; in the context with a strong crowding effect, while selective attention maintains the target feature in the earlier visual area, its effect decreases in the downstream area. Our findings reveal how the human visual system represents the target-specific feature at multiple stages under the limit of attention selection in a cluttered scene.SIGNIFICANCE STATEMENT Using fMRI and a forward encoding model, we reconstructed orientation-selective response profiles for a target embedded in crowded contexts. In the context with a weak crowding effect, attention gradually resolves the target from nearby distractors along the visual hierarchy. In the context with a strong crowding effect, while the feature of the target is preserved in the early visual cortex, it degrades in the later visual processing stage. The increase and decrease of orientation selectivity along the visual hierarchy reveal how the human visual system strikes to present the target-specific feature under the limit of attention selection in a cluttered scene.

Crowding for faces is determined by visual (not holistic) similarity: Evidence from judgements of eye position

Crowding (the disruption of object recognition in clutter) presents the fundamental limitation on peripheral vision. For simple objects, crowding is strong when target/flanker elements are similar and weak when they differ - a selectivity for target-flanker similarity. In contrast, the identification of upright holistically-processed face stimuli is more strongly impaired by upright than inverted flankers, whereas inverted face-targets are impaired by both - a pattern attributed to an additional stage of crowding selective for "holistic similarity" between faces. We propose instead that crowding is selective for target-flanker similarity in all stimuli, but that this selectivity is obscured by task difficulty with inverted face-targets. Using judgements of horizontal eye-position that are minimally affected by inversion, we find that crowding is strong when target-flanker orientations match and weak when they differ for both upright and inverted face-targets. By increasing task difficulty, we show that this selectivity for target-flanker similarity is obscured even for upright face-targets. We further demonstrate that this selectivity follows differences in the spatial order of facial features, rather than "holistic similarity" per se. There is consequently no need to invoke a distinct stage of holistic crowding for faces - crowding is selective for target-flanker similarity, even with complex stimuli such as faces.

Ensemble Perception

To understand visual consciousness, we must understand how the brain represents ensembles of objects at many levels of perceptual analysis. Ensemble perception refers to the visual system's ability to extract summary statistical information from groups of similar objects-often in a brief glance. It defines foundational limits on cognition, memory, and behavior. In this review, we provide an operational definition of ensemble perception and demonstrate that ensemble perception spans across multiple levels of visual analysis, incorporating both low-level visual features and high-level social information. Further, we investigate the functional usefulness of ensemble perception and its efficiency, and we consider possible physiological and cognitive mechanisms that underlie an individual's ability to make accurate and rapid assessments of crowds of objects.

Visual crowding is a combination of an increase of positional uncertainty, source confusion, and featural averaging

Although we perceive a richly detailed visual world, our ability to identify individual objects is severely limited in clutter, particularly in peripheral vision. Models of such “crowding” have generally been driven by the phenomenological misidentifications of crowded targets: using stimuli that do not easily combine to form a unique symbol (e.g. letters or objects), observers typically confuse the source of objects and report either the target or a distractor, but when continuous features are used (e.g. orientated gratings or line positions) observers report a feature somewhere between the target and distractor. To reconcile these accounts, we develop a hybrid method of adjustment that allows detailed analysis of these multiple error categories. Observers reported the orientation of a target, under several distractor conditions, by adjusting an identical foveal target. We apply new modelling to quantify whether perceptual reports show evidence of positional uncertainty, source confusion, and featural averaging on a trial-by-trial basis. Our results show that observers make a large proportion of source-confusion errors. However, our study also reveals the distribution of perceptual reports that underlie performance in this crowding task more generally: aggregate errors cannot be neatly labelled because they are heterogeneous and their structure depends on target-distractor distance.

A Unifying Model of Orientation Crowding in Peripheral Vision

Peripheral vision is fundamentally limited not by the visibility of features, but by the spacing between them [1]. When too close together, visual features can become "crowded" and perceptually indistinguishable. Crowding interferes with basic tasks such as letter and face identification and thus informs our understanding of object recognition breakdown in peripheral vision [2]. Multiple proposals have attempted to explain crowding [3], and each is supported by compelling psychophysical and neuroimaging data [4-6] that are incompatible with competing proposals. In general, perceptual failures have variously been attributed to the averaging of nearby visual signals [7-10], confusion between target and distractor elements [11, 12], and a limited resolution of visual spatial attention [13]. Here we introduce a psychophysical paradigm that allows systematic study of crowded perception within the orientation domain, and we present a unifying computational model of crowding phenomena that reconciles conflicting explanations. Our results show that our single measure produces a variety of perceptual errors that are reported across the crowding literature. Critically, a simple model of the responses of populations of orientation-selective visual neurons accurately predicts all perceptual errors. We thus provide a unifying mechanistic explanation for orientation crowding in peripheral vision. Our simple model accounts for several perceptual phenomena produced by crowding of orientation and raises the possibility that multiple classes of object recognition failures in peripheral vision can be accounted for by a single mechanism.

Multiple Level Crowding: Crowding at the Object Parts Level and at the Object Configural level

In crowding, identification of a peripheral target in the presence of nearby flankers is worse than when the target appears alone. Prevailing theories hold that crowding occurs because of integration or "pooling" of low-level features at a single, relatively early stage of visual processing. Recent studies suggest that crowding can occur also between high-level object representations. The most relevant findings come from studies with faces and may be specific to faces. We examined whether crowding can occur at the object configural level in addition to part-level crowding, using nonface objects. Target (a disconnected square or diamond made of four elements) identification was measured at varying eccentricities. The flankers were similar either to the target parts or to the target configuration. The results showed crowding in both cases: Flankers interfered with target identification such that identification accuracy decreased with an increase in eccentricity, and no interference was observed at the fovea. Crowding by object parts, however, was weaker and had smaller spatial extent than crowding by object configurations; we related this finding to the relationship between crowding and perceptual organization. These results provide strong evidence that crowding occurs not only between object parts but also between configural representations of objects.