Uncorking the bottleneck of crowding: a fresh look at object recognition

The Role of Uniform Textures in Making Texture Elements Visible in the Visual Periphery

There are important differences between central and peripheral vision. With respect to shape, contours retain phenomenal sharpness, although some contours disappear if they are near other contours. This leads to some uniform textures to appear non-uniform (Honeycomb illusion, Bertamini et al., 2016). Unlike other phenomena of shape perception in the periphery, this illusion is showing how continuity of the texture does not contribute to phenomenal continuity. We systematically varied the relationship between central and peripheral regions, and we collected subjective reports (how far can one see lines) as well as judgments of line orientation. We used extended textures created with a square grid and some additional lines that are invisible when they are located at the corners of the grid, or visible when they are separated from the grid (control condition). With respects to subjective reports, we compared the region of visibility for cases in which the texture was uniform (Exp 1a), or when in a central region the lines were different (Exp 1b). There were no differences, showing no role of objective uniformity on visibility. Next, in addition to the region of visibility we measured sensitivity using a forced-choice task (line tilted left or right) (Exp 2). The drop in sensitivity with eccentricity matched the size of the region in which lines were perceived in the illusion condition, but not in the control condition. When participants were offered a choice to report of the lines were present or absent (Exp 3) they confirmed that they did not see them in the illusion condition, but saw them in the control condition. We conclude that mechanisms that control perception of contours operate differently in the periphery, and override prior expectations, including that of uniformity. Conversely, when elements are detected in the periphery, we assign to them properties based on information from central vision, but these shapes cannot be identified correctly when the task requires such discrimination.

Higher susceptibility to central crowding in glaucoma

CLINICAL RELEVANCE

Crowding limits many daily life activities, such as reading and the visual search for objects in cluttered environments. Excessive sensitivity to crowding, especially in central vision, may amplify the difficulties of patients with ocular pathologies. It is thus important to investigate what limits visual activities and how to improve it.

BACKGROUND

Numerous studies have reported reduced contrast sensitivity in central vision in patients with glaucoma. However, deficits have also been observed for letter recognition at high contrast, suggesting that contrast alone cannot completely account for impaired central perception.

METHOD

Seventeen patients and fifteen age-matched controls were randomly presented with letters in central or parafoveal vision at 5° eccentricity for 200 ms. They were asked to decide whether the central T was upright or inverted. The T was either presented in isolation (uncrowded) or flanked by two Hs (crowded) at various spacings. Contrast was manipulated: 60% and 5%.

RESULTS

Compared to controls, patients exhibited a significant effect of crowding in central vision, with higher accuracy for the isolated T than for HTH only at low contrast. In parafoveal vision, an effect of crowding was also observed only in patients. The spacing to escape crowding varied as a function of contrast. Larger spacing was required at low contrast than at high contrast. Susceptibility to crowding was related to central visual field defect for central presentations and to contrast sensitivity for parafoveal presentations, only at low contrast. Controls were at ceiling level both for central and parafoveal presentations.

CONCLUSION

Crowding limits visual perception, impeding reading and object recognition in cluttered environments. Visual field defects and lower contrast sensitivity in glaucoma can increase susceptibility to central and parafoveal crowding, the deleterious effect of which can be improved by manipulating contrast and spacing between elements.

A CODE model bridging crowding in sparse and dense displays

Visual crowding is arguably the strongest limitation imposed on extrafoveal vision, and is a relatively well-understood phenomenon. However, most investigations and theories are based on sparse displays consisting of a target and at most a handful of flanker objects. Recent findings suggest that the laws thought to govern crowding may not hold for densely cluttered displays, and that grouping and nearest neighbour effects may be more important. Here we present a computational model that accounts for crowding effects in both sparse and dense displays. The model is an adaptation and extension of an earlier model that has previously successfully accounted for spatial clustering, numerosity and object-based attention phenomena. Our model combines grouping by proximity and similarity with a nearest neighbour rule, and defines crowding as the extent to which target and flankers fail to segment. We show that when the model is optimized for explaining crowding phenomena in classic, sparse displays, it also does a good job in capturing novel crowding patterns in dense displays, in both existing and new data sets. The model thus ties together different principles governing crowding, specifically Bouma's law, grouping, and nearest neighbour similarity effects.

Large depth differences between target and flankers can increase crowding: Evidence from a multi-depth plane display

Crowding occurs when the presence of nearby features causes highly visible objects to become unrecognizable. Although crowding has implications for many everyday tasks and the tremendous amounts of research reflect its importance, surprisingly little is known about how depth affects crowding. Most available studies show that stereoscopic disparity reduces crowding, indicating that crowding may be relatively unimportant in three-dimensional environments. However, most previous studies tested only small stereoscopic differences in depth in which disparity, defocus blur, and accommodation are inconsistent with the real world. Using a novel multi-depth plane display, this study investigated how large (0.54-2.25 diopters), real differences in target-flanker depth, representative of those experienced between many objects in the real world, affect crowding. Our findings show that large differences in target-flanker depth increased crowding in the majority of observers, contrary to previous work showing reduced crowding in the presence of small depth differences. Furthermore, when the target was at fixation depth, crowding was generally more pronounced when the flankers were behind the target as opposed to in front of it. However, when the flankers were at fixation depth, crowding was generally more pronounced when the target was behind the flankers. These findings suggest that crowding from clutter outside the limits of binocular fusion can still have a significant impact on object recognition and visual perception in the peripheral field.

Measuring uncertainty in human visual segmentation

Segmenting visual stimuli into distinct groups of features and visual objects is central to visual function. Classical psychophysical methods have helped uncover many rules of human perceptual segmentation, and recent progress in machine learning has produced successful algorithms. Yet, the computational logic of human segmentation remains unclear, partially because we lack well-controlled paradigms to measure perceptual segmentation maps and compare models quantitatively. Here we propose a new, integrated approach: given an image, we measure multiple pixel-based same-different judgments and perform model-based reconstruction of the underlying segmentation map. The reconstruction is robust to several experimental manipulations and captures the variability of individual participants. We demonstrate the validity of the approach on human segmentation of natural images and composite textures. We show that image uncertainty affects measured human variability, and it influences how participants weigh different visual features. Because any putative segmentation algorithm can be inserted to perform the reconstruction, our paradigm affords quantitative tests of theories of perception as well as new benchmarks for segmentation algorithms.

The neuroconnectionist research programme

Artificial neural networks (ANNs) inspired by biology are beginning to be widely used to model behavioural and neural data, an approach we call 'neuroconnectionism'. ANNs have been not only lauded as the current best models of information processing in the brain but also criticized for failing to account for basic cognitive functions. In this Perspective article, we propose that arguing about the successes and failures of a restricted set of current ANNs is the wrong approach to assess the promise of neuroconnectionism for brain science. Instead, we take inspiration from the philosophy of science, and in particular from Lakatos, who showed that the core of a scientific research programme is often not directly falsifiable but should be assessed by its capacity to generate novel insights. Following this view, we present neuroconnectionism as a general research programme centred around ANNs as a computational language for expressing falsifiable theories about brain computation. We describe the core of the programme, the underlying computational framework and its tools for testing specific neuroscientific hypotheses and deriving novel understanding. Taking a longitudinal view, we review past and present neuroconnectionist projects and their responses to challenges and argue that the research programme is highly progressive, generating new and otherwise unreachable insights into the workings of the brain.

Attention relieves visual crowding: Dissociable effects of peripheral and central cues

Visual crowding can be reduced when attention is directed to the target by peripheral cues. However, it is unclear whether central cues relieve visual crowding to the same extent as peripheral cues. In this study, we combined the Posner cueing task and the crowding task to investigate the effect of exogenous and endogenous attention on crowding. In Experiment 1, five different stimulus-onset asychronies (SOAs) between the cue and the target and a predictive validity of 100% were adopted. Both attentional cues were shown to significantly reduce the effect of visual crowding, but the peripheral cue was more effective than the central cue. Furthermore, peripheral cues started to relieve visual crowding at the shortest SOA (100 ms), whereas central cues worked only at later SOAs (275 ms or above). When the predictive validity of the cue was decreased to 70% in Experiment 2, similar results to Experiment 1 were found, but the valid cue was less effective in reducing crowding than that in Experiment 1. In Experiment 3, when the predictive validity was decreased to 50%, a valid peripheral cue improved performance but a valid central cue did not, suggesting that endogenous attention but not exogenous attention can be voluntarily controlled when the cues are not predictive of the target's location. These findings collectively suggest that both peripheral and central cues can alleviate crowding, but they differ in terms of strength, time dynamics, and flexibility of voluntary control.

Intact and deficient contextual processing in schizophrenia patients

Schizophrenia patients are known to have deficits in contextual vision. However, results are often very mixed. In some paradigms, patients do not take the context into account and, hence, perform more veridically than healthy controls. In other paradigms, context deteriorates performance much more strongly in patients compared to healthy controls. These mixed results may be explained by differences in the paradigms as well as by small or biased samples, given the large heterogeneity of patients' deficits. Here, we show that mixed results may also come from idiosyncrasies of the stimuli used because in variants of the same visual paradigm, tested with the same participants, we found intact and deficient processing.

Unlocking crowding by ensemble statistics

In crowding,^{1,2,3,4,5,6,7} objects that can be easily recognized in isolation appear jumbled when surrounded by other elements.⁸ Traditionally, crowding is explained by local pooling mechanisms,^{3,6,9,10,11,12,13,14,15} but many findings have shown that the global configuration of the entire stimulus display, rather than local aspects, determines crowding.^{8,16,17,18,19,20,21,22,23,24,25,26,27,28} However, understanding global configurations is challenging because even slight changes can lead from crowding to uncrowding and vice versa.^23,25,28,29 Unfortunately, the number of configurations to explore is virtually infinite. Here, we show that one does not need to know the specific configuration of flankers to determine crowding strength but only their ensemble statistics, which allow for the rapid computation of groups within the stimulus display.^{30,31,32,33,34,35,36,37} To investigate the role of ensemble statistics in (un)crowding, we used a classic vernier offset discrimination task in which the vernier was flanked by multiple squares. We manipulated the orientation statistics of the squares based on the following rationale: a central square with an orientation different from the mean orientation of the other squares stands out from the rest and groups with the vernier, causing strong crowding. If, on the other hand, all squares group together, the vernier is the only element that stands out, and crowding is weak. These effects should depend exclusively on the perceived ensemble statistics, i.e., on the mean orientation of the squares and not on their individual orientations. In two experiments, we confirmed these predictions.

The Irreducibility of Vision: Gestalt, Crowding and the Fundamentals of Vision

What is fundamental in vision has been discussed for millennia. For philosophical realists and the physiological approach to vision, the objects of the outer world are truly given, and failures to perceive objects properly, such as in illusions, are just sporadic misperceptions. The goal is to replace the subjectivity of the mind by careful physiological analyses. Continental philosophy and the Gestaltists are rather skeptical or ignorant about external objects. The percepts themselves are their starting point, because it is hard to deny the truth of one own′s percepts. I will show that, whereas both approaches can well explain many visual phenomena with classic visual stimuli, they both have trouble when stimuli become slightly more complex. I suggest that these failures have a deeper conceptual reason, namely that their foundations (objects, percepts) do not hold true. I propose that only physical states exist in a mind independent manner and that everyday objects, such as bottles and trees, are perceived in a mind-dependent way. The fundamental processing units to process objects are extended windows of unconscious processing, followed by short, discrete conscious percepts.

Multidimensional feature interactions in visual crowding: When configural cues eliminate the polarity advantage

Crowding occurs when surrounding objects (flankers) impair target perception. A key property of crowding is the weaker interference when target and flankers strongly differ on a given dimension. For instance, identification of a target letter is usually superior with flankers of opposite versus the same contrast polarity as the target (the "polarity advantage"). High performance when target-flanker similarity is low has been attributed to the ungrouping of target and flankers. Here, we show that configural cues can override the usual advantage of low target-flanker similarity, and strong target-flanker grouping can reduce - instead of exacerbate - crowding. In Experiment 1, observers were presented with line triplets in the periphery and reported the tilt (left or right) of the central line. Target and flankers had the same (uniform condition) or opposite contrast polarity (alternating condition). Flanker configurations were either upright (||), unidirectionally tilted (\\ or //), or bidirectionally tilted (\/ or /\). Upright flankers yielded stronger crowding than unidirectional flankers, and weaker crowding than bidirectional flankers. Importantly, our results revealed a clear interaction between contrast polarity and flanker configuration. Triplets with upright and bidirectional flankers, but not unidirectional flankers, showed the polarity advantage. In Experiments 2 and 3, we showed that emergent features and redundancy masking (i.e. the reduction of the number of perceived items in repeating configurations) made it easier to discriminate between uniform triplets when flanker tilts were unidirectional (but not when bidirectional). We propose that the spatial configurations of uniform triplets with unidirectional flankers provided sufficient task-relevant information to enable a similar performance as with alternating triplets: strong-target flanker grouping alleviated crowding. We suggest that features which modulate crowding strength can interact non-additively, limiting the validity of typical crowding rules to contexts where only single, independent dimensions determine the effects of target-flanker similarity.

Central-peripheral dichotomy: color-motion and luminance-motion binding show stronger top-down feedback in central vision

Recently a theory (Zhaoping, Vision Research, 136, 32-49, 2017) proposed that top-down feedback from higher to lower visual cortical areas, to aid visual recognition, is stronger in the central than in the peripheral visual fields. Since top-down feedback helps feature binding, a critical visual recognition process, this theory predicts that insufficient feedback in the periphery should make feature misbinding more likely. To test this prediction, this study assessed binding between color and motion features, or between luminance and motion features, at different visual field eccentricities. We first used color-motion stimuli containing equiluminant red and green dots moving in opposite directions, for example, red dots moved leftward while green dots moved rightward. Such stimuli were shown in both a central reference strip and a peripheral test strip; participants reported whether it was the first or second interval in a trial in which the dots of each color moved in the opposite directions between the two strips. The center of the test strip was at 4° or 15° away from the gaze fixation. Participants' performance was much worse when the test strip was more peripheral, suggesting that feature misbinding occurred more frequently there. This held even when the size and density of the dots were adjusted by eccentricity-dependent cortical magnification factors, and even when red/green dots were replaced by yellow/blue dots or black/white dots to suit the retinal input sampling peripherally. Our findings support that top-down feedback is more directed to central vision, which can resolve ambiguities in feature binding at more central visual locations.

Global and high-level effects in crowding cannot be predicted by either high-dimensional pooling or target cueing

In visual crowding, the perception of a target deteriorates in the presence of nearby flankers. Traditionally, target-flanker interactions have been considered as local, mostly deleterious, low-level, and feature specific, occurring when information is pooled along the visual processing hierarchy. Recently, a vast literature of high-level effects in crowding (grouping effects and face-holistic crowding in particular) led to a different understanding of crowding, as a global, complex, and multilevel phenomenon that cannot be captured or explained by simple pooling models. It was recently argued that these high-level effects may still be captured by more sophisticated pooling models, such as the Texture Tiling model (TTM). Unlike simple pooling models, the high-dimensional pooling stage of the TTM preserves rich information about a crowded stimulus and, in principle, this information may be sufficient to drive high-level and global aspects of crowding. In addition, it was proposed that grouping effects in crowding may be explained by post-perceptual target cueing. Here, we extensively tested the predictions of the TTM on the results of six different studies that highlighted high-level effects in crowding. Our results show that the TTM cannot explain any of these high-level effects, and that the behavior of the model is equivalent to a simple pooling model. In addition, we show that grouping effects in crowding cannot be predicted by post-perceptual factors, such as target cueing. Taken together, these results reinforce once more the idea that complex target-flanker interactions determine crowding and that crowding occurs at multiple levels of the visual hierarchy.

The extended present: an informational context for perception

Several previous authors have proposed a kind of specious or subjective present moment that covers a few seconds of recent information. This article proposes a new hypothesis about the subjective present, renamed the extended present, defined not in terms of time covered but as a thematically connected information structure held in working memory and in transiently accessible form in long-term memory. The three key features of the extended present are that information in it is thematically connected, both internally and to current attended perceptual input, it is organised in a hierarchical structure, and all information in it is marked with temporal information, specifically ordinal and duration information. Temporal boundaries to the information structure are determined by hierarchical structure processing and by limits on processing and storage capacity. Supporting evidence for the importance of hierarchical structure analysis is found in the domains of music perception, speech and language processing, perception and production of goal-directed action, and exact arithmetical calculation. Temporal information marking is also discussed and a possible mechanism for representing ordinal and duration information on the time scale of the extended present is proposed. It is hypothesised that the extended present functions primarily as an informational context for making sense of current perceptual input, and as an enabler for perception and generation of complex structures and operations in language, action, music, exact calculation, and other domains.

Dissecting (un)crowding

In crowding, perception of a target deteriorates in the presence of nearby flankers. Surprisingly, perception can be rescued from crowding if additional flankers are added (uncrowding). Uncrowding is a major challenge for all classic models of crowding and vision in general, because the global configuration of the entire stimulus is crucial. However, it is unclear which characteristics of the configuration impact (un)crowding. Here, we systematically dissected flanker configurations and showed that (un)crowding cannot be easily explained by the effects of the sub-parts or low-level features of the stimulus configuration. Our modeling results suggest that (un)crowding requires global processing. These results are well in line with previous studies showing the importance of global aspects in crowding.

A comparative biology approach to DNN modeling of vision: A focus on differences, not similarities

Deep neural networks (DNNs) have revolutionized computer science and are now widely used for neuroscientific research. A hot debate has ensued about the usefulness of DNNs as neuroscientific models of the human visual system; the debate centers on to what extent certain shortcomings of DNNs are real failures and to what extent they are redeemable. Here, we argue that the main problem is that we often do not understand which human functions need to be modeled and, thus, what counts as a falsification. Hence, not only is there a problem on the DNN side, but there is also one on the brain side (i.e., with the explanandum-the thing to be explained). For example, should DNNs reproduce illusions? We posit that we can make better use of DNNs by adopting an approach of comparative biology by focusing on the differences, rather than the similarities, between DNNs and humans to improve our understanding of visual information processing in general.

Unraveling brain interactions in vision: The example of crowding

Crowding, the impairment of target discrimination in clutter, is the standard situation in vision. Traditionally, crowding is explained with (feedforward) models, in which only neighboring elements interact, leading to a "bottleneck" at the earliest stages of vision. It is with this implicit prior that most functional magnetic resonance imaging (fMRI) studies approach the identification of the "neural locus" of crowding, searching for the earliest visual area in which the blood-oxygenation-level-dependent (BOLD) signal is suppressed under crowded conditions. Using this classic approach, we replicated previous findings of crowding-related BOLD suppression starting in V2 and increasing up the visual hierarchy. Surprisingly, under conditions of uncrowding, in which adding flankers improves performance, the BOLD signal was further suppressed. This suggests an important role for top-down connections, which is in line with global models of crowding. To discriminate between various possible models, we used dynamic causal modeling (DCM). We show that recurrent interactions between all visual areas, including higher-level areas like V4 and the lateral occipital complex (LOC), are crucial in crowding and uncrowding. Our results explain the discrepancies in previous findings: in a recurrent visual hierarchy, the crowding effect can theoretically be detected at any stage. Beyond crowding, we demonstrate the need for models like DCM to understand the complex recurrent processing which most likely underlies human perception in general.

Shrinking Bouma's window: How to model crowding in dense displays

In crowding, perception of a target deteriorates in the presence of nearby flankers. Traditionally, it is thought that visual crowding obeys Bouma's law, i.e., all elements within a certain distance interfere with the target, and that adding more elements always leads to stronger crowding. Crowding is predominantly studied using sparse displays (a target surrounded by a few flankers). However, many studies have shown that this approach leads to wrong conclusions about human vision. Van der Burg and colleagues proposed a paradigm to measure crowding in dense displays using genetic algorithms. Displays were selected and combined over several generations to maximize human performance. In contrast to Bouma's law, only the target's nearest neighbours affected performance. Here, we tested various models to explain these results. We used the same genetic algorithm, but instead of selecting displays based on human performance we selected displays based on the model's outputs. We found that all models based on the traditional feedforward pooling framework of vision were unable to reproduce human behaviour. In contrast, all models involving a dedicated grouping stage explained the results successfully. We show how traditional models can be improved by adding a grouping stage.

Investigating Visual Crowding of Objects in Complex Real-World Scenes

Visual crowding, the impairment of object recognition in peripheral vision due to flanking objects, has generally been studied using simple stimuli on blank backgrounds. While crowding is widely assumed to occur in natural scenes, it has not been shown rigorously yet. Given that scene contexts can facilitate object recognition, crowding effects may be dampened in real-world scenes. Therefore, this study investigated crowding using objects in computer-generated real-world scenes. In two experiments, target objects were presented with four flanker objects placed uniformly around the target. Previous research indicates that crowding occurs when the distance between the target and flanker is approximately less than half the retinal eccentricity of the target. In each image, the spacing between the target and flanker objects was varied considerably above or below the standard (0.5) threshold to either suppress or facilitate the crowding effect. Experiment 1 cued the target location and then briefly flashed the scene image before participants could move their eyes. Participants then selected the target object's category from a 15-alternative forced choice response set (including all objects shown in the scene). Experiment 2 used eye tracking to ensure participants were centrally fixating at the beginning of each trial and showed the image for the duration of the participant's fixation. Both experiments found object recognition accuracy decreased with smaller spacing between targets and flanker objects. Thus, this study rigorously shows crowding of objects in semantically consistent real-world scenes.

Topological difference between target and flankers alleviates crowding effect

In the crowding effect, object recognition in the periphery deteriorates when other items flank the target, especially if they share similarities. Here, we report that the similarity defined by topological property (differences in number of holes) influences the crowding effect. Orientation discrimination tasks suggested that the crowding effect was weaker with a topological different (TD) flanker than a topological equivalent (TE) flanker and an existing inward-outward anisotropy phenomenon. In another experiment, both an outer and an inner flanker were used to constitute four different conditions. The performance of an outer TD flanker and an inner TE flanker was superior to that of an outer TE flanker and an inner TD flanker, even though the items of the stimuli were the same. Different stimuli were used to control for local features. To eliminate the possible explanation of confusability, we selected pairs of letters with matched confusability, but one pair was TD and another was TE. The letter identification performance was better for the TD condition. Lastly, we investigated the digit identification under four conditions with varied spacing. Regardless of different spacing, the crowding effect was reduced by a topological different flanker. The results collectively suggest that topological property plays a role in the perceptual grouping, which modulates the crowding effect.

Visual crowding effect in the parvocellular and magnocellular visual pathways

The crowding effect, defined as the detrimental effects of nearby items on visual object recognition, has been extensively investigated. Previous studies have primarily focused on finding the stage(s) in the visual hierarchy where crowding starts to limit target processing, while little attention has been focused on potential differences between the parvocellular (P) and magnocellular (M) pathways in crowding mechanisms. Here, we investigated the crowding effect in these parallel visual pathways. InExperiment 1, stimuli were designed to separately engage the P or M pathway, by tuning stimulus and background features (e.g., temporal frequency and color) to activate the targeted pathway and saturate the other pathway, respectively. Results showed that at the same eccentricity and with the same tasks, targets processed in the M pathway appeared to be more vulnerable to crowding effect. InExperiment 2, crowding effects were studied using three different types of stimuli and visual tasks (form, color, and motion), presumably with different degrees of dependence on the P and M pathways. Results revealed that color, motion, and form discrimination were increasingly more affected by crowding. We conclude that processing in the M and P pathways are differentially impacted by crowding; and importantly, crowding seems to affect processing of spatial forms more than other stimulus properties.

The perceived present: What is it, and what is it there for?

It is proposed that the perceived present is not a moment in time, but an information structure comprising an integrated set of products of perceptual processing. All information in the perceived present carries an informational time marker identifying it as "present". This marker is exclusive to information in the perceived present. There are other kinds of time markers, such as ordinality ("this stimulus occurred before that one") and duration ("this stimulus lasted for 50 ms"). These are different from the "present" time marker and may be attached to information regardless of whether it is in the perceived present or not. It is proposed that the perceived present is a very short-term and very high-capacity holding area for perceptual information. The maximum holding time for any given piece of information is ~100 ms: This is affected by the need to balance the value of informational persistence for further processing against the problem of obsolescence of the information. The main function of the perceived present is to facilitate access by other specialized, automatic processes.

Seven Myths on Crowding and Peripheral Vision

Crowding has become a hot topic in vision research, and some fundamentals are now widely agreed upon. For the classical crowding task, one would likely agree with the following statements. (1) Bouma's law can be stated, succinctly and unequivocally, as saying that critical distance for crowding is about half the target's eccentricity. (2) Crowding is predominantly a peripheral phenomenon. (3) Peripheral vision extends to at most 90° eccentricity. (4) Resolution threshold (the minimal angle of resolution) increases strongly and linearly with eccentricity. Crowding increases at an even steeper rate. (5) Crowding is asymmetric as Bouma has shown. For that inner-outer asymmetry, the peripheral flanker has more effect. (6) Critical crowding distance corresponds to a constant cortical distance in primary visual areas like V1. (7) Except for Bouma's seminal article in 1970, crowding research mostly became prominent starting in the 2000s. I propose the answer is "not really" or "not quite" to these assertions. So should we care? I think we should, before we write the textbook chapters for the next generation.

Dissociable effects of visual crowding on the perception of color and motion

Our ability to recognize objects in peripheral vision is fundamentally limited by crowding, the deleterious effect of clutter that disrupts the recognition of features ranging from orientation and color to motion and depth. Previous research is equivocal on whether this reflects a singular process that disrupts all features simultaneously or multiple processes that affect each independently. We examined crowding for motion and color, two features that allow a strong test of feature independence. "Cowhide" stimuli were presented 15° in peripheral vision, either in isolation or surrounded by flankers to give crowding. Observers reported either the target direction (clockwise/counterclockwise from upward) or its hue (blue/purple). We first established that both features show systematic crowded errors (biased predominantly toward the flanker identities) and selectivity for target-flanker similarity (with reduced crowding for dissimilar target/flanker elements). The multiplicity of crowding was then tested with observers identifying both features. Here, a singular object-selective mechanism predicts that when crowding is weak for one feature and strong for the other that crowding should be all-or-none for both. In contrast, when crowding was weak for color and strong for motion, errors were reduced for color but remained for motion, and vice versa with weak motion and strong color crowding. This double dissociation reveals that crowding disrupts certain combinations of visual features in a feature-specific manner, ruling out a singular object-selective mechanism. Thus, the ability to recognize one aspect of a cluttered scene, like color, offers no guarantees for the correct recognition of other aspects, like motion.

Crowding reveals fundamental differences in local vs. global processing in humans and machines

Feedforward Convolutional Neural Networks (ffCNNs) have become state-of-the-art models both in computer vision and neuroscience. However, human-like performance of ffCNNs does not necessarily imply human-like computations. Previous studies have suggested that current ffCNNs do not make use of global shape information. However, it is currently unclear whether this reflects fundamental differences between ffCNN and human processing or is merely an artefact of how ffCNNs are trained. Here, we use visual crowding as a well-controlled, specific probe to test global shape computations. Our results provide evidence that ffCNNs cannot produce human-like global shape computations for principled architectural reasons. We lay out approaches that may address shortcomings of ffCNNs to provide better models of the human visual system.

Parietal tACS at beta frequency improves vision in a crowding regime

Visual crowding is the inability to discriminate objects when presented with nearby flankers and sets a fundamental limit for conscious perception. Beta oscillations in the parietal cortex were found to be associated to crowding, with higher beta amplitude related to better crowding resilience. An open question is whether beta activity directly and selectively modulates crowding. We employed Transcranial Alternating Current Stimulation (tACS) in the beta band (18-Hz), in the alpha band (10-Hz) or in a sham regime, asking whether 18-Hz tACS would selectively improve the perception of crowded stimuli by increasing parietal beta activity. Resting-state electroencephalography (EEG) was measured before and after stimulation to test the influence of tACS on endogenous oscillations. Consistently with our predictions, we found that 18-Hz tACS, as compared to 10-Hz tACS and sham stimulation, reduced crowding. This improvement was found specifically in the contralateral visual hemifield and was accompanied by an increased amplitude of EEG beta oscillations, confirming an effect on endogenous brain rhythms. These results support a causal relationship between parietal beta oscillations and visual crowding and provide new insights into the precise oscillatory mechanisms involved in human vision.

Exploring the Extent in the Visual Field of the Honeycomb and Extinction Illusions

There are situations in which what is perceived in central vision is different to what is perceived in the periphery, even though the stimulus display is uniform. Here, we studied two cases, known as the Extinction illusion and the Honeycomb illusion, involving small disks and lines, respectively, presented over a large extent of the visual field. Disks and lines are visible in the periphery on their own, but they become invisible when they are presented as part of a pattern (grid). Observers (N = 56) adjusted a circular probe to report the size of the region in which they had seen the lines or the disks. Different images had black or white lines/disks, and we included control stimuli in which these features were spatially separated from the regular grid of squares. We confirmed that the illusion was experienced by the majority of observers and is dependent on the interaction between the elements (i.e., the lines/disks have to be near the squares). We found a dissociation between the two illusions in the dependence on contrast polarity suggesting different mechanisms. We analysed the variability between individuals with respect to schizotypical and autistic-spectrum traits (short version of the Oxford-Liverpool Inventory of Feelings and Experiences [O-LIFE] questionnaire and the Autistic Quotient, respectively) but found no significant relationships. We discuss how illusions relative to what observers are aware of in the periphery may offer a unique tool to study visual awareness.

Running Large-Scale Simulations on the Neurorobotics Platform to Understand Vision - The Case of Visual Crowding

Traditionally, human vision research has focused on specific paradigms and proposed models to explain very specific properties of visual perception. However, the complexity and scope of modern psychophysical paradigms undermine the success of this approach. For example, perception of an element strongly deteriorates when neighboring elements are presented in addition (visual crowding). As it was shown recently, the magnitude of deterioration depends not only on the directly neighboring elements but on almost all elements and their specific configuration. Hence, to fully explain human visual perception, one needs to take large parts of the visual field into account and combine all the aspects of vision that become relevant at such scale. These efforts require sophisticated and collaborative modeling. The Neurorobotics Platform (NRP) of the Human Brain Project offers a unique opportunity to connect models of all sorts of visual functions, even those developed by different research groups, into a coherently functioning system. Here, we describe how we used the NRP to connect and simulate a segmentation model, a retina model, and a saliency model to explain complex results about visual perception. The combination of models highlights the versatility of the NRP and provides novel explanations for inward-outward anisotropy in visual crowding.

Beyond Bouma's window: How to explain global aspects of crowding?

In crowding, perception of an object deteriorates in the presence of nearby elements. Although crowding is a ubiquitous phenomenon, since elements are rarely seen in isolation, to date there exists no consensus on how to model it. Previous experiments showed that the global configuration of the entire stimulus must be taken into account. These findings rule out simple pooling or substitution models and favor models sensitive to global spatial aspects. In order to investigate how to incorporate global aspects into models, we tested a large number of models with a database of forty stimuli tailored for the global aspects of crowding. Our results show that incorporating grouping like components strongly improves model performance.

Visual crowding involves delayed frontoparietal response and enhanced top-down modulation

Crowding, the disrupted recognition of a peripheral target in the presence of nearby flankers, sets a fundamental limit on peripheral vision perception. Debates persist on whether the limit occurs at early visual cortices or is induced by top-down modulation, leaving the neural mechanism for visual crowding largely unclear. To resolve the debate, it is crucial to extract the neural signals elicited by the target from that by the target-flanker clutter, with high temporal resolution. To achieve this purpose, here we employed a temporal response function (TRF) approach to dissociate target-specific response from the overall electroencephalograph (EEG) recordings when the target was presented with (crowded) or without flankers (uncrowded) while subjects were performing a discrimination task on the peripherally presented target. Our results demonstrated two components in the target-specific contrast-tracking TRF response-an early component (100-170 ms) in occipital channels and a late component (210-450 ms) in frontoparietal channels. The late frontoparietal component, which was delayed in time under the crowded condition, was correlated with target discrimination performance, suggesting its involvement in visual crowding. Granger causality analysis further revealed stronger top-down modulation on the target stimulus under the crowded condition. Taken together, our findings support that crowding is associated with a top-down process which modulates the low-level sensory processing and delays the behavioral-relevant response in the high-level region.

Two's company, three's a crowd: Individuation is necessary for object recognition

Object recognition is essential for navigating the real world. Despite decades of research on this topic, the processing steps necessary for recognition remain unclear. In this study, we examined the necessity and role of individuation, the ability to select a small number of spatially distinct objects irrespective of their identity, in the recognition process. More specifically, we tested if the ability to rapidly individuate and enumerate a small number of objects (subitizing) can be impaired by crowding. Crowding is flanker-induced interference that specifically impedes the recognition process. We found that subitizing is impaired when objects are close to each other (Experiment 1), and if the target objects are surrounded by irrelevant but perceptually similar flankers (Experiments 2-4). This impairment cannot be attributed to confusion between targets and flankers, wherein flankers are inadvertently included in or targets are excluded from enumeration (Experiments 3-4). Importantly, the flanker induced interference was comparable in both subitizing and crowding tasks (Experiment 4), suggesting that individuation and identification share a common processing pathway. We conclude that individuation is an essential stage in the object recognition pipeline and argue for a cohesive proposal that both crowding and subitizing are due to limitations of selective attention.

The Temporal Dynamic Relationship Between Attention and Crowding: Electrophysiological Evidence From an Event-Related Potential Study

Visual crowding is the difficulty experienced in identifying a target flanked by other objects within the peripheral visual field. Despite extensive research conducted on this topic, the precise relationship between attention and crowding is still debatable. One perspective suggests that crowding is a bottom-up and pre-attentive process, while another suggests that crowding is top-down and attentional. A third perspective proposes that crowding is a combination of bottom-up and top-down processes. To address this debate, the current study manipulated the attention and distance between targets and flankers, while simultaneously measuring event-related potentials, in human participants. Results indicated that, compared to uncrowded targets, crowded targets elicited more negative frontal N1 and P2 activity and a less negative occipital N1 activity, regardless of whether targets were attended or unattended, and a more positive occipital P2 activity when they were attended. Furthermore, the crowded minus uncrowded difference amplitude was more negative over the frontal region and more positive over the occipital region when the targets were attended, compared to when they were unattended during the N1 and P2 stages. This suggests that crowding, a concept that originates from Gestalt grouping, occurs automatically and can be modulated by attention.

Critical resolution: A superior measure of crowding

Visual object recognition is essential for adaptive interactions with the environment. It is fundamentally limited by crowding, a breakdown of object recognition in clutter. The spatial extent over which crowding occurs is proportional to the eccentricity of the target object, but nevertheless varies substantially depending on various stimulus factors (e.g. viewing time, contrast). However, a lack of studies jointly manipulating such factors precludes predictions of crowding in more heterogeneous scenes, such as the majority of real life situations.

To establish how such co-occurring variations affect crowding, we manipulated combinations of 1) flanker contrast and backward masking, 2) flanker contrast and presentation duration, and 3) flanker preview and pop-out while measuring participants’ ability to correctly report the orientation of a target stimulus. In all three experiments, combining two manipulations consistently modulated the spatial extent of crowding in a way that could not be predicted from an additive combination. However, a simple transformation of the measurement scale completely abolished these interactions and all effects became additive. Precise quantitative predictions of the magnitude of crowding when combining multiple manipulations are thus possible when it is expressed in terms of what we label the ‘critical resolution’. Critical resolution is proportional to the inverse of the smallest flanker free area surrounding the target object necessary for its unimpaired identification. It offers a more parsimonious description of crowding than the traditionally used critical spacing and may thus constitute a measure of fundamental importance for understanding object recognition.

Contour interaction under photopic and scotopic conditions

In the present study, we asked whether contour interaction undergoes significant changes for different luminance levels in the central and peripheral visual field. This study included nine normal observers at two laboratories (five at Palacky University Olomouc, Czech Republic and four at the University of Houston, USA). Observers viewed a randomly selected Sloan letter surrounded by four equally spaced bars for several separations measured edge-to-edge in min arc. Stimuli were viewed foveally under photopic and mesopic luminances and between 5° and 12° peripherally for four different background luminances of the display monitors, corresponding to photopic, mesopic, scotopic, and dim scotopic levels. The extent of the contour interaction in the fovea is approximately 20 times smaller than in the periphery. Whereas the magnitude of foveal contour interaction markedly decreases with decreasing luminance, no consistent luminance-induced change occurs in peripheral contour interaction. The extent of contour interaction does not scale with the size of the target letter, either in the fovea or peripherally. The results support a neural origin of contour interaction consistent with the properties of center-surround antagonism.

Perceptual grouping

A Quick guide to perceptual grouping, the way that we group individual elements that we perceive, such as parts of a visual scene, together to make sense of the world.

The perceived stability of scenes: serial dependence in ensemble representations

We are continuously surrounded by a noisy and ever-changing environment. Instead of analyzing all the elements in a scene, our visual system has the ability to compress an enormous amount of visual information into ensemble representations, such as perceiving a forest instead of every single tree. Still, it is unclear why such complex scenes appear to be the same from moment to moment despite fluctuations, noise, and discontinuities in retinal images. The general effects of change blindness are usually thought to stabilize scene perception, making us unaware of minor inconsistencies between scenes. Here, we propose an alternative, that stable scene perception is actively achieved by the visual system through global serial dependencies: the appearance of scene gist is sequentially dependent on the gist perceived in previous moments. To test this hypothesis, we used summary statistical information as a proxy for “gist” level, global information in a scene. We found evidence for serial dependence in summary statistical representations. Furthermore, we show that this kind of serial dependence occurs at the ensemble level, where local elements are already merged into global representations. Taken together, our results provide a mechanism through which serial dependence can promote the apparent consistency of scenes over time.

Visual crowding is a combination of an increase of positional uncertainty, source confusion, and featural averaging

Although we perceive a richly detailed visual world, our ability to identify individual objects is severely limited in clutter, particularly in peripheral vision. Models of such “crowding” have generally been driven by the phenomenological misidentifications of crowded targets: using stimuli that do not easily combine to form a unique symbol (e.g. letters or objects), observers typically confuse the source of objects and report either the target or a distractor, but when continuous features are used (e.g. orientated gratings or line positions) observers report a feature somewhere between the target and distractor. To reconcile these accounts, we develop a hybrid method of adjustment that allows detailed analysis of these multiple error categories. Observers reported the orientation of a target, under several distractor conditions, by adjusting an identical foveal target. We apply new modelling to quantify whether perceptual reports show evidence of positional uncertainty, source confusion, and featural averaging on a trial-by-trial basis. Our results show that observers make a large proportion of source-confusion errors. However, our study also reveals the distribution of perceptual reports that underlie performance in this crowding task more generally: aggregate errors cannot be neatly labelled because they are heterogeneous and their structure depends on target-distractor distance.

Putting low-level vision into global context: Why vision cannot be reduced to basic circuits

To cope with the complexity of vision, most models in neuroscience and computer vision are of hierarchical and feedforward nature. Low-level vision, such as edge and motion detection, is explained by basic low-level neural circuits, whose outputs serve as building blocks for more complex circuits computing higher level features such as shape and entire objects. There is an isomorphism between states of the outer world, neural circuits, and perception, inspired by the positivistic philosophy of the mind. Here, we show that although such an approach is conceptually and mathematically appealing, it fails to explain many phenomena including crowding, visual masking, and non-retinotopic processing.

Multiple Level Crowding: Crowding at the Object Parts Level and at the Object Configural level

In crowding, identification of a peripheral target in the presence of nearby flankers is worse than when the target appears alone. Prevailing theories hold that crowding occurs because of integration or "pooling" of low-level features at a single, relatively early stage of visual processing. Recent studies suggest that crowding can occur also between high-level object representations. The most relevant findings come from studies with faces and may be specific to faces. We examined whether crowding can occur at the object configural level in addition to part-level crowding, using nonface objects. Target (a disconnected square or diamond made of four elements) identification was measured at varying eccentricities. The flankers were similar either to the target parts or to the target configuration. The results showed crowding in both cases: Flankers interfered with target identification such that identification accuracy decreased with an increase in eccentricity, and no interference was observed at the fovea. Crowding by object parts, however, was weaker and had smaller spatial extent than crowding by object configurations; we related this finding to the relationship between crowding and perceptual organization. These results provide strong evidence that crowding occurs not only between object parts but also between configural representations of objects.