Multi-level Crowding and the Paradox of Object Recognition in Clutter

Enhancing human-AI collaboration: The case of colonoscopy

Diagnostic errors impact patient health and healthcare costs. Artificial Intelligence (AI) shows promise in mitigating this burden by supporting Medical Doctors in decision-making. However, the mere display of excellent or even superhuman performance by AI in specific tasks does not guarantee a positive impact on medical practice. Effective AI assistance should target the primary causes of human errors and foster effective collaborative decision-making with human experts who remain the ultimate decision-makers. In this narrative review, we apply these principles to the specific scenario of AI assistance during colonoscopy. By unraveling the neurocognitive foundations of the colonoscopy procedure, we identify multiple bottlenecks in perception, attention, and decision-making that contribute to diagnostic errors, shedding light on potential interventions to mitigate them. Furthermore, we explored how existing AI devices fare in clinical practice and whether they achieved an optimal integration with the human decision-maker. We argue that to foster optimal Human-AI collaboration, future research should expand our knowledge of factors influencing AI's impact, establish evidence-based cognitive models, and develop training programs based on them. These efforts will enhance human-AI collaboration, ultimately improving diagnostic accuracy and patient outcomes. The principles illuminated in this review hold more general value, extending their relevance to a wide array of medical procedures and beyond.

Iconic logic: the visual art of drawing the right conclusion

Most people, evidence suggests, have a hard time thinking straight. Symbolic logic is a tool that can help remedy this problem. Unfortunately, it is highly abstract and uses symbols whose meanings rely on unintuitive arbitrary conventions. Without sacrificing rigor, iconic logic is more concrete and uses icons that resemble what they stand for and whose meanings are thus easier to picture, process, and remember. Here I review and critique iconic existential graphs and concept diagrams-the former link iconic logic to iconic mathematics; the latter expand popular Euler or Venn diagrams and have, to some degree, been empirically investigated for user-friendliness. I lay out how expertise in perception, cognition, and genetics can inform and improve such empirical research to help make iconic logic more ergonomic. After all, logic is a tool, and tools should not only suit their use but also their user.

The word superiority effect overcomes crowding

Crowding and the word superiority effect are two perceptual phenomena that influence reading. The identification of the inner letters of a word can be hindered by crowding from adjacent letters, but it can be facilitated by the word context itself (the word superiority effect). In the present study, strings of four-letters (words and non-words) with different inter-letter spacings (ranging from an optimal spacing to produce crowding to a spacing too large to produce crowding) were presented briefly in the periphery and participants were asked to identify the third letter of the string. Each word had a partner word that was identical except for its third letter (e.g., COLD, CORD) so that guessing as the source of the improved performance for words could be ruled out. Unsurprisingly, letter identification accuracy for words was better than non-words. For non-words, it was lowest at closer spacings, confirming crowding. However, for words, accuracy remained high at all inter-letter spacings showing that crowding did not prevent identification of the inner letters. This result supports models of "holistic" word recognition where partial cues can lead to recognition without first identifying individual letters. Once the word is recognized, its inner letters can be recovered, despite their feature loss produced by crowding.

Memorability shapes perceived time (and vice versa)

Visual stimuli are known to vary in their perceived duration. Some visual stimuli are also known to linger for longer in memory. Yet, whether these two features of visual processing are linked is unknown. Despite early assumptions that time is an extracted or higher-order feature of perception, more recent work over the past two decades has demonstrated that timing may be instantiated within sensory modality circuits. A primary location for many of these studies is the visual system, where duration-sensitive responses have been demonstrated. Furthermore, visual stimulus features have been observed to shift perceived duration. These findings suggest that visual circuits mediate or construct perceived time. Here we present evidence across a series of experiments that perceived time is affected by the image properties of scene size, clutter and memorability. More specifically, we observe that scene size and memorability dilate time, whereas clutter contracts it. Furthermore, the durations of more memorable images are also perceived more precisely. Conversely, the longer the perceived duration of an image, the more memorable it is. To explain these findings, we applied a recurrent convolutional neural network model of the ventral visual system, in which images are progressively processed over time. We find that more memorable images are processed faster, and that this increase in processing speed predicts both the lengthening and the increased precision of perceived durations. These findings provide evidence for a link between image features, time perception and memory that can be further explored with models of visual processing.

The Role of Uniform Textures in Making Texture Elements Visible in the Visual Periphery

There are important differences between central and peripheral vision. With respect to shape, contours retain phenomenal sharpness, although some contours disappear if they are near other contours. This leads to some uniform textures to appear non-uniform (Honeycomb illusion, Bertamini et al., 2016). Unlike other phenomena of shape perception in the periphery, this illusion is showing how continuity of the texture does not contribute to phenomenal continuity. We systematically varied the relationship between central and peripheral regions, and we collected subjective reports (how far can one see lines) as well as judgments of line orientation. We used extended textures created with a square grid and some additional lines that are invisible when they are located at the corners of the grid, or visible when they are separated from the grid (control condition). With respects to subjective reports, we compared the region of visibility for cases in which the texture was uniform (Exp 1a), or when in a central region the lines were different (Exp 1b). There were no differences, showing no role of objective uniformity on visibility. Next, in addition to the region of visibility we measured sensitivity using a forced-choice task (line tilted left or right) (Exp 2). The drop in sensitivity with eccentricity matched the size of the region in which lines were perceived in the illusion condition, but not in the control condition. When participants were offered a choice to report of the lines were present or absent (Exp 3) they confirmed that they did not see them in the illusion condition, but saw them in the control condition. We conclude that mechanisms that control perception of contours operate differently in the periphery, and override prior expectations, including that of uniformity. Conversely, when elements are detected in the periphery, we assign to them properties based on information from central vision, but these shapes cannot be identified correctly when the task requires such discrimination.

Peripheral vision is mainly for looking rather than seeing

Vision includes looking and seeing. Looking, mainly via gaze shifts, selects a fraction of visual input information for passage through the brain's information bottleneck. The selected input is placed within the attentional spotlight, typically in the central visual field. Seeing decodes, i.e., recognizes and discriminates, the selected inputs. Hence, peripheral vision should be mainly devoted to looking, in particular, deciding where to shift the gaze. Looking is often guided exogenously by a saliency map created by the primary visual cortex (V1), and can be effective with no seeing and limited awareness. In seeing, peripheral vision not only suffers from poor spatial resolution, but is also subject to crowding and is more vulnerable to illusions by misleading, ambiguous, and impoverished visual inputs. Central vision, mainly for seeing, enjoys the top-down feedback that aids seeing in light of the bottleneck which is hypothesized to starts from V1 to higher areas. This feedback queries for additional information from lower visual cortical areas such as V1 for ongoing recognition. Peripheral vision is deficient in this feedback according to the Central-peripheral Dichotomy (CPD) theory. The saccades engendered by peripheral vision allows looking to combine with seeing to give human observers the impression of seeing the whole scene clearly despite inattentional blindness.

Non-uniform contextual interactions in the visual cortex place fundamental limits on spatial vision

A prevailing assumption in our understanding of how neurons in the primary visual cortex (V1) integrate contextual information is that such processes are spatially uniform. Conversely, perceptual phenomena such as visual crowding, the impaired ability to accurately recognize a target stimulus among distractors, suggest that interactions among stimuli are distinctly non-uniform. Prior studies have shown flankers at specific spatial geometries exert differential effects on target perception. To resolve this discrepancy, we investigated how flanker geometry impacted the representation of a target stimulus in the laminar microcircuits of V1. Our study reveals flanker location differentially impairs stimulus representation in excitatory neurons in the superficial and input layers of V1 by tuned suppression and untuned facilitation of orientation responses. Mechanistically, this effect can be explained by asymmetrical spatial kernels in a normalization model of cortical activity. Strikingly, these non-uniform modulations of neural representation mirror perceptual anisotropies. These results establish the non-uniform spatial integration of information in the earliest stages of cortical processing as a fundamental limitation of spatial vision.

Seeing in crowds: Averaging first, then max

Crowding, a fundamental limit in object recognition, is believed to result from excessive integration of nearby items in peripheral vision. To understand its pooling mechanisms, we measured subjects' internal response distributions in an orientation crowding task. Contrary to the prediction of an averaging model, we observed a pattern suggesting that the perceptual judgement is made based on choosing the largest response across the noise-perturbed items. A model featuring first-stage averaging and second-stage signed-max operation predicts the diverse errors made by human observers under various signal strength levels. These findings suggest that different rules operate to resolve the bottleneck at early and high-level stages of visual processing, implementing a combination of linear and nonlinear pooling strategies.

Abnormal basic visual processing functions in binocular fusion disorders

Heterophoria is a common type of binocular fusion disorder that consists of a latent eye misalignment with potential consequences on daily activities such as reading or working on a computer (with CVS). Crowding, a type of contextual modulation, can also impair reading. Our recent studies found an abnormal pattern of low-level visual processing with larger perceptive fields (PF) in heterophoria. The PF is the fundamental processing unit of human vision and both masking and crowding depend on its size. We investigated how heterophoria would impact the PF's size via a lateral masking experiment and consequently affect the foveal crowding at different letter-spacings (the crowding zone). More specifically, we explored the relationship between crowding, lateral masking, the PF's size, and the amount of heterophoria. The binocular horizontal PF's size was larger with heterophoric subjects, in agreement with our previous study. We found a stronger crowding and an extended crowding zone associated with slower response times; this shows that the processing of letter identification under both crowded and uncrowded conditions requires more processing effort in heterophoric individuals. In agreement with previous studies, we found a correlation between the crowding zone and the PF's size; each was strongly correlated with the amount of phoria. These findings resemble those involving the PF size and the extended crowding found at the fovea in amblyopia and young children. We suggest that these findings could help explain the inter-observers' variability found in the masking literature, and the reading difficulties often encountered in subjects with high heterophoria.

Neural correlates of crowding in macaque area V4

Visual crowding refers to the phenomenon where a target object that is easily identifiable in isolation becomes difficult to recognize when surrounded by other stimuli (distractors). Extensive psychophysical studies support two alternative possibilities for the underlying mechanisms. One hypothesis suggests that crowding results from the loss of visual information due to pooled encoding of multiple nearby stimuli in the mid-level processing stages along the ventral visual pathway. Alternatively, crowding may arise from limited resolution in decoding object information during recognition and the encoded information may remain inaccessible unless it is salient. To rigorously test these alternatives, we studied the responses of single neurons in macaque area V4, an intermediate stage of the ventral, object-processing pathway, to parametrically designed crowded displays and their texture-statistics matched metameric counterparts. Our investigations reveal striking parallels between how crowding parameters, e.g., number, distance, and position of distractors, influence human psychophysical performance and V4 shape selectivity. Importantly, we found that enhancing the salience of a target stimulus could reverse crowding effects even in highly cluttered scenes and such reversals could be protracted reflecting a dynamical process. Overall, we conclude that a pooled encoding of nearby stimuli cannot explain the observed responses and we propose an alternative model where V4 neurons preferentially encode salient stimuli in crowded displays.

Large depth differences between target and flankers can increase crowding: Evidence from a multi-depth plane display

Crowding occurs when the presence of nearby features causes highly visible objects to become unrecognizable. Although crowding has implications for many everyday tasks and the tremendous amounts of research reflect its importance, surprisingly little is known about how depth affects crowding. Most available studies show that stereoscopic disparity reduces crowding, indicating that crowding may be relatively unimportant in three-dimensional environments. However, most previous studies tested only small stereoscopic differences in depth in which disparity, defocus blur, and accommodation are inconsistent with the real world. Using a novel multi-depth plane display, this study investigated how large (0.54-2.25 diopters), real differences in target-flanker depth, representative of those experienced between many objects in the real world, affect crowding. Our findings show that large differences in target-flanker depth increased crowding in the majority of observers, contrary to previous work showing reduced crowding in the presence of small depth differences. Furthermore, when the target was at fixation depth, crowding was generally more pronounced when the flankers were behind the target as opposed to in front of it. However, when the flankers were at fixation depth, crowding was generally more pronounced when the target was behind the flankers. These findings suggest that crowding from clutter outside the limits of binocular fusion can still have a significant impact on object recognition and visual perception in the peripheral field.

Cortical Reorganization After Optical Alignment in Strabismic Patients Outside of Critical Period

PURPOSE

To measure visual crowding, an essential bottleneck on object recognition and reliable psychophysical index of cortex organization, in older children and adults with horizontal concomitant strabismus before and after strabismus surgery.

METHODS

Using real-time eye tracking to ensure gaze-contingent display, we examined the peripheral visual crowding effects in older children and adults with horizontal concomitant strabismus but without amblyopia before and after strabismus surgery. Patients were asked to discriminate the orientation of the central tumbling E target letter with flankers arranged along the radial or tangential axis in the nasal or temporal hemifield at different eccentricities (5° or 10°). The critical spacing value, which is the minimum space between the target and the flankers required for correct discrimination, was obtained for comparisons before and after strabismus surgery.

RESULTS

Twelve individuals with exotropia (6 males, 21.75 ± 7.29 years, mean ± SD) and 15 individuals with esotropia (6 males, 24.13 ± 5.96 years) participated in this study. We found that strabismic individuals showed significantly larger critical spacing with nasotemporal asymmetry along the radial axis that related to the strabismus pattern, with exotropes exhibiting stronger temporal field crowding and esotropes exhibiting stronger nasal field crowding before surgical alignment. After surgery, the critical spacing was reduced and rebalanced between the nasal and temporal hemifields. Furthermore, the postoperative recovery of stereopsis was associated with the extent of nasotemporal balance of critical spacing.

CONCLUSIONS

We find that optical realignment (i.e., strabismus surgery) can normalize the enlarged visual crowding effects, a reliable psychophysical index of cortical organization, in the peripheral visual field of older children and adults with strabismus and rebalance the nasotemporal asymmetry of crowding, promoting the recovery of postoperative stereopsis. Our results indicated a potential of experience-dependent cortical organization after axial alignment even for individuals who are out of the critical period of visual development, illuminating the capacity and limitations of optics on sensory plasticity and emphasizing the importance of ocular correction for clinical practice.

The impact of training on the inner-outer asymmetry in crowding

Inner-outer asymmetry, where the outer flanker induces stronger crowding than the inner flanker, is a hallmark property of visual crowding. It is unclear the contribution of inner-outer asymmetry to the pattern of crowding errors (biased predominantly toward the flanker identities) and the role of training on crowding errors. In a typical radial crowding display, 20 observers were asked to report the orientation of a target Gabor (7.5° eccentricity) flanked by either an inner or outer Gabor along the horizontal meridian. The results showed that outer flanker conditions induced stronger crowding, accompanied by assimilative errors to the outer flanker for similar target/flanker elements. In contrast, the inner flanker condition exhibited weaker crowding, with no significant patterns of crowding errors. A population coding model showed that the flanker weights in the outer flanker condition were significantly higher than those in the inner flanker condition. Nine observers continued to train the outer flanker condition for four sessions. Training reduced inner-outer asymmetry and reduced flanker weights to the outer flanker. The learning effects were retained over 4 to 6 months. Individual differences in the appearance of crowding errors, the strength of inner-outer asymmetry, and the training effects were evident. Nevertheless, our findings indicate that different crowding mechanisms may be responsible for the asymmetric crowding effects induced by inner and outer flankers, with the outer flankers dominating the appearance more than the inner ones. Training reduces inner-outer asymmetry by reducing target/flanker confusion, and learning is persistent over months, suggesting that perceptual learning has the potential to improve visual performance by promoting neural plasticity.

Good-enough attentional guidance

Theories of attention posit that attentional guidance operates on information held in a target template within memory. The template is often thought to contain veridical target features, akin to a photograph, and to guide attention to objects that match the exact target features. However, recent evidence suggests that attentional guidance is highly flexible and often guided by non-veridical features, a subset of features, or only associated features. We integrate these findings and propose that attentional guidance maximizes search efficiency based on a 'good-enough' principle to rapidly localize candidate target objects. Candidates are then serially interrogated to make target-match decisions using more precise information. We suggest that good-enough guidance optimizes the speed-accuracy-effort trade-offs inherent in each stage of visual search.

Visual Crowding Reveals Field- and Axis-Specific Cortical Miswiring After Long-Term Axial Misalignment in Strabismic Patients Without Amblyopia

Purpose

Inspired by physiological and neuroimaging findings that revealed squint-induced modification of cortical volume and visual receptive field in early visual areas, we hypothesized that strabismic eyes without amblyopia manifest an increase in critical spacing of visual crowding, an essential bottleneck on object recognition and reliable psychophysical index of cortical organization.

Methods

We used real-time eye tracking to ensure gaze-contingent display and examined visual crowding in patients with horizontal concomitant strabismus (both esotropia and exotropia) but without amblyopia and age-matched normal controls.

Results

Nineteen patients with exotropia (12 men, mean ± SD = 22.89 ± 7.82 years), 21 patients with esotropia (10 men, mean ± SD = 23.48 ± 6.95 years), and 14 age-matched normal controls (7 men, mean ± SD = 23.07 ± 1.07 years) participated in this study. We found that patients with strabismus without amblyopia showed significantly larger critical spacing with nasotemporal asymmetry in only the radial axis that related to the strabismus pattern, with exotropia exhibiting stronger temporal hemifield crowding and esotropia exhibiting stronger nasal hemifield crowding, in both the deviated and fixating eyes. Moreover, the magnitude of crowding change was related to the duration and degree of strabismic deviation.

Conclusions

Using visual crowding as a psychophysical index of cortical organization, our study demonstrated significantly greater peripheral visual crowding with nasotemporal asymmetry in only the radial axis in patients with strabismus without amblyopia, indicating the existence of hemifield- and axis-specific miswiring of cortical processing in object recognition induced by long-term adaptation to ocular misalignment.

Working memory modulates the anger superiority effect in central and peripheral visual fields

Angry faces have been shown to be detected more efficiently in a crowd of distractors compared to happy faces, known as the anger superiority effect (ASE). The present study investigated whether the ASE could be modified by top-down manipulation of working memory (WM), in central and peripheral visual fields. In central vision, participants held a colour in WM for a final memory test while simultaneously performing a visual search task that required them to determine whether a face showed a different expression from other coloured faces. The colour held in WM matched either the colour of the target face (target-matching), the colour of a distractor face (distractor-matching), or neither (non-matching). Results showed that the ASE was observed when the probability of target-matching trials was low. However, when the top-down WM effect was strengthened by raising the probability of target-matching trials, the ASE in the target-matching condition was completely eliminated. Intriguingly, when the visual search task was substituted by a peripheral crowding task, similar results to central vision were found in the target-matching condition. Taken together, our findings indicate that the ASE is subject to the top-down WM effect, regardless of the visual field.

Feature representation under crowding in macaque V1 and V4 neuronal populations

Visual perception depends strongly on spatial context. A profound example is visual crowding, whereby the presence of nearby stimuli impairs the discriminability of object features. Despite extensive work on perceptual crowding and the spatial integrative properties of visual cortical neurons, the link between these two aspects of visual processing remains unclear. To understand better the neural basis of crowding, we recorded activity simultaneously from neuronal populations in V1 and V4 of fixating macaque monkeys. We assessed the information available from the measured responses about the orientation of a visual target both for targets presented in isolation and amid distractors. Both single neuron and population responses had less information about target orientation when distractors were present. Information loss was moderate in V1 and more substantial in V4. Information loss could be traced to systematic divisive and additive changes in neuronal tuning. Additive and multiplicative changes in tuning were more severe in V4; in addition, tuning exhibited other, non-affine transformations that were greater in V4, further restricting the ability of a fixed sensory readout strategy to extract accurate feature information across displays. Our results provide a direct test of crowding effects at different stages of the visual hierarchy. They reveal how crowded visual environments alter the spiking activity of cortical populations by which sensory stimuli are encoded and connect these changes to established mechanisms of neuronal spatial integration.

Unlocking crowding by ensemble statistics

In crowding,^{1,2,3,4,5,6,7} objects that can be easily recognized in isolation appear jumbled when surrounded by other elements.⁸ Traditionally, crowding is explained by local pooling mechanisms,^{3,6,9,10,11,12,13,14,15} but many findings have shown that the global configuration of the entire stimulus display, rather than local aspects, determines crowding.^{8,16,17,18,19,20,21,22,23,24,25,26,27,28} However, understanding global configurations is challenging because even slight changes can lead from crowding to uncrowding and vice versa.^23,25,28,29 Unfortunately, the number of configurations to explore is virtually infinite. Here, we show that one does not need to know the specific configuration of flankers to determine crowding strength but only their ensemble statistics, which allow for the rapid computation of groups within the stimulus display.^{30,31,32,33,34,35,36,37} To investigate the role of ensemble statistics in (un)crowding, we used a classic vernier offset discrimination task in which the vernier was flanked by multiple squares. We manipulated the orientation statistics of the squares based on the following rationale: a central square with an orientation different from the mean orientation of the other squares stands out from the rest and groups with the vernier, causing strong crowding. If, on the other hand, all squares group together, the vernier is the only element that stands out, and crowding is weak. These effects should depend exclusively on the perceived ensemble statistics, i.e., on the mean orientation of the squares and not on their individual orientations. In two experiments, we confirmed these predictions.

Ensemble perception: Stacking the hay to find the needle

The visual clutter we constantly encounter in the world limits object recognition, a phenomenon known as visual crowding. A new study shows that ensemble perception counters this by condensing redundant information into summary statistical representations, which thus releases visual crowding's effect on individual objects.

Crowding results from optimal integration of visual targets with contextual information

Crowding is the inability to recognize an object in clutter, usually considered a fundamental low-level bottleneck to object recognition. Here we advance and test an alternative idea, that crowding, like predictive phenomena such as serial dependence, results from optimizing strategies that exploit redundancies in natural scenes. This notion leads to several testable predictions: crowding should be greatest for unreliable targets and reliable flankers; crowding-induced biases should be maximal when target and flankers have similar orientations, falling off for differences around 20°; flanker interference should be associated with higher precision in orientation judgements, leading to lower overall error rate; effects should be maximal when the orientation of the target is near that of the average of the flankers, rather than to that of individual flankers. Each of these predictions were supported, and could be simulated with ideal-observer models that maximize performance. The results suggest that while crowding can affect object recognition, it may be better understood not as a processing bottleneck, but as a consequence of efficient exploitation of the spatial redundancies of the natural world.

Visual crowding is a phenomenon where objects presented in the visual periphery are not resolved efficiently. Here the authors show that crowding may derive from an optimizing strategy that blends information when it is similar and preserves it when it is dissimilar.

Transient attention equally reduces visual crowding in radial and tangential axes

Crowding refers to the failure to identify a peripheral object due to its proximity to other objects (flankers). This phenomenon can lead to reading and object recognition impairments and is associated with macular degeneration, amblyopia, and dyslexia. Crucially, the maximal target–flanker spacing required for the crowding interference (critical spacing) increases with eccentricity. This spacing is also larger when target and flankers appear along the horizontal meridian (radial arrangement) than when the flankers appear above and below the target (tangential arrangement). This phenomenon is known as radial–tangential anisotropy. Previous studies have demonstrated that transient attention can reduce crowding interference; however, it is still unclear whether and how attention interacts with radial–tangential anisotropy. To address this issue, we manipulated transient attention by using a cue at either the target (valid) or the fixation (neutral) location, in both radial and tangential target–flanker arrangements. Results showed that critical spacing was larger in the radial than in the tangential arrangement and that cueing the target location improved performance and reduced the critical spacing for both radial and tangential arrangements to the same extent. Together, our findings suggest that transient spatial attention plays an essential role in crowding but not in radial–tangential anisotropy.

Ensemble perception without phenomenal awareness of elements

Humans efficiently recognize complex scenes by grouping multiple features and objects into ensembles. It has been suggested that ensemble processing does not require, or even impairs, conscious discrimination of individual element properties. The present study examined whether ensemble perception requires phenomenal awareness of elements. We asked observers to judge the mean orientation of a line-based texture pattern whose central region was made invisible by backward masks. Masks were composed of either a Mondrian pattern (Exp. 1) or of an annular contour (Exp. 2) which, unlike the Mondrian, did not overlap spatially with elements in the central region. In the Mondrian-mask experiment, perceived mean orientation was determined only by visible elements outside the central region. However, in the annular-mask experiment, perceived mean orientation matched the mean orientation of all elements, including invisible elements within the central region. Results suggest that the visual system can compute spatial ensembles even without phenomenal awareness of stimuli.

Morality extracted under crowding impairs face identification

We investigated whether morality associated with faces is perceptible even under less optimal visual conditions such as crowding. A facial image was paired with a sentence describing an immoral act or a neutral act. Participants imagined the person performing the actions described in the sentence during the learning phase. Then, in the crowding phase, the target face was briefly presented in the left or right peripheral visual fields. Participants were required to judge the gender or morality of the target face in Experiment 1 and to choose the target face from two faces in Experiment 2. In both experiments, flankers were presented around the target face in the flanker condition, whereas no flankers were presented in the no-flanker condition. Experiment 1 indicated that the accuracy of judgments about the morality of a crowded face was higher for immoral faces than for neutral faces. This demonstrates that morality is preferentially extracted even when conscious access to facial representations is limited. Experiment 2 showed that the accuracy of selecting the flanked face from two faces was higher for neutral faces than for immoral faces. These indicated that the morality processed under the crowding impaired the discrimination of the facial identity.

Mixture-modeling approach reveals global and local processes in visual crowding

Crowding refers to the inability to recognize objects in clutter, setting a fundamental limit on various perceptual tasks such as reading and facial recognition. While prevailing models suggest that crowding is a unitary phenomenon occurring at an early level of processing, recent studies have shown that crowding might also occur at higher levels of representation. Here we investigated whether local and global crowding interference co-occurs within the same display. To do so, we tested the distinctive contribution of local flanker features and global configurations of the flankers on the pattern of crowding errors. Observers (n = 27) estimated the orientation of a target when presented alone or surrounded by flankers. Flankers were grouped into a global configuration, forming an illusory rectangle when aligned or a rectangular configuration when misaligned. We analyzed the error distributions by fitting probabilistic mixture models. Results showed that participants often misreported the orientation of a flanker instead of that of the target. Interestingly, in some trials the orientation of the global configuration was misreported. These results suggest that crowding occurs simultaneously across multiple levels of visual processing and crucially depends on the spatial configuration of the stimulus. Our results pose a challenge to models of crowding with an early single pooling stage and might be better explained by models which incorporate the possibility of multilevel crowding and account for complex target-flanker interactions.

Relative tuning of holistic face processing towards the fovea

Humans quickly detect and gaze at faces in the world, which reflects their importance in cognition and may lead to tuning of face recognition toward the central visual field. Although sometimes reported, foveal selectivity in face processing is debated: brain imaging studies have found evidence for a central field bias specific to faces, but behavioral studies have found little foveal selectivity in face recognition. These conflicting results are difficult to reconcile, but they could arise from stimulus-specific differences. Recent studies, for example, suggest that individual faces vary in the degree to which they require holistic processing. Holistic processing is the perception of faces as a whole rather than as a set of separate features. We hypothesized that the dissociation between behavioral and neuroimaging studies arises because of this stimulus-specific dependence on holistic processing. Specifically, the central bias found in neuroimaging studies may be specific to holistic processing. Here, we tested whether the eccentricity-dependence of face perception is determined by the degree to which faces require holistic processing. We first measured the holistic-ness of individual Mooney faces (two-tone shadow images readily perceived as faces). In a group of independent observers, we then used a gender discrimination task to measured recognition of these Mooney faces as a function of their eccentricity. Face gender was recognized across the visual field, even at substantial eccentricities, replicating prior work. Importantly, however, holistic face gender recognition was relatively tuned-slightly, but reliably stronger in the central visual field. Our results may reconcile the debate on the eccentricity-dependance of face perception and reveal a spatial inhomogeneity specifically in the holistic representations of faces.

Central-peripheral dichotomy: color-motion and luminance-motion binding show stronger top-down feedback in central vision

Recently a theory (Zhaoping, Vision Research, 136, 32-49, 2017) proposed that top-down feedback from higher to lower visual cortical areas, to aid visual recognition, is stronger in the central than in the peripheral visual fields. Since top-down feedback helps feature binding, a critical visual recognition process, this theory predicts that insufficient feedback in the periphery should make feature misbinding more likely. To test this prediction, this study assessed binding between color and motion features, or between luminance and motion features, at different visual field eccentricities. We first used color-motion stimuli containing equiluminant red and green dots moving in opposite directions, for example, red dots moved leftward while green dots moved rightward. Such stimuli were shown in both a central reference strip and a peripheral test strip; participants reported whether it was the first or second interval in a trial in which the dots of each color moved in the opposite directions between the two strips. The center of the test strip was at 4° or 15° away from the gaze fixation. Participants' performance was much worse when the test strip was more peripheral, suggesting that feature misbinding occurred more frequently there. This held even when the size and density of the dots were adjusted by eccentricity-dependent cortical magnification factors, and even when red/green dots were replaced by yellow/blue dots or black/white dots to suit the retinal input sampling peripherally. Our findings support that top-down feedback is more directed to central vision, which can resolve ambiguities in feature binding at more central visual locations.

Spatial Attention Enhances Crowded Stimulus Encoding Across Modeled Receptive Fields by Increasing Redundancy of Feature Representations

Any visual system, biological or artificial, must make a trade-off between the number of units used to represent the visual environment and the spatial resolution of the sampling array. Humans and some other animals are able to allocate attention to spatial locations to reconfigure the sampling array of receptive fields (RFs), thereby enhancing the spatial resolution of representations without changing the overall number of sampling units. Here, we examine how representations of visual features in a fully convolutional neural network interact and interfere with each other in an eccentricity-dependent RF pooling array and how these interactions are influenced by dynamic changes in spatial resolution across the array. We study these feature interactions within the framework of visual crowding, a well-characterized perceptual phenomenon in which target objects in the visual periphery that are easily identified in isolation are much more difficult to identify when flanked by similar nearby objects. By separately simulating effects of spatial attention on RF size and on the density of the pooling array, we demonstrate that the increase in RF density due to attention is more beneficial than changes in RF size for enhancing target classification for crowded stimuli. Furthermore, by varying target/flanker spacing, as well as the spatial extent of attention, we find that feature redundancy across RFs has more influence on target classification than the fidelity of the feature representations themselves. Based on these findings, we propose a candidate mechanism by which spatial attention relieves visual crowding through enhanced feature redundancy that is mostly due to increased RF density.

Broad attention uncovers benefits of stimulus uniformity in visual crowding

Crowding is the interference by surrounding objects (flankers) with target perception. Low target-flanker similarity usually yields weaker crowding than high similarity ('similarity rule') with less interference, e.g., by opposite- than same-contrast polarity flankers. The advantage of low target-flanker similarity has typically been shown with attentional selection of a single target object. Here, we investigated the validity of the similarity rule when broadening attention to multiple objects. In three experiments, we measured identification for crowded letters (Experiment 1), tumbling Ts (Experiment 2), and tilted lines (Experiment 3). Stimuli consisted of three items that were uniform or alternating in contrast polarity and were briefly presented at ten degrees eccentricity. Observers reported all items (full report) or only the left, central, or right item (single-item report). In Experiments 1 and 2, consistent with the similarity rule, single central item performance was superior with opposite- compared to same-contrast polarity flankers. With full report, the similarity rule was inverted: performance was better for uniform compared to alternating stimuli. In Experiment 3, contrast polarity did not affect performance. We demonstrated a reversal of the similarity rule under broadened attention, suggesting that stimulus uniformity benefits crowded object recognition when intentionally directing attention towards all stimulus elements. We propose that key properties of crowding have only limited validity as they may require a-priori differentiation of target and context.

Global and high-level effects in crowding cannot be predicted by either high-dimensional pooling or target cueing

In visual crowding, the perception of a target deteriorates in the presence of nearby flankers. Traditionally, target-flanker interactions have been considered as local, mostly deleterious, low-level, and feature specific, occurring when information is pooled along the visual processing hierarchy. Recently, a vast literature of high-level effects in crowding (grouping effects and face-holistic crowding in particular) led to a different understanding of crowding, as a global, complex, and multilevel phenomenon that cannot be captured or explained by simple pooling models. It was recently argued that these high-level effects may still be captured by more sophisticated pooling models, such as the Texture Tiling model (TTM). Unlike simple pooling models, the high-dimensional pooling stage of the TTM preserves rich information about a crowded stimulus and, in principle, this information may be sufficient to drive high-level and global aspects of crowding. In addition, it was proposed that grouping effects in crowding may be explained by post-perceptual target cueing. Here, we extensively tested the predictions of the TTM on the results of six different studies that highlighted high-level effects in crowding. Our results show that the TTM cannot explain any of these high-level effects, and that the behavior of the model is equivalent to a simple pooling model. In addition, we show that grouping effects in crowding cannot be predicted by post-perceptual factors, such as target cueing. Taken together, these results reinforce once more the idea that complex target-flanker interactions determine crowding and that crowding occurs at multiple levels of the visual hierarchy.

Sensitivity to Central Crowding for Faces in Patients With Glaucoma

PRECIS

Some patients with glaucoma report difficulties to recognize faces when they are far away. We show that this deficit could result from a higher sensitivity to crowding in central vision.

PURPOSE

The aim of the study is to investigate whether face recognition difficulties reported by some patients with glaucoma result from a greater sensitivity to inner crowding in central vision.

METHODS

Seventeen patients with glaucoma and 17 age-matched normally sighted controls participated in the study. An isolated mouth (uncrowded condition) or a mouth within a face (crowded condition) was randomly displayed centrally for 200 ms. For each condition, participants were asked to decide whether the mouth was closed or open. The stimuli were presented at 3 angular sizes (0.6×0.4, 1×0.72, and 1.5×1.08 degrees). Accuracy was measured.

RESULTS

Crowding affected performance differentially for patients and controls. Consistent with previous studies controls exhibited a "face superiority effect," with a better accuracy when the mouth was located within the face than when it was isolated. Sensitivity to crowding, reflected in a better accuracy with the isolated mouth, was observed in 10 of 17 patients only for small images. Crowding disappeared for larger faces, as the facial features were spaced out. Five patients were not sensitive to crowding. Importantly, no difference was found between the 2 subgroups of patients (sensitive vs. nonsensitive) in terms of mean deviation, contrast sensitivity, acuity, thickness of the retinal nerve fiber layer, or macular ganglion cell-inner plexiform layer.

CONCLUSIONS

An excessive sensitivity to central crowding might explain the difficulties in face perception and reading reported by some patients with glaucoma. The sensory or cognitive processes underlying this excessive sensitivity must be elucidated to improve central perception in glaucoma.

A comparative biology approach to DNN modeling of vision: A focus on differences, not similarities

Deep neural networks (DNNs) have revolutionized computer science and are now widely used for neuroscientific research. A hot debate has ensued about the usefulness of DNNs as neuroscientific models of the human visual system; the debate centers on to what extent certain shortcomings of DNNs are real failures and to what extent they are redeemable. Here, we argue that the main problem is that we often do not understand which human functions need to be modeled and, thus, what counts as a falsification. Hence, not only is there a problem on the DNN side, but there is also one on the brain side (i.e., with the explanandum-the thing to be explained). For example, should DNNs reproduce illusions? We posit that we can make better use of DNNs by adopting an approach of comparative biology by focusing on the differences, rather than the similarities, between DNNs and humans to improve our understanding of visual information processing in general.

Close - but not distant - conditioned flanker emotion affects crowding

Crowding is affected by conditioned stimulus emotion. This effect is clearly observed for conditioned flankers, but only marginally pronounced for conditioned targets. Studies on the processing of emotional stimuli suggest that the magnitude of the emotional effect depends on the presentation depth in that effects of emotion increase with decreasing distance to the observer in depth. Based on respective findings, we investigate crowding with stimuli of conditioned negative and neutral emotion across real depth; that is, stimuli were either presented closer, at or farther away than the fixation depth. Conditioned emotion of flankers affected crowding when flankers were presented closer than or at fixation depth, which is also the distance the target was presented at. Farther away than the fixation depth, flanker emotion did not alter crowding (Experiment 1a). Conditioned target emotion, however, did only show weak effects on crowding; neither when flankers (Experiment 1b) nor when targets were varied in depth (Experiment 2) there was a clear effect of target emotion, replicating findings in two-dimensional settings. Taken together, the results suggest that flanker's emotional associations can become important for crowding, although, it depends on the special processing characteristics of stimulus emotion in depth. The conditioned emotion of targets scarcely affected crowding.

Shrinking Bouma's window: How to model crowding in dense displays

In crowding, perception of a target deteriorates in the presence of nearby flankers. Traditionally, it is thought that visual crowding obeys Bouma's law, i.e., all elements within a certain distance interfere with the target, and that adding more elements always leads to stronger crowding. Crowding is predominantly studied using sparse displays (a target surrounded by a few flankers). However, many studies have shown that this approach leads to wrong conclusions about human vision. Van der Burg and colleagues proposed a paradigm to measure crowding in dense displays using genetic algorithms. Displays were selected and combined over several generations to maximize human performance. In contrast to Bouma's law, only the target's nearest neighbours affected performance. Here, we tested various models to explain these results. We used the same genetic algorithm, but instead of selecting displays based on human performance we selected displays based on the model's outputs. We found that all models based on the traditional feedforward pooling framework of vision were unable to reproduce human behaviour. In contrast, all models involving a dedicated grouping stage explained the results successfully. We show how traditional models can be improved by adding a grouping stage.

Reading: The Confluence of Vision and Language

The scientific study of reading has a rich history that spans disciplines from vision science to linguistics, psychology, cognitive neuroscience, neurology, and education. The study of reading can elucidate important general mechanisms in spatial vision, attentional control, object recognition, and perceptual learning, as well as the principles of plasticity and cortical topography. However, literacy also prompts the development of specific neural circuits to process a unique and artificial stimulus. In this review, we describe the sequence of operations that transforms visual features into language, how the key neural circuits are sculpted by experience during development, and what goes awry in children for whom learning to read is a struggle. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Binocular summation is affected by crowding and tagging

In perceptual crowding, a letter easily recognized on its own, becomes unrecognizable if it is surrounded by other letters, an effect that confers a limit on the visual processing. Models assume that crowding is a hallmark of the periphery but that it is almost absent in the fovea. However, recently it was shown that crowding occurs in the fovea of people with an abnormal development of functional vision (amblyopia), when the stimulus is presented for a very short time. When targets and flankers are dissimilar, the crowding is reduced (tagging). Since a combination of binocular inputs increases the processing load, we investigated whether color tagging the target reduces crowding in the fovea of subjects with normal vision and determined how crowding is combined with binocular vision. The crowding effect at the fovea was significantly reduced by tagging with a color target. Interestingly, whereas binocular summation for a single letter was expected to be about 40%, it was significantly reduced and almost absent under crowding conditions. Our results are consistent with the notion that the crowding effect produces a high processing load on visual processing, which interferes with other processes such as binocular summation. We assume that the tagging effect in our experiment improved the subject's abilities (sensitivity and RT) by creating a "segmentation", i.e., a visual simulated separation between the target letter and the background. Interestingly, tagging the target with a distinct color can eliminate or reduce the crowding effect and consequently, binocular summation recovers.

The Focal Attention Window Size Explains Letter Substitution Errors in Reading

Acquired Neglect Dyslexia is often associated with right-hemisphere brain damage and is mainly characterized by omissions and substitutions in reading single words. Martelli et al. proposed in 2011 that these two types of error are due to different mechanisms. Omissions should depend on neglect plus an oculomotor deficit, whilst substitutions on the difficulty with which the letters are perceptually segregated from each other (i.e., crowding phenomenon). In this study, we hypothesized that a deficit of focal attention could determine a pathological crowding effect, leading to imprecise letter identification and consequently substitution errors. In Experiment 1, three brain-damaged patients, suffering from peripheral dyslexia, mainly characterized by substitutions, underwent an assessment of error distribution in reading pseudowords and a T detection task as a function of cue size and timing, in order to measure focal attention. Each patient, when compared to a control group, showed a deficit in adjusting the attentional focus. In Experiment 2, a group of 17 right-brain-damaged patients were asked to perform the focal attention task and to read single words and pseudowords as a function of inter-letter spacing. The results allowed us to confirm a more general association between substitution-type reading errors and the performance in the focal attention task.

Mixture model investigation of the inner-outer asymmetry in visual crowding reveals a heavier weight towards the visual periphery

Crowding, the failure to identify a peripheral item in clutter, is an essential bottleneck in visual information processing. A hallmark characteristic of crowding is the inner-outer asymmetry in which the outer flanker (more eccentric) produces stronger interference than the inner one (closer to the fovea). We tested the contribution of the inner-outer asymmetry to the pattern of crowding errors in a typical radial crowding display in which both flankers are presented simultaneously on the horizontal meridian. In two experiments, observers were asked to estimate the orientation of a Gabor target. Instead of the target, observers reported the outer flanker much more frequently than the inner one. When the target was the outer Gabor, crowding was reduced. Furthermore, when there were four flankers, two on each side of the target, observers misreported the outer flanker adjacent to the target, not the outermost flanker. Model comparisons suggested that orientation crowding reflects sampling over a weighted sum of the represented features, in which the outer flanker is more heavily weighted compared to the inner one. Our findings reveal a counterintuitive phenomenon: in a radial arrangement of orientation crowding, within a region of selection, the outer item dominates appearance more than the inner one.

Flexible contextual modulation of naturalistic texture perception in peripheral vision

Peripheral vision comprises most of our visual field, and is essential in guiding visual behavior. Its characteristic capabilities and limitations, which distinguish it from foveal vision, have been explained by the most influential theory of peripheral vision as the product of representing the visual input using summary statistics. Despite its success, this account may provide a limited understanding of peripheral vision, because it neglects processes of perceptual grouping and segmentation. To test this hypothesis, we studied how contextual modulation, namely the modulation of the perception of a stimulus by its surrounds, interacts with segmentation in human peripheral vision. We used naturalistic textures, which are directly related to summary-statistics representations. We show that segmentation cues affect contextual modulation, and that this is not captured by our implementation of the summary-statistics model. We then characterize the effects of different texture statistics on contextual modulation, providing guidance for extending the model, as well as for probing neural mechanisms of peripheral vision.

On the contrast dependence of crowding

Visual clutter affects our ability to see. Objects that would be identifiable on their own may become unrecognizable when presented close together ("crowding"), but the psychophysical characteristics of crowding have resisted simplification. Image properties initially thought to produce crowding have paradoxically yielded unexpected results; for example, adding flanking objects can ameliorate crowding (Manassi, Sayim, & Herzog, 2012; Herzog, Sayim, Chcherov, & Manassi, 2015; Pachai, Doerig, & Herzog, 2016). The resulting theory revisions have been sufficiently complex and specialized as to make it difficult to discern what principles may underlie the observed phenomena. Here, a generalized formulation of simple visual contrast energy is presented, arising from straightforward analyses of center and surround neurons in the early visual stream. Extant contrast measures, such as root mean square contrast, are easily shown to fall out as reduced special cases. The new generalized contrast energy metric surprisingly predicts the principal findings of a broad range of crowding studies. These early crowding phenomena may thus be said to arise predominantly from contrast or are, at least, severely confounded by contrast effects. Note that these findings may be distinct from accounts of other, likely downstream, "configural" or "semantic" instances of crowding, suggesting at least two separate forms of crowding that may resist unification. The new fundamental contrast energy formulation provides a candidate explanatory framework that addresses multiple psychophysical phenomena beyond crowding.

Crowding and attention in a framework of neural network model

In this article, I present a framework that would accommodate the classic ideas of visual information processing together with more recent computational approaches. I used the current knowledge about visual crowding, capacity limitations, attention, and saliency to place these phenomena within a standard neural network model. I suggest some revisions to traditional mechanisms of attention and feature integration that are required to fit better into this framework. The results allow us to explain some apparent theoretical controversies in vision research, suggesting a rationale for the limited spatial extent of crowding, a role of saliency in crowding experiments, and several amendments to the feature integration theory. The scheme can be elaborated or modified by future research.

Ensemble coding of crowd speed using biological motion

The accurate perception of human crowds is integral to social understanding and interaction. Previous studies have shown that observers are sensitive to several crowd characteristics such as average facial expression, gender, identity, joint attention, and heading direction. In two experiments, we examined ensemble perception of crowd speed using standard point-light walkers (PLW). Participants were asked to estimate the average speed of a crowd consisting of 12 figures moving at different speeds. In Experiment 1, trials of intact PLWs alternated with trials of scrambled PLWs with a viewing duration of 3 seconds. We found that ensemble processing of crowd speed could rely on local motion alone, although a globally intact configuration enhanced performance. In Experiment 2, observers estimated the average speed of intact-PLW crowds that were displayed at reduced viewing durations across five blocks of trials (between 2500 ms and 500 ms). Estimation of fast crowds was precise and accurate regardless of viewing duration, and we estimated that three to four walkers could still be integrated at 500 ms. For slow crowds, we found a systematic deterioration in performance as viewing time reduced, and performance at 500 ms could not be distinguished from a single-walker response strategy. Overall, our results suggest that rapid and accurate ensemble perception of crowd speed is possible, although sensitive to the precise speed range examined.

The anger superiority effect revisited: a visual crowding task

Visual search studies have shown that threatening facial expressions are more efficiently detected among a crowd of distractor faces than nonthreatening expressions, known as the anger superiority effect (ASE). However, the opposite finding has also been documented. The present study investigated the ASE in the visual periphery with a visual crowding task. In the study, the target face either appeared alone (uncrowded condition) or was crowded by four neutral or emotional faces (crowded condition). Participants were instructed to determine whether the target face was happy or angry. Experiment 1 showed an ASE when crowded by neutral faces. Intriguingly, this superiority vanished when the target face was crowded by emotional faces that had a different expression from the target as well as when the target face was presented alone. Experiment 2 replicated this result in an independent sample of East Asians (vs. Caucasians in Experiment 1) and thus demonstrated the robustness and cross-cultural consistency of our findings. Together, these results suggest that the ASE in the visual periphery is contingent on task demands induced by visual crowding.

Crowding Effects Across Depth are Fixation-Centered for Defocused Flankers and Observer-Centered for Defocused Targets

Depth needs to be considered to understand visual information processing in cluttered environments in the wild. Since differences in depth depend on current gaze position, eye movements were avoided by short presentations in a real depth setup. Thus, allowing only peripheral vision, crowding was tested. That is, the impairment of peripheral target recognition by the presence of nearby flankers was measured. Real depth was presented by a half-transparent mirror that aligned the displays of two orthogonally arranged, distance-adjustable screens. Fixation depth was at a distance of 190 cm, defocused depth planes were presented either near or far, in front of or behind the fixation depth, all within the depth of field. In Experiments 1 and 2, flankers were presented defocused, while the to-be-identified targets were on the fixation depth plane. In Experiments 3-5, targets were presented defocused, while the flankers were kept on the fixation depth plane. Results for defocused flankers indicate increased crowding effects with increased flanker distance from the target at focus (near to far). However, for defocused targets, crowding for targets in front of the focus as compared to behind was increased. Thus, defocused targets produce decreased crowding with increased target distance from the observer. To conclude, the effects of flankers in depth seem to be centered around fixation, while effects of target depth seem to be observer-centered.

Empirical Evidence for Intraspecific Multiple Realization?

Despite the remarkable advances in behavioral and brain sciences over the last decades, the mind-body (brain) problem is still an open debate and one of the most intriguing questions for both cognitive neuroscience and philosophy of mind. Traditional approaches have conceived this problem in terms of a contrast between physicalist monism and Cartesian dualism. However, since the late sixties, the landscape of philosophical views on the problem has become more varied and complex. The Multiple Realization Thesis (MRT) claims that mental properties can be (or are) realized, and mental processes can be (or are) implemented by neural correlates of different kinds. Thus, MRT challenges the psychoneural type-identity theory and the corresponding reductionism. Many philosophers have acknowledged the a priori plausibility of MRT. However, the existence of empirical evidence in favor of intraspecific, human multiple realizations of mental processes and properties is still controversial. Here, we illustrate some cases that provide empirical evidence in support of MRT. Recently, it has been proposed that foveal agnosic vision, like peripheral vision, can be restored by increasing object parts’ spacing (Crutch and Warrington, 2007; Strappini et al., 2017b). Agnosic fovea and normal periphery are both limited by crowding, which impairs object recognition, and provides the signature of visual integration. Here, we define a psychological property of restored object identification, and we cross-reference the data of visually impaired patients with different etiologies. In particular, we compare the data of two stroke patients, two patients with posterior cortical atrophy, six cases of strabismic amblyopia, and one case with restored sight. We also compare these patients with unimpaired subjects tested in the periphery. We show that integration (i.e., restored recognition) seems to describe quite accurately the visual performance in all these cases. Whereas the patients have different etiologies and different neural correlates, the unimpaired subjects have no neural damage. Thus, similarity in the psychological property given the differences in the neural substrate can be interpreted in relation to MRT and provide evidence in its support. Finally, we will frame our contribution within the current debate concerning MRT providing new and compelling empirical evidence.

Visual crowding effect in the parvocellular and magnocellular visual pathways

The crowding effect, defined as the detrimental effects of nearby items on visual object recognition, has been extensively investigated. Previous studies have primarily focused on finding the stage(s) in the visual hierarchy where crowding starts to limit target processing, while little attention has been focused on potential differences between the parvocellular (P) and magnocellular (M) pathways in crowding mechanisms. Here, we investigated the crowding effect in these parallel visual pathways. InExperiment 1, stimuli were designed to separately engage the P or M pathway, by tuning stimulus and background features (e.g., temporal frequency and color) to activate the targeted pathway and saturate the other pathway, respectively. Results showed that at the same eccentricity and with the same tasks, targets processed in the M pathway appeared to be more vulnerable to crowding effect. InExperiment 2, crowding effects were studied using three different types of stimuli and visual tasks (form, color, and motion), presumably with different degrees of dependence on the P and M pathways. Results revealed that color, motion, and form discrimination were increasingly more affected by crowding. We conclude that processing in the M and P pathways are differentially impacted by crowding; and importantly, crowding seems to affect processing of spatial forms more than other stimulus properties.

Excessive visual crowding effects in developmental dyscalculia

Visual crowding refers to the inability to identify objects when surrounded by other similar items. Crowding-like mechanisms are thought to play a key role in numerical perception by determining the sensory mechanisms through which ensembles are perceived. Enhanced visual crowding might hence prevent the normal development of a system involved in segregating and perceiving discrete numbers of items and ultimately the acquisition of more abstract numerical skills. Here, we investigated whether excessive crowding occurs in developmental dyscalculia (DD), a neurodevelopmental disorder characterized by difficulty in learning the most basic numerical and arithmetical concepts, and whether it is found independently of associated major reading and attentional difficulties. We measured spatial crowding in two groups of adult individuals with DD and control subjects. In separate experiments, participants were asked to discriminate the orientation of a Gabor patch either in isolation or under spatial crowding. Orientation discrimination thresholds were comparable across groups when stimuli were shown in isolation, yet they were much higher for the DD group with respect to the control group when the target was crowded by closely neighbouring flanking gratings. The difficulty in discriminating orientation (as reflected by the combination of accuracy and reaction times) in the DD compared to the control group persisted over several larger target flanker distances. Finally, we found that the degree of such spatial crowding correlated with impairments in mathematical abilities even when controlling for visual attention and reading skills. These results suggest that excessive crowding effects might be a characteristic of DD, independent of other associated neurodevelopmental disorders.

Capsule networks as recurrent models of grouping and segmentation

Classically, visual processing is described as a cascade of local feedforward computations. Feedforward Convolutional Neural Networks (ffCNNs) have shown how powerful such models can be. However, using visual crowding as a well-controlled challenge, we previously showed that no classic model of vision, including ffCNNs, can explain human global shape processing. Here, we show that Capsule Neural Networks (CapsNets), combining ffCNNs with recurrent grouping and segmentation, solve this challenge. We also show that ffCNNs and standard recurrent CNNs do not, suggesting that the grouping and segmentation capabilities of CapsNets are crucial. Furthermore, we provide psychophysical evidence that grouping and segmentation are implemented recurrently in humans, and show that CapsNets reproduce these results well. We discuss why recurrence seems needed to implement grouping and segmentation efficiently. Together, we provide mutually reinforcing psychophysical and computational evidence that a recurrent grouping and segmentation process is essential to understand the visual system and create better models that harness global shape computations.

Visual crowding in driving

Visual crowding-the deleterious influence of nearby objects on object recognition-is considered to be a major bottleneck for object recognition in cluttered environments. Although crowding has been studied for decades with static and artificial stimuli, it is still unclear how crowding operates when viewing natural dynamic scenes in real-life situations. For example, driving is a frequent and potentially fatal real-life situation where crowding may play a critical role. In order to investigate the role of crowding in this kind of situation, we presented observers with naturalistic driving videos and recorded their eye movements while they performed a simulated driving task. We found that the saccade localization on pedestrians was impacted by visual clutter, in a manner consistent with the diagnostic criteria of crowding (Bouma's rule of thumb, flanker similarity tuning, and the radial-tangential anisotropy). In order to further confirm that altered saccadic localization is a behavioral consequence of crowding, we also showed that crowding occurs in the recognition of cluttered pedestrians in a more conventional crowding paradigm. We asked participants to discriminate the gender of pedestrians in static video frames and found that the altered saccadic localization correlated with the degree of crowding of the saccade targets. Taken together, our results provide strong evidence that crowding impacts both recognition and goal-directed actions in natural driving situations.

Crowding in humans is unlike that in convolutional neural networks

Object recognition is a primary function of the human visual system. It has recently been claimed that the highly successful ability to recognise objects in a set of emergent computer vision systems-Deep Convolutional Neural Networks (DCNNs)-can form a useful guide to recognition in humans. To test this assertion, we systematically evaluated visual crowding, a dramatic breakdown of recognition in clutter, in DCNNs and compared their performance to extant research in humans. We examined crowding in three architectures of DCNNs with the same methodology as that used among humans. We manipulated multiple stimulus factors including inter-letter spacing, letter colour, size, and flanker location to assess the extent and shape of crowding in DCNNs. We found that crowding followed a predictable pattern across architectures that was different from that in humans. Some characteristic hallmarks of human crowding, such as invariance to size, the effect of target-flanker similarity, and confusions between target and flanker identities, were completely missing, minimised or even reversed. These data show that DCNNs, while proficient in object recognition, likely achieve this competence through a set of mechanisms that are distinct from those in humans. They are not necessarily equivalent models of human or primate object recognition and caution must be exercised when inferring mechanisms derived from their operation.