Substitution and pooling in crowding

Parafoveal letter identification in Russian: Confusion matrices based on error rates

In the present study, we introduce parafoveal letter confusion matrices for the Russian language, which uses the Cyrillic script. To ensure that our confusion rates reflect parafoveal processing and no other effects, we employed an adapted boundary paradigm (Rayner, 1975) that prevented the participants from directly fixating the letter stimuli. Additionally, we assessed confusability under isolated and word-like (crowded) conditions using two modern fonts, since previous research showed that letter recognition depended on crowding and font (Coates, 2015; Pelli et al., 2006). Our additional goal was to gain insight into what letter features or configurational patterns might be essential for letter recognition in Russian; thus, we conducted exploratory clustering analysis on visual confusion scores to identify groups of similar letters. To support this analysis, we conducted a comprehensive review of over 20 studies that proposed crucial properties of Latin letters relevant to character perception. The summary of this review is valuable not only for our current study but also for future research in the field.

When periphery rules: Enhanced sampling weights of the visual periphery in crowding across dimensions

Crowding, our inability to identify a feature or object - the target - due to its proximity to adjacent features or objects - flankers - exhibits a notable inner-outer asymmetry. This asymmetry is characterized by the outer flanker - more peripheral - creating stronger interference than the inner one - closer to the fovea. But crowding is not uniform across different feature dimensions. For example, in the case of orientation, this asymmetry reflects misreport errors: observers are more likely to misidentify the outer flanker as the target than the inner one. However, for spatial frequency (SF), observers tend to average the features of the target and flankers (Yashar et al., 2019). Here, we investigated whether and how the inner-outer asymmetry manifests across various feature dimensions: Gabor orientation and SF, as well as T-shape tilt and color. We reanalyzed continuous estimation reports data published by Yashar et al. (2019), focusing on a previously unanalyzed factor: the relative position of each flanker (inner vs. outer). We fit probabilistic models that assign variable weights to each flanker. Our analysis revealed that observers predominantly misreport the outer flanker as the target with Gabor orientation and T-shape tilt stimuli, and slightly so with color stimuli, whereas with Gabor SF, observers perform a weighted average of all features but also with a bias towards the outer flanker over the inner one. These findings suggest that an increased weighting on the more peripheral items is a general characteristic of crowding in peripheral vision.

Does crowding predict conjunction search? An individual differences approach

Searching for objects in the visual environment is an integral part of human behavior. Most of the information used during such visual search comes from the periphery of our vision, and understanding the basic mechanisms of search therefore requires taking into account the inherent limitations of peripheral vision. Our previous work using an individual differences approach has shown that one of the major factors limiting peripheral vision (crowding) is predictive of single feature search, as reflected in response time and eye movement measures. Here we extended this work, by testing the relationship between crowding and visual search in a conjunction-search paradigm. Given that conjunction search involves more fine-grained discrimination and more serial behavior, we predicted it would be strongly affected by crowding. We tested sixty participants with regard to their sensitivity to both orientation and color-based crowding (as measured by critical spacing) and their efficiency in searching for a color/orientation conjunction (as indicated by manual response times and eye movements). While the correlations between the different crowding tasks were high, the correlations between the different crowding measures and search performance were relatively modest, and no higher than those previously observed for single-feature search. Instead, observers showed very strong color selectivity during search. The results suggest that conjunction search behavior relies more on top-down guidance (here by color) and is therefore relatively less determined by individual differences in sensory limitations as caused by crowding.

Foveal crowding for large and small Landolt Cs: Similarity and Attention

We compare the recognition of foveal crowded Landolt Cs of two sizes: brief (40 ms), large, low-contrast Cs and high-contrast (1 sec) tests at the resolution limit of the visual system. In different series, the test Landolt C was surrounded by two identical distractors located symmetrically along the horizontal or by a single distractor. The distractors were Landolt Cs or rings. At the resolution limit, the critical spacing was similar in the two series and did not depend on the type of distractor. The result supports the hypothesis that crowding at the resolution limit occurs when both the test and the distractors fall into the same smallest receptive field responsible for the target recognition. For large stimuli, at almost all separations distractors of the same shape caused greater impairment than did rings, and recognition errors were non-random. The critical spacing was equal to 0.5 test diameters only in the presence of one distracting Landolt C. This result suggests that attention is involved: When one distractor is added, involuntary attention, which is directed to the centre of gravity of the stimulus, can lead to confusion of features that are present in both tests and distractors and thus to non-random errors.

The impact of training on the inner-outer asymmetry in crowding

Inner-outer asymmetry, where the outer flanker induces stronger crowding than the inner flanker, is a hallmark property of visual crowding. It is unclear the contribution of inner-outer asymmetry to the pattern of crowding errors (biased predominantly toward the flanker identities) and the role of training on crowding errors. In a typical radial crowding display, 20 observers were asked to report the orientation of a target Gabor (7.5° eccentricity) flanked by either an inner or outer Gabor along the horizontal meridian. The results showed that outer flanker conditions induced stronger crowding, accompanied by assimilative errors to the outer flanker for similar target/flanker elements. In contrast, the inner flanker condition exhibited weaker crowding, with no significant patterns of crowding errors. A population coding model showed that the flanker weights in the outer flanker condition were significantly higher than those in the inner flanker condition. Nine observers continued to train the outer flanker condition for four sessions. Training reduced inner-outer asymmetry and reduced flanker weights to the outer flanker. The learning effects were retained over 4 to 6 months. Individual differences in the appearance of crowding errors, the strength of inner-outer asymmetry, and the training effects were evident. Nevertheless, our findings indicate that different crowding mechanisms may be responsible for the asymmetric crowding effects induced by inner and outer flankers, with the outer flankers dominating the appearance more than the inner ones. Training reduces inner-outer asymmetry by reducing target/flanker confusion, and learning is persistent over months, suggesting that perceptual learning has the potential to improve visual performance by promoting neural plasticity.

The role of spatial attention in crowding and feature binding

Crowding refers to the failure to identify a peripheral object due to nearby objects (flankers). A hallmark of crowding is inner-outer asymmetry; that is, the outer flanker (more peripheral) produces stronger interference than the inner one. Here, by manipulating attention, we tested the predictions of two competing accounts: the attentional account, which predicts a positive attentional effect on the inner-outer asymmetry (i.e., attention to the outer flanker will increase asymmetry) and the receptive field size account, which predicts a negative attentional effect. In Experiment 1, observers estimated a Gabor target orientation. A peripheral pre-cue drew attention to one of three locations: target, inner flanker, or outer flanker. Probabilistic mixture modeling demonstrated asymmetry by showing that observers often misreported the outer-flanker orientation as the target. Interestingly, the outer cue led to a higher misreport rate of the outer flanker, and the inner cue led to a lower misreport rate of the outer flanker. Experiment 2 tested the effect of crowding and attention on incoherent object reports (i.e., binding errors, reporting the tilt of one presented item with the color of another item). In each trial, observers estimated both the tilt and color of the target. Attention merely increased coherent target reports, but not coherent flanker reports. The results suggest that the locus of spatial attention plays an essential role in crowding, as well as inner-outer asymmetry, and demonstrate that crowding and feature binding are closely related. However, our findings are inconsistent with the view that covert attention automatically binds features together.

Transient attention equally reduces visual crowding in radial and tangential axes

Crowding refers to the failure to identify a peripheral object due to its proximity to other objects (flankers). This phenomenon can lead to reading and object recognition impairments and is associated with macular degeneration, amblyopia, and dyslexia. Crucially, the maximal target–flanker spacing required for the crowding interference (critical spacing) increases with eccentricity. This spacing is also larger when target and flankers appear along the horizontal meridian (radial arrangement) than when the flankers appear above and below the target (tangential arrangement). This phenomenon is known as radial–tangential anisotropy. Previous studies have demonstrated that transient attention can reduce crowding interference; however, it is still unclear whether and how attention interacts with radial–tangential anisotropy. To address this issue, we manipulated transient attention by using a cue at either the target (valid) or the fixation (neutral) location, in both radial and tangential target–flanker arrangements. Results showed that critical spacing was larger in the radial than in the tangential arrangement and that cueing the target location improved performance and reduced the critical spacing for both radial and tangential arrangements to the same extent. Together, our findings suggest that transient spatial attention plays an essential role in crowding but not in radial–tangential anisotropy.

On the relation between crowding and ensemble perception: Examining the role of attention

Ensemble perception of a crowd of stimuli is very accurate, even when individual stimuli are invisible due to crowding. The ability of high-precision ensemble perception can be an evolved compensatory mechanism for the limited attentional resolution caused by crowding. Thus the relationship of crowding and ensemble coding is like two sides of the same coin wherein attention may play a critical factor for their coexistence. The present study investigated whether crowding and ensemble coding were similarly modulated by attention, which can promote our understanding of their relation. Experiment 1 showed that diverting attention away from the target harmed the performance in both crowding and ensemble perception tasks regardless of stimulus density, but crowding was more severely harmed. Experiment 2 showed that directing attention toward the target bar enhanced the performance of crowding regardless of stimulus density. Ensemble perception of high-density bars was also enhanced but to a lesser extent, while ensemble perception of low-density bars was harmed. Together, our results indicate that crowding is strongly modulated by attention, whereas ensemble perception is only moderately modulated by attention, which conforms to the adaptive view.

Mixture-modeling approach reveals global and local processes in visual crowding

Crowding refers to the inability to recognize objects in clutter, setting a fundamental limit on various perceptual tasks such as reading and facial recognition. While prevailing models suggest that crowding is a unitary phenomenon occurring at an early level of processing, recent studies have shown that crowding might also occur at higher levels of representation. Here we investigated whether local and global crowding interference co-occurs within the same display. To do so, we tested the distinctive contribution of local flanker features and global configurations of the flankers on the pattern of crowding errors. Observers (n = 27) estimated the orientation of a target when presented alone or surrounded by flankers. Flankers were grouped into a global configuration, forming an illusory rectangle when aligned or a rectangular configuration when misaligned. We analyzed the error distributions by fitting probabilistic mixture models. Results showed that participants often misreported the orientation of a flanker instead of that of the target. Interestingly, in some trials the orientation of the global configuration was misreported. These results suggest that crowding occurs simultaneously across multiple levels of visual processing and crucially depends on the spatial configuration of the stimulus. Our results pose a challenge to models of crowding with an early single pooling stage and might be better explained by models which incorporate the possibility of multilevel crowding and account for complex target-flanker interactions.

Binocularly Asymmetric Crowding in Glaucoma and a Lack of Binocular Summation in Crowding

Purpose

Glaucoma is associated with progressive loss of retinal ganglion cells. Here we investigated the impact of glaucomatous damage on monocular and binocular crowding in parafoveal vision. We also examined the binocular summation of crowding to see if crowding is alleviated under binocular viewing.

Methods

The study design included 40 individuals with glaucoma and 24 age-similar normal cohorts. For each subject, the magnitude of crowding was determined by the extent of crowding zone. Crowding zone measurements were made binocularly in parafoveal vision (i.e., at 2° and 4° retinal eccentricities) visual field. For a subgroup of glaucoma subjects (n = 17), crowding zone was also measured monocularly for each eye.

Results

Our results showed that, compared with normal cohorts, individuals with glaucoma exhibited significantly larger crowding—enlargement of crowding zone (an increase by 21%; P < 0.01). Moreover, we also observed a lack of binocular summation (i.e., a binocular ratio of 1): binocular crowding was determined by the better eye. Hence, our results did not provide evidence supporting binocular summation of crowding in glaucomatous vision.

Conclusions

Our findings show that crowding is exacerbated in parafoveal vision in glaucoma and binocularly asymmetric glaucoma seems to induce binocularly asymmetric crowding. Furthermore, the lack of binocular summation for crowding observed in glaucomatous vision combined with the lack of binocular summation reported in a previous study on normal healthy vision support the view that crowding may start in the early stages of visual processing, at least before the process of binocular integration takes place.

Perceptual learning of a crowding task: Effects of anisotropy and optotype

Visual crowding refers to the impairment of recognizing peripherally presented objects flanked by distractors. Crowding effects, exhibiting a certain spatial extent between target and flankers, can be reduced by perceptual learning. In this experiment, we investigated the learning-induced reduction of crowding in normally sighted participants and tested if learning on one optotype (Landolt-C) transfers to another (Tumbling-E) or vice versa. Twenty-three normally sighted participants (18-42 years) trained on a crowding task in the right-upper quadrant (target at 6.5 degrees eccentricity) over four sessions. Half of the participants had the four-alternative forced-choice task to discriminate the orientation of a Landolt-C, the other half of participants had the task to discriminate the orientation of a Tumbling-E, each flanked by distractors. In the fifth session, all participants switched to the other untrained optotype, respectively. Learning success was measured as reduction of the spatial extent of crowding. We found an overall significant and comparable learning-induced reduction of crowding in both conditions (Landolt-C and Tumbling-E). However, only in the group who trained on the Landolt-C task did learning effects transfer to the other optotype. The specific target-flanker-constellations may modulate the transfer effects found here. Perceptual learning of a crowding task with optotypes could be a promising tool in rehabilitation programs to help improve peripheral vision (e.g. in patients with central vision loss), but the dependence of possible transfer effects on the optotype and distractors used requires further clarification.

Investigating Visual Crowding of Objects in Complex Real-World Scenes

Visual crowding, the impairment of object recognition in peripheral vision due to flanking objects, has generally been studied using simple stimuli on blank backgrounds. While crowding is widely assumed to occur in natural scenes, it has not been shown rigorously yet. Given that scene contexts can facilitate object recognition, crowding effects may be dampened in real-world scenes. Therefore, this study investigated crowding using objects in computer-generated real-world scenes. In two experiments, target objects were presented with four flanker objects placed uniformly around the target. Previous research indicates that crowding occurs when the distance between the target and flanker is approximately less than half the retinal eccentricity of the target. In each image, the spacing between the target and flanker objects was varied considerably above or below the standard (0.5) threshold to either suppress or facilitate the crowding effect. Experiment 1 cued the target location and then briefly flashed the scene image before participants could move their eyes. Participants then selected the target object's category from a 15-alternative forced choice response set (including all objects shown in the scene). Experiment 2 used eye tracking to ensure participants were centrally fixating at the beginning of each trial and showed the image for the duration of the participant's fixation. Both experiments found object recognition accuracy decreased with smaller spacing between targets and flanker objects. Thus, this study rigorously shows crowding of objects in semantically consistent real-world scenes.

Mixture model investigation of the inner-outer asymmetry in visual crowding reveals a heavier weight towards the visual periphery

Crowding, the failure to identify a peripheral item in clutter, is an essential bottleneck in visual information processing. A hallmark characteristic of crowding is the inner-outer asymmetry in which the outer flanker (more eccentric) produces stronger interference than the inner one (closer to the fovea). We tested the contribution of the inner-outer asymmetry to the pattern of crowding errors in a typical radial crowding display in which both flankers are presented simultaneously on the horizontal meridian. In two experiments, observers were asked to estimate the orientation of a Gabor target. Instead of the target, observers reported the outer flanker much more frequently than the inner one. When the target was the outer Gabor, crowding was reduced. Furthermore, when there were four flankers, two on each side of the target, observers misreported the outer flanker adjacent to the target, not the outermost flanker. Model comparisons suggested that orientation crowding reflects sampling over a weighted sum of the represented features, in which the outer flanker is more heavily weighted compared to the inner one. Our findings reveal a counterintuitive phenomenon: in a radial arrangement of orientation crowding, within a region of selection, the outer item dominates appearance more than the inner one.

A Comparison of Foveal and Peripheral Contour Interaction and Crowding

SIGNIFICANCE

Performance on clinical tests of visual acuity can be influenced by the presence of nearby targets. This study compared the influence of neighboring flanking bars and letters on foveal and peripheral letter identification.

PURPOSE

Contour interaction and crowding refer to an impairment of visual resolution or discrimination produced by different types of flanking stimuli. This study compared the impairment of percent correct letter identification that is produced in normal observers when a target letter is surrounded by an array of four flanking bars (contour interaction) or four flanking letters (crowding).

METHODS

Performance was measured at the fovea and at eccentricities of 1.25, 2.5, and 5° for photopic (200 cd/m2) and mesopic stimuli (0.5 cd/m2) and a range of target-to-flanker separations.

RESULTS

Consistent with previous reports, foveal contour interaction and crowding were more pronounced for photopic than mesopic targets. However, no statistically significant difference existed between foveal contour-interaction and crowding functions at either luminance level. On the other hand, flanking bars produced much less impairment of letter identification than letter flankers at all three peripheral locations, indicating that crowding is more severe than contour interaction in peripheral vision. In contrast to the fovea, peripheral crowding and contour-interaction functions did not differ systematically for targets of photopic and mesopic luminance.

CONCLUSION

The similarity between foveal contour interaction and crowding and the dissimilarity between peripheral contour interaction and crowding suggest the involvement of different mechanisms at different retinal locations.

Superior Parietal Lobule: A Role in Relative Localization of Multiple Different Elements

Simultanagnosia is an impairment in processing multiple visual elements simultaneously consecutive to bilateral posterior parietal damage, and neuroimaging data have specifically implicated the superior parietal lobule (SPL) in multiple element processing. We previously reported that a patient with focal and bilateral lesions of the SPL performed slower than controls in visual search but only for stimuli consisting of separable lines. Here, we further explored this patient's visual processing of plain object (colored disk) versus object consisting of separable lines (letter), presented in isolation (single object) versus in triplets. Identification of objects was normal in isolation but dropped to chance level when surrounded by distracters, irrespective of eccentricity and spacing. We speculate that this poor performance reflects a deficit in processing objects' relative locations within the triplet (for colored disks), aggravated by a deficit in processing the relative location of each separable line (for letters). Confirming this, performance improved when the patient just had to detect the presence of a specific colored disk within the triplets (visual search instruction), while the inability to identify the middle letter was alleviated when the distracters were identical letters that could be grouped, thereby reducing the number of ways individual lines could be bound.

Technical Report: The Mechanism of Contour Interaction Differs in the Fovea and Periphery

SIGNIFICANCE

Both foveal and peripheral contour interactions are based on, as yet, unexplained neural mechanisms. Our results show that, unlike foveal contour interaction, peripheral contour interaction cannot be explained on the basis of the antagonistic structure of neural receptive fields.

PURPOSE

Foveal contour interaction is markedly reduced for mesopic compared with photopic targets. This finding is consistent with an explanation based on the antagonistic structure of neural receptive fields. However, no reduction was found for low-luminance targets in the periphery, possibly because the luminances used previously remained substantially above peripheral scotopic detection thresholds. In this study, we compared foveal and peripheral contour interactions for long-wavelength photopic and mesopic targets, which would be expected to significantly elevate the peripheral retinal detection threshold.

METHODS

Five normal observers viewed a randomly selected Sloan letter surrounded by four flanking bars at several edge-to-edge separations (min arc). Photopic and mesopic stimuli were viewed foveally and at 6° peripherally through a selective red filter that ensured that mesopic targets were within 1 log unit of detection threshold at both retinal locations.

RESULTS

Whereas the magnitude of foveal contour interaction was substantially less at mesopic compared with photopic luminance (20 vs. 46% reduction of percent correct, on average), no significant difference was observed in peripheral contour interaction, which had average mesopic and photopic magnitudes of 38 and 40%. Moreover, confusion matrices representing photopic and mesopic contour interaction differed in the fovea but not in the periphery. The extent of contour interaction did not change with luminance at either retinal location.

CONCLUSIONS

Our results indicate that, although the characteristics of foveal contour interaction can be accounted for by the antagonistic structure of neural receptive fields, the same mechanism is not compatible with the characteristics of peripheral contour interaction.

Number of flankers influences foveal crowding and contour interaction differently

Nearby flanking objects degrade visual resolution. If the flankers are similar to the acuity target, this influence is called crowding (CW), whereas if the flanking stimuli are simple bars then the phenomenon is known as contour interaction (CI). The aim of this study was to compare the influence of the number and position of flankers on foveal CW and CI to investigate possible differences in mechanism of these two effects. Five normal observers viewed single, foveally presented Sloan letters surrounded by 1, 2 or 4 flankers (either a Sloan letter or one-stroke-width bars), presented at several edge-to-edge separations. Single flankers were presented in the right, left, top or bottom position, 2 flankers were placed equally to the right and left or top and bottom of the central target, and 4 flankers were equally spaced in all four directions. Percent correct letter identification was determined for each type, number, position and separation of flankers and confusion matrices were constructed for separations equal to 20% and 100% letter width. Increasing the number of flankers caused an increase in the magnitude of both phenomena. CW showed a greater magnitude than CI for higher numbers of flankers. Analysis of confusion matrices suggests that in addition to the edge-to-edge interaction that appears to mediate CI, letter substitution and feature pooling contribute significantly to CW when higher numbers of flankers are presented. Foveal CW is more strongly influenced by an increase in the number of flankers than CI, which can be explained by the presence of additional interaction effects.

Seven Myths on Crowding and Peripheral Vision

Crowding has become a hot topic in vision research, and some fundamentals are now widely agreed upon. For the classical crowding task, one would likely agree with the following statements. (1) Bouma's law can be stated, succinctly and unequivocally, as saying that critical distance for crowding is about half the target's eccentricity. (2) Crowding is predominantly a peripheral phenomenon. (3) Peripheral vision extends to at most 90° eccentricity. (4) Resolution threshold (the minimal angle of resolution) increases strongly and linearly with eccentricity. Crowding increases at an even steeper rate. (5) Crowding is asymmetric as Bouma has shown. For that inner-outer asymmetry, the peripheral flanker has more effect. (6) Critical crowding distance corresponds to a constant cortical distance in primary visual areas like V1. (7) Except for Bouma's seminal article in 1970, crowding research mostly became prominent starting in the 2000s. I propose the answer is "not really" or "not quite" to these assertions. So should we care? I think we should, before we write the textbook chapters for the next generation.

Crowding in humans is unlike that in convolutional neural networks

Object recognition is a primary function of the human visual system. It has recently been claimed that the highly successful ability to recognise objects in a set of emergent computer vision systems-Deep Convolutional Neural Networks (DCNNs)-can form a useful guide to recognition in humans. To test this assertion, we systematically evaluated visual crowding, a dramatic breakdown of recognition in clutter, in DCNNs and compared their performance to extant research in humans. We examined crowding in three architectures of DCNNs with the same methodology as that used among humans. We manipulated multiple stimulus factors including inter-letter spacing, letter colour, size, and flanker location to assess the extent and shape of crowding in DCNNs. We found that crowding followed a predictable pattern across architectures that was different from that in humans. Some characteristic hallmarks of human crowding, such as invariance to size, the effect of target-flanker similarity, and confusions between target and flanker identities, were completely missing, minimised or even reversed. These data show that DCNNs, while proficient in object recognition, likely achieve this competence through a set of mechanisms that are distinct from those in humans. They are not necessarily equivalent models of human or primate object recognition and caution must be exercised when inferring mechanisms derived from their operation.

Response selection modulates crowding: a cautionary tale for invoking top-down explanations

Object recognition in the periphery is limited by clutter. This phenomenon of visual crowding is ameliorated when the objects are dissimilar. This effect of inter-object similarity has been extensively studied for low-level features and is thought to reflect bottom-up processes. Recently, crowding was also found to be reduced when objects belonged to explicitly distinct groups; that is, crowding was weak when they had low group membership similarity. It has been claimed that top-down knowledge is necessary to explain this effect of group membership, implying that the effect of similarity on crowding cannot be a purely bottom-up process. We tested the claim that the effect of group membership relies on knowledge in two experiments and found that neither explicit knowledge about differences in group membership nor the possibility of acquiring knowledge about target identities is necessary to produce the effects. These results suggest that top-down processes need not be invoked to explain the effect of group membership. Instead, we suggest that differences in flanker reportability that emerge from the differences in group membership are the source of the effect. That is, when targets and flankers are sampled from distinct groups, flankers cannot be inadvertently reported, leading to fewer errors and hence weaker crowding. Further, we argue that this effect arises at the stage of response selection. This conclusion is well supported by an analytical model based on these principles. We conclude that previously observed effects in crowding attributed to top-down or higher level processes might instead be due to post-perceptual response selection strategies.

Dissociating the functions of superior and inferior parts of the left ventral occipito-temporal cortex during visual word and object processing

During word and object recognition, extensive activation has consistently been observed in the left ventral occipito-temporal cortex (vOT), focused around the occipito-temporal sulcus (OTs). Previous studies have shown that there is a hierarchy of responses from posterior to anterior vOT regions (along the y-axis) that corresponds with increasing levels of recognition - from perceptual to semantic processing, respectively. In contrast, the functional differences between superior and inferior vOT responses (i.e. along the z-axis) have not yet been elucidated. To investigate, we conducted an extensive review of the literature and found that peak activation for reading varies by more than 1 cm in the z-axis. In addition, we investigated functional differences between superior and inferior parts of left vOT by analysing functional MRI data from 58 neurologically normal skilled readers performing 8 different visual processing tasks. We found that group activation in superior vOT was significantly more sensitive than inferior vOT to the type of task, with more superior vOT activation when participants were matching visual stimuli for their semantic or perceptual content than producing speech to the same stimuli. This functional difference along the z-axis was compared to existing boundaries between cytoarchitectonic areas around the OTs. In addition, using dynamic causal modelling, we show that connectivity from superior vOT to anterior vOT increased with semantic content during matching tasks but not during speaking tasks whereas connectivity from inferior vOT to anterior vOT was sensitive to semantic content for matching and speaking tasks. The finding of a functional dissociation between superior and inferior parts of vOT has implications for predicting deficits and response to rehabilitation for patients with partial damage to vOT following stroke or neurosurgery.

Averaging sets of expressive faces is modulated by eccentricity

Research has shown that participants can extract the average facial expression from a set of faces when these were presented at fixation. In this study, we investigated whether this performance would be modulated by eccentricity given that neural resources are limited outside the foveal region. We also examined whether or not there would be compulsory averaging in the parafovea as has been previously reported for the orientation of Gabor patches by Parkes, Lund, Angelucci, Solomon, and Morgan (2001). Participants were presented with expressive faces (alone or in sets of nine, at fixation or at 3° to the left or right) and were asked to identify the expression of the central target face or to estimate the average expression of the set. Our results revealed that, although participants were able to extract average facial expressions in central and parafoveal conditions, their performance was superior in the parafovea, suggesting facilitated averaging outside the fovea by peripheral mechanisms. Furthermore, regardless of whether the task was to judge the expression of the central target or set average, participants had a tendency to identify central targets' expressions in the fovea but were compelled to average in the parafovea, a finding consistent with compulsory averaging. The data also supported averaging over substitution models of crowding. We conclude that the ability to extract average expressions in sets of faces and identify single targets' facial expressions is influenced by eccentricity.

Challenges to pooling models of crowding: Implications for visual mechanisms

A set of phenomena known as crowding reveal peripheral vision's vulnerability in the face of clutter. Crowding is important both because of its ubiquity, making it relevant for many real-world tasks and stimuli, and because of the window it provides onto mechanisms of visual processing. Here we focus on models of the underlying mechanisms. This review centers on a popular class of models known as pooling models, as well as the phenomenology that appears to challenge a pooling account. Using a candidate high-dimensional pooling model, we gain intuitions about whether a pooling model suffices and reexamine the logic behind the pooling challenges. We show that pooling mechanisms can yield substitution phenomena and therefore predict better performance judging the properties of a set versus a particular item. Pooling models can also exhibit some similarity effects without requiring mechanisms that pool at multiple levels of processing, and without constraining pooling to a particular perceptual group. Moreover, we argue that other similarity effects may in part be due to noncrowding influences like cuing. Unlike low-dimensional straw-man pooling models, high-dimensional pooling preserves rich information about the stimulus, which may be sufficient to support high-level processing. To gain insights into the implications for pooling mechanisms, one needs a candidate high-dimensional pooling model and cannot rely on intuitions from low-dimensional models. Furthermore, to uncover the mechanisms of crowding, experiments need to separate encoding from decision effects. While future work must quantitatively examine all of the challenges to a high-dimensional pooling account, insights from a candidate model allow us to conclude that a high-dimensional pooling mechanism remains viable as a model of the loss of information leading to crowding.

The Role of Emotional Expression and Eccentricity on Gaze Perception

The perception of another’s gaze direction and facial expression complements verbal communication and modulates how we interact with other people. However, our perception of these two cues is not always accurate, even when we are looking directly at the person. In addition, in many cases social communication occurs within groups of people where we can’t always look directly at every person in the group. Here, we sought to examine how the presence of other people influences our perception of a target face. We asked participants to judge the direction of gaze of the target face as either looking to their left, to their right or directly at them, when the face was viewed on its own or viewed within a group of other identity faces. The target face either had an angry or a neutral expression and was viewed directly (foveal experiment), or within peripheral vision (peripheral experiment). When the target was viewed within a group, the flanking faces also had either neutral or angry expressions and their gaze was in one of five different directions (from averted leftwards to averted rightwards in steps of 10°). When the target face was viewed foveally there was no effect of target emotion on participants’ judgments of its gaze direction. There was also no effect of the presence of flankers (regardless of expression) on the perception of the target gaze. When the target face was viewed peripherally, participants judged its direction of gaze to be direct over a wider range of gaze deviations than when viewed foveally, and more so for angry faces than neutral faces. We also find that flankers (regardless of emotional expression) did not influence performance. This suggests that observers judge that angry faces were looking at them over a broad range of gaze deviations in the periphery only, possibly resulting from increased uncertainty about the stimulus.

Is excessive visual crowding causally linked to developmental dyslexia?

For about 10% of children reading acquisition is extremely difficult because they are affected by a heritable neurobiological disorder called developmental dyslexia (DD), mainly associated to an auditory-phonological disorder. Visual crowding is a universal phenomenon that impairs the recognition of stimuli in clutter, such as a letter in a word or a word in a text. Several studies have shown an excessive crowding in individuals with DD, but the causal link between excessive crowding and DD is not yet clearly established. An excessive crowding might be, indeed, a simple effect of DD due to reduced reading experience. The results of five experiments in 181 children reveal that: (i) an excessive crowding only at unattended locations characterizes an unselected group of children with DD (Experiment 1); (ii) an extra-large spaced text increases reading accuracy by reducing crowding in an unselected group of children with DD (Experiment 2); (iii) efficient attentional action video game trainings reduce crowding and accelerate reading speed in two unselected groups of children with DD (Experiment 3 and 4), and; (iv) pre-reading crowding longitudinally predicts future poor readers (Experiment 5). Our results show multiple causal links between visual crowding and learning to read. These findings provide new insights for a more efficient remediation and prevention for DD.

Feature contingencies when reading letter strings

Many models posit the use of distinctive spatial features to recognize letters of the alphabet, a fundamental component of reading. It has also been hypothesized that when letters are in close proximity, visual crowding may cause features to mislocalize between nearby letters, causing identification errors. Here, we took a data-driven approach to investigate these aspects of textual processing. Using data collected from subjects identifying each letter in thousands of lower-case letter trigrams presented in the peripheral visual field, we found characteristic error patterns in the results suggestive of the use of particular spatial features. Distinctive features were seldom entirely missed, and we found evidence for errors due to doubling, masking, and migration of features. Dependencies both amongst neighboring letters and in the responses revealed the contingent nature of processing letter strings, challenging the most basic models of reading that ignore either crowding or featural decomposition.

Variations in crowding, saccadic precision, and spatial localization reveal the shared topology of spatial vision

Visual sensitivity varies across the visual field in several characteristic ways. For example, sensitivity declines sharply in peripheral (vs. foveal) vision and is typically worse in the upper (vs. lower) visual field. These variations can affect processes ranging from acuity and crowding (the deleterious effect of clutter on object recognition) to the precision of saccadic eye movements. Here we examine whether these variations can be attributed to a common source within the visual system. We first compared the size of crowding zones with the precision of saccades using an oriented clock target and two adjacent flanker elements. We report that both saccade precision and crowded-target reports vary idiosyncratically across the visual field with a strong correlation across tasks for all participants. Nevertheless, both group-level and trial-by-trial analyses reveal dissociations that exclude a common representation for the two processes. We therefore compared crowding with two measures of spatial localization: Landolt-C gap resolution and three-dot bisection. Here we observe similar idiosyncratic variations with strong interparticipant correlations across tasks despite considerably finer precision. Hierarchical regression analyses further show that variations in spatial precision account for much of the variation in crowding, including the correlation between crowding and saccades. Altogether, we demonstrate that crowding, spatial localization, and saccadic precision show clear dissociations, indicative of independent spatial representations, whilst nonetheless sharing idiosyncratic variations in spatial topology. We propose that these topological idiosyncrasies are established early in the visual system and inherited throughout later stages to affect a range of higher-level representations.

Visual crowding is a combination of an increase of positional uncertainty, source confusion, and featural averaging

Although we perceive a richly detailed visual world, our ability to identify individual objects is severely limited in clutter, particularly in peripheral vision. Models of such “crowding” have generally been driven by the phenomenological misidentifications of crowded targets: using stimuli that do not easily combine to form a unique symbol (e.g. letters or objects), observers typically confuse the source of objects and report either the target or a distractor, but when continuous features are used (e.g. orientated gratings or line positions) observers report a feature somewhere between the target and distractor. To reconcile these accounts, we develop a hybrid method of adjustment that allows detailed analysis of these multiple error categories. Observers reported the orientation of a target, under several distractor conditions, by adjusting an identical foveal target. We apply new modelling to quantify whether perceptual reports show evidence of positional uncertainty, source confusion, and featural averaging on a trial-by-trial basis. Our results show that observers make a large proportion of source-confusion errors. However, our study also reveals the distribution of perceptual reports that underlie performance in this crowding task more generally: aggregate errors cannot be neatly labelled because they are heterogeneous and their structure depends on target-distractor distance.

Can (should) theories of crowding be unified?

Objects in clutter are difficult to recognize, a phenomenon known as crowding. There is little consensus on the underlying mechanisms of crowding, and a large number of models have been proposed. There have also been attempts at unifying the explanations of crowding under a single model, such as the weighted feature model of Harrison and Bex (2015) and the texture synthesis model of Rosenholtz and colleagues (Balas, Nakano, & Rosenholtz, 2009; Keshvari & Rosenholtz, 2016). The goal of this work was to test various models of crowding and to assess whether a unifying account can be developed. Adopting Harrison and Bex's (2015) experimental paradigm, we asked observers to report the orientation of two concentric C-stimuli. Contrary to the predictions of their model, observers' recognition accuracy was worse for the inner C-stimulus. In addition, we demonstrated that the stimulus paradigm used by Harrison and Bex has a crucial confounding factor, eccentricity, which limits its usage to a very narrow range of stimulus parameters. Nevertheless, reporting the orientations of both C-stimuli in this paradigm proved very useful in pitting different crowding models against each other. Specifically, we tested deterministic and probabilistic versions of averaging, substitution, and attentional resolution models as well as the texture synthesis model. None of the models alone was able to explain the entire set of data. Based on these findings, we discuss whether the explanations of crowding can (should) be unified.

Typical magnitude and spatial extent of crowding in autism

Enhanced spatial processing of local visual details has been reported in individuals with autism spectrum conditions (ASC), and crowding is postulated to be a mechanism that may produce this ability. However, evidence for atypical crowding in ASC is mixed, with some studies reporting a complete lack of crowding in autism and others reporting a typical magnitude of crowding between individuals with and without ASC. Here, we aim to disambiguate these conflicting results by testing both the magnitude and the spatial extent of crowding in individuals with ASC (N = 25) and age- and IQ-matched controls (N = 23) during an orientation discrimination task. We find a strong crowding effect in individuals with and without ASC, which falls off as the distance between target and flanker is increased. Both the magnitude and the spatial range of this effect were comparable between individuals with and without ASC. We also find typical (uncrowded) orientation discrimination thresholds in individuals with ASC. These findings suggest that the spatial extent of crowding is unremarkable in ASC, and is therefore unlikely to account for the visual symptoms reported in individuals with the diagnosis.

Perceptual and Cognitive Factors Imposing "Speed Limits" on Reading Rate: A Study with the Rapid Serial Visual Presentation

Adults read at high speed, but estimates of their reading rate vary greatly, i.e., from 100 to 1500 words per minute (wpm). This discrepancy is likely due to different recording methods and to the different perceptual and cognitive processes involved in specific test conditions. The present study investigated the origins of these notable differences in RSVP reading rate (RR). In six experiments we investigated the role of many different perceptual and cognitive variables. The presence of a mask caused a steep decline in reading rate, with an estimated masking cost of about 200 wpm. When the decoding process was isolated, RR approached values of 1200 wpm. When the number of stimuli exceeded the short-term memory span, RR decreased to 800 wpm. The semantic context contributed to reading speed only by a factor of 1.4. Finally, eye movements imposed an upper limit on RR (around 300 wpm). Overall, data indicate a speed limit of 300 wpm, which corresponds to the time needed for eye movement execution, i.e., the most time consuming mechanism. Results reconcile differences in reading rates reported by different laboratories and thus provide suggestions for targeting different components of reading rate.

A New Font, Specifically Designed for Peripheral Vision, Improves Peripheral Letter and Word Recognition, but Not Eye-Mediated Reading Performance

Reading speed is dramatically reduced when readers cannot use their central vision. This is because low visual acuity and crowding negatively impact letter recognition in the periphery. In this study, we designed a new font (referred to as the Eido font) in order to reduce inter-letter similarity and consequently to increase peripheral letter recognition performance. We tested this font by running five experiments that compared the Eido font with the standard Courier font. Letter spacing and x-height were identical for the two monospaced fonts. Six normally-sighted subjects used exclusively their peripheral vision to run two aloud reading tasks (with eye movements), a letter recognition task (without eye movements), a word recognition task (without eye movements) and a lexical decision task. Results show that reading speed was not significantly different between the Eido and the Courier font when subjects had to read single sentences with a round simulated gaze-contingent central scotoma (10° diameter). In contrast, Eido significantly decreased perceptual errors in peripheral crowded letter recognition (-30% errors on average for letters briefly presented at 6° eccentricity) and in peripheral word recognition (-32% errors on average for words briefly presented at 6° eccentricity).

Pooling of continuous features provides a unifying account of crowding

Visual crowding refers to phenomena in which the perception of a peripheral target is strongly affected by nearby flankers. Observers often report seeing the stimuli as "jumbled up," or otherwise confuse the target with the flankers. Theories of visual crowding contend over which aspect of the stimulus gets confused in peripheral vision. Attempts to test these theories have led to seemingly conflicting results, with some experiments suggesting that the mechanism underlying crowding operates on unbound features like color or orientation (Parkes, Lund, Angelucci, Solomon, & Morgan, 2001), while others suggest it "jumbles up" more complex features, or even objects like letters (Korte, 1923). Many of these theories operate on discrete features of the display items, such as the orientation of each line or the identity of each item. By contrast, here we examine the predictions of the Texture Tiling Model, which operates on continuous feature measurements (Balas, Nakano, & Rosenholtz, 2009). We show that the main effects of three studies from the crowding literature are consistent with the predictions of Texture Tiling Model. This suggests that many of the stimulus-specific curiosities surrounding crowding are the inherent result of the informativeness of a rich set of image statistics for the particular tasks.

Crowding in Visual Working Memory Reveals Its Spatial Resolution and the Nature of Its Representations

Spatial resolution fundamentally limits any image representation. Although this limit has been extensively investigated for perceptual representations by assessing how neighboring flankers degrade the perception of a peripheral target with visual crowding, the corresponding limit for representations held in visual working memory (VWM) is unknown. In the present study, we evoked crowding in VWM and directly compared resolution in VWM and perception. Remarkably, the spatial resolution of VWM proved to be no worse than that of perception. However, mixture modeling of errors caused by crowding revealed the qualitatively distinct nature of these representations. Perceptual crowding errors arose from both increased imprecision in target representations and substitution of flankers for targets. By contrast, VWM crowding errors arose exclusively from substitutions, which suggests that VWM transforms analog perceptual representations into discrete items. Thus, although perception and VWM share a common resolution limit, exceeding this limit reveals distinct mechanisms for perceiving images and holding them in mind.

Macaque monkeys experience visual crowding

In peripheral vision, objects that are easily discriminated on their own become less discriminable in the presence of surrounding clutter. This phenomenon is known as crowding.The neural mechanisms underlying crowding are not well understood. Better insight might come from single-neuron recording in nonhuman primates, provided they exhibit crowding; however, previous demonstrations of crowding have been confined to humans. In the present study, we set out to determine whether crowding occurs in rhesus macaque monkeys. We found that animals trained to identify a target letter among flankers displayed three hallmarks of crowding as established in humans. First, at a given eccentricity, increasing the spacing between the target and the flankers improved recognition accuracy. Second, the critical spacing, defined as the minimal spacing at which target discrimination was reliable, was proportional to eccentricity. Third, the critical spacing was largely unaffected by object size. We conclude that monkeys, like humans, experience crowding. These findings open the door to studies of crowding at the neuronal level in the monkey visual system.

Dancing letters'and ticks that buzz around aimlessly: on the origin of crowding

When we see an object, we know where it is. Or do we? Perhaps not in indirect vision, as was observed by the gestalt psychologist Korte in 1923. Objects and object parts appear to 'dance around', and these phenomena may underlie a part of what is called the crowding effect today. From Korte's account of pattern recognition in indirect vision, I select two phenomena: a loss of the positional code for letter parts and a loss of the same for whole letters. Using these examples, I present a novel, speculative explanation for a contradiction in the literature: patterns located more peripherally than a target show more interference than do more centrally located patterns, yet for whole-letter confusions the asymmetry is the other way round. The inward, not the outward, flanker is increasingly confused with a target at increasing target eccentricities. I propose that feature-binding decreases with eccentricity such that free-floating letter parts more often intrude from the periphery and whole letters from the centre. I conclude with a few remarks on computational modelling as, hopefully, a challenge to neural computationalists.

Radial-tangential anisotropy of crowding in the early visual areas

Crowding, the inability to recognize an individual object in clutter (Bouma H. Nature 226: 177-178, 1970), is considered a major impediment to object recognition in peripheral vision. Despite its significance, the cortical loci of crowding are not well understood. In particular, the role of the primary visual cortex (V1) remains unclear. Here we utilize a diagnostic feature of crowding to identify the earliest cortical locus of crowding. Controlling for other factors, radially arranged flankers induce more crowding than tangentially arranged ones (Toet A, Levi DM. Vision Res 32: 1349-1357, 1992). We used functional magnetic resonance imaging (fMRI) to measure the change in mean blood oxygenation level-dependent (BOLD) response due to the addition of a middle letter between a pair of radially or tangentially arranged flankers. Consistent with the previous finding that crowding is associated with a reduced BOLD response [Millin R, Arman AC, Chung ST, Tjan BS. Cereb Cortex (July 5, 2013). doi:10.1093/cercor/bht159], we found that the BOLD signal evoked by the middle letter depended on the arrangement of the flankers: less BOLD response was associated with adding the middle letter between radially arranged flankers compared with adding it between tangentially arranged flankers. This anisotropy in BOLD response was present as early as V1 and remained significant in downstream areas. The effect was observed while subjects' attention was diverted away from the testing stimuli. Contrast detection threshold for the middle letter was unaffected by flanker arrangement, ruling out surround suppression of contrast response as a major factor in the observed BOLD anisotropy. Our findings support the view that V1 contributes to crowding.

Visual crowding cannot be wholly explained by feature pooling

Visual perception is dramatically impaired when a peripheral target is embedded within clutter, a phenomenon known as visual crowding. Despite decades of study, the mechanisms underlying crowding remain a matter of debate. Feature pooling models assert that crowding results from a compulsory pooling (e.g., averaging) of target and distractor features. This view has been extraordinarily influential in recent years, so much so that crowding is typically regarded as synonymous with pooling. However, many demonstrations of feature pooling can also be accommodated by a probabilistic substitution model where observers occasionally report a distractor as the target. Here, we directly compared pooling and substitution using an analytical approach sensitive to both alternatives. In four experiments, we asked observers to report the precise orientation of a target stimulus flanked by two irrelevant distractors. In all cases, the observed data were well described by a quantitative model that assumes probabilistic substitution, and poorly described by a quantitative model that assumes that targets and distractors are averaged. These results challenge the widely held assumption that crowding can be wholly explained by compulsory pooling.

A crowdful of letters: disentangling the role of similarity, eccentricity and spatial frequencies in letter crowding

The present study investigated the joint impact of target-flanker similarity and of spatial frequency content on the crowding effect in letter identification. We presented spatial frequency filtered letters to neurologically intact non-dyslexic readers while manipulating target-flanker distance, target eccentricity and target-flanker confusability (letter similarity metric based on published letter confusion matrices). The results show that high target-flanker confusability magnifies crowding. They also reveal an intricate pattern of interactions of the spatial frequency content of the stimuli with target eccentricity, flanker distance and similarity. The findings are congruent with the notion that crowding results from the inappropriate pooling of target and flanker features and that this integration is more likely to match a response template at a subsequent decision stage with similar than dissimilar flankers. In addition, the evidence suggests that crowding from similar flankers is biased towards relatively high spatial frequencies and that crowding shifts towards lower spatial frequencies as target eccentricity is increased.

Image correlates of crowding in natural scenes

Visual crowding is the inability to identify visible features when they are surrounded by other structure in the peripheral field. Since natural environments are replete with structure and most of our visual field is peripheral, crowding represents the primary limit on vision in the real world. However, little is known about the characteristics of crowding under natural conditions. Here we examine where crowding occurs in natural images. Observers were required to identify which of four locations contained a patch of "dead leaves'' (synthetic, naturalistic contour structure) embedded into natural images. Threshold size for the dead leaves patch scaled with eccentricity in a manner consistent with crowding. Reverse correlation at multiple scales was used to determine local image statistics that correlated with task performance. Stepwise model selection revealed that local RMS contrast and edge density at the site of the dead leaves patch were of primary importance in predicting the occurrence of crowding once patch size and eccentricity had been considered. The absolute magnitudes of the regression weights for RMS contrast at different spatial scales varied in a manner consistent with receptive field sizes measured in striate cortex of primate brains. Our results are consistent with crowding models that are based on spatial averaging of features in the early stages of the visual system, and allow the prediction of where crowding is likely to occur in natural images.