Modulation of transient and sustained response components of V4 neurons by temporal crowding in flashed stimulus sequences

Rethinking simultaneous suppression in visual cortex via compressive spatiotemporal population receptive fields

When multiple visual stimuli are presented simultaneously in the receptive field, the neural response is suppressed compared to presenting the same stimuli sequentially. The prevailing hypothesis suggests that this suppression is due to competition among multiple stimuli for limited resources within receptive fields, governed by task demands. However, it is unknown how stimulus-driven computations may give rise to simultaneous suppression. Using fMRI, we find simultaneous suppression in single voxels, which varies with both stimulus size and timing, and progressively increases up the visual hierarchy. Using population receptive field (pRF) models, we find that compressive spatiotemporal summation rather than compressive spatial summation predicts simultaneous suppression, and that increased simultaneous suppression is linked to larger pRF sizes and stronger compressive nonlinearities. These results necessitate a rethinking of simultaneous suppression as the outcome of stimulus-driven compressive spatiotemporal computations within pRFs, and open new opportunities to study visual processing capacity across space and time.

Rethinking simultaneous suppression in visual cortex via compressive spatiotemporal population receptive fields

When multiple visual stimuli are presented simultaneously in the receptive field, the neural response is suppressed compared to presenting the same stimuli sequentially. The prevailing hypothesis suggests that this suppression is due to competition among multiple stimuli for limited resources within receptive fields, governed by task demands. However, it is unknown how stimulus-driven computations may give rise to simultaneous suppression. Using fMRI, we find simultaneous suppression in single voxels, which varies with both stimulus size and timing, and progressively increases up the visual hierarchy. Using population receptive field (pRF) models, we find that compressive spatiotemporal summation rather than compressive spatial summation predicts simultaneous suppression, and that increased simultaneous suppression is linked to larger pRF sizes and stronger compressive nonlinearities. These results necessitate a rethinking of simultaneous suppression as the outcome of stimulus-driven compressive spatiotemporal computations within pRFs, and open new opportunities to study visual processing capacity across space and time.

Mammals Achieve Common Neural Coverage of Visual Scenes Using Distinct Sampling Behaviors

Most vertebrates use head and eye movements to quickly change gaze orientation and sample different portions of the environment with periods of stable fixation. Visual information must be integrated across fixations to construct a complete perspective of the visual environment. In concert with this sampling strategy, neurons adapt to unchanging input to conserve energy and ensure that only novel information from each fixation is processed. We demonstrate how adaptation recovery times and saccade properties interact and thus shape spatiotemporal tradeoffs observed in the motor and visual systems of mice, cats, marmosets, macaques, and humans. These tradeoffs predict that in order to achieve similar visual coverage over time, animals with smaller receptive field sizes require faster saccade rates. Indeed, we find comparable sampling of the visual environment by neuronal populations across mammals when integrating measurements of saccadic behavior with receptive field sizes and V1 neuronal density. We propose that these mammals share a common statistically driven strategy of maintaining coverage of their visual environment over time calibrated to their respective visual system characteristics.

The Bouma law accounts for crowding in 50 observers

Crowding is the failure to recognize an object due to surrounding clutter. Our visual crowding survey measured 13 crowding distances (or "critical spacings") twice in each of 50 observers. The survey includes three eccentricities (0, 5, and 10 deg), four cardinal meridians, two orientations (radial and tangential), and two fonts (Sloan and Pelli). The survey also tested foveal acuity, twice. Remarkably, fitting a two-parameter model-the well-known Bouma law, where crowding distance grows linearly with eccentricity-explains 82% of the variance for all 13 × 50 measured log crowding distances, cross-validated. An enhanced Bouma law, with factors for meridian, crowding orientation, target kind, and observer, explains 94% of the variance, again cross-validated. These additional factors reveal several asymmetries, consistent with previous reports, which can be expressed as crowding-distance ratios: 0.62 horizontal:vertical, 0.79 lower:upper, 0.78 right:left, 0.55 tangential:radial, and 0.78 Sloan-font:Pelli-font. Across our observers, peripheral crowding is independent of foveal crowding and acuity. Evaluation of the Bouma factor, b (the slope of the Bouma law), as a biomarker of visual health would be easier if there were a way to compare results across crowding studies that use different methods. We define a standardized Bouma factor b' that corrects for differences from Bouma's 25 choice alternatives, 75% threshold criterion, and linearly symmetric flanker placement. For radial crowding on the right meridian, the standardized Bouma factor b' is 0.24 for this study, 0.35 for Bouma (1970), and 0.30 for the geometric mean across five representative modern studies, including this one, showing good agreement across labs, including Bouma's. Simulations, confirmed by data, show that peeking can skew estimates of crowding (e.g., greatly decreasing the mean or doubling the SD of log b). Using gaze tracking to prevent peeking, individual differences are robust, as evidenced by the much larger 0.08 SD of log b across observers than the mere 0.03 test-retest SD of log b measured in half an hour. The ease of measurement of crowding enhances its promise as a biomarker for dyslexia and visual health.

Rapid adaptation of primate LGN neurons to drifting grating stimulation

The visual system needs to dynamically adapt to changing environments. Much is known about the adaptive effects of constant stimulation over prolonged periods. However, there are open questions regarding adaptation to stimuli that are changing over time, interrupted, or repeated. Feature-specific adaptation to repeating stimuli has been shown to occur as early as primary visual cortex (V1), but there is also evidence for more generalized, fatigue-like adaptation that might occur at an earlier stage of processing. Here, we show adaptation in the lateral geniculate nucleus (LGN) of awake, fixating monkeys following brief (1 s) exposure to repeated cycles of a 4-Hz drifting grating. We examined the relative change of each neuron's response across successive (repeated) grating cycles. We found that neurons from all cell classes (parvocellular, magnocellular, and koniocellular) showed significant adaptation. However, only magnocellular neurons showed adaptation when responses were averaged to a population response. In contrast to firing rates, response variability was largely unaffected. Finally, adaptation was comparable between monocular and binocular stimulation, suggesting that rapid LGN adaptation is monocular in nature.NEW & NOTEWORTHY Neural adaptation can be defined as reduction of spiking responses following repeated or prolonged stimulation. Adaptation helps adjust neural responsiveness to avoid saturation and has been suggested to improve perceptual selectivity, information transmission, and predictive coding. Here, we report rapid adaptation to repeated cycles of gratings drifting over the receptive field of neurons at the earliest site of postretinal processing, the lateral geniculate nucleus of the thalamus.

Dynamic divisive normalization circuits explain and predict change detection in monkey area MT

Sudden changes in visual scenes often indicate important events for behavior. For their quick and reliable detection, the brain must be capable to process these changes as independently as possible from its current activation state. In motion-selective area MT, neurons respond to instantaneous speed changes with pronounced transients, often far exceeding the expected response as derived from their speed tuning profile. We here show that this complex, non-linear behavior emerges from the combined temporal dynamics of excitation and divisive inhibition, and provide a comprehensive mathematical analysis. A central prediction derived from this investigation is that attention increases the steepness of the transient response irrespective of the activation state prior to a stimulus change, and irrespective of the sign of the change (i.e. irrespective of whether the stimulus is accelerating or decelerating). Extracellular recordings of attention-dependent representation of both speed increments and decrements confirmed this prediction and suggest that improved change detection derives from basic computations in a canonical cortical circuitry.

Perisaccadic encoding of temporal information in macaque area V4

The accurate processing of temporal information is of critical importance in everyday life. Yet, psychophysical studies in humans have shown that the perception of time is distorted around saccadic eye movements. The neural correlates of this misperception are still poorly understood. Behavioral and neural evidence suggest that it is tightly linked to other known perisaccadic modulations of visual perception. To further our understanding of how temporal processing is affected by saccades, we studied the representations of brief visual time intervals during fixation and saccades in area V4 of two awake macaques. We presented random sequences of vertical bar stimuli and extracted neural responses to double-pulse stimulation at varying interstimulus intervals. Our results show that temporal information about very brief intervals of as brief as 20 ms is reliably represented in the multiunit activity in area V4. Response latencies were not systematically modulated by the saccade. However, a general increase in perisaccadic activity altered the ratio of response amplitudes within stimulus pairs compared with fixation. In line with previous studies showing that the perception of brief time intervals is partly based on response levels, this may be seen as a possible correlate of the perisaccadic misperception of time.NEW & NOTEWORTHY We investigated for the first time how temporal information on very brief timescales is represented in area V4 around the time of saccadic eye movements. Overall, the responses showed an unexpectedly precise representation of time intervals. Our finding of a perisaccadic modulation of relative response amplitudes introduces a new possible correlate of saccade-related perceptual distortions of time.

Altered neural oscillations and connectivity in the beta band underlie detail-oriented visual processing in autism

Sensory and perceptual anomalies may have a major impact on basic cognitive and social skills in humans. Autism Spectrum Disorder (ASD) represents a special perspective to explore this relationship, being characterized by both these features. The present study employed electroencephalography (EEG) to test whether detail-oriented visual perception, a recognized hallmark of ASD, is associated with altered neural oscillations and functional connectivity in the beta frequency band, considering its role in feedback and top-down reentrant signalling in the typical population. Using a visual crowding task, where participants had to discriminate a peripheral target letter surrounded by flankers at different distances, we found that detail-oriented processing in children with ASD, as compared to typically developing peers, could be attributed to anomalous oscillatory activity in the beta band (15-30 Hz), while no differences emerged in the alpha band (8-12 Hz). Altered beta oscillatory response reflected in turn atypical functional connectivity between occipital areas, where the initial stimulus analysis is accomplished, and infero-temporal regions, where objects identity is extracted. Such atypical beta connectivity predicted both ASD symptomatology and their detail-oriented processing. Overall, these results might be explained by an altered feedback connectivity within the visual system, with potential cascade effects in visual scene parsing and higher order functions.

Empirical Evidence for Intraspecific Multiple Realization?

Despite the remarkable advances in behavioral and brain sciences over the last decades, the mind-body (brain) problem is still an open debate and one of the most intriguing questions for both cognitive neuroscience and philosophy of mind. Traditional approaches have conceived this problem in terms of a contrast between physicalist monism and Cartesian dualism. However, since the late sixties, the landscape of philosophical views on the problem has become more varied and complex. The Multiple Realization Thesis (MRT) claims that mental properties can be (or are) realized, and mental processes can be (or are) implemented by neural correlates of different kinds. Thus, MRT challenges the psychoneural type-identity theory and the corresponding reductionism. Many philosophers have acknowledged the a priori plausibility of MRT. However, the existence of empirical evidence in favor of intraspecific, human multiple realizations of mental processes and properties is still controversial. Here, we illustrate some cases that provide empirical evidence in support of MRT. Recently, it has been proposed that foveal agnosic vision, like peripheral vision, can be restored by increasing object parts’ spacing (Crutch and Warrington, 2007; Strappini et al., 2017b). Agnosic fovea and normal periphery are both limited by crowding, which impairs object recognition, and provides the signature of visual integration. Here, we define a psychological property of restored object identification, and we cross-reference the data of visually impaired patients with different etiologies. In particular, we compare the data of two stroke patients, two patients with posterior cortical atrophy, six cases of strabismic amblyopia, and one case with restored sight. We also compare these patients with unimpaired subjects tested in the periphery. We show that integration (i.e., restored recognition) seems to describe quite accurately the visual performance in all these cases. Whereas the patients have different etiologies and different neural correlates, the unimpaired subjects have no neural damage. Thus, similarity in the psychological property given the differences in the neural substrate can be interpreted in relation to MRT and provide evidence in its support. Finally, we will frame our contribution within the current debate concerning MRT providing new and compelling empirical evidence.

Predicting neuronal dynamics with a delayed gain control model

Visual neurons respond to static images with specific dynamics: neuronal responses sum sub-additively over time, reduce in amplitude with repeated or sustained stimuli (neuronal adaptation), and are slower at low stimulus contrast. Here, we propose a simple model that predicts these seemingly disparate response patterns observed in a diverse set of measurements-intracranial electrodes in patients, fMRI, and macaque single unit spiking. The model takes a time-varying contrast time course of a stimulus as input, and produces predicted neuronal dynamics as output. Model computation consists of linear filtering, expansive exponentiation, and a divisive gain control. The gain control signal relates to but is slower than the linear signal, and this delay is critical in giving rise to predictions matched to the observed dynamics. Our model is simpler than previously proposed related models, and fitting the model to intracranial EEG data uncovers two regularities across human visual field maps: estimated linear filters (temporal receptive fields) systematically differ across and within visual field maps, and later areas exhibit more rapid and substantial gain control. The model is further generalizable to account for dynamics of contrast-dependent spike rates in macaque V1, and amplitudes of fMRI BOLD in human V1.

Cortical Reorganization of Peripheral Vision Induced by Simulated Central Vision Loss

When one's central vision is deprived, a spared part of the peripheral retina acts as a pseudofovea for fixation. The neural mechanisms underlying this compensatory adjustment remain unclear. Here we report cortical reorganization induced by simulated central vision loss. Human subjects of both sexes learned to place the target at an eccentric retinal locus outside their blocked visual field for object tracking. Before and after training, we measured visual crowding-a bottleneck of object identification in peripheral vision, using psychophysics and fMRI. We found that training led to an axis-specific reduction of crowding. The change of the crowding effect was reflected in the change of BOLD signal, as a release of cortical suppression in multiple visual areas starting as early as V1. Our findings suggest that the adult visual system is capable of reshaping its oculomotor control and sensory coding to adapt to impoverished visual input.SIGNIFICANCE STATEMENT By simulating central vision loss in normally sighted adults, we found that oculomotor training not only induces PRL, but also facilitates form processing in peripheral vision. As subjects learned to place the target at an eccentric retinal locus, "visual crowding"-the detrimental effect of clutter on peripheral object identification-was reduced. The reduction of the crowding effect was accompanied by a release of response suppression in the visual cortex. These findings indicate that the adult visual system is capable of reshaping the peripheral vision to adapt to central vision loss.

Linkage between retinal ganglion cell density and the nonuniform spatial integration across the visual field

The integration of visual information over space is critical to human pattern vision. For either luminance detection or object recognition, the position of the target in the visual field governs the size of a window within which visual information is integrated. Here we analyze the relationship between the topographic distribution of ganglion cell density and the nonuniform spatial integration across the visual field. We find that the variation in the retinal ganglion cell (RGC) density across the human retina is closely matched to the variation in the extent of spatial integration. Our study suggests that a fixed number of RGCs subserves spatial integration of visual input, independent of the visual-field location.

The ability to integrate visual information over space is a fundamental component of human pattern vision. Regardless of whether it is for detecting luminance contrast or for recognizing objects in a cluttered scene, the position of the target in the visual field governs the size of a window within which visual information is integrated. Here we analyze the relationship between the topographic distribution of ganglion cell density and the nonuniform spatial integration across the visual field. The extent of spatial integration for luminance detection (Ricco’s area) and object recognition (crowding zone) are measured at various target locations. The number of retinal ganglion cells (RGCs) underlying Ricco’s area or crowding zone is estimated by computing the product of Ricco’s area (or crowding zone) and RGC density for a given target location. We find a quantitative agreement between the behavioral data and the RGC density: The variation in the sampling density of RGCs across the human retina is closely matched to the variation in the extent of spatial integration required for either luminance detection or object recognition. Our empirical data combined with the simulation results of computational models suggest that a fixed number of RGCs subserves spatial integration of visual input, independent of the visual-field location.

Saccadic momentum and attentive control in V4 neurons during visual search

Saccadic momentum refers to the increased probability of making a saccade in a forward direction relative to the previous saccade. During visual search and free viewing conditions saccadic probability falls in a gradient from forward to backward directions. It has been considered to reflect an oculomotor bias for a continuing motor plan. Here we report that a saccadic momentum gradient is observed in nonhuman primate behavior and in the visual responses of cortical area V4 neurons during a conjunction style visual search task. This result suggests that saccadic momentum arises in part from a biased spatial distribution of visual responses to stimuli. The effect is independent of feature-based selective attention and overridden by directed spatial attention. The implications of saccadic momentum for search guidance are much broader and robust than the inhibition-of-return's presumed role in preventing refixation of recent locations.

Can Contrast-Response Functions Indicate Visual Processing Levels?

Many visual effects are believed to be processed at several functional and anatomical levels of cortical processing. Determining if and how the levels contribute differentially to these effects is a leading problem in visual perception and visual neuroscience. We review and analyze a combination of extant psychophysical findings in the context of neurophysiological and brain-imaging results. Specifically using findings relating to visual illusions, crowding, and masking as exemplary cases, we develop a theoretical rationale for showing how relative levels of cortical processing contributing to these effects can already be deduced from the psychophysically determined functions relating respectively the illusory, crowding and masking strengths to the contrast of the illusion inducers, of the flankers producing the crowding, and of the mask. The wider implications of this rationale show how it can help to settle or clarify theoretical and interpretive inconsistencies and how it can further psychophysical, brain-recording and brain-imaging research geared to explore the relative functional and cortical levels at which conscious and unconscious processing of visual information occur. Our approach also allows us to make some specific predictions for future studies, whose results will provide empirical tests of its validity.

Stimulus conflation and tuning selectivity in V4 neurons: a model of visual crowding

Visual crowding is a fundamental constraint on our ability to identify peripheral objects in cluttered environments. This study proposes a descriptive model for understanding crowding based on the tuning selectivity for stimuli within the receptive field (RF) and examines potential neural correlates in cortical area V4. For V4 neurons, optimally sized, letter-like stimuli are much smaller than the RF. This permits stimulus conflation, the fusing of separate objects into a single identity, to occur within the RF of single neurons. Flanking interactions between such stimuli were found to be limited to the RF. The response to an optimal stimulus centered in the neuron's RF, is suppressed by the simultaneous presentation of flanking stimuli within the RF. The degree of suppression is a function of the neuron's stimulus tuning properties and the position of the flanker within the RF. A single neuron may show suppression or facilitation depending on the detailed stimulus conditions and the relationship to tuning selectivity. Loss of activity in the set of neurons tuned to a particular stimulus alters its overall representation and potential identification, thus forming a basis for visual crowding effects. The mechanisms that determine the outcome of conflation are associated with object identification, and are not some other independent visual phenomena.

Smokers' Neurological Responses to Novel and Repeated Health Warning Labels (HWLs) From Cigarette Packages

Graphic health warning labels (HWLs) depicting bodily injury due to smoking are effective for producing changes in affect, cognition and smoking behavior in adult smokers. However, little is known about the effects of repeated presentation of graphic HWL's on the aforementioned processes. The goal of this study was to examine neural and behavioral responses to graphic HWL's and evaluate whether the repeated presentation of graphic HWL's leads to repetition suppression (RS). Smokers (N = 16) performed an event-related HWL cue task while blood oxygen level dependent (BOLD) signal was collected during a functional magnetic resonance imaging (fMRI) experimental session. Consistent with prior literature, graphic HWL's, as compared to scrambled images, elicited increased BOLD response in brain regions involved in self-referential and emotion processing. Importantly, BOLD response at sites in this network diminished during repeated presentation of the same HWL. These findings suggest that while novel graphic HWL's may have a significant effect on smokers' brain activity, repeated presentation may lead to muted responses and thus limit their potential to induce behavioral change.

Compressive Temporal Summation in Human Visual Cortex

Combining sensory inputs over space and time is fundamental to vision. Population receptive field models have been successful in characterizing spatial encoding throughout the human visual pathways. A parallel question, how visual areas in the human brain process information distributed over time, has received less attention. One challenge is that the most widely used neuroimaging method, fMRI, has coarse temporal resolution compared with the time-scale of neural dynamics. Here, via carefully controlled temporally modulated stimuli, we show that information about temporal processing can be readily derived from fMRI signal amplitudes in male and female subjects. We find that all visual areas exhibit subadditive summation, whereby responses to longer stimuli are less than the linear prediction from briefer stimuli. We also find fMRI evidence that the neural response to two stimuli is reduced for brief interstimulus intervals (indicating adaptation). These effects are more pronounced in visual areas anterior to V1-V3. Finally, we develop a general model that shows how these effects can be captured with two simple operations: temporal summation followed by a compressive nonlinearity. This model operates for arbitrary temporal stimulation patterns and provides a simple and interpretable set of computations that can be used to characterize neural response properties across the visual hierarchy. Importantly, compressive temporal summation directly parallels earlier findings of compressive spatial summation in visual cortex describing responses to stimuli distributed across space. This indicates that, for space and time, cortex uses a similar processing strategy to achieve higher-level and increasingly invariant representations of the visual world.SIGNIFICANCE STATEMENT Combining sensory inputs over time is fundamental to seeing. Two important temporal phenomena are summation, the accumulation of sensory inputs over time, and adaptation, a response reduction for repeated or sustained stimuli. We investigated these phenomena in the human visual system using fMRI. We built predictive models that operate on arbitrary temporal patterns of stimulation using two simple computations: temporal summation followed by a compressive nonlinearity. Our new temporal compressive summation model captures (1) subadditive temporal summation, and (2) adaptation. We show that the model accounts for systematic differences in these phenomena across visual areas. Finally, we show that for space and time, the visual system uses a similar strategy to achieve increasingly invariant representations of the visual world.

Rehabilitation Approaches in Macular Degeneration Patients

Age related macular degeneration (AMD) is a visual disease that affects elderly population. It entails a progressive loss of central vision whose consequences are dramatic for the patient's quality of life. Current rehabilitation programs are restricted to technical aids based on visual devices. They only temporarily improve specific visual functions such as reading skills. Considering the rapid increase of the aging population worldwide, it is crucial to intensify clinical research on AMD in order to develop simple and efficient methods that improve the patient's visual performances in many different contexts. One very promising approach to face this challenge is based on perceptual learning (PL). Through intensive practice, PL can induce neural plasticity in sensory cortices and result in long-lasting enhancements for various perceptual tasks in both normal and visually impaired populations. A growing number of studies showed how appropriate PL protocols improve visual functions in visual disorders, namely amblyopia, presbyopia or myopia. In order to successfully apply these approaches to more severe conditions such as AMD, numerous challenges have to be overcome. Indeed, the overall elderly age of patients and the reduced cortical surface that is devoted to peripheral vision potentially limit neural plasticity in this population. In addition, ocular fixation becomes much less stable because patients have to rely on peripheral fixation spots outside the scotoma whose size keeps on evolving. The aim of this review article is to discuss the recent literature on this topic and to offer a unified approach for developing new rehabilitation programs of AMD using PL. We argue that with an appropriate experimental and training protocol that is adapted to each patient needs, PL can offer fascinating opportunities for the development of simple, non-expensive rehabilitation approaches a large spectrum of visual functions in AMD patients.

Visual sensitivity of frontal eye field neurons during the preparation of saccadic eye movements

Primate vision is continuously disrupted by saccadic eye movements, and yet this disruption goes unperceived. One mechanism thought to reduce perception of this self-generated movement is saccadic suppression, a global loss of visual sensitivity just before, during, and after saccadic eye movements. The frontal eye field (FEF) is a candidate source of neural correlates of saccadic suppression previously observed in visual cortex, because it contributes to the generation of visually guided saccades and modulates visual cortical responses. However, whether the FEF exhibits a perisaccadic reduction in visual sensitivity that could be transmitted to visual cortex is unknown. To determine whether the FEF exhibits a signature of saccadic suppression, we recorded the visual responses of FEF neurons to brief, full-field visual probe stimuli presented during fixation and before onset of saccades directed away from the receptive field in rhesus macaques (Macaca mulatta) We measured visual sensitivity during both epochs and found that it declines before saccade onset. Visual sensitivity was significantly reduced in visual but not visuomotor neurons. This reduced sensitivity was also present in visual neurons with no movement-related modulation during visually guided saccades and thus occurred independently from movement-related activity. Across the population of visual neurons, sensitivity began declining ∼80 ms before saccade onset. We also observed a similar presaccadic reduction in sensitivity to isoluminant, chromatic stimuli. Our results demonstrate that the signaling of visual information by FEF neurons is reduced during saccade preparation, and thus these neurons exhibit a signature of saccadic suppression.

Spatio-temporal properties of letter crowding

Crowding between adjacent letters has been investigated primarily as a spatial effect. The purpose of this study was to investigate the spatio-temporal properties of letter crowding. Specifically, we examined the systematic changes in the degradation effects in letter identification performance when adjacent letters were presented with a temporal asynchrony, as a function of letter separation and between the fovea and the periphery. We measured proportion-correct performance for identifying the middle target letter in strings of three lowercase letters at the fovea and 10° in the inferior visual field, for a range of center-to-center letter separations and a range of stimulus onset asynchronies (SOA) between the target and flanking letters (positive SOAs: target preceded flankers). As expected, the accuracy for identifying the target letters reduces with decreases in letter separation. This crowding effect shows a strong dependency on SOAs, such that crowding is maximal between 0 and ∼100 ms (depending on conditions) and diminishes for larger SOAs (positive or negative). Maximal crowding does not require the target and flanking letters to physically coexist for the entire presentation duration. Most importantly, crowding can be minimized even for closely spaced letters if there is a large temporal asynchrony between the target and flankers. The reliance of letter identification performance on SOAs and how it changes with letter separations imply that the crowding effect can be traded between space and time. Our findings are consistent with the notion that crowding should be considered as a spatio-temporal, and not simply a spatial, effect.

Radial-tangential anisotropy of crowding in the early visual areas

Crowding, the inability to recognize an individual object in clutter (Bouma H. Nature 226: 177-178, 1970), is considered a major impediment to object recognition in peripheral vision. Despite its significance, the cortical loci of crowding are not well understood. In particular, the role of the primary visual cortex (V1) remains unclear. Here we utilize a diagnostic feature of crowding to identify the earliest cortical locus of crowding. Controlling for other factors, radially arranged flankers induce more crowding than tangentially arranged ones (Toet A, Levi DM. Vision Res 32: 1349-1357, 1992). We used functional magnetic resonance imaging (fMRI) to measure the change in mean blood oxygenation level-dependent (BOLD) response due to the addition of a middle letter between a pair of radially or tangentially arranged flankers. Consistent with the previous finding that crowding is associated with a reduced BOLD response [Millin R, Arman AC, Chung ST, Tjan BS. Cereb Cortex (July 5, 2013). doi:10.1093/cercor/bht159], we found that the BOLD signal evoked by the middle letter depended on the arrangement of the flankers: less BOLD response was associated with adding the middle letter between radially arranged flankers compared with adding it between tangentially arranged flankers. This anisotropy in BOLD response was present as early as V1 and remained significant in downstream areas. The effect was observed while subjects' attention was diverted away from the testing stimuli. Contrast detection threshold for the middle letter was unaffected by flanker arrangement, ruling out surround suppression of contrast response as a major factor in the observed BOLD anisotropy. Our findings support the view that V1 contributes to crowding.

Prominent effects and neural correlates of visual crowding in a neurodegenerative disease population

Crowding is a breakdown in the ability to identify objects in clutter, and is a major constraint on object recognition. Crowding particularly impairs object perception in peripheral, amblyopic and possibly developing vision. Here we argue that crowding is also a critical factor limiting object perception in central vision of individuals with neurodegeneration of the occipital cortices. In the current study, individuals with posterior cortical atrophy (n=26), typical Alzheimer's disease (n=17) and healthy control subjects (n=14) completed centrally-presented tests of letter identification under six different flanking conditions (unflanked, and with letter, shape, number, same polarity and reverse polarity flankers) with two different target-flanker spacings (condensed, spaced). Patients with posterior cortical atrophy were significantly less accurate and slower to identify targets in the condensed than spaced condition even when the target letters were surrounded by flankers of a different category. Importantly, this spacing effect was observed for same, but not reverse, polarity flankers. The difference in accuracy between spaced and condensed stimuli was significantly associated with lower grey matter volume in the right collateral sulcus, in a region lying between the fusiform and lingual gyri. Detailed error analysis also revealed that similarity between the error response and the averaged target and flanker stimuli (but not individual target or flanker stimuli) was a significant predictor of error rate, more consistent with averaging than substitution accounts of crowding. Our findings suggest that crowding in posterior cortical atrophy can be regarded as a pre-attentive process that uses averaging to regularize the pathologically noisy representation of letter feature position in central vision. These results also help to clarify the cortical localization of feature integration components of crowding. More broadly, we suggest that posterior cortical atrophy provides a neurodegenerative disease model for exploring the basis of crowding. These data have significant implications for patients with, or who will go on to develop, dementia-related visual impairment, in whom acquired excessive crowding likely contributes to deficits in word, object, face and scene perception.

Integrating retinotopic features in spatiotopic coordinates

The receptive fields of early visual neurons are anchored in retinotopic coordinates (Hubel and Wiesel, 1962). Eye movements shift these receptive fields and therefore require that different populations of neurons encode an object's constituent features across saccades. Whether feature groupings are preserved across successive fixations or processing starts anew with each fixation has been hotly debated (Melcher and Morrone, 2003; Melcher, 2005, 2010; Knapen et al., 2009; Cavanagh et al., 2010a,b; Morris et al., 2010). Here we show that feature integration initially occurs within retinotopic coordinates, but is then conserved within a spatiotopic coordinate frame independent of where the features fall on the retinas. With human observers, we first found that the relative timing of visual features plays a critical role in determining the spatial area over which features are grouped. We exploited this temporal dependence of feature integration to show that features co-occurring within 45 ms remain grouped across eye movements. Our results thus challenge purely feedforward models of feature integration (Pelli, 2008; Freeman and Simoncelli, 2011) that begin de novo after every eye movement, and implicate the involvement of brain areas beyond early visual cortex. The strong temporal dependence we quantify and its link with trans-saccadic object perception instead suggest that feature integration depends, at least in part, on feedback from higher brain areas (Mumford, 1992; Rao and Ballard, 1999; Di Lollo et al., 2000; Moore and Armstrong, 2003; Stanford et al., 2010).

Cortical reorganization after long-term adaptation to retinal lesions in humans

Single-unit recordings demonstrated that the adult mammalian visual cortex is capable of reorganizing after induced retinal lesions. In humans, whether the adult cortex is capable of reorganizing has only been studied using functional magnetic resonance imaging, with equivocal results. Here, we exploited the phenomenon of visual crowding, a major limitation on object recognition, to show that, in humans with long-standing retinal (macular) lesions that afflict the fovea and thus use their peripheral vision exclusively, the signature properties of crowding are distinctly different from those of the normal periphery. Crowding refers to the inability to recognize objects when the object spacing is smaller than the critical spacing. Critical spacing depends only on the retinal location of the object, scales linearly with its distance from the fovea, and is approximately two times larger in the radial than the tangential direction with respect to the fovea, thus demonstrating the signature radial-tangential anisotropy of the crowding zone. Using retinal imaging combined with behavioral measurements, we mapped out the crowding zone at the precise peripheral retinal locations adopted by individuals with macular lesions as the new visual reference loci. At these loci, the critical spacings are substantially smaller along the radial direction than expected based on the normal periphery, resulting in a lower scaling of critical spacing with the eccentricity of the peripheral locus and a loss in the signature radial-tangential anisotropy of the crowding zone. These results imply a fundamental difference in the substrate of cortical processing in object recognition following long-term adaptation to macular lesions.

Visual crowding in V1

In peripheral vision, objects in clutter are difficult to identify. The exact cause of this "crowding" effect is unclear. To perceive coherent shapes in clutter, the visual system must integrate certain local features across receptive fields while preventing others from being combined. It is believed that this selective feature integration-segmentation process is impaired in peripheral vision, leading to crowding. We used functional magnetic resonance imaging (fMRI) to investigate the neural origin of crowding. We found that crowding was associated with suppressed fMRI signal as early as V1, regardless of whether attention was directed toward or away from a target stimulus. This suppression in early visual cortex was greatest for stimuli that produced the strongest crowding. In contrast, the pattern of activity was mixed in higher level visual areas, such as the lateral occipital cortex. These results support the view that the deficiency in feature integration and segmentation in peripheral vision is present at the earliest stages of cortical processing.

Crowding during restricted and free viewing

Crowding impairs the perception of form in peripheral vision. It is likely to be a key limiting factor of form vision in patients without central vision. Crowding has been extensively studied in normally sighted individuals, typically with a stimulus duration of a few hundred milliseconds to avoid eye movements. These restricted testing conditions do not reflect the natural behavior of a patient with central field loss. Could unlimited stimulus duration and unrestricted eye movements change the properties of crowding in any fundamental way? We studied letter identification in the peripheral vision of normally sighted observers in three conditions: (i) a fixation condition with a brief stimulus presentation of 250 ms, (ii) another fixation condition but with an unlimited viewing time, and (iii) an unrestricted eye movement condition with an artificial central scotoma and an unlimited viewing time. In all conditions, contrast thresholds were measured as a function of target-to-flanker spacing, from which we estimated the spatial extent of crowding in terms of critical spacing. We found that presentation duration beyond 250 ms had little effect on critical spacing with stable gaze. With unrestricted eye movements and a simulated central scotoma, we found a large variability in critical spacing across observers, but more importantly, the variability in critical spacing was well correlated with the variability in target eccentricity. Our results assure that the large body of findings on crowding made with briefly presented stimuli remains relevant to conditions where viewing time is unconstrained. Our results further suggest that impaired oculomotor control associated with central vision loss can confound peripheral form vision beyond the limits imposed by crowding.

Stimulus repetition affects both strength and synchrony of macaque inferior temporal cortical activity

Repetition of a visual stimulus reduces the firing rate of macaque inferior temporal (IT) neurons. The neural mechanisms underlying this adaptation or repetition suppression are still unclear. In particular, we do not know how the IT circuit is affected by stimulus repetition. To address this, we measured local field potentials (LFPs) and multiunit spiking activity (MUA) simultaneously at 16 sites with a laminar electrode in IT while repeating visual images. Stimulus exposures and interstimulus intervals were each 500 ms. The rhesus monkeys were performing a passive fixation task during the recordings. Induced LFP power decreased with repetition for spectral frequencies above 60 Hz but increased with repetition for lower frequencies, the latter because of a delayed decrease in power when repeating a stimulus. LFP-LFP and MUA-LFP coherences decreased with repetition for frequencies above 60 Hz. This repetition suppression of the MUA-LFP coherence was not due to differences in firing rate since it was present when spike counts were equated for the adapter and repeated stimuli. For frequencies between 15 and 40 Hz, the effect of repetition on synchronization depended on the electrode depth: For the putative superficial layers synchronization was enhanced with repetition, while the LFPs of the putative deep layers decreased their synchrony across layers. The between-site, trial-to-trial covariations in MUA (“noise correlations”) decreased with repetition, but this might have reflected repetition suppression of the firing rate. This work demonstrates that short-term stimulus repetition affects the synchronized activity, in addition to response strength, in IT cortex.

The neural correlates of crowding-induced changes in appearance

Object recognition in the peripheral visual field is limited by crowding: the disruptive influence of nearby clutter. Despite its severity, little is known about the cortical locus of crowding. Here, we examined the neural correlates of crowding by combining event-related fMRI adaptation with a change-detection paradigm. Crowding can change the appearance of objects, such that items become perceptually matched to surrounding objects; we used this change in appearance as a signature of crowding and measured brain activity that correlated with the crowded percept. Observers adapted to a peripheral patch of noise surrounded by four Gabor flankers. When crowded, the noise appears oriented and perceptually indistinguishable from the flankers. Consequently, substitution of the noise for a Gabor identical to the flankers ("change-same") is rarely detected, whereas substitution for an orthogonal Gabor ("change-different") is rarely missed. We predicted that brain areas representing the crowded percept would show repetition suppression in change-same trials but release from adaptation in change-different trials. This predicted pattern was observed throughout cortical visual areas V1-V4, increasing in strength from early to late visual areas. These results depict crowding as a multistage process, involving even the earliest cortical visual areas, with perceptual consequences that are increasingly influenced by later visual areas.

Visual adaptation and novelty responses in the superior colliculus

The brain's ability to ignore repeating, often redundant, information while enhancing novel information processing is paramount to survival. When stimuli are repeatedly presented, the response of visually sensitive neurons decreases in magnitude, that is, neurons adapt or habituate, although the mechanism is not yet known. We monitored the activity of visual neurons in the superior colliculus (SC) of rhesus monkeys who actively fixated while repeated visual events were presented. We dissociated adaptation from habituation as mechanisms of the response decrement by using a Bayesian model of adaptation, and by employing a paradigm including rare trials that included an oddball stimulus that was either brighter or dimmer. If the mechanism is adaptation, response recovery should be seen only for the brighter stimulus; if the mechanism is habituation, response recovery ('dishabituation') should be seen for both the brighter and dimmer stimuli. We observed a reduction in the magnitude of the initial transient response and an increase in response onset latency with stimulus repetition for all visually responsive neurons in the SC. Response decrement was successfully captured by the adaptation model, which also predicted the effects of presentation rate and rare luminance changes. However, in a subset of neurons with sustained activity in response to visual stimuli, a novelty signal akin to dishabituation was observed late in the visual response profile for both brighter and dimmer stimuli, and was not captured by the model. This suggests that SC neurons integrate both rapidly discounted information about repeating stimuli and novelty information about oddball events, to support efficient selection in a cluttered dynamic world.

Crowding is tuned for perceived (not physical) location

In the peripheral visual field, nearby objects can make one another difficult to recognize (crowding) in a manner that critically depends on their separation. We manipulated the apparent separation of objects using the illusory shifts in perceived location that arise from local motion to determine if crowding depends on physical or perceived location. Flickering Gabor targets displayed between either flickering or drifting flankers were used to (a) quantify the perceived target-flanker separation and (b) measure discrimination of the target orientation or spatial frequency as a function of physical target-flanker separation. Relative to performance with flickering targets, we find that flankers drifting away from the target improve discrimination, while those drifting toward the target degrade it. When plotted as a function of perceived separation across conditions, the data collapse onto a single function indicating that it is perceived and not physical location that determines the magnitude of visual crowding. There was no measurable spatial distortion of the target that could explain the effects. This suggests that crowding operates predominantly in extrastriate visual cortex and not in early visual areas where the response of neurons is retinotopically aligned with the physical position of a stimulus.

Visual crowding: a fundamental limit on conscious perception and object recognition

Crowding, the inability to recognize objects in clutter, sets a fundamental limit on conscious visual perception and object recognition throughout most of the visual field. Despite how widespread and essential it is to object recognition, reading and visually guided action, a solid operational definition of what crowding is has only recently become clear. The goal of this review is to provide a broad-based synthesis of the most recent findings in this area, to define what crowding is and is not, and to set the stage for future work that will extend our understanding of crowding well beyond low-level vision. Here we define six diagnostic criteria for what counts as crowding, and further describe factors that both escape and break crowding. All of these lead to the conclusion that crowding occurs at multiple stages in the visual hierarchy.

Dynamics of spatial distortions reveal multiple time scales of motion adaptation

Prolonged exposure to consistent visual motion can significantly alter the perceived direction and speed of subsequently viewed objects. These perceptual aftereffects have provided invaluable tools with which to study the mechanisms of motion adaptation and draw inferences about the properties of underlying neural populations. Behavioral studies of the time course of motion aftereffects typically reveal a gradual process of adaptation spanning a period of multiple seconds. In contrast, neurophysiological studies have documented multiple motion adaptation effects operating over similar, or substantially faster (i.e., sub-second) time scales. Here we investigated motion adaptation by measuring time-dependent changes in the ability of moving stimuli to distort the perceived position of briefly presented static objects. The temporal dynamics of these motion-induced spatial distortions reveal the operation of two dissociable mechanisms of motion adaptation with differing properties. The first is rapid (subsecond), acts to limit the distortions induced by continuing motion, but is not sufficient to produce an aftereffect once the motion signal disappears. The second gradually accumulates over a period of seconds, does not modulate the size of distortions produced by continuing motion, and produces repulsive aftereffects after motion offset. These results provide new psychophysical evidence for the operation of multiple mechanisms of motion adaptation operating over distinct time scales.

Attentional modulation of adaptation in V4

Adaptation and visual attention are two processes that alter neural responses to luminance contrast. Rapid contrast adaptation changes response size and dynamics at all stages of visual processing, while visual attention has been shown to modulate both contrast gain and response gain in macaque extrastriate visual cortex. Because attention aims to enhance behaviorally relevant sensory responses while adaptation acts to attenuate neural activity, the question we asked is, how does attention alter adaptation? We present here single-unit recordings from V4 of two rhesus macaques performing a cued target detection task. The study was designed to characterize the effects of attention on the size and dynamics of a sequence of responses produced by a series of flashed oriented gratings parametric in luminance contrast. We found that the effect of attention on the response dynamics of V4 neurons is inconsistent with a mechanism that only alters the effective stimulus contrast, or only rescales the gain of the response. Instead, the action of attention modifies contrast gain early in the task, and modifies both response gain and contrast gain later in the task. We also show that responses to attended stimuli are more closely locked to the stimulus cycle than unattended responses, and that attended responses show less of the phase lag produced by adaptation than unattended responses. The phase advance generated by attention of the adapted responses suggests that the attentional gain control operates in some ways like a contrast gain control utilizing a neural measure of contrast to influence dynamics.

Central V4 receptive fields are scaled by the V1 cortical magnification and correspond to a constant-sized sampling of the V1 surface

The mapping of the topographic representation of the visual field onto cortical areas changes throughout the hierarchy of cortical visual areas. The changes are believed to reflect the establishment of modules with different spatial processing emphasis. The receptive fields (RFs) of neurons within these modules, however, may not be governed by the same spatial topographic map parameters. Here it is shown that the RFs of area V4 neurons (centered 1-12 degrees in eccentricity) are based on a circularly symmetric sampling of the primary visual cortical retinotopic map. No eccentricity dependent magnification beyond that observed in V1 is apparent in the V4 neurons. The size and shape of V4 RFs can be explained by a simple, constant sized, two-dimensional Gaussian sample of visual input from the retinotopic map laid out across the surface of V1. Inferences about the spatial scale of interactions within the receptive fields of neurons cannot be based on a visual area's apparent cortical magnification derived from topographic mapping.

On the importance of the transient visual response in the superior colliculus

A salient event in the environment can initiate a complex orienting response that includes a shift in gaze. The midbrain superior colliculus (SC) contains the appropriate circuitry to generate and distribute a signal of the priority of this event, and co-ordinate the orienting response. The magnitude and timing of the short-latency transient visual response in the SC, when combined with cortical inputs signaling stimulus relevance and expectation, influences the type and latency of the orienting response. This signal in the SC is distributed to higher cortical areas to influence visual processing, to the reinforcement learning system to influence future actions, and to premotor circuits, including neck and shoulder muscles, to influence immediate action.

Neuronal adaptation caused by sequential visual stimulation in the frontal eye field

Images on the retina can change drastically in only a few milliseconds. A robust description of visual temporal processing is therefore necessary to understand visual analysis in the real world. To this end, we studied subsecond visual changes and asked how prefrontal neurons in monkeys respond to stimuli presented in quick succession. We recorded the visual responses of single neurons in the frontal eye field (FEF), a prefrontal area polysynaptically removed from the retina that is involved with higher level cognition. For comparison, we also recorded from small groups of neurons in the superficial superior colliculus (supSC), an area that receives direct retinal input. Two sequential flashes of light at varying interstimulus intervals were presented in a neuron's receptive field. We found pervasive neuronal adaptation in FEF and supSC. Visual responses to the second stimulus were diminished for up to half a second after the first stimulus presentation. Adaptation required a similar amount of time to return to full responsiveness in both structures, but there was significantly more neuronal adaptation overall in FEF. Adaptation was not affected by saccades, although visual responses to single stimuli were transiently suppressed postsaccadically. Our FEF and supSC results systematically document subsecond visual adaptation in prefrontal cortex and show that this adaptation is comparable to, but stronger than, adaptation found earlier in the visual system.

Specialized color modules in macaque extrastriate cortex

Imaging studies are consistent with the existence of brain regions specialized for color, but electrophysiological studies have produced conflicting results. Here we address the neural basis for color, using targeted single-unit recording in alert macaque monkeys, guided by functional magnetic resonance imaging (fMRI) of the same subjects. Distributed within posterior inferior temporal cortex, a large region encompassing V4, PITd, and posterior TEO that some have proposed functions as a single visual complex, we found color-biased fMRI hotspots that we call "globs," each several millimeters wide. Almost all cells located in globs showed strong luminance-invariant color tuning and some shape selectivity. Cells in different globs represented distinct visual field locations, consistent with the coarse retinotopy of this brain region. Cells in "interglob" regions were not color tuned, but were more strongly shape selective. Neither population was direction selective. These results suggest that color perception is mediated by specialized neurons that are clustered within the extrastriate brain.