The time course of segmentation and cue-selectivity in the human visual cortex

Measuring uncertainty in human visual segmentation

Segmenting visual stimuli into distinct groups of features and visual objects is central to visual function. Classical psychophysical methods have helped uncover many rules of human perceptual segmentation, and recent progress in machine learning has produced successful algorithms. Yet, the computational logic of human segmentation remains unclear, partially because we lack well-controlled paradigms to measure perceptual segmentation maps and compare models quantitatively. Here we propose a new, integrated approach: given an image, we measure multiple pixel-based same-different judgments and perform model-based reconstruction of the underlying segmentation map. The reconstruction is robust to several experimental manipulations and captures the variability of individual participants. We demonstrate the validity of the approach on human segmentation of natural images and composite textures. We show that image uncertainty affects measured human variability, and it influences how participants weigh different visual features. Because any putative segmentation algorithm can be inserted to perform the reconstruction, our paradigm affords quantitative tests of theories of perception as well as new benchmarks for segmentation algorithms.

Measuring uncertainty in human visual segmentation

Segmenting visual stimuli into distinct groups of features and visual objects is central to visual function. Classical psychophysical methods have helped uncover many rules of human perceptual segmentation, and recent progress in machine learning has produced successful algorithms. Yet, the computational logic of human segmentation remains unclear, partially because we lack well-controlled paradigms to measure perceptual segmentation maps and compare models quantitatively. Here we propose a new, integrated approach: given an image, we measure multiple pixel-based same-different judgments and perform model-based reconstruction of the underlying segmentation map. The reconstruction is robust to several experimental manipulations and captures the variability of individual participants. We demonstrate the validity of the approach on human segmentation of natural images and composite textures. We show that image uncertainty affects measured human variability, and it influences how participants weigh different visual features. Because any putative segmentation algorithm can be inserted to perform the reconstruction, our paradigm affords quantitative tests of theories of perception as well as new benchmarks for segmentation algorithms.

Neural sources of letter and Vernier acuity

Visual acuity can be measured in many different ways, including with letters and Vernier offsets. Prior psychophysical work has suggested that the two acuities are strongly linked given that they both depend strongly on retinal eccentricity and both are similarly affected in amblyopia. Here we used high-density EEG recordings to ask whether the underlying neural sources are common as suggested by the psychophysics or distinct. To measure visual acuity for letters, we recorded evoked potentials to 3 Hz alternations between intact and scrambled text comprised of letters of varying size. To measure visual acuity for Vernier offsets, we recorded evoked potentials to 3 Hz alternations between bar gratings with and without a set of Vernier offsets. Both alternation types elicited robust activity at the 3 Hz stimulus frequency that scaled in amplitude with both letter and offset size, starting near threshold. Letter and Vernier offset responses differed in both their scalp topography and temporal dynamics. The earliest evoked responses to letters occurred on lateral occipital visual areas, predominantly over the left hemisphere. Later responses were measured at electrodes over early visual cortex, suggesting that letter structure is first extracted in second-tier extra-striate areas and that responses over early visual areas are due to feedback. Responses to Vernier offsets, by contrast, occurred first at medial occipital electrodes, with responses at later time-points being more broadly distributed—consistent with feedforward pathway mediation. The previously observed commonalities between letter and Vernier acuity may be due to common bottlenecks in early visual cortex but not because the two tasks are subserved by a common network of visual areas.

Steady state visual evoked potentials in reading aloud: Effects of lexicality, frequency and orthographic familiarity

The present study explored the possibility to use Steady-State Visual Evoked Potentials (SSVEPs) as a tool to investigate the core mechanisms in visual word recognition. In particular, we investigated three benchmark effects of reading aloud: lexicality (words vs. pseudowords), frequency (high-frequency vs. low-frequency words), and orthographic familiarity ('familiar' versus 'unfamiliar' pseudowords). We found that words and pseudowords elicited robust SSVEPs. Words showed larger SSVEPs than pseudowords and high-frequency words showed larger SSVEPs than low-frequency words. SSVEPs were not sensitive to orthographic familiarity. We further localized the neural generators of the SSVEP effects. The lexicality effect was located in areas associated with early level of visual processing, i.e. in the right occipital lobe and in the right precuneus. Pseudowords produced more activation than words in left sensorimotor areas, rolandic operculum, insula, supramarginal gyrus and in the right temporal gyrus. These areas are devoted to speech processing and/or spelling-to-sound conversion. The frequency effect involved the left temporal pole and orbitofrontal cortex, areas previously implicated in semantic processing and stimulus-response associations respectively, and the right postcentral and parietal inferior gyri, possibly indicating the involvement of the right attentional network.

Dynamics of perceptual decisions about symmetry in visual cortex

Neuroimaging studies have identified multiple extra-striate visual areas that are sensitive to symmetry in planar images (Kohler et al., 2016; Sasaki et al., 2005). Here, we investigated which of these areas are directly involved in perceptual decisions about symmetry, by recording high-density EEG in participants (n = 25) who made rapid judgments about whether an exemplar image contained rotation symmetry or not. Stimulus-locked sensor-level analysis revealed symmetry-specific activity that increased with increasing order of rotation symmetry. Response-locked analysis identified activity occurring between 600 and 200 ms before the button-press, that was directly related to perceptual decision making. We then used fMRI-informed EEG source imaging to characterize the dynamics of symmetry-specific activity within an extended network of areas in visual cortex. The most consistent cortical source of the stimulus-locked activity was VO1, a topographically organized area in ventral visual cortex, that was highly sensitive to symmetry in a previous study (Kohler et al., 2016). Importantly, VO1 activity also contained a strong decision-related component, suggesting that this area plays a crucial role in perceptual decisions about symmetry. Other candidate areas, such as lateral occipital cortex, had weak stimulus-locked symmetry responses and no evidence of correlation with response timing.

Temporal Asymmetry in Dark-Bright Processing Initiates Propagating Activity across Primary Visual Cortex

Differences between visual pathways representing darks and lights have been shown to affect spatial resolution and detection timing. Both psychophysical and physiological studies suggest an underlying retinal origin with amplification in primary visual cortex (V1). Here we show that temporal asymmetries in the processing of darks and lights create motion in terms of propagating activity across V1. Exploiting the high spatiotemporal resolution of voltage-sensitive dye imaging, we captured population responses to abrupt local changes of luminance in cat V1. For stimulation we used two neighboring small squares presented on either bright or dark backgrounds. When a single square changed from dark to bright or vice versa, we found coherent population activity emerging at the respective retinal input locations. However, faster rising and decay times were obtained for the bright to dark than the dark to bright changes. When the two squares changed luminance simultaneously in opposite polarities, we detected a propagating wave front of activity that originated at the cortical location representing the darkened square and rapidly expanded toward the region representing the brightened location. Thus, simultaneous input led to sequential activation across cortical retinotopy. Importantly, this effect was independent of the squares' contrast with the background. We suggest imbalance in dark-bright processing as a driving force in the generation of wave-like activity. Such propagation may convey motion signals and influence perception of shape whenever abrupt shifts in visual objects or gaze cause counterchange of luminance at high-contrast borders.

SIGNIFICANCE STATEMENT

An elementary process in vision is the detection of darks and lights through the retina via ON and OFF channels. Psychophysical and physiological studies suggest that differences between these channels affect spatial resolution and detection thresholds. Here we show that temporal asymmetries in the processing of darks and lights create motion signals across visual cortex. Using two neighboring squares, which simultaneously counterchanged luminance, we discovered propagating activity that was strictly drawn out from cortical regions representing the darkened location. Thus, a synchronous stimulus event translated into sequential wave-like brain activation. Such propagation may convey motion signals accessible in higher brain areas, whenever abrupt shifts in visual objects or gaze cause counterchange of luminance at high-contrast borders.

Response priming evidence for feedforward processing of snake contours but not of ladder contours and textures

In contour integration, increased difficulty in detection and shape discrimination of a chain of parallel elements (a ladder contour) compared to collinear elements (a snake contour) suggests more extensive processing of ladders than of snakes. In addition, conceptual similarities between ladders and textures - which also involve grouping of parallel elements - raises the question whether ladder and texture processing requires feedback from higher visual areas while snakes are processed in a fast feedforward sweep. We tested this in a response priming paradigm, where participants responded as quickly and accurately as possible to the orientation of a diagonal contour in a Gabor array (target). The diagonal was defined either by a snake, ladder, texture, or a continuous line. The target was preceded with varying stimulus onset asynchrony (SOA) by a prime that was either a snake, ladder, or texture, and was consistent or inconsistent to the response demands of the target. Resulting priming effects clearly distinguished between processing of snakes, ladders, and textures. Effects generally increased with SOA but were stronger for snakes and textures compared to ladders. Importantly, only priming effects for snakes were fully present already in the fastest response times, in accordance with a simple feedforward processing model. We conclude that snakes, ladders, and textures do not share similar processing characteristics, with snakes exhibiting a pronounced processing advantage.

The steady-state visual evoked potential in vision research: A review

Periodic visual stimulation and analysis of the resulting steady-state visual evoked potentials were first introduced over 80 years ago as a means to study visual sensation and perception. From the first single-channel recording of responses to modulated light to the present use of sophisticated digital displays composed of complex visual stimuli and high-density recording arrays, steady-state methods have been applied in a broad range of scientific and applied settings.The purpose of this article is to describe the fundamental stimulation paradigms for steady-state visual evoked potentials and to illustrate these principles through research findings across a range of applications in vision science.

Linking brain to behavior for the visual perception of figures and objects

The dissociation of a figure from its background is an essential feat of visual perception, as it allows us to detect, recognize, and interact with shapes and objects in our environment. In order to understand how the human brain gives rise to the perception of figures, we here review experiments that explore the links between activity in visual cortex and performance of perceptual tasks related to figure perception. We organize our review according to a proposed model that attempts to contextualize figure processing within the more general framework of object processing in the brain. Overall, the current literature provides us with individual linking hypotheses as to cortical regions that are necessary for particular tasks related to figure perception. Attempts to reach a more complete understanding of how the brain instantiates figure and object perception, however, will have to consider the temporal interaction between the many regions involved, the details of which may vary widely across different tasks.

Impaired texture segregation but spared contour integration following damage to right posterior parietal cortex

We examined the relations between texture segregation and contour integration in patients with deficits in spatial attention leading to left or right hemisphere extinction. Patients and control participants were presented with texture and contour stimuli consisting of oriented elements. We induced regularity in the stimuli by manipulating the element orientations resulting in an implicit texture border or explicit contour. Participants had to discriminate curved from straight shapes without making eye movements, while the stimulus presentation time was varied using a QUEST procedure. The results showed that only patients with right hemisphere extinction had a spatial bias, needing a longer presentation time to determine the shape of the border or contour on the contralesional side, especially for borders defined by texture. These results indicate that texture segregation is modulated by attention-related brain areas in the right posterior parietal cortex.

The time course of shape discrimination in the human brain

The lateral occipital cortex (LOC) activates selectively to images of intact objects versus scrambled controls, is selective for the figure-ground relationship of a scene, and exhibits at least some degree of invariance for size and position. Because of these attributes, it is considered to be a crucial part of the object recognition pathway. Here we show that human LOC is critically involved in perceptual decisions about object shape. High-density EEG was recorded while subjects performed a threshold-level shape discrimination task on texture-defined figures segmented by either phase or orientation cues. The appearance or disappearance of a figure region from a uniform background generated robust visual evoked potentials throughout retinotopic cortex as determined by inverse modeling of the scalp voltage distribution. Contrasting responses from trials containing shape changes that were correctly detected (hits) with trials in which no change occurred (correct rejects) revealed stimulus-locked, target-selective activity in the occipital visual areas LOC and V4 preceding the subject's response. Activity that was locked to the subjects' reaction time was present in the LOC. Response-locked activity in the LOC was determined to be related to shape discrimination for several reasons: shape-selective responses were silenced when subjects viewed identical stimuli but their attention was directed away from the shapes to a demanding letter discrimination task; shape-selectivity was present across four different stimulus configurations used to define the figure; LOC responses correlated with participants' reaction times. These results indicate that decision-related activity is present in the LOC when subjects are engaged in threshold-level shape discriminations.