Role of onset asynchrony in contour integration

Saccades Influence the Visibility of Targets in Rapid Stimulus Sequences: The Roles of Mislocalization, Retinal Distance and Remapping

Briefly presented targets around the time of a saccade are mislocalized towards the saccadic landing point. This has been taken as evidence for a remapping mechanism that accompanies each eye movement, helping maintain visual stability across large retinal shifts. Previous studies have shown that spatial mislocalization is greatly diminished when trains of brief stimuli are presented at a high frequency rate, which might help to explain why mislocalization is rarely perceived in everyday viewing. Studies in the laboratory have shown that mislocalization can reduce metacontrast masking by causing target stimuli in a masking sequence to be perceived as shifted in space towards the saccadic target and thus more easily discriminated. We investigated the influence of saccades on target discrimination when target and masks were presented in a rapid serial visual presentation (RSVP), as well as with forward masking and with backward masking. In a series of experiments, we found that performance was influenced by the retinal displacement caused by the saccade itself but that an additional component of un-masking occurred even when the retinal location of target and mask was matched. These results speak in favor of a remapping mechanism that begins before the eyes start moving and continues well beyond saccadic termination.

Visual Binding of English and Chinese Word Parts is Limited to Low Temporal Frequencies

Some perceptual mechanisms manifest high temporal precision, allowing reports of visual information even when that information is restricted to windows smaller than 50 ms. Other visual judgments are limited to much coarser time scales. What about visual information extracted at late processing stages, for which we nonetheless have perceptual expertise, such as words? Here, the temporal limits on binding together visual word parts were investigated. In one trial, either the word ‘ball’ was alternated with ‘deck’, or ‘dell’ was alternated with ‘back’, with all stimuli presented at fixation. These stimuli restrict the time scale of the rod identities because the two sets of alternating words form the same image at high alternation frequencies. Observers made a forced choice between the two alternatives. Resulting 75% thresholds are restricted to 5 Hz or less for words and nonword letter strings. A similar result was obtained in an analogous experiment with Chinese participants viewing alternating Chinese characters. These results support the theory that explicit perceptual access to visual information extracted at late stages is limited to coarse time scales.

Inter-element orientation and distance influence the duration of persistent contour integration

Contour integration is a fundamental form of perceptual organization. We introduce a new method of studying the mechanisms responsible for contour integration. This method capitalizes on the perceptual persistence of contours under conditions of impending camouflage. Observers viewed arrays of randomly arranged line segments upon which circular contours comprised of similar line segments were superimposed via abrupt onset. Crucially, these contours remained visible for up to a few seconds following onset, but eventually disappeared due to the camouflaging effects of surrounding background line segments. Our main finding was that the duration of contour visibility depended on the distance and degree of co-alignment between adjacent contour segments such that relatively dense smooth contours persisted longest. The stimulus-related effects reported here parallel similar results from contour detection studies, and complement previous reported top–down influences on contour persistence (Strother et al., 2011). We propose that persistent contour visibility reflects the sustained activity of recurrent processing loops within and between visual cortical areas involved in contour integration and other important stages of visual object recognition.

Speeding up the brain: when spatial facilitation translates into latency shortening

Waves of activity following a focal stimulation are reliably observed to spread across the cortical tissue. The origin of these waves remains unclear and the underlying mechanisms and function are still debated. In this study, we ask whether waves of activity modulate the magnetoencephalography (MEG) signals recorded in humans during visual stimulation with Gabor patches sequentially flashed along a vertical path, eliciting a perception of vertical apparent motion. Building upon the functional properties of long-rang horizontal connections, proposed to contribute to spreading activity, we specifically probe the amplitude and latency of MEG responses as a function of Gabor contrast and orientation. The results indicate that in the left hemisphere the response amplitude is enhanced and the half height response latency is shortened for co-aligned Gabor as compared to misaligned Gabor patches at a low but not at a high contrast. Building upon these findings, we develop a biologically plausible computational model that performs a “spike time alignment” of the responses to elongated contours with varying contrast, endowing them with a phase advance relative to misaligned contours.

Distinct perceptual grouping pathways revealed by temporal carriers and envelopes

S. E. Guttman, L. A. Gilroy, and R. Blake (2005) investigated whether observers could perform temporal grouping in multi-element displays where each local element was stochastically modulated over time along one of several potential dimensions--or "messenger types"--such as contrast, position, orientation, or spatial scale. Guttman et al.'s data revealed that grouping discards messenger type and therefore support a single-pathway model that groups elements with similar temporal waveforms. In the current study, we carried out three experiments in which temporal-grouping information resided either in the carrier, the envelope, or the combined carrier and envelope of each messenger's timecourse. Results revealed that grouping is highly specific for messenger type if carrier envelopes lack grouping information but largely messenger nonspecific if carrier envelopes contain grouping information. These imply that temporal grouping is mediated by several messenger-specific carrier pathways as well as by a messenger-nonspecific envelope pathways. Findings also challenge simple temporal-filtering accounts of perceptual grouping (E. H. Adelson & H. Farid, 1999).

Mobile computation: spatiotemporal integration of the properties of objects in motion

We demonstrate that, as an object moves, color and motion signals from successive, widely spaced locations are integrated, but letter and digit shapes are not. The features that integrate as an object moves match those that integrate when the eyes move but the object is stationary (spatiotopic integration). We suggest that this integration is mediated by large receptive fields gated by attention and that it occurs for surface features (motion and color) that can be summed without precise alignment but not shape features (letters or digits) that require such alignment. Rapidly alternating pairs of colors and motions were presented at several locations around a circle centered at fixation. The same two stimuli alternated at each location with the phase of the alternation reversing from one location to the next. When observers attended to only one location, the stimuli alternated in both retinal coordinates and in the attended stream: feature identification was poor. When the observer's attention shifted around the circle in synchrony with the alternation, the stimuli still alternated at each location in retinal coordinates, but now attention always selected the same color and motion, with the stimulus appearing as a single unchanging object stepping across the locations. The maximum presentation rate at which the color and motion could be reported was twice that for stationary attention, suggesting (as control experiments confirmed) object-based integration of these features. In contrast, the identification of a letter or digit alternating with a mask showed no advantage for moving attention despite the fact that moving attention accessed (within the limits of precision for attentional selection) only the target and never the mask. The masking apparently leaves partial information that cannot be integrated across locations, and we speculate that for spatially defined patterns like letters, integration across large shifts in location may be limited by problems in aligning successive samples. Our results also suggest that as attention moves, the selection of any given location (dwell time) can be as short as 50 ms, far shorter than the typical dwell time for stationary attention. Moving attention can therefore sample a brief instant of a rapidly changing stream if it passes quickly through, giving access to events that are otherwise not seen.

Independent, synchronous access to color and motion features

We investigated the role of attention in pairing superimposed visual features. When moving dots alternate in color and in motion direction, reports of the perceived color and motion reveal an asynchrony: the most accurate reports occur when the motion change precedes the associated color change by approximately 100ms [Moutoussis, K., & Zeki, S. (1997). A direct demonstration of perceptual asynchrony in vision. Proceedings of the Royal Society of London B, 264, 393-399]. This feature binding asynchrony was probed by manipulating endogenous and exogenous attention. First, endogenous attention was manipulated by changing which feature dimension observers were instructed to attend to first. This yielded little effect on the asynchrony. Second, exogenous attention was manipulated by briefly presenting a ring around the target, cueing the report of the color and motion seen within the ring. This reduced or eliminated the apparent latency difference between color and motion. Accuracy was best predicted by timing of each feature relative to the cue rather than the timing of the two features relative to each other, suggesting independent attentional access to the two features with an exogenous attention cue. The timing of attentional cueing affected feature pairing reports as much as the timing of the features themselves.

Threshold recognition of phantom-contour objects requires constant contrast velocity

Recognition of phantom objects--those with contours defined by rapid contrast reversal of adjacent fields of dark and light random dots--was investigated under conditions of abrupt or ramped onset and offset. Discrimination contrast thresholds were determined for a random-dot phantom letter in four possible orientations. For abrupt onset or offset, thresholds were almost independent of the duration of presentation time, over a range that varied tenfold, from 34-340 msec. However, when the onset and offset were shaped by a triangular envelope, thresholds were raised, so that form blindness occurred even when peak d ot contrasts exceeded 60%. Also under ramped onset and offset conditions, threshold contrast varied strictly linearly with stimulus duration in all subjects, suggesting a new construct--contrast velocity, the rate of change of contrast critical for phantom-object recognition.

Spatial versus temporal grouping in a modified Ternus display

The Ternus display can induce a percept of 'element motion' or 'group motion'. Conventionally, this has been attributed to two different motion processes, with different spatial and temporal ranges. In contrast, recent studies have emphasised spatial and temporal grouping principles as underlying the apparent motion percepts in the Ternus display. The present study explored effects of spatial and temporal grouping on the apparent motion percept in a novel Ternus display of oriented Gabor elements with no inter-frame interval. Each frame of this stimulus could be further divided into 'sub-frames', and the orientation of the carriers was changed across these sub-frames. In four experiments transitions were found between the motion percepts with changes in orientation across time (Experiment 1) and space (Experiment 2), and with a temporal offset in the orientation change of the outer element (Experiment 3) to the extent that group motion was not perceived even with large orientation changes over time that previously led to group motion (Experiment 4). Collectively, these results indicate that while spatial properties have an influence in determining the percept of the Ternus display, temporal properties also have a strong influence, and can override spatial grouping. However, these temporal effects cannot be attributed to spatio-temporal limits of low-level motion processes. Some aspects of the observed spatial grouping effects can be accounted for in terms of a modified association field, which may occur through connectivity of orientation selective units in V1. The temporal effects observed are considered in terms of temporal integration, the transitional value at a temporal offset of 40ms being remarkably similar to psychophysical and neurophysiological estimates of the peak temporal impulse response. These temporal responses could be detected at a higher level in the system, providing a basis for apparent motion perception.

Spatial grouping in human vision: temporal structure trumps temporal synchrony

Temporal information promotes visual grouping of local image features into global spatial form. However, experiments demonstrating time-based grouping typically confound two potential sources of information: temporal synchrony (precise timing of changes) and temporal structure (pattern of changes over time). Here, we show that observers prefer temporal structure for determining perceptual organization. That is, human vision groups elements that change according to the same global pattern, even if the changes themselves are not synchronous. This finding prompts an important, testable prediction concerning the neural mechanisms of binding: patterns of neural spiking over time may be more important than absolute spike synchrony.

Perceptual grouping in Gabor lattices: Proximity and alignment

We propose the Gabor lattice as a new stimulus designed to deal with multiple organizations in perceptual grouping, allowing both comparison between psychophysical data and neural findings and a systematic investigation of grouping based on several low-level characteristics and their interactions. A Gabor lattice is a geometric lattice with Gabor patches, evoking a multistable global orientation percept. Visual grouping in Gabor lattices with elements aligned in a global orientation was compared with grouping of nonaligned Gabor patches and of Gaussian blobs. The effect sizes of proximity and alignment were estimated in logistic regression analyses. The results confirmed the importance of proximity and local element alignment as factors in dynamic grouping. We also found a small but consistent enhancement of grouping along the global vector orthogonal to the local patch orientations. In light of these results, we further motivate the relevance of these stimuli and the associated experimental paradigm.

A spatial explanation for synchrony biases in perceptual grouping: consequences for the temporal-binding hypothesis

If two images are shown in rapid sequential order, they are perceived as a single, fused image. Despite this, recent studies have revealed that fundamental perceptual processes are influenced by extremely brief temporal offsets in stimulus presentation. Some researchers have suggested that this is due to the action of a cortical temporal-binding mechanism, which would serve to keep multiple mental representations of one object distinct from those of other objects. There is now gathering evidence that these studies should be reassessed. This article describes evidence for sensitivity to fixational eye and head movements, which provides a purely spatial explanation for the earlier results. Taken in conjunction with other studies, the work serves to undermine the current body of behavioral evidence for a temporal-binding mechanism.

The Perceptual Integration of Auditory Stimulus Edges: An Illusory Short Tone in Stimulus Patterns Consisting of Two Partly Overlapping Glides

Two partly overlapping frequency glides can be perceived as consisting of a long pitch trajectory accompanied by a short tone in the temporal middle. It was found that the appearance of this middle tone could not be related to peripheral processes concerned with spectral splatter or combination tones that could have emerged during the overlap of the glides. Furthermore, it was found that the middle tone was perceived even when components of the 2 glides were separated by more than an equivalent rectangular bandwidth at any time during the overlap. The appearance of the middle tone indicates that auditory events can result from the perceptual integration of component parts-that is, stimulus edges-of acoustically different sounds.

Pop-out from abrupt visual onsets

We report a novel psychophysical paradigm that distinguishes the information present in abrupt stimulus onset from that in the following display. The task is to pick the one odd item from a set added to a pre-existing background of similar items. When all new items are added simultaneously, observers are impaired even at distinguishing one red item amongst several green ones. An asynchrony of about 40 ms between target and distracter items restores performance, with evidence that it is cortical, rather than stimulus timing difference that is significant. The results are consistent with a role for neural synchrony in dynamic grouping.

Rapid image-segmentation and perceptual transparency share a process which utilises X-junctions generated by temporal integration in the visual system

Perceptual transparency requires local same-polarity X-junctions, which can also be generated by temporal integration under natural dynamic conditions. In this study, segmentation performance and target appearance were measured for a uniform gray target embedded in a random-dot frame presented with a temporally adjacent mask. Although static cues for both segmentation and transparency were unavailable, transparency was observed only when collinear same-polarity edges reduced backward masking, in both the fovea and the perifovea. These results suggest that the visual system has a common underlying mechanism for rapid segmentation and transparency, which utilises same-polarity X-junctions generated by temporal integration.

Shape Saliency Modulates Contextual Processing in the Human Lateral Occipital Complex

Visual context influences our perception of target objects in natural scenes. However, little is known about the analysis of context information and its role in shape perception in the human brain. We investigated whether the human lateral occipital complex (LOC), known to be involved in the visual analysis of shapes, also processes information about the context of shapes within cluttered scenes. We employed an fMRI adaptation paradigm in which fMRI responses are lower for two identical than for two different stimuli presented consecutively. The stimuli consisted of closed target contours defined by aligned Gabor elements embedded in a background of randomly oriented Gabors. We measured fMRI adaptation in the LOC across changes in the context of the target shapes by manipulating the position and orientation of the background elements. No adaptation was observed across context changes when the background elements were presented in the same plane as the target elements. However, adaptation was observed when the grouping of the target elements was enhanced in a bottom-up (i.e., grouping by disparity or motion) or top-down (i.e., shape priming) manner and thus the saliency of the target shape increased. These findings suggest that the LOC processes information not only about shapes, but also about their context. This processing of context information in the LOC is modulated by figure–ground segmentation and grouping processes. That is, neural populations in the LOC encode context information when relevant to the perception of target shapes, but represent salient targets independent of context changes.

Contour integration and cortical processing

Our understanding of visual processing in general, and contour integration in particular, has undergone great change over the last 10 years. There is now an accumulation of psychophysical and neurophysiological evidence that the outputs of cells with conjoint orientation preference and spatial position are integrated in the process of explication of rudimentary contours. Recent neuroanatomical and neurophysiological results suggest that this process takes place at the cortical level V1. The code for contour integration may be a temporal one in that it may only manifest itself in the latter part of the spike train as a result of feedback and lateral interactions. Here we review some of the properties of contour integration from a psychophysical perspective and we speculate on their underlying neurophysiological substrate.

Grouping local directional signals into moving contours

We consider how local motion signals are combined to represent the movements of spatially extensive objects. A series of band-pass target dots, whose collective motion defined a moving contour, was positioned within a field of randomly moving noise dots. The visibility of the contours did not depend on the direction of movement relative to local contour orientation unless the contour was constrained to pass through fixation, suggesting that a previously reported advantage for collinear motion trajectories depends on the probability of detecting any of the target elements rather than the integrated contour. Contour visibility was invariant of the spatial frequency of the elements, but it did depend on the speed, number and spacing of elements defining it, as well as the angle and spatial frequency difference between adjacent elements. Local averaging of directional signals is not sufficient to explain these results. The visibility of these moving contours identifies narrow-band grouping processes that are sensitive to the shape defined by the directions of the elements forming the contour.

Overlapping features can be parsed on the basis of rapid temporal cues that produce stable emergent percepts

When two overlapping displays alternate rapidly, it is difficult to resolve the temporal coincidence of objects, parts, or features. However, under certain conditions (at least for luminance-based stimuli) rapid temporal coincidence can be detected on the basis of stable emergent percepts in which parts that oscillate in phase appear more strongly grouped than (or appear distinct from) parts that oscillate out of phase. These emergent percepts appear as depth segregation, enhanced slow orientation rivalry, and oriented shimmer (a new phenomenon that cannot be explained in terms of conventional apparent motion or temporal contrast illusions). These percepts resulted in up to an eightfold decrease in the coincidence detection threshold (alternations as fast as 20 ms/frame or 25 Hz) relative to control conditions that did not yield them; these sensitivity enhancements are unlikely to be due to temporal probability summation. The results provide psychophysical evidence that temporal-phase information can contribute to the parsing of overlapping patterns.

Surfing a spike wave down the ventral stream

Numerous theories of neural processing, often motivated by experimental observations, have explored the computational properties of neural codes based on the absolute or relative timing of spikes in spike trains. Spiking neuron models and theories however, as well as their experimental counterparts, have generally been limited to the simulation or observation of isolated neurons, isolated spike trains, or reduced neural populations. Such theories would therefore seem inappropriate to capture the properties of a neural code relying on temporal spike patterns distributed across large neuronal populations. Here we report a range of computer simulations and theoretical considerations that were designed to explore the possibilities of one such code and its relevance for visual processing. In a unified framework where the relation between stimulus saliency and spike relative timing plays the central role, we describe how the ventral stream of the visual system could process natural input scenes and extract meaningful information, both rapidly and reliably. The first wave of spikes generated in the retina in response to a visual stimulation carries information explicitly in its spatio-temporal structure: the most salient information is represented by the first spikes over the population. This spike wave, propagating through a hierarchy of visual areas, is regenerated at each processing stage, where its temporal structure can be modified by (i). the selectivity of the cortical neurons, (ii). lateral interactions and (iii). top-down attentional influences from higher order cortical areas. The resulting model could account for the remarkable efficiency and rapidity of processing observed in the primate visual system.

On temporal hyperacuity in the human visual system

The spatial grain of the human visual system has always been a central topic for visual sciences, and the optical and physiological basis of perceptual limitations are well described. In particular, we have thorough accounts of spatial hyperacuity, which refers to a precision in the spatial localisation of stimulus contours that is better than the photoreceptor grain that determines spatial resolution. However, although the temporal resolution of the human visual system is comparably well described, we have almost no direct knowledge about the precision of localising visual stimuli in time in the absence of correlated spatial cues. The present study addresses this question by comparing directly the temporal resolution of human observers with their temporal acuity as measured in a temporal bisection task. Despite some improvement with practice, temporal acuity in this task does not fall below 20-30 ms in the best case, which is similar to the temporal resolution limit, and performance does not improve for comparison tasks with multiple stimulus presentations. The absence of visual hyperacuity for purely temporal modulations as tested here contrasts with processing limitations for other types of visual information in comparable tasks, and with other sensory modalities, in particular to those of the auditory system. Such differences can be interpreted in the context of the ecological requirements for organising behaviour, and the functional design of nervous systems.

Latency effects in orientation popout

A target that differs in orientation from neighboring lines and "pops out" has been found to evoke larger responses in cortical V1 cells than lines in the uniform texture surround which do not popout (e.g., Journal of Neurophysiology 67 (1992) 961). If this is more than a coincidence of observations, physiological properties of contextual modulation should be reflected in the perception of salience. In particular, as the differential suppression from texture surround has been reported to be delayed, target salience may be affected by the history of surrounding lines, i.e. by their orientation before the target was presented. This was tested using a feature flicker paradigm in which target and background lines changed their orientations (Experiment 2). All subjects (N = 4) indicated a benefit in target detection when target orientation was not previously present in the surround. A control experiment showed that this effect was not caused by the purely temporal aspects of asynchronous stimulus presentation (Experiment 3). To distinguish this effect from other sources of delayed processing, Experiment 1 compared the performance in target detection and target identification tasks, for single-lines and popout targets. All subjects required longer stimulus presentation time to identify the orientation of a single line than to detect the line itself, indicating that orientation coding needs longer processing than encoding stimulus onset. However, most subjects needed even longer presentations to detect popout, suggesting that the processing of orientation contrast adds to this delay. In an appendix, putative response variations of V1 cells to asynchronous flicker are computed.

Role of synchrony in contour binding: some transient doubts sustained

The temporal correlation hypothesis proposes that neurons signal mutual inclusion in complex features, such as extended contours, by phase-locking their firing [C. M. Gray and W. Singer, Proc. Nat]. Acad. Sci. USA 86, 1698 (1989)]. Although this hypothesis remains controversial, a number of recent psychophysical studies have suggested that temporal correlation among features can indeed promote perceptual grouping. In particular, subjects are better at detecting extended visual contours embedded within a field of distractor elements when a small delay is present between a cycling presentation of the contour and the background [Nature 394, 179 (1988)]. We have replicated this finding and examined three potentially confounding factors. First, we controlled local density and used more curved contours composed of bandpass elements to confirm that the effect was associated with contour integration and not with the operation of coarse-scale spatial filters. Second, we minimized the effects of saccadic eye movements (which could combine with the flicker of the asynchronous display to introduce motion cues at the contour location) both by using a fixation marker that was visible only when observers made a saccade (allowing them to reject these trials) and by retinally stabilizing the stimulus. We report that eye movements contribute to the effect. Third, we asked if either visible persistence or transients at the onset and the offset of the asynchronous stimuli might contribute to the effect. We report that the effect is largely abolished by the inclusion of prestimulus and poststimulus masks and is entirely abolished by ramping the contrast of the stimulus on and off. Neither ramping, masking, nor stabilization should specifically disrupt a contour-binding scheme based on temporal synchrony, and we conclude that it is the transient component at the onset and the offset of these stimuli that is responsible for the reported advantage for asynchronous presentation.