Snakes and ladders: the role of temporal modulation in visual contour integration

Coding of low-level position and orientation information in human naturalistic vision

Orientation and position of small image segments are considered to be two fundamental low-level attributes in early visual processing, yet their encoding in complex natural stimuli is underexplored. By measuring the just-noticeable differences in noise perturbation, we investigated how orientation and position information of a large number of local elements (Gabors) were encoded separately or jointly. Importantly, the Gabors composed various classes of naturalistic stimuli that were equated by all low-level attributes and differed only in their higher-order configural complexity and familiarity. Although unable to consciously tell apart the type of perturbation, observers detected orientation and position noise significantly differently. Furthermore, when the Gabors were perturbed by both types of noise simultaneously, performance adhered to a reliability-based optimal probabilistic combination of individual attribute noises. Our results suggest that orientation and position are independently coded and probabilistically combined for naturalistic stimuli at the earliest stage of visual processing.

Co-circularity opponency in visual texture

It is well known that the human visual system is sensitive to co-circularity among oriented edges, which are ubiquitous features of object contours. Here, we report a novel aftereffect in which the appearance of a texture is dramatically altered after adaptation to a texture composed of elements with co-circular structure. Following prolonged viewing of a texture made of pairs of adjacent Gabor elements arranged to form obtuse angle co-circular pairs, i.e. shallow curves, a subsequently viewed random texture appears to be composed of acute angle, i.e. near-parallel pairs. Conversely, adaptation to a texture made of parallel pairs causes a random texture to appear to be composed of shallow curves. This suggests that mechanisms sensitive to co-circularity are organized in an opponent manner, with one pole sensitive to shallow curves the other parallel shapes. This notion was tested further in a non-adaptation experiment in which co-circular and non-co-circular Gabor pairs were mixed within a single texture. Results revealed summation between pairs that fell on one side of the opponent continuum, and cancellation between pairs that fell on opposite sides of the continuum. Taken together these results support opponent interactions between mechanisms sensitive to pairwise co-circular texture features.

Contour interpolation: A case study in Modularity of Mind

In his monograph Modularity of Mind (1983), philosopher Jerry Fodor argued that mental architecture can be partly decomposed into computational organs termed modules, which were characterized as having nine co-occurring features such as automaticity, domain specificity, and informational encapsulation. Do modules exist? Debates thus far have been framed very generally with few, if any, detailed case studies. The topic is important because it has direct implications on current debates in cognitive science and because it potentially provides a viable framework from which to further understand and make hypotheses about the mind's structure and function. Here, the case is made for the modularity of contour interpolation, which is a perceptual process that represents non-visible edges on the basis of how surrounding visible edges are spatiotemporally configured. There is substantial evidence that interpolation is domain specific, mandatory, fast, and developmentally well-sequenced; that it produces representationally impoverished outputs; that it relies upon a relatively fixed neural architecture that can be selectively impaired; that it is encapsulated from belief and expectation; and that its inner workings cannot be fathomed through conscious introspection. Upon differentiating contour interpolation from a higher-order contour representational ability ("contour abstraction") and upon accommodating seemingly inconsistent experimental results, it is argued that interpolation is modular to the extent that the initiating conditions for interpolation are strong. As interpolated contours become more salient, the modularity features emerge. The empirical data, taken as a whole, show that at least certain parts of the mind are modularly organized.

The equivalent internal orientation and position noise for contour integration

Contour integration is the joining-up of local responses to parts of a contour into a continuous percept. In typical studies observers detect contours formed of discrete wavelets, presented against a background of random wavelets. This measures performance for detecting contours in the limiting external noise that background provides. Our novel task measures contour integration without requiring any background noise. This allowed us to perform noise-masking experiments using orientation and position noise. From these we measure the equivalent internal noise for contour integration. We found an orientation noise of 6° and position noise of 3 arcmin. Orientation noise was 2.6x higher in contour integration compared to an orientation discrimination control task. Comparing against a position discrimination task found position noise in contours to be 2.4x lower. This suggests contour integration involves intermediate processing that enhances the quality of element position representation at the expense of element orientation. Efficiency relative to the ideal observer was lower for the contour tasks (36% in orientation noise, 21% in position noise) compared to the controls (54% and 57%).

Contour Integration in Dynamic Scenes: Impaired Detection Performance in Extended Presentations

Since scenes in nature are highly dynamic, perception requires an on-going and robust integration of local information into global representations. In vision, contour integration (CI) is one of these tasks, and it is performed by our brain in a seemingly effortless manner. Following the rule of good continuation, oriented line segments are linked into contour percepts, thus supporting important visual computations such as the detection of object boundaries. This process has been studied almost exclusively using static stimuli, raising the question of whether the observed robustness and "pop-out" quality of CI carries over to dynamic scenes. We investigate contour detection in dynamic stimuli where targets appear at random times by Gabor elements aligning themselves to form contours. In briefly presented displays (230 ms), a situation comparable to classical paradigms in CI, performance is about 87%. Surprisingly, we find that detection performance decreases to 67% in extended presentations (about 1.9-3.8 s) for the same target stimuli. In order to observe the same reduction with briefly presented stimuli, presentation time has to be drastically decreased to intervals as short as 50 ms. Cueing a specific contour position or shape helps in partially compensating this deterioration, and only in extended presentations combining a location and a shape cue was more efficient than providing a single cue. Our findings challenge the notion of CI as a mainly stimulus-driven process leading to pop-out percepts, indicating that top-down processes play a much larger role in supporting fundamental integration processes in dynamic scenes than previously thought.

Contextual Interactions in Grating Plaid Configurations Are Explained by Natural Image Statistics and Neural Modeling

Processing natural scenes requires the visual system to integrate local features into global object descriptions. To achieve coherent representations, the human brain uses statistical dependencies to guide weighting of local feature conjunctions. Pairwise interactions among feature detectors in early visual areas may form the early substrate of these local feature bindings. To investigate local interaction structures in visual cortex, we combined psychophysical experiments with computational modeling and natural scene analysis. We first measured contrast thresholds for 2 × 2 grating patch arrangements (plaids), which differed in spatial frequency composition (low, high, or mixed), number of grating patch co-alignments (0, 1, or 2), and inter-patch distances (1° and 2° of visual angle). Contrast thresholds for the different configurations were compared to the prediction of probability summation (PS) among detector families tuned to the four retinal positions. For 1° distance the thresholds for all configurations were larger than predicted by PS, indicating inhibitory interactions. For 2° distance, thresholds were significantly lower compared to PS when the plaids were homogeneous in spatial frequency and orientation, but not when spatial frequencies were mixed or there was at least one misalignment. Next, we constructed a neural population model with horizontal laminar structure, which reproduced the detection thresholds after adaptation of connection weights. Consistent with prior work, contextual interactions were medium-range inhibition and long-range, orientation-specific excitation. However, inclusion of orientation-specific, inhibitory interactions between populations with different spatial frequency preferences were crucial for explaining detection thresholds. Finally, for all plaid configurations we computed their likelihood of occurrence in natural images. The likelihoods turned out to be inversely related to the detection thresholds obtained at larger inter-patch distances. However, likelihoods were almost independent of inter-patch distance, implying that natural image statistics could not explain the crowding-like results at short distances. This failure of natural image statistics to resolve the patch distance modulation of plaid visibility remains a challenge to the approach.

Effects of Spatial Frequency Similarity and Dissimilarity on Contour Integration

We examined the effects of spatial frequency similarity and dissimilarity on human contour integration under various conditions of uncertainty. Participants performed a temporal 2AFC contour detection task. Spatial frequency jitter up to 3.0 octaves was applied either to background elements, or to contour and background elements, or to none of both. Results converge on four major findings. (1) Contours defined by spatial frequency similarity alone are only scarcely visible, suggesting the absence of specialized cortical routines for shape detection based on spatial frequency similarity. (2) When orientation collinearity and spatial frequency similarity are combined along a contour, performance amplifies far beyond probability summation when compared to the fully heterogenous condition but only to a margin compatible with probability summation when compared to the fully homogenous case. (3) Psychometric functions are steeper but not shifted for homogenous contours in heterogenous backgrounds indicating an advantageous signal-to-noise ratio. The additional similarity cue therefore not so much improves contour detection performance but primarily reduces observer uncertainty about whether a potential candidate is a contour or just a false positive. (4) Contour integration is a broadband mechanism which is only moderately impaired by spatial frequency dissimilarity.

Do observers like curvature or do they dislike angularity?

Humans have a preference for curved over angular shapes, an effect noted by artists as well as scientists. It may be that people like smooth curves or that people dislike angles, or both. We investigated this phenomenon in four experiments. Using abstract shapes differing in type of contour (angular vs. curved) and complexity, Experiment 1 confirmed a preference for curvature not linked to perceived complexity. Experiment 2 tested whether the effect was modulated by distance. If angular shapes are associated with a threat, the effect may be stronger when they are presented within peripersonal space. This hypothesis was not supported. Experiment 3 tested whether preference for curves occurs when curved lines are compared to straight lines without angles. Sets of coloured lines (angular vs. curved vs. straight) were seen through a circular or square aperture. Curved lines were liked more than either angular or straight lines. Therefore, angles are not necessary to generate a preference for curved shapes. Finally, Experiment 4 used an implicit measure of preference, the manikin task, to measure approach/avoidance behaviour. Results did not confirm a pattern of avoidance for angularity but only a pattern of approach for curvature. Our experiments suggest that the threat association hypothesis cannot fully explain the curvature effect and that curved shapes are, per se, visually pleasant.

A Neurocomputational account of the role of contour facilitation in brightness perception

A new filling-in model is proposed in order to account for challenging brightness illusions, where inducing background elements are spatially separated from the gray target such as dungeon, cube and grating illusions, bullseye display and ring patterns. This model implements the simple idea that neural response to low-contrast contour is enhanced (facilitated) by the presence of collinear or parallel high-contrast contours in its wider neighborhood. Contour facilitation is achieved via dendritic inhibition, which enables the computation of maximum function among inputs to the node. Recurrent application of maximum function leads to the propagation of the neural signal along collinear or parallel contour segments. When a strong global-contour signal is accompanied with a weak local-contour signal at the same location, conditions are met to produce brightness assimilation within the Filling-in Layer. Computer simulations showed that the model correctly predicts brightness appearance in all of the aforementioned illusions as well as in White's effect, Benary's cross, Todorović's illusion, checkerboard contrast, contrast-contrast illusion and various variations of the White's effect. The proposed model offers new insights on how geometric factors (contour colinearity or parallelism), together with contrast magnitude contribute to the brightness perception.

Understanding mid-level representations in visual processing

It is clear that early visual processing provides an image-based representation of the visual scene: Neurons in Striate cortex (V1) encode nothing about the meaning of a scene, but they do provide a great deal of information about the image features within it. The mechanisms of these "low-level" visual processes are relatively well understood. We can construct plausible models for how neurons, up to and including those in V1, build their representations from preceding inputs down to the level of photoreceptors. It is also clear that at some point we have a semantic, "high-level" representation of the visual scene because we can describe verbally the objects that we are viewing and their meaning to us. A huge number of studies are examining these "high-level" visual processes each year. Less well studied are the processes of "mid-level" vision, which presumably provide the bridge between these "low-level" representations of edges, colors, and lights and the "high-level" semantic representations of objects, faces, and scenes. This article and the special issue of papers in which it is published consider the nature of "mid-level" visual processing and some of the reasons why we might not have made as much progress in this domain as we would like.

Inter-element orientation and distance influence the duration of persistent contour integration

Contour integration is a fundamental form of perceptual organization. We introduce a new method of studying the mechanisms responsible for contour integration. This method capitalizes on the perceptual persistence of contours under conditions of impending camouflage. Observers viewed arrays of randomly arranged line segments upon which circular contours comprised of similar line segments were superimposed via abrupt onset. Crucially, these contours remained visible for up to a few seconds following onset, but eventually disappeared due to the camouflaging effects of surrounding background line segments. Our main finding was that the duration of contour visibility depended on the distance and degree of co-alignment between adjacent contour segments such that relatively dense smooth contours persisted longest. The stimulus-related effects reported here parallel similar results from contour detection studies, and complement previous reported top–down influences on contour persistence (Strother et al., 2011). We propose that persistent contour visibility reflects the sustained activity of recurrent processing loops within and between visual cortical areas involved in contour integration and other important stages of visual object recognition.

Component processes in contour integration: a direct comparison between snakes and ladders in a detection and a shape discrimination task

In contour integration, a relevant question is whether snakes and ladders are processed similarly. Higher presentation time thresholds for ladders in detection tasks indicate this is not the case. However, in a detection task only processing differences at the level of element linking and possibly contour localization might be picked up, while differences at the shape encoding level cannot be noticed. In this study, we make a direct comparison of detection and shape discrimination tasks to investigate if processing differences in the visual system between snakes and ladders are limited to contour detection or extend to higher level contour processing, like shape encoding. Stimuli consisted of elements that were oriented collinearly (snakes) or orthogonally (ladders) to the contour path and were surrounded by randomly oriented background elements. In two tasks, six experienced subjects either detected the contour when presented with a contour and a completely random stimulus or performed a shape discrimination task when presented with two contours with different curvature. Presentation time was varied in 9 steps between 8 and 492 ms. By applying a generalized linear mixed model we found that differences in snake and ladder processing are not limited to a detection stage but are also apparent at a shape encoding stage.

The integration of straight contours (snakes and ladders): The role of spatial arrangement, spatial frequency and spatial phase

In the present study we addressed the issue of whether the Gestalt principle of grouping by similarity (iso-orientation) subtends extraction of straight contours made up of disconnected, iso-oriented Gabor elements, whether collinear (snakes) or parallel (ladders). To prevent the use of the most obvious grouping principle of good continuation, which allows us to perceive the relation between local and global orientation along the contour, we manipulated the spatial arrangement of randomly oriented Gabors in the background: they were positioned on an ordered grid, and grouped on the basis of good continuation, or randomly positioned and not grouped. Grid-positioned backgrounds exert a suppressive contextual influence on detection of good continuation along the contour path. Results obtained in a two-interval forced choice task showed that the orderly-positioned background did not completely prevent detection of snakes and ladders. Detection of snakes was hampered at low spatial frequency whereas detection of ladders was improved by the randomly-positioned background at high spatial frequency. These contextual influences support the suggestion that both iso-orientation and good continuation rules are employed by the association field underlying the binding of straight contours. In addition, they are not compatible with integration of snakes and ladders elements within a single receptive field. In support of this suggestion we found that phase constancy within contour elements (as opposed to phase randomization) improved snake detectability at low spatial frequency, and, unexpectedly, impaired ladder detectability at high spatial frequency. This suggests that a low-level mechanism based on the balance between excitatory and inhibitory lateral interactions at a first stage may account for the detection of both straight contours.

The effect of spatial orientation on detecting motion trajectories in noise

A series of experiments investigated the extent to which the spatial orientation of a signal line affects discrimination of its trajectory from the random trajectories of background noise lines. The orientation of the signal line was either parallel (iso-) or orthogonal (ortho-) to its motion direction and it was identical in all respects to the noise (orientation, length and speed) except for its motion direction, rendering the signal line indistinguishable from the noise on a frame-to-frame basis. We found that discrimination of ortho-trajectories was generally better than iso-trajectories. Discrimination of ortho-trajectories was largely immune to the effects of spatial jitter in the trajectory, and to variations in step size and line-length. Discrimination of iso-trajectories was reliable provided that step-size was not too short and did not exceed line length, and that the trajectory was straight. The new result that trajectory discrimination in moving line elements is modulated by line orientation suggests that ortho- and iso-trajectory discrimination rely upon two distinct mechanisms: iso-motion discrimination involves a 'motion-streak' process that combines motion information with information about orientation parallel to the motion trajectory, while ortho-motion discrimination involves extended trajectory facilitation in a network of receptive fields with orthogonal orientation tuning.

Visual exploration and object recognition by lattice deformation

Mechanisms of explicit object recognition are often difficult to investigate and require stimuli with controlled features whose expression can be manipulated in a precise quantitative fashion. Here, we developed a novel method (called "Dots"), for generating visual stimuli, which is based on the progressive deformation of a regular lattice of dots, driven by local contour information from images of objects. By applying progressively larger deformation to the lattice, the latter conveys progressively more information about the target object. Stimuli generated with the presented method enable a precise control of object-related information content while preserving low-level image statistics, globally, and affecting them only little, locally. We show that such stimuli are useful for investigating object recognition under a naturalistic setting--free visual exploration--enabling a clear dissociation between object detection and explicit recognition. Using the introduced stimuli, we show that top-down modulation induced by previous exposure to target objects can greatly influence perceptual decisions, lowering perceptual thresholds not only for object recognition but also for object detection (visual hysteresis). Visual hysteresis is target-specific, its expression and magnitude depending on the identity of individual objects. Relying on the particular features of dot stimuli and on eye-tracking measurements, we further demonstrate that top-down processes guide visual exploration, controlling how visual information is integrated by successive fixations. Prior knowledge about objects can guide saccades/fixations to sample locations that are supposed to be highly informative, even when the actual information is missing from those locations in the stimulus. The duration of individual fixations is modulated by the novelty and difficulty of the stimulus, likely reflecting cognitive demand.

An exploration of the spatial scale over which orientation-dependent surround effects affect contour detection

Contour detection is a crucial component of visual processing; however, performance on contour detection tasks can vary depending on the context of the visual scene. S. C. Dakin and N. J. Baruch (2009) showed that detection of a contour in an array of distracting elements depends on the orientation of flanking elements. Here, using a line of five collinear Gabor elements ("target contour") in a field of distractor Gabor elements, we systematically measured the effects of eccentricity, spacing, and spatial frequency on contour detection performance in three different contexts: randomly oriented distractors (control condition), flanking distractors (on either side of the collinear Gabors) aligned approximately parallel to the target contour, and flanking distractors aligned approximately orthogonal to the target contour. In the control condition, contour detection performance was best for larger Gabors (2 cpd) spaced farther apart (1.2°). Parallel flankers reduced performance for intermediate and large spacings and sizes compared to the control condition, while orthogonal flankers increased performance for the smallest spacing and size compared to the control condition. The results are fit by a model in which collinear facilitation, which is size-dependent but can persist for several degrees of visual angle, competes with orientation-dependent suppression from the flanking context when elements are separated by less than a degree of visual angle.

The effects of aging on contour discrimination in clutter

The present study examined the effect of aging on the detection and discrimination of contours embedded in a dense field of distractors. The minimum stimulus duration required to correctly discriminate (Experiment 1) and detect (Experiment 2) three types of "C" shaped contours was measured. Overall, older subjects required longer stimulus durations than younger subjects in all conditions. Comparing performance for contours comprising elements oriented tangentially to the contour path (aligned) and those oriented orthogonally to the contour (radial) revealed that the effect of local orientation on contour discrimination is slightly greater in older than younger subjects. Control experiments showed that these age differences were not due to differences in retinal illuminance, or the detectability or discriminability of the elements comprising the contours. These findings suggest that ability to extract global contours embedded in clutter declines in older age.

Identification of everyday objects on the basis of Gaborized outline versions

Using outlines derived from a widely used set of line drawings, we created stimuli geared towards the investigation of contour integration and texture segmentation using shapes of everyday objects. Each stimulus consisted of Gabor elements positioned and oriented curvilinearly along the outline of an object, embedded within a larger Gabor array of homogeneous density. We created six versions of the resulting Gaborized outline stimuli by varying the orientations of elements inside and outside the outline. Data from two experiments, in which participants attempted to identify the objects in the stimuli, provide norms for identifiability and name agreement, and show differences in identifiability between stimulus versions. While there was substantial variability between the individual objects in our stimulus set, further analyses suggest a number of stimulus properties which are generally predictive of identification performance. The stimuli and the accompanying normative data, both available on our website (http://www.gestaltrevision.be/sources/gaboroutlines), provide a useful tool to further investigate contour integration and texture segmentation in both normal and clinical populations, especially when top-down influences on these processes, such as the role of prior knowledge of familiar objects, are of main interest.

Learning-dependent plasticity with and without training in the human brain

Long-term experience through development and evolution and shorter-term training in adulthood have both been suggested to contribute to the optimization of visual functions that mediate our ability to interpret complex scenes. However, the brain plasticity mechanisms that mediate the detection of objects in cluttered scenes remain largely unknown. Here, we combine behavioral and functional MRI (fMRI) measurements to investigate the human-brain mechanisms that mediate our ability to learn statistical regularities and detect targets in clutter. We show two different routes to visual learning in clutter with discrete brain plasticity signatures. Specifically, opportunistic learning of regularities typical in natural contours (i.e., collinearity) can occur simply through frequent exposure, generalize across untrained stimulus features, and shape processing in occipitotemporal regions implicated in the representation of global forms. In contrast, learning to integrate discontinuities (i.e., elements orthogonal to contour paths) requires task-specific training (bootstrap-based learning), is stimulus-dependent, and enhances processing in intraparietal regions implicated in attention-gated learning. We propose that long-term experience with statistical regularities may facilitate opportunistic learning of collinear contours, whereas learning to integrate discontinuities entails bootstrap-based training for the detection of contours in clutter. These findings provide insights in understanding how long-term experience and short-term training interact to shape the optimization of visual recognition processes.

Decoding of coherent but not incoherent motion signals in early dorsal visual cortex

When several scattered grating elements are arranged in such a way that their directions of motion are consistent with a common path, observers perceive them as belonging to a globally coherent moving object. Here we investigated how this coherence changes the representation of motion signals in human visual cortex using functional magnetic resonance imaging (fMRI) and multivariate voxel pattern decoding, which have the potential to reveal how well a stimulus is encoded in different contexts. Only during globally coherent motion was it possible to reliably distinguish fMRI signals evoked by different directions of motion in early visual cortex. This effect was specific to the retinotopic representation of the visual field quadrant in V1 traversed by the coherent element path and could not simply be attributed to a general increase in signal strength. Decoding was more reliable for cortical areas corresponding to the lower visual field. Because some previous studies observed poorer speed discrimination when motion was grouped, we also conducted behavioural experiments to investigate this with our stimuli, but did not reveal a consistent relationship between coherence and perceived speed. Taken together, these data show that neuronal populations in early visual cortex represent information that could be used for interpreting motion signals as unified objects.

The aperture problem in contoured stimuli

A moving object elicits responses from V1 neurons tuned to a broad range of locations, directions, and spatiotemporal frequencies. Global pooling of such signals can overcome their intrinsic ambiguity in relation to the object's direction/speed (the "aperture problem"); here we examine the role of low-spatial frequencies (SF) and second-order statistics in this process. Subjects made a 2AFC fine direction-discrimination judgement of 'naturally' contoured stimuli viewed rigidly translating behind a series of small circular apertures. This configuration allowed us to manipulate the scene by randomly switching which portion of the stimulus was presented behind each aperture or by occluding certain spatial frequency bands. We report that global motion integration is (a) largely insensitive to the second-order statistics of such stimuli and (b) is rigidly broadband even in the presence of a disrupted low SF component.

Flexible learning of natural statistics in the human brain

The ability to detect and identify targets in cluttered scenes is a critical skill for survival and interactions. To solve this challenge the brain has optimized mechanisms for capitalizing on frequently occurring regularities in the environment. Although evolution and development have been suggested to shape the brain's architecture in a manner that resembles these natural statistics, we provide novel evidence that short-term experience in adulthood may modify the brain's functional organization to support integration of signals atypical of shape contours in natural scenes. Although collinearity is a prevalent principle for perceptual integration in natural scenes, we show that observers learn to exploit other image regularities (i.e., orthogonal alignments of segments at an angle to the contour path) that typically signify discontinuities. Combining behavioral and functional MRI measurements, we demonstrate that this flexible learning is mediated by changes in the neural representations of behaviorally relevant image regularities primarily in dorsal visual areas. These changes in neural sensitivity are in line with changes in perceptual sensitivity for the detection of orthogonal contours and are evident only in observers that show significant performance improvement. In contrast, changes in the activation extent in frontoparietal regions are evident independent of performance changes, may support the detection of salient regions, and modulate perceptual integration in occipitotemporal areas in a top-down manner. Thus experience at shorter timescales in adulthood supports the adaptive functional optimization of visual circuits for flexible interpretation of natural scenes.

Experience shapes the utility of natural statistics for perceptual contour integration

Segmenting meaningful targets from cluttered scenes is a fundamental function of the visual system. Evolution and development have been suggested to optimize the brain's solution to this computationally challenging task by tuning the visual system to features that co-occur frequently in natural scenes (e.g., collinear edges) [1, 2, 3]. However, the role of shorter-term experience in shaping the utility of scene statistics remains largely unknown. Here, we ask whether collinearity is a specialized case, or whether the brain can learn to recruit any image regularity for the purpose of target identification. Consistent with long-term optimization for typical scene statistics, observers were better at detecting collinear contours than configurations of elements oriented at orthogonal or acute angles to the contour path. However, training resulted in improved detection of orthogonal contours that lasted for several months, suggesting retuning rather than transient changes of visual sensitivity. Improvement was also observed for acute contours but only after longer training. These results demonstrate that the brain flexibly exploits image regularities and learns to use discontinuities typically associated with surface boundaries (orthogonal, acute alignments) for contour linking and target identification. Thus, short-term experience in adulthood shapes the interpretation of scenes by assigning new statistical utility to image regularities.

Apparent Motion by Edge Discontinuities

When the eyes move vertically across a jagged diamond, a local shift (LS) of edge discontinuities and a global shape distortion (GD) (ie expansion/contraction opposite to that expected by the aperture effect) are perceived. These phenomena cannot be accounted for by a local motion signals integration rule based either on the intersection of constraint lines or on the velocity vector summation. The threshold for GD perception and the salience of LS and GD (1 to 10 scale) were measured in two experiments by different methods and displays. In experiment 1 we induced GD through mimicking LS with a kinetic pattern constituted of a set of circular illusory apertures revealing drifting gratings. The point of subjective equality for compression/expansion was reached for gratings the linear extrapolations of which form an angle of 94.4°. In experiment 2 observers followed a dot moving along the vertical elongation axis of a static jagged diamond (with 70° or 90° angles), varying in the shape (triangular, wave, square), frequency, and amplitude of edge discontinuities. GD scores were correlated with LS scores that were inversely related to frequency/amplitude ratios of triangular edge discontinuities. Data are partially accounted for by averaging neighbouring local motion-capture vectors. Results prove that there are strong interrelations between phenomena in which visual motion affects visual localisation and phenomena involving apparent deformation of global shape.

Spatial versus temporal grouping in a modified Ternus display

The Ternus display can induce a percept of 'element motion' or 'group motion'. Conventionally, this has been attributed to two different motion processes, with different spatial and temporal ranges. In contrast, recent studies have emphasised spatial and temporal grouping principles as underlying the apparent motion percepts in the Ternus display. The present study explored effects of spatial and temporal grouping on the apparent motion percept in a novel Ternus display of oriented Gabor elements with no inter-frame interval. Each frame of this stimulus could be further divided into 'sub-frames', and the orientation of the carriers was changed across these sub-frames. In four experiments transitions were found between the motion percepts with changes in orientation across time (Experiment 1) and space (Experiment 2), and with a temporal offset in the orientation change of the outer element (Experiment 3) to the extent that group motion was not perceived even with large orientation changes over time that previously led to group motion (Experiment 4). Collectively, these results indicate that while spatial properties have an influence in determining the percept of the Ternus display, temporal properties also have a strong influence, and can override spatial grouping. However, these temporal effects cannot be attributed to spatio-temporal limits of low-level motion processes. Some aspects of the observed spatial grouping effects can be accounted for in terms of a modified association field, which may occur through connectivity of orientation selective units in V1. The temporal effects observed are considered in terms of temporal integration, the transitional value at a temporal offset of 40ms being remarkably similar to psychophysical and neurophysiological estimates of the peak temporal impulse response. These temporal responses could be detected at a higher level in the system, providing a basis for apparent motion perception.

Grouping local orientation and direction signals to extract spatial contours: empirical tests of "association field" models of contour integration

Over the last decade or so a great deal of psychophysical research has attempted to delineate the principles by which local orientations and motions are combined across space to facilitate the detection of simple spatial contours. This has led to the development of "association field" models of contour detection which suggest that the strength of linking between neighbouring elements in an image, is determined by the degree to which they aligned along smooth (first-order) curves. To test this assumption we used a path detection paradigm to compare the ability of observers to identify the presence of contours defined by either spatial orientation, motion direction or by specific combinations of both types of visual attribute. The relative alignment of the local orientations and/or directions with respect to the axis of the depicted contour was systematically varied. For orientation-defined contours detection was best when the elements were aligned along (parallel with) the contour axis, approached chance levels for obliquely oriented elements and then improved for elements that were orthogonal to the contour axis (i.e., performance was a U-shaped function of degree of orientation misalignment). This pattern of results was found for both straight and curved contours and is not readily explicable in terms of current association field theories. For motion-defined contours, however, performance simply deteriorated as the relative directions of the constituent path elements were progressively misaligned with respect to the contour. Thus the rules by which local orientations are linked to define spatial contours are qualitatively different from those used for linking local directions and each may be mediated by distinct visual mechanisms. When both orientation and motion cues were simultaneously available, contour detection performance was generally enhanced, in a manner that is consistent with probability summation. We suggest that association field models of orientation linking may need to be extended in light of the present findings.

Perceptual grouping in Gabor lattices: Proximity and alignment

We propose the Gabor lattice as a new stimulus designed to deal with multiple organizations in perceptual grouping, allowing both comparison between psychophysical data and neural findings and a systematic investigation of grouping based on several low-level characteristics and their interactions. A Gabor lattice is a geometric lattice with Gabor patches, evoking a multistable global orientation percept. Visual grouping in Gabor lattices with elements aligned in a global orientation was compared with grouping of nonaligned Gabor patches and of Gaussian blobs. The effect sizes of proximity and alignment were estimated in logistic regression analyses. The results confirmed the importance of proximity and local element alignment as factors in dynamic grouping. We also found a small but consistent enhancement of grouping along the global vector orthogonal to the local patch orientations. In light of these results, we further motivate the relevance of these stimuli and the associated experimental paradigm.

The spatial frequency and orientation selectivity of the mechanisms that extract motion-defined contours

The human visual system can undertake a specialized form of motion integration, one that enables the presence of extended spatial contours to be disambiguated from their backgrounds. We have shown previously that the visual system can selectively integrate local motion signals when their directions are along spatial contours and its efficiency is inversely related to the curvature of the contour involved (Ledgeway, T., & Hess, R. F. (2002). Vision Research, 42, 653-659). This integration primarily involves the direction, rather than the speed, of local motion signals. In the present study, we sought to investigate both the spatial frequency and orientation tuning of this specialized contour integration process, using a path detection paradigm. The results show that the tuning for spatial frequency is very broad, in line with previous studies that have examined this issue. In contrast, the orientation selectivity of the mechanism mediating contour extraction under these conditions is relatively narrowband. Thus, spatial frequency but not orientation pooling appears to take place prior to the extraction of motion-defined contours, a situation that is different from that previously shown for spatial contours composed of static, oriented elements.

Contour integration and cortical processing

Our understanding of visual processing in general, and contour integration in particular, has undergone great change over the last 10 years. There is now an accumulation of psychophysical and neurophysiological evidence that the outputs of cells with conjoint orientation preference and spatial position are integrated in the process of explication of rudimentary contours. Recent neuroanatomical and neurophysiological results suggest that this process takes place at the cortical level V1. The code for contour integration may be a temporal one in that it may only manifest itself in the latter part of the spike train as a result of feedback and lateral interactions. Here we review some of the properties of contour integration from a psychophysical perspective and we speculate on their underlying neurophysiological substrate.

The influence of motion-defined form on the perception of spatially-defined form

It is well established that the visual system is sensitive to the global structure--or "form"--of objects defined exclusively by spatial or motion cues, but it remains unclear how form perception combines spatial and motion cues if these are presented concurrently. In the present study, we introduce a novel class of stimuli where spatial-form and motion-form can be superimposed and manipulated independently. In both the spatial and motion domains, global structure consisted of radial-frequency (RF) contours defined by a virtual circle of Gabor elements whose positions and/or drift speeds were sinusoidally modulated at a specified frequency of polar angle. The first two experiments revealed that observers encode the global structure of spatial-RF and motion-RF contours presented in isolation. In a third experiment, observers detected a spatial-RF modulation superimposed on a motion-RF pedestal of identical radial frequency: results showed little facilitation at low pedestal amplitudes but significant masking at higher pedestal amplitudes, especially if the RF modulations of test and pedestal were in anti-phase. Additional experiments demonstrated that masking of the spatial-RF test is abolished if the global structure of the motion-RF pedestal is altered or destroyed while local motion cues are preserved. We argue these results cannot be explained by local neural interactions between spatial and motion cues and propose instead that data reflect higher-level interactions between separate visual pathways encoding spatial-form and motion-form.

Grouping local directional signals into moving contours

We consider how local motion signals are combined to represent the movements of spatially extensive objects. A series of band-pass target dots, whose collective motion defined a moving contour, was positioned within a field of randomly moving noise dots. The visibility of the contours did not depend on the direction of movement relative to local contour orientation unless the contour was constrained to pass through fixation, suggesting that a previously reported advantage for collinear motion trajectories depends on the probability of detecting any of the target elements rather than the integrated contour. Contour visibility was invariant of the spatial frequency of the elements, but it did depend on the speed, number and spacing of elements defining it, as well as the angle and spatial frequency difference between adjacent elements. Local averaging of directional signals is not sufficient to explain these results. The visibility of these moving contours identifies narrow-band grouping processes that are sensitive to the shape defined by the directions of the elements forming the contour.

Modeling the integration of motion signals across space

Experiments by Loffler and Orbach on the integration of motion signals across space [J. Opt. Soc. Am. A 20, 1461 (2003)] revealed that both three-dimensional analysis and object interpretation play a much smaller role than previously assumed. These results motivated the quantitative description of a low-level, bottom-up model presented here. Motion is computed in parallel at different spatial sites, and excitatory interactions operate between sites. The strength of these interactions is determined mainly by distance. Simulations correctly predict behavior for a variety of manipulations on multi-aperture stimuli: aligned and skewed lines, different presentation times, different inter-aperture gaps, and different spatial frequencies. However, strictly distance-dependent mechanisms are too simplistic to account for all experimental data. Mismatches for grossly misoriented lines suggest collinear facilitation as a promising extension. Once incorporated, collinear facilitation not only correctly predicts results for misoriented patterns but also accounts for the lack of motion integration between heterogeneous stimuli such as lines and dots.

The detection of direction-defined and speed-defined spatial contours: one mechanism or two?

It is now accepted that the visual system integrates local orientation information across space to define spatial contours [Vision Research 33 (1993) 173]. More recently, it has been shown that similar integration occurs for the direction of local motion signals, in different parts of the visual field, if they are aligned along the axis of a spatial contour [Vision Research 42 (2002) 653]. Here we ask whether similar spatial-linking rules hold for contours comprised of local elements that share only a common speed (but not direction), in the presence of background elements which collectively have the same mean speed as the contour but considerable random variation in the speeds of the individual elements. Furthermore we investigate the detection of spatial contours that are defined by a common speed that is different (both locally and globally) from that of the background elements. The results show that there is a significant, albeit relatively weak, speed-association field with preferential linking between spatially proximal elements that have similar speeds. Although a salient speed difference between the contour and the background elements enhances detection performance for motion-defined contours, it does so primarily via a different route to that of direction linking. We suggest that for motion-defined contours the Gestalt notions of "common fate" and "good continuity", that describe the parsing of local velocity information into objects, boundaries and contours, are mediated via separate underlying perceptual mechanisms.