Second-order processes in vision: introduction

A Mechanosensory Circuit that Mixes Opponent Channels to Produce Selectivity for Complex Stimulus Features

Johnston's organ is the largest mechanosensory organ in Drosophila; it analyzes movements of the antenna due to sound, wind, gravity, and touch. Different Johnston's organ neurons (JONs) encode distinct stimulus features. Certain JONs respond in a sustained manner to steady displacements, and these JONs subdivide into opponent populations that prefer push or pull displacements. Here, we describe neurons in the brain (aPN3 neurons) that combine excitation and inhibition from push/pull JONs in different ratios. Consequently, different aPN3 neurons are sensitive to movement in different parts of the antenna's range, at different frequencies, or at different amplitude modulation rates. We use a model to show how the tuning of aPN3 neurons can arise from rectification and temporal filtering in JONs, followed by mixing of JON signals in different proportions. These results illustrate how several canonical neural circuit components-rectification, opponency, and filtering-can combine to produce selectivity for complex stimulus features.

Visual chimaeras obtained with the Riesz transform

Similar to an auditory chimaera (Smith et al, 2002 Nature 416 87-90), a visual chimaera can be defined as a synthetic image which has the fine spatial structure of one natural image and the envelope of another image in each spatial frequency band. Visual chimaeras constructed in this way could be useful to vision scientists interested in the study of interactions between first-order and second-order visual processing. Although it is almost trivial to generate 1-D chimaeras by means of the Hilbert transform and the analytic signal, problems arise in multidimensional signals like images given that the partial directional Hilbert transform and current 2-D demodulation algorithms are anisotropic or orientation-variant procedures. Here, we present a computational procedure to synthesise visual chimaeras by means of the Riesz transform--an isotropic generalisation of the Hilbert transform for multidimensional signals--and the associated monogenic signal--the vector-valued function counterpart of the analytic signal in which the Riesz transform replaces the Hilbert transform. Examples of visual chimaeras are shown for same/different category images.

Application of Riesz transforms to the isotropic AM-PM decomposition of geometrical-optical illusion images

The existence of a special second-order mechanism in the human visual system, able to demodulate the envelope of visual stimuli, suggests that spatial information contained in the image envelope may be perceptually relevant. The Riesz transform, a natural isotropic extension of the Hilbert transform to multidimensional signals, was used here to demodulate band-pass filtered images of well-known visual illusions of length, size, direction, and shape. We show that the local amplitude of the monogenic signal or envelope of each illusion image conveys second-order information related to image holistic spatial structure, whereas the local phase component conveys information about the spatial features. Further low-pass filtering of the illusion image envelopes creates physical distortions that correspond to the subjective distortions perceived in the illusory images. Therefore the envelope seems to be the image component that physically carries the spatial information about these illusions. This result contradicts the popular belief that the relevant spatial information to perceive geometrical-optical illusions is conveyed only by the lower spatial frequencies present in their Fourier spectrum.

Perception of edges and visual texture in the camouflage of the common cuttlefish, Sepia officinalis

The cuttlefish, Sepia officinalis, provides a fascinating opportunity to investigate the mechanisms of camouflage as it rapidly changes its body patterns in response to the visual environment. We investigated how edge information determines camouflage responses through the use of spatially high-pass filtered 'objects' and of isolated edges. We then investigated how the body pattern responds to objects defined by texture (second-order information) compared with those defined by luminance. We found that (i) edge information alone is sufficient to elicit the body pattern known as Disruptive, which is the camouflage response given when a whole object is present, and furthermore, isolated edges cause the same response; and (ii) cuttlefish can distinguish and respond to objects of the same mean luminance as the background. These observations emphasize the importance of discrete objects (bounded by edges) in the cuttlefish's choice of camouflage, and more generally imply that figure-ground segregation by cuttlefish is similar to that in vertebrates, as might be predicted by their need to produce effective camouflage against vertebrate predators.

On the flexibility of sustained attention and its effects on a texture segmentation task

Previously we have shown that transient attention--the more automatic, stimulus-driven component of spatial attention--enhances spatial resolution. Specifically, transient attention improves texture segmentation at the periphery, where spatial resolution is too low, but impairs performance at central locations, where spatial resolution is already too high for the task. In the present study we investigated whether sustained attention--the more controlled component of spatial attention-can also affect texture segmentation, and if so whether its effect will be similar to that of transient attention. To that end we combined central, symbolic cues with texture displays in which the target appears at several eccentricities. We found that sustained attention can also affect texture segmentation, but unlike transient attention, sustained attention improved performance at all eccentricities. Comparing the effect of pre-cues and post-cues indicated that the benefit brought about by sustained attention is significantly greater than the effect of location uncertainty reduction. These findings indicate that sustained attention is a more flexible mechanism that can optimize performance at all eccentricities in a task where performance is constrained by spatial resolution.

Structural sparseness and spatial phase alignment in natural scenes

The Fourier phase spectrum plays a central role regarding where in an image contours occur, thereby defining the spatial relationship between those structures in the overall scene. Only a handful of studies have demonstrated psychophysically the relevance of the Fourier phase spectrum with respect to human visual processing, and none have demonstrated the relative amount of local cross-scale spatial phase alignment needed to perceptually extract meaningful structure from an image. We investigated the relative amount of spatial phase alignment needed for humans to perceptually match natural scene image structures at three different spatial frequencies [3, 6, and 12 cycles per degree (cpd)] as a function of the number of structures within the image (i.e., "structural sparseness"). The results showed that (1) the amount of spatial phase alignment needed to match structures depends on structural sparseness, with a bias for matching structures at 6 cpd and (2) the ability to match partially phase-randomized images at a given spatial frequency is independent of structural sparseness at other spatial frequencies. The findings of the current study are discussed in terms of a network of feature integrators in the human visual system.

Visual evoked potentials and reaction time measurements to motion-reversal luminance- and texture-defined stimuli

PURPOSE

Previous studies have suggested that compared to first-order (FO) motion stimuli, second-order (SO) motion stimuli required more cortical time to be processed. The purpose of this study was: 1- to verify this claim with Visual Evoked Potential (VEP) and eye-hand Reaction Time (RT) measurements and 2- examine if the VEP and RT responses are similarly modulated by the same trigger features of the stimuli.

METHODS

The VEPs and eye-hand RT for motion-reversal luminance- (first-order) and texture-defined (second-order) stimuli were recorded from ten normal human subjects. VEPs and RTs were measured for each motion class at eight different modulation depths (from 3 to 100%).

RESULTS

Our results reveal that for stimuli of low contrast levels, the SO-FO timing differences are approximately 100 ms (RT) or 20 ms (VEP), while for contrasts >or= 15-20% (VEP) or >or= 50% (RT), the SO-FO difference is no longer significant (p < 0.007), suggesting either that the brain can no longer distinguish SO from FO stimuli or that in spite of the added complexity of SO stimuli the brain takes equal time to process both.

CONCLUSION

Interestingly, the above contrast discrepancy in SO-FO resolution threshold suggests that, compared to the VEP, the more psychophysical RT measurement can process and thus distinguish a larger spectrum of motion stimuli, thus further confirming the latter measure of the retinocortical processing time as a valid alternative to the VEP.

Contextual effects in fine spatial discriminations

The context in which a pattern is viewed can greatly affect its apparent contrast, a phenomenon commonly attributed to pooled contrast gain control processes. A low-contrast surround may slightly enhance apparent contrast, whereas increasing the contrast of the surround leads to a monotonic decline in contrast appearance. We ask here how the presence of a patterned surround affects the ability to perform fine, suprathreshold orientation, contrast, and spatial frequency discriminations as a function of surround contrast and phase. Our results revealed an unexpected dip in performance when center and surround were in phase and similar in contrast. These results suggest that additional processes, perhaps those involved in scene segregation, play a role in contextual effects on discrimination.

How is complex second-order motion processed?

Converging psychophysical and electrophysiological evidence suggests that first-order (luminance-defined) complex motion types i.e., radial and rotational motion, are processed by specialized extrastriate motion mechanisms. We ask whether radial and rotational second-order (texture-defined) motion patterns are processed in a similar manner. The motion sensitivity to translating, radiating and rotating motion patterns of both first-order (luminance-modulated noise) and second-order (contrast-modulated noise) were measured for patterns presented at four different exposure durations (106, 240, 500 and 750 ms). No significant difference in motion sensitivity was found across motion type for the first-order motion class across exposure duration (i.e., from 240 to 750 ms) whereas direction-identification thresholds for radiating and rotating second-order motion were significantly greater than that of the second-order translational stimuli. Furthermore, thresholds to all second-order motion stimuli increased at a significantly faster rate with decreasing exposure duration compared to those of first-order motion. Interestingly, simple and complex second-order thresholds increased at similar rates. Taken together, the results suggest that complex second-order motion is not analyzed in a sequential manner. Rather, it seems that the same 'hard-wired' mechanisms responsible for complex first-order motion processing also mediate complex second-order motion, but not before the pre-processing (i.e., rectification) of local second-order motion signals.

Gradient-based analysis of non-Fourier motion

A gradient-based image analysis technique is applied to a class of non-Fourier stimuli. To create the stimuli, n translating sine waves with identical spatial and temporal frequencies, but separated by 2pi/n radians, are spatially randomly sampled to produce a P(n) stimulus. For n>or=2, the stimuli are non-Fourier. Local image gradients are represented in the form of a gradient plot, a histogram which shows the frequency of ranges of temporal gradient/spatial gradient pairs occurring. It is shown that the gradient plots contain features, oriented in gradient space, which indicate correct non-Fourier velocity. As n increases, so too does the complexity of the gradient plots, a finding which may account for the concomitant decrease in perceived coherent motion [Vision Res 37 (1997) 1459]. This paper demonstrates that the gradient plot and associated velocity plots are a useful way of assessing gradient-based motion information. Compared to the traditional Fourier based approach, gradient-based analysis can lead to different judgement of the motion information available to standard models of low-level motion processing.

Properties of second-order spatial frequency channels

The segregation of texture patterns may be carried out by a set of linear spatial filters (to enhance one of the constituent textures), a nonlinearity (to convert the higher contrast of response to that constituent to a higher mean response), and finally subsequent ("second-order") linear spatial filters (to provide a strong response to the texture-defined edge itself). In this paper, the properties of such second-order filters are characterized. Observers were required to detect or discriminate textures that were modulated between predominantly horizontally oriented and predominantly vertically oriented noise patterns. Spatial summation for these patterns reached asymptote for a stimulus size of 15 x 15 deg. Modulation contrast sensitivity was nearly flat over a five-octave range of spatial frequency, but was bandpass when stated as efficiency (relative to an idealized observer confronted with the same task). Increment threshold showed the improved performance with a sub-threshold pedestal seen in the "dipper effect", but the typical Weber's law behavior at higher pedestal contrasts was not observed at the highest pedestal modulation contrasts achievable with our stimuli. Sub-threshold summation experiments indicate that second-order filters have a moderate bandwidth.

Marker correspondence, not processing latency, determines temporal binding of visual attributes

BACKGROUND

When simultaneous visual events appear to occur at different times, the discrepancy has generally been ascribed to time differences in neural transmission or cortical processing that lead to asynchronous awareness of the events.

RESULTS

We found, however, that an apparent delay of changes in motion direction relative to synchronous color changes occurs only for rapid alternations, and this delay is not accompanied by a difference in reaction time. We also found that perceptual asynchrony depends on the temporal structure of the stimuli (transitions [first-order temporal change] versus turning points [second-order temporal change]) rather than the attribute type (color versus motion).

CONCLUSIONS

We propose that the perception of the relative time of events is based on the relationship of representations of temporal pattern that we term time markers. We conclude that the perceptual asynchrony effects studied here do not reflect differential neural delays for different attributes; rather, they arise from a faulty correspondence match between color transitions and position transitions (motion), which in turn results from a difficulty in detecting turning points (direction reversals) and a preference for matching markers of the same type.