Detection of chromatic and luminance contrast modulation by the visual system

Spatio-chromatic contrast sensitivity under mesopic and photopic light levels

Contrast sensitivity functions (CSFs) characterize the sensitivity of the human visual system at different spatial scales, but little is known as to how contrast sensitivity for achromatic and chromatic stimuli changes from a mesopic to a highly photopic range reflecting outdoor illumination levels. The purpose of our study was to further characterize the CSF by measuring both achromatic and chromatic sensitivities for background luminance levels from 0.02 cd/m² to 7,000 cd/m². Stimuli consisted of Gabor patches of different spatial frequencies and angular sizes, varying from 0.125 to 6 cpd, which were displayed on a custom high dynamic range (HDR) display with luminance levels up to 15,000 cd/m². Contrast sensitivity was measured in three directions in color space, an achromatic direction, an isoluminant “red-green” direction, and an S-cone isolating “yellow-violet” direction, selected to isolate the luminance, L/M-cone opponent, and S-cone opponent pathways, respectively, of the early postreceptoral processing stages. Within each session, observers were fully adapted to the fixed background luminance (0.02, 2, 20, 200, 2,000, or 7,000 cd/m²). Our main finding is that the background luminance has a differential effect on achromatic contrast sensitivity compared to chromatic contrast sensitivity. The achromatic contrast sensitivity increases with higher background luminance up to 200 cd/m² and then shows a sharp decline when background luminance is increased further. In contrast, the chromatic sensitivity curves do not show a significant sensitivity drop at higher luminance levels. We present a computational luminance-dependent model that predicts the CSF for achromatic and chromatic stimuli of arbitrary size.

Phase-Dependent Interactions in Visual Cortex to Combinations of First- and Second-Order Stimuli

A fundamental task of the visual system is to extract figure-ground boundaries between objects, which are often defined, not only by differences in luminance, but also by "second-order" contrast or texture differences. Responses of cortical neurons to both first- and second-order patterns have been studied extensively, but only for responses to either type of stimulus in isolation. Here, we examined responses of visual cortex neurons to the spatial relationship between superimposed periodic luminance modulation (LM) and contrast modulation (CM) stimuli, the contrasts of which were adjusted to give equated responses when presented alone. Extracellular single-unit recordings were made in area 18 of the cat, the neurons of which show responses to CM and LM stimuli very similar to those in primate area V2 (Li et al., 2014). Most neurons showed a significant dependence on the relative phase of the combined LM and CM patterns, with a clear overall optimal response when they were approximately phase aligned. The degree of this phase preference, and the contributions of suppressive and/or facilitatory interactions, varied considerably from one neuron to another. Such phase-dependent and phase-invariant responses were evident in both simple- and complex-type cells. These results place important constraints on any future model of the underlying neural circuitry for second-order responses. The diversity in the degree of phase dependence between LM and CM stimuli that we observed could help to disambiguate different kinds of boundaries in natural scenes.

SIGNIFICANCE STATEMENT

Many visual cortex neurons exhibit orientation-selective responses to boundaries defined by differences either in luminance or in texture contrast. Previous studies have examined responses to either type of boundary in isolation, but here we measured systematically responses of cortical neurons to the spatial relationship between superimposed periodic luminance-modulated (LM) and contrast-modulated (CM) stimuli with contrasts adjusted to give equated responses. We demonstrate that neuronal responses to these compound stimuli are highly dependent on the relative phase between the LM and CM components. Diversity in the degree of such phase dependence could help to disambiguate different kinds of boundaries in natural scenes, for example, those arising from surface reflectance changes or from illumination gradients such as shading or shadows.

Responses to second-order texture modulations undergo surround suppression

First-order (contrast) surround suppression has been well characterized both psychophysically and physiologically,but relatively little is known as to whether the perception of second-order visual stimuli exhibits analogous center–surround interactions. Second-order surround suppression was characterized by requiring subjects to detect second-order modulation in stimuli presented alone or embedded in a surround.Both contrast- (CM) and orientation-modulated (OM) stimuli were used. For most subjects and both OM and CM stimuli, second-order surrounds caused thresholds to be higher, indicative of second-order suppression. For CM stimuli, suppression was orientation-specific, i.e., higher thresholds for parallel than for orthogonal surrounds. However, the evidence for orientation specificity of suppression for OM stimuli was weaker. These results suggest that normalization, leading to surround suppression, operates at multiple stages in cortical processing.

Visual chimaeras obtained with the Riesz transform

Similar to an auditory chimaera (Smith et al, 2002 Nature 416 87-90), a visual chimaera can be defined as a synthetic image which has the fine spatial structure of one natural image and the envelope of another image in each spatial frequency band. Visual chimaeras constructed in this way could be useful to vision scientists interested in the study of interactions between first-order and second-order visual processing. Although it is almost trivial to generate 1-D chimaeras by means of the Hilbert transform and the analytic signal, problems arise in multidimensional signals like images given that the partial directional Hilbert transform and current 2-D demodulation algorithms are anisotropic or orientation-variant procedures. Here, we present a computational procedure to synthesise visual chimaeras by means of the Riesz transform--an isotropic generalisation of the Hilbert transform for multidimensional signals--and the associated monogenic signal--the vector-valued function counterpart of the analytic signal in which the Riesz transform replaces the Hilbert transform. Examples of visual chimaeras are shown for same/different category images.

The detection of the motion of contrast modulation: a parametric study

Despite a long and productive history as a focus of research interest, the details of how humans detect motion in an image remain controversial. This debate has not been helped by the lack of a clear parametric description of motion discrimination for some of the more simple visual stimuli employed in the literature to date. With this in mind, in the present work, we examined a peculiarity observed in the perception of the motion of second-order (contrast-modulated) stimuli: Under certain stimulus conditions, there is a reversal in the perceived direction of motion of the pattern. The aim was to quantify this phenomenon, relate the reversal to forward (veridical) and ambiguous motion, and place the behavioral data in the context of the window of visibility model of spatiotemporal contrast sensitivity. The direction of motion of contrast-modulated patterns was measured as a function of temporal frequency and carrier contrast, under different critical stimulus conditions. The stimulus properties manipulated were spatial frequency, spatial-phase relationship of carrier and sidebands, color, duration, and, most critically, the retinal location of the stimulus. On a purely empirical basis, the data reconciled several conflicts in the recent literature. From a theoretical standpoint, the data were well explained by the window of visibility approach in the majority of conditions and were partially explained in the remaining conditions. The results raise some interesting questions about underlying motion detection mechanisms and the assumptions embodied in our approach to motion modeling and the visual system in general. Supplemental materials for this article may be downloaded from app.psychonomic-journals.org/content/supplemental.

Contrast detection in infants with fragile X syndrome

Studies have reported that a selective deficit in visual motion processing is present in certain developmental disorders, including Williams syndrome and autism. More recent evidence suggests a visual motion impairment is also present in adults with fragile X syndrome (FXS), the most common form of inherited mental retardation. The goal of the current study was to examine low-level cortical visual processing in infants diagnosed with FXS in order to explore the developmental origin of this putative deficit. We measured contrast detection of first-order (luminance-defined) and second-order (contrast-defined) gratings at two levels of temporal frequency, 0 Hz (static) and 4 Hz (moving). Results indicate that infants with FXS display significantly higher detection thresholds only for the second-order, moving stimuli compared to mental age-matched typically developing controls.

Flexibility of spatial averaging in visual perception

The classical receptive field (RF) concept-the idea that a visual neuron responds to fixed parts and properties of a stimulus-has been challenged by a series of recent physiological results. Here, we extend these findings to human vision, demonstrating that the extent of spatial averaging in contrast perception is also flexible, depending strongly on stimulus contrast and uniformity. At low contrast, spatial averaging is greatest (about 11 min of arc) within uniform regions such as edges, as expected if the relevant neurons have orientation-selective RFs. At high contrast, spatial averaging is minimal. These results can be understood if the visual system is balancing a trade-off between noise reduction, which favours large areas of averaging, and detail preservation, which favours minimal averaging. Two distinct populations of neurons with hard-wired RFs could account for our results, as could the more intriguing possibility of dynamic, contrast-dependent RFs.

Human cortical responses to contrast modulations of visual noise

We studied visual evoked potentials (VEPs) elicited by second-order contrast modulations of binary dynamic noise and first-order luminance modulations. Using a 3-point Laplacian operator centred on Oz, we found that contrast modulations of both low and higher spatial frequencies elicited a negative component whose latency was about 200 ms. The latency of this component was significantly longer than that of the early Laplacian components to first-order luminance modulations. These findings could be due to slower first-stage linear filters and additional processing stages of the second-order pathway. The topographical analysis of scalp recorded VEPs to central and half-field stimulation has suggested that the responses to second-order patterns are likely to be generated by neuronal structures within the primary visual cortex which may have inputs from extrastriate neurons via feedback connections.

The detection of motion in chromatic stimuli: first-order and second-order spatial structure

This study provides evidence for the existence of a low-level chromatic motion mechanism and further elucidates the conditions under which its operation becomes measurable in an experimental stimulus. Observers discriminated the direction of motion of amplitude modulated (AM) gratings that were defined by luminance or chromatic variation and masked with spatiotemporally broadband luminance or chromatic noise. The size and retinal location of the stimuli were varied and the effects of broadband noise and grating masks were both compared with the cohort of stimuli. Some significant disparities in the published literature were well explained by the results. In conclusion, evidence for a chromatically sensitive motion mechanism that evades the, detrimental effects of a luminance mask was found only at the fovea and only when the stimulus was small and centrally placed.

Depth perception from second-order-motion stimuli yoked to head movement

We examined whether depth perception was produced by the parallax of second-order motion (i.e., movement of non-luminance features, such as flicker, texture size modulation, or contrast modulation that moved in synchrony with lateral head movement). The results, obtained with second-order motion from a simple grating stimuli, showed that depth order was judged correctly with probabilities well above chance, but the reported depth magnitude did not co-vary with parallax magnitude. When we used a complex spatial pattern for which feature tracking was difficult, the accuracy of depth-order judgments descended to chance level. Our results suggest that the visual system (a) can detect the correct depth order by tracking a relative shift in the salient features of a stimulus pattern, but (b) cannot determine depth magnitude from a velocity field given by second-order-motion stimuli.

Poor encoding of position by contrast-defined motion

Second-order (contrast-defined) motion stimuli lead to poor performance on a number of tasks, including discriminating form from motion and visual search. To investigate this deficiency, we tested the ability of human observers to monitor multiple regions for motion, to code the relative positions of shapes defined by motion, and to simultaneously encode motion direction and location. Performance with shapes from contrast-defined motion was compared with that obtained from luminance-defined (first-order) stimuli. When the position of coherent motion was uncertain, direction-discrimination thresholds were elevated similarly for both luminance-defined and contrast-defined motion, compared to when the stimulus location was known. The motion of both luminance- and contrast-defined structure can be monitored in multiple visual field locations. Only under conditions that greatly advantaged contrast-defined motion, were observers able to discriminate the positional offset of shapes defined by either type of motion. When shapes from contrast-defined and luminance-defined motion were presented under comparable conditions, the positional accuracy of contrast-defined motion was found to be poorer than its luminance-defined counterpart. These results may explain some, but possibly not all, of the deficits found previously with second-order motion.

Masking effects of low-frequency sinusoidal gratings on the detection of contrast modulation in high-frequency carriers

A modification and extension of Kortum and Geisler's model [Vision Res. 35, 1595 (1995)] of early visual non-linearities that incorporates an expansive nonlinearity (consistent with neurophysiological findings [Vision Res. 35, 2725 (1995)], a normalization based on a local average retinal illumination, similar to Mach's proposal [F. Ratliff, Mach Bands: Quantitative Studies on Neural Networks in the Retina (Holden-Day, San Francisco, Calif, 1965)], and a subsequent compression suggested by Henning et al. [J. Opt. Soc. Am A 17, 1147 (2000)] captures a range of hitherto unexplained interactions between a sinusoidal grating of low spatial frequency and a contrast-modulated grating 2 octaves higher in spatial frequency.

Sensitivity to contrast modulation: the spatial frequency dependence of second-order vision

We consider the overall shape of the second-order modulation sensitivity function (MSF). Because second-order modulations of local contrast or orientation require a carrier signal, it is necessary to evaluate modulation sensitivity against a variety of carriers before reaching a general conclusion about second-order sensitivity. Here we present second-order sensitivity functions for new carrier types (low pass (1/f) noise, and high pass noise) and demonstrate that, when first-order artefacts have been accounted for, the shape of the resulting MSFs are similar to one another and to those for white and broad band noise. They are all low pass with a likely upper frequency limit in the range 10-20 c/deg, suggesting that detection of second-order stimuli is relatively insensitive to the structure of the carrier signal. This result contrasts strongly with that found for (first-order) luminance modulations of the same noise types. Here the noise acts as mask and each noise type masks most those frequencies that are dominant in its spectrum. Thus the shape of second-order MSFs are largely independent of the spectrum of their noise carrier, but first-order CSFs depend on the spectrum of an additive noise mask. This provides further evidence for the separation of first- and second-order vision and characterises second-order vision as a low pass mechanism.

Is second-order spatial loss in amblyopia explained by the loss of first-order spatial input?

The purpose of the study was to determine whether amblyopes show detection loss for second-order spatial information, and if present, whether the loss is explained by the loss of first-order spatial input. We psychophysically determined detection thresholds for the amblyopic and non-amblyopic eyes of five adult amblyopes and the dominant eyes of three control observers. We found that four amblyopic eyes and two non-amblyopic eyes showed second-order loss relative to the control eyes. The second-order loss was greater than the first-order loss at the carrier spatial frequency (first-order input). The extra second-order loss indicates an early amplification of cortical neural loss that we speculate is due to deficient binocular input to second-order neurons.

Motion of contrast envelopes: peace and noise

We examined the effect of changing the composition of the carrier on the perception of motion in a drifting contrast envelope. Human observers were required to discriminate the direction of motion of contrast modulations of an underlying carrier as a function of temporal frequency and scaled (carrier) contrast. The carriers were modulations of both color and luminance, defined within a cardinal color space. Random-noise carriers had either binary luminance profiles or flat (gray-scale-white) or 1/f (pink) spectral power functions. Independent variables investigated were the envelope spatial frequency and temporal-drift frequency and the fundamental spatial frequency, color, and temporal-update frequency of the carrier. The results show that observers were able to discriminate correctly the direction of envelope motion for binary-noise carriers at both high (16 Hz) and low (2 Hz) temporal-drift frequencies. Changing the carrier format from binary noise to a flat (gray-scale) or 1/f amplitude profile reduced discrimination performance slightly but only in the high-temporal-frequency condition. Manipulation of the fundamental frequency of the carrier elicited no change in performance at the low temporal frequencies but produced ambiguous or reversed motion at the higher temporal frequencies as soon as the fundamental frequency was higher than the envelope modulation frequency. We found that envelope motion detection was sensitive to the structure of the carrier.

Blur tolerance for luminance and chromatic stimuli

We investigated the blur tolerance of human observers for stimuli modulated along the isoluminant red-green, the isoluminant yellow-blue, and the luminance (black-white) direction in color space. We report the following results: (i) Blur difference thresholds for red-green and luminance stimuli (of equal cone contrast) are very similar and as low as 0.5 min of visual angle; for yellow-blue the lowest blur thresholds are much higher (1.5 min of visual angle). (ii) The smallest blur thresholds are found for slightly blurred square waves (reference blur of 1 arc min) and not for sharp edges. (iii) Blur thresholds for red-green and black-white follow a Weber law for reference (pedestal) blurs greater than the optimum blur. (iv) Using the model proposed by Watt and Morgan [Vision Res. 24, 1387 (1984)] we estimated the internal blur of the visual system for the black-white and the red-green color directions and arrived at the following estimates: 1.2 arc min for black-white stimuli at 10% contrast and 0.9 arc min for red-green stimuli at 10% cone contrast. Blur tolerance for yellow-blue is independent of external blur and cannot be predicted by the model. (v) The contrast dependence of blur sensitivity is similar for red-green and luminance modulations (slopes of -0.15 and -0.16 in log-log coordinates, respectively) and slightly stronger for yellow-blue (slope = -0.75). Blur discrimination thresholds are not predicted by the contrast sensitivity function of the visual system. Our findings are useful for predicting blur tolerance for complex images and provide a spatial frequency cutoff point when Gaussian low-pass filters are used for noise removal in colored images. They are also useful as a baseline for the study of visual disorders such as amblyopia.

Nonlocal interactions in color perception: nonlinear processing of chromatic signals from remote inducers

The perceived color of an object depends on the chromaticity of its immediate background. But color appearance is also influenced by remote chromaticities. To quantify these influences, the effects of remote color fields on the appearance of a fixated 2 degrees test field were measured using a forced-choice method. Changes in the appearance of the test field were induced by chromaticity changes of the background and of 2 degrees color fields not adjacent to the test field. The appearance changes induced by the color of the background corresponded to a fraction of between 0.5 and 0.95 of the cone contrast of the background change, depending on the observer. The magnitude of induction by the background color was modulated on average by 7.6% by chromaticity changes in the remote color fields. Chromaticity changes in the remote fields had virtually no inducing effect when they occurred without a change in background color. The spatial range of these chromatic interactions extended over at least 10 degrees from the fovea. They were established within the first few hundred milliseconds after the change of background color and depended only weakly on the number of inducing fields. These results may be interpreted as reflecting rapid chromatic interactions that support robustness of color vision under changing viewing conditions.

Interaction between first- and second-order orientation channels revealed by the tilt illusion: psychophysics and computational modelling

This paper examines the interaction between first- and second-order contours in the orientation domain. Using the simultaneous tilt illusion (TI), we show that the apparent rotation of a vertical test grating away from that of a surrounding inducing grating (repulsion effect) occurs when both the inducing and test grating are either first- or second-order. Furthermore, a significant repulsion effect is obtained when a first-order inducing grating surrounds a second-order test. If lateral inhibitory interactions between populations of orientation selective neurons provides a plausible explanation for orientation repulsion effects [Blakemore, C. B. Carpenter, R. H. S. & Georgeson, M. A. (1970) Nature, 228, 37-39], it is likely that the cue-invariant mechanisms that encodes the orientation of first- and second-order contours also exhibit inhibitory interactions. A two-channel computational model of orientation encoding is presented where one channel encodes only first-order stimuli while the second channel encodes both first- and second-order contours. In addition to predicting the orientation repulsion effects we observed, the model also provides a functional account of orientation attraction effects in terms of the responses of populations of orientation-tuned neurons.

A binocular site for contrast-modulated masking

Contrast-modulated (CM) gratings, composed of two luminance-modulated sinusoids of similar spatial frequency, mask the detection of test sinusoids at the difference frequency. However, the mechanism underlying masking by CM gratings remains poorly understood. In this paper, we aimed to determine whether the masking of 1 cycle deg(-1) LM test gratings by a 1 cycle deg(-1) beat (formed from a pair of carriers at 8 and 9 cycles deg(-1)) occurs in monocular channels or after the site of binocular combination, or both. Threshold elevations for the detection of a 1 cycle deg(-1) test grating were obtained for a number of stimulus conditions, including: (1) dichoptic CM (both 8 and 9 cycles deg(-1) mask components presented to one eye, with the 1 cycle deg(-1) test grating to the other); (2) dichoptic variant (8 and 9 cycles deg(-1) mask gratings presented to separate eyes, with the 1 cycle deg(-1) test grating presented to one eye); (3) binocular CM (all mask and test gratings presented to both eyes). As a control, masking magnitude was also measured for LM mask gratings of similar frequency (1 cycle deg(-1)) and effective contrast (3%) to that of the beat. For both LM and CM masks, the dichoptic condition yielded threshold elevations that were similar or greater than the binocular condition. When 8 and 9 cycles deg(-1) mask components were presented to separate eyes (the dichoptic variant condition), no beat pattern was visible and no elevations in detection threshold occurred. The results demonstrate that, like LM masking, detection of a target in the presence of a CM mask does not involve purely monocular mechanisms. Further, that the site of CM masking must occur beyond the stage at which monocular matching for stereopsis takes place. This is consistent with other studies which suggest that dichoptic masking is contingent on stereo matching, and thus occurs relatively late in the hierarchy of binocular visual processing.

Visual sensitivity to temporal modulations of temporal noise

The present endeavor is meant (a) to provide a direct comparison between first- and second-order temporal modulation and, by so doing, (b) to eliminate all spatial clues that might have contaminated previous assessments of the second-order temporal modulation transfer function (TMTF). The second aim was achieved by means of the temporal modulation of a purely temporal white noise, a stimulus used frequently in psychoacoustics but not used as yet in visual stimulation. Luminance and contrast temporal modulation thresholds were measured with a 2AFC staircase procedure. In the first case, the mean luminance of a spatially homogeneous, 30 degrees field was modulated sinusoidally over time (first-order modulation). In the second case, the luminance of the same or of a 60 degrees field was randomized over time at a rate of 150 Hz and this temporal white noise (the carrier) was modulated sinusoidally over time (second-order modulation). First-order thresholds reproduce the classical (large field) flicker sensitivity. Second-order thresholds (measured for the first time with purely temporal stimuli) are at least 100 times higher than first-order ones, display a low-pass characteristic (at least up to 0.5 Hz) and yield a critical fusion frequency (measured at 100% modulation) of approximately 10 Hz. The data are in accord with other estimates of the TMTF of the second-order system and thus confirm the effective neutralization of the spatial cues present in these previous studies.

Orientation processing mechanisms revealed by the plaid tilt illusion

The tilt after-effect (TAE) and tilt illusion (TI) have revealed a great deal about the nature of orientation coding of 1-dimensional (1D) lines and gratings. Comparatively little research however has addressed the mechanisms responsible for encoding the orientation of 2-dimensional (2D) plaid stimuli. A multi-stage model of edge detection has recently been proposed [Georgeson, M. A. (1998) Image & Vision Computing, 16(6-7), 389-405] to account for the perceived structure of a plaid stimulus that incorporates extraction of the zero-crossings (ZCs) of the plaid. Data is presented showing that the ZCs of a plaid inducing stimulus can interact with vertical grating test stimulus to induce a standard tilt illusion. However, by considering the second-order structure of a plaid rather than ZCs, it was shown that the perceived orientation of the vertical test grating results from the combination of orientation illusions due to the first- and second-order components of an inducing plaid. The data suggest that the mechanisms encoding the orientation of second-order contours are similar to, and interact directly with, those that encode first-order contours.

Comparing contrast-modulated and luminance-modulated masking: effects of spatial frequency and phase

The masking of a sinusoidal test grating by contrast-modulated (CM) gratings could, in principle, be attributable to the presence of a distortion product, injected into the stimulus during some nonlinear transformation at an early level of visual processing (e.g. Nachmias, 1989 Vision Research 29 137-142). If so, CM gratings and luminance-modulated (LM) gratings of similar effective contrast and spatial frequency should mask the detection of sinusoids in a similar fashion. We compared the effects of masking by 1 cycle deg-1 CM gratings [both simple beats (8 + 9 cycles deg-1) and amplitude-modulated gratings (8 + 9 + 10 cycles deg-1)], with those of masking by 1 cycle deg-1 LM gratings of low contrast. We found that: (i) CM and low-contrast LM grating masks yielded similar spatial-frequency tuning functions around the modulation frequency of 1 cycle deg-1; (ii) low-contrast LM gratings masked the detection of test sinusoids in a highly phase-dependent fashion, while masking by CM gratings did not vary systematically with relative spatial phase. The results suggest that masking produced by CM gratings cannot simply be explained by the presence of a distortion product at the beat or modulation frequency.

Sensitivity to contrast modulation depends on carrier spatial frequency and orientation

We consider how the detection of second-order contrast structure depends on the orientation and spatial frequency of first-order luminance structure. For patterns composed of a bandpass noise carrier multiplied by a contrast envelope function, we show that sensitivity to the envelope varies in proportion to the spatial frequency of the carrier. For oriented carriers at low spatial-frequencies, detection of the contrast envelope is easier when the envelope and carrier are perpendicular, but this dependency diminishes as the spatial frequency of the carrier increases. These differences are not attributable to either the detection of side-bands, or the presence of spurious contrast structure in unmodulated carrier images. A final experiment measured envelope detection in the presence of noise masks. Results indicate that orientationally and spatially-band pass filtering precedes the detection of second-order structure.