Nonlinear combination of luminance excursions during flicker, simultaneous contrast, afterimages and binocular fusion

Effects of binocular disparity on binocular luminance combination

This study aimed to examine the effects of binocular disparity on binocular combination of brightness information coming from luminance increments and decrements. The point of subjective equality was determined by asking the observers to judge which stimulus appeared brighter-a bar stimulus with variable disparity or another stimulus with zero disparity. For the bar stimulus, the interocular luminance ratio was varied to trace an equal brightness curve. Binocular disparity had no effect on luminance increments presented on a gray or black background. In contrast, when luminance decrements were presented on a gray background, non-zero disparities elevated points of subjective equality for stimuli with interocular luminance differences. This means that the binocular brightness combination of the two monocular signals shifted from winner-take-all summation toward linear averaging. It has been argued that this effect may be caused by non-zero binocular disparities attenuating interocular suppression, which is deemed to operate normally with zero disparity.

Different rules for binocular combination of luminance flicker in cortical and subcortical pathways

How does the human brain combine information across the eyes? It has been known for many years that cortical normalization mechanisms implement 'ocularity invariance': equalizing neural responses to spatial patterns presented either monocularly or binocularly. Here, we used a novel combination of electrophysiology, psychophysics, pupillometry, and computational modeling to ask whether this invariance also holds for flickering luminance stimuli with no spatial contrast. We find dramatic violations of ocularity invariance for these stimuli, both in the cortex and also in the subcortical pathways that govern pupil diameter. Specifically, we find substantial binocular facilitation in both pathways with the effect being strongest in the cortex. Near-linear binocular additivity (instead of ocularity invariance) was also found using a perceptual luminance matching task. Ocularity invariance is, therefore, not a ubiquitous feature of visual processing, and the brain appears to repurpose a generic normalization algorithm for different visual functions by adjusting the amount of interocular suppression.

Perceptual enhancement of suprathreshold luminance modulation in stereoscopic patterns

Previous studies have shown that the binocular summation of luminance contrast signals depends on the parameters involved in stereopsis when the luminance contrast is at the detection threshold. However, less attention has been paid to the perception of luminance modulation in stereoscopic patterns at suprathreshold contrast. To address this issue, we determined the contrast of stereoscopic patterns at the perceptual match to a standard contrast as a function of binocular disparity. The matched contrast was close to the standard contrast at 0 degrees disparity, but decreased as disparity deviated from 0 degrees, suggesting that sufficient disparity perceptually enhances luminance contrast. The reduction of matched contrast was more evident for uncrossed disparities than for crossed disparities, which almost disappeared when the contrast was near the threshold and also occurred when vertical disparity was introduced. We argue that the perceptual enhancement of the luminance contrast is due to the weaker interocular suppression for stimuli with large disparities.

Salience-Based Edge Selection in Flicker and Binocular Color Vision

A test cross that flickers between light yellow and dark blue at 5 to 8Hz looks apparently yellow on a dark gray surround and apparently blue on a light gray surround (flicker augmented contrast). The achromatic surround cannot be inducing the perceived colors. Instead, the visual system selects the more salient apparent color with the higher Michelson contrast. The same is true for dichoptic vision. When one eye views a steady, light yellow cross and the other eye views a congruent steady dark blue cross, the binocular combination of colors looks apparently yellow on a dark gray surround and apparently blue on a light gray surround. Thus, when competing stimuli are distributed over time (flicker) or space (dichoptic vision), the visual system overweights the stimulus with the higher contrast. To see objects clearly, we accept the best view of any object and downplay inferior alternatives.

Deadeye Visualization Revisited: Investigation of Preattentiveness and Applicability in Virtual Environments

Visualizations rely on highlighting to attract and guide our attention. To make an object of interest stand out independently from a number of distractors, the underlying visual cue, e.g., color, has to be preattentive. In our prior work, we introduced Deadeye as an instantly recognizable highlighting technique that works by rendering the target object for one eye only. In contrast to prior approaches, Deadeye excels by not modifying any visual properties of the target. However, in the case of 2D visualizations, the method requires an additional setup to allow dichoptic presentation, which is a considerable drawback. As a follow-up to requests from the community, this paper explores Deadeye as a highlighting technique for 3D visualizations, because such stereoscopic scenarios support dichoptic presentation out of the box. Deadeye suppresses binocular disparities for the target object, so we cannot assume the applicability of our technique as a given fact. With this motivation, the paper presents quantitative evaluations of Deadeye in VR, including configurations with multiple heterogeneous distractors as an important robustness challenge. After confirming the preserved preattentiveness (all average accuracies above 90%) under such real-world conditions, we explore VR volume rendering as an example application scenario for Deadeye. We depict a possible workflow for integrating our technique, conduct an exploratory survey to demonstrate benefits and limitations, and finally provide related design implications.

Deadeye: A Novel Preattentive Visualization Technique Based on Dichoptic Presentation

Preattentive visual features such as hue or flickering can effectively draw attention to an object of interest - for instance, an important feature in a scientific visualization. These features appear to pop out and can be recognized by our visual system, independently from the number of distractors. Most cues do not take advantage of the fact that most humans have two eyes. In cases where binocular vision is applied, it is almost exclusively used to convey depth by exposing stereo pairs. We present Deadeye, a novel preattentive visualization technique based on presenting different stimuli to each eye. The target object is rendered for one eye only and is instantly detected by our visual system. In contrast to existing cues, Deadeye does not modify any visual properties of the target and, thus, is particularly suited for visualization applications. Our evaluation confirms that Deadeye is indeed perceived preattentively. We also explore a conjunction search based on our technique and show that, in contrast to 3D depth, the task cannot be processed in parallel.

Binocular combination of luminance profiles

We develop and test a new two-dimensional model for binocular combination of the two eyes' luminance profiles. For first-order stimuli, the model assumes that one eye's luminance profile first goes through a luminance compressor, receives gain-control and gain-enhancement from the other eye, and then linearly combines the other eye's output profile. For second-order stimuli, rectification is added in the signal path of the model before the binocular combination site. Both the total contrast and luminance energies, weighted sums over both the space and spatial-frequency domains, were used in the interocular gain-control, while only the total contrast energy was used in the interocular gain-enhancement. To challenge the model, we performed a binocular brightness matching experiment over a large range of background and target luminances. The target stimulus was a dichoptic disc with a sharp edge that has an increment or decrement luminance from its background. The disk's interocular luminance ratio varied from trial to trial. To refine the model we tested three luminance compressors, five nested binocular combination models (including the Ding–Sperling and the DSKL models), and examined the presence or absence of total luminance energy in the model. We found that (1) installing a luminance compressor, either a logarithmic luminance function or luminance gain-control, (2) including both contrast and luminance energies, and (3) adding interocular gain-enhancement (the DSKL model) to a combined model significantly improved its performance. The combined model provides a systematic account of binocular luminance summation over a large range of luminance input levels. It gives a unified explanation of Fechner's paradox observed on a dark background, and a winner-take-all phenomenon observed on a light background. To further test the model, we conducted two additional experiments: luminance summation of discs with asymmetric contour information (Experiment 2), similar to Levelt (1965) and binocular combination of second-order contrast-modulated gratings (Experiment 3). We used the model obtained in Experiment 1 to predict the results of Experiments 2 and 3 and the results of our previous studies. Model simulations further refined the contrast space weight and contrast sensitivity functions that are installed in the model, and provide a reasonable account for rebalancing of imbalanced binocular vision by reducing the mean luminance in the dominant eye.

Measurement of crosstalk in stereoscopic display systems used for vision research

Studying binocular vision requires precise control over the stimuli presented to the left and right eyes. A popular technique is to segregate signals either temporally (frame interleaving), spectrally (using colored filters), or through light polarization. None of these segregation methods achieves perfect isolation, and so a degree of crosstalk is usually apparent, in which signals intended for one eye are faintly visible to the other eye. Previous studies have reported crosstalk values mostly for consumer-grade systems. Here we measure crosstalk for eight systems, many of which are intended for use in vision research. We provide benchmark crosstalk values, report a negative crosstalk effect in some LCD-based systems, and give guidelines for dealing with crosstalk in different experimental paradigms.

Negative Afterimages From Flicker-Augmented Colors

A patch that alternates between two hues such as dark green and light blue looks greenish on a light gray surround and bluish on a dark gray surround ("flicker-augmented contrast"). Thus, when an edge alternates between two hues in the same location, the visual system selects the more salient hue-the one with the higher Michelson contrast. However, the afterimage is the same pink, driven by the time integral of the physical, not the perceptual, adapting hues and regardless of the surround luminance. So the process of edge biasing does not transfer to the mechanism that creates afterimages.

Binocular combination of second-order stimuli

Phase information is a fundamental aspect of visual stimuli. However, the nature of the binocular combination of stimuli defined by modulations in contrast, so-called second-order stimuli, is presently not clear. To address this issue, we measured binocular combination for first- (luminance modulated) and second-order (contrast modulated) stimuli using a binocular phase combination paradigm in seven normal adults. We found that the binocular perceived phase of second-order gratings depends on the interocular signal ratio as has been previously shown for their first order counterparts; the interocular signal ratios when the two eyes were balanced was close to 1 in both first- and second-order phase combinations. However, second-order combination is more linear than previously found for first-order combination. Furthermore, binocular combination of second-order stimuli was similar regardless of whether the carriers in the two eyes were correlated, anti-correlated, or uncorrelated. This suggests that, in normal adults, the binocular phase combination of second-order stimuli occurs after the monocular extracting of the second-order modulations. The sensory balance associated with this second-order combination can be obtained from binocular phase combination measurements.

Optimizing the temporal dynamics of light to human perception

No previous research has tuned the temporal characteristics of light-emitting devices to enhance brightness perception in human vision, despite the potential for significant power savings. The role of stimulus duration on perceived contrast is unclear, due to contradiction between the models proposed by Bloch and by Broca and Sulzer over 100 years ago. We propose that the discrepancy is accounted for by the observer's "inherent expertise bias," a type of experimental bias in which the observer's life-long experience with interpreting the sensory world overcomes perceptual ambiguities and biases experimental outcomes. By controlling for this and all other known biases, we show that perceived contrast peaks at durations of 50-100 ms, and we conclude that the Broca-Sulzer effect best describes human temporal vision. We also show that the plateau in perceived brightness with stimulus duration, described by Bloch's law, is a previously uncharacterized type of temporal brightness constancy that, like classical constancy effects, serves to enhance object recognition across varied lighting conditions in natural vision-although this is a constancy effect that normalizes perception across temporal modulation conditions. A practical outcome of this study is that tuning light-emitting devices to match the temporal dynamics of the human visual system's temporal response function will result in significant power savings.

Nonlinearities in the binocular combination of luminance and contrast

We studied the rules by which visual responses to luminous targets are combined across the two eyes. Previous work has found very different forms of binocular combination for targets defined by increments and by decrements of luminance, with decrement data implying a severe nonlinearity before binocular combination. We ask whether this difference is due to the luminance of the target, the luminance of the background, or the sign of the luminance excursion. We estimated the pre-binocular nonlinearity (power exponent) by fitting a computational model to ocular equibrightness matches. The severity of the nonlinearity had a monotonic dependence on the signed difference between target and background luminance. For dual targets, in which there was both a luminance increment and a luminance decrement (e.g. contrast), perception was governed largely by the decrement. The asymmetry in the nonlinearities derived from the subjective matching data made a clear prediction for visual performance: there should be more binocular summation for detecting luminance increments than for detecting luminance decrements. This prediction was confirmed by the results of a subsequent experiment. We discuss the relation between these results and luminance nonlinearities such as a logarithmic transform, as well as the involvement of contemporary model architectures of binocular vision.

Binocular fusion of luminance, color, motion and flicker--two eyes are worse than one

How much information is conserved, or discarded, as it travels through the visual system? A target (an oriented bar) was defined by dark grey spots embedded in light spots for the left eye, and by light grey spots embedded in dark spots for the right eye. With one eye open, the target bars were clearly visible, but with both eyes open the light and dark spots fused binocularly into medium grey and the target vanished. Results were similar for color; the target comprised greenish spots embedded in reddish spots for the left eye, and by reddish spots embedded in greenish spots for the right eye (somewhat like an Isihara color plate).

RESULT

The colored targets were invisible when both eyes were open but reappeared when one eye was closed. Small targets that moved in opposite directions in the two eyes were visible to each eye alone by common fate, but motion averaging made them disappear when both eyes were open.

CONCLUSION

Opposed retinal luminances or colors were averaged out by binocular fusion, but could be retrieved by a special afterimage technique in Experiment 6. Conversely, in Experiment 7 dichoptic target spots flickered in counterphase but background spots flickered in-phase to the two eyes.

RESULT

The targets were invisible monocularly but became visible as reduced-flicker when fused binocularly. We conclude that two eyes were worse than one eye when opposite colors or movements were fused (Experiments 1-6) but were better than one when binocular correlations could be extracted (Experiment 7). These experiments show how much of the visual information gets transmitted, gets discarded, can still be retrieved, or reaches conscious awareness.

The role of layered scene representations in color appearance

The chromatic appearance of a surface depends on its surrounding scene. A variety of mechanisms have been proposed to account for such phenomena, ranging from low-level gain control or adaptation processes that adjust for such properties as a scene's chromatic mean and covariance structure, to higher-level computations that compensate for the chromatic content of the illuminant. Despite their differences, a shared prediction of all such processes is that color induction should be limited to a full discounting of the surround or illuminant color--that is, an opponent color shift equal in magnitude to the chromatic bias of the surround or illuminant. Here, we report new forms of chromatic induction that can be significantly larger than predicted by all such models. We show that when the geometric and chromatic relationships between a target and its surround support a decomposition of an image into multiple layers, the induced color can significantly exceed the full discounting prediction. Similar phenomena are also observed with achromatic stimuli, suggesting that common processes of perceptual decomposition are involved in both forms of induction. These results demonstrate that information about the geometric and photometric relationship between a target and its surround is utilized by the mechanisms involved in color induction.

The psychophysics of detecting binocular discrepancies of luminance

In the natural world, a binocular discrepancy of luminance can signal a glossy surface. Using a spatial forced choice task, we have measured the ability of subjects to detect binocular luminance disparities. We show that the detection of binocular luminance disparity shares several basic psychophysical features with the detection of surface properties such as lightness and chromaticity: an approximation to Weber's Law, spatial summation, temporal summation, and a deterioration with increasing eccentricity. We also discuss whether color-deficient subjects could derive reliable information about chromaticity from the binocular disparities of luminance induced by a monocularly worn color filter.

Induced temporal variation at frequencies not in the stimulus: evidence for a neural nonlinearity

The perceived color of a light depends on surrounding light. When the surround varies slowly over time, a central, physically steady light is perceived to vary also. This perceived temporal variation of the central region, however, is strongly attenuated when the surround varies faster than approximately 3 Hz (R. L. De Valois, M. A. Webster, K. K. De Valois, & B. Lingelbach, 1986). The classical explanation is low-pass temporal filtering at a cortical stage that attenuates the neural representation of temporal frequencies capable of causing induced temporal variation. This theory assumes neural responses are linear, so only temporal frequencies in the stimulus are represented in the neural response. The current experiments revealed that temporal frequencies above 3 Hz are capable of inducing temporal variation. Specifically, with two temporal frequencies superimposed in the surround, the induced temporal variation in the uniform region is at the difference frequency of these two frequencies, even though this frequency is not physically present in the stimulus. The results are accounted for by a nonlinear neural process, which causes temporal variation at the difference frequency, and a following linear temporal filter.

Overlapping features can be parsed on the basis of rapid temporal cues that produce stable emergent percepts

When two overlapping displays alternate rapidly, it is difficult to resolve the temporal coincidence of objects, parts, or features. However, under certain conditions (at least for luminance-based stimuli) rapid temporal coincidence can be detected on the basis of stable emergent percepts in which parts that oscillate in phase appear more strongly grouped than (or appear distinct from) parts that oscillate out of phase. These emergent percepts appear as depth segregation, enhanced slow orientation rivalry, and oriented shimmer (a new phenomenon that cannot be explained in terms of conventional apparent motion or temporal contrast illusions). These percepts resulted in up to an eightfold decrease in the coincidence detection threshold (alternations as fast as 20 ms/frame or 25 Hz) relative to control conditions that did not yield them; these sensitivity enhancements are unlikely to be due to temporal probability summation. The results provide psychophysical evidence that temporal-phase information can contribute to the parsing of overlapping patterns.

A purely temporal transparency mechanism in the visual system

I report evidence for a purely temporal perceptual transparency mechanism. Rapid alternation of two images in the same location can result in the simultaneous experience of both, accompanied by a sense of transparency. This is true even when the sum of the two images does not appear transparent, which suggests that the percept is not mediated by the static transparency processes. At slow rates, alternating gratings were experienced as successive. As the rate was increased, by 8 Hz observers experienced the gratings as simultaneous. The rapidly alternating gratings are apparently processed separately before being combined for awareness by a process that integrates over about 120 ms. A final experiment tested whether the common presentation time of different parts of an image in alternation with another would cause the parts to perceptually bind. Observers did not distinguish between a rapidly alternating intact grating display and one in which halves of the display were exchanged in time. In other words, temporal binding across space did not occur. The temporal transparency phenomenon, in addition to informing theories of transparency and the dynamics of visual processing, may also be useful for the creation of transparent displays for electronic devices.

Monocular lustre from flicker

A spot that flickers at 16 Hz between two luminance levels (on a grey surround) has an appearance of metallic lustre, which we call 'monocular lustre'. Binocular and monocular lustre were measured in comparable conditions by a rating procedure, and both were reported only when the light and dark values of the flickering (or binocularly fused) spot straddled the surround luminance, so that the spot was alternately brighter and darker than the surround. We attribute lustre to competition between ON and OFF visual pathways.

Luminance processing in apparent motion, Vernier offset and stereoscopic depth

We obtained (apparently) linear responses to luminance from three special displays of apparent motion, Vernier offset and stereoscopic depth. In our motion stimulus a dark and a light bar exchanged luminances repetitively on a grey surround. Motion was attributed to the bar that differed more from the surround, that is, on a dark surround the light bar appeared to jump, and on a light surround the dark bar appeared to jump. The apparent motion disappeared when the luminance of the surround lay halfway between that of the bars--on a linear, not a logarithmic scale. Similar results were obtained for special Vernier offset and stereo stimuli. These results cannot be explained if all luminances are processed within the same luminance pathway and that pathway transforms input luminance using non-linear compression. However, the apparent linearity of our results could arise from opposite and equal non-linearities cancelling out within separate ON- and OFF-spatial luminance pathways. A second set of experiments presented one bar separately into each eye on different surrounds (dichoptic presentation of competing apparent motion signals) or manipulated the display spatially so that different surrounds were associated with different bars (binocular presentation of competing Vernier targets). Results showed that apparent motion and Vernier signals of equal Weber contrast (normalisation of linear difference to surround luminance) evoked equal-motion and equal Vernier offset strengths. Given that motion and Vernier strength followed Weber's law, we infer that the ON- and OFF-pathways transform luminance non-linearly. Our third experiment presents an example of a brightness bisection task in which we were able to influence the bisection steps, to follow either a linear or non-linear series. The benefits of parsing the visual scene so that visual information is processed within two opposite luminance pathways is discussed.

Neural dynamics of binocular brightness perception

How does the visual cortex combine information from both eyes to generate perceptual representations of object surfaces? Important clues about this process may be derived from data about the perceived brightness of surface regions under binocular viewing conditions, including data about binocular brightness summation in response to Ganzfelds, the U-shaped data of Fechner's paradox that violates binocular brightness summation, and the effects of different combinations of monocular and binocular contours and surface luminance differences on threshold sensitivity to monocular flashes of light. How to reconcile these apparently contradictory data properties has been a severe challenge to previous models, and none has explained them all. The present article quantitatively simulates them all by further developing the FACADE vision model. Key model processes discount the illuminant and compute image contrasts in each monocular channel using shunting on-center off-surround networks; binocularly fuse these discounted monocular signals using shunting on-center off-surround networks with nonlinear excitatory and inhibitory signals; and use these binocularly fused activities to trigger filling-in of a binocular surface representation that represents perceived surface brightness. Previous models that have suggested explanations of subsets of these data are discussed.