Explaining brightness illusions using spatial filtering and local response normalization

What Fechner could not do: Separating perceptual encoding and decoding with difference scaling

A key question in perception research is how stimulus variations translate into perceptual magnitudes, that is, the perceptual encoding process. As experimenters, we cannot probe perceptual magnitudes directly, but infer the encoding process from responses obtained in a psychophysical experiment. The most prominent experimental technique to measure perceptual appearance is matching, where observers adjust a probe stimulus to match a target in its appearance along the dimension of interest. The resulting data quantify the perceived magnitude of the target in physical units of the probe, and are thus an indirect expression of the underlying encoding process. In this paper, we show analytically and in simulation that data from matching tasks do not sufficiently constrain perceptual encoding functions, because there exist an infinite number of pairs of encoding functions that generate the same matching data. We use simulation to demonstrate that maximum likelihood conjoint measurement (Ho, Landy, & Maloney, 2008; Knoblauch & Maloney, 2012) does an excellent job of recovering the shape of ground truth encoding functions from data that were generated with these very functions. Finally, we measure perceptual scales and matching data for White's effect (White, 1979) and show that the matching data can be predicted from the estimated encoding functions, down to individual differences.

Vertical anisotropy in lightness perception not caused by lighting assumption

Our recent study found an illusory effect whereby an image of an upward-facing gray panel appears darker than its 180-degree rotated image. We attributed this inversion effect to the observer's implicit assumption that light from above is more intense than light from below. This paper aims to explore the possibility that low-level visual anisotropy also contributes to the effect. In Experiment 1, we investigated whether the effect could be observed even when the position, the contrast polarity, and the existence of the edge were manipulated. In Experiments 2 and 3, the effect was further examined using stimuli that contained no depth cues. Experiment 4 confirmed the effect using stimuli of even simpler configuration. The results of all the experiments demonstrated that brighter edges on the upper side of the target make it appear lighter, indicating that low-level anisotropy contributes to the inversion effect, even without depth orientation information. However, darker edges on the upper side of the target produced ambiguous results. We speculate that the perceived lightness of the target might be affected by two kinds of vertical anisotropy, one of which is dependent on contrast polarity while the other is independent of it. Moreover, the results also replicated the previous finding that the lighting assumption contributes to perceived lightness. Overall, the present study demonstrates that both low-level vertical anisotropy and mid-level lighting assumption influence lightness.

Empirical evaluation of computational models of lightness perception

Lightness of a surface depends not only on its physical characteristics, but also on the properties of the surrounding context. As a result, varying the context can significantly alter surface lightness, an effect exploited in many lightness illusions. Computational models can produce outcomes similar to human illusory percepts, allowing for demonstrable assessment of the applied mechanisms and principles. We tested 8 computational models on 13 typical displays used in lightness research (11 Illusions and 2 Mondrians), and compared them with results from human participants (N = 85). Results show that HighPass and MIR models predict empirical results for simultaneous lightness contrast (SLC) and its close variations. ODOG and its newer variants (ODOG-2 and L-ODOG) in addition to SLC displays were able to predict effect of White's illusion. RETINEX was able to predict effects of both SLC displays and Dungeon illusion. Dynamic decorrelation model was able to predict obtained effects for all tested stimuli except two SLC variations. Finally, FL-ODOG model was best at simulating human data, as it was able to predict empirical results for all displays, bar the Reversed contrast illusion. Finally, most models underperform on the Mondrian displays that represent most natural stimuli for the human visual system.

Fixational eye movements enable robust edge detection

Human vision relies on mechanisms that respond to luminance edges in space and time. Most edge models use orientation-selective mechanisms on multiple spatial scales and operate on static inputs assuming that edge processing occurs within a single fixational instance. Recent studies, however, demonstrate functionally relevant temporal modulations of the sensory input due to fixational eye movements. Here we propose a spatiotemporal model of human edge detection that combines elements of spatial and active vision. The model augments a spatial vision model by temporal filtering and shifts the input images over time, mimicking an active sampling scheme via fixational eye movements. The first model test was White's illusion, a lightness effect that has been shown to depend on edges. The model reproduced the spatial-frequency-specific interference with the edges by superimposing narrowband noise (1–5 cpd), similar to the psychophysical interference observed in White's effect. Second, we compare the model's edge detection performance in natural images in the presence and absence of Gaussian white noise with human-labeled contours for the same (noise-free) images. Notably, the model detects edges robustly against noise in both test cases without relying on orientation-selective processes. Eliminating model components, we demonstrate the relevance of multiscale spatiotemporal filtering and scale-specific normalization for edge detection. The proposed model facilitates efficient edge detection in (artificial) vision systems and challenges the notion that orientation-selective mechanisms are required for edge detection.

Simple Assumptions to Improve Markov Illuminance and Reflectance

Murray recently introduced a novel computational lightness model, Markov illuminance and reflectance (MIR). MIR is a promising new approach that simulates human lightness processing using a conditional random field (CRF) where natural-scene statistics of reflectance and illumination are implemented. Although MIR can account for various lightness illusions and phenomena, it has limitations, such as the inability to predict reverse-contrast phenomena. In this study, we improved MIR performance by modifying its inference process, the prior on X-junctions, and that on general illumination changes. Our modified model improved predictions for Checkerboard assimilation, the simplified Checkershadow and its control figure, the influence of luminance noise, and White's effect and its several variants. In particular, White's effect is a partial reverse contrast that is challenging for computational models, so this improvement is a significant advance for the MIR framework. This study showed the high extensibility and potential of MIR, which shows the promise for further sophistication.

A discrete magno-parvo additive model in early vision for explaining brightness perception in varying contrastive contexts

A varying contrastive context filter (VCCF)-based model of brightness perception has been proposed. It is motivated first by a recently proposed difference of difference-of-Gaussian (DDOG) filter. Alongside, it is also inspired from the fact that the nature evolves various discrete systems and mechanisms to carry out many of its complex tasks. A weight factor, used for the linear combination of two filters representing the magnocellular and parvocellular channels in the central visual pathway, has been defined and termed as the factor of contrastive context (FOCC) in the present model. This is a binary variable that lends a property of discretization to the DDOG filter. By analyzing important brightness contrast as well as brightness assimilation illusions, we arrive at the minimal set of values (only two) for FOCC, using which one is able to successfully predict the direction of brightness shift in both situations of brightness contrast, claimed and categorized here as low contrastive context, and those of brightness assimilation, claimed and categorized here as high contrastive context perception, depending upon whether the initial M-channel-filtered stimulus is above or below a threshold of the contrastive context. As distinct from Michelson/Weber/RMS contrast, high or low, the contrastive context claimed is dependent on the edge information in the stimulus determined by the Laplacian operator, also used in the DDOG model. We compared the proposed model against the already well-established oriented difference-of-Gaussian (ODOG) model of brightness perception. Extensive simulations suggest that though for most illusions both ODOG and VCCF produce correct output, for certain intricate cases in which the ODOG filter fails to correctly predict the illusory effect, our proposed VCCF model continues to remain effective.

Lightness Perception in Complex Scenes

Lightness perception is the perception of achromatic surface colors: black, white, and shades of grey. Lightness has long been a central research topic in experimental psychology, as perceiving surface color is an important visual task but also a difficult one due to the deep ambiguity of retinal images. In this article, I review psychophysical work on lightness perception in complex scenes over the past 20 years, with an emphasis on work that supports the development of computational models. I discuss Bayesian models, equivalent illumination models, multidimensional scaling, anchoring theory, spatial filtering models, natural scene statistics, and related work in computer vision. I review open topics in lightness perception that seem ready for progress, including the relationship between lightness and brightness, and developing more sophisticated computational models of lightness in complex scenes. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Home Textile Pattern Emotion Labeling Using Deep Multi-View Feature Learning

Different home textile patterns have different emotional expressions. Emotion evaluation of home textile patterns can effectively improve the retrieval performance of home textile patterns based on semantics. It can not only help designers make full use of existing designs and stimulate creative inspiration but also help users select designs and products that are more in line with their needs. In this study, we develop a three-stage framework for home textile pattern emotion labeling based on artificial intelligence. To be specific, first of all, three kinds of aesthetic features, i.e., shape, texture, and salient region, are extracted from the original home textile patterns. Then, a CNN (convolutional neural network)-based deep feature extractor is constructed to extract deep features from the aesthetic features acquired in the previous stage. Finally, a novel multi-view classifier is designed to label home textile patterns that can automatically learn the weight of each view. The three-stage framework is evaluated by our data and the experimental results show its promising performance in home textile patterns labeling.

Fully multi-target segmentation for breast ultrasound image based on fully convolutional network

Ultrasound image segmentation plays an important role in computer-aided diagnosis of breast cancer. Existing approaches focused on extracting the tumor tissue to characterize the tumor class. However, other tissues are also helpful for providing the references. In this paper, a multi-target semantic segmentation approach is proposed based on the fully convolutional network for segmenting the breast ultrasound image into different target tissue regions. For handling the uncertain affiliation of pixels in blurry boundaries, the certain outputs of pixel characteristics in AlexNet are transformed into the fuzzy decision expression. For improving the image detail representation, the AlexNet network structure of fully convolutional network is optimized with fully connected skip structure. In addition, the output of net model is optimized with fully connected conditional random field to improve the characterization of spatial consistency and pixels' correlation of the image. Moreover, a data training optimization method is developed for improving the efficiency of network training. In the experiment, 325 ultrasound images and four error metrics are utilized for validating the segmentation performance. Comparing with existing methods, experimental results show that the proposed approach is effective for handling the breast ultrasound images accurately and reliably. Graphical abstract.

Lightness induction enhancements and limitations at low frequency modulations across a variety of stimulus contexts

Lightness illusions are often studied under static viewing conditions with figures varying in geometric design, containing different types of perceptual grouping and figure-ground cues. A few studies have explored the perception of lightness induction while modulating lightness illusions continuously in time, where changes in perceived lightness are often linked to the temporal modulation frequency, up to around 2–4 Hz. These findings support the concept of a cut-off frequency for lightness induction. However, another critical change (enhancement) in the magnitude of perceived lightness during slower temporal modulation conditions has not been addressed in previous temporal modulation studies. Moreover, it remains unclear whether this critical change applies to a variety of lightness illusion stimuli, and the degree to which different stimulus configurations can demonstrate enhanced lightness induction in low modulation frequencies. Therefore, we measured lightness induction strength by having participants cancel out any perceived modulation in lightness detected over time within a central target region, while the surrounding context, which ultimately drives the lightness illusion, was viewed in a static state or modulated continuously in time over a low frequency range (0.25–2 Hz). In general, lightness induction decreased as temporal modulation frequency was increased, with the strongest perceived lightness induction occurring at lower modulation frequencies for visual illusions with strong grouping and figure-ground cues. When compared to static viewing conditions, we found that slow continuous surround modulation induces a strong and significant increase in perceived lightness for multiple types of lightness induction stimuli. Stimuli with perceptually ambiguous grouping and figure-ground cues showed weaker effects of slow modulation lightness enhancement. Our results demonstrate that, in addition to the existence of a cut-off frequency, an additional critical temporal modulation frequency of lightness induction exists (0.25–0.5 Hz), which instead maximally enhances lightness induction and seems to be contingent upon the prevalence of figure-ground and grouping organization.

p 1-FP: Extraction, Classification, and Prediction of Website Fingerprints with Deep Learning

Recent advances in Deep Neural Network (DNN) architectures have received a great deal of attention due to their ability to outperform state-of-the-art machine learning techniques across a wide range of application, as well as automating the feature engineering process. In this paper, we broadly study the applicability of deep learning to website fingerprinting. First, we show that unsupervised DNNs can generate lowdimensional informative features that improve the performance of state-of-the-art website fingerprinting attacks. Second, when used as classifiers, we show that they can exceed performance of existing attacks across a range of application scenarios, including fingerprinting Tor website traces, fingerprinting search engine queries over Tor, defeating fingerprinting defenses, and fingerprinting TLS-encrypted websites. Finally, we investigate which site-level features of a website influence its fingerprintability by DNNs.

Color induction in equiluminant flashed stimuli

Color induction is the influence of the surrounding color (inducer) on the perceived color of a central region. There are two different types of color induction: color contrast (the color of the central region shifts away from that of the inducer) and color assimilation (the color shifts towards the color of the inducer). Several studies on these effects have used uniform and striped surrounds, reporting color contrast and color assimilation, respectively. Other authors [J. Vis.12(1), 22 (2012)1534-736210.1167/12.12.1] have studied color induction using flashed uniform surrounds, reporting that the contrast is higher for shorter flash duration. Extending their study, we present new psychophysical results using both flashed and static (i.e., non-flashed) equiluminant stimuli for both striped and uniform surrounds. Similarly to them, for uniform surround stimuli we observed color contrast, but we did not obtain the maximum contrast for the shortest (10 ms) flashed stimuli, but for 40 ms. We only observed this maximum contrast for red, green, and lime inducers, while for a purple inducer we obtained an asymptotic profile along the flash duration. For striped stimuli, we observed color assimilation only for the static (infinite flash duration) red-green surround inducers (red first inducer, green second inducer). For the other inducers' configurations, we observed color contrast or no induction. Since other studies showed that non-equiluminant striped static stimuli induce color assimilation, our results also suggest that luminance differences could be a key factor to induce it.

What image features guide lightness perception?

Lightness constancy is the ability to perceive black and white surface colors under a wide range of lighting conditions. This fundamental visual ability is not well understood, and current theories differ greatly on what image features are important for lightness perception. Here we measured classification images for human observers and four models of lightness perception to determine which image regions influenced lightness judgments. The models were a high-pass-filter model, an oriented difference-of-Gaussians model, an anchoring model, and an atmospheric-link-function model. Human and model observers viewed three variants of the argyle illusion (Adelson, 1993) and judged which of two test patches appeared lighter. Classification images showed that human lightness judgments were based on local, anisotropic stimulus regions that were bounded by regions of uniform lighting. The atmospheric-link-function and anchoring models predicted the lightness illusion perceived by human observers, but the high-pass-filter and oriented-difference-of-Gaussians models did not. Furthermore, all four models produced classification images that were qualitatively different from those of human observers, meaning that the model lightness judgments were guided by different image regions than human lightness judgments. These experiments provide a new test of models of lightness perception, and show that human observers' lightness computations can be highly local, as in low-level models, and nevertheless depend strongly on lighting boundaries, as suggested by midlevel models.

A parsimonious model of brightness induction

We present a parsimonious model of brightness induction which can account for various brightness illusions of both brightness-contrast and brightness-assimilation types. Our model is based on a difference of difference-of-Gaussian filter and a two-pass model of attentive vision based on the parallel channels in the central visual pathway. It overcomes some of the problems that could not be addressed by the well-known oriented difference of Gaussian model like those associated with Mach band and checkerboard illusions. This model attempts to provide insight to the mechanism of attention in brightness perception through the two major complimentary visual channels, viz. the magnocellular and the parvocellular.

Remote controls illusion: strange interactions across space cannot be explained by simple contrast filters

The visual system has separable visual encoding for luminance and for contrast modulation [J. Vis.8(1), B152 (2008)1534-736210.1167/8.6.1]; the two dimensions can be represented with a luminance contrast versus luminance plane. Here we use a contrast asynchrony paradigm to explore contextual effects on luminance contrast modulation: two identical rectangular bars (0.5°×2.5°) have luminance levels that modulate at 2 Hz; when one bar is placed on a bright field and the other bar on a dark field, observers perceive the bars modulating in antiphase with each other and yet becoming light and dark at the same time. The antiphase perception corresponds to the change in contrast between the bars and their surrounds (a change along the contrast axis of the plane); the in-phase perception corresponds to the luminance modulation (a change along the luminance axis of the plane). We examine spatial interaction by adding bright rectangular (0.5°×2.5°) flankers on both sides of the dark-field bar and dark flankers on both sides of the bright-field bar. Remarkably, flankers produce an in-phase appearance when separated from the bars by between 2^' and 4^' of visual angle, and produce antiphase appearance when they directly adjoin the bars or are separated by more than 8^'. To estimate the dimensions of the spatial interaction, we parametrically adjust the size of the gap between bars and flankers and the length of the flankers. We attempt to account for the results with models based on rectified difference of Gaussian filters and with rectified oriented difference of Gaussian filters. The models can account for the results when the flankers are the same height as bars, but are unable to account for the effects of increasing the flanker length. The models therefore suggest that the spatial interaction across distances requires more complex interactions of contrast filters.

Kitaoka's Tomato: Two Simple Explanations Based on Information in the Stimulus

Kitaoka’s Tomato is a color illusion in which a semitransparent blue-green field is placed on top of a red object (a tomato). The tomato appears red even though the pixels would appear green if viewed in isolation. We show that this phenomenon can be explained by a high-pass filter and by histogram equalization. The results suggest that this illusion does not require complex inferences about color constancy; rather, the tomato’s red is available in the physical stimulus at the appropriate spatial scale and dynamic range.

Bioplausible multiscale filtering in retino-cortical processing as a mechanism in perceptual grouping

Why does our visual system fail to reconstruct reality, when we look at certain patterns? Where do Geometrical illusions start to emerge in the visual pathway? How far should we take computational models of vision with the same visual ability to detect illusions as we do? This study addresses these questions, by focusing on a specific underlying neural mechanism involved in our visual experiences that affects our final perception. Among many types of visual illusion, 'Geometrical' and, in particular, 'Tilt Illusions' are rather important, being characterized by misperception of geometric patterns involving lines and tiles in combination with contrasting orientation, size or position. Over the last decade, many new neurophysiological experiments have led to new insights as to how, when and where retinal processing takes place, and the encoding nature of the retinal representation that is sent to the cortex for further processing. Based on these neurobiological discoveries, we provide computer simulation evidence from modelling retinal ganglion cells responses to some complex Tilt Illusions, suggesting that the emergence of tilt in these illusions is partially related to the interaction of multiscale visual processing performed in the retina. The output of our low-level filtering model is presented for several types of Tilt Illusion, predicting that the final tilt percept arises from multiple-scale processing of the Differences of Gaussians and the perceptual interaction of foreground and background elements. The model is a variation of classical receptive field implementation for simple cells in early stages of vision with the scales tuned to the object/texture sizes in the pattern. Our results suggest that this model has a high potential in revealing the underlying mechanism connecting low-level filtering approaches to mid- and high-level explanations such as 'Anchoring theory' and 'Perceptual grouping'.

A DOG filter model of the occurrence of Mach bands on spatial contrast discontinuities

The present work proposes a unified model to explain two previously reported properties of the Mach band illusion. The first is the frequently referenced fact that Mach bands are prominently visible at ramps, but practically vanish at intensity steps. The second property, less studied, on the other hand may also be related to the first. It concerns the fact that the width of the illusory Mach bands appears to be a function of the slope of the ramp itself. The model proposed here combines the difference of Gaussians (DOG) model of lateral inhibition in receptive fields with the models of feature detection, based on a holistic approach. The sharpness of discontinuity (SOD) concept for Mach band stimulus has been defined and is related to the slope of the ramp. It is suggested that calculation of SOD leads to an adaptive change in inhibitory surround, a notion that has the support of physiological experiments too.

Limitations of the Oriented Difference of Gaussian Filter in Special Cases of Brightness Perception Illusions

The Oriented Difference of Gaussian (ODOG) filter of Blakeslee and McCourt has been successfully employed to explain several brightness perception illusions which include illusions of both brightness-contrast type, for example, Simultaneous Brightness Contrast and Grating Induction and the brightness-assimilation type, for example, the White effect and the shifted White effect. Here, we demonstrate some limitations of the ODOG filter in predicting perceived brightness by comparing the ODOG responses to various stimuli (generated by varying two parameters, namely, test patch length and spatial frequency) with experimental observations of the same.

Testing the role of luminance edges in White's illusion with contour adaptation

White's illusion is the perceptual effect that two equiluminant gray patches superimposed on a black-and-white square-wave grating appear different in lightness: A test patch placed on a dark stripe of the grating looks lighter than one placed on a light stripe. Although the effect does not depend on the aspect ratio of the test patches, and thus on the amount of border that is shared with either the dark or the light stripe, the context of each patch must, in a yet to be specified way, influence their lightness. We employed a contour adaptation paradigm (Anstis, 2013) to test the contribution of each of the test patches' edges to the perceived lightness of the test patches. We found that adapting to the edges that are oriented parallel to the grating slightly increased the lightness illusion, whereas adapting to the orthogonal edges abolished, or for some observers even reversed, the lightness illusion. We implemented a temporal adaptation mechanism in three spatial filtering models of lightness perception, and show that the models cannot account for the observed adaptation effects. We conclude that White's illusion is largely determined by edge contrast across the edge orthogonal to the grating, whereas the parallel edge has little or no influence. We suggest mechanisms that could explain this asymmetry.

An exponential filter model predicts lightness illusions

Lightness, or perceived reflectance of a surface, is influenced by surrounding context. This is demonstrated by the Simultaneous Contrast Illusion (SCI), where a gray patch is perceived lighter against a black background and vice versa. Conversely, assimilation is where the lightness of the target patch moves toward that of the bounding areas and can be demonstrated in White's effect. Blakeslee and McCourt (1999) introduced an oriented difference-of-Gaussian (ODOG) model that is able to account for both contrast and assimilation in a number of lightness illusions and that has been subsequently improved using localized normalization techniques. We introduce a model inspired by image statistics that is based on a family of exponential filters, with kernels spanning across multiple sizes and shapes. We include an optional second stage of normalization based on contrast gain control. Our model was tested on a well-known set of lightness illusions that have previously been used to evaluate ODOG and its variants, and model lightness values were compared with typical human data. We investigate whether predictive success depends on filters of a particular size or shape and whether pooling information across filters can improve performance. The best single filter correctly predicted the direction of lightness effects for 21 out of 27 illusions. Combining two filters together increased the best performance to 23, with asymptotic performance at 24 for an arbitrarily large combination of filter outputs. While normalization improved prediction magnitudes, it only slightly improved overall scores in direction predictions. The prediction performance of 24 out of 27 illusions equals that of the best performing ODOG variant, with greater parsimony. Our model shows that V1-style orientation-selectivity is not necessary to account for lightness illusions and that a low-level model based on image statistics is able to account for a wide range of both contrast and assimilation effects.

What visual illusions tell us about underlying neural mechanisms and observer strategies for tackling the inverse problem of achromatic perception

Research in lightness perception centers on understanding the prior assumptions and processing strategies the visual system uses to parse the retinal intensity distribution (the proximal stimulus) into the surface reflectance and illumination components of the scene (the distal stimulus—ground truth). It is agreed that the visual system must compare different regions of the visual image to solve this inverse problem; however, the nature of the comparisons and the mechanisms underlying them are topics of intense debate. Perceptual illusions are of value because they reveal important information about these visual processing mechanisms. We propose a framework for lightness research that resolves confusions and paradoxes in the literature, and provides insight into the mechanisms the visual system employs to tackle the inverse problem. The main idea is that much of the debate and confusion in the literature stems from the fact that lightness, defined as apparent reflectance, is underspecified and refers to three different types of judgments that are not comparable. Under stimulus conditions containing a visible illumination component, such as a shadow boundary, observers can distinguish and match three independent dimensions of achromatic experience: apparent intensity (brightness), apparent local intensity ratio (brightness-contrast), and apparent reflectance (lightness). In the absence of a visible illumination boundary, however, achromatic vision reduces to two dimensions and, depending on stimulus conditions and observer instructions, judgments of lightness are identical to judgments of brightness or brightness-contrast. Furthermore, because lightness judgments are based on different information under different conditions, they can differ greatly in their degree of difficulty and in their accuracy. This may, in part, explain the large variability in lightness constancy across studies.

The Oriented Difference of Gaussians (ODOG) model of brightness perception: Overview and executable Mathematica notebooks

The Oriented Difference of Gaussians (ODOG) model of brightness (perceived intensity) by Blakeslee and McCourt (Vision Research 39:4361-4377, 1999), which is based on linear spatial filtering by oriented receptive fields followed by contrast normalization, has proven highly successful in parsimoniously predicting the perceived intensity (brightness) of regions in complex visual stimuli such as White's effect, which had been believed to defy filter-based explanations. Unlike competing explanations such as anchoring theory, filling-in, edge-integration, or layer decomposition, the spatial filtering approach embodied by the ODOG model readily accounts for the often overlooked but ubiquitous gradient structure of induction which, while most striking in grating induction, also occurs within the test fields of classical simultaneous brightness contrast and the White stimulus. Also, because the ODOG model does not require defined regions of interest, it is generalizable to any stimulus, including natural images. The ODOG model has motivated other researchers to develop modified versions (LODOG and FLODOG), and has served as an important counterweight and proof of concept to constrain high-level theories which rely on less well understood or justified mechanisms such as unconscious inference, transparency, perceptual grouping, and layer decomposition. Here we provide a brief but comprehensive description of the ODOG model as it has been implemented since 1999, as well as working Mathematica (Wolfram, Inc.) notebooks which users can employ to generate ODOG model predictions for their own stimuli.

A Neurocomputational account of the role of contour facilitation in brightness perception

A new filling-in model is proposed in order to account for challenging brightness illusions, where inducing background elements are spatially separated from the gray target such as dungeon, cube and grating illusions, bullseye display and ring patterns. This model implements the simple idea that neural response to low-contrast contour is enhanced (facilitated) by the presence of collinear or parallel high-contrast contours in its wider neighborhood. Contour facilitation is achieved via dendritic inhibition, which enables the computation of maximum function among inputs to the node. Recurrent application of maximum function leads to the propagation of the neural signal along collinear or parallel contour segments. When a strong global-contour signal is accompanied with a weak local-contour signal at the same location, conditions are met to produce brightness assimilation within the Filling-in Layer. Computer simulations showed that the model correctly predicts brightness appearance in all of the aforementioned illusions as well as in White's effect, Benary's cross, Todorović's illusion, checkerboard contrast, contrast-contrast illusion and various variations of the White's effect. The proposed model offers new insights on how geometric factors (contour colinearity or parallelism), together with contrast magnitude contribute to the brightness perception.

The impact of stereoscopic depth on the Munker-White illusion

The current study investigated the impact of stereoscopic depth information on adults' perception of a coloured version of the Munker-White illusion. In one half of the illusory figure red patches were embedded in black stripes and flanked by yellow stripes. In the other half of the illusory figure red patches were embedded in yellow stripes and flanked by black stripes. The red patches either remained in the same depth plane as the black and yellow inducing stripes (zero horizontal disparity condition) or were shifted into the foreground (crossed horizontal disparity condition) or into the background (uncrossed horizontal disparity condition). According to the results, the illusory effect was robust across all viewing conditions. The illusion mainly consisted of a subjective darkening of the red patches superimposed on the yellow stripes, a perceived hue shift of the red patches superimposed on the black stripes toward yellow, and a subjective saturation decrease in both kinds of red patches. Moreover, the study established a partial confirmation of Anderson's scission theory, according to which the Munker-White illusion should be largest in the crossed horizontal disparity condition, intermediate in the zero horizontal disparity condition, and smallest in the uncrossed horizontal disparity condition.

Noise masking of White's illusion exposes the weakness of current spatial filtering models of lightness perception

Spatial filtering models are currently a widely accepted mechanistic account of human lightness perception. Their popularity can be ascribed to two reasons: They correctly predict how human observers perceive a variety of lightness illusions, and the processing steps involved in the models bear an apparent resemblance with known physiological mechanisms at early stages of visual processing. Here, we tested the adequacy of these models by probing their response to stimuli that have been modified by adding narrowband noise. Psychophysically, it has been shown that noise in the range of one to five cycles per degree (cpd) can drastically reduce the strength of some lightness phenomena, while noise outside this range has little or no effect on perceived lightness. Choosing White's illusion (White, 1979) as a test case, we replicated and extended the psychophysical results, and found that none of the spatial filtering models tested was able to reproduce the spatial frequency specific effect of narrowband noise. We discuss the reasons for failure for each model individually, but we argue that the failure is indicative of the general inadequacy of this class of spatial filtering models. Given the present evidence we do not believe that spatial filtering models capture the mechanisms that are responsible for producing many of the lightness phenomena observed in human perception. Instead we think that our findings support the idea that low-level contributions to perceived lightness are primarily determined by the luminance contrast at surface boundaries.

Luminance gradient configuration determines perceived lightness in a simple geometric illusion

Accurate perception of surface reflectance poses a significant computational problem for the visual system. The amount of light reflected by a surface is affected by a combination of factors including the surface's reflectance properties and illumination conditions. The latter are not limited by the strength of the illuminant but also include the relative placement of the light illuminating the surface, the orientation of the surface and its 3d shape, all of which result in a pattern of luminance gradients across the surface. In this study we explore how luminance gradients contribute to lightness perception. We introduce a novel, simple lightness illusion. It consists of six separate checks, organized in rows of two. Each check has a negative luminance gradient across it. The top and the bottom rows are the same: with the darker check on the left, and the lighter check on the right. Two checks in the middle row are identical; however, the check on the right appears darker than the check on the left. As there are no shared borders between the checks, simultaneous contrast cannot explain the effect. However, there are multiple possible explanations including spatial filtering (Blakeslee and McCourt, 2004) or some higher-order mechanism such as perceptual grouping or amodal completion. Here, we explore these possibilities by manipulating the luminance configurations and the gradient slopes of the checks.

Mach bands explained by response normalization

Mach bands are the illusory dark and bright bars seen at the foot and knee of a luminance trapezoid. First demonstrated by Ernst Mach in the latter part of the 19th century, Mach bands are a test bed not only for models of brightness illusions but of spatial vision in general. Up until 50 years ago the dominant explanation of Mach Bands was that they were caused by lateral inhibition among retinal neurons. More recently, the dominant idea has been that Mach bands are a consequence of a visual process that generates a sparse, binary description of the image in terms of "edges" and "bars". Another recent explanation is that Mach bands result from learned expectations about the pattern of light typically found on sharply curved surfaces. In keeping with recent multi-scale filtering accounts of brightness illusions as well as current physiology, I show however that Mach bands are most simply explained by response normalization, whereby the gains of early visual channels are adjusted on a local basis to make their responses more equal. I show that a simple one-dimensional model of response normalization explains the range of conditions under which Mach bands occur, and as importantly, the conditions under which they do not occur.

The roles of image decomposition and edge curvature in the 'snake' lightness illusion

The snake illusion is an effect in which the lightness of target patches is strongly affected by the luminance of remote patches. One explanation is that such images are decomposed into a pattern of illumination and a pattern of reflectance, involving a classification of luminance edges into illumination and reflectance edges. Based on this decomposition, perceived reflectance is determined by discounting the illumination. A problem for this account is that image decomposition is not unique, and that different decompositions may lead to different lightness predictions. One way to rule out alternative decompositions and ensure correct predictions is to postulate that the visual system tends to classify curved luminance edges as reflectance edges rather than illumination edges. We have constructed several variations of the basic snake display in order to test the proposed curvature constraint and the more general image decomposition hypothesis. Although the results from some displays have confirmed previous findings of the effect of curvature, the general pattern of data questions the relevance of the shape of luminance edges for the determination of lightness in this class of displays. The data also argue against an image decomposition mechanism as an explanation of this effect. As an alternative, a tentative neurally based account is sketched.

Scale-invariance in brightness illusions implicates object-level visual processing

Brightness illusions demonstrate that an object's perceived brightness depends on its visual context, leading to theoretical explanations ranging from simple lateral inhibition to those based on the influence of knowledge of and experience with the world. We measure the relative brightness of mid-luminance test disks embedded in gray-scale images, and show that rankings of test disk brightness are independent of viewing distance, implying that the rankings depend on the physical object size, not the size of disks subtended on the retina. A single filter that removes low spatial frequency content, adjusted to the diameters of the test disks, can account for the relative brightness of the disks. We note that the removal of low spatial frequency content is a principle common to many different approaches to brightness/lightness phenomena; furthermore, object-size representations--as opposed to retinal-size representations--inherently remove low spatial frequency content, therefore, any process that creates object representations should also produce brightness illusions.

Depth effect on lightness revisited: The role of articulation, proximity and fields of illumination

The coplanar ratio principle proposes that when the luminance range in an image is larger than the canonical reflectance range of 30:1, the lightness of a target surface depends on the luminance ratio between that target and its adjacent coplanar neighbor (Gilchrist, 1980). This conclusion is based on experiments in which changes in the perceived target depth produced large changes in its perceived lightness without significantly altering the observers' retinal image. Using the same paradigm, we explored how this depth effect on lightness depends on display complexity (articulation), proximity of the target to its highest coplanar luminance and spatial distribution of fields of illumination. Importantly, our experiments allowed us to test differing predictions made by the anchoring theory (Gilchrist et al., 1999), the coplanar ratio principle, as well as other models. We report three main findings, generally consistent with anchoring theory predictions: (1) Articulation can substantially increase the depth effect. (2) Target lightness depends not on the adjacent luminance but on the highest coplanar luminance, irrespective of its position relative to the target. (3) When a plane contains multiple fields of illumination, target lightness depends on the highest luminance in its field of illumination, not on the highest coplanar luminance.

Retinal mesopic adaptation model for brightness perception under transient glare

A glare source in the visual field modifies the brightness of a test patch surrounded by a mesopic background. In this study, we investigated the effect of two levels of transient glare on brightness perception for several combinations of mesopic reference test luminances (Lts) and background luminances (Lbs). While brightness perception was affected by Lb, there were no appreciable effects for changes in the Lt. The highest brightness reduction was found for Lbs in the low mesopic range. Considering the main proposal that brightness can be inferred from contrast and the Lb sets the mesopic luminance adaptation, we hypothesized that contrast gain and retinal adaptation mechanisms would act when a transient glare source was present in the visual field. A physiology-based model that adequately fitted the present and previous results was developed.

Orientation enhancement in early visual processing can explain time course of brightness contrast and White's illusion

Dynamics of orientation tuning in V1 indicates that computational model of V1 should not only comprise of bank of static spatially oriented filters but also include the contribution for dynamical response facilitation or suppression along orientation. Time evolution of orientation response in V1 can emerge due to time- dependent excitation and lateral inhibition in the orientation domain. Lateral inhibition in the orientation domain suggests that Ernst Mach's proposition can be applied for the enhancement of initial orientation distribution that is generated due to interaction of visual stimulus with spatially oriented filters and subcortical temporal filter. Oriented spatial filtering that appears much early (<70 ms) in the sequence of visual information processing can account for many of the brightness illusions observed at steady state. It is therefore expected that time evolution of orientation response might be reflecting in the brightness percept over time. Our numerical study suggests that only spatio-temporal filtering at early phase can explain experimentally observed temporal dynamics of brightness contrast illusion. But, enhancement of orientation response at early phase of visual processing is the key mechanism that can guide visual system to predict the brightness by "Max-rule" or "Winner Takes All" (WTA) estimation and thus producing White's illusions at any exposure.

A neurodynamical model of brightness induction in v1

Brightness induction is the modulation of the perceived intensity of an area by the luminance of surrounding areas. Recent neurophysiological evidence suggests that brightness information might be explicitly represented in V1, in contrast to the more common assumption that the striate cortex is an area mostly responsive to sensory information. Here we investigate possible neural mechanisms that offer a plausible explanation for such phenomenon. To this end, a neurodynamical model which is based on neurophysiological evidence and focuses on the part of V1 responsible for contextual influences is presented. The proposed computational model successfully accounts for well known psychophysical effects for static contexts and also for brightness induction in dynamic contexts defined by modulating the luminance of surrounding areas. This work suggests that intra-cortical interactions in V1 could, at least partially, explain brightness induction effects and reveals how a common general architecture may account for several different fundamental processes, such as visual saliency and brightness induction, which emerge early in the visual processing pathway.

Brightness induction magnitude declines with increasing distance from the inducing field edge

Brightness induction refers to a class of visual illusions where the perceived intensity of a region of space is influenced by the luminance of surrounding regions. These illusions are significant because they provide insight into the neural organization and processing strategies employed by the visual system. The nature of these processing strategies, however, has long been debated. Here we investigate the spatial characteristics of grating induction as a function of the distance from the inducing field edge to evaluate the viability of various competing models. In particular multiscale spatial filtering models and homogeneous filling-in models make very different predictions in regard to the magnitude of induction as a function of this distance. Filling-in explanations predict that the brightness/lightness of the filled-in region will be homogeneous, whereas multiscale filtering predicts a fall-off in induction magnitude with distance from the inducing field edge. Induction magnitude was measured using a narrow probe version of the quadrature-phase motion-cancellation paradigm (Blakeslee & McCourt, 2011) and a point-by-point brightness matching paradigm (Blakeslee & McCourt, 1997, 1999; McCourt, 1994). Both techniques reveal a decrease in the magnitude of induction with increasing distance from the inducing edge. A homogeneous filling-in mechanism cannot explain the induced structure in the test fields of these stimuli. The results argue strongly against filling-in mechanisms as well as against any mechanism that posits that induction is homogeneous. The structure of the induction is, however, well accounted for by the multiscale filtering (ODOG) model of Blakeslee and McCourt (1999). These results support models of brightness/lightness, such as filtering models, which preserve these gradients of induction.

A possible role and basis of visual pathway selection in brightness induction

It is a well-known fact that the perceived brightness of any surface depends on the brightness of the surfaces that surround it. This phenomenon is termed as brightness induction. Isotropic arrays of multi-scale DoG (Difference of Gaussians) as well as cortical Oriented DoG (ODOG) and extensions thereof, like the Frequency-specific Locally Normalized ODOG (FLODOG) functions have been employed towards prediction of the direction of brightness induction in many brightness perception effects. But the neural basis of such spatial filters is seldom obvious. For instance, the visual information from retinal ganglion cells to such spatial filters, which have been generally speculated to appear at the early stage of cortical processing, are fed by at least three parallel channels viz. Parvocellular (P), Magnocellular (M) and Koniocellular (K) in the subcortical pathway, but the role of such pathways in brightness induction is generally not implicit. In this work, three different spatial filters based on an extended classical receptive field (ECRF) model of retinal ganglion cells, have been approximately related to the spatial contrast sensitivity functions of these three parallel channels. Based on our analysis involving different brightness perception effects, we propose that the M channel, with maximum conduction velocity, may have a special role for an initial sensorial perception. As a result, brightness assimilation may be the consequence of vision at a glance through the M pathway; contrast effect may be the consequence of a subsequent vision with scrutiny through the P channel; and the K pathway response may represent an intermediate situation resulting in ambiguity in brightness perception. The present work attempts to correlate this phenomenon of pathway selection with the complementary nature of these channels in terms of spatial frequency as well as contrast.

Relative Brightness in Natural Images Can Be Accounted for by Removing Blurry Content

One critical question regarding visual cognition concerns how the physical properties of the visual world are represented in early vision and then relayed to high-level vision. Here, we posit a simple theory: Processes that encode object appearance reduce their response to spatial content that is coarser than the size of the attended object. We show that a filtering procedure based on this theory can account for the relative brightness levels of test patches placed in images of natural scenes and for many hard-to-explain brightness illusions. The implication is that the perception of brightness differences in most brightness illusions actually corresponds to physical differences present in the images. Portions of the visual system may encode these physical differences by means of neural processes that adaptively reduce their response to low-spatial-frequency content.

Spatiotemporal analysis of brightness induction

Brightness induction refers to a class of visual illusions in which the perceived intensity of a region of space is influenced by the luminance of surrounding regions. These illusions are significant because they provide insight into the neural organization of the visual system. A novel quadrature-phase motion cancelation technique was developed to measure the magnitude of the grating induction brightness illusion across a wide range of spatial frequencies, temporal frequencies and test field heights. Canceling contrast is greatest at low frequencies and declines with increasing frequency in both dimensions, and with increasing test field height. Canceling contrast scales as the product of inducing grating spatial frequency and test field height (the number of inducing grating cycles per test field height). When plotted using a spatial axis which indexes this product, the spatiotemporal induction surfaces for four test field heights can be described as four partially overlapping sections of a single larger surface. These properties of brightness induction are explained in the context of multiscale spatial filtering. The present study is the first to measure the magnitude of grating induction as a function of temporal frequency. Taken in conjunction with several other studies (Blakeslee & McCourt, 2008; Magnussen & Glad, 1975; Robinson & de Sa, 2008) the results of this study illustrate that at least one form of brightness induction is very much faster than that reported by DeValois, Webster, DeValois, and Lingelbach (1986) and Rossi and Paradiso (1996), and are inconsistent with the proposition that brightness induction results from a slow "filling in" process.

The effect of figural manipulations on brightness differences in the Benary cross

The Benary cross is a classical demonstration showing that the perceived brightness f an area is not solely determined by its luminance, but also by the context in which it is embedded. Despite the fact that two identical grey triangles are flanked by an equal amount of black and white, one of the triangles is perceived as being lighter than the other. It has been argued that the junctions surrounding a test area are crucial in determining brightness. Here, we explored how different aspects influencing perceptual organisation influence perceived figure-background relations in the Benary cross and, with that, the perceived brightness of the triangular patches in our stimuli. The results of a cancellation task confirm that the alignment of contours at junctions indeed has a strong influence on an area's brightness. At the same time, however, the Benary effect is also influenced by the overall symmetry of the cross and its orientation.

Multiresolution wavelet framework models brightness induction effects

A new multiresolution wavelet model is presented here, which accounts for brightness assimilation and contrast effects in a unified framework, and includes known psychophysical and physiological attributes of the primate visual system (such as spatial frequency channels, oriented receptive fields, contrast sensitivity function, contrast non-linearities, and a unified set of parameters). Like other low-level models, such as the ODOG model [Blakeslee, B., & McCourt, M. E. (1999). A multiscale spatial filtering account of the white effect, simultaneous brightness contrast and grating induction. Vision Research, 39, 4361-4377], this formulation reproduces visual effects such as simultaneous contrast, the White effect, grating induction, the Todorović effect, Mach bands, the Chevreul effect and the Adelson-Logvinenko tile effects, but it also reproduces other previously unexplained effects such as the dungeon illusion, all using a single set of parameters.