Object knowledge modulates colour appearance

The role of memory color in visual attention

The expected color of an object influences how it is perceived. For example, a banana in a greyscale photo may appear slightly yellow because bananas are expected to be yellow. This phenomenon is known as the memory color effect (MCE), and the objects with a memory color are called "color-diagnostic." The MCE is theorized to be a top-down influence of color knowledge on visual perception. However, its validity has been questioned because most evidence for the MCE is based on subjective reports. Here a change detection task is used as an objective measure of the effect and the results show that change detection differs for color-diagnostic objects. Specifically, it was predicted and found that unnaturally colored color-diagnostic objects (e.g., a blue banana) would attract attention and thus be discovered more quickly and accurately. In the experiment, two arrays alternated with the target present in one array and absent in the other while all other objects remained unchanged. Participants had to find the target as quickly and accurately as possible. In the experimental condition, the targets were color-diagnostic objects (e.g., a banana) presented in either their natural (yellow) or an unnatural (blue) color. In the control condition, non-color-diagnostic objects (e.g., a mug) were presented with the same colors as the color-diagnostic objects. Unnaturally colored color-diagnostic objects were found more quickly, which suggests that the MCE is a top-down, preattentive process that can influence a nonsubjective visual perceptual task such as change detection.

Assessing the representational structure of softness activated by words

Softness is an important material property that can be judged directly, by interacting with an object, but also indirectly, by simply looking at an image of a material. The latter is likely possible by filling in relevant multisensory information from prior experiences with soft materials. Such experiences are thought to lead to associations that make up our representations about perceptual softness. Here, we investigate the structure of this representational space when activated by words, and compare it to haptic and visual perceptual spaces that we obtained in earlier work. To this end, we performed an online study where people rated different sensory aspects of soft materials, presented as written names. We compared the results with the previous studies where identical ratings were made on the basis of visual and haptic information. Correlation and Procrustes analyses show that, overall, the representational spaces of verbally presented materials were similar to those obtained from haptic and visual experiments. However, a classifier analysis showed that verbal representations could better be predicted from those obtained from visual than from haptic experiments. In a second study we rule out that these larger discrepancies in representations between verbal and haptic conditions could be due to difficulties in material identification in haptic experiments. We discuss the results with respect to the recent idea that at perceived softness is a multidimensional construct.

A model of colour appearance based on efficient coding of natural images

An object's colour, brightness and pattern are all influenced by its surroundings, and a number of visual phenomena and "illusions" have been discovered that highlight these often dramatic effects. Explanations for these phenomena range from low-level neural mechanisms to high-level processes that incorporate contextual information or prior knowledge. Importantly, few of these phenomena can currently be accounted for in quantitative models of colour appearance. Here we ask to what extent colour appearance is predicted by a model based on the principle of coding efficiency. The model assumes that the image is encoded by noisy spatio-chromatic filters at one octave separations, which are either circularly symmetrical or oriented. Each spatial band's lower threshold is set by the contrast sensitivity function, and the dynamic range of the band is a fixed multiple of this threshold, above which the response saturates. Filter outputs are then reweighted to give equal power in each channel for natural images. We demonstrate that the model fits human behavioural performance in psychophysics experiments, and also primate retinal ganglion responses. Next, we systematically test the model's ability to qualitatively predict over 50 brightness and colour phenomena, with almost complete success. This implies that much of colour appearance is potentially attributable to simple mechanisms evolved for efficient coding of natural images, and is a well-founded basis for modelling the vision of humans and other animals.

Aesthetic experience models human learning

Aesthetic experiences have the potential to promote learning and creativity by enhancing the ability to understand complexity and to integrate novel or disparate information. Offering a theoretical framework for understanding the cognitive benefits of aesthetic experiences, this paper argues they are the necessary outcome of human learning, in which natural objects or artworks are evaluated in a multi-dimensional preference space shaped by Bayesian prediction. In addition, it contends that the brain-states underlying aesthetic experiences harness configurations of the apex three transmodal neural systems-the default mode network, the central executive network, and the salience network-that may offer information-processing advantages by recruiting the brain's high-power communication hubs, thus enhancing potential for learning gain.

Self-Supervised Color-Concept Association via Image Colorization

The interpretation of colors in visualizations is facilitated when the assignments between colors and concepts in the visualizations match human's expectations, implying that the colors can be interpreted in a semantic manner. However, manually creating a dataset of suitable associations between colors and concepts for use in visualizations is costly, as such associations would have to be collected from humans for a large variety of concepts. To address the challenge of collecting this data, we introduce a method to extract color-concept associations automatically from a set of concept images. While the state-of-the-art method extracts associations from data with supervised learning, we developed a self-supervised method based on colorization that does not require the preparation of ground truth color-concept associations. Our key insight is that a set of images of a concept should be sufficient for learning color-concept associations, since humans also learn to associate colors to concepts mainly from past visual input. Thus, we propose to use an automatic colorization method to extract statistical models of the color-concept associations that appear in concept images. Specifically, we take a colorization model pre-trained on ImageNet and fine-tune it on the set of images associated with a given concept, to predict pixel-wise probability distributions in Lab color space for the images. Then, we convert the predicted probability distributions into color ratings for a given color library and aggregate them for all the images of a concept to obtain the final color-concept associations. We evaluate our method using four different evaluation metrics and via a user study. Experiments show that, although the state-of-the-art method based on supervised learning with user-provided ratings is more effective at capturing relative associations, our self-supervised method obtains overall better results according to metrics like Earth Mover's Distance (EMD) and Entropy Difference (ED), which are closer to human perception of color distributions.

Newly learned shape-color associations show signatures of reliability-weighted averaging without forced fusion or a memory color effect

Reliability-weighted averaging of multiple perceptual estimates (or cues) can improve precision. Research suggests that newly learned statistical associations can be rapidly integrated in this way for efficient decision-making. Yet, it remains unclear if the integration of newly learned statistics into decision-making can directly influence perception, rather than taking place only at the decision stage. In two experiments, we implicitly taught observers novel associations between shape and color. Observers made color matches by adjusting the color of an oval to match a simultaneously presented reference. As the color of the oval changed across trials, so did its shape according to a novel mapping of axis ratio to color. Observers showed signatures of reliability-weighted averaging-a precision improvement in both experiments and reweighting of the newly learned shape cue with changes in uncertainty in Experiment 2. To ask whether this was accompanied by perceptual effects, Experiment 1 tested for forced fusion by measuring color discrimination thresholds with and without incongruent novel cues. Experiment 2 tested for a memory color effect, observers adjusting the color of ovals with different axis ratios until they appeared gray. There was no evidence for forced fusion and the opposite of a memory color effect. Overall, our results suggest that the ability to quickly learn novel cues and integrate them with familiar cues is not immediately (within the short duration of our experiments and in the domain of color and shape) accompanied by common perceptual effects.

Why bananas look yellow: The dominant hue of object colours

In this study, we propose a new approach to the perceptual representation of object colours. Three-dimensional objects have a polychromatic colour distribution. Yet, human observers abstract from the variation along the three perceptual colour dimensions when describing objects, such as when we say, "a banana is yellow". We propose that the perceived object colour is determined by the dominant hue. The dominant hue corresponds to the first principal component of an object's chromaticities. Across three experiments, we show for a sample of objects that the chromatic variation away from the dominant hue is almost completely neglected by human observers under non-laboratory viewing conditions. This is partly due to the low visibility of this variation, and partly to attentional change blindness. These findings reveal the potential role of dominant hue in the perception of object colours. Dominant hue may enable us to determine the most representative colours of objects because perceived object colours tend to be maximally bright and saturated. The present findings also imply that we can simplify the colour distributions of objects by projecting them onto their dominant hue. This may be useful for computational applications.

Influence of colour on object motor representation

Colour conveys specific information about the status/quality of an object; whereas its role in object recognition has been widely studied, little is known about its role in sensorimotor processes. We performed three experiments to assess whether colour influences the motor representation of graspable objects. In Experiment 1, we used a grasp compatibility task, in which participants categorized each object as natural or artifact, by performing reach-to-grasp movements. Response grasps could be compatible or incompatible with the ones normally used to manipulate the objects. Results showed faster reaction times for natural objects displayed in the correct colour compared with both opposite colour and correct colour artifact objects. In Experiment 2, to directly assess the effect of colour on object motor representation, we used an interference task in which an irrelevant object was shown while performing a pre-specified reach-to-grasp movement (i.e., verbal cues: small vs. large). Results highlighted a reversed compatibility effect when objects were shown in their correct colour, but only at the beginning of the movement (10 ms SOA). Finally, we run a third experiment using the same task as in Experiment 2. In this experiment, we compared the grasp compatibility effect driven by natural objects with the grasp compatibility effect driven by dangerous natural objects (e.g., cactus), which are objects that should not elicit a grasping program. The results of Experiment 3 confirm those of Experiment 2, highlighting also specific processes related to dangerous objects. Taken together, these results revealed that colour can be significant for the motor system, highlighting the close link between colour and shape, and also specific processes related to dangerous objects.

Context Matters: A Theory of Semantic Discriminability for Perceptual Encoding Systems

People's associations between colors and concepts influence their ability to interpret the meanings of colors in information visualizations. Previous work has suggested such effects are limited to concepts that have strong, specific associations with colors. However, although a concept may not be strongly associated with any colors, its mapping can be disambiguated in the context of other concepts in an encoding system. We articulate this view in semantic discriminability theory, a general framework for understanding conditions determining when people can infer meaning from perceptual features. Semantic discriminability is the degree to which observers can infer a unique mapping between visual features and concepts. Semantic discriminability theory posits that the capacity for semantic discriminability for a set of concepts is constrained by the difference between the feature-concept association distributions across the concepts in the set. We define formal properties of this theory and test its implications in two experiments. The results show that the capacity to produce semantically discriminable colors for sets of concepts was indeed constrained by the statistical distance between color-concept association distributions (Experiment 1). Moreover, people could interpret meanings of colors in bar graphs insofar as the colors were semantically discriminable, even for concepts previously considered "non-colorable" (Experiment 2). The results suggest that colors are more robust for visual communication than previously thought.

The look and feel of soft are similar across different softness dimensions

The softness of objects can be perceived through several senses. For instance, to judge the softness of a cat's fur, we do not only look at it, we often also run our fingers through its coat. Recently, we have shown that haptically perceived softness covaries with the compliance, viscosity, granularity, and furriness of materials (Dovencioglu, Üstün, Doerschner, & Drewing, 2020). However, it is unknown whether vision can provide similar information about the various aspects of perceived softness. Here, we investigated this question in an experiment with three conditions: in the haptic condition, blindfolded participants explored materials with their hands, in the static visual condition participants were presented with close-up photographs of the same materials, and in the dynamic visual condition participants watched videos of the hand-material interactions that were recorded in the haptic condition. After haptically or visually exploring the materials, participants rated them on various attributes. Our results show a high overall perceptual correspondence among the three experimental conditions. With a few exceptions, this correspondence tended to be strongest between haptic and dynamic visual conditions. These results are discussed with respect to information potentially available through the senses, or through prior experience, when judging the softness of materials.

Task Construal Influences Estimations of the Environment

People's characteristics can affect their perception of the physical environment, and the judgments and estimates they make about their surroundings. Estimates of the environment change based on observers' metabolic state, physical properties, and the potential effort they would need to exert for a certain action. The functional role of such scaling is to provide agents with information on possible actions and their energetic costs. Combining actions with costs facilitates both higher-level planning (e.g., choosing an optimal running speed on a marathon) as well as planning on lower levels of an action hierarchy, such as determining the best movement trajectories for energy-efficient action. Recently, some of the findings on reported effects of effort on perception have been challenged as arising from task demands-participants guessing the purpose of the experimental manipulation and adjusting their estimates as a result. Arguably however, the failed replications used overly distracting cover stories which may have introduced task demands of their own, and masked other effects. The current study tested the generality of effects of potential effort on height and distance perception, employing effective yet not distracting cover stories. Four experiments attempted to identify conditions under which anticipated effort may systematically change perceptual estimates. Experiment 1 found that height estimates were not influenced by the effort required to place objects of different weights onto surfaces of varying heights. Experiments 2, 3 used two different effort manipulations (walking vs. hopping; and carrying an empty vs. a heavy backpack, respectively) and found that these did not influence estimates of distance (to be) traveled. Experiment 4 also used backpack weight to manipulate effort but critically, unlike Exp. 1-3 it did not employ a cover story and participants did not traverse distances after giving estimates. In contrast with the first three experiments, distances in the final experiment were estimated as longer when participants were encumbered by a backpack. Combined, these results suggest that the measured effects on the estimation of distance were due to how participants construed the task rather than being of a perceptual nature.

The Influence of Object-Color Knowledge on Emerging Object Representations in the Brain

The ability to rapidly and accurately recognize complex objects is a crucial function of the human visual system. To recognize an object, we need to bind incoming visual features, such as color and form, together into cohesive neural representations and integrate these with our preexisting knowledge about the world. For some objects, typical color is a central feature for recognition; for example, a banana is typically yellow. Here, we applied multivariate pattern analysis on time-resolved neuroimaging (MEG) data to examine how object-color knowledge affects emerging object representations over time. Our results from 20 participants (11 female) show that the typicality of object-color combinations influences object representations, although not at the initial stages of object and color processing. We find evidence that color decoding peaks later for atypical object-color combinations compared with typical object-color combinations, illustrating the interplay between processing incoming object features and stored object knowledge. Together, these results provide new insights into the integration of incoming visual information with existing conceptual object knowledge.SIGNIFICANCE STATEMENT To recognize objects, we have to be able to bind object features, such as color and shape, into one coherent representation and compare it with stored object knowledge. The MEG data presented here provide novel insights about the integration of incoming visual information with our knowledge about the world. Using color as a model to understand the interaction between seeing and knowing, we show that there is a unique pattern of brain activity for congruently colored objects (e.g., a yellow banana) relative to incongruently colored objects (e.g., a red banana). This effect of object-color knowledge only occurs after single object features are processed, demonstrating that conceptual knowledge is accessed relatively late in the visual processing hierarchy.

A neurodynamic model of the interaction between color perception and color memory

The memory color effect and Spanish castle illusion have been taken as evidence of the cognitive penetrability of vision. In the same manner, the successful decoding of color-related brain signals in functional neuroimaging studies suggests the retrieval of memory colors associated with a perceived gray object. Here, we offer an alternative account of these findings based on the design principles of adaptive resonance theory (ART). In ART, conscious perception is a consequence of a resonant state. Resonance emerges in a recurrent cortical circuit when a bottom-up spatial pattern agrees with the top-down expectation. When they do not agree, a special control mechanism is activated that resets the network and clears off erroneous expectation, thus allowing the bottom-up activity to always dominate in perception. We developed a color ART circuit and evaluated its behavior in computer simulations. The model helps to explain how traces of erroneous expectations about incoming color are eventually removed from the color perception, although their transient effect may be visible in behavioral responses or in brain imaging. Our results suggest that the color ART circuit, as a predictive computational system, is almost never penetrable, because it is equipped with computational mechanisms designed to constrain the impact of the top-down predictions on ongoing perceptual processing.

Hues of Color Afterimages

We studied the relationship between color afterimages and complementary colors. The hues of afterimages of 24 inducer hues, uniformly distributed over the rgb color circle, were measured by an iterative method of adjustment. The judgment of equality of hue of the afterimage and a synthesized patch was effectively judged at the moment immediately after the switch-off of the inducer, when the synthesized patch went through any number of iterative adjustments. The two patches—both phenomenally present, but only one optically presented—appeared to the left and right of a fixation mark that was fixated throughout the whole procedure. Thus, both patches were present in eccentric vision. The hues of afterimages were found to be quite different from the hue of the complementary of the inducer. Almost one half of the color circle (orange to chartreuse) leads to afterimage hues in a narrow region of purples. This implies that color circles based on diametrically opposed inducer–afterimage hues are necessarily inconsistent. Yet, perhaps surprisingly, the relation between primary and afterimage hues is still approximately an involution (they are reciprocally related).

Interaction of top-down category-level expectation and bottom-up sensory input in early stages of visual-orthographic processing

How and when top-down information modulates visual-orthographic processing is an essential question in reading research. In a previous study, we showed that task modulation of print-tuning started at around 170 ms after stimulus presentation in the N1 offset of the ERP, while the N1 onset was yet unaffected. Here we test how prior category-level expectation affects visual-orthographic processing. Familiar, left/right-structured Chinese characters and stroke number matched, unfamiliar Korean characters were presented, while expectation about the upcoming stimuli was manipulated with green and blue colored frames (high Chinese vs. high Korean expectation). EEG data of 18 native Chinese speakers were recorded while participants performed an expectation judgment task. Results from occipito-temporal and whole map analyses revealed that effects of prior expectation changed throughout the N1. Accordingly in the N1 onset, a print tuning main effect was found, with a stronger N1 to Chinese characters than Korean characters, irrespective of expectation. In the N1 offset, an expectation-by-character interaction was observed at the whole map level, with a more negative N1 to Korean characters than Chinese characters when expecting a Chinese character, but no such difference when expecting a Korean character. Moreover, the expectation-by-character interaction continued to the N250, with similar responses to Chinese and Korean characters under the Chinese expectation condition, while less negative N250 to Korean than Chinese under the Korean expectation condition. Taken together, the current study provides evidence that prior category-level expectation starts to take effect at an early stage even within 200 ms by facilitating the processing of expected stimuli, suggesting that category-level expectation can influence early visual-orthographic processing during word recognition.

What Cognitive Neurology Teaches Us about Our Experience of Color

Color provides valuable information about the environment, yet the exact mechanisms explaining how colors appear to us remain poorly understood. Retinal signals are processed in the visual cortex through high-level mechanisms that link color perception with top-down expectations and knowledge. Here, we review the neuroimaging evidence about color processing in the brain, and how it is affected by acquired brain lesions in humans. Evidence from patients with brain-damage suggests that high-level color processing may be divided into at least three modules: perceptual color experience, color naming, and color knowledge. These modules appear to be functionally independent but richly interconnected, and serve as cortical relays linking sensory and semantic information, with the final goal of directing object-related behavior. We argue that the relations between colors and their objects are key mechanisms to understand high-level color processing.

Chromatic adaptation from achromatic stimuli with implied color

Previous research has shown that the typical or memory color of an object is perceived in images of that object, even when the image is achromatic. We performed an experiment to investigate whether the implied color in greyscale images could influence the perceived color of subsequent, simple stimuli. We used a standard top-up adaptation technique along with a roving-pedestal, two-alternative spatial forced-choice method for measuring perceptual bias without contamination from any response or decision biases. Adaptors were achromatic images of natural objects that are normally seen with diagnostic color. We found that, in some circumstances, greyscale adapting images had a biasing effect, shifting the achromatic point toward the implied color, in comparison with phase-scrambled images. We interpret this effect as evidence of adaptation in chromatic signaling mechanisms that receive top-down input from knowledge of object color. This implied color adaptation effect was particularly strong from images of bananas, which are popular stimuli in memory color experiments. We also consider the effect in a color constancy context, in which the implied color is used by the visual system to estimate an illuminant, but find our results inconsistent with this explanation.

Human Depth Sensitivity Is Affected by Object Plausibility

Using behavioral and fMRI paradigms, we asked how the physical plausibility of complex 3-D objects, as defined by the object's congruence with 3-D Euclidean geometry, affects behavioral thresholds and neural responses to depth information. Stimuli were disparity-defined geometric objects rendered as random dot stereograms, presented in plausible and implausible variations. In the behavior experiment, observers were asked to complete (1) a noise-based depth task that involved judging the depth position of a target embedded in noise and (2) a fine depth judgment task that involved discriminating the nearer of two consecutively presented targets. Interestingly, results indicated greater behavioral sensitivities of depth judgments for implausible versus plausible objects across both tasks. In the fMRI experiment, we measured fMRI responses concurrently with behavioral depth responses. Although univariate responses for depth judgments were largely similar across cortex regardless of object plausibility, multivariate representations for plausible and implausible objects were notably distinguishable along depth-relevant intermediate regions V3 and V3A, in addition to object-relevant LOC. Our data indicate significant modulations of both behavioral judgments of and neural responses to depth by object context. We conjecture that disparity mechanisms interact dynamically with the object recognition problem in the visual system such that disparity computations are adjusted based on object familiarity.

Memory influences haptic perception of softness

The memory of an object’s property (e.g. its typical colour) can affect its visual perception. We investigated whether memory of the softness of every-day objects influences their haptic perception. We produced bipartite silicone rubber stimuli: one half of the stimuli was covered with a layer of an object (sponge, wood, tennis ball, foam ball); the other half was uncovered silicone. Participants were not aware of the partition. They first used their bare finger to stroke laterally over the covering layer to recognize the well-known object and then indented the other half of the stimulus with a probe to compare its softness to that of an uncovered silicone stimulus. Across four experiments with different methods we showed that silicon stimuli covered with a layer of rather hard objects (tennis ball and wood) were perceived harder than the same silicon stimuli when being covered with a layer of rather soft objects (sponge and foam ball), indicating that haptic perception of softness is affected by memory.

Combining Bottom-Up and Top-Down Visual Mechanisms for Color Constancy Under Varying Illumination

Multi-illuminant-based color constancy (MCC) is quite a challenging task. In this paper, we proposed a novel model motivated by the bottom-up and top-down mechanisms of human visual system (HVS) to estimate the spatially varying illumination in a scene. The motivation for bottom-up based estimation is from our finding that the bright and dark parts in a scene play different roles in encoding illuminants. However, handling the color shift of large colorful objects is difficult using pure bottom-up processing. Thus, we further introduce a top-down constraint inspired by the findings in visual psychophysics, in which high-level information (e.g., the prior of light source colors) plays a key role in visual color constancy. In order to implement the top-down hypothesis, we simply learn a color mapping between the illuminant distribution estimated by bottom-up processing and the ground truth maps provided by the dataset. We evaluated our model on four datasets and the results show that our method obtains very competitive performance compared with the state-of-the-art MCC algorithms. Moreover, the robustness of our model is more tangible considering that our results were obtained using the same parameters for all the datasets or the parameters of our model were learned from the inputs, that is, mimicking how HVS operates. We also show the color correction results on some real-world images taken from the web.

Methods for Assessing Quantity and Quality of Illumination

Human vision provides useful information about the shape and color of the objects around us. It works well in many, but not all, lighting conditions. Since the advent of human-made light sources, it has been important to understand how illumination affects vision quality, but this has been surprisingly difficult. The widespread introduction of solid-state light emitters has increased the urgency of this problem. Experts still debate how lighting can best enable high-quality vision-a key issue since about one-fifth of global electrical power production is used to make light. Photometry, the measurement of the visual quantity of light, is well established, yet significant uncertainties remain. Colorimetry, the measurement of color, has achieved good reproducibility, but researchers still struggle to understand how illumination can best enable high-quality color vision. Fortunately, in recent years, considerable progress has been made. Here, we summarize the current understanding and discuss key areas for future study.

Semantic Knowledge Enhances Conscious Awareness of Visual Objects

It is becoming increasingly established that information from long-term memory can influence early perceptual processing, a finding that is in line with recent theoretical approaches to cognition such as the predictive coding framework. Notwithstanding, the impact of semantic knowledge on conscious perception and the temporal dynamics of such an influence remain unclear. To address this question, we presented pictures of novel objects to participants as the second of two targets in an attentional blink paradigm. We found that associating newly acquired semantic knowledge to objects increased overall conscious detection in comparison to objects associated with minimal knowledge while controlling for object familiarity. Additionally, event-related brain potentials revealed a corresponding modulation beginning 100 msec after stimulus presentation in the P1 component. Furthermore, the size of this modulation was correlated with participant's subjective reports of conscious perception. These findings suggest that semantic knowledge can shape the contents of consciousness by affecting early stages of perceptual processing.

Color Perception: Objects, Constancy, and Categories

Color has been scientifically investigated by linking color appearance to colorimetric measurements of the light that enters the eye. However, the main purpose of color perception is not to determine the properties of incident light, but to aid the visual perception of objects and materials in our environment. We review the state of the art on object colors, color constancy, and color categories to gain insight into the functional aspects of color perception. The common ground between these areas of research is that color appearance is tightly linked to the identification of objects and materials and the communication across observers. In conclusion, we argue that research should focus on how color processing is adapted to the surface properties of objects in the natural environment in order to bridge the gap between the known early stages of color perception and the subjective appearance of color.

Cognitive penetration of early vision in face perception

Cognitive and affective penetration of perception refers to the influence that higher mental states such as beliefs and emotions have on perceptual systems. Psychological and neuroscientific studies appear to show that these states modulate the visual system at the visuomotor, attentional, and late levels of processing. However, empirical evidence showing that similar consequences occur in early stages of visual processing seems to be scarce. In this paper, I argue that psychological evidence does not seem to be either sufficient or necessary to argue in favour of or against the cognitive penetration of perception in either late or early vision. In order to do that we need to have recourse to brain imaging techniques. Thus, I introduce a neuroscientific study and argue that it seems to provide well-grounded evidence for the cognitive penetration of early vision in face perception. I also examine and reject alternative explanations to my conclusion.

A Bayesian Model of the Memory Colour Effect

According to the memory colour effect, the colour of a colour-diagnostic object is not perceived independently of the object itself. Instead, it has been shown through an achromatic adjustment method that colour-diagnostic objects still appear slightly in their typical colour, even when they are colourimetrically grey. Bayesian models provide a promising approach to capture the effect of prior knowledge on colour perception and to link these effects to more general effects of cue integration. Here, we model memory colour effects using prior knowledge about typical colours as priors for the grey adjustments in a Bayesian model. This simple model does not involve any fitting of free parameters. The Bayesian model roughly captured the magnitude of the measured memory colour effect for photographs of objects. To some extent, the model predicted observed differences in memory colour effects across objects. The model could not account for the differences in memory colour effects across different levels of realism in the object images. The Bayesian model provides a particularly simple account of memory colour effects, capturing some of the multiple sources of variation of these effects.

Gray Bananas and a Red Letter A - From Synesthetic Sensation to Memory Colors

Grapheme-color synesthesia is a condition in which objectively achromatic graphemes induce concurrent color experiences. While it was long thought that the colors emerge during perception, there is growing support for the view that colors are integral to synesthetes' cognitive representations of graphemes. In this work, we review evidence for two opposing theories positing either a perceptual or cognitive origin of concurrent colors: the cross-activation theory and the conceptual-mediation model. The review covers results on inducer and concurrent color processing as well as findings concerning the brain structure and grapheme-color mappings in synesthetes and trained mappings in nonsynesthetes. The results support different aspects of both theories. Finally, we discuss how research on memory colors could provide a new perspective in the debate about the level of processing at which the synesthetic colors occur.

Categorizing natural color distributions

The natural objects that we are surrounded with virtually always contain many different shades of color, yet the visual system usually categorizes them into a single color category. We examined various image statistics and their role in categorizing the color of leaves. Our subjects categorized photographs of autumn leaves and versions that were manipulated, including: randomly repositioned pixels, leaves uniformly colored with their mean color, leaves that were made by reflecting the original leaves' chromaticity distribution about their mean ("flipped leaves"), and simple patches colored with the mean colors of the original leaves. We trained a linear classifier with a set of image statistics in order to predict the category that each object was assigned to. Our results show that the mean hue of an object is highly predictive of the natural object's color category (>90% accuracy) and observers' choices are consistent with their use of unique yellow as a decision boundary for classification. The flipped leaves produced consistent changes in color categorization that are possibly explained by an interaction between the color distributions and the texture of the leaves.

When is cognitive penetration a plausible explanation?

Albert Newen and Petra Vetter argue that neurophysiological considerations and psychophysical studies provide striking evidence for cognitive penetration. This commentary focuses mainly on the neurophysiological considerations, which have thus far remained largely absent in the philosophical debate concerning cognitive penetration, and on the cognitive penetration of perceptual experiences, which is the form of cognitive penetration philosophers have debated about the most. It is argued that Newen and Vetter's evidence for cognitive penetration is unpersuasive because they do not sufficiently scrutinize the details of the empirical studies they make use of-the details of the empirical studies are crucial also when the studies are used in philosophical debates. The previous does not mean that cognitive penetration could not occur. Quite the contrary, details of the feedback connections to the visual perceptual module and one of the candidates presented by Newen and Vetter suggest that cognitive penetration can occur in rare cases.

The dress and individual differences in the perception of surface properties

This study investigates systematic individual differences in the way observers perceive different kinds of surface properties and their relationship to the dress, which shows striking individual differences in colour perception. We tested whether these individual differences have a common source, namely differences in perceptual strategies according to which observers attribute features in two-dimensional images to surfaces or to their illumination. First, we reanalysed data from two previous experiments on the dress and colour constancy. The comparison of the two experiments revealed that the colour perception of the dress is strongly related to individual differences in colour constancy. Second, two online surveys measured individual differences in the perception of colour-ambiguous images including the dress, in colour constancy, in gloss perception, in the subjective grey-point, in colour naming, and in the perception of an image with ambiguous shading. The results of the surveys replicated and extended previous findings according to which individual differences in the colour perception of the dress are due to implicit assumptions about the illumination. However, results also showed that the individual differences for other phenomena were independent of the dress and of each other. Overall, these results suggest that the striking individual differences in dress colour perception are due to individual differences in the interpretation of illumination cues to achieve colour constancy. At the same time, they undermine the idea of an overall perceptual strategy that encompasses other phenomena more generally related to the interpretation of illumination and surface properties.

Determinants of Colour Constancy and the Blue Bias

We investigated several sensory and cognitive determinants of colour constancy across 40 illumination hues. In the first experiment, we measured colour naming for the illumination and for the colour induced by the illumination on the colorimetric grey. Results confirmed that the induced colours are approximately complementary to the colour of the illumination. In the second experiment, we measured colour constancy using achromatic adjustments. Average colour constancy was perfect under the blue daylight illumination and decreased in colour directions away from the blue daylight illumination due to undershooting and a strong blue bias. Apart from this blue bias, colour constancy was not related to illumination discrimination and to chromatic detection measured previously with the same setup and stimuli. We also observed a strong negative relationship between the degree of colour constancy and the consensus of naming the illumination colour. Constancy coincided with a low naming consensus, in particular because bluish illumination colours were sometimes seen as achromatic. Blue bias and category consensus alone explained >68%, and all determinants together explained >94% of the variance of achromatic adjustments. These findings suggest that colour constancy is optimised for blue daylight.

Long-term memory color investigation: culture effect and experimental setting factors

Memory colors generated continuous interest in the color community. Previous studies focused on reflecting color chips and color samples in real scenes or on monitors. The cognitive effect of culture was rarely considered. In this paper, we performed a comprehensive investigation of the long-term memory colors of 26 familiar objects using the asymmetric color matching method among Chinese and German observers on a display. Three experiments were conducted to evaluate the variations introduced by culture, context-based gray image, and initial matching color. Memory colors of important objects were collected and representative memory colors were quantified in terms of CIELAB L^*, a^*, and b^* values. The intra- and inter-observer variations were analyzed by mean-color-difference-from-mean values and chromatic ellipses. The effects of different cultural groups and experimental settings were also shown.

No Evidence of Narrowly Defined Cognitive Penetrability in Unambiguous Vision

The classical notion of cognitive impenetrability suggests that perceptual processing is an automatic modular system and not under conscious control. Near consensus is now emerging that this classical notion is untenable. However, as recently pointed out by Firestone and Scholl, this consensus is built on quicksand. In most studies claiming perception is cognitively penetrable, it remains unclear which actual process has been affected (perception, memory, imagery, input selection or judgment). In fact, the only available "proofs" for cognitive penetrability are proxies for perception, such as behavioral responses and neural correlates. We suggest that one can interpret cognitive penetrability in two different ways, a broad sense and a narrow sense. In the broad sense, attention and memory are not considered as "just" pre- and post-perceptual systems but as part of the mechanisms by which top-down processes influence the actual percept. Although many studies have proven top-down influences in this broader sense, it is still debatable whether cognitive penetrability remains tenable in a narrow sense. The narrow sense states that cognitive penetrability only occurs when top-down factors are flexible and cause a clear illusion from a first person perspective. So far, there is no strong evidence from a first person perspective that visual illusions can indeed be driven by high-level flexible factors. One cannot be cognitively trained to see and unsee visual illusions. We argue that this lack of convincing proof for cognitive penetrability in the narrow sense can be explained by the fact that most research focuses on foveal vision only. This type of perception may be too unambiguous for transient high-level factors to control perception. Therefore, illusions in more ambiguous perception, such as peripheral vision, can offer a unique insight into the matter. They produce a clear subjective percept based on unclear, degraded visual input: the optimal basis to study narrowly defined cognitive penetrability.

Changing What You See by Changing What You Know: The Role of Attention

Attending is a cognitive process that incorporates a person's knowledge, goals, and expectations. What we perceive when we attend to one thing is different from what we perceive when we attend to something else. Yet, it is often argued that attentional effects do not count as evidence that perception is influenced by cognition. I investigate two arguments often given to justify excluding attention. The first is arguing that attention is a post-perceptual process reflecting selection between fully constructed perceptual representations. The second is arguing that attention as a pre-perceptual process that simply changes the input to encapsulated perceptual systems. Both of these arguments are highly problematic. Although some attentional effects can indeed be construed as post-perceptual, others operate by changing perceptual content across the entire visual hierarchy. Although there is a natural analogy between spatial attention and a change of input, the analogy falls apart when we consider other forms of attention. After dispelling these arguments, I make a case for thinking of attention not as a confound, but as one of the mechanisms by which cognitive states affect perception by going through cases in which the same or similar visual inputs are perceived differently depending on the observer's cognitive state, and instances where cuing an observer using language affects what one sees. Lastly, I provide two compelling counter-examples to the critique that although cognitive influences on perception can be demonstrated in the laboratory, it is impossible to really experience them for oneself in a phenomenologically compelling way. Taken together, the current evidence strongly supports the thesis that what we know routinely influences what we see, that the same sensory input can be perceived differently depending on the current cognitive state of the viewer, and that phenomenologically salient demonstrations are possible if certain conditions are met.

Threefoldness

Theories of picture perception aim to understand our perceptual relation to both the picture surface and the depicted object. I argue that we should talk about not two, but three entities when understanding picture perception: (A) the picture surface, (B) the three dimensional object the picture surface visually encodes and (C) the three dimensional depicted object. As (B) and (C) can come apart, we get a more complex picture of picture perception than normally assumed and one where the notion of twofoldness, which has played an important albeit controversial role in understanding picture perception is replaced by the concept of threefoldness.

Memory colours affect colour appearance

Memory colour effects show that colour perception is affected by memory and prior knowledge and hence by cognition. None of Firestone & Scholl's (F&S's) potential pitfalls apply to our work on memory colours. We present a Bayesian model of colour appearance to illustrate that an interaction between perception and memory is plausible from the perspective of vision science.

Cortical Color and the Cognitive Sciences

Back when researchers thought about the various forms that color vision could take, the focus was primarily on the retinal mechanisms. Since that time, research on human color vision has shifted from an interest in retinal mechanisms to cortical color processing. This has allowed color research to provide insight into questions that are not limited to early vision but extend to cognition. Direct cortical connections from higher-level areas to lower-level areas have been found throughout the brain. One of the classic questions in cognitive science is whether perception is influenced, and if so to what extent, by cognition and whether a clear distinction can be drawn between perception and cognition. Since perception is seen as providing justification for our beliefs about properties in the external world, these questions also have metaphysical and epistemological significance. The aim of this paper is to highlight some of the areas where research on color perception can shed new light on questions in the cognitive sciences. A further aim of the paper is to raise some questions about color research that are in dire need of further reflection and investigation.

The Myth of Color Sensations, or How Not to See a Yellow Banana

I argue against a class of philosophical views of color perception, especially insofar as such views posit the existence of color sensations. I argue against the need to posit such nonconceptual mental intermediaries between the stimulus and the eventual conceptualized perceptual judgment. Central to my arguments are considerations of certain color illusions. Such illusions are best explained by reference to high-level, conceptualized knowledge concerning, for example, object identity, likely lighting conditions, and material composition of the distal stimulus. Such explanations obviate the need to appeal to nonconceputal mental links in the causal chains eventuating in conceptualized color discriminations.

Is Color Experience Cognitively Penetrable?

Is color experience cognitively penetrable? Some philosophers have recently argued that it is. In this paper, we take issue with the claim that color experience is cognitively penetrable. We argue that the notion of cognitive penetration that has recently dominated the literature is flawed since it fails to distinguish between the modulation of perceptual content by non-perceptual principles and genuine cognitive penetration. We use this distinction to show that studies suggesting that color experience can be modulated by factors of the cognitive system do not establish that color experience is cognitively penetrable. Additionally, we argue that even if color experience turns out to be modulated by color-related beliefs and knowledge beyond non-perceptual principles, it does not follow that color experience is cognitively penetrable since the experiences of determinate hues involve post-perceptual processes. We conclude with a brief discussion of the implications that these ideas may have on debates in philosophy.

An Easy Way to Show Memory Color Effects

This study proposes and evaluates a simple stimulus display that allows one to measure memory color effects (the effect of object knowledge and memory on color perception). The proposed approach is fast and easy and does not require running an extensive experiment. It shows that memory color effects are robust to minor variations due to a lack of color calibration.

Colour Vision: Understanding #TheDress

A widely-viewed image of a dress elicits striking individual variation in colour perception. Experiments with multiple variants of the image suggest that the individual differences may arise through the action of visual mechanisms that normally stabilise object colour.

Object knowledge changes visual appearance: semantic effects on color afterimages

According to predictive coding models of perception, what we see is determined jointly by the current input and the priors established by previous experience, expectations, and other contextual factors. The same input can thus be perceived differently depending on the priors that are brought to bear during viewing. Here, I show that expected (diagnostic) colors are perceived more vividly than arbitrary or unexpected colors, particularly when color input is unreliable. Participants were tested on a version of the 'Spanish Castle Illusion' in which viewing a hue-inverted image renders a subsequently shown achromatic version of the image in vivid color. Adapting to objects with intrinsic colors (e.g., a pumpkin) led to stronger afterimages than adapting to arbitrarily colored objects (e.g., a pumpkin-colored car). Considerably stronger afterimages were also produced by scenes containing intrinsically colored elements (grass, sky) compared to scenes with arbitrarily colored objects (books). The differences between images with diagnostic and arbitrary colors disappeared when the association between the image and color priors was weakened by, e.g., presenting the image upside-down, consistent with the prediction that color appearance is being modulated by color knowledge. Visual inputs that conflict with prior knowledge appear to be phenomenologically discounted, but this discounting is moderated by input certainty, as shown by the final study which uses conventional images rather than afterimages. As input certainty is increased, unexpected colors can become easier to detect than expected ones, a result consistent with predictive-coding models.

Cognition does not affect perception: Evaluating the evidence for “top-down” effects

What determines what we see? In contrast to the traditional “modular” understanding of perception, according to which visual processing is encapsulated from higher-level cognition, a tidal wave of recent research alleges that states such as beliefs, desires, emotions, motivations, intentions, and linguistic representations exert direct, top-down influences on what we see. There is a growing consensus that such effects are ubiquitous, and that the distinction between perception and cognition may itself be unsustainable. We argue otherwise: None of these hundreds of studies – either individually or collectively – provides compelling evidence for true top-down effects on perception, or “cognitive penetrability.” In particular, and despite their variety, we suggest that these studies all fall prey to only a handful of pitfalls. And whereas abstract theoretical challenges have failed to resolve this debate in the past, our presentation of these pitfalls is empirically anchored: In each case, we show not only how certain studies could be susceptible to the pitfall (in principle), but also how several alleged top-down effects actually are explained by the pitfall (in practice). Moreover, these pitfalls are perfectly general, with each applying to dozens of other top-down effects. We conclude by extracting the lessons provided by these pitfalls into a checklist that future work could use to convincingly demonstrate top-down effects on visual perception. The discovery of substantive top-down effects of cognition on perception would revolutionize our understanding of how the mind is organized; but without addressing these pitfalls, no such empirical report will license such exciting conclusions.

Striking individual differences in color perception uncovered by 'the dress' photograph

'The dress' is a peculiar photograph: by themselves the dress' pixels are brown and blue, colors associated with natural illuminants, but popular accounts (#TheDress) suggest the dress appears either white/gold or blue/black. Could the purported categorical perception arise because the original social-media question was an alternative-forced-choice? In a free-response survey (N = 1401), we found that most people, including those naïve to the image, reported white/gold or blue/black, but some said blue/brown. Reports of white/gold over blue/black were higher among older people and women. On re-test, some subjects reported a switch in perception, showing the image can be multistable. In a language-independent measure of perception, we asked subjects to identify the dress' colors from a complete color gamut. The results showed three peaks corresponding to the main descriptive categories, providing additional evidence that the brain resolves the image into one of three stable percepts. We hypothesize that these reflect different internal priors: some people favor a cool illuminant (blue sky), discount shorter wavelengths, and perceive white/gold; others favor a warm illuminant (incandescent light), discount longer wavelengths, and see blue/black. The remaining subjects may assume a neutral illuminant, and see blue/brown. We show that by introducing overt cues to the illumination, we can flip the dress color.