Semantic versus perceptual influences of color in object recognition

Improved Color Mapping Methods for Multiband Nighttime Image Fusion

Previously, we presented two color mapping methods for the application of daytime colors to fused nighttime (e.g., intensified and longwave infrared or thermal (LWIR)) imagery. These mappings not only impart a natural daylight color appearance to multiband nighttime images but also enhance their contrast and the visibility of otherwise obscured details. As a result, it has been shown that these colorizing methods lead to an increased ease of interpretation, better discrimination and identification of materials, faster reaction times and ultimately improved situational awareness. A crucial step in the proposed coloring process is the choice of a suitable color mapping scheme. When both daytime color images and multiband sensor images of the same scene are available, the color mapping can be derived from matching image samples (i.e., by relating color values to sensor output signal intensities in a sample-based approach). When no exact matching reference images are available, the color transformation can be derived from the first-order statistical properties of the reference image and the multiband sensor image. In the current study, we investigated new color fusion schemes that combine the advantages of both methods (i.e., the efficiency and color constancy of the sample-based method with the ability of the statistical method to use the image of a different but somewhat similar scene as a reference image), using the correspondence between multiband sensor values and daytime colors (sample-based method) in a smooth transformation (statistical method). We designed and evaluated three new fusion schemes that focus on (i) a closer match with the daytime luminances; (ii) an improved saliency of hot targets; and (iii) an improved discriminability of materials. We performed both qualitative and quantitative analyses to assess the weak and strong points of all methods.

The color "fruit": object memories defined by color

Most fruits and other highly color-diagnostic objects have color as a central aspect of their identity, which can facilitate detection and visual recognition. It has been theorized that there may be a large amount of overlap between the neural representations of these objects and processing involved in color perception. In accordance with this theory we sought to determine if the recognition of highly color diagnostic fruit objects could be facilitated by the visual presentation of their known color associates. In two experiments we show that color associate priming is possible, but contingent upon multiple factors. Color priming was found to be maximally effective for the most highly color diagnostic fruits, when low spatial-frequency information was present in the image, and when determination of the object's specific identity, not merely its category, was required. These data illustrate the importance of color for determining the identity of certain objects, and support the theory that object knowledge involves sensory specific systems.

Haptic object perception: spatial dimensionality and relation to vision

Enabled by the remarkable dexterity of the human hand, specialized haptic exploration is a hallmark of object perception by touch. Haptic exploration normally takes place in a spatial world that is three-dimensional; nevertheless, stimuli of reduced spatial dimensionality are also used to display spatial information. This paper examines the consequences of full (three-dimensional) versus reduced (two-dimensional) spatial dimensionality for object processing by touch, particularly in comparison with vision. We begin with perceptual recognition of common human-made artefacts, then extend our discussion of spatial dimensionality in touch and vision to include faces, drawing from research on haptic recognition of facial identity and emotional expressions. Faces have often been characterized as constituting a specialized input for human perception. We find that contrary to vision, haptic processing of common objects is impaired by reduced spatial dimensionality, whereas haptic face processing is not. We interpret these results in terms of fundamental differences in object perception across the modalities, particularly the special role of manual exploration in extracting a three-dimensional structure.

The interaction between surface color and color knowledge: behavioral and electrophysiological evidence

In this study, we used event-related potentials (ERPs) to evaluate the contribution of surface color and color knowledge information in object identification. We constructed two color-object verification tasks - a surface and a knowledge verification task - using high color diagnostic objects; both typical and atypical color versions of the same object were presented. Continuous electroencephalogram was recorded from 26 subjects. A cluster randomization procedure was used to explore the differences between typical and atypical color objects in each task. In the color knowledge task, we found two significant clusters that were consistent with the N350 and late positive complex (LPC) effects. Atypical color objects elicited more negative ERPs compared to typical color objects. The color effect found in the N350 time window suggests that surface color is an important cue that facilitates the selection of a stored object representation from long-term memory. Moreover, the observed LPC effect suggests that surface color activates associated semantic knowledge about the object, including color knowledge representations. We did not find any significant differences between typical and atypical color objects in the surface color verification task, which indicates that there is little contribution of color knowledge to resolve the surface color verification. Our main results suggest that surface color is an important visual cue that triggers color knowledge, thereby facilitating object identification.

The role of color information on object recognition: a review and meta-analysis

In this study, we systematically review the scientific literature on the effect of color on object recognition. Thirty-five independent experiments, comprising 1535 participants, were included in a meta-analysis. We found a moderate effect of color on object recognition (d=0.28). Specific effects of moderator variables were analyzed and we found that color diagnosticity is the factor with the greatest moderator effect on the influence of color in object recognition; studies using color diagnostic objects showed a significant color effect (d=0.43), whereas a marginal color effect was found in studies that used non-color diagnostic objects (d=0.18). The present study did not permit the drawing of specific conclusions about the moderator effect of the object recognition task; while the meta-analytic review showed that color information improves object recognition mainly in studies using naming tasks (d=0.36), the literature review revealed a large body of evidence showing positive effects of color information on object recognition in studies using a large variety of visual recognition tasks. We also found that color is important for the ability to recognize artifacts and natural objects, to recognize objects presented as types (line-drawings) or as tokens (photographs), and to recognize objects that are presented without surface details, such as texture or shadow. Taken together, the results of the meta-analysis strongly support the contention that color plays a role in object recognition. This suggests that the role of color should be taken into account in models of visual object recognition.

The influence of color information on the recognition of color diagnostic and noncolor diagnostic objects

In the present study, the authors explore in detail the level of visual object recognition at which perceptual color information improves the recognition of color diagnostic and noncolor diagnostic objects. To address this issue, 3 object recognition tasks with different cognitive demands were designed: (a) an object verification task; (b) a category verification task; and (c) a name verification task. The authors found that perceptual color information improved color diagnostic object recognition mainly in tasks for which access to the semantic knowledge about the object was necessary to perform the task; that is, in category and name verification. In contrast, the authors found that perceptual color information facilitates noncolor diagnostic object recognition when access to the object's structural description from long-term memory was necessary--that is, object verification. In summary, the present study shows that the role of perceptual color information in object recognition is dependent on color diagnosticity.

Object knowledge modulates colour appearance

We investigated the memory colour effect for colour diagnostic artificial objects. Since knowledge about these objects and their colours has been learned in everyday life, these stimuli allow the investigation of the influence of acquired object knowledge on colour appearance. These investigations are relevant for questions about how object and colour information in high-level vision interact as well as for research about the influence of learning and experience on perception in general. In order to identify suitable artificial objects, we developed a reaction time paradigm that measures (subjective) colour diagnosticity. In the main experiment, participants adjusted sixteen such objects to their typical colour as well as to grey. If the achromatic object appears in its typical colour, then participants should adjust it to the opponent colour in order to subjectively perceive it as grey. We found that knowledge about the typical colour influences the colour appearance of artificial objects. This effect was particularly strong along the daylight axis.

Looking at anything that is green when hearing "frog": how object surface colour and stored object colour knowledge influence language-mediated overt attention

Three eye-tracking experiments investigated the influence of stored colour knowledge, perceived surface colour, and conceptual category of visual objects on language-mediated overt attention. Participants heard spoken target words whose concepts are associated with a diagnostic colour (e.g., "spinach"; spinach is typically green) while their eye movements were monitored to (a) objects associated with a diagnostic colour but presented in black and white (e.g., a black-and-white line drawing of a frog), (b) objects associated with a diagnostic colour but presented in an appropriate but atypical colour (e.g., a colour photograph of a yellow frog), and (c) objects not associated with a diagnostic colour but presented in the diagnostic colour of the target concept (e.g., a green blouse; blouses are not typically green). We observed that colour-mediated shifts in overt attention are primarily due to the perceived surface attributes of the visual objects rather than stored knowledge about the typical colour of the object. In addition our data reveal that conceptual category information is the primary determinant of overt attention if both conceptual category and surface colour competitors are copresent in the visual environment.

Independent effects of colour on object identification and memory

We examined the effects of colour on object identification and memory using a study–test priming procedure with a coloured-object decision task at test (i.e., deciding whether an object is correctly coloured). Objects were selected to have a single associated colour and were either correctly or incorrectly coloured. In addition, object shape and colour were either spatially integrated (i.e., colour fell on the object surface) or spatially separated (i.e., colour formed the background to the object). Transforming the colour of an object from study to test (e.g., from a yellow banana to a purple banana) reduced priming of response times, as compared to when the object was untransformed. This utilization of colour information in object memory was not contingent upon colour falling on the object surface or whether the resulting configuration was of a correctly or incorrectly coloured object. In addition, we observed independent effects of colour on response times, whereby coloured-object decisions were more efficient for correctly than for incorrectly coloured objects but only when colour fell on the object surface. These findings provide evidence for two distinct mechanisms of shape–colour binding in object processing.

The influence of colour and sound on neuronal activation during visual object naming

This paper investigates how neuronal activation for naming photographs of objects is influenced by the addition of appropriate colour or sound. Behaviourally, both colour and sound are known to facilitate object recognition from visual form. However, previous functional imaging studies have shown inconsistent effects. For example, the addition of appropriate colour has been shown to reduce antero-medial temporal activation whereas the addition of sound has been shown to increase posterior superior temporal activation. Here we compared the effect of adding colour or sound cues in the same experiment. We found that the addition of either the appropriate colour or sound increased activation for naming photographs of objects in bilateral occipital regions and the right anterior fusiform. Moreover, the addition of colour reduced left antero-medial temporal activation but this effect was not observed for the addition of object sound. We propose that activation in bilateral occipital and right fusiform areas precedes the integration of visual form with either its colour or associated sound. In contrast, left antero-medial temporal activation is reduced because object recognition is facilitated after colour and form have been integrated.

Colour agnosia impairs the recognition of natural but not of non-natural scenes

Scene recognition can be enhanced by appropriate colour information, yet the level of visual processing at which colour exerts its effects is still unclear. It has been suggested that colour supports low-level sensory processing, while others have claimed that colour information aids semantic categorization and recognition of objects and scenes. We investigated the effect of colour on scene recognition in a case of colour agnosia, M.A.H. In a scene identification task, participants had to name images of natural or non-natural scenes in six different formats. Irrespective of scene format, M.A.H. was much slower on the natural than on the non-natural scenes. As expected, neither M.A.H. nor control participants showed any difference in performance for the non-natural scenes. However, for the natural scenes, appropriate colour facilitated scene recognition in control participants (i.e., shorter reaction times), whereas M.A.H.'s performance did not differ across formats. Our data thus support the hypothesis that the effect of colour occurs at the level of learned associations.

Effects of Delay on Color Priming for Natural Objects

Using a standard study-test procedure, color priming was examined through effects of color transformation, from correctly colored to incorrectly colored and vice versa, for natural objects with pre-existing color-shape associations, e.g., yellow banana. More specifically these effects were examined at study-test delays of 0, 24, and 48 hr. When deciding whether an object was correctly colored, color transformation eliminated priming. Furthermore, there was evidence that for objects that were not transformed, priming was stronger for correctly as compared with incorrectly colored objects. In addition, the introduction of 24- and 48-hr. delays between the study and the test phase of the task reduced the effects of color transformation on priming. These findings are discussed in terms of the representations that mediate implicit memory performance.

Prefrontal engagement during source memory retrieval depends on the prior encoding task

The prefrontal cortex is strongly engaged by some, but not all, episodic memory tests. Prior work has shown that source recognition tests--those that require memory for conjunctions of studied attributes--yield deficient performance in patients with prefrontal damage and greater prefrontal activity in healthy subjects, as compared to simple recognition tests. Here, we tested the hypothesis that there is no intrinsic relationship between the prefrontal cortex and source memory, but that the prefrontal cortex is engaged by the demand to retrieve weakly encoded relationships. Subjects attempted to remember object/color conjunctions after an encoding task that focused on object identity alone, and an integrative encoding task that encouraged attention to object/color relationships. After the integrative encoding task, the late prefrontal brain electrical activity that typically occurs in source memory tests was eliminated. Earlier brain electrical activity related to successful recognition of the objects was unaffected by the nature of prior encoding.

Cortical regions associated with different aspects of object recognition performance

In the present object recognition study, we examined the relationship between brain activation and four behavioral measures: error rate, reaction time, observer sensitivity, and response bias. Subjects perceptually matched object pairs in which structural similarity (SS), an index of structural differentiation, and exposure duration (DUR), an index of task difficulty, were manipulated. The SS manipulation affected the fMRI signal in the left anterior fusiform and parietal cortices, which in turn reflected a bias to respond same. Conversely, an SS-modulated fMRI signal in the right middle frontal gyrus reflected a bias to respond different. The DUR manipulation affected the fMRI signal in occipital and posterior fusiform regions, which in turn reflected greater sensitivity, longer reaction times, and greater accuracy. These findings demonstrate that the regions most strongly implicated in processing object shape (SS-modulated regions) are associated with response bias, whereas regions that are not directly involved in shape processing are associated with successful recognition performance.

The effect of colour congruency on shape discriminations of novel objects

Although visual object recognition is primarily shape driven, colour assists the recognition of some objects. It is unclear, however, just how colour information is coded with respect to shape in long-term memory and how the availability of colour in the visual image facilitates object recognition. We examined the role of colour in the recognition of novel, 3-D objects by manipulating the congruency of object colour across the study and test phases, using an old/new shape-identification task. In experiment 1, we found that participants were faster at correctly identifying old objects on the basis of shape information when these objects were presented in their original colour, rather than in a different colour. In experiments 2 and 3, we found that participants were faster at correctly identifying old objects on the basis of shape information when these objects were presented with their original part-colour conjunctions, rather than in different or in reversed part-colour conjunctions. In experiment 4, we found that participants were quite poor at the verbal recall of part-colour conjunctions for correctly identified old objects, presented as grey-scale images at test. In experiment 5, we found that participants were significantly slower at correctly identifying old objects when object colour was incongruent across study and test, than when background colour was incongruent across study and test. The results of these experiments suggest that both shape and colour information are stored as part of the long-term representation of these novel objects. Results are discussed in terms of how colour might be coded with respect to shape in stored object representations.

The role of colour in implicit and explicit memory performance

We present two experiments that examine the effects of colour transformation between study and test (from black and white to colour and vice versa, of from incorrectly coloured to correctly coloured and vice versa) on implicit and explicit measures of memory for diagnostically coloured natural objects (e.g., yellow banana). For naming and coloured-object decision (i.e., deciding whether an object is correctly coloured), there were shorter response times to correctly coloured-objects than to black-and-white and incorrectly coloured-objects. Repetition priming was equivalent for the different stimulus types. Colour transformation did not influence priming of picture naming, but for coloured-object decision priming was evident only for objects remaining the same from study to test. This was the case for both naming and coloured-object decision as study tasks. When participants were asked to consciously recognize objects that they had named or made coloured-object decisions to previously, whilst ignoring their colour, colour transformation reduced recognition efficiency. We discuss these results in terms of the flexibility of object representations that mediate priming and recognition.

Gustave Caillebotte, French impressionism, and mere exposure

Gustave Caillebotte was a painter, a collector of some of his colleagues' most renowned works, and a major force in the creation of the late 19th century French Impressionist canon. Six studies are presented as a naturalistic investigation of the effects of mere exposure to images in his collection and to those matched to them. The probabilities of cultural exposure to the 132 stimulus images were indexed by the frequencies of their separate appearances in Cornell University library books--a total of 4,232 times in 980 different books. Across the studies, adult preferences were correlated with differences in image frequencies, but not with recognition, complexity, or prototypicality judgments; children's preferences were not correlated with frequency. Prior cultural exposure also interacted with experimental exposure in predictable ways. The results suggest that mere exposure helps to maintain an artistic canon.

Effects of structural similarity on neural substrates for object recognition

Human occipitotemporal cortex (OTC) is critically involved in object recognition, but the functional organization of this brain region is controversial. In the present study, functional magnetic resonance imaging (fMRI) signal changes were recorded in humans during an animal-matching task that parametrically varied degree of structural (i.e., shape) similarity among the items. fMRI signal in the mid- to anterior-fusiform gyrus increased as animals overlapped more in terms of structure and as reaction time increased. In contrast, relatively more posterior aspects of the fusiform gyrus and inferior occipital cortex showed greater fMRI signal when the animals overlapped less in terms of structure. A similar organization emerged when three-dimensional geometric shapes were matched, indicating that OTC is differentially tuned to varying degrees of overlap in object structure, regardless of taxonomic category. We discuss how the present findings fit in with current functional neuroanatomical approaches to object recognition.

Visual object categorisation at distinct levels of abstraction: a new stimulus set

We designed a new stimulus set with 269 line drawings of everyday artifacts and animals. The stimulus set contains several typical exemplars from a sample of 25 basic-level categories. We determined to what extent these stimuli were named at the basic level and at a more subordinate level. An additional experiment showed the validity of this calibration: typicality ratings were correlated significantly with the level of naming. In a final experiment we found that this effect depends largely on the global configuration of a stimulus as it was still apparent with degraded images obtained by locally shifting small fragments of the drawings.

Surface cues reduce the latency to name rotated images of objects

Jolicoeur (1985, Memory & Cognition 13 289-303) found a linear increase in the latency to name line drawings of objects rotated (0 degrees to 120 degrees) from the upright (0 degrees) in the initial trial block. This effect was much shallower in later blocks. He proposed that the initial effect may indicate that mental rotation is the default process for recognising rotated objects, and that the decrease in this effect, seen with practice, may reflect the increased use of learned orientation-invariant features. Initially, we were interested in whether object-colour associations that may be learned during the initial block, could account for the reduced latency to name rotated objects, seen in later blocks. In experiment 1 we used full-cue colour images of objects that depicted colour and other surface cues. Surprisingly, given that Jolicoeur's findings were replicated several times with line drawings, we found that even the initial linear trend in naming latency was shallow. We replicated this result in follow-up experiments. In contrast, when we used less-realistic depictions of the same objects that had fewer visual cues (ie line drawings, coloured drawings, greyscale images), the results were comparable to those of Jolicoeur. Also, the initial linear trends were steeper for these depictions than for full-cue colour images. The results suggest that, when multiple surface cues are available in the image, mental rotation may not be the default recognition process.

The role of color in high-level vision

Traditional theories of object recognition have emphasized the role of shape information in high-level vision. However, the accumulating behavioral, neuroimaging and neuropsychological evidence indicates that the surface color of an object affects its recognition. In this article, we discuss the research that examines the conditions under which color influences the operations of high-level vision and the neural substrates that might mediate these operations. The relationship between object color and object recognition is summarized in the 'Shape+Surface' model of high-level vision.

P and M channel-specific interference in the what and where pathway

It was the purpose of this experiment to study interference effects of colour or luminance peripheral flicker (in order to saturate either the parvocellular or the magnocellular stream) on object identity and spatial location memory. Colour flicker interfered with object identity recognition, whereas luminance flicker affected memory for spatial location. Moreover, overall performance was worse if coloured rather than grey-scaled objects were used in the stimulus display. There was no selective effect of coloured flicker affecting coloured objects and chromatic flicker affecting chromatic objects. These results provide strong evidence for the theoretical position that the what pathway relies heavily on information derived from the P stream and the where pathway on the M stream.

Color diagnosticity in object recognition

Does color influence object recognition? In the present study, the degree to which an object was associated with a specific color was referred to as color diagnosticity. Using a feature listing and typicality measure, objects were identified as either high in color diagnosticity or low in color diagnosticity. According to the color diagnosticity hypothesis, color should more strongly influence the recognition of high color diagnostic (HCD) objects (e.g., a banana) than the recognition of low color diagnostic (LCD) objects (e.g., a lamp). This prediction was supported by results from classification, naming, and verification experiments, in which subjects were faster to identify color versions of HCD objects than they were to identify achromatic versions and incongruent color versions. In contrast, subjects were no faster to identify color versions of LCD objects than they were to identify achromatic and incongruent color versions. Moreover, when shape information was degraded but color information preserved, subjects were less impaired in their recognition of degraded HCD objects than of degraded LCD objects, relative to their nondegraded versions. Collectively, these results suggest that color plays a role in the recognition of HCD objects.

Cortical regions associated with perceiving, naming, and knowing about colors

Positron emission tomography (PET) was used to investigate whether retrieving information about a specific object attribute requires reactivation of brain areas that mediate perception of that attribute. During separate PET scans, subjects passively viewed colored and equiluminant gray-scale Mondrians, named colored and achromatic objects, named the color of colored objects, and generated color names associated with achromatic objects. Color perception was associated with activations in the lingual and fusiform gyri of the occipital lobes, consistent with previous neuroimaging and human lesion studies. Retrieving information about object color (generating color names for achromatic objects relative to naming achromatic objects) activated the left inferior temporal, left frontal, and left posterior parietal cortices, replicating previous findings from this laboratory. When subjects generated color names for achromatic objects relative to the low-level baseline of viewing gray-scale Mondrians, additional activations in the left fusiform/lateral occipital region were detected. However, these activations were lateral to the occipital regions associated with color perception and identical to occipital regions activated when subjects simply named achromatic objects relative to the same low-level baseline. This suggests that the occipital activations associated with retrieving color information were due to the perception of object form rather than to the top-down influence of brain areas that mediate color perception. Taken together, these results indicate that retrieving previously acquired information about an object's typical color does not require reactivation of brain regions that subserve color perception.