1
|
de Vries JP, Flachot A, Morimoto T, Gegenfurtner KR. A deep new look at color. Behav Brain Sci 2023; 46:e389. [PMID: 38054295 DOI: 10.1017/s0140525x23001620] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/07/2023]
Abstract
Bowers et al. counter deep neural networks (DNNs) as good models of human visual perception. From our color perspective we feel their view is based on three misconceptions: A misrepresentation of the state-of-the-art of color perception; the type of model required to move the field forward; and the attribution of shortcomings to DNN research that are already being resolved.
Collapse
Affiliation(s)
- Jelmer Philip de Vries
- Department of Psychology, Justus Liebig Universitat, Giessen, Germany ; www.jelmerdevries.com; https://www.allpsych.uni-giessen.de/karl/
| | - Alban Flachot
- Department of Psychology, York University, Toronto, ON, Canada
| | - Takuma Morimoto
- Department of Psychology, Justus Liebig Universitat, Giessen, Germany ; www.jelmerdevries.com; https://www.allpsych.uni-giessen.de/karl/
- Department of Experimental Psychology, University of Oxford, Oxford, UK ; https://sites.google.com/view/tmorimoto
| | - Karl R Gegenfurtner
- Department of Psychology, Justus Liebig Universitat, Giessen, Germany ; www.jelmerdevries.com; https://www.allpsych.uni-giessen.de/karl/
| |
Collapse
|
2
|
Bun LM, Horwitz GD. Color and luminance processing in V1 complex cells and artificial neural networks. COLOR RESEARCH AND APPLICATION 2023; 48:841-852. [PMID: 38145033 PMCID: PMC10746296 DOI: 10.1002/col.22903] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 09/03/2023] [Indexed: 12/26/2023]
Abstract
Object recognition by natural and artificial visual systems benefits from the identification of object boundaries. A useful cue for the detection of object boundaries is the superposition of luminance and color edges. To gain insight into the suitability of this cue for object recognition, we examined convolutional neural network models that had been trained to recognize objects in natural images. We focused specifically on units in the second convolutional layer whose activations are invariant to the spatial phase of a sinusoidal grating. Some of these units were tuned for a nonlinear combination of color and luminance, which is broadly consistent with a role in object boundary detection. Others were tuned for luminance alone, but very few were tuned for color alone. A literature review reveals that V1 complex cells have a similar distribution of tuning. We speculate that this pattern of sensitivity provides an efficient basis for object recognition, perhaps by mitigating the effects of lighting on luminance contrast polarity. The absence of a contrast polarity-invariant representation of chromaticity alone suggests that it is redundant with other representations.
Collapse
Affiliation(s)
- Luke M. Bun
- Department of Bioengineering
- Washington National Primate Research Center
| | - Gregory D. Horwitz
- Department of Bioengineering
- Washington National Primate Research Center
- Department of Physiology and Biophysics, University of Washington, Seattle, WA, 98195
| |
Collapse
|
3
|
Sanchez-Cesteros O, Rincon M, Bachiller M, Valladares-Rodriguez S. A Long Skip Connection for Enhanced Color Selectivity in CNN Architectures. SENSORS (BASEL, SWITZERLAND) 2023; 23:7582. [PMID: 37688036 PMCID: PMC10490730 DOI: 10.3390/s23177582] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/21/2023] [Revised: 08/25/2023] [Accepted: 08/29/2023] [Indexed: 09/10/2023]
Abstract
Some recent studies show that filters in convolutional neural networks (CNNs) have low color selectivity in datasets of natural scenes such as Imagenet. CNNs, bio-inspired by the visual cortex, are characterized by their hierarchical learning structure which appears to gradually transform the representation space. Inspired by the direct connection between the LGN and V4, which allows V4 to handle low-level information closer to the trichromatic input in addition to processed information that comes from V2/V3, we propose the addition of a long skip connection (LSC) between the first and last blocks of the feature extraction stage to allow deeper parts of the network to receive information from shallower layers. This type of connection improves classification accuracy by combining simple-visual and complex-abstract features to create more color-selective ones. We have applied this strategy to classic CNN architectures and quantitatively and qualitatively analyzed the improvement in accuracy while focusing on color selectivity. The results show that, in general, skip connections improve accuracy, but LSC improves it even more and enhances the color selectivity of the original CNN architectures. As a side result, we propose a new color representation procedure for organizing and filtering feature maps, making their visualization more manageable for qualitative color selectivity analysis.
Collapse
Affiliation(s)
- Oscar Sanchez-Cesteros
- Department of Artificial Intelligence, National University of Distance Education (UNED), 28040 Madrid, Spain; (M.R.); (M.B.); (S.V.-R.)
| | - Mariano Rincon
- Department of Artificial Intelligence, National University of Distance Education (UNED), 28040 Madrid, Spain; (M.R.); (M.B.); (S.V.-R.)
| | - Margarita Bachiller
- Department of Artificial Intelligence, National University of Distance Education (UNED), 28040 Madrid, Spain; (M.R.); (M.B.); (S.V.-R.)
| | - Sonia Valladares-Rodriguez
- Department of Artificial Intelligence, National University of Distance Education (UNED), 28040 Madrid, Spain; (M.R.); (M.B.); (S.V.-R.)
- Department of Electronics and Computing, University of Santiago de Compostela (USC), 15705 Santiago de Compostela, Spain
| |
Collapse
|
4
|
Taylor J, Xu Y. Comparing the Dominance of Color and Form Information across the Human Ventral Visual Pathway and Convolutional Neural Networks. J Cogn Neurosci 2023; 35:816-840. [PMID: 36877074 PMCID: PMC11283826 DOI: 10.1162/jocn_a_01979] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/07/2023]
Abstract
Color and form information can be decoded in every region of the human ventral visual hierarchy, and at every layer of many convolutional neural networks (CNNs) trained to recognize objects, but how does the coding strength of these features vary over processing? Here, we characterize for these features both their absolute coding strength-how strongly each feature is represented independent of the other feature-and their relative coding strength-how strongly each feature is encoded relative to the other, which could constrain how well a feature can be read out by downstream regions across variation in the other feature. To quantify relative coding strength, we define a measure called the form dominance index that compares the relative influence of color and form on the representational geometry at each processing stage. We analyze brain and CNN responses to stimuli varying based on color and either a simple form feature, orientation, or a more complex form feature, curvature. We find that while the brain and CNNs largely differ in how the absolute coding strength of color and form vary over processing, comparing them in terms of their relative emphasis of these features reveals a striking similarity: For both the brain and for CNNs trained for object recognition (but not for untrained CNNs), orientation information is increasingly de-emphasized, and curvature information is increasingly emphasized, relative to color information over processing, with corresponding processing stages showing largely similar values of the form dominance index.
Collapse
|
5
|
Heidari-Gorji H, Gegenfurtner KR. Object-based color constancy in a deep neural network. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2023; 40:A48-A56. [PMID: 37133003 DOI: 10.1364/josaa.479451] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Color constancy refers to our capacity to see consistent colors under different illuminations. In computer vision and image processing, color constancy is often approached by explicit estimation of the scene's illumination, followed by an image correction. In contrast, color constancy in human vision is typically measured as the capacity to extract color information about objects and materials in a scene consistently throughout various illuminations, which goes beyond illumination estimation and might require some degree of scene and color understanding. Here, we pursue an approach with deep neural networks that tries to assign reflectances to individual objects in the scene. To circumvent the lack of massive ground truth datasets labeled with reflectances, we used computer graphics to render images. This study presents a model that recognizes colors in an image pixel by pixel under different illumination conditions.
Collapse
|
6
|
Flachot A, Akbarinia A, Schütt HH, Fleming RW, Wichmann FA, Gegenfurtner KR. Deep neural models for color classification and color constancy. J Vis 2022; 22:17. [PMID: 35353153 PMCID: PMC8976922 DOI: 10.1167/jov.22.4.17] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Color constancy is our ability to perceive constant colors across varying illuminations. Here, we trained deep neural networks to be color constant and evaluated their performance with varying cues. Inputs to the networks consisted of two-dimensional images of simulated cone excitations derived from three-dimensional (3D) rendered scenes of 2,115 different 3D shapes, with spectral reflectances of 1,600 different Munsell chips, illuminated under 278 different natural illuminations. The models were trained to classify the reflectance of the objects. Testing was done with four new illuminations with equally spaced CIEL*a*b* chromaticities, two along the daylight locus and two orthogonal to it. High levels of color constancy were achieved with different deep neural networks, and constancy was higher along the daylight locus. When gradually removing cues from the scene, constancy decreased. Both ResNets and classical ConvNets of varying degrees of complexity performed well. However, DeepCC, our simplest sequential convolutional network, represented colors along the three color dimensions of human color vision, while ResNets showed a more complex representation.
Collapse
Affiliation(s)
- Alban Flachot
- Abteilung Allgemeine Psychologie, Justus Liebig University, Giessen, Germany.,
| | - Arash Akbarinia
- Abteilung Allgemeine Psychologie, Justus Liebig University, Giessen, Germany.,
| | - Heiko H Schütt
- Center for Neural Science, New York University, New York, NY, USA.,
| | - Roland W Fleming
- Experimental Psychology, Justus Liebig University, Giessen, Germany.,
| | - Felix A Wichmann
- Neural Information Processing Group, University of Tübingen, Germany.,
| | - Karl R Gegenfurtner
- Abteilung Allgemeine Psychologie, Justus Liebig University, Giessen, Germany.,
| |
Collapse
|
7
|
Evaluation of fiber degree for fish muscle based on the edge feature attention net. FOOD BIOSCI 2022. [DOI: 10.1016/j.fbio.2022.101658] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
|
8
|
de Vries JP, Akbarinia A, Flachot A, Gegenfurtner KR. Emergent color categorization in a neural network trained for object recognition. eLife 2022; 11:76472. [PMID: 36511778 PMCID: PMC9797187 DOI: 10.7554/elife.76472] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2021] [Accepted: 12/11/2022] [Indexed: 12/14/2022] Open
Abstract
Color is a prime example of categorical perception, yet it is unclear why and how color categories emerge. On the one hand, prelinguistic infants and several animals treat color categorically. On the other hand, recent modeling endeavors have successfully utilized communicative concepts as the driving force for color categories. Rather than modeling categories directly, we investigate the potential emergence of color categories as a result of acquiring visual skills. Specifically, we asked whether color is represented categorically in a convolutional neural network (CNN) trained to recognize objects in natural images. We systematically trained new output layers to the CNN for a color classification task and, probing novel colors, found borders that are largely invariant to the training colors. The border locations were confirmed using an evolutionary algorithm that relies on the principle of categorical perception. A psychophysical experiment on human observers, analogous to our primary CNN experiment, shows that the borders agree to a large degree with human category boundaries. These results provide evidence that the development of basic visual skills can contribute to the emergence of a categorical representation of color.
Collapse
Affiliation(s)
| | | | - Alban Flachot
- Experimental Psychology, Giessen UniversityGiessenGermany,Center for Vision Research, Department of Psychology, York UniversityTorontoCanada
| | | |
Collapse
|
9
|
da Silva Cotrim W, Felix LB, Minim VPR, Campos RC, Minim LA. Development of a hybrid system based on convolutional neural networks and support vector machines for recognition and tracking color changes in food during thermal processing. Chem Eng Sci 2021. [DOI: 10.1016/j.ces.2021.116679] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
10
|
Abstract
Visual images can be described in terms of the illuminants and objects that are causal to the light reaching the eye, the retinal image, its neural representation, or how the image is perceived. Respecting the differences among these distinct levels of description can be challenging but is crucial for a clear understanding of color vision. This article approaches color by reviewing what is known about its neural representation in the early visual cortex, with a brief description of signals in the eye and the thalamus for context. The review focuses on the properties of single neurons and advances the general theme that experimental approaches based on knowledge of feedforward signals have promoted greater understanding of the neural code for color than approaches based on correlating single-unit responses with color perception. New data from area V1 illustrate the strength of the feedforward approach. Future directions for progress in color neurophysiology are discussed: techniques for improved single-neuron characterization, for investigations of neural populations and small circuits, and for the analysis of natural image statistics.
Collapse
Affiliation(s)
- Gregory D Horwitz
- Department of Physiology and Biophysics, University of Washington, Seattle, Washington 98195, USA; .,Washington National Primate Research Center, University of Washington, Seattle, Washington 98121, USA
| |
Collapse
|
11
|
Taylor J, Xu Y. Joint representation of color and form in convolutional neural networks: A stimulus-rich network perspective. PLoS One 2021; 16:e0253442. [PMID: 34191815 PMCID: PMC8244861 DOI: 10.1371/journal.pone.0253442] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2020] [Accepted: 06/05/2021] [Indexed: 11/18/2022] Open
Abstract
To interact with real-world objects, any effective visual system must jointly code the unique features defining each object. Despite decades of neuroscience research, we still lack a firm grasp on how the primate brain binds visual features. Here we apply a novel network-based stimulus-rich representational similarity approach to study color and form binding in five convolutional neural networks (CNNs) with varying architecture, depth, and presence/absence of recurrent processing. All CNNs showed near-orthogonal color and form processing in early layers, but increasingly interactive feature coding in higher layers, with this effect being much stronger for networks trained for object classification than untrained networks. These results characterize for the first time how multiple basic visual features are coded together in CNNs. The approach developed here can be easily implemented to characterize whether a similar coding scheme may serve as a viable solution to the binding problem in the primate brain.
Collapse
Affiliation(s)
- JohnMark Taylor
- Department of Psychology, Vision Sciences Laboratory, Harvard University, Cambridge, MA, United States of America
| | - Yaoda Xu
- Department of Psychology, Yale University, New Haven, CT, United States of America
| |
Collapse
|
12
|
Harris E, Mihai D, Hare J. How Convolutional Neural Network Architecture Biases Learned Opponency and Color Tuning. Neural Comput 2021; 33:858-898. [PMID: 33400902 DOI: 10.1162/neco_a_01356] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2020] [Accepted: 10/06/2020] [Indexed: 11/04/2022]
Abstract
Recent work suggests that changing convolutional neural network (CNN) architecture by introducing a bottleneck in the second layer can yield changes in learned function. To understand this relationship fully requires a way of quantitatively comparing trained networks. The fields of electrophysiology and psychophysics have developed a wealth of methods for characterizing visual systems that permit such comparisons. Inspired by these methods, we propose an approach to obtaining spatial and color tuning curves for convolutional neurons that can be used to classify cells in terms of their spatial and color opponency. We perform these classifications for a range of CNNs with different depths and bottleneck widths. Our key finding is that networks with a bottleneck show a strong functional organization: almost all cells in the bottleneck layer become both spatially and color opponent, and cells in the layer following the bottleneck become nonopponent. The color tuning data can further be used to form a rich understanding of how color a network encodes color. As a concrete demonstration, we show that shallower networks without a bottleneck learn a complex nonlinear color system, whereas deeper networks with tight bottlenecks learn a simple channel opponent code in the bottleneck layer. We develop a method of obtaining a hue sensitivity curve for a trained CNN that enables high-level insights that complement the low-level findings from the color tuning data. We go on to train a series of networks under different conditions to ascertain the robustness of the discussed results. Ultimately our methods and findings coalesce with prior art, strengthening our ability to interpret trained CNNs and furthering our understanding of the connection between architecture and learned representation. Trained models and code for all experiments are available at https://github.com/ecs-vlc/opponency.
Collapse
Affiliation(s)
- Ethan Harris
- Vision Learning and Control, Electronics and Computer Science, University of Southampton, Southampton SO17 1B J, U.K.,
| | - Daniela Mihai
- Vision Learning and Control, Electronics and Computer Science, University of Southampton, Southampton SO17 1B J, U.K.,
| | - Jonathon Hare
- Vision Learning and Control, Electronics and Computer Science, University of Southampton, Southampton SO17 1B J, U.K.,
| |
Collapse
|
13
|
Color for object recognition: Hue and chroma sensitivity in the deep features of convolutional neural networks. Vision Res 2021; 182:89-100. [PMID: 33611127 DOI: 10.1016/j.visres.2020.09.010] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2020] [Revised: 09/02/2020] [Accepted: 09/18/2020] [Indexed: 11/22/2022]
Abstract
In this work, we examined the color tuning of units in the hidden layers of AlexNet, VGG-16 and VGG-19 convolutional neural networks and their relevance for the successful recognition of an object. We first selected the patches for which the units are maximally responsive among the 1.2 M images of the ImageNet training dataset. We segmented these patches using a k-means clustering algorithm on their chromatic distribution. Then we independently varied the color of these segments, both in hue and chroma, to measure the unit's chromatic tuning. The models exhibited properties at times similar or opposed to the known chromatic processing of biological system. We found that, similarly to the most anterior occipital visual areas in primates, the last convolutional layer exhibited high color sensitivity. We also found the gradual emergence of single to double opponent kernels. Contrary to cells in the visual system, however, these kernels were selective for hues that gradually transit from being broadly distributed in early layers, to mainly falling along the blue-orange axis in late layers. In addition, we found that the classification performance of our models varies as we change the color of our stimuli following the models' kernels properties. Performance was highest for colors the kernels maximally responded to, and images responsible for the activation of color sensitive kernels were more likely to be mis-classified as we changed their color. These observations were shared by all three networks, thus suggesting that they are general properties of current convolutional neural networks trained for object recognition.
Collapse
|
14
|
Fennell JG, Talas L, Baddeley RJ, Cuthill IC, Scott-Samuel NE. The Camouflage Machine: Optimizing protective coloration using deep learning with genetic algorithms. Evolution 2021; 75:614-624. [PMID: 33415740 DOI: 10.1111/evo.14162] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2020] [Revised: 12/23/2020] [Accepted: 12/23/2020] [Indexed: 12/20/2022]
Abstract
Evolutionary biologists frequently wish to measure the fitness of alternative phenotypes using behavioral experiments. However, many phenotypes are complex. One example is coloration: camouflage aims to make detection harder, while conspicuous signals (e.g., for warning or mate attraction) require the opposite. Identifying the hardest and easiest to find patterns is essential for understanding the evolutionary forces that shape protective coloration, but the parameter space of potential patterns (colored visual textures) is vast, limiting previous empirical studies to a narrow range of phenotypes. Here, we demonstrate how deep learning combined with genetic algorithms can be used to augment behavioral experiments, identifying both the best camouflage and the most conspicuous signal(s) from an arbitrarily vast array of patterns. To show the generality of our approach, we do so for both trichromatic (e.g., human) and dichromatic (e.g., typical mammalian) visual systems, in two different habitats. The patterns identified were validated using human participants; those identified as the best for camouflage were significantly harder to find than a tried-and-tested military design, while those identified as most conspicuous were significantly easier to find than other patterns. More generally, our method, dubbed the "Camouflage Machine," will be a useful tool for identifying the optimal phenotype in high dimensional state spaces.
Collapse
Affiliation(s)
- John G Fennell
- School of Psychological Science, University of Bristol, Bristol, UK
| | - Laszlo Talas
- School of Psychological Science, University of Bristol, Bristol, UK
| | | | - Innes C Cuthill
- School of Biological Sciences, University of Bristol, Bristol, UK
| | | |
Collapse
|
15
|
Rafegas I, Vanrell M, Alexandre LA, Arias G. Understanding trained CNNs by indexing neuron selectivity. Pattern Recognit Lett 2020. [DOI: 10.1016/j.patrec.2019.10.013] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
16
|
Cotrim WDS, Minim VPR, Felix LB, Minim LA. Short convolutional neural networks applied to the recognition of the browning stages of bread crust. J FOOD ENG 2020. [DOI: 10.1016/j.jfoodeng.2020.109916] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
17
|
Akbarinia A, Gil-Rodríguez R. Deciphering image contrast in object classification deep networks. Vision Res 2020; 173:61-76. [PMID: 32480109 DOI: 10.1016/j.visres.2020.04.015] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2019] [Revised: 04/24/2020] [Accepted: 04/28/2020] [Indexed: 11/16/2022]
Abstract
The ultimate goal of neuroscience is to explain how complex behaviour arises from neuronal activity. A comparable level of complexity also emerges in deep neural networks (DNNs) while exhibiting human-level performance in demanding visual tasks. Unlike in biological systems, all parameters and operations of DNNs are accessible. Therefore, in theory, it should be possible to decipher the exact mechanisms learnt by these artificial networks. Here, we investigate the concept of contrast invariance within the framework of DNNs. We start by discussing how a network can achieve robustness to changes in local and global image contrast. We used a technique from neuroscience-"kernel lesion"-to measure the degree of performance degradation when individual kernels are eliminated from a network. We further compared contrast normalisation, a mechanism used in biological systems, to the strategies that DNNs learn to cope with changes of contrast. The results of our analysis suggest that (i) contrast is a low-level feature for these networks, and it is encoded in the shallow layers; (ii) a handful of kernels appear to have a greater impact on this feature, and their removal causes a substantially larger accuracy loss for low-contrast images; (iii) edges are a distinct visual feature within the internal representation of object classification DNNs.
Collapse
Affiliation(s)
- Arash Akbarinia
- Department of General Psychology, Justus-Liebig University, D-35394 Giessen, Germany.
| | - Raquel Gil-Rodríguez
- Department of General Psychology, Justus-Liebig University, D-35394 Giessen, Germany
| |
Collapse
|
18
|
Fennell JG, Talas L, Baddeley RJ, Cuthill IC, Scott-Samuel NE. Optimizing colour for camouflage and visibility using deep learning: the effects of the environment and the observer's visual system. J R Soc Interface 2019; 16:20190183. [PMID: 31138092 PMCID: PMC6544896 DOI: 10.1098/rsif.2019.0183] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2019] [Accepted: 05/01/2019] [Indexed: 11/22/2022] Open
Abstract
Avoiding detection can provide significant survival advantages for prey, predators, or the military; conversely, maximizing visibility would be useful for signalling. One simple determinant of detectability is an animal's colour relative to its environment. But identifying the optimal colour to minimize (or maximize) detectability in a given natural environment is complex, partly because of the nature of the perceptual space. Here for the first time, using image processing techniques to embed targets into realistic environments together with psychophysics to estimate detectability and deep neural networks to interpolate between sampled colours, we propose a method to identify the optimal colour that either minimizes or maximizes visibility. We apply our approach in two natural environments (temperate forest and semi-arid desert) and show how a comparatively small number of samples can be used to predict robustly the most and least effective colours for camouflage. To illustrate how our approach can be generalized to other non-human visual systems, we also identify the optimum colours for concealment and visibility when viewed by simulated red-green colour-blind dichromats, typical for non-human mammals. Contrasting the results from these visual systems sheds light on why some predators seem, at least to humans, to have colouring that would appear detrimental to ambush hunting. We found that for simulated dichromatic observers, colour strongly affected detection time for both environments. In contrast, trichromatic observers were more effective at breaking camouflage.
Collapse
Affiliation(s)
- J. G. Fennell
- School of Psychological Science, University of Bristol, 12a Priory Road, Bristol BS8 1TU, UK
| | - L. Talas
- School of Psychological Science, University of Bristol, 12a Priory Road, Bristol BS8 1TU, UK
| | - R. J. Baddeley
- School of Psychological Science, University of Bristol, 12a Priory Road, Bristol BS8 1TU, UK
| | - I. C. Cuthill
- School of Biological Sciences, University of Bristol, Bristol Life Sciences Building, 24 Tyndall Avenue, Bristol BS8 1TQ, UK
| | - N. E. Scott-Samuel
- School of Psychological Science, University of Bristol, 12a Priory Road, Bristol BS8 1TU, UK
| |
Collapse
|
19
|
Conway BR, Eskew RT, Martin PR, Stockman A. A tour of contemporary color vision research. Vision Res 2018; 151:2-6. [PMID: 29959956 PMCID: PMC6345392 DOI: 10.1016/j.visres.2018.06.009] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2018] [Revised: 06/12/2018] [Accepted: 06/23/2018] [Indexed: 11/17/2022]
Abstract
The study of color vision encompasses many disciplines, including art, biochemistry, biophysics, brain imaging, cognitive neuroscience, color preferences, colorimetry, computer modelling, design, electrophysiology, language and cognition, molecular genetics, neuroscience, physiological optics, psychophysics and physiological optics. Coupled with the elusive nature of the subjective experience of color, this wide range of disciplines makes the study of color as challenging as it is fascinating. This overview of the special issue Color: Cone Opponency and Beyond outlines the state of the science of color, and points to some of the many questions that remain to be answered in this exciting field.
Collapse
Affiliation(s)
- Bevil R Conway
- National Eye Institute and National Institute of Mental Health, National Institutes of Health, Bethesda, MD 20892, USA
| | - Rhea T Eskew
- Department of Psychology, 125 Nightingale Hall, Northeastern University, Boston, MA 02115, USA
| | - Paul R Martin
- Save Sight Institute and School of Medical Sciences, The University of Sydney, Sydney, New South Wales, Australia
| | - Andrew Stockman
- UCL Institute of Ophthalmology, University College London, 11-43 Bath Street, London EC1V 9EL, England, United Kingdom
| |
Collapse
|