1
|
Ponting S, Morimoto T, Smithson HE. Modeling surface color discrimination under different lighting environments using image chromatic statistics and convolutional neural networks. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2023; 40:A149-A159. [PMID: 36846077 PMCID: PMC7614229 DOI: 10.1364/josaa.479986] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Revised: 01/12/2023] [Accepted: 01/12/2023] [Indexed: 08/10/2023]
Abstract
We modeled discrimination thresholds for object colors under different lighting environments [J. Opt. Soc. Am. 35, B244 (2018)]. First, we built models based on chromatic statistics, testing 60 models in total. Second, we trained convolutional neural networks (CNNs), using 160,280 images labeled by either the ground-truth or human responses. No single chromatic statistics model was sufficient to describe human discrimination thresholds across conditions, while human-response-trained CNNs nearly perfectly predicted human thresholds. Guided by region-of-interest analysis of the network, we modified the chromatic statistics models to use only the lower regions of the objects, which substantially improved performance.
Collapse
Affiliation(s)
- Samuel Ponting
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- These authors contributed equally to this paper
| | - Takuma Morimoto
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Department of Psychology, Justus-Liebig-Universitat-Giessen, Giessen, Germany
- These authors contributed equally to this paper
| | | |
Collapse
|
2
|
Ponting S, Morimoto T, Smithson HE. Modeling surface color discrimination under different lighting environments using image chromatic statistics and convolutional neural networks. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2023; 40:A149-A159. [PMID: 36846077 PMCID: PMC7614229 DOI: 10.1364/josaa.4799861] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Figures] [Subscribe] [Scholar Register] [Indexed: 05/05/2023]
Abstract
We modeled discrimination thresholds for object colors under different lighting environments [J. Opt. Soc. Am. 35, B244 (2018)]. First, we built models based on chromatic statistics, testing 60 models in total. Second, we trained convolutional neural networks (CNNs), using 160,280 images labeled by either the ground-truth or human responses. No single chromatic statistics model was sufficient to describe human discrimination thresholds across conditions, while human-response-trained CNNs nearly perfectly predicted human thresholds. Guided by region-of-interest analysis of the network, we modified the chromatic statistics models to use only the lower regions of the objects, which substantially improved performance.
Collapse
Affiliation(s)
- Samuel Ponting
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Takuma Morimoto
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Department of Psychology, Justus-Liebig-Universitat-Giessen, Giessen, Germany
- Corresponding author:
| | | |
Collapse
|
3
|
Singh V, Burge J, Brainard DH. Equivalent noise characterization of human lightness constancy. J Vis 2022; 22:2. [PMID: 35394508 PMCID: PMC8994201 DOI: 10.1167/jov.22.5.2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2021] [Accepted: 02/19/2022] [Indexed: 12/03/2022] Open
Abstract
A goal of visual perception is to provide stable representations of task-relevant scene properties (e.g. object reflectance) despite variation in task-irrelevant scene properties (e.g. illumination and reflectance of other nearby objects). To study such stability in the context of the perceptual representation of lightness, we introduce a threshold-based psychophysical paradigm. We measure how thresholds for discriminating the achromatic reflectance of a target object (task-relevant property) in rendered naturalistic scenes are impacted by variation in the reflectance functions of background objects (task-irrelevant property), using a two-alternative forced-choice paradigm in which the reflectance of the background objects is randomized across the two intervals of each trial. We control the amount of background reflectance variation by manipulating a statistical model of naturally occurring surface reflectances. For low background object reflectance variation, discrimination thresholds were nearly constant, indicating that observers' internal noise determines threshold in this regime. As background object reflectance variation increases, its effects start to dominate performance. A model based on signal detection theory allows us to express the effects of task-irrelevant variation in terms of the equivalent noise, that is relative to the intrinsic precision of the task-relevant perceptual representation. The results indicate that although naturally occurring background object reflectance variation does intrude on the perceptual representation of target object lightness, the effect is modest - within a factor of two of the equivalent noise level set by internal noise.
Collapse
Affiliation(s)
- Vijay Singh
- Department of Physics, North Carolina Agricultural and Technical State University, Greensboro, NC, USA
- Computational Neuroscience Initiative, University of Pennsylvania, Philadelphia, PA, USA
| | - Johannes Burge
- Computational Neuroscience Initiative, University of Pennsylvania, Philadelphia, PA, USA
- Department of Psychology, University of Pennsylvania, Philadelphia, PA, USA
- Neuroscience Graduate Group, University of Pennsylvania, Philadelphia, PA, USA
- Bioengineering Graduate Group, University of Pennsylvania, Philadelphia, PA, USA
| | - David H Brainard
- Computational Neuroscience Initiative, University of Pennsylvania, Philadelphia, PA, USA
- Department of Psychology, University of Pennsylvania, Philadelphia, PA, USA
- Neuroscience Graduate Group, University of Pennsylvania, Philadelphia, PA, USA
- Bioengineering Graduate Group, University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
4
|
Morimoto T, Kusuyama T, Fukuda K, Uchikawa K. Human color constancy based on the geometry of color distributions. J Vis 2021; 21:7. [PMID: 33661281 PMCID: PMC7937993 DOI: 10.1167/jov.21.3.7] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2020] [Accepted: 01/14/2021] [Indexed: 11/26/2022] Open
Abstract
The physical inputs to our visual system are dictated by the interplay between lights and surfaces; thus, for surface color to be stably perceived, the influence of the illuminant must be discounted. To reveal our strategy to infer the illuminant color, we conducted three psychophysical experiments designed to test our optimal color hypothesis that we internalize the physical color gamut under various illuminants and apply the prior to estimate the illuminant color. In each experiment, we presented 61 hexagons arranged without spatial gaps, where the surrounding 60 hexagons were set to have a specific shape in their color distribution. We asked participants to adjust the color of a center test field so that it appeared to be a full-white surface placed under a test illuminant. Results and computational modeling suggested that, although our proposed model is limited in accounting for estimation of illuminant intensity by human observers, it agrees fairly well with the estimates of illuminant chromaticity in most tested conditions. The accuracy of estimation generally outperformed other tested conventional color constancy models. These results support the hypothesis that our visual system can utilize the geometry of scene color distribution to achieve color constancy.
Collapse
Affiliation(s)
- Takuma Morimoto
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Takahiro Kusuyama
- Department of Information Processing, Tokyo Institute of Technology, Yokohama, Japan
| | - Kazuho Fukuda
- Department of Information Design, Kogakuin University, Tokyo, Japan
| | - Keiji Uchikawa
- Human Media Research Center, Kanagawa Institute of Technology, Atsugi, Japan
| |
Collapse
|
5
|
Morimoto T, Kishigami S, Linhares JMM, Nascimento SMC, Smithson HE. Hyperspectral environmental illumination maps: characterizing directional spectral variation in natural environments. OPTICS EXPRESS 2019; 27:32277-32293. [PMID: 31684444 PMCID: PMC7028397 DOI: 10.1364/oe.27.032277] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
Objects placed in real-world scenes receive incident light from every direction, and the spectral content of this light may vary from one direction to another. In computer graphics, environmental illumination is approximated using maps that specify illumination at a point as a function of incident angle. However, to-date, existing public databases of environmental illumination specify only three colour channels (RGB). We have captured a new set of 12 environmental illumination maps (eight outdoor scenes; four indoor scenes) using a hyperspectral imaging system with 33 spectral channels. The data reveal a striking directional variation of spectral distribution of lighting in natural environments. We discuss limitations of using daylight models to describe natural environmental illumination.
Collapse
Affiliation(s)
- Takuma Morimoto
- University of Oxford, Department of Experimental Psychology, Oxford, UK
| | - Sho Kishigami
- Department of Computer Science and Engineering, Toyohashi University of Technology, Toyohashi, Japan
| | | | | | | |
Collapse
|
6
|
Foster DH, Amano K. Hyperspectral imaging in color vision research: tutorial. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2019; 36:606-627. [PMID: 31044981 DOI: 10.1364/josaa.36.000606] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/01/2018] [Accepted: 01/11/2019] [Indexed: 06/09/2023]
Abstract
This tutorial offers an introduction to terrestrial and close-range hyperspectral imaging and some of its uses in human color vision research. The main types of hyperspectral cameras are described together with procedures for image acquisition, postprocessing, and calibration for either radiance or reflectance data. Image transformations are defined for colorimetric representations, color rendering, and cone receptor and postreceptor coding. Several example applications are also presented. These include calculating the color properties of scenes, such as gamut volume and metamerism, and analyzing the utility of color in observer tasks, such as identifying surfaces under illuminant changes. The effects of noise and uncertainty are considered in both image acquisition and color vision applications.
Collapse
|