A multi-stage color model

A circuit motif for color in the human foveal retina

The neural pathways that start human color vision begin in the complex synaptic network of the foveal retina where signals originating in long (L), middle (M), and short (S) wavelength-sensitive cone photoreceptor types are compared through antagonistic interactions, referred to as opponency. In nonhuman primates, two cone opponent pathways are well established: an L vs. M cone circuit linked to the midget ganglion cell type, often called the red-green pathway, and an S vs. L + M cone circuit linked to the small bistratified ganglion cell type, often called the blue-yellow pathway. These pathways have been taken to correspond in human vision to cardinal directions in a trichromatic color space, providing the parallel inputs to higher-level color processing. Yet linking cone opponency in the nonhuman primate retina to color mechanisms in human vision has proven particularly difficult. Here, we apply connectomic reconstruction to the human foveal retina to trace parallel excitatory synaptic outputs from the S-ON (or "blue-cone") bipolar cell to the small bistratified cell and two additional ganglion cell types: a large bistratified ganglion cell and a subpopulation of ON-midget ganglion cells, whose synaptic connections suggest a significant and unique role in color vision. These two ganglion cell types are postsynaptic to both S-ON and L vs. M opponent midget bipolar cells and thus define excitatory pathways in the foveal retina that merge the cardinal red-green and blue-yellow circuits, with the potential for trichromatic cone opponency at the first stage of human vision.

The mechanism of human color vision and potential implanted devices for artificial color vision

Vision plays a major role in perceiving external stimuli and information in our daily lives. The neural mechanism of color vision is complicated, involving the co-ordinated functions of a variety of cells, such as retinal cells and lateral geniculate nucleus cells, as well as multiple levels of the visual cortex. In this work, we reviewed the history of experimental and theoretical studies on this issue, from the fundamental functions of the individual cells of the visual system to the coding in the transmission of neural signals and sophisticated brain processes at different levels. We discuss various hypotheses, models, and theories related to the color vision mechanism and present some suggestions for developing novel implanted devices that may help restore color vision in visually impaired people or introduce artificial color vision to those who need it.

Unique yellow shifts for small and brief stimuli in the central retina

The spectral locus of unique yellow was determined for flashes of different sizes (<11 arcmin) and durations (<500 ms) presented in and near the fovea. An adaptive optics scanning laser ophthalmoscope was used to minimize the effects of higher-order aberrations during simultaneous stimulus delivery and retinal imaging. In certain subjects, parafoveal cones were classified as L, M, or S, which permitted the comparison of unique yellow measurements with variations in local L/M ratios within and between observers. Unique yellow shifted to longer wavelengths as stimulus size or duration was reduced. This effect is most pronounced for changes in size and more apparent in the fovea than in the parafovea. The observed variations in unique yellow are not entirely predicted from variations in L/M ratio and therefore implicate neural processes beyond photoreception.

Cone-Opponent Ganglion Cells in the Primate Fovea Tuned to Non-Cardinal Color Directions

A long-standing question in vision science is how the three cone photoreceptor types - long (L), medium (M) and short (S) wavelength sensitive - combine to generate our perception of color. Hue perception can be described along two opponent axes: red-green and blue-yellow. Psychophysical measurements of color appearance indicate that the cone inputs to the red-green and blue-yellow opponent axes are M vs. L+S and L vs. M+S, respectively. However, the "cardinal directions of color space" revealed by psychophysical measurements of color detection thresholds are L vs. M and S vs. L+M. The cardinal directions match the most common cone-opponent retinal ganglion cells (RGCs) in the primate retina. Accordingly, the cone opponency necessary for color appearance is thought to be established in cortex. However, small populations with the appropriate M vs. L+S and L vs. M+S cone-opponency have been reported in large surveys of cone inputs to primate RGCs and their projections to the lateral geniculate nucleus (LGN) yet their existence continues to be debated. Resolving this long-standing open question is needed as a complete account of the cone-opponency in the retinal output is critical for efforts to understand how downstream neural circuits process color. Here, we performed adaptive optics calcium imaging to longitudinally and noninvasively measurements of the foveal RGC light responses in the living macaque eye. We confirm the presence of L vs. M+S and M vs. L+S neurons with non-cardinal cone-opponency and demonstrate that cone-opponent signals in the retinal output are substantially more diverse than classically thought.

Seeing color following gene augmentation therapy in achromatopsia

How will people who spent their visual lives with only rods respond to cone function restoration? Will they be able suddenly see the colors of the rainbow? CNGA3-achromatopsia is a congenital hereditary disease in which cone dysfunction leads patients to have rod photoreceptor-driven vision only in daylight,1,2,3,4 seeing the world in blurry shades of gray.5,6 We studied color perception in four CNGA3-achromatopsia patients following monocular retinal gene augmentation therapy.7,8,9 Following treatment, although some cortical changes were reported,3,4 patients did not report a dramatic change in their vision.3,9 However, in accordance with the fact that sensitivity of rods and cones is most different at long wavelengths, they consistently reported seeing red objects on dark backgrounds differently than they did before surgery.3 Because clinical color assessments failed to find any indication of color vision, we conducted a gamut of tailored tests to better define patients' descriptions. We evaluated patients' perceived lightness of different colors, color detection, and saliency, comparing their treated with their untreated eyes. Although the perceived lightness of different colors was generally similar between the eyes and matched a rod-input model, patients could detect a colored stimulus only in their treated eyes. In a search task, long response times, which were further extended with array size, suggested low saliency. We suggest that treated CNGA3-achromatopsia patients can perceive a stimulus's color attribute, although in a manner that is different and very limited compared with sighted individuals. We discuss the retinal and cortical obstacles that might explain this perceptual gap.

Pattern forming mechanisms of color vision

While our understanding of the way single neurons process chromatic stimuli in the early visual pathway has advanced significantly in recent years, we do not yet know how these cells interact to form stable representations of hue. Drawing on physiological studies, we offer a dynamical model of how the primary visual cortex tunes for color, hinged on intracortical interactions and emergent network effects. After detailing the evolution of network activity through analytical and numerical approaches, we discuss the effects of the model's cortical parameters on the selectivity of the tuning curves. In particular, we explore the role of the model's thresholding nonlinearity in enhancing hue selectivity by expanding the region of stability, allowing for the precise encoding of chromatic stimuli in early vision. Finally, in the absence of a stimulus, the model is capable of explaining hallucinatory color perception via a Turing-like mechanism of biological pattern formation.

Comparative connectomics reveals noncanonical wiring for color vision in human foveal retina

The Old World macaque monkey and New World common marmoset provide fundamental models for human visual processing, yet the human ancestral lineage diverged from these monkey lineages over 25 Mya. We therefore asked whether fine-scale synaptic wiring in the nervous system is preserved across these three primate families, despite long periods of independent evolution. We applied connectomic electron microscopy to the specialized foveal retina where circuits for highest acuity and color vision reside. Synaptic motifs arising from the cone photoreceptor type sensitive to short (S) wavelengths and associated with "blue-yellow" (S-ON and S-OFF) color-coding circuitry were reconstructed. We found that distinctive circuitry arises from S cones for each of the three species. The S cones contacted neighboring L and M (long- and middle-wavelength sensitive) cones in humans, but such contacts were rare or absent in macaques and marmosets. We discovered a major S-OFF pathway in the human retina and established its absence in marmosets. Further, the S-ON and S-OFF chromatic pathways make excitatory-type synaptic contacts with L and M cone types in humans, but not in macaques or marmosets. Our results predict that early-stage chromatic signals are distinct in the human retina and imply that solving the human connectome at the nanoscale level of synaptic wiring will be critical for fully understanding the neural basis of human color vision.

Protocol for isolation of melanopsin and rhodopsin in the human eye using silent substitution

Melanopsin-mediated visual and non-visual functions are difficult to study in vivo. To isolate melanopsin responses, non-standard light stimulation instruments are required, with at least as many primaries as photoreceptor classes in the eye. In this protocol, we describe the physical light calibrations of the display instrumentation, control of stimulus artefacts, and correction of individual between-eye differences in human observers. The protocol achieves complete photoreceptor silent substitution in psychophysical, pupillometry, and electroretinographic experiments for probing melanopsin, rod, and cone function. For complete details on the use and execution of this protocol, please refer to Uprety et al. (2022).1.

Decoding of EEG signals reveals non-uniformities in the neural geometry of colour

The idea of colour opponency maintains that colour vision arises through the comparison of two chromatic mechanisms, red versus green and yellow versus blue. The four unique hues, red, green, blue, and yellow, are assumed to appear at the null points of these the two chromatic systems. Here we hypothesise that, if unique hues represent a tractable cortical state, they should elicit more robust activity compared to other, non-unique hues. We use a spatiotemporal decoding approach to report that electroencephalographic (EEG) responses carry robust information about the tested isoluminant unique hues within a 100-350 ms window from stimulus onset. Decoding is possible in both passive and active viewing tasks, but is compromised when concurrent high luminance contrast is added to the colour signals. For large hue-differences, the efficiency of hue decoding can be predicted by mutual distance in a nominally uniform perceptual colour space. However, for small perceptual neighbourhoods around unique hues, the encoding space shows pivotal non-uniformities which suggest that anisotropies in neurometric hue-spaces may reflect perceptual unique hues.

From cones to color vision: a neurobiological model that explains the unique hues

The irreducible unique hues-red, green, blue, and yellow-remain one of the great mysteries of vision science. Attempts to create a physiologically parsimonious model that can predict the spectral locations of the unique hues all rely on at least one post hoc adjustment to produce appropriate loci for unique green and unique red, and struggle to explain the non-linearity of the Blue/Yellow system. We propose a neurobiological color vision model that overcomes these challenges by using physiological cone ratios, cone-opponent normalization to equal-energy white, and a simple adaptation mechanism to produce color-opponent mechanisms that accurately predict the spectral locations and variability of the unique hues.

Cone opponent functional domains in primary visual cortex combine signals for color appearance mechanisms

Studies of color perception have led to mechanistic models of how cone-opponent signals from retinal ganglion cells are integrated to generate color appearance. But it is unknown how this hypothesized integration occurs in the brain. Here we show that cone-opponent signals transmitted from retina to primary visual cortex (V1) are integrated through highly organized circuits within V1 to implement the color opponent interactions required for color appearance. Combining intrinsic signal optical imaging (ISI) and 2-photon calcium imaging (2PCI) at single cell resolution, we demonstrate cone-opponent functional domains (COFDs) that combine L/M cone-opponent and S/L + M cone-opponent signals following the rules predicted from psychophysical studies of color perception. These give rise to an orderly organization of hue preferences of the neurons within the COFDs and the generation of hue "pinwheels". Thus, spatially organized neural circuits mediate an orderly transition from cone-opponency to color appearance that begins in V1.

How We See Black and White: The Role of Midget Ganglion Cells

According to classical opponent color theory, hue sensations are mediated by spectrally opponent neurons that are excited by some wavelengths of light and inhibited by others, while black-and-white sensations are mediated by spectrally non-opponent neurons that respond with the same sign to all wavelengths. However, careful consideration of the morphology and physiology of spectrally opponent L vs. M midget retinal ganglion cells (RGCs) in the primate retina indicates that they are ideally suited to mediate black-and-white sensations and poorly suited to mediate color. Here we present a computational model that demonstrates how the cortex could use unsupervised learning to efficiently separate the signals from L vs. M midget RGCs into distinct signals for black and white based only correlation of activity over time. The model also reveals why it is unlikely that these same ganglion cells could simultaneously mediate our perception of red and green, and shows how, in theory, a separate small population of midget RGCs with input from S, M, and L cones would be ideally suited to mediating hue perception.

Evolution in Approach to Colour in Tall Buildings’ Architecture on the Isle of Dogs, London, UK

Architects’ approach towards colour in architectural design evolved radically in the recent 50 years, and ranges from a modernist aversion to a vernacular appreciation. These changes were linked to the development of culture, technology and scientific knowledge in different areas connected to human functioning. The authors have examined evolution in design of tall buildings in the Isle of Dogs in London (UK) since the 1980s. The area experienced major growth spurs in the turn of the 20th and 21st centuries, resulting in the greatest concentration of tall buildings in London today. The Island has been a playground for architects who have developed a range of approaches to the design of towers. The authors observed the evolution of architectural style, analyzed application of colour and made connections between scale, beauty and human behaviour. They concluded that colour in tall buildings’ architecture on the Isle of Dogs is predominantly used to disguise their massing. Colour detail facilitates the domestic feel of a public realm. Therefore, alongside decorative quality, and if considerately applied, colour may positively influence the quality of living and working environments.

Brain-computer interface-based assessment of color vision

Objective.Present methods for assessing color vision require the person's active participation. Here we describe a brain-computer interface-based method for assessing color vision that does not require the person's participation.Approach.This method uses steady-state visual evoked potentials to identify metamers-two light sources that have different spectral distributions but appear to the person to be the same color.Main results.We demonstrate that: minimization of the visual evoked potential elicited by two flickering light sources identifies the metamer; this approach can distinguish people with color-vision deficits from those with normal color vision; and this metamer-identification process can be automated.Significance.This new method has numerous potential clinical, scientific, and industrial applications.

Cultural Components of Sex Differences in Color Preference

Preferences for pink and blue were tested in children aged 4-11 years in three small-scale societies: Shipibo villages in the Peruvian Amazon, kastom villages in the highlands of Tanna Island, Vanuatu, and BaYaka foragers in the northern Republic of Congo; and compared to children from an Australian global city (total N = 232). No sex differences were found in preference for pink in any of the three societies not influenced by global culture (ds - 0.31-0.23), in contrast to a female preference for pink in the global city (d = 1.24). Results suggest that the pairing of female and pink is a cultural phenomenon and is not driven by an essential preference for pink in girls.

Image Retrieval Using the Fused Perceptual Color Histogram

Extracting visual features for image retrieval by mimicking human cognition remains a challenge. Opponent color and HSV color spaces can mimic human visual perception well. In this paper, we improve and extend the CDH method using a multi-stage model to extract and represent an image in a way that mimics human perception. Our main contributions are as follows: (1) a visual feature descriptor is proposed to represent an image. It has the advantages of a histogram-based method and is consistent with visual perception factors such as spatial layout, intensity, edge orientation, and the opponent colors. (2) We improve the distance formula of CDHs; it can effectively adjust the similarity between images according to two parameters. The proposed method provides efficient performance in similar image retrieval rather than instance retrieval. Experiments with four benchmark datasets demonstrate that the proposed method can describe color, texture, and spatial features and performs significantly better than the color volume histogram, color difference histogram, local binary pattern histogram, and multi-texton histogram, and some SURF-based approaches.

Plasticity in perception: insights from color vision deficiencies

Inherited color vision deficiencies typically result from a loss or alteration of the visual photopigments absorbing light and thus impact the very first step of seeing. There is growing interest in how subsequent steps in the visual pathway might be calibrated to compensate for the altered receptor signals, with the possibility that color coding and color percepts might be less severely impacted than the receptor differences predict. These compensatory adjustments provide important insights into general questions about sensory plasticity and the sensory and cognitive processes underlying how we experience color.

Amplitude modulation encoding in the auditory cortex: comparisons between the primary and middle lateral belt regions

In macaques, the middle lateral auditory cortex (ML) is a belt region adjacent to the primary auditory cortex (A1) and believed to be at a hierarchically higher level. Although ML single-unit responses have been studied for several auditory stimuli, the ability of ML cells to encode amplitude modulation (AM)-an ability that has been widely studied in A1-has not yet been characterized. Here, we compared the responses of A1 and ML neurons to amplitude-modulated (AM) noise in awake macaques. Although several of the basic properties of A1 and ML responses to AM noise were similar, we found several key differences. ML neurons were less likely to phase lock, did not phase lock as strongly, and were more likely to respond in a nonsynchronized fashion than A1 cells, consistent with a temporal-to-rate transformation as information ascends the auditory hierarchy. ML neurons tended to have lower temporally (phase-locking) based best modulation frequencies than A1 neurons. Neurons that decreased their firing rate in response to AM noise relative to their firing rate in response to unmodulated noise became more common at the level of ML than they were in A1. In both A1 and ML, we found a prevalent class of neurons that usually have enhanced rate responses relative to responses to the unmodulated noise at lower modulation frequencies and suppressed rate responses relative to responses to the unmodulated noise at middle modulation frequencies.NEW & NOTEWORTHY ML neurons synchronized less than A1 neurons, consistent with a hierarchical temporal-to-rate transformation. Both A1 and ML had a class of modulation transfer functions previously unreported in the cortex with a low-modulation-frequency (MF) peak, a middle-MF trough, and responses similar to unmodulated noise responses at high MFs. The results support a hierarchical shift toward a two-pool opponent code, where subtraction of neural activity between two populations of oppositely tuned neurons encodes AM.

Enhanced luminance sensitivity on color and luminance pedestals: Threshold measurements and a model of parvocellular luminance processing

Psychophysical interactions between chromatic and achromatic stimuli may inform our understanding of the cortical processing of signals of parvocellular origin, which carry both luminance and color information. We measured observers' sensitivity in discriminating the luminance of circular patch stimuli with a range of baseline ("pedestal") luminance and chromaticity. Pedestal stimuli were defined along vectors in cone-contrast space in a plane spanned by the red-green cone-opponent (L-M) and achromatic (L + M + S) axes. For a range of pedestal directions and intensities within this plane, we measured thresholds for discriminating increments from decrements along the achromatic axis. Low-contrast pedestals lowered luminance thresholds for every pedestal type. Thresholds began to increase with higher pedestal contrasts, forming a "dipper"-shaped function. Dipper functions varied systematically with pedestal chromaticity: Compared to the achromatic case, chromatic pedestals were effective at lower contrast. We suggest that the enhancement of luminance sensitivity caused by both achromatic and chromatic pedestals stems from a single mechanism, which is maximally sensitive to chromatic stimuli. We fit our data with a computational model of such a mechanism, in which luminance is computed from the rectified output of cone-opponent mechanisms similar to parvocellular neurons.

Multiplicative modulations enhance diversity of hue-selective cells

There is still much to understand about the brain's colour processing mechanisms and the transformation from cone-opponent representations to perceptual hues. Moreover, it is unclear which area(s) in the brain represent unique hues. We propose a hierarchical model inspired by the neuronal mechanisms in the brain for local hue representation, which reveals the contributions of each visual cortical area in hue representation. Hue encoding is achieved through incrementally increasing processing nonlinearities beginning with cone input. Besides employing nonlinear rectifications, we propose multiplicative modulations as a form of nonlinearity. Our simulation results indicate that multiplicative modulations have significant contributions in encoding of hues along intermediate directions in the MacLeod-Boynton diagram and that our model V2 neurons have the capacity to encode unique hues. Additionally, responses of our model neurons resemble those of biological colour cells, suggesting that our model provides a novel formulation of the brain's colour processing pathway.

Seeing Visual Gamma Oscillations in a New Light

Gamma oscillations have been argued to support visual perception by synchronizing the processing and transfer of information within and across areas of visual cortex. Here, we highlight recent findings implicating the influence of color on visual gamma oscillations and how these observations may relate to local cortical tuning and organization.

The Verriest Lecture: Adventures in blue and yellow

Conventional models of color vision assume that blue and yellow (along with red and green) are the fundamental building blocks of color appearance, yet how these hues are represented in the brain and whether and why they might be special are questions that remain shrouded in mystery. Many studies have explored the visual encoding of color categories, from the statistics of the environment to neural processing to perceptual experience. Blue and yellow are tied to salient features of the natural color world, and these features have likely shaped several important aspects of color vision. However, it remains less certain that these dimensions are encoded as primary or "unique" in the visual representation of color. There are also striking differences between blue and yellow percepts that may reflect high-level inferences about the world, specifically about the colors of light and surfaces. Moreover, while the stimuli labeled as blue or yellow or other basic categories show a remarkable degree of constancy within the observer, they all vary independently of one another across observers. This pattern of variation again suggests that blue and yellow and red and green are not a primary or unitary dimension of color appearance, and instead suggests a representation in which different hues reflect qualitatively different categories rather than quantitative differences within an underlying low-dimensional "color space."

Demonstration of a Human Color Vision Mimic in the Infrared

Color vision results from the interaction of retinal photopigments with reflected or transmitted visible light. The International Commission on Illumination (CIE) developed the CIE color-matching chart, which separates colors on the basis of the interaction of their spectral profiles with three retinal photopigments in the human eye. We report the development of an infrared chromaticity (CIE-IR) chart, which mimics the CIE chart, in order to discriminate between different chemicals on the basis of the interactions of their IR signatures with three different IR optical filters, instead of the retinal photopigments in the human eye. Our results demonstrate that the CIE-IR chart enables separation of different classes of chemicals, as the visible CIE chart does with color, except for those in the IR spectral region. Such results clearly show that the biomimetic sensing method based on human color vision is in fact a true analogue to color vision and that the proposed CIE-IR chart can be used as a classification method unique to this biomimetic sensing modality.

Reconciling Color Vision Models With Midget Ganglion Cell Receptive Fields

Midget retinal ganglion cells (RGCs) make up the majority of foveal RGCs in the primate retina. The receptive fields of midget RGCs exhibit both spectral and spatial opponency and are implicated in both color and achromatic form vision, yet the exact mechanisms linking their responses to visual perception remain unclear. Efforts to develop color vision models that accurately predict all the features of human color and form vision based on midget RGCs provide a case study connecting experimental and theoretical neuroscience, drawing on diverse research areas such as anatomy, physiology, psychophysics, and computer vision. Recent technological advances have allowed researchers to test some predictions of color vision models in new and precise ways, producing results that challenge traditional views. Here, we review the progress in developing models of color-coding receptive fields that are consistent with human psychophysics, the biology of the primate visual system and the response properties of midget RGCs.

An S-cone circuit for edge detection in the primate retina

Midget retinal ganglion cells (RGCs) are the most common RGC type in the primate retina. Their responses have been proposed to mediate both color and spatial vision, yet the specific links between midget RGC responses and visual perception are unclear. Previous research on the dual roles of midget RGCs has focused on those comparing long (L) vs. middle (M) wavelength sensitive cones. However, there is evidence for several other rare midget RGC subtypes receiving S-cone input, but their role in color and spatial vision is uncertain. Here, we confirm the existence of the single S-cone center OFF midget RGC circuit in the central retina of macaque monkey both structurally and functionally. We investigated the receptive field properties of the S-OFF midget circuit with single cell electrophysiology and 3D electron microscopy reconstructions of the upstream circuitry. Like the well-studied L vs. M midget RGCs, the S-OFF midget RGCs have a center-surround receptive field consistent with a role in spatial vision. While spectral opponency in a primate RGC is classically assumed to contribute to hue perception, a role supporting edge detection is more consistent with the S-OFF midget RGC receptive field structure and studies of hue perception.

Unique hue stimulus selection using Munsell color chips under different chroma levels and illumination conditions

To determine the role of chroma and illumination conditions on assessments of unique hue stimuli (UHs: Y, B, R, and G) 25 color normal observers selected Munsell color chips under controlled but different light sources that included simulated D65 daylight, incandescent (A), CWF and TL84, from two rotatable trays-one containing low and the other high chroma samples. Three independent evaluations were obtained from each observer with a gap of at least 24 h between assessments. A total of 2400 UH assessments were thus obtained. The mean, as well as the range and variability, in UH selections was calculated according to gender, illumination condition, and chroma. Results show statistically significant differences for uB and uR stimuli selections from the low to the high chroma sets under all lighting conditions, and for uY under light source A and CWF while differences in uG selections in different lighting conditions were statistically insignificant. No significant difference was found between male and female UH selections. Hue shift predictions based on CAT02 transform were found to agree well with observer responses under different illumination conditions. It was also found that observers' UH choices varied more for low chroma samples under different lighting conditions compared to high chroma samples. The highest variability was obtained under light source CWF, while the simulated D65 source resulted in the lowest variability in selections.

The spectral identity of foveal cones is preserved in hue perception

Organisms are faced with the challenge of making inferences about the physical world from incomplete incoming sensory information. One strategy to combat ambiguity in this process is to combine new information with prior experiences. We investigated the strategy of combining these information sources in color vision. Single cones in human subjects were stimulated and the associated percepts were recorded. Subjects rated each flash for brightness, hue, and saturation. Brightness ratings were proportional to stimulus intensity. Saturation was independent of intensity, but varied between cones. Hue, in contrast, was assigned in a stereotyped manner that was predicted by cone type. These experiments revealed that, near the fovea, long and middle wavelength sensitive cones produce sensations that can be reliably distinguished on the basis of hue, but not saturation or brightness. Taken together, these observations implicate the high-resolution, color-opponent parvocellular pathway in this low-level visual task.

Mechanisms contributing to increment threshold and decrement threshold spectral sensitivities

The shape of the human spectral sensitivity function depends on how it is measured. In the increment threshold (IT) technique, sensitivity is typically measured as the inverse of threshold for detection of increments of monochromatic light presented for relatively long durations on achromatic pedestals. Spectral sensitivity functions derived from IT techniques have long been used to reveal contribution from opponent color channels. Although IT functions have been studied extensively, little attention has been given to functions derived from decrement thresholds (DT), partly due to technical challenges of producing appropriate stimuli. Comparison of IT and DT spectral sensitivities may be of interest because there are known asymmetries in the visual system between on- and off-pathways and between increment and decrement responses within these pathways. Consequently, spectral sensitivity functions obtained using DT measures may reveal a different complement of contributing mechanisms than those that produce IT functions. We report here that IT and DT derived spectral sensitivities were essentially identical over much of the visible spectrum. However, decrement sensitivity was slightly greater than increment sensitivity in the shorter wavelengths at modest light levels. This difference was not present at higher light levels, implicating rod pathways as a possible source of the difference. In sum, it appears that under conditions shown to reveal strong contribution from opponent mechanisms, decrement functions are either 1) determined by a similar complement of spectrally opponent mechanisms as those that define increment spectral sensitivities or 2) that the present conditions are insensitive to underlying asymmetries.

Probing Computation in the Primate Visual System at Single-Cone Resolution

Daylight vision begins when light activates cone photoreceptors in the retina, creating spatial patterns of neural activity. These cone signals are then combined and processed in downstream neural circuits, ultimately producing visual perception. Recent technical advances have made it possible to deliver visual stimuli to the retina that probe this processing by the visual system at its elementary resolution of individual cones. Physiological recordings from nonhuman primate retinas reveal the spatial organization of cone signals in retinal ganglion cells, including how signals from cones of different types are combined to support both spatial and color vision. Psychophysical experiments with human subjects characterize the visual sensations evoked by stimulating a single cone, including the perception of color. Future combined physiological and psychophysical experiments focusing on probing the elementary visual inputs are likely to clarify how neural processing generates our perception of the visual world.

A Neuronal Network Model of the Primate Visual System: Color Mechanisms in the Retina, LGN and V1

Color plays a key role in human vision but the neural machinery that underlies the transformation from stimulus to perception is not well understood. Here, we implemented a two-dimensional network model of the first stages in the primate parvocellular pathway (retina, lateral geniculate nucleus and layer 4C[Formula: see text] in V1) consisting of conductance-based point neurons. Model parameters were tuned based on physiological and anatomical data from the primate foveal and parafoveal vision, the most relevant visual field areas for color vision. We exhaustively benchmarked the model against well-established chromatic and achromatic visual stimuli, showing spatial and temporal responses of the model to disk- and ring-shaped light flashes, spatially uniform squares and sine-wave gratings of varying spatial frequency. The spatiotemporal patterns of parvocellular cells and cortical cells are consistent with their classification into chromatically single-opponent and double-opponent groups, and nonopponent cells selective for luminance stimuli. The model was implemented in the widely used neural simulation tool NEST and released as open source software. The aim of our modeling is to provide a biologically realistic framework within which a broad range of neuronal interactions can be examined at several different levels, with a focus on understanding how color information is processed.

V1-based modeling of discrimination between natural scenes within the luminance and isoluminant color planes

We have been developing a computational visual difference predictor model that can predict how human observers rate the perceived magnitude of suprathreshold differences between pairs of full-color naturalistic scenes (To, Lovell, Troscianko, & Tolhurst, 2010). The model is based closely on V1 neurophysiology and has recently been updated to more realistically implement sequential application of nonlinear inhibitions (contrast normalization followed by surround suppression; To, Chirimuuta, & Tolhurst, 2017). The model is based originally on a reliable luminance model (Watson & Solomon, 1997) which we have extended to the red/green and blue/yellow opponent planes, assuming that the three planes (luminance, red/green, and blue/yellow) can be modeled similarly to each other with narrow-band oriented filters. This paper examines whether this may be a false assumption, by decomposing our original full-color stimulus images into monochromatic and isoluminant variants, which observers rate separately and which we model separately. The ratings for the original full-color scenes correlate better with the new ratings for the monochromatic variants than for the isoluminant ones, suggesting that luminance cues carry more weight in observers' ratings to full-color images. The ratings for the original full-color stimuli can be predicted from the new monochromatic and isoluminant rating data by combining them by Minkowski summation with power m = 2.71, consistent with other studies involving feature summation. The model performed well at predicting ratings for monochromatic stimuli, but was weaker for isoluminant stimuli, indicating that mirroring the monochromatic models is not sufficient to model the color planes. We discuss several alternative strategies to improve the color modeling.

A Blind Stereoscopic Image Quality Evaluator with Segmented Stacked Autoencoders Considering The Whole Visual Perception Route

Most of the current blind stereoscopic image quality assessment (SIQA) algorithms cannot show reliable accuracy. One reason is that they do not have the deep architectures and the other reason is that they are designed on the relatively weak biological basis, compared with findings on human visual system (HVS). In this paper, we propose a Deep Edge and COlor Signal INtegrity Evaluator (DECOSINE) based on the whole visual perception route from eyes to the frontal lobe, and especially focus on edge and color signal processing in retinal ganglion cells (RGC) and lateral geniculate nucleus (LGN). Furthermore, to model the complex and deep structure of the visual cortex, Segmented Stacked Auto-encoder (S-SAE) is used, which has not utilized for SIQA before. The utilization of the S-SAE complements weakness of deep learning-based SIQA metrics that require a very long training time. Experiments are conducted on popular SIQA databases, and the superiority of DECOSINE in terms of prediction accuracy and monotonicity is proved. The experimental results show that our model about the whole visual perception route and utilization of S-SAE are effective for SIQA.

A tour of contemporary color vision research

The study of color vision encompasses many disciplines, including art, biochemistry, biophysics, brain imaging, cognitive neuroscience, color preferences, colorimetry, computer modelling, design, electrophysiology, language and cognition, molecular genetics, neuroscience, physiological optics, psychophysics and physiological optics. Coupled with the elusive nature of the subjective experience of color, this wide range of disciplines makes the study of color as challenging as it is fascinating. This overview of the special issue Color: Cone Opponency and Beyond outlines the state of the science of color, and points to some of the many questions that remain to be answered in this exciting field.

Color Perception: Objects, Constancy, and Categories

Color has been scientifically investigated by linking color appearance to colorimetric measurements of the light that enters the eye. However, the main purpose of color perception is not to determine the properties of incident light, but to aid the visual perception of objects and materials in our environment. We review the state of the art on object colors, color constancy, and color categories to gain insight into the functional aspects of color perception. The common ground between these areas of research is that color appearance is tightly linked to the identification of objects and materials and the communication across observers. In conclusion, we argue that research should focus on how color processing is adapted to the surface properties of objects in the natural environment in order to bridge the gap between the known early stages of color perception and the subjective appearance of color.

Sensations from a single M-cone depend on the activity of surrounding S-cones

Color vision requires the activity of cone photoreceptors to be compared in post-receptoral circuitry. Decades of psychophysical measurements have quantified the nature of these comparative interactions on a coarse scale. How such findings generalize to a cellular scale remains unclear. To answer that question, we quantified the influence of surrounding light on the appearance of spots targeted to individual cones. The eye's aberrations were corrected with adaptive optics and retinal position was precisely tracked in real-time to compensate for natural movement. Subjects reported the color appearance of each spot. A majority of L-and M-cones consistently gave rise to the sensation of white, while a smaller group repeatedly elicited hue sensations. When blue sensations were reported they were more likely mediated by M- than L-cones. Blue sensations were elicited from M-cones against a short-wavelength light that preferentially elevated the quantal catch in surrounding S-cones, while stimulation of the same cones against a white background elicited green sensations. In one of two subjects, proximity to S-cones increased the probability of blue reports when M-cones were probed. We propose that M-cone increments excited both green and blue opponent pathways, but the relative activity of neighboring cones favored one pathway over the other.

Long-wavelength (reddish) hues induce unusually large gamma oscillations in the primate primary visual cortex

Electrophysiological signals recorded from the brain exhibit prominent gamma oscillations (∼30–80 Hz) under sensory stimulation. These oscillations are modulated by stimulus properties and behavioral state, and implicated to play a role in cognitive functions, such as attention and object-binding. In visual areas, gamma oscillations have mainly been studied using achromatic gratings/gabors. Here we show that color stimuli generate unprecedented levels of gamma oscillations in the local field potentials recorded from primate area V1. The strongest oscillations are induced by long-wavelength (reddish) hues. Importantly, their strength depends on color saturation but not on luminance, and is strongly correlated with the L−M cone contrast produced by stimuli, suggesting that gamma oscillations may represent key components of the processing of visual chromatic information.

Gamma oscillations (∼30–80 Hz) are a prominent signature of electrophysiological signals, with a purported role in natural vision. Previous studies in the primary visual cortex (area V1) have shown that achromatic gratings or gabor stimuli generate salient gamma oscillations, whose strength and frequency depend on stimulus properties such as their size, contrast, and orientation. Surprisingly, although natural images are rarely achromatic, the effect of chromatic input on gamma has not been thoroughly investigated. Recording from primate V1, we show that gamma oscillations of extremely high magnitude (peak increase of ∼300-fold in some cases), far exceeding the gamma generated by optimally tuned achromatic gratings, are induced in the local field potentials by full-field color stimuli of different hues. Furthermore, gamma oscillations are sensitive to the hue of the chromatic input, with the strongest oscillations for long-wavelength (reddish) hues and another, smaller gamma response peak for hues in the short-wavelength (bluish) range, which lie approximately on the two cardinal chromatic response axes of the upstream lateral geniculate nucleus neurons. These oscillations depended critically on the purity of the hue, decreasing with hue desaturation, but remained robust for pure hue stimuli even at reduced luminance. Importantly, the magnitude of gamma oscillations was highly correlated with positive L−M cone contrast produced by the stimuli, suggesting that gamma could be a marker of the specific mechanisms underlying this computation. These findings provide insights into the generation of gamma oscillations, as well as the processing of color along the visual pathway.

Contrast-dependent red-green hue shift

On bright surrounds, red-green-balanced yellow targets become greenish brown with decreased target luminance, and red-green-balanced brown targets become reddish yellow with increased target luminance. These effects imply luminance- and/or contrast-dependent weighting of M- and L-cone signals in post-receptoral pathways. We show psychophysically that luminance contrast between the surround and the target is the primary determinant of the magnitude of red-green hue shift, requiring surround luminance at least twice the target luminance and increasing with further increases of surround/target contrast. There is a much smaller effect of absolute stimulus luminance, with dimmer stimuli showing slightly larger hue shifts. To evaluate a possible retinal origin of the changes in cone-signal weightings underlying the hue shift, we recorded spike responses from both ON- and OFF-center midget ganglion cells in peripheral primate retina. We found no evidence that the relative strength of L- and M-cone post-receptoral responses changed systematically with change of surround irradiance. Nor was there any systematic difference between ON- and OFF-subtypes. This suggests that the change in cone signal weighting occurs later in the visual system.

Hue opponency: chromatic valence functions, individual differences, cortical winner-take-all opponent modeling, and the relationship between spikes and sensitivity

Neural spike rate data are more restricted in range than related psychophysical data. For example, several studies suggest a compressive (roughly cube root) nonlinear relationship between wavelength-opponent spike rates in primate midbrain and color appearance in humans, two rather widely separated domains. This presents an opportunity to partially bridge a chasm between these two domains and to probe the putative nonlinearity with other psychophysical data. Here neural wavelength-opponent data are used to create cortical competition models for hue opponency. This effort led to creation of useful models of spiking neuron winner-take-all (WTA) competition and MAX selection. When fed with actual primate data, the spiking WTA models generate reasonable wavelength-opponent spike rate behaviors. An average psychophysical observer for red-green and blue-yellow opponency is curated from eight applicable studies in the refereed and dissertation literatures, with cancellation data roughly every 10 nm in 18 subjects for yellow-blue opponency and 15 subjects for red-green opponency. A direct mapping between spiking neurons with broadband wavelength sensitivity and human psychophysical luminance yields a power law exponent of 0.27, similar to the cube root nonlinearity. Similarly, direct mapping between the WTA model opponent spike rates and psychophysical opponent data suggests power law relationships with exponents between 0.24 and 0.41.

Age-related changes in ON and OFF responses to luminance increments and decrements

Impulse response functions for an incremental luminous pulse (ON flash) or a decremental luminous pulse (OFF flash) were derived for twelve young (19-24 years old) and ten old (65-84 years old) observers. Thresholds were measured for two pulses separated by stimulus-onset-asynchronies from 13.3 to 186.7 ms. The pulses had a spatial Gaussian shape and were presented as increments or decrements on a 15  cd/m² equal-energy white background, having the same chromaticity as the pulse. A spatial four-alternative forced-choice method was combined with a staircase procedure. Retinal illuminance was equated individually by heterochromatic flicker photometry and using a 2.3-mm exit pupil in a Maxwellian-view optical system to reduce the effects of age-related changes and individual variations in lens density and pupil size. Luminance ON- and OFF-impulse response functions calculated from the threshold data revealed significant age-related changes in the response amplitude of both first excitatory and first inhibitory phases. However, there were no significant changes in the time to the first peak or the second peak. These age-related changes in luminance varying ON- and OFF-impulse response functions (IRFs), reflecting putative properties of the magnocellular pathway, are discussed in relation to motion detection and the balance of ON and OFF pathways across the life span.

Perceptual Color Space Representations in the Oculomotor System Are Modulated by Surround Suppression and Biased Selection

The oculomotor system utilizes color extensively for planning saccades. Therefore, we examined how the oculomotor system actually encodes color and several factors that modulate these representations: attention-based surround suppression and inherent biases in selecting and encoding color categories. We measured saccade trajectories while human participants performed a memory-guided saccade task with color targets and distractors and examined whether oculomotor target selection processing was functionally related to the CIE (x,y) color space distances between color stimuli and whether there were hierarchical differences between color categories in the strength and speed of encoding potential saccade goals. We observed that saccade planning was modulated by the CIE (x,y) distances between stimuli thus demonstrating that color is encoded in perceptual color space by the oculomotor system. Furthermore, these representations were modulated by (1) cueing attention to a particular color thereby eliciting surround suppression in oculomotor color space and (2) inherent selection and encoding biases based on color category independent of cueing and perceptual discriminability. Since surround suppression emerges from recurrent feedback attenuation of sensory projections, observing oculomotor surround suppression suggested that oculomotor encoding of behavioral relevance results from integrating sensory and cognitive signals that are pre-attenuated based on task demands and that the oculomotor system therefore does not functionally contribute to this process. Second, although perceptual discriminability did partially account for oculomotor processing differences between color categories, we also observed preferential processing of the red color category across various behavioral metrics. This is consistent with numerous previous studies and could not be simply explained by perceptual discriminability. Since we utilized a memory-guided saccade task, this indicates that the biased processing of the red color category does not rely on sustained sensory input and must therefore involve cortical areas associated with the highest levels of visual processing involved in visual working memory.

Photopigments and the dimensionality of animal color vision

Early color-matching studies established that normal human color vision is trichromatic. Subsequent research revealed a causal link between trichromacy and the presence in the retina of three classes of cone photopigments. Over the years, measurements of the photopigment complements of other species have expanded greatly and these are frequently used to predict the dimensionality of an animal's color vision. This review provides an account of how the linkage between the number of active photopigments and the dimensions of human color vision developed, summarizes the various mechanisms that can impact photopigment spectra and number, and provides an across-species survey to examine cases where the photopigment link to the dimensionality of color vision has been claimed. The literature reveals numerous instances where the human model fails to account for the ways in which the visual systems of other animals exploit information obtained from the presence of multiple photopigments in support of their behavior.

Bioplausible multiscale filtering in retino-cortical processing as a mechanism in perceptual grouping

Why does our visual system fail to reconstruct reality, when we look at certain patterns? Where do Geometrical illusions start to emerge in the visual pathway? How far should we take computational models of vision with the same visual ability to detect illusions as we do? This study addresses these questions, by focusing on a specific underlying neural mechanism involved in our visual experiences that affects our final perception. Among many types of visual illusion, 'Geometrical' and, in particular, 'Tilt Illusions' are rather important, being characterized by misperception of geometric patterns involving lines and tiles in combination with contrasting orientation, size or position. Over the last decade, many new neurophysiological experiments have led to new insights as to how, when and where retinal processing takes place, and the encoding nature of the retinal representation that is sent to the cortex for further processing. Based on these neurobiological discoveries, we provide computer simulation evidence from modelling retinal ganglion cells responses to some complex Tilt Illusions, suggesting that the emergence of tilt in these illusions is partially related to the interaction of multiscale visual processing performed in the retina. The output of our low-level filtering model is presented for several types of Tilt Illusion, predicting that the final tilt percept arises from multiple-scale processing of the Differences of Gaussians and the perceptual interaction of foreground and background elements. The model is a variation of classical receptive field implementation for simple cells in early stages of vision with the scales tuned to the object/texture sizes in the pattern. Our results suggest that this model has a high potential in revealing the underlying mechanism connecting low-level filtering approaches to mid- and high-level explanations such as 'Anchoring theory' and 'Perceptual grouping'.

Circuitry to explain how the relative number of L and M cones shapes color experience

The wavelength of light that appears unique yellow is surprisingly consistent across people even though the ratio of middle (M) to long (L) wavelength sensitive cones is strikingly variable. This observation has been explained by normalization to the mean spectral distribution of our shared environment. Our purpose was to reconcile the nearly perfect alignment of everyone's unique yellow through a normalization process with the striking variability in unique green, which varies by as much as 60 nm between individuals. The spectral location of unique green was measured in a group of volunteers whose cone ratios were estimated with a technique that combined genetics and flicker photometric electroretinograms. In contrast to unique yellow, unique green was highly dependent upon relative cone numerosity. We hypothesized that the difference in neural architecture of the blue-yellow and red-green opponent systems in the presence of a normalization process creates the surprising dependence of unique green on cone ratio. We then compared the predictions of different theories of color vision processing that incorporate L and M cone ratio and a normalization process. The results of this analysis reveal that—contrary to prevailing notions--postretinal contributions may not be required to explain the phenomena of unique hues.

Development and evaluation of colour control interfaces for LED lighting

To capitalise on the colour tuning capabilities of LED lighting, a model for converting device-specific control signals to chromaticity coordinates was used in a psychophysical experiment evaluating the usability of three colour control interfaces based on RGB (red, green, blue), HSB (hue, saturation, brightness) and opponent colour mixing systems. Although common and well accepted, the RGB interface had lowest usability based on both psychophysical results and subjective ratings. The usability of HSB and opponent colour interfaces was not significantly different. These findings can guide the development of useful and efficient colour control interfaces for tunable LED lighting systems.

A neural signature of the unique hues

Since at least the 17^th century there has been the idea that there are four simple and perceptually pure “unique” hues: red, yellow, green, and blue, and that all other hues are perceived as mixtures of these four hues. However, sustained scientific investigation has not yet provided solid evidence for a neural representation that separates the unique hues from other colors. We measured event-related potentials elicited from unique hues and the ‘intermediate’ hues in between them. We find a neural signature of the unique hues 230 ms after stimulus onset at a post-perceptual stage of visual processing. Specifically, the posterior P2 component over the parieto-occipital lobe peaked significantly earlier for the unique than for the intermediate hues (Z = −2.9, p = 0.004). Having identified a neural marker for unique hues, fundamental questions about the contribution of neural hardwiring, language and environment to the unique hues can now be addressed.

Variations in normal color vision. VI. Factors underlying individual differences in hue scaling and their implications for models of color appearance

Observers with normal color vision vary widely in their judgments of color appearance, such as the specific spectral stimuli they perceive as pure or unique hues. We examined the basis of these individual differences by using factor analysis to examine the variations in hue-scaling functions from both new and previously published data. Observers reported the perceived proportion of red, green, blue or yellow in chromatic stimuli sampling angles at fixed intervals within the LM and S cone-opponent plane. These proportions were converted to hue angles in a perceptual-opponent space defined by red vs. green and blue vs. yellow axes. Factors were then extracted from the correlation matrix using PCA and Varimax rotation. These analyses revealed that inter-observer differences depend on seven or more narrowly-tuned factors. Moreover, although the task required observers to decompose the stimuli into four primary colors, there was no evidence for factors corresponding to these four primaries, or for opponent relationships between primaries. Perceptions of "redness" in orange, red, and purple, for instance, involved separate factors rather than one shared process for red. This pattern was compared to factor analyses of Monte Carlo simulations of the individual differences in scaling predicted by variations in standard opponent mechanisms, such as their spectral tuning or relative sensitivity. The observed factor pattern is inconsistent with these models and thus with conventional accounts of color appearance based on the Hering primaries. Instead, our analysis points to a perceptual representation of color in terms of multiple mechanisms or decision rules that each influence the perception of only a relatively narrow range of hues, potentially consistent with a population code for color suggested by cortical physiology.

Binocular vs. monocular hue perception

Hue perception has been shown to differ for the same stimulus when presented to the temporal and the nasal areas of the retina. The present study investigated perceptual differences in stimuli viewed binocularly or monocularly in the peripheral retina to determine how hue information combines across the two retinas for a stimulus falling on the temporal retina of one eye and the nasal retina of the other. A hue-scaling procedure was utilized to ascertain hue perception for three color- and binocular-normal observers viewing monochromatic stimuli (450-670nm, 20nm steps) ranging in size from 1.0° to 3.7°. Peripherally-presented binocular stimuli fell upon the nasal retina of one eye and the temporal retina of the other. Hue-scaling results indicated that peripheral binocular hue and saturation perceptions for smaller stimuli were more similar to those of stimuli falling on the temporal retina in the monocular condition. Hue-scaling data were also used to determine perceptive field sizes for the four elemental hues. Binocular perceptive field sizes were more similar to those obtained for stimuli falling on the temporal retina in the monocular conditions. Eye dominance did not appear to have an effect on hue perception. The results seem to indicate that visual information from the temporal retina is weighted more heavily when information from the two eyes is combined cortically. This finding may relate to differences in V1 cortical activation for stimuli presented to the nasal retina versus the temporal retina.