Spatial reconstruction of signals from short-wavelength cones

Color, Pattern, and the Retinal Cone Mosaic

Textbook trichromacy accounts for human color vision in terms of spectral sampling by three classes of cone photoreceptors. This account neglects entangling of color and pattern information created by wavelength-dependent optical blur (chromatic aberrations) and interleaved spatial sampling of the retinal image by the three classes of cones. Recent experimental, computational, and neurophysiological work is now considering color and pattern vision at the elementary scale of daylight vison, that is at the scale of individual cones. The results provide insight about rich interactions between color and pattern vision as well as the role of the statistical structure of natural scenes in shaping visual processing.

Color and the Cone Mosaic

Vision is limited by the measurements taken by the cone photoreceptors. To provide useful perceptual representations, the brain must go beyond the measurements and make inferences about the scene being viewed. This article considers the first stages of spatiochromatic vision. We show how spatial and chromatic information become intertwined by the optics of the eye and because of the structure of the retinal cone mosaic, and we consider the consequent implications for perception. Because there is at most one cone at each retinal location, the standard treatment of human trichromacy does not apply at fine spatial scales. Rather, trichromacy results from a perceptual inference based on measurements from cones of different classes at different locations. Our treatment emphasizes linking physics, biology, and computation with the goal of providing a framework for a larger understanding of how the brain interprets photoreceptor excitations to see objects and their properties.

Humans make efficient use of natural image statistics when performing spatial interpolation

Visual systems learn through evolution and experience over the lifespan to exploit the statistical structure of natural images when performing visual tasks. Understanding which aspects of this statistical structure are incorporated into the human nervous system is a fundamental goal in vision science. To address this goal, we measured human ability to estimate the intensity of missing image pixels in natural images. Human estimation accuracy is compared with various simple heuristics (e.g., local mean) and with optimal observers that have nearly complete knowledge of the local statistical structure of natural images. Human estimates are more accurate than those of simple heuristics, and they match the performance of an optimal observer that knows the local statistical structure of relative intensities (contrasts). This optimal observer predicts the detailed pattern of human estimation errors and hence the results place strong constraints on the underlying neural mechanisms. However, humans do not reach the performance of an optimal observer that knows the local statistical structure of the absolute intensities, which reflect both local relative intensities and local mean intensity. As predicted from a statistical analysis of natural images, human estimation accuracy is negligibly improved by expanding the context from a local patch to the whole image. Our results demonstrate that the human visual system exploits efficiently the statistical structure of natural images.

Trichromatic reconstruction from the interleaved cone mosaic: Bayesian model and the color appearance of small spots

Observers use a wide range of color names, including white, to describe monochromatic flashes with a retinal size comparable to that of a single cone. We model such data as a consequence of information loss arising from trichromatic sampling. The model starts with the simulated responses of the individual L, M, and S cones actually present in the cone mosaic and uses these to reconstruct the L-, M-, and S-cone signals that were present at every image location. We incorporate the optics and the mosaic topography of individual observers, as well as the spatio-chromatic statistics of natural images. We simulated the experiment of H. Hofer, B. Singer, & D. R. Williams (2005) and predicted the color name on each simulated trial from the average chromaticity of the spot reconstructed by our model. Broad features of the data across observers emerged naturally as a consequence of the measured individual variation in the relative numbers of L, M, and S cones. The model's output is also consistent with the appearance of larger spots and of sinusoidal contrast modulations. Finally, the model makes testable predictions for future experiments that study how color naming varies with the fine structure of the retinal mosaic.

Evidence for the existence of colour mechanisms producing unique hues as derived from a colour illusion based on spatio-chromatic interactions

When its spatial frequency is high enough, a grid of grey horizontal strips presented on a coloured background may change its neutral colour. It was found that some background colours induce a strong illusion and some no illusion at all. The effect of the background colour on the illusion was studied for the spatial frequencies of 0.5, 2.5, 4, and 8 c/deg. Thirty chromaticities (evenly distributed across the colour gamut triangle) of the backgrounds in the equiluminant plane, and 24 in the ML plane (where S-contrast was zero), were tested. Five matches were made for each frequency and each background chromaticity. Viewing was binocular. For the low (0.5 c/deg) frequency strips, the backgrounds were found to induce the colour, if any, approximately complimentary to that of the background (i.e., chromatic simultaneous contrast). For the high (8 c/deg) frequency, most backgrounds induced only illusory colours close to unique hues (yellow, blue, and green), with a few backgrounds inducing a mixture of green with blue. Then, the method of adjustment was used to determine the unique hues for the same three observers. A remarkable similarity was found between unique hues and illusory loci, suggesting that the illusion is due to a difference in the spatial resolution of the post-receptor channels producing the unique hues.

An effect of luminance contrast on high-spatial-frequency tritanopia

Luminance contrast was found to affect high-spatial-frequency tritanopia despite the S-cone coordinates of the stimuli being kept constant. This proves that a traditional account of artificial tritanopias based on the spatial resolution differences between the S-cone channel and the M- and L-cone channels is not applicable here. We suggest that the lack of spatial resolution in one of the post-receptor (rather than receptor) spatio-chromatic channels could be a cause of high-spatial-frequency tritanopia.

One blue colour channel or two?

Contrary to the general belief that the yellow-blue mechanism has lower spatial resolution than the red-green mechanism, it has been recently claimed that both mechanisms have similar spatial sensitivity (McKeefry et al, 2001 Vision Research 41 245-255). Studying high-spatial-frequency tritanopia (a colour illusion based on spatio-chromatic interactions in human vision), we found strong evidence for the existence of two blue mechanisms-with low and high spatial-frequency resolution. If confirmed, this may resolve the apparent paradox concerning spatial resolution of the yellow-blue mechanism.

Seeing with S cones

The S cone is highly conserved across mammalian species, sampling the retinal image with less spatial frequency than other cone photoreceptors. In human and monkey retina, the S cone represents typically 5-10% of the cone mosaic and distributes in a quasi-regular fashion over most of the retina. In the fovea, the S cone mosaic recedes from a central "S-free" zone whose size depends on the optics of the eye for a particular primate species: the smaller the eye, the less extreme the blurring of short wavelengths, and the smaller the zone. In the human retina, the density of the S mosaic predicts well the spatial acuity for S-isolating targets across the retina. This acuity is likely supported by a bistratified retinal ganglion cell whose spatial density is about that of the S cone. The dendrites of this cell collect a depolarizing signal from S cones that opposes a summed signal from M and L cones. The source of this depolarizing signal is a specialized circuit that begins with expression of the L-AP4 or mGluR6 glutamate receptor at the S cone-->bipolar cell synapse. The pre-synaptic circuitry of this bistratified ganglion cell is consistent with its S-ON/(M+L)-OFF physiological receptive field and with a role for the ganglion cell in blue/yellow color discrimination. The S cone also provides synapses to other types of retinal circuit that may underlie a contribution to the cortical areas involved with motion discrimination.

High-spatial-frequency tritanopia: S-filling-in or S-filtering-out?

It has long been an accepted fact that a small test field presented against a large background may change its colour appearance because the test-field background contrast is attenuated by the receptor colour channels unequally (Willmer, 1944 Nature 153 774-775; Hartridge, 1947 Philosophical Transactions of the Royal Society of London, Series B 232 519-671). Such an effect is usually called small-field tritanopia. However, as shown in the present report, a similar colour illusion can be achieved with a large test field as well, provided its spatial-frequency content is high enough to reveal the differential drop of contrast sensitivity for the receptor colour channels (high-spatial-frequency tritanopia). A few demonstrations are presented which show that a traditional explanation of high-spatial-frequency tritanopia (including small-field tritanopia), based on the hypothetical process of filling-in, is not correct. An alternative account, based on spatial filtering within the receptor colour channels, is put forward.

Spatial order in short-wavelength-sensitive cone photoreceptors: a comparative study of the primate retina

We compared the spatial distribution of short-wavelength-sensitive (SWS or blue) cone photoreceptors in the retinas of eight primate species. The regularity of the SWS cone array was quantified with a statistic (packing factor) that varies between a random distribution (0) and a triangular array (1). We find wide variability among species, with packing factors varying between 0.06 and 0.3. The SWS cone array in at least two New World monkey species is indistinguishable from a random array. The SWS cone density gradient across the retina was measured in the capuchin monkey Cebus apella and the squirrel monkey Saimiri sciureus. Both species show a peak density of 5,000-8,000 cells/mm2 at the fovea and a 50-fold central-peripheral density gradient. In contrast to the wide variation in local regularity, the spatial density and the topography of SWS cones are well preserved across primates.

Binocular rivalry with isoluminant stimuli visible only via short-wavelength-sensitive cones

To test whether the binocular contour rivalry mechanism is tritanopic, we presented isoluminant, rival stimuli visible only via the short-wavelength-sensitive (S) cones. We stimulated only the S cones with violet gratings superimposed on a bright yellow field that adapted the responses of the middle- and long-wavelength-sensitive (M and L) cones. We found that an S-cone grating presented to one eye rivalled with an orthogonal grating presented to the other. Rivalry persisted over a range of luminances and contrasts of the S-cone stimuli, and was greater than could be accounted for by nonrival fading. The spatial spread of rivalry from S-cone stimuli is similar to that for the same stimuli when visible also to the M and L cones (luminance stimuli). We found that an S-cone stimulus would rival with a luminance stimulus, and exploited this to determine the equivalent luminance contrast of S-cone stimuli by putting them in a rivalry competition with luminance stimuli. For rivalry, the equivalent luminance contrast of isoluminant, S-cone stimuli is much less than their S-cone contrast. The existence of rivalry with isoluminant stimuli, along with earlier evidence that such stimuli can support stereopsis, challenges the view that an achromatic channel alone drives certain higher level functions such as depth perception.