Interaction of the long-wave cones and the rods to produce color sensations

The Paton prize lecture 2021: A colourful experience leading to a reassessment of colour vision and its theories

In this lecture, given in honour of Sir William Paton, a brilliant scientist and one of Britain's great patrons of biology, I give a personal account of the fundamental issues in colour vision that I have tackled since 1973, when I discovered a cortical zone lying outside the primary visual cortex that is rich in cells with chromatic properties. I do not provide an exhaustive review of colour vision but summarise how my views on colour vision and theories surrounding it have changed in light of that discovery.

Diverse Cell Types, Circuits, and Mechanisms for Color Vision in the Vertebrate Retina

Synaptic interactions to extract information about wavelength, and thus color, begin in the vertebrate retina with three classes of light-sensitive cells: rod photoreceptors at low light levels, multiple types of cone photoreceptors that vary in spectral sensitivity, and intrinsically photosensitive ganglion cells that contain the photopigment melanopsin. When isolated from its neighbors, a photoreceptor confounds photon flux with wavelength and so by itself provides no information about color. The retina has evolved elaborate color opponent circuitry for extracting wavelength information by comparing the activities of different photoreceptor types broadly tuned to different parts of the visible spectrum. We review studies concerning the circuit mechanisms mediating opponent interactions in a range of species, from tetrachromatic fish with diverse color opponent cell types to common dichromatic mammals where cone opponency is restricted to a subset of specialized circuits. Distinct among mammals, primates have reinvented trichromatic color vision using novel strategies to incorporate evolution of an additional photopigment gene into the foveal structure and circuitry that supports high-resolution vision. Color vision is absent at scotopic light levels when only rods are active, but rods interact with cone signals to influence color perception at mesopic light levels. Recent evidence suggests melanopsin-mediated signals, which have been identified as a substrate for setting circadian rhythms, may also influence color perception. We consider circuits that may mediate these interactions. While cone opponency is a relatively simple neural computation, it has been implemented in vertebrates by diverse neural mechanisms that are not yet fully understood.

Rod and cone interactions in the retina

We have long known that rod and cone signals interact within the retina and can even contribute to color vision, but the extent of these influences has remained unclear. New results with more powerful methods of RNA expression profiling, specific cell labeling, and single-cell recording have provided greater clarity and are showing that rod and cone signals can mix at virtually every level of signal processing. These interactions influence the integration of retinal signals and make an important contribution to visual perception.

Vision under mesopic and scotopic illumination

Evidence has accumulated that rod activation under mesopic and scotopic light levels alters visual perception and performance. Here we review the most recent developments in the measurement of rod and cone contributions to mesopic color perception and temporal processing, with a focus on data measured using a four-primary photostimulator method that independently controls rod and cone excitations. We discuss the findings in the context of rod inputs to the three primary retinogeniculate pathways to understand rod contributions to mesopic vision. Additionally, we present evidence that hue perception is possible under scotopic, pure rod-mediated conditions that involves cortical mechanisms.

What can fish brains tell us about visual perception?

Fish are a complex taxonomic group, whose diversity and distance from other vertebrates well suits the comparative investigation of brain and behavior: in fish species we observe substantial differences with respect to the telencephalic organization of other vertebrates and an astonishing variety in the development and complexity of pallial structures. We will concentrate on the contribution of research on fish behavioral biology for the understanding of the evolution of the visual system. We shall review evidence concerning perceptual effects that reflect fundamental principles of the visual system functioning, highlighting the similarities and differences between distant fish groups and with other vertebrates. We will focus on perceptual effects reflecting some of the main tasks that the visual system must attain. In particular, we will deal with subjective contours and optical illusions, invariance effects, second order motion and biological motion and, finally, perceptual binding of object properties in a unified higher level representation.

The color of night: Surface color perception under dim illuminations

Several studies document rudimentary color vision under dim illumination. Here, hue perceptions of paper color samples were determined for a wide range of light levels, including very low light levels where rods alone mediate vision. The appearances of 24 paper color samples from the OSA Uniform Color Scales were gauged under successively dimmer illuminations from 10-0.0003 Lux. Triads of samples were chosen representing each of eight basic color categories; red, pink, orange, yellow, green, blue, purple, and gray. Samples within each triad varied in lightness. Observers sorted samples into groups that they could categorize with specific color names. Above 0.32 Lux, observers sorted the samples into the originally chosen color groups with few exceptions. For 0.1-0.01 Lux, the red and orange samples were usually correctly identified as either red or orange. The remaining samples tended to be grouped into two categories, associated with the scotopic sample reflectance. The lowest reflectance samples were below threshold and were named black. The higher reflectance group was named predominately as green or blue-green (three observers; the fourth observer used blue or achromatic). At the three dimmest levels (< or = 0.0032 Lux) there continued to be conspicuous color percepts. Color categories were reliably assigned based on relative sample scotopic lightness. Of the samples above threshold, those with lower reflectance were classified as red or orange (all observers) and the higher reflectance samples as green or blue-green (three observers) or achromatic or blue (the fourth observer). Rods and L-cones presumably mediated color percepts at the intermediate light levels used in the study. At the three lowest light levels there were distinct color appearances mediated exclusively by rods. We speculate that at these light levels the visual system estimates probable colors based on prior natural experience.

Matching rod percepts with cone stimuli

Traditional methods for studying the effects of rod activity on color vision make it hard to assess the underlying physiological mechanisms. In this study, rod-mediated changes in color appearance were assessed by matching them with cone-mediated color changes. A four-primary photostimulator allowed independent control of rod and cone stimulation and identification of the cone types that generate color sensations equivalent to rod color sensations. The results showed that increases in rod stimulation required matches with cone stimuli that excited M-cones more than L-cones for all conditions. Matches for low-luminance conditions also required some S-cone stimulation. A subsidiary experiment showed that increases in rod modulation of an inducing field produced chromatic contrast effects like those produced by the M-cone system. The data are consistent with a hypothesis of perceptual normalization of scotopic vision to the chromatic appearance of objects under photopic conditions.

Rod inputs to macaque ganglion cells

The strength of rod inputs to ganglion cells was assessed in the macaque retina at retinal positions within 3-15 deg eccentricity. The experimental paradigm used temporally modulated heterochromatic lights whose relative phase was varied. This paradigm provided a sensitive test to detect rod input. In parvocellular (PC) pathway cells, the gain of the cone-driven signal decreased with decrease in luminance. At 2 td a weak rod response, of a few impulses per second for 100% rod modulation, was revealed in about 60% of cells. For blue-on cells, the cone-driven response also decreased with retinal illuminance, but no rod response could be found. In magnocellular (MC) pathway cells, rod input was much more apparent. Responses became rod dominated at and below 20 td; we cannot exclude rod intrusion at higher retinal illuminances. Responsivity was maintained even at low retinal illuminances. Temporal-frequency dependent rod-cone interactions were observed in MC-pathway cells. Rod responses were of longer latency than cone responses, but there was no evidence of any difference in rod latency between parvocellular and magnocellular pathways.

Partial additivity of rod signals with M- and L-cone signals in increment detection

Test additivity experiments revealed the combination rules for increment detection by rods and either M- or L-cone-dominated mechanisms isolated by means of chromatic adaptation (Stiles' pi 4 and pi 5, respectively). Increment thresholds were measured for single test wavelengths detected by each mechanism. Pairs of test wavelengths were then superimposed, and increment thresholds were measured for simultaneous detection by both rod and cone mechanisms. The observed degree of additivity was corrected (reduced) to compensate for the partial detection by each mechanism of both test wavelengths in the combined stimuli. We find that subthreshold rod signals are partially additive with subthreshold signals from both M- and L-cones. The degree of additivity is high and similar for both M- and L-cones: less than the ideal prediction of linear addition, but greater than that predicted by either probability summation of independent mechanisms or orthogonal vector addition.

Mechanisms of chromatic rod vision in scotopic illumination

After viewing a coloured patch for 30 sec, successive contrast colours were triggered by stimulating either rods or cones. The conditions were arranged so that the rod and cone stimuli matched both with respect to chromaticness and brightness in a chromatically neutral state of adaptation. The results showed that the contrast colours triggered by rods were strikingly similar to those triggered by cones. Yet, the scotopic contrast colours, as compared with the photopic ones, were generally found to be somewhat displaced toward blue. This displacement was attributed to the difference in test conditions. Thus, it was suggested that, although rods may excite all the different types of spectrally opponent cells, they generally tend to prefer the short-wave cells. Moreover, it was concluded that the scotopic successive contrast colours are triggered by rod signals feeding into the primary rod pathway and therefore must originate centrally to the receptor level.

Color constancy. I. Basic theory of two-stage linear recovery of spectral descriptions for lights and surfaces

Changing a scene's illuminant causes the chromatic properties of reflected lights to change. This change in the lights from surfaces provides spectral information about surface reflectances and illuminants. We examine conditions under which these properties may be recovered by using bilinear models. Necessary conditions that follow from comparing the number of equations and the number of unknowns in the recovery procedure are not sufficient for unique recovery. Necessary and sufficient conditions follow from demanding a one-to-one relationship between quantum catch data and sets of lit surfaces. We present an algorithm for determining whether spectral descriptions of lights and surfaces can be recovered uniquely from reflected lights.

Rod flicker perception: scotopic duality, phase lags and destructive interference

Rod vision has a duality of organization: at mesopic luminances rod signals have access to a slow, sensitive pathway (which we refer to, following Stiles, as pi 0) and a fast, insensitive pathway (pi' 0). The phase lag between the two rod signals increases with frequency until at 15-Hz the rod signals transmitted through the two pathways emerge out-of-phase, so that destructive interference produces a nulling of the apparent flicker. Relative to the cones, the phase lag of pi' 0 is roughly half that of pi 0. Thus at 15-Hz pi' 0 signals can be out-of-phase with cone signals, so that the signals from the slower pathway, pi 0, are actually in phase with cone signals. We have investigated the frequency response, adaptation behavior and phase characteristics of the two rod processes. The slower process, pi 0 is more sensitive than pi' 0, and dominates from absolute threshold up to low mesopic levels. The adaptation of pi 0 seems not to be associated with a change in time constant, but rather with simple response compression or sensitivity scaling. The time constant of pi' 0, however, does change with adaptation. There are large differences in the way that light adaptation changes the sensitivity of the two processes: signals from pi'0 may evade part of the postreceptoral sensitivity regulating mechanism normally associated with rod vision. The ability of signals from pi 0 and pi' 0 to reinforce or cancel each other, however, suggests that they are later reunited in a common pathway.

Color perception under chromatic adaptation: red/green equilibria with adapted short-wavelength-sensitive cones

Chromatic adaptation can dramatically alter the color appearance of a light. The specific effect of adapting short-wavelength-sensitive (SWS) cones is examined by using two adapting wavelengths that lie on a tritanopic confusion line. The change in color appearance caused by signals from adapted SWS cones is isolated by restricting the wavelengths of the test light to 550 nm or longer. Thus the test negligibly stimulates SWS cones, so their sensitivity does not affect the test's appearance. The results show that adapted SWS cones contribute redness to the appearance of a superimposed test light, while not affecting sensitivity of MWS and LWS cones. Quantitatively, the redness from SWS cones illuminated by a large adapting field approaches physical admixture of test and adapting lights. This is very different from an adapting field that stimulates only MWS and LWS cones which, due to a postreceptoral process, contributes much less redness to a small superimposed test than expected from admixture. The difference between the adapted SWS-cone and the adapted MWS/LWS-cone contributions to the color of a small test explains a surprising result: a bluish-green (491 nm) adapting field contributes redness to a superimposed test light.

Rod influence in dichromatic surface color perception

Two protanopes, two deuteranopes, and two normal subjects named 424 OSA Uniform Color Scales samples using single-word color terms of their choice under three different experimental conditions. When viewing a stimulus field subtending about 4 deg, the performance of the dichromats revealed a substantial ability to discriminate colors along the red-green axis. When the stimuli were limited to the central fovea, or when rods were excluded with a bleach, dichromats could no longer categorize colors in the red-green dimension. The different conditions did not affect the performance of the normals. The results suggest that rods contribute signals used by dichromats, along with lightness cues, to help discriminate and categorize surface colors.

Mesopic spectral responses and the Purkinje shift of macaque lateral geniculate nucleus cells

Spectral responsiveness of different classes of macaque LGN cells at eccentricities smaller than 12 degrees was studied at low light intensities. Cone thresholds of cells varied from 1 to 10 td. Rod inputs were found in all classes of cells, including inhibitory inputs in some cells. Rod inputs were not apparent above 10-40 td, giving a total mesopic range of about 1-40 td. Strong rod-mediated responses could be evoked in broadband phasic cells and in spectrally opponent cells excited by short wavelengths. Only weak if any excitatory responses could be evoked by short wavelengths at scotopic levels in spectrally opponent long-wavelength excited cells. Hence, rod inputs do not confound the spectral responsiveness of cells because no spectrally opponent cell excited by long-wavelength stimuli at photopic levels became significantly responsive to short wavelength stimuli at mesopic or scotopic intensities. The so-called "rod color" may be blue. An increase in the dominance of wide-band cell responses that may explain the Bezold-Brücke hue shift was observed at higher stimulus radiances at wavelengths near 450 and 650 nm.

Effect of retinal location on cone critical flicker frequency

This experiment investigated the differential sensitivity of various areas of the retina using flicker. For 12 subjects testing was carried out in the fovea, and 5 degrees and 6 degrees temporal to the fovea using a wavelength of 555 nm. Testing was done both in the presence of a surround beam and in its absence. In all cases, there was a 2.5- to 4-fold increase in the amount of energy needed to perceive flicker as testing was shifted from the fovea to the periphery. A number of possible explanations are suggested to account for these findings.

Noise in ultrasonic imaging

The feasibility of noise measurement in medical ultrasonic imaging has been studied and a theoretical evaluation of stochastic noise influence upon the perceived image has been undertaken. The influence of noise on a perceived image has been calculated by considering the light intensity perception characteristic of the human eye and it has been found taht the additive noise in the video signal pushes the small signals in the image towards the invisible (black) region. Impulse noise measurements of amplitude and time jitter have been set up using a standard water/CCl4 interface and pulse amplitude analysis to take into account the random nature of noise by distribution measurements.

Evidence that the colored shadow effect is retinal

Polarized lenses were employed to present the images of the colored shadows effect separately to each eye. The phenomenon was perceived only when both images were viewed on the same retina, leading to the conclusion that the effect is entirely retinal in origin and does not involve the central nervous system.

Scotopic luminosity function and color-mixture data

The hypothesis that rods contribute to color information has not been generally accepted primarily because of the apparent lack of a connection between the scotopic luminosity function and color-mixture data. We present analyses showing that the scotopic luminosity function is intimately related to color data over the entire spectrum, indicating that rods play a central role in normal color vision. These results, not readily explainable in terms of the trichromatic theory, suggest an alternate idea of sensing in terms of the psychophysical quantities called brightness, hue, and saturation.

Large-field trichromacy in protanopes and deuteranopes

Protanopes and deuteranopes do not accept the classical dichromatic matches when field size extends to 8 degrees visual angle. Their unique matches of spectral yellow to a mixture of red and green are then mediated by the photoreceptors of small-field dichromacy interacting with a photoreceptor with the spectral sensitivity of rhodopsin. Our data suggest that large-field trichromacy is a general feature of protanopia and deuteranopia.

Variegated color sensations from rod-cone interactions: flicker-fusion experiments

Multicolored images were produced by the combination of two different color-separation images, one illuminated with 656 nm light and the other illuminated with 500 nm light. Flicker-fusion frequency measurements were used to identify the conditions in which the 500 nm light was below cone threshold. Under those conditions the multicolored images were shown to be generated by rod and long-wave cone interactions.

Interactions of rod and cone signals in the mudpuppy retina

Interactions between rod and cone signals in mudpuppy retinal neurones were investigated by intracellular recording. 2. The mudpuppy retina contains one kind of rod (lambda max = 525 nm) and one kind of cone (lambda max = 572 nm). The responses of receptors can be distinguished on the basis of their spectral sensitivities. 3. Rod and cone responses have different time courses of recovery and absolute sensitivities. Differences between receptor responses can be used to describe inputs to interneurones. 4. There are two spectral classes of horizontal cells: L-type and C-type. L-type cells are hyperpolarized by rods and cones in varying proportion, with some cells receiving little rod input. C-type cells are hyperpolarized by rods and depolarized by cones. 5. Bipolar cell receptive field centres receive input from cones or from rods and cones. There is no correlation between the spectral properties of centre responses and their polarity. 6. Antagonistic surrounds of bipolar cells show cone or rod and cone sensitivity. They are believed to be generated by the L-type horizontal cells. 7. Some bipolar cells exhibit chromatic interactions between cone signals in the centre and rod signals in the surround, which resemble those observed between the signals of different spectral classes of cones in species known to possess colour discrimination. 8. Amacrine and on-off ganglion cells have L-type responses showing both rod and cone sensitivity. 9. It is proposed that interactions between rod and cone signals observed in mudpuppy also exist in primate retina and are at least partially responsible for certain psychophysical observations of rod-cone interactions.

Colour receptors, and their synaptic connexions, in the retina of a cyprinid fish

Morphologically speaking, there are five kinds of cone cells in the retina of the rudd ( Scardinius erythrophthalmus ). But two of them, the principal elements of the double cones and the free principal cones, are probably functionally equivalent, while another, sparse, population of small ( oblique ) cones (which disappear in older fish), is unlikely to make a significant contribution to visual spectral sensitivity. Thus, principal and accessory cones (usually paired with one another), and single cones seem to be the three receptors which underlie the fish’s trichromacy. Photographic densitometry of individual cone cells was used to provide evidence that accessory cones contain a green-absorbing photopigment and the single cones a blue one. Other arguments are given in support of those identifications, and they also strongly suggest that principal cones contain the red-absorbing pigment. Golgi-impregnated bipolar cells were examined electron-microscopically to determine the specific patterns of synaptic connexion they make with these different, anatomically identifiable, colour cones and with the retinal rods. Three principal arrangements were distinguished (see figure 69, page 190). (1) Rod bipolar cells comprise two distinct morphological types, both of which connect exclusively to principal (red) cones as well as to the rods within the outlines of their dendritic fields. (2) Selective cone bipolar cells, more delicate neurons with considerably wider dendritic fields, connect (according to type) to one or other of the different colour cone populations. Examples analysed were specific for the accessory (green) or for the single (blue) cones; no bipolar cells were found connected only to red cones. (3) Mixed cone bipolars have the smallest dendritic fields, and connect to combinations of cones (for example, red and green, or green and blue, but not red and blue). They also have synaptic input (usually relatively sparse) from the rods. Cells were encountered connecting to all three cone types, but they were only partially analysed, and are not described at length. The light microscopic morphology of these bipolar cell types consistently reflects the detailed pattern of connexion each makes with the different receptor populations (just as the morphology of the cones reflects the spectral properties of their photopigment). But while their synaptic connectivity is generally highly specific for cone type, they do occasionally make anomalous connexions with the ‘wrong’ receptors. There is a high degree of divergence (page 85) at the receptor-bipolar synapses, and the different kinds of cones each characteristically connect to different numbers of bipolar cells. Principal (red) cones, which are the most numerous, individually connect to more bipolars than cones of other types, whose characteristic synaptic divergence is likewise related to the frequency with which they occur in the retina. However, rods, which are much more numerous than cones, do not conform with this generalization. The selectivity with which the synaptic terminals of the different cones are connected together by their invaginating basal processes was also examined. These processes link neighbouring synaptic terminals of differently coloured cones: specifically, principal (red) cone basal processes invaginate accessory (green) cone pedicles, and vice versa. Single (blue) cone basal processes connect only to accessory cone pedicles, but that synaptic relation is not reciprocated. These synapses between the cones have important bearing upon interpretation of the bipolar cell connectivity patterns. In their light, the interaction between colour channels which the convergence of different cones onto the mixed cone bipolar dendrites mediates, seems to re-iterate a process already undertaken more peripherally. Likewise, whereas the anatomy of the selective cone bipolars appears designed to convey activity from the individual cone populations, the responses of the receptors they sample must already be influenced by activity in other colour channels.

Rod-cone interactions: different color sensations from identical stimuli

Different color sensations were generated by two areas in a complex scene, even though both areas sent to the eye the same 656-nanometer radiance that excited the long-wave cones and excited only the rods.

Rod-cone interaction in human scotopic vision

Thresholds of a test flash were measured at various time intervals from onset of a conditioning flash under parafoveal scotopic conditions; rods or cones were selectively stimulated by utilizing either 420- or 680-nanometer light. Rod-cone interaction was indicated because conditioning flash presentation increased test threshold above control level for heterochromatic as well as for homochromatic stimulus pairs. The time course of these t.. reshold changes indicates that the rod system has a longer latency than the cone system.