Selectivity of human retinotopic visual cortex to S-cone-opponent, L/M-cone-opponent and achromatic stimulation

Chromatic visual evoked potentials: A review of physiology, methods and clinical applications

Objective assessment of the visual system can be performed electrophysiologically using the visual evoked potential (VEP). In many clinical circumstances, this is performed using high contrast achromatic patterns or diffuse flash stimuli. These methods are clinically valuable but they may only assess a subset of possible physiological circuitries within the visual system, particularly those involved in achromatic (luminance) processing. The use of chromatic VEPs (cVEPs) in addition to standard VEPs can inform us of the function or dysfunction of chromatic pathways. The chromatic VEP has been well studied in human health and disease. Yet, to date our knowledge of their underlying mechanisms and applications remains limited. This likely reflects a heterogeneity in the methodology, analysis and conclusions of different works, which leads to ambiguity in their clinical use. This review sought to identify the primary methodologies employed for recording cVEPs. Furthermore cVEP maturation and application in understanding the function of the chromatic system under healthy and diseased conditions are reviewed. We first briefly describe the physiology of normal colour vision, before describing the methodologies and historical developments which have led to our understanding of cVEPs. We thereafter describe the expected maturation of the cVEP, followed by reviewing their application in several disorders: congenital colour vision deficiencies, retinal disease, glaucoma, optic nerve and neurological disorders, diabetes, amblyopia and dyslexia. We finalise the review with recommendations for testing and future directions.

In primary visual cortex fMRI responses to chromatic and achromatic stimuli are interdependent and predict contrast detection thresholds

Chromatic and achromatic signals in primary visual cortex have historically been considered independent of each other but have since shown evidence of interdependence. Here, we investigated the combination of two components of a stimulus; an achromatic dynamically changing check background and a chromatic (L-M or S cone) target grating. We found that combinations of chromatic and achromatic signals in primary visual cortex were interdependent, with the dynamic range of responses to chromatic contrast decreasing as achromatic contrast increased. A contrast detection threshold study also revealed interdependence of background and target, with increasing chromatic contrast detection thresholds as achromatic background contrast increased. A model that incorporated a normalising effect of achromatic contrast on chromatic responses, but not vice versa, best predicted our V1 data as well as behavioural thresholds. Further along the visual hierarchy, the dynamic range of chromatic responses was maintained when compared to achromatic responses, which became increasingly compressive.

Increasing spatial frequency of S-cone defined gratings reduces their visibility and brain response more than for gratings defined by L-M cone contrast

Chromatic sensitivity reduces as spatial frequency increases. Here, we explore the behavioural and neuronal responses to chromatic stimuli at two spatial frequencies for which the difference in sensitivity will be greater for S-cone than L-M stimuli. Luminance artefacts were removed using the Random Luminance Modulation (RLM) technique. As expected, doubling the spatial frequency increased the detection threshold more for S-cone than for isoluminant L-M gratings. We then used fMRI to measure the cortical BOLD responses to the same two chromatic stimuli (S and L-M) at the same two spatial frequencies. Responses were measured in six visual areas (V1, V2, V3, V3a, hV4, TO1/2). We found a significant interaction between spatial frequency in V1, V2 and V4 suggesting that the behaviourally observed increase in contrast threshold for high spatial frequency S-cone stimuli is reflected in these retinotopic areas. Our measurements show that neural responses consistent with psychophysical behaviour in a colour detection task can be observed as early as primary visual cortex.

Color-biased regions in the ventral visual pathway are food selective

Color-biased regions have been found between face- and place-selective areas in the ventral visual pathway. To investigate the function of the color-biased regions in a pathway responsible for object recognition, we analyzed the natural scenes dataset (NSD), a large 7T fMRI dataset from 8 participants who each viewed up to 30,000 trials of images of colored natural scenes over more than 30 scanning sessions. In a whole-brain analysis, we correlated the average color saturation of the images with voxel responses, revealing color-biased regions that diverge into two streams, beginning in V4 and extending medially and laterally relative to the fusiform face area in both hemispheres. We drew regions of interest (ROIs) for the two streams and found that the images for each ROI that evoked the largest responses had certain characteristics: they contained food, circular objects, warmer hues, and had higher color saturation. Further analyses showed that food images were the strongest predictor of activity in these regions, implying the existence of medial and lateral ventral food streams (VFSs). We found that color also contributed independently to voxel responses, suggesting that the medial and lateral VFSs use both color and form to represent food. Our findings illustrate how high-resolution datasets such as the NSD can be used to disentangle the multifaceted contributions of many visual features to the neural representations of natural scenes.

Magnetoencephalography contrast adaptation reflects perceptual adaptation

Contrast adaptation is a fundamental visual process that has been extensively investigated and used to infer the selectivity of visual cortex. We recently reported an apparent disconnect between the effects of contrast adaptation on perception and functional magnetic resonance imaging BOLD response adaptation, in which adaptation between chromatic and achromatic stimuli measured psychophysically showed greater selectivity than adaptation measured using BOLD signals. Here we used magnetoencephalography (MEG) recordings of neural responses to the same chromatic and achromatic adaptation conditions to characterize the neural effects of contrast adaptation and to determine whether BOLD adaptation or MEG better reflect the measured perceptual effects. Participants viewed achromatic, L-M isolating, or S-cone isolating radial sinusoids before adaptation and after adaptation to each of the three contrast directions. We measured adaptation-related changes in the neural response to a range of stimulus contrast amplitudes using two measures of the MEG response: the overall response amplitude, and a novel time-resolved measure of the contrast response function, derived from a classification analysis combined with multidimensional scaling. Within-stimulus adaptation effects on the contrast response functions in each case showed a pattern of contrast-gain or a combination of contrast-gain and response-gain effects. Cross-stimulus adaptation conditions showed that adaptation effects were highly stimulus selective across early, ventral, and dorsal visual cortical areas, consistent with the perceptual effects.

Spatial and Feature-selective Attention Have Distinct, Interacting Effects on Population-level Tuning

Attention can be deployed in different ways: When searching for a taxi in New York City, we can decide where to attend (e.g., to the street) and what to attend to (e.g., yellow cars). Although we use the same word to describe both processes, nonhuman primate data suggest that these produce distinct effects on neural tuning. This has been challenging to assess in humans, but here we used an opportunity afforded by multivariate decoding of MEG data. We found that attending to an object at a particular location and attending to a particular object feature produced effects that interacted multiplicatively. The two types of attention induced distinct patterns of enhancement in occipital cortex, with feature-selective attention producing relatively more enhancement of small feature differences and spatial attention producing relatively larger effects for larger feature differences. An information flow analysis further showed that stimulus representations in occipital cortex were Granger-caused by coding in frontal cortices earlier in time and that the timing of this feedback matched the onset of attention effects. The data suggest that spatial and feature-selective attention rely on distinct neural mechanisms that arise from frontal-occipital information exchange, interacting multiplicatively to selectively enhance task-relevant information.

Structural Differences Across Multiple Visual Cortical Regions in the Absence of Cone Function in Congenital Achromatopsia

Most individuals with congenital achromatopsia (ACHM) carry mutations that affect the retinal phototransduction pathway of cone photoreceptors, fundamental to both high acuity vision and colour perception. As the central fovea is occupied solely by cones, achromats have an absence of retinal input to the visual cortex and a small central area of blindness. Additionally, those with complete ACHM have no colour perception, and colour processing regions of the ventral cortex also lack typical chromatic signals from the cones. This study examined the cortical morphology (grey matter volume, cortical thickness, and cortical surface area) of multiple visual cortical regions in ACHM (n = 15) compared to normally sighted controls (n = 42) to determine the cortical changes that are associated with the retinal characteristics of ACHM. Surface-based morphometry was applied to T1-weighted MRI in atlas-defined early, ventral and dorsal visual regions of interest. Reduced grey matter volume in V1, V2, V3, and V4 was found in ACHM compared to controls, driven by a reduction in cortical surface area as there was no significant reduction in cortical thickness. Cortical surface area (but not thickness) was reduced in a wide range of areas (V1, V2, V3, TO1, V4, and LO1). Reduction in early visual areas with large foveal representations (V1, V2, and V3) suggests that the lack of foveal input to the visual cortex was a major driving factor in morphological changes in ACHM. However, the significant reduction in ventral area V4 coupled with the lack of difference in dorsal areas V3a and V3b suggest that deprivation of chromatic signals to visual cortex in ACHM may also contribute to changes in cortical morphology. This research shows that the congenital lack of cone input to the visual cortex can lead to widespread structural changes across multiple visual areas.

From mathematics to medicine: A practical primer on topological data analysis (TDA) and the development of related analytic tools for the functional discovery of latent structure in fMRI data

fMRI is the preeminent method for collecting signals from the human brain in vivo, for using these signals in the service of functional discovery, and relating these discoveries to anatomical structure. Numerous computational and mathematical techniques have been deployed to extract information from the fMRI signal. Yet, the application of Topological Data Analyses (TDA) remain limited to certain sub-areas such as connectomics (that is, with summarized versions of fMRI data). While connectomics is a natural and important area of application of TDA, applications of TDA in the service of extracting structure from the (non-summarized) fMRI data itself are heretofore nonexistent. “Structure” within fMRI data is determined by dynamic fluctuations in spatially distributed signals over time, and TDA is well positioned to help researchers better characterize mass dynamics of the signal by rigorously capturing shape within it. To accurately motivate this idea, we a) survey an established method in TDA (“persistent homology”) to reveal and describe how complex structures can be extracted from data sets generally, and b) describe how persistent homology can be applied specifically to fMRI data. We provide explanations for some of the mathematical underpinnings of TDA (with expository figures), building ideas in the following sequence: a) fMRI researchers can and should use TDA to extract structure from their data; b) this extraction serves an important role in the endeavor of functional discovery, and c) TDA approaches can complement other established approaches toward fMRI analyses (for which we provide examples). We also provide detailed applications of TDA to fMRI data collected using established paradigms, and offer our software pipeline for readers interested in emulating our methods. This working overview is both an inter-disciplinary synthesis of ideas (to draw researchers in TDA and fMRI toward each other) and a detailed description of methods that can motivate collaborative research.

A multiplicity of color-responsive cortical mechanisms revealed by the dynamics of cVEPs

Our results connect higher-order color mechanisms deduced from psychophysics with the known diversity of populations of double-opponent, color-responsive cells in V1. We used the chromatic visual evoked potential, the cVEP, to study responses in human visual cortex to equiluminant color patterns. Stimuli were modulated along three directions in color space: the cardinal directions, L-M and S, and along the line in color space from the white point to the color of the Red LED in the display screen (the Red direction). The Red direction is roughly intermediate between L-M and S in DKL space in cone-contrast coordinates. While cVEP response amplitude, latency, and width--and their dependences on cone contrast-- were similar in the L-M and Red directions, the Transientness of the Red response was significantly greater than for responses to stimuli in the L-M direction and in the S direction. This difference in response dynamics supports the concept that there are multiple, distinct neuronal populations, so-called higher- order color mechanisms, for color perception within human V1 cortex.

A quadratic model captures the human V1 response to variations in chromatic direction and contrast

An important goal for vision science is to develop quantitative models of the representation of visual signals at post-receptoral sites. To this end, we develop the quadratic color model (QCM) and examine its ability to account for the BOLD fMRI response in human V1 to spatially-uniform, temporal chromatic modulations that systematically vary in chromatic direction and contrast. We find that the QCM explains the same, cross-validated variance as a conventional general linear model, with far fewer free parameters. The QCM generalizes to allow prediction of V1 responses to a large range of modulations. We replicate the results for each subject and find good agreement across both replications and subjects. We find that within the LM cone contrast plane, V1 is most sensitive to L-M contrast modulations and least sensitive to L+M contrast modulations. Within V1, we observe little to no change in chromatic sensitivity as a function of eccentricity.

Suppression of Luminance Contrast Sensitivity by Weak Color Presentation

The results of psychophysical studies suggest that color in a visual scene affects luminance contrast perception. In our brain imaging studies we have found evidence of an effect of chromatic information on luminance information. The dependency of saturation on brain activity in the visual cortices was measured by functional magnetic resonance imaging (fMRI) while the subjects were observing visual stimuli consisting of colored patches of various hues manipulated in saturation (Chroma value in the Munsell color system) on an achromatic background. The results indicate that the patches suppressed luminance driven brain activity. Furthermore, the suppression was stronger rather than weaker for patches with lower saturation colors, although suppression was absent when gray patches were presented instead of colored patches. We also measured brain activity while the subjects observed only the patches (on a uniformly black background) and confirmed that the colored patches alone did not give rise to differences in brain activity for different Chroma values. The chromatic information affects the luminance information in V1, since the effect was observed in early visual cortices (V2 and V3) and the ventral pathway (hV4), as well as in the dorsal pathway (V3A/B). In addition, we conducted a psychophysical experiment in which the ability to discriminate luminance contrast on a grating was measured. Discrimination was worse when weak (less saturated) colored patches were attached to the grating than when strong (saturated) colored patches or achromatic patches were attached. The results of both the fMRI and psychophysical experiments were consistent in that the effects of color were greater in the conditions with low saturation colors.

Attention selectively enhances stimulus information for surround over foveal stimulus representations in occipital cortex

By attending to part of a visual scene, we can prioritize processing of the most relevant visual information and so use our limited resources effectively. Previous functional magnetic resonance imaging (fMRI) work has shown that attention can increase overall blood-oxygen-level-dependent (BOLD) signal responsiveness but also enhances the stimulus information in terms of classifier performance. Here, we investigate how these effects vary across the visual field. We compare attention-enhanced fMRI-BOLD amplitude responses and classifier accuracy in fovea and surrounding stimulus regions using a set of four simple stimuli subdivided into a foveal region (1.4° diameter) and a surround region (15° diameter). We found dissociations between the effects of attention on average response and in enhancing stimulus information. In early visual cortex, we found that attention increased the amplitude of responses to both foveal and surround parts of the stimuli and increased classifier performance only for the surround stimulus. Conversely, ventral visual areas showed less change in average response but greater changes in decoding. Unlike for early visual cortex, in the ventral visual cortex attention produced similar changes in decoding for center and surround stimuli.

Color for object recognition: Hue and chroma sensitivity in the deep features of convolutional neural networks

In this work, we examined the color tuning of units in the hidden layers of AlexNet, VGG-16 and VGG-19 convolutional neural networks and their relevance for the successful recognition of an object. We first selected the patches for which the units are maximally responsive among the 1.2 M images of the ImageNet training dataset. We segmented these patches using a k-means clustering algorithm on their chromatic distribution. Then we independently varied the color of these segments, both in hue and chroma, to measure the unit's chromatic tuning. The models exhibited properties at times similar or opposed to the known chromatic processing of biological system. We found that, similarly to the most anterior occipital visual areas in primates, the last convolutional layer exhibited high color sensitivity. We also found the gradual emergence of single to double opponent kernels. Contrary to cells in the visual system, however, these kernels were selective for hues that gradually transit from being broadly distributed in early layers, to mainly falling along the blue-orange axis in late layers. In addition, we found that the classification performance of our models varies as we change the color of our stimuli following the models' kernels properties. Performance was highest for colors the kernels maximally responded to, and images responsible for the activation of color sensitive kernels were more likely to be mis-classified as we changed their color. These observations were shared by all three networks, thus suggesting that they are general properties of current convolutional neural networks trained for object recognition.

The effects of visual discomfort and chromaticity separation on neural processing during a visual task

Visual stimuli that are uncomfortable to look at evoke a large neural response suggesting altered processing. While there is some evidence linking uncomfortable achromatic stimuli to impaired visual processing, the effect of uncomfortable chromatic patterns on visual cognition has yet to be explored. Large differences in chromaticity separation (e.g. red and blue) elicit visual discomfort, larger metabolic responses, larger visual evoked potentials, and greater alpha suppression compared to small chromaticity separations (e.g. pink and purple). We investigated the impact of stimuli that varied in their chromaticity separation (calculated in perceptual color space) on a visual task and their effect on neural responses across the cortex. Thirty participants completed a continuous pairs task (letters changed at 3 Hz) while grating patterns that differed in their chromaticity separation alternated with a grey screen at 5 Hz. The different temporal frequencies allowed for steady-state visual evoked potentials (SSVEPs) to the two stimulus-types to be measured simultaneously using electroencephalography (EEG). A subset of participants rated the gratings on a 9-point scale of discomfort. We observed greater ratings of discomfort and increased power at 5 Hz with the larger chromaticity separations. The increase in 5 Hz power with greater chromaticity separation was evident across the cortex. However, there was no significant effect of chromaticity separation on power at 3 Hz, or on reaction times, and no consistent effect on behavioral accuracy. Despite eliciting heightened neural responses across the cortex, short term exposure to uncomfortable chromatic stimuli does not adversely impact visual task performance.

Color Compensation in Anomalous Trichromats Assessed with fMRI

Anomalous trichromacy is a common form of congenital color deficiency resulting from a genetic alteration in the photopigments of the eye's light receptors. The changes reduce sensitivity to reddish and greenish hues, yet previous work suggests that these observers may experience the world to be more colorful than their altered receptor sensitivities would predict, potentially indicating an amplification of post-receptoral signals. However, past evidence suggesting such a gain adjustment rests on subjective measures of color appearance or salience. We directly tested for neural amplification by using fMRI to measure cortical responses in color-anomalous and normal control observers. Color contrast response functions were measured in two experiments with different tasks to control for attentional factors. Both experiments showed a predictable reduction in chromatic responses for anomalous trichromats in primary visual cortex. However, in later areas V2v and V3v, chromatic responses in the two groups were indistinguishable. Our results provide direct evidence for neural plasticity that compensates for the deficiency in the initial receptor color signals and suggest that the site of this compensation is in early visual cortex.

Steady-State Visual Evoked Potentials Elicited from Early Visual Cortex Reflect Both Perceptual Color Space and Cone-Opponent Mechanisms

Colors are represented in the cone-opponent signals, L-M versus S cones, at least up to the level of inputs to the primary visual cortex. We explored the hue selective responses in early cortical visual areas through recordings of steady-state visual evoked potentials (SSVEPs), elicited by a flickering checkerboard whose color smoothly swept around the hue circle defined in a cone-opponent color space. If cone opponency dominates hue representation in the source of SSVEP signals, SSVEP amplitudes as a function of hue should form a profile that is line-symmetric along the cardinal axes of the cone-opponent color space. Observed SSVEP responses were clearly chromatic ones with increased SSVEP amplitudes and reduced response latencies for higher contrast conditions. The overall elliptic amplitude profile was significantly tilted away from the cardinal axes to have the highest amplitudes in the "lime-magenta" direction, indicating that the hue representation in question is not dominated by cone-opponency. The observed SSVEP amplitude hue profile was better described as a summation of a perceptual response and cone-opponent responses with a larger weight to the former. These results indicate that hue representations in the early visual cortex, measured by the SSVEP technique, are possibly related to perceptual color contrast.

Neural correlates of perceptual color inferences as revealed by #thedress

Color constancy involves disambiguating the spectral characteristics of lights and surfaces, for example to distinguish red in white light from white in red light. Solving this problem appears especially challenging for bluish tints, which may be attributed more often to shading, and this bias may underlie the individual differences in whether people described the widely publicized image of #thedress as blue-black or white-gold. To probe these higher-level color inferences, we examined neural correlates of the blue-bias, using frequency-tagging and high-density electroencephalography to monitor responses to 3-Hz alternations between different color versions of #thedress. Specifically, we compared relative neural responses to the original “blue” dress image alternated with the complementary “yellow” image (formed by inverting the chromatic contrast of each pixel). This image pair produced a large modulation of the electroencephalography amplitude at the alternation frequency, consistent with a perceived contrast difference between the blue and yellow images. Furthermore, decoding topographical differences in the blue-yellow asymmetries over occipitoparietal channels predicted blue-black and white-gold observers with over 80% accuracy. The blue-yellow asymmetry was stronger than for a “red” versus “green” pair matched for the same component differences in L versus M or S versus LM chromatic contrast as the blue-yellow pair and thus cannot be accounted for by asymmetries within either precortical cardinal mechanism. Instead, the results may point to neural correlates of a higher-level perceptual representation of surface colors.

fMRI representational similarity analysis reveals graded preferences for chromatic and achromatic stimulus contrast across human visual cortex

Human visual cortex is partitioned into different functional areas that, from lower to higher, become increasingly selective and responsive to complex feature dimensions. Here we use a Representational Similarity Analysis (RSA) of fMRI-BOLD signals to make quantitative comparisons across LGN and multiple visual areas of the low-level stimulus information encoded in the patterns of voxel responses. Our stimulus set was picked to target the four functionally distinct subcortical channels that input visual cortex from the LGN: two achromatic sinewave stimuli that favor the responses of the high-temporal magnocellular and high-spatial parvocellular pathways, respectively, and two chromatic stimuli isolating the L/M-cone opponent and S-cone opponent pathways, respectively. Each stimulus type had three spatial extents to sample both foveal and para-central visual field. With the RSA, we compare quantitatively the response specializations for individual stimuli and combinations of stimuli in each area and how these change across visual cortex. First, our results replicate the known response preferences for motion/flicker in the dorsal visual areas. In addition, we identify two distinct gradients along the ventral visual stream. In the early visual areas (V1-V3), the strongest differential representation is for the achromatic high spatial frequency stimuli, suitable for form vision, and a very weak differentiation of chromatic versus achromatic contrast. Emerging in ventral occipital areas (V4, VO1 and VO2), however, is an increasingly strong separation of the responses to chromatic versus achromatic contrast and a decline in the high spatial frequency representation. These gradients provide new insight into how visual information is transformed across the visual cortex.

What Cognitive Neurology Teaches Us about Our Experience of Color

Color provides valuable information about the environment, yet the exact mechanisms explaining how colors appear to us remain poorly understood. Retinal signals are processed in the visual cortex through high-level mechanisms that link color perception with top-down expectations and knowledge. Here, we review the neuroimaging evidence about color processing in the brain, and how it is affected by acquired brain lesions in humans. Evidence from patients with brain-damage suggests that high-level color processing may be divided into at least three modules: perceptual color experience, color naming, and color knowledge. These modules appear to be functionally independent but richly interconnected, and serve as cortical relays linking sensory and semantic information, with the final goal of directing object-related behavior. We argue that the relations between colors and their objects are key mechanisms to understand high-level color processing.

Color contrast adaptation: fMRI fails to predict behavioral adaptation

fMRI-adaptation is a valuable tool for inferring the selectivity of neural responses. Here we use it in human color vision to test the selectivity of responses to S-cone opponent (blue-yellow), L/M-cone opponent (red-green), and achromatic (Ach) contrast across nine regions of interest in visual cortex. We measure psychophysical adaptation, using comparable stimuli to the fMRI-adaptation, and find significant selective adaptation for all three stimulus types, implying separable visual responses to each. For fMRI-adaptation, we find robust adaptation but, surprisingly, much less selectivity due to high levels of cross-stimulus adaptation in all conditions. For all BY and Ach test/adaptor pairs, selectivity is absent across all ROIs. For RG/Ach stimulus pairs, this paradigm has previously shown selectivity for RG in ventral areas and for Ach in dorsal areas. For chromatic stimulus pairs (RG/BY), we find a trend for selectivity in ventral areas. In conclusion, we find an overall lack of correspondence between BOLD and behavioral adaptation suggesting they reflect different aspects of the underlying neural processes. For example, raised cross-stimulus adaptation in fMRI may reflect adaptation of the broadly-tuned normalization pool. Finally, we also identify a longer-timescale adaptation (1h) in both BOLD and behavioral data. This is greater for chromatic than achromatic contrast. The longer-timescale BOLD effect was more evident in the higher ventral areas than in V1, consistent with increasing windows of temporal integration for higher-order areas.

Reevaluating hMT+ and hV4 functional specialization for motion and static contrast using fMRI-guided repetitive transcranial magnetic stimulation

Although visual areas hMT+ and hV4 are considered to have segregated functions for the processing of motion and form within dorsal and ventral streams, respectively, more recent evidence favors some functional overlap. Here we use fMRI-guided online repetitive transcranial magnetic stimulation (rTMS) to test two associated hypotheses: that area hV4 is causally involved in the perception of motion and hMT+ in the perception of static form. We use variations of a common global stimulus to test two dynamic motion-based tasks and two static form-based tasks in ipsilateral and contralateral visual fields. We find that rTMS to both hMT+ and hV4 significantly impairs direction discrimination and causes a perceptual slowing of motion, implicating hV4 in motion perception. Stimulation of hMT+ impairs motion in both visual fields, implying that disruption to one hMT+ disrupts the other with both needed for optimal performance. For the second hypothesis, we find the novel result that hV4 stimulation markedly reduces perceived contrast of a static stimulus. hMT+ stimulation also produces an effect, implicating it in static contrast perception. Our findings are the first to show that rTMS of hV4 can produce a large perceptual effect and, taken together, suggest a less rigid functional segregation between hMT+ and hV4 than previously thought.

Columnar organization of mid-spectral and end-spectral hue preferences in human visual cortex

Multiple color-selective areas have been described in visual cortex, in both humans and non-human primates. In macaques, hue-selective columns have been reported in several areas. In V2, it has been proposed that such hue-selective columns are mapped so as to mirror the order of wavelength through the visible spectrum, within thin-type stripes. Other studies have suggested a neural segregation of mid-spectral vs. end-spectral hue preferences (e.g. red and blue vs. green and yellow), within thin- and thick-type stripes, respectively. This latter segregation could reduce the spatial 'blur' due to chromatic aberration in the encoding of fine spatial details in the thick-type stripes. To distinguish between these and related models, we tested the organization of hue preferences in human visual cortex using fMRI at high spatial resolution. We used a high field (7T) scanner in humans (n = 7), measuring responses to four independent hues, including end-spectral (i.e. red-gray and blue-gray) and mid-spectral (i.e. green-gray and yellow-gray) isoluminant gratings, and also relative to achromatic luminance-varying (control) stimuli. In each subject, thin- and thick-type columns in V2 and V3 were localized using an independent set of stimuli and scans. We found distinct hue-selective differences along the dimension of mid-vs. end-spectral hues, in striate and early extrastriate visual cortex. First, as reported previously in macaques, V1 responded more strongly to end-spectral hues, compared to mid-spectral hues. Second, the color-selective thin-type stripes in V2 and V3 showed a greater response to end- and mid-spectral hues, relative to luminance-varying gratings. Third, thick-type stripes in V2/V3 showed a significantly stronger response to mid-spectral (compared to end-spectral) hues. Fourth, in the higher-tier color-selective area in occipital temporal cortex (n = 4), responses to all four hues were statistically equivalent to each other. These results suggest that early visual cortex segregates the processing of mid-vs. end-spectral hues, perhaps to counter the challenging optical constraint of chromatic aberration.

Processing of chromatic information in a deep convolutional neural network

Deep convolutional neural networks are a class of machine-learning algorithms capable of solving non-trivial tasks, such as object recognition, with human-like performance. Little is known about the exact computations that deep neural networks learn, and to what extent these computations are similar to the ones performed by the primate brain. Here, we investigate how color information is processed in the different layers of the AlexNet deep neural network, originally trained on object classification of over 1.2M images of objects in their natural contexts. We found that the color-responsive units in the first layer of AlexNet learned linear features and were broadly tuned to two directions in color space, analogously to what is known of color responsive cells in the primate thalamus. Moreover, these directions are decorrelated and lead to statistically efficient representations, similar to the cardinal directions of the second-stage color mechanisms in primates. We also found, in analogy to the early stages of the primate visual system, that chromatic and achromatic information were segregated in the early layers of the network. Units in the higher layers of AlexNet exhibit on average a lower responsivity for color than units at earlier stages.

Source localization of S-cone and L/M-cone driven signals using silent substitution flash stimulation

This study aimed to analyze the neuronal sources of the visual evoked potentials after flash stimulation of the S- and the L/M-cone driven channels of the visual system. For 11 volunteers a 64-channel electroencephalography (EEG) was recorded during selective excitation of both color opponent channels. Individual and grand average data were analyzed topographically. Source localization was carried out using a realistically shaped three compartment boundary element model (BEM) and a mirrored moving dipole model. We found two main components (N1, P1) in all subjects, as well as a third late component in most subjects. For these components significant latency differences (N1=33 ms, P1=22 ms; p<0.05) between both color opponent channels were found. The results showed no differences in the topography and no differences in dipole localization between both color channels. Talairach coordinates of grand averages indicated activation in area 18. Comparison of results of separately stimulated eyes revealed no differences. Our findings showed that neural processing occurs in the same areas of the visual cortex for stimuli with different spectral properties. The signals of S- and L/M-cone driven channels are transmitted in distinct pathways to the cortex. Thus, the observed latency differences might be caused by different anatomical and functional properties of these pathways.

Human Visual Cortex Responses to Rapid Cone and Melanopsin-Directed Flicker

Signals from cones are recombined in postreceptoral channels [luminance, L + M; red-green, L - M; blue-yellow, S - (L + M)]. The melanopsin-containing retinal ganglion cells are also active at daytime light levels and recent psychophysical results suggest that melanopsin contributes to conscious vision in humans. Here, we measured BOLD fMRI responses to spectral modulations that separately targeted the postreceptoral cone channels and melanopsin. Responses to spatially uniform (27.5° field size, central 5° obscured) flicker at 0.5, 1, 2, 4, 8, 16, 32, and 64 Hz were recorded from areas V1, V2/V3, motion-sensitive area MT, and the lateral occipital complex. In V1 and V2/V3, higher temporal sensitivity was observed to L + M + S (16 Hz) compared with L - M flicker (8 Hz), consistent with psychophysical findings. Area MT was most sensitive to rapid (32 Hz) flicker of either L + M + S or L - M. We found S cone responses only in areas V1 and V2/V3 (peak frequency: 4-8 Hz). In addition, we studied an L + M modulation and found responses that were effectively identical at all temporal frequencies to those recorded for the L + M + S modulation. Finally, we measured the cortical response to melanopsin-directed flicker and compared this response with control modulations that addressed stimulus imprecision and the possibility of stimulation of cones in the shadow of retinal blood vessels (penumbral cones). For our stimulus conditions, melanopsin flicker did not elicit a cortical response exceeding that of the control modulations. We note that failure to control for penumbral cone stimulation could be mistaken for a melanopsin response.

SIGNIFICANCE STATEMENT

The retina contains cone photoreceptors and ganglion cells that contain the photopigment melanopsin. Cones provide brightness and color signals to visual cortex. Melanopsin influences circadian rhythm and the pupil, but its contribution to cortex and perception is less clear. We measured the response of human visual cortex with fMRI using spectral modulations tailored to stimulate the cones and melanopsin separately. We found that cortical responses to cone signals vary systematically across visual areas. Differences in temporal sensitivity for achromatic, red-green, and blue-yellow stimuli generally reflect the known perceptual properties of vision. We found that melanopsin signals do not produce a measurable response in visual cortex at temporal frequencies between 0.5 and 64 Hz at daytime light levels.

Differences in Brain Hemodynamics in Response to Achromatic and Chromatic Cards of the Rorschach: A fMRI Study

In order to investigate the effects of color stimuli of the Rorschach inkblot method (RIM), the cerebral activity of 40 participants with no history of neurological or psychiatric illness was scanned while they engaged in the Rorschach task. A scanned image of the ten RIM inkblots was projected onto a screen in the MRI scanner. Cerebral activation in response to five achromatic color cards and five chromatic cards were compared. As a result, a significant increase in brain activity was observed in bilateral visual areas V2 and V3, parietooccipital junctions, pulvinars, right superior temporal gyrus, and left premotor cortex for achromatic color cards (p < .001). For the cards with chromatic color, significant increase in brain activity was observed in left visual area V4 and left orbitofrontal cortex (p < .001). Furthermore, a conjoint analysis revealed various regions were activated in responding to the RIM. The neuropsychological underpinnings of the response process, as described by Acklin and Wu-Holt (1996), were largely confirmed.

Color responses and their adaptation in human superior colliculus and lateral geniculate nucleus

We use an fMRI adaptation paradigm to explore the selectivity of human responses in the lateral geniculate nucleus (LGN) and superior colliculus (SC) to red-green color and achromatic contrast. We measured responses to red-green (RG) and achromatic (ACH) high contrast sinewave counter-phasing rings with and without adaptation, within a block design. The signal for the RG test stimulus was reduced following both RG and ACH adaptation, whereas the signal for the ACH test was unaffected by either adaptor. These results provide compelling evidence that the human LGN and SC have significant capacity for color adaptation. Since in the LGN red-green responses are mediated by P cells, these findings are in contrast to earlier neurophysiological data from non-human primates that have shown weak or no contrast adaptation in the P pathway. Cross-adaptation of the red-green color response by achromatic contrast suggests unselective response adaptation and points to a dual role for P cells in responding to both color and achromatic contrast. We further show that subcortical adaptation is not restricted to the geniculostriate system, but is also present in the superior colliculus (SC), an oculomotor region that until recently, has been thought to be color-blind. Our data show that the human SC not only responds to red-green color contrast, but like the LGN, shows reliable but unselective adaptation.

Dependence of chromatic responses in V1 on visual field eccentricity and spatial frequency: an fMRI study

Psychophysical sensitivity to red-green chromatic modulation decreases with visual eccentricity, compared to sensitivity to luminance modulation, even after appropriate stimulus scaling. This is likely to occur at a central, rather than a retinal, site. Blood-oxygenation-level-dependent (BOLD) functional magnetic resonance imaging (fMRI) responses to stimuli designed to separately stimulate different afferent channels' [red-green, luminance, and short-wavelength (S)-cone] circular gratings were recorded as a function of visual eccentricity (±10  deg) and spatial frequency (SF) in human primary visual cortex (V1) and further visual areas (V2v, V3v). In V1, the SF tuning of BOLD fMRI responses became coarser with eccentricity. For red-green and luminance gratings, similar SF tuning curves were found at all eccentricities. The pattern for S-cone modulation differed, with SF tuning changing more slowly with eccentricity than for the other two modalities. This may be due to the different retinal distribution with eccentricity of this receptor type. A similar pattern held in V2v and V3v. This would suggest that transformation or spatial filtering of the chromatic (red-green) signal occurs beyond these areas.

The selectivity of responses to red-green colour and achromatic contrast in the human visual cortex: an fMRI adaptation study

There is controversy as to how responses to colour in the human brain are organized within the visual pathways. A key issue is whether there are modular pathways that respond selectively to colour or whether there are common neural substrates for both colour and achromatic (Ach) contrast. We used functional magnetic resonance imaging (fMRI) adaptation to investigate the responses of early and extrastriate visual areas to colour and Ach contrast. High-contrast red-green (RG) and Ach sinewave rings (0.5 cycles/degree, 2 Hz) were used as both adapting stimuli and test stimuli in a block design. We found robust adaptation to RG or Ach contrast in all visual areas. Cross-adaptation between RG and Ach contrast occurred in all areas indicating the presence of integrated, colour and Ach responses. Notably, we revealed contrasting trends for the two test stimuli. For the RG test, unselective processing (robust adaptation to both RG and Ach contrast) was most evident in the early visual areas (V1 and V2), but selective responses, revealed as greater adaptation between the same stimuli than cross-adaptation between different stimuli, emerged in the ventral cortex, in V4 and VO in particular. For the Ach test, unselective responses were again most evident in early visual areas but Ach selectivity emerged in the dorsal cortex (V3a and hMT+). Our findings support a strong presence of integrated mechanisms for colour and Ach contrast across the visual hierarchy, with a progression towards selective processing in extrastriate visual areas.

Hue Selectivity in Human Visual Cortex Revealed by Functional Magnetic Resonance Imaging

The variability of color-selective neurons in human visual cortex is considered more diverse than cone-opponent mechanisms. We addressed this issue by deriving histograms of hue-selective voxels measured using fMRI with a novel stimulation paradigm, where the stimulus hue changed continuously. Despite the large between-subject difference in hue-selective histograms, individual voxels exhibited selectivity for intermediate hues, such as purple, cyan, and orange, in addition to those along cone-opponent axes. In order to rule the possibility out that the selectivity for intermediate hues emerged through spatial summation of activities of neurons selectively responding to cone-opponent signals, we further tested hue-selective adaptations in intermediate directions of cone-opponent axes, by measuring responses to 4 diagonal hues during concurrent adaptation to 1 of the 4 hues. The selective and unidirectional reduction in response to the adapted hue lends supports to our argument that cortical neurons respond selectively to intermediate hues.

Brightness-color interactions in human early visual cortex

The interaction between brightness and color causes there to be different color appearance when one and the same object is viewed against surroundings of different brightness. Brightness contrast causes color to be desaturated, as has been found in perceptual experiments on color induction and color-gamut expansion in human vision. However, it is not clear yet where in the cerebral cortex the brightness-color interaction that causes these major perceptual effects is located. One hypothesis is that brightness and color signals are processed separately and in parallel within the primary visual cortex V1 and only interact in extrastriate cortex. Another hypothesis is that color and brightness contrast interact strongly already within V1. We localized the brightness-color interaction in human V1 by means of recording the chromatic visual-evoked potential. The chromatic visual-evoked potential measurements decisively support the idea that brightness-color interaction arises in a recurrent inhibitory network in V1. Furthermore, our results show that the inhibitory signal for brightness-color interaction is generated by local brightness contrast at the boundary between target and surround, instead of by the luminance difference between the interior of the color target and its large background.

Color coding in the primate visual pathway: a historical view

The physiology and anatomy of the primate visual pathway are reviewed from a historical perspective, especially in relation to color vision. From the work of the last decades, certain issues have been selected which remain unresolved and still pose a challenge for neurobiologists and psychophysicists. It is suggested that the structure of the primate visual pathway has been colored by the evolution of trichromacy and that many features of the parvocellular pathway represent adaptations to this end.

Abnormal late visual responses and alpha oscillations in neurofibromatosis type 1: a link to visual and attention deficits

Background

Neurofibromatosis type 1 (NF1) affects several areas of cognitive function including visual processing and attention. We investigated the neural mechanisms underlying the visual deficits of children and adolescents with NF1 by studying visual evoked potentials (VEPs) and brain oscillations during visual stimulation and rest periods.

Methods

Electroencephalogram/event-related potential (EEG/ERP) responses were measured during visual processing (NF1 n = 17; controls n = 19) and idle periods with eyes closed and eyes open (NF1 n = 12; controls n = 14). Visual stimulation was chosen to bias activation of the three detection mechanisms: achromatic, red-green and blue-yellow.

Results

We found significant differences between the groups for late chromatic VEPs and a specific enhancement in the amplitude of the parieto-occipital alpha amplitude both during visual stimulation and idle periods. Alpha modulation and the negative influence of alpha oscillations in visual performance were found in both groups.

Conclusions

Our findings suggest abnormal later stages of visual processing and enhanced amplitude of alpha oscillations supporting the existence of deficits in basic sensory processing in NF1. Given the link between alpha oscillations, visual perception and attention, these results indicate a neural mechanism that might underlie the visual sensitivity deficits and increased lapses of attention observed in individuals with NF1.

The role of the foreshortening cue in the perception of 3D object slant

Slant is the degree to which a surface recedes or slopes away from the observer about the horizontal axis. The perception of surface slant may be derived from static monocular cues, including linear perspective and foreshortening, applied to single shapes or to multi-element textures. It is still unclear the extent to which color vision can use these cues to determine slant in the absence of achromatic contrast. Although previous demonstrations have shown that some pictures and images may lose their depth when presented at isoluminance, this has not been tested systematically using stimuli within the spatio-temporal passband of color vision. Here we test whether the foreshortening cue from surface compression (change in the ratio of width to length) can induce slant perception for single shapes for both color and luminance vision. We use radial frequency patterns with narrowband spatio-temporal properties. In the first experiment, both a manual task (lever rotation) and a visual task (line rotation) are used as metrics to measure the perception of slant for achromatic, red-green isoluminant and S-cone isolating stimuli. In the second experiment, we measure slant discrimination thresholds as a function of depicted slant in a 2AFC paradigm and find similar thresholds for chromatic and achromatic stimuli. We conclude that both color and luminance vision can use the foreshortening of a single surface to perceive slant, with performances similar to those obtained using other strong cues for slant, such as texture. This has implications for the role of color in monocular 3D vision, and the cortical organization used in 3D object perception.

Processing of the S-cone signals in the early visual cortex of primates

The short-wavelength-sensitive (S) cones play an important role in color vision of primates, and may also contribute to the coding of other visual features, such as luminance and motion. The color signals carried by the S cones and other cone types are largely separated in the subcortical visual pathway. Studies on nonhuman primates or humans have suggested that these signals are combined in the striate cortex (V1) following a substantial amplification of the S-cone signals in the same area. In addition to reviewing these studies, this review describes the circuitry in V1 that may underlie the processing of the S-cone signals and the dynamics of this processing. It also relates the interaction between various cone signals in V1 to the results of some psychophysical and physiological studies on color perception, which leads to a discussion of a previous model, in which color perception is produced by a multistage processing of the cone signals. Finally, I discuss the processing of the S-cone signals in the extrastriate area V2.

BOLD human responses to chromatic spatial features

Animal physiological and human psychophysical studies suggest that an early step in visual processing involves the detection and identification of features such as lines and edges, by neural mechanisms with even- and odd-symmetric receptive fields. Functional imaging studies also demonstrate mechanisms with even- and odd-receptive fields in early visual areas, in response to luminance-modulated stimuli. In this study we measured fMRI BOLD responses to 2-D stimuli composed of only even or only odd symmetric features, and to an amplitude-matched random noise control, modulated in red-green equiluminant colour contrast. All these stimuli had identical power but different phase spectra, either highly congruent (even or odd symmetry stimuli) or random (noise). At equiluminance, V1 BOLD activity showed no preference between congruent- and random-phase stimuli, as well as no preference between even and odd symmetric stimuli. Areas higher in the visual hierarchy, both along the dorsal pathway (caudal part of the intraparietal sulcus, dorsal LO and V3A) and the ventral pathway (V4), responded preferentially to odd symmetry over even symmetry stimuli, and to congruent over random phase stimuli. Interestingly, V1 showed an equal increase in BOLD activity at each alternation between stimuli of different symmetry, suggesting the existence of specialised mechanisms for the detection of edges and lines such as even- and odd-chromatic receptive fields. Overall the results indicate a high selectivity of colour-selective neurons to spatial phase along both the dorsal and the ventral pathways in humans.

A multimodal perspective on the composition of cortical oscillations

An expanding corpus of research details the relationship between functional magnetic resonance imaging (fMRI) measures and neuronal network oscillations. Typically, integrated electroencephalography and fMRI, or parallel magnetoencephalography (MEG) and fMRI are used to draw inference about the consanguinity of BOLD and electrical measurements. However, there is a relative dearth of information about the relationship between E/MEG and the focal networks from which these signals emanate. Consequently, the genesis and composition of E/MEG oscillations requires further clarification. Here we aim to contribute to understanding through a series of parallel measurements of primary motor cortex (M1) oscillations, using human MEG and in vitro rodent local field potentials. We compare spontaneous activity in the ∼10 Hz mu and 15–30 Hz beta frequency ranges and compare MEG signals with independent and integrated layers III and V (LIII/LV) from in vitro recordings. We explore the mechanisms of oscillatory generation, using specific pharmacological modulation with the GABA-A alpha-1 subunit modulator zolpidem. Finally, to determine the contribution of cortico-cortical connectivity, we recorded in vitro M1, during an incision to sever lateral connections between M1 and S1 cortices. We demonstrate that frequency distribution of MEG signals appear have closer statistically similarity with signals from integrated rather than independent LIII/LV laminae. GABAergic modulation in both modalities elicited comparable changes in the power of the beta band. Finally, cortico-cortical connectivity in sensorimotor cortex (SMC) appears to directly influence the power of the mu rhythm in LIII. These findings suggest that the MEG signal is an amalgam of outputs from LIII and LV, that multiple frequencies can arise from the same cortical area and that in vitro and MEG M1 oscillations are driven by comparable mechanisms. Finally, cortico-cortical connectivity is reflected in the power of the SMC mu rhythm.

Tracking blue cone signals in the primate brain

In this paper, we review the path taken by signals originating from the short wavelength sensitive cones (S-cones) in Old World and New World primates. Two types of retinal ganglion cells (RGCs) carrying S-cone signals (blue-On and blue-Off cells) project to the dorsal lateral geniculate nucleus (dLGN) in the thalamus. In all primates, these S-cone signals are relayed through the 'dust-like' (konis in classical Greek) dLGN cells. In New World primates such as common marmoset, these very small cells are known to form distinct and spatially extensive, koniocellular layers. Although in Old World primates, such as macaques, koniocellular layers tend to be very thin, the adjacent parvocellular layers contain distinct koniocellular extensions. It appears that all S-cone signals are relayed through such konio cells, whether they are in the main koniocellular layers or in their colonies within the parvocellular layers of the dLGN. In the primary visual cortex, these signals begin to merge with the signals carried by the other two principal parallel channels, namely the magnocellular and parvocellular channels. This article will also review the possible routes taken by the S-cone signals to reach one of the topographically organised extrastriate visual cortical areas, the middle temporal area (area MT). This area is the major conduit for signals reaching the parietal cortex. Alternative visual inputs to area MT not relayed via the primary visual cortex area (V1) may provide the neurological basis for the phenomenon of 'blindsight' observed in human and non-human primates, who have partial or complete damage to the primary visual cortex. Short wavelength sensitive cone (S-cone) signals to area MT may also play a role in directing visual attention with possible implications for understanding the pathology in dyslexia and some of its treatment options.

Relationship between neural response and adaptation selectivity to form and color: an ERP study

Adaptation is widely used as a tool for studying selectivity to visual features. In these studies it is usually assumed that the loci of feature selective neural responses and adaptation coincide. We used an adaptation paradigm to investigate the relationship between response and adaptation selectivity in event-related potentials (ERPs). ERPs were evoked by the presentation of colored Glass patterns in a form discrimination task. Response selectivities to form and, to some extent, color of the patterns were reflected in the C1 and N1 ERP components. Adaptation selectivity to color was reflected in N1 and was followed by a late (300–500 ms after stimulus onset) effect of form adaptation. Thus for form, response and adaptation selectivity were manifested in non-overlapping intervals. These results indicate that adaptation and response selectivity can be associated with different processes. Therefore, inferring selectivity from an adaptation paradigm requires analysis of both adaptation and neural response data.

Synaesthetic perception of colour and visual space in a blind subject: an fMRI case study

In spatial sequence synaesthesia (SSS) ordinal stimuli are perceived as arranged in peripersonal space. Using fMRI, we examined the neural bases of SSS and colour synaesthesia for spoken words in a late-blind synaesthete, JF. He reported days of the week and months of the year as both coloured and spatially ordered in peripersonal space; parts of the days and festivities of the year were spatially ordered but uncoloured. Words that denote time-units and triggered no concurrents were used in a control condition. Both conditions inducing SSS activated the occipito-parietal, infero-frontal and insular cortex. The colour area hOC4v was engaged when the synaesthetic experience included colour. These results confirm the continued recruitment of visual colour cortex in this late-blind synaesthetes. Synaesthesia also involved activation in inferior frontal cortex, which may be related to spatial memory and detection, and in the insula, which might contribute to audiovisual integration related to the processing of inducers and concurrents.

Toward a unified theory of visual area V4

Visual area V4 is a midtier cortical area in the ventral visual pathway. It is crucial for visual object recognition and has been a focus of many studies on visual attention. However, there is no unifying view of V4's role in visual processing. Neither is there an understanding of how its role in feature processing interfaces with its role in visual attention. This review captures our current knowledge of V4, largely derived from electrophysiological and imaging studies in the macaque monkey. Based on recent discovery of functionally specific domains in V4, we propose that the unifying function of V4 circuitry is to enable selective extraction of specific functional domain-based networks, whether it be by bottom-up specification of object features or by top-down attentionally driven selection.

Which "neural activity" do you mean? fMRI, MEG, oscillations and neurotransmitters

Over the last 20 years, BOLD-FMRI has proved itself to be a powerful and versatile tool for the study of the neural substrate underpinning many of our cognitive and perceptual functions. However, exactly how it is coupled to the underlying neurophysiology, and how this coupling varies across the brain, across tasks and across individuals is still unclear. The story is further complicated by the fact that within the same cortical region, multiple evoked and induced oscillatory effects may be modulated during task execution, supporting different cognitive roles, and any or all of these may have metabolic demands that then drive the BOLD response. In this paper I shall concentrate on one experimental approach to shedding light on this problem i.e. the execution of the same experimental tasks using MEG and fMRI in order to reveal which electrophysiological responses best match the BOLD response spatially, temporally and functionally. The results demonstrate a rich and complex story that does not fit with a simplistic view of BOLD reflecting "neural activity" and suggests that we could consider the coupling between BOLD and the various parameters of neural function as an ill-posed inverse problem. Finally, I describe recent work linking individual variability in both cortical oscillations and the BOLD-fMRI response to variability in endogenous GABA concentration.