Poster Session: A direct measure of adaptation and visual salience

One hypothesized function of adaptation is to increase the salience of novel targets by discounting the properties of the ambient environment. Previous studies have suggested this by finding faster search times for novel targets when searching on backgrounds observers are currently adapted to. However, this provides only an indirect measure of salience. Here, we developed a more direct measure of the impact of adaptation on feature salience. Backgrounds were oriented 1/f noise images with power confined within 15 deg of horizontal or vertical. Targets were 5 c/deg Gabor patches centered on the 8 deg backgrounds. Observers simultaneously adapted to the horizontal or vertical backgrounds shown on the left or right of fixation. A 250ms test probe then showed the Gabor patch on the same background (horizontal or vertical) on both sides. The target orientation was adjusted on one side until it appeared as conspicuous as a fixed target on the other side. Settings were made for fixed targets ranging from 10 to 45 deg from the backgrounds. For most conditions/observers, the salience matches required a smaller orientation offset on the same- vs. different-adapt background. These results support a functional role of adaptation in highlighting novelty by potentially "unmasking" the target from its background, and emphasize the importance of considering adaptation aftereffects not only for isolated targets but within the stimulus contexts they are embedded in.

Automatic, Early Color-Specific Neural Responses to Object Color Knowledge

Some familiar objects are associated with specific colors, e.g., rubber ducks with yellow. Whether and at what stage neural responses occur to these color associations remain open questions. We recorded frequency-tagged electroencephalogram (EEG) responses to periodic presentations of yellow-associated objects, shown among sequences of non-periodic blue-, red-, and green-associated objects. Both color and grayscale versions of the objects elicited yellow-specific responses, indicating an automatic activation of color knowledge from object shape. Follow-up experiments replicated these effects with green-specific responses, and demonstrated modulated responses for incongruent color/object associations. Importantly, the onset of color-specific responses was as early to grayscale as actually colored stimuli (before 100 ms), the latter additionally eliciting a conventional later response (approximately 140-230 ms) to actual stimulus color. This suggests that the neural representation of familiar objects includes both diagnostic shape and color properties, such that shape can elicit associated color-specific responses before actual color-specific responses occur.

Poster Session: Perceptual scaling and natural image statistics

Visual coding is thought to be matched to many of the properties of the natural visual environment, such as the characteristic amplitude spectra or fractal geometry of natural scenes. This match has been probed in a wide variety of ways. Here we used the paradigm of "maximum likelihood difference scaling" (MLDS, Maloney and Knoblauch Ann Rev Vis Sci 2020) to explore the perceptual representation of the spatial structure of images. The MLDS task involves presenting pairs of images drawn from different levels along a dimension and judging which pair has greater difference. In our case the stimulus dimension corresponded to the slope of the image amplitude spectrum. Grayscale noise images were filtered to form a range of slopes from 0 to -2 in steps of 0.2. Further image arrays were generated by first binarizing the image intensities and then extracting only the edges to isolate the fractal structure (corresponding to fractal dimension range from 1 to 2 in steps of 0.1). For each array the MLDS task was used to estimate the perceived differences. Scaling for fractal dimension did not differentially favor natural fractal values. However for the amplitude spectra, the derived perceptual scales tended to be steeper for intermediate levels of the array and shallower for both strongly blurred or sharpened levels, consistent with greater perceptual salience for image differences that have more naturalistic spectra.

Using equiluminance settings to estimate the cardinal chromatic directions for individuals

Color information is processed by the retina and lateral geniculate along principal dimensions known as the cardinal directions of color space. Normal differences in spectral sensitivity can impact the stimulus directions that isolate these axes for individual observers and can arise from variation in lens and macular pigment density, photopigment opsins, photoreceptor optical density, and relative cone numbers. Some of these factors that influence the chromatic cardinal axes also impact luminance sensitivity. We modeled and empirically tested how well tilts on the individual's equiluminant plane are correlated with rotations in the directions of their cardinal chromatic axes. Our results show that, especially for the SvsLM axis, the chromatic axes can be partially predicted by luminance settings, providing a potential procedure for efficiently characterizing the cardinal chromatic axes for observers.

Visual adaptation to medical images: a comparison of digital mammography and tomosynthesis

Purpose

Radiologists and other image readers spend prolonged periods inspecting medical images. The visual system can rapidly adapt or adjust sensitivity to the images that an observer is currently viewing, and previous studies have demonstrated that this can lead to pronounced changes in the perception of mammogram images. We compared these adaptation effects for images from different imaging modalities to explore both general and modality-specific consequences of adaptation in medical image perception.

Approach

We measured perceptual changes induced by adaptation to images acquired by digital mammography (DM) or digital breast tomosynthesis (DBT), which have both similar and distinct textural properties. Participants (nonradiologists) adapted to images from the same patient acquired from each modality or for different patients with American College of Radiology-Breast Imaging Reporting and Data System (BI-RADS) classification of dense or fatty tissue. The participants then judged the appearance of composite images formed by blending the two adapting images (i.e., DM versus DBT or dense versus fatty in each modality).

Results

Adaptation to either modality produced similar significant shifts in the perception of dense and fatty textures, reducing the salience of the adapted component in the test images. In side-by-side judgments, a modality-specific adaptation effect was not observed. However, when the images were directly fixated during adaptation and testing, so that the textural differences between the modalities were more visible, significantly different changes in the sensitivity to the noise in the images were observed.

Conclusions

These results confirm that observers can readily adapt to the visual properties or spatial textures of medical images in ways that can bias their perception of the images, and that adaptation can also be selective for the distinctive visual features of images acquired by different modalities.

Calibrating Vision: Concepts and Questions

The idea that visual coding and perception are shaped by experience and adjust to changes in the environment or the observer is universally recognized as a cornerstone of visual processing, yet the functions and processes mediating these calibrations remain in many ways poorly understood. In this article we review a number of facets and issues surrounding the general notion of calibration, with a focus on plasticity within the encoding and representational stages of visual processing. These include how many types of calibrations there are - and how we decide; how plasticity for encoding is intertwined with other principles of sensory coding; how it is instantiated at the level of the dynamic networks mediating vision; how it varies with development or between individuals; and the factors that may limit the form or degree of the adjustments. Our goal is to give a small glimpse of an enormous and fundamental dimension of vision, and to point to some of the unresolved questions in our understanding of how and why ongoing calibrations are a pervasive and essential element of vision.

Temporal dynamics of face adaptation

The appearance of a face can be strongly affected by adaptation to faces seen previously. A number of studies have examined the time course of these aftereffects, but the integration time over which adaptation pools signals to control the adaptation state remains uncertain. Here we examined the effects of temporal frequency on face gender aftereffects induced by a pair of faces alternating between the two genders to assess when the aftereffects were pooled over successive faces versus driven by the last face seen. In the first experiment, we found that temporal frequencies between 0.25 and 2.00 Hz all failed to produce an aftereffect, suggesting a fairly long integration time. In the second experiment, we therefore probed slower alternation rates of 0.03 to 0.25 Hz. A rate of 0.0625 Hz (i.e., 8 seconds per face) was required to generate significant aftereffects from the last presented face and was consistent with an average time constant of 15 to 20 seconds for an exponentially decaying integration window. This integration time is substantially longer than found previously for analogous effects for alternating colors, and thus points to a potentially slower mechanism of adaptation for faces compared with chromatic adaptation.

Adapting to an enhanced color gamut - implications for color vision and color deficiencies

One strategy for aiding color deficiencies is to use three narrow passbands to filter the light spectrum to increase the saturation of colors. This filtering is analogous to the narrow emission bands used in wide gamut lighting or displays. We examined how perception adapts to the greater color gamut area produced by such devices, testing color-normal observers and simulated environments. Narrowband spectra increased chromatic contrasts but also increased contrast adaptation, partially offsetting the perceived contrast enhancements. Such adaptation adjustments are important for understanding the perceptual consequences of exposure to naturally or artificially enhanced color gamut areas for both color-deficient and color-normal observers.

Using Smooth Metamers to Estimate Color Appearance Metrics for Diverse Color-Normal Observers

Color-normal subjects sometimes disagree about metameric matches involving highly structured SPDs, because their cone fundamentals differ slightly, but non-negligibly. This has significant implications for the design of light sources and displays, so it should be estimated. We propose a broadly applicable estimation method based on a simple adaptive "front-end" interface that can be used with any selected standard color appearance model. The interface accepts, as input, any set of color matching functions for the individual subject (for example, these could be that person's cone response functions) and also the associated tristimulus values for the test stimulus and also for the reference stimulus (i.e. reference white). The interface converts this data into tristimulus values of the form used by the selected color appearance model (which could, for example, be X, Y, Z), while also carrying out the needed transform, which is based on an estimate of the subject's likely previous long-term adaptations to their unique cone fundamentals. The selected standard color appearance model then provides color appearance data that is an estimate of the color appearance of the test stimulus, for that individual subject. This information has the advantage of being interpretable within that model's well-known color space. The adaptive front end is based on the fact that, for any selected input SPD and the subject's unique color matching functions, there can be many different SPDs that are metameric for that individual. Since observer-to-observer color perception differences are minimized for spectrally smooth SPDs, smooth metamers predict color appearances reasonably accurately.

Effects of event number and adaptation duration on blur and face aftereffects

Previous studies have revealed that the strength of adaptation aftereffects can depend on both the number of adaptation events (trials) and the duration of each event. However, how these effects depend on the stimulus property adapted remains unknown. In the present study, we compared the influence of adaptation event number vs event duration on the strength of adaptation for blur or face aftereffects. For blur adaptation, we filtered a natural image's amplitude spectrum over slopes between -1 (blurred) to +1 (sharpened) relative to the original image to create a series of blurred/sharpened images. For face adaptation, we morphed an Asian and Caucasian face image resulting in a finely graded series of images spanning the two ethnicities. During each top-up adapting period, we varied the number of adaptation events (4 or 16) and duration of each event (250ms or 1s), resulting in 4 event number-event duration conditions. Our results suggest that the effects of event number and event duration on the strength of aftereffects are similar for blur and face adaptation.

Assessing the neural coding of image blur using multivariate pattern analysis

Blur is a fundamental perceptual attribute of images, but the way in which the visual system encodes this attribute remains poorly understood. Previously, we examined the neural correlates of blur by measuring BOLD responses to in-focus images and their blurred or sharpened counterparts, formed by varying the slope of the amplitude spectra but maintaining constant RMS contrast (Tregillus et al. 2014). In visual cortex (V1-V4), highest activation occurred for in-focus images compared to blurred or sharpened images - a finding which counters expectations from norm-based or predictive coding but is consistent with other studies examining the effects of manipulating the 1/f amplitude spectrum (Olman et al. 2004; Isherwood et al. 2017). To further examine the representation of blur, here we reanalysed this dataset using multivariate pattern analysis and also expanded the analysis to include additional visual areas (VO1, VO2, V3AB, LO, TO). A linear classifier trained to distinguish blurred vs. sharpened images provided significant decoding irrespective of visual area, suggesting that information about blur may be preserved across much of the visual hierarchy. The decoding may reflect larger scale differences in the representation of spatial frequency information within regions (e.g. as a function of eccentricity) or a finer scale columnar organization.

Contrast Adaptation in Face Perception Revealed Through EEG and Behavior

Exposure to a face can produce biases in the perception of subsequent faces. Typically, these face aftereffects are studied by adapting to an individual face or category (e.g., faces of a given gender) and can result in renormalization of perceptions such that the adapting face appears more neutral. These shifts are analogous to chromatic adaptation, where a renormalization for the average adapting color occurs. However, in color vision, adaptation can also adjust to the variance or range of colors in the distribution. We examined whether this variance or contrast adaptation also occurs for faces, using an objective EEG measure to assess response changes following adaptation. An average female face was contracted or expanded along the horizontal or vertical axis to form four images. Observers viewed a 20 s sequence of the four images presented in a fixed order at a rate of 6 Hz, while responses to the faces were recorded with EEG. A 6 Hz signal was observed over right occipito-temporal channels, indicating symmetric responses to the four images. This test sequence was repeated after 20 s adaptation to alternations between two of the faces (e.g., horizontal contracted and expanded). This adaptation resulted in an additional signal at 3 Hz, consistent with asymmetric responses to adapted and non-adapted test faces. Adapting pairs have the same mean (undistorted) as the test sequence and thus should not bias responses driven only by the mean. Instead, the results are consistent with selective adaptation to the distortion axis. A 3 Hz signal was also observed after adapting to face pairs selected to induce a mean bias (e.g., expanded vertical and expanded horizontal), and this signal was not significantly different from that observed following adaption to a single image that did not form part of the test sequence (e.g., a single image expanded both vertically and horizontally). In a further experiment, we found that this variance adaptation can also be observed behaviorally. Our results suggest that adaptation calibrates face perception not only for the average characteristics of the faces we experience but also for the gamut of faces to which we are exposed.

The magnitude of monocular light attenuation required to elicit the Pulfrich illusion

In the Pulfrich illusion, the depth of a moving object is misperceived due to induced retinal disparity and/or interocular velocity differences arising from differences in luminance, contrast, or spatial frequency between the two eyes. These effects have been shown to occur both for visual deficits and for optical corrections that introduce significant binocular differences between the retinal images. However, it remains unknown to what extent the illusion might arise given normal variation between the eyes, such as natural interocular variation in pupil diameter (anisocoria). To assess this, we examined the threshold interocular retinal illuminance difference required to experience illusory depth in two random-dot fields moving in opposite directions in 24 normally-sighted observers with dilated pupils. Interocular difference in retinal illuminance was induced by placing neutral density filters of different intensities before the left eye. A minority of subjects (n = 8) did not provide meaningful data on changes in the experience of illusory depth with interocular difference in retinal illuminance and four subjects showed biases >±10% from the 50% point of subjective equality in the psychometric function. For the remaining 12 participants, the retinal illuminance had to differ by approximately 40% for the depth between the planes to become visible at threshold levels. This difference was approximately constant over a range of absolute luminance levels from 10 to 80 cd/m2. Our results suggest that while motion-in-depth illusions due to interocular differences in retinal illuminance may be pronounced in certain ophthalmic diseases or following certain optical interventions, it is unlikely to be manifest as a result of normal interocular variations in retinal illuminance. Further, our results also point towards the existence of substantial individual differences in the experience of what is otherwise thought of as a readily appreciable motion-in-depth illusion.

Changes of tuning but not dynamics of contrast adaptation with age

Normal aging results in pronounced optical and neural changes in the visual system. Processes of adaptation are thought to help compensate for many of these changes in order to maintain perceptual constancy, but it is uncertain how stable adaptation itself remains with aging. We compared the dynamics of adaptation in young (aged 19-24 years) and older (aged 66-74) adults. Contrast thresholds for Gabor patterns were tracked during and after 300 s adaptation to vertical and horizontal Gabor patches. The time course of contrast adaptation and asymptotic adaptation magnitude were similar between older and young adults when normalized for their respective baseline thresholds. Older adults showed stronger transfer of adaptation to the orthogonal orientation and there was an asymmetry between the transfer of adaptation between the horizontal and vertical orientations for both groups. These results suggest age-related changes in orientation tuning while the processes of cortical contrast adaptation remain largely intact with aging.

Color appearance model incorporating contrast adaptation - implications for individual differences in color vision

Color appearance models use standard color matching functions to derive colorimetric information from spectral radiometric measurements of a visual environment, and they process that information to predict color perceptual attributes such as hue, chroma and lightness. That processing is usually done by equations with fixed numerical coefficients that were predetermined to yield optimal agreement for a given standard observer. Here we address the well-known fact that, among color-normal observers, there are significant differences of color matching functions. These cause disagreements between individuals as to whether certain colors match, an important effect that is often called observer metamerism. Yet how these individual sensitivity differences translate into differences in perceptual metrics is not fully addressed by many appearance models. It might seem that appearance could be predicted by substituting an individual's color matching functions into an otherwise-unchanged color appearance model, but this is problematic because the model's coefficients were not optimized for the new observer. Here we explore a solution guided by the idea that processes of adaptation in the visual system tend to compensate color perception for differences in cone responses and consequent color matching functions. For this purpose, we developed a simple color appearance model that uses only a few numerical coefficients, yet accurately predicts the perceptual attributes of Munsell samples under a selected standard lighting condition. We then added a feedback loop to automatically adjust the model coefficients, in response to switching between cone fundamentals simulating different observers and color matching functions. This adjustment is intended to model long term contrast adaptation in the vision system by maintaining average overall color contrast levels. Incorporating this adaptation principle into color appearance models could allow better assessments of displays and illumination systems, to help improve color appearances for most observers.

Varying Stimulus Duration Reveals Consistent Neural Activity and Behavior for Human Face Individuation

Establishing consistent relationships between neural activity and behavior is a challenge in human cognitive neuroscience research. We addressed this issue using variable time constraints in an oddball frequency-sweep design for visual discrimination of complex images (face exemplars). Sixteen participants viewed sequences of ascending presentation durations, from 25 to 333 ms (40-3 Hz stimulation rate) while their electroencephalogram (EEG) was recorded. Throughout each sequence, the same unfamiliar face picture was repeated with variable size and luminance changes while different unfamiliar facial identities appeared every 1 s (1 Hz). A neural face individuation response, tagged at 1 Hz and its unique harmonics, emerged over the occipito-temporal cortex at 50 ms stimulus duration (25-100 ms across individuals), with an optimal response reached at 170 ms stimulus duration. In a subsequent experiment, identity changes appeared non-periodically within fixed-frequency sequences while the same participants performed an explicit face individuation task. The behavioral face individuation response also emerged at 50 ms presentation time, and behavioral accuracy correlated with individual participants' neural response amplitude in a weighted middle stimulus duration range (50-125 ms). Moreover, the latency of the neural response peaking between 180 and 200 ms correlated strongly with individuals' behavioral accuracy in this middle duration range, as measured independently. These observations point to the minimal (50 ms) and optimal (170 ms) stimulus durations for human face individuation and provide novel evidence that inter-individual differences in the magnitude and latency of early, high-level neural responses are predictive of behavioral differences in performance at this function.

Duration Comparisons for Vision and Touch Are Dependent on Presentation Order and Temporal Context

Integrating visual and tactile information in the temporal domain is critical for active perception. To accomplish this, coordinated timing is required. Here, we study perceived duration within and across these two modalities. Specifically, we examined how duration comparisons within and across vision and touch were influenced by temporal context and presentation order using a two-interval forced choice task. We asked participants to compare the duration of two temporal intervals defined by tactile or visual events. Two constant standard durations (700 ms and 1,000 ms in 'shorter' sessions; 1,000 ms and 1,500 ms in 'longer' sessions) were compared to variable comparison durations in different sessions. In crossmodal trials, standard and comparison durations were presented in different modalities, whereas in the intramodal trials, the two durations were presented in the same modality. The standard duration was either presented first (<sc>) or followed the comparison duration (<cs>). In both crossmodal and intramodal conditions, we found that the longer standard duration was overestimated in <cs> trials and underestimated in <sc> trials whereas the estimation of shorter standard duration was unbiased. Importantly, the estimation of 1,000ms was biased when it was the longer standard duration within the shorter sessions but not when it was the shorter standard duration within the longer sessions, indicating an effect of temporal context. The effects of presentation order can be explained by a central tendency effect applied in different ways to different presentation orders. Both crossmodal and intramodal conditions showed better discrimination performance for <sc> trials than <cs> trials, supporting the Type B effect for both crossmodal and intramodal duration comparison. Moreover, these results were not dependent on whether the standard duration was defined using tactile or visual stimuli. Overall, our results indicate that duration comparison between vision and touch is dependent on presentation order and temporal context, but not modality.

ENHANCED PERIPHERAL FACE PROCESSING IN DEAF INDIVIDUALS

Studies of compensatory changes in visual functions in response to auditory loss have shown that enhancements tend to be restricted to the processing of specific visual features, such as motion in the periphery. Previous studies have also shown that deaf individuals can show greater face processing abilities in the central visual field. Enhancements in the processing of peripheral stimuli are thought to arise from a lack of auditory input and a subsequent increase in the allocation of attentional resources to peripheral locations, while enhancements in face processing abilities are thought to be driven by experience with ASL and not necessarily hearing loss. This combined with the fact that face processing abilities typically decline with eccentricity suggests that face processing enhancements may not extend to the periphery for deaf individuals. Using a face matching task, we examined whether deaf individuals' enhanced ability to discriminate between faces extends to the peripheral visual field. Deaf participants were more accurate than hearing participants in discriminating faces presented both centrally and in the periphery. Our results support earlier findings that deaf individuals possess enhanced face discrimination abilities in the central visual field and further extend them by showing that these enhancements also occur in the periphery for more complex stimuli.

Task-dependent contrast gain in anomalous trichromats

Anomalous trichromacy is a form of color vision deficiency characterized by the presence of three cone types, but with shifted spectral sensitivities for L or M cones, causing a red-green color deficiency. However, long-term adaptation to this impoverished opponent input may allow for a more normal color experience at the suprathreshold level ("compensation"). Recent experimental evidence points to the presence of compensation in some tasks. The current study used threshold detection, suprathreshold contrast matching, and a reaction-time task to compare contrast coding in normal and anomalous observers along the cardinal cone-opponent axes. Compared to color normals, anomals required more L-M contrast, but not S contrast, to detect stimuli and to match an achromatic reference stimulus. Reaction times were measured for several contrast levels along the two cone-opponent axes. Anomals had higher overall reaction times, but their reaction-time versus contrast functions could be matched to those of controls simply by scaling contrast by the detection thresholds. Anomalous participants were impaired relative to controls for L-M stimuli in all three tasks. However, the contrast losses were three times greater for thresholds and reaction times than for suprathreshold matches. These data provide evidence for compensation in anomalous trichromats, but highlight the role that the experimental task plays in revealing it.

Color perception and compensation in color deficiencies assessed with hue scaling

Anomalous trichromats have three classes of cone receptors but with smaller separation in the spectral sensitivities of their longer-wave (L or M) cones compared to normal trichromats. As a result, the differences in the responses of the longer-wave cones are smaller, resulting in a weaker input to opponent mechanisms that compare the LvsM responses. Despite this, previous studies have found that their color percepts are more similar to normal trichromats than the smaller LvsM differences predict, suggesting that post-receptoral processes might amplify their responses to compensate for the weaker opponent inputs. We evaluated the degree and form of compensation using a hue-scaling task, in which the appearance of different hues is described by the perceived proportions of red-green or blue-yellow primary colors. The scaling functions were modeled to estimate the relative salience of the red-green to blue-yellow components. The red-green amplitudes of the 10 anomalous observers were 1.5 times weaker than for a group of 26 normal controls. However, their relative sensitivity at threshold for detecting LvsM chromatic contrast was on average 6 times higher, consistent with a 4-fold gain in the suprathreshold hue-scaling responses. Within-observer variability in the settings was similar for the two groups, suggesting that the suprathreshold gain did not similarly amplify the noise, at least for the dimension of hue. While the compensation was pronounced it was nevertheless partial, and anomalous observers differed systematically from the controls in the shapes of the hue-scaling functions and the corresponding loci of their color categories. Factor analyses further revealed different patterns of individual differences between the groups. We discuss the implications of these results for understanding both the processes of compensation for a color deficiency and the limits of these processes.

Color Vision: Decoding Color Space

A new study has used magnetoencephalography to track cortical responses to color as they emerge in time. Similarities and differences within these neural responses parallel characteristics of the perceptual experience of color.

Color Compensation in Anomalous Trichromats Assessed with fMRI

Anomalous trichromacy is a common form of congenital color deficiency resulting from a genetic alteration in the photopigments of the eye's light receptors. The changes reduce sensitivity to reddish and greenish hues, yet previous work suggests that these observers may experience the world to be more colorful than their altered receptor sensitivities would predict, potentially indicating an amplification of post-receptoral signals. However, past evidence suggesting such a gain adjustment rests on subjective measures of color appearance or salience. We directly tested for neural amplification by using fMRI to measure cortical responses in color-anomalous and normal control observers. Color contrast response functions were measured in two experiments with different tasks to control for attentional factors. Both experiments showed a predictable reduction in chromatic responses for anomalous trichromats in primary visual cortex. However, in later areas V2v and V3v, chromatic responses in the two groups were indistinguishable. Our results provide direct evidence for neural plasticity that compensates for the deficiency in the initial receptor color signals and suggest that the site of this compensation is in early visual cortex.

Plasticity in perception: insights from color vision deficiencies

Inherited color vision deficiencies typically result from a loss or alteration of the visual photopigments absorbing light and thus impact the very first step of seeing. There is growing interest in how subsequent steps in the visual pathway might be calibrated to compensate for the altered receptor signals, with the possibility that color coding and color percepts might be less severely impacted than the receptor differences predict. These compensatory adjustments provide important insights into general questions about sensory plasticity and the sensory and cognitive processes underlying how we experience color.

Color Vision: Glasses Half Full

A new study finds that individuals with color deficiencies report long-term changes in their color vision after only a few days of wearing glasses that boost color contrasts, potentially because they learn to see or interpret color in new ways.

Neural correlates of perceptual color inferences as revealed by #thedress

Color constancy involves disambiguating the spectral characteristics of lights and surfaces, for example to distinguish red in white light from white in red light. Solving this problem appears especially challenging for bluish tints, which may be attributed more often to shading, and this bias may underlie the individual differences in whether people described the widely publicized image of #thedress as blue-black or white-gold. To probe these higher-level color inferences, we examined neural correlates of the blue-bias, using frequency-tagging and high-density electroencephalography to monitor responses to 3-Hz alternations between different color versions of #thedress. Specifically, we compared relative neural responses to the original “blue” dress image alternated with the complementary “yellow” image (formed by inverting the chromatic contrast of each pixel). This image pair produced a large modulation of the electroencephalography amplitude at the alternation frequency, consistent with a perceived contrast difference between the blue and yellow images. Furthermore, decoding topographical differences in the blue-yellow asymmetries over occipitoparietal channels predicted blue-black and white-gold observers with over 80% accuracy. The blue-yellow asymmetry was stronger than for a “red” versus “green” pair matched for the same component differences in L versus M or S versus LM chromatic contrast as the blue-yellow pair and thus cannot be accounted for by asymmetries within either precortical cardinal mechanism. Instead, the results may point to neural correlates of a higher-level perceptual representation of surface colors.

All-or-none face categorization in the human brain

Visual categorization is integral for our interaction with the natural environment. In this process, similar selective responses are produced to a class of variable visual inputs. Whether categorization is supported by partial (graded) or absolute (all-or-none) neural responses in high-level human brain regions is largely unknown. We address this issue with a novel frequency-sweep paradigm probing the evolution of face categorization responses between the minimal and optimal stimulus presentation times. In a first experiment, natural images of variable non-face objects were progressively swept from 120 to 3 ​Hz (8.33-333 ​ms duration) in rapid serial visual presentation sequences. Widely variable face exemplars appeared every 1 ​s, enabling an implicit frequency-tagged face-categorization electroencephalographic (EEG) response at 1 ​Hz. Face-categorization activity emerged with stimulus durations as brief as 17 ​ms (17-83 ​ms across individual participants) but was significant with 33 ​ms durations at the group level. The face categorization response amplitude increased until 83 ​ms stimulus duration (12 ​Hz), implying graded categorization responses. In a second EEG experiment, faces appeared non-periodically throughout such sequences at fixed presentation rates, while participants explicitly categorized faces. A strong correlation between response amplitude and behavioral accuracy across frequency rates suggested that dilution from missed categorizations, rather than a decreased response to each face stimulus, accounted for the graded categorization responses as found in Experiment 1. This was supported by (1) the absence of neural responses to faces that participants failed to categorize explicitly in Experiment 2 and (2) equivalent amplitudes and spatio-temporal signatures of neural responses to behaviorally categorized faces across presentation rates. Overall, these observations provide original evidence that high-level visual categorization of faces, starting at about 100 ​ms following stimulus onset in the human brain, is variable across observers tested under tight temporal constraints, but occurs in an all-or-none fashion.

Visual discomfort from flicker: Effects of mean light level and contrast

Uncomfortable images generally have a particular spatial structure, which deviates from a reciprocal relationship between amplitude and spatial frequency (f) in the Fourier domain (1/f). Although flickering patterns with similar temporal structure also appear uncomfortable, the discomfort is affected by not only the amplitude spectrum but also the phase spectrum. Here we examined how discomfort from flicker with differing temporal profiles also varies as a function of the mean light level and luminance contrast of the stimulus. Participants were asked to rate discomfort for a 17° flickering uniform field at different light levels from scotopic to photopic. The flicker waveform was varied with a square wave or random phase spectrum and filtered by modulating the slope of the amplitude spectrum relative to 1/f. At photopic levels, the 1/f square wave flicker appeared most comfortable, whereas the discomfort from the random flicker increased monotonically as the slope of the amplitude spectrum decreased. This special status for the 1/f square wave condition was limited to photopic light levels. At the lower mesopic or scotopic levels, the effect of phase spectrum on the discomfort was diminished, with both phase spectra showing a monotonic change with the slope of the amplitude spectrum. We show that these changes cannot be accounted for by changes in the effective luminance contrast of the stimuli or by the responses from a linear model based on the temporal impulse responses under different light levels. However, discomfort from flicker is robustly correlated with judgments of the perceived naturalness of flicker across different contrasts and mean luminance levels.

Comparison of two methods of hue scaling

Hue-scaling functions are designed to characterize color appearance by assessing the relative strength of the red versus green and blue versus yellow opponent sensations comprising different hues. However, these judgments can be non-intuitive and may pose difficulties for measurement and analysis. We explored an alternative scaling method based on positioning a dial to represent the relative similarity or distance of each hue from the labeled positions for the opponent categories. The hue-scaling and hue-similarity rating methods were compared for 28 observers. Settings on both tasks were comparable though the similarity ratings showed less inter-observer variability and weaker categorical bias, suggesting that these categorical biases may reflect properties of the task rather than the percepts. Alternatively, properties that are concordant for the two paradigms provide evidence for characteristics that do reflect color appearance. Individual differences on both tasks suggest that color appearance depends on multiple, narrowly tuned color processes, which are inconsistent with conventional color-opponent theory.

Color variance and achromatic settings

The average color in a scene is a potentially important cue to the illuminant and thus for color constancy, but it remains unknown how well and in what ways observers can estimate the mean chromaticity. We examined this by measuring the variability in "achromatic" settings for stimuli composed of different distributions of colors with varying contrast ranges along the luminance, SvsLM, and LvsM cardinal axes. Observers adjusted the mean chromaticity of the palette to set the average to gray. Variability in the settings increased as chromatic contrast or (to a lesser extent) luminance contrast increased. Signals along the cardinal axes are relatively independent in many detection and discrimination tasks, but showed strong interference in the white estimates. This "cross-masking" and the effects of chromatic variance in general may occur because observers cannot explicitly perceive or represent the mean of a set of qualitatively different hues (e.g., that red and green hues average to gray), and thus may infer the mean only indirectly (e.g., from the relative saturation of different hues).

Predicting color matches from luminance matches

Color vision and spectral sensitivity vary among individuals with normal color vision; thus, for many applications, it is important to measure and correct for an observer's sensitivity. Full correction would require measuring color and luminance matches and is rarely implemented. However, luminance matches (equiluminance settings) are routinely measured and simple to conduct. We modeled how well an observer's color matches could be approximated by measuring only luminance sensitivity, since both depend on a common set of factors. We show that lens and macular pigment density and $L/M$L/M cone ratios alter equiluminance settings in different ways and can therefore be estimated from the settings. In turn, the density variations can account for a large proportion of the normal variation in color matching. Thus, luminance matches may provide a simple method to at least partially predict an observer's color matches without requiring more complex tasks or equipment.

The Verriest Lecture: Adventures in blue and yellow

Conventional models of color vision assume that blue and yellow (along with red and green) are the fundamental building blocks of color appearance, yet how these hues are represented in the brain and whether and why they might be special are questions that remain shrouded in mystery. Many studies have explored the visual encoding of color categories, from the statistics of the environment to neural processing to perceptual experience. Blue and yellow are tied to salient features of the natural color world, and these features have likely shaped several important aspects of color vision. However, it remains less certain that these dimensions are encoded as primary or "unique" in the visual representation of color. There are also striking differences between blue and yellow percepts that may reflect high-level inferences about the world, specifically about the colors of light and surfaces. Moreover, while the stimuli labeled as blue or yellow or other basic categories show a remarkable degree of constancy within the observer, they all vary independently of one another across observers. This pattern of variation again suggests that blue and yellow and red and green are not a primary or unitary dimension of color appearance, and instead suggests a representation in which different hues reflect qualitatively different categories rather than quantitative differences within an underlying low-dimensional "color space."

Individual differences and their implications for color perception

Individual differences are a conspicuous feature of color vision and arise from many sources, in both the observer and the world. These differences have important practical implications for comparing and correcting perception and performance, and important theoretical implications for understanding the design principles underlying color coding. Color percepts within and between individuals often vary less than the variations in spectral sensitivity might predict. This stability is achieved by a variety of processes that compensate perception for the sensitivity limits of the eye and brain. Yet judgments of color between individuals can also vary widely, and in ways that are not readily explained by differences in sensitivity or the environment. These differences are uncorrelated across different color categories, and could reflect how these categories are learned or represented.

Directional Visual Motion Is Represented in the Auditory and Association Cortices of Early Deaf Individuals

Individuals who are deaf since early life may show enhanced performance at some visual tasks, including discrimination of directional motion. The neural substrates of such behavioral enhancements remain difficult to identify in humans, although neural plasticity has been shown for early deaf people in the auditory and association cortices, including the primary auditory cortex (PAC) and STS region, respectively. Here, we investigated whether neural responses in auditory and association cortices of early deaf individuals are reorganized to be sensitive to directional visual motion. To capture direction-selective responses, we recorded fMRI responses frequency-tagged to the 0.1-Hz presentation of central directional (100% coherent random dot) motion persisting for 2 sec contrasted with nondirectional (0% coherent) motion for 8 sec. We found direction-selective responses in the STS region in both deaf and hearing participants, but the extent of activation in the right STS region was 5.5 times larger for deaf participants. Minimal but significant direction-selective responses were also found in the PAC of deaf participants, both at the group level and in five of six individuals. In response to stimuli presented separately in the right and left visual fields, the relative activation across the right and left hemispheres was similar in both the PAC and STS region of deaf participants. Notably, the enhanced right-hemisphere activation could support the right visual field advantage reported previously in behavioral studies. Taken together, these results show that the reorganized auditory cortices of early deaf individuals are sensitive to directional motion. Speculatively, these results suggest that auditory and association regions can be remapped to support enhanced visual performance.

Dynamics of contrast adaptation in central and peripheral vision

Adaptation aftereffects are generally stronger for peripheral than for foveal viewing. We examined whether there are also differences in the dynamics of visual adaptation in central and peripheral vision. We tracked the time course of contrast adaptation to binocularly presented Gabor patterns in both the central visual field (within 5°) and in the periphery (beyond 10° eccentricity) using a yes/no detection task to monitor contrast thresholds. Consistent with previous studies, sensitivity losses were stronger in the periphery than in the center when adapting to equivalent high contrast (90% contrast) patterns. The time course of the threshold changes was fitted with separate exponential functions to estimate the time constants during the adapt and post-adapt phases. When adapting to equivalent high contrast, adaptation effects built up and decayed more slowly in the periphery compared with central adaptation. Surprisingly, the aftereffect in the periphery did not decay completely to the baseline within the monitored post-adapt period (400 s), and instead asymptoted to a higher level than for central adaptation. Even when contrast was reduced to one-third (30% contrast) of the central contrast, peripheral adaptation remained stronger and decayed more slowly. This slower dynamic was also confirmed at suprathreshold test contrasts by tracking tilt-aftereffects with a 2AFC orientation discrimination task. Our results indicate that the dynamics of contrast adaptation differ between central and peripheral vision, with the periphery adapting not only more strongly but also more slowly, and provide another example of potential qualitative processing differences between central and peripheral vision.

Adaptation and visual discomfort from flicker

Spatial images with unnatural amplitude spectra tend to appear uncomfortable. Analogous effects are found in the temporal domain, yet discomfort in flickering patterns is also strongly dependent on the phase spectrum. Here we examined how discomfort in temporal flicker is affected by adaptation to different amplitude and phase spectra. Adapting and test flicker were square wave or random phase transitions in a uniform field filtered by increasing (blurred) or decreasing (sharpened) the slope of the amplitude spectrum. Participants rated the level of discomfort or sharpness/blur for the test flicker. Before adaptation, square wave transitions were rated as most comfortable when they had "focused" edges, which were defined as characterized by 1/f amplitude spectra, while random phase transitions instead appeared more comfortable the more blurred they were. After adapting to blurred or sharpened transitions, both square wave and random phase flicker appeared more sharpened or blurred, respectively, and these effects were consistent with renormalization of perceived temporal focus. In comparison, adaptation affected discomfort in the two waveforms in qualitatively different ways, and exposure to the adapting stimulus tended to increase rather than decreased its perceived discomfort. These results point to a dissociation between the perceived amplitude spectrum and perceived discomfort, suggesting they in part depend on distinct processes. The results further illustrate the importance of the phase spectrum in determining visual discomfort from flickering patterns.