Designing for sensory adaptation: what you see depends on what you've been looking at - Recommendations, guidelines and standards should reflect this

Sensory systems continuously recalibrate their responses according to the current stimulus environment. As a result, perception is strongly affected by the current and recent context. These adaptative changes affect both sensitivity (e.g., habituating to noise, seeing better in the dark) and appearance (e.g. how things look, what catches attention) and adjust to many perceptual properties (e.g. from light level to the characteristics of someone's face). They therefore have a profound effect on most perceptual experiences, and on how and how well the senses work in different settings. Characterizing the properties of adaptation, how it manifests, and when it influences perception in modern environments can provide insights into the diversity of human experience. Adaptation could also be leveraged both to optimize perceptual abilities (e.g. in visual inspection tasks like radiology) and to mitigate unwanted consequences (e.g. exposure to potentially unhealthy stimulus environments).

Poster Session: A direct measure of adaptation and visual salience

One hypothesized function of adaptation is to increase the salience of novel targets by discounting the properties of the ambient environment. Previous studies have suggested this by finding faster search times for novel targets when searching on backgrounds observers are currently adapted to. However, this provides only an indirect measure of salience. Here, we developed a more direct measure of the impact of adaptation on feature salience. Backgrounds were oriented 1/f noise images with power confined within 15 deg of horizontal or vertical. Targets were 5 c/deg Gabor patches centered on the 8 deg backgrounds. Observers simultaneously adapted to the horizontal or vertical backgrounds shown on the left or right of fixation. A 250ms test probe then showed the Gabor patch on the same background (horizontal or vertical) on both sides. The target orientation was adjusted on one side until it appeared as conspicuous as a fixed target on the other side. Settings were made for fixed targets ranging from 10 to 45 deg from the backgrounds. For most conditions/observers, the salience matches required a smaller orientation offset on the same- vs. different-adapt background. These results support a functional role of adaptation in highlighting novelty by potentially "unmasking" the target from its background, and emphasize the importance of considering adaptation aftereffects not only for isolated targets but within the stimulus contexts they are embedded in.

Automatic, Early Color-Specific Neural Responses to Object Color Knowledge

Some familiar objects are associated with specific colors, e.g., rubber ducks with yellow. Whether and at what stage neural responses occur to these color associations remain open questions. We recorded frequency-tagged electroencephalogram (EEG) responses to periodic presentations of yellow-associated objects, shown among sequences of non-periodic blue-, red-, and green-associated objects. Both color and grayscale versions of the objects elicited yellow-specific responses, indicating an automatic activation of color knowledge from object shape. Follow-up experiments replicated these effects with green-specific responses, and demonstrated modulated responses for incongruent color/object associations. Importantly, the onset of color-specific responses was as early to grayscale as actually colored stimuli (before 100 ms), the latter additionally eliciting a conventional later response (approximately 140-230 ms) to actual stimulus color. This suggests that the neural representation of familiar objects includes both diagnostic shape and color properties, such that shape can elicit associated color-specific responses before actual color-specific responses occur.

Poster Session: Perceptual scaling and natural image statistics

Visual coding is thought to be matched to many of the properties of the natural visual environment, such as the characteristic amplitude spectra or fractal geometry of natural scenes. This match has been probed in a wide variety of ways. Here we used the paradigm of "maximum likelihood difference scaling" (MLDS, Maloney and Knoblauch Ann Rev Vis Sci 2020) to explore the perceptual representation of the spatial structure of images. The MLDS task involves presenting pairs of images drawn from different levels along a dimension and judging which pair has greater difference. In our case the stimulus dimension corresponded to the slope of the image amplitude spectrum. Grayscale noise images were filtered to form a range of slopes from 0 to -2 in steps of 0.2. Further image arrays were generated by first binarizing the image intensities and then extracting only the edges to isolate the fractal structure (corresponding to fractal dimension range from 1 to 2 in steps of 0.1). For each array the MLDS task was used to estimate the perceived differences. Scaling for fractal dimension did not differentially favor natural fractal values. However for the amplitude spectra, the derived perceptual scales tended to be steeper for intermediate levels of the array and shallower for both strongly blurred or sharpened levels, consistent with greater perceptual salience for image differences that have more naturalistic spectra.

Using equiluminance settings to estimate the cardinal chromatic directions for individuals

Color information is processed by the retina and lateral geniculate along principal dimensions known as the cardinal directions of color space. Normal differences in spectral sensitivity can impact the stimulus directions that isolate these axes for individual observers and can arise from variation in lens and macular pigment density, photopigment opsins, photoreceptor optical density, and relative cone numbers. Some of these factors that influence the chromatic cardinal axes also impact luminance sensitivity. We modeled and empirically tested how well tilts on the individual's equiluminant plane are correlated with rotations in the directions of their cardinal chromatic axes. Our results show that, especially for the SvsLM axis, the chromatic axes can be partially predicted by luminance settings, providing a potential procedure for efficiently characterizing the cardinal chromatic axes for observers.

Visual adaptation to medical images: a comparison of digital mammography and tomosynthesis

Purpose

Radiologists and other image readers spend prolonged periods inspecting medical images. The visual system can rapidly adapt or adjust sensitivity to the images that an observer is currently viewing, and previous studies have demonstrated that this can lead to pronounced changes in the perception of mammogram images. We compared these adaptation effects for images from different imaging modalities to explore both general and modality-specific consequences of adaptation in medical image perception.

Approach

We measured perceptual changes induced by adaptation to images acquired by digital mammography (DM) or digital breast tomosynthesis (DBT), which have both similar and distinct textural properties. Participants (nonradiologists) adapted to images from the same patient acquired from each modality or for different patients with American College of Radiology-Breast Imaging Reporting and Data System (BI-RADS) classification of dense or fatty tissue. The participants then judged the appearance of composite images formed by blending the two adapting images (i.e., DM versus DBT or dense versus fatty in each modality).

Results

Adaptation to either modality produced similar significant shifts in the perception of dense and fatty textures, reducing the salience of the adapted component in the test images. In side-by-side judgments, a modality-specific adaptation effect was not observed. However, when the images were directly fixated during adaptation and testing, so that the textural differences between the modalities were more visible, significantly different changes in the sensitivity to the noise in the images were observed.

Conclusions

These results confirm that observers can readily adapt to the visual properties or spatial textures of medical images in ways that can bias their perception of the images, and that adaptation can also be selective for the distinctive visual features of images acquired by different modalities.

Calibrating Vision: Concepts and Questions

The idea that visual coding and perception are shaped by experience and adjust to changes in the environment or the observer is universally recognized as a cornerstone of visual processing, yet the functions and processes mediating these calibrations remain in many ways poorly understood. In this article we review a number of facets and issues surrounding the general notion of calibration, with a focus on plasticity within the encoding and representational stages of visual processing. These include how many types of calibrations there are - and how we decide; how plasticity for encoding is intertwined with other principles of sensory coding; how it is instantiated at the level of the dynamic networks mediating vision; how it varies with development or between individuals; and the factors that may limit the form or degree of the adjustments. Our goal is to give a small glimpse of an enormous and fundamental dimension of vision, and to point to some of the unresolved questions in our understanding of how and why ongoing calibrations are a pervasive and essential element of vision.

Temporal dynamics of face adaptation

The appearance of a face can be strongly affected by adaptation to faces seen previously. A number of studies have examined the time course of these aftereffects, but the integration time over which adaptation pools signals to control the adaptation state remains uncertain. Here we examined the effects of temporal frequency on face gender aftereffects induced by a pair of faces alternating between the two genders to assess when the aftereffects were pooled over successive faces versus driven by the last face seen. In the first experiment, we found that temporal frequencies between 0.25 and 2.00 Hz all failed to produce an aftereffect, suggesting a fairly long integration time. In the second experiment, we therefore probed slower alternation rates of 0.03 to 0.25 Hz. A rate of 0.0625 Hz (i.e., 8 seconds per face) was required to generate significant aftereffects from the last presented face and was consistent with an average time constant of 15 to 20 seconds for an exponentially decaying integration window. This integration time is substantially longer than found previously for analogous effects for alternating colors, and thus points to a potentially slower mechanism of adaptation for faces compared with chromatic adaptation.

Adapting to an enhanced color gamut - implications for color vision and color deficiencies

One strategy for aiding color deficiencies is to use three narrow passbands to filter the light spectrum to increase the saturation of colors. This filtering is analogous to the narrow emission bands used in wide gamut lighting or displays. We examined how perception adapts to the greater color gamut area produced by such devices, testing color-normal observers and simulated environments. Narrowband spectra increased chromatic contrasts but also increased contrast adaptation, partially offsetting the perceived contrast enhancements. Such adaptation adjustments are important for understanding the perceptual consequences of exposure to naturally or artificially enhanced color gamut areas for both color-deficient and color-normal observers.

Using Smooth Metamers to Estimate Color Appearance Metrics for Diverse Color-Normal Observers

Color-normal subjects sometimes disagree about metameric matches involving highly structured SPDs, because their cone fundamentals differ slightly, but non-negligibly. This has significant implications for the design of light sources and displays, so it should be estimated. We propose a broadly applicable estimation method based on a simple adaptive "front-end" interface that can be used with any selected standard color appearance model. The interface accepts, as input, any set of color matching functions for the individual subject (for example, these could be that person's cone response functions) and also the associated tristimulus values for the test stimulus and also for the reference stimulus (i.e. reference white). The interface converts this data into tristimulus values of the form used by the selected color appearance model (which could, for example, be X, Y, Z), while also carrying out the needed transform, which is based on an estimate of the subject's likely previous long-term adaptations to their unique cone fundamentals. The selected standard color appearance model then provides color appearance data that is an estimate of the color appearance of the test stimulus, for that individual subject. This information has the advantage of being interpretable within that model's well-known color space. The adaptive front end is based on the fact that, for any selected input SPD and the subject's unique color matching functions, there can be many different SPDs that are metameric for that individual. Since observer-to-observer color perception differences are minimized for spectrally smooth SPDs, smooth metamers predict color appearances reasonably accurately.

Effects of event number and adaptation duration on blur and face aftereffects

Previous studies have revealed that the strength of adaptation aftereffects can depend on both the number of adaptation events (trials) and the duration of each event. However, how these effects depend on the stimulus property adapted remains unknown. In the present study, we compared the influence of adaptation event number vs event duration on the strength of adaptation for blur or face aftereffects. For blur adaptation, we filtered a natural image's amplitude spectrum over slopes between -1 (blurred) to +1 (sharpened) relative to the original image to create a series of blurred/sharpened images. For face adaptation, we morphed an Asian and Caucasian face image resulting in a finely graded series of images spanning the two ethnicities. During each top-up adapting period, we varied the number of adaptation events (4 or 16) and duration of each event (250ms or 1s), resulting in 4 event number-event duration conditions. Our results suggest that the effects of event number and event duration on the strength of aftereffects are similar for blur and face adaptation.

Assessing the neural coding of image blur using multivariate pattern analysis

Blur is a fundamental perceptual attribute of images, but the way in which the visual system encodes this attribute remains poorly understood. Previously, we examined the neural correlates of blur by measuring BOLD responses to in-focus images and their blurred or sharpened counterparts, formed by varying the slope of the amplitude spectra but maintaining constant RMS contrast (Tregillus et al. 2014). In visual cortex (V1-V4), highest activation occurred for in-focus images compared to blurred or sharpened images - a finding which counters expectations from norm-based or predictive coding but is consistent with other studies examining the effects of manipulating the 1/f amplitude spectrum (Olman et al. 2004; Isherwood et al. 2017). To further examine the representation of blur, here we reanalysed this dataset using multivariate pattern analysis and also expanded the analysis to include additional visual areas (VO1, VO2, V3AB, LO, TO). A linear classifier trained to distinguish blurred vs. sharpened images provided significant decoding irrespective of visual area, suggesting that information about blur may be preserved across much of the visual hierarchy. The decoding may reflect larger scale differences in the representation of spatial frequency information within regions (e.g. as a function of eccentricity) or a finer scale columnar organization.

Contrast Adaptation in Face Perception Revealed Through EEG and Behavior

Exposure to a face can produce biases in the perception of subsequent faces. Typically, these face aftereffects are studied by adapting to an individual face or category (e.g., faces of a given gender) and can result in renormalization of perceptions such that the adapting face appears more neutral. These shifts are analogous to chromatic adaptation, where a renormalization for the average adapting color occurs. However, in color vision, adaptation can also adjust to the variance or range of colors in the distribution. We examined whether this variance or contrast adaptation also occurs for faces, using an objective EEG measure to assess response changes following adaptation. An average female face was contracted or expanded along the horizontal or vertical axis to form four images. Observers viewed a 20 s sequence of the four images presented in a fixed order at a rate of 6 Hz, while responses to the faces were recorded with EEG. A 6 Hz signal was observed over right occipito-temporal channels, indicating symmetric responses to the four images. This test sequence was repeated after 20 s adaptation to alternations between two of the faces (e.g., horizontal contracted and expanded). This adaptation resulted in an additional signal at 3 Hz, consistent with asymmetric responses to adapted and non-adapted test faces. Adapting pairs have the same mean (undistorted) as the test sequence and thus should not bias responses driven only by the mean. Instead, the results are consistent with selective adaptation to the distortion axis. A 3 Hz signal was also observed after adapting to face pairs selected to induce a mean bias (e.g., expanded vertical and expanded horizontal), and this signal was not significantly different from that observed following adaption to a single image that did not form part of the test sequence (e.g., a single image expanded both vertically and horizontally). In a further experiment, we found that this variance adaptation can also be observed behaviorally. Our results suggest that adaptation calibrates face perception not only for the average characteristics of the faces we experience but also for the gamut of faces to which we are exposed.

The magnitude of monocular light attenuation required to elicit the Pulfrich illusion

In the Pulfrich illusion, the depth of a moving object is misperceived due to induced retinal disparity and/or interocular velocity differences arising from differences in luminance, contrast, or spatial frequency between the two eyes. These effects have been shown to occur both for visual deficits and for optical corrections that introduce significant binocular differences between the retinal images. However, it remains unknown to what extent the illusion might arise given normal variation between the eyes, such as natural interocular variation in pupil diameter (anisocoria). To assess this, we examined the threshold interocular retinal illuminance difference required to experience illusory depth in two random-dot fields moving in opposite directions in 24 normally-sighted observers with dilated pupils. Interocular difference in retinal illuminance was induced by placing neutral density filters of different intensities before the left eye. A minority of subjects (n = 8) did not provide meaningful data on changes in the experience of illusory depth with interocular difference in retinal illuminance and four subjects showed biases >±10% from the 50% point of subjective equality in the psychometric function. For the remaining 12 participants, the retinal illuminance had to differ by approximately 40% for the depth between the planes to become visible at threshold levels. This difference was approximately constant over a range of absolute luminance levels from 10 to 80 cd/m2. Our results suggest that while motion-in-depth illusions due to interocular differences in retinal illuminance may be pronounced in certain ophthalmic diseases or following certain optical interventions, it is unlikely to be manifest as a result of normal interocular variations in retinal illuminance. Further, our results also point towards the existence of substantial individual differences in the experience of what is otherwise thought of as a readily appreciable motion-in-depth illusion.

Changes of tuning but not dynamics of contrast adaptation with age

Normal aging results in pronounced optical and neural changes in the visual system. Processes of adaptation are thought to help compensate for many of these changes in order to maintain perceptual constancy, but it is uncertain how stable adaptation itself remains with aging. We compared the dynamics of adaptation in young (aged 19-24 years) and older (aged 66-74) adults. Contrast thresholds for Gabor patterns were tracked during and after 300 s adaptation to vertical and horizontal Gabor patches. The time course of contrast adaptation and asymptotic adaptation magnitude were similar between older and young adults when normalized for their respective baseline thresholds. Older adults showed stronger transfer of adaptation to the orthogonal orientation and there was an asymmetry between the transfer of adaptation between the horizontal and vertical orientations for both groups. These results suggest age-related changes in orientation tuning while the processes of cortical contrast adaptation remain largely intact with aging.

Color appearance model incorporating contrast adaptation - implications for individual differences in color vision

Color appearance models use standard color matching functions to derive colorimetric information from spectral radiometric measurements of a visual environment, and they process that information to predict color perceptual attributes such as hue, chroma and lightness. That processing is usually done by equations with fixed numerical coefficients that were predetermined to yield optimal agreement for a given standard observer. Here we address the well-known fact that, among color-normal observers, there are significant differences of color matching functions. These cause disagreements between individuals as to whether certain colors match, an important effect that is often called observer metamerism. Yet how these individual sensitivity differences translate into differences in perceptual metrics is not fully addressed by many appearance models. It might seem that appearance could be predicted by substituting an individual's color matching functions into an otherwise-unchanged color appearance model, but this is problematic because the model's coefficients were not optimized for the new observer. Here we explore a solution guided by the idea that processes of adaptation in the visual system tend to compensate color perception for differences in cone responses and consequent color matching functions. For this purpose, we developed a simple color appearance model that uses only a few numerical coefficients, yet accurately predicts the perceptual attributes of Munsell samples under a selected standard lighting condition. We then added a feedback loop to automatically adjust the model coefficients, in response to switching between cone fundamentals simulating different observers and color matching functions. This adjustment is intended to model long term contrast adaptation in the vision system by maintaining average overall color contrast levels. Incorporating this adaptation principle into color appearance models could allow better assessments of displays and illumination systems, to help improve color appearances for most observers.

Varying Stimulus Duration Reveals Consistent Neural Activity and Behavior for Human Face Individuation

Establishing consistent relationships between neural activity and behavior is a challenge in human cognitive neuroscience research. We addressed this issue using variable time constraints in an oddball frequency-sweep design for visual discrimination of complex images (face exemplars). Sixteen participants viewed sequences of ascending presentation durations, from 25 to 333 ms (40-3 Hz stimulation rate) while their electroencephalogram (EEG) was recorded. Throughout each sequence, the same unfamiliar face picture was repeated with variable size and luminance changes while different unfamiliar facial identities appeared every 1 s (1 Hz). A neural face individuation response, tagged at 1 Hz and its unique harmonics, emerged over the occipito-temporal cortex at 50 ms stimulus duration (25-100 ms across individuals), with an optimal response reached at 170 ms stimulus duration. In a subsequent experiment, identity changes appeared non-periodically within fixed-frequency sequences while the same participants performed an explicit face individuation task. The behavioral face individuation response also emerged at 50 ms presentation time, and behavioral accuracy correlated with individual participants' neural response amplitude in a weighted middle stimulus duration range (50-125 ms). Moreover, the latency of the neural response peaking between 180 and 200 ms correlated strongly with individuals' behavioral accuracy in this middle duration range, as measured independently. These observations point to the minimal (50 ms) and optimal (170 ms) stimulus durations for human face individuation and provide novel evidence that inter-individual differences in the magnitude and latency of early, high-level neural responses are predictive of behavioral differences in performance at this function.

Duration Comparisons for Vision and Touch Are Dependent on Presentation Order and Temporal Context

Integrating visual and tactile information in the temporal domain is critical for active perception. To accomplish this, coordinated timing is required. Here, we study perceived duration within and across these two modalities. Specifically, we examined how duration comparisons within and across vision and touch were influenced by temporal context and presentation order using a two-interval forced choice task. We asked participants to compare the duration of two temporal intervals defined by tactile or visual events. Two constant standard durations (700 ms and 1,000 ms in 'shorter' sessions; 1,000 ms and 1,500 ms in 'longer' sessions) were compared to variable comparison durations in different sessions. In crossmodal trials, standard and comparison durations were presented in different modalities, whereas in the intramodal trials, the two durations were presented in the same modality. The standard duration was either presented first (<sc>) or followed the comparison duration (<cs>). In both crossmodal and intramodal conditions, we found that the longer standard duration was overestimated in <cs> trials and underestimated in <sc> trials whereas the estimation of shorter standard duration was unbiased. Importantly, the estimation of 1,000ms was biased when it was the longer standard duration within the shorter sessions but not when it was the shorter standard duration within the longer sessions, indicating an effect of temporal context. The effects of presentation order can be explained by a central tendency effect applied in different ways to different presentation orders. Both crossmodal and intramodal conditions showed better discrimination performance for <sc> trials than <cs> trials, supporting the Type B effect for both crossmodal and intramodal duration comparison. Moreover, these results were not dependent on whether the standard duration was defined using tactile or visual stimuli. Overall, our results indicate that duration comparison between vision and touch is dependent on presentation order and temporal context, but not modality.

ENHANCED PERIPHERAL FACE PROCESSING IN DEAF INDIVIDUALS

Studies of compensatory changes in visual functions in response to auditory loss have shown that enhancements tend to be restricted to the processing of specific visual features, such as motion in the periphery. Previous studies have also shown that deaf individuals can show greater face processing abilities in the central visual field. Enhancements in the processing of peripheral stimuli are thought to arise from a lack of auditory input and a subsequent increase in the allocation of attentional resources to peripheral locations, while enhancements in face processing abilities are thought to be driven by experience with ASL and not necessarily hearing loss. This combined with the fact that face processing abilities typically decline with eccentricity suggests that face processing enhancements may not extend to the periphery for deaf individuals. Using a face matching task, we examined whether deaf individuals' enhanced ability to discriminate between faces extends to the peripheral visual field. Deaf participants were more accurate than hearing participants in discriminating faces presented both centrally and in the periphery. Our results support earlier findings that deaf individuals possess enhanced face discrimination abilities in the central visual field and further extend them by showing that these enhancements also occur in the periphery for more complex stimuli.

Task-dependent contrast gain in anomalous trichromats

Anomalous trichromacy is a form of color vision deficiency characterized by the presence of three cone types, but with shifted spectral sensitivities for L or M cones, causing a red-green color deficiency. However, long-term adaptation to this impoverished opponent input may allow for a more normal color experience at the suprathreshold level ("compensation"). Recent experimental evidence points to the presence of compensation in some tasks. The current study used threshold detection, suprathreshold contrast matching, and a reaction-time task to compare contrast coding in normal and anomalous observers along the cardinal cone-opponent axes. Compared to color normals, anomals required more L-M contrast, but not S contrast, to detect stimuli and to match an achromatic reference stimulus. Reaction times were measured for several contrast levels along the two cone-opponent axes. Anomals had higher overall reaction times, but their reaction-time versus contrast functions could be matched to those of controls simply by scaling contrast by the detection thresholds. Anomalous participants were impaired relative to controls for L-M stimuli in all three tasks. However, the contrast losses were three times greater for thresholds and reaction times than for suprathreshold matches. These data provide evidence for compensation in anomalous trichromats, but highlight the role that the experimental task plays in revealing it.