Equivalent noise characterization of human lightness constancy

Orthogonal neural representations support perceptual judgments of natural stimuli

In natural visually guided behavior, observers must separate relevant information from a barrage of irrelevant information. Many studies have investigated the neural underpinnings of this ability using artificial stimuli presented on blank backgrounds. Natural images, however, contain task-irrelevant background elements that might interfere with the perception of object features. Recent studies suggest that visual feature estimation can be modeled through the linear decoding of task-relevant information from visual cortex. So, if the representations of task-relevant and irrelevant features are not orthogonal in the neural population, then variation in the task-irrelevant features would impair task performance. We tested this hypothesis using human psychophysics and monkey neurophysiology combined with parametrically variable naturalistic stimuli. We demonstrate that (1) the neural representation of one feature (the position of an object) in visual area V4 is orthogonal to those of several background features, (2) the ability of human observers to precisely judge object position was largely unaffected by those background features, and (3) many features of the object and the background (and of objects from a separate stimulus set) are orthogonally represented in V4 neural population responses. Our observations are consistent with the hypothesis that orthogonal neural representations can support stable perception of object features despite the richness of natural visual scenes.

Characterization of human lightness discrimination thresholds for independent spectral variations

The lightness of an object is an intrinsic property that depends on its surface reflectance spectrum. The visual system estimates an object's lightness from the light reflected off its surface. However, the reflected light also depends on object extrinsic properties of the scene, such as the light source. For stable perception, the visual system needs to discount the variations due to the object extrinsic properties. We characterize this perceptual stability for variation in two spectral properties of the scene: the reflectance spectra of background objects and the intensity of light sources. We measure human observers' thresholds of discriminating computer-generated images of 3D scenes based on the lightness of a spherical target object in the scene. We measured change in discrimination thresholds as we varied the reflectance spectra of the objects and the intensity of the light sources in the scene, both individually and simultaneously. For small amounts of extrinsic variations, the discrimination thresholds remained nearly constant indicating that the thresholds were dominated by observers' intrinsic representation of lightness. As extrinsic variation increased, it started affecting observers' lightness judgment and the thresholds increased. We estimated that the effects of extrinsic variations were comparable to observers' intrinsic variation in the representation of object lightness. Moreover, for simultaneous variation of these spectral properties, the increase in threshold squared compared to the no-variation condition was a linear sum of the corresponding increase in threshold squared for the individual properties, indicating that the variations from these independent sources combine linearly.

Orthogonal neural representations support perceptual judgements of natural stimuli

In natural behavior, observers must separate relevant information from a barrage of irrelevant information. Many studies have investigated the neural underpinnings of this ability using artificial stimuli presented on simple backgrounds. Natural viewing, however, carries a set of challenges that are inaccessible using artificial stimuli, including neural responses to background objects that are task-irrelevant. An emerging body of evidence suggests that the visual abilities of humans and animals can be modeled through the linear decoding of task-relevant information from visual cortex. This idea suggests the hypothesis that irrelevant features of a natural scene should impair performance on a visual task only if their neural representations intrude on the linear readout of the task relevant feature, as would occur if the representations of task-relevant and irrelevant features are not orthogonal in the underlying neural population. We tested this hypothesis using human psychophysics and monkey neurophysiology, in response to parametrically variable naturalistic stimuli. We demonstrate that 1) the neural representation of one feature (the position of a central object) in visual area V4 is orthogonal to those of several background features, 2) the ability of human observers to precisely judge object position was largely unaffected by task-irrelevant variation in those background features, and 3) many features of the object and the background are orthogonally represented by V4 neural responses. Our observations are consistent with the hypothesis that orthogonal neural representations can support stable perception of objects and features despite the tremendous richness of natural visual scenes.

The Perceptual Science of Augmented Reality

Augmented reality (AR) systems aim to alter our view of the world and enable us to see things that are not actually there. The resulting discrepancy between perception and reality can create compelling entertainment and can support innovative approaches to education, guidance, and assistive tools. However, building an AR system that effectively integrates with our natural visual experience is hard. AR systems often suffer from visual limitations and artifacts, and addressing these flaws requires basic knowledge of perception. At the same time, AR system development can serve as a catalyst that drives innovative new research in perceptual science. This review describes recent perceptual research pertinent to and driven by modern AR systems, with the goal of highlighting thought-provoking areas of inquiry and open questions.