We infer light in space

Estimating lighting direction in scenes with multiple objects

To recover the reflectance and shape of an object in a scene, the human visual system must account for the properties of the light illuminating the object. Here, we examine the extent to which multiple objects within a scene are utilised to estimate the direction of lighting in a scene. In Experiment 1, we presented participants with rendered scenes that contained 1, 9, or 25 unfamiliar blob-like objects and measured their capacity to discriminate whether a directional light source was left or right of the participants' vantage point. Trends reported for ensemble perception suggest that the number of utilised objects-and, consequently, discrimination sensitivity-would increase with set size. However, we find little indication that increasing the number of objects in a scene increased discrimination sensitivity. In Experiment 2, an equivalent noise analysis was used to measure participants' internal noise and the number of objects used to judge the average light source direction in a scene, finding that participants relied on 1 or 2 objects to make their judgement regardless of whether 9 or 25 objects were present. In Experiment 3, participants completed a shape identification task that required an implicit judgement of light source direction, rather than an explicit judgement as in Experiments 1 and 2. We find that sensitivity for identifying surface shape was comparable for scenes containing 1, 9, and 25 objects. Our results suggest that the visual system relied on a small number of objects to estimate the direction of lighting in our rendered scenes.

Lightness Perception in Complex Scenes

Lightness perception is the perception of achromatic surface colors: black, white, and shades of grey. Lightness has long been a central research topic in experimental psychology, as perceiving surface color is an important visual task but also a difficult one due to the deep ambiguity of retinal images. In this article, I review psychophysical work on lightness perception in complex scenes over the past 20 years, with an emphasis on work that supports the development of computational models. I discuss Bayesian models, equivalent illumination models, multidimensional scaling, anchoring theory, spatial filtering models, natural scene statistics, and related work in computer vision. I review open topics in lightness perception that seem ready for progress, including the relationship between lightness and brightness, and developing more sophisticated computational models of lightness in complex scenes. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Effects of light map orientation and shape on the visual perception of canonical materials

We previously presented a systematic optics-based canonical approach to test material-lighting interactions in their full natural ecology, combining canonical material and lighting modes. Analyzing the power of the spherical harmonics components of the lighting allowed us to predict the lighting effects on material perception for generic natural illumination environments. To further understand how material properties can be brought out or communicated visually, in the current study, we tested whether and how light map orientation and shape affect these interactions in a rating experiment: For combinations of four materials, three shapes, and three light maps, we rotated the light maps in 15 different configurations. For the velvety objects, there were main and interaction effects of lighting and light map orientation. The velvety ratings decreased when the main light source was coming from the back of the objects. For the specular objects, there were main and interaction effects of lighting and shape. The specular ratings increased when the environment in the specular reflections was clearly visible in the stimuli. For the glittery objects, there were main and interaction effects of shape and light map orientation. The glittery ratings correlated with the coverage of the glitter reflections as the shape and light map orientation varied. For the matte objects, results were robust across all conditions. Last, we propose combining the canonical modes approach with so-called importance maps to analyze the appearance features of the proximal stimulus, the image, in contradistinction to the physical parameters as an approach for optimization of material communication.

Light: Toward a Transdisciplinary Science of Appearance and Atmosphere

To understand the processes behind seeing light, we need to integrate knowledge about the incoming optical structure, its perception, and how light interacts with material, shape, and space-objectively and subjectively. To that end, we need a novel approach to the science of light, namely, a transdisciplinary science of appearance, integrating optical, perceptual, and design knowledge and methods. In this article, I review existing literature as a basis for such a synthesis, which should discuss light in its full complexity, including its spatial properties and interactions with materials, shape, and space. I propose to investigate this by representing the endless variety of light, materials, shapes, and space as canonical modes and their combinations.

An Upward-Facing Surface Appears Darker: The Role Played by the Light-From-Above Assumption in Lightness Perception

The human visual system can extract information on surface reflectance (lightness) from light intensity; this, however, confounds information on reflectance and illumination. We hypothesized that the visual system, to solve this lightness problem, utilizes the internally held prior assumption that illumination falls from above. Experiment 1 showed that an upward-facing surface is perceived to be darker than a downward-facing surface, proving our hypothesis. Experiment 2 showed the same results in the absence of explicit illumination cues. The effect of the light-from-left prior assumption was not observed in Experiment 3. The upward- and downward-facing surface stimuli in Experiments 1 and 2 showed no difference in a two-dimensional configuration or three-dimensional structure, and the participants' perceived lightness appeared to be affected by the observers' prior assumption that illumination is always from above. Other studies have not accounted for this illusory effect, and this study's finding provides additional insights into the study of lightness perception.

Asymmetric perceptual confounds between canonical lightings and materials

To better understand the interactions between material perception and light perception, we further developed our material probe MatMix 1.0 into MixIM 1.0, which allows optical mixing of canonical lighting modes. We selected three canonical lighting modes (ambient, focus, and brilliance) and created scenes to represent the three illuminations. Together with four canonical material modes (matte, velvety, specular, glittery), this resulted in 12 basis images (the "bird set"). These images were optically mixed in our probing method. Three experiments were conducted with different groups of observers. In Experiment 1, observers were instructed to manipulate MixIM 1.0 and match optically mixed lighting modes while discounting the materials. In Experiment 2, observers were shown a pair of stimuli and instructed to simultaneously judge whether the materials and lightings were the same or different in a four-category discrimination task. In Experiment 3, observers performed both the matching and discrimination tasks in which only the ambient and focus light were implemented. Overall, the matching and discrimination results were comparable as (a) robust asymmetric perceptual confounds were found and confirmed in both types of tasks, (b) performances were consistent and all above chance levels, and (c) observers had higher sensitivities to our canonical materials than to our canonical lightings. The latter result may be explained in terms of a generic insensitivity for naturally occurring variations in light conditions. Our findings suggest that midlevel image features are more robust across different materials than across different lightings and, thus, more diagnostic for materials than for lightings, causing the asymmetric perceptual confounds.

Sensual Light? Subjective Dimensions of Ambient Illumination

This work concerns the subjective impression of perceived illumination. The purpose of the study is to test attributes expressing qualitative experiences referring to ambient lighting that can be applied as descriptors. Seventy participants viewed an actual model room, with the fourth wall removed (viewing booth). Walls, floor, and ceiling were achromatic. Two achromatic cubes were placed inside the room: One was a reflectance increment to the walls, the other a decrement. The room was illuminated by two different light sources, Artificial Daylight (D65) or Tungsten Filament (F), the order of which was randomized across participants. The participants' task was to evaluate ambient illumination for each light source. A semantic differential method was employed with 27 pairs of adjectives on 1 to 7 rating scales, categorized in three groups: characteristics of atmosphere, time, and cross-modal. Only the ratings of nine pairs of adjectives were not influenced by the type of illumination. The most differentiated couples under different illuminants were hot/cold and modern/old, but large differences also appeared with the following couples: hard/soft, technological/primitive, summery/wintry, warm/cool, sensual/frigid, natural/artificial, and hospitable/inhospitable. The hypothesis that there would be consistency among the subjects in evaluations of the characteristics tested and that these would be differently perceived under different illuminants was confirmed. The results show that it is possible to identify subjective perceived illumination as a phenomenon endowed with specific filling-in qualities and that as a perceptual experience it can be categorized, with implications for application in architecture and design.

Illumination discrimination in the absence of a fixed surface-reflectance layout

Previous studies have shown that humans can discriminate spectral changes in illumination and that this sensitivity depends both on the chromatic direction of the illumination change and on the ensemble of surfaces in the scene. These studies, however, always used stimulus scenes with a fixed surface-reflectance layout. Here we compared illumination discrimination for scenes in which the surface reflectance layout remains fixed (fixed-surfaces condition) to those in which surface reflectances were shuffled randomly across scenes, but with the mean scene reflectance held approximately constant (shuffled-surfaces condition). Illumination discrimination thresholds in the fixed-surfaces condition were commensurate with previous reports. Thresholds in the shuffled-surfaces condition, however, were considerably elevated. Nonetheless, performance in the shuffled-surfaces condition exceeded that attainable through random guessing. Analysis of eye fixations revealed that in the fixed-surfaces condition, low illumination discrimination thresholds (across observers) were predicted by low overall fixation spread and high consistency of fixation location and fixated surface reflectances across trial intervals. Performance in the shuffled-surfaces condition was not systematically related to any of the eye-fixation characteristics we examined for that condition, but was correlated with performance in the fixed-surfaces condition.

Visual Light Zones

In this article, we studied perception of a particular case of light fields that is characterized by a difference in its consistent structure between parts of a scene. In architectural lighting design, such a consistent structure in a part of a light field is called a light zone. First, we explored whether human observers are sensitive to light zones, that is, zones determined primarily by light flow differences, for a natural-looking scene. We found that observers were able to distinguish the light conditions between the zones. The results suggested an effect of light zones' orientation. Therefore, in Experiment 2, we systematically examined how the orientation of light zones (left-right or front-back) with respect to a viewer influences light inferences in symmetric scenes. We found that observers are quite sensitive to the difference in the light flow of the light zones. In addition, we found that participants showed idiosyncratic behavior, especially for front-back-oriented light zones. Our findings show that observers are sensitive to differences in light field structure between two parts of a scene, which we call visual light zones.

Separate and Simultaneous Adjustment of Light Qualities in a Real Scene

Humans are able to estimate light field properties in a scene in that they have expectations of the objects’ appearance inside it. Previously, we probed such expectations in a real scene by asking whether a “probe object” fitted a real scene with regard to its lighting. But how well are observers able to interactively adjust the light properties on a “probe object” to its surrounding real scene? Image ambiguities can result in perceptual interactions between light properties. Such interactions formed a major problem for the “readability” of the illumination direction and diffuseness on a matte smooth spherical probe. We found that light direction and diffuseness judgments using a rough sphere as probe were slightly more accurate than when using a smooth sphere, due to the three-dimensional (3D) texture. We here extended the previous work by testing independent and simultaneous (i.e., the light field properties separated one by one or blended together) adjustments of light intensity, direction, and diffuseness using a rough probe. Independently inferred light intensities were close to the veridical values, and the simultaneously inferred light intensity interacted somewhat with the light direction and diffuseness. The independently inferred light directions showed no statistical difference with the simultaneously inferred directions. The light diffuseness inferences correlated with but contracted around medium veridical values. In summary, observers were able to adjust the basic light properties through both independent and simultaneous adjustments. The light intensity, direction, and diffuseness are well “readable” from our rough probe. Our method allows “tuning the light” (adjustment of its spatial distribution) in interfaces for lighting design or perception research.

The visual light field in real scenes

Human observers' ability to infer the light field in empty space is known as the “visual light field.” While most relevant studies were performed using images on computer screens, we investigate the visual light field in a real scene by using a novel experimental setup. A “probe” and a scene were mixed optically using a semitransparent mirror. Twenty participants were asked to judge whether the probe fitted the scene with regard to the illumination intensity, direction, and diffuseness. Both smooth and rough probes were used to test whether observers use the additional cues for the illumination direction and diffuseness provided by the 3D texture over the rough probe. The results confirmed that observers are sensitive to the intensity, direction, and diffuseness of the illumination also in real scenes. For some lighting combinations on scene and probe, the awareness of a mismatch between the probe and scene was found to depend on which lighting condition was on the scene and which on the probe, which we called the “swap effect.” For these cases, the observers judged the fit to be better if the average luminance of the visible parts of the probe was closer to the average luminance of the visible parts of the scene objects. The use of a rough instead of smooth probe was found to significantly improve observers' abilities to detect mismatches in lighting diffuseness and directions.