Human cortical areas involved in perception of surface glossiness

Glossiness perception and its pupillary response

Recent studies have revealed that pupillary response changes depend on perceptual factors such as subjective brightness caused by optical illusions and luminance. However, the manner in which the perceptual factor that is derived from the glossiness perception of object surfaces affects the pupillary response remains unclear. We investigated the relationship between the glossiness perception and pupillary response through a glossiness rating experiment that included recording the pupil diameter. We prepared general object images (original) and randomized images (shuffled) that comprised the same images with randomized small square regions as stimuli. The image features were controlled by matching the luminance histogram. The observers were asked to rate the perceived glossiness of the stimuli presented for 3,000 ms and the changes in their pupil diameters were recorded. Images with higher glossiness ratings constricted the pupil size more than those with lower glossiness ratings at the peak constriction of the pupillary responses during the stimulus duration. The linear mixed-effects model demonstrated that the glossiness rating, image category (original/shuffled), variance of the luminance histogram, and stimulus area were most effective in predicting the pupillary responses. These results suggest that the illusory brightness obtained by the image regions of high-glossiness objects, such as specular highlights, induce pupil constriction.

Conditions of a Multi-View 3D Display for Accurate Reproduction of Perceived Glossiness

Visualizing objects as they are perceived in the real world is often critical in our daily experiences. We previously focused on objects' surface glossiness visualized with a 3D display and found that a multi-view 3D display reproduces perceived glossiness more accurately than a 2D display. This improvement of glossiness reproduction can be explained by the fact that a glossy surface visualized by a multi-view 3D display appropriately provides luminance differences between the two eyes and luminance changes accompanying the viewer's lateral head motion. In the present study, to determine the requirements of a multi-view 3D display for the accurate reproduction of perceived glossiness, we developed a simulator of a multi-view 3D display to independently and simultaneously manipulate the viewpoint interval and the magnitude of the optical inter-view crosstalk. Using the simulator, we conducted a psychophysical experiment and found that glossiness reproduction is most accurate when the viewpoint interval is small and there is just a small (but not too small) amount of crosstalk. We proposed a simple yet perceptually valid model that quantitatively predicts the reproduction accuracy of perceived glossiness.

Gloss perception: Searching for a deep neural network that behaves like humans

The visual computations underlying human gloss perception remain poorly understood, and to date there is no image-computable model that reproduces human gloss judgments independent of shape and viewing conditions. Such a model could provide a powerful platform for testing hypotheses about the detailed workings of surface perception. Here, we made use of recent developments in artificial neural networks to test how well we could recreate human responses in a high-gloss versus low-gloss discrimination task. We rendered >70,000 scenes depicting familiar objects made of either mirror-like or near-matte textured materials. We trained numerous classifiers to distinguish the two materials in our images-ranging from linear classifiers using simple pixel statistics to convolutional neural networks (CNNs) with up to 12 layers-and compared their classifications with human judgments. To determine which classifiers made the same kinds of errors as humans, we painstakingly identified a set of 60 images in which human judgments are consistently decoupled from ground truth. We then conducted a Bayesian hyperparameter search to identify which out of several thousand CNNs most resembled humans. We found that, although architecture has only a relatively weak effect, high correlations with humans are somewhat more typical in networks of shallower to intermediate depths (three to five layers). We also trained deep convolutional generative adversarial networks (DCGANs) of different depths to recreate images based on our high- and low-gloss database. Responses from human observers show that two layers in a DCGAN can recreate gloss recognizably for human observers. Together, our results indicate that human gloss classification can best be explained by computations resembling early to mid-level vision.

Structure and function of neural circuit related to gloss perception in the macaque inferior temporal cortex: a case report

The inferior temporal (IT) cortex of the macaque monkey plays a pivotal role in the visual recognition of objects. In the IT cortex, a feature-selective network formed by connecting subregions specialized for common visual features seems to be a basic strategy for processing biologically important visual features. Gloss perception plays an important role in the judgment of materials and conditions of objects and is a biologically significant visual function. In the present study, we attempted to determine whether a neural circuit specialized for processing information related to gloss perception exists in the IT cortex in one monkey. We injected retrograde tracer into a gloss-selective subregion in the IT cortex where gloss-selective neurons were clustered in the neural recording experiment, and anatomically examined its neural connections. We observed that retrogradely labeled neurons were densely accumulated in multiple locations in the posterior and anterior IT cortices. Based on the results of this case study, we will discuss the possibility that, together with the injection site, the sites with a dense cluster of labeled neurons form feature-selective neural circuits for the processing of gloss information in the IT cortex.

The effect of visual distractors on visual working memory for surface roughness in the human brain

Research has confirmed that the visual working memory representation of objects' roughness is robust against illumination changes in the human ventral visual cortex and intraparietal sulcus, but not yet against visual distractors during memory maintenance. Thus, this study investigated the effects of visual distractors on roughness-related brain regions during the maintenance phase using multi-voxel pattern analysis (MVPA). We conducted an fMRI experiment in which participants were asked to memorize a sphere's roughness against visual distractors, presented during the delay period in random trials. Region of interest-based MVPA showed no contribution of the ventral visual cortex and intraparietal sulcus to the roughness memory, regardless of behavioral performance. Post hoc searchlight MVPA revealed an above-chance decoding performance level in the brain regions presumably related to haptic processing when no visual distractors were shown. In contrast, when visual distractors appeared in the delay period, decoding performance exceeded the chance level in the ventral visual cortex. These results suggest that when visual distractors are presented during the memory phase, both visual and haptic processing are related to visual working memory for roughness, and the weighting of modality changes depending on the presence of visual distractors.

Human brain activity reflecting facial attractiveness from skin reflection

Facial attraction has a great influence on our daily social interactions. Previous studies have mainly focused on the attraction from facial shape and expression. We recently found that faces with radiant skin appear to be more attractive than those with oily-shiny or matte skin. In the present study, we conducted functional magnetic resonance imaging (fMRI) and psychological experiments to determine the human brain activity that reflects facial attractiveness modulated by these skin reflection types. In the fMRI experiment, female subjects were shown successive images of unfamiliar female faces with matte, oily-shiny, or radiant skin. The subjects compared each face with the immediately preceding face in terms of attractiveness, age, and skin reflection, all based on the skin. The medial part of the orbitofrontal cortex (mOFC) was significantly more active when comparing attractiveness than when comparing skin reflection, suggesting that the mOFC is involved in processing facial attractiveness from skin reflection. In the psychological experiment, attractiveness rating was highest for radiant skin, followed by oily-shiny, and then matte skin. Comparison of the results of these experiments showed that mOFC activation level increased with attractiveness rating. These results suggest that the activation level of the mOFC reflects facial attractiveness from skin reflection.

Relationship Between the Activities of Gloss-Selective Neurons in the Macaque Inferior Temporal Cortex and the Gloss Discrimination Behavior of the Monkey

In the macaque monkey, neurons that selectively respond to specific gloss are present in a restricted region of the central part of the inferior temporal (IT) cortex. Although the population activity of these neurons is known to represent the perceptual gloss space, the involvement of their activity in gloss perception has not been directly tested. In the present study, we examined the causal relationship between the activities of gloss-selective neurons and gloss perception by applying electrical microstimulation or injection of small amounts of muscimol (GABA_A agonist) to manipulate neural activities while monkeys performed a gloss discrimination task. We found that microstimulation within or in the vicinity of the region where gloss-selective neurons were recorded induced bias toward higher gloss judgment. With muscimol injection, gloss discrimination performance was degraded in one monkey after the first injection into the region where gloss-selective neurons were recorded. These results suggest that gloss discrimination behavior is mediated by the activities of a gloss-selective network that includes the gloss-selective region in the central IT cortex examined here.

The limited contribution of early visual cortex in visual working memory for surface roughness

Are visual representations in the human early visual cortex necessary for visual working memory (VWM)? Previous studies suggest that VWM is underpinned by distributed representations across several brain regions, including the early visual cortex. Notably, in these studies, participants had to memorize images under consistent visual conditions. However, in our daily lives, we must retain the essential visual properties of objects despite changes in illumination or viewpoint. The role of brain regions-particularly the early visual cortices-in these situations remains unclear. The present study investigated whether the early visual cortex was essential for achieving stable VWM. Focusing on VWM for object surface properties, we conducted fMRI experiments, while male and female participants performed a delayed roughness discrimination task in which sample and probe spheres were presented under varying illumination. By applying multi-voxel pattern analysis to brain activity in regions of interest, we found that the ventral visual cortex and intraparietal sulcus were involved in roughness VWM under changing illumination conditions. In contrast, VWM was not supported as robustly by the early visual cortex. These findings show that visual representations in the early visual cortex alone are insufficient for the robust roughness VWM representation required during changes in illumination.

Differences in Stereoscopic Luster Evoked by Static and Dynamic Stimuli

We compared the classic static stereoscopic luster phenomenon with a recently described dynamic variant ("counter modulation") to investigate whether they are related to the same or different processes. In the experiments, we presented pairs of center-surround stimuli haploscopically and measured the effect of the contrast between center colors on perceived luster. The center colors were either static or temporally modulated. In addition, we examined five color conditions (one achromatic, two equiluminant, and two mixed conditions) and three background conditions that influence the channel-wise polarities of the contrast of the two centers to the common surround. The results for static and dynamic stimuli differed in several ways, suggesting that they depend on different mechanisms: Compared with the static version, in dynamic stimuli, luster was perceived at markedly lower contrasts, did not depend on the sign of the contrast polarities, and appeared more steady. However, both phenomena seem also similar in some respects: In both cases, equiluminant stimuli led to lustrous impressions that were considerably less strong than those evoked by stimuli containing luminance variation, and the strength of the perceived luster was generally boosted with reversed contrast polarities.

Neural Mechanisms of Material Perception: Quest on Shitsukan

In recent years, a growing body of research has addressed the nature and mechanism of material perception. Material perception entails perceiving and recognizing a material, surface quality or internal state of an object based on sensory stimuli such as visual, tactile, and/or auditory sensations. This process is ongoing in every aspect of daily life. We can, for example, easily distinguish whether an object is made of wood or metal, or whether a surface is rough or smooth. Judging whether the ground is wet or dry or whether a fish is fresh also involves material perception. Information obtained through material perception can be used to govern actions toward objects and to make decisions about whether to approach an object or avoid it. Because the physical processes leading to sensory signals related to material perception is complicated, it has been difficult to manipulate experimental stimuli in a rigorous manner. However, that situation is now changing thanks to advances in technology and knowledge in related fields. In this article, we will review what is currently known about the neural mechanisms responsible for material perception. We will show that cortical areas in the ventral visual pathway are strongly involved in material perception. Our main focus is on vision, but every sensory modality is involved in material perception. Information obtained through different sensory modalities is closely linked in material perception. Such cross-modal processing is another important feature of material perception, and will also be covered in this review.

Individual differences in learning correlate with modulation of brain activity induced by transcranial direct current stimulation

Transcranial direct current stimulation (tDCS) has been shown to enhance cognitive performance on a variety of tasks. It is hypothesized that tDCS enhances performance by affecting task related cortical excitability changes in networks underlying or connected to the site of stimulation facilitating long term potentiation. However, many recent studies have called into question the reliability and efficacy of tDCS to induce modulatory changes in brain activity. In this study, our goal is to investigate the individual differences in tDCS induced modulatory effects on brain activity related to the degree of enhancement in performance, providing insight into this lack of reliability. In accomplishing this goal, we used functional magnetic resonance imaging (fMRI) concurrently with tDCS stimulation (1 mA, 30 minutes duration) using a visual search task simulating real world conditions. The experiment consisted of three fMRI sessions: pre-training (no performance feedback), training (performance feedback which included response accuracy and target location and either real tDCS or sham stimulation given), and post-training (no performance feedback). The right posterior parietal cortex was selected as the site of anodal tDCS based on its known role in visual search and spatial attention processing. Our results identified a region in the right precentral gyrus, known to be involved with visual spatial attention and orienting, that showed tDCS induced task related changes in cortical excitability that were associated with individual differences in improved performance. This same region showed greater activity during the training session for target feedback of incorrect (target-error feedback) over correct trials for the tDCS stim over sham group indicating greater attention to target features during training feedback when trials were incorrect. These results give important insight into the nature of neural excitability induced by tDCS as it relates to variability in individual differences in improved performance shedding some light the apparent lack of reliability found in tDCS research.

Look but don't touch: Visual cues to surface structure drive somatosensory cortex

When planning interactions with nearby objects, our brain uses visual information to estimate shape, material composition, and surface structure before we come into contact with them. Here we analyse brain activations elicited by different types of visual appearance, measuring fMRI responses to objects that are glossy, matte, rough, or textured. In addition to activation in visual areas, we found that fMRI responses are evoked in the secondary somatosensory area (S2) when looking at glossy and rough surfaces. This activity could be reliably discriminated on the basis of tactile-related visual properties (gloss, rough, and matte), but importantly, other visual properties (i.e., coloured texture) did not substantially change fMRI activity. The activity could not be solely due to tactile imagination, as asking explicitly to imagine such surface properties did not lead to the same results. These findings suggest that visual cues to an object's surface properties evoke activity in neural circuits associated with tactile stimulation. This activation may reflect the a-priori probability of the physics of the interaction (i.e., the expectation of upcoming friction) that can be used to plan finger placement and grasp force.

Differential processing of binocular and monocular gloss cues in human visual cortex

The visual impression of an object's surface reflectance (“gloss”) relies on a range of visual cues, both monocular and binocular. Whereas previous imaging work has identified processing within ventral visual areas as important for monocular cues, little is known about cortical areas involved in processing binocular cues. Here, we used human functional MRI (fMRI) to test for brain areas selectively involved in the processing of binocular cues. We manipulated stereoscopic information to create four conditions that differed in their disparity structure and in the impression of surface gloss that they evoked. We performed multivoxel pattern analysis to find areas whose fMRI responses allow classes of stimuli to be distinguished based on their depth structure vs. material appearance. We show that higher dorsal areas play a role in processing binocular gloss information, in addition to known ventral areas involved in material processing, with ventral area lateral occipital responding to both object shape and surface material properties. Moreover, we tested for similarities between the representation of gloss from binocular cues and monocular cues. Specifically, we tested for transfer in the decoding performance of an algorithm trained on glossy vs. matte objects defined by either binocular or by monocular cues. We found transfer effects from monocular to binocular cues in dorsal visual area V3B/kinetic occipital (KO), suggesting a shared representation of the two cues in this area. These results indicate the involvement of mid- to high-level visual circuitry in the estimation of surface material properties, with V3B/KO potentially playing a role in integrating monocular and binocular cues.

Human visual cortical responses to specular and matte motion flows

Determining the compositional properties of surfaces in the environment is an important visual capacity. One such property is specular reflectance, which encompasses the range from matte to shiny surfaces. Visual estimation of specular reflectance can be informed by characteristic motion profiles; a surface with a specular reflectance that is difficult to determine while static can be confidently disambiguated when set in motion. Here, we used fMRI to trace the sensitivity of human visual cortex to such motion cues, both with and without photometric cues to specular reflectance. Participants viewed rotating blob-like objects that were rendered as images (photometric) or dots (kinematic) with either matte-consistent or shiny-consistent specular reflectance profiles. We were unable to identify any areas in low and mid-level human visual cortex that responded preferentially to surface specular reflectance from motion. However, univariate and multivariate analyses identified several visual areas; V1, V2, V3, V3A/B, and hMT+, capable of differentiating shiny from matte surface flows. These results indicate that the machinery for extracting kinematic cues is present in human visual cortex, but the areas involved in integrating such information with the photometric cues necessary for surface specular reflectance remain unclear.

Human V4 and ventral occipital retinotopic maps

The ventral surface of the human occipital lobe contains multiple retinotopic maps. The most posterior of these maps is considered a potential homolog of macaque V4, and referred to as human V4 ("hV4"). The location of the hV4 map, its retinotopic organization, its role in visual encoding, and the cortical areas it borders have been the subject of considerable investigation and debate over the last 25 years. We review the history of this map and adjacent maps in ventral occipital cortex, and consider the different hypotheses for how these ventral occipital maps are organized. Advances in neuroimaging, computational modeling, and characterization of the nearby anatomical landmarks and functional brain areas have improved our understanding of where human V4 is and what kind of visual representations it contains.

fMRI evidence for areas that process surface gloss in the human visual cortex

•

Glossiness information is mainly processed along ventral visual pathway.

•

The posterior fusiform sulcus (pFs) is especially selective to surface gloss.

•

V3B/KO responds to gloss, but differentially from the pFs.

Surface gloss is an important cue to the material properties of objects. Recent progress in the study of macaque’s brain has increased our understating of the areas involved in processing information about gloss, however the homologies with the human brain are not yet fully understood. Here we used human functional magnetic resonance imaging (fMRI) measurements to localize brain areas preferentially responding to glossy objects. We measured cortical activity for thirty-two rendered three-dimensional objects that had either Lambertian or specular surface properties. To control for differences in image structure, we overlaid a grid on the images and scrambled its cells. We found activations related to gloss in the posterior fusiform sulcus (pFs) and in area V3B/KO. Subsequent analysis with Granger causality mapping indicated that V3B/KO processes gloss information differently than pFs. Our results identify a small network of mid-level visual areas whose activity may be important in supporting the perception of surface gloss.