The extraction of 3D shape from texture and shading in the human brain

The large-scale organization of shape processing in the ventral and dorsal pathways is dissociable from attention

The two visual pathways model posits that visual information is processed through two distinct cortical systems: The ventral pathway promotes visual recognition, while the dorsal pathway supports visuomotor control. Recent evidence suggests the dorsal pathway is also involved in shape processing and may contribute to object perception, but it remains unclear whether this sensitivity is independent of attentional mechanisms that were localized to overlapping cortical regions. To address this question, we conducted two fMRI experiments that utilized different parametric scrambling manipulations in which human participants viewed novel objects in different levels of scrambling and were instructed to attend to either the object or to another aspect of the image (e.g. color of the background). Univariate and multivariate analyses revealed that the large-scale organization of shape selectivity along the dorsal and ventral pathways was preserved regardless of the focus of attention. Attention did modulate shape sensitivity, but these effects were similar across the two pathways. These findings support the idea that shape processing is at least partially dissociable from attentional processes and relies on a distributed set of cortical regions across the visual pathways.

Extraction of three-dimensional shapes in glaucoma patients in response to monocular depth cues

PURPOSE

To assess the impact of glaucoma on perceiving three-dimensional (3D) shapes based on monocular depth cues.

STUDY DESIGN

Clinical observational study.

METHODS

Twenty glaucoma patients, subjected to binocular visual-field sensitivity (binocular-VFS) tests using a Humphrey Visual Field Analyzer, and 20 age-matched healthy volunteers, underwent two tasks: identifying the nearest vertex of a 3D shape using monocular shading (3D-SfS), texture (3D-SfT), or motion (3D-SfM) cues, and distinguishing elementary one-dimensional (1D) features of these cues. The association of the visual-field index (VFI) of binocular-VFS with 3D shape perception in glaucoma patients was also examined.

RESULTS

Glaucoma patients demonstrated reduced accuracy in distinguishing 1D luminance brightness and a larger "error-in-depth" between the perceived and actual depths for 3D-SfM and 3D-SfS compared to healthy volunteers. Six glaucoma patients with a 100% VFI for binocular-VFS exhibited a similar error-in-depth to the other fourteen glaucoma patients; they had a larger error-in-depth for 3D-SfM compared to healthy volunteers. No correlation between the error-in-depth values and the VFI values of binocular-VFS was observed.

CONCLUSIONS

The 3D shape perception in glaucoma patients varies based on the depth cue's characteristics. Impaired 1D discrimination and larger thresholds for 3D-SfM in glaucoma patients with a 100% VFI for binocular-VFS indicate more pronounced perceptual deficits of lower-level elementary features for 3D-SfS and higher-level visual processing of 3D shapes for 3D-SfM. The effects of the location and degree of binocular visual-field defects on 3D shape perception remain to be elucidated. Our research provides insights into the 3D shape extraction mechanism in glaucoma.

A focus on the multiple interfaces between action and perception and their neural correlates

Successful behaviour relies on the appropriate interplay between action and perception. The well-established dorsal and ventral stream theories depicted two distinct functional pathways for the processes of action and perception, respectively. In physiological conditions, the two pathways closely cooperate in order to produce successful adaptive behaviour. As the coupling between perception and action exists, this requires an interface that is responsible for a common reading of the two functions. Several studies have proposed different types of perception and action interfaces, suggesting their role in the creation of the shared interaction channel. In the present review, we describe three possible perception and action interfaces: i) the motor code, including common coding approaches, ii) attention, and iii) object affordance; we highlight their potential neural correlates. From this overview, a recurrent neural substrate that underlies all these interface functions appears to be crucial: the parieto-frontal circuit. This network is involved in the mirror mechanism which underlies the perception and action interfaces identified as common coding and motor code theories. The same network is also involved in the spotlight of attention and in the encoding of potential action towards objects; these are manifested in the perception and action interfaces for common attention and object affordance, respectively. Within this framework, most studies were dedicated to the description of the role of the inferior parietal lobule; growing evidence, however, suggests that the superior parietal lobule also plays a crucial role in the interplay between action and perception. The present review proposes a novel model that is inclusive of the superior parietal regions and their relative contribution to the different action and perception interfaces.

Hands off, brain off? A meta-analysis of neuroimaging data during active and passive driving

BACKGROUND

Car driving is more and more automated, to such an extent that driving without active steering control is becoming a reality. Although active driving requires the use of visual information to guide actions (i.e., steering the vehicle), passive driving only requires looking at the driving scene without any need to act (i.e., the human is passively driven).

MATERIALS & METHODS

After a careful search of the scientific literature, 11 different studies, providing 17 contrasts, were used to run a comprehensive meta-analysis contrasting active driving with passive driving.

RESULTS

Two brain regions were recruited more consistently for active driving compared to passive driving, the left precentral gyrus (BA3 and BA4) and the left postcentral gyrus (BA4 and BA3/40), whereas a set of brain regions was recruited more consistently in passive driving compared to active driving: the left middle frontal gyrus (BA6), the right anterior lobe and the left posterior lobe of the cerebellum, the right sub-lobar thalamus, the right anterior prefrontal cortex (BA10), the right inferior occipital gyrus (BA17/18/19), the right inferior temporal gyrus (BA37), and the left cuneus (BA17).

DISCUSSION

From a theoretical perspective, these findings support the idea that the output requirement of the visual scanning process engaged for the same activity can trigger different cerebral pathways, associated with different cognitive processes. A dorsal stream dominance was found during active driving, whereas a ventral stream dominance was obtained during passive driving. From a practical perspective, and contrary to the dominant position in the Human Factors community, our findings support the idea that a transition from passive to active driving would remain challenging as passive and active driving engage distinct neural networks.

Deficits of Visual Cortex Function in Acute Acquired Concomitant Esotropia Patients

Purpose

The purpose of this study was to explore the cortical deficits of patients with acquired concomitant esotropia (AACE) using the resting-state functional magnetic resonance imaging (rs-fMRI) technique.

Methods

Rs-fMRI signals from 25 patients with AACE and 25 matched controls were collected. The repeated-measures analysis of variance (RM-ANOVA) test and two-sample t-test were used to investigate statistical differences of the amplitudes of low-frequency fluctuation (ALFF) signals and correlation analysis was performed to validate the relationship of signal change and clinical features.

Results

The AACE group showed decreased ALFF in both hemispheres symmetrically (t = 0.38, P = 0.71), with peak t in both middle occipital gyrus. The ALFF signal from the upper left inferior frontal gyrus was negatively correlated with the age of onset (r = 0.62, P = 0.0008), and the ALFF signal from the right superior temporal gyrus was negatively correlated with the near work hours (r = 0.63, P = 0.0008). The ALFF signal in the left fusiform gyrus was positively correlated with both near (r = 0.48, P = 0.01) and far (r = 0.44, P = 0.03) deviation, whereas it was only positively correlated with far deviation (r = 0.44, P = 0.03) in the right. Besides, the age of onset and the near work hour were independent factors of signal changes.

Conclusions

Using the ALFF signal of rs-fMRI, we found functional deficits in the primary visual cortex and dorsal pathway in patients with AACE. There were functional changes in the fusiform gyrus, and the greater the deviation angle, the higher the changing level. These findings reveal the association of AACE and the visual center, giving us more clues about the treatment of AACE.

The perception of translucency from surface gloss

Translucent objects (like fruit and wax) reflect and transmit incident light to generate complex retinal image structure. Understanding how we visually perceive translucency from these images is challenging, but previous studies have demonstrated that perceived shape and shading is important for perceiving translucency. We considered the possibility that perceived translucency might also depend on 3D shape inferred from surface gloss (i.e., shape from specular highlights). Here, we performed experiments to test whether interactions between specular and non-specular image properties generated by different 3D shape information influences perceived translucency. Results revealed that perceived translucency could be explained by incongruence in 3D shape used to generate specular and non-specular image components. We proposed a new computational model based on measurable image features informative of shading relative to specular highlights that accounted for 59% of the variability in judgments of perceived translucency from the result of 10-fold cross validation. This model was found to outperform other models based on explicit subjective measures of perceived surface shape, suggesting it implicitly taps much of the relevant geometric information necessary for predicting observer judgments of translucency for glossy materials. These results provide new insight into how the visual system might infer translucency from the structure of specular and non-specular shading generated by glossy semi-opaque materials.

From pictures to reality: modelling the phenomenology and psychophysics of 3D perception

The dominant inferential approach to human 3D perception assumes a model of spatial encoding based on a physical description of objects and space. Prevailing models based on this physicalist approach assume that the visual system infers an objective, unitary and mostly veridical representation of the external world. However, careful consideration of the phenomenology of 3D perception challenges these assumptions. I review important aspects of phenomenology, psychophysics and neurophysiology which suggest that human visual perception of 3D  objects and space is underwritten by distinct and dissociated spatial encodings that are optimized for specific regions of space. Specifically, I argue that 3D perception is underwritten by at least three distinct encodings for (1) egocentric distance perception at the ambulatory scale, (2) exocentric distance (scaled depth) perception optimized for near space, and (3) perception of object shape and layout (unscaled depth). This tripartite division can more satisfactorily account for the phenomenology, psychophysics and adaptive logic of human 3D perception. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Cortical Deficits are Correlated with Impaired Stereopsis in Patients with Strabismus

In this study, we explored the neural mechanism underlying impaired stereopsis and possible functional plasticity after strabismus surgery. We enrolled 18 stereo-deficient patients with intermittent exotropia before and after surgery, along with 18 healthy controls. Functional magnetic resonance imaging data were collected when participants viewed three-dimensional stimuli. Compared with controls, preoperative patients showed hypoactivation in higher-level dorsal (visual and parietal) areas and ventral visual areas. Pre- and postoperative activation did not significantly differ in patients overall; patients with improved stereopsis showed stronger postoperative activation than preoperative activation in the right V3A and left intraparietal sulcus. Worse stereopsis and fusional control were correlated with preoperative hypoactivation, suggesting that cortical deficits along the two streams might reflect impaired stereopsis in intermittent exotropia. The correlation between improved stereopsis and activation in the right V3A after surgery indicates that functional plasticity may underlie the improvement of stereopsis. Thus, additional postoperative strategies are needed to promote functional plasticity and enhance the recovery of stereopsis.

Conditions of a Multi-View 3D Display for Accurate Reproduction of Perceived Glossiness

Visualizing objects as they are perceived in the real world is often critical in our daily experiences. We previously focused on objects' surface glossiness visualized with a 3D display and found that a multi-view 3D display reproduces perceived glossiness more accurately than a 2D display. This improvement of glossiness reproduction can be explained by the fact that a glossy surface visualized by a multi-view 3D display appropriately provides luminance differences between the two eyes and luminance changes accompanying the viewer's lateral head motion. In the present study, to determine the requirements of a multi-view 3D display for the accurate reproduction of perceived glossiness, we developed a simulator of a multi-view 3D display to independently and simultaneously manipulate the viewpoint interval and the magnitude of the optical inter-view crosstalk. Using the simulator, we conducted a psychophysical experiment and found that glossiness reproduction is most accurate when the viewpoint interval is small and there is just a small (but not too small) amount of crosstalk. We proposed a simple yet perceptually valid model that quantitatively predicts the reproduction accuracy of perceived glossiness.

Altered Temporal Dynamics of the Amplitude of Low-Frequency Fluctuations in Comitant Exotropia Patients

Purpose

Growing evidence reported that patients with comitant exotropia (CE) were accompanied by static cerebral neural activity changes. However, whether the dynamic time-varying of neural activity changes in patients with CE remains unknown.

Methods

A total of 36 patients with CE (25 men and 11 women) and 36 well-matched healthy controls are enrolled in the study. The dynamic amplitude of low-frequency fluctuation (dALFF) combined with the sliding window method was used to assess the dynamic neural activity changes in patients with CE.

Results

Compared with HCs, patients with CE had decreased dALFF values in the right superior parietal lobule (SPL) and right precuneus gyrus (PreCUN). Moreover, we found that the dALFF maps showed an accuracy of 48.61% and an area under the curve of.54 for distinguishing the patients with CE from HCs.

Conclusion

Our study demonstrated that patients with CE showed altered dynamic neural activity changes in the right SPL and right PreCUN, which might indicate the neuropathological mechanism of stereoscopic dysfunction in patients with CE.

The Dorsal Visual Pathway Represents Object-Centered Spatial Relations for Object Recognition

Although there is mounting evidence that input from the dorsal visual pathway is crucial for object processes in the ventral pathway, the specific functional contributions of dorsal cortex to these processes remain poorly understood. Here, we hypothesized that dorsal cortex computes the spatial relations among an object's parts, a process crucial for forming global shape percepts, and transmits this information to the ventral pathway to support object categorization. Using fMRI with human participants (females and males), we discovered regions in the intraparietal sulcus (IPS) that were selectively involved in computing object-centered part relations. These regions exhibited task-dependent functional and effective connectivity with ventral cortex, and were distinct from other dorsal regions, such as those representing allocentric relations, 3D shape, and tools. In a subsequent experiment, we found that the multivariate response of posterior (p)IPS, defined on the basis of part-relations, could be used to decode object category at levels comparable to ventral object regions. Moreover, mediation and multivariate effective connectivity analyses further suggested that IPS may account for representations of part relations in the ventral pathway. Together, our results highlight specific contributions of the dorsal visual pathway to object recognition. We suggest that dorsal cortex is a crucial source of input to the ventral pathway and may support the ability to categorize objects on the basis of global shape.SIGNIFICANCE STATEMENT Humans categorize novel objects rapidly and effortlessly. Such categorization is achieved by representing an object's global shape structure, that is, the relations among object parts. Yet, despite their importance, it is unclear how part relations are represented neurally. Here, we hypothesized that object-centered part relations may be computed by the dorsal visual pathway, which is typically implicated in visuospatial processing. Using fMRI, we identified regions selective for the part relations in dorsal cortex. We found that these regions can support object categorization, and even mediate representations of part relations in the ventral pathway, the region typically thought to support object categorization. Together, these findings shed light on the broader network of brain regions that support object categorization.

Dynamic dot displays reveal material motion network in the human brain

There is growing research interest in the neural mechanisms underlying the recognition of material categories and properties. This research field, however, is relatively more recent and limited compared to investigations of the neural mechanisms underlying object and scene category recognition. Motion is particularly important for the perception of non-rigid materials, but the neural basis of non-rigid material motion remains unexplored. Using fMRI, we investigated which brain regions respond preferentially to material motion versus other types of motion. We introduce a new database of stimuli - dynamic dot materials - that are animations of moving dots that induce vivid percepts of various materials in motion, e.g. flapping cloth, liquid waves, wobbling jelly. Control stimuli were scrambled versions of these same animations and rigid three-dimensional rotating dots. Results showed that isolating material motion properties with dynamic dots (in contrast with other kinds of motion) activates a network of cortical regions in both ventral and dorsal visual pathways, including areas normally associated with the processing of surface properties and shape, and extending to somatosensory and premotor cortices. We suggest that such a widespread preference for material motion is due to strong associations between stimulus properties. For example viewing dots moving in a specific pattern not only elicits percepts of material motion; one perceives a flexible, non-rigid shape, identifies the object as a cloth flapping in the wind, infers the object's weight under gravity, and anticipates how it would feel to reach out and touch the material. These results are a first important step in mapping out the cortical architecture and dynamics in material-related motion processing.

What Does Dorsal Cortex Contribute to Perception?

According to the influential "Two Visual Pathways" hypothesis, the cortical visual system is segregated into two pathways, with the ventral, occipitotemporal pathway subserving object perception, and the dorsal, occipitoparietal pathway subserving the visuomotor control of action. However, growing evidence suggests that the dorsal pathway also plays a functional role in object perception. In the current article, we present evidence that the dorsal pathway contributes uniquely to the perception of a range of visuospatial attributes that are not redundant with representations in ventral cortex. We describe how dorsal cortex is recruited automatically during perception, even when no explicit visuomotor response is required. Importantly, we propose that dorsal cortex may selectively process visual attributes that can inform the perception of potential actions on objects and environments, and we consider plausible developmental and cognitive mechanisms that might give rise to these representations. As such, we consider whether naturalistic stimuli, such as real-world solid objects, might engage dorsal cortex more so than simplified or artificial stimuli such as images that do not afford action, and how the use of suboptimal stimuli might limit our understanding of the functional contribution of dorsal cortex to visual perception.

Neural bases of self- and object-motion in a naturalistic vision

To plan movements toward objects our brain must recognize whether retinal displacement is due to self-motion and/or to object-motion. Here, we aimed to test whether motion areas are able to segregate these types of motion. We combined an event-related functional magnetic resonance imaging experiment, brain mapping techniques, and wide-field stimulation to study the responsivity of motion-sensitive areas to pure and combined self- and object-motion conditions during virtual movies of a train running within a realistic landscape. We observed a selective response in MT to the pure object-motion condition, and in medial (PEc, pCi, CSv, and CMA) and lateral (PIC and LOR) areas to the pure self-motion condition. Some other regions (like V6) responded more to complex visual stimulation where both object- and self-motion were present. Among all, we found that some motion regions (V3A, LOR, MT, V6, and IPSmot) could extract object-motion information from the overall motion, recognizing the real movement of the train even when the images remain still (on the screen), or moved, because of self-movements. We propose that these motion areas might be good candidates for the "flow parsing mechanism," that is the capability to extract object-motion information from retinal motion signals by subtracting out the optic flow components.

Generalized Representation of Stereoscopic Surface Shape and Orientation in the Human Visual Cortex

The brain's ability to extract three-dimensional (3D) shape and orientation information from viewed objects is vital in daily life. Stereoscopic 3D surface perception relies on binocular disparity. Neurons selective to binocular disparity are widely distributed among visual areas, but the manner in these areas are involved in stereoscopic 3D surface representation is unclear. To address this, participants were instructed to observe random dot stereograms (RDS) depicting convex and concave curved surfaces and the blood oxygenation level-dependent (BOLD) signal of visual cortices was recorded. Two surface types were: (i) horizontally positioned surfaces defined by shear disparity; and (ii) vertically positioned surfaces defined by compression disparity. The surfaces were presented at different depth positions per trial. Functional magnetic resonance imaging (fMRI) data were classified from early visual areas to higher visual areas. We determined whether cortical areas were selective to shape and orientation by assessing same-type stimuli classification accuracies based on multi-voxel activity patterns per area. To identify whether some areas were related to a more generalized sign of curvature or orientation representation, transfer classification was used by training classifiers on one dataset type and testing classifiers on another type. Same-type stimuli classification results showed that most selected visual areas were selective to shape and all were selective to the orientation of disparity-defined 3D surfaces. Transfer classification results showed that in the dorsal visual area V3A, classification accuracies for the discriminate sign of surface curvature were higher than the baseline of statistical significance for all types of classifications, demonstrating that V3A is related to generalized shape representation. Classification accuracies for discriminating horizontal-vertical surfaces in higher dorsal areas V3A and V7 and ventral area lateral occipital complex (LOC) as well as in some areas of intraparietal sulcus (IPS) were higher than the baseline of statistical significance, indicating their relation to the generalized representation of 3D surface orientation.

Single-cell selectivity and functional architecture of human lateral occipital complex

The human lateral occipital complex (LOC) is more strongly activated by images of objects compared to scrambled controls, but detailed information at the neuronal level is currently lacking. We recorded with microelectrode arrays in the LOC of 2 patients and obtained highly selective single-unit, multi-unit, and high-gamma responses to images of objects. Contrary to predictions derived from functional imaging studies, all neuronal properties indicated that the posterior subsector of LOC we recorded from occupies an unexpectedly high position in the hierarchy of visual areas. Notably, the response latencies of LOC neurons were long, the shape selectivity was spatially clustered, LOC receptive fields (RFs) were large and bilateral, and a number of LOC neurons exhibited three-dimensional (3D)-structure selectivity (a preference for convex or concave stimuli), which are all properties typical of end-stage ventral stream areas. Thus, our results challenge prevailing ideas about the position of the more posterior subsector of LOC in the hierarchy of visual areas.

Identifying Cortical Substrates Underlying the Phenomenology of Stereopsis and Realness: A Pilot fMRI Study

Viewing a real scene or a stereoscopic image (e.g., 3D movies) with both eyes yields a vivid subjective impression of object solidity, tangibility, immersive negative space and sense of realness; something that is not experienced when viewing single pictures of 3D scenes normally with both eyes. This phenomenology, sometimes referred to as stereopsis, is conventionally ascribed to the derivation of depth from the differences in the two eye's images (binocular disparity). Here we report on a pilot study designed to explore if dissociable neural activity associated with the phenomenology of realness can be localized in the cortex. In order to dissociate subjective impression from disparity processing, we capitalized on the finding that the impression of realness associated with stereoscopic viewing can also be generated when viewing a single picture of a 3D scene with one eye through an aperture. Under a blocked fMRI design, subjects viewed intact and scrambled images of natural 3-D objects, and scenes under three viewing conditions: (1) single pictures viewed normally with both eyes (binocular); (2) single pictures viewed with one eye through an aperture (monocular-aperture); and (3) stereoscopic anaglyph images of the same scenes viewed with both eyes (binocular stereopsis). Fixed-effects GLM contrasts aimed at isolating the phenomenology of stereopsis demonstrated a selective recruitment of similar posterior parietal regions for both monocular and binocular stereopsis conditions. Our findings provide preliminary evidence that the cortical processing underlying the subjective impression of realness may be dissociable and distinct from the derivation of depth from disparity.

Dissociating neural activity associated with the subjective phenomenology of monocular stereopsis: An EEG study

The subjective phenomenology associated with stereopsis, of solid tangible objects separated by a palpable negative space, is conventionally thought to be a by-product of the derivation of depth from binocular disparity. However, the same qualitative impression has been reported in the absence of disparity, e.g., when viewing pictorial images monocularly through an aperture. Here we aimed to explore if we could identify dissociable neural activity associated with the qualitative impression of stereopsis in the absence of the processing of binocular disparities. We measured EEG activity while subjects viewed pictorial (non-stereoscopic) images of 2D and 3D geometric forms under four different viewing conditions (binocular, monocular, binocular aperture, monocular aperture). EEG activity was analysed by oscillatory source localization (beamformer technique) to examine power change in occipital and parietal regions across viewing and stimulus conditions in targeted frequency bands (alpha: 8-13 Hz & gamma: 60-90 Hz). We observed expected event-related gamma synchronization and alpha desynchronization in occipital cortex and predominant gamma synchronization in parietal cortex across viewing and stimulus conditions. However, only the viewing condition predicted to generate the strongest impression of stereopsis (monocular aperture) revealed significantly elevated gamma synchronization within the parietal cortex for the critical contrasts (3D vs. 2D form). These findings suggest dissociable neural processes specific to the qualitative impression of stereopsis as distinguished from disparity processing.

Magnetoencephalography adaptation reveals depth-cue-invariant object representations in the visual cortex

Independent of edges and 2-D shape that can be highly informative of object identity, depth cues alone can also give rise to vivid and effective object percepts. The processing of different depth cues engages segregated cortical areas, and an efficient object representation would be one that is invariant to depth cues. Here, we investigated depth-cue invariance of object representations by measuring the category-specific response to faces-the M170 response measured with magnetoencephalography. The M170 response is strongest to faces and is sensitive to adaptation, such that repeated presentation of a face diminishes subsequent M170 responses. We used this feature of the M170 and measured the degree to which the adaptation effect is affected by variations in depth cue and 3-D object shape. Subjects viewed a rapid presentation of two stimuli-an adaptor and a test stimulus. The adaptor was either a face, a chair, or a face-like oval surface, and rendered with a single depth cue (shading, structure from motion, or texture). The test stimulus was always a shaded face of a random identity, thus completely controlling for low-level influences on the M170 response to the test stimulus. In the left fusiform face area, we found strong M170 adaptation when the adaptor was a face regardless of its depth cue. This adaptation was marginal in the right fusiform and negligible in the occipital regions. Our results support the presence of depth-cue-invariant representations in the human visual system, alongside size, position, and viewpoint invariance.

More than Action: The Dorsal Pathway Contributes to the Perception of 3-D Structure

An evolving view in cognitive neuroscience is that the dorsal visual pathway not only plays a key role in visuomotor behavior but that it also contributes functionally to the recognition of objects. To characterize the nature of the object representations derived by the dorsal pathway, we assessed perceptual performance in the context of the continuous flash suppression paradigm, which suppresses object processing in the ventral pathway while sparing computation in the dorsal pathway. In a series of experiments, prime stimuli, which were rendered imperceptible by the continuous flash suppression, still contributed to perceptual decisions related to the subsequent perceptible target stimuli. However, the contribution of the prime to perception was contingent on the prime's structural coherence, in that a perceptual advantage was observed only for targets primed by objects with legitimate 3-D structure. Finally, we obtained additional evidence to demonstrate that the processing of the suppressed objects was contingent on the magnocellular, rather than the parvocellular, system, further linking the processing of the suppressed stimuli to the dorsal pathway. Together, these results provide novel evidence that the dorsal pathway does not only support visuomotor control but, rather, that it also derives the structural description of 3-D objects and contributes to shape perception.

Binocular stereo acuity affects monocular three-dimensional shape perception in patients with strabismus

Background/aims

To evaluate the perception of three-dimensional (3D) shape in patients with strabismus and the contributions of stereopsis and monocular cues to this perception.

Methods

Twenty-one patients with strabismus with and 20 without stereo acuity as well as 25 age-matched normal volunteers performed two tasks: (1) identifying the closest vertices of 3D shapes from monocular shading (3D-SfS), texture (3D-SfT) or motion cues (3D-SfM) and from binocular disparity (3D-SfD), (2) discriminating 1D elementary features of these cues.

Results

Discrimination of the elementary features of luminance, texture and motion did not differ across groups. When the distances between reported and actual closest vertices were resolved into sagittal and frontoparallel plane components, sagittal components in 3D-SfS and frontoparallel components in 3D-SfT indicated larger errors in patients with strabismus without stereo acuity than in normal subjects. These patients could not discriminate one-dimensional elementary features of binocular disparity. Patients with strabismus with stereo acuity performed worse for both components of 3D-SfD and frontoparallel components of 3D-SfT compared with normal subjects. No differences were observed in the perception of 3D-SfM across groups. A comparison between normal subjects and patients with strabismus with normal stereopsis revealed no deficit in 3D shape perception from any cue.

Conclusions

Binocular stereopsis is essential for fine perception of 3D shape, even when 3D shape is defined by monocular static cues. Interaction between these cues may occur in ventral occipitotemporal regions, where 3D-SfS, 3D-SfT and 3D-SfD are processed in the same or neighbouring cortical regions. Our findings demonstrate the perceptual benefit of binocular stereopsis in patients with strabismus.

Effects of cortical damage on binocular depth perception

Stereoscopic depth perception requires considerable neural computation, including the initial correspondence of the two retinal images, comparison across the local regions of the visual field and integration with other cues to depth. The most common cause for loss of stereoscopic vision is amblyopia, in which one eye has failed to form an adequate input to the visual cortex, usually due to strabismus (deviating eye) or anisometropia. However, the significant cortical processing required to produce the percept of depth means that, even when the retinal input is intact from both eyes, brain damage or dysfunction can interfere with stereoscopic vision. In this review, I examine the evidence for impairment of binocular vision and depth perception that can result from insults to the brain, including both discrete damage, temporal lobectomy and more systemic diseases such as posterior cortical atrophy.

This article is part of the themed issue ‘Vision in our three-dimensional world’.

Functional interactions between the macaque dorsal and ventral visual pathways during three-dimensional object vision

The division of labor between the dorsal and the ventral visual stream in the primate brain has inspired numerous studies on the visual system in humans and in nonhuman primates. However, how and under which circumstances the two visual streams interact is still poorly understood. Here we review evidence from anatomy, modelling, electrophysiology, electrical microstimulation (EM), reversible inactivation and functional imaging in the macaque monkey aimed at clarifying at which levels in the hierarchy of visual areas the two streams interact, and what type of information might be exchanged between the two streams during three-dimensional (3D) object viewing. Neurons in both streams encode 3D structure from binocular disparity, synchronized activity between parietal and inferotemporal areas is present during 3D structure categorization, and clusters of 3D structure-selective neurons in parietal cortex are anatomically connected to ventral stream areas. In addition, caudal intraparietal cortex exerts a causal influence on 3D-structure related activations in more anterior parietal cortex and in inferotemporal cortex. Thus, both anatomical and functional evidence indicates that the dorsal and the ventral visual stream interact during 3D object viewing.

Smooth versus Textured Surfaces: Feature-Based Category Selectivity in Human Visual Cortex

In fMRI studies, human lateral occipital (LO) cortex is thought to respond selectively to images of objects, compared with nonobjects. However, it remains unresolved whether all objects evoke equivalent levels of activity in LO, and, if not, which image features produce stronger activation. Here, we used an unbiased parametric texture model to predict preferred versus nonpreferred stimuli in LO. Observation and psychophysical results showed that predicted preferred stimuli (both objects and nonobjects) had smooth (rather than textured) surfaces. These predictions were confirmed using fMRI, for objects and nonobjects. Similar preferences were also found in the fusiform face area (FFA). Consistent with this: (1) FFA and LO responded more strongly to nonfreckled (smooth) faces, compared with otherwise identical freckled (textured) faces; and (2) strong functional connections were found between LO and FFA. Thus, LO and FFA may be part of an information-processing stream distinguished by feature-based category selectivity (smooth > textured).

Biologically Inspired Model for Inference of 3D Shape from Texture

A biologically inspired model architecture for inferring 3D shape from texture is proposed. The model is hierarchically organized into modules roughly corresponding to visual cortical areas in the ventral stream. Initial orientation selective filtering decomposes the input into low-level orientation and spatial frequency representations. Grouping of spatially anisotropic orientation responses builds sketch-like representations of surface shape. Gradients in orientation fields and subsequent integration infers local surface geometry and globally consistent 3D depth. From the distributions in orientation responses summed in frequency, an estimate of the tilt and slant of the local surface can be obtained. The model suggests how 3D shape can be inferred from texture patterns and their image appearance in a hierarchically organized processing cascade along the cortical ventral stream. The proposed model integrates oriented texture gradient information that is encoded in distributed maps of orientation-frequency representations. The texture energy gradient information is defined by changes in the grouped summed normalized orientation-frequency response activity extracted from the textured object image. This activity is integrated by directed fields to generate a 3D shape representation of a complex object with depth ordering proportional to the fields output, with higher activity denoting larger distance in relative depth away from the viewer.

'What' Is Happening in the Dorsal Visual Pathway

The cortical visual system is almost universally thought to be segregated into two anatomically and functionally distinct pathways: a ventral occipitotemporal pathway that subserves object perception, and a dorsal occipitoparietal pathway that subserves object localization and visually guided action. Accumulating evidence from both human and non-human primate studies, however, challenges this binary distinction and suggests that regions in the dorsal pathway contain object representations that are independent of those in ventral cortex and that play a functional role in object perception. We review here the evidence implicating dorsal object representations, and we propose an account of the anatomical organization, functional contributions, and origins of these representations in the service of perception.

Brain activation difference evoked by different binocular disparities of stereograms: An fMRI study

The binocular disparity of two retina images is a main cue of stereoscopic vision. However, the global dependency between brain response and binocular disparity still remains unclear. Here, we used functional Magnetic Resonance Imaging (fMRI) to identify stereopsis-related brain regions with a modified Random Dot Stereogram (RDS) and plotted the activation variation curves under different disparity size. In order to eliminate the confounding shape difference between the stereogram and the plane, commonly seen in RDS, we modified the RDS to a checkerboard version. We found that V3A, V7 and MT+/V5 in dorsal visual stream were activated in stereoscopic experiment, while little activation was found in ventral visual regions. According to the activation trends, 13 subjects were divided into three groups: 5 subjects with turning points (a shift from increased to decreased activation), 5 subjects without turning points and 3 subjects with activation unrelated to disparity. We inferred that the dorsal visual stream primarily processes spatial depth information, rather than shape information.

Stereopsis after anterior temporal lobectomy

Brain areas critical for stereopsis have been investigated in non-human primates but are largely unknown in the human brain. Microelectrode recordings and functional MRI (fMRI) studies in monkeys have shown that in monkeys the inferior temporal cortex is critically involved in 3D shape categorization. Furthermore, some human fMRI studies similarly suggest an involvement of visual areas in the temporal lobe in depth perception. We aimed to investigate the role of the human anterior temporal neocortex in stereopsis by assessing stereoscopic depth perception before and after anterior temporal lobectomy. Eighteen epilepsy surgery patients were tested, pre- and postoperatively, in 3 different depth discrimination tasks. Sensitivity for local and global disparity was tested in a near-far discrimination task and sensitivity for 3D curvature was assessed in a convex-concave discrimination task, where 3D shapes were presented at different positions in depth. We found no evidence that temporal lobe epilepsy surgery has a significant effect on stereopsis. In contrast with earlier findings, we conclude that local as well as global stereopsis is maintained after unilateral resection of the temporal pole in epilepsy surgery patients. Our findings, together with previous studies, suggest that in humans more posterior visual regions underlie depth perception.

Learning the 3-D structure of objects from 2-D views depends on shape, not format

Humans can learn to recognize new objects just from observing example views. However, it is unknown what structural information enables this learning. To address this question, we manipulated the amount of structural information given to subjects during unsupervised learning by varying the format of the trained views. We then tested how format affected participants' ability to discriminate similar objects across views that were rotated 90° apart. We found that, after training, participants' performance increased and generalized to new views in the same format. Surprisingly, the improvement was similar across line drawings, shape from shading, and shape from shading + stereo even though the latter two formats provide richer depth information compared to line drawings. In contrast, participants' improvement was significantly lower when training used silhouettes, suggesting that silhouettes do not have enough information to generate a robust 3-D structure. To test whether the learned object representations were format-specific or format-invariant, we examined if learning novel objects from example views transfers across formats. We found that learning objects from example line drawings transferred to shape from shading and vice versa. These results have important implications for theories of object recognition because they suggest that (a) learning the 3-D structure of objects does not require rich structural cues during training as long as shape information of internal and external features is provided and (b) learning generates shape-based object representations independent of the training format.

Temporal Frequency Tuning Reveals Interactions between the Dorsal and Ventral Visual Streams

Visual processing of complex objects is supported by the ventral visual pathway in the service of object identification and by the dorsal visual pathway in the service of object-directed reaching and grasping. Here, we address how these two streams interact during tool processing, by exploiting the known asymmetry in projections of subcortical magnocellular and parvocellular inputs to the dorsal and ventral streams. The ventral visual pathway receives both parvocellular and magnocellular input, whereas the dorsal visual pathway receives largely magnocellular input. We used fMRI to measure tool preferences in parietal cortex when the images were presented at either high or low temporal frequencies, exploiting the fact that parvocellular channels project principally to the ventral but not dorsal visual pathway. We reason that regions of parietal cortex that exhibit tool preferences for stimuli presented at frequencies characteristic of the parvocellular pathway receive their inputs from the ventral stream. We found that the left inferior parietal lobule, in the vicinity of the supramarginal gyrus, exhibited tool preferences for images presented at low temporal frequencies, whereas superior and posterior parietal regions exhibited tool preferences for images present at high temporal frequencies. These data indicate that object identity, processed within the ventral stream, is communicated to the left inferior parietal lobule and may there combine with inputs from the dorsal visual pathway to allow for functionally appropriate object manipulation.

The neural representation of objects formed through the spatiotemporal integration of visual transients

Oftentimes, objects are only partially and transiently visible as parts of them become occluded during observer or object motion. The visual system can integrate such object fragments across space and time into perceptual wholes or spatiotemporal objects. This integrative and dynamic process may involve both ventral and dorsal visual processing pathways, along which shape and spatial representations are thought to arise. We measured fMRI BOLD response to spatiotemporal objects and used multi-voxel pattern analysis (MVPA) to decode shape information across 20 topographic regions of visual cortex. Object identity could be decoded throughout visual cortex, including intermediate (V3A, V3B, hV4, LO1-2,) and dorsal (TO1-2, and IPS0-1) visual areas. Shape-specific information, therefore, may not be limited to early and ventral visual areas, particularly when it is dynamic and must be integrated. Contrary to the classic view that the representation of objects is the purview of the ventral stream, intermediate and dorsal areas may play a distinct and critical role in the construction of object representations across space and time.

Look but don't touch: Visual cues to surface structure drive somatosensory cortex

When planning interactions with nearby objects, our brain uses visual information to estimate shape, material composition, and surface structure before we come into contact with them. Here we analyse brain activations elicited by different types of visual appearance, measuring fMRI responses to objects that are glossy, matte, rough, or textured. In addition to activation in visual areas, we found that fMRI responses are evoked in the secondary somatosensory area (S2) when looking at glossy and rough surfaces. This activity could be reliably discriminated on the basis of tactile-related visual properties (gloss, rough, and matte), but importantly, other visual properties (i.e., coloured texture) did not substantially change fMRI activity. The activity could not be solely due to tactile imagination, as asking explicitly to imagine such surface properties did not lead to the same results. These findings suggest that visual cues to an object's surface properties evoke activity in neural circuits associated with tactile stimulation. This activation may reflect the a-priori probability of the physics of the interaction (i.e., the expectation of upcoming friction) that can be used to plan finger placement and grasp force.

3D Shape Perception in Posterior Cortical Atrophy: A Visual Neuroscience Perspective

Posterior cortical atrophy (PCA) is a rare focal neurodegenerative syndrome characterized by progressive visuoperceptual and visuospatial deficits, most often due to atypical Alzheimer's disease (AD). We applied insights from basic visual neuroscience to analyze 3D shape perception in humans affected by PCA. Thirteen PCA patients and 30 matched healthy controls participated, together with two patient control groups with diffuse Lewy body dementia (DLBD) and an amnestic-dominant phenotype of AD, respectively. The hierarchical study design consisted of 3D shape processing for 4 cues (shading, motion, texture, and binocular disparity) with corresponding 2D and elementary feature extraction control conditions. PCA and DLBD exhibited severe 3D shape-processing deficits and AD to a lesser degree. In PCA, deficient 3D shape-from-shading was associated with volume loss in the right posterior inferior temporal cortex. This region coincided with a region of functional activation during 3D shape-from-shading in healthy controls. In PCA patients who performed the same fMRI paradigm, response amplitude during 3D shape-from-shading was reduced in this region. Gray matter volume in this region also correlated with 3D shape-from-shading in AD. 3D shape-from-disparity in PCA was associated with volume loss slightly more anteriorly in posterior inferior temporal cortex as well as in ventral premotor cortex. The findings in right posterior inferior temporal cortex and right premotor cortex are consistent with neurophysiologically based models of the functional anatomy of 3D shape processing. However, in DLBD, 3D shape deficits rely on mechanisms distinct from inferior temporal structural integrity.

SIGNIFICANCE STATEMENT

Posterior cortical atrophy (PCA) is a neurodegenerative syndrome characterized by progressive visuoperceptual dysfunction and most often an atypical presentation of Alzheimer's disease (AD) affecting the ventral and dorsal visual streams rather than the medial temporal system. We applied insights from fundamental visual neuroscience to analyze 3D shape perception in PCA. 3D shape-processing deficits were affected beyond what could be accounted for by lower-order processing deficits. For shading and disparity, this was related to volume loss in regions previously implicated in 3D shape processing in the intact human and nonhuman primate brain. Typical amnestic-dominant AD patients also exhibited 3D shape deficits. Advanced visual neuroscience provides insight into the pathogenesis of PCA that also bears relevance for vision in typical AD.

Aesthetic preference recognition of 3D shapes using EEG

Recognition and identification of aesthetic preference is indispensable in industrial design. Humans tend to pursue products with aesthetic values and make buying decisions based on their aesthetic preferences. The existence of neuromarketing is to understand consumer responses toward marketing stimuli by using imaging techniques and recognition of physiological parameters. Numerous studies have been done to understand the relationship between human, art and aesthetics. In this paper, we present a novel preference-based measurement of user aesthetics using electroencephalogram (EEG) signals for virtual 3D shapes with motion. The 3D shapes are designed to appear like bracelets, which is generated by using the Gielis superformula. EEG signals were collected by using a medical grade device, the B-Alert X10 from advance brain monitoring, with a sampling frequency of 256 Hz and resolution of 16 bits. The signals obtained when viewing 3D bracelet shapes were decomposed into alpha, beta, theta, gamma and delta rhythm by using time-frequency analysis, then classified into two classes, namely like and dislike by using support vector machines and K-nearest neighbors (KNN) classifiers respectively. Classification accuracy of up to 80 % was obtained by using KNN with the alpha, theta and delta rhythms as the features extracted from frontal channels, Fz, F3 and F4 to classify two classes, like and dislike.

Organization and evolution of parieto-frontal processing streams in macaque monkeys and humans

The functional organization of the parieto-frontal system is crucial for understanding cognitive-motor behavior and provides the basis for interpreting the consequences of parietal lesions in humans from a neurobiological perspective. The parieto-frontal connectivity defines some main information streams that, rather than being devoted to restricted functions, underlie a rich behavioral repertoire. Surprisingly, from macaque to humans, evolution has added only a few, new functional streams, increasing however their complexity and encoding power. In fact, the characterization of the conduction times of parietal and frontal areas to different target structures has recently opened a new window on cortical dynamics, suggesting that evolution has amplified the probability of dynamic interactions between the nodes of the network, thanks to communication patterns based on temporally-dispersed conduction delays. This might allow the representation of sensory-motor signals within multiple neural assemblies and reference frames, as to optimize sensory-motor remapping within an action space characterized by different and more complex demands across evolution.

Sensitivity to object impossibility in the human visual cortex: evidence from functional connectivity

Processing spatial configuration is a fundamental requirement for object recognition. Using fMRI, the neural basis underlying this ability was examined while human participants viewed possible and visually similar, but spatially impossible, objects presented for either long or short exposure duration. Response profiles in object-selective cortical regions exhibited sensitivity to object possibility, but only for the long exposure duration. Contrary, functional connectivity, indexed by the pairwise correlations between activation profiles across ROIs, revealed sensitivity to possibility, evident in enhanced correlations for impossible compared with possible objects. Such sensitivity was found even following a brief exposure duration, which allowed only minimal awareness of possibility. Importantly, this sensitivity was correlated with participants' general spatial ability as assessed by an independent neuropsychological test. These results suggest that the visual system is highly susceptible to objects' 3-D structural information even with minimal perceptual awareness. Such sensitivity is captured at the level of functional connectivity between object-selective regions, rather than the absolute level of within-region activity, implicating the role of interregional synchronization in the representation of objects' 3-D structure.

Shape from shading in starlings (Sturnus vulgaris)

Birds behave as if they quickly and accurately perceive an object-filled visual world. Beyond the extensive research with pigeons, however, there is a large and important gap in our knowledge about the mechanisms of object perception and recognition in other avian visual systems. The pattern of shading reflected from the surfaces of objects is one important optical feature that provides fundamental information about shape. To better understand how surface and object shading is processed by a passerine species, 5 starlings were tested with differentially illuminated convex and concave curved surfaces in 3 experiments using a simultaneous visual discrimination procedure. Starlings rapidly learned this shape-from-shading discrimination independent of varied lighting direction, surface color, and camera perspective. Variations in the pattern of lighting through experimental manipulations of camera perspective, surface height, contrast, material specularity, and surface shape were consistent with the hypothesis that the starlings perceived these illuminated surfaces as having 3-dimensional shape, similar to results previously collected with pigeons. These similarities across different orders of birds indicate that the relative shading of objects in a visual scene is a highly salient feature for shape processing in birds and is likely a highly conserved visual process that is widely distributed within this class of animal.

Efficient transfer entropy analysis of non-stationary neural time series

Information theory allows us to investigate information processing in neural systems in terms of information transfer, storage and modification. Especially the measure of information transfer, transfer entropy, has seen a dramatic surge of interest in neuroscience. Estimating transfer entropy from two processes requires the observation of multiple realizations of these processes to estimate associated probability density functions. To obtain these necessary observations, available estimators typically assume stationarity of processes to allow pooling of observations over time. This assumption however, is a major obstacle to the application of these estimators in neuroscience as observed processes are often non-stationary. As a solution, Gomez-Herrero and colleagues theoretically showed that the stationarity assumption may be avoided by estimating transfer entropy from an ensemble of realizations. Such an ensemble of realizations is often readily available in neuroscience experiments in the form of experimental trials. Thus, in this work we combine the ensemble method with a recently proposed transfer entropy estimator to make transfer entropy estimation applicable to non-stationary time series. We present an efficient implementation of the approach that is suitable for the increased computational demand of the ensemble method's practical application. In particular, we use a massively parallel implementation for a graphics processing unit to handle the computationally most heavy aspects of the ensemble method for transfer entropy estimation. We test the performance and robustness of our implementation on data from numerical simulations of stochastic processes. We also demonstrate the applicability of the ensemble method to magnetoencephalographic data. While we mainly evaluate the proposed method for neuroscience data, we expect it to be applicable in a variety of fields that are concerned with the analysis of information transfer in complex biological, social, and artificial systems.

The transition in the ventral stream from feature to real-world entity representations

We propose that the ventral visual pathway of human and non-human primates is organized into three levels: (1) ventral retinotopic cortex including what is known as TEO in the monkey but corresponds to V4A and PITd/v, and the phPIT cluster in humans, (2) area TE in the monkey and its homolog LOC and neighboring fusiform regions, and more speculatively, (3) TGv in the monkey and its possible human equivalent, the temporal pole. We attribute to these levels the visual representations of features, partial real-world entities (RWEs), and known, complete RWEs, respectively. Furthermore, we propose that the middle level, TE and its homolog, is organized into three parallel substreams, lower bank STS, dorsal convexity of TE, and ventral convexity of TE, as are their corresponding human regions. These presumably process shape in depth, 2D shape and material properties, respectively, to construct RWE representations.

Human cortical areas involved in perception of surface glossiness

Glossiness is the visual appearance of an object's surface as defined by its surface reflectance properties. Despite its ecological importance, little is known about the neural substrates underlying its perception. In this study, we performed the first human neuroimaging experiments that directly investigated where the processing of glossiness resides in the visual cortex. First, we investigated the cortical regions that were more activated by observing high glossiness compared with low glossiness, where the effects of simple luminance and luminance contrast were dissociated by controlling the illumination conditions (Experiment 1). As cortical regions that may be related to the processing of glossiness, V2, V3, hV4, VO-1, VO-2, collateral sulcus (CoS), LO-1, and V3A/B were identified, which also showed significant correlation with the perceived level of glossiness. This result is consistent with the recent monkey studies that identified selective neural response to glossiness in the ventral visual pathway, except for V3A/B in the dorsal visual pathway, whose involvement in the processing of glossiness could be specific to the human visual system. Second, we investigated the cortical regions that were modulated by selective attention to glossiness (Experiment 2). The visual areas that showed higher activation to attention to glossiness than that to either form or orientation were identified as right hV4, right VO-2, and right V3A/B, which were commonly identified in Experiment 1. The results indicate that these commonly identified visual areas in the human visual cortex may play important roles in glossiness perception.

Resting state functional connectivity in early blind humans

Task-based neuroimaging studies in early blind humans (EB) have demonstrated heightened visual cortex responses to non-visual paradigms. Several prior functional connectivity studies in EB have shown altered connections consistent with these task-based results. But these studies generally did not consider behavioral adaptations to lifelong blindness typically observed in EB. Enhanced cognitive abilities shown in EB include greater serial recall and attention to memory. Here, we address the question of the extent to which brain intrinsic activity in EB reflects such adaptations. We performed a resting-state functional magnetic resonance imaging study contrasting 14 EB with 14 age/gender matched normally sighted controls (NS). A principal finding was markedly greater functional connectivity in EB between visual cortex and regions typically associated with memory and cognitive control of attention. In contrast, correlations between visual cortex and non-deprived sensory cortices were significantly lower in EB. Thus, the available data, including that obtained in prior task-based and resting state fMRI studies, as well as the present results, indicate that visual cortex in EB becomes more heavily incorporated into functional systems instantiating episodic recall and attention to non-visual events. Moreover, EB appear to show a reduction in interactions between visual and non-deprived sensory cortices, possibly reflecting suppression of inter-sensory distracting activity.