Attention to Form or Surface Properties Modulates Different Regions of Human Occipitotemporal Cortex

Human visual cortical responses to specular and matte motion flows

Determining the compositional properties of surfaces in the environment is an important visual capacity. One such property is specular reflectance, which encompasses the range from matte to shiny surfaces. Visual estimation of specular reflectance can be informed by characteristic motion profiles; a surface with a specular reflectance that is difficult to determine while static can be confidently disambiguated when set in motion. Here, we used fMRI to trace the sensitivity of human visual cortex to such motion cues, both with and without photometric cues to specular reflectance. Participants viewed rotating blob-like objects that were rendered as images (photometric) or dots (kinematic) with either matte-consistent or shiny-consistent specular reflectance profiles. We were unable to identify any areas in low and mid-level human visual cortex that responded preferentially to surface specular reflectance from motion. However, univariate and multivariate analyses identified several visual areas; V1, V2, V3, V3A/B, and hMT+, capable of differentiating shiny from matte surface flows. These results indicate that the machinery for extracting kinematic cues is present in human visual cortex, but the areas involved in integrating such information with the photometric cues necessary for surface specular reflectance remain unclear.

The relationship between the neural computations for speech and music perception is context-dependent: an activation likelihood estimate study

The relationship between the neurobiology of speech and music has been investigated for more than a century. There remains no widespread agreement regarding how (or to what extent) music perception utilizes the neural circuitry that is engaged in speech processing, particularly at the cortical level. Prominent models such as Patel's Shared Syntactic Integration Resource Hypothesis (SSIRH) and Koelsch's neurocognitive model of music perception suggest a high degree of overlap, particularly in the frontal lobe, but also perhaps more distinct representations in the temporal lobe with hemispheric asymmetries. The present meta-analysis study used activation likelihood estimate analyses to identify the brain regions consistently activated for music as compared to speech across the functional neuroimaging (fMRI and PET) literature. Eighty music and 91 speech neuroimaging studies of healthy adult control subjects were analyzed. Peak activations reported in the music and speech studies were divided into four paradigm categories: passive listening, discrimination tasks, error/anomaly detection tasks and memory-related tasks. We then compared activation likelihood estimates within each category for music vs. speech, and each music condition with passive listening. We found that listening to music and to speech preferentially activate distinct temporo-parietal bilateral cortical networks. We also found music and speech to have shared resources in the left pars opercularis but speech-specific resources in the left pars triangularis. The extent to which music recruited speech-activated frontal resources was modulated by task. While there are certainly limitations to meta-analysis techniques particularly regarding sensitivity, this work suggests that the extent of shared resources between speech and music may be task-dependent and highlights the need to consider how task effects may be affecting conclusions regarding the neurobiology of speech and music.

Development of Tool Representations in the Dorsal and Ventral Visual Object Processing Pathways

Tools represent a special class of objects, because they are processed across both the dorsal and ventral visual object processing pathways. Three core regions are known to be involved in tool processing: the left posterior middle temporal gyrus, the medial fusiform gyrus (bilaterally), and the left inferior parietal lobule. A critical and relatively unexplored issue concerns whether, in development, tool preferences emerge at the same time and to a similar degree across all regions of the tool-processing network. To test this issue, we used functional magnetic resonance imaging to measure the neural amplitude, peak location, and the dispersion of tool-related neural responses in the youngest sample of children tested to date in this domain (ages 4-8 years). We show that children recruit overlapping regions of the adult tool-processing network and also exhibit similar patterns of co-activation across the network to adults. The amplitude and co-activation data show that the core components of the tool-processing network are established by age 4. Our findings on the distributions of peak location and dispersion of activation indicate that the tool network undergoes refinement between ages 4 and 8 years.

The Representation of Object-Directed Action and Function Knowledge in the Human Brain

The appropriate use of everyday objects requires the integration of action and function knowledge. Previous research suggests that action knowledge is represented in frontoparietal areas while function knowledge is represented in temporal lobe regions. Here we used multivoxel pattern analysis to investigate the representation of object-directed action and function knowledge while participants executed pantomimes of familiar tool actions. A novel approach for decoding object knowledge was used in which classifiers were trained on one pair of objects and then tested on a distinct pair; this permitted a measurement of classification accuracy over and above object-specific information. Region of interest (ROI) analyses showed that object-directed actions could be decoded in tool-preferring regions of both parietal and temporal cortex, while no independently defined tool-preferring ROI showed successful decoding of object function. However, a whole-brain searchlight analysis revealed that while frontoparietal motor and peri-motor regions are engaged in the representation of object-directed actions, medial temporal lobe areas in the left hemisphere are involved in the representation of function knowledge. These results indicate that both action and function knowledge are represented in a topographically coherent manner that is amenable to study with multivariate approaches, and that the left medial temporal cortex represents knowledge of object function.

Grasping with the eyes: the role of elongation in visual recognition of manipulable objects

Processing within the dorsal visual stream subserves object-directed action, whereas visual object recognition is mediated by the ventral visual stream. Recent findings suggest that the computations performed by the dorsal stream can nevertheless influence object recognition. Little is known, however, about the type of dorsal stream information that is available to assist in object recognition. Here, we present a series of experiments that explored different psychophysical manipulations known to bias the processing of a stimulus toward the dorsal visual stream in order to isolate its contribution to object recognition. We show that elongated-shaped stimuli, regardless of their semantic category and familiarity, when processed by the dorsal stream, elicit visuomotor grasp-related information that affects how we categorize manipulable objects. Elongated stimuli may reduce ambiguity during grasp preparation by providing a coarse cue to hand shaping and orientation that is sufficient to support action planning. We propose that this dorsal-stream-based analysis of elongation along a principal axis is the basis for how the dorsal visual object processing stream can affect categorization of manipulable objects.

The Effect of Task Instruction on Haptic Texture Processing: The Neural Underpinning of Roughness and Spatial Density Perception

Perceived roughness is associated with a variety of physical factors and multiple peripheral afferent types. The current study investigated whether this complexity of the mapping between physical and perceptual space is reflected at the cortical level. In an integrative psychophysical and imaging approach, we used dot pattern stimuli for which previous studies reported a simple linear relationship of interdot spacing and perceived spatial density and a more complex function of perceived roughness. Thus, by using both a roughness and a spatial estimation task, the physical and perceived stimulus characteristics could be dissociated, with the spatial density task controlling for the processing of low-level sensory aspects. Multivoxel pattern analysis was used to investigate which brain regions hold information indicative of the level of the perceived texture characteristics. While information about differences in perceived roughness was primarily available in higher-order cortices, that is, the operculo-insular cortex and a ventral visual cortex region, information about perceived spatial density could already be derived from early somatosensory and visual regions. This result indicates that cortical processing reflects the different complexities of the evaluated haptic texture dimensions. Furthermore, this study is to our knowledge the first to show a contribution of the visual cortex to tactile roughness perception.

fMRI evidence for areas that process surface gloss in the human visual cortex

•

Glossiness information is mainly processed along ventral visual pathway.

•

The posterior fusiform sulcus (pFs) is especially selective to surface gloss.

•

V3B/KO responds to gloss, but differentially from the pFs.

Surface gloss is an important cue to the material properties of objects. Recent progress in the study of macaque’s brain has increased our understating of the areas involved in processing information about gloss, however the homologies with the human brain are not yet fully understood. Here we used human functional magnetic resonance imaging (fMRI) measurements to localize brain areas preferentially responding to glossy objects. We measured cortical activity for thirty-two rendered three-dimensional objects that had either Lambertian or specular surface properties. To control for differences in image structure, we overlaid a grid on the images and scrambled its cells. We found activations related to gloss in the posterior fusiform sulcus (pFs) and in area V3B/KO. Subsequent analysis with Granger causality mapping indicated that V3B/KO processes gloss information differently than pFs. Our results identify a small network of mid-level visual areas whose activity may be important in supporting the perception of surface gloss.

Photographs of manipulable objects are named more quickly than the same objects depicted as line-drawings: Evidence that photographs engage embodiment more than line-drawings

Previous research has shown that photographs of manipulable objects (i.e., those that can be grasped for use with one hand) are named more quickly than non-manipulable objects when they have been matched for object familiarity and age of acquisition. The current study tested the hypothesis that the amount of visual detail present in object depictions moderates these “manipulability” effects on object naming. The same objects were presented as photographs and line-drawings during a speeded naming task. Forty-six participants named 222 objects depicted in both formats. A significant object depiction (photographs versus line drawing) by manipulability interaction confirmed our hypothesis that manipulable objects are identified more quickly when shown as photographs; whereas, non-manipulable objects are identified equally quickly when shown as photographs versus line-drawings. These results indicate that factors such as surface detail and texture moderate the role of “action” and/or “manipulability” effects during object identification tasks, and suggest that photographs of manipulable objects are associated with more embodied representations of those objects than when they are depicted as line-drawings.

Contextual modulation as de-texturizer

Contextual modulation refers to the effect of texture placed outside of a neuron's classical receptive field as well as the effect of surround texture on the perceptual properties of variegated regions within. In this minireview, we argue that one role of contextual modulation is to enhance the perception of contours at the expense of textures, in short to de-texturize the image. The evidence for this role comes mainly from three sources: psychophysical studies of shape after-effects, computational models of neurons that exhibit iso-orientation surround inhibition, and fMRI studies revealing specialized areas for contour as opposed to texture processing. The relationship between psychophysical studies that support the notion of contextual modulation as de-texturizer and those that investigate contour integration and crowding is discussed.

Accuracy and speed of material categorization in real-world images

It is easy to visually distinguish a ceramic knife from one made of steel, a leather jacket from one made of denim, and a plush toy from one made of plastic. Most studies of material appearance have focused on the estimation of specific material properties such as albedo or surface gloss, and as a consequence, almost nothing is known about how we recognize material categories like leather or plastic. We have studied judgments of high-level material categories with a diverse set of real-world photographs, and we have shown (Sharan, 2009) that observers can categorize materials reliably and quickly. Performance on our tasks cannot be explained by simple differences in color, surface shape, or texture. Nor can the results be explained by observers merely performing shape-based object recognition. Rather, we argue that fast and accurate material categorization is a distinct, basic ability of the visual system.

Parahippocampal cortex is involved in material processing via echoes in blind echolocation experts

Some blind humans use sound to navigate by emitting mouth-clicks and listening to the echoes that reflect from silent objects and surfaces in their surroundings. These echoes contain information about the size, shape, location, and material properties of objects. Here we present results from an fMRI experiment that investigated the neural activity underlying the processing of materials through echolocation. Three blind echolocation experts (as well as three blind and three sighted non-echolocating control participants) took part in the experiment. First, we made binaural sound recordings in the ears of each echolocator while he produced clicks in the presence of one of three different materials (fleece, synthetic foliage, or whiteboard), or while he made clicks in an empty room. During fMRI scanning these recordings were played back to participants. Remarkably, all participants were able to identify each of the three materials reliably, as well as the empty room. Furthermore, a whole brain analysis, in which we isolated the processing of just the reflected echoes, revealed a material-related increase in BOLD activation in a region of left parahippocampal cortex in the echolocating participants, but not in the blind or sighted control participants. Our results, in combination with previous findings about brain areas involved in material processing, are consistent with the idea that material processing by means of echolocation relies on a multi-modal material processing area in parahippocampal cortex.

Representation of object weight in human ventral visual cortex

Skilled manipulation requires the ability to predict the weights of viewed objects based on learned associations linking object weight to object visual appearance. However, the neural mechanisms involved in extracting weight information from viewed object properties are unknown. Given that ventral visual pathway areas represent a wide variety of object features, one intriguing but as yet untested possibility is that these areas also represent object weight, a nonvisual motor-relevant object property. Here, using event-related fMRI and pattern classification techniques, we tested the novel hypothesis that object-sensitive regions in occipitotemporal cortex (OTC), in addition to traditional motor-related brain areas, represent object weight when preparing to lift that object. In two studies, the same participants prepared and then executed lifting actions with objects of varying weight. In the first study, we show that when lifting visually identical objects, where predicted weight is based solely on sensorimotor memory, weight is represented in object-sensitive OTC. In the second study, we show that when object weight is associated with a particular surface texture, that texture-sensitive OTC areas also come to represent object weight. Notably, these texture-sensitive areas failed to carry information about weight in the first study, when object surface properties did not specify weight. Our results indicate that the integration of visual and motor-relevant object information occurs at the level of single OTC areas and provide evidence that the ventral visual pathway is actively and flexibly engaged in processing object weight, an object property critical for action planning and control.

Visuo-haptic multisensory object recognition, categorization, and representation

Visual and haptic unisensory object processing show many similarities in terms of categorization, recognition, and representation. In this review, we discuss how these similarities contribute to multisensory object processing. In particular, we show that similar unisensory visual and haptic representations lead to a shared multisensory representation underlying both cross-modal object recognition and view-independence. This shared representation suggests a common neural substrate and we review several candidate brain regions, previously thought to be specialized for aspects of visual processing, that are now known also to be involved in analogous haptic tasks. Finally, we lay out the evidence for a model of multisensory object recognition in which top-down and bottom-up pathways to the object-selective lateral occipital complex are modulated by object familiarity and individual differences in object and spatial imagery.

The transition in the ventral stream from feature to real-world entity representations

We propose that the ventral visual pathway of human and non-human primates is organized into three levels: (1) ventral retinotopic cortex including what is known as TEO in the monkey but corresponds to V4A and PITd/v, and the phPIT cluster in humans, (2) area TE in the monkey and its homolog LOC and neighboring fusiform regions, and more speculatively, (3) TGv in the monkey and its possible human equivalent, the temporal pole. We attribute to these levels the visual representations of features, partial real-world entities (RWEs), and known, complete RWEs, respectively. Furthermore, we propose that the middle level, TE and its homolog, is organized into three parallel substreams, lower bank STS, dorsal convexity of TE, and ventral convexity of TE, as are their corresponding human regions. These presumably process shape in depth, 2D shape and material properties, respectively, to construct RWE representations.

The Impact of Density and Ratio on Object-Ensemble Representation in Human Anterior-Medial Ventral Visual Cortex

Behavioral research has demonstrated that observers can extract summary statistics from ensembles of multiple objects. We recently showed that a region of anterior-medial ventral visual cortex, overlapping largely with the scene-sensitive parahippocampal place area (PPA), participates in object-ensemble representation. Here we investigated the encoding of ensemble density in this brain region using fMRI-adaptation. In Experiment 1, we varied density by changing the spacing between objects and found no sensitivity in PPA to such density changes. Thus, density may not be encoded in PPA, possibly because object spacing is not perceived as an intrinsic ensemble property. In Experiment 2, we varied relative density by changing the ratio of 2 types of objects comprising an ensemble, and observed significant sensitivity in PPA to such ratio change. Although colorful ensembles were shown in Experiment 2, Experiment 3 demonstrated that sensitivity to object ratio change was not driven mainly by a change in the ratio of colors. Thus, while anterior-medial ventral visual cortex is insensitive to density (object spacing) changes, it does code relative density (object ratio) within an ensemble. Object-ensemble processing in this region may thus depend on high-level visual information, such as object ratio, rather than low-level information, such as spacing/spatial frequency.

Flexible establishment of functional brain networks supports attentional modulation of unconscious cognition

In classical theories of attention, unconscious automatic processes are thought to be independent of higher-level attentional influences. Here, we propose that unconscious processing depends on attentional enhancement of task-congruent processing pathways implemented by a dynamic modulation of the functional communication between brain regions. Using functional magnetic resonance imaging, we tested our model with a subliminally primed lexical decision task preceded by an induction task preparing either a semantic or a perceptual task set. Subliminal semantic priming was significantly greater after semantic compared to perceptual induction in ventral occipito-temporal (vOT) and inferior frontal cortex, brain areas known to be involved in semantic processing. The functional connectivity pattern of vOT varied depending on the induction task and successfully predicted the magnitude of behavioral and neural priming. Together, these findings support the proposal that dynamic establishment of functional networks by task sets is an important mechanism in the attentional control of unconscious processing.

When concepts lose their color: a case of object-color knowledge impairment

Color is important in our daily interactions with objects, and plays a role in both low- and high-level visual processing. Previous neuropsychological studies have shown that color perception and object-color knowledge can doubly dissociate, and that both can dissociate from processing of object form. We present a case study of an individual who displayed an impairment for knowledge of the typical colors of objects, with preserved color perception and color naming. Our case also presented with a pattern of, if anything, worse performance for naming living items compared to non-living things. The findings of the experimental investigation are evaluated in light of two theories of conceptual organization in the brain: the Sensory/Functional Theory and the Domain-Specific Hypothesis. The dissociations observed in this case compel a model in which sensory/motor modality and semantic domain jointly constrain the organization of object knowledge.

Human cortical areas involved in perception of surface glossiness

Glossiness is the visual appearance of an object's surface as defined by its surface reflectance properties. Despite its ecological importance, little is known about the neural substrates underlying its perception. In this study, we performed the first human neuroimaging experiments that directly investigated where the processing of glossiness resides in the visual cortex. First, we investigated the cortical regions that were more activated by observing high glossiness compared with low glossiness, where the effects of simple luminance and luminance contrast were dissociated by controlling the illumination conditions (Experiment 1). As cortical regions that may be related to the processing of glossiness, V2, V3, hV4, VO-1, VO-2, collateral sulcus (CoS), LO-1, and V3A/B were identified, which also showed significant correlation with the perceived level of glossiness. This result is consistent with the recent monkey studies that identified selective neural response to glossiness in the ventral visual pathway, except for V3A/B in the dorsal visual pathway, whose involvement in the processing of glossiness could be specific to the human visual system. Second, we investigated the cortical regions that were modulated by selective attention to glossiness (Experiment 2). The visual areas that showed higher activation to attention to glossiness than that to either form or orientation were identified as right hV4, right VO-2, and right V3A/B, which were commonly identified in Experiment 1. The results indicate that these commonly identified visual areas in the human visual cortex may play important roles in glossiness perception.

Brain regions involved in processing facial identity and expression are differentially selective for surface and edge information

Although different brain regions are widely considered to be involved in the recognition of facial identity and expression, it remains unclear how these regions process different properties of the visual image. Here, we ask how surface-based reflectance information and edge-based shape cues contribute to the perception and neural representation of facial identity and expression. Contrast-reversal was used to generate images in which normal contrast relationships across the surface of the image were disrupted, but edge information was preserved. In a behavioural experiment, contrast-reversal significantly attenuated judgements of facial identity, but only had a marginal effect on judgements of expression. An fMR-adaptation paradigm was then used to ask how brain regions involved in the processing of identity and expression responded to blocks comprising all normal, all contrast-reversed, or a mixture of normal and contrast-reversed faces. Adaptation in the posterior superior temporal sulcus--a region directly linked with processing facial expression--was relatively unaffected by mixing normal with contrast-reversed faces. In contrast, the response of the fusiform face area--a region linked with processing facial identity--was significantly affected by contrast-reversal. These results offer a new perspective on the reasons underlying the neural segregation of facial identity and expression in which brain regions involved in processing invariant aspects of faces, such as identity, are very sensitive to surface-based cues, whereas regions involved in processing changes in faces, such as expression, are relatively dependent on edge-based cues.

Representation of the material properties of objects in the visual cortex of nonhuman primates

Information about the material from which objects are made provide rich and useful clues that enable us to categorize and identify those objects, know their state (e.g., ripeness of fruits), and properly act on them. However, despite its importance, little is known about the neural processes that underlie material perception in nonhuman primates. Here we conducted an fMRI experiment in awake macaque monkeys to explore how information about various real-world materials is represented in the visual areas of monkeys, how these neural representations correlate with perceptual material properties, and how they correspond to those in human visual areas that have been studied previously. Using a machine-learning technique, the representation in each visual area was read out from multivoxel patterns of regional activity elicited in response to images of nine real-world material categories (metal, wood, fur, etc.). The congruence of the neural representations with either a measure of low-level image properties, such as spatial frequency content, or with the visuotactile properties of materials, such as roughness, hardness, and warmness, were tested. We show that monkey V1 shares a common representation with human early visual areas reflecting low-level image properties. By contrast, monkey V4 and the posterior inferior temporal cortex represent the visuotactile properties of material, as in human ventral higher visual areas, although there were some interspecies differences in the representational structures. We suggest that, in monkeys, V4 and the posterior inferior temporal cortex are important stages for constructing information about the material properties of objects from their low-level image features.

The representation of material categories in the brain

Using textures mapped onto virtual nonsense objects, it has recently been shown that early visual cortex plays an important role in processing material properties. Here, we examined brain activation to photographs of materials, consisting of wood, stone, metal and fabric surfaces. These photographs were close-ups in the sense that the materials filled the image. In the first experiment, observers categorized the material in each image (i.e., wood, stone, metal, or fabric), while in an fMRI-scanner. We predicted the assigned material category using the obtained voxel patterns using a linear classifier. Region-of-interest and whole-brain analyses demonstrated material coding in the early visual regions, with lower accuracies for more anterior regions. There was little evidence for material coding in other brain regions. In the second experiment, we used an adaptation paradigm to reveal additional brain areas involved in the perception of material categories. Participants viewed images of wood, stone, metal, and fabric, presented in blocks with images of either different material categories (no adaptation) or images of different samples from the same material category (material adaptation). To measure baseline activation, blocks with the same material sample were presented (baseline adaptation). Material adaptation effects were found mainly in the parahippocampal gyrus, in agreement with fMRI-studies of texture perception. Our findings suggest that the parahippocampal gyrus, early visual cortex, and possibly the supramarginal gyrus are involved in the perception of material categories, but in different ways. The different outcomes from the two studies are likely due to inherent differences between the two paradigms. A third experiment suggested, based on anatomical overlap between activations, that spatial frequency information is important for within-category material discrimination.

Encoding-Stage Crosstalk Between Object- and Spatial Property-Based Scene Processing Pathways

Scene categorization draws on 2 information sources: The identities of objects scenes contain and scenes' intrinsic spatial properties. Because these resources are formally independent, it is possible for them to leads to conflicting judgments of scene category. We tested the hypothesis that the potential for such conflicts is mitigated by a system of "crosstalk" between object- and spatial layout-processing pathways, under which the encoded spatial properties of scenes are biased by scenes' object contents. Specifically, we show that the presence of objects strongly associated with a given scene category can bias the encoded spatial properties of scenes containing them toward the average of that category, an effect which is evident both in behavioral measures of scenes' perceived spatial properties and in scene-evoked multivoxel patterns recorded with functional magnetic resonance imaging from the parahippocampal place area (PPA), a region associated with the processing of scenes' spatial properties. These results indicate that harmonization of object- and spatial property-based estimates of scene identity begins when spatial properties are encoded, and that the PPA plays a central role in this process.

Altered spontaneous brain activity in patients with Parkinson's disease accompanied by depressive symptoms, as revealed by regional homogeneity and functional connectivity in the prefrontal-limbic system

As patients with Parkinson’s disease (PD) are at high risk for comorbid depression, it is hypothesized that these two diseases are sharing common pathogenic pathways. Using regional homogeneity (ReHo) and functional connectivity approaches, we characterized human regional brain activity at resting state to examine specific brain networks in patients with PD and those with PD and depression (PDD). This study comprised 41 PD human patients and 25 normal human subjects. The patients completed the Hamilton Depression Rating Scale and were further divided into two groups: patients with depressive symptoms and non-depressed PD patients (nD-PD). Compared with the non-depressed patients, those with depressive symptoms exhibited significantly increased regional activity in the left middle frontal gyrus and right inferior frontal gyrus, and decreased ReHo in the left amygdala and bilateral lingual gyrus. Brain network connectivity analysis revealed decreased functional connectivity within the prefrontal-limbic system and increased functional connectivity in the prefrontal cortex and lingual gyrus in PDD compared with the nD-PD group. In summary, the findings showed regional brain activity alterations and disruption of the mood regulation network in PDD patients. The pathogenesis of PDD may be attributed to abnormal neural activity in multiple brain regions.

Frequency-based heuristics for material perception

People often make rapid visual judgments of the properties of surfaces they are going to walk on or touch. How do they do this when the interactions of illumination geometry with 3-D material structure and object shape result in images that inverse optics algorithms cannot resolve without externally imposed constraints? A possibly effective strategy would be to use heuristics based on information that can be gleaned rapidly from retinal images. By using perceptual scaling of a large sample of images, combined with correspondence and canonical correlation analyses, we discovered that material properties, such as roughness, thickness, and undulations, are characterized by specific scales of luminance variations. Using movies, we demonstrate that observers' percepts of these 3-D qualities vary continuously as a function of the relative energy in corresponding 2-D frequency bands. In addition, we show that judgments of roughness, thickness, and undulations are predictably altered by adaptation to dynamic noise at the corresponding scales. These results establish that the scale of local 3-D structure is critical in perceiving material properties, and that relative contrast at particular spatial frequencies is important for perceiving the critical 3-D structure from shading cues, so that cortical mechanisms for estimating material properties could be constructed by combining the parallel outputs of sets of frequency-selective neurons. These results also provide methods for remote sensing of material properties in machine vision, and rapid synthesis, editing and transfer of material properties for computer graphics and animation.

Incongruent object/context relationships in visual scenes: where are they processed in the brain?

Rapid object visual categorization in briefly flashed natural scenes is influenced by the surrounding context. The neural correlates underlying reduced categorization performance in response to incongruent object/context associations remain unclear and were investigated in the present study using fMRI. Participants were instructed to categorize objects in briefly presented scenes (exposure duration=100ms). Half of the scenes consisted of objects pasted in an expected (congruent) context, whereas for the other half, objects were embedded in incongruent contexts. Object categorization was more accurate and faster in congruent relative to incongruent scenes. Moreover, we found that the two types of scenes elicited different patterns of cerebral activation. In particular, the processing of incongruent scenes induced increased activations in the parahippocampal cortex, as well as in the right frontal cortex. This higher activity may indicate additional neural processing of the novel (non experienced) contextual associations that were inherent to the incongruent scenes. Moreover, our results suggest that the locus of object categorization impairment due to contextual incongruence is in the right anterior parahippocampal cortex. Indeed in this region activity was correlated with the reaction time increase observed with incongruent scenes. Representations for associations between objects and their usual context of appearance might be encoded in the right anterior parahippocampal cortex.

Two visual streams: Interconnections do not imply duplication of function

Abstract Schenk and McIntosh (S&M) provide a useful review of the perception-action model (PAM), highlighting some of the gaps that need to be filled, and counteracting the erroneous belief held by some that the PAM implies two mutually independent streams. Although we agree with S&M's contention that the functional independence of the two streams has been overestimated, we reject their speculation that "the specializations proposed may be relative rather than absolute." We argue that the contributions made by the two streams are quite distinct, and that establishing how they work together is the key to a full understanding of visually guided behavior.

Differential connectivity within the Parahippocampal Place Area

The Parahippocampal Place Area (PPA) has traditionally been considered a homogeneous region of interest, but recent evidence from both human studies and animal models has suggested that PPA may be composed of functionally distinct subunits. To investigate this hypothesis, we utilize a functional connectivity measure for fMRI that can estimate connectivity differences at the voxel level. Applying this method to whole-brain data from two experiments, we provide the first direct evidence that anterior and posterior PPA exhibit distinct connectivity patterns, with anterior PPA more strongly connected to regions in the default mode network (including the parieto-medial temporal pathway) and posterior PPA more strongly connected to occipital visual regions. We show that object sensitivity in PPA also has an anterior-posterior gradient, with stronger responses to abstract objects in posterior PPA. These findings cast doubt on the traditional view of PPA as a single coherent region, and suggest that PPA is composed of one subregion specialized for the processing of low-level visual features and object shape, and a separate subregion more involved in memory and scene context.

Tool manipulation knowledge is retrieved by way of the ventral visual object processing pathway

Using functional Magnetic Resonance Imaging (fMRI), we find that object manipulation knowledge is accessed by way of the ventral object processing pathway. We exploit the fact that parvocellular channels project to the ventral but not the dorsal stream, and show that increased neural responses for tool stimuli are observed in the inferior parietal lobule when those stimuli are visible only to the ventral object processing stream. In a control condition, tool-preferences were observed in a superior and posterior parietal region for stimuli titrated so as to be visible by the dorsal visual pathway. Functional connectivity analyses confirm the dissociation between sub-regions of parietal cortex according to whether their principal afferent input is via the ventral or dorsal visual pathway. These results challenge the 'Embodied Hypothesis of Tool Recognition', according to which tool identification critically depends on simulation of object manipulation knowledge. Instead, these data indicate that retrieval of object-associated manipulation knowledge is contingent on accessing the identity of the object, a process that is subserved by the ventral visual pathway.

Automatic processing of unattended object features by functional connectivity

Observers can selectively attend to object features that are relevant for a task. However, unattended task-irrelevant features may still be processed and possibly integrated with the attended features. This study investigated the neural mechanisms for processing both task-relevant (attended) and task-irrelevant (unattended) object features. The Garner paradigm was adapted for functional magnetic resonance imaging (fMRI) to test whether specific brain areas process the conjunction of features or whether multiple interacting areas are involved in this form of feature integration. Observers attended to shape, color, or non-rigid motion of novel objects while unattended features changed from trial to trial (change blocks) or remained constant (no-change blocks) during a given block. This block manipulation allowed us to measure the extent to which unattended features affected neural responses which would reflect the extent to which multiple object features are automatically processed. We did not find Garner interference at the behavioral level. However, we designed the experiment to equate performance across block types so that any fMRI results could not be due solely to differences in task difficulty between change and no-change blocks. Attention to specific features localized several areas known to be involved in object processing. No area showed larger responses on change blocks compared to no-change blocks. However, psychophysiological interaction (PPI) analyses revealed that several functionally-localized areas showed significant positive interactions with areas in occipito-temporal and frontal areas that depended on block type. Overall, these findings suggest that both regional responses and functional connectivity are crucial for processing multi-featured objects.

Activity patterns in the category-selective occipitotemporal cortex predict upcoming motor actions

Converging lines of evidence point to the occipitotemporal cortex (OTC) as a critical structure in visual perception. For instance, human functional magnetic resonance imaging (fMRI) has revealed a modular organisation of object-selective, face-selective, body-selective and scene-selective visual areas in the OTC, and disruptions to the processing within these regions, either in neuropsychological patients or through transcranial magnetic stimulation, can produce category-specific deficits in visual recognition. Here we show, using fMRI and pattern classification methods, that the activity in the OTC also represents how stimuli will be interacted with by the body--a level of processing more traditionally associated with the preparatory activity in sensorimotor circuits of the brain. Combining functional mapping of different OTC areas with a real object-directed delayed movement task, we found that the pre-movement spatial activity patterns across the OTC could be used to predict both the action of an upcoming hand movement (grasping vs. reaching) and the effector (left hand vs. right hand) to be used. Interestingly, we were able to extract this wide range of predictive movement information even though nearly all OTC areas showed either baseline-level or below baseline-level activity prior to action onset. Our characterisation of different OTC areas according to the features of upcoming movements that they could predict also revealed a general gradient of effector-to-action-dependent movement representations along the posterior-anterior OTC axis. These findings suggest that the ventral visual pathway, which is well known to be involved in object recognition and perceptual processing, plays a larger than previously expected role in preparing object-directed hand actions.

Crossmodal interactions of haptic and visual texture information in early sensory cortex

Both visual and haptic information add to the perception of surface texture. While prior studies have reported crossmodal interactions of both sensory modalities at the behavioral level, neuroimaging studies primarily investigated texture perception in separate visual and haptic paradigms. These experimental designs, however, only allowed to identify overlap in both sensory processing streams but no interaction of visual and haptic texture processing. By varying texture characteristics in a bimodal task, the current study investigated how these crossmodal interactions are reflected at the cortical level. We used fMRI to compare cortical activation in response to matching versus non-matching visual-haptic texture information. We expected that passive simultaneous presentation of matching visual-haptic input would be sufficient to induce BOLD responses graded with varying texture characteristics. Since no cognitive evaluation of the stimuli was required, we expected to find changes primarily at a rather early processing stage. Our results confirmed our assumptions by showing crossmodal interactions of visual-haptic texture information in early somatosensory and visual cortex. However, the nature of the crossmodal effects was slightly different in both sensory cortices. In early visual cortex, matching visual-haptic information increased the average activation level and induced parametric BOLD signal variations with varying texture characteristics. In early somatosensory cortex only the latter was true. These results challenge the notion that visual and haptic texture information is processed independently and indicate a crossmodal interaction of sensory information already at an early cortical processing stage.

Evidence for crossmodal interactions across depth on target localisation performance in a spatial array

Auditory stimuli are known to improve visual target recognition and detection when both are presented in the same spatial location. However, most studies have focused on crossmodal spatial congruency along the horizontal plane and the effects of audio-visual spatial congruency in depth (i.e., along the depth axis) are relatively less well understood. In the following experiments we presented a visual (face) or auditory (voice) target stimulus in a location on a spatial array which was either spatially congruent or incongruent in depth (i.e., positioned directly in front or behind) with a crossmodal stimulus. The participant's task was to determine whether a visual (experiments 1 and 3) or auditory (experiment 2) target was located in the foreground or background of this array. We found that both visual and auditory targets were less accurately located when crossmodal stimuli were presented from different, compared to congruent, locations in depth. Moreover, this effect was particularly found for visual targets located in the periphery, although spatial incongruency affected the location of auditory targets across both locations. The relative distance of the array to the observer did not seem to modulate this congruency effect (experiment 3). Our results add to the growing evidence for multisensory influences on search performance and extend these findings to the localisation of targets in the depth plane.

Neural selectivity and representation of gloss in the monkey inferior temporal cortex

When we view an object, its appearance depends in large part on specific surface reflectance properties; among these is surface gloss, which provides important information about the material composition of the object and the fine structure of its surface. To study how gloss is represented in the visual cortical areas related to object recognition, we examined the responses of neurons in the inferior temporal (IT) cortex of the macaque monkey to a set of object images exhibiting various combinations of specular reflection, diffuse reflection, and roughness, which are important physical parameters of surface gloss. We found that there are neurons in the lower bank of the superior temporal sulcus that selectively respond to specific gloss. This neuronal selectivity was largely maintained when the shape or illumination of the object was modified and perceived glossiness was unchanged. By contrast, neural responses were significantly altered when the pixels of the images were randomly rearranged, and perceived glossiness was dramatically changed. The stimulus preference of these neurons differed from cell to cell, and, as a population, they systematically represented a variety of surface glosses. We conclude that, within the visual cortex, there are mechanisms operating to integrate local image features and extract information about surface gloss and that this information is systematically represented in the IT cortex, an area playing an important role in object recognition.

Neural correlates of changes in a visual search task due to cognitive training in seniors

This study aimed to elucidate the underlying neural sources of near transfer after a multidomain cognitive training in older participants in a visual search task. Participants were randomly assigned to a social control, a no-contact control and a training group, receiving a 4-month paper-pencil and PC-based trainer guided cognitive intervention. All participants were tested in a before and after session with a conjunction visual search task. Performance and event-related potentials (ERPs) suggest that the cognitive training improved feature processing of the stimuli which was expressed in an increased rate of target detection compared to the control groups. This was paralleled by enhanced amplitudes of the frontal P2 in the ERP and by higher activation in lingual and parahippocampal brain areas which are discussed to support visual feature processing. Enhanced N1 and N2 potentials in the ERP for nontarget stimuli after cognitive training additionally suggest improved attention and subsequent processing of arrays which were not immediately recognized as targets. Possible test repetition effects were confined to processes of stimulus categorisation as suggested by the P3b potential. The results show neurocognitive plasticity in aging after a broad cognitive training and allow pinpointing the functional loci of effects induced by cognitive training.

Object ensemble processing in human anterior-medial ventral visual cortex

Our visual system can extract summary statistics from large collections of similar objects without forming detailed representations of the individual objects in the ensemble. Such object ensemble representation is adaptive and allows us to overcome the capacity limitation associated with representing specific objects. Surprisingly, little is known about the neural mechanisms supporting such object ensemble representation. Here we showed human observers identical photographs of the same object ensemble, different photographs depicting the same ensemble, or different photographs depicting different ensembles. We observed fMRI adaptation in anterior-medial ventral visual cortex whenever object ensemble statistics repeated, even when local image features differed across photographs. Interestingly, such object ensemble processing is closely related to texture and scene processing in the brain. In contrast, the lateral occipital area, a region involved in object-shape processing, showed adaptation only when identical photographs were repeated. These results provide the first step toward understanding the neural underpinnings of real-world object ensemble representation.

Selective responses to specular surfaces in the macaque visual cortex revealed by fMRI

The surface properties of objects, such as gloss, transparency and texture, provide important information about the material characteristics of objects in our visual environment. However, because there have been few reports on the neuronal responses to surface properties in primates, we still lack information about where and how surface properties are processed in the primate visual cortex. In this study, we used functional magnetic resonance imaging (fMRI) to examine the cortical responses to specular surfaces in the macaque visual cortex. Using computer graphics, we generated images of specular and matte objects and prepared scrambled images by locally randomizing the luminance phases of the images with specular and matte objects. In experiment 1, we contrasted the responses to specular images with those to matte and scrambled images. Activation was observed along the ventral visual pathway, including V1, V2, V3, V4 and the posterior inferior temporal (IT) cortex. In experiment 2, we manipulated the contrasts of images and found that the activation observed in these regions could not be explained solely by the global or local contrasts. These results suggest that image features related to specular surface are processed along the ventral visual pathway from V1 to specific regions in the IT cortex. This is consistent with previous human fMRI experiments that showed surface properties are processed in the ventral visual pathway.

Frontoparietal involvement in passively guided shape and length discrimination: a comparison between subcortical stroke patients and healthy controls

Fifty to 85 % of patients with sensorimotor hemiparesis following stroke encounter impaired tactile processing and proprioception. Sensory feedback is, however, paramount for motor recovery. Sensory feedback through passively guided somatosensory discrimination exercises has been used in therapy, but so far, no studies have investigated which brain areas are involved in this process. Therefore, we performed a study with functional magnetic resonance imaging (fMRI) to examine brain areas related to discriminating passively guided shape and length discrimination in stroke patients and evaluate whether they differed from healthy age-matched controls. Eight subcortical stroke patients discriminated different shapes or length based on passive finger movements provided by an fMRI compatible robot. The data were contrasted to a control condition whereby patients discriminated music fragments. Passively guided somatosensory discrimination versus music discrimination elicited activation in similar frontoparietal areas in stroke patients compared to the healthy control group. Still, patients had increased activation in the right angular gyrus, left superior lingual gyrus, and right cerebellar lobule VI compared to healthy volunteers. Conversely, healthy volunteers activated the right precentral gyrus to a greater extent than patients. In both groups, shape discrimination resulted in anterior intraparietal sulcus and premotor activation, while length discrimination elicited a more medially located parietal activation with mainly right-sided premotor activity. The current study is a first step in clarifying brain activations during passively guided shape and length discrimination in subcortical stroke patients. Research into the effects of the use of sensory discrimination exercises on brain reorganization and brain plasticity is encouraged.

Micro-valences: perceiving affective valence in everyday objects

Perceiving the affective valence of objects influences how we think about and react to the world around us. Conversely, the speed and quality with which we visually recognize objects in a visual scene can vary dramatically depending on that scene’s affective content. Although typical visual scenes contain mostly “everyday” objects, the affect perception in visual objects has been studied using somewhat atypical stimuli with strong affective valences (e.g., guns or roses). Here we explore whether affective valence must be strong or overt to exert an effect on our visual perception. We conclude that everyday objects carry subtle affective valences – “micro-valences” – which are intrinsic to their perceptual representation.

Combination of texture and color cues in visual segmentation

The visual system can use various cues to segment the visual scene into figure and background. We studied how human observers combine two of these cues, texture and color, in visual segmentation. In our task, the observers identified the orientation of an edge that was defined by a texture difference, a color difference, or both (cue combination). In a fourth condition, both texture and color information were available, but the texture and color edges were not spatially aligned (cue conflict). Performance markedly improved when the edges were defined by two cues, compared to the single-cue conditions. Observers only benefited from the two cues, however, when they were spatially aligned. A simple signal-detection model that incorporates interactions between texture and color processing accounts for the performance in all conditions. In a second experiment, we studied whether the observers are able to ignore a task-irrelevant cue in the segmentation task or whether it interferes with performance. Observers identified the orientation of an edge defined by one cue and were instructed to ignore the other cue. Three types of trial were intermixed: neutral trials, in which the second cue was absent; congruent trials, in which the second cue signaled the same edge as the target cue; and conflict trials, in which the second cue signaled an edge orthogonal to the target cue. Performance improved when the second cue was congruent with the target cue. Performance was impaired when the second cue was in conflict with the target cue, indicating that observers could not discount the second cue. We conclude that texture and color are not processed independently in visual segmentation.

Neural correlates of visual aesthetics--beauty as the coalescence of stimulus and internal state

How do external stimuli and our internal state coalesce to create the distinctive aesthetic pleasures that give vibrance to human experience? Neuroaesthetics has so far focused on the neural correlates of observing beautiful stimuli compared to neutral or ugly stimuli, or on neural correlates of judging for beauty as opposed to other judgments. Our group questioned whether this approach is sufficient. In our view, a brain region that assesses beauty should show beauty-level-dependent activation during the beauty judgment task, but not during other, unrelated tasks. We therefore performed an fMRI experiment in which subjects judged visual textures for beauty, naturalness and roughness. Our focus was on finding brain activation related to the rated beauty level of the stimuli, which would take place exclusively during the beauty judgment. An initial whole-brain analysis did not reveal such interactions, yet a number of the regions showing main effects of the judgment task or the beauty level of stimuli were selectively sensitive to beauty level during the beauty task. Of the regions that were more active during beauty judgments than roughness judgments, the frontomedian cortex and the amygdala demonstrated the hypothesized interaction effect, while the posterior cingulate cortex did not. The latter region, which only showed a task effect, may play a supporting role in beauty assessments, such as attending to one's internal state rather than the external world. Most of the regions showing interaction effects of judgment and beauty level correspond to regions that have previously been implicated in aesthetics using different stimulus classes, but based on either task or beauty effects alone. The fact that we have now shown that task-stimulus interactions are also present during the aesthetic judgment of visual textures implies that these areas form a network that is specifically devoted to aesthetic assessment, irrespective of the stimulus type.