Functional imaging of human visual recognition

Class Information Predicts Activation by Object Fragments in Human Object Areas

Object-related areas in the ventral visual system in humans are known from imaging studies to be preferentially activated by object images compared with noise or texture patterns. It is unknown, however, which features of the object images are extracted and represented in these areas. Here we tested the extent to which the representation of visual classes used object fragments selected by maximizing the information delivered about the class. We tested functional magnetic resonance imaging blood oxygenation level-dependent activation of highly informative object features in low- and high-level visual areas, compared with noninformative object features matched for low-level image properties. Activation in V1 was similar, but in the lateral occipital area and in the posterior fusiform gyrus, activation by “informative” fragments was significantly higher for three object classes. Behavioral studies also revealed high correlation between performance and fragments information. The results show that an objective class-information measure can predict classification performance and activation in human object-related areas.

fMRI mapping of a morphed continuum of 3D shapes within inferior temporal cortex

Here, we mapped fMRI responses to incrementally changing shapes along a continuous 3D morph, ranging from a head ("face") to a house ("place"). The response to each shape was mapped independently by using single-stimulus imaging, and stimulus shapes were equated for lower-level visual cues. We measured activity in 2-mm samples across human inferior temporal cortex from the fusiform face area (FFA) (apparently selective for faces) to the parahippocampal place area (PPA) (apparently selective for places), testing for (i) incremental changes in the topography of FFA and PPA (predicted by the continuous-mapping model) or (ii) little or no response to the intermediate morphed shapes (predicted by the category model). Neither result occurred; instead, we found approximately linearly graded changes in the response amplitudes to graded-shape changes, without changes in topography-similar to visual responses in different lower-tier cortical areas.

Two hierarchically organized neural systems for object information in human visual cortex

The primate visual system is broadly organized into two segregated processing pathways, a ventral stream for object vision and a dorsal stream for space vision. Here, evidence from functional brain imaging in humans demonstrates that object representations are not confined to the ventral pathway, but can also be found in several areas along the dorsal pathway. In both streams, areas at intermediate processing stages in extrastriate cortex (V4, V3A, MT and V7) showed object-selective but viewpoint- and size-specific responses. In contrast, higher-order areas in lateral occipital and posterior parietal cortex (LOC, IPS1 and IPS2) responded selectively to objects independent of image transformations. Contrary to the two-pathways hypothesis, our findings indicate that basic object information related to shape, size and viewpoint may be represented similarly in two parallel and hierarchically organized neural systems in the ventral and dorsal visual pathways.

Changes in hemoglobin concentration in the lateral occipital regions during shape recognition: a near-infrared spectroscopy study

By using near-infrared spectroscopy (NIRS), we measured the changes in the oxygenated and deoxygenated hemoglobin (oxy-Hb and deoxy-Hb, respectively) concentrations while performing visual tasks. We conducted experiments using two tasks: a shape recognition task and a position recognition task. It was found that the oxy-Hb concentration was substantially higher in the lateral occipital regions during shape recognition than during position recognition. The changes in the oxy-Hb concentration were considered to reflect the activation difference between the two tasks. No difference was observed in the oxy-Hb concentration during the memorization of shape and memorization of position. The deoxy-Hb concentration was different between the two tasks only when different stimuli were used but not when identical stimuli were used. In addition, it was suggested that the deoxy-Hb concentration is more sensitive to activation difference between the hemispheres and the activation at some regions. Measurements of the oxy-Hb and deoxy-Hb concentrations would reflect different aspects of cortical activations. The present results showed that measuring the oxy-Hb and deoxy-Hb concentrations separately can differentiate the activation of the regional cortical functions.

Simultanagnosia: effects of semantic category and repetition blindness

When confronted with two identical stimuli in a very brief period of time subjects often fail to report the second stimulus, a phenomenon termed "repetition blindness". The "type-token" account attributes the phenomenon to a failure to individuate the exemplars. We report a subject, KE, who developed simultanagnosia (the inability to see more than one item in an array) as a consequence of bilateral parietal lobe infarctions. With presentation of two words, pictures or letters for an unlimited time, KE typically reported both stimuli on less than half of trials. Performance was significantly influenced by the semantic relationship between items in the array. He reported both items significantly more frequently if they were semantically related; in contrast, when presented either identical or visually different depictions of the same item, he reported both items on only 2-4% of trials. Performance was not influenced by the visual similarity between the stimuli; he reported visually dissimilar objects less frequently than visually similar but different objects. We suggest that KE's bilateral parietal lesions prevent the binding of preserved object representations to a representation computed by the dorsal visual system. More generally, these data are consistent with the claim that the posterior parietal cortex is crucial for individuating a stimulus by computing its unique spatio-temporal characteristics.

Timing of the earliest ERP correlate of visual awareness

The earliest reliably occurring event-related brain potential (ERP) correlate of visual awareness (visual awareness negativity, VAN) emerges after 100 ms and peaks between 200 and 300 ms from stimulus onset. In a study using low-contrast stimuli, VAN was significantly delayed, peaking at 460 ms (V. Ojanen, A. Revonsuo, & M. Sams, 2003). In that study physical differences between the conscious and nonconscious stimuli may have confounded the results. Here we explored whether VAN is similarly delayed for physically identical stimuli. We presented low-contrast stimuli near an individually determined subjective contrast threshold. A delayed VAN peaked at 400 ms at occipito-temporal sites to subjectively perceived stimuli. Our results support the interpretation that VAN is the earliest ERP correlate of phenomenal visual awareness. The electrophysiological processes eliciting VAN may become delayed as a function of the difficulty of the early perceptual discrimination.

Processing the socially relevant parts of faces

Faces are processed by a distributed neural system in the visual as well as in the non-visual cortex [the "core" and the "extended" systems, J.V. Haxby, E.A. Hoffman, M.I. Gobbini, The distributed human neural system for face perception, Trends Cogn. Sci. 4 (2000) 223-233]. Yet, the functions of the different brain regions included in the face processing system are far from clear. On the basis of the case study of a patient unable to recognize fearful faces, Adolphs et al. [R. Adolphs, F. Gosselin, T.W. Buchanan, D. Tranel, P. Schyns, A.R. Damasio, A mechanism for impaired fear recognition after amygdala damage, Nature 433 (2005) 68-72] suggested that the amygdala might play a role in orienting attention towards the eyes, i.e. towards the region of face conveying most information about fear. In a functional magnetic resonance (fMRI) study comparing patterns of activation during observation of whole faces and parts of faces displaying neutral expressions, we evaluated the neural systems for face processing when only partial information is provided, as well as those involved in processing two socially relevant facial areas (the eyes and the mouth). Twenty-four subjects were asked to perform a gender decision task on pictures showing whole faces, upper faces (eyes and eyebrows), and lower faces (mouth). Our results showed that the amygdala was activated more in response to the whole faces than to parts of faces, indicating that the amygdala is involved in orienting attention toward eye and mouth. Processing of parts of faces in isolation was found to activate other regions within both the "core" and the "extended" systems, as well as structures outside this network, thus suggesting that these structures are involved in building up the representation of the whole face from its parts.

Quality of life and the evolution of the brain

The dual problem of explaining brain evolution and the way in which it has led to wide species differences in behaviour and physiology has often appeared intractable to scientists. The main limiting factor is that we do not understand enough about how brains work to appreciate why gross or fine morphological differences can lead to the considerable across- as well as within-species differences in behaviour. Even at a molecular level, while two-thirds of our genes are involved in regulating brain function, there is a high degree of homology within different phyla. In the context of quality of life (QoL), arguably the most important consideration is that the brain you have evolved is adapted to the environment you are living in and is capable of generating ‘conscious’ experience. When that environment is radically altered, issues arise regarding whether there is sufficient adaptability to cope and the extent to which mental as well as physical suffering might be experienced as a consequence. At the other end of the spectrum there is the question of how enriched social and physical environments might enhance QoL through promoting positive affect. Here I will discuss potential functional contributions of differences in brain size and organisation and the impact of experience. I will mainly focus on mental functioning and show particularly that capacities for consciousness, emotional experience, social interaction and cognition and behavioural flexibility are likely to be widespread in other animal species, even if less developed than in humans.

Contour discontinuities subserve two types of form analysis that underlie motion processing

Form analysis subserves motion processing in at least two ways: first, in terms of figural segmentation dedicated to solving the problem of figure-to-figure matching over time, and second, in terms of defining trackable features whose unambiguous motion signals can be generalized to ambiguously moving portions of an object. The former is a primarily ventral process involving the lateral occipital complex and also retinotopic areas such as V2 and V4, and the latter is a dorsal process involving V3A. Contour discontinuities, such as corners, deep concavities, maxima of positive curvature, junctions, and terminators, play a central role in both types of form analysis. Transformational apparent motion will be discussed in the context of figural segmentation and matching, and rotational motion in the context of trackable features. In both cases the analysis of form must proceed in parallel with the analysis of motion, in order to constrain the ongoing analysis of motion.

Task-related laterality effects in the lateral occipital complex

Using functional imaging, we investigated the effects of two different tasks on activation in the lateral occipital complex (LOC). Alternating blocks of intact and scrambled objects were presented. In one task, subjects responded when an object repeated (matching task). In a second task subjects silently named objects (naming task). Identical objects (tools, animals and letters) were presented for both tasks. A relative measure of the number of voxels activated in LOC in left and right hemispheres was calculated for each task across a range of thresholds. Also the effects of task demands on category specific areas in LOC were examined. The object matching task resulted in proportionally more activity in the right hemisphere. The object naming task resulted in proportionally more activity in the left hemisphere, most prominently in the anterior portion of LOC. Effectively, changing the task changed the lateralization of activation to intact objects in LOC. In contrast, changing the task did not change the lateralization of category-specific activations. The results suggest that there are task-related top-down influences on the activation of neural populations in LOC as a whole, but the lateralization of category-specific regions in LOC is independent of task demands and may reflect bottom-up processing.

Neural correlates of monocular and binocular depth cues based on natural images: A LORETA analysis

Functional imaging studies investigating perception of depth rely solely on one type of depth cue based on non-natural stimulus material. To overcome these limitations and to provide a more realistic and complete set of depth cues natural stereoscopic images were used in this study. Using slow cortical potentials and source localization we aimed to identify the neural correlates of monocular and binocular depth cues. This study confirms and extends functional imaging studies, showing that natural images provide a good, reliable, and more realistic alternative to artificial stimuli, and demonstrates the possibility to separate the processing of different depth cues.

Concurrent visuomotor behaviour improves form discrimination in a patient with visual form agnosia

It is now well established that the visual brain is divided into two visual streams, the ventral and the dorsal stream. Milner and Goodale have suggested that the ventral stream is dedicated for processing vision for perception and the dorsal stream vision for action [A.D. Milner & M.A. Goodale (1995) The Visual Brain in Action, Oxford University Press, Oxford]. However, it is possible that ongoing processes in the visuomotor stream will nevertheless have an effect on perceptual processes. This possibility was examined in the present study. We have examined the visual form-discrimination performance of the form-agnosic patient D.F. with and without a concurrent visuomotor task, and found that her performance was significantly improved in the former condition. This suggests that the visuomotor behaviour provides cues that enhance her ability to recognize the form of the target object. In control experiments we have ruled out proprioceptive and efferent cues, and therefore propose that D.F. can, to a significant degree, access the object's visuomotor representation in the dorsal stream. Moreover, we show that the grasping-induced perceptual improvement disappears if the target objects only differ with respect to their shape but not their width. This suggests that shape information per se is not used for this grasping task.

Uncertainty and invariance in the human visual cortex

The way in which input noise perturbs the behavior of a system depends on the internal processing structure of the system. In visual psychophysics, there is a long tradition of using external noise methods (i.e., adding noise to visual stimuli) as tools for system identification. Here, we demonstrate that external noise affects processing of visual scenes at different cortical areas along the human ventral visual pathway, from retinotopic regions to higher occipitotemporal areas implicated in visual shape processing. We found that when the contrast of the stimulus was held constant, the further away from the retinal input a cortical area was the more its activity, as measured with functional magnetic resonance imaging (fMRI), depended on the signal-to-noise ratio (SNR) of the visual stimulus. A similar pattern of results was observed when trials with correct and incorrect responses were analyzed separately. We interpret these findings by extending signal detection theory to fMRI data analysis. This approach reveals the sequential ordering of decision stages in the cortex by exploiting the relation between fMRI response and stimulus SNR. In particular, our findings provide novel evidence that occipitotemporal areas in the ventral visual pathway form a cascade of decision stages with increasing degree of signal uncertainty and feature invariance.

Bistable illusory rebound motion: Event-related functional magnetic resonance imaging of perceptual states and switches

The neural correlates of a recently discovered visual illusion that we call 'illusory rebound motion' (IRM) are described. This illusion is remarkable because motion is perceived in the absence of any net motion energy in the stimulus. When viewing bars alternating between white and black on a gray background, the percept alternates between one of flashing bars (veridical) and the IRM illusion, where the bars appear to shoot back and forth rather like the opening and closing of a zipper. The event-related functional magnetic resonance imaging (fMRI) data reported here reveal that (1) the blood-oxygen-level-dependent (BOLD) signal in the human analog of macaque motion processing area MT (hMT+) increases when there is a perceptual change from "no-IRM" to "see-IRM" and decreases when there is a perceptual change from "see-IRM" to "no-IRM," although the stimulus remains constant; and (2) the BOLD signal in early retinotopic areas (V1, V2, and V3d) shows switch-related activation whenever there is a perceptual change, regardless whether from IRM to no-IRM or vice versa. We conclude that hMT+ is a neural correlate of this novel illusory motion percept because BOLD signal in hMT+ modulates with the perception of IRM.

Modulation of V1 Activity by Shape: Image-Statistics or Shape-Based Perception?

It is current dogma that neurons in primary visual cortex extract local edges from the scene from which later visual areas reconstruct more meaningful shapes. Recent neuroimaging studies, however, have shown V1 modulations by the degree of structure in the image (shape). These V1 modulations due to the level of shape coherence have been explained in one of two possible ways: due to changes in image statistics or shape-based perceptual influences from higher visual areas. Here we compare both hypotheses using stimuli composed of Gabor arrays constructed to form circular shapes that can be successively degraded by manipulating the orientations of individual Gabors while maintaining local and global statistics. In a first experiment, we confirm that V1 responses are inversely correlated with the degree of structure in the image. In a second experiment, stimulus predictions are compared based on the degree of circular shape or change in the image statistic varied (orientation variance) in the image. We find that these V1 modulations to shape change are correlated with low-level changes in orientation contrast rather than shape perception per se.

Neural correlates of transformational apparent motion

When a figure discretely and instantaneously changes its shape, observers typically do not perceive the abrupt transition between shapes that in fact occurs. Rather, a continuous shape change is perceived. Although this illusory "transformational apparent motion" (TAM) is a faulty construction of the visual system, it is not arbitrary. From the many possible shape changes that could have been inferred, usually just one is perceived because only one is consistent with the shape-based rules that the visual system uses to (1) segment figures from one another within a scene and (2) match figures to themselves across successive scenes. TAM requires an interaction between neuronal circuits that process form relationships with circuits that compute motion trajectories. In particular, this form-motion interaction must happen before TAM is perceived because the direction of perceived motion is dictated by form relationships among figures in successive images. The present fMRI study (n = 19) provides the first evidence that both form (LOC, posterior fusiform gyrus) and motion (hMT+) processing areas are more active when TAM is perceived than in a control stimulus where it is not. Retinotopic areas (n = 10), hMT+ (n = 7), and LOC (n = 7) were mapped in a subset of subjects.

RESULTS

There is greater BOLD response to TAM than to the control condition in V1 and all subsequent retinotopic areas, as well as in hMT+ and the LOC, suggesting that areas that process form interact with hMT+ to construct the perception of moving figures.

Time-resolved fMRI of mental rotation revisited--dissociating visual perception from mental rotation in female subjects

Functional neuroimaging studies have demonstrated that mental rotation paradigms activate a network of spatially distributed cortical areas rather than a discrete brain region. Although the neuro-anatomical nodes of the rotation network are well established, their specific functional role is less well identified. It was the aim of the present study to dissociate network components involved in the visual perception of 3D cubic objects from regions involved in their mental spatial transformation. This was achieved by desynchronizing the time course of the perceptional process (i.e., stimulus duration) from the duration of the cognitive process (i.e., reaction times) and by comparing these with the temporal characteristics of the hemodynamic response functions (HRFs) in regions of interest. To minimize intersubject variability, an all-female subject group was chosen for this investigation. Time-resolved fMRI analysis revealed a significant increase in the full width at half maximum (FWHM) of the HRF with reaction times in the supplementary motor area (pre-SMA), in the bilateral premotor cortex (PMd-proper), and in the left parietal lobe (PP). The FWHM in visual system components such as the bilateral lateral occipital complex (LOC) and dorsal extrastriate visual areas (DE) was constant across trials and roughly equal to the stimulus duration. These findings suggest that visual system activation during mental rotation reflects visual perception and can be dissociated from other network components whose response characteristics indicates an involvement in the mental spatial transformation itself.

Attention to Form or Surface Properties Modulates Different Regions of Human Occipitotemporal Cortex

We carried out 2 functional magnetic resonance imaging experiments to investigate the cortical mechanisms underlying the contribution of form and surface properties to object recognition. In experiment 1, participants performed same-different judgments in separate blocks of trials on pairs of unfamiliar "nonsense" objects on the basis of their form, surface properties (i.e., both color and texture), or orientation. Attention to form activated the lateral occipital (LO) area, whereas attention to surface properties activated the collateral sulcus (CoS) and the inferior occipital gyrus (IOG). In experiment 2, participants were required to make same-different judgments on the basis of texture, color, or form. Again attention to form activated area LO, whereas attention to texture activated regions in the IOG and the CoS, as well as regions in the lingual sulcus and the inferior temporal sulcus. Within these last 4 regions, activation associated with texture was higher than activation associated with color. No color-specific cortical areas were identified in these regions, although parts of V1 and the cuneus yielded higher activation for color as opposed to texture. These results suggest that there are separate form and surface-property pathways in extrastriate cortex. The extraction of information about an object's color seems to occur relatively early in visual analysis as compared with the extraction of surface texture, perhaps because the latter requires more complex computations.

Memory in Early Adolescents Born Prematurely: A Functional Magnetic Resonance Imaging Investigation

This study employed functional magnetic resonance imaging to examine the functional neuroanatomy of the hippocampus and head of the caudate nucleus during 2 different types of memory tasks in a sample of 9 early adolescent children who were born preterm (neonatal intensive care unit [NICU] sample) and a group of 9 age-matched control children who were born at term. The investigation employed delayed match to sample (DMS), delayed nonmatch to sample (DNMS), and spatial memory span tasks, as well as 2 analogous perceptuomotor tasks that placed no demands on memory. The general question examined was whether preterm children show different levels of hippocampal and caudate activation during these tasks when compared to children born at term. The findings indicated that the 2 groups did not differ in functional activation of the hippocampus during the DMS and DNMS tasks. During the encoding phase of the spatial memory span task, the DMS perceptuomotor task, and the spatial memory span perceptuomotor task, the NICU sample showed greater activation change in the right caudate nucleus, and less right caudate activation change during the test phase. During the spatial span perceptuomotor task, the preterm group showed reduced activation change in the left caudate nucleus during both the encoding and test phase. Also, during the DMS perceptuomotor task, the NICU group showed increased activation change in the left caudate nucleus during encoding and decreased activation change at test. The implications of these findings for understanding the functional neuroanatomy of memory deficits are discussed, as is the potential for distinguishing the effects of neural plasticity from those of typical brain maturational processes.

Location and spatial profile of category-specific regions in human extrastriate cortex

Subjects were scanned in a single functional MRI (fMRI) experiment that enabled us to localize cortical regions in each subject in the occipital and temporal lobes that responded significantly in a variety of contrasts: faces>objects, body parts>objects, scenes>objects, objects>scrambled objects, and moving>stationary stimuli. The resulting activation maps were co-registered across subjects using spherical surface coordinates [Fischl et al., Hum Brain Mapp 1999;8:272-284] to produce a "percentage overlap map" indicating the percentage of subjects who showed a significant response for each contrast at each point on the surface. Prominent among the overlapping activations in these contrasts were the fusiform face area (FFA), extrastriate body area (EBA), parahippocampal place area (PPA), lateral occipital complex (LOC), and MT+/V5; only a few other areas responded consistently across subjects in these contrasts. Another analysis showed that the spatial profile of the selective response drops off quite sharply outside the standard borders of the FFA and PPA (less so for the EBA and MT+/V5), indicating that these regions are not simply peaks of very broad selectivities spanning centimeters of cortex, but fairly discrete regions of cortex with distinctive functional profiles. The data also yielded a surprise that challenges our understanding of the function of area MT+: a higher response to body parts than to objects. The anatomical consistency of each of our functionally defined regions across subjects and the spatial sharpness of their activation profiles within subjects highlight the fact that these regions constitute replicable and distinctive landmarks in the functional organization of the human brain.

Exploring the neural organization of person-related knowledge: Lateralization of lesion, category specificity, and stimulus modality effects

While it is generally agreed that the right fusiform gyrus is specialized for face recognition, the question of whether knowledge about persons is lateralized in the temporal lobes is more contentious. Does knowledge about people differ from other kinds of object knowledge with respect to brain laterality? Are side-of-lesion effects mediated by stimulus modality? This study aimed to investigate these questions by comparing patients with left temporal (LT) (n=8) and right temporal (RT) (n=11) lesions to control subjects (n=12) on verbal and visual tests of people, buildings, and objects. The RT group was impaired at recognizing famous faces, but not at choosing the picture of a famous building or a famous name from nonfamous distracters. The LT group was impaired at naming people, buildings, and objects, regardless of stimulus modality. When presemantic processing was controlled for, neither patient group was impaired in producing person-specific knowledge to faces or names, supporting the notion that semantic knowledge for people as for other kinds of objects, is stored in a distributed network across both hemispheres, regardless of stimulus modality.

The Neural Basis of the Behavioral Face-Inversion Effect

Two of the most robust markers for "special" face processing are the behavioral face-inversion effect (FIE)-the disproportionate drop in recognition of upside-down (inverted) stimuli relative to upright faces-and the face-selective fMRI response in the fusiform face area (FFA). However, the relationship between these two face-selective markers is unknown. Here we report that the behavioral FIE is closely associated with the fMRI response in the FFA, but not in other face-selective or object-selective regions. The FFA and the face-selective region in the superior temporal sulcus (f_STS), but not the occipital face-selective region (OFA), showed a higher response to upright than inverted faces. However, only in the FFA was this fMRI-FIE positively correlated across subjects with the behavioral FIE. Second, the FFA, but not the f_STS, showed greater neural sensitivity to differences between faces when they were upright than inverted, suggesting a possible neural mechanism for the behavioral FIE. Although a similar trend was found in the occipital face area (OFA), it was less robust than the FFA. Taken together, our data suggest that among the face-selective and object-selective regions, the FFA is a primary neural source of the behavioral FIE.

Mapping multiple visual areas in the human brain with a short fMRI sequence

It is a fundamental insight of neuroscience that the cerebral cortex is divided into spatially separated and functionally distinct areas. In this study, we tried to map a large number of visual areas in individual subjects passively viewing a simple stimulus sequence during functional magnetic resonance imaging (fMRI) at 1.5 T. The blocked stimulus sequence contrasted static object photographs with video takes of movement through natural indoor and outdoor scenes, alternated with a control fixation task. Two runs of the 5-min sequence sufficed to invoke 29 distinguishable activations, 16 (13 bilateral) of which were observed in all 10 participants. At the ventral side, object responsive activations were organized along the lateral occipital-temporal surface and near the collateral and occipital-temporal sulci. The latter activations, corresponding to the lateral occipital complex, showed a different activation profile from those near the collateral sulcus, most likely corresponding to the color constancy areas V4/V8-V4alpha. A potentially new fusiform object area was seen in 6 subjects, even more anterior than the parahippocampal place area. At the dorsal side, consistent activations were mainly related to motion stimuli and included the well-known areas V3a, VIPS, POIPS, hV5+, STS and the cingulate sulcus. There was consistent activation in the parietal-occipital sulcus, containing the areas V6a and V6. In addition, all subjects showed activation in the superior-anterior precuneus. Thus, the short stimulus sequence robustly invoked multiple visual areas and can be used to map the organization of the visual system in normal and brain-damaged individuals.

Correlations of cortical activation and behavior during the application of newly learned categories

Large individual differences are commonly observed during the early stages of category learning in both functional MRI (fMRI) activation maps and behavioral data. The current investigation characterizes this variability by correlating the volume of activation with behavioral performance. Healthy subjects were trained to classify patterns of random dots into categories. Training was carried out using a 4-choice categorization task with feedback. Functional MRI was performed prior to any training and then following each of 3 training sessions. The fMRI sessions involved the presentation of 3 separate paradigms which required the skill imparted by the training to determine whether two patterns of dots belonged to the same category. Contrasts between the 3 paradigms allowed the examination of the effects of training and of familiarity with the task. For fMRI performed with those materials used during training, increases in the volume of activation were observed initially. As behavioral performance continued to improve, reductions in activation were observed across regions involved in visuospatial processing and spatial attention. These reductions in activation were observed only for those materials used in training and only after high levels of performance were achieved. The magnitude of these reductions in activation correlated with each individual's own rate of learning. The present data support the observation that at least two stages of cortical activation underlie the use of newly learned categories. The first, recruitment of nearby tissue, is observed as initial increases in the volumes of activation. These initial stages of recruitment are followed by specialization across the same network which is observed as a reduction in activation with continued improvements in behavioral performance.

Spatiotemporal characteristics of form analysis in the human visual cortex revealed by rapid event-related fMRI adaptation

The integration of local elements to coherent forms is at the core of understanding visual perception. Accumulating evidence suggests that both early retinotopic and higher occipitotemporal areas contribute to the integration of local elements to global forms. However, the spatiotemporal characteristics of form analysis in the human visual cortex remain largely unknown. The aim of this study was to investigate form analysis at different spatial (global vs. local structure) and temporal (different stimulus presentation rates) scales across stages of visual analysis (from V1 to the lateral occipital complex-LOC) in the human brain. We used closed contours rendered by Gabor elements and manipulated either the global contour structure or the orientation of the local Gabor elements. Our rapid event-related fMRI adaptation studies suggest that contour integration and form processing in early visual areas is transient and limited within the local neighborhood of their cells' receptive field. In contrast, higher visual areas appear to process the perceived global form in a more sustained manner. Finally, we demonstrate that these spatiotemporal properties of form processing in the visual cortex are modulated by attention. Attention to the global form maintains sustained processing in occipitotemporal areas, whereas attention to local elements enhances their integration in early visual areas. These findings provide novel neuroimaging evidence for form analysis at different spatiotemporal scales across human visual areas and validate the use of rapid event-related fMRI adaptation for investigating processing across stages of visual analysis in the human brain.

Temporal Characterization of the Neural Correlates of Perceptual Decision Making in the Human Brain

Single and multi-unit recordings in primates have identified spatially localized neuronal activity correlating with an animal's behavioral performance. Due to the invasive nature of these experiments, it has been difficult to identify such correlates in humans. We report the first non-invasive neural measurements of perceptual decision making, via single-trial EEG analysis, that lead to neurometric functions predictive of psychophysical performance for a face versus car categorization task. We identified two major discriminating components. The earliest correlating with psychophysical performance was consistent with the well-known face-selective N170. The second component, which was a better match to the psychometric function, did not occur until at least 130 ms later. As evidence for faces versus cars decreased, onset of the later, but not the earlier, component systematically shifted forward in time. In addition, a choice probability analysis indicated strong correlation between the neural responses of the later component and our subjects' behavioral judgements. These findings demonstrate a temporal evolution of component activity indicative of an evidence accumulation process which begins after early visual perception and has a processing time that depends on the strength of the evidence.

Interactions between the dorsal and the ventral pathways in mental rotation: an fMRI study

In this fMRI study, we examined the relationship between activations in the inferotemporal region (ventral pathway) and the parietal region (dorsal pathway), as well as in the prefrontal cortex (associated with working memory), in a modified mental rotation task. We manipulated figural complexity (simple vs. complex) to affect the figure recognition process (associated with the ventral pathway) and the amount of rotation (0 degrees vs. 90 degrees), typically associated with the dorsal pathway. The pattern of activation not only showed that both streams are affected by both manipulations, but also showed an overadditive interaction. The effect of figural complexity was greater for 90 degrees rotation than for 0 degrees in multiple regions, including the ventral, dorsal, and prefrontal regions. In addition, functional connectivity analyses on the correlations across the time courses of activation between regions of interest showed increased synchronization among multiple brain areas as task demand increased. The results indicate that both the dorsal and the ventral pathways show interactive effects of object and spatial processing, and they suggest that multiple regions interact to perform mental rotation.

Distinct and shared cortical regions of the human brain activated by pictorial depictions versus verbal descriptions: an fMRI study

Using fMRI, we observed that there were functionally disjunctive regions in the human brain that were specifically activated during the silent reading of sentences (i.e., the symbolical representation at the propositional level) but not during the perception of arranged objects (i.e., analogical representation), or vice versa: Parts of the left and right lingual gyri, the left fusiform gyrus, the left and right inferior occipital gyri, the right cuneus, and the left middle occipital gyrus were activated exclusively during the silent reading of sentences, whereas perception of the arranged objects activated distinct regions in the lingual gyrus, the declive, the fusiform gyrus, and the cuneus in the right hemisphere. A large proportion (86% in cortical volume) of the occipito-temporal regions was functionally conjunctive: these neural structures were activated during both silent reading of sentences and perception of arranged objects. We observed a similar trend of functional disjunction and conjunction between single words (the symbolical mode at the lexical level) and single objects (analogical mode): the degree of functional conjunction in the latter case was about 96%. These results suggest that the degree of functional disjunction between the pictorial depictions and the verbal descriptions tended to increase as the complexity of mental representation increased from the single word (lexical) level (4%) to the sentence (propositional) level (14%).

Changing patterns of brain activation during category learning revealed by functional MRI

Functional magnetic resonance imaging (fMRI) was used to investigate neural changes as a function of category learning in normals (n=8). Subjects were trained to classify patterns of dots into four categories over 4 consecutive days. fMRI monitored the changes that occurred during learning prior to training and then following each training session. During fMRI, subjects determined whether two patterns of dots were members of the same category. The behavioral changes that occurred as a result of the training were observed as increases in response accuracy within shortened response times. fMRI illustrated initial increases in volumes of activation distributed across the known visuospatial processing networks. The regions affected by learning were identified as those involved in the planning and execution of eye movements (frontal and supplementary eye fields, FEF and SEF), spatial attention (superior and inferior parietal lobules, SPL and IPL) and visual processing (primary, secondary, and tertiary visual cortices). The volumes of activation then decreased as training progressed further. Of the two proposed mechanisms for learning, that of strengthened connectivity on a given network and that of selection of different networks, our data supports the former.

Perceptual continuity and the emergence of perceptual persistence in the ventral visual pathway

Perceptual continuity is an important aspect of our experience of the visual world. In this study, we focus on an example of perceptual continuity involving the maintenance of figure-ground segregation despite the removal of binding cues that initiated the segregation. Fragmented line drawings of objects were superimposed on a background of randomly oriented lines. Global forms could be discriminated from the background based on differences in motion or differences in color/brightness. Furthermore, perception of a global form persisted after the binding cue had been removed. A comparison between the persistence of forms constructed from motion or color demonstrated that both forms produced persistence after the object defining cues were removed. Functional imaging showed a gradual increase in the persistence of brain activity in the lower visual areas (V1, V2, VP), which reached significance in V4v and peaked in the lateral occipital area. There was no difference in the location of persistence for color- or motion-defined forms. These results suggest that the retention of a global percept is an emerging property of the ventral visual processing stream and the maintenance of grouped visual elements is independent of cue type. We postulated that perceptual persistence depends on a system of perceptual memory reflecting the state of perceptual organization.

Face perception: domain specific, not process specific

Evidence that face perception is mediated by special cognitive and neural mechanisms comes from fMRI studies of the fusiform face area (FFA) and behavioral studies of the face inversion effect. Here, we used these two methods to ask whether face perception mechanisms are stimulus specific, process specific, or both. Subjects discriminated pairs of upright or inverted faces or house stimuli that differed in either the spatial distance among parts (configuration) or the shape of the parts. The FFA showed a much higher response to faces than to houses, but no preference for the configuration task over the part task. Similarly, the behavioral inversion effect was as large in the part task as the configuration task for faces, but absent in both part and configuration tasks for houses. These findings indicate that face perception mechanisms are not process specific for parts or configuration but are domain specific for face stimuli per se.

Similar cortical correlates underlie visual object identification and orientation judgment

Visual object perception has been suggested to follow two different routes in the human brain: a ventral, view-invariant occipital-temporal route processes object identity, whereas a dorsal, view-dependent occipital-parietal route processes spatial properties of an object. Using fMRI, we addressed the question whether these routes are exclusively involved in either object recognition or spatial representation. We presented subjects with images of natural objects and involved them either in object identification or object orientation judgment task. For both tasks, we observed activation in ventro-temporal as well as parietal areas bilaterally, with significantly stronger responses for the orientation judgment in both ventro-temporal as well as parietal areas. Our findings suggest that object identification and orientation judgment do not follow strictly separable cortical pathways, but rather involve both the dorsal and the ventral stream.

Segregation and persistence of form in the lateral occipital complex

While the lateral occipital complex (LOC) has been shown to be implicated in object recognition, it is unclear whether this brain area is responsive to low-level stimulus-driven features or high-level representational processes. We used scrambled shape-from-motion displays to disambiguate the presence of contours from figure-ground segregation and to measure the strength of the binding process for shapes without contours. We found persisting brain activation in the LOC for scrambled displays after the motion stopped indicating that this brain area subserves and maintains figure-ground segregation processes, a low-level function in the object processing hierarchy. In our second experiment, we found that the figure-ground segregation process has some form of spatial constancy indicating top-down influences. The persisting activation after the motion stops suggests an intermediate role in object recognition processes for this brain area and might provide further evidence for the idea that the lateral occipital complex subserves mnemonic functions mediating between iconic and short-term memory.

The Integration of Colour and Motion by the Human Visual Brain

Objects in the visual scene are defined by different cues such as colour and motion. Through the integration of these cues the visual system is able to utilize different sources of information, thus enhancing its ability to discriminate objects from their backgrounds. In the following experiments, we investigate the neural mechanisms of cue integration in the human. We show, using functional magnetic resonance imaging (fMRI), that both colour and motion defined shapes activate the lateral occipital complex (LOC) and that shapes defined by both colour and motion simultaneously activate the anterior-ventral margins of this area more strongly than shapes defined by either cue alone. This suggests that colour and motion cues are integrated in the LOC and possibly a neighbouring, more anterior, region. We support this result using an fMR adaptation technique, demonstrating that a region of the LOC adapts on repeated presentations of a shape regardless of the cue that is used to define it and even if the cue is varied. This result raises the possibility that the LOC contains cue-invariant neurons that respond to shapes regardless of the cue that is used to define them. We propose that such neurons could integrate signals from different cues, making them more responsive to objects defined by more than one cue, thus increasing the ability of the observer to recognize them.

THE HUMAN VISUAL CORTEX

The discovery and analysis of cortical visual areas is a major accomplishment of visual neuroscience. In the past decade the use of noninvasive functional imaging, particularly functional magnetic resonance imaging (fMRI), has dramatically increased our detailed knowledge of the functional organization of the human visual cortex and its relation to visual perception. The fMRI method offers a major advantage over other techniques applied in neuroscience by providing a large-scale neuroanatomical perspective that stems from its ability to image the entire brain essentially at once. This bird's eye view has the potential to reveal large-scale principles within the very complex plethora of visual areas. Thus, it could arrange the entire constellation of human visual areas in a unified functional organizational framework. Here we review recent findings and methods employed to uncover the functional properties of the human visual cortex focusing on two themes: functional specialization and hierarchical processing.

Shape Saliency Modulates Contextual Processing in the Human Lateral Occipital Complex

Visual context influences our perception of target objects in natural scenes. However, little is known about the analysis of context information and its role in shape perception in the human brain. We investigated whether the human lateral occipital complex (LOC), known to be involved in the visual analysis of shapes, also processes information about the context of shapes within cluttered scenes. We employed an fMRI adaptation paradigm in which fMRI responses are lower for two identical than for two different stimuli presented consecutively. The stimuli consisted of closed target contours defined by aligned Gabor elements embedded in a background of randomly oriented Gabors. We measured fMRI adaptation in the LOC across changes in the context of the target shapes by manipulating the position and orientation of the background elements. No adaptation was observed across context changes when the background elements were presented in the same plane as the target elements. However, adaptation was observed when the grouping of the target elements was enhanced in a bottom-up (i.e., grouping by disparity or motion) or top-down (i.e., shape priming) manner and thus the saliency of the target shape increased. These findings suggest that the LOC processes information not only about shapes, but also about their context. This processing of context information in the LOC is modulated by figure–ground segmentation and grouping processes. That is, neural populations in the LOC encode context information when relevant to the perception of target shapes, but represent salient targets independent of context changes.

Brain areas engaged during visual judgments by involuntary access to novel semantic information

Theories of visual recognition place different emphasis on the role of non-stimulus factors. Previously, we showed that arbitrary semantic associations influenced visual recognition of novel objects. Here, the neural substrate of this effect was investigated. During a visual task, novel objects associated with arbitrary semantic features produced more activation in frontal and parietal cortex than objects associated with names. Because the task required no semantic retrieval, access to semantics appears to be involuntary. The brain regions involved have been implicated in semantic processing, thus recently acquired semantics activate a similar network to semantics learned over a lifetime.

Neural correlates of implicit object identification

The present study sought to assess neural correlates of implicit identification of objects by means of fMRI, using tasks that require matching of the physical properties of objects. Behavioural data suggests that there is automatic access to object identity when observers attend to a physical property of the form of an object (e.g. the object's orientation) and no evidence for semantic processing when subjects attend to colour. We evaluated whether, in addition to neural areas associated with decisions to specific perceptual properties, areas associated with access to semantic information were activated when tasks demanded processing of the global configuration of pictures. We used two perceptual matching tasks based on the global orientation or on the colour of line drawings. Our results confirmed behavioural data. Activations in the inferior occipital cortex, fusiform and inferior temporal gyri in both tasks (orientation and colour) account for perceptual and structural processing involved in each task. In contrast, activations in the posterior and medial parts of the fusiform gyrus, shown to be involved in explicit semantic judgements, were more pronounced in the orientation-matching task, suggesting that semantic information from the pictures is processed in an implicit way even when not required by the task. Thus, this study suggests that cortical regions usually involved in explicit semantic processing are also activated when implicit processing of objects occurs.

Ventral occipital lesions impair object recognition but not object-directed grasping: an fMRI study

D.F., a patient with severe visual form agnosia, has been the subject of extensive research during the past decade. The fact that she could process visual input accurately for the purposes of guiding action despite being unable to perform visual discriminations on the same visual input inspired a novel interpretation of the functions of the two main cortical visual pathways or 'streams'. Within this theoretical context, the authors proposed that D.F. had suffered severe bilateral damage to her occipitotemporal visual system (the 'ventral stream'), while retaining the use of her occipitoparietal visual system (the 'dorsal stream'). The present paper reports a direct test of this idea, which was initially derived from purely behavioural data, before the advent of modern functional neuroimaging. We used functional MRI to examine activation in her ventral and dorsal streams during object recognition and object-directed grasping tasks. We found that D.F. showed no difference in activation when presented with line drawings of common objects compared with scrambled line drawings in the lateral occipital cortex (LO) of the ventral stream, an area that responded differentially to these stimuli in healthy individuals. Moreover, high-resolution anatomical MRI showed that her lesion corresponded bilaterally with the location of LO in healthy participants. The lack of activation with line drawings in D.F. mirrors her poor performance in identifying the objects depicted in the drawings. With coloured and greyscale pictures, stimuli that she can identify more often, D.F. did show some ventral-stream activation. These activations were, however, more widely distributed than those seen in control participants and did not include LO. In contrast to the absent or abnormal activation observed during these perceptual tasks, D.F. showed robust activation in the expected dorsal stream regions during object grasping, despite considerable atrophy in some regions of the parietal lobes. In particular, an area in the anterior intraparietal sulcus was activated more for grasping an object than for just reaching to that object, for both D.F. and controls. In conclusion, we have been able to confirm directly that D.F.'s visual form agnosia is associated with extensive damage to the ventral stream, and that her spared visuomotor skills are associated with visual processing in the dorsal stream.

Visual awareness of low-contrast stimuli is reflected in event-related brain potentials

Visual awareness was studied in 11 subjects by using coherent and scrambled objects as stimuli. The stimuli were presented near the subjective perceptual threshold. Explicitly recognized stimuli elicited a specific negative ERP deflection peaking at 460 ms. Our reaction time experiment suggests that this visual awareness negativity (VAN) is similar to that found previously at a shorter latency. We propose that VAN might reflect the exact moment of becoming aware of the task-relevant stimulus properties and provide an online marker of the temporal dynamics of visual awareness.

Perceptual organization of local elements into global shapes in the human visual cortex

The question of how local image features on the retina are integrated into perceived global shapes is central to our understanding of human visual perception. Psychophysical investigations have suggested that the emergence of a coherent visual percept, or a "good-Gestalt", is mediated by the perceptual organization of local features based on their similarity. However, the neural mechanisms that mediate unified shape perception in the human brain remain largely unknown. Using human fMRI, we demonstrate that not only higher occipitotemporal but also early retinotopic areas are involved in the perceptual organization and detection of global shapes. Specifically, these areas showed stronger fMRI responses to global contours consisting of collinear elements than to patterns of randomly oriented local elements. More importantly, decreased detection performance and fMRI activations were observed when misalignment of the contour elements disturbed the perceptual coherence of the contours. However, grouping of the misaligned contour elements by disparity resulted in increased performance and fMRI activations, suggesting that similar neural mechanisms may underlie grouping of local elements to global shapes by different visual features (orientation or disparity). Thus, these findings provide novel evidence for the role of both early feature integration processes and higher stages of visual analysis in coherent visual perception.

fMRI activation in response to illusory contours and salient regions in the human lateral occipital complex

Regions in the human Lateral Occipital Complex (LOC) show fMRI responses to illusory surfaces. We show that the LOC activation is due to the globally completed region and occurs even when the region is not bounded by illusory contours (ICs). Kanizsa-type stimuli were modified by rounding the corners of the "pacmen" inducers and misaligning them slightly. The impression of an enclosed, salient region (SR) remained, although ICs were no longer perceived (psychophysical data). fMRI activity was elevated for both the IC and SR stimuli, compared to their control stimuli. The LOC response to salient regions may be the result of fast but crude region-based segmentation processes, which are useful for selecting parts of cluttered images for more detailed, computationally intensive processing.

Integration of local features into global shapes: monkey and human FMRI studies

The integration of local image features into global shapes was investigated in monkeys and humans using fMRI. An adaptation paradigm was used, in which stimulus selectivity was deduced by changes in the course of adaptation of a pattern of randomly oriented elements. Accordingly, we observed stronger activity when orientation changes in the adapting stimulus resulted in a collinear contour than a different random pattern. This selectivity to collinear contours was observed not only in higher visual areas that are implicated in shape processing, but also in early visual areas where selectivity depended on the receptive field size. These findings suggest that unified shape perception in both monkeys and humans involves multiple visual areas that may integrate local elements to global shapes at different spatial scales.

Effects of Alzheimer's disease on the recognition of novel versus familiar words: neuropsychological and clinico-metabolic data

This study explored recognition memory performance for novel versus familiar words in Alzheimer's disease (AD) patients and normal controls (NCs), using an adaptation of E. Tulving and N. Kroll's (1995) procedure. Results showed that both groups exhibited more hits and more false alarms for familiar than for novel words. The groups did not differ in the recognition of familiar words, reflecting preserved familiarity processes in AD. However, AD patients made more false alarms than NCs in the recognition of novel words, reflecting impairment of recollection processes in AD. A positron emission tomography analysis of clinico-metabolic correlations in AD patients showed a correlation between recognition of novel words and right hippocampal activity, whereas recognition of familiar words was more related to metabolic activity in the left posterior orbitofrontal cortex.

Attention to faces: a change-blindness study

What strategies does human vision use to attend to faces and their features? How are such strategies altered by 2-D inversion or photographic negation? We report two experiments in which these questions were studied with the flicker task of the change-blindness literature. In experiment 1 we studied detection of configural changes to the eyes or mouth, and found that upright faces receive more efficient attention than inverted faces, and that faces shown with normal contrast receive more efficient attention than faces shown in photographic negative. Moreover, eyes receive greater attention than the mouth. In experiment 2 we studied detection of local changes to the eyes or mouth, and found the same results. It is well known that inversion and negation impair the perception and recognition of faces. The experiments presented here extend previous findings by showing that inversion and negation also impair attention to faces.

Shape perception reduces activity in human primary visual cortex

Visual perception involves the grouping of individual elements into coherent patterns that reduce the descriptive complexity of a visual scene. The physiological basis of this perceptual simplification remains poorly understood. We used functional MRI to measure activity in a higher object processing area, the lateral occipital complex, and in primary visual cortex in response to visual elements that were either grouped into objects or randomly arranged. We observed significant activity increases in the lateral occipital complex and concurrent reductions of activity in primary visual cortex when elements formed coherent shapes, suggesting that activity in early visual areas is reduced as a result of grouping processes performed in higher areas. These findings are consistent with predictive coding models of vision that postulate that inferences of high-level areas are subtracted from incoming sensory information in lower areas through cortical feedback.