Processing of contrast polarity of visual images in inferotemporal cortex of the macaque monkey

Neural responses to dynamic adaptation reveal the dissociation between the processing of the shape of contours and textures

Shape-adaptation studies show that surround textures can inhibit the processing of contours. Using event-related potentials (ERP), we examined the time-course of neural processes involved in contour-shape and texture-shape processing following adaptation to contours and textures. Contours were made of Gabor strings whose orientations were either tangential or orthogonal to the contour path, while textures were made of a series of contours arranged in parallel. We focused on two ERP components -P1, related to low-level visual processes and N1, broadly indicative of mid-level vision- and, on ERP difference waves (no-adaptor minus with-adaptor) to isolate the effects of adaptation, which are fundamentally distinct from individual processes driving P1 and N1 components. We found that in the absence of adaptation, the N1 component for contour-tests peaked later and increased in amplitude compared to the N1 for texture-tests. Following adaptation, the ERP difference wave for contour-tests revealed an early and a late component that were differentially affected by the presence of surround texture, but critically not by its orientation. For texture-tests, the early component was of opposite polarity for contours compared to texture adaptors. From the temporal sequence of ERP modulations, we conclude that texture processing begins before contour processing and encompasses the stages of perceptual processing reflected in both the low-level P1 and the mid-level N1 vision-related components. Our study provides novel evidence on the nature of separable and temporally distinct texture and contour processing mechanisms, shown in two difference wave components, that highlights the multi-faceted nature of dynamic adaptation to shape when presented in isolation and in context.

Face Representations via Tensorfaces of Various Complexities

Neurons selective for faces exist in humans and monkeys. However, characteristics of face cell receptive fields are poorly understood. In this theoretical study, we explore the effects of complexity, defined as algorithmic information (Kolmogorov complexity) and logical depth, on possible ways that face cells may be organized. We use tensor decompositions to decompose faces into a set of components, called tensorfaces, and their associated weights, which can be interpreted as model face cells and their firing rates. These tensorfaces form a high-dimensional representation space in which each tensorface forms an axis of the space. A distinctive feature of the decomposition algorithm is the ability to specify tensorface complexity. We found that low-complexity tensorfaces have blob-like appearances crudely approximating faces, while high-complexity tensorfaces appear clearly face-like. Low-complexity tensorfaces require a larger population to reach a criterion face reconstruction error than medium- or high-complexity tensorfaces, and thus are inefficient by that criterion. Low-complexity tensorfaces, however, generalize better when representing statistically novel faces, which are faces falling beyond the distribution of face description parameters found in the tensorface training set. The degree to which face representations are parts based or global forms a continuum as a function of tensorface complexity, with low and medium tensorfaces being more parts based. Given the computational load imposed in creating high-complexity face cells (in the form of algorithmic information and logical depth) and in the absence of a compelling advantage to using high-complexity cells, we suggest face representations consist of a mixture of low- and medium-complexity face cells.

Modeling diverse responses to filled and outline shapes in macaque V4

Visual area V4 is an important midlevel cortical processing stage that subserves object recognition in primates. Studies investigating shape coding in V4 have largely probed neuronal responses with filled shapes, i.e., shapes defined by both a boundary and an interior fill. As a result, we do not know whether form-selective V4 responses are dictated by boundary features alone or if interior fill is also important. We studied 43 V4 neurons in two male macaque monkeys ( Macaca mulatta) with a set of 362 filled shapes and their corresponding outlines to determine how interior fill modulates neuronal responses in shape-selective neurons. Only a minority of neurons exhibited similar response strength and shape preferences for filled and outline stimuli. A majority responded preferentially to one stimulus category (either filled or outline shapes) and poorly to the other. Our findings are inconsistent with predictions of the hierarchical-max (HMax) V4 model that builds form selectivity from oriented boundary features and takes little account of attributes related to object surface, such as the phase of the boundary edge. We modified the V4 HMax model to include sensitivity to interior fill by either removing phase-pooling or introducing unoriented units at the V1 level; both modifications better explained our data without increasing the number of free parameters. Overall, our results suggest that boundary orientation and interior surface information are both maintained until at least the midlevel visual representation, consistent with the idea that object fill is important for recognition and perception in natural vision. NEW & NOTEWORTHY The shape of an object's boundary is critical for identification; consistent with this idea, models of object recognition predict that filled and outline versions of a shape are encoded similarly. We report that many neurons in a midlevel visual cortical area respond differently to filled and outline shapes and modify a biologically plausible model to account for our data. Our results suggest that representations of boundary shape and surface fill are interrelated in visual cortex.

Neural representation for object recognition in inferotemporal cortex

We suggest that population representation of objects in inferotemporal cortex lie on a continuum between a purely structural, parts-based description and a purely holistic description. The intrinsic dimensionality of object representation is estimated to be around 100, perhaps with lower dimensionalities for object representations more toward the holistic end of the spectrum. Cognitive knowledge in the form of semantic information and task information feed back to inferotemporal cortex from perirhinal and prefrontal cortex respectively, providing high-level multimodal-based expectations that assist in the interpretation of object stimuli. Integration of object information across eye movements may also contribute to object recognition through a process of active vision.

What makes a cell face selective? The importance of contrast

Faces are robustly detected by computer vision algorithms that search for characteristic coarse contrast features. Here, we investigated whether face-selective cells in the primate brain exploit contrast features as well. We recorded from face-selective neurons in macaque inferotemporal cortex, while presenting a face-like collage of regions whose luminances were changed randomly. Modulating contrast combinations between regions induced activity changes ranging from no response to a response greater than that to a real face in 50% of cells. The critical stimulus factor determining response magnitude was contrast polarity, for example, nose region brighter than left eye. Contrast polarity preferences were consistent across cells, suggesting a common computational strategy across the population, and matched features used by computer vision algorithms for face detection. Furthermore, most cells were tuned both for contrast polarity and for the geometry of facial features, suggesting cells encode information useful both for detection and recognition.

Differential roles of frequency-following and frequency-doubling visual responses revealed by evoked neural harmonics

Frequency-following and frequency-doubling neurons are ubiquitous in both striate and extrastriate visual areas. However, responses from these two types of neural populations have not been effectively compared in humans because previous EEG studies have not successfully dissociated responses from these populations. We devised a light-dark flicker stimulus that unambiguously distinguished these responses as reflected in the first and second harmonics in the steady-state visual evoked potentials. These harmonics revealed the spatial and functional segregation of frequency-following (the first harmonic) and frequency-doubling (the second harmonic) neural populations. Spatially, the first and second harmonics in steady-state visual evoked potentials exhibited divergent posterior scalp topographies for a broad range of EEG frequencies. The scalp maximum was medial for the first harmonic and contralateral for the second harmonic, a divergence not attributable to absolute response frequency. Functionally, voluntary visual-spatial attention strongly modulated the second harmonic but had negligible effects on the simultaneously elicited first harmonic. These dissociations suggest an intriguing possibility that frequency-following and frequency-doubling neural populations may contribute complementary functions to resolve the conflicting demands of attentional enhancement and signal fidelity--the frequency-doubling population may mediate substantial top-down signal modulation for attentional selection, whereas the frequency-following population may simultaneously preserve relatively undistorted sensory qualities regardless of the observer's cognitive state.

A model for learning topographically organized parts-based representations of objects in visual cortex: topographic nonnegative matrix factorization

Object representation in the inferior temporal cortex (IT), an area of visual cortex critical for object recognition in the primate, exhibits two prominent properties: (1) objects are represented by the combined activity of columnar clusters of neurons, with each cluster representing component features or parts of objects, and (2) closely related features are continuously represented along the tangential direction of individual columnar clusters. Here we propose a learning model that reflects these properties of parts-based representation and topographic organization in a unified framework. This model is based on a nonnegative matrix factorization (NMF) basis decomposition method. NMF alone provides a parts-based representation where nonnegative inputs are approximated by additive combinations of nonnegative basis functions. Our proposed model of topographic NMF (TNMF) incorporates neighborhood connections between NMF basis functions arranged on a topographic map and attains the topographic property without losing the parts-based property of the NMF. The TNMF represents an input by multiple activity peaks to describe diverse information, whereas conventional topographic models, such as the self-organizing map (SOM), represent an input by a single activity peak in a topographic map. We demonstrate the parts-based and topographic properties of the TNMF by constructing a hierarchical model for object recognition where the TNMF is at the top tier for learning high-level object features. The TNMF showed better generalization performance over NMF for a data set of continuous view change of an image and more robustly preserving the continuity of the view change in its object representation. Comparison of the outputs of our model with actual neural responses recorded in the IT indicates that the TNMF reconstructs the neuronal responses better than the SOM, giving plausibility to the parts-based learning of the model.

Spatial properties of curvature-encoding mechanisms revealed through the shape-frequency and shape-amplitude after-effects

The shape-frequency and shape-amplitude after-effects, or SFAE and SAAE, are phenomena in which adaptation to a sinusoidal-shaped contour results in a shift in, respectively, the perceived shape-frequency and perceived shape-amplitude of a test contour in a direction away from that of the adapting stimulus. Recent evidence shows that the SFAE and SAAE are mediated by mechanisms sensitive to curvature [Gheorghiu, E., & Kingdom, F. A. A. (2007a). The spatial feature underlying the shape-frequency and shape-amplitude after-effects. Vision Research, 47(6), 834-844]. Therefore we have used the SFAE and SAAE as a tool to study curvature processing. We examined whether curvature-encoding mechanisms are selective for (i) shape-phase, (ii) curvature polarity (or sign) and (iii) local orientation. We also investigated whether (iv) the two orthogonal dimensions of a curve, the sag and the cord, are encoded independently, and (v) whether curvature encoders are organized in an opponent manner. SFAEs/SAAEs were measured for adapting and test contours that differed or not in a given spatial property, the rationale being that if the after-effects were smaller when adaptor and test differed in a particular spatial property then curvature-encoding mechanisms must be selective for that spatial property. Our results reveal that SFAEs and SAAEs show (i) a degree of selectivity to curves that are mirror symmetric (in our stimuli half-cycle sine-wave contours in cosine (0/180deg) shape-phase); (ii) a degree of selectivity to the sign or polarity of curvature; (iii) a degree of selectivity to local orientation; (iv) independent coding of the sag and the cord of the curve, and (v) no evidence for opponent-curvature coding. The results agree with neurophysiological studies showing that simple shape dimensions are encoded independently.

Long-term speeding in perceptual switches mediated by attention-dependent plasticity in cortical visual processing

Binocular rivalry has been extensively studied to understand the mechanisms that control switches in visual awareness and much has been revealed about the contributions of stimulus and cognitive factors. Because visual processes are fundamentally adaptive, however, it is also important to understand how experience alters the dynamics of perceptual switches. When observers viewed binocular rivalry repeatedly over many days, the rate of perceptual switches increased as much as 3-fold. This long-term rivalry speeding exhibited a pattern of image-feature specificity that ruled out primary contributions from strategic and nonsensory factors and implicated neural plasticity occurring in both low- and high-level visual processes in the ventral stream. Furthermore, the speeding occurred only when the rivaling patterns were voluntarily attended, suggesting that the underlying neural plasticity selectively engages when stimuli are behaviorally relevant. Long-term rivalry speeding may thus reflect broader mechanisms that facilitate quick assessments of signals that contain multiple behaviorally relevant interpretations.

Early face processing specificity: it's in the eyes!

Unlike most other objects that are processed analytically, faces are processed configurally. This configural processing is reflected early in visual processing following face inversion and contrast reversal, as an increase in the N170 amplitude, a scalp-recorded event-related potential. Here, we show that these face-specific effects are mediated by the eye region. That is, they occurred only when the eyes were present, but not when eyes were removed from the face. The N170 recorded to inverted and negative faces likely reflects the processing of the eyes. We propose a neural model of face processing in which face- and eye-selective neurons situated in the superior temporal sulcus region of the human brain respond differently to the face configuration and to the eyes depending on the face context. This dynamic response modulation accounts for the N170 variations reported in the literature. The eyes may be central to what makes faces so special.

Chromatic tuning of contour-shape mechanisms revealed through the shape-frequency and shape-amplitude after-effects

We investigated whether contour-shape processing mechanisms are selective for color direction using the shape-frequency and shape-amplitude after-effects, or SFAE and SAAE [Gheorghiu, E. & Kingdom, F. A. A. (2006). Luminance-contrast properties of contour-shape processing revealed through the shape-frequency after-effect. Vision Research, 46(21), 3603-3615. Gheorghiu, E. & Kingdom, F. A. A. (2007). The spatial feature underlying the shape-frequency and shape-amplitude after-effects. Vision Research, 47(6), 834-844]. All contours were defined along the 'red-green', 'blue-yellow' and 'luminance' axes of cardinal color space. Adapting and test contours were defined along the same or along opposite polarities within a cardinal axis, and along the same or along different cardinal axes. We found (i) little transfer of the after-effects across different within-axis polarities, for all cardinal axes and for both even-symmetric and odd-symmetric contours; (ii) little transfer between the red-green and blue-yellow cardinal axes; (iii) little transfer between the chromatic and luminance cardinal directions for the SAAE; (iv) large transfer between the chromatic and luminance cardinal directions for the SFAE. We conclude that contour-shape mechanisms are selective for within-cardinal axis polarity and for the chromatic axes within the isoluminant plane. However for certain types of contour-shape processing they are poorly selective along the chromatic versus luminance dimension. Overall our results suggest that contour-shape encoding mechanisms are selective for color direction and that color is important for contour-shape processing.

Object category structure in response patterns of neuronal population in monkey inferior temporal cortex

Our mental representation of object categories is hierarchically organized, and our rapid and seemingly effortless categorization ability is crucial for our daily behavior. Here, we examine responses of a large number (>600) of neurons in monkey inferior temporal (IT) cortex with a large number (>1,000) of natural and artificial object images. During the recordings, the monkeys performed a passive fixation task. We found that the categorical structure of objects is represented by the pattern of activity distributed over the cell population. Animate and inanimate objects created distinguishable clusters in the population code. The global category of animate objects was divided into bodies, hands, and faces. Faces were divided into primate and nonprimate faces, and the primate-face group was divided into human and monkey faces. Bodies of human, birds, and four-limb animals clustered together, whereas lower animals such as fish, reptile, and insects made another cluster. Thus the cluster analysis showed that IT population responses reconstruct a large part of our intuitive category structure, including the global division into animate and inanimate objects, and further hierarchical subdivisions of animate objects. The representation of categories was distributed in several respects, e.g., the similarity of response patterns to stimuli within a category was maintained by both the cells that maximally responded to the category and the cells that responded weakly to the category. These results advance our understanding of the nature of the IT neural code, suggesting an inherently categorical representation that comprises a range of categories including the amply investigated face category.

The spatial feature underlying the shape-frequency and shape-amplitude after-effects

The shape-frequency and shape-amplitude after-effects, or SFAE and SAAE, refer respectively to the shifts observed in the perceived shape-frequency and shape-amplitude of a sinusoidal test contour following adaptation to a similar-shaped contour. As with other shape after-effects the shifts are in a direction away from that of the adapting stimulus. Using a variety of procedures we tested whether the spatial feature that was adapted in the SFAE and SAAE was (a) local orientation, (b) average unsigned curvature, (c) periodicity/density, (d) shape-amplitude and (e) local curvature. Our results suggest that the last of these, local curvature, underlies both the SFAE and SAAE. The evidence in favour of local curvature was that the after-effect reached its maximum value when just half-a-cycle of the test contour, in +/-cosine phase, was present. We suggest that the SFAE and SAAE are mediated by intermediate-level mechanisms that encode the shapes of contour fragments with constant sign of curvature. Given the neurophysiological evidence that neurons in area V4 encode parts of shapes with constant sign of curvature, we suggest V4 is the likely neural substrate for both the SFAE and SAAE.

Luminance-contrast properties of contour-shape processing revealed through the shape-frequency after-effect

We investigated the first-order inputs to contour-shape mechanisms using the shape-frequency after-effect (SFAE), in which adaptation to a sinusoidally modulated contour causes a shift in the apparent shape-frequency of a test contour in a direction away from that of the adapting stimulus [Kingdom F. A. A., & Prins N. (2005a). Different mechanisms encode the shapes of contours and contour-textures. Journal of Vision 5(8), 463, (Abstract)]. We measured SFAEs for adapting and test contours (and edges) that differed in the contrast-polarity, scale (or blur) and magnitude of luminance contrast. The rationale was that if the SFAE was found to be reduced when adaptor and test differed along a particular dimension of luminance contrast, contour-shape mechanisms must be tuned to that dimension. Our results reveal that SFAEs manifest (i) a degree of selectivity to luminance contrast polarity for both even-symmetric (contours only) and odd-symmetric (both contours and edges) luminance profiles; (ii) a degree of selectivity to luminance scale (or blur); (iii) higher selectivity to fine compared to coarse scale for broadband edges (iv) a small preference for equal-in-contrast adaptors and tests. These results suggest that contour shapes are not encoded in the form of a sparse, cartoon-like sketch, as might be presumed by local energy (i.e. non-phase-selective) or form-cue invariant models, but instead in a form that is relatively 'feature-rich.'

Face, eye and object early processing: What is the face specificity?

We investigated the human face specificity by comparing the effects of inversion and contrast reversal, two manipulations known to disrupt configural face processing, on human and ape faces, isolated eyes and objects, using event-related potentials. The face sensitive marker, N170, was shortest to human faces and delayed by inversion and contrast reversal for all categories and not only for human faces. Most importantly, N170 to inverted or contrast-reversed faces was not different from N170 to eyes that did not differ across manipulations. This suggests the disruption of facial configuration by these manipulations isolates the eye region from the face context, to which eye neurons respond. Our data suggest that (i) the inversion and contrast reversal effects on N170 latency are not specific to human faces and (ii) the similar increase of N170 amplitude by inversion and contrast reversal is unique to human faces and is driven by the eye region. Thus, while inversion and contrast reversal effects on N170 latency are not category-specific, their effects on amplitude are face-specific and reflect mainly the contribution of the eye region.

Quantitative analysis of functional clustering of neurons in the macaque inferior temporal cortex

Neurons with similar preferences for two-dimensional shapes of intermediate complexity cluster in area TE of the monkey inferior temporal cortex. To further characterize the functional structure of area TE, we quantitatively analyzed various aspects of the visual responses of closely located neurons by applying multiple single-unit recording techniques in anesthetized monkeys. Examination of the visual responses elicited with a large, predetermined set of visual stimuli confirmed previous findings that nearby neurons, on average, exhibited positively correlated preferences for a set of visual stimuli. Nearby neurons also tended to be similar in their receptive-field organization and contrast-polarity preference. In contrast, no correlation was found in the size tuning of neighboring neurons. Pooling or subtraction of activities between a pair of nearby neurons was shown to improve stimulus discriminability, if the neuron pair had positively or negatively correlated stimulus preferences, respectively. These results indicate that nearby TE neurons share some aspects of stimulus preference, but their response selectivity differ in other aspects. Both pooling and subtraction between nearby neurons can reduce across-trial response variability, if these decoding strategies are applied to appropriate neuronal pools.

Spatiotemporal analysis of event-related potentials to upright, inverted, and contrast-reversed faces: Effects on encoding and recognition

In an n-back face recognition task where subjects responded to repeated stimuli, ERPs were recorded to upright, inverted, and contrast-reversed faces. The effects of inversion and contrast reversal on face encoding and recognition were investigated using the multivariate spatiotemporal partial least squares (PLS) analysis. The configural manipulations affected early processing (100-200 ms) at posterior sites: Inversion effects were parietal and lateral, whereas contrast-reversal effects were more occipital and medial, suggesting different underlying generators. A later reactivation of face processing areas was unique to inverted faces, likely due to processing difficulties. PLS also indicated that the "old-new" repetition effect was maximal for upright faces and likely involved frontotemporal areas. Marked processing differences between inverted and contrast-reversed faces were seen, but these effects were similar at encoding and recognition.

Attention during adaptation weakens negative afterimages

The effect of attention during adaptation on subsequent negative afterimages was examined. One of 2 overlapped outline figures was attended during a 7-10-s adaptation period. When the figures were readily perceptually segregated (on the basis of color or motion), the subsequent afterimages were initially weaker for the previously attended figure. This effect was confirmed by demonstrations that the onset of a single afterimage was delayed when an afterimage inducer was attended during adaptation compared with when a central digit stream or an overlapped (brightness-balanced) figure that did not generate an afterimage was attended. The attention effect was further confirmed using a criterion-independent (dot-integration) paradigm. The fact that selective attention during adaptation weakened or delayed afterimages suggests that attention primarily facilitates the adaptation of polarity-independent processes that modulate the visibility of afterimages rather than facilitating the adaptation of polarity-selective processes that mediate the formation of afterimages.

Attentional selection of overlapped shapes: a study using brief shape aftereffects

Prior studies using brief stimulus sequences revealed "opponent shape aftereffects", indicative of direct opponent coding of global shape attributes such as aspect ratio, skew, taper, curvature, and convexity (perhaps in IT). Further, aftereffects from overlapped opponent pairs of adaptor shapes (e.g., concave and convex shapes) were substantially modulated by attention [Vision Res. 41 (2001) 3883]. Hypothetically, (1) attention might weight the attended and ignored contours at early stages of processing, or (2) it might sway opposing neural activity (e.g., of convex- vs. concave-tuned units) at the stage of opponent shape coding. Attentional modulation was equivalent for opponent pairs (producing opposite aftereffects) and non-opponent pairs (producing orthogonal aftereffects) of overlapped adaptor shapes, whether convexity or aspect-ratio aftereffects were measured. Further, the degree of attentional modulation obtained for these aftereffects (approximately 60%) was comparable to that obtained for V4 cells [J. Neurosci. 19 (1999) 1736]. Taken together, differential contour weighting appears to be the primary mechanism of attentional modulation of brief shape aftereffects.

Attentional control of texture orientation judgments

Recent models of texture processing use low level, spatially parallel computations to extract texture properties. The rapid, preattentive nature of texture segregation suggests that these computations are bottom-up in nature. However, the immunity of texture judgments to top-down influences remains to be tested. Here we investigate the degree to which judgments of texture orientation are susceptible to top-down attentional control. Observers view a brief display composed of variously luminant texture elements (line segments) alternately (in checkerboard arrangement) oriented up/right (at 71.5 degrees ) or up/left (at 108.5 degrees ), and are asked to make various judgments. In a given task, the observer attempts on each trial to judge which oriented population of line segments has an intensity histogram that best matches a given target histogram. Performance demonstrates adaptive flexibility across different tasks, suggesting that observers are able to exercise significant top-down control over texture orientation computations. Specifically, observers can attend selectively to positive contrast texture elements, to negative contrast texture elements, or to high (positive and negative) contrast texture elements. More generally, observers perform well if the target histogram can be approximated by a weighted average of positive and negative half-wave rectifiers. Performance is poor for histograms that cannot be captured in this way. These results suggest that attentional control in these tasks is limited to adjusting the relative gain of the on- and off-center systems.

Attention-dependent brief adaptation to contour orientation: a high-level aftereffect for convexity?

In contrast to the abundant literature investigating how orientation coding depends on edges defined by various image features, relatively little is known about how coding of orientation might also depend on the two distinct functional roles that oriented edges commonly play. Oriented lines can delineate outline contours of a figure or they can form texture. The results of five experiments using orientation aftereffects measured with brief tests (27 ms, backward masked; adapt-to-test interval=201 ms) provided evidence that brief stimuli (<135 ms) selectively adapt coding of contour-line orientation rather than coding of line-texture orientation. Furthermore, parametric results revealed that the rapidly adapting aftereffects for contour orientation are characterized by (1) broad orientation tuning (peaking at +/-30 degrees to +/-50 degrees from test orientation), (2) indifference as to how the contours are defined (e.g. bright lines, high-pass-filtered lines, faint lines generated by the spatial inhomogeneity of visual sensitivity), (3) rapid saturation at low contrast energy, (4) strong modulation by selective attention, and (5) relative size tolerance. These characteristics appear to parallel those of cells in the high end of the visual form processing pathway (such as inferotemporal cortex). It is thus suggested that the rapidly adapting contour orientation aftereffects reported here may be mediated by high-level neural units that encode global configurations of orientation (e.g. convexity and concavity).

The lateral occipital complex and its role in object recognition

Here we review recent findings that reveal the functional properties of extra-striate regions in the human visual cortex that are involved in the representation and perception of objects. We characterize both the invariant and non-invariant properties of these regions and we discuss the correlation between activation of these regions and recognition. Overall, these results indicate that the lateral occipital complex plays an important role in human object recognition.

fMR-adaptation: a tool for studying the functional properties of human cortical neurons

The invariant properties of human cortical neurons cannot be studied directly by fMRI due to its limited spatial resolution. One voxel obtained from a fMRI scan contains several hundred thousands neurons. Therefore, the fMRI signal may average out a heterogeneous group of highly selective neurons. Here, we present a novel experimental paradigm for fMRI, functional magnetic resonance-adaptation (fMR-A), that enables to tag specific neuronal populations within an area and investigate their functional properties. This approach contrasts with conventional mapping methods that measure the averaged activity of a region. The application of fMR-A to study the functional properties of cortical neurons proceeds in two stages: First, the neuronal population is adapted by repeated presentation of a single stimulus. Second, some property of the stimulus is varied and the recovery from adaptation is assessed. If the signal remains adapted, it will indicate that the neurons are invariant to that attribute. However, if the fMRI signal will recover from the adapted state it would imply that the neurons are sensitive to the property that was varied. Here, an application of fMR-A for studying the invariant properties of high-order object areas (lateral occipital complex--LOC) to changes in object size, position, illumination and rotation is presented. The results show that LOC is less sensitive to changes in object size and position compared to changes of illumination and viewpoint. fMR-A can be extended to other neuronal systems in which adaptation is manifested and can be used with event-related paradigms as well. By manipulating experimental parameters and testing recovery from adaptation it should be possible to gain insight into the functional properties of cortical neurons which are beyond the spatial resolution limits imposed by conventional fMRI.

Cortical activity related to cue-invariant shape perception in humans

We used magnetoencephalography to search spatio-temporally for cortical activity related to the perception of shape defined by various visual cues in humans. The visual stimuli were three kinds of two-dimensional figures: two had fixed shapes (Diamond and Cross), the other did not (Noise). These figures were defined by three visual cues: difference of flicker, texture or luminance between the foreground and the background in the random dot pattern. Using this stimulus, we recorded the magnetic responses from the temporo-occipital regions of nine healthy subjects. Additionally, we measured the reaction time for the subjects to detect the figure by button-pressing. A magnetic component was identified in the responses. The properties of the first magnetic component differed for stimulus condition. The peak latency of the first magnetic component was different for the cues (270 ms for flicker, 360 ms for texture and 250 ms for luminance), but not for the figures. In contrast, the peak amplitude of the first magnetic component was different for the figures (96-144 fT for Diamond or Cross and 52-80 fT for Noise), but not for the cues. The signal source of the first magnetic component was estimated to lie on the ventral side of the extrastriate cortex: In the posterior part of the inferior temporal cortex, probably in the fusiform gyrus in four subjects, and in the lateral part of the occipital cortex which was outside of the primary visual cortex (visual area 1) in one subject. The signal source location was different inter-individually, but almost the same within each subject. Reaction time was 471 ms for flicker, 569 ms for texture and 426 ms for luminance, but the interval between the reaction time and the peak latency was constant (about 200 ms) for each cue. The first magnetic component was more clearly recorded from the right hemisphere than from the left.We found that the shape defined by the different visual cues activates the same localized site in the lateral extrastriate cortex. This spatial convergence suggests that there is a restricted locus that processes the visual shape regardless of the difference of the visual cue. The correspondence between the peak latency and the reaction time suggests that the activity of the area is responsible for the perception of visual shape. The inter-hemispheric difference suggests a dominance of the right hemisphere in visual shape processing.

Selectivity for 3D shape that reveals distinct areas within macaque inferior temporal cortex

The anterior part of the macaque inferior temporal cortex, area TE, occupies a large portion of the temporal lobe and is critical for object recognition. Thus far, no relation between anatomical subdivisions of TE and neuronal selectivity has been described. Here, we present evidence that neurons selective for three-dimensional (3D) shape are concentrated in the lower bank of the superior temporal sulcus, whereas neurons in lateral TE are generally unselective for 3D shape, though equally selective for 2D shape. These findings reveal that TE consists of at least two distinct areas, one of which processes a specific object property.

Differential processing of objects under various viewing conditions in the human lateral occipital complex

The invariant properties of human cortical neurons cannot be studied directly by fMRI due to its limited spatial resolution. Here, we circumvented this limitation by using fMR adaptation, namely, reduction of the fMR signal due to repeated presentation of identical images. Object-selective regions (lateral occipital complex [LOC]) showed a monotonic signal decrease as repetition frequency increased. The invariant properties of fMR adaptation were studied by presenting the same object in different viewing conditions. LOC exhibited stronger fMR adaptation to changes in size and position (more invariance) compared to illumination and viewpoint. The effect revealed two putative subdivisions within LOC: caudal-dorsal (LO), which exhibited substantial recovery from adaptation under all transformations, and posterior fusiform (PF/LOa), which displayed stronger adaptation. This study demonstrates the utility of fMR adaptation for revealing functional characteristics of neurons in fMRI studies.

Functional architecture in monkey inferotemporal cortex revealed by in vivo optical imaging

To investigate the functional organization in the monkey inferotemporal cortex, which is the last exclusively visual area along the ventral visual cortical pathway, optical imaging based on intrinsic signals was carried out. We first conducted single-cell recordings with microelectrodes and determined the features critical for the activation of single cells. For the subsequent optical imaging, each critical feature was presented, which evoked multiple dark spots. Individual spots were approximately 0.5 mm in diameter and one of them covered the site of the electrode penetration at which the particular critical feature had been determined. The degree of stimulus selectivity varied from spot to spot, and from region to region even within a spot. Some regions were activated only by one of 12- 16 stimuli, while others by more than three stimuli. There were spots specifically activated by faces, and the position of activation spot changed gradually along the cortical surface as the stimulus face was rotated in depth. The length of the overall region along the direction of shift of these spots was approximately 1 mm. These results confirm the regional clustering of cells with similar stimulus selectivity and suggest larger units in which some parameters of object features are continuously mapped.

Mechanisms of stereoscopic vision: the disparity energy model

The past year has seen significant advances in our understanding of the role played by the primary visual cortex (V1) in stereoscopic vision. Recently, the mechanism by which complex cells in V1 respond to random-dot stereograms has been characterized; it appears that their response properties greatly reduce the complexity of one of the critical links for stereopsis, the correspondence problem.

Effects of shape-discrimination training on the selectivity of inferotemporal cells in adult monkeys

Through extensive training, humans can become "visual experts, " able to visually distinguish subtle differences among similar objects with greater ease than those who are untrained. To understand the neural mechanisms behind this acquired discrimination ability, adult monkeys were fully trained to discriminate 28 moderately complex shapes. The training effects on the stimulus selectivity of cells in area TE of the inferotemporal cortex were then examined in anesthetized preparations. Area TE represents a later stage of the ventral visual cortical pathway that is known to mediate visual object discrimination and recognition. The recordings from the trained monkeys and untrained controls showed that the proportion of TE cells responsive to some member of the 28 stimuli was significantly greater in the trained monkeys than that in the control monkeys. Cell responses recorded from the trained monkeys were not sharply tuned to single training stimuli, but rather broadly covered several training stimuli. The distances among the training stimuli in the response space spanned by responses of the recorded TE cells were significantly greater in the trained monkeys than those in the control monkeys. The subset of training stimuli to which individual cells responded differed from cell to cell with only partial overlaps, suggesting that the cells responded to features common to several stimuli. These results are consistent with a model in which visual expertise is acquired through the development of differential responses by inferotemporal cells to the images of relevant objects.

Shape Quantization and Recognition with Randomized Trees

We explore a new approach to shape recognition based on a virtually infinite family of binary features (queries) of the image data, designed to accommodate prior information about shape invariance and regularity. Each query corresponds to a spatial arrangement of several local topographic codes (or tags), which are in themselves too primitive and common to be informative about shape. All the discriminating power derives from relative angles and distances among the tags. The important attributes of the queries are a natural partial ordering corresponding to increasing structure and complexity; semi-invariance, meaning that most shapes of a given class will answer the same way to two queries that are successive in the ordering; and stability, since the queries are not based on distinguished points and substructures.

No classifier based on the full feature set can be evaluated, and it is impossible to determine a priori which arrangements are informative. Our approach is to select informative features and build tree classifiers at the same time by inductive learning. In effect, each tree provides an approximation to the full posterior where the features chosen depend on the branch that is traversed. Due to the number and nature of the queries, standard decision tree construction based on a fixed-length feature vector is not feasible. Instead we entertain only a small random sample of queries at each node, constrain their complexity to increase with tree depth, and grow multiple trees. The terminal nodes are labeled by estimates of the corresponding posterior distribution over shape classes. An image is classified by sending it down every tree and aggregating the resulting distributions.

The method is applied to classifying handwritten digits and synthetic linear and nonlinear deformations of three hundred [Formula: see text] symbols. State-of-the-art error rates are achieved on the National Institute of Standards and Technology database of digits. The principal goal of the experiments on [Formula: see text] symbols is to analyze invariance, generalization error and related issues, and a comparison with artificial neural networks methods is presented in this context.

[Figure: see text]

Representation of Visual Features of Objects in the Inferotemporal Cortex

Cells in area TE of the inferotemporal cortex of the monkey brain selectively respond to various moderately complex object features, and those that respond to similar features cluster in a columnar region elongated vertical to the cortical surface. Columns representing related but different features partially overlap, and at least in some cases they comprise a continuous map of a piece of complex feature space. This continuous mapping is likely used for various computations, such as production of the image of the object at different viewing angles, illumination conditions, and articulation poses. Copyright 1996 Elsevier Science Ltd.

Divergent projections from the anterior inferotemporal area TE to the perirhinal and entorhinal cortices in the macaque monkey

Area TE is located at the latter part of the ventral visual cortical pathway, which is essential for visual recognition of objects. TE projects heavily to the perirhinal region, which is important for visual recognition memory of objects. To study the organization of projections from TE to the perirhinal (areas 35 and 36) and entorhinal (area 28) cortices, we made focal injections of Phaseolus vulgaris leucoagglutinin (PHA-L) and large injections of biocytin or wheat germ agglutinin conjugated to horseradish peroxidase (WGA-HRP) into anterior levels of TE in macaque monkeys. Injections of PHA-L into the ventral part of anterior TE (TEav) resulted in labeling of terminals distributed widely in area 36 (approximately one-half of its total extent), although the injection sites were limited to 0.7 mm in width. The labeled terminals tended to be denser in the medial part of area 36. There was less dense but definite labeling in area 35 and the lateral part of area 28. After a single injection of PHA-L or WGA-HRP into the dorsal part of anterior TE (TEad), labeled terminals were confined to a small region at the lateral part of area 36 (less than one-tenth of its total extent). The projections to areas 35 and 28 from TEad were much sparser than those from TEav. The different patterns of projections to the perirhinal and entorhinal cortices, together with previously reported differences in their afferent and other efferent connections, suggest the functional differentiation between TEav and TEad. The divergent projection from TEav to the perirhinal cortex may facilitate the association of different visual features in the perirhinal cortex.

Optical imaging of functional organization in the monkey inferotemporal cortex

To investigate the functional organization of object recognition, the technique of optical imaging was applied to the primate inferotemporal cortex, which is thought to be essential for object recognition. The features critical for the activation of single cells were first determined in unit recordings with electrodes. In the subsequent optical imaging, presentation of the critical features activated patchy regions around 0.5 millimeters in diameter, covering the site of the electrode penetration at which the critical feature had been determined. Because signals in optical imaging reflect average neuronal activities in the regions, the result directly indicates the regional clustering of cells responding to similar features.

Intrinsic Connections in the macaque inferior temporal cortex

Intrinsic connections in the inferior temporal cortex were analyzed by making extracellular injections of biocytin in Japanese macaques. Analysis was focused mainly on the dorsal part of area TE, in which a functional columnar organization has been shown. Interlaminar connections were analyzed in coronal section after laminar-specific microinjections, and intralaminar connections were examined from tangential sections. After injections at various depths in the dorsal TE, both axons and cell bodies were strongly labeled above or below the injection site in a columnar appearance. Axons from layer 3 ran in bundles towards the white matter and gave off prominent collaterals in layer 5. Ascending axons from lower to upper layers were also present (e.g., layers 4, 5, and 6 to layer 3). In tangential sections, there were abundant axons running parallel to the pia mater. These horizontal axons, particularly those in layers 2 and 3, produced patches of terminals 0.5 +/- 0.1 mm (mean +/- s.d.) in size and cylindrical in shape, spanning layers 1-3 or even to layers 4 and 5. In the tangential plane, they were distributed in an anisotropic manner around the injection. The farthest patch appeared at 4 mm from the injection site. The center-to-center distance between nearest-neighbor patches was 0.7 +/- 0.3 mm. These patches were found only within the dorsal TE and did not extend into the lower bank of the superior temporal sulcus or into the ventral part of area TE. Area TEO, which is a major afferent source to area TE, had axonal patches with spacing similar to those in area TE but with smaller sizes (0.4 +/- 0.1 mm). The results show that intrinsic horizontal axons both in area TE and in area TEO arborize in a patchy manner, as has been reported for several other cortical areas. In are TE, the size and spacing of the terminal patches match those of columns with similar stimulus selectivity. Thus, these patches may be related to the functional modularity in area TE. Vertical connections across layers and cylindrical patches of horizontal axons most likely contribute to the shared stimulus selectivity among cells within a column.