Generalisation to novel exemplars of learned shape categories based on visual and auditory spatial cues does not benefit from multisensory information

Although the integration of information across multiple senses can enhance object representations in memory, how multisensory information affects the formation of categories is uncertain. In particular, it is unclear to what extent categories formed from multisensory information benefit object recognition over unisensory inputs. Two experiments investigated the categorisation of novel auditory and visual objects, with categories defined by spatial similarity, and tested generalisation to novel exemplars. Participants learned to categorise exemplars based on visual-only (geometric shape), auditory-only (spatially defined soundscape) or audio-visual spatial cues. Categorisation to learned as well as novel exemplars was then tested under the same sensory learning conditions. For all learning modalities, categorisation generalised to novel exemplars. However, there was no evidence of enhanced categorisation performance for learned multisensory exemplars. At best, bimodal performance approximated that of the most accurate unimodal condition, although this was observed only for a subset of exemplars within a category. These findings provide insight into the perceptual processes involved in the formation of categories and have relevance for understanding the sensory nature of object representations underpinning these categories.

Superordinate Categorization Based on the Perceptual Organization of Parts

Plants and animals are among the most behaviorally significant superordinate categories for humans. Visually assigning objects to such high-level classes is challenging because highly distinct items must be grouped together (e.g., chimpanzees and geckos) while more similar items must sometimes be separated (e.g., stick insects and twigs). As both animals and plants typically possess complex multi-limbed shapes, the perceptual organization of shape into parts likely plays a crucial rule in identifying them. Here, we identify a number of distinctive growth characteristics that affect the spatial arrangement and properties of limbs, yielding useful cues for differentiating plants from animals. We developed a novel algorithm based on shape skeletons to create many novel object pairs that differ in their part structure but are otherwise very similar. We found that particular part organizations cause stimuli to look systematically more like plants or animals. We then generated other 110 sequences of shapes morphing from animal- to plant-like appearance by modifying three aspects of part structure: sprouting parts, curvedness of parts, and symmetry of part pairs. We found that all three parameters correlated strongly with human animal/plant judgments. Together our findings suggest that subtle changes in the properties and organization of parts can provide powerful cues in superordinate categorization.

How Visual Expertise Changes Representational Geometry: A Behavioral and Neural Perspective

Many people develop expertise in specific domains of interest, such as chess, microbiology, radiology, and, the case in point in our study: ornithology. It is poorly understood to what extent such expertise alters brain function. Previous neuroimaging studies of expertise have typically focused upon the category level, for example, selectivity for birds versus nonbird stimuli. We present a multivariate fMRI study focusing upon the representational similarity among objects of expertise at the subordinate level. We compare the neural representational spaces of experts and novices to behavioral judgments. At the behavioral level, ornithologists (n = 20) have more fine-grained and task-dependent representations of item similarity that are more consistent among experts compared to control participants. At the neural level, the neural patterns of item similarity are more distinct and consistent in experts than in novices, which is in line with the behavioral results. In addition, these neural patterns in experts show stronger correlations with behavior compared to novices. These findings were prominent in frontal regions, and some effects were also found in occipitotemporal regions. This study illustrates the potential of an analysis of representational geometry to understand to what extent expertise changes neural information processing.

Object recognition is enabled by an experience-dependent appraisal of visual features in the brain's value system

This paper addresses perceptual synthesis by comparing responses evoked by visual stimuli before and after they are recognized, depending on prior exposure. Using magnetoencephalography, we analyzed distributed patterns of neuronal activity - evoked by Mooney figures - before and after they were recognized as meaningful objects. Recognition induced changes were first seen at 100-120 ​ms, for both faces and tools. These early effects - in right inferior and middle occipital regions - were characterized by an increase in power in the absence of any changes in spatial patterns of activity. Within a later 210-230 ​ms window, a quite different type of recognition effect appeared. Regions of the brain's value system (insula, entorhinal cortex and cingulate of the right hemisphere for faces and right orbitofrontal cortex for tools) evinced a reorganization of their neuronal activity without an overall power increase in the region. Finally, we found that during the perception of disambiguated face stimuli, a face-specific response in the right fusiform gyrus emerged at 240-290 ​ms, with a much greater latency than the well-known N170m component, and, crucially, followed the recognition effect in the value system regions. These results can clarify one of the most intriguing issues of perceptual synthesis, namely, how a limited set of high-level predictions, which is required to reduce the uncertainty when resolving the ill-posed inverse problem of perception, can be available before category-specific processing in visual cortex. We suggest that a subset of local spatial features serves as partial cues for a fast re-activation of object-specific appraisal by the value system. The ensuing top-down feedback from value system to visual cortex, in particular, the fusiform gyrus enables high levels of processing to form category-specific predictions. This descending influence of the value system was more prominent for faces than for tools, the fact that reflects different dependence of these categories on value-related information.

An investigation of the effect of temporal contiguity training on size-tolerant representations in object-selective cortex

The human visual system has a remarkable ability to reliably identify objects across variations in appearance, such as variations in viewpoint, lighting and size. Here we used fMRI in humans to test whether temporal contiguity training with natural and altered image dynamics can respectively build and break neural size tolerance for objects. Participants (N ​= ​23) were presented with sequences of images of "growing" and "shrinking" objects. In half of the trials, the object also changed identity when the size change happened. According to the temporal contiguity hypothesis, and studies with a similar paradigm in monkeys, this training process should alter size tolerance. After the training phase, BOLD responses to each of the object images were measured in the scanner. Neural patterns in LOC and V1 contained information on size, similarity and identity. In LOC, the representation of object identity was partially invariant to changes in size. However, temporal contiguity training did not affect size tolerance in LOC. Size tolerance in human object-selective cortex is more robust to variations in input statistics than expected based on prior work in monkeys supporting the temporal contiguity hypothesis.

Forced conceptual thought induced by electrical stimulation of the left prefrontal gyrus involves widespread neural networks

BACKGROUND

Early accounts of forced thought were reported at the onset of a focal seizure, and characterized as vague, repetitive, and involuntary intellectual auras distinct from perceptual or psychic hallucinations or illusions. Here, we examine the neural underpinnings involved in conceptual thought by presenting a series of 3 patients with epilepsy reporting intrusive thoughts during electrical stimulation of the left lateral prefrontal cortex (PFC) during invasive surgical evaluation. We illustrate the widespread networks involved through two independent brain imaging modalities: resting state functional magnetic resonance imaging (fMRI) (rs-fMRI) and task-based meta-analytic connectivity modeling (MACM).

METHODS

We report the clinical and stimulation characteristics of three patients with left hemispheric language dominance who demonstrate forced thought with functional mapping. To examine the brain networks underlying this phenomenon, we used the regions of interest (ROI) centered at the active electrode pairs. We modeled functional networks using two approaches: (1) rs-fMRI functional connectivity analysis, representing 81 healthy controls and (2) meta-analytic connectivity modeling (MACM), representing 8260 healthy subjects. We also determined the overlapping regions between these three subjects' rs-fMRI and MACM networks through a conjunction analysis.

RESULTS

We identified that left PFC was associated with a large-scale functional network including frontal, temporal, and parietal regions, a network that has been associated with multiple cognitive functions including semantics, speech, attention, working memory, and explicit memory.

CONCLUSIONS

We illustrate the neural networks involved in conceptual thought through a unique patient population and argue that PFC supports this function through activation of a widespread network.

Category learning can alter perception and its neural correlates

Learned Categorical Perception (CP) occurs when the members of different categories come to look more dissimilar ("between-category separation") and/or members of the same category come to look more similar ("within-category compression") after a new category has been learned. To measure learned CP and its physiological correlates we compared dissimilarity judgments and Event Related Potentials (ERPs) before and after learning to sort multi-featured visual textures into two categories by trial and error with corrective feedback. With the same number of training trials and feedback, about half the subjects succeeded in learning the categories ("Learners": criterion 80% accuracy) and the rest did not ("Non-Learners"). At both lower and higher levels of difficulty, successful Learners showed significant between-category separation-and, to a lesser extent, within-category compression-in pairwise dissimilarity judgments after learning, compared to before; their late parietal ERP positivity (LPC, usually interpreted as decisional) also increased and their occipital N1 amplitude (usually interpreted as perceptual) decreased. LPC amplitude increased with response accuracy and N1 amplitude decreased with between-category separation for the Learners. Non-Learners showed no significant changes in dissimilarity judgments, LPC or N1, within or between categories. This is behavioral and physiological evidence that category learning can alter perception. We sketch a neural net model predictive of this effect.

Training Humans to Categorize Monkey Calls: Auditory Feature- and Category-Selective Neural Tuning Changes

Grouping auditory stimuli into common categories is essential for a variety of auditory tasks, including speech recognition. We trained human participants to categorize auditory stimuli from a large novel set of morphed monkey vocalizations. Using fMRI-rapid adaptation (fMRI-RA) and multi-voxel pattern analysis (MVPA) techniques, we gained evidence that categorization training results in two distinct sets of changes: sharpened tuning to monkey call features (without explicit category representation) in left auditory cortex and category selectivity for different types of calls in lateral prefrontal cortex. In addition, the sharpness of neural selectivity in left auditory cortex, as estimated with both fMRI-RA and MVPA, predicted the steepness of the categorical boundary, whereas categorical judgment correlated with release from adaptation in the left inferior frontal gyrus. These results support the theory that auditory category learning follows a two-stage model analogous to the visual domain, suggesting general principles of perceptual category learning in the human brain.

Shape discrimination along morph-spaces

We investigated the dimensions defining mental shape space, by measuring shape discrimination thresholds along "morph-spaces" defined by pairs of shapes. Given any two shapes, one can construct a morph-space by taking weighted averages of their boundary vertices (after normalization), creating a continuum of shapes ranging from the first shape to the second. Previous studies of morphs between highly familiar shape categories (e.g. truck and turkey) have shown elevated discrimination at category boundaries, reflecting a kind of "categorical perception" in shape space. Here, we use this technique to explore the underlying representation of unfamiliar shapes. Subjects were shown two shapes at nearby points along a morph-space, and asked to judge whether they were the same or different, with an adaptive procedure used to estimate discrimination thresholds at each point along the morph-space. We targeted several potentially important categorical distinctions, such one- vs. two-part shapes, two- vs. three-part shapes, changes in symmetry structure, and other potentially important distinctions. Observed discrimination thresholds showed substantial and systematic deviations from uniformity at different points along each shape continuum, meaning that subjects were consistently better at discriminating at certain points along each morph-space than at others. We introduce a shape similarity measure, based on Bayesian skeletal shape representations, which gives a good account of the observed variations in shape sensitivity.

EEG frequency tagging reveals higher order intermodulation components as neural markers of learned holistic shape representations

Shape perception is intrinsically holistic: combinations of features give rise to configurations with emergent properties that are different from the sum of the parts. The current study investigated neural markers of holistic shape representations learned by means of categorization training. We used the EEG frequency tagging technique, where two parts of a shape stimulus were 'tagged' by modifying their contrast at different temporal frequencies. Signals from both parts are integrated and, as a result, emergent frequency components (so-called, intermodulation responses, IMs), caused by nonlinear interaction of two frequency signals, are observed in the EEG spectrum. First, participants were trained in 4 sessions to discriminate highly similar, unfamiliar shapes into two categories, defined based on the combination of features. After training, EEG was recorded while frequency-tagged shapes from either the trained or the untrained shape family were presented. For all IMs combined, no learning effects were detected, but post hoc analyses of higher-order IMs revealed stronger occipital and occipito-temporal IMs for both trained and untrained exemplars of the trained shape family as compared to the untrained shape family. In line with recent findings, we suggest that the higher-order IMs may reflect high-level visual computations, like holistic shape categorization, resulting from a cascade of non-linear operations. Higher order frequency responses are relatively low in power, hence results should be interpreted cautiously and future research is needed to confirm these effects. In general, these findings are, to our knowledge, the first to show IMs as a neural correlate of perceptual learning.

Temporal Contiguity Training Influences Behavioral and Neural Measures of Viewpoint Tolerance

Humans can often recognize faces across viewpoints despite the large changes in low-level image properties a shift in viewpoint introduces. We present a behavioral and an fMRI adaptation experiment to investigate whether this viewpoint tolerance is reflected in the neural visual system and whether it can be manipulated through training. Participants saw training sequences of face images creating the appearance of a rotating head. Half of the sequences showed faces undergoing veridical changes in appearance across the rotation (non-morph condition). The other half were non-veridical: during rotation, the face simultaneously morphed into another face. This procedure should successfully associate frontal face views with side views of the same or a different identity, and, according to the temporal contiguity hypothesis, thus enhance viewpoint tolerance in the non-morph condition and/or break tolerance in the morph condition. Performance on the same/different task in the behavioral experiment (N = 20) was affected by training. There was a significant interaction between training (associated/not associated) and identity (same/different), mostly reflecting a higher confusion of different identities when they were associated during training. In the fMRI study (N = 20), fMRI adaptation effects were found for same-viewpoint images of untrained faces, but no adaptation for untrained faces was present across viewpoints. Only trained faces which were not morphed during training elicited a slight adaptation across viewpoints in face-selective regions. However, both in the behavioral and in the neural data the effects were small and weak from a statistical point of view. Overall, we conclude that the findings are not inconsistent with the proposal that temporal contiguity can influence viewpoint tolerance, with more evidence for tolerance when faces are not morphed during training.

Incremental learning of perceptual and conceptual representations and the puzzle of neural repetition suppression

Incremental learning models of long-term perceptual and conceptual knowledge hold that neural representations are gradually acquired over many individual experiences via Hebbian-like activity-dependent synaptic plasticity across cortical connections of the brain. In such models, variation in task relevance of information, anatomic constraints, and the statistics of sensory inputs and motor outputs lead to qualitative alterations in the nature of representations that are acquired. Here, the proposal that behavioral repetition priming and neural repetition suppression effects are empirical markers of incremental learning in the cortex is discussed, and research results that both support and challenge this position are reviewed. Discussion is focused on a recent fMRI-adaptation study from our laboratory that shows decoupling of experience-dependent changes in neural tuning, priming, and repetition suppression, with representational changes that appear to work counter to the explicit task demands. Finally, critical experiments that may help to clarify and resolve current challenges are outlined.

Attention to distinguishing features in object recognition: An interactive-iterative framework

This article advances a framework that casts object recognition as a process of discrimination between alternative object identities, in which top-down and bottom-up processes interact-iteratively when necessary-with attention to distinguishing features playing a critical role. In two experiments, observers discriminated between different types of artificial fish. In parallel, a secondary, variable-SOA visual-probe detection task was used to examine the dynamics of visual attention. In Experiment 1, the fish varied in three distinguishing features: one indicating the general category (saltwater, freshwater), and one of the two other features indicating the specific type of fish within each category. As predicted, in the course of recognizing each fish, attention was allocated iteratively to the distinguishing features in an optimal manner: first to the general category feature, and then, based on its value, to the second feature that identified the specific fish. In Experiment 2, two types of fish could be discriminated on the basis of either of two distinguishing features, one more visually discriminable than the other. On some of the trials, one of the two alternative distinguishing features was occluded. As predicted, in the course of recognizing each fish, attention was directed initially to the more discriminable distinguishing feature, but when this feature was occluded, it was then redirected to the less discriminable feature. The implications of these findings, and the interactive-iterative framework they support, are discussed with regard to several fundamental issues having a long history in the literatures on object recognition, object categorization, and visual perception in general.

Changing similarity: Stable and flexible modulations of psychological dimensions

Successfully categorizing objects requires discriminating between relevant and irrelevant dimensions (e.g., shape, color). Categorization can lead to changes in the visual system that stretch psychological space, making relevant dimensions more distinct and irrelevant dimensions more similar. These changes are known as dimensional modulation (DM) and they can be both stable and flexible in nature. The current study examined the interaction between stable DM and flexible DM, as well as the time course of relative changes in similarity. Using a two-dimensional space of cars, participants learned to categorize the space and then completed a target identification task during EEG recording. We found that attention, operationally defined as the selection negativity, was sensitive to category-relevance and appeared to selectively enhance previously irrelevant differences in the service of a target detection task. In contrast, we found that late decisional stages, operationally defined as the P3 b, were less sensitive to relevance and instead more sensitive to the number of morphsteps that separated targets from non-targets. Thus, it appears that relative similarity between targets and non-targets dynamically changed over the time course of individual decisions. Similarity between exemplars was greater along the irrelevant than the relevant dimension early on in the time course but a compensatory allocation of attention led to similarity being optimized among all dimensions for later stages. This finding is important because it 1) provides a new source of converging evidence for stable DM and 2) links a neural measure of attentional modulation with facilitation of an unpracticed, but task-relevant perceptual dimension.

Neuropsychological evidence for the temporal dynamics of category-specific naming

Multiple accounts have been proposed to explain category-specific recognition impairments. Some suggest that category-specific deficits may be caused by a deficit in recurrent processing between the levels of a hierarchically organized visual object recognition system. Here, we tested predictions of interactive processing theories on the emergence of category-selective naming deficits in neurologically intact observers and in patient GA, a single case showing a category-specific impairment for natural objects after a herpes simplex encephalitis infection. Fragmented object outlines were repeatedly presented until correct naming occurred (maximum 10 times), and the fragments increased in length with every repetition. We studied how shape complexity, object category, and fragment curvature influence the timing of correct object identification. The results of a survival analysis are consistent with the idea that deficits in recurrent processing between low- and high-level visual object representations can cause category-selective impairments.

Multivoxel Object Representations in Adult Human Visual Cortex Are Flexible: An Associative Learning Study

Learning associations between co-occurring events enables us to extract structure from our environment. Medial-temporal lobe structures are critical for associative learning. However, the role of the ventral visual pathway (VVP) in associative learning is not clear. Do multivoxel object representations in the VVP reflect newly formed associations? We show that VVP multivoxel representations become more similar to each other after human participants learn arbitrary new associations between pairs of unrelated objects (faces, houses, cars, chairs). Participants were scanned before and after 15 days of associative learning. To evaluate how object representations changed, a classifier was trained on discriminating two nonassociated categories (e.g., faces/houses) and tested on discriminating their paired associates (e.g., cars/chairs). Because the associations were arbitrary and counterbalanced across participants, there was initially no particular reason for this cross-classification decision to tend toward either alternative. Nonetheless, after learning, cross-classification performance increased in the VVP (but not hippocampus), on average by 3.3%, with some voxels showing increases of up to 10%. For example, a chair multivoxel representation that initially resembled neither face nor house representations was, after learning, classified as more similar to that of faces for participants who associated chairs with faces and to that of houses for participants who associated chairs with houses. Additionally, learning produced long-lasting perceptual consequences. In a behavioral priming experiment performed several months later, the change in cross-classification performance was correlated with the degree of priming. Thus, VVP multivoxel representations are not static but become more similar to each other after associative learning.

Auditory Perceptual Category Formation Does Not Require Perceptual Warping

Categorical perception occurs when a perceiver's stimulus classifications affect their ability to make fine perceptual discriminations and is the most intensively studied form of category learning. On the basis of categorical perception studies, it has been proposed that category learning proceeds by the deformation of an initially homogeneous perceptual space (“perceptual warping”), so that stimuli within the same category are perceived as more similar to each other (more difficult to tell apart) than stimuli that are the same physical distance apart but that belong to different categories. Here, we present a significant counterexample in which robust category learning occurs without these differential perceptual space deformations. Two artificial categories were defined along the dimension of pitch for a perceptually unfamiliar, multidimensional class of sounds. A group of participants (selected on the basis of their listening abilities) were trained to sort sounds into these two arbitrary categories. Category formation, verified empirically, was accompanied by a heightened sensitivity along the entire pitch range, as indicated by changes in an EEG index of implicit perceptual distance (mismatch negativity), with no significant resemblance to the local perceptual deformations predicted by categorical perception. This demonstrates that robust categories can be initially formed within a continuous perceptual dimension without perceptual warping. We suggest that perceptual category formation is a flexible, multistage process sequentially combining different types of learning mechanisms rather than a single process with a universal set of behavioral and neural correlates.

Category Learning Stretches Neural Representations in Visual Cortex

We review recent work that shows how learning to categorize objects changes how those objects are represented in the mind and the brain. After category learning, visual perception of objects is enhanced along perceptual dimensions that were relevant to the learned categories, an effect we call dimensional modulation (DM). DM stretches object representations along category-relevant dimensions and shrinks them along category-irrelevant dimensions. The perceptual advantage for category-relevant dimensions extends beyond categorization and can be observed during visual discrimination and other tasks that do not depend on the learned categories. fMRI shows that category learning causes ventral stream neural populations in visual cortex representing objects along a category-relevant dimension to become more distinct. These results are consistent with a view that specific aspects of cognitive tasks associated with objects can account for how our visual system responds to objects.

Knowledge is power: how conceptual knowledge transforms visual cognition

In this review, we synthesize the existing literature demonstrating the dynamic interplay between conceptual knowledge and visual perceptual processing. We consider two theoretical frameworks that demonstrate interactions between processes and brain areas traditionally considered perceptual or conceptual. Specifically, we discuss categorical perception, in which visual objects are represented according to category membership, and highlight studies showing that category knowledge can penetrate early stages of visual analysis. We next discuss the embodied account of conceptual knowledge, which holds that concepts are instantiated in the same neural regions required for specific types of perception and action, and discuss the limitations of this framework. We additionally consider studies showing that gaining abstract semantic knowledge about objects and faces leads to behavioral and electrophysiological changes that are indicative of more efficient stimulus processing. Finally, we consider the role that perceiver goals and motivation may play in shaping the interaction between conceptual and perceptual processing. We hope to demonstrate how pervasive such interactions between motivation, conceptual knowledge, and perceptual processing are in our understanding of the visual environment, and to demonstrate the need for future research aimed at understanding how such interactions arise in the brain.

The perceptual effects of learning object categories that predict perceptual goals

In classic category learning studies, subjects typically learn to assign items to 1 of 2 categories, with no further distinction between how items on each side of the category boundary should be treated. In real life, however, we often learn categories that dictate further processing goals, for instance, with objects in only 1 category requiring further individuation. Using methods from category learning and perceptual expertise, we studied the perceptual consequences of experience with objects in tasks that rely on attention to different dimensions in different parts of the space. In 2 experiments, subjects first learned to categorize complex objects from a single morphspace into 2 categories based on 1 morph dimension, and then learned to perform a different task, either naming or a local feature judgment, for each of the 2 categories. A same-different discrimination test before and after each training measured sensitivity to feature dimensions of the space. After initial categorization, sensitivity increased along the category-diagnostic dimension. After task association, sensitivity increased more for the category that was named, especially along the nondiagnostic dimension. The results demonstrate that local attentional weights, associated with individual exemplars as a function of task requirements, can have lasting effects on perceptual representations.

Categorical interoception: perceptual organization of sensations from inside

Adequate perception of bodily sensations is essential to protect health. However, misinterpretation of signals from within the body is common and can be fatal, for example, in asthma or cardiovascular disease. We suggest that placing interoceptive stimuli into interoceptive categories (e.g., the category of symptoms vs. the category of benign sensations) leads to perceptual generalization effects that may underlie misinterpretation. In two studies, we presented stimuli inducing respiratory effort (respiratory loads) either organized into categories or located on a continuous dimension. We found pervasive effects of categorization on magnitude estimations, affective stimulus evaluations, stimulus recognition, and breathing behavior. These findings indicate the need for broadening perspectives on interoception to include basal processes of stimulus organization, in order for interoceptive bias to be understood. The results are relevant to a wide range of interoception-related phenomena, from emotion to symptom perception.

Task- and experience-dependent cortical selectivity to features informative for categorization

In this study, we bridge the gap between monkey electrophysiological recordings that showed selective responses to informative features and human fMRI data that demonstrated increased and selective responses to trained objects. Human participants trained with computer-generated fish stimuli. For each participant, two features of the fish were informative for category membership and two features were uninformative. After training, participants showed higher perceptual sensitivity to the informative dimensions. An fMRI adaptation paradigm revealed that during categorization the right inferior frontal gyrus and occipitotemporal cortex were selectively responsive to the informative features. These selective cortical responses were experience dependent; they were not present for the entire trained object, but specific for those features that were informative for categorization. Responses in the inferior frontal gyrus showed category selectivity. Moreover, selectivity to the informative features correlated with performance on the categorization task during scanning. This all suggests that the frontal cortex is involved in actively categorizing objects and that it uses informative features to do so while ignoring those features that do not contribute category information. Occipitotemporal cortex also showed selectivity to the informative features during the categorization task. Interestingly, this area showed a positive correlation of performance during training and selectivity to the informative features and a negative correlation with selectivity to the uninformative features. This indicates that training enhanced sensitivity to trained items and decreased sensitivity to uninformative features. The absence of sensitivity for informative features during a color change detection task indicates that there is a strong component of task-related processing of these features.

Time course of shape and category selectivity revealed by EEG rapid adaptation

A hallmark of human cognition is the ability to rapidly assign meaning to sensory stimuli. It has been suggested that this fast visual object categorization ability is accomplished by a feedforward processing hierarchy consisting of shape-selective neurons in occipito-temporal cortex that feed into task circuits in frontal cortex computing conceptual category membership. We performed an EEG rapid adaptation study to test this hypothesis. Participants were trained to categorize novel stimuli generated with a morphing system that precisely controlled both stimulus shape and category membership. We subsequently performed EEG recordings while participants performed a category matching task on pairs of successively presented stimuli. We used space-time cluster analysis to identify channels and latencies exhibiting selective neural responses. Neural signals before 200 msec on posterior channels demonstrated a release from adaptation for shape changes, irrespective of category membership, compatible with a shape- but not explicitly category-selective neural representation. A subsequent cluster with anterior topography appeared after 200 msec and exhibited release from adaptation consistent with explicit categorization. These signals were subsequently modulated by perceptual uncertainty starting around 300 msec. The degree of category selectivity of the anterior signals was strongly predictive of behavioral performance. We also observed a posterior category-selective signal after 300 msec exhibiting significant functional connectivity with the initial anterior category-selective signal. In summary, our study supports the proposition that perceptual categorization is accomplished by the brain within a quarter second through a largely feedforward process culminating in frontal areas, followed by later category-selective signals in posterior regions.

Toward a unified model of face and object recognition in the human visual system

Our understanding of the mechanisms and neural substrates underlying visual recognition has made considerable progress over the past 30 years. During this period, accumulating evidence has led many scientists to conclude that objects and faces are recognised in fundamentally distinct ways, and in fundamentally distinct cortical areas. In the psychological literature, in particular, this dissociation has led to a palpable disconnect between theories of how we process and represent the two classes of object. This paper follows a trend in part of the recognition literature to try to reconcile what we know about these two forms of recognition by considering the effects of learning. Taking a widely accepted, self-organizing model of object recognition, this paper explains how such a system is affected by repeated exposure to specific stimulus classes. In so doing, it explains how many aspects of recognition generally regarded as unusual to faces (holistic processing, configural processing, sensitivity to inversion, the other-race effect, the prototype effect, etc.) are emergent properties of category-specific learning within such a system. Overall, the paper describes how a single model of recognition learning can and does produce the seemingly very different types of representation associated with faces and objects.

Motor training increases the stability of activation patterns in the primary motor cortex

Learning to be skillful is an endowed talent of humans, but neural mechanisms underlying behavioral improvement remain largely unknown. Some studies have reported that the mean magnitude of neural activation is increased after learning, whereas others have instead shown decreased activation. In this study, we used functional magnetic resonance imaging (fMRI) to investigate learning-induced changes in the neural activation in the human brain with a classic motor training task. Specifically, instead of comparing the mean magnitudes of activation before and after training, we analyzed the learning-induced changes in multi-voxel spatial patterns of neural activation. We observed that the stability of the activation patterns, or the similarity of the activation patterns between the even and odd runs of the fMRI scans, was significantly increased in the primary motor cortex (M1) after training. By contrast, the mean magnitude of neural activation remained unchanged. Therefore, our study suggests that learning shapes the brain by increasing the stability of the activation patterns, therefore providing a new perspective in understanding the neural mechanisms underlying learning.

How category learning affects object representations: not all morphspaces stretch alike

How does learning to categorize objects affect how people visually perceive them? Behavioral, neurophysiological, and neuroimaging studies have tested the degree to which category learning influences object representations, with conflicting results. Some studies have found that objects become more visually discriminable along dimensions relevant to previously learned categories, while others have found no such effect. One critical factor we explore here lies in the structure of the morphspaces used in different studies. Studies finding no increase in discriminability often use blended morphspaces, with morphparents lying at corners of the space. By contrast, studies finding increases in discriminability use factorial morphspaces, defined by separate morphlines forming axes of the space. Using the same 4 morphparents, we created both factorial and blended morphspaces matched in pairwise discriminability. Category learning caused a selective increase in discriminability along the relevant dimension of the factorial space, but not in the blended space, and led to the creation of functional dimensions in the factorial space, but not in the blended space. These findings demonstrate that not all morphspaces stretch alike: Only some morphspaces support enhanced discriminability to relevant object dimensions following category learning. Our results have important implications for interpreting neuroimaging studies reporting little or no effect of category learning on object representations in the visual system: Those studies may have been limited by their use of blended morphspaces.

The time course of shape discrimination in the human brain

The lateral occipital cortex (LOC) activates selectively to images of intact objects versus scrambled controls, is selective for the figure-ground relationship of a scene, and exhibits at least some degree of invariance for size and position. Because of these attributes, it is considered to be a crucial part of the object recognition pathway. Here we show that human LOC is critically involved in perceptual decisions about object shape. High-density EEG was recorded while subjects performed a threshold-level shape discrimination task on texture-defined figures segmented by either phase or orientation cues. The appearance or disappearance of a figure region from a uniform background generated robust visual evoked potentials throughout retinotopic cortex as determined by inverse modeling of the scalp voltage distribution. Contrasting responses from trials containing shape changes that were correctly detected (hits) with trials in which no change occurred (correct rejects) revealed stimulus-locked, target-selective activity in the occipital visual areas LOC and V4 preceding the subject's response. Activity that was locked to the subjects' reaction time was present in the LOC. Response-locked activity in the LOC was determined to be related to shape discrimination for several reasons: shape-selective responses were silenced when subjects viewed identical stimuli but their attention was directed away from the shapes to a demanding letter discrimination task; shape-selectivity was present across four different stimulus configurations used to define the figure; LOC responses correlated with participants' reaction times. These results indicate that decision-related activity is present in the LOC when subjects are engaged in threshold-level shape discriminations.

Learning selective top-down control enhances performance in a visual categorization task

We model the putative neuronal and synaptic mechanisms involved in learning a visual categorization task, taking inspiration from single-cell recordings in inferior temporal cortex (ITC). Our working hypothesis is that learning the categorization task involves both bottom-up, ITC to prefrontal cortex (PFC), and top-down (PFC to ITC) synaptic plasticity and that the latter enhances the selectivity of the ITC neurons encoding the task-relevant features of the stimuli, thereby improving the signal-to-noise ratio. We test this hypothesis by modeling both areas and their connections with spiking neurons and plastic synapses, ITC acting as a feature-selective layer and PFC as a category coding layer. This minimal model gives interesting clues as to properties and function of the selective feedback signal from PFC to ITC that help solving a categorization task. In particular, we show that, when the stimuli are very noisy because of a large number of nonrelevant features, the feedback structure helps getting better categorization performance and decreasing the reaction time. It also affects the speed and stability of the learning process and sharpens tuning curves of ITC neurons. Furthermore, the model predicts a modulation of neural activities during error trials, by which the differential selectivity of ITC neurons to task-relevant and task-irrelevant features diminishes or is even reversed, and modulations in the time course of neural activities that appear when, after learning, corrupted versions of the stimuli are input to the network.

A Developmental Difference in Shape Processing and Word-Shape Associations between 4 and 6.5 Year Olds

In distinguishing individual shapes (defined by their contours), older children (6.5 years of age on average) performed better than younger children (4 years of age on average), and, although the task did not involve any categorization or generalization, the error pattern was qualitatively affected by shape differences that are generally common distinctions between objects belonging to different categories. The influence of these shape differences was also observed for unfamiliar shapes, demonstrating that the influence of categorization experience was not modulated by the retrieval of shape features from known categories but rather related to a different perception of shape by age. The results suggest a direct influence of categorization experience on more abstract shape processing. When children were distinguishing shapes, new words were paired with the target shapes, and in 2 additional tasks, the acquired name–shape associations were tested. The younger age group was able to remember more words correctly.

Category learning increases discriminability of relevant object dimensions in visual cortex

Learning to categorize objects can transform how they are perceived, causing relevant perceptual dimensions predictive of object category to become enhanced. For example, an expert mycologist might become attuned to species-specific patterns of spacing between mushroom gills but learn to ignore cap textures attributable to varying environmental conditions. These selective changes in perception can persist beyond the act of categorizing objects and influence our ability to discriminate between them. Using functional magnetic resonance imaging adaptation, we demonstrate that such category-specific perceptual enhancements are associated with changes in the neural discriminability of object representations in visual cortex. Regions within the anterior fusiform gyrus became more sensitive to small variations in shape that were relevant during prior category learning. In addition, extrastriate occipital areas showed heightened sensitivity to small variations in shape that spanned the category boundary. Visual representations in cortex, just like our perception, are sensitive to an object's history of categorization.

Human brain activity predicts individual differences in prior knowledge use during decisions

Studies by cognitive psychologists, psychophysicists, neuroscientists, and economists provide ample evidence that humans use prior knowledge to bias decisions adaptively. In this study, we sought to locate and investigate the brain areas mediating this behavior. Participants viewed ambiguous abstract shapes and decided whether a shape was of Category A (smoother) or B (bumpier). The decision was made in the context of one of two prior knowledge cues, 80/20 and 50/50. The 80/20 cue indicated that upcoming shapes had an 80% probability of being of one category, for example, B, and a 20% probability of being of the other. The 50/50 cue indicated that upcoming shapes had an equal probability of being of either category. The shift in bias produced by the 80/20 cue relative to the 50/50 cue was of the predicted sign for every subject but varied in magnitude. We searched for brain regions in which activity changes correlated with the extent of the bias shift; these were dorsolateral pFC (middle frontal gyrus), inferior frontal junction, anterior insula, inferior parietal lobule, intraparietal sulcus, head of the caudate, posterior cingulate cortex, and fusiform gyrus. The findings indicate that an individual's brain activity in these regions reflects the extent to which that individual makes use of prior knowledge to bias decisions. We also created within-ROI tuning curves by binning the shape curvature levels and plotting brain activity levels at each of the nine bins. In the fronto-parietal and anterior insula ROIs, the tuning curves peaked at targets contraindicated by the prior knowledge cue (e.g., Category B targets if the 80/20 cue meant 20% probability B). The increased activity in these regions likely indicates a no-go response when sufficient perceptual evidence favored the alternative contraindicated by the 80/20 cue.

Improving perception to make distant connections closer

One of the challenges for perceptually grounded accounts of high-level cognition is to explain how people make connections and draw inferences between situations that superficially have little in common. Evidence suggests that people draw these connections even without having explicit, verbalizable knowledge of their bases. Instead, the connections are based on sub-symbolic representations that are grounded in perception, action, and space. One reason why people are able to spontaneously see relations between situations that initially appear to be unrelated is that their eventual perceptions are not restricted to initial appearances. Training and strategic deployment allow our perceptual processes to deliver outputs that would have otherwise required abstract or formal reasoning. Even without people having any privileged access to the internal operations of perceptual modules, these modules can be systematically altered so as to better serve our high-level reasoning needs. Moreover, perceptually based processes can be altered in a number of ways to closely approximate formally sanctioned computations. To be concrete about mechanisms of perceptual change, we present 21 illustrations of ways in which we alter, adjust, and augment our perceptual systems with the intention of having them better satisfy our needs.

Activation of Fusiform Face Area by Greebles Is Related to Face Similarity but Not Expertise

Some of the brain areas in the ventral temporal lobe, such as the fusiform face area (FFA), are critical for face perception in humans, but what determines this specialization is a matter of debate. The face specificity hypothesis claims that faces are processed in a domain-specific way. Alternatively, the expertise hypothesis states that the FFA is specialized in processing objects of expertise. To disentangle these views, some previous experiments used an artificial class of novel objects called Greebles. These experiments combined a learning and fMRI paradigm. Given the high impact of the results in the literature, we replicated and further investigated this paradigm. In our experiment, eight participants were trained for ten 1-hr sessions at identifying Greebles. We scanned participants before and after training and examined responses in FFA and lateral occipital complex. Most importantly and in contrast to previous reports, we found a neural inversion effect for Greebles before training. This result suggests that people process the “novel” Greebles as faces, even before training. This prediction was confirmed in a postexperimental debriefing. In addition, we did not find an increase of the inversion effect for Greebles in the FFA after training. This indicates that the activity in the FFA for Greebles does not depend on the degree of expertise acquired with the objects but on the interpretation of the stimuli as face-related.

Similarity dependency of the change in ERP component N1 accompanying with the object recognition learning

Performance during object recognition across views is largely dependent on inter-object similarity. The present study was designed to investigate the similarity dependency of object recognition learning on the changes in ERP component N1. Human subjects were asked to train themselves to recognize novel objects with different inter-object similarity by performing object recognition tasks. During the tasks, images of an object had to be discriminated from the images of other objects irrespective of the viewpoint. When objects had a high inter-object similarity, the ERP component, N1 exhibited a significant increase in both the amplitude and the latency variation across objects during the object recognition learning process, and the N1 amplitude and latency variation across the views of the same objects decreased significantly. In contrast, no significant changes were found during the learning process when using objects with low inter-object similarity. The present findings demonstrate that the changes in the variation of N1 that accompany the object recognition learning process are dependent upon the inter-object similarity and imply that there is a difference in the neuronal representation for object recognition when using objects with high and low inter-object similarity.

Neural representations for object perception: structure, category, and adaptive coding

Object perception is one of the most remarkable capacities of the primate brain. Owing to the large and indeterminate dimensionality of object space, the neural basis of object perception has been difficult to study and remains controversial. Recent work has provided a more precise picture of how 2D and 3D object structure is encoded in intermediate and higher-level visual cortices. Yet, other studies suggest that higher-level visual cortex represents categorical identity rather than structure. Furthermore, object responses are surprisingly adaptive to changes in environmental statistics, implying that learning through evolution, development, and also shorter-term experience during adulthood may optimize the object code. Future progress in reconciling these findings will depend on more effective sampling of the object domain and direct comparison of these competing hypotheses.

Towards a new kind of experimental psycho-aesthetics? Reflections on the Parallellepipeda project

Experimental psycho-aesthetics-the science aimed at understanding the factors that determine aesthetic experience-is reviewed briefly as background to describe the Parallellepipeda project, a cross-over project between artists and scientists in Leuven. In particular, I sketch how it started and developed further, with close interactions between the participating artists and scientists. A few examples of specific research projects are mentioned to illustrate the kind of research questions we address and the methodological approach we have taken. We often found an effect of providing participants with additional information, a difference between novice and expert participants, and a shift with increasing experience with an artwork, in the direction of tolerating more complexity and acquiring more order from it. By establishing more connections between parts of an artwork and more associations to the artwork, it becomes a stronger Gestalt, which is more easily mastered by the viewer and leads to increased appreciation. In the final part of the paper, I extract some general lessons from the project regarding a possible new way of doing psycho-aesthetics research, which is able to solve some of the problems of traditional experimental psycho-aesthetics (eg, trade-off between experimental control and ecological validity).

Division of labor between lateral and ventral extrastriate representations of faces, bodies, and objects

The occipito-temporal cortex is strongly implicated in carrying out the high-level computations associated with vision. In human neuroimaging studies, focal regions are consistently found within this broad region that respond strongly and selectively to faces, bodies, or objects. A notable feature of these selective regions is that they are found in pairs. In the posterior-lateral occipito-temporal cortex, focal selectivity is found for faces (occipital face area), bodies (extrastriate body area), and objects (lateral occipital). These three areas are found bilaterally and at close quarters to each other. Likewise, in the ventro-medial occipito-temporal cortex, three similar category-selective regions are found, also in proximity to each other: for faces (fusiform face area), bodies (fusiform body area), and objects (posterior fusiform). Here we review some of the extensive evidence on the functional properties of these areas with two aims. First, we seek to identify principles that distinguish the posterior-lateral and ventro-medial clusters of selective regions but that apply generally within each cluster across the three stimulus kinds. Our review identifies and elaborates several principles by which these relationships hold. In brief, the posterior-lateral representations are more primitive, local, and stimulus-driven relative to the ventro-medial representations, which in contrast are more invariant to visual features, global, and linked to the subjective percept. Second, because the evidence base of studies that compare both posterior-lateral and ventro-medial representations of faces, bodies, and objects is still relatively small, we seek to provoke more cross-talk among the research strands that are traditionally separate. We identify several promising approaches for such future work.

Development of differential sensitivity for shape changes resulting from linear and nonlinear planar transformations

A shape bias for extending names to objects that look visually similar has been commonly accepted but it is hard to define which kind of shape dissimilarities are diagnostic for the identity of an object. Here, we present a transformational approach to describe shape differences that can incorporate many significant shape features. We introduce two kinds of transformations: one kind concerns linear transformations of the image plane (affine transformations), generally limiting shape variations within the borders of basic-level categories; the other kind concerns nonlinear continuous transformations of the image plane (topological transformations), allowing all kinds of shape variation crossing and not crossing the borders of basic-level categories. We administered stimulus pairs differing in these shape transformations to children of 3 years to 7 years old in a delayed match-to-sample task. With increasing age, especially between 5 years and 6 years, children became more sensitive to the topological deformations that are relevant for between-category distinctions, indicating that acquired categorical knowledge in early years induces perceptual learning of the relevant generic shape differences between categories.

Learning alters the tuning of functional magnetic resonance imaging patterns for visual forms

Learning is thought to facilitate the recognition of objects by optimizing the tuning of visual neurons to behaviorally relevant features. However, the learning mechanisms that shape neural selectivity for visual forms in the human brain remain essentially unknown. Here, we combine behavioral and functional magnetic resonance imaging (fMRI) measurements to test the mechanisms that mediate enhanced behavioral sensitivity in the discrimination of visual forms after training. In particular, we used high-resolution fMRI and multivoxel pattern classification methods to investigate fine learning-dependent changes in neural preference for global forms. We measured the observers' choices when discriminating between concentric and radial patterns presented in noise before and after training. Similarly, we measured the choices of a pattern classifier when predicting each stimulus from fMRI activity. Comparing the performance of human observers and classifiers demonstrated that learning alters the observers' sensitivity to visual forms and the tuning of fMRI activation patterns in visual areas selective for task-relevant features. In particular, training on low-signal stimuli enhanced the amplitude but reduced the width of pattern-based tuning functions in higher dorsal and ventral visual areas. Thus, our findings suggest that learning of visual patterns is implemented by enhancing the response to the preferred stimulus category and reducing the response to nonpreferred stimuli in higher extrastriate visual cortex.

Dynamic norm-based encoding for unfamiliar shapes in human visual cortex

Previous studies have argued that faces and other objects are encoded in terms of their deviation from a class prototype or norm. This prototype is associated with a smaller neural population response compared with nonprototype objects. However, it is still unclear (1) whether a norm-based representation can emerge for unfamiliar or novel object classes through visual experience at the time scale of an experiment and (2) whether the results from previous studies are caused by the prototypicality of a stimulus, by the physical properties of individual stimuli independent from the stimulus distribution, and/or by the trial-to-trial adaptation. Here we show with a combined behavioral and event-related fMRI study in humans that a short amount of visual experience with exemplars from novel object classes determines which stimulus is represented as the norm. Prototypicality effects were observed at the behavioral level by behavioral asymmetries during a stimulus comparison task. The fMRI data revealed that class exemplars closest to the prototypes--the perceived average of each class--were associated with a smaller response in the anterior part of the visual object-selective cortex compared with other class exemplars. By dissociating between the physical characteristics and the prototypicality status of the stimuli and by controlling for trial-to-trial adaptation, we can firmly conclude for the first time that high-level visual areas represent the identity of exemplars using a dynamic, norm-based encoding principle.

The role of top-down task context in learning to perceive objects

In high-level perceptual regions of the ventral visual pathway in humans, experience shapes the functional properties of the cortex: the fusiform face area responds most strongly to faces of familiar rather than unfamiliar races, and the visual word form area (VWFA) is tuned only to familiar orthographies. But are these regions affected only by the bottom-up stimulus information they receive during learning, or does the effect of perceptual experience depend on the way that stimulus information is used during learning? Here, we test the hypothesis that top-down influences (i.e., task context) modulate the effect of perceptual experience on functional selectivities of the high-level visual cortex. Specifically, we test whether experience with novel visual stimuli produces a greater effect on the VWFA when those stimuli are associated with meanings (via association learning) but produces a greater effect on shape-processing regions when trained in a discrimination task without associated meanings. Our result supports this hypothesis and further shows that learning is transferred to novel objects that share parts with the trained objects. Thus, the effects of experience on selectivities of the high-level visual cortex depend on the task context in which that experience occurs and the perceptual processing strategy by which objects are encoded during learning.

Broad and narrow conceptual tuning in the human frontal lobes

Previous work has implicated prefrontal cortices in selecting among and retrieving conceptual information stored elsewhere. However, recent neurophysiological work in monkeys suggests that prefrontal cortex may play a more direct role in representing conceptual information in a flexible context-specific manner. Here, we investigate the nature of visual object representations from perceptual to conceptual levels in an unbiased data-driven manner using a functional magnetic resonance imaging adaptation paradigm with pictures of animals. Throughout much of occipital cortex, activity was highly sensitive to changes in 2D stimulus form, consistent with tuning to form and position within retinotopic coordinates and matching an automated measure of shape similarity. Broad superordinate conceptual information was represented as early as extrastriate and posterior ventral temporal cortex. These regions were not completely invariant to form, suggesting that form similarity remains an important organizational constraint into the temporal cortex. Separate sites within prefrontal cortex represented broad and narrow conceptual tuning, with more anterior sites tuned narrowly to close conceptual associates in a manner that was invariant to stimulus form/position and that matched independent similarity ratings of the stimuli. The combination of broad and narrow conceptual tuning within prefrontal cortex may support flexible selection, retrieval, and classification of objects at different levels of categorical abstraction.

Category training induces cross-modal object representations in the adult human brain

The formation of cross-modal object representations was investigated using a novel paradigm that was previously successful in establishing unimodal visual category learning in monkeys and humans. The stimulus set consisted of six categories of bird shapes and sounds that were morphed to create different exemplars of each category. Subjects learned new cross-modal bird categories using a one-back task. Over time, the subjects became faster and more accurate in categorizing the birds. After 3 days of training, subjects were scanned while passively viewing and listening to trained and novel bird types. Stimulus blocks consisted of bird sounds only, bird pictures only, matching pictures and sounds (cross-modal congruent), and mismatching pictures and sounds (cross-modal incongruent). fMRI data showed unimodal and cross-modal training effects in the right fusiform gyrus. In addition, the left STS showed cross-modal training effects in the absence of unimodal training effects. Importantly, for both the right fusiform gyrus and the left STS, the newly formed cross-modal representation was specific for the trained categories. Learning did not generalize to incongruent combinations of learned sounds and shapes; their response did not differ from the response to novel cross-modal bird types. Moreover, responses were larger for congruent than for incongruent cross-modal bird types in the right fusiform gyrus and STS, providing further evidence that categorization training induced the formation of meaningful cross-modal object representations.

Beyond shape: how you learn about objects affects how they are represented in visual cortex

BACKGROUND

Experience can alter how objects are represented in the visual cortex. But experience can take different forms. It is unknown whether the kind of visual experience systematically alters the nature of visual cortical object representations.

METHODOLOGY/PRINCIPAL FINDINGS

We take advantage of different training regimens found to produce qualitatively different types of perceptual expertise behaviorally in order to contrast the neural changes that follow different kinds of visual experience with the same objects. Two groups of participants went through training regimens that required either subordinate-level individuation or basic-level categorization of a set of novel, artificial objects, called "Ziggerins". fMRI activity of a region in the right fusiform gyrus increased after individuation training and was correlated with the magnitude of configural processing of the Ziggerins observed behaviorally. In contrast, categorization training caused distributed changes, with increased activity in the medial portion of the ventral occipito-temporal cortex relative to more lateral areas.

CONCLUSIONS/SIGNIFICANCE

Our results demonstrate that the kind of experience with a category of objects can systematically influence how those objects are represented in visual cortex. The demands of prior learning experience therefore appear to be one factor determining the organization of activity patterns in visual cortex.

The neural basis of visual object learning

Object vision in human and nonhuman primates is often cited as a primary example of adult plasticity in neural information processing. It has been hypothesized that visual experience leads to single neurons in the monkey brain with strong selectivity for complex objects, and to regions in the human brain with a preference for particular categories of highly familiar objects. This view suggests that adult visual experience causes dramatic local changes in the response properties of high-level visual cortex. Here, we review the current neurophysiological and neuroimaging evidence and find that the available data support a different conclusion: adult visual experience introduces moderate, relatively distributed effects that modulate a pre-existing, rich and flexible set of neural object representations.