Category-specific organization in the human brain does not require visual experience

Representational geometry: integrating cognition, computation, and the brain

•

Representational geometry is a framework that enables us to relate brain, computation, and cognition.

•

Representations in brains and models can be characterized by representational distance matrices.

•

Distance matrices can be readily compared to test computational models.

•

We review recent insights into perception, cognition, memory, and action and discuss current challenges.

The cognitive concept of representation plays a key role in theories of brain information processing. However, linking neuronal activity to representational content and cognitive theory remains challenging. Recent studies have characterized the representational geometry of neural population codes by means of representational distance matrices, enabling researchers to compare representations across stages of processing and to test cognitive and computational theories. Representational geometry provides a useful intermediate level of description, capturing both the information represented in a neuronal population code and the format in which it is represented. We review recent insights gained with this approach in perception, memory, cognition, and action. Analyses of representational geometry can compare representations between models and the brain, and promise to explain brain computation as transformation of representational similarity structure.

Representations of individuals in ventral temporal cortex defined by faces and biographies

The fusiform gyrus responds more strongly to faces than to other categories of objects. This response could reflect either categorical detection of faces or recognition of particular facial identities. Recent fMRI studies have attempted to address the question of what information is encoded in these regions, but have reported mixed results. We tested whether the creation of richer identity representations via training on visual and social information, and the use of an adaptation design, would reveal more robust representations of these identities in ventral temporal cortex. Examining the patterns of activation across voxels in bilateral fusiform gyri, we identified unique patterns for particular identities. Attaching distinctive biographical information to identities did not increase the strength of these representations, but did produce a grouping effect: faces associated with the same amount of biographical information were represented more similarly to each other. These results are consistent with the possibility that identity exemplars are represented in posterior visual areas best known for their role in representing categorical information, and suggest that these areas may be sensitive to some forms of non-visual information, including from the social domain.

Ultra rapid object categorization: effects of level, animacy and context

It is widely agreed that in object categorization bottom-up and top-down influences interact. How top-down processes affect categorization has been primarily investigated in isolation, with only one higher level process at a time being manipulated. Here, we investigate the combination of different top-down influences (by varying the level of category, the animacy and the background of the object) and their effect on rapid object categorization. Subjects participated in a two-alternative forced choice rapid categorization task, while we measured accuracy and reaction times. Subjects had to categorize objects on the superordinate, basic or subordinate level. Objects belonged to the category animal or vehicle and each object was presented on a gray, congruent (upright) or incongruent (inverted) background. The results show that each top-down manipulation impacts object categorization and that they interact strongly. The best categorization was achieved on the superordinate level, providing no advantage for basic level in rapid categorization. Categorization between vehicles was faster than between animals on the basic level and vice versa on the subordinate level. Objects in homogenous gray background (context) yielded better overall performance than objects embedded in complex scenes, an effect most prominent on the subordinate level. An inverted background had no negative effect on object categorization compared to upright scenes. These results show how different top-down manipulations, such as category level, category type and background information, are related. We discuss the implications of top-down interactions on the interpretation of categorization results.

Neural pathways conveying novisual information to the visual cortex

The visual cortex has been traditionally considered as a stimulus-driven, unimodal system with a hierarchical organization. However, recent animal and human studies have shown that the visual cortex responds to non-visual stimuli, especially in individuals with visual deprivation congenitally, indicating the supramodal nature of the functional representation in the visual cortex. To understand the neural substrates of the cross-modal processing of the non-visual signals in the visual cortex, we firstly showed the supramodal nature of the visual cortex. We then reviewed how the nonvisual signals reach the visual cortex. Moreover, we discussed if these non-visual pathways are reshaped by early visual deprivation. Finally, the open question about the nature (stimulus-driven or top-down) of non-visual signals is also discussed.

Tool selectivity in left occipitotemporal cortex develops without vision

Previous studies have provided evidence for a tool-selective region in left lateral occipitotemporal cortex (LOTC). This region responds selectively to pictures of tools and to characteristic visual tool motion. The present human fMRI study tested whether visual experience is required for the development of tool-selective responses in left LOTC. Words referring to tools, animals, and nonmanipulable objects were presented auditorily to 14 congenitally blind and 16 sighted participants. Sighted participants additionally viewed pictures of these objects. In whole-brain group analyses, sighted participants showed tool-selective activity in left LOTC in both visual and auditory tasks. Importantly, virtually identical tool-selective LOTC activity was found in the congenitally blind group performing the auditory task. Furthermore, both groups showed equally strong tool-selective activity for auditory stimuli in a tool-selective LOTC region defined by the picture-viewing task in the sighted group. Detailed analyses in individual participants showed significant tool-selective LOTC activity in 13 of 14 blind participants and 14 of 16 sighted participants. The strength and anatomical location of this activity were indistinguishable across groups. Finally, both blind and sighted groups showed significant resting state functional connectivity between left LOTC and a bilateral frontoparietal network. Together, these results indicate that tool-selective activity in left LOTC develops without ever having seen a tool or its motion. This finding puts constraints on the possible role that this region could have in tool processing and, more generally, provides new insights into the principles shaping the functional organization of OTC.

Seeing human: distinct and overlapping neural signatures associated with two forms of dehumanization

The process of dehumanization, or thinking of others as less than human, is a phenomenon with significant societal implications. According to Haslam's (2006) model, two concepts of humanness derive from comparing humans with either animals or machines: individuals may be dehumanized by likening them to either animals or machines, or humanized by emphasizing differences from animals or machines. Recent work in cognitive neuroscience emphasizes understanding cognitive processes in terms of interactions between distributed cortical networks. It has been found that reasoning about internal mental states is associated with activation of the default mode network (DMN) and deactivation of the task positive network (TPN); whereas reasoning about mechanical processes produces the opposite pattern. We conducted two neuroimaging studies. The first examined the neural bases of dehumanization and its relation to these two brain networks, using images and voice-over social narratives which either implicitly contrasted or implicitly likened humans to either animals or machines. The second study addressed a discrepancy between findings from the first study and prior work on the neural correlates of dehumanization: using a design similar to prior work we examined neural responses to pictures of humans, animals and machines, presented without any social context. In both studies, human and humanizing conditions were associated with relatively high activity in the DMN and relatively low activity in the TPN. However, the non-human and dehumanizing conditions deviated in different ways: they demonstrated more marked changes either in the DMN or in the TPN. Notably, differences between the animal dehumanizing and humanizing conditions were most evident in regions associated with mechanistic reasoning, not in the mentalizing network. Conjunction analysis of contrasts from both paradigms revealed that only one region was consistently more active when participants saw human, a medial parietal region regarded as the central hub of the DMN. These findings provide a neural basis for Haslam's distinction between two types of dehumanization, and suggest that the DMN and TPN play opposing roles in creating a sense of moral concern.

Multivoxel pattern analysis reveals auditory motion information in MT+ of both congenitally blind and sighted individuals

Cross-modal plasticity refers to the recruitment of cortical regions involved in the processing of one modality (e.g. vision) for processing other modalities (e.g. audition). The principles determining how and where cross-modal plasticity occurs remain poorly understood. Here, we investigate these principles by testing responses to auditory motion in visual motion area MT+ of congenitally blind and sighted individuals. Replicating previous reports, we find that MT+ as a whole shows a strong and selective responses to auditory motion in congenitally blind but not sighted individuals, suggesting that the emergence of this univariate response depends on experience. Importantly, however, multivoxel pattern analyses showed that MT+ contained information about different auditory motion conditions in both blind and sighted individuals. These results were specific to MT+ and not found in early visual cortex. Basic sensitivity to auditory motion in MT+ is thus experience-independent, which may be a basis for the region's strong cross-modal recruitment in congenital blindness.

Selectivity for large nonmanipulable objects in scene-selective visual cortex does not require visual experience

The principles that determine the organization of object representations in ventral temporal cortex (VTC) remain elusive. Here, we focus on the parahippocampal place area (PPA), a region in medial VTC that has been shown to respond selectively to pictures of scenes. Recent studies further observed that this region also shows a preference for large nonmanipulable objects relative to other objects, which might reflect the suitability of large objects for navigation. The mechanisms underlying this selectivity remain poorly understood. We examined the extent to which PPA selectivity requires visual experience. Fourteen congenitally blind and matched sighted participants were tested on an auditory size judgment experiment involving large nonmanipulable objects, small objects (tools), and animals. Sighted participants additionally participated in a picture-viewing experiment. Replicating previous work, we found that the PPA responded selectively to large nonmanipulable objects, relative to tools and animals, in the sighted group viewing pictures. Importantly, this selectivity was also observed in the auditory experiment in both sighted and congenitally blind groups. In both groups, selectivity for large nonmanipulable objects was additionally observed in the retrosplenial complex (RSC) and the transverse occipital sulcus (TOS), regions previously implicated in scene perception and navigation. Finally, in both groups the PPA showed resting-state functional connectivity with TOS and RSC. These results provide new evidence that large object selectivity in PPA, and the intrinsic connectivity between PPA and other navigation-relevant regions, do not require visual experience. More generally, they show that the organization of object representations in VTC can develop, at least partly, without visual experience.

Alive and grasping: stable and rapid semantic access to an object category but not object graspability

How quickly do different kinds of conceptual knowledge become available following visual word perception? Resolving this question will inform neural and computational theories of visual word recognition and semantic memory use. We measured real-time brain activity using event-related brain potentials (ERPs) during a go/nogo task to determine the upper limit by which category-related knowledge (living/nonliving) and action-related knowledge (graspable/ungraspable) must have been accessed to influence a downstream decision process. We find that decision processes can be influenced by the living/nonliving distinction by 160ms after stimulus onset whereas information about (one-hand) graspability is not available before 300ms. We also provide evidence that rapid access to category-related knowledge occurs for all items, not just a subset of living, nonliving, graspable, or ungraspable ones, and for all participants regardless of their response speed. The latency of the N200 nogo effect by contrast is sensitive to decision speed. We propose a tentative hypothesis of the neural mechanisms underlying semantic access and a subsequent decision process.

Human Object-Similarity Judgments Reflect and Transcend the Primate-IT Object Representation

Primate inferior temporal (IT) cortex is thought to contain a high-level representation of objects at the interface between vision and semantics. This suggests that the perceived similarity of real-world objects might be predicted from the IT representation. Here we show that objects that elicit similar activity patterns in human IT (hIT) tend to be judged as similar by humans. The IT representation explained the human judgments better than early visual cortex, other ventral-stream regions, and a range of computational models. Human similarity judgments exhibited category clusters that reflected several categorical divisions that are prevalent in the IT representation of both human and monkey, including the animate/inanimate and the face/body division. Human judgments also reflected the within-category representation of IT. However, the judgments transcended the IT representation in that they introduced additional categorical divisions. In particular, human judgments emphasized human-related additional divisions between human and non-human animals and between man-made and natural objects. hIT was more similar to monkey IT than to human judgments. One interpretation is that IT has evolved visual-feature detectors that distinguish between animates and inanimates and between faces and bodies because these divisions are fundamental to survival and reproduction for all primate species, and that other brain systems serve to more flexibly introduce species-dependent and evolutionarily more recent divisions.

Insights into the origins of knowledge from the cognitive neuroscience of blindness

Children learn about the world through senses such as touch, smell, vision, and audition, but they conceive of the world in terms of objects, events, agents, and their mental states. A fundamental question in cognitive science is how nature and nurture contribute to the development of such conceptual categories. What innate mechanisms do children bring to the learning problem? How does experience contribute to development? In this article we discuss insights into these longstanding questions from cognitive neuroscience studies of blindness. Despite drastically different sensory experiences, behavioural and neuroscientific work suggests that blind children acquire typical concepts of objects, actions, and mental states. Blind people think and talk about these categories in ways that are similar to sighted people. Neuroimaging reveals that blind people make such judgements relying on the same neural mechanisms as sighted people. One way to interpret these findings is that neurocognitive development is largely hardwired, and so differences in experience have little consequence. Contrary to this interpretation, neuroimaging studies also show that blindness profoundly reorganizes the visual system. Most strikingly, developmental blindness enables "visual" circuits to participate in high-level cognitive functions, including language processing. Thus, blindness qualitatively changes sensory representations, but leaves conceptual representations largely unchanged. The effect of sensory experience on concepts is modest, despite the brain's potential for neuroplasticity.

Relative category-specific preservation in semantic dementia? Evidence from 35 cases

Category-specific deficits have rarely been reported in semantic dementia (SD). To our knowledge, only four previous studies have documented category-specific deficits, and these have focused on the living versus non-living things contrast rather than on more fine-grained semantic categories. This study aimed to determine whether a category-specific effect could be highlighted by a semantic sorting task administered to 35 SD patients once at baseline and again after 2 years and to 10 Alzheimer's disease patients (AD). We found a relative preservation of fruit and vegetables only in SD. This relative preservation of fruit and vegetables could be considered with regard to the importance of color knowledge in their discrimination. Indeed, color knowledge retrieval is known to depend on the left posterior fusiform gyrus which is relatively spared in SD. Finally, according to predictions of semantic memory models, our findings best fitted the Devlin and Gonnerman's computational account.

Spatial frequency tuning reveals interactions between the dorsal and ventral visual systems

It is widely argued that the ability to recognize and identify manipulable objects depends on the retrieval and simulation of action-based information associated with using those objects. Evidence for that view comes from fMRI studies that have reported differential BOLD contrast in dorsal visual stream regions when participants view manipulable objects compared with a range of baseline categories. An alternative interpretation is that processes internal to the ventral visual pathway are sufficient to support the visual identification of manipulable objects and that the retrieval of object-associated use information is contingent on analysis of the visual input by the ventral stream. Here, we sought to distinguish these two perspectives by exploiting the fact that the dorsal stream is largely driven by magnocellular input, which is biased toward low spatial frequency visual information. Thus, any tool-selective responses in parietal cortex that are driven by high spatial frequencies would be indicative of inputs from the ventral visual pathway. Participants viewed images of tools and animals containing only low, or only high, spatial frequencies during fMRI. We find an internal parcellation of left parietal "tool-preferring" voxels: Inferior aspects of left parietal cortex are driven by high spatial frequency information and have privileged connectivity with ventral stream regions that show similar category preferences, whereas superior regions are driven by low spatial frequency information. Our findings suggest that the automatic activation of complex object-associated manipulation knowledge is contingent on analysis of the visual input by the ventral visual pathway.

Early visual experience and the recognition of basic facial expressions: involvement of the middle temporal and inferior frontal gyri during haptic identification by the early blind

Face perception is critical for social communication. Given its fundamental importance in the course of evolution, the innate neural mechanisms can anticipate the computations necessary for representing faces. However, the effect of visual deprivation on the formation of neural mechanisms that underlie face perception is largely unknown. We previously showed that sighted individuals can recognize basic facial expressions by haptics surprisingly well. Moreover, the inferior frontal gyrus (IFG) and posterior superior temporal sulcus (pSTS) in the sighted subjects are involved in haptic and visual recognition of facial expressions. Here, we conducted both psychophysical and functional magnetic-resonance imaging (fMRI) experiments to determine the nature of the neural representation that subserves the recognition of basic facial expressions in early blind individuals. In a psychophysical experiment, both early blind and sighted subjects haptically identified basic facial expressions at levels well above chance. In the subsequent fMRI experiment, both groups haptically identified facial expressions and shoe types (control). The sighted subjects then completed the same task visually. Within brain regions activated by the visual and haptic identification of facial expressions (relative to that of shoes) in the sighted group, corresponding haptic identification in the early blind activated regions in the inferior frontal and middle temporal gyri. These results suggest that the neural system that underlies the recognition of basic facial expressions develops supramodally even in the absence of early visual experience.

The development of visual areas depends differently on visual experience

Visual experience plays an important role in the development of the visual cortex; however, recent functional imaging studies have shown that the functional organization is preserved in several higher-tier visual areas in congenitally blind subjects, indicating that maturation of visual areas depend unequally on visual experience. In this study, we aim to validate this hypothesis using a multimodality MRI approach. We found increased cortical thickness in the congenitally blind was present in the early visual areas and absent in the higher-tier ones, suggesting that the structural development of the visual cortex depends hierarchically on visual experience. In congenitally blind subjects, the decreased resting-state functional connectivity with the primary somatosensory cortex was more prominent in the early visual areas than in the higher-tier ones and were more pronounced in the ventral stream than in the dorsal one, suggesting that the development of functional organization of the visual cortex also depends differently on visual experience. Moreover, congenitally blind subjects showed normal or increased functional connectivity between ipsilateral higher-tier and early visual areas, suggesting an indirect corticocortical pathway through which somatosenroy information can reach the early visual areas. These findings support our hypothesis that the development of visual areas depends differently on visual experience.

Cross-modal plasticity preserves functional specialization in posterior parietal cortex

In congenitally blind individuals, many regions of the brain that are typically heavily involved in visual processing are recruited for a variety of nonvisual sensory and cognitive tasks (Rauschecker 1995; Pascual-Leone et al. 2005). This phenomenon-cross-modal plasticity-has been widely documented, but the principles that determine where and how cross-modal changes occur remain poorly understood (Bavelier and Neville 2002). Here, we evaluate the hypothesis that cross-modal plasticity respects the type of computations performed by a region, even as it changes the modality of the inputs over which they are carried out (Pascual-Leone and Hamilton 2001). We compared the fMRI signal in sighted and congenitally blind participants during proprioceptively guided reaching. We show that parietooccipital reach-related regions retain their functional role-encoding of the spatial position of the reach target-even as the dominant modality in this region changes from visual to nonvisual inputs. This suggests that the computational role of a region, independently of the processing modality, codetermines its potential cross-modal recruitment. Our findings demonstrate that preservation of functional properties can serve as a guiding principle for cross-modal plasticity even in visuomotor cortical regions, i.e. beyond the early visual cortex and other traditional visual areas.

Conceptual representations of perceptual knowledge

Many neuroimaging studies of semantic memory have argued that knowledge of an object's perceptual properties are represented in a modality-specific manner. These studies often base their argument on finding activation in the left-hemisphere fusiform gyrus-a region assumed to be involved in perceptual processing-when the participant is verifying verbal statements about objects and properties. In this paper, we report an extension of one of these influential papers-Kan, Barsalou, Solomon, Minor, and Thompson-Schill (2003 )-and present evidence for an amodal component in the representation and processing of perceptual knowledge. Participants were required to verify object-property statements (e.g., "cat-whiskers?"; "bear-wings?") while they were being scanned by functional magnetic resonance imaging (fMRI). We replicated Kan et al.'s activation in the left fusiform gyrus, but also found activation in regions of left inferior frontal gyrus (IFG) and middle-temporal gyrus, areas known to reflect amodal processes or representations. Further, only activations in the left IFG, an amodal area, were correlated with measures of behavioural performance.

Decoding semantics across fMRI sessions with different stimulus modalities: a practical MVPA study

Both embodied and symbolic accounts of conceptual organization would predict partial sharing and partial differentiation between the neural activations seen for concepts activated via different stimulus modalities. But cross-participant and cross-session variability in BOLD activity patterns makes analyses of such patterns with MVPA methods challenging. Here, we examine the effect of cross-modal and individual variation on the machine learning analysis of fMRI data recorded during a word property generation task. We present the same set of living and non-living concepts (land-mammals, or work tools) to a cohort of Japanese participants in two sessions: the first using auditory presentation of spoken words; the second using visual presentation of words written in Japanese characters. Classification accuracies confirmed that these semantic categories could be detected in single trials, with within-session predictive accuracies of 80–90%. However cross-session prediction (learning from auditory-task data to classify data from the written-word-task, or vice versa) suffered from a performance penalty, achieving 65–75% (still individually significant at p « 0.05). We carried out several follow-on analyses to investigate the reason for this shortfall, concluding that distributional differences in neither time nor space alone could account for it. Rather, combined spatio-temporal patterns of activity need to be identified for successful cross-session learning, and this suggests that feature selection strategies could be modified to take advantage of this.

Increased BOLD variability in the parietal cortex and enhanced parieto-occipital connectivity during tactile perception in congenitally blind individuals

Previous studies in early blind individuals posited a possible role of parieto-occipital connections in conveying nonvisual information to the visual occipital cortex. As a consequence of blindness, parietal areas would thus become able to integrate a greater amount of multimodal information than in sighted individuals. To verify this hypothesis, we compared fMRI-measured BOLD signal temporal variability, an index of efficiency in functional information integration, in congenitally blind and sighted individuals during tactile spatial discrimination and motion perception tasks. In both tasks, the BOLD variability analysis revealed many cortical regions with a significantly greater variability in the blind as compared to sighted individuals, with an overlapping cluster located in the left inferior parietal/anterior intraparietal cortex. A functional connectivity analysis using this region as seed showed stronger correlations in both tasks with occipital areas in the blind as compared to sighted individuals. As BOLD variability reflects neural integration and processing efficiency, these cross-modal plastic changes in the parietal cortex, even if described in a limited sample, reinforce the hypothesis that this region may play an important role in processing nonvisual information in blind subjects and act as a hub in the cortico-cortical pathway from somatosensory cortex to the reorganized occipital areas.

Body-part-specific Representations of Semantic Noun Categories

Word meaning processing in the brain involves ventrolateral temporal cortex, but a semantic contribution of the dorsal stream, especially frontocentral sensorimotor areas, has been controversial. We here examine brain activation during passive reading of object-related nouns from different semantic categories, notably animal, food, and tool words, matched for a range of psycholinguistic features. Results show ventral stream activation in temporal cortex along with category-specific activation patterns in both ventral and dorsal streams, including sensorimotor systems and adjacent pFC. Precentral activation reflected action-related semantic features of the word categories. Cortical regions implicated in mouth and face movements were sparked by food words, and hand area activation was seen for tool words, consistent with the actions implicated by the objects the words are used to speak about. Furthermore, tool words specifically activated the right cerebellum, and food words activated the left orbito-frontal and fusiform areas. We discuss our results in the context of category-specific semantic deficits in the processing of words and concepts, along with previous neuroimaging research, and conclude that specific dorsal and ventral areas in frontocentral and temporal cortex index visual and affective–emotional semantic attributes of object-related nouns and action-related affordances of their referent objects.

Neural representations of unfamiliar objects are modulated by sensorimotor experience

Sensory/functional accounts of semantic memory organization emphasize that object representations in the brain reflect the modalities involved in object knowledge acquisition. The present study aimed to elucidate the impact of different types of object-related sensorimotor experience on the neural representations of novel objects. Sixteen subjects engaged in an object matching task while their brain activity was assessed with functional magnetic resonance imaging (fMRI), before and after they acquired knowledge about previously unfamiliar objects. In three training sessions subjects learned about object function, actively manipulating only one set of objects (manipulation training objects, MTO), and visually exploring a second set (visual training objects, VTO). A third object set served as control condition and was not part of the training (no training objects, NTO). While training-related activation increases were observed in the fronto-parietal cortex for both VTO and MTO, post training activity in the left inferior/middle frontal gyrus and the left posterior inferior parietal lobule was higher for MTO than VTO and NTO. As revealed by Dynamic Causal Modeling of effective connectivity between the regions with enhanced post training activity, these effects were likely caused, respectively, by a down-regulation of a fronto-parietal tool use network in response to VTO, and by an increased connectivity for MTO. This pattern of findings indicates that the modalities involved in sensorimotor experience influence the formation of neural representations of objects in semantic memory, with manipulation experience specifically yielding higher activity in regions of the fronto-parietal cortex.

The representation of biological classes in the human brain

Evidence of category specificity from neuroimaging in the human visual system is generally limited to a few relatively coarse categorical distinctions-e.g., faces versus bodies, or animals versus artifacts-leaving unknown the neural underpinnings of fine-grained category structure within these large domains. Here we use fMRI to explore brain activity for a set of categories within the animate domain, including six animal species-two each from three very different biological classes: primates, birds, and insects. Patterns of activity throughout ventral object vision cortex reflected the biological classes of the stimuli. Specifically, the abstract representational space-measured as dissimilarity matrices defined between species-specific multivariate patterns of brain activity-correlated strongly with behavioral judgments of biological similarity of the same stimuli. This biological class structure was uncorrelated with structure measured in retinotopic visual cortex, which correlated instead with a dissimilarity matrix defined by a model of V1 cortex for the same stimuli. Additionally, analysis of the shape of the similarity space in ventral regions provides evidence for a continuum in the abstract representational space-with primates at one end and insects at the other. Further investigation into the cortical topography of activity that contributes to this category structure reveals the partial engagement of brain systems active normally for inanimate objects in addition to animate regions.

The nature of experience determines object representations in the visual system

Visual perceptual learning (PL) and perceptual expertise (PE) traditionally lead to different training effects and recruit different brain areas, but reasons for these differences are largely unknown. Here, we tested how the learning history influences visual object representations. Two groups were trained with tasks typically used in PL or PE studies, with the same novel objects, training duration and parafoveal stimulus presentation. We observed qualitatively different changes in the cortical representations of these objects following PL and PE training, replicating typical training effects in each field. These effects were also modulated by testing tasks, suggesting that experience interacts with attentional set and that the choice of testing tasks critically determines the pattern of training effects one can observe after a short-term visual training. Experience appears sufficient to account for prior differences in the neural locus of learning between PL and PE. The nature of the experience with an object's category can determine its representation in the visual system.

'Visual' acuity of the congenitally blind using visual-to-auditory sensory substitution

Sensory Substitution Devices (SSDs) convey visual information through sounds or touch, thus theoretically enabling a form of visual rehabilitation in the blind. However, for clinical use, these devices must provide fine-detailed visual information which was not yet shown for this or other means of visual restoration. To test the possible functional acuity conveyed by such devices, we used the Snellen acuity test conveyed through a high-resolution visual-to-auditory SSD (The vOICe). We show that congenitally fully blind adults can exceed the World Health Organization (WHO) blindness acuity threshold using SSDs, reaching the highest acuity reported yet with any visual rehabilitation approach. This demonstrates the potential capacity of SSDs as inexpensive, non-invasive visual rehabilitation aids, alone or when supplementing visual prostheses.

Fast and automatic activation of an abstract representation of money in the human ventral visual pathway

Money, when used as an incentive, activates the same neural circuits as rewards associated with physiological needs. However, unlike physiological rewards, monetary stimuli are cultural artifacts: how are monetary stimuli identified in the first place? How and when does the brain identify a valid coin, i.e. a disc of metal that is, by social agreement, endowed with monetary properties? We took advantage of the changes in the Euro area in 2002 to compare neural responses to valid coins (Euros, Australian Dollars) with neural responses to invalid coins that have lost all monetary properties (French Francs, Finnish Marks). We show in magneto-encephalographic recordings, that the ventral visual pathway automatically distinguishes between valid and invalid coins, within only ∼150 ms. This automatic categorization operates as well on coins subjects were familiar with as on unfamiliar coins. No difference between neural responses to scrambled controls could be detected. These results could suggest the existence of a generic, all-purpose neural representation of money that is independent of experience. This finding is reminiscent of a central assumption in economics, money fungibility, or the fact that a unit of money is substitutable to another. From a neural point of view, our findings may indicate that the ventral visual pathway, a system previously thought to analyze visual features such as shape or color and to be influenced by daily experience, could also able to use conceptual attributes such as monetary validity to categorize familiar as well as unfamiliar visual objects. The symbolic abilities of the posterior fusiform region suggested here could constitute an efficient neural substrate to deal with culturally defined symbols, independently of experience, which probably fostered money's cultural emergence and success.

Closely overlapping responses to tools and hands in left lateral occipitotemporal cortex

The perception of object-directed actions performed by either hands or tools recruits regions in left fronto-parietal cortex. Here, using functional MRI (fMRI), we tested whether the common role of hands and tools in object manipulation is also reflected in the distribution of response patterns to these categories in visual cortex. In two experiments we found that static pictures of hands and tools activated closely overlapping regions in left lateral occipitotemporal cortex (LOTC). Left LOTC responses to tools selectively overlapped with responses to hands but not with responses to whole bodies, nonhand body parts, other objects, or visual motion. Multivoxel pattern analysis in left LOTC indicated a high degree of similarity between response patterns to hands and tools but not between hands or tools and other body parts. Finally, functional connectivity analysis showed that the left LOTC hand/tool region was selectively connected, relative to neighboring body-, motion-, and object-responsive regions, with regions in left intraparietal sulcus and left premotor cortex that have previously been implicated in hand/tool action-related processing. Taken together, these results suggest that action-related object properties shared by hands and tools are reflected in the organization of high-order visual cortex. We propose that the functional organization of high-order visual cortex partly reflects the organization of downstream functional networks, such as the fronto-parietal action network, due to differences within visual cortex in the connectivity to these networks.

Conceptual structure: Towards an integrated neuro-cognitive account

How are the meanings of concepts represented and processed? We present a cognitive model of conceptual representations and processing - the Conceptual Structure Account (CSA; Tyler & Moss, 2001) - as an example of a distributed, feature-based approach. In a first section, we describe the CSA and evaluate relevant neuropsychological and experimental behavioral data. We discuss studies using linguistic and non-linguistic stimuli, which are both presumed to access the same conceptual system. We then take the CSA as a framework for hypothesising how conceptual knowledge is represented and processed in the brain. This neuro-cognitive approach attempts to integrate the distributed feature-based characteristics of the CSA with a distributed and feature-based model of sensory object processing. Based on a review of relevant functional imaging and neuropsychological data, we argue that distributed accounts of feature-based representations have considerable explanatory power, and that a cognitive model of conceptual representations is needed to understand their neural bases.

Decoding patterns of human brain activity

Considerable information about mental states can be decoded from noninvasive measures of human brain activity. Analyses of brain activity patterns can reveal what a person is seeing, perceiving, attending to, or remembering. Moreover, multidimensional models can be used to investigate how the brain encodes complex visual scenes or abstract semantic information. Such feats of "brain reading" or "mind reading," though impressive, raise important conceptual, methodological, and ethical issues. What does successful decoding reveal about the cognitive functions performed by a brain region? How should brain signals be spatially selected and mathematically combined to ensure that decoding reflects inherent computations of the brain rather than those performed by the decoder? We highlight recent advances and describe how multivoxel pattern analysis can provide a window into mind-brain relationships with unprecedented specificity, when carefully applied. However, as brain-reading technology advances, issues of neuroethics and mental privacy will be important to consider.

Spatial language processing in the blind: evidence for a supramodal representation and cortical reorganization

Neuropsychological and imaging studies have shown that the left supramarginal gyrus (SMG) is specifically involved in processing spatial terms (e.g. above, left of), which locate places and objects in the world. The current fMRI study focused on the nature and specificity of representing spatial language in the left SMG by combining behavioral and neuronal activation data in blind and sighted individuals. Data from the blind provide an elegant way to test the supramodal representation hypothesis, i.e. abstract codes representing spatial relations yielding no activation differences between blind and sighted. Indeed, the left SMG was activated during spatial language processing in both blind and sighted individuals implying a supramodal representation of spatial and other dimensional relations which does not require visual experience to develop. However, in the absence of vision functional reorganization of the visual cortex is known to take place. An important consideration with respect to our finding is the amount of functional reorganization during language processing in our blind participants. Therefore, the participants also performed a verb generation task. We observed that only in the blind occipital areas were activated during covert language generation. Additionally, in the first task there was functional reorganization observed for processing language with a high linguistic load. As the visual cortex was not specifically active for spatial contents in the first task, and no reorganization was observed in the SMG, the latter finding further supports the notion that the left SMG is the main node for a supramodal representation of verbal spatial relations.

Division of labor between lateral and ventral extrastriate representations of faces, bodies, and objects

The occipito-temporal cortex is strongly implicated in carrying out the high-level computations associated with vision. In human neuroimaging studies, focal regions are consistently found within this broad region that respond strongly and selectively to faces, bodies, or objects. A notable feature of these selective regions is that they are found in pairs. In the posterior-lateral occipito-temporal cortex, focal selectivity is found for faces (occipital face area), bodies (extrastriate body area), and objects (lateral occipital). These three areas are found bilaterally and at close quarters to each other. Likewise, in the ventro-medial occipito-temporal cortex, three similar category-selective regions are found, also in proximity to each other: for faces (fusiform face area), bodies (fusiform body area), and objects (posterior fusiform). Here we review some of the extensive evidence on the functional properties of these areas with two aims. First, we seek to identify principles that distinguish the posterior-lateral and ventro-medial clusters of selective regions but that apply generally within each cluster across the three stimulus kinds. Our review identifies and elaborates several principles by which these relationships hold. In brief, the posterior-lateral representations are more primitive, local, and stimulus-driven relative to the ventro-medial representations, which in contrast are more invariant to visual features, global, and linked to the subjective percept. Second, because the evidence base of studies that compare both posterior-lateral and ventro-medial representations of faces, bodies, and objects is still relatively small, we seek to provoke more cross-talk among the research strands that are traditionally separate. We identify several promising approaches for such future work.

What drives the organization of object knowledge in the brain?

Various forms of category-specificity have been described at both the cognitive and neural levels, inviting the inference that different semantic domains are processed by distinct, dedicated mechanisms. In this paper, we argue for an extension of a domain-specific interpretation to these phenomena that is based on network-level analyses of functional coupling among brain regions. On this view, domain-specificity in one region of the brain emerges because of innate connectivity with a network of regions that also process information about that domain. Recent findings are reviewed that converge with this framework, and a new direction is outlined for understanding the neural principles that shape the organization of conceptual knowledge.

Typical neural representations of action verbs develop without vision

Many empiricist theories hold that concepts are composed of sensory-motor primitives. For example, the meaning of the word "run" is in part a visual image of running. If action concepts are partly visual, then the concepts of congenitally blind individuals should be altered in that they lack these visual features. We compared semantic judgments and neural activity during action verb comprehension in congenitally blind and sighted individuals. Participants made similarity judgments about pairs of nouns and verbs that varied in the visual motion they conveyed. Blind adults showed the same pattern of similarity judgments as sighted adults. We identified the left middle temporal gyrus (lMTG) brain region that putatively stores visual-motion features relevant to action verbs. The functional profile and location of this region was identical in sighted and congenitally blind individuals. Furthermore, the lMTG was more active for all verbs than nouns, irrespective of visual-motion features. We conclude that the lMTG contains abstract representations of verb meanings rather than visual-motion images. Our data suggest that conceptual brain regions are not altered by the sensory modality of learning.

Modality-independent coding of spatial layout in the human brain

In many nonhuman species, neural computations of navigational information such as position and orientation are not tied to a specific sensory modality [1, 2]. Rather, spatial signals are integrated from multiple input sources, likely leading to abstract representations of space. In contrast, the potential for abstract spatial representations in humans is not known, because most neuroscientific experiments on human navigation have focused exclusively on visual cues. Here, we tested the modality independence hypothesis with two functional magnetic resonance imaging (fMRI) experiments that characterized computations in regions implicated in processing spatial layout [3]. According to the hypothesis, such regions should be recruited for spatial computation of 3D geometric configuration, independent of a specific sensory modality. In support of this view, sighted participants showed strong activation of the parahippocampal place area (PPA) and the retrosplenial cortex (RSC) for visual and haptic exploration of information-matched scenes but not objects. Functional connectivity analyses suggested that these effects were not related to visual recoding, which was further supported by a similar preference for haptic scenes found with blind participants. Taken together, these findings establish the PPA/RSC network as critical in modality-independent spatial computations and provide important evidence for a theory of high-level abstract spatial information processing in the human brain.

The unique role of the visual word form area in reading

Reading systematically activates the left lateral occipitotemporal sulcus, at a site known as the visual word form area (VWFA). This site is reproducible across individuals/scripts, attuned to reading-specific processes, and partially selective for written strings relative to other categories such as line drawings. Lesions affecting the VWFA cause pure alexia, a selective deficit in word recognition. These findings must be reconciled with the fact that human genome evolution cannot have been influenced by such a recent and culturally variable activity as reading. Capitalizing on recent functional magnetic resonance imaging experiments, we provide strong corroborating evidence for the hypothesis that reading acquisition partially recycles a cortical territory evolved for object and face recognition, the prior properties of which influenced the form of writing systems.

EEG decoding of semantic category reveals distributed representations for single concepts

Achieving a clearer picture of categorial distinctions in the brain is essential for our understanding of the conceptual lexicon, but much more fine-grained investigations are required in order for this evidence to contribute to lexical research. Here we present a collection of advanced data-mining techniques that allows the category of individual concepts to be decoded from single trials of EEG data. Neural activity was recorded while participants silently named images of mammals and tools, and category could be detected in single trials with an accuracy well above chance, both when considering data from single participants, and when group-training across participants. By aggregating across all trials, single concepts could be correctly assigned to their category with an accuracy of 98%. The pattern of classifications made by the algorithm confirmed that the neural patterns identified are due to conceptual category, and not any of a series of processing-related confounds. The time intervals, frequency bands and scalp locations that proved most informative for prediction permit physiological interpretation: the widespread activation shortly after appearance of the stimulus (from 100 ms) is consistent both with accounts of multi-pass processing, and distributed representations of categories. These methods provide an alternative to fMRI for fine-grained, large-scale investigations of the conceptual lexicon.

Cortical organization of environmental sounds by attribute

Semantic knowledge is supported by a widely distributed neuronal network, with differential patterns of activation depending upon experimental stimulus or task demands. Despite a wide body of knowledge on semantic object processing from the visual modality, the response of this semantic network to environmental sounds remains relatively unknown. Here, we used fMRI to investigate how access to different conceptual attributes from environmental sound input modulates this semantic network. Using a range of living and manmade sounds, we scanned participants whilst they carried out an object attribute verification task. Specifically, we tested visual perceptual, encyclopedic, and categorical attributes about living and manmade objects relative to a high-level auditory perceptual baseline to investigate the differential patterns of response to these contrasting types of object-related attributes, whilst keeping stimulus input constant across conditions. Within the bilateral distributed network engaged for processing environmental sounds across all conditions, we report here a highly significant dissociation within the left hemisphere between the processing of visual perceptual and encyclopedic attributes of objects.