NEUROPSYCHOLOGICAL AND NEUROIMAGING PERSPECTIVES ON CONCEPTUAL KNOWLEDGE: AN INTRODUCTION

Temporal differences and commonalities between hand and tool neural processing

Object recognition is a complex cognitive process that relies on how the brain organizes object-related information. While spatial principles have been extensively studied, less studied temporal dynamics may also offer valuable insights into this process, particularly when neural processing overlaps for different categories, as it is the case of the categories of hands and tools. Here we focus on the differences and/or similarities between the time-courses of hand and tool processing under electroencephalography (EEG). Using multivariate pattern analysis, we compared, for different time points, classification accuracy for images of hands or tools when compared to images of animals. We show that for particular time intervals (~ 136-156 ms and ~ 252-328 ms), classification accuracy for hands and for tools differs. Furthermore, we show that classifiers trained to differentiate between tools and animals generalize their learning to classification of hand stimuli between ~ 260-320 ms and ~ 376-500 ms after stimulus onset. Classifiers trained to distinguish between hands and animals, on the other hand, were able to extend their learning to the classification of tools at ~ 150 ms. These findings suggest variations in semantic features and domain-specific differences between the two categories, with later-stage similarities potentially related to shared action processing for hands and tools.

Preliminary normative data for 12 categories using semantic verbal fluency: The role of animacy

Verbal fluency tasks are a common part of neuropsychological batteries and are frequently used in clinical and research practices to support the diagnosis of neurological impairments. Semantic verbal fluency is most frequently examined using only the category of animals. Little is known about the differences other semantic categories may present on semantic verbal fluency performance. The purpose of this study was to establish preliminary categorical normative data across twelve categories comprising of different animate and inanimate categories using semantic verbal fluency in neurotypical Lebanese-speaking adults and to determine the impact of category affiliation (animacy) on semantic verbal fluency performance. The task was administered to seventy female and male adults aged between 19 and 79 years having different educational levels. Participants generated the greatest number of exemplars for the category of body parts. Participants then produced exemplars in the following decreasing order: animals, vegetables, fruits, clothes, kitchen utensils, naturals, electronics, furniture, means of transportation, tools, and accessories. The animate categories were associated with the greatest number of exemplars compared to the inanimate. Clustering strategy might have been reinforced by the shared properties of animates.

Primacy of Functional Knowledge in Semantic Representations: The Case of Living and Nonliving Things

In 3 experiments, participants decided whether sensory and functional features were true of living and nonliving concepts. In Experiments 1 and 2, concepts were presented twice: test phase followed study phase after either 3 min (Experiment 1) or 3 s (Experiment 2). At test, concepts were paired with the same feature as that at study, or a different feature from either the same modality (within-modality priming) or another modality (cross-modality priming). In both experiments functional decisions were faster than sensory decisions for living and nonliving concepts. Whilst no semantic priming occurred between study and test in Experiment 1, the shorter study–test interval of Experiment 2 did lead to test phase semantic priming. Here there was greater within- than cross-modality priming for sensory decisions, but equivalent within- and cross-modality priming for functional decisions owing to significantly greater facilitation of functional decisions from prior sensory decisions than vice versa. Experiment 3 involved a single verification phase: For half the participants the feature name preceded the concept name, and for half the concept name preceded the feature name. The functional processing advantage persisted irrespective of presentation order. Results suggest that functional information is central to the representation of all concepts: Function is processed faster than sensory information and is activated obligatorily.

Limiting factors for mapping corpus-based semantic representations to brain activity

To help understand how semantic information is represented in the human brain, a number of previous studies have explored how a linear mapping from corpus derived semantic representations to corresponding patterns of fMRI brain activations can be learned. They have demonstrated that such a mapping for concrete nouns is able to predict brain activations with accuracy levels significantly above chance, but the more recent elaborations have achieved relatively little performance improvement over the original study. In fact, the absolute accuracies of all these models are still currently rather limited, and it is not clear which aspects of the approach need improving in order to achieve performance levels that might lead to better accounts of human capabilities. This paper presents a systematic series of computational experiments designed to identify the limiting factors of the approach. Two distinct series of artificial brain activation vectors with varying levels of noise are introduced to characterize how the brain activation data restricts performance, and improved corpus based semantic vectors are developed to determine how the word set and model inputs affect the results. These experiments lead to the conclusion that the current state-of-the-art input semantic representations are already operating nearly perfectly (at least for non-ambiguous concrete nouns), and that it is primarily the quality of the fMRI data that is limiting what can be achieved with this approach. The results allow the study to end with empirically informed suggestions about the best directions for future research in this area.

Shapes, scents and sounds: quantifying the full multi-sensory basis of conceptual knowledge

Contemporary neuroscience theories assume that concepts are formed through experience in multiple sensory-motor modalities. Quantifying the contribution of each modality to different object categories is critical to understanding the structure of the conceptual system and to explaining category-specific knowledge deficits. Verbal feature listing is typically used to elicit this information but has a number of drawbacks: sensory knowledge often cannot easily be translated into verbal features and many features are experienced in multiple modalities. Here, we employed a more direct approach in which subjects rated their knowledge of objects in each sensory-motor modality separately. Compared with these ratings, feature listing over-estimated the importance of visual form and functional knowledge and under-estimated the contributions of other sensory channels. An item's sensory rating proved to be a better predictor of lexical-semantic processing speed than the number of features it possessed, suggesting that ratings better capture the overall quantity of sensory information associated with a concept. Finally, the richer, multi-modal rating data not only replicated the sensory-functional distinction between animals and non-living things but also revealed novel distinctions between different types of artefact. Hierarchical cluster analyses indicated that mechanical devices (e.g., vehicles) were distinct from other non-living objects because they had strong sound and motion characteristics, making them more similar to animals in this respect. Taken together, the ratings align with neuroscience evidence in suggesting that a number of distinct sensory processing channels make important contributions to object knowledge. Multi-modal ratings for 160 objects are provided as supplementary materials.

The Brain's concepts: the role of the Sensory-motor system in conceptual knowledge

Concepts are the elementary units of reason and linguistic meaning. They are conventional and relatively stable. As such, they must somehow be the result of neural activity in the brain. The questions are: Where? and How? A common philosophical position is that all concepts-even concepts about action and perception-are symbolic and abstract, and therefore must be implemented outside the brain's sensory-motor system. We will argue against this position using (1) neuroscientific evidence; (2) results from neural computation; and (3) results about the nature of concepts from cognitive linguistics. We will propose that the sensory-motor system has the right kind of structure to characterise both sensory-motor and more abstract concepts. Central to this picture are the neural theory of language and the theory of cogs, according to which, brain structures in the sensory-motor regions are exploited to characterise the so-called "abstract" concepts that constitute the meanings of grammatical constructions and general inference patterns.

Comprehension

Comprehension is the aspect of cognition in which information is retrieved and consciously integrated. It occurs between the input of perception and memory, and the output of language and executive functioning. This article focuses on disorders of comprehension of linguistic information and semantic knowledge. Comprehension of speech begins with the interpretation of acoustic-phonetic input as word forms. These concepts to which the word forms are semantically associated must then be retrieved, in parallel with interpretation of word order and grammatical marking, to achieve comprehension of discourse. Neural systems for semantic memory are closely related to those for lexical processing. Disorders of language comprehension and semantic knowledge for concrete entities give insight into the relationship of these processes to neural systems.

The neurobiology of semantic memory

Semantic memory includes all acquired knowledge about the world and is the basis for nearly all human activity, yet its neurobiological foundation is only now becoming clear. Recent neuroimaging studies demonstrate two striking results: the participation of modality-specific sensory, motor, and emotion systems in language comprehension, and the existence of large brain regions that participate in comprehension tasks but are not modality-specific. These latter regions, which include the inferior parietal lobe and much of the temporal lobe, lie at convergences of multiple perceptual processing streams. These convergences enable increasingly abstract, supramodal representations of perceptual experience that support a variety of conceptual functions including object recognition, social cognition, language, and the remarkable human capacity to remember the past and imagine the future.

Modulation of BOLD response in motion-sensitive lateral temporal cortex by real and fictive motion sentences

Can linguistic semantics affect neural processing in feature-specific visual regions? Specifically, when we hear a sentence describing a situation that includes motion, do we engage neural processes that are part of the visual perception of motion? How about if a motion verb was used figuratively, not literally? We used fMRI to investigate whether semantic content can "penetrate" and modulate neural populations that are selective to specific visual properties during natural language comprehension. Participants were presented audiovisually with three kinds of sentences: motion sentences ("The wild horse crossed the barren field."), static sentences, ("The black horse stood in the barren field."), and fictive motion sentences ("The hiking trail crossed the barren field."). Motion-sensitive visual areas (MT+) were localized individually in each participant as well as face-selective visual regions (fusiform face area; FFA). MT+ was activated significantly more for motion sentences than the other sentence types. Fictive motion sentences also activated MT+ more than the static sentences. Importantly, no modulation of neural responses was found in FFA. Our findings suggest that the neural substrates of linguistic semantics include early visual areas specifically related to the represented semantics and that figurative uses of motion verbs also engage these neural systems, but to a lesser extent. These data are consistent with a view of language comprehension as an embodied process, with neural substrates as far reaching as early sensory brain areas that are specifically related to the represented semantics.

Language disorders in children with central nervous system injury

Children with injury to the central nervous system (CNS) exhibit a variety of language disorders that have been described by members of different disciplines, in different journals, using different descriptors and taxonomies. This paper is an overview of language deficits in children with CNS injury, whether congenital or acquired after a period of normal development. It first reviews the principal CNS conditions associated with language disorders in childhood. It then describes a functional taxonomy of language, with examples of the phenomenology and neurobiology of clinical deficits in children with CNS insults. Finally, it attempts to situate language in the broader realm of cognition and in current theoretical accounts of embodied cognition.

Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies

Semantic memory refers to knowledge about people, objects, actions, relations, self, and culture acquired through experience. The neural systems that store and retrieve this information have been studied for many years, but a consensus regarding their identity has not been reached. Using strict inclusion criteria, we analyzed 120 functional neuroimaging studies focusing on semantic processing. Reliable areas of activation in these studies were identified using the activation likelihood estimate (ALE) technique. These activations formed a distinct, left-lateralized network comprised of 7 regions: posterior inferior parietal lobe, middle temporal gyrus, fusiform and parahippocampal gyri, dorsomedial prefrontal cortex, inferior frontal gyrus, ventromedial prefrontal cortex, and posterior cingulate gyrus. Secondary analyses showed specific subregions of this network associated with knowledge of actions, manipulable artifacts, abstract concepts, and concrete concepts. The cortical regions involved in semantic processing can be grouped into 3 broad categories: posterior multimodal and heteromodal association cortex, heteromodal prefrontal cortex, and medial limbic regions. The expansion of these regions in the human relative to the nonhuman primate brain may explain uniquely human capacities to use language productively, plan, solve problems, and create cultural and technological artifacts, all of which depend on the fluid and efficient retrieval and manipulation of semantic knowledge.

Category-specific neural processing for naming pictures of animals and naming pictures of tools: an ALE meta-analysis

Using activation-likelihood estimation (ALE) meta-analysis, we identified brain areas that are invoked when people name pictures of animals and pictures of tools. We found that naming animals and naming tools invoked separate distributed networks in the brain. Specifically, we found that naming animals invoked greater responses than naming tools in frontal lobe structures that are typically modulated by emotional content and task demands, and in a number of visual areas in the ventral stream. In contrast, naming tools invoked greater responses in a different set of areas in the ventral stream than those invoked by naming animals. Naming tools also invoked greater responses than naming animals in motor areas in the frontal lobe as well as in sensory areas in the parietal lobe. The only overlapping sites of activation that we found for naming these two categories of objects were in the left pars triangularis, the left inferior temporal gyrus, and the left parahippocampal gyrus. Taken together, our meta-analysis reveals that animals and tools are categorically represented in visual areas but show convergence in higher-order associative areas in the temporal and frontal lobes in regions that are typically regarded as being involved in memory and/or semantic processing. Our results also reveal that naming tools not only engages visual areas in the ventral stream but also a fronto-parietal network associated with tool use. Whether or not this network associated with tool use contributes directly to recognition will require further investigation.

Effects of stimulus duration and choice delay on visual categorization in pigeons

We (Lazareva, Freiburger, & Wasserman, 2004) previously trained four pigeons to classify color photographs into their basic-level categories (cars, chairs, flowers, or people) or into their superordinate-level categories (natural or artificial). Here, we found that brief stimulus durations had the most detrimental effect on the basic-level discrimination of natural stimuli by the same pigeons. Increasing the delay between stimulus presentation and choice responding had greater detrimental effect on the basic-level discrimination than the superordinate-level discrimination. These results suggest that basic-level discriminations required longer stimulus durations and were more subject to forgetting than were superordinate-level discriminations. Additionally, categorization of natural stimuli required longer stimulus durations than categorization of artificial stimuli, but only at the basic level. Together, these findings suggest that basic-level categorization may not always be superior to superordinate-level categorization and provide additional evidence of a dissociation between natural and artificial stimuli in pigeons' categorization.

Further lesion evidence for the neural basis of conceptual knowledge for persons and other concrete entities

The neural underpinnings of conceptual knowledge have been studied intensively, but many unanswered questions remain. In a previous study examining recognition of persons, animals, and tools in 116 participants with unilateral brain lesions, we found no instance of a patient who manifested defective recognition in all three categories. We reasoned that the spatial distribution of the lesion loci critical for the appearance of recognition defects for these different categories explained why this 'three-way' defect could not be found in patients with unilateral lesions, and we proposed that only a suitable bilateral lesion would be likely to produce such a combined defect. In the study reported here, we tested this hypothesis by investigating recognition performances in 55 participants with bilateral cortical lesions. In support of the hypothesis, nine patients, all of whose lesions included bilateral occipitotemporal and/or temporal cortices, had a three-way recognition impairment (persons, M = 18.3%; animals, M = 35.7%; tools, M = 71.3%; all scores >2 SDs below normal). As expected, bilateral lesions to other neural sectors, for example prefrontal cortices, did not lead to recognition impairments. These findings provide further support for the notion that retrieval of knowledge for concrete entities from different conceptual categories depends on partially segregated neural systems, located in different sectors of occipitotemporal and temporal regions in right and left hemisphere.

ERP correlates of superordinate category activation

The human semantic network is hierarchically organized, containing superordinate, basic and subordinate levels. Various impairments are thought to be connected with abnormal access to superordinate concepts. We devised an ERP paradigm to examine the activation of superordinate versus otherwise related concepts in 20 healthy participants. Following the presentation of a typical category member an arrow indicated whether the appropriate superordinate category had to be generated (categorization task) or an otherwise related word (relation task). To control task execution, a second word was presented for which a match-mismatch-judgment was required. Reaction times, accuracy rates and ERPs after the second word showed that participants successfully accessed the superordinate category name and that verification in the categorization task was faster and easier than in the relation task. Comparison of ERPs after the arrow revealed topographical, Global Field Power (GFP), and onset latency differences between the two tasks and thus indicated the involvement of at least partially different neural generators. Source localization analysis confirmed that brain regions were activated that were also identified in previous experiments with semantic task. The paradigm seems to be suitable for further examination of superordinate activation processes and evaluation of impairments such as thought disorders in schizophrenic patients.

Cognitive and Neural Contributions to Understanding the Conceptual System

The conceptual system contains categorical knowledge about experience that supports the spectrum of cognitive processes. Cognitive science theories assume that categorical knowledge resides in a modular and amodal semantic memory, whereas neuroscience theories assume that categorical knowledge is grounded in the brain's modal systems for perception, action, and affect. Neuroscience has influenced theories of the conceptual system by stressing principles of neural processing in neural networks and by motivating grounded theories of cognition, which propose that simulations of experience represent knowledge. Cognitive science has influenced theories of the conceptual system by documenting conceptual phenomena and symbolic operations that must be grounded in the brain. Significant progress in understanding the conceptual system is most likely to occur if cognitive and neural approaches achieve successful integration.

Language beyond action

The discovery of mirror neurons in macaques and of a similar system in humans has provided a new and fertile neurobiological ground for rooting a variety of cognitive faculties. Automatic sensorimotor resonance has been invoked as the key elementary process accounting for disparate (dys)functions, like imitation, ideomotor apraxia, autism, and schizophrenia. In this paper, we provide a critical appraisal of three of these claims that deal with the relationship between language and the motor system. Does language comprehension require the motor system? Was there an evolutionary switch from manual gestures to speech as the primary mode of language? Is human communication explained by automatic sensorimotor resonances? A positive answer to these questions would open the tantalizing possibility of bringing language and human communication within the fold of the motor system. We argue that the available empirical evidence does not appear to support these claims, and their theoretical scope fails to account for some crucial features of the phenomena they are supposed to explain. Without denying the enormous importance of the discovery of mirror neurons, we highlight the limits of their explanatory power for understanding language and communication.

Sources of error in picture naming under time pressure

We used a deadline procedure to investigate how time pressure may influence the processes involved in picture naming. The deadline exaggerated errors found under naming without deadline. There were also category differences in performance between living and nonliving things and, in particular, for animals versus fruit and vegetables. The majority of errors were visuallyand semantically related to the target (e. celery-asparagus), and there was a greater proportion of these errors made to living things. Importantly, there were also more visual-semantic errors to animals than to fruit and vegetables. In addition, there were a smaller number of pure semantic errors (e.g., nut-bolt), which were made predominantly to nonliving things. The different kinds of error were correlated with different variables. Overall, visual-semantic errors were associated with visual complexity and visual similarity, whereas pure semantic errors were associated with imageability and age of acquisition. However, for animals, visual-semantic errors were associated with visual complexity, whereas for fruit and vegetables they were associated with visual similarity. We discuss these findings in terms of theories of category-specific semantic impairment and models of picture naming.

Perceptual knowledge retrieval activates sensory brain regions

Although knowledge indexes our experiences of the world, the neural basis of this relationship remains to be determined. Previous neuroimaging research, especially involving knowledge biased to visual and functional information, suggests that semantic representations depend on modality-specific brain mechanisms. However, it is unclear whether sensory cortical regions, in general, support retrieval of perceptual knowledge. Using neuroimaging methods, we show that semantic decisions that index tactile, gustatory, auditory, and visual knowledge specifically activate brain regions associated with encoding these sensory experiences. Retrieval of tactile knowledge was specifically associated with increased activation in somatosensory, motor, and premotor cortical regions. In contrast, decisions involving flavor knowledge increased activation in an orbitofrontal region previously implicated in processing semantic comparisons among edible items. Perceptual knowledge retrieval that references visual and auditory experiences was associated with increased activity in distinct temporal brain regions involved in the respective sensory processing. These results indicate that retrieval of perceptual knowledge relies on brain regions used to mediate sensory experiences with the referenced objects.

Anatomical functional and cognitive determinants of semantic memory disorders

Contemporary debates on the 'semantic memory' construct revolve around three main topics: (1) the functional and anatomical relationships between episodic and semantic memory; (2) the format of semantic representations and their relationships with the underlying sensory-motor processes; (3) the categorical organization of semantic memory. The aim of the present review is to demonstrate that there is a common thread linking these different aspects of semantic memory. This thread is represented by the interdependence of mechanisms involved in the construction of semantic memory and the content of semantic representations. In particular, I suggest there is a continuity between: (a) the mechanisms of acquisition of episodic and semantic memory; (b) semantic representations and sensory-motor processes preliminary to the acquisition of these representations. This continuity has important implications for the format of semantic representations and the brain structures subserving the organisation of various categories of knowledge.

Evaluating feature-category relations using semantic fluency tasks

The issue of the relationship between semantic features and semantic categories has been raised by Warrington and colleagues, who claimed that sensory and functional-associative features are differentially important in determining the meaning of living and nonliving things (Warrington & McCarthy, 1983, 1987; Warrington & Shallice, 1984). In the present study, the effectiveness of semantic memory search for living and nonliving things with sensory and functional-associative search cues was evaluated through eight different adaptations of the semantic fluency task. More living thing responses and clusters were generated from sensory than from functional-associative search cues, while the reverse pattern holds for nonliving things responses and clusters. The results thus provide consistent empirical support for the assumption that sensory properties are fundamental in the representation of living things, while functional-associative properties are fundamental in the semantic representation of nonliving things.

The neural bases of complex tool use in humans

The behaviors involved in complex human tool use cut across boundaries traditionally drawn between social, cognitive, perceptual and motor processes. Longstanding neuropsychological evidence suggests a distinction between brain systems responsible for representing: (1) semantic knowledge about familiar tools and their uses, and (2) the acquired skills necessary for performing these actions. Contemporary findings in functional neuroimaging support and refine this distinction by revealing the distributed neural systems that support these processes and the conditions under which they interact. Together, these findings indicate that behaviors associated with complex tool use arise from functionally specialized networks involving temporal, parietal and frontal areas within the left cerebral hemisphere.

Electrophysiological correlates of object categorization: back to basics

The time course of visual object categorization as a function of electrophysiological activity in the brain was investigated using a variant of the "oddball" design. Category level was manipulated by sequentially presenting subordinate, basic or superordinate target objects among a variety of non-target objects. It was found that superordinate categorizations were performed more quickly and differentiated from basic level categorizations in amplitude early in visual processing (320-420 ms). In contrast, subordinate categorizations took longer to perform and differentiated from basic level categorizations in amplitude and latency at later stages (450-550 ms). Notably, these effects were observed using the same objects categorized at different levels suggesting that visually categorizing objects at varying levels of abstraction engaged specific cognitive processes. These results are consistent with research on rapid visual categorization that challenges the generality of basic category level superiority effects.

Cortical regions associated with different aspects of object recognition performance

In the present object recognition study, we examined the relationship between brain activation and four behavioral measures: error rate, reaction time, observer sensitivity, and response bias. Subjects perceptually matched object pairs in which structural similarity (SS), an index of structural differentiation, and exposure duration (DUR), an index of task difficulty, were manipulated. The SS manipulation affected the fMRI signal in the left anterior fusiform and parietal cortices, which in turn reflected a bias to respond same. Conversely, an SS-modulated fMRI signal in the right middle frontal gyrus reflected a bias to respond different. The DUR manipulation affected the fMRI signal in occipital and posterior fusiform regions, which in turn reflected greater sensitivity, longer reaction times, and greater accuracy. These findings demonstrate that the regions most strongly implicated in processing object shape (SS-modulated regions) are associated with response bias, whereas regions that are not directly involved in shape processing are associated with successful recognition performance.

Object concepts and action: extracting affordances from objects parts

Two experiments with a part-generation task show that rated salience and production order of parts in artifacts are first predicted by their relevance for canonical actions, but also that they vary, depending on the current situation. In three further experiments participants read sentences describing actions (e.g., 'The woman shares the orange') followed by objects' parts from which it was easy or not to extract affordances (e.g., 'slice' vs. 'pulp'). They had to perform a part verification task or to evaluate whether or not the combination made sense. Parts from which it was easy to derive affordances were processed earlier and the combination was evaluated as the one which made more sense. Overall, results support the view that sensory-motor simulations underlie conceptualization and that concepts are action-based.