Experiential grounding of abstract concepts: Processing of abstract mental state concepts engages brain regions involved in mentalizing, automatic speech, and lip movements

concepts like mental state concepts lack a physical referent, which can be directly perceived. Classical theories therefore claim that abstract concepts require amodal representations detached from experiential brain systems. However, grounded cognition approaches suggest an involvement of modal experiential brain regions in the processing of abstract concepts. In the present functional magnetic resonance imaging study, we investigated the relation of the processing of abstract mental state concepts to modal experiential brain systems in a fine-grained fashion. Participants performed lexical decisions on abstract mental state as well as on verbal association concepts as control category. Experiential brain systems related to the processing of mental states, generating verbal associations, automatic speech as well as hand and lip movements were determined by corresponding localizer tasks. Processing of abstract mental state concepts neuroanatomically overlapped with activity patterns associated with processing of mental states, generating verbal associations, automatic speech and lip movements. Hence, mental state concepts activate the mentalizing brain network, complemented by perceptual-motor brain regions involved in simulation of visual or action features associated with social interactions, linguistic brain regions as well as face-motor brain regions recruited for articulation. The present results provide compelling evidence for the rich grounding of abstract mental state concepts in experiential brain systems related to mentalizing, verbal communication and mouth action.

Early modulations of neural oscillations during the processing of emotional body language

The processing of threat-related emotional body language (EBL) has been shown to engage sensorimotor cortical areas early on and induce freezing in the observers' motor system, particularly when observing fearful EBL. To provide insights into the interplay between somatosensory and motor areas during observation of EBL, here, we used high-density electroencephalography (hd-EEG) in healthy humans while they observed EBL stimuli involving fearful and neutral expressions. To capture early sensorimotor brain response, we focused on P100 fronto-central event-related potentials (ERPs) and event-related desynchronization/synchronization (ERD/ERS) in the mu-alpha (8-13 Hz) and lower beta (13-20 Hz) bands over the primary motor (M1) and somatosensory (S1) cortices. Source-level ERP and ERD/ERS analyses were conducted using eLORETA. Results revealed higher P100 amplitudes in motor and premotor channels for 'Neutral' compared with 'Fear'. Additionally, analysis of ERD/ERS showed increased beta band desynchronization in M1 for 'Neutral', and the opposite pattern in S1. Source-level estimation showed significant differences between conditions mainly observed in the beta band over sensorimotor areas. These findings provide high-temporal resolution evidence suggesting that seeing fearful EBL induces early activation of somatosensory areas, which in turn could suppress M1 activity. These findings highlight early dynamics within the observer's sensorimotor system and hint at a sensorimotor mechanism supporting freezing during the processing of EBL.

White matter tract disconnection in Gerstmann's syndrome: Insights from a single case study

It has been suggested that Gerstmann's syndrome is the result of subcortical disconnection rather than emerging from damage of a multifunctional brain region within the parietal lobe. However, patterns of white matter tract disconnection following parietal damage have been barely investigated. This single case study allows characterising Gerstmann's syndrome in terms of disconnected networks. We report the case of a left parietal patient affected by Gerstmann's tetrad: agraphia, acalculia, left/right orientation problems, and finger agnosia. Lesion mapping, atlas-based estimation of probability of disconnection, and DTI-based tractography revealed that the lesion was mainly located in the superior parietal lobule, and it caused disruption of both intraparietal tracts passing through the inferior parietal lobule (e.g., tracts connecting the angular, supramarginal, postcentral gyri, and the superior parietal lobule) and fronto-parietal long tracts (e.g., the superior longitudinal fasciculus). The lesion site appears to be located more superiorly as compared to the cerebral regions shown active by other studies during tasks impaired in the syndrome, and it reached the subcortical area potentially critical in the emergence of the syndrome, as hypothesised in previous studies. Importantly, the reconstruction of tracts connecting regions within the parietal lobe indicates that this critical subcortical area is mainly crossed by white matter tracts connecting the angular gyrus and the superior parietal lobule. Taken together, these findings suggest that this case study might be considered as empirical evidence of Gerstmann's tetrad caused by disconnection of intraparietal white matter tracts.

Conceptual representations in the default, control and attention networks are task-dependent and cross-modal

Conceptual knowledge is central to human cognition. Neuroimaging studies suggest that conceptual processing involves modality-specific and multimodal brain regions in a task-dependent fashion. However, it remains unclear (1) to what extent conceptual feature representations are also modulated by the task, (2) whether conceptual representations in multimodal regions are indeed cross-modal, and (3) how the conceptual system relates to the large-scale functional brain networks. To address these issues, we conducted multivariate pattern analyses on fMRI data. 40 participants performed three tasks-lexical decision, sound judgment, and action judgment-on written words. We found that (1) conceptual feature representations are strongly modulated by the task, (2) conceptual representations in several multimodal regions are cross-modal, and (3) conceptual feature retrieval involves the default, frontoparietal control, and dorsal attention networks. Conceptual representations in these large-scale networks are task-dependent and cross-modal. Our findings support theories that assume conceptual processing to rely on a flexible, multi-level architecture.

Academic training increases grounding of scientific concepts in experiential brain systems

Scientific concepts typically transcendent our sensory experiences. Traditional approaches to science education therefore assume a shift towards amodal or verbal knowledge representations during academic training. Grounded cognition approaches, in contrast, predict a maintenance of grounding of the concepts in experiential brain networks or even an increase. To test these competing approaches, the present study investigated the semantic content of scientific psychological concepts and identified the corresponding neural circuits using functional magnetic resonance imaging (fMRI) in undergraduate psychology students (beginners) and in graduated psychologists (advanced learners). During fMRI scanning, participants were presented with words denoting scientific psychological concepts within a lexical decision task (e.g. "conditioning", "habituation"). The individual semantic property content of each concept was related to brain activity during abstract concept processing. In both beginners and advanced learners, visual and motor properties activated brain regions also involved in perception and action, while mental state properties increased activity in brain regions also recruited by emotional-social scene observation. Only in advanced learners, social constellation properties elicited brain activity overlapping with emotional-social scene observation. In line with grounded cognition approaches, the present results highlight the importance of experiential information for constituting the meaning of abstract scientific concepts during the course of academic training.

Neural decoding dissociates perceptual grouping between proximity and similarity in visual perception

Unlike single grouping principle, cognitive neural mechanism underlying the dissociation across two or more grouping principles is still unclear. In this study, a dimotif lattice paradigm that can adjust the strength of one grouping principle was used to inspect how, when, and where the processing of two grouping principles (proximity and similarity) were carried out in human brain. Our psychophysical findings demonstrated that similarity grouping effect was enhanced with reduced proximity effect when the grouping cues of proximity and similarity were presented simultaneously. Meanwhile, EEG decoding was performed to reveal the specific cognitive patterns involved in each principle by using time-resolved MVPA. More importantly, the onsets of dissociation between 2 grouping principles coincided within 3 time windows: the early-stage proximity-defined local visual element arrangement in middle occipital cortex, the middle-stage processing for feature selection modulating low-level visual cortex such as inferior occipital cortex and fusiform cortex, and the high-level cognitive integration to make decisions for specific grouping preference in the parietal areas. In addition, it was discovered that the brain responses were highly correlated with behavioral grouping. Therefore, our study provides direct evidence for a link between the human perceptual space of grouping decision-making and neural space of brain activation patterns.

A strategy of model space search for dynamic causal modeling in task fMRI data exploratory analysis

Dynamic causal modeling (DCM) is a tool used for effective connectivity (EC) estimation in neuroimage analysis. But it is a model-driven analysis method, and the structure of the EC network needs to be determined in advance based on a large amount of prior knowledge. This characteristic makes it difficult to apply DCM to the exploratory brain network analysis. The exploratory analysis of DCM can be realized from two perspectives: one is to reduce the computational cost of the model; the other is to reduce the model space. From the perspective of model space reduction, a model space exploration strategy is proposed, including two algorithms. One algorithm, named GreedyEC, starts with reducing EC from full model, and the other, named GreedyROI, start with adding EC from one node model. Then the two algorithms were applied to the task state functional magnetic resonance imaging (fMRI) data of visual object recognition and selected the best DCM model from the perspective of model comparison based on Bayesian model compare method. Results show that combining the results of the two algorithms can further improve the effect of DCM exploratory analysis. For convenience in application, the algorithms were encapsulated into MATLAB function based on SPM to help neuroscience researchers to analyze the brain causal information flow network. The strategy provides a model space exploration tool that may obtain the best model from the perspective of model comparison and lower the threshold of DCM analysis.

Principal Component Analysis of Oxford Cognitive Screen in Patients With Stroke

Cognitive deficits occur in most patients with stroke and are the important predictors of adverse long-term outcome. Early identification is fundamental to plan the most appropriate care, including rehabilitation and discharge decisions. The Oxford Cognitive Screen (OCS) is a simple, valid, and reliable tool for the assessment of cognitive deficits in patients with stroke. It contains 10 subtests, providing 14 scores referring to 5 theoretically derived cognitive domains: attention, language, number, praxis, and memory. However, an empirical verification of the domain composition of the OCS subtests in stroke data is still lacking in the literature. A principal component analysis (PCA) was performed on 1,973 patients with stroke who were enrolled in OCS studies in the UK and in Italy. A number of six main components were identified relating to the domains of language and arithmetic, memory, visuomotor ability, orientation, spatial exploration, and executive functions. Bootstrapped split-half reliability analysis on patients and comparison between patients and 498 healthy participants, as that between patients with left and right hemisphere damage, confirmed the results obtained by the principal component analysis. A clarification about the contribution of each score to the theoretical original domains and to the components identified by the PCA is provided with the aim to foster the usability of OCS for both clinicians and researchers.

Using High-Density Electroencephalography to Explore Spatiotemporal Representations of Object Categories in Visual Cortex

Visual object perception involves neural processes that unfold over time and recruit multiple regions of the brain. Here, we use high-density EEG to investigate the spatiotemporal representations of object categories across the dorsal and ventral pathways. In , human participants were presented with images from two animate object categories (birds and insects) and two inanimate categories (tools and graspable objects). In , participants viewed images of tools and graspable objects from a different stimulus set, one in which a shape confound that often exists between these categories (elongation) was controlled for. To explore the temporal dynamics of object representations, we employed time-resolved multivariate pattern analysis on the EEG time series data. This was performed at the electrode level as well as in source space of two regions of interest: one encompassing the ventral pathway and another encompassing the dorsal pathway. Our results demonstrate shape, exemplar, and category information can be decoded from the EEG signal. Multivariate pattern analysis within source space revealed that both dorsal and ventral pathways contain information pertaining to shape, inanimate object categories, and animate object categories. Of particular interest, we note striking similarities obtained in both ventral stream and dorsal stream regions of interest. These findings provide insight into the spatio-temporal dynamics of object representation and contribute to a growing literature that has begun to redefine the traditional role of the dorsal pathway.

Differential temporo-spatial pattern of electrical brain activity during the processing of abstract concepts related to mental states and verbal associations

Refined grounded cognition accounts propose that abstract concepts might be grounded in brain circuits involved in mentalizing. In the present event-related potential (ERP) study, we compared the time course of neural processing in response to semantically predefined abstract mental states and verbal association concepts during a lexical decision task. In addition to scalp ERPs, source estimates of underlying volume brain activity were determined to reveal spatio-temporal clusters of greater electrical brain activity to abstract mental state vs. verbal association concepts, and vice versa. Source estimates suggested early (onset 194 ms), but short-lived enhanced activity (offset 210 ms) to verbal association concepts in left occipital regions. Increased occipital activity might reflect retrieval of visual word form or access to visual conceptual features of associated words. Increased estimated source activity to mental state concepts was obtained in visuo-motor (superior parietal, pre- and postcentral areas) and mentalizing networks (lateral and medial prefrontal areas, insula, precuneus, temporo-parietal junction) with an onset of 212 ms, which extended to later time windows. The time course data indicated two processing phases: An initial conceptual access phase, in which linguistic and modal brain circuits rapidly process features depending on their relevance, and a later conceptual elaboration phase, in which elaborative processing within feature-specific networks further refines the concept. This study confirms the proposal that abstract concepts are based on representations in distinct neural circuits depending on their semantic feature content. The present research also highlights the importance of investigating sets of abstract concepts with a defined semantic content.

Dynamic Causal Modeling on the Identification of Interacting Networks in the Brain: A Systematic Review

Dynamic causal modeling (DCM) has long been used to characterize effective connectivity within networks of distributed neuronal responses. Previous reviews have highlighted the understanding of the conceptual basis behind DCM and its variants from different aspects. However, no detailed summary or classification research on the task-related effective connectivity of various brain regions has been made formally available so far, and there is also a lack of application analysis of DCM for hemodynamic and electrophysiological measurements. This review aims to analyze the effective connectivity of different brain regions using DCM for different measurement data. We found that, in general, most studies focused on the networks between different cortical regions, and the research on the networks between other deep subcortical nuclei or between them and the cerebral cortex are receiving increasing attention, but far from the same scale. Our analysis also reveals a clear bias towards some task types. Based on these results, we identify and discuss several promising research directions that may help the community to attain a clear understanding of the brain network interactions under different tasks.

Neural signatures of bullying experience and social rejection in teenagers

Relational bullying in schools is one of the most frequent forms of violence and can have severe negative health impact, e.g. depression. Social exclusion is the most prominent form of relational bullying that can be operationalized experimentally. The present study used MR-based perfusion imaging (pCASL) to investigate the neural signatures of social exclusion and its relationship with individually different extent of previous bullying experience. Twenty-four teenagers reporting bullying experience at different extent were scanned during a virtual ball-tossing (Cyberball game). Our findings showed that social exclusion (relative to social inclusion) activated frontal brain areas: sub- and perigenual anterior cingulate cortex (sg/pgACC), left inferior frontal cortex (IFG), and dorsolateral prefrontal cortex. Positive relationship between exclusion-specific signal increase and individually different extents of prior bullying experience was for the first time observed in left IFG and sgACC. This suggests that more frequent prior experience has conditioned greater mentalizing and/or rumination, in order to cope with the situation. While this interpretation remains speculative, the present data show that the experience of being bullied partly sensitizes the neural substrate relevant for the processing of social exclusion.

Semantic and action tool knowledge in the brain: Identifying common and distinct networks

Most cognitive models of apraxia assume that impaired tool use results from a deficit occurring at the conceptual level, which contains dedicated information about tool use, namely, semantic and action tool knowledge. Semantic tool knowledge contains information about the prototypical use of familiar tools, such as function (e.g., a hammer and a mallet share the same purpose) and associative relations (e.g., a hammer goes with a nail). Action tool knowledge contains information about how to manipulate tools, such as hand posture and kinematics. The present review aimed to better understand the neural correlates of action and semantic tool knowledge, by focusing on activation, stimulation and patients' studies (left brain-damaged patients). We found that action and semantic tool knowledge rely upon a large brain network including temporal and parietal regions. Yet, while action tool knowledge calls into play the intraparietal sulcus, function relations mostly involve the anterior and posterior temporal lobe. Associative relations engaged the angular and the posterior middle temporal gyrus. Moreover, we found that hand posture and kinematics both tapped into the inferior parietal lobe and the lateral occipital temporal cortex, but no region specificity was found for one or the other representation. Our results point out the major role of both posterior middle temporal gyrus and inferior parietal lobe for action and semantic tool knowledge. They highlight the common and distinct brain networks involved in action and semantic tool networks and spur future directions on this topic.

Task-Dependent Functional and Effective Connectivity during Conceptual Processing

Conceptual knowledge is central to cognition. Previous neuroimaging research indicates that conceptual processing involves both modality-specific perceptual-motor areas and multimodal convergence zones. For example, our previous functional magnetic resonance imaging (fMRI) study revealed that both modality-specific and multimodal regions respond to sound and action features of concepts in a task-dependent fashion (Kuhnke P, Kiefer M, Hartwigsen G. 2020b. Task-dependent recruitment of modality-specific and multimodal regions during conceptual processing. Cereb Cortex. 30:3938–3959.). However, it remains unknown whether and how modality-specific and multimodal areas interact during conceptual tasks. Here, we asked 1) whether multimodal and modality-specific areas are functionally coupled during conceptual processing, 2) whether their coupling depends on the task, 3) whether information flows top-down, bottom-up or both, and 4) whether their coupling is behaviorally relevant. We combined psychophysiological interaction analyses with dynamic causal modeling on the fMRI data of our previous study. We found that functional coupling between multimodal and modality-specific areas strongly depended on the task, involved both top-down and bottom-up information flow, and predicted conceptually guided behavior. Notably, we also found coupling between different modality-specific areas and between different multimodal areas. These results suggest that functional coupling in the conceptual system is extensive, reciprocal, task-dependent, and behaviorally relevant. We propose a new model of the conceptual system that incorporates task-dependent functional interactions between modality-specific and multimodal areas.

You won't believe what this guy is doing with the potato: The ObjAct stimulus-set depicting human actions on congruent and incongruent objects

Perception famously involves both bottom-up and top-down processes. The latter are influenced by our previous knowledge and expectations about the world. In recent years, many studies have focused on the role of expectations in perception in general, and in object processing in particular. Yet studying this question is not an easy feat, requiring-among other things-the creation and validation of appropriate stimuli. Here, we introduce the ObjAct stimulus-set of free-to-use, highly controlled real-life scenes, on which critical objects are pasted. All scenes depict human agents performing an action with an object that is either congruent or incongruent with the action. The focus on human actions yields highly constraining contexts, strengthening congruency effects. The stimuli were analyzed for low-level properties, using the SHINE toolbox to control for luminance and contrast, and using a deep convolutional neural network to mimic V1 processing and potentially discover other low-level factors that might differ between congruent and incongruent scenes. Two online validation studies (N = 500) were also conducted to assess the congruency manipulation and collect additional ratings of our images (e.g., arousal, likeability, visual complexity). We also provide full descriptions of the online sources from which all images were taken, as well as verbal descriptions of their content. Taken together, this extensive validation and characterization procedure makes the ObjAct stimulus-set highly informative and easy to use for future researchers in multiple fields, from object and scene processing, through top-down contextual effects, to the study of actions.

Left posterior inferior parietal cortex causally supports the retrieval of action knowledge

Conceptual knowledge is central to human cognition. The left posterior inferior parietal lobe (pIPL) is implicated by neuroimaging studies as a multimodal hub representing conceptual knowledge related to various perceptual-motor modalities. However, the causal role of left pIPL in conceptual processing remains unclear. Here, we transiently disrupted left pIPL function with transcranial magnetic stimulation (TMS) to probe its causal relevance for the retrieval of action and sound knowledge. We compared effective TMS over left pIPL with sham TMS, while healthy participants performed three different tasks-lexical decision, action judgment, and sound judgment-on words with a high or low association to actions and sounds. We found that pIPL-TMS selectively impaired action judgments on low sound-low action words. For the first time, we directly related computational simulations of the TMS-induced electrical field to behavioral performance, which revealed that stronger stimulation of left pIPL is associated with worse performance for action but not sound judgments. These results indicate that left pIPL causally supports conceptual processing when action knowledge is task-relevant and cannot be compensated by sound knowledge. Our findings suggest that left pIPL is specialized for the retrieval of action knowledge, challenging the view of left pIPL as a multimodal conceptual hub.

Task-Dependent Recruitment of Modality-Specific and Multimodal Regions during Conceptual Processing

Conceptual knowledge is central to cognitive abilities such as word comprehension. Previous neuroimaging evidence indicates that concepts are at least partly composed of perceptual and motor features that are represented in the same modality-specific brain regions involved in actual perception and action. However, it is unclear to what extent the retrieval of perceptual-motor features and the resulting engagement of modality-specific regions depend on the concurrent task. To address this issue, we measured brain activity in 40 young and healthy participants using functional magnetic resonance imaging, while they performed three different tasks-lexical decision, sound judgment, and action judgment-on words that independently varied in their association with sounds and actions. We found neural activation for sound and action features of concepts selectively when they were task-relevant in brain regions also activated during auditory and motor tasks, respectively, as well as in higher-level, multimodal regions which were recruited during both sound and action feature retrieval. For the first time, we show that not only modality-specific perceptual-motor areas but also multimodal regions are engaged in conceptual processing in a flexible, task-dependent fashion, responding selectively to task-relevant conceptual features.

Literacy Training of Kindergarten Children With Pencil, Keyboard or Tablet Stylus: The Influence of the Writing Tool on Reading and Writing Performance at the Letter and Word Level

During the last years, digital writing devices are increasingly replacing handwriting with pencil and paper. As reading and writing skills are central for education, it is important to know, which writing tool is optimal for initial literacy education. The present training study was therefore set up to test the influence of the writing tool on the acquisition of literacy skills at the letter and word level with various tests in a large sample of kindergarten children (n = 147). Using closely matched letter learning games, children were trained with 16 letters by handwriting with a pencil on a sheet of paper, by writing with a stylus on a tablet computer, or by typing letters using a virtual keyboard on a tablet across 7 weeks. Training using a stylus on a touchscreen is an interesting comparison condition for traditional handwriting, because the slippery surface of a touchscreen has lower friction than paper and thus increases difficulty of motor control. Before training, immediately after training and four to five weeks after training, we assessed reading and writing performance using standardized tests. We also assessed visuo-spatial skills before and after training, in order to test, whether the different training regimens affected cognitive domains other than written language. Children of the pencil group showed superior performance in letter recognition and improved visuo-spatial skills compared with keyboard training. The performance of the stylus group did not differ significantly neither from the keyboard nor from the pencil group. Keyboard training, however, resulted in superior performance in word writing and reading compared with handwriting training with a stylus on the tablet, but not compared with the pencil group. Our results suggest that handwriting with pencil fosters acquisition of letter knowledge and improves visuo-spatial skills compared with keyboarding. At least given the current technological state, writing with a stylus on a touchscreen seems to be the least favorable writing tool, possibly because of increased demands on motor control. Future training studies covering a more extended observation period over years are needed to allow conclusions about long-term effects of writing tools on literacy acquisition as well as on general cognitive development.

Generalized Representation of Stereoscopic Surface Shape and Orientation in the Human Visual Cortex

The brain's ability to extract three-dimensional (3D) shape and orientation information from viewed objects is vital in daily life. Stereoscopic 3D surface perception relies on binocular disparity. Neurons selective to binocular disparity are widely distributed among visual areas, but the manner in these areas are involved in stereoscopic 3D surface representation is unclear. To address this, participants were instructed to observe random dot stereograms (RDS) depicting convex and concave curved surfaces and the blood oxygenation level-dependent (BOLD) signal of visual cortices was recorded. Two surface types were: (i) horizontally positioned surfaces defined by shear disparity; and (ii) vertically positioned surfaces defined by compression disparity. The surfaces were presented at different depth positions per trial. Functional magnetic resonance imaging (fMRI) data were classified from early visual areas to higher visual areas. We determined whether cortical areas were selective to shape and orientation by assessing same-type stimuli classification accuracies based on multi-voxel activity patterns per area. To identify whether some areas were related to a more generalized sign of curvature or orientation representation, transfer classification was used by training classifiers on one dataset type and testing classifiers on another type. Same-type stimuli classification results showed that most selected visual areas were selective to shape and all were selective to the orientation of disparity-defined 3D surfaces. Transfer classification results showed that in the dorsal visual area V3A, classification accuracies for the discriminate sign of surface curvature were higher than the baseline of statistical significance for all types of classifications, demonstrating that V3A is related to generalized shape representation. Classification accuracies for discriminating horizontal-vertical surfaces in higher dorsal areas V3A and V7 and ventral area lateral occipital complex (LOC) as well as in some areas of intraparietal sulcus (IPS) were higher than the baseline of statistical significance, indicating their relation to the generalized representation of 3D surface orientation.

Gesturing tool use and tool transport actions modulates inferior parietal functional connectivity with the dorsal and ventral object processing pathways

Interacting with manipulable objects (tools) requires the integration of diverse computations supported by anatomically remote regions. Previous functional neuroimaging research has demonstrated the left supramarginal (SMG) exhibits functional connectivity to both ventral and dorsal pathways, supporting the integration of ventrally-mediated tool properties and conceptual knowledge with dorsally-computed volumetric and structural representations of tools. This architecture affords us the opportunity to test whether interactions between the left SMG, ventral visual pathway, and dorsal visual pathway are differentially modulated when participants plan and generate tool-directed gestures emphasizing functional manipulation (tool use gesturing) or structure-based grasping (tool transport gesturing). We found that functional connectivity between the left SMG, ventral temporal cortex (bilateral fusiform gyri), and dorsal visual pathway (left superior parietal lobule/posterior intraparietal sulcus) was maximal for tool transport planning and gesturing, whereas functional connectivity between the left SMG, left ventral anterior temporal lobe, and left frontal operculum was maximal for tool use planning and gesturing. These results demonstrate that functional connectivity to the left SMG is differentially modulated by tool use and tool transport gesturing, suggesting that distinct tool features computed by the two object processing pathways are integrated in the parietal lobe in the service of tool-directed action.

Real-world size coding of solid objects, but not 2-D or 3-D images, in visual agnosia patients with bilateral ventral lesions

Patients with visual agnosia show severe deficits in recognizing two-dimensional (2-D) images of objects, despite the fact that early visual processes such as figure-ground segmentation, and stereopsis, are largely intact. Strikingly, however, these patients can nevertheless show a preservation in their ability to recognize real-world objects -a phenomenon known as the 'real-object advantage' (ROA) in agnosia. To uncover the mechanisms that support the ROA, patients were asked to identify objects whose size was congruent or incongruent with typical real-world size, presented in different display formats (real objects, 2-D and 3-D images). While recognition of images was extremely poor, real object recognition was surprisingly preserved, but only when physical size matched real-world size. Analogous display format and size manipulations did not influence the recognition of common geometric shapes that lacked real-world size associations. These neuropsychological data provide evidence for a surprising preservation of size-coding of real-world-sized tangible objects in patients for whom ventral contributions to image processing are severely disrupted. We propose that object size information is largely mediated by dorsal visual cortex and that this information, together with detailed representation of object shape which is also subserved by dorsal cortex, serve as the basis of the ROA.

The dominant role of functional action representation in object recognition

Action representation of manipulable objects has been found to be involved in object recognition. Recently, studies have indicated the existence of two distinct action systems: functional action specifying how to use an object and structural action concerning how to grasp an object. Despite evidence revealing the systems' anatomical and functional differences, few preceding studies have dissociated their respective roles in object recognition. The present study aimed to tease apart their roles in the recognition of manipulable objects with a priming paradigm. Specifically, we used static stimuli (photos, Experiments 1 and 2) and dynamic stimuli (video clips, Experiments 3 and 4) depicting functional and structural action hand gestures as primes and measured the magnitude of functional and structural action priming effect in object recognition. We found that static and dynamic priming stimuli induced a robust action priming effect only for functional action prime-target pairs. Naming latencies of the target objects were shorter when functional action representations of the prime and target were congruent than when they were incongruent. Moreover, as compared to static priming photos, dynamic priming stimuli induced a larger functional action priming effect. By contrast, neither static nor dynamic priming stimuli elicited a structural action priming effect. Behavioral data from our four experiments provide consistent evidence of the dominant role of functional action representation in the recognition of manipulable objects, suggesting that action knowledge regarding how to use rather than grasp an object is more likely an intrinsic component of objects' conceptual representation.

Action and semantic tool knowledge - Effective connectivity in the underlying neural networks

Evidence from neuropsychological and imaging studies indicate that action and semantic knowledge about tools draw upon distinct neural substrates, but little is known about the underlying interregional effective connectivity. With fMRI and dynamic causal modeling (DCM) we investigated effective connectivity in the left-hemisphere (LH) while subjects performed (i) a function knowledge and (ii) a value knowledge task, both addressing semantic tool knowledge, and (iii) a manipulation (action) knowledge task. Overall, the results indicate crosstalk between action nodes and semantic nodes. Interestingly, effective connectivity was weakened between semantic nodes and action nodes during the manipulation task. Furthermore, pronounced modulations of effective connectivity within the fronto-parietal action system of the LH (comprising lateral occipito-temporal cortex, intraparietal sulcus, supramarginal gyrus, inferior frontal gyrus) were observed in a bidirectional manner during the processing of action knowledge. In contrast, the function and value knowledge tasks resulted in a significant strengthening of the effective connectivity between visual cortex and fusiform gyrus. Importantly, this modulation was present in both semantic tasks, indicating that processing different aspects of semantic knowledge about tools evokes similar effective connectivity patterns. Data revealed that interregional effective connectivity during the processing of tool knowledge occurred in a bidirectional manner with a weakening of connectivity between areas engaged in action and semantic knowledge about tools during the processing of action knowledge. Moreover, different semantic tool knowledge tasks elicited similar effective connectivity patterns.

Preserved Object Weight Processing after Bilateral Lateral Occipital Complex Lesions

Object interaction requires knowledge of the weight of an object, as well as its shape. The lateral occipital complex (LOC), an area within the ventral visual pathway, is well known to be critically involved in processing visual shape information. Recently, however, LOC has also been implicated in coding object weight before grasping-a result that is surprising because weight is a nonvisual object property that is more relevant for motor interaction than visual perception. Here, we examined the causal role of LOC in perceiving heaviness and in determining appropriate fingertip forces during object lifting. We studied perceptions of heaviness and lifting behavior in a neuropsychological patient (M.C.) who has large bilateral occipitotemporal lesions that include LOC. We compared the patient's performance to a group of 18 neurologically healthy age-matched controls. Participants were asked to lift and report the perceived heaviness of a set of equally weighted spherical objects of various sizes-stimuli which typically induce the size-weight illusion, in which the smaller objects feel heavier than the larger objects despite having identical mass. Despite her ventral stream lesions, M.C. experienced a robust size-weight illusion induced by visual cues to object volume, and the magnitude of the illusion in M.C. was comparable to age-matched controls. Similarly, M.C. evinced predictive fingertip force scaling to visual size cues during her initial lifts of the objects that were well within the normal range. These single-case neuropsychological findings suggest that LOC is unlikely to play a causal role in computing object weight.

Functional reorganization of the conceptual brain system after deafness in early childhood

The neurodevelopmental consequences of deafness on the functional neuroarchitecture of the conceptual system have not been intensively investigated so far. Using functional magnetic resonance imaging (fMRI), we therefore identified brain areas involved in conceptual processing in deaf and hearing participants. Conceptual processing was probed by a pictorial animacy decision task. Furthermore, brain areas sensitive to observing verbal signs and to observing non-verbal visual hand actions were identified in deaf participants. In hearing participants, brain areas responsive to environmental sounds and the observation of visual hand actions were determined. We found a stronger recruitment of superior and middle temporal cortex in deaf compared to hearing participants during animacy decisions. This region, which forms auditory cortex in hearing people according to the sound listening task, was also activated in deaf participants, when they observed sign language, but not when they observed non-verbal hand actions. These results indicate that conceptual processing in deaf people more strongly depends on language representations compared to hearing people. Furthermore, additionally enhanced activation in visual and motor areas of deaf versus hearing participants during animacy decisions and a more frequent report of visual and motor features in the property listing task suggest that the loss of the auditory channel is partially compensated by an increased importance of visual and motor information for constituting object knowledge. Hence, our results indicate that conceptual processing in deaf compared to hearing people is more strongly based on the language system, complemented by an enhanced contribution of the visuo-motor system.

Towards a unified perspective of object shape and motion processing in human dorsal cortex

Although object-related areas were discovered in human parietal cortex a decade ago, surprisingly little is known about the nature and purpose of these representations, and how they differ from those in the ventral processing stream. In this article, we review evidence for the unique contribution of object areas of dorsal cortex to three-dimensional (3-D) shape representation, the localization of objects in space, and in guiding reaching and grasping actions. We also highlight the role of dorsal cortex in form-motion interaction and spatiotemporal integration, possible functional relationships between 3-D shape and motion processing, and how these processes operate together in the service of supporting goal-directed actions with objects. Fundamental differences between the nature of object representations in the dorsal versus ventral processing streams are considered, with an emphasis on how and why dorsal cortex supports veridical (rather than invariant) representations of objects to guide goal-directed hand actions in dynamic visual environments.

Investigating grounded conceptualization: motor system state-dependence facilitates familiarity judgments of novel tools

Theories of embodied cognition propose that we recognize tools in part by reactivating sensorimotor representations of tool use in a process of simulation. If motor simulations play a causal role in tool recognition then performing a concurrent motor task should differentially modulate recognition of experienced vs. non-experienced tools. We sought to test the hypothesis that an incompatible concurrent motor task modulates conceptual processing of learned vs. non-learned objects by directly manipulating the embodied experience of participants. We trained one group to use a set of novel, 3-D printed tools under the pretense that they were preparing for an archeological expedition to Mars (manipulation group); we trained a second group to report declarative information about how the tools are stored (storage group). With this design, familiarity and visual attention to different object parts was similar for both groups, though their qualitative interactions differed. After learning, participants made familiarity judgments of auditorily presented tool names while performing a concurrent motor task or simply sitting at rest. We showed that familiarity judgments were facilitated by motor state-dependence; specifically, in the manipulation group, familiarity was facilitated by a concurrent motor task, whereas in the spatial group familiarity was facilitated while sitting at rest. These results are the first to directly show that manipulation experience differentially modulates conceptual processing of familiar vs. unfamiliar objects, suggesting that embodied representations contribute to recognizing tools.

Three-Dimensional Representations of Objects in Dorsal Cortex are Dissociable from Those in Ventral Cortex

An established conceptualization of visual cortical function is one in which ventral regions mediate object perception while dorsal regions support spatial information processing and visually guided action. This division has been contested by evidence showing that dorsal regions are also engaged in the representation of object shape, even when actions are not required. The critical question is whether these dorsal, object-based representations are dissociable from ventral representations, and whether they play a functional role in object recognition. We examined the neural and behavioral profile of patients with impairments in object recognition following ventral cortex damage. In a functional magnetic resonanace imaging experiment, the blood oxygen level-dependent response in the ventral, but not dorsal, cortex of the patients evinced less sensitivity to object 3D structure compared with that of healthy controls. Consistently, in psychophysics experiments, the patients exhibited significant impairments in object perception, but still revealed residual sensitivity to object-based structural information. Together, these findings suggest that, although in the intact system there is considerable crosstalk between dorsal and ventral cortices, object representations in dorsal cortex can be computed independently from those in ventral cortex. While dorsal representations alone are unable to support normal object perception, they can, nevertheless, support a coarse description of object structural information.

Human Movement Responses to the Rorschach and Mirroring Activity: An fMRI Study

It has been suggested that the Rorschach human movement (M) response could be associated with an embodied simulation mechanism mediated by the mirror neuron system (MNS). To date, evidence for this hypothesis comes from two electroencephalogram studies and one repetitive transcranial magnetic stimulation study. To provide additional data on this topic, the Rorschach was administered during fMRI to a sample of 26 healthy adult volunteers. Activity in MNS-related brain areas temporally associated with M responses was compared with such activity for other, non-M Rorschach responses. Data analyses focused on MNS regions of interest identified by Neurosynth, a web-based platform for large scale, automated meta-analysis of fMRI data. Consistent with the hypothesis that M responses involve embodied simulation and MNS activity, univariate region of interest analyses showed that production of M responses associated with significantly greater activity in MNS-related brain areas when compared with non-M Rorschach responses. This finding is consistent with the traditional interpretation of the M code.

'What' Is Happening in the Dorsal Visual Pathway

The cortical visual system is almost universally thought to be segregated into two anatomically and functionally distinct pathways: a ventral occipitotemporal pathway that subserves object perception, and a dorsal occipitoparietal pathway that subserves object localization and visually guided action. Accumulating evidence from both human and non-human primate studies, however, challenges this binary distinction and suggests that regions in the dorsal pathway contain object representations that are independent of those in ventral cortex and that play a functional role in object perception. We review here the evidence implicating dorsal object representations, and we propose an account of the anatomical organization, functional contributions, and origins of these representations in the service of perception.

Somatotopic Semantic Priming and Prediction in the Motor System

The recognition of action-related sounds and words activates motor regions, reflecting the semantic grounding of these symbols in action information; in addition, motor cortex exerts causal influences on sound perception and language comprehension. However, proponents of classic symbolic theories still dispute the role of modality-preferential systems such as the motor cortex in the semantic processing of meaningful stimuli. To clarify whether the motor system carries semantic processes, we investigated neurophysiological indexes of semantic relationships between action-related sounds and words. Event-related potentials revealed that action-related words produced significantly larger stimulus-evoked (Mismatch Negativity-like) and predictive brain responses (Readiness Potentials) when presented in body-part-incongruent sound contexts (e.g., “kiss” in footstep sound context; “kick” in whistle context) than in body-part-congruent contexts, a pattern reminiscent of neurophysiological correlates of semantic priming. Cortical generators of the semantic relatedness effect were localized in areas traditionally associated with semantic memory, including left inferior frontal cortex and temporal pole, and, crucially, in motor areas, where body-part congruency of action sound–word relationships was indexed by a somatotopic pattern of activation. As our results show neurophysiological manifestations of action-semantic priming in the motor cortex, they prove semantic processing in the motor system and thus in a modality-preferential system of the human brain.

Handwriting or Typewriting? The Influence of Pen- or Keyboard-Based Writing Training on Reading and Writing Performance in Preschool Children

Digital writing devices associated with the use of computers, tablet PCs, or mobile phones are increasingly replacing writing by hand. It is, however, controversially discussed how writing modes influence reading and writing performance in children at the start of literacy. On the one hand, the easiness of typing on digital devices may accelerate reading and writing in young children, who have less developed sensory-motor skills. On the other hand, the meaningful coupling between action and perception during handwriting, which establishes sensory-motor memory traces, could facilitate written language acquisition. In order to decide between these theoretical alternatives, for the present study, we developed an intense training program for preschool children attending the German kindergarten with 16 training sessions. Using closely matched letter learning games, eight letters of the German alphabet were trained either by handwriting with a pen on a sheet of paper or by typing on a computer keyboard. Letter recognition, naming, and writing performance as well as word reading and writing performance were assessed. Results did not indicate a superiority of typing training over handwriting training in any of these tasks. In contrast, handwriting training was superior to typing training in word writing, and, as a tendency, in word reading. The results of our study, therefore, support theories of action-perception coupling assuming a facilitatory influence of sensory-motor representations established during handwriting on reading and writing.

Wearing weighted backpack dilates subjective visual duration: the role of functional linkage between weight experience and visual timing

Bodily state plays a critical role in our perception. In the present study, we asked the question whether and how bodily experience of weights influences time perception. Participants judged durations of a picture (a backpack or a trolley bag) presented on the screen, while wearing different weight backpacks or without backpack. The results showed that the subjective duration of the backpack picture was dilated when participants wore a medium weighted backpack relative to an empty backpack or without backpack, regardless of identity (e.g., color) of the visual backpack. However, the duration dilation was not manifested for the picture of trolley bag. These findings suggest that weight experience modulates visual duration estimation through the linkage between the wore backpack and to-be-estimated visual target. The congruent action affordance between the wore backpack and visual inputs plays a critical role in the functional linkage between inner experience and time perception. We interpreted our findings within the framework of embodied time perception.

Manipulation gesture effect in visual and auditory presentations: the link between tools in perceptual and motor tasks

There is much behavioral and neurophysiological evidence in support of the idea that seeing a tool activates motor components of action related to the perceived object (e.g., grasping, use manipulation). However, the question remains as to whether the processing of the motor components associated with the tool is automatic or depends on the situation, including the task and the modality of tool presentation. The present study investigated whether the activation of motor components involved in tool use in response to the simple perception of a tool is influenced by the link between prime and target tools, as well as by the modality of presentation, in perceptual or motor tasks. To explore this issue, we manipulated the similarity of gesture involved in the use of the prime and target (identical, similar, different) with two tool presentation modalities of the presentation tool (visual or auditory) in perceptual and motor tasks. Across the experiments, we also manipulated the relevance of the prime (i.e., associated or not with the current task). The participants saw a first tool (or heard the sound it makes), which was immediately followed by a second tool on which they had to perform a perceptual task (i.e., indicate whether the second tool was identical to or different from the first tool) or a motor task (i.e., manipulate the second tool as if it were the first tool). In both tasks, the similarity between the gestures employed for the first and the second tool was manipulated (Identical, Similar or Different gestures). The results showed that responses were faster when the manipulation gestures for the two tools were identical or similar, but only in the motor task. This effect was observed irrespective of the modality of presentation of the first tool, i.e., visual or auditory. We suggest that the influence of manipulation gesture on response time depends on the relevance of the first tool in motor tasks. We discuss these motor activation results in terms of the relevance and demands of the tasks.

Suggestion-Induced Modulation of Semantic Priming during Functional Magnetic Resonance Imaging

Using functional magnetic resonance imaging during a primed visual lexical decision task, we investigated the neural and functional mechanisms underlying modulations of semantic word processing through hypnotic suggestions aimed at altering lexical processing of primes. The priming task was to discriminate between target words and pseudowords presented 200 ms after the prime word which was semantically related or unrelated to the target. In a counterbalanced study design, each participant performed the task once at normal wakefulness and once after the administration of hypnotic suggestions to perceive the prime as a meaningless symbol of a foreign language. Neural correlates of priming were defined as significantly lower activations upon semantically related compared to unrelated trials. We found significant suggestive treatment-induced reductions in neural priming, albeit irrespective of the degree of suggestibility. Neural priming was attenuated upon suggestive treatment compared with normal wakefulness in brain regions supporting automatic (fusiform gyrus) and controlled semantic processing (superior and middle temporal gyri, pre- and postcentral gyri, and supplementary motor area). Hence, suggestions reduced semantic word processing by conjointly dampening both automatic and strategic semantic processes.

A saw is first identified as an object used on wood: ERP evidence for temporal differences between Thematic and Functional similarity relations

The role of functional and motor information in manipulable artifact object semantic organization is still poorly understood. In particular, several types of semantic relations involving object functional knowledge may be distinguished. Functional similarity relations group objects with similar functions at relatively specific (e.g. saw-axe, both used to cut wood) or general (saw-knife, both used to cut) levels. Thematic relations group objects based on their complementarity in events (saw used upon/with wood). Recent eye-tracking data showed distinct temporal time courses for the different semantic relations, with fastest processing of thematic relations and slowest processing of general function similarity relations. Behavioral data suggest the involvement of distinct cognitive mechanisms in manipulable artifact object semantic processing. The aim of the present study was to assess the neural correlates of thematic, and specific and general function similarity relation processing. Specifically, we investigated whether time course differences between semantic relations could be highlighted at the neurophysiological level. We used a protocol combining semantic priming with electroencephalography, and manipulated the type of semantic relation and the duration of the interval between prime and target objects. Two consistent and complementary results were shown. On N1 and P3 components, semantic priming was observed for thematic relations only. On N400 component, the type of semantic relation interacted with interval duration, and semantic priming was visible for all 3 relations after the longest interval only. Results revealed graded processing time courses for thematic, specific function similarity, and general function similarity relations at the neural level, and further indicate that thematic relations impact object processing during the early stages of object recognition. Findings suggest a hierarchical organization of three types of semantic relations based on functional knowledge. The parallel between semantic relations involving manipulable artifact objects and levels of action representations is discussed.