1
|
Walbrin J, Sossounov N, Mahdiani M, Vaz I, Almeida J. Fine-grained knowledge about manipulable objects is well-predicted by contrastive language image pre-training. iScience 2024; 27:110297. [PMID: 39040066 PMCID: PMC11261149 DOI: 10.1016/j.isci.2024.110297] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2023] [Revised: 02/23/2024] [Accepted: 06/14/2024] [Indexed: 07/24/2024] Open
Abstract
Object recognition is an important ability that relies on distinguishing between similar objects (e.g., deciding which utensil(s) to use at different stages of meal preparation). Recent work describes the fine-grained organization of knowledge about manipulable objects via the study of the constituent dimensions that are most relevant to human behavior, for example, vision, manipulation, and function-based properties. A logical extension of this work concerns whether or not these dimensions are uniquely human, or can be approximated by deep learning. Here, we show that behavioral dimensions are generally well-predicted by CLIP-ViT - a multimodal network trained on a large and diverse set of image-text pairs. Moreover, this model outperforms comparison networks pre-trained on smaller, image-only datasets. These results demonstrate the impressive capacity of CLIP-ViT to approximate fine-grained object knowledge. We discuss the possible sources of this benefit relative to other models (e.g., multimodal vs. image-only pre-training, dataset size, architecture).
Collapse
Affiliation(s)
- Jon Walbrin
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
| | - Nikita Sossounov
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
| | | | - Igor Vaz
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
| | - Jorge Almeida
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
| |
Collapse
|
2
|
Mahon BZ, Almeida J. Reciprocal interactions among parietal and occipito-temporal representations support everyday object-directed actions. Neuropsychologia 2024; 198:108841. [PMID: 38430962 DOI: 10.1016/j.neuropsychologia.2024.108841] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2023] [Revised: 02/19/2024] [Accepted: 02/25/2024] [Indexed: 03/05/2024]
Abstract
Everyday interactions with common manipulable objects require the integration of conceptual knowledge about objects and actions with real-time sensory information about the position, orientation and volumetric structure of the grasp target. The ability to successfully interact with everyday objects involves analysis of visual form and shape, surface texture, material properties, conceptual attributes such as identity, function and typical context, and visuomotor processing supporting hand transport, grasp form, and object manipulation. Functionally separable brain regions across the dorsal and ventral visual pathways support the processing of these different object properties and, in cohort, are necessary for functional object use. Object-directed grasps display end-state-comfort: they anticipate in form and force the shape and material properties of the grasp target, and how the object will be manipulated after it is grasped. End-state-comfort is the default for everyday interactions with manipulable objects and implies integration of information across the ventral and dorsal visual pathways. We propose a model of how visuomotor and action representations in parietal cortex interact with object representations in ventral and lateral occipito-temporal cortex. One pathway, from the supramarginal gyrus to the middle and inferior temporal gyrus, supports the integration of action-related information, including hand and limb position (supramarginal gyrus) with conceptual attributes and an appreciation of the action goal (middle temporal gyrus). A second pathway, from posterior IPS to the fusiform gyrus and collateral sulcus supports the integration of grasp parameters (IPS) with the surface texture and material properties (e.g., weight distribution) of the grasp target. Reciprocal interactions among these regions are part of a broader network of regions that support everyday functional object interactions.
Collapse
Affiliation(s)
- Bradford Z Mahon
- Department of Psychology, Carnegie Mellon University, USA; Neuroscience Institute, Carnegie Mellon University, USA; Department of Neurosurgery, University of Rochester Medical Center, USA.
| | - Jorge Almeida
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra, Portugal; CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Portugal
| |
Collapse
|
3
|
Luo X, Li M, Zeng J, Dai Z, Cui Z, Zhu M, Tian M, Wu J, Han Z. Mechanisms underlying category learning in the human ventral occipito-temporal cortex. Neuroimage 2024; 287:120520. [PMID: 38242489 DOI: 10.1016/j.neuroimage.2024.120520] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Revised: 01/07/2024] [Accepted: 01/17/2024] [Indexed: 01/21/2024] Open
Abstract
The human ventral occipito-temporal cortex (VOTC) has evolved into specialized regions that process specific categories, such as words, tools, and animals. The formation of these areas is driven by bottom-up visual and top-down nonvisual experiences. However, the specific mechanisms through which top-down nonvisual experiences modulate category-specific regions in the VOTC are still unknown. To address this question, we conducted a study in which participants were trained for approximately 13 h to associate three sets of novel meaningless figures with different top-down nonvisual features: the wordlike category with word features, the non-wordlike category with nonword features, and the visual familiarity condition with no nonvisual features. Pre- and post-training functional MRI (fMRI) experiments were used to measure brain activity during stimulus presentation. Our results revealed that training induced a categorical preference for the two training categories within the VOTC. Moreover, the locations of two training category-specific regions exhibited a notable overlap. Remarkably, within the overlapping category-specific region, training resulted in a dissociation in activation intensity and pattern between the two training categories. These findings provide important insights into how different nonvisual categorical information is encoded in the human VOTC.
Collapse
Affiliation(s)
- Xiangqi Luo
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China
| | - Mingyang Li
- Key Laboratory for Biomedical Engineering of Ministry of Education, Department of Biomedical Engineering, College of Biomedical Engineering & Instrument Science, Zhejiang University, Hangzhou 310027, PR China
| | - Jiahong Zeng
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China
| | - Zhiyun Dai
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China
| | - Zhenjiang Cui
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China
| | - Minhong Zhu
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China
| | - Mengxin Tian
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China
| | - Jiahao Wu
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China
| | - Zaizhu Han
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China.
| |
Collapse
|
4
|
Amaral L, Besson G, Caparelli-Dáquer E, Bergström F, Almeida J. Temporal differences and commonalities between hand and tool neural processing. Sci Rep 2023; 13:22270. [PMID: 38097608 PMCID: PMC10721913 DOI: 10.1038/s41598-023-48180-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Accepted: 11/23/2023] [Indexed: 12/17/2023] Open
Abstract
Object recognition is a complex cognitive process that relies on how the brain organizes object-related information. While spatial principles have been extensively studied, less studied temporal dynamics may also offer valuable insights into this process, particularly when neural processing overlaps for different categories, as it is the case of the categories of hands and tools. Here we focus on the differences and/or similarities between the time-courses of hand and tool processing under electroencephalography (EEG). Using multivariate pattern analysis, we compared, for different time points, classification accuracy for images of hands or tools when compared to images of animals. We show that for particular time intervals (~ 136-156 ms and ~ 252-328 ms), classification accuracy for hands and for tools differs. Furthermore, we show that classifiers trained to differentiate between tools and animals generalize their learning to classification of hand stimuli between ~ 260-320 ms and ~ 376-500 ms after stimulus onset. Classifiers trained to distinguish between hands and animals, on the other hand, were able to extend their learning to the classification of tools at ~ 150 ms. These findings suggest variations in semantic features and domain-specific differences between the two categories, with later-stage similarities potentially related to shared action processing for hands and tools.
Collapse
Affiliation(s)
- L Amaral
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal.
- Department of Neuroscience, Georgetown University Medical Center, Washington, DC, USA.
| | - G Besson
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
| | - E Caparelli-Dáquer
- Laboratory of Electrical Stimulation of the Nervous System (LabEEL), Rio de Janeiro State University, Rio de Janeiro, Brazil
| | - F Bergström
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- Department of Psychology, University of Gothenburg, Gothenburg, Sweden
| | - J Almeida
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal.
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal.
| |
Collapse
|
5
|
Almeida J, Fracasso A, Kristensen S, Valério D, Bergström F, Chakravarthi R, Tal Z, Walbrin J. Neural and behavioral signatures of the multidimensionality of manipulable object processing. Commun Biol 2023; 6:940. [PMID: 37709924 PMCID: PMC10502059 DOI: 10.1038/s42003-023-05323-x] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Accepted: 09/04/2023] [Indexed: 09/16/2023] Open
Abstract
Understanding how we recognize objects requires unravelling the variables that govern the way we think about objects and the neural organization of object representations. A tenable hypothesis is that the organization of object knowledge follows key object-related dimensions. Here, we explored, behaviorally and neurally, the multidimensionality of object processing. We focused on within-domain object information as a proxy for the decisions we typically engage in our daily lives - e.g., identifying a hammer in the context of other tools. We extracted object-related dimensions from subjective human judgments on a set of manipulable objects. We show that the extracted dimensions are cognitively interpretable and relevant - i.e., participants are able to consistently label them, and these dimensions can guide object categorization; and are important for the neural organization of knowledge - i.e., they predict neural signals elicited by manipulable objects. This shows that multidimensionality is a hallmark of the organization of manipulable object knowledge.
Collapse
Affiliation(s)
- Jorge Almeida
- Proaction Lab, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal.
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal.
| | - Alessio Fracasso
- School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK
| | - Stephanie Kristensen
- Proaction Lab, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
| | - Daniela Valério
- Proaction Lab, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
| | - Fredrik Bergström
- Proaction Lab, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- Department of Psychology, University of Gothenburg, Gothenburg, Sweden
| | | | - Zohar Tal
- Proaction Lab, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
| | - Jonathan Walbrin
- Proaction Lab, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
| |
Collapse
|
6
|
Almeida J, Martins AR, Amaral L, Valério D, Bukhari Q, Schu G, Nogueira J, Spínola M, Soleimani G, Fernandes F, Silva AR, Fregni F, Simis M, Simões M, Peres A. The cerebellum is causally involved in episodic memory under aging. GeroScience 2023; 45:2267-2287. [PMID: 36749471 PMCID: PMC10651631 DOI: 10.1007/s11357-023-00738-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2022] [Accepted: 01/17/2023] [Indexed: 02/08/2023] Open
Abstract
Episodic memory decline is a major signature of both normal and pathological aging. Many neural regions have been implicated in the processes subserving both episodic memory and typical aging decline. Here, we demonstrate that the cerebellum is causally involved episodic memory under aging. We show that a 12-day neurostimulation program delivered to the right cerebellum led to improvements in episodic memory performance under healthy aging that long outlast the stimulation period - healthy elderly individuals show episodic memory improvement both immediately after the intervention program and in a 4-month follow-up. These results demonstrate the causal relevance of the cerebellum in processes associated with long-term episodic memory, potentially highlighting its role in regulating and maintaining cognitive processing. Moreover, they point to the importance of non-pharmacological interventions that prevent or diminish cognitive decline in healthy aging.
Collapse
Affiliation(s)
- Jorge Almeida
- Proaction Lab, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal.
- CINEICC, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal.
| | - Ana R Martins
- Proaction Lab, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
| | - Lénia Amaral
- Proaction Lab, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
- CINEICC, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
- Department of Neuroscience, Georgetown University Medical Center, Washington, USA
| | - Daniela Valério
- Proaction Lab, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
- CINEICC, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
| | - Qasim Bukhari
- Proaction Lab, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
- CINEICC, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
| | - Guilherme Schu
- Proaction Lab, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
- CINEICC, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
| | - Joana Nogueira
- CINEICC, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
- Psychological Assessment and Psychometrics Laboratory, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
| | - Mónica Spínola
- CINEICC, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
- Psychological Assessment and Psychometrics Laboratory, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
- NOVA LINCS, University of Madeira, Caminho da Penteada, 9020-105, Funchal, Portugal
| | - Ghazaleh Soleimani
- Department of Biomedical Engineering, Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran
- Department of Psychiatry, University of Minnesota, Minneapolis, USA
| | | | - Ana R Silva
- CINEICC, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
- Psychological Assessment and Psychometrics Laboratory, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
| | - Felipe Fregni
- Spaulding Neuromodulation Center, Department of Physical Medicine & Rehabilitation, Spaulding Rehabilitation Hospital and Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Marcel Simis
- Faculdade de Medicina, Hospital das Clinicas HCFMUSP, Universidade de São Paulo, São Paulo, Brazil
| | - Mário Simões
- CINEICC, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
- Psychological Assessment and Psychometrics Laboratory, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
| | - André Peres
- Proaction Lab, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
- CINEICC, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
| |
Collapse
|
7
|
Amaral L, Donato R, Valério D, Caparelli-Dáquer E, Almeida J, Bergström F. Disentangling hand and tool processing: Distal effects of neuromodulation. Cortex 2022; 157:142-154. [PMID: 36283136 DOI: 10.1016/j.cortex.2022.08.011] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2021] [Revised: 05/29/2022] [Accepted: 08/24/2022] [Indexed: 12/15/2022]
Abstract
Neural processing within a local brain region that responds to more than one object category (e.g., hands and tools) nonetheless have different functional connectivity patterns with other distal brain areas, which suggests that local processing can affect and/or be affected by processing in distal areas, in a category-specific way. Here we wanted to test whether administering either a hand- or tool-related training task in tandem with transcranial direct current stimulation (tDCS) to a region that responds both to hands and tools (posterior middle temporal gyrus; pMTG), modulated local and distal neural processing more for the trained than the untrained category in a subsequent fMRI task. After each combined tDCS/training session, participants viewed images of tools, hands, and animals, in an fMRI scanner. Using multivoxel pattern analysis, we found that tDCS stimulation to pMTG indeed improved the classification accuracy between tools vs. animals, but only when combined with a tool and not a hand training task. Surprisingly, tDCS stimulation to pMTG also improved classification accuracy between hands vs. animals when combined with a tool but not a hand training task. Our findings suggest that overlapping but functionally-specific networks may be engaged separately by using a category-specific training task together with tDCS - a strategy that can be applied more broadly to other cognitive domains using tDCS. By hypothesis, these effects on local processing are a direct result of within-domain connectivity constraints from domain-specific networks that are at play in the processing and organization of object representations.
Collapse
Affiliation(s)
- Lénia Amaral
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra. Portugal; CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra. Portugal
| | - Rita Donato
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra. Portugal; Department of General Psychology, University of Padova, Italy; Human Inspired Technology Centre, University of Padova, Italy
| | - Daniela Valério
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra. Portugal; CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra. Portugal
| | - Egas Caparelli-Dáquer
- Laboratory of Electrical Stimulation of the Nervous System (LabEEL), Rio de Janeiro State University, Brazil
| | - Jorge Almeida
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra. Portugal; CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra. Portugal.
| | - Fredrik Bergström
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra. Portugal; CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra. Portugal; Department of Psychology, University of Gothenburg, Sweden.
| |
Collapse
|
8
|
Pilacinski A, De Haan S, Donato R, Almeida J. Tool heads prime saccades. Sci Rep 2021; 11:11954. [PMID: 34099787 PMCID: PMC8184872 DOI: 10.1038/s41598-021-91254-8] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2021] [Accepted: 05/21/2021] [Indexed: 11/09/2022] Open
Abstract
Tools are wielded by their handles, but a lot of information about their function comes from their heads (the action-ends). Here we investigated whether eye saccadic movements are primed by tool handles, or whether they are primed by tool heads. We measured human saccadic reaction times while subjects were performing an attentional task. We found that saccades were executed quicker when performed to the side congruent with the tool head, even though "toolness" was irrelevant for the task. Our results show that heads are automatically processed by the visual system to orient eye movements, indicating that eyes are attracted by functional parts of manipulable objects and by the characteristic information these parts convey.
Collapse
Affiliation(s)
- Artur Pilacinski
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra, 3000-115, Coimbra, Portugal. .,CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal.
| | - Stella De Haan
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra, 3000-115, Coimbra, Portugal.,CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal.,GIGA-CRC In Vivo Imaging, University of Liège, Liège, Belgium
| | - Rita Donato
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra, 3000-115, Coimbra, Portugal.,CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal.,Department of General Psychology, University of Padova, Padova, Italy.,Human Inspired Technology Research Centre, University of Padova, Padova, Italy
| | - Jorge Almeida
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra, 3000-115, Coimbra, Portugal.,CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
| |
Collapse
|
9
|
Valério D, Santana I, Aguiar de Sousa D, Schu G, Leal G, Pavão Martins I, Almeida J. Knowing how to do it or doing it? A double dissociation between tool-gesture production and tool-gesture knowledge. Cortex 2021; 141:449-464. [PMID: 34147827 DOI: 10.1016/j.cortex.2021.05.008] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2021] [Revised: 04/05/2021] [Accepted: 05/11/2021] [Indexed: 10/21/2022]
Abstract
Deciding how to manipulate an object to fulfill a goal requires accessing different types of object-related information. How these different types of information are integrated and represented in the brain is still an open question. Here, we focus on examining two types of object-related information-tool-gesture knowledge (i.e., how to manipulate an object), and tool-gesture production (i.e., the actual manipulation of an object). We show a double dissociation between tool-gesture knowledge and tool-gesture production: Patient FP presents problems in pantomiming tool use in the context of a spared ability to perform judgments about an object's manipulation, whereas Patient LS can pantomime tool use, but is impaired at performing manipulation judgments. Moreover, we compared the location of the lesions in FP and LS with those sustained by two classic ideomotor apraxic patients (IMA), using a cortical thickness approach. Patient FP presented lesions in common with our classic IMA that included the left inferior parietal lobule (IPL), and specifically the supramarginal gyrus, the left parietal operculum, the left premotor cortex and the left inferior frontal gyrus, whereas Patient LS and our classic IMA patients presented common lesions in regions of the superior parietal lobule (SPL), motor areas (as primary somatosensory cortex, premotor cortex and primary motor cortex), and frontal areas. Our results show that tool-gesture production and tool-gesture knowledge can be behaviorally and neurally doubly dissociated and put strong constraints on extant theories of action and object recognition and use.
Collapse
Affiliation(s)
- Daniela Valério
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra, Portugal; CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Portugal
| | - Isabel Santana
- Neurology Department and Dementia Clinic, Centro Hospitalar e Universitário de Coimbra, Coimbra, Portugal; Centre for Neuroscience and Cell Biology (CNC), University of Coimbra, Coimbra, Portugal; Faculty of Medicine, University of Coimbra, Coimbra, Portugal
| | | | - Guilherme Schu
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra, Portugal; CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Portugal
| | - Gabriela Leal
- Language Research Laboratory, Faculty of Medicine, University of Lisbon, Portugal
| | - Isabel Pavão Martins
- Neurology Department, Hospital de Santa Maria, Lisbon, Portugal; Language Research Laboratory, Faculty of Medicine, University of Lisbon, Portugal
| | - Jorge Almeida
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra, Portugal; CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Portugal.
| |
Collapse
|
10
|
Walbrin J, Almeida J. High-Level Representations in Human Occipito-Temporal Cortex Are Indexed by Distal Connectivity. J Neurosci 2021; 41:4678-4685. [PMID: 33849949 PMCID: PMC8260247 DOI: 10.1523/jneurosci.2857-20.2021] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2020] [Revised: 03/09/2021] [Accepted: 03/15/2021] [Indexed: 12/30/2022] Open
Abstract
Human object recognition is dependent on occipito-temporal cortex (OTC), but a complete understanding of the complex functional architecture of this area must account for how it is connected to the wider brain. Converging functional magnetic resonance imaging evidence shows that univariate responses to different categories of information (e.g., faces, bodies, and nonhuman objects) are strongly related to, and potentially shaped by, functional and structural connectivity to the wider brain. However, to date, there have been no systematic attempts to determine how distal connectivity and complex local high-level responses in occipito-temporal cortex (i.e., multivoxel response patterns) are related. Here, we show that distal functional connectivity is related to, and can reliably index, high-level representations for several visual categories (i.e., tools, faces, and places) within occipito-temporal cortex; that is, voxel sets that are strongly connected to distal brain areas show higher pattern discriminability than less well-connected sets do. We further show that in several cases, pattern discriminability is higher in sets of well-connected voxels than sets defined by local activation (e.g., strong amplitude responses to faces in fusiform face area). Together, these findings demonstrate the important relationship between the complex functional organization of occipito-temporal cortex and wider brain connectivity.SIGNIFICANCE STATEMENT Human object recognition relies strongly on OTC, yet responses in this broad area are often considered in relative isolation to the rest of the brain. We employ a novel connectivity-guided voxel selection approach with functional magnetic resonance imaging data to show higher sensitivity to information (i.e., higher multivoxel pattern discriminability) in voxel sets that share strong connectivity to distal brain areas, relative to (1) voxel sets that are less strongly connected, and in several cases, (2) voxel sets that are defined by strong local response amplitude. These findings underscore the importance of distal contributions to local processing in OTC.
Collapse
Affiliation(s)
- Jon Walbrin
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra, 3004-531 Coimbra, Portugal
| | - Jorge Almeida
- Proaction Laboratory, Faculty of Psychology and Educational Sciences, University of Coimbra, 3004-531 Coimbra, Portugal
| |
Collapse
|