1
|
Kabulska Z, Lingnau A. The cognitive structure underlying the organization of observed actions. Behav Res Methods 2023; 55:1890-1906. [PMID: 35788973 PMCID: PMC10250259 DOI: 10.3758/s13428-022-01894-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/26/2022] [Indexed: 11/08/2022]
Abstract
In daily life, we frequently encounter actions performed by other people. Here we aimed to examine the key categories and features underlying the organization of a wide range of actions in three behavioral experiments (N = 378 participants). In Experiment 1, we used a multi-arrangement task of 100 different actions. Inverse multidimensional scaling and hierarchical clustering revealed 11 action categories, including Locomotion, Communication, and Aggressive actions. In Experiment 2, we used a feature-listing paradigm to obtain a wide range of action features that were subsequently reduced to 59 key features and used in a rating study (Experiment 3). A direct comparison of the feature ratings obtained in Experiment 3 between actions belonging to the categories identified in Experiment 1 revealed a number of features that appear to be critical for the distinction between these categories, e.g., the features Harm and Noise for the category Aggressive actions, and the features Targeting a person and Contact with others for the category Interaction. Finally, we found that a part of the category-based organization is explained by a combination of weighted features, whereas a significant proportion of variability remained unexplained, suggesting that there are additional sources of information that contribute to the categorization of observed actions. The characterization of action categories and their associated features serves as an important extension of previous studies examining the cognitive structure of actions. Moreover, our results may serve as the basis for future behavioral, neuroimaging and computational modeling studies.
Collapse
Affiliation(s)
- Zuzanna Kabulska
- Department of Psychology, Faculty of Human Sciences, University of Regensburg, Universitätsstraße 31, 93053, Regensburg, Germany
| | - Angelika Lingnau
- Department of Psychology, Faculty of Human Sciences, University of Regensburg, Universitätsstraße 31, 93053, Regensburg, Germany.
| |
Collapse
|
2
|
Abstract
Visual representations of bodies, in addition to those of faces, contribute to the recognition of con- and heterospecifics, to action recognition, and to nonverbal communication. Despite its importance, the neural basis of the visual analysis of bodies has been less studied than that of faces. In this article, I review what is known about the neural processing of bodies, focusing on the macaque temporal visual cortex. Early single-unit recording work suggested that the temporal visual cortex contains representations of body parts and bodies, with the dorsal bank of the superior temporal sulcus representing bodily actions. Subsequent functional magnetic resonance imaging studies in both humans and monkeys showed several temporal cortical regions that are strongly activated by bodies. Single-unit recordings in the macaque body patches suggest that these represent mainly body shape features. More anterior patches show a greater viewpoint-tolerant selectivity for body features, which may reflect a processing principle shared with other object categories, including faces. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.
Collapse
Affiliation(s)
- Rufin Vogels
- Laboratorium voor Neuro- en Psychofysiologie, KU Leuven, Belgium; .,Leuven Brain Institute, KU Leuven, Belgium
| |
Collapse
|
3
|
Brand J, Piccirelli M, Hepp-Reymond MC, Eng K, Michels L. Brain Activation During Visually Guided Finger Movements. Front Hum Neurosci 2020; 14:309. [PMID: 32922274 PMCID: PMC7456884 DOI: 10.3389/fnhum.2020.00309] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2020] [Accepted: 07/13/2020] [Indexed: 11/13/2022] Open
Abstract
Computer interaction via visually guided hand movements often employs either abstract cursor-based feedback or virtual hand (VH) representations of varying degrees of realism. The effect of changing this visual feedback in virtual reality settings is currently unknown. In this study, 19 healthy right-handed adults performed index finger movements (“action”) and observed movements (“observation”) with four different types of visual feedback: a simple circular cursor (CU), a point light (PL) pattern indicating finger joint positions, a shadow cartoon hand (SH) and a realistic VH. Finger movements were recorded using a data glove, and eye-tracking was recorded optically. We measured brain activity using functional magnetic resonance imaging (fMRI). Both action and observation conditions showed stronger fMRI signal responses in the occipitotemporal cortex compared to baseline. The action conditions additionally elicited elevated bilateral activations in motor, somatosensory, parietal, and cerebellar regions. For both conditions, feedback of a hand with a moving finger (SH, VH) led to higher activations than CU or PL feedback, specifically in early visual regions and the occipitotemporal cortex. Our results show the stronger recruitment of a network of cortical regions during visually guided finger movements with human hand feedback when compared to a visually incomplete hand and abstract feedback. This information could have implications for the design of visually guided tasks involving human body parts in both research and application or training-related paradigms.
Collapse
Affiliation(s)
- Johannes Brand
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland.,Neuroscience Center Zurich, Zurich, Switzerland
| | - Marco Piccirelli
- Department of Neuroradiology, University Hospital Zurich, Zurich, Switzerland.,Klinisches Neurozentrum, University Hospital Zurich, Zurich, Switzerland
| | - Marie-Claude Hepp-Reymond
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland.,Neuroscience Center Zurich, Zurich, Switzerland
| | - Kynan Eng
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland.,Neuroscience Center Zurich, Zurich, Switzerland
| | - Lars Michels
- Department of Neuroradiology, University Hospital Zurich, Zurich, Switzerland.,Klinisches Neurozentrum, University Hospital Zurich, Zurich, Switzerland
| |
Collapse
|
4
|
Poyo Solanas M, Vaessen M, de Gelder B. Computation-Based Feature Representation of Body Expressions in the Human Brain. Cereb Cortex 2020; 30:6376-6390. [DOI: 10.1093/cercor/bhaa196] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2020] [Revised: 06/04/2020] [Accepted: 06/26/2020] [Indexed: 01/31/2023] Open
Abstract
Abstract
Humans and other primate species are experts at recognizing body expressions. To understand the underlying perceptual mechanisms, we computed postural and kinematic features from affective whole-body movement videos and related them to brain processes. Using representational similarity and multivoxel pattern analyses, we showed systematic relations between computation-based body features and brain activity. Our results revealed that postural rather than kinematic features reflect the affective category of the body movements. The feature limb contraction showed a central contribution in fearful body expression perception, differentially represented in action observation, motor preparation, and affect coding regions, including the amygdala. The posterior superior temporal sulcus differentiated fearful from other affective categories using limb contraction rather than kinematics. The extrastriate body area and fusiform body area also showed greater tuning to postural features. The discovery of midlevel body feature encoding in the brain moves affective neuroscience beyond research on high-level emotion representations and provides insights in the perceptual features that possibly drive automatic emotion perception.
Collapse
Affiliation(s)
- Marta Poyo Solanas
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Limburg 6200 MD, The Netherlands
| | - Maarten Vaessen
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Limburg 6200 MD, The Netherlands
| | - Beatrice de Gelder
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Limburg 6200 MD, The Netherlands
- Department of Computer Science, University College London, London WC1E 6BT, UK
| |
Collapse
|
5
|
Ross P, Flack T. Removing Hand Form Information Specifically Impairs Emotion Recognition for Fearful and Angry Body Stimuli. Perception 2019; 49:98-112. [PMID: 31801026 DOI: 10.1177/0301006619893229] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Emotion perception research has largely been dominated by work on facial expressions, but emotion is also strongly conveyed from the body. Research exploring emotion recognition from the body tends to refer to “the body” as a whole entity. However, the body is made up of different components (hands, arms, trunk, etc.), all of which could be differentially contributing to emotion recognition. We know that the hands can help to convey actions and, in particular, are important for social communication through gestures, but we currently do not know to what extent the hands influence emotion recognition from the body. Here, 93 adults viewed static emotional body stimuli with either the hands, arms, or both components removed and completed a forced-choice emotion recognition task. Removing the hands significantly reduced recognition accuracy for fear and anger but made no significant difference to the recognition of happiness and sadness. Removing the arms had no effect on emotion recognition accuracy compared with the full-body stimuli. These results suggest the hands may play a key role in the recognition of emotions from the body.
Collapse
Affiliation(s)
- Paddy Ross
- Department of Psychology, Durham University, UK
| | - Tessa Flack
- School of Psychology, University of Lincoln, UK
| |
Collapse
|
6
|
Kim Y, Usui N, Miyazaki A, Haji T, Matsumoto K, Taira M, Nakamura K, Katsuyama N. Cortical Regions Encoding Hardness Perception Modulated by Visual Information Identified by Functional Magnetic Resonance Imaging With Multivoxel Pattern Analysis. Front Syst Neurosci 2019; 13:52. [PMID: 31632245 PMCID: PMC6779815 DOI: 10.3389/fnsys.2019.00052] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2019] [Accepted: 09/11/2019] [Indexed: 01/09/2023] Open
Abstract
Recent studies have revealed that hardness perception is determined by visual information along with the haptic input. This study investigated the cortical regions involved in hardness perception modulated by visual information using functional magnetic resonance imaging (fMRI) and multivoxel pattern analysis (MVPA). Twenty-two healthy participants were enrolled. They were required to place their left and right hands at the front and back, respectively, of a mirror attached to a platform placed above them while lying in a magnetic resonance scanner. In conditions SFT, MED, and HRD, one of three polyurethane foam pads of varying hardness (soft, medium, and hard, respectively) was presented to the left hand in a given trial, while only the medium pad was presented to the right hand in all trials. MED was defined as the control condition, because the visual and haptic information was congruent. During the scan, the participants were required to push the pad with the both hands while observing the reflection of the left hand and estimate the hardness of the pad perceived by the right (hidden) hand based on magnitude estimation. Behavioral results showed that the perceived hardness was significantly biased toward softer or harder in >73% of the trials in conditions SFT and HRD; we designated these trials as visually modulated (SFTvm and HRDvm, respectively). The accuracy map was calculated individually for each of the pair-wise comparisons of (SFTvm vs. MED), (HRDvm vs. MED), and (SFTvm vs. HRDvm) by a searchlight MVPA, and the cortical regions encoding the perceived hardness with visual modulation were identified by conjunction of the three accuracy maps in group analysis. The cluster was observed in the right sensory motor cortex, left anterior intraparietal sulcus (aIPS), bilateral parietal operculum (PO), and occipito-temporal cortex (OTC). Together with previous findings on such cortical regions, we conclude that the visual information of finger movements processed in the OTC may be integrated with haptic input in the left aIPS, and the subjective hardness perceived by the right hand with visual modulation may be processed in the cortical network between the left PO and aIPS.
Collapse
Affiliation(s)
- Yuri Kim
- Primate Research Institute, Kyoto University, Inuyama, Japan.,Department of Cognitive Neurobiology, Tokyo Medical and Dental University, Tokyo, Japan.,Center for Brain Integration Research, Tokyo Medical and Dental University, Tokyo, Japan
| | - Nobuo Usui
- Department of Cognitive Neurobiology, Tokyo Medical and Dental University, Tokyo, Japan.,Center for Brain Integration Research, Tokyo Medical and Dental University, Tokyo, Japan
| | | | - Tomoki Haji
- Tamagawa University Brain Science Institute, Tokyo, Japan
| | | | - Masato Taira
- Department of Cognitive Neurobiology, Tokyo Medical and Dental University, Tokyo, Japan.,Center for Brain Integration Research, Tokyo Medical and Dental University, Tokyo, Japan
| | | | - Narumi Katsuyama
- Primate Research Institute, Kyoto University, Inuyama, Japan.,Department of Cognitive Neurobiology, Tokyo Medical and Dental University, Tokyo, Japan.,Center for Brain Integration Research, Tokyo Medical and Dental University, Tokyo, Japan
| |
Collapse
|
7
|
van den Heiligenberg FMZ, Orlov T, Macdonald SN, Duff EP, Henderson Slater D, Beckmann CF, Johansen-Berg H, Culham JC, Makin TR. Artificial limb representation in amputees. Brain 2019. [PMID: 29534154 PMCID: PMC5917779 DOI: 10.1093/brain/awy054] [Citation(s) in RCA: 40] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022] Open
Abstract
The human brain contains multiple hand-selective areas, in both the sensorimotor and visual systems. Could our brain repurpose neural resources, originally developed for supporting hand function, to represent and control artificial limbs? We studied individuals with congenital or acquired hand-loss (hereafter one-handers) using functional MRI. We show that the more one-handers use an artificial limb (prosthesis) in their everyday life, the stronger visual hand-selective areas in the lateral occipitotemporal cortex respond to prosthesis images. This was found even when one-handers were presented with images of active prostheses that share the functionality of the hand but not necessarily its visual features (e.g. a ‘hook’ prosthesis). Further, we show that daily prosthesis usage determines large-scale inter-network communication across hand-selective areas. This was demonstrated by increased resting state functional connectivity between visual and sensorimotor hand-selective areas, proportional to the intensiveness of everyday prosthesis usage. Further analysis revealed a 3-fold coupling between prosthesis activity, visuomotor connectivity and usage, suggesting a possible role for the motor system in shaping use-dependent representation in visual hand-selective areas, and/or vice versa. Moreover, able-bodied control participants who routinely observe prosthesis usage (albeit less intensively than the prosthesis users) showed significantly weaker associations between degree of prosthesis observation and visual cortex activity or connectivity. Together, our findings suggest that altered daily motor behaviour facilitates prosthesis-related visual processing and shapes communication across hand-selective areas. This neurophysiological substrate for prosthesis embodiment may inspire rehabilitation approaches to improve usage of existing substitutionary devices and aid implementation of future assistive and augmentative technologies.
Collapse
Affiliation(s)
- Fiona M Z van den Heiligenberg
- Institute of Cognitive Neuroscience, University College London, London, UK.,FMRIB Centre, Nuffield Department of Clinical Neuroscience, University of Oxford, Oxford, UK.,Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, The Netherlands
| | - Tanya Orlov
- Neurobiology Department, Life Sciences Institute, Hebrew University of Jerusalem, Jerusalem, Israel
| | - Scott N Macdonald
- Brain and Mind Institute, Department of Psychology, University of Western Ontario, Canada
| | - Eugene P Duff
- FMRIB Centre, Nuffield Department of Clinical Neuroscience, University of Oxford, Oxford, UK
| | - David Henderson Slater
- FMRIB Centre, Nuffield Department of Clinical Neuroscience, University of Oxford, Oxford, UK.,Oxford Centre for Enablement, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Christian F Beckmann
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, The Netherlands
| | - Heidi Johansen-Berg
- FMRIB Centre, Nuffield Department of Clinical Neuroscience, University of Oxford, Oxford, UK
| | - Jody C Culham
- Brain and Mind Institute, Department of Psychology, University of Western Ontario, Canada
| | - Tamar R Makin
- Institute of Cognitive Neuroscience, University College London, London, UK.,FMRIB Centre, Nuffield Department of Clinical Neuroscience, University of Oxford, Oxford, UK.,Wellcome Centre for Human Neuroimaging, University College London, London, UK
| |
Collapse
|
8
|
Orlandi A, Proverbio AM. Bilateral engagement of the occipito-temporal cortex in response to dance kinematics in experts. Sci Rep 2019; 9:1000. [PMID: 30700799 PMCID: PMC6353946 DOI: 10.1038/s41598-018-37876-x] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2018] [Accepted: 12/14/2018] [Indexed: 01/04/2023] Open
Abstract
Previous evidence has shown neuroplastic changes in brain anatomy and connectivity associated with the acquisition of professional visuomotor skills. Reduced hemispherical asymmetry was found in the sensorimotor and visual areas in expert musicians and athletes compared with non-experts. Moreover, increased expertise with faces, body, and objects resulted in an enhanced engagement of the occipito-temporal cortex (OTC) during stimulus observation. The present study aimed at investigating whether intense and extended practice with dance would result in an enhanced symmetric response of OTC at an early stage of action processing. Expert ballet dancers and non-dancer controls were presented with videos depicting ballet steps during EEG recording. The observation of the moving dancer elicited a posterior N2 component, being larger over the left hemisphere in dancers than controls. The source reconstruction (swLORETA) of the negativity showed the engagement of the bilateral inferior and middle temporal regions in experts, while right-lateralized activity was found in controls. The dancers also showed an early P2 and enhanced P300 responses, indicating faster stimulus processing and subsequent recognition. This evidence seemed to suggest expertise-related increased sensitivity of the OTC in encoding body kinematics. Thus, we speculated that long-term whole-body practice would result in enriched and refined action processing.
Collapse
Affiliation(s)
- Andrea Orlandi
- Neuro-MI, Milan Center for Neuroscience, Department of Psychology, University of Milano - Bicocca, Milan, Italy.
| | - Alice Mado Proverbio
- Neuro-MI, Milan Center for Neuroscience, Department of Psychology, University of Milano - Bicocca, Milan, Italy
| |
Collapse
|
9
|
N1 responses to images of hands in occipito-temporal event-related potentials. Neuropsychologia 2017; 106:83-89. [DOI: 10.1016/j.neuropsychologia.2017.09.012] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2017] [Revised: 08/09/2017] [Accepted: 09/12/2017] [Indexed: 11/18/2022]
|
10
|
Action Categories in Lateral Occipitotemporal Cortex Are Organized Along Sociality and Transitivity. J Neurosci 2017; 37:562-575. [PMID: 28100739 DOI: 10.1523/jneurosci.1717-16.2016] [Citation(s) in RCA: 70] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2016] [Revised: 11/07/2016] [Accepted: 11/11/2016] [Indexed: 01/04/2023] Open
Abstract
How neural specificity for distinct conceptual knowledge categories arises is central for understanding the organization of semantic memory in the human brain. Although there is a large body of research on the neural processing of distinct object categories, the organization of action categories remains largely unknown. In particular, it is unknown whether different action categories follow a specific topographical organization on the cortical surface analogously to the category-specific organization of object knowledge. Here, we tested whether the neural representation of action knowledge is organized in terms of nonsocial versus social and object-unrelated versus object-related actions (sociality and transitivity, respectively, hereafter). We hypothesized a major distinction of sociality and transitivity along dorsal and ventral lateral occipitotemporal cortex (LOTC), respectively. Using fMRI-based multivoxel pattern analysis, we identified neural representations of action information associated with sociality and transitivity in bilateral LOTC. Representational similarity analysis revealed a dissociation between dorsal and ventral LOTC. We found that action representations in dorsal LOTC are segregated along features of sociality, whereas action representations in ventral LOTC are segregated along features of transitivity. In addition, representations of sociality and transitivity features were found more anteriorly in LOTC than representations of specific subtypes of actions, suggesting a posterior-anterior gradient from concrete to abstract action features. These findings elucidate how the neural representations of perceptually and conceptually diverse actions are organized in distinct subsystems in the LOTC. SIGNIFICANCE STATEMENT The lateral occipitotemporal cortex (LOTC) is critically involved in the recognition of objects and actions, but our knowledge about the underlying organizing principles is limited. Here, we discovered a dorsal-ventral distinction of actions in LOTC: dorsal LOTC represents actions based on sociality (how much an action is directed to another person) in proximity to person knowledge. In contrast, ventral LOTC represents actions based on transitivity (how much an action involves the interaction with inanimate objects) in proximity to tools/artifacts in ventral LOTC, suggesting a mutually dependent organization of actions and objects. In addition, we found a posterior-to-anterior organization of the LOTC for concrete and abstract representations, respectively. Our findings provide important insights about the organization of actions in LOTC.
Collapse
|
11
|
Neural Representations of Observed Actions Generalize across Static and Dynamic Visual Input. J Neurosci 2017; 37:3056-3071. [PMID: 28209734 DOI: 10.1523/jneurosci.2496-16.2017] [Citation(s) in RCA: 37] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2016] [Revised: 01/16/2017] [Accepted: 02/03/2017] [Indexed: 11/21/2022] Open
Abstract
People interact with entities in the environment in distinct and categorizable ways (e.g., kicking is making contact with foot). We can recognize these action categories across variations in actors, objects, and settings; moreover, we can recognize them from both dynamic and static visual input. However, the neural systems that support action recognition across these perceptual differences are unclear. Here, we used multivoxel pattern analysis of fMRI data to identify brain regions that support visual action categorization in a format-independent way. Human participants were scanned while viewing eight categories of interactions (e.g., pulling) depicted in two visual formats: (1) visually controlled videos of two interacting actors and (2) visually varied photographs selected from the internet involving different actors, objects, and settings. Action category was decodable across visual formats in bilateral inferior parietal, bilateral occipitotemporal, left premotor, and left middle frontal cortex. In most of these regions, the representational similarity of action categories was consistent across subjects and visual formats, a property that can contribute to a common understanding of actions among individuals. These results suggest that the identified brain regions support action category codes that are important for action recognition and action understanding.SIGNIFICANCE STATEMENT Humans tend to interpret the observed actions of others in terms of categories that are invariant to incidental features: whether a girl pushes a boy or a button and whether we see it in real-time or in a single snapshot, it is still pushing Here, we investigated the brain systems that facilitate the visual recognition of these action categories across such differences. Using fMRI, we identified several areas of parietal, occipitotemporal, and frontal cortex that exhibit action category codes that are similar across viewing of dynamic videos and still photographs. Our results provide strong evidence for the involvement of these brain regions in recognizing the way that people interact physically with objects and other people.
Collapse
|
12
|
Wurm MF, Caramazza A, Lingnau A. Action Categories in Lateral Occipitotemporal Cortex Are Organized Along Sociality and Transitivity. J Neurosci 2017; 37:562-575. [PMID: 28100739 DOI: 10.1523/jneurosci.1717-16.2017] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2016] [Revised: 11/07/2016] [Accepted: 11/11/2016] [Indexed: 05/28/2023] Open
Abstract
UNLABELLED How neural specificity for distinct conceptual knowledge categories arises is central for understanding the organization of semantic memory in the human brain. Although there is a large body of research on the neural processing of distinct object categories, the organization of action categories remains largely unknown. In particular, it is unknown whether different action categories follow a specific topographical organization on the cortical surface analogously to the category-specific organization of object knowledge. Here, we tested whether the neural representation of action knowledge is organized in terms of nonsocial versus social and object-unrelated versus object-related actions (sociality and transitivity, respectively, hereafter). We hypothesized a major distinction of sociality and transitivity along dorsal and ventral lateral occipitotemporal cortex (LOTC), respectively. Using fMRI-based multivoxel pattern analysis, we identified neural representations of action information associated with sociality and transitivity in bilateral LOTC. Representational similarity analysis revealed a dissociation between dorsal and ventral LOTC. We found that action representations in dorsal LOTC are segregated along features of sociality, whereas action representations in ventral LOTC are segregated along features of transitivity. In addition, representations of sociality and transitivity features were found more anteriorly in LOTC than representations of specific subtypes of actions, suggesting a posterior-anterior gradient from concrete to abstract action features. These findings elucidate how the neural representations of perceptually and conceptually diverse actions are organized in distinct subsystems in the LOTC. SIGNIFICANCE STATEMENT The lateral occipitotemporal cortex (LOTC) is critically involved in the recognition of objects and actions, but our knowledge about the underlying organizing principles is limited. Here, we discovered a dorsal-ventral distinction of actions in LOTC: dorsal LOTC represents actions based on sociality (how much an action is directed to another person) in proximity to person knowledge. In contrast, ventral LOTC represents actions based on transitivity (how much an action involves the interaction with inanimate objects) in proximity to tools/artifacts in ventral LOTC, suggesting a mutually dependent organization of actions and objects. In addition, we found a posterior-to-anterior organization of the LOTC for concrete and abstract representations, respectively. Our findings provide important insights about the organization of actions in LOTC.
Collapse
Affiliation(s)
- Moritz F Wurm
- Cognitive Neuropsychology Laboratory, Harvard University, Cambridge, Massachusetts 02138,
- Center for Mind/Brain Sciences, University of Trento, 38068 Rovereto (TN), Italy
| | - Alfonso Caramazza
- Cognitive Neuropsychology Laboratory, Harvard University, Cambridge, Massachusetts 02138
- Center for Mind/Brain Sciences, University of Trento, 38068 Rovereto (TN), Italy
| | - Angelika Lingnau
- Center for Mind/Brain Sciences, University of Trento, 38068 Rovereto (TN), Italy
- Department of Psychology, Royal Holloway University of London, TW20 0EX Egham, Surrey, United Kingdom, and
- Department of Cognitive Sciences, University of Trento, 38068 Rovereto (TN), Italy
| |
Collapse
|
13
|
Buchanan JJ. The Coordination Dynamics of Observational Learning: Relative Motion Direction and Relative Phase as Informational Content Linking Action-Perception to Action-Production. ADVANCES IN EXPERIMENTAL MEDICINE AND BIOLOGY 2016; 957:209-228. [DOI: 10.1007/978-3-319-47313-0_12] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
|
14
|
Gallivan JP, Culham JC. Neural coding within human brain areas involved in actions. Curr Opin Neurobiol 2015; 33:141-9. [DOI: 10.1016/j.conb.2015.03.012] [Citation(s) in RCA: 165] [Impact Index Per Article: 18.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2014] [Revised: 03/12/2015] [Accepted: 03/19/2015] [Indexed: 12/16/2022]
|
15
|
Lingnau A, Downing PE. The lateral occipitotemporal cortex in action. Trends Cogn Sci 2015; 19:268-77. [PMID: 25843544 DOI: 10.1016/j.tics.2015.03.006] [Citation(s) in RCA: 181] [Impact Index Per Article: 20.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2015] [Revised: 03/05/2015] [Accepted: 03/09/2015] [Indexed: 12/31/2022]
Abstract
Understanding and responding to other people's actions is fundamental for social interactions. Whereas many studies emphasize the importance of parietal and frontal regions for these abilities, several lines of recent research show that the human lateral occipitotemporal cortex (LOTC) represents varied aspects of action, ranging from perception of tools and bodies and the way they typically move, to understanding the meaning of actions, to performing overt actions. Here, we highlight common themes across these lines of work, which have informed theories related to high-level vision, concepts, social cognition, and apraxia. We propose that patterns of activity in LOTC form representational spaces, the dimensions of which capture perceptual, semantic, and motor knowledge of how actions change the state of the world.
Collapse
Affiliation(s)
- Angelika Lingnau
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Rovereto, 38068, Italy; Department of Psychology and Cognitive Science, University of Trento, Rovereto, 38068, Italy
| | - Paul E Downing
- Wales Institute of Cognitive Neuroscience, School of Psychology, Bangor University, Bangor, LL57 2AS, UK.
| |
Collapse
|
16
|
Perini F, Caramazza A, Peelen MV. Left occipitotemporal cortex contributes to the discrimination of tool-associated hand actions: fMRI and TMS evidence. Front Hum Neurosci 2014; 8:591. [PMID: 25140142 PMCID: PMC4122187 DOI: 10.3389/fnhum.2014.00591] [Citation(s) in RCA: 27] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2014] [Accepted: 07/16/2014] [Indexed: 11/30/2022] Open
Abstract
Functional neuroimaging studies have implicated the left lateral occipitotemporal cortex (LOTC) in both tool and hand perception but the functional role of this region is not fully known. Here, by using a task manipulation, we tested whether tool-/hand-selective LOTC contributes to the discrimination of tool-associated hand actions. Participants viewed briefly presented pictures of kitchen and garage tools while they performed one of two tasks: in the action task, they judged whether the tool is associated with a hand rotation action (e.g., screwdriver) or a hand squeeze action (e.g., garlic press), while in the location task they judged whether the tool is typically found in the kitchen (e.g., garlic press) or in the garage (e.g., screwdriver). Both tasks were performed on the same stimulus set and were matched for difficulty. Contrasting fMRI responses between these tasks showed stronger activity during the action task than the location task in both tool- and hand-selective LOTC regions, which closely overlapped. No differences were found in nearby object- and motion-selective control regions. Importantly, these findings were confirmed by a TMS study, which showed that effective TMS over the tool-/hand-selective LOTC region significantly slowed responses for tool action discriminations relative to tool location discriminations, with no such difference during sham TMS. We conclude that left LOTC contributes to the discrimination of tool-associated hand actions.
Collapse
Affiliation(s)
- Francesca Perini
- Center for Mind/Brain Sciences, CIMeC, University of Trento Rovereto, Trento, Italy
| | - Alfonso Caramazza
- Center for Mind/Brain Sciences, CIMeC, University of Trento Rovereto, Trento, Italy ; Department of Psychology, Harvard University Cambridge, MA, USA
| | - Marius V Peelen
- Center for Mind/Brain Sciences, CIMeC, University of Trento Rovereto, Trento, Italy
| |
Collapse
|