1
|
Truman A, Kutas M. Flexible Conceptual Representations. Cogn Sci 2024; 48:e13475. [PMID: 38923016 DOI: 10.1111/cogs.13475] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Revised: 05/14/2024] [Accepted: 06/05/2024] [Indexed: 06/28/2024]
Abstract
A view that has been gaining prevalence over the past decade is that the human conceptual system is malleable, dynamic, context-dependent, and task-dependent, that is, flexible. Within the flexible conceptual representation framework, conceptual representations are constructed ad hoc, forming a different, idiosyncratic instantiation upon each occurrence. In this review, we scrutinize the neurocognitive literature to better understand the nature of this flexibility. First, we identify some key characteristics of these representations. Next, we consider how these flexible representations are constructed by addressing some of the open questions in this framework: We review the age-old question of how to reconcile flexibility with the apparent need for shareable stable definitions to anchor meaning and come to mutual understanding, as well as some newer questions we find critical, namely, the nature of relations among flexible representations, the role of feature saliency in activation, and the viability of all-or-none feature activations. We suggest replacing the debate about the existence of a definitional stable core that is obligatorily activated with a question of the degree and probability of activation of the information constituting a conceptual representation. We rely on published works to suggest that (1) prior featural salience matters, (2) feature activation may be graded, and (3) Bayesian updating of prior information according to current demands offers a viable account of how flexible representations are constructed. This proposal provides a theoretical mechanism for incorporating a changing momentary context into a constructed representation, while still preserving some of the concept's constituent meaning.
Collapse
Affiliation(s)
- Alyssa Truman
- Department of Cognitive Science, University of California, San Diego
| | - Marta Kutas
- Department of Cognitive Science, University of California, San Diego
| |
Collapse
|
2
|
Anand T, Patterson K, Rowe JB, Cope TE. Drawing from name in semantic dementia reveals graded object knowledge representations in anterior temporal lobe. Mem Cognit 2024:10.3758/s13421-024-01578-9. [PMID: 38777996 DOI: 10.3758/s13421-024-01578-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/22/2024] [Indexed: 05/25/2024]
Abstract
Semantic dementia (SD) is characterized by progressive impairment in conceptual knowledge due to anterior temporal lobe (ATL) neurodegeneration. Extended neuropsychological assessments can quantitatively demonstrate the semantic impairment, but this graded loss of knowledge can also be readily observed in the qualitative observation of patients' recall of single concepts. Here, we present the results of a simple task of object drawing-from-name, by patients with SD (N = 19), who have isolated atrophy of the ATL bilaterally. Both cross-sectionally and longitudinally, patient drawings demonstrated a pattern of degradation in which rare and distinctive features (such as the hump on a camel) were lost earliest in disease course, and there was an increase in the intrusion of prototypical features (such as the typical small ears of most mammals on an elephant) with more advanced disease. Crucially, patient drawings showed a continuum of conceptual knowledge loss rather than a binary 'present' or 'absent' state. Overall, we demonstrate that qualitative evaluation of line drawings of animals and objects provides fascinating insights into the transmodal semantic deficit in SD. Our results are consistent with a distributed-plus-hub model of semantic memory. The graded nature of the deficit in semantic performance observed in our subset of longitudinally observed patients suggests that the temporal lobe binds feature-based semantic attributes in its central convergence zone.
Collapse
Affiliation(s)
- Tanmay Anand
- Cambridge University Hospitals NHS Trust, Cambridge, CB2 0QQ, UK
| | - Karalyn Patterson
- Medical Research Council Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, CB2 7EF, UK
- Department of Clinical Neurosciences, University of Cambridge, Cambridge, CB2 0SZ, UK
| | - James B Rowe
- Cambridge University Hospitals NHS Trust, Cambridge, CB2 0QQ, UK
- Medical Research Council Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, CB2 7EF, UK
- Department of Clinical Neurosciences, University of Cambridge, Cambridge, CB2 0SZ, UK
| | - Thomas E Cope
- Cambridge University Hospitals NHS Trust, Cambridge, CB2 0QQ, UK.
- Medical Research Council Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, CB2 7EF, UK.
- Department of Clinical Neurosciences, University of Cambridge, Cambridge, CB2 0SZ, UK.
| |
Collapse
|
3
|
Chapman AF, Störmer VS. Representational structures as a unifying framework for attention. Trends Cogn Sci 2024; 28:416-427. [PMID: 38280837 PMCID: PMC11290436 DOI: 10.1016/j.tics.2024.01.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Revised: 01/04/2024] [Accepted: 01/05/2024] [Indexed: 01/29/2024]
Abstract
Our visual system consciously processes only a subset of the incoming information. Selective attention allows us to prioritize relevant inputs, and can be allocated to features, locations, and objects. Recent advances in feature-based attention suggest that several selection principles are shared across these domains and that many differences between the effects of attention on perceptual processing can be explained by differences in the underlying representational structures. Moving forward, it can thus be useful to assess how attention changes the structure of the representational spaces over which it operates, which include the spatial organization, feature maps, and object-based coding in visual cortex. This will ultimately add to our understanding of how attention changes the flow of visual information processing more broadly.
Collapse
Affiliation(s)
- Angus F Chapman
- Department of Psychological and Brain Sciences, Boston University, Boston, MA, USA.
| | - Viola S Störmer
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA.
| |
Collapse
|
4
|
Vinton LC, Preston C, de la Rosa S, Mackie G, Tipper SP, Barraclough NE. Four fundamental dimensions underlie the perception of human actions. Atten Percept Psychophys 2024; 86:536-558. [PMID: 37188862 PMCID: PMC10185378 DOI: 10.3758/s13414-023-02709-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/30/2023] [Indexed: 05/17/2023]
Abstract
We evaluate the actions of other individuals based upon a variety of movements that reveal critical information to guide decision making and behavioural responses. These signals convey a range of information about the actor, including their goals, intentions and internal mental states. Although progress has been made to identify cortical regions involved in action processing, the organising principles underlying our representation of actions still remains unclear. In this paper we investigated the conceptual space that underlies action perception by assessing which qualities are fundamental to the perception of human actions. We recorded 240 different actions using motion-capture and used these data to animate a volumetric avatar that performed the different actions. 230 participants then viewed these actions and rated the extent to which each action demonstrated 23 different action characteristics (e.g., avoiding-approaching, pulling-pushing, weak-powerful). We analysed these data using Exploratory Factor Analysis to examine the latent factors underlying visual action perception. The best fitting model was a four-dimensional model with oblique rotation. We named the factors: friendly-unfriendly, formidable-feeble, planned-unplanned, and abduction-adduction. The first two factors of friendliness and formidableness explained approximately 22% of the variance each, compared to planned and abduction, which explained approximately 7-8% of the variance each; as such we interpret this representation of action space as having 2 + 2 dimensions. A closer examination of the first two factors suggests a similarity to the principal factors underlying our evaluation of facial traits and emotions, whilst the last two factors of planning and abduction appear unique to actions.
Collapse
Affiliation(s)
- Laura C Vinton
- Department of Psychology, University of York, Heslington, York, YO10 5DD, UK
| | - Catherine Preston
- Department of Psychology, University of York, Heslington, York, YO10 5DD, UK
| | - Stephan de la Rosa
- Department of Social Sciences, IU University of Applied Sciences, Juri-Gagarin-Ring 152, 99084, Erfurt, Germany
| | - Gabriel Mackie
- Department of Psychology, University of York, Heslington, York, YO10 5DD, UK
| | - Steven P Tipper
- Department of Psychology, University of York, Heslington, York, YO10 5DD, UK
| | - Nick E Barraclough
- Department of Psychology, University of York, Heslington, York, YO10 5DD, UK.
| |
Collapse
|
5
|
McMahon E, Bonner MF, Isik L. Hierarchical organization of social action features along the lateral visual pathway. Curr Biol 2023; 33:5035-5047.e8. [PMID: 37918399 PMCID: PMC10841461 DOI: 10.1016/j.cub.2023.10.015] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Revised: 09/01/2023] [Accepted: 10/10/2023] [Indexed: 11/04/2023]
Abstract
Recent theoretical work has argued that in addition to the classical ventral (what) and dorsal (where/how) visual streams, there is a third visual stream on the lateral surface of the brain specialized for processing social information. Like visual representations in the ventral and dorsal streams, representations in the lateral stream are thought to be hierarchically organized. However, no prior studies have comprehensively investigated the organization of naturalistic, social visual content in the lateral stream. To address this question, we curated a naturalistic stimulus set of 250 3-s videos of two people engaged in everyday actions. Each clip was richly annotated for its low-level visual features, mid-level scene and object properties, visual social primitives (including the distance between people and the extent to which they were facing), and high-level information about social interactions and affective content. Using a condition-rich fMRI experiment and a within-subject encoding model approach, we found that low-level visual features are represented in early visual cortex (EVC) and middle temporal (MT) area, mid-level visual social features in extrastriate body area (EBA) and lateral occipital complex (LOC), and high-level social interaction information along the superior temporal sulcus (STS). Communicative interactions, in particular, explained unique variance in regions of the STS after accounting for variance explained by all other labeled features. Taken together, these results provide support for representation of increasingly abstract social visual content-consistent with hierarchical organization-along the lateral visual stream and suggest that recognizing communicative actions may be a key computational goal of the lateral visual pathway.
Collapse
Affiliation(s)
- Emalie McMahon
- Department of Cognitive Science, Zanvyl Krieger School of Arts & Sciences, Johns Hopkins University, 237 Krieger Hall, 3400 N. Charles Street, Baltimore, MD 21218, USA.
| | - Michael F Bonner
- Department of Cognitive Science, Zanvyl Krieger School of Arts & Sciences, Johns Hopkins University, 237 Krieger Hall, 3400 N. Charles Street, Baltimore, MD 21218, USA
| | - Leyla Isik
- Department of Cognitive Science, Zanvyl Krieger School of Arts & Sciences, Johns Hopkins University, 237 Krieger Hall, 3400 N. Charles Street, Baltimore, MD 21218, USA; Department of Biomedical Engineering, Whiting School of Engineering, Johns Hopkins University, Suite 400 West, Wyman Park Building, 3400 N. Charles Street, Baltimore, MD 21218, USA
| |
Collapse
|
6
|
Jiahui G, Feilong M, Visconti di Oleggio Castello M, Nastase SA, Haxby JV, Gobbini MI. Modeling naturalistic face processing in humans with deep convolutional neural networks. Proc Natl Acad Sci U S A 2023; 120:e2304085120. [PMID: 37847731 PMCID: PMC10614847 DOI: 10.1073/pnas.2304085120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Accepted: 09/11/2023] [Indexed: 10/19/2023] Open
Abstract
Deep convolutional neural networks (DCNNs) trained for face identification can rival and even exceed human-level performance. The ways in which the internal face representations in DCNNs relate to human cognitive representations and brain activity are not well understood. Nearly all previous studies focused on static face image processing with rapid display times and ignored the processing of naturalistic, dynamic information. To address this gap, we developed the largest naturalistic dynamic face stimulus set in human neuroimaging research (700+ naturalistic video clips of unfamiliar faces). We used this naturalistic dataset to compare representational geometries estimated from DCNNs, behavioral responses, and brain responses. We found that DCNN representational geometries were consistent across architectures, cognitive representational geometries were consistent across raters in a behavioral arrangement task, and neural representational geometries in face areas were consistent across brains. Representational geometries in late, fully connected DCNN layers, which are optimized for individuation, were much more weakly correlated with cognitive and neural geometries than were geometries in late-intermediate layers. The late-intermediate face-DCNN layers successfully matched cognitive representational geometries, as measured with a behavioral arrangement task that primarily reflected categorical attributes, and correlated with neural representational geometries in known face-selective topographies. Our study suggests that current DCNNs successfully capture neural cognitive processes for categorical attributes of faces but less accurately capture individuation and dynamic features.
Collapse
Affiliation(s)
- Guo Jiahui
- Center for Cognitive Neuroscience, Dartmouth College, Hanover, NH03755
| | - Ma Feilong
- Center for Cognitive Neuroscience, Dartmouth College, Hanover, NH03755
| | | | - Samuel A. Nastase
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ08544
| | - James V. Haxby
- Center for Cognitive Neuroscience, Dartmouth College, Hanover, NH03755
| | - M. Ida Gobbini
- Department of Medical and Surgical Sciences, University of Bologna, Bologna40138, Italy
- Istituti di Ricovero e Cura a Carattere Scientifico, Istituto delle Scienze Neurologiche di Bologna, Bologna40139, Italia
| |
Collapse
|
7
|
Deniz F, Tseng C, Wehbe L, Dupré la Tour T, Gallant JL. Semantic Representations during Language Comprehension Are Affected by Context. J Neurosci 2023; 43:3144-3158. [PMID: 36973013 PMCID: PMC10146529 DOI: 10.1523/jneurosci.2459-21.2023] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Revised: 02/17/2023] [Accepted: 02/26/2023] [Indexed: 03/29/2023] Open
Abstract
The meaning of words in natural language depends crucially on context. However, most neuroimaging studies of word meaning use isolated words and isolated sentences with little context. Because the brain may process natural language differently from how it processes simplified stimuli, there is a pressing need to determine whether prior results on word meaning generalize to natural language. fMRI was used to record human brain activity while four subjects (two female) read words in four conditions that vary in context: narratives, isolated sentences, blocks of semantically similar words, and isolated words. We then compared the signal-to-noise ratio (SNR) of evoked brain responses, and we used a voxelwise encoding modeling approach to compare the representation of semantic information across the four conditions. We find four consistent effects of varying context. First, stimuli with more context evoke brain responses with higher SNR across bilateral visual, temporal, parietal, and prefrontal cortices compared with stimuli with little context. Second, increasing context increases the representation of semantic information across bilateral temporal, parietal, and prefrontal cortices at the group level. In individual subjects, only natural language stimuli consistently evoke widespread representation of semantic information. Third, context affects voxel semantic tuning. Finally, models estimated using stimuli with little context do not generalize well to natural language. These results show that context has large effects on the quality of neuroimaging data and on the representation of meaning in the brain. Thus, neuroimaging studies that use stimuli with little context may not generalize well to the natural regime.SIGNIFICANCE STATEMENT Context is an important part of understanding the meaning of natural language, but most neuroimaging studies of meaning use isolated words and isolated sentences with little context. Here, we examined whether the results of neuroimaging studies that use out-of-context stimuli generalize to natural language. We find that increasing context improves the quality of neuro-imaging data and changes where and how semantic information is represented in the brain. These results suggest that findings from studies using out-of-context stimuli may not generalize to natural language used in daily life.
Collapse
Affiliation(s)
- Fatma Deniz
- Helen Wills Neuroscience Institute, University of California, Berkeley, California 94720
- Institute of Software Engineering and Theoretical Computer Science, Technische Universität Berlin, Berlin 10623, Germany
| | - Christine Tseng
- Helen Wills Neuroscience Institute, University of California, Berkeley, California 94720
| | - Leila Wehbe
- Machine Learning Department, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213
| | - Tom Dupré la Tour
- Helen Wills Neuroscience Institute, University of California, Berkeley, California 94720
| | - Jack L Gallant
- Helen Wills Neuroscience Institute, University of California, Berkeley, California 94720
- Department of Psychology, University of California, Berkeley, California 94720
| |
Collapse
|
8
|
Palenciano AF, Senoussi M, Formica S, González-García C. Canonical template tracking: Measuring the activation state of specific neural representations. FRONTIERS IN NEUROIMAGING 2023; 1:974927. [PMID: 37555182 PMCID: PMC10406196 DOI: 10.3389/fnimg.2022.974927] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/21/2022] [Accepted: 12/13/2022] [Indexed: 08/10/2023]
Abstract
Multivariate analyses of neural data have become increasingly influential in cognitive neuroscience since they allow to address questions about the representational signatures of neurocognitive phenomena. Here, we describe Canonical Template Tracking: a multivariate approach that employs independent localizer tasks to assess the activation state of specific representations during the execution of cognitive paradigms. We illustrate the benefits of this methodology in characterizing the particular content and format of task-induced representations, comparing it with standard (cross-)decoding and representational similarity analyses. Then, we discuss relevant design decisions for experiments using this analysis approach, focusing on the nature of the localizer tasks from which the canonical templates are derived. We further provide a step-by-step tutorial of this method, stressing the relevant analysis choices for functional magnetic resonance imaging and magneto/electroencephalography data. Importantly, we point out the potential pitfalls linked to canonical template tracking implementation and interpretation of the results, together with recommendations to mitigate them. To conclude, we provide some examples from previous literature that highlight the potential of this analysis to address relevant theoretical questions in cognitive neuroscience.
Collapse
Affiliation(s)
- Ana F. Palenciano
- Mind, Brain, and Behavior Research Center, University of Granada, Granada, Spain
| | - Mehdi Senoussi
- CLLE Lab, CNRS UMR 5263, University of Toulouse, Toulouse, France
- Department of Experimental Psychology, Ghent University, Ghent, Belgium
| | - Silvia Formica
- Department of Psychology, Berlin School of Mind and Brain, Humboldt Universität zu Berlin, Berlin, Germany
| | | |
Collapse
|
9
|
Sargent M, LePage A, Kenett YN, Matheson HE. The Effects of Environmental Scene and Body Posture on Embodied Strategies in Creative Thinking. CREATIVITY RESEARCH JOURNAL 2023. [DOI: 10.1080/10400419.2022.2160563] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
Affiliation(s)
- Matthew Sargent
- Psychology Department, University of Northern British Columbia
| | - Alex LePage
- Psychology Department, University of Northern British Columbia
| | - Yoed N. Kenett
- The Faculty of Data and Decision Sciences, Technion - Israel Institute of Technology
| | | |
Collapse
|
10
|
Shahdloo M, Çelik E, Urgen BA, Gallant JL, Çukur T. Task-Dependent Warping of Semantic Representations during Search for Visual Action Categories. J Neurosci 2022; 42:6782-6799. [PMID: 35863889 PMCID: PMC9436022 DOI: 10.1523/jneurosci.1372-21.2022] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2021] [Revised: 06/29/2022] [Accepted: 07/06/2022] [Indexed: 11/21/2022] Open
Abstract
Object and action perception in cluttered dynamic natural scenes relies on efficient allocation of limited brain resources to prioritize the attended targets over distractors. It has been suggested that during visual search for objects, distributed semantic representation of hundreds of object categories is warped to expand the representation of targets. Yet, little is known about whether and where in the brain visual search for action categories modulates semantic representations. To address this fundamental question, we studied brain activity recorded from five subjects (one female) via functional magnetic resonance imaging while they viewed natural movies and searched for either communication or locomotion actions. We find that attention directed to action categories elicits tuning shifts that warp semantic representations broadly across neocortex and that these shifts interact with intrinsic selectivity of cortical voxels for target actions. These results suggest that attention serves to facilitate task performance during social interactions by dynamically shifting semantic selectivity toward target actions and that tuning shifts are a general feature of conceptual representations in the brain.SIGNIFICANCE STATEMENT The ability to swiftly perceive the actions and intentions of others is a crucial skill for humans that relies on efficient allocation of limited brain resources to prioritize the attended targets over distractors. However, little is known about the nature of high-level semantic representations during natural visual search for action categories. Here, we provide the first evidence showing that attention significantly warps semantic representations by inducing tuning shifts in single cortical voxels, broadly spread across occipitotemporal, parietal, prefrontal, and cingulate cortices. This dynamic attentional mechanism can facilitate action perception by efficiently allocating neural resources to accentuate the representation of task-relevant action categories.
Collapse
Affiliation(s)
- Mo Shahdloo
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford OX3 9DU, United Kingdom
- National Magnetic Resonance Research Centre, Bilkent University, 06800 Ankara, Turkey
- Departments of Electrical and Electronics Engineering and
| | - Emin Çelik
- National Magnetic Resonance Research Centre, Bilkent University, 06800 Ankara, Turkey
- Neuroscience Program, Aysel Sabuncu Brain Research Centre, Bilkent University, 06800 Ankara, Turkey
| | - Burcu A Urgen
- National Magnetic Resonance Research Centre, Bilkent University, 06800 Ankara, Turkey
- Psychology, Bilkent University, 06800 Ankara, Turkey
- Neuroscience Program, Aysel Sabuncu Brain Research Centre, Bilkent University, 06800 Ankara, Turkey
| | - Jack L Gallant
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, California 94720
| | - Tolga Çukur
- National Magnetic Resonance Research Centre, Bilkent University, 06800 Ankara, Turkey
- Departments of Electrical and Electronics Engineering and
- Neuroscience Program, Aysel Sabuncu Brain Research Centre, Bilkent University, 06800 Ankara, Turkey
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, California 94720
| |
Collapse
|
11
|
|
12
|
Meersmans K, Storms G, De Deyne S, Bruffaerts R, Dupont P, Vandenberghe R. Orienting to Different Dimensions of Word Meaning Alters the Representation of Word Meaning in Early Processing Regions. Cereb Cortex 2021; 32:3302-3317. [PMID: 34963135 PMCID: PMC9340395 DOI: 10.1093/cercor/bhab416] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2021] [Revised: 10/19/2021] [Accepted: 10/20/2021] [Indexed: 11/14/2022] Open
Abstract
Conscious processing of word meaning can be guided by attention. In this event-related functional magnetic resonance imaging study in 22 healthy young volunteers, we examined in which regions orienting attention to two fundamental and generic dimensions of word meaning, concreteness versus valence, alters the semantic representations coded in activity patterns. The stimuli consisted of 120 nouns in written or spoken modality which varied factorially along the concreteness and valence axis. Participants performed a forced-choice judgement of either concreteness or valence. Rostral and subgenual anterior cingulate were strongly activated during valence judgement, and precuneus and the dorsal attention network during concreteness judgement. Task and stimulus type interacted in right posterior fusiform gyrus, left lingual gyrus, precuneus, and insula. In the right posterior fusiform gyrus and the left lingual gyrus, the correlation between the pairwise similarity in activity patterns evoked by words and the pairwise distance in valence and concreteness was modulated by the direction of attention, word valence or concreteness. The data indicate that orienting attention to basic dimensions of word meaning exerts effects on the representation of word meaning in more peripheral nodes, such as the ventral occipital cortex, rather than the core perisylvian language regions.
Collapse
Affiliation(s)
- Karen Meersmans
- Laboratory for Cognitive Neurology, Department of Neurosciences, Leuven Brain Institute, 3000 Leuven, Belgium
| | - Gerrit Storms
- Laboratory of Experimental Psychology, 3000 Leuven, Belgium
| | - Simon De Deyne
- Computational Cognitive Science Lab, Melbourne School of Psychological Sciences, University of Melbourne, 3010 Melbourne, Australia
| | - Rose Bruffaerts
- Laboratory for Cognitive Neurology, Department of Neurosciences, Leuven Brain Institute, 3000 Leuven, Belgium
| | - Patrick Dupont
- Laboratory for Cognitive Neurology, Department of Neurosciences, Leuven Brain Institute, 3000 Leuven, Belgium
| | - Rik Vandenberghe
- Laboratory for Cognitive Neurology, Department of Neurosciences, Leuven Brain Institute, 3000 Leuven, Belgium.,Neurology Department, University Hospitals Leuven, 3000 Leuven, Belgium
| |
Collapse
|
13
|
Wurm MF, Caramazza A. Two 'what' pathways for action and object recognition. Trends Cogn Sci 2021; 26:103-116. [PMID: 34702661 DOI: 10.1016/j.tics.2021.10.003] [Citation(s) in RCA: 38] [Impact Index Per Article: 12.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2021] [Revised: 09/03/2021] [Accepted: 10/01/2021] [Indexed: 10/20/2022]
Abstract
The ventral visual stream is conceived as a pathway for object recognition. However, we also recognize the actions an object can be involved in. Here, we show that action recognition critically depends on a pathway in lateral occipitotemporal cortex, partially overlapping and topographically aligned with object representations that are precursors for action recognition. By contrast, object features that are more relevant for object recognition, such as color and texture, are typically found in ventral occipitotemporal cortex. We argue that occipitotemporal cortex contains similarly organized lateral and ventral 'what' pathways for action and object recognition, respectively. This account explains a number of observed phenomena, such as the duplication of object domains and the specific representational profiles in lateral and ventral cortex.
Collapse
Affiliation(s)
- Moritz F Wurm
- Center for Mind/Brain Sciences - CIMeC, University of Trento, Corso Bettini 31, 38068 Rovereto, Italy.
| | - Alfonso Caramazza
- Center for Mind/Brain Sciences - CIMeC, University of Trento, Corso Bettini 31, 38068 Rovereto, Italy; Department of Psychology, Harvard University, 33 Kirkland St, Cambridge, MA 02138, USA
| |
Collapse
|
14
|
Uithol S, Bryant KL, Toni I, Mars RB. The Anticipatory and Task-Driven Nature of Visual Perception. Cereb Cortex 2021; 31:5354-5362. [PMID: 34491289 PMCID: PMC8567999 DOI: 10.1093/cercor/bhab163] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2020] [Revised: 05/20/2021] [Accepted: 05/20/2021] [Indexed: 11/29/2022] Open
Abstract
Humans have a remarkable capacity to arrange and rearrange perceptual input according to different categorizations. This begs the question whether the categorization is exclusively a higher visual or amodal process, or whether categorization processes influence early visual areas as well. To investigate this we scanned healthy participants in a magnetic resonance imaging scanner during a conceptual decision task in which participants had to answer questions about upcoming images of animals. Early visual cortices (V1 and V2) contained information about the current visual input, about the granularity of the forthcoming categorical decision, as well as perceptual expectations about the upcoming visual stimulus. The middle temporal gyrus, the anterior temporal lobe, and the inferior frontal gyrus were also involved in the categorization process, constituting an attention and control network that modulates perceptual processing. These findings provide further evidence that early visual processes are driven by conceptual expectations and task demands.
Collapse
Affiliation(s)
- Sebo Uithol
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, 6525 HR, Nijmegen, the Netherlands.,Cognitive Psychology Unit, Institute of Psychology, Leiden University, 2333 AK Leiden, the Netherlands
| | - Katherine L Bryant
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, 6525 HR, Nijmegen, the Netherlands.,Wellcome Centre for Integrative Neuroimaging, Centre for Functional MRI of the Brain (FMRIB), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford OX3 9DU, UK
| | - Ivan Toni
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, 6525 HR, Nijmegen, the Netherlands
| | - Rogier B Mars
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, 6525 HR, Nijmegen, the Netherlands.,Wellcome Centre for Integrative Neuroimaging, Centre for Functional MRI of the Brain (FMRIB), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford OX3 9DU, UK
| |
Collapse
|
15
|
Chen HT, Manning JR, van der Meer MAA. Between-subject prediction reveals a shared representational geometry in the rodent hippocampus. Curr Biol 2021; 31:4293-4304.e5. [PMID: 34428470 DOI: 10.1016/j.cub.2021.07.061] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2020] [Revised: 05/27/2021] [Accepted: 07/26/2021] [Indexed: 11/24/2022]
Abstract
The rodent hippocampus constructs statistically independent representations across environments ("global remapping") and assigns individual neuron firing fields to locations within an environment in an apparently random fashion, processes thought to contribute to the role of the hippocampus in episodic memory. This random mapping implies that it should be challenging to predict hippocampal encoding of a given experience in one subject based on the encoding of that same experience in another subject. Contrary to this prediction, we find that by constructing a common representational space across rats in which neural activity is aligned using geometric operations (rotation, reflection, and translation; "hyperalignment"), we can predict data of "right" trials (R) on a T-maze in a target rat based on (1) the "left" trials (L) of the target rat and (2) the relationship between L and R trials from a different source rat. These cross-subject predictions relied on ensemble activity patterns, including both firing rate and field location, and outperformed a number of control mappings, such as those based on permuted data that broke the relationship between L and R activity for individual neurons and those based solely on within-subject prediction. This work constitutes proof of principle for successful cross-subject prediction of ensemble activity patterns in the hippocampus and provides new insights in understanding how different experiences are structured, enabling further work identifying what aspects of experience encoding are shared versus unique to an individual.
Collapse
Affiliation(s)
- Hung-Tu Chen
- Department of Psychological & Brain Sciences, Dartmouth College, Hanover, NH 03755, USA
| | - Jeremy R Manning
- Department of Psychological & Brain Sciences, Dartmouth College, Hanover, NH 03755, USA
| | | |
Collapse
|
16
|
Nastase SA, Liu YF, Hillman H, Zadbood A, Hasenfratz L, Keshavarzian N, Chen J, Honey CJ, Yeshurun Y, Regev M, Nguyen M, Chang CHC, Baldassano C, Lositsky O, Simony E, Chow MA, Leong YC, Brooks PP, Micciche E, Choe G, Goldstein A, Vanderwal T, Halchenko YO, Norman KA, Hasson U. The "Narratives" fMRI dataset for evaluating models of naturalistic language comprehension. Sci Data 2021; 8:250. [PMID: 34584100 PMCID: PMC8479122 DOI: 10.1038/s41597-021-01033-3] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2021] [Accepted: 08/18/2021] [Indexed: 02/08/2023] Open
Abstract
The "Narratives" collection aggregates a variety of functional MRI datasets collected while human subjects listened to naturalistic spoken stories. The current release includes 345 subjects, 891 functional scans, and 27 diverse stories of varying duration totaling ~4.6 hours of unique stimuli (~43,000 words). This data collection is well-suited for naturalistic neuroimaging analysis, and is intended to serve as a benchmark for models of language and narrative comprehension. We provide standardized MRI data accompanied by rich metadata, preprocessed versions of the data ready for immediate use, and the spoken story stimuli with time-stamped phoneme- and word-level transcripts. All code and data are publicly available with full provenance in keeping with current best practices in transparent and reproducible neuroimaging.
Collapse
Affiliation(s)
- Samuel A Nastase
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, USA.
| | - Yun-Fei Liu
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD, USA
| | - Hanna Hillman
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, USA
| | - Asieh Zadbood
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, USA
| | - Liat Hasenfratz
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, USA
| | - Neggin Keshavarzian
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, USA
| | - Janice Chen
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD, USA
| | - Christopher J Honey
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD, USA
| | - Yaara Yeshurun
- School of Psychological Sciences, Tel Aviv University, Tel Aviv, Israel
| | - Mor Regev
- Montreal Neurological Institute, McGill University, Montreal, QC, Canada
| | - Mai Nguyen
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, USA
| | - Claire H C Chang
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, USA
| | | | - Olga Lositsky
- Department of Cognitive, Linguistic and Psychological Sciences, Brown University, Providence, RI, USA
| | - Erez Simony
- Faculty of Electrical Engineering, Holon Institute of Technology, Holon, Israel
- Department of Neurobiology, Weizmann Institute of Science, Rehovot, Israel
| | | | - Yuan Chang Leong
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA
| | - Paula P Brooks
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, USA
| | - Emily Micciche
- Peabody College, Vanderbilt University, Nashville, TN, USA
| | - Gina Choe
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, USA
| | - Ariel Goldstein
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, USA
| | - Tamara Vanderwal
- Department of Psychiatry, University of British Columbia, and BC Children's Hospital Research Institute, Vancouver, BC, Canada
| | - Yaroslav O Halchenko
- Department of Psychological and Brain Sciences and Department of Computer Science, Dartmouth College, Hanover, NH, USA
| | - Kenneth A Norman
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, USA
| | - Uri Hasson
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, USA
| |
Collapse
|
17
|
Marrazzo G, Vaessen MJ, de Gelder B. Decoding the difference between explicit and implicit body expression representation in high level visual, prefrontal and inferior parietal cortex. Neuroimage 2021; 243:118545. [PMID: 34478822 DOI: 10.1016/j.neuroimage.2021.118545] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2020] [Revised: 08/30/2021] [Accepted: 08/31/2021] [Indexed: 11/28/2022] Open
Abstract
Recent studies provide an increasing understanding of how visual objects categories like faces or bodies are represented in the brain and also raised the question whether a category based or more dynamic network inspired models are more powerful. Two important and so far sidestepped issues in this debate are, first, how major category attributes like the emotional expression directly influence category representation and second, whether category and attribute representation are sensitive to task demands. This study investigated the impact of a crucial category attribute like emotional expression on category area activity and whether this varies with the participants' task. Using (fMRI) we measured BOLD responses while participants viewed whole body expressions and performed either an explicit (emotion) or an implicit (shape) recognition task. Our results based on multivariate methods show that the type of task is the strongest determinant of brain activity and can be decoded in EBA, VLPFC and IPL. Brain activity was higher for the explicit task condition in VLPFC and was not emotion specific. This pattern suggests that during explicit recognition of the body expression, body category representation may be strengthened, and emotion and action related activity suppressed. Taken together these results stress the importance of the task and of the role of category attributes for understanding the functional organization of high level visual cortex.
Collapse
Affiliation(s)
- Giuseppe Marrazzo
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Limburg 6200 MD, Maastricht, the Netherlands
| | - Maarten J Vaessen
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Limburg 6200 MD, Maastricht, the Netherlands
| | - Beatrice de Gelder
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Limburg 6200 MD, Maastricht, the Netherlands; Department of Computer Science, University College London, London WC1E 6BT, United Kingdom.
| |
Collapse
|
18
|
Ritchie JB, Zeman AA, Bosmans J, Sun S, Verhaegen K, Op de Beeck HP. Untangling the Animacy Organization of Occipitotemporal Cortex. J Neurosci 2021; 41:7103-7119. [PMID: 34230104 PMCID: PMC8372013 DOI: 10.1523/jneurosci.2628-20.2021] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2020] [Revised: 04/20/2021] [Accepted: 05/20/2021] [Indexed: 11/21/2022] Open
Abstract
Some of the most impressive functional specializations in the human brain are found in the occipitotemporal cortex (OTC), where several areas exhibit selectivity for a small number of visual categories, such as faces and bodies, and spatially cluster based on stimulus animacy. Previous studies suggest this animacy organization reflects the representation of an intuitive taxonomic hierarchy, distinct from the presence of face- and body-selective areas in OTC. Using human functional magnetic resonance imaging, we investigated the independent contribution of these two factors-the face-body division and taxonomic hierarchy-in accounting for the animacy organization of OTC and whether they might also be reflected in the architecture of several deep neural networks that have not been explicitly trained to differentiate taxonomic relations. We found that graded visual selectivity, based on animal resemblance to human faces and bodies, masquerades as an apparent animacy continuum, which suggests that taxonomy is not a separate factor underlying the organization of the ventral visual pathway.SIGNIFICANCE STATEMENT Portions of the visual cortex are specialized to determine whether types of objects are animate in the sense of being capable of self-movement. Two factors have been proposed as accounting for this animacy organization: representations of faces and bodies and an intuitive taxonomic continuum of humans and animals. We performed an experiment to assess the independent contribution of both of these factors. We found that graded visual representations, based on animal resemblance to human faces and bodies, masquerade as an apparent animacy continuum, suggesting that taxonomy is not a separate factor underlying the organization of areas in the visual cortex.
Collapse
Affiliation(s)
- J Brendan Ritchie
- Laboratory of Biological Psychology, Department of Brain and Cognition, Leuven Brain Institute, Katholieke Universiteit Leuven, 3000 Leuven, Belgium
| | - Astrid A Zeman
- Laboratory of Biological Psychology, Department of Brain and Cognition, Leuven Brain Institute, Katholieke Universiteit Leuven, 3000 Leuven, Belgium
| | - Joyce Bosmans
- Faculty of Medicine and Health Sciences, University of Antwerp, 2000 Antwerp, Belgium
| | - Shuo Sun
- Laboratory of Biological Psychology, Department of Brain and Cognition, Leuven Brain Institute, Katholieke Universiteit Leuven, 3000 Leuven, Belgium
| | - Kirsten Verhaegen
- Laboratory of Biological Psychology, Department of Brain and Cognition, Leuven Brain Institute, Katholieke Universiteit Leuven, 3000 Leuven, Belgium
| | - Hans P Op de Beeck
- Laboratory of Biological Psychology, Department of Brain and Cognition, Leuven Brain Institute, Katholieke Universiteit Leuven, 3000 Leuven, Belgium
| |
Collapse
|
19
|
Shashidhara S, Erez Y. Reward motivation increases univariate activity but has limited effect on coding of task-relevant information across the frontoparietal cortex. Neuropsychologia 2021; 160:107981. [PMID: 34332993 PMCID: PMC8434417 DOI: 10.1016/j.neuropsychologia.2021.107981] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2021] [Revised: 07/24/2021] [Accepted: 07/27/2021] [Indexed: 11/05/2022]
Abstract
Selection and integration of information based on current goals is fundamental for goal-directed behavior. Reward motivation has been shown to improve behavioral performance, yet the neural mechanisms that link motivation and control processes, and in particular its effect on context-dependent information processing, remain unclear. We used functional magnetic resonance imaging (fMRI) in 24 human volunteers (13 females) to test whether reward motivation enhances the coding of task-relevant information across the frontoparietal cortex, as would be predicted based on previous experimental evidence and theoretical accounts. In a cued target detection task, participants detected whether an object from a cued visual category was present in a subsequent display. The combination of the cue and the object visual category determined the behavioral status of the objects. To manipulate reward motivation, half of all trials offered the possibility of a monetary reward. We observed an increase with reward in overall univariate activity across the frontoparietal control network when the cue and subsequent object were presented. Multivariate pattern analysis (MVPA) showed that behavioral status information for the objects was conveyed across the network. However, in contrast to our prediction, reward did not increase the discrimination between behavioral status conditions in the stimulus epoch of a trial when object information was processed depending on a current context. In the high-level general-object visual region, the lateral occipital complex, the representation of behavioral status was driven by visual differences and was not modulated by reward. Our study provides useful evidence for the limited effects of reward motivation on task-related neural representations and highlights the necessity to unravel the diverse forms and extent of these effects. Reward motivation leads to increased activity in the frontoparietal control network. Task-relevant information is coded across the control network. Reward does not increase coding of behavioral relevance of task information. Visual category coding in the general-object region does not increase with reward.
Collapse
Affiliation(s)
- Sneha Shashidhara
- MRC Cognition and Brain Sciences Unit, University of Cambridge, 15 Chaucer Road, Cambridge, CB2 7EF, UK.
| | - Yaara Erez
- MRC Cognition and Brain Sciences Unit, University of Cambridge, 15 Chaucer Road, Cambridge, CB2 7EF, UK.
| |
Collapse
|
20
|
Kaiser D, Häberle G, Cichy RM. Coherent natural scene structure facilitates the extraction of task-relevant object information in visual cortex. Neuroimage 2021; 240:118365. [PMID: 34233220 PMCID: PMC8456750 DOI: 10.1016/j.neuroimage.2021.118365] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2020] [Revised: 04/22/2021] [Accepted: 07/03/2021] [Indexed: 11/24/2022] Open
Abstract
Looking for objects within complex natural environments is a task everybody performs multiple times each day. In this study, we explore how the brain uses the typical composition of real-world environments to efficiently solve this task. We recorded fMRI activity while participants performed two different categorization tasks on natural scenes. In the object task, they indicated whether the scene contained a person or a car, while in the scene task, they indicated whether the scene depicted an urban or a rural environment. Critically, each scene was presented in an "intact" way, preserving its coherent structure, or in a "jumbled" way, with information swapped across quadrants. In both tasks, participants' categorization was more accurate and faster for intact scenes. These behavioral benefits were accompanied by stronger responses to intact than to jumbled scenes across high-level visual cortex. To track the amount of object information in visual cortex, we correlated multi-voxel response patterns during the two categorization tasks with response patterns evoked by people and cars in isolation. We found that object information in object- and body-selective cortex was enhanced when the object was embedded in an intact, rather than a jumbled scene. However, this enhancement was only found in the object task: When participants instead categorized the scenes, object information did not differ between intact and jumbled scenes. Together, these results indicate that coherent scene structure facilitates the extraction of object information in a task-dependent way, suggesting that interactions between the object and scene processing pathways adaptively support behavioral goals.
Collapse
Affiliation(s)
- Daniel Kaiser
- Department of Psychology, University of York, York, UK.
| | - Greta Häberle
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany; Charité - Universitätsmedizin Berlin, Einstein Center for Neurosciences Berlin, Berlin, Germany; Humboldt-Universität zu Berlin, Faculty of Philosophy, Berlin School of Mind and Brain, Berlin, Germany
| | - Radoslaw M Cichy
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany; Charité - Universitätsmedizin Berlin, Einstein Center for Neurosciences Berlin, Berlin, Germany; Humboldt-Universität zu Berlin, Faculty of Philosophy, Berlin School of Mind and Brain, Berlin, Germany; Bernstein Center for Computational Neuroscience Berlin, Berlin, Germany
| |
Collapse
|
21
|
Stehr DA, Zhou X, Tisby M, Hwu PT, Pyles JA, Grossman ED. Top-Down Attention Guidance Shapes Action Encoding in the pSTS. Cereb Cortex 2021; 31:3522-3535. [PMID: 33629729 DOI: 10.1093/cercor/bhab029] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2020] [Revised: 01/07/2021] [Accepted: 01/25/2021] [Indexed: 11/12/2022] Open
Abstract
The posterior superior temporal sulcus (pSTS) is a brain region characterized by perceptual representations of human body actions that promote the understanding of observed behavior. Increasingly, action observation is recognized as being strongly shaped by the expectations of the observer (Kilner 2011; Koster-Hale and Saxe 2013; Patel et al. 2019). Therefore, to characterize top-down influences on action observation, we evaluated the statistical structure of multivariate activation patterns from the action observation network (AON) while observers attended to the different dimensions of action vignettes (the action kinematics, goal, or identity of avatars jumping or crouching). Decoding accuracy varied as a function of attention instruction in the right pSTS and left inferior frontal cortex (IFC), with the right pSTS classifying actions most accurately when observers attended to the action kinematics and the left IFC classifying most accurately when observed attended to the actor's goal. Functional connectivity also increased between the right pSTS and right IFC when observers attended to the actions portrayed in the vignettes. Our findings are evidence that the attentive state of the viewer modulates sensory representations in the pSTS, consistent with proposals that the pSTS occupies an interstitial zone mediating top-down context and bottom-up perceptual cues during action observation.
Collapse
Affiliation(s)
- Daniel A Stehr
- Department of Cognitive Sciences, University of California, Irvine, Irvine, CA 92697, USA
| | - Xiaojue Zhou
- Department of Cognitive Sciences, University of California, Irvine, Irvine, CA 92697, USA
| | - Mariel Tisby
- Department of Cognitive Sciences, University of California, Irvine, Irvine, CA 92697, USA
| | - Patrick T Hwu
- Department of Cognitive Sciences, University of California, Irvine, Irvine, CA 92697, USA
| | - John A Pyles
- Department of Psychology, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| | - Emily D Grossman
- Department of Cognitive Sciences, University of California, Irvine, Irvine, CA 92697, USA
| |
Collapse
|
22
|
Zhang T, Gao JS, Çukur T, Gallant JL. Voxel-Based State Space Modeling Recovers Task-Related Cognitive States in Naturalistic fMRI Experiments. Front Neurosci 2021; 14:565976. [PMID: 34045937 PMCID: PMC8145286 DOI: 10.3389/fnins.2020.565976] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2020] [Accepted: 09/24/2020] [Indexed: 11/23/2022] Open
Abstract
Complex natural tasks likely recruit many different functional brain networks, but it is difficult to predict how such tasks will be represented across cortical areas and networks. Previous electrophysiology studies suggest that task variables are represented in a low-dimensional subspace within the activity space of neural populations. Here we develop a voxel-based state space modeling method for recovering task-related state spaces from human fMRI data. We apply this method to data acquired in a controlled visual attention task and a video game task. We find that each task induces distinct brain states that can be embedded in a low-dimensional state space that reflects task parameters, and that attention increases state separation in the task-related subspace. Our results demonstrate that the state space framework offers a powerful approach for modeling human brain activity elicited by complex natural tasks.
Collapse
Affiliation(s)
- Tianjiao Zhang
- Program in Bioengineering, University of California, Berkeley, Berkeley, CA, United States
| | - James S Gao
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, United States
| | - Tolga Çukur
- Department of Electrical and Electronics Engineering, Bilkent University, Ankara, Turkey.,National Magnetic Resonance Research Center (UMRAM), Bilkent University, Ankara, Turkey.,Neuroscience Program, Sabuncu Brain Research Center, Bilkent University, Ankara, Turkey
| | - Jack L Gallant
- Program in Bioengineering, University of California, Berkeley, Berkeley, CA, United States.,Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, United States.,Department of Psychology, University of California, Berkeley, Berkeley, CA, United States
| |
Collapse
|
23
|
Beach SD, Ozernov-Palchik O, May SC, Centanni TM, Gabrieli JDE, Pantazis D. Neural Decoding Reveals Concurrent Phonemic and Subphonemic Representations of Speech Across Tasks. NEUROBIOLOGY OF LANGUAGE (CAMBRIDGE, MASS.) 2021; 2:254-279. [PMID: 34396148 PMCID: PMC8360503 DOI: 10.1162/nol_a_00034] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/28/2020] [Accepted: 02/21/2021] [Indexed: 06/13/2023]
Abstract
Robust and efficient speech perception relies on the interpretation of acoustically variable phoneme realizations, yet prior neuroimaging studies are inconclusive regarding the degree to which subphonemic detail is maintained over time as categorical representations arise. It is also unknown whether this depends on the demands of the listening task. We addressed these questions by using neural decoding to quantify the (dis)similarity of brain response patterns evoked during two different tasks. We recorded magnetoencephalography (MEG) as adult participants heard isolated, randomized tokens from a /ba/-/da/ speech continuum. In the passive task, their attention was diverted. In the active task, they categorized each token as ba or da. We found that linear classifiers successfully decoded ba vs. da perception from the MEG data. Data from the left hemisphere were sufficient to decode the percept early in the trial, while the right hemisphere was necessary but not sufficient for decoding at later time points. We also decoded stimulus representations and found that they were maintained longer in the active task than in the passive task; however, these representations did not pattern more like discrete phonemes when an active categorical response was required. Instead, in both tasks, early phonemic patterns gave way to a representation of stimulus ambiguity that coincided in time with reliable percept decoding. Our results suggest that the categorization process does not require the loss of subphonemic detail, and that the neural representation of isolated speech sounds includes concurrent phonemic and subphonemic information.
Collapse
Affiliation(s)
- Sara D. Beach
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA
- Program in Speech and Hearing Bioscience and Technology, Harvard University, Cambridge, MA, USA
| | - Ola Ozernov-Palchik
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Sidney C. May
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA
- Lynch School of Education and Human Development, Boston College, Chestnut Hill, MA, USA
| | - Tracy M. Centanni
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Psychology, Texas Christian University, Fort Worth, TX, USA
| | - John D. E. Gabrieli
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Dimitrios Pantazis
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA
| |
Collapse
|
24
|
Matheson HE, Garcea FE, Buxbaum LJ. Scene context shapes category representational geometry during processing of tools. Cortex 2021; 141:1-15. [PMID: 34020166 DOI: 10.1016/j.cortex.2021.03.021] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2020] [Revised: 02/05/2021] [Accepted: 03/12/2021] [Indexed: 10/21/2022]
Abstract
Tools are ubiquitous in human environments and to think about them we use concepts. Increasingly, conceptual representation is thought to be dynamic and sensitive to the goals of the observer. Indeed, observer goals can reshape representational geometry within cortical networks supporting concepts. In the present study, we investigated the novel hypothesis that task-irrelevant scene context may implicitly alter the representational geometry of regions within the tool network. Participants performed conceptual judgments on images of tools embedded in scenes that either suggested their use (i.e., a kitchen timer sitting on a kitchen counter with vegetables in a frying pan) or that they would simply be moved (i.e., a kitchen timer sitting in an open drawer with other miscellaneous kitchen items around). We investigated whether representations in the tool network reflect category, grip, and shape information using a representational similarity analysis (RSA). We show that a) a number of regions of the tool network reflect category information about tools and b) category information predicts patterns in supramarginal gyrus more strongly in use contexts than in move contexts. Together, these results show that information about tool category is distributed across different regions of the tool network and that scene context helps shape the representational geometry of the tool network.
Collapse
Affiliation(s)
- Heath E Matheson
- University of Northern British Columbia, Prince George, BC, Canada.
| | - Frank E Garcea
- Moss Rehabilitation Research Institute, Elkins Park, PA, USA; Department of Neurosurgery, University of Rochester Medical Center, New York, USA
| | | |
Collapse
|
25
|
Karimi-Rouzbahani H, Woolgar A, Rich AN. Neural signatures of vigilance decrements predict behavioural errors before they occur. eLife 2021; 10:e60563. [PMID: 33830017 PMCID: PMC8060034 DOI: 10.7554/elife.60563] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2020] [Accepted: 04/02/2021] [Indexed: 11/24/2022] Open
Abstract
There are many monitoring environments, such as railway control, in which lapses of attention can have tragic consequences. Problematically, sustained monitoring for rare targets is difficult, with more misses and longer reaction times over time. What changes in the brain underpin these 'vigilance decrements'? We designed a multiple-object monitoring (MOM) paradigm to examine how the neural representation of information varied with target frequency and time performing the task. Behavioural performance decreased over time for the rare target (monitoring) condition, but not for a frequent target (active) condition. This was mirrored in neural decoding using magnetoencephalography: coding of critical information declined more during monitoring versus active conditions along the experiment. We developed new analyses that can predict behavioural errors from the neural data more than a second before they occurred. This facilitates pre-empting behavioural errors due to lapses in attention and provides new insight into the neural correlates of vigilance decrements.
Collapse
Affiliation(s)
- Hamid Karimi-Rouzbahani
- Perception in Action Research Centre, Faculty of Human Sciences, Macquarie UniversitySydneyAustralia
- Medical Research Council Cognition and Brain Sciences Unit, University of CambridgeCambridgeUnited Kingdom
- Department of Cognitive Science, Faculty of Human Sciences, Macquarie UniversitySydneyAustralia
| | - Alexandra Woolgar
- Perception in Action Research Centre, Faculty of Human Sciences, Macquarie UniversitySydneyAustralia
- Medical Research Council Cognition and Brain Sciences Unit, University of CambridgeCambridgeUnited Kingdom
- Department of Cognitive Science, Faculty of Human Sciences, Macquarie UniversitySydneyAustralia
| | - Anina N Rich
- Perception in Action Research Centre, Faculty of Human Sciences, Macquarie UniversitySydneyAustralia
- Department of Cognitive Science, Faculty of Human Sciences, Macquarie UniversitySydneyAustralia
| |
Collapse
|
26
|
Jolly E, Chang LJ. Multivariate spatial feature selection in fMRI. Soc Cogn Affect Neurosci 2021; 16:795-806. [PMID: 33501987 PMCID: PMC8343556 DOI: 10.1093/scan/nsab010] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2020] [Revised: 09/16/2020] [Accepted: 01/25/2021] [Indexed: 01/20/2023] Open
Abstract
Multivariate neuroimaging analyses constitute a powerful class of techniques to identify psychological representations. However, not all psychological processes are represented at the same spatial scale throughout the brain. This heterogeneity is apparent when comparing hierarchically organized local representations of perceptual processes to flexible transmodal representations of more abstract cognitive processes such as social and affective operations. An open question is how the spatial scale of analytic approaches interacts with the spatial scale of the representations under investigation. In this article, we describe how multivariate analyses can be viewed as existing on a spatial spectrum, anchored by searchlights used to identify locally distributed patterns of information on one end, whole brain approach used to identify diffuse neural representations at the other and region-based approaches in between. We describe how these distinctions are an important and often overlooked analytic consideration and provide heuristics to compare these different techniques to choose based on the analyst’s inferential goals.
Collapse
Affiliation(s)
- E Jolly
- Computational Social Affective Neuroscience Laboratory, Department of Psychological and Brain Science, Dartmouth College, Hanover, NH 03755, USA
| | - L J Chang
- Computational Social Affective Neuroscience Laboratory, Department of Psychological and Brain Science, Dartmouth College, Hanover, NH 03755, USA
| |
Collapse
|
27
|
Levine SM, Schwarzbach JV. Individualizing Representational Similarity Analysis. Front Psychiatry 2021; 12:729457. [PMID: 34707520 PMCID: PMC8542717 DOI: 10.3389/fpsyt.2021.729457] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/23/2021] [Accepted: 09/10/2021] [Indexed: 11/13/2022] Open
Abstract
Representational similarity analysis (RSA) is a popular multivariate analysis technique in cognitive neuroscience that uses functional neuroimaging to investigate the informational content encoded in brain activity. As RSA is increasingly being used to investigate more clinically-geared questions, the focus of such translational studies turns toward the importance of individual differences and their optimization within the experimental design. In this perspective, we focus on two design aspects: applying individual vs. averaged behavioral dissimilarity matrices to multiple participants' neuroimaging data and ensuring the congruency between tasks when measuring behavioral and neural representational spaces. Incorporating these methods permits the detection of individual differences in representational spaces and yields a better-defined transfer of information from representational spaces onto multivoxel patterns. Such design adaptations are prerequisites for optimal translation of RSA to the field of precision psychiatry.
Collapse
Affiliation(s)
- Seth M Levine
- Institute of Cognitive and Clinical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
| | - Jens V Schwarzbach
- Department of Psychiatry and Psychotherapy, University of Regensburg, Regensburg, Germany
| |
Collapse
|
28
|
Poncet M, Fabre‐Thorpe M, Chakravarthi R. A simple rule to describe interactions between visual categories. Eur J Neurosci 2020; 52:4639-4666. [DOI: 10.1111/ejn.14890] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2019] [Revised: 06/12/2020] [Accepted: 06/24/2020] [Indexed: 11/27/2022]
Affiliation(s)
- Marlene Poncet
- CerCo Université de ToulouseCNRSUPS Toulouse France
- School of Psychology University of St Andrews St Andrews UK
| | | | | |
Collapse
|
29
|
Reaction times predict dynamic brain representations measured with MEG for only some object categorisation tasks. Neuropsychologia 2020; 151:107687. [PMID: 33212137 DOI: 10.1016/j.neuropsychologia.2020.107687] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2020] [Revised: 09/29/2020] [Accepted: 11/10/2020] [Indexed: 11/21/2022]
Abstract
Behavioural categorisation reaction times (RTs) provide a useful way to link behaviour to brain representations measured with neuroimaging. In this framework, objects are assumed to be represented in a multidimensional activation space, with the distances between object representations indicating their degree of neural similarity. Faster RTs have been reported to correlate with greater distances from a classification decision boundary for animacy. Objects inherently belong to more than one category, yet it is not known whether the RT-distance relationship, and its evolution over the time-course of the neural response, is similar across different categories. Here we used magnetoencephalography (MEG) to address this question. Our stimuli included typically animate and inanimate objects, as well as more ambiguous examples (i.e., robots and toys). We conducted four semantic categorisation tasks on the same stimulus set assessing animacy, living, moving, and human-similarity concepts, and linked the categorisation RTs to MEG time-series decoding data. Our results show a sustained RT-distance relationship throughout the time course of object processing for not only animacy, but also categorisation according to human-similarity. Interestingly, this sustained RT-distance relationship was not observed for the living and moving category organisations, despite comparable classification accuracy of the MEG data across all four category organisations. Our findings show that behavioural RTs predict representational distance for an organisational principle other than animacy, however further research is needed to determine why this relationship is observed only for some category organisations and not others.
Collapse
|
30
|
Ebitz RB, Tu JC, Hayden BY. Rules warp feature encoding in decision-making circuits. PLoS Biol 2020; 18:e3000951. [PMID: 33253163 PMCID: PMC7728226 DOI: 10.1371/journal.pbio.3000951] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2020] [Revised: 12/10/2020] [Accepted: 11/02/2020] [Indexed: 01/22/2023] Open
Abstract
We have the capacity to follow arbitrary stimulus-response rules, meaning simple policies that guide our behavior. Rule identity is broadly encoded across decision-making circuits, but there are less data on how rules shape the computations that lead to choices. One idea is that rules could simplify these computations. When we follow a rule, there is no need to encode or compute information that is irrelevant to the current rule, which could reduce the metabolic or energetic demands of decision-making. However, it is not clear if the brain can actually take advantage of this computational simplicity. To test this idea, we recorded from neurons in 3 regions linked to decision-making, the orbitofrontal cortex (OFC), ventral striatum (VS), and dorsal striatum (DS), while macaques performed a rule-based decision-making task. Rule-based decisions were identified via modeling rules as the latent causes of decisions. This left us with a set of physically identical choices that maximized reward and information, but could not be explained by simple stimulus-response rules. Contrasting rule-based choices with these residual choices revealed that following rules (1) decreased the energetic cost of decision-making; and (2) expanded rule-relevant coding dimensions and compressed rule-irrelevant ones. Together, these results suggest that we use rules, in part, because they reduce the costs of decision-making through a distributed representational warping in decision-making circuits.
Collapse
Affiliation(s)
- R. Becket Ebitz
- Department of Neuroscience, Center for Magnetic Resonance Research, and Center for Neuroengineering University of Minnesota, Minneapolis, Minnesota, United States of America
| | - Jiaxin Cindy Tu
- Department of Neuroscience, Center for Magnetic Resonance Research, and Center for Neuroengineering University of Minnesota, Minneapolis, Minnesota, United States of America
| | - Benjamin Y. Hayden
- Department of Neuroscience, Center for Magnetic Resonance Research, and Center for Neuroengineering University of Minnesota, Minneapolis, Minnesota, United States of America
| |
Collapse
|
31
|
Haxby JV, Gobbini MI, Nastase SA. Naturalistic stimuli reveal a dominant role for agentic action in visual representation. Neuroimage 2020; 216:116561. [DOI: 10.1016/j.neuroimage.2020.116561] [Citation(s) in RCA: 32] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2019] [Revised: 01/07/2020] [Accepted: 01/14/2020] [Indexed: 11/26/2022] Open
|
32
|
DuPre E, Hanke M, Poline JB. Nature abhors a paywall: How open science can realize the potential of naturalistic stimuli. Neuroimage 2020; 216:116330. [PMID: 31704292 PMCID: PMC7198323 DOI: 10.1016/j.neuroimage.2019.116330] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2019] [Revised: 10/31/2019] [Accepted: 10/31/2019] [Indexed: 11/26/2022] Open
Abstract
Naturalistic stimuli show significant potential to inform behavioral, cognitive, and clinical neuroscience. To date, this impact is still limited by the relative inaccessibility of both generated neuroimaging data as well as the supporting naturalistic stimuli. In this perspective, we highlight currently available naturalistic datasets and technical solutions such as DataLad that continue to advance our ability to share this data. We also review scientific and sociological challenges in selecting naturalistic stimuli for reproducible research. Overall, we encourage researchers to share their naturalistic datasets to the full extent possible under local copyright law.
Collapse
Affiliation(s)
| | - Michael Hanke
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Jülich, Jülich, Germany; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | | |
Collapse
|
33
|
Tovar DA, Murray MM, Wallace MT. Selective Enhancement of Object Representations through Multisensory Integration. J Neurosci 2020; 40:5604-5615. [PMID: 32499378 PMCID: PMC7363464 DOI: 10.1523/jneurosci.2139-19.2020] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2019] [Revised: 04/17/2020] [Accepted: 05/21/2020] [Indexed: 11/21/2022] Open
Abstract
Objects are the fundamental building blocks of how we create a representation of the external world. One major distinction among objects is between those that are animate versus those that are inanimate. In addition, many objects are specified by more than a single sense, yet the nature by which multisensory objects are represented by the brain remains poorly understood. Using representational similarity analysis of male and female human EEG signals, we show enhanced encoding of audiovisual objects when compared with their corresponding visual and auditory objects. Surprisingly, we discovered that the often-found processing advantages for animate objects were not evident under multisensory conditions. This was due to a greater neural enhancement of inanimate objects-which are more weakly encoded under unisensory conditions. Further analysis showed that the selective enhancement of inanimate audiovisual objects corresponded with an increase in shared representations across brain areas, suggesting that the enhancement was mediated by multisensory integration. Moreover, a distance-to-bound analysis provided critical links between neural findings and behavior. Improvements in neural decoding at the individual exemplar level for audiovisual inanimate objects predicted reaction time differences between multisensory and unisensory presentations during a Go/No-Go animate categorization task. Links between neural activity and behavioral measures were most evident at intervals of 100-200 ms and 350-500 ms after stimulus presentation, corresponding to time periods associated with sensory evidence accumulation and decision-making, respectively. Collectively, these findings provide key insights into a fundamental process the brain uses to maximize the information it captures across sensory systems to perform object recognition.SIGNIFICANCE STATEMENT Our world is filled with ever-changing sensory information that we are able to seamlessly transform into a coherent and meaningful perceptual experience. We accomplish this feat by combining different stimulus features into objects. However, despite the fact that these features span multiple senses, little is known about how the brain combines the various forms of sensory information into object representations. Here, we used EEG and machine learning to study how the brain processes auditory, visual, and audiovisual objects. Surprisingly, we found that nonliving (i.e., inanimate) objects, which are more difficult to process with one sense alone, benefited the most from engaging multiple senses.
Collapse
Affiliation(s)
- David A Tovar
- School of Medicine, Vanderbilt University, Nashville, Tennessee 37240
- Vanderbilt Brain Institute, Vanderbilt University, Nashville, Tennessee 37240
| | - Micah M Murray
- The Laboratory for Investigative Neurophysiology (The LINE), Department of Radiology, Lausanne University Hospital and University of Lausanne (CHUV-UNIL), 1011 Lausanne, Switzerland
- Sensory, Cognitive and Perceptual Neuroscience Section, Center for Biomedical Imaging (CIBM) of Lausanne and Geneva, 1015 Lausanne, Switzerland
- Department of Ophthalmology, Fondation Asile des aveugles and University of Lausanne, 1002 Lausanne, Switzerland
- Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, Tennessee 37240
| | - Mark T Wallace
- School of Medicine, Vanderbilt University, Nashville, Tennessee 37240
- Vanderbilt Brain Institute, Vanderbilt University, Nashville, Tennessee 37240
- Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, Tennessee 37240
- Department of Psychology, Vanderbilt University, Nashville, Tennessee 37240
- Department of Psychiatry and Behavioral Sciences, Vanderbilt University Medical Center, Nashville, Tennessee 37240
- Department of Pharmacology, Vanderbilt University, Nashville, Tennessee 37240
| |
Collapse
|
34
|
Weaverdyck ME, Lieberman MD, Parkinson C. Tools of the Trade Multivoxel pattern analysis in fMRI: a practical introduction for social and affective neuroscientists. Soc Cogn Affect Neurosci 2020; 15:487-509. [PMID: 32364607 PMCID: PMC7308652 DOI: 10.1093/scan/nsaa057] [Citation(s) in RCA: 57] [Impact Index Per Article: 14.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2019] [Revised: 04/08/2020] [Accepted: 04/15/2020] [Indexed: 12/26/2022] Open
Abstract
The family of neuroimaging analytical techniques known as multivoxel pattern analysis (MVPA) has dramatically increased in popularity over the past decade, particularly in social and affective neuroscience research using functional magnetic resonance imaging (fMRI). MVPA examines patterns of neural responses, rather than analyzing single voxel- or region-based values, as is customary in conventional univariate analyses. Here, we provide a practical introduction to MVPA and its most popular variants (namely, representational similarity analysis (RSA) and decoding analyses, such as classification using machine learning) for social and affective neuroscientists of all levels, particularly those new to such methods. We discuss how MVPA differs from traditional mass-univariate analyses, the benefits MVPA offers to social neuroscientists, experimental design and analysis considerations, step-by-step instructions for how to implement specific analyses in one's own dataset and issues that are currently facing research using MVPA methods.
Collapse
Affiliation(s)
- Miriam E Weaverdyck
- Department of Psychology, University of California, Los Angeles, CA 90095, USA
| | - Matthew D Lieberman
- Department of Psychology, University of California, Los Angeles, CA 90095, USA
| | - Carolyn Parkinson
- Department of Psychology, University of California, Los Angeles, CA 90095, USA
- Brain Research Institute, University of California, Los Angeles, CA 90095, USA
| |
Collapse
|
35
|
Haxby JV, Guntupalli JS, Nastase SA, Feilong M. Hyperalignment: Modeling shared information encoded in idiosyncratic cortical topographies. eLife 2020; 9:e56601. [PMID: 32484439 PMCID: PMC7266639 DOI: 10.7554/elife.56601] [Citation(s) in RCA: 67] [Impact Index Per Article: 16.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2020] [Accepted: 05/14/2020] [Indexed: 01/13/2023] Open
Abstract
Information that is shared across brains is encoded in idiosyncratic fine-scale functional topographies. Hyperalignment captures shared information by projecting pattern vectors for neural responses and connectivities into a common, high-dimensional information space, rather than by aligning topographies in a canonical anatomical space. Individual transformation matrices project information from individual anatomical spaces into the common model information space, preserving the geometry of pairwise dissimilarities between pattern vectors, and model cortical topography as mixtures of overlapping, individual-specific topographic basis functions, rather than as contiguous functional areas. The fundamental property of brain function that is preserved across brains is information content, rather than the functional properties of local features that support that content. In this Perspective, we present the conceptual framework that motivates hyperalignment, its computational underpinnings for joint modeling of a common information space and idiosyncratic cortical topographies, and discuss implications for understanding the structure of cortical functional architecture.
Collapse
Affiliation(s)
- James V Haxby
- Center for Cognitive Neuroscience, Dartmouth CollegeHanoverUnited States
| | | | | | - Ma Feilong
- Center for Cognitive Neuroscience, Dartmouth CollegeHanoverUnited States
| |
Collapse
|
36
|
A study in affect: Predicting valence from fMRI data. Neuropsychologia 2020; 143:107473. [DOI: 10.1016/j.neuropsychologia.2020.107473] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2019] [Revised: 04/10/2020] [Accepted: 04/19/2020] [Indexed: 12/19/2022]
|
37
|
Zhang RY, Kay K. Flexible top-down modulation in human ventral temporal cortex. Neuroimage 2020; 218:116964. [PMID: 32439537 DOI: 10.1016/j.neuroimage.2020.116964] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2019] [Revised: 05/13/2020] [Accepted: 05/14/2020] [Indexed: 01/10/2023] Open
Abstract
Visual neuroscientists have long characterized attention as inducing a scaling or additive effect on fixed parametric functions describing neural responses (e.g., contrast response functions). Here, we instead propose that top-down effects are more complex and manifest in ways that depend not only on attention but also other cognitive processes involved in executing a task. To substantiate this theory, we analyze fMRI responses in human ventral temporal cortex (VTC) in a study where stimulus eccentricity and cognitive task are varied. We find that as stimuli are presented farther into the periphery, bottom-up stimulus-driven responses decline but top-down attentional enhancement increases substantially. This disproportionate enhancement of weak responses cannot be easily explained by conventional models of attention. Furthermore, we find that attentional effects depend on the specific cognitive task performed by the subject, indicating the influence of additional cognitive processes other than attention (e.g., decision-making). The effects we observe replicate in an independent experiment from the same study, and also generalize to a separate study involving different stimulus manipulations (contrast and phase coherence). Our results suggest that a quantitative understanding of top-down modulation requires more nuanced characterization of the multiple cognitive factors involved in completing a perceptual task.
Collapse
Affiliation(s)
- Ru-Yuan Zhang
- Shanghai Key Laboratory of Psychotic Disorders, Shanghai Mental Health Center, School of Medicine, Shanghai Jiao Tong University, Shanghai, 200030, China; Institute of Psychology and Behavioral Science, Shanghai Jiao Tong University, Shanghai, 200030, China; Center for Magnetic Resonance Research, Department of Radiology, University of Minnesota, Minneapolis, MN, 55455, USA.
| | - Kendrick Kay
- Center for Magnetic Resonance Research, Department of Radiology, University of Minnesota, Minneapolis, MN, 55455, USA
| |
Collapse
|
38
|
Levine SM, Alahäivälä ALI, Wechsler TF, Wackerle A, Rupprecht R, Schwarzbach JV. Linking Personality Traits to Individual Differences in Affective Spaces. Front Psychol 2020; 11:448. [PMID: 32231631 PMCID: PMC7082752 DOI: 10.3389/fpsyg.2020.00448] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2019] [Accepted: 02/26/2020] [Indexed: 11/13/2022] Open
Abstract
Different individuals respond differently to emotional stimuli in their environment. Therefore, to understand how emotions are represented mentally will ultimately require investigations into individual-level information. Here we tasked participants with freely arranging emotionally charged images on a computer screen according to their subjective emotional similarity (yielding a unique affective space for each participant) and subsequently sought external validity of the layout of the individuals’ affective spaces through the five-factor personality model (Neuroticism, Extraversion, Openness to Experience, Agreeableness, Conscientiousness) assessed via the NEO Five-Factor Inventory. Applying agglomerative hierarchical clustering to the group-level affective space revealed a set of underlying affective clusters whose within-cluster dissimilarity, per individual, was then correlated with individuals’ personality scores. These cluster-based analyses predominantly revealed that the dispersion of the negative cluster showed a positive relationship with Neuroticism and a negative relationship with Conscientiousness, a finding that would be predicted by prior work. Such results demonstrate the non-spurious structure of individualized emotion information revealed by data-driven analyses of a behavioral task (and validated by incorporating psychological measures of personality) and corroborate prior knowledge of the interaction between affect and personality. Future investigations can similarly combine hypothesis- and data-driven methods to extend such findings, potentially yielding new perspectives on underlying cognitive processes, disease susceptibility, or even diagnostic/prognostic markers for mental disorders involving emotion dysregulation.
Collapse
Affiliation(s)
- Seth M Levine
- Department of Psychiatry and Psychotherapy, University of Regensburg, Regensburg, Germany
| | - Aino L I Alahäivälä
- Department of Psychiatry and Psychotherapy, University of Regensburg, Regensburg, Germany
| | - Theresa F Wechsler
- Department of Clinical Psychology and Psychotherapy, University of Regensburg, Regensburg, Germany
| | - Anja Wackerle
- Department of Psychiatry and Psychotherapy, University of Regensburg, Regensburg, Germany
| | - Rainer Rupprecht
- Department of Psychiatry and Psychotherapy, University of Regensburg, Regensburg, Germany
| | - Jens V Schwarzbach
- Department of Psychiatry and Psychotherapy, University of Regensburg, Regensburg, Germany
| |
Collapse
|
39
|
Functional Imaging of Visuospatial Attention in Complex and Naturalistic Conditions. Curr Top Behav Neurosci 2020. [PMID: 30547430 DOI: 10.1007/7854_2018_73] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/30/2023]
Abstract
One of the ultimate goals of cognitive neuroscience is to understand how the brain works in the real world. Functional imaging with naturalistic stimuli provides us with the opportunity to study the brain in situations similar to the everyday life. This includes the processing of complex stimuli that can trigger many types of signals related both to the physical characteristics of the external input and to the internal knowledge that we have about natural objects and environments. In this chapter, I will first outline different types of stimuli that have been used in naturalistic imaging studies. These include static pictures, short video clips, full-length movies, and virtual reality, each comprising specific advantages and disadvantages. Next, I will turn to the main issue of visual-spatial orienting in naturalistic conditions and its neural substrates. I will discuss different classes of internal signals, related to objects, scene structure, and long-term memory. All of these, together with external signals about stimulus salience, have been found to modulate the activity and the connectivity of the frontoparietal attention networks. I will conclude by pointing out some promising future directions for functional imaging with naturalistic stimuli. Despite this field of research is still in its early days, I consider that it will play a major role in bridging the gap between standard laboratory paradigms and mechanisms of brain functioning in the real world.
Collapse
|
40
|
Jiahui G, Feilong M, Visconti di Oleggio Castello M, Guntupalli JS, Chauhan V, Haxby JV, Gobbini MI. Predicting individual face-selective topography using naturalistic stimuli. Neuroimage 2019; 216:116458. [PMID: 31843709 DOI: 10.1016/j.neuroimage.2019.116458] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2019] [Revised: 11/16/2019] [Accepted: 12/09/2019] [Indexed: 01/28/2023] Open
Abstract
Subject-specific, functionally defined areas are conventionally estimated with functional localizers and a simple contrast analysis between responses to different stimulus categories. Compared with functional localizers, naturalistic stimuli provide several advantages such as stronger and widespread brain activation, greater engagement, and increased subject compliance. In this study we demonstrate that a subject's idiosyncratic functional topography can be estimated with high fidelity from that subject's fMRI data obtained while watching a naturalistic movie using hyperalignment to project other subjects' localizer data into that subject's idiosyncratic cortical anatomy. These findings lay the foundation for developing an efficient tool for mapping functional topographies for a wide range of perceptual and cognitive functions in new subjects based only on fMRI data collected while watching an engaging, naturalistic stimulus and other subjects' localizer data from a normative sample.
Collapse
Affiliation(s)
- Guo Jiahui
- Center for Cognitive Neuroscience, Dartmouth College, NH, USA
| | - Ma Feilong
- Center for Cognitive Neuroscience, Dartmouth College, NH, USA
| | | | | | - Vassiki Chauhan
- Center for Cognitive Neuroscience, Dartmouth College, NH, USA
| | - James V Haxby
- Center for Cognitive Neuroscience, Dartmouth College, NH, USA
| | - M Ida Gobbini
- Cognitive Science, Dartmouth College, NH, USA; Dipartimento di Medicina Specialistica, Diagnostica e Sperimentale, Università di Bologna, Bologna, 40126, Italy.
| |
Collapse
|
41
|
Popal H, Wang Y, Olson IR. A Guide to Representational Similarity Analysis for Social Neuroscience. Soc Cogn Affect Neurosci 2019; 14:1243-1253. [PMID: 31989169 PMCID: PMC7057283 DOI: 10.1093/scan/nsz099] [Citation(s) in RCA: 49] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2019] [Revised: 10/13/2019] [Accepted: 10/22/2019] [Indexed: 01/04/2023] Open
Abstract
Representational similarity analysis (RSA) is a computational technique that uses pairwise comparisons of stimuli to reveal their representation in higher-order space. In the context of neuroimaging, mass-univariate analyses and other multivariate analyses can provide information on what and where information is represented but have limitations in their ability to address how information is represented. Social neuroscience is a field that can particularly benefit from incorporating RSA techniques to explore hypotheses regarding the representation of multidimensional data, how representations can predict behavior, how representations differ between groups and how multimodal data can be compared to inform theories. The goal of this paper is to provide a practical as well as theoretical guide to implementing RSA in social neuroscience studies.
Collapse
Affiliation(s)
- Haroon Popal
- Department of Psychology, Temple University, Philadelphia, PA
| | | | - Ingrid R Olson
- Department of Psychology, Temple University, Philadelphia, PA
| |
Collapse
|
42
|
McClure P, Rho N, Lee JA, Kaczmarzyk JR, Zheng CY, Ghosh SS, Nielson DM, Thomas AG, Bandettini P, Pereira F. Knowing What You Know in Brain Segmentation Using Bayesian Deep Neural Networks. Front Neuroinform 2019; 13:67. [PMID: 31749693 PMCID: PMC6843052 DOI: 10.3389/fninf.2019.00067] [Citation(s) in RCA: 22] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2019] [Accepted: 09/18/2019] [Indexed: 01/03/2023] Open
Abstract
In this paper, we describe a Bayesian deep neural network (DNN) for predicting FreeSurfer segmentations of structural MRI volumes, in minutes rather than hours. The network was trained and evaluated on a large dataset (n = 11,480), obtained by combining data from more than a hundred different sites, and also evaluated on another completely held-out dataset (n = 418). The network was trained using a novel spike-and-slab dropout-based variational inference approach. We show that, on these datasets, the proposed Bayesian DNN outperforms previously proposed methods, in terms of the similarity between the segmentation predictions and the FreeSurfer labels, and the usefulness of the estimate uncertainty of these predictions. In particular, we demonstrated that the prediction uncertainty of this network at each voxel is a good indicator of whether the network has made an error and that the uncertainty across the whole brain can predict the manual quality control ratings of a scan. The proposed Bayesian DNN method should be applicable to any new network architecture for addressing the segmentation problem.
Collapse
Affiliation(s)
- Patrick McClure
- Machine Learning Team, National Institute of Mental Health, Bethesda, MD, United States
- Section on Functional Imaging Methods, National Institute of Mental Health, Bethesda, MD, United States
| | - Nao Rho
- Machine Learning Team, National Institute of Mental Health, Bethesda, MD, United States
- Section on Functional Imaging Methods, National Institute of Mental Health, Bethesda, MD, United States
| | - John A. Lee
- Section on Functional Imaging Methods, National Institute of Mental Health, Bethesda, MD, United States
- Data Sharing and Science Team, National Institute of Mental Health, Bethesda, MD, United States
| | - Jakub R. Kaczmarzyk
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, United States
| | - Charles Y. Zheng
- Machine Learning Team, National Institute of Mental Health, Bethesda, MD, United States
- Section on Functional Imaging Methods, National Institute of Mental Health, Bethesda, MD, United States
| | - Satrajit S. Ghosh
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, United States
| | - Dylan M. Nielson
- Section on Functional Imaging Methods, National Institute of Mental Health, Bethesda, MD, United States
- Data Sharing and Science Team, National Institute of Mental Health, Bethesda, MD, United States
| | - Adam G. Thomas
- Data Sharing and Science Team, National Institute of Mental Health, Bethesda, MD, United States
| | - Peter Bandettini
- Section on Functional Imaging Methods, National Institute of Mental Health, Bethesda, MD, United States
| | - Francisco Pereira
- Machine Learning Team, National Institute of Mental Health, Bethesda, MD, United States
- Section on Functional Imaging Methods, National Institute of Mental Health, Bethesda, MD, United States
| |
Collapse
|
43
|
Palenciano AF, González-García C, Arco JE, Pessoa L, Ruz M. Representational Organization of Novel Task Sets during Proactive Encoding. J Neurosci 2019; 39:8386-8397. [PMID: 31427394 PMCID: PMC6794921 DOI: 10.1523/jneurosci.0725-19.2019] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2019] [Revised: 07/19/2019] [Accepted: 08/13/2019] [Indexed: 11/21/2022] Open
Abstract
Recent multivariate analyses of brain data have boosted our understanding of the organizational principles that shape neural coding. However, most of this progress has focused on perceptual visual regions (Connolly et al., 2012), whereas far less is known about the organization of more abstract, action-oriented representations. In this study, we focused on humans' remarkable ability to turn novel instructions into actions. While previous research shows that instruction encoding is tightly linked to proactive activations in frontoparietal brain regions, little is known about the structure that orchestrates such anticipatory representation. We collected fMRI data while participants (both males and females) followed novel complex verbal rules that varied across control-related variables (integrating within/across stimuli dimensions, response complexity, target category) and reward expectations. Using representational similarity analysis (Kriegeskorte et al., 2008), we explored where in the brain these variables explained the organization of novel task encoding, and whether motivation modulated these representational spaces. Instruction representations in the lateral PFC were structured by the three control-related variables, whereas intraparietal sulcus encoded response complexity and the fusiform gyrus and precuneus organized its activity according to the relevant stimulus category. Reward exerted a general effect, increasing the representational similarity among different instructions, which was robustly correlated with behavioral improvements. Overall, our results highlight the flexibility of proactive task encoding, governed by distinct representational organizations in specific brain regions. They also stress the variability of motivation-control interactions, which appear to be highly dependent on task attributes, such as complexity or novelty.SIGNIFICANCE STATEMENT In comparison with other primates, humans display a remarkable success in novel task contexts thanks to our ability to transform instructions into effective actions. This skill is associated with proactive task-set reconfigurations in frontoparietal cortices. It remains yet unknown, however, how the brain encodes in anticipation the flexible, rich repertoire of novel tasks that we can achieve. Here we explored cognitive control and motivation-related variables that might orchestrate the representational space for novel instructions. Our results showed that different dimensions become relevant for task prospective encoding, depending on the brain region, and that the lateral PFC simultaneously organized task representations following different control-related variables. Motivation exerted a general modulation upon this process, diminishing rather than increasing distances among instruction representations.
Collapse
Affiliation(s)
- Ana F Palenciano
- Mind, Brain, and Behavior Research Center, University of Granada, 18011, Granada, Spain
| | | | - Juan E Arco
- Mind, Brain, and Behavior Research Center, University of Granada, 18011, Granada, Spain
| | - Luiz Pessoa
- Psychology Department, University of Maryland 20742
| | - María Ruz
- Mind, Brain, and Behavior Research Center, University of Granada, 18011, Granada, Spain,
| |
Collapse
|
44
|
Cichy RM, Kriegeskorte N, Jozwik KM, van den Bosch JJ, Charest I. The spatiotemporal neural dynamics underlying perceived similarity for real-world objects. Neuroimage 2019; 194:12-24. [PMID: 30894333 PMCID: PMC6547050 DOI: 10.1016/j.neuroimage.2019.03.031] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2018] [Revised: 01/25/2019] [Accepted: 03/13/2019] [Indexed: 01/19/2023] Open
Abstract
The degree to which we perceive real-world objects as similar or dissimilar structures our perception and guides categorization behavior. Here, we investigated the neural representations enabling perceived similarity using behavioral judgments, fMRI and MEG. As different object dimensions co-occur and partly correlate, to understand the relationship between perceived similarity and brain activity it is necessary to assess the unique role of multiple object dimensions. We thus behaviorally assessed perceived object similarity in relation to shape, function, color and background. We then used representational similarity analyses to relate these behavioral judgments to brain activity. We observed a link between each object dimension and representations in visual cortex. These representations emerged rapidly within 200 ms of stimulus onset. Assessing the unique role of each object dimension revealed partly overlapping and distributed representations: while color-related representations distinctly preceded shape-related representations both in the processing hierarchy of the ventral visual pathway and in time, several dimensions were linked to high-level ventral visual cortex. Further analysis singled out the shape dimension as neither fully accounted for by supra-category membership, nor a deep neural network trained on object categorization. Together our results comprehensively characterize the relationship between perceived similarity of key object dimensions and neural activity.
Collapse
Affiliation(s)
- Radoslaw M. Cichy
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany,Bernstein Center for Computational Neuroscience Berlin, Berlin, Germany,Berlin School of Mind and Brain, Berlin, Germany,Corresponding author. Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany.
| | - Nikolaus Kriegeskorte
- Department of Psychology, Zuckerman Mind Brain Behavior Institute, Columbia University, New York, USA
| | - Kamila M. Jozwik
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany
| | | | - Ian Charest
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK,School of Psychology, University of Birmingham, Birmingham, UK
| |
Collapse
|
45
|
Wen Z, Yu T, Yang X, Li Y. Goal-Directed Processing of Naturalistic Stimuli Modulates Large-Scale Functional Connectivity. Front Neurosci 2019; 12:1003. [PMID: 30760971 PMCID: PMC6361838 DOI: 10.3389/fnins.2018.01003] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2018] [Accepted: 12/12/2018] [Indexed: 11/25/2022] Open
Abstract
Humans selectively process external information according to their internal goals. Previous studies have found that cortical activity and interactions between specific cortical areas such as frontal-parietal regions are modulated by behavioral goals. However, these results are largely based on simple stimuli and task rules in laboratory settings. Here, we investigated how top-down goals modulate whole-brain functional connectivity (FC) under naturalistic conditions. Analyses were conducted on a publicly available functional magnetic resonance imaging (fMRI) dataset (OpenfMRI database, accession number: ds000233) collected on twelve participants who made either behavioral or taxonomic judgments of behaving animals containing in naturalistic video clips. The task-evoked FC patterns of the participants were extracted using a novel inter-subject functional correlation (ISFC) method that increases the signal-to-noise ratio for detecting task-induced inter-regional correlation compared with standard FC analysis. Using multivariate pattern analysis (MVPA) methods, we successfully predicted the task goals of the participants with ISFC patterns but not with standard FC patterns, suggests that the ISFC method may be an efficient tool for exploring subtle network differences between brain states. We further examined the predictive power of several canonical brain networks and found that many within-network and across-network ISFC measures supported task goals classification. Our findings suggest that goal-directed processing of naturalistic stimuli systematically modulates large-scale brain networks but is not limited to the local neural activity or connectivity of specific regions.
Collapse
Affiliation(s)
- Zhenfu Wen
- Center for Brain Computer Interfaces and Brain Information Processing, South China University of Technology, Guangzhou, China.,Guangzhou Key Laboratory of Brain Computer Interaction and Application, Guangzhou, China
| | - Tianyou Yu
- Center for Brain Computer Interfaces and Brain Information Processing, South China University of Technology, Guangzhou, China.,Guangzhou Key Laboratory of Brain Computer Interaction and Application, Guangzhou, China
| | - Xinbin Yang
- Department of Surgical Thoracic Oncology, Affiliated Cancer Hospital and Institute of Guangzhou Medical University, Guangzhou, China
| | - Yuanqing Li
- Center for Brain Computer Interfaces and Brain Information Processing, South China University of Technology, Guangzhou, China.,Guangzhou Key Laboratory of Brain Computer Interaction and Application, Guangzhou, China
| |
Collapse
|
46
|
Brady TF, Störmer VS, Shafer-Skelton A, Williams JR, Chapman AF, Schill HM. Scaling up visual attention and visual working memory to the real world. PSYCHOLOGY OF LEARNING AND MOTIVATION 2019. [DOI: 10.1016/bs.plm.2019.03.001] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/16/2023]
|
47
|
fMRIPrep: a robust preprocessing pipeline for functional MRI. Nat Methods 2018; 16:111-116. [PMID: 30532080 PMCID: PMC6319393 DOI: 10.1038/s41592-018-0235-4] [Citation(s) in RCA: 1372] [Impact Index Per Article: 228.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2018] [Accepted: 10/29/2018] [Indexed: 12/20/2022]
Abstract
Preprocessing of functional MRI (fMRI) involves numerous steps to clean and standardize data before statistical analysis. Generally, researchers create ad-hoc preprocessing workflows for each new dataset, building upon a large inventory of tools available. The complexity of these workflows has snowballed with rapid advances in acquisition and processing. We introduce fMRIPrep, an analysis-agnostic tool that addresses the challenge of robust and reproducible preprocessing for fMRI data. FMRIPrep automatically adapts a best-in-breed workflow to the idiosyncrasies of virtually any dataset, ensuring high-quality preprocessing with no manual intervention. By introducing visual assessment checkpoints into an iterative integration framework for software-testing, we show that fMRIPrep robustly produces high-quality results on a diverse fMRI data collection. Additionally, fMRIPrep introduces less uncontrolled spatial smoothness than commonly used preprocessing tools. FMRIPrep equips neuroscientists with a high-quality, robust, easy-to-use and transparent preprocessing workflow, which can help ensure the validity of inference and the interpretability of their results.
Collapse
|
48
|
van Loon AM, Olmos-Solis K, Fahrenfort JJ, Olivers CNL. Current and future goals are represented in opposite patterns in object-selective cortex. eLife 2018; 7:e38677. [PMID: 30394873 PMCID: PMC6279347 DOI: 10.7554/elife.38677] [Citation(s) in RCA: 46] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2018] [Accepted: 10/31/2018] [Indexed: 11/21/2022] Open
Abstract
Adaptive behavior requires the separation of current from future goals in working memory. We used fMRI of object-selective cortex to determine the representational (dis)similarities of memory representations serving current and prospective perceptual tasks. Participants remembered an object drawn from three possible categories as the target for one of two consecutive visual search tasks. A cue indicated whether the target object should be looked for first (currently relevant), second (prospectively relevant), or if it could be forgotten (irrelevant). Prior to the first search, representations of current, prospective and irrelevant objects were similar, with strongest decoding for current representations compared to prospective (Experiment 1) and irrelevant (Experiment 2). Remarkably, during the first search, prospective representations could also be decoded, but revealed anti-correlated voxel patterns compared to currently relevant representations of the same category. We propose that the brain separates current from prospective memories within the same neuronal ensembles through opposite representational patterns.
Collapse
Affiliation(s)
- Anouk Mariette van Loon
- Department of Experimental and Applied PsychologyVrije Universiteit AmsterdamAmsterdamThe Netherlands
- Institute of Brain and Behavior AmsterdamVrije Universiteit AmsterdamAmsterdamThe Netherlands
| | - Katya Olmos-Solis
- Department of Experimental and Applied PsychologyVrije Universiteit AmsterdamAmsterdamThe Netherlands
| | - Johannes Jacobus Fahrenfort
- Department of Experimental and Applied PsychologyVrije Universiteit AmsterdamAmsterdamThe Netherlands
- Department of Brain and CognitionUniversity of AmsterdamAmsterdamThe Netherlands
| | - Christian NL Olivers
- Department of Experimental and Applied PsychologyVrije Universiteit AmsterdamAmsterdamThe Netherlands
- Institute of Brain and Behavior AmsterdamVrije Universiteit AmsterdamAmsterdamThe Netherlands
| |
Collapse
|
49
|
Decoding Brain States for Planning Functional Grasps of Tools: A Functional Magnetic Resonance Imaging Multivoxel Pattern Analysis Study. J Int Neuropsychol Soc 2018; 24:1013-1025. [PMID: 30196800 DOI: 10.1017/s1355617718000590] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/05/2023]
Abstract
OBJECTIVES We used multivoxel pattern analysis (MVPA) to investigate neural selectivity for grasp planning within the left-lateralized temporo-parieto-frontal network of areas (praxis representation network, PRN) typically associated with tool-related actions, as studied with traditional neuroimaging contrasts. METHODS We used data from 20 participants whose task was to plan functional grasps of tools, with either right or left hands. Region of interest and whole-brain searchlight analyses were performed to show task-related neural patterns. RESULTS MVPA revealed significant contributions to functional grasp planning from the anterior intraparietal sulcus (aIPS) and its immediate vicinities, supplemented by inputs from posterior subdivisions of IPS, and the ventral lateral occipital complex (vLOC). Moreover, greater local selectivity was demonstrated in areas near the superior parieto-occipital cortex and dorsal premotor cortex, putatively forming the dorso-dorsal stream. CONCLUSIONS A contribution from aIPS, consistent with its role in prospective grasp formation and/or encoding of relevant tool properties (e.g., potential graspable parts), is likely to accompany the retrieval of manipulation and/or mechanical knowledge subserved by the supramarginal gyrus for achieving action goals. An involvement of vLOC indicates that MVPA is particularly sensitive to coding of object properties, their identities and even functions, for a support of grip formation. Finally, the engagement of the superior parieto-frontal regions as revealed by MVPA is consistent with their selectivity for transient features of tools (i.e., variable affordances) for anticipatory hand postures. These outcomes support the notion that, compared to traditional approaches, MVPA can reveal more fine-grained patterns of neural activity. (JINS, 2018, 24, 1013-1025).
Collapse
|
50
|
Feilong M, Nastase SA, Guntupalli JS, Haxby JV. Reliable individual differences in fine-grained cortical functional architecture. Neuroimage 2018; 183:375-386. [PMID: 30118870 DOI: 10.1016/j.neuroimage.2018.08.029] [Citation(s) in RCA: 39] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2018] [Revised: 08/10/2018] [Accepted: 08/13/2018] [Indexed: 12/29/2022] Open
Abstract
Fine-grained functional organization of cortex is not well-conserved across individuals. As a result, individual differences in cortical functional architecture are confounded by topographic idiosyncrasies-i.e., differences in functional-anatomical correspondence. In this study, we used hyperalignment to align information encoded in topographically variable patterns to study individual differences in fine-grained cortical functional architecture in a common representational space. We characterized the structure of individual differences using three common functional indices, and assessed the reliability of this structure across independent samples of data in a natural vision paradigm. Hyperalignment markedly improved the reliability of individual differences across all three indices by resolving topographic idiosyncrasies and accommodating information encoded in spatially fine-grained response patterns. Our results demonstrate that substantial individual differences in cortical functional architecture exist at fine spatial scales, but are inaccessible with anatomical normalization alone.
Collapse
Affiliation(s)
- Ma Feilong
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| | - Samuel A Nastase
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA; Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - J Swaroop Guntupalli
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA; Vicarious AI, Union City, CA, USA
| | - James V Haxby
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA.
| |
Collapse
|