1
|
Fairhall SL. Sentence-level embeddings reveal dissociable word- and sentence-level cortical representation across coarse- and fine-grained levels of meaning. BRAIN AND LANGUAGE 2024; 250:105389. [PMID: 38306958 DOI: 10.1016/j.bandl.2024.105389] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/31/2023] [Revised: 01/09/2024] [Accepted: 01/26/2024] [Indexed: 02/04/2024]
Abstract
In this large-sample (N = 64) fMRI study, sentence embeddings (text-embedding-ada-002, OpenAI) and representational similarity analysis were used to contrast sentence-level and word-level semantic representation. Overall, sentence-level information resulted in a 20-25 % increase in the model's ability to captures neural representation when compared to word-level only information (word-order scrambled embeddings). This increase was relatively undifferentiated across the cortex. However, when coarse-grained (across thematic category) and fine-grained (within thematic category) combinatorial meaning were separately assessed, word- and sentence-level representations were seen to strongly dissociate across the cortex and to do so differently as a function of grain. Coarse-grained sentence-level representations were evident in occipitotemporal, ventral temporal and medial prefrontal cortex, while fine-grained differences were seen in lateral prefrontal and parietal cortex, middle temporal gyrus, the precuneus, and medial prefrontal cortex. This result indicates dissociable cortical substrates underly single concept versus combinatorial meaning and that different cortical regions specialise for fine- and coarse-grained meaning.
Collapse
Affiliation(s)
- Scott L Fairhall
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Italy.
| |
Collapse
|
2
|
Zhang Y, Wu W, Mirman D, Hoffman P. Representation of event and object concepts in ventral anterior temporal lobe and angular gyrus. Cereb Cortex 2024; 34:bhad519. [PMID: 38185997 PMCID: PMC10839851 DOI: 10.1093/cercor/bhad519] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2023] [Revised: 12/07/2023] [Accepted: 12/08/2023] [Indexed: 01/09/2024] Open
Abstract
Semantic knowledge includes understanding of objects and their features and also understanding of the characteristics of events. The hub-and-spoke theory holds that these conceptual representations rely on multiple information sources that are integrated in a central hub in the ventral anterior temporal lobes. The dual-hub theory expands this framework with the claim that the ventral anterior temporal lobe hub is specialized for object representation, while a second hub in angular gyrus is specialized for event representation. To test these ideas, we used representational similarity analysis, univariate and psychophysiological interaction analyses of fMRI data collected while participants processed object and event concepts (e.g. "an apple," "a wedding") presented as images and written words. Representational similarity analysis showed that angular gyrus encoded event concept similarity more than object similarity, although the left angular gyrus also encoded object similarity. Bilateral ventral anterior temporal lobes encoded both object and event concept structure, and left ventral anterior temporal lobe exhibited stronger coding for events. Psychophysiological interaction analysis revealed greater connectivity between left ventral anterior temporal lobe and right pMTG, and between right angular gyrus and bilateral ITG and middle occipital gyrus, for event concepts compared to object concepts. These findings support the specialization of angular gyrus for event semantics, though with some involvement in object coding, but do not support ventral anterior temporal lobe specialization for object concepts.
Collapse
Affiliation(s)
- Yueyang Zhang
- School of Philosophy, Psychology & Language Sciences, University of Edinburgh, Edinburgh EH8 9JZ, United Kingdom
| | - Wei Wu
- School of Philosophy, Psychology & Language Sciences, University of Edinburgh, Edinburgh EH8 9JZ, United Kingdom
| | - Daniel Mirman
- School of Philosophy, Psychology & Language Sciences, University of Edinburgh, Edinburgh EH8 9JZ, United Kingdom
| | - Paul Hoffman
- School of Philosophy, Psychology & Language Sciences, University of Edinburgh, Edinburgh EH8 9JZ, United Kingdom
| |
Collapse
|
3
|
Huang C, Li A, Pang Y, Yang J, Zhang J, Wu X, Mei L. How the intrinsic functional connectivity patterns of the semantic network support semantic processing. Brain Imaging Behav 2024:10.1007/s11682-024-00849-y. [PMID: 38261218 DOI: 10.1007/s11682-024-00849-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/05/2024] [Indexed: 01/24/2024]
Abstract
Semantic processing, a core of language comprehension, involves the activation of brain regions dispersed extensively across the frontal, temporal, and parietal cortices that compose the semantic network. To comprehend the functional structure of this semantic network and how it prepares for semantic processing, we investigated its intrinsic functional connectivity (FC) and the relation between this pattern and semantic processing ability in a large sample from the Human Connectome Project (HCP) dataset. We first defined a well-studied brain network for semantic processing, and then we characterized the within-network connectivity (WNC) and the between-network connectivity (BNC) within this network using a voxel-based global brain connectivity (GBC) method based on resting-state functional magnetic resonance imaging (fMRI). The results showed that 97.73% of the voxels in the semantic network displayed considerably greater WNC than BNC, demonstrating that the semantic network is a fairly encapsulated network. Moreover, multiple connector hubs in the semantic network were identified after applying the criterion of WNC > 1 SD above the mean WNC of the semantic network. More importantly, three of these connector hubs (i.e., the left anterior temporal lobe, angular gyrus, and orbital part of the inferior frontal gyrus) were reliably associated with semantic processing ability. Our findings suggest that the three identified regions use WNC as the central mechanism for supporting semantic processing and that task-independent spontaneous connectivity in the semantic network is essential for semantic processing.
Collapse
Affiliation(s)
- Chengmei Huang
- Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, 510631, China
- School of Psychology, South China Normal University, Guangzhou, 510631, China
- Center for Studies of Psychological Application, South China Normal University, Guangzhou, 510631, China
| | - Aqian Li
- Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, 510631, China
- School of Psychology, South China Normal University, Guangzhou, 510631, China
- Center for Studies of Psychological Application, South China Normal University, Guangzhou, 510631, China
| | - Yingdan Pang
- Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, 510631, China
- School of Psychology, South China Normal University, Guangzhou, 510631, China
- Center for Studies of Psychological Application, South China Normal University, Guangzhou, 510631, China
| | - Jiayi Yang
- Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, 510631, China
- School of Psychology, South China Normal University, Guangzhou, 510631, China
- Center for Studies of Psychological Application, South China Normal University, Guangzhou, 510631, China
| | - Jingxian Zhang
- Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, 510631, China
- School of Psychology, South China Normal University, Guangzhou, 510631, China
- Center for Studies of Psychological Application, South China Normal University, Guangzhou, 510631, China
| | - Xiaoyan Wu
- Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, 510631, China
- School of Psychology, South China Normal University, Guangzhou, 510631, China
- Center for Studies of Psychological Application, South China Normal University, Guangzhou, 510631, China
| | - Leilei Mei
- Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, 510631, China.
| |
Collapse
|
4
|
Wu H, Zuo Z, Yuan Z, Zhou T, Zhuo Y, Zheng N, Chen B. Neural representation of gestalt grouping and attention effect in human visual cortex. J Neurosci Methods 2023; 399:109980. [PMID: 37783351 DOI: 10.1016/j.jneumeth.2023.109980] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2023] [Revised: 08/29/2023] [Accepted: 09/29/2023] [Indexed: 10/04/2023]
Abstract
BACKGROUND The brain aggregates meaningless local sensory elements to form meaningful global patterns in a process called perceptual grouping. Current brain imaging studies have found that neural activities in V1 are modulated during visual grouping. However, how grouping is represented in each of the early visual areas, and how attention alters these representations, is still unknown. NEW METHOD We adopted MVPA to decode the specific content of perceptual grouping by comparing neural activity patterns between gratings and dot lattice stimuli which can be grouped with proximity law. Furthermore, we quantified the grouping effect by defining the strength of grouping, and assessed the effect of attention on grouping. RESULTS We found that activity patterns to proximity grouped stimuli in early visual areas resemble these to grating stimuli with the same orientations. This similarity exists even when there is no attention focused on the stimuli. The results also showed a progressive increase of representational strength of grouping from V1 to V3, and attention modulation to grouping is only significant in V3 among all the visual areas. COMPARISON WITH EXISTING METHODS Most of the previous work on perceptual grouping has focused on how activity amplitudes are modulated by grouping. Using MVPA, the present work successfully decoded the contents of neural activity patterns corresponding to proximity grouping stimuli, thus shed light on the availability of content-decoding approach in the research on perceptual grouping. CONCLUSIONS Our work found that the content of the neural activity patterns during perceptual grouping can be decoded in the early visual areas under both attended and unattended task, and provide novel evidence that there is a cascade processing for proximity grouping through V1 to V3. The strength of grouping was larger in V3 than in any other visual areas, and the attention modulation to the strength of grouping was only significant in V3 among all the visual areas, implying that V3 plays an important role in proximity grouping.
Collapse
Affiliation(s)
- Hao Wu
- School of Electrical Engineering, Xi'an University of Technology, Xi'an, Shaanxi 710048, China
| | - Zhentao Zuo
- State Key Laboratory of Brain and Cognitive Science, Institute of Biophysics, Chinese Academy of Sciences, Beijing 100101, China; University of the Chinese Academy of Sciences, Chinese Academy of Sciences, Beijing, China.
| | - Zejian Yuan
- National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi'an, Shaanxi 710049, China; Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, China
| | - Tiangang Zhou
- State Key Laboratory of Brain and Cognitive Science, Institute of Biophysics, Chinese Academy of Sciences, Beijing 100101, China; University of the Chinese Academy of Sciences, Chinese Academy of Sciences, Beijing, China
| | - Yan Zhuo
- State Key Laboratory of Brain and Cognitive Science, Institute of Biophysics, Chinese Academy of Sciences, Beijing 100101, China; University of the Chinese Academy of Sciences, Chinese Academy of Sciences, Beijing, China
| | - Nanning Zheng
- National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi'an, Shaanxi 710049, China; Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, China
| | - Badong Chen
- National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi'an, Shaanxi 710049, China; Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, China.
| |
Collapse
|
5
|
Rabini G, Ubaldi S, Fairhall SL. Task-based activation and resting-state connectivity predict individual differences in semantic capacity for complex semantic knowledge. Commun Biol 2023; 6:1020. [PMID: 37813935 PMCID: PMC10562439 DOI: 10.1038/s42003-023-05400-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2022] [Accepted: 09/29/2023] [Indexed: 10/11/2023] Open
Abstract
Our ability to know and access complex factual information has far reaching effects, influencing our scholastic, professional and social lives. Here we employ functional MRI to assess the relationship between individual differences in semantic aptitude in the task-based activation and resting-state functional connectivity. Using psychometric and behavioural measures, we quantified the semantic and executive aptitude of individuals and had them perform a general-knowledge semantic-retrieval task (N = 41) and recorded resting-state data (N = 43). During the semantic-retrieval task, participants accessed general-knowledge facts drawn from four different knowledge-domains (people, places, objects and 'scholastic'). Individuals with greater executive capacity more strongly recruit anterior sections of prefrontal cortex (PFC) and the precuneus, and individuals with lower semantic capacity more strongly activate a posterior section of the dorsomedial PFC (dmPFC). The role of these regions in semantic processing was validated by analysis of independent resting-state data, where increased connectivity between a left anterior PFC and the precuneus predict higher semantic aptitude, and increased connectivity between left anterior PFC and posterior dmPFC predict lower semantic aptitude. Results suggest that coordination between core semantic regions in the precuneus and anterior prefrontal regions associated with executive processes support greater semantic aptitude.
Collapse
Affiliation(s)
- Giuseppe Rabini
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Trento, Italy.
| | - Silvia Ubaldi
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Trento, Italy
| | - Scott L Fairhall
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Trento, Italy
| |
Collapse
|
6
|
Frick A, Besson G, Salmon E, Delhaye E. Perirhinal cortex is associated with fine-grained discrimination of conceptually confusable objects in Alzheimer's disease. Neurobiol Aging 2023; 130:1-11. [PMID: 37419076 DOI: 10.1016/j.neurobiolaging.2023.06.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2022] [Revised: 06/01/2023] [Accepted: 06/03/2023] [Indexed: 07/09/2023]
Abstract
The perirhinal cortex (PrC) stands among the first brain areas to deteriorate in Alzheimer's disease (AD). This study tests to what extent the PrC is involved in representing and discriminating confusable objects based on the conjunction of their perceptual and conceptual features. To this aim, AD patients and control counterparts performed 3 tasks: a naming, a recognition memory, and a conceptual matching task, where we manipulated conceptual and perceptual confusability. A structural MRI of the antero-lateral parahippocampal subregions was obtained for each participant. We found that the sensitivity to conceptual confusability was associated with the left PrC volume in both AD patients and control participants for the recognition memory task, while it was specifically associated with the volume of the left PrC in AD patients for the conceptual matching task. This suggests that a decreased volume of the PrC is related to the ability to disambiguate conceptually confusable items. Therefore, testing recognition memory or conceptual matching of easily conceptually confusable items can provide a potential cognitive marker of PrC atrophy.
Collapse
Affiliation(s)
- Aurélien Frick
- GIGA-CRC In Vivo Imaging, University of Liège, Liège, Belgium; Psychology and Neuroscience of Cognition Research Unit, University of Liège, Liège, Belgium.
| | - Gabriel Besson
- CINEICC, Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
| | - Eric Salmon
- GIGA-CRC In Vivo Imaging, University of Liège, Liège, Belgium
| | - Emma Delhaye
- GIGA-CRC In Vivo Imaging, University of Liège, Liège, Belgium; Psychology and Neuroscience of Cognition Research Unit, University of Liège, Liège, Belgium
| |
Collapse
|
7
|
Liuzzi AG, Meersmans K, Storms G, De Deyne S, Dupont P, Vandenberghe R. Independency of Coding for Affective Similarities and for Word Co-occurrences in Temporal Perisylvian Neocortex. NEUROBIOLOGY OF LANGUAGE (CAMBRIDGE, MASS.) 2023; 4:257-279. [PMID: 37229512 PMCID: PMC10205158 DOI: 10.1162/nol_a_00095] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/26/2022] [Accepted: 12/09/2022] [Indexed: 05/27/2023]
Abstract
Word valence is one of the principal dimensions in the organization of word meaning. Co-occurrence-based similarities calculated by predictive natural language processing models are relatively poor at representing affective content, but very powerful in their own way. Here, we determined how these two canonical but distinct ways of representing word meaning relate to each other in the human brain both functionally and neuroanatomically. We re-analysed an fMRI study of word valence. A co-occurrence-based model was used and the correlation with the similarity of brain activity patterns was compared to that of affective similarities. The correlation between affective and co-occurrence-based similarities was low (r = 0.065), confirming that affect was captured poorly by co-occurrence modelling. In a whole-brain representational similarity analysis, word embedding similarities correlated significantly with the similarity between activity patterns in a region confined to the superior temporal sulcus to the left, and to a lesser degree to the right. Affective word similarities correlated with the similarity in activity patterns in this same region, confirming previous findings. The affective similarity effect extended more widely beyond the superior temporal cortex than the effect of co-occurrence-based similarities did. The effect of co-occurrence-based similarities remained unaltered after partialling out the effect of affective similarities (and vice versa). To conclude, different aspects of word meaning, derived from affective judgements or from word co-occurrences, are represented in superior temporal language cortex in a neuroanatomically overlapping but functionally independent manner.
Collapse
Affiliation(s)
- Antonietta Gabriella Liuzzi
- Laboratory for Cognitive Neurology, Department of Neurosciences, Leuven Brain Institute, KU Leuven, Leuven, Belgium
| | - Karen Meersmans
- Laboratory for Cognitive Neurology, Department of Neurosciences, Leuven Brain Institute, KU Leuven, Leuven, Belgium
| | - Gerrit Storms
- Laboratory of Experimental Psychology, KU Leuven, Leuven, Belgium
| | - Simon De Deyne
- Computational Cognitive Science Lab, University of Melbourne, Melbourne, Australia
| | - Patrick Dupont
- Laboratory for Cognitive Neurology, Department of Neurosciences, Leuven Brain Institute, KU Leuven, Leuven, Belgium
| | - Rik Vandenberghe
- Laboratory for Cognitive Neurology, Department of Neurosciences, Leuven Brain Institute, KU Leuven, Leuven, Belgium
- Neurology Department, University Hospitals Leuven, Leuven, Belgium
| |
Collapse
|
8
|
Delhaye E, Coco MI, Bahri MA, Raposo A. Typicality in the brain during semantic and episodic memory decisions. Neuropsychologia 2023; 184:108529. [PMID: 36898662 DOI: 10.1016/j.neuropsychologia.2023.108529] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2022] [Revised: 11/25/2022] [Accepted: 03/02/2023] [Indexed: 03/11/2023]
Abstract
Concept typicality is a key semantic dimension supporting the categorical organization of items based on their features, such that typical items share more features with other members of their category than atypical items, which are more distinctive. Typicality effects manifest in better accuracy and faster response times during categorization tasks, but higher performance for atypical items in episodic memory tasks, due to their distinctiveness. At a neural level, typicality has been linked to the anterior temporal lobe (ATL) and the inferior frontal gyrus (IFG) in semantic decision tasks, but patterns of brain activity during episodic memory tasks remain to be understood. We investigated the neural correlates of typicality in semantic and episodic memory to determine the brain regions associated with semantic typicality and uncover effects arising when items are reinstated during retrieval. In an fMRI study, 26 healthy young subjects first performed a category verification task on words representing typical and atypical concepts (encoding), and then completed a recognition memory task (retrieval). In line with previous literature, we observed higher accuracy and faster response times for typical items in the category verification task, while atypical items were better recognized in the episodic memory task. During category verification, univariate analyses revealed a greater involvement of the angular gyrus for typical items and the inferior frontal gyrus for atypical items. During the correct recognition of old items, regions belonging to the core recollection network were activated. We then compared the similarity of the representations from encoding to retrieval (ERS) using Representation Similarity Analyses. Results showed that typical items were reinstated more than atypical ones in several regions including the left precuneus and left anterior temporal lobe (ATL). This suggests that the correct retrieval of typical items requires finer-grained processing, evidenced by greater item-specific reinstatement, which is needed to resolve their confusability with other members of the category due to their higher feature similarity. Our findings confirm the centrality of the ATL in the processing of typicality while extending it to memory retrieval.
Collapse
Affiliation(s)
- Emma Delhaye
- GIGA-CRC IVI, Liege University, Belgium; CICPSI, Faculdade de Psicologia, Universidade de Lisboa, Portugal.
| | - Moreno I Coco
- CICPSI, Faculdade de Psicologia, Universidade de Lisboa, Portugal; Department of Psychology, Sapienza, University of Rome, Italy; IRCCS Santa Lucia, Rome, Italy
| | | | - Ana Raposo
- CICPSI, Faculdade de Psicologia, Universidade de Lisboa, Portugal
| |
Collapse
|
9
|
Hehl M, Swinnen SP, Van Malderen S, Cuypers K. No evidence for a difference in lateralization and distinctiveness level of transcranial magnetic stimulation-derived cortical motor representations over the adult lifespan. Front Aging Neurosci 2022; 14:971858. [PMID: 36313026 PMCID: PMC9608504 DOI: 10.3389/fnagi.2022.971858] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2022] [Accepted: 09/15/2022] [Indexed: 11/30/2022] Open
Abstract
This study aimed to investigate the presence and patterns of age-related differences in TMS-based measures of lateralization and distinctiveness of the cortical motor representations of two different hand muscles. In a sample of seventy-three right-handed healthy participants over the adult lifespan, the first dorsal interosseus (FDI) and abductor digiti minimi (ADM) cortical motor representations of both hemispheres were acquired using transcranial magnetic stimulation (TMS). In addition, dexterity and maximum force levels were measured. Lateralization quotients were calculated for homolog behavioral and TMS measures, whereas the distinctiveness between the FDI and ADM representation within one hemisphere was quantified by the center of gravity (CoG) distance and cosine similarity. The presence and patterns of age-related changes were examined using linear, polynomial, and piecewise linear regression. No age-related differences could be identified for the lateralization quotient of behavior or cortical motor representations of both intrinsic hand muscles. Furthermore, no evidence for a change in the distinctiveness of the FDI and ADM representation with advancing age was found. In conclusion this work showed that lateralization and distinctiveness of cortical motor representations, as determined by means of TMS-based measures, remain stable over the adult lifespan.
Collapse
Affiliation(s)
- Melina Hehl
- Movement Control & Neuroplasticity Research Group, Department of Movement Sciences, Group Biomedical Sciences, KU Leuven, Heverlee, Belgium
- Neuroplasticity and Movement Control Research Group, Rehabilitation Research Institute (REVAL), Hasselt University, Diepenbeek, Belgium
| | - Stephan P. Swinnen
- Movement Control & Neuroplasticity Research Group, Department of Movement Sciences, Group Biomedical Sciences, KU Leuven, Heverlee, Belgium
- Leuven Brain Institute (LBI), KU Leuven, Leuven, Belgium
| | - Shanti Van Malderen
- Movement Control & Neuroplasticity Research Group, Department of Movement Sciences, Group Biomedical Sciences, KU Leuven, Heverlee, Belgium
- Neuroplasticity and Movement Control Research Group, Rehabilitation Research Institute (REVAL), Hasselt University, Diepenbeek, Belgium
| | - Koen Cuypers
- Movement Control & Neuroplasticity Research Group, Department of Movement Sciences, Group Biomedical Sciences, KU Leuven, Heverlee, Belgium
- Neuroplasticity and Movement Control Research Group, Rehabilitation Research Institute (REVAL), Hasselt University, Diepenbeek, Belgium
- Leuven Brain Institute (LBI), KU Leuven, Leuven, Belgium
- *Correspondence: Koen Cuypers,
| |
Collapse
|
10
|
Tong J, Binder JR, Humphries C, Mazurchuk S, Conant LL, Fernandino L. A Distributed Network for Multimodal Experiential Representation of Concepts. J Neurosci 2022; 42:7121-7130. [PMID: 35940877 PMCID: PMC9480893 DOI: 10.1523/jneurosci.1243-21.2022] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2021] [Revised: 07/11/2022] [Accepted: 07/13/2022] [Indexed: 11/21/2022] Open
Abstract
Neuroimaging, neuropsychological, and psychophysical evidence indicate that concept retrieval selectively engages specific sensory and motor brain systems involved in the acquisition of the retrieved concept. However, it remains unclear which supramodal cortical regions contribute to this process and what kind of information they represent. Here, we used representational similarity analysis of two large fMRI datasets with a searchlight approach to generate a detailed map of human brain regions where the semantic similarity structure across individual lexical concepts can be reliably detected. We hypothesized that heteromodal cortical areas typically associated with the default mode network encode multimodal experiential information about concepts, consistent with their proposed role as cortical integration hubs. In two studies involving different sets of concepts and different participants (both sexes), we found a distributed, bihemispheric network engaged in concept representation, composed of high-level association areas in the anterior, lateral, and ventral temporal lobe; inferior parietal lobule; posterior cingulate gyrus and precuneus; and medial, dorsal, ventrolateral, and orbital prefrontal cortex. In both studies, a multimodal model combining sensory, motor, affective, and other types of experiential information explained significant variance in the neural similarity structure observed in these regions that was not explained by unimodal experiential models or by distributional semantics (i.e., word2vec similarity). These results indicate that during concept retrieval, lexical concepts are represented across a vast expanse of high-level cortical regions, especially in the areas that make up the default mode network, and that these regions encode multimodal experiential information.SIGNIFICANCE STATEMENT Conceptual knowledge includes information acquired through various modalities of experience, such as visual, auditory, tactile, and emotional information. We investigated which brain regions encode mental representations that combine information from multiple modalities when participants think about the meaning of a word. We found that such representations are encoded across a widely distributed network of cortical areas in both hemispheres, including temporal, parietal, limbic, and prefrontal association areas. Several areas not traditionally associated with semantic cognition were also implicated. Our results indicate that the retrieval of conceptual knowledge during word comprehension relies on a much larger portion of the cerebral cortex than previously thought and that multimodal experiential information is represented throughout the entire network.
Collapse
Affiliation(s)
- Jiaqing Tong
- Departments of Neurology
- Biophysics, Medical College of Wisconsin, Milwaukee, Wisconsin 53226
| | - Jeffrey R Binder
- Departments of Neurology
- Biophysics, Medical College of Wisconsin, Milwaukee, Wisconsin 53226
| | | | - Stephen Mazurchuk
- Departments of Neurology
- Biophysics, Medical College of Wisconsin, Milwaukee, Wisconsin 53226
| | | | - Leonardo Fernandino
- Departments of Neurology
- Department of Biomedical Engineering, Marquette University and Medical College of Wisconsin, Milwaukee, Wisconsin 53233
| |
Collapse
|
11
|
Leonardelli E, Fairhall SL. Similarity-based fMRI-MEG fusion reveals hierarchical organisation within the brain's semantic system. Neuroimage 2022; 259:119405. [PMID: 35752412 DOI: 10.1016/j.neuroimage.2022.119405] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2021] [Revised: 06/11/2022] [Accepted: 06/21/2022] [Indexed: 01/17/2023] Open
Abstract
Our ability to understand and interact with our environment relies upon conceptual knowledge of the meaning of objects. This process is supported by a distributed network of frontal, parietal, and temporal brain regions. Insight into the differential roles of various elements of this system can be inferred from the timing of activation, and here we use similarity-based fMRI-MEG fusion to understand when the representational spaces in different elements of the semantic system converge with representational spaces in the evolving MEG signal. Participants performed a semantic-typicality judgment of written words drawn from nine different semantic categories in separate fMRI and MEG sessions. Results indicate an initial period of congruence between MEG and fMRI informational spaces dominated by the posterior inferior temporal gyrus and the ventral temporal cortex between 350-450 msec. This is followed by a second period of convergence between 450-795 msec where MEG and fMRI representational spaces conform in left angular gyrus and precuneus in addition to ventral temporal cortex. Results are consistent with the multistage recruitment of the semantic system, initially involving automatic aspects of the representational system and later extending to broader elements of the semantic system more strongly associated with internalised cognition.
Collapse
|
12
|
Topologic Reorganization of White Matter Connectivity Networks in Early-Blind Adolescents. Neural Plast 2022; 2022:8034757. [PMID: 35529452 PMCID: PMC9072039 DOI: 10.1155/2022/8034757] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2021] [Revised: 07/28/2021] [Accepted: 04/05/2022] [Indexed: 11/18/2022] Open
Abstract
Blindness studies are important models for the comprehension of human brain development and reorganization, after visual deprivation early in life. To investigate the global and local topologic alterations and to identify specific reorganized neural patterns in early-blind adolescents (EBAs), we applied diffusion tensor tractography and graph theory to establish and analyze the white matter connectivity networks in 21 EBAs and 22 age- and sex-matched normal-sighted controls (NSCs). The network profiles were compared between the groups using a linear regression model, and the associations between clinical variables and network profiles were analyzed. Graph theory analysis revealed “small-world” attributes in the structural connection networks of both EBA and NSC cohorts. The EBA cohort exhibited significant lower network density and global and local efficiency, as well as significantly elevated shortest path length, compared to the NSC group. The network efficiencies were markedly reduced in the EBA cohort, with the largest alterations in the default-mode, visual, and limbic areas. Moreover, decreased regional efficiency and increased nodal path length in some visual and default-mode areas were strongly associated with the period of blindness in EBA cohort, suggesting that the function of these areas would gradually weaken in the early-blind brains. Additionally, the differences in hub distribution between the two groups were mainly within the occipital and frontal areas, suggesting that neural reorganization occurred in these brain regions after early visual deprivation during adolescence. This study revealed that the EBA brain structural network undergoes both convergent and divergent topologic reorganizations to circumvent early visual deprivation. Our research will add to the growing knowledge of underlying neural mechanisms that govern brain reorganization and development, under conditions of early visual deprivation.
Collapse
|
13
|
Acunzo DJ, Low DM, Fairhall SL. Deep neural networks reveal topic-level representations of sentences in medial prefrontal cortex, lateral anterior temporal lobe, precuneus, and angular gyrus. Neuroimage 2022; 251:119005. [PMID: 35176493 PMCID: PMC10184870 DOI: 10.1016/j.neuroimage.2022.119005] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2021] [Revised: 02/11/2022] [Accepted: 02/13/2022] [Indexed: 11/17/2022] Open
Abstract
When reading a sentence, individual words can be combined to create more complex meaning. In this study, we sought to uncover brain regions that reflect the representation of the meaning of sentences at the topic level, as opposed to the meaning of their individual constituent words when considered irrespective of their context. Using fMRI, we recorded the neural activity of participants while reading sentences. We constructed a topic-level sentence representations using the final layer of a convolutional neural network (CNN) trained to classify Wikipedia sentences into broad semantic categories. This model was contrasted with word-level sentence representations constructed using the average of the word embeddings constituting the sentence. Using representational similarity analysis, we found that the medial prefrontal cortex, lateral anterior temporal lobe, precuneus, and angular gyrus more strongly represent sentence topic-level, compared to word-level, meaning, uncovering the important role of these semantic system regions in the representation of topic-level meaning. Results were comparable when sentence meaning was modelled with a multilayer perceptron that was not sensitive to word order within a sentence, suggesting that the learning objective, in the terms of the topic being modelled, is the critical factor in capturing these neural representational spaces.
Collapse
Affiliation(s)
- David J Acunzo
- CIMeC/University of Trento, Corso Bettini 31, Rovereto 38068, Italy
| | - Daniel M Low
- Program in Speech and Hearing Bioscience and Technology, Harvard Medical School, United States; Brain and Cognitive Sciences Department, MIT, United States
| | - Scott L Fairhall
- CIMeC/University of Trento, Corso Bettini 31, Rovereto 38068, Italy.
| |
Collapse
|
14
|
Ferko KM, Blumenthal A, Martin CB, Proklova D, Minos AN, Saksida LM, Bussey TJ, Khan AR, Köhler S. Activity in perirhinal and entorhinal cortex predicts perceived visual similarities among category exemplars with highest precision. eLife 2022; 11:66884. [PMID: 35311645 PMCID: PMC9020819 DOI: 10.7554/elife.66884] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2021] [Accepted: 03/17/2022] [Indexed: 01/22/2023] Open
Abstract
Vision neuroscience has made great strides in understanding the hierarchical organization of object representations along the ventral visual stream (VVS). How VVS representations capture fine-grained visual similarities between objects that observers subjectively perceive has received limited examination so far. In the current study, we addressed this question by focussing on perceived visual similarities among subordinate exemplars of real-world categories. We hypothesized that these perceived similarities are reflected with highest fidelity in neural activity patterns downstream from inferotemporal regions, namely in perirhinal (PrC) and anterolateral entorhinal cortex (alErC) in the medial temporal lobe. To address this issue with functional magnetic resonance imaging (fMRI), we administered a modified 1-back task that required discrimination between category exemplars as well as categorization. Further, we obtained observer-specific ratings of perceived visual similarities, which predicted behavioural discrimination performance during scanning. As anticipated, we found that activity patterns in PrC and alErC predicted the structure of perceived visual similarity relationships among category exemplars, including its observer-specific component, with higher precision than any other VVS region. Our findings provide new evidence that subjective aspects of object perception that rely on fine-grained visual differentiation are reflected with highest fidelity in the medial temporal lobe.
Collapse
Affiliation(s)
- Kayla M Ferko
- Brain and Mind Institute, University of Western Ontario, London, Canada.,Robarts Research Institute Schulich School of Medicine and Dentistry, University of Western Ontario, London, Canada
| | - Anna Blumenthal
- Brain and Mind Institute, University of Western Ontario, London, Canada.,Cervo Brain Research Center, University of Laval, Quebec, Canada
| | - Chris B Martin
- Department of Psychology, Florida State University, Tallahassee, United States
| | - Daria Proklova
- Brain and Mind Institute, University of Western Ontario, London, Canada
| | - Alexander N Minos
- Brain and Mind Institute, University of Western Ontario, London, Canada
| | - Lisa M Saksida
- Brain and Mind Institute, University of Western Ontario, London, Canada.,Robarts Research Institute Schulich School of Medicine and Dentistry, University of Western Ontario, London, Canada.,Department of Physiology and Pharmacology, University of Western Ontario, London, Canada
| | - Timothy J Bussey
- Brain and Mind Institute, University of Western Ontario, London, Canada.,Robarts Research Institute Schulich School of Medicine and Dentistry, University of Western Ontario, London, Canada.,Department of Physiology and Pharmacology, University of Western Ontario, London, Canada
| | - Ali R Khan
- Brain and Mind Institute, University of Western Ontario, London, Canada.,Robarts Research Institute Schulich School of Medicine and Dentistry, University of Western Ontario, London, Canada.,School of Biomedical Engineering, University of Western Ontario, London, Canada.,Department of Medical Biophysics, University of Western Ontario, London, Canada
| | - Stefan Köhler
- Brain and Mind Institute, University of Western Ontario, London, Canada.,Department of Psychology, University of Western Ontario, London, Canada
| |
Collapse
|
15
|
Riegel M, Wierzba M, Wypych M, Ritchey M, Jednoróg K, Grabowska A, Vuilleumier P, Marchewka A. Distinct medial-tempora lobe mechanisms of encoding and amygdala-mediated memory reinstatement for disgust and fear. Neuroimage 2022; 251:118889. [PMID: 35065268 DOI: 10.1016/j.neuroimage.2022.118889] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2021] [Revised: 12/22/2021] [Accepted: 01/06/2022] [Indexed: 11/29/2022] Open
Abstract
Current models of episodic memory posit that retrieval involves the reenactment of encoding processes. Recent evidence has shown that this reinstatement process - indexed by subsequent encoding-retrieval similarity of brain activity patterns - is related to the activity in the hippocampus during encoding. However, we tend to re-experience emotional events in memory more richly than dull events. The role of amygdala - a critical hub of emotion processing - in reinstatement of emotional events was poorly understood. To investigate it, we leveraged a previously overlooked divergence in the role of amygdala in memory modulation by distinct emotions - disgust and fear. Here we used a novel paradigm in which participants encoded complex events (word pairs) and their memory was tested after 3 weeks, both phases during fMRI scanning. Using representational similarity analysis and univariate analyses, we show that the strength of amygdala activation during encoding was correlated with memory reinstatement of individual event representations in emotion-specific regions. Critically, amygdala modulated reinstatement more for disgust than fear. This was in line with other differences observed at the level of memory performance and neural mechanisms of encoding. Specifically, amygdala and perirhinal cortex were more involved during encoding of disgust-related events, whereas hippocampus and parahippocampal gyrus during encoding of fear-related events. Together, these findings shed a new light on the role of the amygdala and medial temporal lobe regions in encoding and reinstatement of specific emotional memories.
Collapse
Affiliation(s)
- Monika Riegel
- Laboratory of Brain Imaging, Nencki Institute of Experimental Biology of Polish Academy of Sciences, Warsaw 02-093, Poland; Department of Psychology, Columbia University, New York 10027, United States of America; Centre interfacultaire de gérontologie et d'études des vulnerabilities, University of Geneva, CH-Geneva 1211, Switzerland.
| | - Małgorzata Wierzba
- Laboratory of Brain Imaging, Nencki Institute of Experimental Biology of Polish Academy of Sciences, Warsaw 02-093, Poland
| | - Marek Wypych
- Laboratory of Brain Imaging, Nencki Institute of Experimental Biology of Polish Academy of Sciences, Warsaw 02-093, Poland
| | - Maureen Ritchey
- Department of Psychology and Neuroscience, Boston College, Chestnut Hill, MA 02467, United States of America
| | - Katarzyna Jednoróg
- Laboratory of Brain Imaging, Nencki Institute of Experimental Biology of Polish Academy of Sciences, Warsaw 02-093, Poland
| | - Anna Grabowska
- SWPS University of Social Sciences and Humanities, Warsaw 03-815, Poland
| | - Patrik Vuilleumier
- Department of Neuroscience, University Medical Center, Geneva CH-1211, Switzerland; Swiss Center for Affective Sciences, University of Geneva, Campus Biotech, CH-Geneva 1211, Switzerland; Geneva Neuroscience Center, University of Geneva, Geneva CH-1211, Switzerland
| | - Artur Marchewka
- Laboratory of Brain Imaging, Nencki Institute of Experimental Biology of Polish Academy of Sciences, Warsaw 02-093, Poland
| |
Collapse
|
16
|
Rogers TT, Cox CR, Lu Q, Shimotake A, Kikuchi T, Kunieda T, Miyamoto S, Takahashi R, Ikeda A, Matsumoto R, Lambon Ralph MA. Evidence for a deep, distributed and dynamic code for animacy in human ventral anterior temporal cortex. eLife 2021; 10:66276. [PMID: 34704935 PMCID: PMC8550752 DOI: 10.7554/elife.66276] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2021] [Accepted: 10/09/2021] [Indexed: 12/01/2022] Open
Abstract
How does the human brain encode semantic information about objects? This paper reconciles two seemingly contradictory views. The first proposes that local neural populations independently encode semantic features; the second, that semantic representations arise as a dynamic distributed code that changes radically with stimulus processing. Combining simulations with a well-known neural network model of semantic memory, multivariate pattern classification, and human electrocorticography, we find that both views are partially correct: information about the animacy of a depicted stimulus is distributed across ventral temporal cortex in a dynamic code possessing feature-like elements posteriorly but with elements that change rapidly and nonlinearly in anterior regions. This pattern is consistent with the view that anterior temporal lobes serve as a deep cross-modal ‘hub’ in an interactive semantic network, and more generally suggests that tertiary association cortices may adopt dynamic distributed codes difficult to detect with common brain imaging methods.
Collapse
Affiliation(s)
- Timothy T Rogers
- Department of Psychology, University of Wisconsin- Madison, Madison, United States
| | - Christopher R Cox
- Department of Psychology, Louisiana State University, Baton Rouge, United States
| | - Qihong Lu
- Department of Psychology, Princeton University, Princeton, United States
| | - Akihiro Shimotake
- Department of Neurology, Kyoto University Graduate School of Medicine, Kyoto, Japan
| | - Takayuki Kikuchi
- Department of Neurosurgery, Kyoto University Graduate School of Medicine, Kyoto, Japan
| | - Takeharu Kunieda
- Department of Neurosurgery, Kyoto University Graduate School of Medicine, Kyoto, Japan.,Department of Neurosurgery, Ehime University Graduate School of Medicine, Ehime, Japan
| | - Susumu Miyamoto
- Department of Neurosurgery, Kyoto University Graduate School of Medicine, Kyoto, Japan
| | - Ryosuke Takahashi
- Department of Neurology, Kyoto University Graduate School of Medicine, Kyoto, Japan
| | - Akio Ikeda
- Department of Epilepsy, Movement Disorders and Physiology, Kyoto University Graduate School ofMedicine, Kyoto, Japan
| | - Riki Matsumoto
- Department of Neurology, Kyoto University Graduate School of Medicine, Kyoto, Japan.,Division of Neurology, Kobe University Graduate School of Medicine, Kusunoki-cho, Kobe, Japan
| | - Matthew A Lambon Ralph
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, United Kingdom
| |
Collapse
|
17
|
Wurm MF, Caramazza A. Two 'what' pathways for action and object recognition. Trends Cogn Sci 2021; 26:103-116. [PMID: 34702661 DOI: 10.1016/j.tics.2021.10.003] [Citation(s) in RCA: 38] [Impact Index Per Article: 12.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2021] [Revised: 09/03/2021] [Accepted: 10/01/2021] [Indexed: 10/20/2022]
Abstract
The ventral visual stream is conceived as a pathway for object recognition. However, we also recognize the actions an object can be involved in. Here, we show that action recognition critically depends on a pathway in lateral occipitotemporal cortex, partially overlapping and topographically aligned with object representations that are precursors for action recognition. By contrast, object features that are more relevant for object recognition, such as color and texture, are typically found in ventral occipitotemporal cortex. We argue that occipitotemporal cortex contains similarly organized lateral and ventral 'what' pathways for action and object recognition, respectively. This account explains a number of observed phenomena, such as the duplication of object domains and the specific representational profiles in lateral and ventral cortex.
Collapse
Affiliation(s)
- Moritz F Wurm
- Center for Mind/Brain Sciences - CIMeC, University of Trento, Corso Bettini 31, 38068 Rovereto, Italy.
| | - Alfonso Caramazza
- Center for Mind/Brain Sciences - CIMeC, University of Trento, Corso Bettini 31, 38068 Rovereto, Italy; Department of Psychology, Harvard University, 33 Kirkland St, Cambridge, MA 02138, USA
| |
Collapse
|
18
|
Zaiser AK, Bader R, Meyer P. High feature overlap reveals the importance of anterior and medial temporal lobe structures for learning by means of fast mapping. Cortex 2021; 146:74-88. [PMID: 34839220 DOI: 10.1016/j.cortex.2021.07.017] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2021] [Revised: 07/15/2021] [Accepted: 07/18/2021] [Indexed: 11/03/2022]
Abstract
Contrary to traditional theories of declarative memory, it has recently been shown that novel, arbitrary associations can rapidly and directly be integrated into cortical memory networks by means of a learning procedure called fast mapping (FM), possibly bypassing time-consuming hippocampal-neocortical consolidation processes. In the typical FM paradigm, a picture of a previously unknown item is presented next to a picture of a previously known item and participants answer a question referring to an unfamiliar label, thereby incidentally creating associations between the unknown item and the label. However, contradictory findings have been reported and factors moderating rapid cortical integration through FM yet need to be identified. Previous behavioral results showed that rapid semantic integration through FM was boosted if the unknown and the known item shared many features. In light of this, we propose that the perirhinal cortex might be especially qualified to support the rapid incorporation of these associations into cortical memory networks within the FM paradigm, due to its computational mechanisms during the processing of complex and particularly highly similar objects. We therefore expected that a high degree of feature overlap between the unknown and the known item would trigger strong engagement of the perirhinal cortex at encoding, which in turn might enhance rapid cortical integration of the novel picture-label associations. Within an fMRI experiment, we observed greater subsequent memory effects (i.e., stronger activation for subsequent hits than misses) during encoding in the perirhinal cortex and an associated anterior temporal network if the items shared many features than if they shared few features. This indicates that the perirhinal cortex indeed contributes to the acquisition of novel associations by means of FM if feature overlap is high.
Collapse
Affiliation(s)
- Ann-Kathrin Zaiser
- Experimental Neuropsychology Unit, Department of Psychology, Saarland University, Saarbrücken, Germany; Department of Applied Psychology, SRH University Heidelberg, Germany.
| | - Regine Bader
- Experimental Neuropsychology Unit, Department of Psychology, Saarland University, Saarbrücken, Germany
| | - Patric Meyer
- Department of Applied Psychology, SRH University Heidelberg, Germany; Network Aging Research (NAR), Heidelberg University, Heidelberg, Germany
| |
Collapse
|
19
|
Perceptual and Semantic Representations at Encoding Contribute to True and False Recognition of Objects. J Neurosci 2021; 41:8375-8389. [PMID: 34413205 DOI: 10.1523/jneurosci.0677-21.2021] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2021] [Revised: 06/30/2021] [Accepted: 07/28/2021] [Indexed: 11/21/2022] Open
Abstract
When encoding new episodic memories, visual and semantic processing is proposed to make distinct contributions to accurate memory and memory distortions. Here, we used fMRI and preregistered representational similarity analysis to uncover the representations that predict true and false recognition of unfamiliar objects. Two semantic models captured coarse-grained taxonomic categories and specific object features, respectively, while two perceptual models embodied low-level visual properties. Twenty-eight female and male participants encoded images of objects during fMRI scanning, and later had to discriminate studied objects from similar lures and novel objects in a recognition memory test. Both perceptual and semantic models predicted true memory. When studied objects were later identified correctly, neural patterns corresponded to low-level visual representations of these object images in the early visual cortex, lingual, and fusiform gyri. In a similar fashion, alignment of neural patterns with fine-grained semantic feature representations in the fusiform gyrus also predicted true recognition. However, emphasis on coarser taxonomic representations predicted forgetting more anteriorly in the anterior ventral temporal cortex, left inferior frontal gyrus and, in an exploratory analysis, left perirhinal cortex. In contrast, false recognition of similar lure objects was associated with weaker visual analysis posteriorly in early visual and left occipitotemporal cortex. The results implicate multiple perceptual and semantic representations in successful memory encoding and suggest that fine-grained semantic as well as visual analysis contributes to accurate later recognition, while processing visual image detail is critical for avoiding false recognition errors.SIGNIFICANCE STATEMENT People are able to store detailed memories of many similar objects. We offer new insights into the encoding of these specific memories by combining fMRI with explicit models of how image properties and object knowledge are represented in the brain. When people processed fine-grained visual properties in occipital and posterior temporal cortex, they were more likely to recognize the objects later and less likely to falsely recognize similar objects. In contrast, while object-specific feature representations in fusiform gyrus predicted accurate memory, coarse-grained categorical representations in frontal and temporal regions predicted forgetting. The data provide the first direct tests of theoretical assumptions about encoding true and false memories, suggesting that semantic representations contribute to specific memories as well as errors.
Collapse
|
20
|
Henderson SK, Dev SI, Ezzo R, Quimby M, Wong B, Brickhouse M, Hochberg D, Touroutoglou A, Dickerson BC, Cordella C, Collins JA. A category-selective semantic memory deficit for animate objects in semantic variant primary progressive aphasia. Brain Commun 2021; 3:fcab210. [PMID: 34622208 PMCID: PMC8493104 DOI: 10.1093/braincomms/fcab210] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2021] [Revised: 07/16/2021] [Accepted: 07/26/2021] [Indexed: 11/13/2022] Open
Abstract
Data are mixed on whether patients with semantic variant primary progressive aphasia exhibit a category-selective semantic deficit for animate objects. Moreover, there is little consensus regarding the neural substrates of this category-selective semantic deficit, though prior literature has suggested that the perirhinal cortex and the lateral posterior fusiform gyrus may support semantic memory functions important for processing animate objects. In this study, we investigated whether patients with semantic variant primary progressive aphasia exhibited a category-selective semantic deficit for animate objects in a word-picture matching task, controlling for psycholinguistic features of the stimuli, including frequency, familiarity, typicality and age of acquisition. We investigated the neural bases of this category selectivity by examining its relationship with cortical atrophy in two primary regions of interest: bilateral perirhinal cortex and lateral posterior fusiform gyri. We analysed data from 20 patients with semantic variant primary progressive aphasia (mean age = 64 years, S.D. = 6.94). For each participant, we calculated an animacy index score to denote the magnitude of the category-selective semantic deficit for animate objects. Multivariate regression analysis revealed a main effect of animacy (β = 0.52, t = 4.03, P < 0.001) even after including all psycholinguistic variables in the model, such that animate objects were less likely to be identified correctly relative to inanimate objects. Inspection of each individual patient's data indicated the presence of a disproportionate impairment in animate objects in most patients. A linear regression analysis revealed a relationship between the right perirhinal cortex thickness and animacy index scores (β = -0.57, t = -2.74, P = 0.015) such that patients who were more disproportionally impaired for animate relative to inanimate objects exhibited thinner right perirhinal cortex. A vertex-wise general linear model analysis restricted to the temporal lobes revealed additional associations between positive animacy index scores (i.e. a disproportionately poorer performance on animate objects) and cortical atrophy in the right perirhinal and entorhinal cortex, superior, middle, and inferior temporal gyri, and the anterior fusiform gyrus, as well as the left anterior fusiform gyrus. Taken together, our results indicate that a category-selective semantic deficit for animate objects is a characteristic feature of semantic variant primary progressive aphasia that is detectable in most individuals. Our imaging findings provide further support for the role of the right perirhinal cortex and other temporal lobe regions in the semantic processing of animate objects.
Collapse
Affiliation(s)
- Shalom K Henderson
- Frontotemporal Disorders Unit and Alzheimer’s Disease Research Center, Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| | - Sheena I Dev
- Frontotemporal Disorders Unit and Alzheimer’s Disease Research Center, Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| | - Rania Ezzo
- Frontotemporal Disorders Unit and Alzheimer’s Disease Research Center, Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| | - Megan Quimby
- Frontotemporal Disorders Unit and Alzheimer’s Disease Research Center, Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| | - Bonnie Wong
- Frontotemporal Disorders Unit and Alzheimer’s Disease Research Center, Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| | - Michael Brickhouse
- Frontotemporal Disorders Unit and Alzheimer’s Disease Research Center, Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| | - Daisy Hochberg
- Frontotemporal Disorders Unit and Alzheimer’s Disease Research Center, Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| | - Alexandra Touroutoglou
- Frontotemporal Disorders Unit and Alzheimer’s Disease Research Center, Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA, USA
| | - Bradford C Dickerson
- Frontotemporal Disorders Unit and Alzheimer’s Disease Research Center, Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA, USA
| | - Claire Cordella
- Frontotemporal Disorders Unit and Alzheimer’s Disease Research Center, Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| | - Jessica A Collins
- Frontotemporal Disorders Unit and Alzheimer’s Disease Research Center, Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| |
Collapse
|
21
|
Jamoulle T, Ran Q, Meersmans K, Schaeverbeke J, Dupont P, Vandenberghe R. Posterior Intraparietal Sulcus Mediates Detection of Salient Stimuli Outside the Endogenous Focus of Attention. Cereb Cortex 2021; 32:1455-1469. [PMID: 34467392 PMCID: PMC8971085 DOI: 10.1093/cercor/bhab299] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2021] [Revised: 07/16/2021] [Accepted: 07/21/2021] [Indexed: 11/21/2022] Open
Abstract
Visual consciousness is shaped by the interplay between endogenous selection and exogenous capture. If stimulus saliency is aligned with a subject’s attentional priorities, endogenous selection will be facilitated. In case of a misalignment, endogenous selection may be compromised as attentional capture is a strong and automatic process. We manipulated task-congruent versus -incongruent saliency in a functional magnetic resonance imaging change-detection task and analyzed brain activity patterns in the cortex surrounding the intraparietal sulcus (IPS) within the Julich-Brain probabilistic cytoarchitectonic mapping reference frame. We predicted that exogenous effects would be seen mainly in the posterior regions of the IPS (hIP4–hIP7–hIP8), whereas a conflict between endogenous and exogenous orienting would elicit activity from more anterior cytoarchitectonic areas (hIP1–hIP2–hIP3). Contrary to our hypothesis, a conflict between endogenous and exogenous orienting had an effect early in the IPS (mainly in hIP7 and hIP8). This is strong evidence for an endogenous component in hIP7/8 responses to salient stimuli beyond effects of attentional bottom-up sweep. Our results suggest that hIP7 and hIP8 are implicated in the individuation of attended locations based on saliency as well as endogenous instructions.
Collapse
Affiliation(s)
- Tarik Jamoulle
- Laboratory for Cognitive Neurology, Department of Neurosciences, KU Leuven, Leuven, Belgium
| | - Qian Ran
- Laboratory for Cognitive Neurology, Department of Neurosciences, KU Leuven, Leuven, Belgium
| | - Karen Meersmans
- Laboratory for Cognitive Neurology, Department of Neurosciences, KU Leuven, Leuven, Belgium
| | - Jolien Schaeverbeke
- Laboratory for Cognitive Neurology, Department of Neurosciences, KU Leuven, Leuven, Belgium
| | - Patrick Dupont
- Laboratory for Cognitive Neurology, Department of Neurosciences, KU Leuven, Leuven, Belgium
| | - Rik Vandenberghe
- Laboratory for Cognitive Neurology, Department of Neurosciences, KU Leuven, Leuven, Belgium.,Neurology Department, University Hospitals Leuven, Leuven, Belgium
| |
Collapse
|
22
|
Rabini G, Ubaldi S, Fairhall SL. Combining concepts across categorical domains: a linking role of the precuneus. NEUROBIOLOGY OF LANGUAGE (CAMBRIDGE, MASS.) 2021; 2:354-371. [PMID: 34595480 PMCID: PMC7611750 DOI: 10.1162/nol_a_00039] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/05/2023]
Abstract
The human capacity for semantic knowledge entails not only the representation of single concepts but the capacity to combine these concepts into the increasingly complex ideas that underlie human thought. This process involves not only the combination of concepts from within the same semantic category but frequently the conceptual combination across semantic domains. In this fMRI study (N=24) we investigate the cortical mechanisms underlying our ability to combine concepts across different semantic domains. Using five different semantic domains (People, Places, Food, Objects and Animals), we present sentences depicting concepts drawn from a single semantic domain as well as sentences that combine concepts from two of these domains. Contrasting single-category and combined-category sentences reveals that the precuneus is more active when concepts from different domains have to be combined. At the same time, we observe that distributed category selectivity representations persist when higher-order meaning involves the combination of categories and that this category-selective response is captured by the combination of the single categories composing the sentence. Collectively, these results suggest that the combination of concepts across different semantic domains is mediated by the precuneus, which functions to link together category-selective representations distributed across the cortex.
Collapse
Affiliation(s)
- Giuseppe Rabini
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Italy
| | - Silvia Ubaldi
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Italy
| | | |
Collapse
|
23
|
Liuzzi AG, Ubaldi S, Fairhall SL. Representations of conceptual information during automatic and active semantic access. Neuropsychologia 2021; 160:107953. [PMID: 34252416 DOI: 10.1016/j.neuropsychologia.2021.107953] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2021] [Revised: 06/29/2021] [Accepted: 07/06/2021] [Indexed: 02/05/2023]
Abstract
When we read a word or see an object, conceptual meaning is automatically accessed. However, previous research investigating non-perceptual sensitivity to semantic class has employed active tasks. In this fMRI study, we tested whether conceptual representations in regions constituting the semantic network are invoked during passive semantic access and whether these representations are modulated by the need to access deeper knowledge. Seventeen healthy subjects performed a semantically active typicality judgment task and a semantically passive phonetic decision task, in both the written and the spoken input-modalities. Stimuli consisted of one hundred forty-four concepts drawn from six semantic categories. Multivariate Pattern Analysis (MVPA) revealed that the left posterior middle temporal gyrus (pMTG), posterior ventral temporal cortex (pVTC) and pars triangularis of the left inferior frontal gyrus (IFG) showed a stronger sensitivity to semantic category when active rather than passive semantic access is required. Using a cross-task training/testing classifier, we determined that conceptual representations were not only active in these regions during passive semantic access but that the neural representation of these categories was common to both active and passive access. Collectively, these results show that while representations in the pMTG, pVTC and IFG are strongly modulated by active conceptual access, consistent representational patterns are present during active and passive conceptual access in these same regions.
Collapse
Affiliation(s)
| | - Silvia Ubaldi
- Center for Mind/Brain Sciences, University of Trento, Trento, 38068, Italy
| | | |
Collapse
|
24
|
Chan HY, Scholz C, Baek EC, O'Donnell MB, Falk EB. Being the Gatekeeper: How Thinking about Sharing Affects Neural Encoding of Information. Cereb Cortex 2021; 31:3939-3949. [PMID: 33792682 PMCID: PMC8258440 DOI: 10.1093/cercor/bhab060] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2020] [Revised: 02/05/2021] [Accepted: 02/18/2021] [Indexed: 11/13/2022] Open
Abstract
Information transmission in a society depends on individuals' intention to share or not. Yet, little is known about whether being the gatekeeper shapes the brain's processing of incoming information. Here, we examine how thinking about sharing affects neural encoding of information, and whether this effect is moderated by the person's real-life social network position. In an functional magnetic resonance imaging study, participants rated abstracts of news articles on how much they wanted to read for themselves (read) or-as information gatekeepers-to share with a specific other (narrowcast) or to post on their social media feed (broadcast). In all conditions, consistent spatial blood oxygen level-dependent patterns associated with news articles were observed across participants in brain regions involved in perceptual and language processing as well as higher-order processes. However, when thinking about sharing, encoding consistency decreased in higher-order processing areas (e.g., default mode network), suggesting that the gatekeeper role involves more individualized processing in the brain, that is, person- and context-specific. Moreover, participants whose social networks had high ego-betweenness centrality (i.e., more likely to be information gatekeeper in real life) showed more individualized encoding when thinking about broadcasting. This study reveals how gatekeeping shapes our brain's processing of incoming information.
Collapse
Affiliation(s)
- Hang-Yee Chan
- Amsterdam School of Communication Research, University of Amsterdam, 1018 WV Amsterdam, the Netherlands
| | - Christin Scholz
- Amsterdam School of Communication Research, University of Amsterdam, 1018 WV Amsterdam, the Netherlands
| | - Elisa C Baek
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA 90095, USA
| | - Matthew B O'Donnell
- Annenberg School for Communication, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Emily B Falk
- Annenberg School for Communication, University of Pennsylvania, Philadelphia, PA 19104, USA
| |
Collapse
|
25
|
Fiorilli J, Bos JJ, Grande X, Lim J, Düzel E, Pennartz CMA. Reconciling the object and spatial processing views of the perirhinal cortex through task-relevant unitization. Hippocampus 2021; 31:737-755. [PMID: 33523577 PMCID: PMC8359385 DOI: 10.1002/hipo.23304] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2020] [Revised: 11/27/2020] [Accepted: 01/02/2021] [Indexed: 12/21/2022]
Abstract
The perirhinal cortex is situated on the border between sensory association cortex and the hippocampal formation. It serves an important function as a transition area between the sensory neocortex and the medial temporal lobe. While the perirhinal cortex has traditionally been associated with object coding and the "what" pathway of the temporal lobe, current evidence suggests a broader function of the perirhinal cortex in solving feature ambiguity and processing complex stimuli. Besides fulfilling functions in object coding, recent neurophysiological findings in freely moving rodents indicate that the perirhinal cortex also contributes to spatial and contextual processing beyond individual sensory modalities. Here, we address how these two opposing views on perirhinal cortex-the object-centered and spatial-contextual processing hypotheses-may be reconciled. The perirhinal cortex is consistently recruited when different features can be merged perceptually or conceptually into a single entity. Features that are unitized in these entities include object information from multiple sensory domains, reward associations, semantic features and spatial/contextual associations. We propose that the same perirhinal network circuits can be flexibly deployed for multiple cognitive functions, such that the perirhinal cortex performs similar unitization operations on different types of information, depending on behavioral demands and ranging from the object-related domain to spatial, contextual and semantic information.
Collapse
Affiliation(s)
- Julien Fiorilli
- Cognitive and Systems Neuroscience Group, SILS Center for NeuroscienceUniversity of AmsterdamAmsterdamThe Netherlands
- Research Priority Area Brain and CognitionUniversity of AmsterdamAmsterdamThe Netherlands
| | - Jeroen J. Bos
- Cognitive and Systems Neuroscience Group, SILS Center for NeuroscienceUniversity of AmsterdamAmsterdamThe Netherlands
- Research Priority Area Brain and CognitionUniversity of AmsterdamAmsterdamThe Netherlands
- Donders Institute for Brain, Cognition and BehaviorRadboud University and Radboud University Medical CentreNijmegenThe Netherlands
| | - Xenia Grande
- Institute of Cognitive Neurology and Dementia ResearchOtto‐von‐Guericke University MagdeburgMagdeburgGermany
- German Center for Neurodegenerative DiseasesMagdeburgGermany
| | - Judith Lim
- Cognitive and Systems Neuroscience Group, SILS Center for NeuroscienceUniversity of AmsterdamAmsterdamThe Netherlands
- Research Priority Area Brain and CognitionUniversity of AmsterdamAmsterdamThe Netherlands
| | - Emrah Düzel
- Institute of Cognitive Neurology and Dementia ResearchOtto‐von‐Guericke University MagdeburgMagdeburgGermany
- German Center for Neurodegenerative DiseasesMagdeburgGermany
- Institute of Cognitive NeuroscienceUniversity College LondonLondonUK
| | - Cyriel M. A. Pennartz
- Cognitive and Systems Neuroscience Group, SILS Center for NeuroscienceUniversity of AmsterdamAmsterdamThe Netherlands
- Research Priority Area Brain and CognitionUniversity of AmsterdamAmsterdamThe Netherlands
| |
Collapse
|
26
|
Abstract
Memory and language are important high-level cognitive functions of humans, and the study of conceptual representation of the human brain is a key approach to reveal the principles of cognition. However, this research is often constrained by the availability of stimulus materials. The research on concept representation often needs to be based on a standardized and large-scale database of conceptual semantic features. Although Western scholars have established a variety of English conceptual semantic feature datasets, there is still a lack of a comprehensive Chinese version. In the present study, a Chinese Conceptual semantic Feature Dataset (CCFD) was established with 1,410 concepts including their semantic features and the similarity between concepts. The concepts were grouped into 28 subordinate categories and seven superior categories artificially. The results showed that concepts within the same category were closer to each other, while concepts between categories were farther apart. The CCFD proposed in this study can provide stimulation materials and data support for related research fields. All the data and supplementary materials can be found at https://osf.io/ug5dt/ .
Collapse
|
27
|
Neural pattern similarity across concept exemplars predicts memory after a long delay. Neuroimage 2020; 219:117030. [PMID: 32526388 DOI: 10.1016/j.neuroimage.2020.117030] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2020] [Revised: 05/14/2020] [Accepted: 06/05/2020] [Indexed: 11/21/2022] Open
Abstract
The irregularities of the world ensure that each interaction we have with a concept is unique. In order to generalize across these unique encounters to form a high-level representation of a concept, we must draw on similarities between exemplars to form new conceptual knowledge that is maintained over a long time. Two neural similarity measures - pattern robustness and encoding-retrieval similarity - are particularly important for predicting memory outcomes. In this study, we used fMRI to measure activity patterns while people encoded and retrieved novel pairings between unfamiliar (Dutch) words and visually presented animal species. We address two underexplored questions: 1) whether neural similarity measures can predict memory outcomes, despite perceptual variability between presentations of a concept and 2) if pattern similarity measures can predict subsequent memory over a long delay (i.e., one month). Our findings indicate that pattern robustness during encoding in brain regions that include parietal and medial temporal areas is an important predictor of subsequent memory. In addition, we found significant encoding-retrieval similarity in the left ventrolateral prefrontal cortex after a month's delay. These findings demonstrate that pattern similarity is an important predictor of memory for novel word-animal pairings even when the concept includes multiple exemplars. Importantly, we show that established predictive relationships between pattern similarity and subsequent memory do not require visually identical stimuli (i.e., are not simply due to low-level visual overlap between stimulus presentations) and are maintained over a month.
Collapse
|
28
|
Giari G, Leonardelli E, Tao Y, Machado M, Fairhall SL. Spatiotemporal properties of the neural representation of conceptual content for words and pictures - an MEG study. Neuroimage 2020; 219:116913. [PMID: 32389730 PMCID: PMC7343530 DOI: 10.1016/j.neuroimage.2020.116913] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2019] [Revised: 04/26/2020] [Accepted: 05/01/2020] [Indexed: 11/26/2022] Open
Abstract
The entwined nature of perceptual and conceptual processes renders an understanding of the interplay between perceptual recognition and conceptual access a continuing challenge. Here, to disentangle perceptual and conceptual processing in the brain, we combine magnetoencephalography (MEG), picture and word presentation and representational similarity analysis (RSA). We replicate previous findings of early and robust sensitivity to semantic distances between objects presented as pictures and show earlier (~105 msec), but not later, representations can be accounted for by contemporary computer models of visual similarity (AlexNet). Conceptual content for word stimuli is reliably present in two temporal clusters, the first ranging from 230 to 335 msec, the second from 360 to 585 msec. The time-course of picture induced semantic content and the spatial location of conceptual representation were highly convergent, and the spatial distribution of both differed from that of words. While this may reflect differences in picture and word induced conceptual access, this underscores potential confounds in visual perceptual and conceptual processing. On the other hand, using the stringent criterion that neural and conceptual spaces must align, the robust representation of semantic content by 230-240 msec for visually unconfounded word stimuli significantly advances estimates of the timeline of semantic access and its orthographic and lexical precursors.
Collapse
Affiliation(s)
- Giuliano Giari
- Center for Mind/Brain Science, University of Trento, Trento, Italy
| | | | - Yuan Tao
- Department of Cognitive Science, Johns Hopkins University, Baltimore, United States
| | - Mayara Machado
- Max Planck Institute for Human Development, Berlin, Germany
| | - Scott L Fairhall
- Center for Mind/Brain Science, University of Trento, Trento, Italy.
| |
Collapse
|
29
|
Hoffman P, Tamm A. Barking up the right tree: Univariate and multivariate fMRI analyses of homonym comprehension. Neuroimage 2020; 219:117050. [PMID: 32534964 PMCID: PMC7443701 DOI: 10.1016/j.neuroimage.2020.117050] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2019] [Revised: 06/09/2020] [Accepted: 06/10/2020] [Indexed: 11/21/2022] Open
Abstract
Homonyms are a critical test case for investigating how the brain resolves ambiguity in language and, more generally, how context influences semantic processing. Previous neuroimaging studies have associated processing of homonyms with greater engagement of regions involved in executive control of semantic processing. However, the precise role of these areas and the involvement of semantic representational regions in homonym comprehension remain elusive. We addressed this by combining univariate and multivariate fMRI analyses of homonym processing. We tested whether multi-voxel activation patterns could discriminate between presentations of the same homonym in different contexts (e.g., bark following tree vs. bark following dog). The ventral anterior temporal lobe, implicated in semantic representation but not previously in homonym comprehension, showed this meaning-specific coding, despite not showing increased mean activation for homonyms. Within inferior frontal gyrus (IFG), a key site for semantic control, there was a dissociation between pars orbitalis, which also showed meaning-specific coding, and pars triangularis, which discriminated more generally between semantically related and unrelated word pairs. IFG effects were goal-dependent, only occurring when the task required semantic decisions, in line with a top-down control function. Finally, posterior middle temporal cortex showed a hybrid pattern of responses, supporting the idea that it acts as an interface between semantic representations and the control system. The study provides new evidence for context-dependent coding in the semantic system and clarifies the role of control regions in processing ambiguity. It also highlights the importance of combining univariate and multivariate neuroimaging data to fully elucidate the role of a brain region in semantic cognition.
Collapse
Affiliation(s)
- Paul Hoffman
- School of Philosophy, Psychology & Language Sciences, University of Edinburgh, UK.
| | - Andres Tamm
- School of Philosophy, Psychology & Language Sciences, University of Edinburgh, UK
| |
Collapse
|
30
|
Clarke A. Dynamic activity patterns in the anterior temporal lobe represents object semantics. Cogn Neurosci 2020; 11:111-121. [PMID: 32249714 PMCID: PMC7446031 DOI: 10.1080/17588928.2020.1742678] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2019] [Revised: 02/07/2020] [Indexed: 02/07/2023]
Abstract
The anterior temporal lobe (ATL) is considered a crucial area for the representation of transmodal concepts. Recent evidence suggests that specific regions within the ATL support the representation of individual object concepts, as shown by studies combining multivariate analysis methods and explicit measures of semantic knowledge. This research looks to further our understanding by probing conceptual representations at a spatially and temporally resolved neural scale. Representational similarity analysis was applied to human intracranial recordings from anatomically defined lateral to medial ATL sub-regions. Neural similarity patterns were tested against semantic similarity measures, where semantic similarity was defined by a hybrid corpus-based and feature-based approach. Analyses show that the perirhinal cortex, in the medial ATL, significantly related to semantic effects around 200 to 400 ms, and were greater than more lateral ATL regions. Further, semantic effects were present in low frequency (theta and alpha) oscillatory phase signals. These results provide converging support that more medial regions of the ATL support the representation of basic-level visual object concepts within the first 400 ms, and provide a bridge between prior fMRI and MEG work by offering detailed evidence for the presence of conceptual representations within the ATL.
Collapse
Affiliation(s)
- Alex Clarke
- Department of Psychology, University of Cambridge, Cambridge, UK
| |
Collapse
|
31
|
Representation of associative and affective semantic similarity of abstract words in the lateral temporal perisylvian language regions. Neuroimage 2020; 217:116892. [PMID: 32371118 DOI: 10.1016/j.neuroimage.2020.116892] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2020] [Revised: 04/23/2020] [Accepted: 04/28/2020] [Indexed: 12/18/2022] Open
Abstract
The examination of semantic cognition has traditionally identified word concreteness as well as valence as two of the principal dimensions in the representation of conceptual knowledge. More recently, corpus-based vector space models as well as graph-theoretical analysis of large-scale task-related behavioural responses have revolutionized our insight into how the meaning of words is structured. In this fMRI study, we apply representational similarity analysis to investigate the conceptual representation of abstract words. Brain activity patterns were related to a cued-association based graph as well as to a vector-based co-occurrence model of word meaning. Twenty-six subjects (19 females and 7 males) performed an overt repetition task during fMRI. First, we performed a searchlight classification procedure to identify regions where activity is discriminable between abstract and concrete words. These regions were left inferior frontal gyrus, the upper and lower bank of the superior temporal sulcus bilaterally, posterior middle temporal gyrus and left fusiform gyrus. Representational Similarity Analysis demonstrated that for abstract words, the similarity of activity patterns in the cortex surrounding the superior temporal sulcus bilaterally and in the left anterior superior temporal gyrus reflects the similarity in word meaning. These effects were strongest for semantic similarity derived from the cued association-based graph and for affective similarity derived from either of the two models. The latter effect was mainly driven by positive valence words. This research highlights the close neurobiological link between the information structure of abstract and affective word content and the similarity in activity pattern in the lateral and anterior temporal language system.
Collapse
|
32
|
Familiarity impairments after anterior temporal-lobe resection with hippocampal sparing: Lessons learned from case NB. Neuropsychologia 2020; 138:107339. [PMID: 31930957 DOI: 10.1016/j.neuropsychologia.2020.107339] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2019] [Revised: 12/29/2019] [Accepted: 01/09/2020] [Indexed: 01/27/2023]
Abstract
We review evidence from an extensive single case study in an individual (NB) who underwent a rare left-sided anterior temporal-lobe resection with sparing of the hippocampus. Our study aimed to determine whether memory functions of perirhinal cortex, which was largely removed in the resection, can be impaired against a background of preserved hippocampus-dependent memory processing. This research was guided by the proposal that item-based familiarity assessment relies on contributions of perirhinal cortex, and that the hippocampus plays a unique role in the relational binding of items to episodic contexts, which is critical for recollection. Seven sets of findings have emerged from our research on NB (synthesized from five primary research articles), and from follow-up work in other patients: (i) Familiarity impairments can be selective and be revealed with multiple methods; (ii) selective familiarity and selective recollection impairments can be double dissociated; (iii) selective familiarity impairments show material specificity; (iv) selective familiarity impairments extend to assessment of cumulative lifetime experience; (v) selective familiarity impairments are sensitive to degree of feature overlap between object concepts; (vi) selective familiarity impairments are associated with preserved task-related fMRI signals in the hippocampus; (vii) selective familiarity impairments can be observed in other lesion cases. Despite our main focus on the dual-process framework, we also discuss implications for the functional organization of the medial temporal lobes in broader terms. We argue that our findings shed light on this organization even if the functional specialization of different medial temporal structures is ultimately not fully captured with reference to the cognitive distinction between familiarity and recollection.
Collapse
|
33
|
Bruffaerts R, Schaeverbeke J, De Weer AS, Nelissen N, Dries E, Van Bouwel K, Sieben A, Bergmans B, Swinnen C, Pijnenburg Y, Sunaert S, Vandenbulcke M, Vandenberghe R. Multivariate analysis reveals anatomical correlates of naming errors in primary progressive aphasia. Neurobiol Aging 2019; 88:71-82. [PMID: 31955981 DOI: 10.1016/j.neurobiolaging.2019.12.016] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2019] [Revised: 12/10/2019] [Accepted: 12/15/2019] [Indexed: 12/30/2022]
Abstract
Primary progressive aphasia (PPA) is an overarching term for a heterogeneous group of neurodegenerative diseases which affect language processing. Impaired picture naming has been linked to atrophy of the anterior temporal lobe in the semantic variant of PPA. Although atrophy of the anterior temporal lobe proposedly impairs picture naming by undermining access to semantic knowledge, picture naming also entails object recognition and lexical retrieval. Using multivariate analysis, we investigated whether cortical atrophy relates to different types of naming errors generated during picture naming in 43 PPA patients (13 semantic, 9 logopenic, 11 nonfluent, and 10 mixed variant). Omissions were associated with atrophy of the anterior temporal lobes. Semantic errors, for example, mistaking a rhinoceros for a hippopotamus, were associated with atrophy of the left mid and posterior fusiform cortex and the posterior middle and inferior temporal gyrus. Semantic errors and atrophy in these regions occurred in each PPA subtype, without major between-subtype differences. We propose that pathological changes to neural mechanisms associated with semantic errors occur across the PPA spectrum.
Collapse
Affiliation(s)
- Rose Bruffaerts
- Laboratory for Cognitive Neurology, Department of Neurosciences, KU Leuven, Leuven, Belgium; Neurology Department, University Hospitals Leuven, Leuven, Belgium.
| | - Jolien Schaeverbeke
- Laboratory for Cognitive Neurology, Department of Neurosciences, KU Leuven, Leuven, Belgium
| | - An-Sofie De Weer
- Laboratory for Cognitive Neurology, Department of Neurosciences, KU Leuven, Leuven, Belgium
| | - Natalie Nelissen
- Laboratory for Cognitive Neurology, Department of Neurosciences, KU Leuven, Leuven, Belgium
| | - Eva Dries
- Neurology Department, University Hospitals Leuven, Leuven, Belgium
| | - Karen Van Bouwel
- Neurology Department, University Hospitals Leuven, Leuven, Belgium
| | - Anne Sieben
- Neurology Department, University Hospital Ghent, Ghent, Belgium
| | - Bruno Bergmans
- Neurology Department, University Hospital Ghent, Ghent, Belgium; Neurology Department, AZ Sint-Jan Brugge-Oostende AV, Bruges, Belgium
| | | | - Yolande Pijnenburg
- Neurology Department, Alzheimer Center Amsterdam, Amsterdam Neuroscience, Vrije Universiteit Amsterdam, Amsterdam UMC, Amsterdam, The Netherlands
| | - Stefan Sunaert
- Radiology Department, University Hospitals Leuven, Leuven, Belgium
| | | | - Rik Vandenberghe
- Laboratory for Cognitive Neurology, Department of Neurosciences, KU Leuven, Leuven, Belgium; Neurology Department, University Hospitals Leuven, Leuven, Belgium
| |
Collapse
|
34
|
The Emotional Facet of Subjective and Neural Indices of Similarity. Brain Topogr 2019; 32:956-964. [PMID: 31728708 PMCID: PMC6882781 DOI: 10.1007/s10548-019-00743-7] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2019] [Accepted: 11/02/2019] [Indexed: 12/27/2022]
Abstract
Emotional similarity refers to the tendency to group stimuli together because they evoke the same feelings in us. The majority of research on similarity perception that has been conducted to date has focused on non-emotional stimuli. Different models have been proposed to explain how we represent semantic concepts, and judge the similarity among them. They are supported from behavioural and neural evidence, often combined by using Multivariate Pattern Analyses. By contrast, less is known about the cognitive and neural mechanisms underlying the judgement of similarity between real-life emotional experiences. This review summarizes the major findings, debates and limitations in the semantic similarity literature. They will serve as background to the emotional facet of similarity that will be the focus of this review. A multi-modal and overarching approach, which relates different levels of neuroscientific explanation (i.e., computational, algorithmic and implementation), would be the key to further unveil what makes emotional experiences similar to each other.
Collapse
|
35
|
Liuzzi AG, Bruffaerts R, Vandenberghe R. The medial temporal written word processing system. Cortex 2019; 119:287-300. [DOI: 10.1016/j.cortex.2019.05.002] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2018] [Revised: 03/14/2019] [Accepted: 05/08/2019] [Indexed: 10/26/2022]
|
36
|
Liu F, Han J, Zhang L, Li F. Inductive Reasoning Differs Between Taxonomic and Thematic Contexts: Electrophysiological Evidence. Front Psychol 2019; 10:1702. [PMID: 31402887 PMCID: PMC6669940 DOI: 10.3389/fpsyg.2019.01702] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2019] [Accepted: 07/08/2019] [Indexed: 11/13/2022] Open
Abstract
Inductive reasoning can be performed in different contexts, but it is unclear whether the neural mechanism of reasoning performed in a thematic context (e.g., bee has x, so honey has x) is the same as that performed in a taxonomic context (e.g., bee has x, so butterfly has x). In the present study, participants were required to judge whether a conclusion was acceptable or not based on its premise, for which the taxonomic or thematic distances between premise and conclusion objects were either far or near. The Event related potential (ERP) results indicated that the effect of context (taxonomic vs. thematic) was initially observed in the P2 component; while the distance effect (far vs. near) was observed in N400 and late components. Moreover, the distance effect on thematic-based inductive reasoning was found in the anterior regions, while the distance effect on taxonomic-based inductive reasoning conditions was found in the posterior regions. These results support the view that inductive reasoning is performed differently under different semantic contexts.
Collapse
Affiliation(s)
- Fangfang Liu
- School of Psychology, Jiangxi Normal University, Nanchang, China.,School of Educational Science, Minnan Normal University, Zhangzhou, China
| | - Jiahui Han
- School of Psychology, Jiangxi Normal University, Nanchang, China
| | - Lingcong Zhang
- School of Educational Science, Minnan Normal University, Zhangzhou, China
| | - Fuhong Li
- School of Psychology, Jiangxi Normal University, Nanchang, China
| |
Collapse
|
37
|
Hu B, von der Heydt R, Niebur E. Figure-Ground Organization in Natural Scenes: Performance of a Recurrent Neural Model Compared with Neurons of Area V2. eNeuro 2019; 6:ENEURO.0479-18.2019. [PMID: 31167850 PMCID: PMC6635809 DOI: 10.1523/eneuro.0479-18.2019] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2018] [Revised: 04/15/2019] [Accepted: 05/07/2019] [Indexed: 12/02/2022] Open
Abstract
A crucial step in understanding visual input is its organization into meaningful components, in particular object contours and partially occluded background structures. This requires that all contours are assigned to either the foreground or the background (border ownership assignment). While earlier studies showed that neurons in primate extrastriate cortex signal border ownership for simple geometric shapes, recent studies show consistent border ownership coding also for complex natural scenes. In order to understand how the brain performs this task, we developed a biologically plausible recurrent neural network that is fully image computable. Our model uses local edge detector ( B ) cells and grouping ( G ) cells whose activity represents proto-objects based on the integration of local feature information. G cells send modulatory feedback connections to those B cells that caused their activation, making the B cells border ownership selective. We found close agreement between our model and neurophysiological results in terms of the timing of border ownership signals (BOSs) as well as the consistency of BOSs across scenes. We also benchmarked our model on the Berkeley Segmentation Dataset and achieved performance comparable to recent state-of-the-art computer vision approaches. Our proposed model provides insight into the cortical mechanisms of figure-ground organization.
Collapse
Affiliation(s)
- Brian Hu
- Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University, Baltimore, MD 21218
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD 21205
| | - Rüdiger von der Heydt
- Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University, Baltimore, MD 21218
- Solomon Snyder Department of Neuroscience, Johns Hopkins University, Baltimore, MD 21205
| | - Ernst Niebur
- Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University, Baltimore, MD 21218
- Solomon Snyder Department of Neuroscience, Johns Hopkins University, Baltimore, MD 21205
| |
Collapse
|
38
|
Folstein JR, Dieciuc MA. The Cognitive Neuroscience of Stable and Flexible Semantic Typicality. Front Psychol 2019; 10:1265. [PMID: 31214079 PMCID: PMC6554317 DOI: 10.3389/fpsyg.2019.01265] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2018] [Accepted: 05/13/2019] [Indexed: 12/15/2022] Open
Abstract
Typicality effects are among the most well-studied phenomena in the study of concepts. The classical notion of typicality is that typical concepts share many features with category co-members and few features with members of contrast categories. However, this notion was challenged by evidence that typicality is highly context dependent and not always dependent on central tendency. Dieciuc and Folstein (2019) argued that there is strong evidence for both views and that the two types of typicality effects might depend on different mechanisms. A recent theoretical framework, the controlled semantic cognition framework (Lamdon Ralph et al., 2017) strongly emphasizes the classical view, but includes mechanisms that could potentially account for both kinds of typicality. In contrast, the situated cognition framework (Barsalou, 2009b) articulates the context-dependent view. Here, we review evidence from cognitive neuroscience supporting the two frameworks. We also briefly evaluate the ability of computational models associated with the CSC to account for phenomena supporting SitCog (Rogers and McClelland, 2004). Many predictions of both frameworks are borne out by recent cognitive neuroscience evidence. While the CSC framework can at least potentially account for many of the typicality phenomena reviewed, challenges remain, especially with regard to ad hoc categories.
Collapse
Affiliation(s)
- Jonathan R. Folstein
- Department of Psychology, Florida State University, Tallahassee, FL, United States
| | | |
Collapse
|
39
|
Bruffaerts R, De Deyne S, Meersmans K, Liuzzi AG, Storms G, Vandenberghe R. Redefining the resolution of semantic knowledge in the brain: Advances made by the introduction of models of semantics in neuroimaging. Neurosci Biobehav Rev 2019; 103:3-13. [PMID: 31132379 DOI: 10.1016/j.neubiorev.2019.05.015] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2019] [Revised: 05/15/2019] [Accepted: 05/17/2019] [Indexed: 12/12/2022]
Abstract
The boundaries of our understanding of conceptual representation in the brain have been redrawn since the introduction of explicit models of semantics. These models are grounded in vast behavioural datasets acquired in healthy volunteers. Here, we review the most important techniques which have been applied to detect semantic information in neuroimaging data and argue why semantic models are possibly the most valuable addition to the research of semantics in recent years. Using multivariate analysis, predictions based on patient lesion data have been confirmed during semantic processing in healthy controls. Secondly, this new method has given rise to new research avenues, e.g. the detection of semantic processing outside of the temporal cortex. As a future line of work, the same research strategy could be useful to study neurological conditions such as the semantic variant of primary progressive aphasia, which is characterized by pathological semantic processing.
Collapse
Affiliation(s)
- Rose Bruffaerts
- Laboratory for Cognitive Neurology, Department of Neurosciences, KU Leuven, Belgium; Neurology Department, University Hospitals Leuven, 3000 Leuven, Belgium.
| | - Simon De Deyne
- Laboratory of Experimental Psychology, Humanities and Social Sciences Group, KU Leuven, Belgium
| | - Karen Meersmans
- Laboratory for Cognitive Neurology, Department of Neurosciences, KU Leuven, Belgium
| | | | - Gert Storms
- Laboratory of Experimental Psychology, Humanities and Social Sciences Group, KU Leuven, Belgium
| | - Rik Vandenberghe
- Laboratory for Cognitive Neurology, Department of Neurosciences, KU Leuven, Belgium; Neurology Department, University Hospitals Leuven, 3000 Leuven, Belgium
| |
Collapse
|
40
|
Liuzzi AG, Dupont P, Peeters R, Bruffaerts R, De Deyne S, Storms G, Vandenberghe R. Left perirhinal cortex codes for semantic similarity between written words defined from cued word association. Neuroimage 2019; 191:127-139. [DOI: 10.1016/j.neuroimage.2019.02.011] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2018] [Revised: 01/31/2019] [Accepted: 02/05/2019] [Indexed: 10/27/2022] Open
|
41
|
Rice GE, Hoffman P, Binney RJ, Lambon Ralph MA. Concrete versus abstract forms of social concept: an fMRI comparison of knowledge about people versus social terms. Philos Trans R Soc Lond B Biol Sci 2019; 373:rstb.2017.0136. [PMID: 29915004 PMCID: PMC6015823 DOI: 10.1098/rstb.2017.0136] [Citation(s) in RCA: 40] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/10/2018] [Indexed: 12/14/2022] Open
Abstract
The anterior temporal lobes (ATLs) play a key role in conceptual knowledge representation. The hub-and-spoke theory suggests that the contribution of the ATLs to semantic representation is (a) transmodal, i.e. integrating information from multiple sensorimotor and verbal modalities, and (b) pan-categorical, representing concepts from all categories. Another literature, however, suggests that this region's responses are modality- and category-selective; prominent examples include category selectivity for socially relevant concepts and face recognition. The predictions of each approach have never been directly compared. We used data from three studies to compare category-selective responses within the ATLs. Study 1 compared ATL responses to famous people versus another conceptual category (landmarks) from visual versus auditory inputs. Study 2 compared ATL responses to famous people from pictorial and written word inputs. Study 3 compared ATL responses to a different kind of socially relevant stimuli, namely abstract non-person-related words, in order to ascertain whether ATL subregions are engaged for social concepts more generally or only for person-related knowledge. Across all three studies a dominant bilateral ventral ATL cluster responded to all categories in all modalities. Anterior to this ‘pan-category’ transmodal region, a second cluster responded more weakly overall yet selectively for people, but did so equally for spoken names and faces (Study 1). A third region in the anterior superior temporal gyrus responded selectively to abstract socially relevant words (Study 3), but did not respond to concrete socially relevant words (i.e. written names; Study 2). These findings can be accommodated by the graded hub-and-spoke model of concept representation. On this view, the ventral ATL is the centre point of a bilateral ATL hub, which contributes to conceptual representation through transmodal distillation of information arising from multiple modality-specific association cortices. Partial specialization occurs across the graded ATL hub as a consequence of gradedly differential connectivity across the region. This article is part of the theme issue ‘Varieties of abstract concepts: development, use and representation in the brain’.
Collapse
Affiliation(s)
- Grace E Rice
- Neuroscience and Aphasia Research Unit (NARU), University of Manchester, Manchester, UK
| | - Paul Hoffman
- Centre for Cognitive Ageing and Cognitive Epidemiology (CCACE), Department of Psychology, University of Edinburgh, Edinburgh, UK
| | | | | |
Collapse
|
42
|
Anderson AJ, Lin F. How pattern information analyses of semantic brain activity elicited in language comprehension could contribute to the early identification of Alzheimer's Disease. Neuroimage Clin 2019; 22:101788. [PMID: 30991624 PMCID: PMC6451171 DOI: 10.1016/j.nicl.2019.101788] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2018] [Revised: 01/28/2019] [Accepted: 03/22/2019] [Indexed: 12/19/2022]
Abstract
Alzheimer's disease (AD) is associated with a loss of semantic knowledge reflecting brain pathophysiology that begins years before dementia. Identifying early signs of pathophysiology induced dysfunction in the neural systems that access and process words' meaning could therefore help forecast dementia. This article reviews pioneering studies demonstrating that abnormal functional Magnetic Resonance Imaging (fMRI) response patterns elicited in semantic tasks reflect both AD-pathophysiology and the hereditary risk of AD, and also can help forecast cognitive decline. However, to bring current semantic task-based fMRI research up to date with new AD research guidelines the relationship with different types of AD-pathophysiology needs to be more thoroughly examined. We shall argue that new analytic techniques and experimental paradigms will be critical for this. Previous work has relied on specialized tests of specific components of semantic knowledge/processing (e.g. famous name recognition) to reveal coarse AD-related changes in activation across broad brain regions. Recent computational advances now enable more detailed tests of the semantic information that is represented within brain regions during more natural language comprehension. These new methods stand to more directly index how pathophysiology alters neural information processing, whilst using language comprehension as the basis for a more comprehensive examination of semantic brain function. We here connect the semantic pattern information analysis literature up with AD research to raise awareness to potential cross-disciplinary research opportunities.
Collapse
Affiliation(s)
- Andrew James Anderson
- Department of Neuroscience, University of Rochester Medical Center, United States of America.
| | - Feng Lin
- Department of Neuroscience, University of Rochester Medical Center, United States of America; School of Nursing, University of Rochester Medical Center, United States of America; Department of Psychiatry, University of Rochester Medical Center, United States of America; Department of Neurology, University of Rochester Medical Center, United States of America; Department of Brain and Cognitive Sciences, University of Rochester, United States of America.
| |
Collapse
|
43
|
Kivisaari SL, van Vliet M, Hultén A, Lindh-Knuutila T, Faisal A, Salmelin R. Reconstructing meaning from bits of information. Nat Commun 2019; 10:927. [PMID: 30804334 PMCID: PMC6389990 DOI: 10.1038/s41467-019-08848-0] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2018] [Accepted: 01/29/2019] [Indexed: 01/09/2023] Open
Abstract
Modern theories of semantics posit that the meaning of words can be decomposed into a unique combination of semantic features (e.g., "dog" would include "barks"). Here, we demonstrate using functional MRI (fMRI) that the brain combines bits of information into meaningful object representations. Participants receive clues of individual objects in form of three isolated semantic features, given as verbal descriptions. We use machine-learning-based neural decoding to learn a mapping between individual semantic features and BOLD activation patterns. The recorded brain patterns are best decoded using a combination of not only the three semantic features that were in fact presented as clues, but a far richer set of semantic features typically linked to the target object. We conclude that our experimental protocol allowed us to demonstrate that fragmented information is combined into a complete semantic representation of an object and to identify brain regions associated with object meaning.
Collapse
Affiliation(s)
- Sasa L Kivisaari
- Department of Neuroscience and Biomedical Engineering, Aalto University, P.O. Box 12200, FI-00076, Aalto, Finland. .,Aalto NeuroImaging, Aalto University, P.O. Box 12200, Aalto, FI-00076, Finland.
| | - Marijn van Vliet
- Department of Neuroscience and Biomedical Engineering, Aalto University, P.O. Box 12200, FI-00076, Aalto, Finland.,Aalto NeuroImaging, Aalto University, P.O. Box 12200, Aalto, FI-00076, Finland
| | - Annika Hultén
- Department of Neuroscience and Biomedical Engineering, Aalto University, P.O. Box 12200, FI-00076, Aalto, Finland.,Aalto NeuroImaging, Aalto University, P.O. Box 12200, Aalto, FI-00076, Finland
| | - Tiina Lindh-Knuutila
- Department of Neuroscience and Biomedical Engineering, Aalto University, P.O. Box 12200, FI-00076, Aalto, Finland
| | - Ali Faisal
- Department of Neuroscience and Biomedical Engineering, Aalto University, P.O. Box 12200, FI-00076, Aalto, Finland
| | - Riitta Salmelin
- Department of Neuroscience and Biomedical Engineering, Aalto University, P.O. Box 12200, FI-00076, Aalto, Finland.,Aalto NeuroImaging, Aalto University, P.O. Box 12200, Aalto, FI-00076, Finland
| |
Collapse
|
44
|
Leonardelli E, Fait E, Fairhall SL. Temporal dynamics of access to amodal representations of category-level conceptual information. Sci Rep 2019; 9:239. [PMID: 30659237 PMCID: PMC6338759 DOI: 10.1038/s41598-018-37429-2] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2018] [Accepted: 12/06/2018] [Indexed: 11/08/2022] Open
Abstract
Categories describe semantic divisions between classes of objects and category-based models are widely used for investigation of the conceptual system. One critical issue in this endeavour is the isolation of conceptual from perceptual contributions to category-differences. An unambiguous way to address this confound is combining multiple input-modalities. To this end, we showed participants person/place stimuli using name and picture modalities. Using multivariate methods, we searched for category-sensitive neural patterns shared across input-modalities and thus independent from perceptual properties. The millisecond temporal resolution of magnetoencephalography (MEG) allowed us to consider the precise timing of conceptual access and, by confronting latencies between the two modalities ("time generalization"), how latencies of processing depends on the input-modality. Our results identified category-sensitive conceptual representations common between modalities at three stages and that conceptual access for words was delayed by about 90 msec with respect to pictures. We also show that for pictures, the first conceptual pattern of activity (shared between both words and pictures) occurs as early as 110 msec. Collectively, our results indicated that conceptual access at the category-level is a multistage process and that different delays in access across these two input-modalities determine when these representations are activated.
Collapse
Affiliation(s)
- Elisa Leonardelli
- Center for Mind/Brain Sciences, University of Trento, Trento, 38068, Italy.
| | - Elisa Fait
- Center for Mind/Brain Sciences, University of Trento, Trento, 38068, Italy
| | - Scott L Fairhall
- Center for Mind/Brain Sciences, University of Trento, Trento, 38068, Italy
| |
Collapse
|
45
|
Yi YG, Kim DY, Shim WH, Oh JY, Kim HS, Jung M. Perilesional and homotopic area activation during proverb comprehension after stroke. Brain Behav 2019; 9:e01202. [PMID: 30588768 PMCID: PMC6346665 DOI: 10.1002/brb3.1202] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/27/2018] [Revised: 10/22/2018] [Accepted: 11/30/2018] [Indexed: 12/19/2022] Open
Abstract
INTRODUCTION The mechanism of functional recovery in right hemisphere (RH) stroke patients when attempting to comprehend a proverb has not been identified. We previously reported that there is bilateral hemisphere involvement during proverb comprehension in the normal population. However, the underlying mechanisms of proverb comprehension following a right middle cerebral artery (MCA) infarction have not yet been fully elucidated. METHODS We here compared the brain regions activated by literal sentences and by opaque or transparent proverbs in right MCA infarction patients using functional magnetic resonance imaging (fMRI). Experimental stimuli included 18 opaque proverbs, 18 transparent proverbs, and 18 literal sentences that were presented pseudorandomly in 1 of 3 predesigned sequences. RESULTS Fifteen normal adults and 17 right MCA infarction patients participated in this study. The areas of the brain in the stroke patients involved in understanding a proverb compared with a literal sentence included the right middle frontal gyrus (MFG) and frontal pole, right anterior cingulate gyrus/paracingulate gyrus and left inferior frontal gyrus (IFG), middle temporal gyrus (MTG), precuneus, and supramarginal gyrus (SMG). When the proverbs were presented to these stroke patients in the comprehension tests, the left supramarginal, postcentral gyrus, and right paracingulate gyrus were activated for the opaque proverbs compared to the transparent proverbs. CONCLUSIONS These findings suggest that the functional recovery of language in stroke patients can be explained by perilesional activation, which is thought to arise from the regulation of the excitatory and inhibitory neurotransmitter system, and by homotopic area activation which has been characterized by decreased transcallosal inhibition and astrocyte involvement.
Collapse
Affiliation(s)
- You Gyoung Yi
- Department of Rehabilitation Medicine, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, Korea
| | - Dae Yul Kim
- Department of Rehabilitation Medicine, Asan Medical Center, Seoul, Korea
| | - Woo Hyun Shim
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, Seoul, Korea
| | - Joo Young Oh
- Asan Institute for Life Science, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea
| | - Ho Sung Kim
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, Seoul, Korea
| | - Minji Jung
- Department of Rehabilitation Medicine, Asan Medical Center, Seoul, Korea
| |
Collapse
|
46
|
Ostarek M, Joosen D, Ishag A, de Nijs M, Huettig F. Are visual processes causally involved in "perceptual simulation" effects in the sentence-picture verification task? Cognition 2018; 182:84-94. [PMID: 30219635 DOI: 10.1016/j.cognition.2018.08.017] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2017] [Revised: 08/27/2018] [Accepted: 08/27/2018] [Indexed: 10/28/2022]
Abstract
Many studies have shown that sentences implying an object to have a certain shape produce a robust reaction time advantage for shape-matching pictures in the sentence-picture verification task. Typically, this finding has been interpreted as evidence for perceptual simulation, i.e., that access to implicit shape information involves the activation of modality-specific visual processes. It follows from this proposal that disrupting visual processing during sentence comprehension should interfere with perceptual simulation and obliterate the match effect. Here we directly test this hypothesis. Participants listened to sentences while seeing either visual noise that was previously shown to strongly interfere with basic visual processing or a blank screen. Experiments 1 and 2 replicated the match effect but crucially visual noise did not modulate it. When an interference technique was used that targeted high-level semantic processing (Experiment 3) however the match effect vanished. Visual noise specifically targeting high-level visual processes (Experiment 4) only had a minimal effect on the match effect. We conclude that the shape match effect in the sentence-picture verification paradigm is unlikely to rely on perceptual simulation.
Collapse
Affiliation(s)
- Markus Ostarek
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands; International Max Planck Research School for Language Sciences, The Netherlands.
| | - Dennis Joosen
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands
| | - Adil Ishag
- International University of Africa, Khartoum, Sudan
| | - Monique de Nijs
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands
| | - Falk Huettig
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands
| |
Collapse
|
47
|
Nguyen M, Vanderwal T, Hasson U. Shared understanding of narratives is correlated with shared neural responses. Neuroimage 2018; 184:161-170. [PMID: 30217543 DOI: 10.1016/j.neuroimage.2018.09.010] [Citation(s) in RCA: 129] [Impact Index Per Article: 21.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2018] [Revised: 08/10/2018] [Accepted: 09/04/2018] [Indexed: 12/21/2022] Open
Abstract
Humans have a striking ability to infer meaning from even the sparsest and most abstract forms of narratives. At the same time, flexibility in the form of a narrative is matched by inherent ambiguity in its interpretation. How does the brain represent subtle, idiosyncratic differences in the interpretation of abstract and ambiguous narratives? In this fMRI study, subjects were scanned either watching a novel 7-min animation depicting a complex narrative through the movement of geometric shapes, or listening to a narration of the animation's social story. Using an intersubject representational similarity analysis that compared interpretation similarity and neural similarity across subjects, we found that the more similar two people's interpretations of the abstract shapes animation were, the more similar were their neural responses in regions of the default mode network (DMN) and fronto-parietal network. Moreover, these shared responses were modality invariant: the shapes movie and the verbal interpretation of the movie elicited shared responses in linguistic areas and a subset of the DMN when subjects shared interpretations. Together, these results suggest a network of high-level regions that are not only sensitive to subtle individual differences in narrative interpretation during naturalistic conditions, but also resilient to large differences in the modality of the narrative.
Collapse
Affiliation(s)
- Mai Nguyen
- Department of Psychology, Princeton University, Princeton, NJ, 08540, USA.
| | - Tamara Vanderwal
- Yale Child Study Center, Yale University, 230 S Frontage Rd, New Haven, CT, 06520, USA
| | - Uri Hasson
- Department of Psychology, Princeton University, Princeton, NJ, 08540, USA; Princeton Neuroscience Institute, Princeton University, Princeton, NJ, 08540, USA
| |
Collapse
|
48
|
Recognition memory shielded from semantic but not perceptual interference in normal aging. Neuropsychologia 2018; 119:448-463. [PMID: 30071206 DOI: 10.1016/j.neuropsychologia.2018.07.031] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2017] [Revised: 07/02/2018] [Accepted: 07/28/2018] [Indexed: 12/15/2022]
Abstract
Normal aging impairs long-term declarative memory, and evidence suggests that this impairment may be driven partly by structural or functional changes in the medial temporal lobe (MTL). Theories of MTL memory function therefore make predictions for age-related memory loss. One theory - the Representational-Hierarchical account - makes two specific predictions. First, recognition memory performance in older participants should be impaired by feature-level interference, in which studied items contain many shared, and thus repeatedly appearing, perceptual features. Second, if the interference in a recognition memory task - i.e., the information that repeats across items - resides at a higher level of complexity than simple perceptual features, such as semantic gist, older adults should be less impacted by such interference than young adults. We tested these predictions using the Deese-Roediger-McDermott paradigm, by creating feature-level (i.e., perceptual) interference with phonemically/orthographically related word categories, and higher-level associative interference with semantically related word categories. We manipulated category size in order to compare the effect of less versus more interference (i.e., small versus large category size), which served to (1) avoid potential item confounds arising from systematic differences between words belonging to perceptually- versus semantically-related categories, and (2) ensure that any effect of interference was due to information encoded at study, rather than pre-experimentally. Further, we used signal detection theory (SDT) to interpret our data, rather than examining false alarm (FA) rates in isolation. The d' measure derived from SDT avoids contamination of the memory measure by response bias, and lies on an interval scale, allowing memory performance in different conditions to be compared without violating assumptions of the statistical tests. Older participants were relatively more impaired by perceptual interference and less impaired by semantic interference than young adults. This pattern is at odds with many current theories of age-related memory loss, but is in line with the Representational-Hierarchical account.
Collapse
|
49
|
Blumenthal A, Stojanoski B, Martin CB, Cusack R, Köhler S. Animacy and real-world size shape object representations in the human medial temporal lobes. Hum Brain Mapp 2018; 39:3779-3792. [PMID: 29947037 DOI: 10.1002/hbm.24212] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2017] [Revised: 04/11/2018] [Accepted: 04/30/2018] [Indexed: 12/15/2022] Open
Abstract
Identifying what an object is, and whether an object has been encountered before, is a crucial aspect of human behavior. Despite this importance, we do not yet have a complete understanding of the neural basis of these abilities. Investigations into the neural organization of human object representations have revealed category specific organization in the ventral visual stream in perceptual tasks. Interestingly, these categories fall within broader domains of organization, with reported distinctions between animate, inanimate large, and inanimate small objects. While there is some evidence for category specific effects in the medial temporal lobe (MTL), in particular in perirhinal and parahippocampal cortex, it is currently unclear whether domain level organization is also present across these structures. To this end, we used fMRI with a continuous recognition memory task. Stimuli were images of objects from several different categories, which were either animate or inanimate, or large or small within the inanimate domain. We employed representational similarity analysis (RSA) to test the hypothesis that object-evoked responses in MTL structures during recognition-memory judgments also show evidence for domain-level organization along both dimensions. Our data support this hypothesis. Specifically, object representations were shaped by either animacy, real-world size, or both, in perirhinal and parahippocampal cortex, and the hippocampus. While sensitivity to these dimensions differed across structures when probed individually, hinting at interesting links to functional differentiation, similarities in organization across MTL structures were more prominent overall. These results argue for continuity in the organization of object representations in the ventral visual stream and the MTL.
Collapse
Affiliation(s)
- Anna Blumenthal
- Department of Psychology, University of Western Ontario, London, Ontario, Canada.,The Brain and Mind Institute, University of Western Ontario, London, Ontario, Canada
| | - Bobby Stojanoski
- Department of Psychology, University of Western Ontario, London, Ontario, Canada.,The Brain and Mind Institute, University of Western Ontario, London, Ontario, Canada
| | - Chris B Martin
- Department of Psychology, University of Toronto, Toronto, Ontario, Canada
| | - Rhodri Cusack
- Department of Psychology, Trinity College, Dublin, Ireland
| | - Stefan Köhler
- Department of Psychology, University of Western Ontario, London, Ontario, Canada.,The Brain and Mind Institute, University of Western Ontario, London, Ontario, Canada.,Rotman Research Institute, Baycrest, Toronto, Ontario, Canada
| |
Collapse
|
50
|
Zhao L, Chen C, Shao L, Wang Y, Xiao X, Chen C, Yang J, Zevin J, Xue G. Orthographic and Phonological Representations in the Fusiform Cortex. Cereb Cortex 2018; 27:5197-5210. [PMID: 27664959 DOI: 10.1093/cercor/bhw300] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2016] [Accepted: 09/06/2016] [Indexed: 11/12/2022] Open
Abstract
Mental and neural representations of words are at the core of understanding the cognitive and neural mechanisms of reading. Despite extensive studies, the nature of visual word representation remains highly controversial due to methodological limitations. In particular, it is unclear whether the fusiform cortex contains only abstract orthographic representation, or represents both lower and higher level orthography as well as phonology. Using representational similarity analysis, we integrated behavioral ratings, computational models of reading and visual object recognition, and neuroimaging data to examine the nature of visual word representations in the fusiform cortex. Our results provided clear evidence that the middle and anterior fusiform represented both phonological and orthographic information. Whereas lower level orthographic information was represented at every stage of the ventral visual stream, abstract orthographic information was increasingly represented along the posterior-to-anterior axis. Furthermore, the left and right hemispheres were tuned to high- and low-frequency orthographic information, respectively. These results help to resolve the long-standing debates regarding the role of the fusiform in reading, and have significant implications for the development of psychological, neural, and computational theories of reading.
Collapse
Affiliation(s)
- Libo Zhao
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China.,Center for Collaboration and Innovation in Brain and Learning Sciences, Beijing Normal University, Beijing 100875, PR China
| | - Chunhui Chen
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China.,Center for Collaboration and Innovation in Brain and Learning Sciences, Beijing Normal University, Beijing 100875, PR China
| | - Luying Shao
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China.,Center for Collaboration and Innovation in Brain and Learning Sciences, Beijing Normal University, Beijing 100875, PR China
| | - Yapeng Wang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China.,Center for Collaboration and Innovation in Brain and Learning Sciences, Beijing Normal University, Beijing 100875, PR China
| | - Xiaoqian Xiao
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China.,Center for Collaboration and Innovation in Brain and Learning Sciences, Beijing Normal University, Beijing 100875, PR China
| | - Chuansheng Chen
- Department of Psychology and Social Behavior, University of California, Irvine, CA 92697, USA
| | - Jianfeng Yang
- School of Psychology, Shanxi Normal University, Xi'an 710062, PR China
| | - Jason Zevin
- Department of Linguistics, University of Southern California, Los Angeles, CA 90089, USA
| | - Gui Xue
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China.,Center for Collaboration and Innovation in Brain and Learning Sciences, Beijing Normal University, Beijing 100875, PR China
| |
Collapse
|